2021-03-26 04:17:55,632 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:17:55,632 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): WordEmbeddings('ar') (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) (list_embedding_2): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=206, bias=True) (beta): 1.0 (weights): None (weight_tensor) None )" 2021-03-26 04:17:55,633 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:17:55,633 Corpus: "MultiCorpus: 1573 train + 176 dev + 195 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" 2021-03-26 04:17:55,633 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:17:55,634 Parameters: 2021-03-26 04:17:55,634 - learning_rate: "0.3" 2021-03-26 04:17:55,634 - mini_batch_size: "32" 2021-03-26 04:17:55,634 - patience: "3" 2021-03-26 04:17:55,635 - anneal_factor: "0.5" 2021-03-26 04:17:55,635 - max_epochs: "150" 2021-03-26 04:17:55,635 - shuffle: "True" 2021-03-26 04:17:55,635 - train_with_dev: "False" 2021-03-26 04:17:55,635 - batch_growth_annealing: "False" 2021-03-26 04:17:55,636 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:17:55,636 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.3_202103260417" 2021-03-26 04:17:55,636 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:17:55,636 Device: cuda:0 2021-03-26 04:17:55,636 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:17:55,637 Embeddings storage mode: cpu 2021-03-26 04:17:55,638 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:17:58,321 epoch 1 - iter 5/50 - loss 75.10696945 - samples/sec: 59.66 - lr: 0.300000 2021-03-26 04:18:00,696 epoch 1 - iter 10/50 - loss 71.54744225 - samples/sec: 67.43 - lr: 0.300000 2021-03-26 04:18:03,249 epoch 1 - iter 15/50 - loss 65.56100286 - samples/sec: 62.69 - lr: 0.300000 2021-03-26 04:18:05,739 epoch 1 - iter 20/50 - loss 62.89946499 - samples/sec: 64.29 - lr: 0.300000 2021-03-26 04:18:08,362 epoch 1 - iter 25/50 - loss 60.70132797 - samples/sec: 61.05 - lr: 0.300000 2021-03-26 04:18:10,893 epoch 1 - iter 30/50 - loss 57.30805092 - samples/sec: 63.25 - lr: 0.300000 2021-03-26 04:18:13,216 epoch 1 - iter 35/50 - loss 55.10309862 - samples/sec: 68.92 - lr: 0.300000 2021-03-26 04:18:15,733 epoch 1 - iter 40/50 - loss 53.12539721 - samples/sec: 63.62 - lr: 0.300000 2021-03-26 04:18:18,247 epoch 1 - iter 45/50 - loss 51.49834357 - samples/sec: 63.70 - lr: 0.300000 2021-03-26 04:18:20,689 epoch 1 - iter 50/50 - loss 49.88232040 - samples/sec: 65.55 - lr: 0.300000 2021-03-26 04:18:20,690 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:18:20,690 EPOCH 1 done: loss 49.8823 - lr 0.3000000 2021-03-26 04:18:22,048 DEV : loss 30.73543357849121 - score 0.507 2021-03-26 04:18:22,069 BAD EPOCHS (no improvement): 0 2021-03-26 04:18:31,763 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:18:33,775 epoch 2 - iter 5/50 - loss 32.49798546 - samples/sec: 79.62 - lr: 0.300000 2021-03-26 04:18:35,664 epoch 2 - iter 10/50 - loss 31.19827633 - samples/sec: 84.79 - lr: 0.300000 2021-03-26 04:18:37,726 epoch 2 - iter 15/50 - loss 30.67719905 - samples/sec: 77.69 - lr: 0.300000 2021-03-26 04:18:39,723 epoch 2 - iter 20/50 - loss 29.86658115 - samples/sec: 80.18 - lr: 0.300000 2021-03-26 04:18:41,766 epoch 2 - iter 25/50 - loss 29.56375023 - samples/sec: 78.41 - lr: 0.300000 2021-03-26 04:18:43,775 epoch 2 - iter 30/50 - loss 28.81621901 - samples/sec: 79.72 - lr: 0.300000 2021-03-26 04:18:45,721 epoch 2 - iter 35/50 - loss 28.27086552 - samples/sec: 82.30 - lr: 0.300000 2021-03-26 04:18:47,632 epoch 2 - iter 40/50 - loss 27.63589411 - samples/sec: 83.85 - lr: 0.300000 2021-03-26 04:18:49,710 epoch 2 - iter 45/50 - loss 27.50761443 - samples/sec: 77.06 - lr: 0.300000 2021-03-26 04:18:51,660 epoch 2 - iter 50/50 - loss 27.20592896 - samples/sec: 82.13 - lr: 0.300000 2021-03-26 04:18:51,661 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:18:51,661 EPOCH 2 done: loss 27.2059 - lr 0.3000000 2021-03-26 04:18:52,475 DEV : loss 17.981204986572266 - score 0.6999 2021-03-26 04:18:52,501 BAD EPOCHS (no improvement): 0 2021-03-26 04:19:02,566 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:19:04,559 epoch 3 - iter 5/50 - loss 19.67391281 - samples/sec: 80.41 - lr: 0.300000 2021-03-26 04:19:06,670 epoch 3 - iter 10/50 - loss 20.82917442 - samples/sec: 75.83 - lr: 0.300000 2021-03-26 04:19:08,786 epoch 3 - iter 15/50 - loss 21.23497098 - samples/sec: 75.69 - lr: 0.300000 2021-03-26 04:19:10,635 epoch 3 - iter 20/50 - loss 20.67392168 - samples/sec: 86.64 - lr: 0.300000 2021-03-26 04:19:12,548 epoch 3 - iter 25/50 - loss 19.83488708 - samples/sec: 83.76 - lr: 0.300000 2021-03-26 04:19:14,539 epoch 3 - iter 30/50 - loss 19.89728591 - samples/sec: 80.40 - lr: 0.300000 2021-03-26 04:19:16,432 epoch 3 - iter 35/50 - loss 19.76958389 - samples/sec: 84.61 - lr: 0.300000 2021-03-26 04:19:18,402 epoch 3 - iter 40/50 - loss 19.47361774 - samples/sec: 81.30 - lr: 0.300000 2021-03-26 04:19:20,338 epoch 3 - iter 45/50 - loss 19.23065883 - samples/sec: 82.73 - lr: 0.300000 2021-03-26 04:19:22,103 epoch 3 - iter 50/50 - loss 19.27288631 - samples/sec: 90.77 - lr: 0.300000 2021-03-26 04:19:22,103 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:19:22,104 EPOCH 3 done: loss 19.2729 - lr 0.3000000 2021-03-26 04:19:22,902 DEV : loss 13.695486068725586 - score 0.7567 2021-03-26 04:19:22,928 BAD EPOCHS (no improvement): 0 2021-03-26 04:19:32,480 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:19:34,387 epoch 4 - iter 5/50 - loss 14.91749001 - samples/sec: 84.04 - lr: 0.300000 2021-03-26 04:19:36,328 epoch 4 - iter 10/50 - loss 16.39088926 - samples/sec: 82.48 - lr: 0.300000 2021-03-26 04:19:38,223 epoch 4 - iter 15/50 - loss 16.06209895 - samples/sec: 84.54 - lr: 0.300000 2021-03-26 04:19:40,308 epoch 4 - iter 20/50 - loss 16.54561882 - samples/sec: 76.79 - lr: 0.300000 2021-03-26 04:19:42,262 epoch 4 - iter 25/50 - loss 16.21666988 - samples/sec: 81.94 - lr: 0.300000 2021-03-26 04:19:44,231 epoch 4 - iter 30/50 - loss 16.23734223 - samples/sec: 81.36 - lr: 0.300000 2021-03-26 04:19:46,120 epoch 4 - iter 35/50 - loss 16.14153720 - samples/sec: 84.81 - lr: 0.300000 2021-03-26 04:19:48,060 epoch 4 - iter 40/50 - loss 15.98076320 - samples/sec: 82.52 - lr: 0.300000 2021-03-26 04:19:50,062 epoch 4 - iter 45/50 - loss 15.74216646 - samples/sec: 79.97 - lr: 0.300000 2021-03-26 04:19:51,884 epoch 4 - iter 50/50 - loss 15.68846325 - samples/sec: 87.92 - lr: 0.300000 2021-03-26 04:19:51,885 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:19:51,885 EPOCH 4 done: loss 15.6885 - lr 0.3000000 2021-03-26 04:19:52,680 DEV : loss 12.137931823730469 - score 0.7942 2021-03-26 04:19:52,706 BAD EPOCHS (no improvement): 0 2021-03-26 04:20:02,622 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:20:04,608 epoch 5 - iter 5/50 - loss 15.45747547 - samples/sec: 80.65 - lr: 0.300000 2021-03-26 04:20:06,776 epoch 5 - iter 10/50 - loss 15.33912888 - samples/sec: 73.90 - lr: 0.300000 2021-03-26 04:20:08,865 epoch 5 - iter 15/50 - loss 15.48315074 - samples/sec: 76.70 - lr: 0.300000 2021-03-26 04:20:10,899 epoch 5 - iter 20/50 - loss 14.89399667 - samples/sec: 78.77 - lr: 0.300000 2021-03-26 04:20:12,904 epoch 5 - iter 25/50 - loss 14.98029118 - samples/sec: 79.87 - lr: 0.300000 2021-03-26 04:20:14,756 epoch 5 - iter 30/50 - loss 14.63713500 - samples/sec: 86.50 - lr: 0.300000 2021-03-26 04:20:16,879 epoch 5 - iter 35/50 - loss 14.07365886 - samples/sec: 75.45 - lr: 0.300000 2021-03-26 04:20:18,870 epoch 5 - iter 40/50 - loss 14.03398926 - samples/sec: 80.41 - lr: 0.300000 2021-03-26 04:20:20,746 epoch 5 - iter 45/50 - loss 13.89231139 - samples/sec: 85.41 - lr: 0.300000 2021-03-26 04:20:22,557 epoch 5 - iter 50/50 - loss 13.74129833 - samples/sec: 88.42 - lr: 0.300000 2021-03-26 04:20:22,558 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:20:22,559 EPOCH 5 done: loss 13.7413 - lr 0.3000000 2021-03-26 04:20:23,442 DEV : loss 10.568546295166016 - score 0.8127 2021-03-26 04:20:23,460 BAD EPOCHS (no improvement): 0 2021-03-26 04:20:33,013 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:20:34,946 epoch 6 - iter 5/50 - loss 12.02973881 - samples/sec: 82.90 - lr: 0.300000 2021-03-26 04:20:37,336 epoch 6 - iter 10/50 - loss 12.09904261 - samples/sec: 67.01 - lr: 0.300000 2021-03-26 04:20:39,441 epoch 6 - iter 15/50 - loss 11.87269878 - samples/sec: 76.07 - lr: 0.300000 2021-03-26 04:20:41,745 epoch 6 - iter 20/50 - loss 11.90691776 - samples/sec: 69.51 - lr: 0.300000 2021-03-26 04:20:43,681 epoch 6 - iter 25/50 - loss 11.86476704 - samples/sec: 82.70 - lr: 0.300000 2021-03-26 04:20:45,619 epoch 6 - iter 30/50 - loss 11.94253292 - samples/sec: 82.64 - lr: 0.300000 2021-03-26 04:20:47,465 epoch 6 - iter 35/50 - loss 12.14770309 - samples/sec: 86.76 - lr: 0.300000 2021-03-26 04:20:49,570 epoch 6 - iter 40/50 - loss 12.19662697 - samples/sec: 76.07 - lr: 0.300000 2021-03-26 04:20:51,449 epoch 6 - iter 45/50 - loss 12.11139940 - samples/sec: 85.25 - lr: 0.300000 2021-03-26 04:20:53,099 epoch 6 - iter 50/50 - loss 11.94814116 - samples/sec: 97.09 - lr: 0.300000 2021-03-26 04:20:53,100 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:20:53,100 EPOCH 6 done: loss 11.9481 - lr 0.3000000 2021-03-26 04:20:53,894 DEV : loss 9.031622886657715 - score 0.8425 2021-03-26 04:20:53,920 BAD EPOCHS (no improvement): 0 2021-03-26 04:21:03,930 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:21:06,014 epoch 7 - iter 5/50 - loss 12.33336906 - samples/sec: 76.88 - lr: 0.300000 2021-03-26 04:21:08,062 epoch 7 - iter 10/50 - loss 10.94311547 - samples/sec: 78.20 - lr: 0.300000 2021-03-26 04:21:09,922 epoch 7 - iter 15/50 - loss 11.11774225 - samples/sec: 86.08 - lr: 0.300000 2021-03-26 04:21:11,821 epoch 7 - iter 20/50 - loss 11.46735899 - samples/sec: 84.37 - lr: 0.300000 2021-03-26 04:21:13,675 epoch 7 - iter 25/50 - loss 11.08219643 - samples/sec: 86.40 - lr: 0.300000 2021-03-26 04:21:15,552 epoch 7 - iter 30/50 - loss 11.15123660 - samples/sec: 85.30 - lr: 0.300000 2021-03-26 04:21:17,521 epoch 7 - iter 35/50 - loss 11.02137399 - samples/sec: 81.31 - lr: 0.300000 2021-03-26 04:21:19,751 epoch 7 - iter 40/50 - loss 11.08207619 - samples/sec: 71.83 - lr: 0.300000 2021-03-26 04:21:21,811 epoch 7 - iter 45/50 - loss 11.05912957 - samples/sec: 77.73 - lr: 0.300000 2021-03-26 04:21:23,589 epoch 7 - iter 50/50 - loss 10.84630838 - samples/sec: 90.08 - lr: 0.300000 2021-03-26 04:21:23,590 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:21:23,591 EPOCH 7 done: loss 10.8463 - lr 0.3000000 2021-03-26 04:21:24,378 DEV : loss 9.201812744140625 - score 0.8337 2021-03-26 04:21:24,403 BAD EPOCHS (no improvement): 1 2021-03-26 04:21:24,404 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:21:26,358 epoch 8 - iter 5/50 - loss 9.95667782 - samples/sec: 81.97 - lr: 0.300000 2021-03-26 04:21:28,455 epoch 8 - iter 10/50 - loss 10.97621489 - samples/sec: 76.36 - lr: 0.300000 2021-03-26 04:21:30,506 epoch 8 - iter 15/50 - loss 10.84686445 - samples/sec: 78.10 - lr: 0.300000 2021-03-26 04:21:32,501 epoch 8 - iter 20/50 - loss 10.96737380 - samples/sec: 80.26 - lr: 0.300000 2021-03-26 04:21:34,422 epoch 8 - iter 25/50 - loss 10.52111063 - samples/sec: 83.41 - lr: 0.300000 2021-03-26 04:21:36,394 epoch 8 - iter 30/50 - loss 10.18619401 - samples/sec: 81.20 - lr: 0.300000 2021-03-26 04:21:38,854 epoch 8 - iter 35/50 - loss 10.46665453 - samples/sec: 65.07 - lr: 0.300000 2021-03-26 04:21:41,130 epoch 8 - iter 40/50 - loss 10.44879591 - samples/sec: 70.37 - lr: 0.300000 2021-03-26 04:21:43,139 epoch 8 - iter 45/50 - loss 10.45465293 - samples/sec: 79.70 - lr: 0.300000 2021-03-26 04:21:45,011 epoch 8 - iter 50/50 - loss 10.33866776 - samples/sec: 85.56 - lr: 0.300000 2021-03-26 04:21:45,012 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:21:45,012 EPOCH 8 done: loss 10.3387 - lr 0.3000000 2021-03-26 04:21:45,817 DEV : loss 7.98112678527832 - score 0.8624 2021-03-26 04:21:45,842 BAD EPOCHS (no improvement): 0 2021-03-26 04:21:55,706 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:21:57,603 epoch 9 - iter 5/50 - loss 9.39634228 - samples/sec: 84.46 - lr: 0.300000 2021-03-26 04:21:59,638 epoch 9 - iter 10/50 - loss 9.66039915 - samples/sec: 78.68 - lr: 0.300000 2021-03-26 04:22:01,612 epoch 9 - iter 15/50 - loss 9.76121337 - samples/sec: 81.13 - lr: 0.300000 2021-03-26 04:22:03,619 epoch 9 - iter 20/50 - loss 9.80530524 - samples/sec: 79.83 - lr: 0.300000 2021-03-26 04:22:05,705 epoch 9 - iter 25/50 - loss 9.52289055 - samples/sec: 76.78 - lr: 0.300000 2021-03-26 04:22:07,848 epoch 9 - iter 30/50 - loss 9.68419746 - samples/sec: 74.75 - lr: 0.300000 2021-03-26 04:22:09,921 epoch 9 - iter 35/50 - loss 9.60363786 - samples/sec: 77.29 - lr: 0.300000 2021-03-26 04:22:11,889 epoch 9 - iter 40/50 - loss 9.56846452 - samples/sec: 81.38 - lr: 0.300000 2021-03-26 04:22:13,679 epoch 9 - iter 45/50 - loss 9.52679759 - samples/sec: 89.44 - lr: 0.300000 2021-03-26 04:22:15,610 epoch 9 - iter 50/50 - loss 9.51809608 - samples/sec: 82.98 - lr: 0.300000 2021-03-26 04:22:15,611 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:22:15,611 EPOCH 9 done: loss 9.5181 - lr 0.3000000 2021-03-26 04:22:16,446 DEV : loss 7.542094707489014 - score 0.8666 2021-03-26 04:22:16,467 BAD EPOCHS (no improvement): 0 2021-03-26 04:22:26,106 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:22:28,089 epoch 10 - iter 5/50 - loss 8.68427620 - samples/sec: 80.79 - lr: 0.300000 2021-03-26 04:22:30,020 epoch 10 - iter 10/50 - loss 8.22454109 - samples/sec: 83.05 - lr: 0.300000 2021-03-26 04:22:32,134 epoch 10 - iter 15/50 - loss 8.22477700 - samples/sec: 75.75 - lr: 0.300000 2021-03-26 04:22:34,141 epoch 10 - iter 20/50 - loss 8.36938996 - samples/sec: 79.82 - lr: 0.300000 2021-03-26 04:22:35,979 epoch 10 - iter 25/50 - loss 8.27269264 - samples/sec: 87.13 - lr: 0.300000 2021-03-26 04:22:37,934 epoch 10 - iter 30/50 - loss 8.49550018 - samples/sec: 81.94 - lr: 0.300000 2021-03-26 04:22:40,038 epoch 10 - iter 35/50 - loss 8.60381297 - samples/sec: 76.17 - lr: 0.300000 2021-03-26 04:22:41,867 epoch 10 - iter 40/50 - loss 8.52518930 - samples/sec: 87.61 - lr: 0.300000 2021-03-26 04:22:43,828 epoch 10 - iter 45/50 - loss 8.66577231 - samples/sec: 81.65 - lr: 0.300000 2021-03-26 04:22:45,532 epoch 10 - iter 50/50 - loss 8.64119944 - samples/sec: 94.00 - lr: 0.300000 2021-03-26 04:22:45,532 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:22:45,533 EPOCH 10 done: loss 8.6412 - lr 0.3000000 2021-03-26 04:22:46,324 DEV : loss 6.95101261138916 - score 0.874 2021-03-26 04:22:46,348 BAD EPOCHS (no improvement): 0 2021-03-26 04:22:56,214 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:22:58,403 epoch 11 - iter 5/50 - loss 8.42196026 - samples/sec: 73.20 - lr: 0.300000 2021-03-26 04:23:00,322 epoch 11 - iter 10/50 - loss 8.40741711 - samples/sec: 83.50 - lr: 0.300000 2021-03-26 04:23:02,263 epoch 11 - iter 15/50 - loss 8.24041354 - samples/sec: 82.53 - lr: 0.300000 2021-03-26 04:23:04,403 epoch 11 - iter 20/50 - loss 8.41640503 - samples/sec: 74.84 - lr: 0.300000 2021-03-26 04:23:06,478 epoch 11 - iter 25/50 - loss 8.47183853 - samples/sec: 77.21 - lr: 0.300000 2021-03-26 04:23:08,442 epoch 11 - iter 30/50 - loss 8.52703635 - samples/sec: 81.53 - lr: 0.300000 2021-03-26 04:23:10,264 epoch 11 - iter 35/50 - loss 8.45981086 - samples/sec: 87.98 - lr: 0.300000 2021-03-26 04:23:12,391 epoch 11 - iter 40/50 - loss 8.43065636 - samples/sec: 75.27 - lr: 0.300000 2021-03-26 04:23:14,376 epoch 11 - iter 45/50 - loss 8.42949425 - samples/sec: 80.67 - lr: 0.300000 2021-03-26 04:23:16,426 epoch 11 - iter 50/50 - loss 8.36444000 - samples/sec: 78.16 - lr: 0.300000 2021-03-26 04:23:16,427 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:23:16,427 EPOCH 11 done: loss 8.3644 - lr 0.3000000 2021-03-26 04:23:17,267 DEV : loss 6.946200847625732 - score 0.8772 2021-03-26 04:23:17,293 BAD EPOCHS (no improvement): 0 2021-03-26 04:23:26,956 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:23:29,007 epoch 12 - iter 5/50 - loss 6.99961243 - samples/sec: 78.12 - lr: 0.300000 2021-03-26 04:23:30,967 epoch 12 - iter 10/50 - loss 7.69392977 - samples/sec: 81.69 - lr: 0.300000 2021-03-26 04:23:33,135 epoch 12 - iter 15/50 - loss 8.04296926 - samples/sec: 73.88 - lr: 0.300000 2021-03-26 04:23:35,160 epoch 12 - iter 20/50 - loss 8.05934105 - samples/sec: 79.08 - lr: 0.300000 2021-03-26 04:23:37,187 epoch 12 - iter 25/50 - loss 7.93668758 - samples/sec: 79.03 - lr: 0.300000 2021-03-26 04:23:39,866 epoch 12 - iter 30/50 - loss 8.06570826 - samples/sec: 59.75 - lr: 0.300000 2021-03-26 04:23:41,897 epoch 12 - iter 35/50 - loss 7.95962033 - samples/sec: 78.92 - lr: 0.300000 2021-03-26 04:23:44,121 epoch 12 - iter 40/50 - loss 7.95873179 - samples/sec: 71.98 - lr: 0.300000 2021-03-26 04:23:46,042 epoch 12 - iter 45/50 - loss 7.85535208 - samples/sec: 83.38 - lr: 0.300000 2021-03-26 04:23:47,838 epoch 12 - iter 50/50 - loss 7.84040368 - samples/sec: 89.18 - lr: 0.300000 2021-03-26 04:23:47,839 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:23:47,839 EPOCH 12 done: loss 7.8404 - lr 0.3000000 2021-03-26 04:23:48,675 DEV : loss 6.661767482757568 - score 0.8829 2021-03-26 04:23:48,703 BAD EPOCHS (no improvement): 0 2021-03-26 04:23:58,778 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:24:00,931 epoch 13 - iter 5/50 - loss 7.16372118 - samples/sec: 74.43 - lr: 0.300000 2021-03-26 04:24:02,729 epoch 13 - iter 10/50 - loss 7.43499422 - samples/sec: 89.11 - lr: 0.300000 2021-03-26 04:24:04,635 epoch 13 - iter 15/50 - loss 7.52820756 - samples/sec: 84.04 - lr: 0.300000 2021-03-26 04:24:06,834 epoch 13 - iter 20/50 - loss 7.58331962 - samples/sec: 72.83 - lr: 0.300000 2021-03-26 04:24:08,895 epoch 13 - iter 25/50 - loss 7.71773420 - samples/sec: 77.70 - lr: 0.300000 2021-03-26 04:24:11,069 epoch 13 - iter 30/50 - loss 7.67236060 - samples/sec: 73.67 - lr: 0.300000 2021-03-26 04:24:13,130 epoch 13 - iter 35/50 - loss 7.62056946 - samples/sec: 77.68 - lr: 0.300000 2021-03-26 04:24:15,003 epoch 13 - iter 40/50 - loss 7.53186643 - samples/sec: 85.50 - lr: 0.300000 2021-03-26 04:24:16,956 epoch 13 - iter 45/50 - loss 7.55881039 - samples/sec: 82.04 - lr: 0.300000 2021-03-26 04:24:18,742 epoch 13 - iter 50/50 - loss 7.46832686 - samples/sec: 89.62 - lr: 0.300000 2021-03-26 04:24:18,743 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:24:18,743 EPOCH 13 done: loss 7.4683 - lr 0.3000000 2021-03-26 04:24:19,542 DEV : loss 6.98322057723999 - score 0.8823 2021-03-26 04:24:19,560 BAD EPOCHS (no improvement): 1 2021-03-26 04:24:19,561 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:24:21,655 epoch 14 - iter 5/50 - loss 6.10111704 - samples/sec: 76.47 - lr: 0.300000 2021-03-26 04:24:23,668 epoch 14 - iter 10/50 - loss 6.29064140 - samples/sec: 79.60 - lr: 0.300000 2021-03-26 04:24:25,618 epoch 14 - iter 15/50 - loss 6.46209361 - samples/sec: 82.10 - lr: 0.300000 2021-03-26 04:24:27,656 epoch 14 - iter 20/50 - loss 6.76686928 - samples/sec: 78.57 - lr: 0.300000 2021-03-26 04:24:29,664 epoch 14 - iter 25/50 - loss 6.86153515 - samples/sec: 79.73 - lr: 0.300000 2021-03-26 04:24:31,549 epoch 14 - iter 30/50 - loss 6.75523327 - samples/sec: 84.97 - lr: 0.300000 2021-03-26 04:24:33,582 epoch 14 - iter 35/50 - loss 6.96098984 - samples/sec: 78.75 - lr: 0.300000 2021-03-26 04:24:35,489 epoch 14 - iter 40/50 - loss 6.94293357 - samples/sec: 84.01 - lr: 0.300000 2021-03-26 04:24:37,446 epoch 14 - iter 45/50 - loss 6.96709191 - samples/sec: 81.85 - lr: 0.300000 2021-03-26 04:24:39,275 epoch 14 - iter 50/50 - loss 7.01545926 - samples/sec: 87.59 - lr: 0.300000 2021-03-26 04:24:39,275 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:24:39,276 EPOCH 14 done: loss 7.0155 - lr 0.3000000 2021-03-26 04:24:40,079 DEV : loss 6.541140556335449 - score 0.8916 2021-03-26 04:24:40,105 BAD EPOCHS (no improvement): 0 2021-03-26 04:24:49,775 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:24:51,891 epoch 15 - iter 5/50 - loss 7.28769703 - samples/sec: 75.72 - lr: 0.300000 2021-03-26 04:24:53,966 epoch 15 - iter 10/50 - loss 7.33594060 - samples/sec: 77.18 - lr: 0.300000 2021-03-26 04:24:55,862 epoch 15 - iter 15/50 - loss 6.93184964 - samples/sec: 84.43 - lr: 0.300000 2021-03-26 04:24:57,869 epoch 15 - iter 20/50 - loss 6.87248054 - samples/sec: 79.79 - lr: 0.300000 2021-03-26 04:24:59,767 epoch 15 - iter 25/50 - loss 6.95717945 - samples/sec: 84.43 - lr: 0.300000 2021-03-26 04:25:01,869 epoch 15 - iter 30/50 - loss 7.01571414 - samples/sec: 76.18 - lr: 0.300000 2021-03-26 04:25:03,809 epoch 15 - iter 35/50 - loss 7.14505937 - samples/sec: 82.55 - lr: 0.300000 2021-03-26 04:25:05,793 epoch 15 - iter 40/50 - loss 7.03667165 - samples/sec: 80.75 - lr: 0.300000 2021-03-26 04:25:07,699 epoch 15 - iter 45/50 - loss 6.90432454 - samples/sec: 84.02 - lr: 0.300000 2021-03-26 04:25:09,309 epoch 15 - iter 50/50 - loss 6.88903606 - samples/sec: 99.54 - lr: 0.300000 2021-03-26 04:25:09,310 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:25:09,310 EPOCH 15 done: loss 6.8890 - lr 0.3000000 2021-03-26 04:25:10,121 DEV : loss 6.795278072357178 - score 0.8813 2021-03-26 04:25:10,146 BAD EPOCHS (no improvement): 1 2021-03-26 04:25:10,147 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:25:12,174 epoch 16 - iter 5/50 - loss 6.26615057 - samples/sec: 78.98 - lr: 0.300000 2021-03-26 04:25:14,157 epoch 16 - iter 10/50 - loss 6.86684566 - samples/sec: 80.79 - lr: 0.300000 2021-03-26 04:25:16,097 epoch 16 - iter 15/50 - loss 6.87310921 - samples/sec: 82.56 - lr: 0.300000 2021-03-26 04:25:17,936 epoch 16 - iter 20/50 - loss 6.56489854 - samples/sec: 87.12 - lr: 0.300000 2021-03-26 04:25:19,906 epoch 16 - iter 25/50 - loss 6.47284828 - samples/sec: 81.28 - lr: 0.300000 2021-03-26 04:25:21,812 epoch 16 - iter 30/50 - loss 6.31857347 - samples/sec: 84.05 - lr: 0.300000 2021-03-26 04:25:23,865 epoch 16 - iter 35/50 - loss 6.49668576 - samples/sec: 77.97 - lr: 0.300000 2021-03-26 04:25:25,833 epoch 16 - iter 40/50 - loss 6.44836339 - samples/sec: 81.39 - lr: 0.300000 2021-03-26 04:25:27,753 epoch 16 - iter 45/50 - loss 6.49273729 - samples/sec: 83.43 - lr: 0.300000 2021-03-26 04:25:29,478 epoch 16 - iter 50/50 - loss 6.52631361 - samples/sec: 92.83 - lr: 0.300000 2021-03-26 04:25:29,478 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:25:29,479 EPOCH 16 done: loss 6.5263 - lr 0.3000000 2021-03-26 04:25:30,312 DEV : loss 6.253621578216553 - score 0.8935 2021-03-26 04:25:30,338 BAD EPOCHS (no improvement): 0 2021-03-26 04:25:40,033 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:25:42,008 epoch 17 - iter 5/50 - loss 6.54145203 - samples/sec: 81.17 - lr: 0.300000 2021-03-26 04:25:44,025 epoch 17 - iter 10/50 - loss 6.48270683 - samples/sec: 79.37 - lr: 0.300000 2021-03-26 04:25:46,119 epoch 17 - iter 15/50 - loss 6.22515707 - samples/sec: 76.49 - lr: 0.300000 2021-03-26 04:25:47,974 epoch 17 - iter 20/50 - loss 6.20492010 - samples/sec: 86.36 - lr: 0.300000 2021-03-26 04:25:49,913 epoch 17 - iter 25/50 - loss 6.13051668 - samples/sec: 82.62 - lr: 0.300000 2021-03-26 04:25:51,864 epoch 17 - iter 30/50 - loss 6.13832904 - samples/sec: 82.09 - lr: 0.300000 2021-03-26 04:25:53,769 epoch 17 - iter 35/50 - loss 6.12548726 - samples/sec: 84.10 - lr: 0.300000 2021-03-26 04:25:55,680 epoch 17 - iter 40/50 - loss 6.18831244 - samples/sec: 83.84 - lr: 0.300000 2021-03-26 04:25:57,818 epoch 17 - iter 45/50 - loss 6.28022619 - samples/sec: 74.89 - lr: 0.300000 2021-03-26 04:25:59,757 epoch 17 - iter 50/50 - loss 6.22215544 - samples/sec: 82.58 - lr: 0.300000 2021-03-26 04:25:59,757 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:25:59,758 EPOCH 17 done: loss 6.2222 - lr 0.3000000 2021-03-26 04:26:00,587 DEV : loss 5.916778564453125 - score 0.8984 2021-03-26 04:26:00,612 BAD EPOCHS (no improvement): 0 2021-03-26 04:26:10,517 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:26:12,469 epoch 18 - iter 5/50 - loss 5.82463799 - samples/sec: 82.16 - lr: 0.300000 2021-03-26 04:26:14,563 epoch 18 - iter 10/50 - loss 5.79875684 - samples/sec: 76.45 - lr: 0.300000 2021-03-26 04:26:16,654 epoch 18 - iter 15/50 - loss 5.68639402 - samples/sec: 76.57 - lr: 0.300000 2021-03-26 04:26:18,633 epoch 18 - iter 20/50 - loss 5.87758648 - samples/sec: 80.94 - lr: 0.300000 2021-03-26 04:26:20,589 epoch 18 - iter 25/50 - loss 5.76586943 - samples/sec: 81.89 - lr: 0.300000 2021-03-26 04:26:22,563 epoch 18 - iter 30/50 - loss 5.90244009 - samples/sec: 81.19 - lr: 0.300000 2021-03-26 04:26:24,687 epoch 18 - iter 35/50 - loss 5.86991540 - samples/sec: 75.39 - lr: 0.300000 2021-03-26 04:26:26,702 epoch 18 - iter 40/50 - loss 5.96687905 - samples/sec: 79.48 - lr: 0.300000 2021-03-26 04:26:28,538 epoch 18 - iter 45/50 - loss 5.96893187 - samples/sec: 87.22 - lr: 0.300000 2021-03-26 04:26:30,673 epoch 18 - iter 50/50 - loss 6.12081981 - samples/sec: 74.98 - lr: 0.300000 2021-03-26 04:26:30,674 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:26:30,675 EPOCH 18 done: loss 6.1208 - lr 0.3000000 2021-03-26 04:26:31,500 DEV : loss 6.017233848571777 - score 0.899 2021-03-26 04:26:31,525 BAD EPOCHS (no improvement): 0 2021-03-26 04:26:41,353 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:26:43,368 epoch 19 - iter 5/50 - loss 5.00032930 - samples/sec: 79.47 - lr: 0.300000 2021-03-26 04:26:45,421 epoch 19 - iter 10/50 - loss 5.22974877 - samples/sec: 78.03 - lr: 0.300000 2021-03-26 04:26:47,424 epoch 19 - iter 15/50 - loss 5.28217349 - samples/sec: 79.97 - lr: 0.300000 2021-03-26 04:26:49,427 epoch 19 - iter 20/50 - loss 5.33064126 - samples/sec: 79.96 - lr: 0.300000 2021-03-26 04:26:51,311 epoch 19 - iter 25/50 - loss 5.43961421 - samples/sec: 85.00 - lr: 0.300000 2021-03-26 04:26:53,174 epoch 19 - iter 30/50 - loss 5.54057804 - samples/sec: 85.97 - lr: 0.300000 2021-03-26 04:26:55,322 epoch 19 - iter 35/50 - loss 5.59828159 - samples/sec: 74.54 - lr: 0.300000 2021-03-26 04:26:57,570 epoch 19 - iter 40/50 - loss 5.63219038 - samples/sec: 71.24 - lr: 0.300000 2021-03-26 04:26:59,866 epoch 19 - iter 45/50 - loss 5.62833966 - samples/sec: 69.77 - lr: 0.300000 2021-03-26 04:27:02,021 epoch 19 - iter 50/50 - loss 5.73004192 - samples/sec: 74.29 - lr: 0.300000 2021-03-26 04:27:02,022 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:27:02,022 EPOCH 19 done: loss 5.7300 - lr 0.3000000 2021-03-26 04:27:02,832 DEV : loss 5.986006736755371 - score 0.9011 2021-03-26 04:27:02,856 BAD EPOCHS (no improvement): 0 2021-03-26 04:27:12,395 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:27:14,577 epoch 20 - iter 5/50 - loss 4.56922116 - samples/sec: 73.43 - lr: 0.300000 2021-03-26 04:27:16,568 epoch 20 - iter 10/50 - loss 5.21878107 - samples/sec: 80.42 - lr: 0.300000 2021-03-26 04:27:18,629 epoch 20 - iter 15/50 - loss 5.54959607 - samples/sec: 77.68 - lr: 0.300000 2021-03-26 04:27:20,511 epoch 20 - iter 20/50 - loss 5.40102017 - samples/sec: 85.12 - lr: 0.300000 2021-03-26 04:27:22,577 epoch 20 - iter 25/50 - loss 5.53657951 - samples/sec: 77.50 - lr: 0.300000 2021-03-26 04:27:24,522 epoch 20 - iter 30/50 - loss 5.57236757 - samples/sec: 82.34 - lr: 0.300000 2021-03-26 04:27:26,745 epoch 20 - iter 35/50 - loss 5.51763896 - samples/sec: 72.02 - lr: 0.300000 2021-03-26 04:27:28,880 epoch 20 - iter 40/50 - loss 5.48591815 - samples/sec: 75.02 - lr: 0.300000 2021-03-26 04:27:30,937 epoch 20 - iter 45/50 - loss 5.49633989 - samples/sec: 77.91 - lr: 0.300000 2021-03-26 04:27:32,852 epoch 20 - iter 50/50 - loss 5.57298695 - samples/sec: 83.60 - lr: 0.300000 2021-03-26 04:27:32,853 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:27:32,853 EPOCH 20 done: loss 5.5730 - lr 0.3000000 2021-03-26 04:27:33,652 DEV : loss 5.930326461791992 - score 0.8968 2021-03-26 04:27:33,676 BAD EPOCHS (no improvement): 1 2021-03-26 04:27:33,677 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:27:35,714 epoch 21 - iter 5/50 - loss 6.82391586 - samples/sec: 78.62 - lr: 0.300000 2021-03-26 04:27:37,885 epoch 21 - iter 10/50 - loss 5.70353832 - samples/sec: 73.74 - lr: 0.300000 2021-03-26 04:27:39,715 epoch 21 - iter 15/50 - loss 5.08343248 - samples/sec: 87.50 - lr: 0.300000 2021-03-26 04:27:41,571 epoch 21 - iter 20/50 - loss 5.16639459 - samples/sec: 86.31 - lr: 0.300000 2021-03-26 04:27:43,425 epoch 21 - iter 25/50 - loss 5.16516088 - samples/sec: 86.37 - lr: 0.300000 2021-03-26 04:27:45,541 epoch 21 - iter 30/50 - loss 5.25475041 - samples/sec: 75.71 - lr: 0.300000 2021-03-26 04:27:47,672 epoch 21 - iter 35/50 - loss 5.19062170 - samples/sec: 75.17 - lr: 0.300000 2021-03-26 04:27:49,833 epoch 21 - iter 40/50 - loss 5.14780552 - samples/sec: 74.10 - lr: 0.300000 2021-03-26 04:27:51,757 epoch 21 - iter 45/50 - loss 5.16311884 - samples/sec: 83.27 - lr: 0.300000 2021-03-26 04:27:53,831 epoch 21 - iter 50/50 - loss 5.24303764 - samples/sec: 77.21 - lr: 0.300000 2021-03-26 04:27:53,831 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:27:53,832 EPOCH 21 done: loss 5.2430 - lr 0.3000000 2021-03-26 04:27:54,631 DEV : loss 5.876016139984131 - score 0.9041 2021-03-26 04:27:54,653 BAD EPOCHS (no improvement): 0 2021-03-26 04:28:04,386 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:28:06,584 epoch 22 - iter 5/50 - loss 5.56373177 - samples/sec: 72.87 - lr: 0.300000 2021-03-26 04:28:08,505 epoch 22 - iter 10/50 - loss 5.34489045 - samples/sec: 83.37 - lr: 0.300000 2021-03-26 04:28:10,423 epoch 22 - iter 15/50 - loss 5.28978802 - samples/sec: 83.54 - lr: 0.300000 2021-03-26 04:28:12,292 epoch 22 - iter 20/50 - loss 5.25375621 - samples/sec: 85.72 - lr: 0.300000 2021-03-26 04:28:14,329 epoch 22 - iter 25/50 - loss 5.11982265 - samples/sec: 78.62 - lr: 0.300000 2021-03-26 04:28:16,538 epoch 22 - iter 30/50 - loss 5.22046140 - samples/sec: 72.49 - lr: 0.300000 2021-03-26 04:28:19,075 epoch 22 - iter 35/50 - loss 5.05933951 - samples/sec: 63.11 - lr: 0.300000 2021-03-26 04:28:21,098 epoch 22 - iter 40/50 - loss 5.06113884 - samples/sec: 79.16 - lr: 0.300000 2021-03-26 04:28:23,017 epoch 22 - iter 45/50 - loss 5.06400488 - samples/sec: 83.44 - lr: 0.300000 2021-03-26 04:28:25,049 epoch 22 - iter 50/50 - loss 5.00299379 - samples/sec: 78.79 - lr: 0.300000 2021-03-26 04:28:25,050 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:28:25,050 EPOCH 22 done: loss 5.0030 - lr 0.3000000 2021-03-26 04:28:25,858 DEV : loss 6.265571594238281 - score 0.9001 2021-03-26 04:28:25,882 BAD EPOCHS (no improvement): 1 2021-03-26 04:28:25,883 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:28:27,807 epoch 23 - iter 5/50 - loss 4.88015375 - samples/sec: 83.26 - lr: 0.300000 2021-03-26 04:28:29,610 epoch 23 - iter 10/50 - loss 4.93044343 - samples/sec: 88.80 - lr: 0.300000 2021-03-26 04:28:31,525 epoch 23 - iter 15/50 - loss 4.89805943 - samples/sec: 83.63 - lr: 0.300000 2021-03-26 04:28:33,514 epoch 23 - iter 20/50 - loss 4.81811663 - samples/sec: 80.51 - lr: 0.300000 2021-03-26 04:28:35,611 epoch 23 - iter 25/50 - loss 4.94307913 - samples/sec: 76.41 - lr: 0.300000 2021-03-26 04:28:37,546 epoch 23 - iter 30/50 - loss 4.94647453 - samples/sec: 82.73 - lr: 0.300000 2021-03-26 04:28:39,574 epoch 23 - iter 35/50 - loss 5.03459977 - samples/sec: 79.00 - lr: 0.300000 2021-03-26 04:28:41,466 epoch 23 - iter 40/50 - loss 4.94717068 - samples/sec: 84.63 - lr: 0.300000 2021-03-26 04:28:43,578 epoch 23 - iter 45/50 - loss 5.00173565 - samples/sec: 75.83 - lr: 0.300000 2021-03-26 04:28:45,557 epoch 23 - iter 50/50 - loss 5.05484855 - samples/sec: 80.93 - lr: 0.300000 2021-03-26 04:28:45,557 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:28:45,557 EPOCH 23 done: loss 5.0548 - lr 0.3000000 2021-03-26 04:28:46,352 DEV : loss 6.006520748138428 - score 0.9057 2021-03-26 04:28:46,373 BAD EPOCHS (no improvement): 0 2021-03-26 04:28:55,851 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:28:57,870 epoch 24 - iter 5/50 - loss 3.85893168 - samples/sec: 79.32 - lr: 0.300000 2021-03-26 04:28:59,880 epoch 24 - iter 10/50 - loss 4.09285135 - samples/sec: 79.73 - lr: 0.300000 2021-03-26 04:29:01,886 epoch 24 - iter 15/50 - loss 4.19558508 - samples/sec: 79.80 - lr: 0.300000 2021-03-26 04:29:03,783 epoch 24 - iter 20/50 - loss 4.47492408 - samples/sec: 84.48 - lr: 0.300000 2021-03-26 04:29:05,795 epoch 24 - iter 25/50 - loss 4.56721498 - samples/sec: 79.56 - lr: 0.300000 2021-03-26 04:29:07,745 epoch 24 - iter 30/50 - loss 4.72045338 - samples/sec: 82.12 - lr: 0.300000 2021-03-26 04:29:09,649 epoch 24 - iter 35/50 - loss 4.74745126 - samples/sec: 84.18 - lr: 0.300000 2021-03-26 04:29:11,558 epoch 24 - iter 40/50 - loss 4.66962092 - samples/sec: 83.88 - lr: 0.300000 2021-03-26 04:29:13,521 epoch 24 - iter 45/50 - loss 4.73731511 - samples/sec: 81.60 - lr: 0.300000 2021-03-26 04:29:15,381 epoch 24 - iter 50/50 - loss 4.78475182 - samples/sec: 86.14 - lr: 0.300000 2021-03-26 04:29:15,382 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:29:15,382 EPOCH 24 done: loss 4.7848 - lr 0.3000000 2021-03-26 04:29:16,207 DEV : loss 6.140750885009766 - score 0.9007 2021-03-26 04:29:16,232 BAD EPOCHS (no improvement): 1 2021-03-26 04:29:16,233 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:29:18,379 epoch 25 - iter 5/50 - loss 4.88933020 - samples/sec: 74.64 - lr: 0.300000 2021-03-26 04:29:20,394 epoch 25 - iter 10/50 - loss 4.66825428 - samples/sec: 79.44 - lr: 0.300000 2021-03-26 04:29:22,242 epoch 25 - iter 15/50 - loss 4.67216385 - samples/sec: 86.68 - lr: 0.300000 2021-03-26 04:29:24,195 epoch 25 - iter 20/50 - loss 4.49212998 - samples/sec: 82.01 - lr: 0.300000 2021-03-26 04:29:26,091 epoch 25 - iter 25/50 - loss 4.58727453 - samples/sec: 84.47 - lr: 0.300000 2021-03-26 04:29:28,017 epoch 25 - iter 30/50 - loss 4.55948855 - samples/sec: 83.18 - lr: 0.300000 2021-03-26 04:29:29,965 epoch 25 - iter 35/50 - loss 4.46674612 - samples/sec: 82.24 - lr: 0.300000 2021-03-26 04:29:31,937 epoch 25 - iter 40/50 - loss 4.52352757 - samples/sec: 81.23 - lr: 0.300000 2021-03-26 04:29:33,897 epoch 25 - iter 45/50 - loss 4.58539889 - samples/sec: 81.71 - lr: 0.300000 2021-03-26 04:29:35,781 epoch 25 - iter 50/50 - loss 4.57801357 - samples/sec: 84.99 - lr: 0.300000 2021-03-26 04:29:35,782 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:29:35,782 EPOCH 25 done: loss 4.5780 - lr 0.3000000 2021-03-26 04:29:36,609 DEV : loss 6.105770111083984 - score 0.9059 2021-03-26 04:29:36,635 BAD EPOCHS (no improvement): 0 2021-03-26 04:29:46,517 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:29:48,522 epoch 26 - iter 5/50 - loss 4.06674557 - samples/sec: 79.91 - lr: 0.300000 2021-03-26 04:29:50,450 epoch 26 - iter 10/50 - loss 4.42879331 - samples/sec: 83.06 - lr: 0.300000 2021-03-26 04:29:52,488 epoch 26 - iter 15/50 - loss 4.60063888 - samples/sec: 78.55 - lr: 0.300000 2021-03-26 04:29:54,433 epoch 26 - iter 20/50 - loss 4.58111991 - samples/sec: 82.38 - lr: 0.300000 2021-03-26 04:29:56,492 epoch 26 - iter 25/50 - loss 4.44824369 - samples/sec: 77.74 - lr: 0.300000 2021-03-26 04:29:58,473 epoch 26 - iter 30/50 - loss 4.39979011 - samples/sec: 80.85 - lr: 0.300000 2021-03-26 04:30:00,547 epoch 26 - iter 35/50 - loss 4.48905651 - samples/sec: 77.23 - lr: 0.300000 2021-03-26 04:30:02,655 epoch 26 - iter 40/50 - loss 4.54615889 - samples/sec: 75.93 - lr: 0.300000 2021-03-26 04:30:04,549 epoch 26 - iter 45/50 - loss 4.57882829 - samples/sec: 84.55 - lr: 0.300000 2021-03-26 04:30:06,321 epoch 26 - iter 50/50 - loss 4.49696365 - samples/sec: 90.39 - lr: 0.300000 2021-03-26 04:30:06,322 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:30:06,322 EPOCH 26 done: loss 4.4970 - lr 0.3000000 2021-03-26 04:30:07,134 DEV : loss 6.628708839416504 - score 0.8974 2021-03-26 04:30:07,156 BAD EPOCHS (no improvement): 1 2021-03-26 04:30:07,156 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:30:08,969 epoch 27 - iter 5/50 - loss 4.78592529 - samples/sec: 88.35 - lr: 0.300000 2021-03-26 04:30:11,153 epoch 27 - iter 10/50 - loss 4.92443051 - samples/sec: 73.30 - lr: 0.300000 2021-03-26 04:30:13,078 epoch 27 - iter 15/50 - loss 4.73327821 - samples/sec: 83.24 - lr: 0.300000 2021-03-26 04:30:15,045 epoch 27 - iter 20/50 - loss 4.75421574 - samples/sec: 81.39 - lr: 0.300000 2021-03-26 04:30:17,141 epoch 27 - iter 25/50 - loss 4.66145059 - samples/sec: 76.43 - lr: 0.300000 2021-03-26 04:30:19,233 epoch 27 - iter 30/50 - loss 4.61727355 - samples/sec: 76.54 - lr: 0.300000 2021-03-26 04:30:21,261 epoch 27 - iter 35/50 - loss 4.65609898 - samples/sec: 78.95 - lr: 0.300000 2021-03-26 04:30:23,220 epoch 27 - iter 40/50 - loss 4.61374962 - samples/sec: 81.81 - lr: 0.300000 2021-03-26 04:30:25,081 epoch 27 - iter 45/50 - loss 4.52198956 - samples/sec: 86.08 - lr: 0.300000 2021-03-26 04:30:26,948 epoch 27 - iter 50/50 - loss 4.54031735 - samples/sec: 85.77 - lr: 0.300000 2021-03-26 04:30:26,949 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:30:26,949 EPOCH 27 done: loss 4.5403 - lr 0.3000000 2021-03-26 04:30:27,744 DEV : loss 6.179070949554443 - score 0.9033 2021-03-26 04:30:27,762 BAD EPOCHS (no improvement): 2 2021-03-26 04:30:27,762 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:30:29,742 epoch 28 - iter 5/50 - loss 4.99302526 - samples/sec: 80.93 - lr: 0.300000 2021-03-26 04:30:31,815 epoch 28 - iter 10/50 - loss 4.83438566 - samples/sec: 77.24 - lr: 0.300000 2021-03-26 04:30:33,693 epoch 28 - iter 15/50 - loss 4.58077038 - samples/sec: 85.31 - lr: 0.300000 2021-03-26 04:30:35,654 epoch 28 - iter 20/50 - loss 4.53201609 - samples/sec: 81.64 - lr: 0.300000 2021-03-26 04:30:37,621 epoch 28 - iter 25/50 - loss 4.37551620 - samples/sec: 81.42 - lr: 0.300000 2021-03-26 04:30:39,540 epoch 28 - iter 30/50 - loss 4.24277393 - samples/sec: 83.44 - lr: 0.300000 2021-03-26 04:30:41,456 epoch 28 - iter 35/50 - loss 4.30316538 - samples/sec: 83.61 - lr: 0.300000 2021-03-26 04:30:43,577 epoch 28 - iter 40/50 - loss 4.33001579 - samples/sec: 75.47 - lr: 0.300000 2021-03-26 04:30:45,534 epoch 28 - iter 45/50 - loss 4.35705420 - samples/sec: 81.82 - lr: 0.300000 2021-03-26 04:30:47,427 epoch 28 - iter 50/50 - loss 4.31629688 - samples/sec: 84.59 - lr: 0.300000 2021-03-26 04:30:47,428 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:30:47,428 EPOCH 28 done: loss 4.3163 - lr 0.3000000 2021-03-26 04:30:48,250 DEV : loss 5.845078945159912 - score 0.9038 2021-03-26 04:30:48,270 BAD EPOCHS (no improvement): 3 2021-03-26 04:30:48,271 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:30:50,432 epoch 29 - iter 5/50 - loss 4.47561340 - samples/sec: 74.12 - lr: 0.300000 2021-03-26 04:30:52,438 epoch 29 - iter 10/50 - loss 4.05306280 - samples/sec: 79.88 - lr: 0.300000 2021-03-26 04:30:54,531 epoch 29 - iter 15/50 - loss 4.26914605 - samples/sec: 76.51 - lr: 0.300000 2021-03-26 04:30:56,485 epoch 29 - iter 20/50 - loss 4.22125438 - samples/sec: 81.99 - lr: 0.300000 2021-03-26 04:30:58,434 epoch 29 - iter 25/50 - loss 4.19066468 - samples/sec: 82.18 - lr: 0.300000 2021-03-26 04:31:00,471 epoch 29 - iter 30/50 - loss 4.11440060 - samples/sec: 78.61 - lr: 0.300000 2021-03-26 04:31:02,363 epoch 29 - iter 35/50 - loss 4.13777413 - samples/sec: 84.71 - lr: 0.300000 2021-03-26 04:31:04,222 epoch 29 - iter 40/50 - loss 4.09913780 - samples/sec: 86.15 - lr: 0.300000 2021-03-26 04:31:06,304 epoch 29 - iter 45/50 - loss 4.15816302 - samples/sec: 76.93 - lr: 0.300000 2021-03-26 04:31:08,093 epoch 29 - iter 50/50 - loss 4.18301320 - samples/sec: 89.62 - lr: 0.300000 2021-03-26 04:31:08,094 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:31:08,094 EPOCH 29 done: loss 4.1830 - lr 0.3000000 2021-03-26 04:31:08,926 DEV : loss 6.092258453369141 - score 0.9051 2021-03-26 04:31:08,950 BAD EPOCHS (no improvement): 4 2021-03-26 04:31:08,951 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:31:11,072 epoch 30 - iter 5/50 - loss 4.20534439 - samples/sec: 75.49 - lr: 0.150000 2021-03-26 04:31:13,164 epoch 30 - iter 10/50 - loss 4.13369124 - samples/sec: 76.52 - lr: 0.150000 2021-03-26 04:31:15,215 epoch 30 - iter 15/50 - loss 3.90392586 - samples/sec: 78.13 - lr: 0.150000 2021-03-26 04:31:17,081 epoch 30 - iter 20/50 - loss 3.84214165 - samples/sec: 85.89 - lr: 0.150000 2021-03-26 04:31:19,071 epoch 30 - iter 25/50 - loss 3.86618098 - samples/sec: 80.45 - lr: 0.150000 2021-03-26 04:31:21,025 epoch 30 - iter 30/50 - loss 3.75677759 - samples/sec: 82.00 - lr: 0.150000 2021-03-26 04:31:23,051 epoch 30 - iter 35/50 - loss 3.81993992 - samples/sec: 79.05 - lr: 0.150000 2021-03-26 04:31:25,031 epoch 30 - iter 40/50 - loss 3.82808222 - samples/sec: 80.90 - lr: 0.150000 2021-03-26 04:31:26,997 epoch 30 - iter 45/50 - loss 3.81757127 - samples/sec: 81.46 - lr: 0.150000 2021-03-26 04:31:28,853 epoch 30 - iter 50/50 - loss 3.96143827 - samples/sec: 86.29 - lr: 0.150000 2021-03-26 04:31:28,854 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:31:28,854 EPOCH 30 done: loss 3.9614 - lr 0.1500000 2021-03-26 04:31:29,685 DEV : loss 5.713150978088379 - score 0.9096 2021-03-26 04:31:29,711 BAD EPOCHS (no improvement): 0 2021-03-26 04:31:39,433 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:31:41,513 epoch 31 - iter 5/50 - loss 3.97427645 - samples/sec: 77.01 - lr: 0.150000 2021-03-26 04:31:43,538 epoch 31 - iter 10/50 - loss 4.07154245 - samples/sec: 79.13 - lr: 0.150000 2021-03-26 04:31:45,428 epoch 31 - iter 15/50 - loss 3.84109275 - samples/sec: 84.73 - lr: 0.150000 2021-03-26 04:31:47,265 epoch 31 - iter 20/50 - loss 3.63473686 - samples/sec: 87.20 - lr: 0.150000 2021-03-26 04:31:49,372 epoch 31 - iter 25/50 - loss 3.69866761 - samples/sec: 75.98 - lr: 0.150000 2021-03-26 04:31:51,259 epoch 31 - iter 30/50 - loss 3.57191054 - samples/sec: 84.97 - lr: 0.150000 2021-03-26 04:31:53,281 epoch 31 - iter 35/50 - loss 3.60921316 - samples/sec: 79.19 - lr: 0.150000 2021-03-26 04:31:55,098 epoch 31 - iter 40/50 - loss 3.56092208 - samples/sec: 88.21 - lr: 0.150000 2021-03-26 04:31:57,014 epoch 31 - iter 45/50 - loss 3.53439759 - samples/sec: 83.57 - lr: 0.150000 2021-03-26 04:31:58,797 epoch 31 - iter 50/50 - loss 3.53695005 - samples/sec: 89.86 - lr: 0.150000 2021-03-26 04:31:58,798 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:31:58,798 EPOCH 31 done: loss 3.5370 - lr 0.1500000 2021-03-26 04:31:59,609 DEV : loss 5.8165435791015625 - score 0.9118 2021-03-26 04:31:59,635 BAD EPOCHS (no improvement): 0 2021-03-26 04:32:09,492 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:32:11,392 epoch 32 - iter 5/50 - loss 4.09934840 - samples/sec: 84.37 - lr: 0.150000 2021-03-26 04:32:13,356 epoch 32 - iter 10/50 - loss 3.60038693 - samples/sec: 81.54 - lr: 0.150000 2021-03-26 04:32:15,493 epoch 32 - iter 15/50 - loss 3.67805559 - samples/sec: 74.95 - lr: 0.150000 2021-03-26 04:32:17,365 epoch 32 - iter 20/50 - loss 3.61150162 - samples/sec: 85.60 - lr: 0.150000 2021-03-26 04:32:19,404 epoch 32 - iter 25/50 - loss 3.41172076 - samples/sec: 78.51 - lr: 0.150000 2021-03-26 04:32:21,448 epoch 32 - iter 30/50 - loss 3.48814637 - samples/sec: 78.38 - lr: 0.150000 2021-03-26 04:32:23,451 epoch 32 - iter 35/50 - loss 3.48316173 - samples/sec: 79.92 - lr: 0.150000 2021-03-26 04:32:25,542 epoch 32 - iter 40/50 - loss 3.47152191 - samples/sec: 76.57 - lr: 0.150000 2021-03-26 04:32:27,716 epoch 32 - iter 45/50 - loss 3.48673698 - samples/sec: 73.68 - lr: 0.150000 2021-03-26 04:32:29,340 epoch 32 - iter 50/50 - loss 3.40475936 - samples/sec: 98.62 - lr: 0.150000 2021-03-26 04:32:29,341 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:32:29,341 EPOCH 32 done: loss 3.4048 - lr 0.1500000 2021-03-26 04:32:31,473 DEV : loss 5.617457389831543 - score 0.9136 2021-03-26 04:32:31,510 BAD EPOCHS (no improvement): 0 2021-03-26 04:32:41,354 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:32:43,608 epoch 33 - iter 5/50 - loss 2.96983914 - samples/sec: 71.05 - lr: 0.150000 2021-03-26 04:32:45,636 epoch 33 - iter 10/50 - loss 3.19039111 - samples/sec: 79.00 - lr: 0.150000 2021-03-26 04:32:47,564 epoch 33 - iter 15/50 - loss 3.03779140 - samples/sec: 83.09 - lr: 0.150000 2021-03-26 04:32:49,550 epoch 33 - iter 20/50 - loss 3.12249979 - samples/sec: 80.64 - lr: 0.150000 2021-03-26 04:32:51,600 epoch 33 - iter 25/50 - loss 3.13500700 - samples/sec: 78.09 - lr: 0.150000 2021-03-26 04:32:53,471 epoch 33 - iter 30/50 - loss 3.13828891 - samples/sec: 85.65 - lr: 0.150000 2021-03-26 04:32:55,516 epoch 33 - iter 35/50 - loss 3.20323824 - samples/sec: 78.33 - lr: 0.150000 2021-03-26 04:32:57,567 epoch 33 - iter 40/50 - loss 3.16869446 - samples/sec: 78.06 - lr: 0.150000 2021-03-26 04:32:59,656 epoch 33 - iter 45/50 - loss 3.27099452 - samples/sec: 76.67 - lr: 0.150000 2021-03-26 04:33:01,478 epoch 33 - iter 50/50 - loss 3.27532232 - samples/sec: 87.91 - lr: 0.150000 2021-03-26 04:33:01,479 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:33:01,479 EPOCH 33 done: loss 3.2753 - lr 0.1500000 2021-03-26 04:33:02,316 DEV : loss 5.714725494384766 - score 0.9124 2021-03-26 04:33:02,342 BAD EPOCHS (no improvement): 1 2021-03-26 04:33:02,342 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:33:04,309 epoch 34 - iter 5/50 - loss 2.89319005 - samples/sec: 81.46 - lr: 0.150000 2021-03-26 04:33:06,232 epoch 34 - iter 10/50 - loss 2.75303411 - samples/sec: 83.27 - lr: 0.150000 2021-03-26 04:33:08,338 epoch 34 - iter 15/50 - loss 3.11904596 - samples/sec: 76.02 - lr: 0.150000 2021-03-26 04:33:10,602 epoch 34 - iter 20/50 - loss 3.22779783 - samples/sec: 70.74 - lr: 0.150000 2021-03-26 04:33:12,540 epoch 34 - iter 25/50 - loss 3.27530612 - samples/sec: 82.64 - lr: 0.150000 2021-03-26 04:33:14,553 epoch 34 - iter 30/50 - loss 3.29493998 - samples/sec: 79.52 - lr: 0.150000 2021-03-26 04:33:16,467 epoch 34 - iter 35/50 - loss 3.31849298 - samples/sec: 83.68 - lr: 0.150000 2021-03-26 04:33:18,275 epoch 34 - iter 40/50 - loss 3.23385530 - samples/sec: 88.59 - lr: 0.150000 2021-03-26 04:33:20,195 epoch 34 - iter 45/50 - loss 3.17170356 - samples/sec: 83.49 - lr: 0.150000 2021-03-26 04:33:22,037 epoch 34 - iter 50/50 - loss 3.18908337 - samples/sec: 86.92 - lr: 0.150000 2021-03-26 04:33:22,038 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:33:22,038 EPOCH 34 done: loss 3.1891 - lr 0.1500000 2021-03-26 04:33:22,852 DEV : loss 5.6389312744140625 - score 0.9136 2021-03-26 04:33:22,873 BAD EPOCHS (no improvement): 2 2021-03-26 04:33:22,874 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:33:24,832 epoch 35 - iter 5/50 - loss 2.92917271 - samples/sec: 81.80 - lr: 0.150000 2021-03-26 04:33:26,841 epoch 35 - iter 10/50 - loss 3.03267279 - samples/sec: 79.73 - lr: 0.150000 2021-03-26 04:33:28,869 epoch 35 - iter 15/50 - loss 3.13962334 - samples/sec: 78.96 - lr: 0.150000 2021-03-26 04:33:30,717 epoch 35 - iter 20/50 - loss 3.20024943 - samples/sec: 86.65 - lr: 0.150000 2021-03-26 04:33:32,584 epoch 35 - iter 25/50 - loss 3.23139182 - samples/sec: 85.76 - lr: 0.150000 2021-03-26 04:33:34,464 epoch 35 - iter 30/50 - loss 3.11910969 - samples/sec: 85.18 - lr: 0.150000 2021-03-26 04:33:36,597 epoch 35 - iter 35/50 - loss 3.06835806 - samples/sec: 75.12 - lr: 0.150000 2021-03-26 04:33:38,676 epoch 35 - iter 40/50 - loss 3.11270922 - samples/sec: 77.00 - lr: 0.150000 2021-03-26 04:33:40,518 epoch 35 - iter 45/50 - loss 3.10661338 - samples/sec: 86.92 - lr: 0.150000 2021-03-26 04:33:42,433 epoch 35 - iter 50/50 - loss 3.14953357 - samples/sec: 83.64 - lr: 0.150000 2021-03-26 04:33:42,434 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:33:42,434 EPOCH 35 done: loss 3.1495 - lr 0.1500000 2021-03-26 04:33:43,271 DEV : loss 5.767107963562012 - score 0.9122 2021-03-26 04:33:43,298 BAD EPOCHS (no improvement): 3 2021-03-26 04:33:43,298 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:33:45,465 epoch 36 - iter 5/50 - loss 3.18284845 - samples/sec: 73.89 - lr: 0.150000 2021-03-26 04:33:47,373 epoch 36 - iter 10/50 - loss 3.61262190 - samples/sec: 83.94 - lr: 0.150000 2021-03-26 04:33:49,321 epoch 36 - iter 15/50 - loss 3.56332256 - samples/sec: 82.22 - lr: 0.150000 2021-03-26 04:33:51,328 epoch 36 - iter 20/50 - loss 3.38617566 - samples/sec: 79.77 - lr: 0.150000 2021-03-26 04:33:53,248 epoch 36 - iter 25/50 - loss 3.27218135 - samples/sec: 83.45 - lr: 0.150000 2021-03-26 04:33:55,266 epoch 36 - iter 30/50 - loss 3.26657916 - samples/sec: 79.35 - lr: 0.150000 2021-03-26 04:33:57,401 epoch 36 - iter 35/50 - loss 3.25774821 - samples/sec: 74.98 - lr: 0.150000 2021-03-26 04:33:59,259 epoch 36 - iter 40/50 - loss 3.29469748 - samples/sec: 86.29 - lr: 0.150000 2021-03-26 04:34:01,156 epoch 36 - iter 45/50 - loss 3.27597940 - samples/sec: 84.39 - lr: 0.150000 2021-03-26 04:34:02,977 epoch 36 - iter 50/50 - loss 3.32588531 - samples/sec: 87.99 - lr: 0.150000 2021-03-26 04:34:02,977 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:34:02,978 EPOCH 36 done: loss 3.3259 - lr 0.1500000 2021-03-26 04:34:03,795 DEV : loss 5.621600151062012 - score 0.9114 2021-03-26 04:34:03,821 BAD EPOCHS (no improvement): 4 2021-03-26 04:34:03,822 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:34:05,909 epoch 37 - iter 5/50 - loss 3.24834247 - samples/sec: 76.73 - lr: 0.075000 2021-03-26 04:34:07,829 epoch 37 - iter 10/50 - loss 3.15661476 - samples/sec: 83.48 - lr: 0.075000 2021-03-26 04:34:09,776 epoch 37 - iter 15/50 - loss 3.05580727 - samples/sec: 82.25 - lr: 0.075000 2021-03-26 04:34:11,596 epoch 37 - iter 20/50 - loss 2.99344176 - samples/sec: 87.97 - lr: 0.075000 2021-03-26 04:34:13,697 epoch 37 - iter 25/50 - loss 2.99454953 - samples/sec: 76.23 - lr: 0.075000 2021-03-26 04:34:15,759 epoch 37 - iter 30/50 - loss 3.03316812 - samples/sec: 77.64 - lr: 0.075000 2021-03-26 04:34:17,722 epoch 37 - iter 35/50 - loss 3.03424073 - samples/sec: 81.60 - lr: 0.075000 2021-03-26 04:34:19,646 epoch 37 - iter 40/50 - loss 3.06192328 - samples/sec: 83.23 - lr: 0.075000 2021-03-26 04:34:22,044 epoch 37 - iter 45/50 - loss 3.01700522 - samples/sec: 66.75 - lr: 0.075000 2021-03-26 04:34:24,021 epoch 37 - iter 50/50 - loss 3.06639231 - samples/sec: 81.03 - lr: 0.075000 2021-03-26 04:34:24,021 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:34:24,022 EPOCH 37 done: loss 3.0664 - lr 0.0750000 2021-03-26 04:34:24,840 DEV : loss 5.518418788909912 - score 0.9183 2021-03-26 04:34:24,859 BAD EPOCHS (no improvement): 0 2021-03-26 04:34:34,545 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:34:36,543 epoch 38 - iter 5/50 - loss 3.03597603 - samples/sec: 80.20 - lr: 0.075000 2021-03-26 04:34:38,828 epoch 38 - iter 10/50 - loss 2.76591871 - samples/sec: 70.06 - lr: 0.075000 2021-03-26 04:34:40,861 epoch 38 - iter 15/50 - loss 3.07190213 - samples/sec: 78.78 - lr: 0.075000 2021-03-26 04:34:42,892 epoch 38 - iter 20/50 - loss 3.16678113 - samples/sec: 78.86 - lr: 0.075000 2021-03-26 04:34:44,907 epoch 38 - iter 25/50 - loss 3.06128056 - samples/sec: 79.52 - lr: 0.075000 2021-03-26 04:34:46,942 epoch 38 - iter 30/50 - loss 3.09672271 - samples/sec: 78.69 - lr: 0.075000 2021-03-26 04:34:48,862 epoch 38 - iter 35/50 - loss 3.00959673 - samples/sec: 83.42 - lr: 0.075000 2021-03-26 04:34:50,782 epoch 38 - iter 40/50 - loss 3.02298063 - samples/sec: 83.42 - lr: 0.075000 2021-03-26 04:34:52,693 epoch 38 - iter 45/50 - loss 3.00045899 - samples/sec: 83.81 - lr: 0.075000 2021-03-26 04:34:54,592 epoch 38 - iter 50/50 - loss 2.97319459 - samples/sec: 84.31 - lr: 0.075000 2021-03-26 04:34:54,593 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:34:54,593 EPOCH 38 done: loss 2.9732 - lr 0.0750000 2021-03-26 04:34:55,436 DEV : loss 5.583366394042969 - score 0.9142 2021-03-26 04:34:55,455 BAD EPOCHS (no improvement): 1 2021-03-26 04:34:55,456 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:34:57,401 epoch 39 - iter 5/50 - loss 2.80546980 - samples/sec: 82.30 - lr: 0.075000 2021-03-26 04:34:59,405 epoch 39 - iter 10/50 - loss 3.02134721 - samples/sec: 79.92 - lr: 0.075000 2021-03-26 04:35:01,331 epoch 39 - iter 15/50 - loss 2.87578603 - samples/sec: 83.13 - lr: 0.075000 2021-03-26 04:35:03,179 epoch 39 - iter 20/50 - loss 2.81496171 - samples/sec: 86.66 - lr: 0.075000 2021-03-26 04:35:05,238 epoch 39 - iter 25/50 - loss 2.82369418 - samples/sec: 77.80 - lr: 0.075000 2021-03-26 04:35:07,215 epoch 39 - iter 30/50 - loss 2.83144122 - samples/sec: 81.04 - lr: 0.075000 2021-03-26 04:35:09,314 epoch 39 - iter 35/50 - loss 2.85793325 - samples/sec: 76.26 - lr: 0.075000 2021-03-26 04:35:11,180 epoch 39 - iter 40/50 - loss 2.88679821 - samples/sec: 85.82 - lr: 0.075000 2021-03-26 04:35:13,101 epoch 39 - iter 45/50 - loss 2.85130506 - samples/sec: 83.38 - lr: 0.075000 2021-03-26 04:35:14,930 epoch 39 - iter 50/50 - loss 2.84462428 - samples/sec: 87.53 - lr: 0.075000 2021-03-26 04:35:14,931 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:35:14,931 EPOCH 39 done: loss 2.8446 - lr 0.0750000 2021-03-26 04:35:15,708 DEV : loss 5.578202724456787 - score 0.9146 2021-03-26 04:35:15,733 BAD EPOCHS (no improvement): 2 2021-03-26 04:35:15,734 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:35:17,691 epoch 40 - iter 5/50 - loss 3.21980987 - samples/sec: 81.84 - lr: 0.075000 2021-03-26 04:35:19,685 epoch 40 - iter 10/50 - loss 2.90155966 - samples/sec: 80.31 - lr: 0.075000 2021-03-26 04:35:21,654 epoch 40 - iter 15/50 - loss 2.88553217 - samples/sec: 81.31 - lr: 0.075000 2021-03-26 04:35:23,646 epoch 40 - iter 20/50 - loss 2.84431648 - samples/sec: 80.43 - lr: 0.075000 2021-03-26 04:35:25,461 epoch 40 - iter 25/50 - loss 2.83064776 - samples/sec: 88.21 - lr: 0.075000 2021-03-26 04:35:27,363 epoch 40 - iter 30/50 - loss 2.81809179 - samples/sec: 84.26 - lr: 0.075000 2021-03-26 04:35:29,390 epoch 40 - iter 35/50 - loss 2.87427166 - samples/sec: 78.99 - lr: 0.075000 2021-03-26 04:35:31,415 epoch 40 - iter 40/50 - loss 2.90430885 - samples/sec: 79.08 - lr: 0.075000 2021-03-26 04:35:33,373 epoch 40 - iter 45/50 - loss 2.86053363 - samples/sec: 81.78 - lr: 0.075000 2021-03-26 04:35:35,158 epoch 40 - iter 50/50 - loss 2.90439391 - samples/sec: 89.76 - lr: 0.075000 2021-03-26 04:35:35,159 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:35:35,159 EPOCH 40 done: loss 2.9044 - lr 0.0750000 2021-03-26 04:35:35,978 DEV : loss 5.7049994468688965 - score 0.9118 2021-03-26 04:35:36,004 BAD EPOCHS (no improvement): 3 2021-03-26 04:35:36,005 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:35:38,161 epoch 41 - iter 5/50 - loss 2.84771671 - samples/sec: 74.30 - lr: 0.075000 2021-03-26 04:35:40,251 epoch 41 - iter 10/50 - loss 2.94583633 - samples/sec: 76.62 - lr: 0.075000 2021-03-26 04:35:42,371 epoch 41 - iter 15/50 - loss 2.72260372 - samples/sec: 75.54 - lr: 0.075000 2021-03-26 04:35:44,450 epoch 41 - iter 20/50 - loss 2.76606383 - samples/sec: 77.02 - lr: 0.075000 2021-03-26 04:35:46,377 epoch 41 - iter 25/50 - loss 2.76165267 - samples/sec: 83.13 - lr: 0.075000 2021-03-26 04:35:48,253 epoch 41 - iter 30/50 - loss 2.85141704 - samples/sec: 85.36 - lr: 0.075000 2021-03-26 04:35:50,314 epoch 41 - iter 35/50 - loss 2.80431631 - samples/sec: 77.72 - lr: 0.075000 2021-03-26 04:35:52,563 epoch 41 - iter 40/50 - loss 2.77395735 - samples/sec: 71.21 - lr: 0.075000 2021-03-26 04:35:54,603 epoch 41 - iter 45/50 - loss 2.82585680 - samples/sec: 78.54 - lr: 0.075000 2021-03-26 04:35:56,527 epoch 41 - iter 50/50 - loss 2.76732529 - samples/sec: 83.24 - lr: 0.075000 2021-03-26 04:35:56,528 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:35:56,528 EPOCH 41 done: loss 2.7673 - lr 0.0750000 2021-03-26 04:35:57,451 DEV : loss 5.642393112182617 - score 0.9177 2021-03-26 04:35:57,477 BAD EPOCHS (no improvement): 4 2021-03-26 04:35:57,478 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:35:59,690 epoch 42 - iter 5/50 - loss 3.21034460 - samples/sec: 72.39 - lr: 0.037500 2021-03-26 04:36:02,342 epoch 42 - iter 10/50 - loss 3.35436149 - samples/sec: 60.40 - lr: 0.037500 2021-03-26 04:36:04,451 epoch 42 - iter 15/50 - loss 3.15462260 - samples/sec: 75.98 - lr: 0.037500 2021-03-26 04:36:06,524 epoch 42 - iter 20/50 - loss 3.04606973 - samples/sec: 77.25 - lr: 0.037500 2021-03-26 04:36:08,527 epoch 42 - iter 25/50 - loss 2.99213443 - samples/sec: 79.98 - lr: 0.037500 2021-03-26 04:36:10,483 epoch 42 - iter 30/50 - loss 2.90308900 - samples/sec: 81.90 - lr: 0.037500 2021-03-26 04:36:12,373 epoch 42 - iter 35/50 - loss 2.88003882 - samples/sec: 84.74 - lr: 0.037500 2021-03-26 04:36:14,356 epoch 42 - iter 40/50 - loss 2.84434015 - samples/sec: 80.74 - lr: 0.037500 2021-03-26 04:36:16,462 epoch 42 - iter 45/50 - loss 2.76004986 - samples/sec: 76.07 - lr: 0.037500 2021-03-26 04:36:18,278 epoch 42 - iter 50/50 - loss 2.71170438 - samples/sec: 88.20 - lr: 0.037500 2021-03-26 04:36:18,279 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:36:18,279 EPOCH 42 done: loss 2.7117 - lr 0.0375000 2021-03-26 04:36:19,157 DEV : loss 5.596333980560303 - score 0.9191 2021-03-26 04:36:19,181 BAD EPOCHS (no improvement): 0 2021-03-26 04:36:28,982 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:36:31,053 epoch 43 - iter 5/50 - loss 2.53796349 - samples/sec: 77.34 - lr: 0.037500 2021-03-26 04:36:33,117 epoch 43 - iter 10/50 - loss 2.67341330 - samples/sec: 77.59 - lr: 0.037500 2021-03-26 04:36:35,206 epoch 43 - iter 15/50 - loss 2.67378864 - samples/sec: 76.66 - lr: 0.037500 2021-03-26 04:36:37,234 epoch 43 - iter 20/50 - loss 2.65061052 - samples/sec: 78.95 - lr: 0.037500 2021-03-26 04:36:39,252 epoch 43 - iter 25/50 - loss 2.57634023 - samples/sec: 79.35 - lr: 0.037500 2021-03-26 04:36:41,419 epoch 43 - iter 30/50 - loss 2.63065434 - samples/sec: 73.90 - lr: 0.037500 2021-03-26 04:36:43,523 epoch 43 - iter 35/50 - loss 2.63950038 - samples/sec: 76.12 - lr: 0.037500 2021-03-26 04:36:45,589 epoch 43 - iter 40/50 - loss 2.64751761 - samples/sec: 77.55 - lr: 0.037500 2021-03-26 04:36:47,700 epoch 43 - iter 45/50 - loss 2.74424743 - samples/sec: 75.88 - lr: 0.037500 2021-03-26 04:36:49,751 epoch 43 - iter 50/50 - loss 2.72231129 - samples/sec: 78.06 - lr: 0.037500 2021-03-26 04:36:49,752 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:36:49,752 EPOCH 43 done: loss 2.7223 - lr 0.0375000 2021-03-26 04:36:50,556 DEV : loss 5.48341703414917 - score 0.9203 2021-03-26 04:36:50,582 BAD EPOCHS (no improvement): 0 2021-03-26 04:37:00,171 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:37:02,366 epoch 44 - iter 5/50 - loss 2.98065414 - samples/sec: 73.00 - lr: 0.037500 2021-03-26 04:37:04,384 epoch 44 - iter 10/50 - loss 3.02249229 - samples/sec: 79.34 - lr: 0.037500 2021-03-26 04:37:06,401 epoch 44 - iter 15/50 - loss 2.85490796 - samples/sec: 79.42 - lr: 0.037500 2021-03-26 04:37:08,446 epoch 44 - iter 20/50 - loss 2.86533296 - samples/sec: 78.29 - lr: 0.037500 2021-03-26 04:37:10,490 epoch 44 - iter 25/50 - loss 2.80952433 - samples/sec: 78.33 - lr: 0.037500 2021-03-26 04:37:12,411 epoch 44 - iter 30/50 - loss 2.80893228 - samples/sec: 83.40 - lr: 0.037500 2021-03-26 04:37:14,429 epoch 44 - iter 35/50 - loss 2.75288236 - samples/sec: 79.36 - lr: 0.037500 2021-03-26 04:37:16,351 epoch 44 - iter 40/50 - loss 2.72165121 - samples/sec: 83.34 - lr: 0.037500 2021-03-26 04:37:18,338 epoch 44 - iter 45/50 - loss 2.73430733 - samples/sec: 80.57 - lr: 0.037500 2021-03-26 04:37:20,362 epoch 44 - iter 50/50 - loss 2.72831919 - samples/sec: 79.12 - lr: 0.037500 2021-03-26 04:37:20,363 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:37:20,363 EPOCH 44 done: loss 2.7283 - lr 0.0375000 2021-03-26 04:37:21,167 DEV : loss 5.58450984954834 - score 0.9177 2021-03-26 04:37:21,191 BAD EPOCHS (no improvement): 1 2021-03-26 04:37:21,192 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:37:23,156 epoch 45 - iter 5/50 - loss 2.30449636 - samples/sec: 81.54 - lr: 0.037500 2021-03-26 04:37:25,015 epoch 45 - iter 10/50 - loss 2.32092698 - samples/sec: 86.15 - lr: 0.037500 2021-03-26 04:37:27,011 epoch 45 - iter 15/50 - loss 2.29700800 - samples/sec: 80.25 - lr: 0.037500 2021-03-26 04:37:29,031 epoch 45 - iter 20/50 - loss 2.43191308 - samples/sec: 79.28 - lr: 0.037500 2021-03-26 04:37:30,997 epoch 45 - iter 25/50 - loss 2.51080059 - samples/sec: 81.45 - lr: 0.037500 2021-03-26 04:37:32,982 epoch 45 - iter 30/50 - loss 2.52594601 - samples/sec: 80.73 - lr: 0.037500 2021-03-26 04:37:34,963 epoch 45 - iter 35/50 - loss 2.51447598 - samples/sec: 80.89 - lr: 0.037500 2021-03-26 04:37:37,083 epoch 45 - iter 40/50 - loss 2.48263493 - samples/sec: 75.55 - lr: 0.037500 2021-03-26 04:37:39,153 epoch 45 - iter 45/50 - loss 2.48890561 - samples/sec: 77.40 - lr: 0.037500 2021-03-26 04:37:41,196 epoch 45 - iter 50/50 - loss 2.48906700 - samples/sec: 78.40 - lr: 0.037500 2021-03-26 04:37:41,197 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:37:41,197 EPOCH 45 done: loss 2.4891 - lr 0.0375000 2021-03-26 04:37:42,141 DEV : loss 5.558457374572754 - score 0.9191 2021-03-26 04:37:42,165 BAD EPOCHS (no improvement): 2 2021-03-26 04:37:42,166 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:37:44,169 epoch 46 - iter 5/50 - loss 2.66223438 - samples/sec: 79.97 - lr: 0.037500 2021-03-26 04:37:46,352 epoch 46 - iter 10/50 - loss 2.64043747 - samples/sec: 73.35 - lr: 0.037500 2021-03-26 04:37:48,443 epoch 46 - iter 15/50 - loss 2.68533693 - samples/sec: 76.58 - lr: 0.037500 2021-03-26 04:37:50,538 epoch 46 - iter 20/50 - loss 2.90953861 - samples/sec: 76.46 - lr: 0.037500 2021-03-26 04:37:52,680 epoch 46 - iter 25/50 - loss 2.93996088 - samples/sec: 74.79 - lr: 0.037500 2021-03-26 04:37:54,552 epoch 46 - iter 30/50 - loss 2.83967495 - samples/sec: 85.57 - lr: 0.037500 2021-03-26 04:37:56,390 epoch 46 - iter 35/50 - loss 2.74778470 - samples/sec: 87.13 - lr: 0.037500 2021-03-26 04:37:58,518 epoch 46 - iter 40/50 - loss 2.73911650 - samples/sec: 75.25 - lr: 0.037500 2021-03-26 04:38:00,941 epoch 46 - iter 45/50 - loss 2.68517716 - samples/sec: 66.09 - lr: 0.037500 2021-03-26 04:38:02,810 epoch 46 - iter 50/50 - loss 2.67263688 - samples/sec: 85.68 - lr: 0.037500 2021-03-26 04:38:02,811 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:38:02,811 EPOCH 46 done: loss 2.6726 - lr 0.0375000 2021-03-26 04:38:03,601 DEV : loss 5.60237979888916 - score 0.9211 2021-03-26 04:38:03,625 BAD EPOCHS (no improvement): 0 2021-03-26 04:38:13,494 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:38:15,481 epoch 47 - iter 5/50 - loss 2.47122407 - samples/sec: 80.64 - lr: 0.037500 2021-03-26 04:38:17,378 epoch 47 - iter 10/50 - loss 2.47364318 - samples/sec: 84.41 - lr: 0.037500 2021-03-26 04:38:19,518 epoch 47 - iter 15/50 - loss 2.48115468 - samples/sec: 74.84 - lr: 0.037500 2021-03-26 04:38:21,375 epoch 47 - iter 20/50 - loss 2.65376525 - samples/sec: 86.29 - lr: 0.037500 2021-03-26 04:38:23,460 epoch 47 - iter 25/50 - loss 2.67308200 - samples/sec: 76.81 - lr: 0.037500 2021-03-26 04:38:25,329 epoch 47 - iter 30/50 - loss 2.60481541 - samples/sec: 85.70 - lr: 0.037500 2021-03-26 04:38:27,280 epoch 47 - iter 35/50 - loss 2.56278869 - samples/sec: 82.09 - lr: 0.037500 2021-03-26 04:38:29,182 epoch 47 - iter 40/50 - loss 2.61160978 - samples/sec: 84.22 - lr: 0.037500 2021-03-26 04:38:31,447 epoch 47 - iter 45/50 - loss 2.58385894 - samples/sec: 70.68 - lr: 0.037500 2021-03-26 04:38:33,414 epoch 47 - iter 50/50 - loss 2.51415426 - samples/sec: 81.44 - lr: 0.037500 2021-03-26 04:38:33,415 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:38:33,415 EPOCH 47 done: loss 2.5142 - lr 0.0375000 2021-03-26 04:38:34,253 DEV : loss 5.621846675872803 - score 0.9215 2021-03-26 04:38:34,274 BAD EPOCHS (no improvement): 0 2021-03-26 04:38:43,880 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:38:46,076 epoch 48 - iter 5/50 - loss 2.57413118 - samples/sec: 72.95 - lr: 0.037500 2021-03-26 04:38:48,107 epoch 48 - iter 10/50 - loss 2.61529049 - samples/sec: 78.86 - lr: 0.037500 2021-03-26 04:38:50,015 epoch 48 - iter 15/50 - loss 2.66174406 - samples/sec: 83.90 - lr: 0.037500 2021-03-26 04:38:52,139 epoch 48 - iter 20/50 - loss 2.75963891 - samples/sec: 75.39 - lr: 0.037500 2021-03-26 04:38:54,253 epoch 48 - iter 25/50 - loss 2.66337224 - samples/sec: 75.75 - lr: 0.037500 2021-03-26 04:38:56,278 epoch 48 - iter 30/50 - loss 2.67778315 - samples/sec: 79.08 - lr: 0.037500 2021-03-26 04:38:58,402 epoch 48 - iter 35/50 - loss 2.64026489 - samples/sec: 75.41 - lr: 0.037500 2021-03-26 04:39:00,478 epoch 48 - iter 40/50 - loss 2.59820676 - samples/sec: 77.12 - lr: 0.037500 2021-03-26 04:39:02,539 epoch 48 - iter 45/50 - loss 2.64941340 - samples/sec: 77.68 - lr: 0.037500 2021-03-26 04:39:04,142 epoch 48 - iter 50/50 - loss 2.60686312 - samples/sec: 100.01 - lr: 0.037500 2021-03-26 04:39:04,143 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:39:04,143 EPOCH 48 done: loss 2.6069 - lr 0.0375000 2021-03-26 04:39:04,964 DEV : loss 5.574971675872803 - score 0.9211 2021-03-26 04:39:04,990 BAD EPOCHS (no improvement): 1 2021-03-26 04:39:04,990 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:39:06,953 epoch 49 - iter 5/50 - loss 2.26608279 - samples/sec: 81.62 - lr: 0.037500 2021-03-26 04:39:08,952 epoch 49 - iter 10/50 - loss 2.48984057 - samples/sec: 80.06 - lr: 0.037500 2021-03-26 04:39:10,891 epoch 49 - iter 15/50 - loss 2.41273263 - samples/sec: 82.59 - lr: 0.037500 2021-03-26 04:39:13,164 epoch 49 - iter 20/50 - loss 2.46317679 - samples/sec: 70.44 - lr: 0.037500 2021-03-26 04:39:15,268 epoch 49 - iter 25/50 - loss 2.48580365 - samples/sec: 76.10 - lr: 0.037500 2021-03-26 04:39:17,266 epoch 49 - iter 30/50 - loss 2.52267183 - samples/sec: 80.20 - lr: 0.037500 2021-03-26 04:39:19,066 epoch 49 - iter 35/50 - loss 2.55116108 - samples/sec: 88.96 - lr: 0.037500 2021-03-26 04:39:21,139 epoch 49 - iter 40/50 - loss 2.50691254 - samples/sec: 77.22 - lr: 0.037500 2021-03-26 04:39:23,089 epoch 49 - iter 45/50 - loss 2.54569444 - samples/sec: 82.13 - lr: 0.037500 2021-03-26 04:39:24,887 epoch 49 - iter 50/50 - loss 2.50360020 - samples/sec: 89.10 - lr: 0.037500 2021-03-26 04:39:24,887 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:39:24,888 EPOCH 49 done: loss 2.5036 - lr 0.0375000 2021-03-26 04:39:25,683 DEV : loss 5.674612998962402 - score 0.9167 2021-03-26 04:39:25,707 BAD EPOCHS (no improvement): 2 2021-03-26 04:39:25,708 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:39:27,531 epoch 50 - iter 5/50 - loss 2.49787436 - samples/sec: 87.87 - lr: 0.037500 2021-03-26 04:39:29,470 epoch 50 - iter 10/50 - loss 2.54350984 - samples/sec: 82.65 - lr: 0.037500 2021-03-26 04:39:31,617 epoch 50 - iter 15/50 - loss 2.55895030 - samples/sec: 74.60 - lr: 0.037500 2021-03-26 04:39:33,746 epoch 50 - iter 20/50 - loss 2.53379130 - samples/sec: 75.19 - lr: 0.037500 2021-03-26 04:39:35,777 epoch 50 - iter 25/50 - loss 2.46748724 - samples/sec: 78.90 - lr: 0.037500 2021-03-26 04:39:37,667 epoch 50 - iter 30/50 - loss 2.53201131 - samples/sec: 84.77 - lr: 0.037500 2021-03-26 04:39:39,587 epoch 50 - iter 35/50 - loss 2.58285206 - samples/sec: 83.45 - lr: 0.037500 2021-03-26 04:39:41,597 epoch 50 - iter 40/50 - loss 2.54398156 - samples/sec: 79.71 - lr: 0.037500 2021-03-26 04:39:43,507 epoch 50 - iter 45/50 - loss 2.51924728 - samples/sec: 83.86 - lr: 0.037500 2021-03-26 04:39:45,276 epoch 50 - iter 50/50 - loss 2.56936092 - samples/sec: 90.53 - lr: 0.037500 2021-03-26 04:39:45,277 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:39:45,277 EPOCH 50 done: loss 2.5694 - lr 0.0375000 2021-03-26 04:39:46,087 DEV : loss 5.591398239135742 - score 0.9179 2021-03-26 04:39:46,113 BAD EPOCHS (no improvement): 3 2021-03-26 04:39:46,114 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:39:48,144 epoch 51 - iter 5/50 - loss 2.60005298 - samples/sec: 78.89 - lr: 0.037500 2021-03-26 04:39:50,046 epoch 51 - iter 10/50 - loss 2.50737890 - samples/sec: 84.21 - lr: 0.037500 2021-03-26 04:39:51,950 epoch 51 - iter 15/50 - loss 2.58061322 - samples/sec: 84.13 - lr: 0.037500 2021-03-26 04:39:53,823 epoch 51 - iter 20/50 - loss 2.49857840 - samples/sec: 85.48 - lr: 0.037500 2021-03-26 04:39:56,201 epoch 51 - iter 25/50 - loss 2.54270968 - samples/sec: 67.33 - lr: 0.037500 2021-03-26 04:39:58,075 epoch 51 - iter 30/50 - loss 2.56095430 - samples/sec: 85.51 - lr: 0.037500 2021-03-26 04:40:00,130 epoch 51 - iter 35/50 - loss 2.56534418 - samples/sec: 77.90 - lr: 0.037500 2021-03-26 04:40:02,152 epoch 51 - iter 40/50 - loss 2.54329309 - samples/sec: 79.28 - lr: 0.037500 2021-03-26 04:40:04,114 epoch 51 - iter 45/50 - loss 2.54833703 - samples/sec: 81.60 - lr: 0.037500 2021-03-26 04:40:05,955 epoch 51 - iter 50/50 - loss 2.54080164 - samples/sec: 87.01 - lr: 0.037500 2021-03-26 04:40:05,955 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:40:05,955 EPOCH 51 done: loss 2.5408 - lr 0.0375000 2021-03-26 04:40:06,771 DEV : loss 5.714648246765137 - score 0.916 2021-03-26 04:40:06,801 BAD EPOCHS (no improvement): 4 2021-03-26 04:40:06,802 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:40:08,792 epoch 52 - iter 5/50 - loss 2.18448515 - samples/sec: 80.49 - lr: 0.018750 2021-03-26 04:40:10,804 epoch 52 - iter 10/50 - loss 2.40422935 - samples/sec: 79.58 - lr: 0.018750 2021-03-26 04:40:12,773 epoch 52 - iter 15/50 - loss 2.46985007 - samples/sec: 81.38 - lr: 0.018750 2021-03-26 04:40:14,770 epoch 52 - iter 20/50 - loss 2.40598304 - samples/sec: 80.20 - lr: 0.018750 2021-03-26 04:40:16,798 epoch 52 - iter 25/50 - loss 2.37731241 - samples/sec: 78.96 - lr: 0.018750 2021-03-26 04:40:18,969 epoch 52 - iter 30/50 - loss 2.41620742 - samples/sec: 73.77 - lr: 0.018750 2021-03-26 04:40:21,096 epoch 52 - iter 35/50 - loss 2.39733038 - samples/sec: 75.34 - lr: 0.018750 2021-03-26 04:40:23,180 epoch 52 - iter 40/50 - loss 2.37697385 - samples/sec: 76.83 - lr: 0.018750 2021-03-26 04:40:25,171 epoch 52 - iter 45/50 - loss 2.41595177 - samples/sec: 80.45 - lr: 0.018750 2021-03-26 04:40:27,080 epoch 52 - iter 50/50 - loss 2.47739849 - samples/sec: 83.89 - lr: 0.018750 2021-03-26 04:40:27,081 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:40:27,082 EPOCH 52 done: loss 2.4774 - lr 0.0187500 2021-03-26 04:40:27,914 DEV : loss 5.6428985595703125 - score 0.9215 2021-03-26 04:40:27,940 BAD EPOCHS (no improvement): 1 2021-03-26 04:40:27,941 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:40:29,869 epoch 53 - iter 5/50 - loss 2.80800118 - samples/sec: 83.07 - lr: 0.018750 2021-03-26 04:40:31,897 epoch 53 - iter 10/50 - loss 2.71635209 - samples/sec: 79.00 - lr: 0.018750 2021-03-26 04:40:33,762 epoch 53 - iter 15/50 - loss 2.40103800 - samples/sec: 85.88 - lr: 0.018750 2021-03-26 04:40:35,795 epoch 53 - iter 20/50 - loss 2.41370821 - samples/sec: 78.77 - lr: 0.018750 2021-03-26 04:40:37,757 epoch 53 - iter 25/50 - loss 2.45257851 - samples/sec: 81.64 - lr: 0.018750 2021-03-26 04:40:39,823 epoch 53 - iter 30/50 - loss 2.40813193 - samples/sec: 77.52 - lr: 0.018750 2021-03-26 04:40:41,944 epoch 53 - iter 35/50 - loss 2.44468188 - samples/sec: 75.50 - lr: 0.018750 2021-03-26 04:40:43,768 epoch 53 - iter 40/50 - loss 2.44019739 - samples/sec: 87.82 - lr: 0.018750 2021-03-26 04:40:45,736 epoch 53 - iter 45/50 - loss 2.48135226 - samples/sec: 81.40 - lr: 0.018750 2021-03-26 04:40:47,540 epoch 53 - iter 50/50 - loss 2.48608873 - samples/sec: 88.79 - lr: 0.018750 2021-03-26 04:40:47,541 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:40:47,541 EPOCH 53 done: loss 2.4861 - lr 0.0187500 2021-03-26 04:40:48,354 DEV : loss 5.656498908996582 - score 0.9187 2021-03-26 04:40:48,376 BAD EPOCHS (no improvement): 2 2021-03-26 04:40:48,376 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:40:50,448 epoch 54 - iter 5/50 - loss 2.60976076 - samples/sec: 77.31 - lr: 0.018750 2021-03-26 04:40:52,395 epoch 54 - iter 10/50 - loss 2.35027825 - samples/sec: 82.28 - lr: 0.018750 2021-03-26 04:40:54,351 epoch 54 - iter 15/50 - loss 2.36107490 - samples/sec: 81.90 - lr: 0.018750 2021-03-26 04:40:56,955 epoch 54 - iter 20/50 - loss 2.33634679 - samples/sec: 61.48 - lr: 0.018750 2021-03-26 04:40:58,967 epoch 54 - iter 25/50 - loss 2.40109615 - samples/sec: 79.57 - lr: 0.018750 2021-03-26 04:41:00,997 epoch 54 - iter 30/50 - loss 2.39482352 - samples/sec: 78.91 - lr: 0.018750 2021-03-26 04:41:03,058 epoch 54 - iter 35/50 - loss 2.35816078 - samples/sec: 77.67 - lr: 0.018750 2021-03-26 04:41:05,099 epoch 54 - iter 40/50 - loss 2.32958505 - samples/sec: 78.46 - lr: 0.018750 2021-03-26 04:41:06,923 epoch 54 - iter 45/50 - loss 2.29746844 - samples/sec: 87.84 - lr: 0.018750 2021-03-26 04:41:08,791 epoch 54 - iter 50/50 - loss 2.40223788 - samples/sec: 85.72 - lr: 0.018750 2021-03-26 04:41:08,791 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:41:08,792 EPOCH 54 done: loss 2.4022 - lr 0.0187500 2021-03-26 04:41:09,622 DEV : loss 5.651618957519531 - score 0.9195 2021-03-26 04:41:09,648 BAD EPOCHS (no improvement): 3 2021-03-26 04:41:09,649 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:41:11,510 epoch 55 - iter 5/50 - loss 2.56955271 - samples/sec: 86.06 - lr: 0.018750 2021-03-26 04:41:13,555 epoch 55 - iter 10/50 - loss 2.34034249 - samples/sec: 78.29 - lr: 0.018750 2021-03-26 04:41:15,529 epoch 55 - iter 15/50 - loss 2.39607155 - samples/sec: 81.13 - lr: 0.018750 2021-03-26 04:41:17,510 epoch 55 - iter 20/50 - loss 2.40379005 - samples/sec: 80.84 - lr: 0.018750 2021-03-26 04:41:19,606 epoch 55 - iter 25/50 - loss 2.42211420 - samples/sec: 76.42 - lr: 0.018750 2021-03-26 04:41:21,654 epoch 55 - iter 30/50 - loss 2.49672683 - samples/sec: 78.15 - lr: 0.018750 2021-03-26 04:41:23,777 epoch 55 - iter 35/50 - loss 2.52185025 - samples/sec: 75.41 - lr: 0.018750 2021-03-26 04:41:25,738 epoch 55 - iter 40/50 - loss 2.51877766 - samples/sec: 81.67 - lr: 0.018750 2021-03-26 04:41:27,698 epoch 55 - iter 45/50 - loss 2.46066902 - samples/sec: 81.76 - lr: 0.018750 2021-03-26 04:41:29,483 epoch 55 - iter 50/50 - loss 2.42432467 - samples/sec: 89.70 - lr: 0.018750 2021-03-26 04:41:29,484 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:41:29,484 EPOCH 55 done: loss 2.4243 - lr 0.0187500 2021-03-26 04:41:30,304 DEV : loss 5.660974025726318 - score 0.9203 2021-03-26 04:41:30,330 BAD EPOCHS (no improvement): 4 2021-03-26 04:41:30,331 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:41:32,470 epoch 56 - iter 5/50 - loss 2.67091203 - samples/sec: 74.90 - lr: 0.009375 2021-03-26 04:41:34,414 epoch 56 - iter 10/50 - loss 2.46447575 - samples/sec: 82.34 - lr: 0.009375 2021-03-26 04:41:36,460 epoch 56 - iter 15/50 - loss 2.54704790 - samples/sec: 78.26 - lr: 0.009375 2021-03-26 04:41:38,353 epoch 56 - iter 20/50 - loss 2.60251430 - samples/sec: 84.60 - lr: 0.009375 2021-03-26 04:41:40,128 epoch 56 - iter 25/50 - loss 2.57725821 - samples/sec: 90.22 - lr: 0.009375 2021-03-26 04:41:42,056 epoch 56 - iter 30/50 - loss 2.54962186 - samples/sec: 83.10 - lr: 0.009375 2021-03-26 04:41:44,079 epoch 56 - iter 35/50 - loss 2.56064803 - samples/sec: 79.12 - lr: 0.009375 2021-03-26 04:41:46,153 epoch 56 - iter 40/50 - loss 2.56441651 - samples/sec: 77.23 - lr: 0.009375 2021-03-26 04:41:48,222 epoch 56 - iter 45/50 - loss 2.51202908 - samples/sec: 77.36 - lr: 0.009375 2021-03-26 04:41:50,244 epoch 56 - iter 50/50 - loss 2.49388498 - samples/sec: 79.22 - lr: 0.009375 2021-03-26 04:41:50,244 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:41:50,245 EPOCH 56 done: loss 2.4939 - lr 0.0093750 2021-03-26 04:41:51,083 DEV : loss 5.629773139953613 - score 0.9203 2021-03-26 04:41:51,109 BAD EPOCHS (no improvement): 1 2021-03-26 04:41:51,109 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:41:53,098 epoch 57 - iter 5/50 - loss 2.37442803 - samples/sec: 80.55 - lr: 0.009375 2021-03-26 04:41:55,213 epoch 57 - iter 10/50 - loss 2.48838272 - samples/sec: 75.69 - lr: 0.009375 2021-03-26 04:41:57,093 epoch 57 - iter 15/50 - loss 2.43360175 - samples/sec: 85.25 - lr: 0.009375 2021-03-26 04:41:58,931 epoch 57 - iter 20/50 - loss 2.46973049 - samples/sec: 87.16 - lr: 0.009375 2021-03-26 04:42:01,042 epoch 57 - iter 25/50 - loss 2.48110202 - samples/sec: 75.83 - lr: 0.009375 2021-03-26 04:42:03,177 epoch 57 - iter 30/50 - loss 2.52635504 - samples/sec: 75.02 - lr: 0.009375 2021-03-26 04:42:05,553 epoch 57 - iter 35/50 - loss 2.52847188 - samples/sec: 67.40 - lr: 0.009375 2021-03-26 04:42:07,300 epoch 57 - iter 40/50 - loss 2.49505937 - samples/sec: 91.65 - lr: 0.009375 2021-03-26 04:42:09,169 epoch 57 - iter 45/50 - loss 2.49363049 - samples/sec: 85.67 - lr: 0.009375 2021-03-26 04:42:11,128 epoch 57 - iter 50/50 - loss 2.52488782 - samples/sec: 81.79 - lr: 0.009375 2021-03-26 04:42:11,129 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:42:11,130 EPOCH 57 done: loss 2.5249 - lr 0.0093750 2021-03-26 04:42:12,062 DEV : loss 5.650582313537598 - score 0.9211 2021-03-26 04:42:12,098 BAD EPOCHS (no improvement): 2 2021-03-26 04:42:12,099 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:42:14,152 epoch 58 - iter 5/50 - loss 2.54769721 - samples/sec: 77.99 - lr: 0.009375 2021-03-26 04:42:16,131 epoch 58 - iter 10/50 - loss 2.53803582 - samples/sec: 80.96 - lr: 0.009375 2021-03-26 04:42:18,098 epoch 58 - iter 15/50 - loss 2.54988208 - samples/sec: 81.40 - lr: 0.009375 2021-03-26 04:42:19,767 epoch 58 - iter 20/50 - loss 2.41201873 - samples/sec: 95.94 - lr: 0.009375 2021-03-26 04:42:21,656 epoch 58 - iter 25/50 - loss 2.37423041 - samples/sec: 84.80 - lr: 0.009375 2021-03-26 04:42:23,469 epoch 58 - iter 30/50 - loss 2.37531073 - samples/sec: 88.32 - lr: 0.009375 2021-03-26 04:42:25,436 epoch 58 - iter 35/50 - loss 2.32408940 - samples/sec: 81.38 - lr: 0.009375 2021-03-26 04:42:27,178 epoch 58 - iter 40/50 - loss 2.32021767 - samples/sec: 91.94 - lr: 0.009375 2021-03-26 04:42:29,348 epoch 58 - iter 45/50 - loss 2.34878608 - samples/sec: 73.79 - lr: 0.009375 2021-03-26 04:42:31,262 epoch 58 - iter 50/50 - loss 2.37261997 - samples/sec: 83.68 - lr: 0.009375 2021-03-26 04:42:31,262 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:42:31,263 EPOCH 58 done: loss 2.3726 - lr 0.0093750 2021-03-26 04:42:32,056 DEV : loss 5.631865501403809 - score 0.9199 2021-03-26 04:42:32,079 BAD EPOCHS (no improvement): 3 2021-03-26 04:42:32,080 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:42:34,065 epoch 59 - iter 5/50 - loss 2.36048121 - samples/sec: 80.69 - lr: 0.009375 2021-03-26 04:42:36,004 epoch 59 - iter 10/50 - loss 2.70518186 - samples/sec: 82.74 - lr: 0.009375 2021-03-26 04:42:37,866 epoch 59 - iter 15/50 - loss 2.60348249 - samples/sec: 86.04 - lr: 0.009375 2021-03-26 04:42:39,778 epoch 59 - iter 20/50 - loss 2.60562146 - samples/sec: 83.77 - lr: 0.009375 2021-03-26 04:42:41,668 epoch 59 - iter 25/50 - loss 2.54358743 - samples/sec: 84.77 - lr: 0.009375 2021-03-26 04:42:43,594 epoch 59 - iter 30/50 - loss 2.49220409 - samples/sec: 83.17 - lr: 0.009375 2021-03-26 04:42:45,377 epoch 59 - iter 35/50 - loss 2.49576769 - samples/sec: 89.86 - lr: 0.009375 2021-03-26 04:42:47,357 epoch 59 - iter 40/50 - loss 2.47423758 - samples/sec: 80.89 - lr: 0.009375 2021-03-26 04:42:49,368 epoch 59 - iter 45/50 - loss 2.44007614 - samples/sec: 79.62 - lr: 0.009375 2021-03-26 04:42:51,451 epoch 59 - iter 50/50 - loss 2.43062379 - samples/sec: 76.90 - lr: 0.009375 2021-03-26 04:42:51,451 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:42:51,452 EPOCH 59 done: loss 2.4306 - lr 0.0093750 2021-03-26 04:42:52,262 DEV : loss 5.658109188079834 - score 0.9207 2021-03-26 04:42:52,287 BAD EPOCHS (no improvement): 4 2021-03-26 04:42:52,288 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:42:54,144 epoch 60 - iter 5/50 - loss 2.35025871 - samples/sec: 86.30 - lr: 0.004687 2021-03-26 04:42:56,046 epoch 60 - iter 10/50 - loss 2.37897037 - samples/sec: 84.19 - lr: 0.004687 2021-03-26 04:42:58,067 epoch 60 - iter 15/50 - loss 2.31608614 - samples/sec: 79.26 - lr: 0.004687 2021-03-26 04:42:59,960 epoch 60 - iter 20/50 - loss 2.26537485 - samples/sec: 84.60 - lr: 0.004687 2021-03-26 04:43:02,052 epoch 60 - iter 25/50 - loss 2.26644080 - samples/sec: 76.55 - lr: 0.004687 2021-03-26 04:43:04,049 epoch 60 - iter 30/50 - loss 2.31688050 - samples/sec: 80.19 - lr: 0.004687 2021-03-26 04:43:05,871 epoch 60 - iter 35/50 - loss 2.28219617 - samples/sec: 87.90 - lr: 0.004687 2021-03-26 04:43:07,753 epoch 60 - iter 40/50 - loss 2.31520236 - samples/sec: 85.11 - lr: 0.004687 2021-03-26 04:43:09,875 epoch 60 - iter 45/50 - loss 2.37652651 - samples/sec: 75.44 - lr: 0.004687 2021-03-26 04:43:11,657 epoch 60 - iter 50/50 - loss 2.42843126 - samples/sec: 89.89 - lr: 0.004687 2021-03-26 04:43:11,657 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:43:11,657 EPOCH 60 done: loss 2.4284 - lr 0.0046875 2021-03-26 04:43:12,453 DEV : loss 5.651933193206787 - score 0.9215 2021-03-26 04:43:12,474 BAD EPOCHS (no improvement): 1 2021-03-26 04:43:12,475 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:43:14,399 epoch 61 - iter 5/50 - loss 2.08282530 - samples/sec: 83.25 - lr: 0.004687 2021-03-26 04:43:16,399 epoch 61 - iter 10/50 - loss 2.24820324 - samples/sec: 80.05 - lr: 0.004687 2021-03-26 04:43:18,466 epoch 61 - iter 15/50 - loss 2.41197851 - samples/sec: 77.50 - lr: 0.004687 2021-03-26 04:43:20,499 epoch 61 - iter 20/50 - loss 2.40561969 - samples/sec: 78.75 - lr: 0.004687 2021-03-26 04:43:22,264 epoch 61 - iter 25/50 - loss 2.32103359 - samples/sec: 90.74 - lr: 0.004687 2021-03-26 04:43:24,306 epoch 61 - iter 30/50 - loss 2.34994412 - samples/sec: 78.46 - lr: 0.004687 2021-03-26 04:43:26,241 epoch 61 - iter 35/50 - loss 2.32774382 - samples/sec: 82.75 - lr: 0.004687 2021-03-26 04:43:28,374 epoch 61 - iter 40/50 - loss 2.37477439 - samples/sec: 75.08 - lr: 0.004687 2021-03-26 04:43:30,341 epoch 61 - iter 45/50 - loss 2.39251536 - samples/sec: 81.44 - lr: 0.004687 2021-03-26 04:43:32,049 epoch 61 - iter 50/50 - loss 2.39368558 - samples/sec: 93.81 - lr: 0.004687 2021-03-26 04:43:32,050 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:43:32,050 EPOCH 61 done: loss 2.3937 - lr 0.0046875 2021-03-26 04:43:32,862 DEV : loss 5.637948513031006 - score 0.9219 2021-03-26 04:43:32,887 BAD EPOCHS (no improvement): 0 2021-03-26 04:43:42,750 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:43:44,828 epoch 62 - iter 5/50 - loss 2.27241840 - samples/sec: 77.08 - lr: 0.004687 2021-03-26 04:43:46,844 epoch 62 - iter 10/50 - loss 2.29186729 - samples/sec: 79.50 - lr: 0.004687 2021-03-26 04:43:48,784 epoch 62 - iter 15/50 - loss 2.39521653 - samples/sec: 82.55 - lr: 0.004687 2021-03-26 04:43:50,634 epoch 62 - iter 20/50 - loss 2.37930581 - samples/sec: 86.59 - lr: 0.004687 2021-03-26 04:43:52,580 epoch 62 - iter 25/50 - loss 2.40283502 - samples/sec: 82.29 - lr: 0.004687 2021-03-26 04:43:54,497 epoch 62 - iter 30/50 - loss 2.36301094 - samples/sec: 83.55 - lr: 0.004687 2021-03-26 04:43:56,460 epoch 62 - iter 35/50 - loss 2.35278542 - samples/sec: 81.57 - lr: 0.004687 2021-03-26 04:43:58,571 epoch 62 - iter 40/50 - loss 2.35601684 - samples/sec: 75.85 - lr: 0.004687 2021-03-26 04:44:00,361 epoch 62 - iter 45/50 - loss 2.27771665 - samples/sec: 89.47 - lr: 0.004687 2021-03-26 04:44:02,302 epoch 62 - iter 50/50 - loss 2.30656911 - samples/sec: 82.48 - lr: 0.004687 2021-03-26 04:44:02,303 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:44:02,303 EPOCH 62 done: loss 2.3066 - lr 0.0046875 2021-03-26 04:44:03,125 DEV : loss 5.627078056335449 - score 0.9227 2021-03-26 04:44:03,150 BAD EPOCHS (no improvement): 0 2021-03-26 04:44:12,835 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:44:14,853 epoch 63 - iter 5/50 - loss 2.72502465 - samples/sec: 79.39 - lr: 0.004687 2021-03-26 04:44:16,844 epoch 63 - iter 10/50 - loss 2.34835476 - samples/sec: 80.43 - lr: 0.004687 2021-03-26 04:44:18,808 epoch 63 - iter 15/50 - loss 2.35418735 - samples/sec: 81.55 - lr: 0.004687 2021-03-26 04:44:20,744 epoch 63 - iter 20/50 - loss 2.42223947 - samples/sec: 82.74 - lr: 0.004687 2021-03-26 04:44:22,567 epoch 63 - iter 25/50 - loss 2.34308921 - samples/sec: 87.86 - lr: 0.004687 2021-03-26 04:44:24,548 epoch 63 - iter 30/50 - loss 2.36702642 - samples/sec: 80.82 - lr: 0.004687 2021-03-26 04:44:26,579 epoch 63 - iter 35/50 - loss 2.38747502 - samples/sec: 78.86 - lr: 0.004687 2021-03-26 04:44:28,479 epoch 63 - iter 40/50 - loss 2.41883632 - samples/sec: 84.27 - lr: 0.004687 2021-03-26 04:44:30,417 epoch 63 - iter 45/50 - loss 2.35734841 - samples/sec: 82.64 - lr: 0.004687 2021-03-26 04:44:32,130 epoch 63 - iter 50/50 - loss 2.43212230 - samples/sec: 93.49 - lr: 0.004687 2021-03-26 04:44:32,131 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:44:32,131 EPOCH 63 done: loss 2.4321 - lr 0.0046875 2021-03-26 04:44:32,941 DEV : loss 5.634641170501709 - score 0.9219 2021-03-26 04:44:32,959 BAD EPOCHS (no improvement): 1 2021-03-26 04:44:32,959 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:44:34,944 epoch 64 - iter 5/50 - loss 2.49137278 - samples/sec: 80.71 - lr: 0.004687 2021-03-26 04:44:36,845 epoch 64 - iter 10/50 - loss 2.36707425 - samples/sec: 84.24 - lr: 0.004687 2021-03-26 04:44:38,788 epoch 64 - iter 15/50 - loss 2.28359599 - samples/sec: 82.43 - lr: 0.004687 2021-03-26 04:44:40,820 epoch 64 - iter 20/50 - loss 2.38758727 - samples/sec: 78.82 - lr: 0.004687 2021-03-26 04:44:42,799 epoch 64 - iter 25/50 - loss 2.38408761 - samples/sec: 80.93 - lr: 0.004687 2021-03-26 04:44:44,723 epoch 64 - iter 30/50 - loss 2.34029110 - samples/sec: 83.23 - lr: 0.004687 2021-03-26 04:44:46,698 epoch 64 - iter 35/50 - loss 2.27839816 - samples/sec: 81.06 - lr: 0.004687 2021-03-26 04:44:48,669 epoch 64 - iter 40/50 - loss 2.30497065 - samples/sec: 81.25 - lr: 0.004687 2021-03-26 04:44:50,880 epoch 64 - iter 45/50 - loss 2.31095112 - samples/sec: 72.44 - lr: 0.004687 2021-03-26 04:44:52,645 epoch 64 - iter 50/50 - loss 2.27450335 - samples/sec: 90.75 - lr: 0.004687 2021-03-26 04:44:52,646 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:44:52,646 EPOCH 64 done: loss 2.2745 - lr 0.0046875 2021-03-26 04:44:53,447 DEV : loss 5.627133846282959 - score 0.9219 2021-03-26 04:44:53,465 BAD EPOCHS (no improvement): 2 2021-03-26 04:44:53,466 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:44:55,469 epoch 65 - iter 5/50 - loss 2.32387009 - samples/sec: 79.94 - lr: 0.004687 2021-03-26 04:44:57,327 epoch 65 - iter 10/50 - loss 2.41782620 - samples/sec: 86.23 - lr: 0.004687 2021-03-26 04:44:59,270 epoch 65 - iter 15/50 - loss 2.43123651 - samples/sec: 82.47 - lr: 0.004687 2021-03-26 04:45:01,246 epoch 65 - iter 20/50 - loss 2.31962968 - samples/sec: 81.04 - lr: 0.004687 2021-03-26 04:45:03,205 epoch 65 - iter 25/50 - loss 2.49893980 - samples/sec: 81.73 - lr: 0.004687 2021-03-26 04:45:05,153 epoch 65 - iter 30/50 - loss 2.51514649 - samples/sec: 82.21 - lr: 0.004687 2021-03-26 04:45:07,174 epoch 65 - iter 35/50 - loss 2.49580066 - samples/sec: 79.23 - lr: 0.004687 2021-03-26 04:45:09,077 epoch 65 - iter 40/50 - loss 2.48866832 - samples/sec: 84.16 - lr: 0.004687 2021-03-26 04:45:11,216 epoch 65 - iter 45/50 - loss 2.44491422 - samples/sec: 74.89 - lr: 0.004687 2021-03-26 04:45:13,019 epoch 65 - iter 50/50 - loss 2.40319876 - samples/sec: 88.88 - lr: 0.004687 2021-03-26 04:45:13,019 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:45:13,020 EPOCH 65 done: loss 2.4032 - lr 0.0046875 2021-03-26 04:45:13,845 DEV : loss 5.6327924728393555 - score 0.9211 2021-03-26 04:45:13,862 BAD EPOCHS (no improvement): 3 2021-03-26 04:45:13,863 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:45:15,852 epoch 66 - iter 5/50 - loss 2.18675711 - samples/sec: 80.51 - lr: 0.004687 2021-03-26 04:45:17,915 epoch 66 - iter 10/50 - loss 2.41399018 - samples/sec: 77.62 - lr: 0.004687 2021-03-26 04:45:19,903 epoch 66 - iter 15/50 - loss 2.34998701 - samples/sec: 80.58 - lr: 0.004687 2021-03-26 04:45:21,768 epoch 66 - iter 20/50 - loss 2.41503422 - samples/sec: 85.86 - lr: 0.004687 2021-03-26 04:45:23,785 epoch 66 - iter 25/50 - loss 2.48407470 - samples/sec: 79.36 - lr: 0.004687 2021-03-26 04:45:25,725 epoch 66 - iter 30/50 - loss 2.56181589 - samples/sec: 82.56 - lr: 0.004687 2021-03-26 04:45:27,754 epoch 66 - iter 35/50 - loss 2.55085475 - samples/sec: 78.96 - lr: 0.004687 2021-03-26 04:45:29,807 epoch 66 - iter 40/50 - loss 2.59409171 - samples/sec: 78.00 - lr: 0.004687 2021-03-26 04:45:31,803 epoch 66 - iter 45/50 - loss 2.54425696 - samples/sec: 80.21 - lr: 0.004687 2021-03-26 04:45:33,703 epoch 66 - iter 50/50 - loss 2.56617797 - samples/sec: 84.30 - lr: 0.004687 2021-03-26 04:45:33,704 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:45:33,704 EPOCH 66 done: loss 2.5662 - lr 0.0046875 2021-03-26 04:45:34,510 DEV : loss 5.638400077819824 - score 0.9219 2021-03-26 04:45:34,536 BAD EPOCHS (no improvement): 4 2021-03-26 04:45:34,536 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:45:36,572 epoch 67 - iter 5/50 - loss 2.58468990 - samples/sec: 78.70 - lr: 0.002344 2021-03-26 04:45:38,449 epoch 67 - iter 10/50 - loss 2.50182832 - samples/sec: 85.32 - lr: 0.002344 2021-03-26 04:45:40,440 epoch 67 - iter 15/50 - loss 2.59449295 - samples/sec: 80.43 - lr: 0.002344 2021-03-26 04:45:42,515 epoch 67 - iter 20/50 - loss 2.48832155 - samples/sec: 77.18 - lr: 0.002344 2021-03-26 04:45:44,387 epoch 67 - iter 25/50 - loss 2.36310293 - samples/sec: 85.54 - lr: 0.002344 2021-03-26 04:45:46,359 epoch 67 - iter 30/50 - loss 2.38655980 - samples/sec: 81.23 - lr: 0.002344 2021-03-26 04:45:48,268 epoch 67 - iter 35/50 - loss 2.30594007 - samples/sec: 83.87 - lr: 0.002344 2021-03-26 04:45:50,207 epoch 67 - iter 40/50 - loss 2.31153803 - samples/sec: 82.57 - lr: 0.002344 2021-03-26 04:45:52,310 epoch 67 - iter 45/50 - loss 2.30876457 - samples/sec: 76.18 - lr: 0.002344 2021-03-26 04:45:54,232 epoch 67 - iter 50/50 - loss 2.38392017 - samples/sec: 83.37 - lr: 0.002344 2021-03-26 04:45:54,233 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:45:54,234 EPOCH 67 done: loss 2.3839 - lr 0.0023437 2021-03-26 04:45:55,041 DEV : loss 5.640754699707031 - score 0.9223 2021-03-26 04:45:55,059 BAD EPOCHS (no improvement): 1 2021-03-26 04:45:55,060 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:45:57,046 epoch 68 - iter 5/50 - loss 2.16239529 - samples/sec: 80.64 - lr: 0.002344 2021-03-26 04:45:58,987 epoch 68 - iter 10/50 - loss 2.30322955 - samples/sec: 82.52 - lr: 0.002344 2021-03-26 04:46:00,860 epoch 68 - iter 15/50 - loss 2.18448377 - samples/sec: 85.58 - lr: 0.002344 2021-03-26 04:46:02,924 epoch 68 - iter 20/50 - loss 2.20179555 - samples/sec: 77.58 - lr: 0.002344 2021-03-26 04:46:05,128 epoch 68 - iter 25/50 - loss 2.22976829 - samples/sec: 72.65 - lr: 0.002344 2021-03-26 04:46:07,365 epoch 68 - iter 30/50 - loss 2.25753242 - samples/sec: 71.55 - lr: 0.002344 2021-03-26 04:46:09,277 epoch 68 - iter 35/50 - loss 2.28681386 - samples/sec: 83.78 - lr: 0.002344 2021-03-26 04:46:11,277 epoch 68 - iter 40/50 - loss 2.32609110 - samples/sec: 80.06 - lr: 0.002344 2021-03-26 04:46:13,328 epoch 68 - iter 45/50 - loss 2.32432895 - samples/sec: 78.07 - lr: 0.002344 2021-03-26 04:46:15,228 epoch 68 - iter 50/50 - loss 2.35555371 - samples/sec: 84.31 - lr: 0.002344 2021-03-26 04:46:15,228 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:46:15,228 EPOCH 68 done: loss 2.3556 - lr 0.0023437 2021-03-26 04:46:16,039 DEV : loss 5.634337425231934 - score 0.9219 2021-03-26 04:46:16,058 BAD EPOCHS (no improvement): 2 2021-03-26 04:46:16,058 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:46:18,204 epoch 69 - iter 5/50 - loss 2.39963605 - samples/sec: 74.63 - lr: 0.002344 2021-03-26 04:46:20,128 epoch 69 - iter 10/50 - loss 2.52453003 - samples/sec: 83.23 - lr: 0.002344 2021-03-26 04:46:22,015 epoch 69 - iter 15/50 - loss 2.39732469 - samples/sec: 84.88 - lr: 0.002344 2021-03-26 04:46:24,063 epoch 69 - iter 20/50 - loss 2.45164004 - samples/sec: 78.19 - lr: 0.002344 2021-03-26 04:46:26,350 epoch 69 - iter 25/50 - loss 2.34485009 - samples/sec: 70.01 - lr: 0.002344 2021-03-26 04:46:28,440 epoch 69 - iter 30/50 - loss 2.39694957 - samples/sec: 76.65 - lr: 0.002344 2021-03-26 04:46:30,466 epoch 69 - iter 35/50 - loss 2.38782328 - samples/sec: 79.05 - lr: 0.002344 2021-03-26 04:46:32,426 epoch 69 - iter 40/50 - loss 2.42859703 - samples/sec: 81.72 - lr: 0.002344 2021-03-26 04:46:34,551 epoch 69 - iter 45/50 - loss 2.44324458 - samples/sec: 75.37 - lr: 0.002344 2021-03-26 04:46:36,287 epoch 69 - iter 50/50 - loss 2.40381415 - samples/sec: 92.26 - lr: 0.002344 2021-03-26 04:46:36,287 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:46:36,288 EPOCH 69 done: loss 2.4038 - lr 0.0023437 2021-03-26 04:46:37,101 DEV : loss 5.629244804382324 - score 0.9227 2021-03-26 04:46:37,128 BAD EPOCHS (no improvement): 3 2021-03-26 04:46:37,128 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:46:39,318 epoch 70 - iter 5/50 - loss 2.64318256 - samples/sec: 73.14 - lr: 0.002344 2021-03-26 04:46:41,224 epoch 70 - iter 10/50 - loss 2.71038888 - samples/sec: 84.09 - lr: 0.002344 2021-03-26 04:46:43,243 epoch 70 - iter 15/50 - loss 2.52488495 - samples/sec: 79.31 - lr: 0.002344 2021-03-26 04:46:45,187 epoch 70 - iter 20/50 - loss 2.47664029 - samples/sec: 82.37 - lr: 0.002344 2021-03-26 04:46:47,639 epoch 70 - iter 25/50 - loss 2.42149183 - samples/sec: 65.34 - lr: 0.002344 2021-03-26 04:46:49,665 epoch 70 - iter 30/50 - loss 2.40650520 - samples/sec: 79.08 - lr: 0.002344 2021-03-26 04:46:51,770 epoch 70 - iter 35/50 - loss 2.41927083 - samples/sec: 76.09 - lr: 0.002344 2021-03-26 04:46:53,764 epoch 70 - iter 40/50 - loss 2.44539106 - samples/sec: 80.32 - lr: 0.002344 2021-03-26 04:46:56,201 epoch 70 - iter 45/50 - loss 2.41105876 - samples/sec: 65.72 - lr: 0.002344 2021-03-26 04:46:58,028 epoch 70 - iter 50/50 - loss 2.42444482 - samples/sec: 87.70 - lr: 0.002344 2021-03-26 04:46:58,028 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:46:58,029 EPOCH 70 done: loss 2.4244 - lr 0.0023437 2021-03-26 04:46:58,886 DEV : loss 5.630977630615234 - score 0.9227 2021-03-26 04:46:58,911 BAD EPOCHS (no improvement): 4 2021-03-26 04:46:58,912 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:47:00,991 epoch 71 - iter 5/50 - loss 2.40335622 - samples/sec: 77.04 - lr: 0.001172 2021-03-26 04:47:02,794 epoch 71 - iter 10/50 - loss 2.38454808 - samples/sec: 88.90 - lr: 0.001172 2021-03-26 04:47:04,708 epoch 71 - iter 15/50 - loss 2.48061834 - samples/sec: 83.68 - lr: 0.001172 2021-03-26 04:47:06,716 epoch 71 - iter 20/50 - loss 2.37713302 - samples/sec: 79.75 - lr: 0.001172 2021-03-26 04:47:08,771 epoch 71 - iter 25/50 - loss 2.34294545 - samples/sec: 77.92 - lr: 0.001172 2021-03-26 04:47:10,700 epoch 71 - iter 30/50 - loss 2.42969246 - samples/sec: 83.04 - lr: 0.001172 2021-03-26 04:47:12,515 epoch 71 - iter 35/50 - loss 2.39980226 - samples/sec: 88.26 - lr: 0.001172 2021-03-26 04:47:14,409 epoch 71 - iter 40/50 - loss 2.35062600 - samples/sec: 84.58 - lr: 0.001172 2021-03-26 04:47:16,304 epoch 71 - iter 45/50 - loss 2.38621088 - samples/sec: 84.48 - lr: 0.001172 2021-03-26 04:47:18,211 epoch 71 - iter 50/50 - loss 2.40086455 - samples/sec: 83.99 - lr: 0.001172 2021-03-26 04:47:18,212 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:47:18,212 EPOCH 71 done: loss 2.4009 - lr 0.0011719 2021-03-26 04:47:18,999 DEV : loss 5.630173683166504 - score 0.9231 2021-03-26 04:47:19,025 BAD EPOCHS (no improvement): 0 2021-03-26 04:47:28,586 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:47:30,506 epoch 72 - iter 5/50 - loss 2.07270017 - samples/sec: 83.44 - lr: 0.001172 2021-03-26 04:47:32,500 epoch 72 - iter 10/50 - loss 2.45771600 - samples/sec: 80.36 - lr: 0.001172 2021-03-26 04:47:34,636 epoch 72 - iter 15/50 - loss 2.47021635 - samples/sec: 74.99 - lr: 0.001172 2021-03-26 04:47:36,580 epoch 72 - iter 20/50 - loss 2.45928237 - samples/sec: 82.37 - lr: 0.001172 2021-03-26 04:47:38,553 epoch 72 - iter 25/50 - loss 2.39187438 - samples/sec: 81.18 - lr: 0.001172 2021-03-26 04:47:40,357 epoch 72 - iter 30/50 - loss 2.37344064 - samples/sec: 88.81 - lr: 0.001172 2021-03-26 04:47:42,555 epoch 72 - iter 35/50 - loss 2.39183445 - samples/sec: 72.86 - lr: 0.001172 2021-03-26 04:47:44,500 epoch 72 - iter 40/50 - loss 2.40346884 - samples/sec: 82.29 - lr: 0.001172 2021-03-26 04:47:46,573 epoch 72 - iter 45/50 - loss 2.39540144 - samples/sec: 77.26 - lr: 0.001172 2021-03-26 04:47:48,432 epoch 72 - iter 50/50 - loss 2.39621652 - samples/sec: 86.20 - lr: 0.001172 2021-03-26 04:47:48,433 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:47:48,433 EPOCH 72 done: loss 2.3962 - lr 0.0011719 2021-03-26 04:47:49,272 DEV : loss 5.62816047668457 - score 0.9223 2021-03-26 04:47:49,301 BAD EPOCHS (no improvement): 1 2021-03-26 04:47:49,302 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:47:51,341 epoch 73 - iter 5/50 - loss 2.44974518 - samples/sec: 78.55 - lr: 0.001172 2021-03-26 04:47:53,368 epoch 73 - iter 10/50 - loss 2.35539858 - samples/sec: 79.03 - lr: 0.001172 2021-03-26 04:47:55,307 epoch 73 - iter 15/50 - loss 2.29412109 - samples/sec: 82.56 - lr: 0.001172 2021-03-26 04:47:57,467 epoch 73 - iter 20/50 - loss 2.45073715 - samples/sec: 74.15 - lr: 0.001172 2021-03-26 04:47:59,439 epoch 73 - iter 25/50 - loss 2.43627182 - samples/sec: 81.18 - lr: 0.001172 2021-03-26 04:48:01,643 epoch 73 - iter 30/50 - loss 2.40200396 - samples/sec: 72.64 - lr: 0.001172 2021-03-26 04:48:03,974 epoch 73 - iter 35/50 - loss 2.32076040 - samples/sec: 68.70 - lr: 0.001172 2021-03-26 04:48:05,935 epoch 73 - iter 40/50 - loss 2.28614058 - samples/sec: 81.67 - lr: 0.001172 2021-03-26 04:48:08,167 epoch 73 - iter 45/50 - loss 2.32075121 - samples/sec: 71.74 - lr: 0.001172 2021-03-26 04:48:10,007 epoch 73 - iter 50/50 - loss 2.29757083 - samples/sec: 87.00 - lr: 0.001172 2021-03-26 04:48:10,008 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:48:10,008 EPOCH 73 done: loss 2.2976 - lr 0.0011719 2021-03-26 04:48:10,837 DEV : loss 5.6278395652771 - score 0.9223 2021-03-26 04:48:10,857 BAD EPOCHS (no improvement): 2 2021-03-26 04:48:10,857 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:48:12,812 epoch 74 - iter 5/50 - loss 1.83286681 - samples/sec: 81.90 - lr: 0.001172 2021-03-26 04:48:14,874 epoch 74 - iter 10/50 - loss 2.12402097 - samples/sec: 77.68 - lr: 0.001172 2021-03-26 04:48:16,757 epoch 74 - iter 15/50 - loss 2.21711052 - samples/sec: 85.10 - lr: 0.001172 2021-03-26 04:48:18,867 epoch 74 - iter 20/50 - loss 2.23210856 - samples/sec: 75.88 - lr: 0.001172 2021-03-26 04:48:20,803 epoch 74 - iter 25/50 - loss 2.21322096 - samples/sec: 82.73 - lr: 0.001172 2021-03-26 04:48:22,924 epoch 74 - iter 30/50 - loss 2.25693878 - samples/sec: 75.48 - lr: 0.001172 2021-03-26 04:48:24,900 epoch 74 - iter 35/50 - loss 2.25561285 - samples/sec: 81.07 - lr: 0.001172 2021-03-26 04:48:26,926 epoch 74 - iter 40/50 - loss 2.30016709 - samples/sec: 79.03 - lr: 0.001172 2021-03-26 04:48:28,873 epoch 74 - iter 45/50 - loss 2.32310001 - samples/sec: 82.27 - lr: 0.001172 2021-03-26 04:48:30,848 epoch 74 - iter 50/50 - loss 2.28137210 - samples/sec: 81.07 - lr: 0.001172 2021-03-26 04:48:30,849 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:48:30,849 EPOCH 74 done: loss 2.2814 - lr 0.0011719 2021-03-26 04:48:31,665 DEV : loss 5.626428127288818 - score 0.9227 2021-03-26 04:48:31,691 BAD EPOCHS (no improvement): 3 2021-03-26 04:48:31,692 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:48:33,724 epoch 75 - iter 5/50 - loss 2.25622938 - samples/sec: 78.81 - lr: 0.001172 2021-03-26 04:48:35,719 epoch 75 - iter 10/50 - loss 2.31577545 - samples/sec: 80.29 - lr: 0.001172 2021-03-26 04:48:37,656 epoch 75 - iter 15/50 - loss 2.28079580 - samples/sec: 82.73 - lr: 0.001172 2021-03-26 04:48:39,560 epoch 75 - iter 20/50 - loss 2.29470981 - samples/sec: 84.09 - lr: 0.001172 2021-03-26 04:48:41,617 epoch 75 - iter 25/50 - loss 2.29392951 - samples/sec: 77.84 - lr: 0.001172 2021-03-26 04:48:43,732 epoch 75 - iter 30/50 - loss 2.40328041 - samples/sec: 75.71 - lr: 0.001172 2021-03-26 04:48:45,581 epoch 75 - iter 35/50 - loss 2.35129993 - samples/sec: 86.61 - lr: 0.001172 2021-03-26 04:48:47,623 epoch 75 - iter 40/50 - loss 2.34900579 - samples/sec: 78.42 - lr: 0.001172 2021-03-26 04:48:49,596 epoch 75 - iter 45/50 - loss 2.37384658 - samples/sec: 81.22 - lr: 0.001172 2021-03-26 04:48:51,544 epoch 75 - iter 50/50 - loss 2.37896151 - samples/sec: 82.21 - lr: 0.001172 2021-03-26 04:48:51,545 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:48:51,545 EPOCH 75 done: loss 2.3790 - lr 0.0011719 2021-03-26 04:48:52,366 DEV : loss 5.630959987640381 - score 0.9223 2021-03-26 04:48:52,392 BAD EPOCHS (no improvement): 4 2021-03-26 04:48:52,393 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:48:54,461 epoch 76 - iter 5/50 - loss 2.13097723 - samples/sec: 77.45 - lr: 0.000586 2021-03-26 04:48:56,410 epoch 76 - iter 10/50 - loss 2.27908762 - samples/sec: 82.15 - lr: 0.000586 2021-03-26 04:48:58,379 epoch 76 - iter 15/50 - loss 2.36238278 - samples/sec: 81.35 - lr: 0.000586 2021-03-26 04:49:00,472 epoch 76 - iter 20/50 - loss 2.38692014 - samples/sec: 76.53 - lr: 0.000586 2021-03-26 04:49:02,415 epoch 76 - iter 25/50 - loss 2.39408958 - samples/sec: 82.47 - lr: 0.000586 2021-03-26 04:49:04,457 epoch 76 - iter 30/50 - loss 2.45271692 - samples/sec: 78.41 - lr: 0.000586 2021-03-26 04:49:06,291 epoch 76 - iter 35/50 - loss 2.41970376 - samples/sec: 87.34 - lr: 0.000586 2021-03-26 04:49:08,060 epoch 76 - iter 40/50 - loss 2.39700398 - samples/sec: 90.57 - lr: 0.000586 2021-03-26 04:49:10,153 epoch 76 - iter 45/50 - loss 2.42808259 - samples/sec: 76.57 - lr: 0.000586 2021-03-26 04:49:12,061 epoch 76 - iter 50/50 - loss 2.42875048 - samples/sec: 83.94 - lr: 0.000586 2021-03-26 04:49:12,062 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:49:12,062 EPOCH 76 done: loss 2.4288 - lr 0.0005859 2021-03-26 04:49:12,880 DEV : loss 5.630477428436279 - score 0.9219 2021-03-26 04:49:12,906 BAD EPOCHS (no improvement): 1 2021-03-26 04:49:12,907 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:49:14,791 epoch 77 - iter 5/50 - loss 1.92447205 - samples/sec: 85.01 - lr: 0.000586 2021-03-26 04:49:16,874 epoch 77 - iter 10/50 - loss 2.23509965 - samples/sec: 76.87 - lr: 0.000586 2021-03-26 04:49:18,971 epoch 77 - iter 15/50 - loss 2.45340130 - samples/sec: 76.36 - lr: 0.000586 2021-03-26 04:49:20,915 epoch 77 - iter 20/50 - loss 2.32647941 - samples/sec: 82.38 - lr: 0.000586 2021-03-26 04:49:22,837 epoch 77 - iter 25/50 - loss 2.29459015 - samples/sec: 83.34 - lr: 0.000586 2021-03-26 04:49:24,741 epoch 77 - iter 30/50 - loss 2.29548467 - samples/sec: 84.12 - lr: 0.000586 2021-03-26 04:49:26,796 epoch 77 - iter 35/50 - loss 2.29987510 - samples/sec: 77.94 - lr: 0.000586 2021-03-26 04:49:28,659 epoch 77 - iter 40/50 - loss 2.34307029 - samples/sec: 86.02 - lr: 0.000586 2021-03-26 04:49:30,601 epoch 77 - iter 45/50 - loss 2.45157159 - samples/sec: 82.45 - lr: 0.000586 2021-03-26 04:49:32,374 epoch 77 - iter 50/50 - loss 2.42382906 - samples/sec: 90.34 - lr: 0.000586 2021-03-26 04:49:32,375 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:49:32,375 EPOCH 77 done: loss 2.4238 - lr 0.0005859 2021-03-26 04:49:33,174 DEV : loss 5.631059169769287 - score 0.9219 2021-03-26 04:49:33,198 BAD EPOCHS (no improvement): 2 2021-03-26 04:49:33,199 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:49:35,186 epoch 78 - iter 5/50 - loss 2.57229795 - samples/sec: 80.60 - lr: 0.000586 2021-03-26 04:49:37,103 epoch 78 - iter 10/50 - loss 2.69179895 - samples/sec: 83.59 - lr: 0.000586 2021-03-26 04:49:39,270 epoch 78 - iter 15/50 - loss 2.49594205 - samples/sec: 73.89 - lr: 0.000586 2021-03-26 04:49:41,248 epoch 78 - iter 20/50 - loss 2.53078225 - samples/sec: 81.16 - lr: 0.000586 2021-03-26 04:49:43,871 epoch 78 - iter 25/50 - loss 2.45043825 - samples/sec: 61.05 - lr: 0.000586 2021-03-26 04:49:46,014 epoch 78 - iter 30/50 - loss 2.41474845 - samples/sec: 74.72 - lr: 0.000586 2021-03-26 04:49:48,078 epoch 78 - iter 35/50 - loss 2.42673413 - samples/sec: 77.56 - lr: 0.000586 2021-03-26 04:49:50,134 epoch 78 - iter 40/50 - loss 2.39748089 - samples/sec: 77.90 - lr: 0.000586 2021-03-26 04:49:52,214 epoch 78 - iter 45/50 - loss 2.35223733 - samples/sec: 76.97 - lr: 0.000586 2021-03-26 04:49:54,078 epoch 78 - iter 50/50 - loss 2.39032073 - samples/sec: 85.94 - lr: 0.000586 2021-03-26 04:49:54,078 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:49:54,079 EPOCH 78 done: loss 2.3903 - lr 0.0005859 2021-03-26 04:49:54,880 DEV : loss 5.631186485290527 - score 0.9219 2021-03-26 04:49:54,905 BAD EPOCHS (no improvement): 3 2021-03-26 04:49:54,906 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:49:56,860 epoch 79 - iter 5/50 - loss 2.32237132 - samples/sec: 81.96 - lr: 0.000586 2021-03-26 04:49:58,699 epoch 79 - iter 10/50 - loss 2.59322809 - samples/sec: 87.15 - lr: 0.000586 2021-03-26 04:50:00,731 epoch 79 - iter 15/50 - loss 2.58026641 - samples/sec: 78.79 - lr: 0.000586 2021-03-26 04:50:02,869 epoch 79 - iter 20/50 - loss 2.56619343 - samples/sec: 74.92 - lr: 0.000586 2021-03-26 04:50:05,041 epoch 79 - iter 25/50 - loss 2.51247024 - samples/sec: 73.74 - lr: 0.000586 2021-03-26 04:50:06,903 epoch 79 - iter 30/50 - loss 2.44964033 - samples/sec: 86.01 - lr: 0.000586 2021-03-26 04:50:08,977 epoch 79 - iter 35/50 - loss 2.48183150 - samples/sec: 77.23 - lr: 0.000586 2021-03-26 04:50:11,005 epoch 79 - iter 40/50 - loss 2.50194048 - samples/sec: 78.93 - lr: 0.000586 2021-03-26 04:50:12,895 epoch 79 - iter 45/50 - loss 2.50711123 - samples/sec: 84.78 - lr: 0.000586 2021-03-26 04:50:14,643 epoch 79 - iter 50/50 - loss 2.52246664 - samples/sec: 91.65 - lr: 0.000586 2021-03-26 04:50:14,644 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:50:14,644 EPOCH 79 done: loss 2.5225 - lr 0.0005859 2021-03-26 04:50:15,449 DEV : loss 5.631038188934326 - score 0.9219 2021-03-26 04:50:15,467 BAD EPOCHS (no improvement): 4 2021-03-26 04:50:15,468 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:50:17,448 epoch 80 - iter 5/50 - loss 2.07633672 - samples/sec: 80.89 - lr: 0.000293 2021-03-26 04:50:19,470 epoch 80 - iter 10/50 - loss 2.18866901 - samples/sec: 79.23 - lr: 0.000293 2021-03-26 04:50:21,519 epoch 80 - iter 15/50 - loss 2.38815840 - samples/sec: 78.15 - lr: 0.000293 2021-03-26 04:50:23,621 epoch 80 - iter 20/50 - loss 2.39637839 - samples/sec: 76.16 - lr: 0.000293 2021-03-26 04:50:25,735 epoch 80 - iter 25/50 - loss 2.34935032 - samples/sec: 75.78 - lr: 0.000293 2021-03-26 04:50:27,678 epoch 80 - iter 30/50 - loss 2.37969813 - samples/sec: 82.43 - lr: 0.000293 2021-03-26 04:50:29,757 epoch 80 - iter 35/50 - loss 2.39405797 - samples/sec: 76.99 - lr: 0.000293 2021-03-26 04:50:31,573 epoch 80 - iter 40/50 - loss 2.35879222 - samples/sec: 88.17 - lr: 0.000293 2021-03-26 04:50:33,602 epoch 80 - iter 45/50 - loss 2.38336645 - samples/sec: 78.92 - lr: 0.000293 2021-03-26 04:50:35,459 epoch 80 - iter 50/50 - loss 2.35637002 - samples/sec: 86.27 - lr: 0.000293 2021-03-26 04:50:35,459 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:50:35,460 EPOCH 80 done: loss 2.3564 - lr 0.0002930 2021-03-26 04:50:36,305 DEV : loss 5.630791664123535 - score 0.9219 2021-03-26 04:50:36,331 BAD EPOCHS (no improvement): 1 2021-03-26 04:50:36,331 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:50:38,254 epoch 81 - iter 5/50 - loss 2.48897581 - samples/sec: 83.31 - lr: 0.000293 2021-03-26 04:50:40,041 epoch 81 - iter 10/50 - loss 2.25514839 - samples/sec: 89.62 - lr: 0.000293 2021-03-26 04:50:42,071 epoch 81 - iter 15/50 - loss 2.26627936 - samples/sec: 78.90 - lr: 0.000293 2021-03-26 04:50:43,990 epoch 81 - iter 20/50 - loss 2.30013109 - samples/sec: 83.47 - lr: 0.000293 2021-03-26 04:50:45,953 epoch 81 - iter 25/50 - loss 2.23416708 - samples/sec: 81.57 - lr: 0.000293 2021-03-26 04:50:47,965 epoch 81 - iter 30/50 - loss 2.24264871 - samples/sec: 79.60 - lr: 0.000293 2021-03-26 04:50:49,973 epoch 81 - iter 35/50 - loss 2.27044076 - samples/sec: 79.79 - lr: 0.000293 2021-03-26 04:50:51,941 epoch 81 - iter 40/50 - loss 2.25232525 - samples/sec: 81.36 - lr: 0.000293 2021-03-26 04:50:53,968 epoch 81 - iter 45/50 - loss 2.24451494 - samples/sec: 78.99 - lr: 0.000293 2021-03-26 04:50:55,794 epoch 81 - iter 50/50 - loss 2.26902369 - samples/sec: 87.71 - lr: 0.000293 2021-03-26 04:50:55,795 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:50:55,795 EPOCH 81 done: loss 2.2690 - lr 0.0002930 2021-03-26 04:50:56,602 DEV : loss 5.631392955780029 - score 0.9219 2021-03-26 04:50:56,627 BAD EPOCHS (no improvement): 2 2021-03-26 04:50:56,628 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:50:58,539 epoch 82 - iter 5/50 - loss 1.69027870 - samples/sec: 83.80 - lr: 0.000293 2021-03-26 04:51:00,590 epoch 82 - iter 10/50 - loss 2.31625203 - samples/sec: 78.09 - lr: 0.000293 2021-03-26 04:51:02,529 epoch 82 - iter 15/50 - loss 2.20893728 - samples/sec: 82.59 - lr: 0.000293 2021-03-26 04:51:04,414 epoch 82 - iter 20/50 - loss 2.22896555 - samples/sec: 84.93 - lr: 0.000293 2021-03-26 04:51:06,511 epoch 82 - iter 25/50 - loss 2.23054180 - samples/sec: 76.38 - lr: 0.000293 2021-03-26 04:51:08,485 epoch 82 - iter 30/50 - loss 2.26120172 - samples/sec: 81.14 - lr: 0.000293 2021-03-26 04:51:10,333 epoch 82 - iter 35/50 - loss 2.23025372 - samples/sec: 86.64 - lr: 0.000293 2021-03-26 04:51:12,270 epoch 82 - iter 40/50 - loss 2.27022659 - samples/sec: 82.68 - lr: 0.000293 2021-03-26 04:51:14,213 epoch 82 - iter 45/50 - loss 2.26758277 - samples/sec: 82.42 - lr: 0.000293 2021-03-26 04:51:16,044 epoch 82 - iter 50/50 - loss 2.28401194 - samples/sec: 87.44 - lr: 0.000293 2021-03-26 04:51:16,045 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:51:16,045 EPOCH 82 done: loss 2.2840 - lr 0.0002930 2021-03-26 04:51:16,881 DEV : loss 5.632561683654785 - score 0.9219 2021-03-26 04:51:16,907 BAD EPOCHS (no improvement): 3 2021-03-26 04:51:16,908 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:51:19,242 epoch 83 - iter 5/50 - loss 2.13644736 - samples/sec: 68.62 - lr: 0.000293 2021-03-26 04:51:21,256 epoch 83 - iter 10/50 - loss 2.38941020 - samples/sec: 79.51 - lr: 0.000293 2021-03-26 04:51:23,131 epoch 83 - iter 15/50 - loss 2.38207877 - samples/sec: 85.41 - lr: 0.000293 2021-03-26 04:51:25,093 epoch 83 - iter 20/50 - loss 2.34289131 - samples/sec: 81.61 - lr: 0.000293 2021-03-26 04:51:26,872 epoch 83 - iter 25/50 - loss 2.32033530 - samples/sec: 90.10 - lr: 0.000293 2021-03-26 04:51:28,995 epoch 83 - iter 30/50 - loss 2.27905010 - samples/sec: 75.43 - lr: 0.000293 2021-03-26 04:51:30,992 epoch 83 - iter 35/50 - loss 2.28741706 - samples/sec: 80.22 - lr: 0.000293 2021-03-26 04:51:32,890 epoch 83 - iter 40/50 - loss 2.29978756 - samples/sec: 84.39 - lr: 0.000293 2021-03-26 04:51:34,879 epoch 83 - iter 45/50 - loss 2.33553922 - samples/sec: 80.52 - lr: 0.000293 2021-03-26 04:51:36,738 epoch 83 - iter 50/50 - loss 2.34764516 - samples/sec: 86.15 - lr: 0.000293 2021-03-26 04:51:36,739 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:51:36,739 EPOCH 83 done: loss 2.3476 - lr 0.0002930 2021-03-26 04:51:37,557 DEV : loss 5.633981227874756 - score 0.9219 2021-03-26 04:51:37,580 BAD EPOCHS (no improvement): 4 2021-03-26 04:51:37,581 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:51:39,566 epoch 84 - iter 5/50 - loss 2.83537569 - samples/sec: 80.69 - lr: 0.000146 2021-03-26 04:51:41,620 epoch 84 - iter 10/50 - loss 2.61301935 - samples/sec: 78.01 - lr: 0.000146 2021-03-26 04:51:44,781 epoch 84 - iter 15/50 - loss 2.47842774 - samples/sec: 50.63 - lr: 0.000146 2021-03-26 04:51:46,845 epoch 84 - iter 20/50 - loss 2.54841444 - samples/sec: 77.59 - lr: 0.000146 2021-03-26 04:51:48,858 epoch 84 - iter 25/50 - loss 2.49058902 - samples/sec: 79.60 - lr: 0.000146 2021-03-26 04:51:50,774 epoch 84 - iter 30/50 - loss 2.46437473 - samples/sec: 83.55 - lr: 0.000146 2021-03-26 04:51:52,676 epoch 84 - iter 35/50 - loss 2.44853312 - samples/sec: 84.21 - lr: 0.000146 2021-03-26 04:51:54,564 epoch 84 - iter 40/50 - loss 2.42928336 - samples/sec: 84.81 - lr: 0.000146 2021-03-26 04:51:56,550 epoch 84 - iter 45/50 - loss 2.44210018 - samples/sec: 80.63 - lr: 0.000146 2021-03-26 04:51:58,319 epoch 84 - iter 50/50 - loss 2.36999966 - samples/sec: 90.51 - lr: 0.000146 2021-03-26 04:51:58,320 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:51:58,320 EPOCH 84 done: loss 2.3700 - lr 0.0001465 2021-03-26 04:51:59,138 DEV : loss 5.633601665496826 - score 0.9219 2021-03-26 04:51:59,158 BAD EPOCHS (no improvement): 1 2021-03-26 04:51:59,159 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:52:00,936 epoch 85 - iter 5/50 - loss 2.53912203 - samples/sec: 90.07 - lr: 0.000146 2021-03-26 04:52:02,884 epoch 85 - iter 10/50 - loss 2.55454174 - samples/sec: 82.22 - lr: 0.000146 2021-03-26 04:52:05,097 epoch 85 - iter 15/50 - loss 2.37489824 - samples/sec: 72.38 - lr: 0.000146 2021-03-26 04:52:07,291 epoch 85 - iter 20/50 - loss 2.44851355 - samples/sec: 72.95 - lr: 0.000146 2021-03-26 04:52:09,311 epoch 85 - iter 25/50 - loss 2.45759084 - samples/sec: 79.33 - lr: 0.000146 2021-03-26 04:52:11,666 epoch 85 - iter 30/50 - loss 2.37488166 - samples/sec: 67.99 - lr: 0.000146 2021-03-26 04:52:13,863 epoch 85 - iter 35/50 - loss 2.32282099 - samples/sec: 72.92 - lr: 0.000146 2021-03-26 04:52:15,805 epoch 85 - iter 40/50 - loss 2.35236027 - samples/sec: 82.48 - lr: 0.000146 2021-03-26 04:52:17,833 epoch 85 - iter 45/50 - loss 2.34360903 - samples/sec: 78.96 - lr: 0.000146 2021-03-26 04:52:19,618 epoch 85 - iter 50/50 - loss 2.32948160 - samples/sec: 89.72 - lr: 0.000146 2021-03-26 04:52:19,619 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:52:19,619 EPOCH 85 done: loss 2.3295 - lr 0.0001465 2021-03-26 04:52:20,430 DEV : loss 5.6335673332214355 - score 0.9219 2021-03-26 04:52:20,454 BAD EPOCHS (no improvement): 2 2021-03-26 04:52:20,454 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:52:22,441 epoch 86 - iter 5/50 - loss 2.27040057 - samples/sec: 80.61 - lr: 0.000146 2021-03-26 04:52:24,461 epoch 86 - iter 10/50 - loss 2.20236114 - samples/sec: 79.26 - lr: 0.000146 2021-03-26 04:52:26,490 epoch 86 - iter 15/50 - loss 2.20784380 - samples/sec: 78.93 - lr: 0.000146 2021-03-26 04:52:28,401 epoch 86 - iter 20/50 - loss 2.25163724 - samples/sec: 83.80 - lr: 0.000146 2021-03-26 04:52:30,217 epoch 86 - iter 25/50 - loss 2.24298689 - samples/sec: 88.23 - lr: 0.000146 2021-03-26 04:52:32,301 epoch 86 - iter 30/50 - loss 2.31922689 - samples/sec: 76.85 - lr: 0.000146 2021-03-26 04:52:34,212 epoch 86 - iter 35/50 - loss 2.33380191 - samples/sec: 83.76 - lr: 0.000146 2021-03-26 04:52:36,177 epoch 86 - iter 40/50 - loss 2.40758548 - samples/sec: 81.50 - lr: 0.000146 2021-03-26 04:52:38,150 epoch 86 - iter 45/50 - loss 2.38650574 - samples/sec: 81.17 - lr: 0.000146 2021-03-26 04:52:39,975 epoch 86 - iter 50/50 - loss 2.44259843 - samples/sec: 87.79 - lr: 0.000146 2021-03-26 04:52:39,976 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:52:39,976 EPOCH 86 done: loss 2.4426 - lr 0.0001465 2021-03-26 04:52:40,774 DEV : loss 5.63382625579834 - score 0.9219 2021-03-26 04:52:40,799 BAD EPOCHS (no improvement): 3 2021-03-26 04:52:40,800 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:52:42,804 epoch 87 - iter 5/50 - loss 2.61555619 - samples/sec: 79.90 - lr: 0.000146 2021-03-26 04:52:44,669 epoch 87 - iter 10/50 - loss 2.35265634 - samples/sec: 85.90 - lr: 0.000146 2021-03-26 04:52:46,667 epoch 87 - iter 15/50 - loss 2.35643369 - samples/sec: 80.17 - lr: 0.000146 2021-03-26 04:52:48,625 epoch 87 - iter 20/50 - loss 2.32868401 - samples/sec: 81.79 - lr: 0.000146 2021-03-26 04:52:50,842 epoch 87 - iter 25/50 - loss 2.40707584 - samples/sec: 72.21 - lr: 0.000146 2021-03-26 04:52:53,435 epoch 87 - iter 30/50 - loss 2.38372990 - samples/sec: 61.76 - lr: 0.000146 2021-03-26 04:52:55,452 epoch 87 - iter 35/50 - loss 2.35168895 - samples/sec: 79.39 - lr: 0.000146 2021-03-26 04:52:57,491 epoch 87 - iter 40/50 - loss 2.29494644 - samples/sec: 78.54 - lr: 0.000146 2021-03-26 04:52:59,466 epoch 87 - iter 45/50 - loss 2.30548906 - samples/sec: 81.15 - lr: 0.000146 2021-03-26 04:53:01,369 epoch 87 - iter 50/50 - loss 2.35296534 - samples/sec: 84.16 - lr: 0.000146 2021-03-26 04:53:01,369 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:01,370 EPOCH 87 done: loss 2.3530 - lr 0.0001465 2021-03-26 04:53:02,177 DEV : loss 5.633864402770996 - score 0.9219 2021-03-26 04:53:02,201 BAD EPOCHS (no improvement): 4 2021-03-26 04:53:02,202 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:02,203 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:02,203 learning rate too small - quitting training! 2021-03-26 04:53:02,203 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:11,683 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:11,684 Testing using best model ... 2021-03-26 04:53:11,685 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.3_202103260417/best-model.pt 2021-03-26 04:53:19,342 0.914 2021-03-26 04:53:19,343 Results: - F-score (micro): 0.9105 - F-score (macro): 0.5251 - Accuracy (incl. no class): 0.914 By class: precision recall f1-score support PROPN 0.8824 0.9375 0.9091 16 NUM 1.0000 0.9444 0.9714 18 ADV 0.8785 0.8868 0.8826 106 PART 0.9679 0.9679 0.9679 218 VERB 0.9653 0.9586 0.9619 145 ADP 0.9804 0.9709 0.9756 103 DET 0.9434 0.9434 0.9434 53 NOUN 0.9067 0.9577 0.9315 426 CCONJ 0.9740 0.9868 0.9804 76 ADJ 0.8699 0.8231 0.8458 130 PUNCT 1.0000 1.0000 1.0000 26 AUX 0.9524 0.9524 0.9524 42 PRON 0.9706 0.9565 0.9635 207 INTJ 0.8824 1.0000 0.9375 15 SCONJ 0.9200 0.9583 0.9388 24 CONJ 1.0000 1.0000 1.0000 40 NOUN+PRON 0.7647 0.8667 0.8125 60 PUNC 1.0000 1.0000 1.0000 177 V+PRON 0.8305 0.8448 0.8376 58 DET+NOUN 0.9222 0.9881 0.9540 84 V 0.8642 0.8861 0.8750 79 EOS 1.0000 1.0000 1.0000 70 FOREIGN 1.0000 0.8571 0.9231 7 MENTION 0.8947 1.0000 0.9444 17 CONJ+PART 0.8889 0.7273 0.8000 11 CONJ+NOUN 0.5000 0.8182 0.6207 11 CONJ+ADJ 0.0000 0.0000 0.0000 3 PREP+NOUN 0.8000 0.9524 0.8696 21 ADJ+NSUFF 0.7297 0.8182 0.7714 33 CONJ+PRON 1.0000 1.0000 1.0000 8 PROG_PART+V 0.8710 0.8710 0.8710 31 PART+PRON 1.0000 0.8095 0.8947 21 FUT_PART 1.0000 1.0000 1.0000 3 PREP 0.9846 0.9846 0.9846 65 PRON+DET+NOUN+NSUFF 1.0000 1.0000 1.0000 1 DET+ADJ+NSUFF 0.5000 1.0000 0.6667 2 PREP+DET+NUM+NSUFF 1.0000 0.0000 0.0000 1 PREP+DET+NOUN 0.8462 0.9167 0.8800 12 NOUN+CASE 1.0000 0.8000 0.8889 5 PREP+PART 1.0000 1.0000 1.0000 2 NOUN+NSUFF 0.8393 0.8246 0.8319 57 FUT_PART+V 0.8889 0.6154 0.7273 13 CONJ+PREP 1.0000 0.8000 0.8889 5 ADJ+PRON 0.6667 0.1818 0.2857 11 V+PRON+PRON 0.7143 0.5556 0.6250 9 CONJ+PREP+DET+NOUN 1.0000 1.0000 1.0000 1 V+PRON+PREP+PRON 0.0000 0.0000 0.0000 3 NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 DET+NOUN+NSUFF 0.8750 0.8235 0.8485 17 CONJ+PREP+NOUN+PRON 1.0000 0.0000 0.0000 1 CONJ+NOUN+PRON 0.0000 0.0000 0.0000 2 PART+ADJ 1.0000 0.0000 0.0000 1 PART+NOUN+PRON 0.0000 0.0000 0.0000 2 CONJ+PREP+NOUN 1.0000 0.0000 0.0000 1 URL 1.0000 1.0000 1.0000 6 CONJ+FUT_PART 1.0000 0.0000 0.0000 1 PREP+DET+NOUN+NSUFF 0.5000 0.5000 0.5000 2 HASH 1.0000 1.0000 1.0000 13 PREP+ADJ 1.0000 0.0000 0.0000 1 EMOT 1.0000 0.9615 0.9804 26 CONJ+FUT_PART+V 1.0000 0.0000 0.0000 2 CONJ+NOUN+NSUFF 1.0000 0.6667 0.8000 3 DET+ADJ 1.0000 0.4545 0.6250 11 NOUN+NSUFF+PRON 0.5714 0.8000 0.6667 5 PREP+PRON 0.9091 1.0000 0.9524 20 CONJ+PREP+PART 1.0000 0.0000 0.0000 2 V+PREP+PRON 0.8333 0.5556 0.6667 9 PART+V 1.0000 0.0000 0.0000 1 ADJ+PREP+PRON 0.0000 1.0000 0.0000 0 PROG_PART+V+PREP+PRON 0.0000 0.0000 0.0000 3 PROG_PART+V+PRON 0.7500 0.9375 0.8333 16 CONJ+ADJ+CASE 1.0000 0.0000 0.0000 1 CONJ+V+PRON+PREP+PRON 0.0000 1.0000 0.0000 0 CONJ+PROG_PART+V 0.6667 1.0000 0.8000 2 CONJ+PREP+V 1.0000 0.0000 0.0000 1 PREP+V+PRON 1.0000 0.0000 0.0000 1 DET+NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 PREP+NOUN+PRON 1.0000 0.4000 0.5714 5 CONJ+DET+NOUN 1.0000 1.0000 1.0000 1 CONJ+V 0.4000 1.0000 0.5714 2 PREP+NOUN+NSUFF 0.5000 1.0000 0.6667 2 V+NEG_PART 1.0000 0.0000 0.0000 1 PROG_PART+V+PRON+PRON 0.0000 1.0000 0.0000 0 CONJ+V+PRON 0.5000 0.5000 0.5000 2 PREP+ADJ+NSUFF 1.0000 0.0000 0.0000 1 PRON+DET+NOUN 1.0000 0.0000 0.0000 1 PREP+DET 1.0000 0.0000 0.0000 1 NOUN+PREP+PRON 1.0000 0.0000 0.0000 1 PART+NOUN 1.0000 0.6000 0.7500 5 CONJ+V+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+ADJ+NSUFF 0.0000 0.0000 0.0000 1 PART+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+V+NEG_PART 0.4000 0.4000 0.4000 5 CONJ+PART+V+NEG_PART 1.0000 0.0000 0.0000 2 CONJ+PART+PREP+NEG_PART 0.0000 1.0000 0.0000 0 PART+V+PRON+NEG_PART 0.5000 0.8000 0.6154 5 FUT_PART+V+PREP+PRON 1.0000 1.0000 1.0000 1 PART+PREP+NEG_PART 1.0000 1.0000 1.0000 5 FUT_PART+V+PRON+PRON 0.0000 0.0000 0.0000 1 FUT_PART+V+PRON 0.0000 0.0000 0.0000 2 PART+PROG_PART+V+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+NSUFF 1.0000 0.0000 0.0000 1 PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 CONJ+PROG_PART+V+PRON 1.0000 1.0000 1.0000 2 ADJ+NSUFF+PRON 1.0000 1.0000 1.0000 1 PART+PROG_PART+V+NEG_PART 1.0000 0.0000 0.0000 3 PROG_PART+V+NEG_PART 0.0000 1.0000 0.0000 0 PART+PART+V+NEG_PART 1.0000 0.0000 0.0000 1 NUM+NSUFF 1.0000 1.0000 1.0000 1 CONJ+PART+V+PRON+NEG_PART 0.0000 1.0000 0.0000 0 PART+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+ADV+NSUFF 1.0000 0.0000 0.0000 1 CONJ+ADV 0.0000 1.0000 0.0000 0 CONJ+PART+ADJ 0.0000 1.0000 0.0000 0 micro avg 0.9105 0.9105 0.9105 2737 macro avg 0.7807 0.6032 0.5251 2737 weighted avg 0.9164 0.9105 0.9047 2737 2021-03-26 04:53:19,343 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:19,343 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:23,166 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC 2021-03-26 04:53:23,167 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu 2021-03-26 04:53:23,167 Dev: None 2021-03-26 04:53:23,167 Test: None 2021-03-26 04:53:23,462 Reading data from ../../Datasets_adhoc/UD_MADAR 2021-03-26 04:53:23,463 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu 2021-03-26 04:53:23,463 Dev: None 2021-03-26 04:53:23,463 Test: None 2021-03-26 04:53:23,512 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 04:53:23,513 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt 2021-03-26 04:53:23,513 Dev: None 2021-03-26 04:53:23,513 Test: None 2021-03-26 04:53:23,689 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 04:53:23,689 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt 2021-03-26 04:53:23,690 Dev: None 2021-03-26 04:53:23,690 Test: None 2021-03-26 04:53:25,752 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 04:53:25,753 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt 2021-03-26 04:53:25,753 Dev: None 2021-03-26 04:53:25,753 Test: None 2021-03-26 04:53:25,901 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 04:53:25,901 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt 2021-03-26 04:53:25,902 Dev: None 2021-03-26 04:53:25,902 Test: None 2021-03-26 04:53:26,053 Filtering long sentences 2021-03-26 04:53:26,092 MultiCorpus: 1574 train + 177 dev + 193 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences 2021-03-26 04:53:26,501 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:26,502 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): WordEmbeddings('ar') (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) (list_embedding_2): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=206, bias=True) (beta): 1.0 (weights): None (weight_tensor) None )" 2021-03-26 04:53:26,502 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:26,502 Corpus: "MultiCorpus: 1574 train + 177 dev + 193 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" 2021-03-26 04:53:26,503 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:26,503 Parameters: 2021-03-26 04:53:26,503 - learning_rate: "0.3" 2021-03-26 04:53:26,503 - mini_batch_size: "64" 2021-03-26 04:53:26,503 - patience: "3" 2021-03-26 04:53:26,504 - anneal_factor: "0.5" 2021-03-26 04:53:26,504 - max_epochs: "150" 2021-03-26 04:53:26,504 - shuffle: "True" 2021-03-26 04:53:26,504 - train_with_dev: "False" 2021-03-26 04:53:26,504 - batch_growth_annealing: "False" 2021-03-26 04:53:26,505 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:26,505 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260453" 2021-03-26 04:53:26,505 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:26,505 Device: cuda:0 2021-03-26 04:53:26,505 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:26,506 Embeddings storage mode: cpu 2021-03-26 04:53:26,507 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:28,213 epoch 1 - iter 2/25 - loss 83.97468567 - samples/sec: 75.08 - lr: 0.300000 2021-03-26 04:53:29,534 epoch 1 - iter 4/25 - loss 75.63530731 - samples/sec: 97.04 - lr: 0.300000 2021-03-26 04:53:30,856 epoch 1 - iter 6/25 - loss 72.50162125 - samples/sec: 96.91 - lr: 0.300000 2021-03-26 04:53:32,192 epoch 1 - iter 8/25 - loss 70.06266451 - samples/sec: 95.92 - lr: 0.300000 2021-03-26 04:53:33,721 epoch 1 - iter 10/25 - loss 68.25133667 - samples/sec: 83.77 - lr: 0.300000 2021-03-26 04:53:35,053 epoch 1 - iter 12/25 - loss 66.84262911 - samples/sec: 96.26 - lr: 0.300000 2021-03-26 04:53:36,500 epoch 1 - iter 14/25 - loss 66.02898734 - samples/sec: 88.51 - lr: 0.300000 2021-03-26 04:53:37,772 epoch 1 - iter 16/25 - loss 63.99918914 - samples/sec: 100.80 - lr: 0.300000 2021-03-26 04:53:39,457 epoch 1 - iter 18/25 - loss 62.62216356 - samples/sec: 76.01 - lr: 0.300000 2021-03-26 04:53:41,244 epoch 1 - iter 20/25 - loss 61.69286957 - samples/sec: 71.68 - lr: 0.300000 2021-03-26 04:53:42,598 epoch 1 - iter 22/25 - loss 60.76663035 - samples/sec: 94.66 - lr: 0.300000 2021-03-26 04:53:43,979 epoch 1 - iter 24/25 - loss 59.23835262 - samples/sec: 92.75 - lr: 0.300000 2021-03-26 04:53:44,540 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:44,540 EPOCH 1 done: loss 58.6116 - lr 0.3000000 2021-03-26 04:53:45,781 DEV : loss 42.79204559326172 - score 0.3643 2021-03-26 04:53:45,806 BAD EPOCHS (no improvement): 0 2021-03-26 04:53:55,344 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:53:56,735 epoch 2 - iter 2/25 - loss 43.24363708 - samples/sec: 92.21 - lr: 0.300000 2021-03-26 04:53:58,021 epoch 2 - iter 4/25 - loss 40.99682331 - samples/sec: 99.68 - lr: 0.300000 2021-03-26 04:53:59,501 epoch 2 - iter 6/25 - loss 40.01383527 - samples/sec: 86.57 - lr: 0.300000 2021-03-26 04:54:00,815 epoch 2 - iter 8/25 - loss 40.72169018 - samples/sec: 97.57 - lr: 0.300000 2021-03-26 04:54:02,099 epoch 2 - iter 10/25 - loss 40.42348213 - samples/sec: 99.81 - lr: 0.300000 2021-03-26 04:54:03,096 epoch 2 - iter 12/25 - loss 39.97247760 - samples/sec: 128.64 - lr: 0.300000 2021-03-26 04:54:04,152 epoch 2 - iter 14/25 - loss 39.15356309 - samples/sec: 121.35 - lr: 0.300000 2021-03-26 04:54:05,258 epoch 2 - iter 16/25 - loss 38.90821433 - samples/sec: 116.00 - lr: 0.300000 2021-03-26 04:54:06,347 epoch 2 - iter 18/25 - loss 38.21725549 - samples/sec: 117.69 - lr: 0.300000 2021-03-26 04:54:07,491 epoch 2 - iter 20/25 - loss 37.46524897 - samples/sec: 112.16 - lr: 0.300000 2021-03-26 04:54:08,417 epoch 2 - iter 22/25 - loss 36.69194898 - samples/sec: 138.43 - lr: 0.300000 2021-03-26 04:54:09,446 epoch 2 - iter 24/25 - loss 36.02944136 - samples/sec: 124.54 - lr: 0.300000 2021-03-26 04:54:09,833 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:54:09,833 EPOCH 2 done: loss 35.7397 - lr 0.3000000 2021-03-26 04:54:10,594 DEV : loss 29.750354766845703 - score 0.5246 2021-03-26 04:54:10,620 BAD EPOCHS (no improvement): 0 2021-03-26 04:54:20,425 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:54:21,484 epoch 3 - iter 2/25 - loss 32.57054138 - samples/sec: 121.30 - lr: 0.300000 2021-03-26 04:54:22,551 epoch 3 - iter 4/25 - loss 31.07599831 - samples/sec: 120.16 - lr: 0.300000 2021-03-26 04:54:23,771 epoch 3 - iter 6/25 - loss 31.57134374 - samples/sec: 105.05 - lr: 0.300000 2021-03-26 04:54:24,751 epoch 3 - iter 8/25 - loss 30.99461079 - samples/sec: 130.76 - lr: 0.300000 2021-03-26 04:54:25,700 epoch 3 - iter 10/25 - loss 30.21249199 - samples/sec: 135.03 - lr: 0.300000 2021-03-26 04:54:26,682 epoch 3 - iter 12/25 - loss 29.47812446 - samples/sec: 130.56 - lr: 0.300000 2021-03-26 04:54:27,686 epoch 3 - iter 14/25 - loss 28.78943321 - samples/sec: 127.70 - lr: 0.300000 2021-03-26 04:54:28,700 epoch 3 - iter 16/25 - loss 28.61816049 - samples/sec: 126.48 - lr: 0.300000 2021-03-26 04:54:29,711 epoch 3 - iter 18/25 - loss 28.34320969 - samples/sec: 126.82 - lr: 0.300000 2021-03-26 04:54:30,777 epoch 3 - iter 20/25 - loss 27.82139015 - samples/sec: 120.25 - lr: 0.300000 2021-03-26 04:54:31,755 epoch 3 - iter 22/25 - loss 27.48131596 - samples/sec: 131.15 - lr: 0.300000 2021-03-26 04:54:32,841 epoch 3 - iter 24/25 - loss 27.17227077 - samples/sec: 117.97 - lr: 0.300000 2021-03-26 04:54:33,275 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:54:33,275 EPOCH 3 done: loss 26.8413 - lr 0.3000000 2021-03-26 04:54:34,027 DEV : loss 19.97711753845215 - score 0.6439 2021-03-26 04:54:34,052 BAD EPOCHS (no improvement): 0 2021-03-26 04:54:43,808 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:54:45,019 epoch 4 - iter 2/25 - loss 19.83089352 - samples/sec: 105.94 - lr: 0.300000 2021-03-26 04:54:46,080 epoch 4 - iter 4/25 - loss 19.36055803 - samples/sec: 120.83 - lr: 0.300000 2021-03-26 04:54:47,235 epoch 4 - iter 6/25 - loss 20.59924157 - samples/sec: 111.03 - lr: 0.300000 2021-03-26 04:54:48,384 epoch 4 - iter 8/25 - loss 20.88819456 - samples/sec: 111.60 - lr: 0.300000 2021-03-26 04:54:49,409 epoch 4 - iter 10/25 - loss 21.39904594 - samples/sec: 125.20 - lr: 0.300000 2021-03-26 04:54:50,527 epoch 4 - iter 12/25 - loss 21.46397877 - samples/sec: 114.64 - lr: 0.300000 2021-03-26 04:54:51,584 epoch 4 - iter 14/25 - loss 21.64387935 - samples/sec: 121.24 - lr: 0.300000 2021-03-26 04:54:52,581 epoch 4 - iter 16/25 - loss 21.79656446 - samples/sec: 128.51 - lr: 0.300000 2021-03-26 04:54:53,606 epoch 4 - iter 18/25 - loss 22.06780073 - samples/sec: 125.06 - lr: 0.300000 2021-03-26 04:54:54,615 epoch 4 - iter 20/25 - loss 21.88907843 - samples/sec: 127.22 - lr: 0.300000 2021-03-26 04:54:55,736 epoch 4 - iter 22/25 - loss 21.68766655 - samples/sec: 114.33 - lr: 0.300000 2021-03-26 04:54:56,755 epoch 4 - iter 24/25 - loss 21.61118587 - samples/sec: 125.98 - lr: 0.300000 2021-03-26 04:54:57,265 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:54:57,265 EPOCH 4 done: loss 21.4647 - lr 0.3000000 2021-03-26 04:54:58,036 DEV : loss 16.356971740722656 - score 0.7141 2021-03-26 04:54:58,057 BAD EPOCHS (no improvement): 0 2021-03-26 04:55:08,244 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:55:09,312 epoch 5 - iter 2/25 - loss 19.59161377 - samples/sec: 120.15 - lr: 0.300000 2021-03-26 04:55:10,471 epoch 5 - iter 4/25 - loss 19.70559645 - samples/sec: 110.70 - lr: 0.300000 2021-03-26 04:55:11,506 epoch 5 - iter 6/25 - loss 19.31401348 - samples/sec: 123.87 - lr: 0.300000 2021-03-26 04:55:12,619 epoch 5 - iter 8/25 - loss 19.14877057 - samples/sec: 115.21 - lr: 0.300000 2021-03-26 04:55:14,014 epoch 5 - iter 10/25 - loss 18.98953533 - samples/sec: 91.90 - lr: 0.300000 2021-03-26 04:55:15,354 epoch 5 - iter 12/25 - loss 18.87965854 - samples/sec: 95.67 - lr: 0.300000 2021-03-26 04:55:16,455 epoch 5 - iter 14/25 - loss 18.58909457 - samples/sec: 116.37 - lr: 0.300000 2021-03-26 04:55:17,485 epoch 5 - iter 16/25 - loss 18.31753743 - samples/sec: 124.45 - lr: 0.300000 2021-03-26 04:55:18,589 epoch 5 - iter 18/25 - loss 18.08833377 - samples/sec: 116.17 - lr: 0.300000 2021-03-26 04:55:19,616 epoch 5 - iter 20/25 - loss 17.97942266 - samples/sec: 124.98 - lr: 0.300000 2021-03-26 04:55:20,678 epoch 5 - iter 22/25 - loss 17.93916269 - samples/sec: 120.72 - lr: 0.300000 2021-03-26 04:55:21,612 epoch 5 - iter 24/25 - loss 17.80934910 - samples/sec: 137.27 - lr: 0.300000 2021-03-26 04:55:22,034 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:55:22,036 EPOCH 5 done: loss 17.8025 - lr 0.3000000 2021-03-26 04:55:22,800 DEV : loss 13.64643669128418 - score 0.7617 2021-03-26 04:55:22,825 BAD EPOCHS (no improvement): 0 2021-03-26 04:55:32,596 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:55:33,686 epoch 6 - iter 2/25 - loss 15.59965611 - samples/sec: 117.74 - lr: 0.300000 2021-03-26 04:55:34,792 epoch 6 - iter 4/25 - loss 16.48317146 - samples/sec: 115.95 - lr: 0.300000 2021-03-26 04:55:35,785 epoch 6 - iter 6/25 - loss 16.92143854 - samples/sec: 129.11 - lr: 0.300000 2021-03-26 04:55:36,888 epoch 6 - iter 8/25 - loss 16.72712636 - samples/sec: 116.07 - lr: 0.300000 2021-03-26 04:55:37,882 epoch 6 - iter 10/25 - loss 16.70757866 - samples/sec: 129.16 - lr: 0.300000 2021-03-26 04:55:38,963 epoch 6 - iter 12/25 - loss 16.57004619 - samples/sec: 118.55 - lr: 0.300000 2021-03-26 04:55:40,075 epoch 6 - iter 14/25 - loss 16.66024719 - samples/sec: 115.27 - lr: 0.300000 2021-03-26 04:55:41,105 epoch 6 - iter 16/25 - loss 16.44063133 - samples/sec: 124.54 - lr: 0.300000 2021-03-26 04:55:42,329 epoch 6 - iter 18/25 - loss 16.27063513 - samples/sec: 104.70 - lr: 0.300000 2021-03-26 04:55:43,481 epoch 6 - iter 20/25 - loss 16.08676958 - samples/sec: 111.24 - lr: 0.300000 2021-03-26 04:55:44,554 epoch 6 - iter 22/25 - loss 15.83572531 - samples/sec: 119.47 - lr: 0.300000 2021-03-26 04:55:45,590 epoch 6 - iter 24/25 - loss 15.76636434 - samples/sec: 123.78 - lr: 0.300000 2021-03-26 04:55:45,975 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:55:45,975 EPOCH 6 done: loss 15.6612 - lr 0.3000000 2021-03-26 04:55:46,744 DEV : loss 11.731863021850586 - score 0.7912 2021-03-26 04:55:46,762 BAD EPOCHS (no improvement): 0 2021-03-26 04:55:56,548 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:55:57,537 epoch 7 - iter 2/25 - loss 16.28380728 - samples/sec: 129.78 - lr: 0.300000 2021-03-26 04:55:58,585 epoch 7 - iter 4/25 - loss 15.14429021 - samples/sec: 122.35 - lr: 0.300000 2021-03-26 04:55:59,648 epoch 7 - iter 6/25 - loss 14.99506346 - samples/sec: 120.47 - lr: 0.300000 2021-03-26 04:56:00,693 epoch 7 - iter 8/25 - loss 14.78533983 - samples/sec: 122.68 - lr: 0.300000 2021-03-26 04:56:01,754 epoch 7 - iter 10/25 - loss 14.55287571 - samples/sec: 120.78 - lr: 0.300000 2021-03-26 04:56:02,840 epoch 7 - iter 12/25 - loss 14.56592727 - samples/sec: 118.11 - lr: 0.300000 2021-03-26 04:56:03,893 epoch 7 - iter 14/25 - loss 14.49070978 - samples/sec: 121.72 - lr: 0.300000 2021-03-26 04:56:04,868 epoch 7 - iter 16/25 - loss 14.42348069 - samples/sec: 131.41 - lr: 0.300000 2021-03-26 04:56:05,909 epoch 7 - iter 18/25 - loss 14.40482251 - samples/sec: 123.29 - lr: 0.300000 2021-03-26 04:56:06,910 epoch 7 - iter 20/25 - loss 14.48805385 - samples/sec: 128.04 - lr: 0.300000 2021-03-26 04:56:08,036 epoch 7 - iter 22/25 - loss 14.39600455 - samples/sec: 113.87 - lr: 0.300000 2021-03-26 04:56:09,130 epoch 7 - iter 24/25 - loss 14.09759820 - samples/sec: 117.17 - lr: 0.300000 2021-03-26 04:56:09,678 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:56:09,680 EPOCH 7 done: loss 14.0652 - lr 0.3000000 2021-03-26 04:56:10,490 DEV : loss 11.188261985778809 - score 0.8053 2021-03-26 04:56:10,518 BAD EPOCHS (no improvement): 0 2021-03-26 04:56:20,507 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:56:21,589 epoch 8 - iter 2/25 - loss 12.58739758 - samples/sec: 118.76 - lr: 0.300000 2021-03-26 04:56:22,604 epoch 8 - iter 4/25 - loss 13.04474783 - samples/sec: 126.36 - lr: 0.300000 2021-03-26 04:56:23,632 epoch 8 - iter 6/25 - loss 12.55590312 - samples/sec: 124.70 - lr: 0.300000 2021-03-26 04:56:24,546 epoch 8 - iter 8/25 - loss 12.62532973 - samples/sec: 140.18 - lr: 0.300000 2021-03-26 04:56:25,643 epoch 8 - iter 10/25 - loss 12.26407042 - samples/sec: 116.98 - lr: 0.300000 2021-03-26 04:56:26,633 epoch 8 - iter 12/25 - loss 12.92811076 - samples/sec: 129.49 - lr: 0.300000 2021-03-26 04:56:27,553 epoch 8 - iter 14/25 - loss 12.84461852 - samples/sec: 139.37 - lr: 0.300000 2021-03-26 04:56:28,491 epoch 8 - iter 16/25 - loss 12.82955539 - samples/sec: 136.64 - lr: 0.300000 2021-03-26 04:56:29,583 epoch 8 - iter 18/25 - loss 12.95769893 - samples/sec: 117.41 - lr: 0.300000 2021-03-26 04:56:30,550 epoch 8 - iter 20/25 - loss 12.84561110 - samples/sec: 133.25 - lr: 0.300000 2021-03-26 04:56:31,578 epoch 8 - iter 22/25 - loss 12.85654727 - samples/sec: 124.80 - lr: 0.300000 2021-03-26 04:56:32,956 epoch 8 - iter 24/25 - loss 12.81559539 - samples/sec: 93.12 - lr: 0.300000 2021-03-26 04:56:33,390 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:56:33,392 EPOCH 8 done: loss 12.8214 - lr 0.3000000 2021-03-26 04:56:34,251 DEV : loss 10.346247673034668 - score 0.8115 2021-03-26 04:56:34,276 BAD EPOCHS (no improvement): 0 2021-03-26 04:56:44,165 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:56:45,225 epoch 9 - iter 2/25 - loss 12.45041800 - samples/sec: 121.07 - lr: 0.300000 2021-03-26 04:56:46,207 epoch 9 - iter 4/25 - loss 11.98195148 - samples/sec: 130.71 - lr: 0.300000 2021-03-26 04:56:47,232 epoch 9 - iter 6/25 - loss 12.15954542 - samples/sec: 124.98 - lr: 0.300000 2021-03-26 04:56:48,177 epoch 9 - iter 8/25 - loss 11.93493176 - samples/sec: 135.86 - lr: 0.300000 2021-03-26 04:56:49,332 epoch 9 - iter 10/25 - loss 12.10410681 - samples/sec: 110.98 - lr: 0.300000 2021-03-26 04:56:50,297 epoch 9 - iter 12/25 - loss 12.03921103 - samples/sec: 133.04 - lr: 0.300000 2021-03-26 04:56:51,347 epoch 9 - iter 14/25 - loss 12.02120093 - samples/sec: 122.14 - lr: 0.300000 2021-03-26 04:56:52,434 epoch 9 - iter 16/25 - loss 11.79854709 - samples/sec: 117.91 - lr: 0.300000 2021-03-26 04:56:53,404 epoch 9 - iter 18/25 - loss 11.61538373 - samples/sec: 132.20 - lr: 0.300000 2021-03-26 04:56:54,874 epoch 9 - iter 20/25 - loss 11.94448543 - samples/sec: 87.18 - lr: 0.300000 2021-03-26 04:56:56,111 epoch 9 - iter 22/25 - loss 11.81401552 - samples/sec: 103.63 - lr: 0.300000 2021-03-26 04:56:57,187 epoch 9 - iter 24/25 - loss 11.70084679 - samples/sec: 119.14 - lr: 0.300000 2021-03-26 04:56:57,702 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:56:57,704 EPOCH 9 done: loss 11.6376 - lr 0.3000000 2021-03-26 04:56:58,547 DEV : loss 9.364679336547852 - score 0.8449 2021-03-26 04:56:58,572 BAD EPOCHS (no improvement): 0 2021-03-26 04:57:08,513 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:57:09,634 epoch 10 - iter 2/25 - loss 10.44735384 - samples/sec: 114.43 - lr: 0.300000 2021-03-26 04:57:10,698 epoch 10 - iter 4/25 - loss 10.18703008 - samples/sec: 120.58 - lr: 0.300000 2021-03-26 04:57:11,751 epoch 10 - iter 6/25 - loss 10.51863972 - samples/sec: 121.84 - lr: 0.300000 2021-03-26 04:57:12,786 epoch 10 - iter 8/25 - loss 10.90037954 - samples/sec: 123.85 - lr: 0.300000 2021-03-26 04:57:13,822 epoch 10 - iter 10/25 - loss 11.50496826 - samples/sec: 123.63 - lr: 0.300000 2021-03-26 04:57:14,889 epoch 10 - iter 12/25 - loss 11.35811400 - samples/sec: 120.19 - lr: 0.300000 2021-03-26 04:57:15,945 epoch 10 - iter 14/25 - loss 11.25915929 - samples/sec: 121.32 - lr: 0.300000 2021-03-26 04:57:16,881 epoch 10 - iter 16/25 - loss 11.20458829 - samples/sec: 136.99 - lr: 0.300000 2021-03-26 04:57:17,882 epoch 10 - iter 18/25 - loss 11.15681087 - samples/sec: 128.10 - lr: 0.300000 2021-03-26 04:57:18,788 epoch 10 - iter 20/25 - loss 11.15392790 - samples/sec: 141.52 - lr: 0.300000 2021-03-26 04:57:19,925 epoch 10 - iter 22/25 - loss 11.13716615 - samples/sec: 112.77 - lr: 0.300000 2021-03-26 04:57:20,924 epoch 10 - iter 24/25 - loss 11.03530443 - samples/sec: 128.44 - lr: 0.300000 2021-03-26 04:57:21,328 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:57:21,329 EPOCH 10 done: loss 10.9343 - lr 0.3000000 2021-03-26 04:57:22,082 DEV : loss 8.626761436462402 - score 0.8509 2021-03-26 04:57:22,106 BAD EPOCHS (no improvement): 0 2021-03-26 04:57:31,973 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:57:32,981 epoch 11 - iter 2/25 - loss 9.40605307 - samples/sec: 127.39 - lr: 0.300000 2021-03-26 04:57:34,071 epoch 11 - iter 4/25 - loss 10.33374119 - samples/sec: 117.67 - lr: 0.300000 2021-03-26 04:57:35,139 epoch 11 - iter 6/25 - loss 10.77863773 - samples/sec: 120.04 - lr: 0.300000 2021-03-26 04:57:36,180 epoch 11 - iter 8/25 - loss 10.52217364 - samples/sec: 123.06 - lr: 0.300000 2021-03-26 04:57:37,194 epoch 11 - iter 10/25 - loss 10.40267553 - samples/sec: 126.47 - lr: 0.300000 2021-03-26 04:57:38,278 epoch 11 - iter 12/25 - loss 10.46414081 - samples/sec: 118.37 - lr: 0.300000 2021-03-26 04:57:39,458 epoch 11 - iter 14/25 - loss 10.36640521 - samples/sec: 108.61 - lr: 0.300000 2021-03-26 04:57:40,503 epoch 11 - iter 16/25 - loss 10.39492095 - samples/sec: 122.63 - lr: 0.300000 2021-03-26 04:57:41,447 epoch 11 - iter 18/25 - loss 10.42191193 - samples/sec: 135.87 - lr: 0.300000 2021-03-26 04:57:42,485 epoch 11 - iter 20/25 - loss 10.45917497 - samples/sec: 123.45 - lr: 0.300000 2021-03-26 04:57:43,656 epoch 11 - iter 22/25 - loss 10.40492392 - samples/sec: 109.56 - lr: 0.300000 2021-03-26 04:57:44,757 epoch 11 - iter 24/25 - loss 10.45924691 - samples/sec: 116.42 - lr: 0.300000 2021-03-26 04:57:45,237 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:57:45,239 EPOCH 11 done: loss 10.4692 - lr 0.3000000 2021-03-26 04:57:46,067 DEV : loss 8.071310997009277 - score 0.8623 2021-03-26 04:57:46,101 BAD EPOCHS (no improvement): 0 2021-03-26 04:57:55,909 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:57:57,009 epoch 12 - iter 2/25 - loss 9.19157362 - samples/sec: 116.70 - lr: 0.300000 2021-03-26 04:57:58,023 epoch 12 - iter 4/25 - loss 10.71440196 - samples/sec: 126.52 - lr: 0.300000 2021-03-26 04:57:59,154 epoch 12 - iter 6/25 - loss 10.32768408 - samples/sec: 113.33 - lr: 0.300000 2021-03-26 04:58:00,159 epoch 12 - iter 8/25 - loss 10.20629978 - samples/sec: 127.54 - lr: 0.300000 2021-03-26 04:58:01,389 epoch 12 - iter 10/25 - loss 9.83114662 - samples/sec: 104.13 - lr: 0.300000 2021-03-26 04:58:02,493 epoch 12 - iter 12/25 - loss 9.89521742 - samples/sec: 116.14 - lr: 0.300000 2021-03-26 04:58:03,608 epoch 12 - iter 14/25 - loss 9.87386608 - samples/sec: 114.93 - lr: 0.300000 2021-03-26 04:58:04,675 epoch 12 - iter 16/25 - loss 9.74479634 - samples/sec: 120.12 - lr: 0.300000 2021-03-26 04:58:05,716 epoch 12 - iter 18/25 - loss 9.72899850 - samples/sec: 123.32 - lr: 0.300000 2021-03-26 04:58:06,762 epoch 12 - iter 20/25 - loss 9.79978871 - samples/sec: 122.56 - lr: 0.300000 2021-03-26 04:58:07,844 epoch 12 - iter 22/25 - loss 9.69491967 - samples/sec: 118.46 - lr: 0.300000 2021-03-26 04:58:08,816 epoch 12 - iter 24/25 - loss 9.54800685 - samples/sec: 131.85 - lr: 0.300000 2021-03-26 04:58:09,262 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:58:09,263 EPOCH 12 done: loss 9.6618 - lr 0.3000000 2021-03-26 04:58:10,044 DEV : loss 7.919666290283203 - score 0.8589 2021-03-26 04:58:10,061 BAD EPOCHS (no improvement): 1 2021-03-26 04:58:10,062 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:58:11,195 epoch 13 - iter 2/25 - loss 8.74641275 - samples/sec: 113.13 - lr: 0.300000 2021-03-26 04:58:12,270 epoch 13 - iter 4/25 - loss 8.56014812 - samples/sec: 119.38 - lr: 0.300000 2021-03-26 04:58:13,245 epoch 13 - iter 6/25 - loss 8.30085142 - samples/sec: 131.41 - lr: 0.300000 2021-03-26 04:58:14,181 epoch 13 - iter 8/25 - loss 8.65799254 - samples/sec: 137.01 - lr: 0.300000 2021-03-26 04:58:15,270 epoch 13 - iter 10/25 - loss 8.63084922 - samples/sec: 117.73 - lr: 0.300000 2021-03-26 04:58:16,307 epoch 13 - iter 12/25 - loss 8.77607254 - samples/sec: 123.64 - lr: 0.300000 2021-03-26 04:58:17,439 epoch 13 - iter 14/25 - loss 8.87740670 - samples/sec: 113.25 - lr: 0.300000 2021-03-26 04:58:18,402 epoch 13 - iter 16/25 - loss 8.94392380 - samples/sec: 133.12 - lr: 0.300000 2021-03-26 04:58:19,401 epoch 13 - iter 18/25 - loss 8.93109330 - samples/sec: 128.22 - lr: 0.300000 2021-03-26 04:58:20,386 epoch 13 - iter 20/25 - loss 8.86445279 - samples/sec: 130.20 - lr: 0.300000 2021-03-26 04:58:21,511 epoch 13 - iter 22/25 - loss 8.93233819 - samples/sec: 114.04 - lr: 0.300000 2021-03-26 04:58:22,598 epoch 13 - iter 24/25 - loss 8.89520804 - samples/sec: 118.00 - lr: 0.300000 2021-03-26 04:58:23,028 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:58:23,028 EPOCH 13 done: loss 8.9468 - lr 0.3000000 2021-03-26 04:58:23,801 DEV : loss 7.6833624839782715 - score 0.8617 2021-03-26 04:58:23,827 BAD EPOCHS (no improvement): 2 2021-03-26 04:58:23,827 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:58:24,972 epoch 14 - iter 2/25 - loss 7.01421189 - samples/sec: 112.04 - lr: 0.300000 2021-03-26 04:58:25,993 epoch 14 - iter 4/25 - loss 7.10888994 - samples/sec: 125.71 - lr: 0.300000 2021-03-26 04:58:27,016 epoch 14 - iter 6/25 - loss 7.76643205 - samples/sec: 125.28 - lr: 0.300000 2021-03-26 04:58:28,055 epoch 14 - iter 8/25 - loss 7.94494444 - samples/sec: 123.36 - lr: 0.300000 2021-03-26 04:58:29,058 epoch 14 - iter 10/25 - loss 8.00102415 - samples/sec: 127.96 - lr: 0.300000 2021-03-26 04:58:30,062 epoch 14 - iter 12/25 - loss 8.27631696 - samples/sec: 127.63 - lr: 0.300000 2021-03-26 04:58:31,116 epoch 14 - iter 14/25 - loss 8.18886886 - samples/sec: 121.81 - lr: 0.300000 2021-03-26 04:58:32,138 epoch 14 - iter 16/25 - loss 8.23848945 - samples/sec: 125.42 - lr: 0.300000 2021-03-26 04:58:33,058 epoch 14 - iter 18/25 - loss 8.27528975 - samples/sec: 139.32 - lr: 0.300000 2021-03-26 04:58:34,087 epoch 14 - iter 20/25 - loss 8.41562710 - samples/sec: 124.63 - lr: 0.300000 2021-03-26 04:58:35,189 epoch 14 - iter 22/25 - loss 8.42097157 - samples/sec: 116.36 - lr: 0.300000 2021-03-26 04:58:36,156 epoch 14 - iter 24/25 - loss 8.52386268 - samples/sec: 132.58 - lr: 0.300000 2021-03-26 04:58:36,556 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:58:36,557 EPOCH 14 done: loss 8.5702 - lr 0.3000000 2021-03-26 04:58:37,316 DEV : loss 7.460776329040527 - score 0.8689 2021-03-26 04:58:37,342 BAD EPOCHS (no improvement): 0 2021-03-26 04:58:47,205 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:58:48,451 epoch 15 - iter 2/25 - loss 8.41517854 - samples/sec: 102.99 - lr: 0.300000 2021-03-26 04:58:49,597 epoch 15 - iter 4/25 - loss 8.74319327 - samples/sec: 111.78 - lr: 0.300000 2021-03-26 04:58:50,617 epoch 15 - iter 6/25 - loss 8.38857420 - samples/sec: 125.83 - lr: 0.300000 2021-03-26 04:58:51,743 epoch 15 - iter 8/25 - loss 8.21414596 - samples/sec: 113.80 - lr: 0.300000 2021-03-26 04:58:52,739 epoch 15 - iter 10/25 - loss 8.17341042 - samples/sec: 128.92 - lr: 0.300000 2021-03-26 04:58:53,723 epoch 15 - iter 12/25 - loss 8.06812867 - samples/sec: 130.47 - lr: 0.300000 2021-03-26 04:58:54,790 epoch 15 - iter 14/25 - loss 8.05767250 - samples/sec: 120.11 - lr: 0.300000 2021-03-26 04:58:55,810 epoch 15 - iter 16/25 - loss 7.91986740 - samples/sec: 125.62 - lr: 0.300000 2021-03-26 04:58:56,808 epoch 15 - iter 18/25 - loss 8.05734306 - samples/sec: 128.59 - lr: 0.300000 2021-03-26 04:58:57,799 epoch 15 - iter 20/25 - loss 8.08630199 - samples/sec: 129.33 - lr: 0.300000 2021-03-26 04:58:58,976 epoch 15 - iter 22/25 - loss 8.16225732 - samples/sec: 108.90 - lr: 0.300000 2021-03-26 04:59:00,036 epoch 15 - iter 24/25 - loss 8.11989534 - samples/sec: 120.91 - lr: 0.300000 2021-03-26 04:59:00,437 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:59:00,437 EPOCH 15 done: loss 8.1712 - lr 0.3000000 2021-03-26 04:59:01,181 DEV : loss 7.299462795257568 - score 0.8706 2021-03-26 04:59:01,205 BAD EPOCHS (no improvement): 0 2021-03-26 04:59:11,000 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:59:11,995 epoch 16 - iter 2/25 - loss 6.63340878 - samples/sec: 129.13 - lr: 0.300000 2021-03-26 04:59:13,026 epoch 16 - iter 4/25 - loss 7.67077863 - samples/sec: 124.28 - lr: 0.300000 2021-03-26 04:59:14,055 epoch 16 - iter 6/25 - loss 7.59115497 - samples/sec: 124.53 - lr: 0.300000 2021-03-26 04:59:15,050 epoch 16 - iter 8/25 - loss 7.71391118 - samples/sec: 128.85 - lr: 0.300000 2021-03-26 04:59:16,109 epoch 16 - iter 10/25 - loss 7.66146083 - samples/sec: 121.09 - lr: 0.300000 2021-03-26 04:59:17,075 epoch 16 - iter 12/25 - loss 7.44220650 - samples/sec: 132.90 - lr: 0.300000 2021-03-26 04:59:18,117 epoch 16 - iter 14/25 - loss 7.54290489 - samples/sec: 123.04 - lr: 0.300000 2021-03-26 04:59:19,145 epoch 16 - iter 16/25 - loss 7.69426468 - samples/sec: 124.63 - lr: 0.300000 2021-03-26 04:59:20,158 epoch 16 - iter 18/25 - loss 7.78613400 - samples/sec: 126.58 - lr: 0.300000 2021-03-26 04:59:21,273 epoch 16 - iter 20/25 - loss 7.89610012 - samples/sec: 114.96 - lr: 0.300000 2021-03-26 04:59:22,354 epoch 16 - iter 22/25 - loss 7.92611876 - samples/sec: 118.72 - lr: 0.300000 2021-03-26 04:59:23,421 epoch 16 - iter 24/25 - loss 7.84850351 - samples/sec: 120.08 - lr: 0.300000 2021-03-26 04:59:23,821 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:59:23,822 EPOCH 16 done: loss 7.8628 - lr 0.3000000 2021-03-26 04:59:24,579 DEV : loss 7.393617153167725 - score 0.8748 2021-03-26 04:59:24,604 BAD EPOCHS (no improvement): 0 2021-03-26 04:59:34,384 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:59:35,451 epoch 17 - iter 2/25 - loss 8.34224653 - samples/sec: 120.25 - lr: 0.300000 2021-03-26 04:59:36,452 epoch 17 - iter 4/25 - loss 7.54335344 - samples/sec: 128.08 - lr: 0.300000 2021-03-26 04:59:37,404 epoch 17 - iter 6/25 - loss 7.42643118 - samples/sec: 134.66 - lr: 0.300000 2021-03-26 04:59:38,517 epoch 17 - iter 8/25 - loss 7.16716546 - samples/sec: 115.23 - lr: 0.300000 2021-03-26 04:59:39,689 epoch 17 - iter 10/25 - loss 7.25732112 - samples/sec: 109.28 - lr: 0.300000 2021-03-26 04:59:40,758 epoch 17 - iter 12/25 - loss 7.16744749 - samples/sec: 119.97 - lr: 0.300000 2021-03-26 04:59:41,835 epoch 17 - iter 14/25 - loss 7.25663158 - samples/sec: 118.92 - lr: 0.300000 2021-03-26 04:59:42,817 epoch 17 - iter 16/25 - loss 7.41065523 - samples/sec: 130.45 - lr: 0.300000 2021-03-26 04:59:43,875 epoch 17 - iter 18/25 - loss 7.50303170 - samples/sec: 121.18 - lr: 0.300000 2021-03-26 04:59:44,911 epoch 17 - iter 20/25 - loss 7.51523950 - samples/sec: 123.69 - lr: 0.300000 2021-03-26 04:59:46,000 epoch 17 - iter 22/25 - loss 7.52067937 - samples/sec: 117.71 - lr: 0.300000 2021-03-26 04:59:46,976 epoch 17 - iter 24/25 - loss 7.43268249 - samples/sec: 131.49 - lr: 0.300000 2021-03-26 04:59:47,387 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:59:47,388 EPOCH 17 done: loss 7.4542 - lr 0.3000000 2021-03-26 04:59:48,170 DEV : loss 6.938924789428711 - score 0.881 2021-03-26 04:59:48,194 BAD EPOCHS (no improvement): 0 2021-03-26 04:59:57,881 ---------------------------------------------------------------------------------------------------- 2021-03-26 04:59:59,032 epoch 18 - iter 2/25 - loss 6.95253706 - samples/sec: 111.43 - lr: 0.300000 2021-03-26 04:59:59,949 epoch 18 - iter 4/25 - loss 6.14172232 - samples/sec: 139.96 - lr: 0.300000 2021-03-26 05:00:00,996 epoch 18 - iter 6/25 - loss 6.56162341 - samples/sec: 122.60 - lr: 0.300000 2021-03-26 05:00:01,986 epoch 18 - iter 8/25 - loss 6.49303353 - samples/sec: 129.52 - lr: 0.300000 2021-03-26 05:00:02,974 epoch 18 - iter 10/25 - loss 6.57700791 - samples/sec: 129.71 - lr: 0.300000 2021-03-26 05:00:04,099 epoch 18 - iter 12/25 - loss 6.94862250 - samples/sec: 113.97 - lr: 0.300000 2021-03-26 05:00:05,061 epoch 18 - iter 14/25 - loss 6.86058249 - samples/sec: 133.17 - lr: 0.300000 2021-03-26 05:00:06,167 epoch 18 - iter 16/25 - loss 6.81615442 - samples/sec: 116.05 - lr: 0.300000 2021-03-26 05:00:07,428 epoch 18 - iter 18/25 - loss 6.82973880 - samples/sec: 101.67 - lr: 0.300000 2021-03-26 05:00:08,594 epoch 18 - iter 20/25 - loss 6.90358000 - samples/sec: 109.95 - lr: 0.300000 2021-03-26 05:00:09,602 epoch 18 - iter 22/25 - loss 7.04449615 - samples/sec: 127.11 - lr: 0.300000 2021-03-26 05:00:10,567 epoch 18 - iter 24/25 - loss 7.04610141 - samples/sec: 132.83 - lr: 0.300000 2021-03-26 05:00:10,969 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:00:10,969 EPOCH 18 done: loss 7.0938 - lr 0.3000000 2021-03-26 05:00:11,734 DEV : loss 6.572293758392334 - score 0.8845 2021-03-26 05:00:11,758 BAD EPOCHS (no improvement): 0 2021-03-26 05:00:21,476 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:00:22,656 epoch 19 - iter 2/25 - loss 6.50636125 - samples/sec: 108.71 - lr: 0.300000 2021-03-26 05:00:23,684 epoch 19 - iter 4/25 - loss 6.44323480 - samples/sec: 124.77 - lr: 0.300000 2021-03-26 05:00:24,684 epoch 19 - iter 6/25 - loss 6.69485251 - samples/sec: 128.42 - lr: 0.300000 2021-03-26 05:00:25,731 epoch 19 - iter 8/25 - loss 6.91241974 - samples/sec: 122.59 - lr: 0.300000 2021-03-26 05:00:26,727 epoch 19 - iter 10/25 - loss 6.92695212 - samples/sec: 128.70 - lr: 0.300000 2021-03-26 05:00:27,693 epoch 19 - iter 12/25 - loss 6.99924886 - samples/sec: 132.68 - lr: 0.300000 2021-03-26 05:00:28,765 epoch 19 - iter 14/25 - loss 6.83471465 - samples/sec: 119.55 - lr: 0.300000 2021-03-26 05:00:29,737 epoch 19 - iter 16/25 - loss 7.05365744 - samples/sec: 131.98 - lr: 0.300000 2021-03-26 05:00:30,739 epoch 19 - iter 18/25 - loss 7.09827262 - samples/sec: 127.87 - lr: 0.300000 2021-03-26 05:00:31,759 epoch 19 - iter 20/25 - loss 7.02771170 - samples/sec: 125.77 - lr: 0.300000 2021-03-26 05:00:32,731 epoch 19 - iter 22/25 - loss 7.03901434 - samples/sec: 131.91 - lr: 0.300000 2021-03-26 05:00:33,745 epoch 19 - iter 24/25 - loss 7.00594433 - samples/sec: 126.41 - lr: 0.300000 2021-03-26 05:00:34,200 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:00:34,200 EPOCH 19 done: loss 6.8745 - lr 0.3000000 2021-03-26 05:00:34,975 DEV : loss 7.177085876464844 - score 0.8767 2021-03-26 05:00:35,003 BAD EPOCHS (no improvement): 1 2021-03-26 05:00:35,004 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:00:35,956 epoch 20 - iter 2/25 - loss 5.38716412 - samples/sec: 134.74 - lr: 0.300000 2021-03-26 05:00:36,939 epoch 20 - iter 4/25 - loss 5.85706186 - samples/sec: 130.37 - lr: 0.300000 2021-03-26 05:00:37,893 epoch 20 - iter 6/25 - loss 6.37915858 - samples/sec: 134.51 - lr: 0.300000 2021-03-26 05:00:38,895 epoch 20 - iter 8/25 - loss 6.47841519 - samples/sec: 127.89 - lr: 0.300000 2021-03-26 05:00:39,861 epoch 20 - iter 10/25 - loss 6.55083828 - samples/sec: 132.77 - lr: 0.300000 2021-03-26 05:00:40,893 epoch 20 - iter 12/25 - loss 6.38643750 - samples/sec: 124.10 - lr: 0.300000 2021-03-26 05:00:41,941 epoch 20 - iter 14/25 - loss 6.53920797 - samples/sec: 122.31 - lr: 0.300000 2021-03-26 05:00:42,988 epoch 20 - iter 16/25 - loss 6.48416033 - samples/sec: 122.47 - lr: 0.300000 2021-03-26 05:00:43,993 epoch 20 - iter 18/25 - loss 6.45129516 - samples/sec: 127.53 - lr: 0.300000 2021-03-26 05:00:45,022 epoch 20 - iter 20/25 - loss 6.51733649 - samples/sec: 124.62 - lr: 0.300000 2021-03-26 05:00:46,087 epoch 20 - iter 22/25 - loss 6.48917937 - samples/sec: 120.25 - lr: 0.300000 2021-03-26 05:00:47,154 epoch 20 - iter 24/25 - loss 6.53120399 - samples/sec: 120.22 - lr: 0.300000 2021-03-26 05:00:47,624 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:00:47,624 EPOCH 20 done: loss 6.5602 - lr 0.3000000 2021-03-26 05:00:48,393 DEV : loss 6.406036376953125 - score 0.8849 2021-03-26 05:00:48,419 BAD EPOCHS (no improvement): 0 2021-03-26 05:00:57,975 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:00:58,986 epoch 21 - iter 2/25 - loss 5.07812023 - samples/sec: 126.91 - lr: 0.300000 2021-03-26 05:00:59,931 epoch 21 - iter 4/25 - loss 5.87068987 - samples/sec: 135.69 - lr: 0.300000 2021-03-26 05:01:00,899 epoch 21 - iter 6/25 - loss 5.90569671 - samples/sec: 132.46 - lr: 0.300000 2021-03-26 05:01:02,123 epoch 21 - iter 8/25 - loss 5.89209396 - samples/sec: 104.74 - lr: 0.300000 2021-03-26 05:01:03,354 epoch 21 - iter 10/25 - loss 5.82085629 - samples/sec: 104.07 - lr: 0.300000 2021-03-26 05:01:04,387 epoch 21 - iter 12/25 - loss 6.16112550 - samples/sec: 124.28 - lr: 0.300000 2021-03-26 05:01:05,424 epoch 21 - iter 14/25 - loss 6.25945626 - samples/sec: 123.54 - lr: 0.300000 2021-03-26 05:01:06,523 epoch 21 - iter 16/25 - loss 6.27312291 - samples/sec: 116.64 - lr: 0.300000 2021-03-26 05:01:07,581 epoch 21 - iter 18/25 - loss 6.25390930 - samples/sec: 121.21 - lr: 0.300000 2021-03-26 05:01:08,642 epoch 21 - iter 20/25 - loss 6.37842085 - samples/sec: 120.83 - lr: 0.300000 2021-03-26 05:01:09,671 epoch 21 - iter 22/25 - loss 6.37292236 - samples/sec: 124.55 - lr: 0.300000 2021-03-26 05:01:10,834 epoch 21 - iter 24/25 - loss 6.36688979 - samples/sec: 110.18 - lr: 0.300000 2021-03-26 05:01:11,319 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:01:11,320 EPOCH 21 done: loss 6.3817 - lr 0.3000000 2021-03-26 05:01:12,084 DEV : loss 6.48774528503418 - score 0.8919 2021-03-26 05:01:12,109 BAD EPOCHS (no improvement): 0 2021-03-26 05:01:22,011 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:01:23,013 epoch 22 - iter 2/25 - loss 5.74258280 - samples/sec: 128.14 - lr: 0.300000 2021-03-26 05:01:24,013 epoch 22 - iter 4/25 - loss 5.54190350 - samples/sec: 128.14 - lr: 0.300000 2021-03-26 05:01:24,976 epoch 22 - iter 6/25 - loss 5.63025824 - samples/sec: 133.34 - lr: 0.300000 2021-03-26 05:01:25,973 epoch 22 - iter 8/25 - loss 5.75732350 - samples/sec: 128.54 - lr: 0.300000 2021-03-26 05:01:26,991 epoch 22 - iter 10/25 - loss 5.95047402 - samples/sec: 126.02 - lr: 0.300000 2021-03-26 05:01:27,996 epoch 22 - iter 12/25 - loss 5.88446764 - samples/sec: 127.57 - lr: 0.300000 2021-03-26 05:01:29,167 epoch 22 - iter 14/25 - loss 5.96128140 - samples/sec: 109.47 - lr: 0.300000 2021-03-26 05:01:30,173 epoch 22 - iter 16/25 - loss 5.94008636 - samples/sec: 127.36 - lr: 0.300000 2021-03-26 05:01:31,295 epoch 22 - iter 18/25 - loss 5.92830536 - samples/sec: 114.24 - lr: 0.300000 2021-03-26 05:01:32,361 epoch 22 - iter 20/25 - loss 5.93459527 - samples/sec: 120.31 - lr: 0.300000 2021-03-26 05:01:33,365 epoch 22 - iter 22/25 - loss 5.97066474 - samples/sec: 127.60 - lr: 0.300000 2021-03-26 05:01:34,466 epoch 22 - iter 24/25 - loss 5.97097691 - samples/sec: 116.40 - lr: 0.300000 2021-03-26 05:01:34,874 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:01:34,874 EPOCH 22 done: loss 6.0013 - lr 0.3000000 2021-03-26 05:01:35,606 DEV : loss 6.178506374359131 - score 0.8919 2021-03-26 05:01:35,631 BAD EPOCHS (no improvement): 0 2021-03-26 05:01:45,379 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:01:46,594 epoch 23 - iter 2/25 - loss 5.67629218 - samples/sec: 105.53 - lr: 0.300000 2021-03-26 05:01:47,589 epoch 23 - iter 4/25 - loss 5.56583500 - samples/sec: 129.01 - lr: 0.300000 2021-03-26 05:01:48,595 epoch 23 - iter 6/25 - loss 5.36302352 - samples/sec: 127.60 - lr: 0.300000 2021-03-26 05:01:49,597 epoch 23 - iter 8/25 - loss 5.49623519 - samples/sec: 127.93 - lr: 0.300000 2021-03-26 05:01:50,643 epoch 23 - iter 10/25 - loss 5.41584888 - samples/sec: 122.65 - lr: 0.300000 2021-03-26 05:01:51,595 epoch 23 - iter 12/25 - loss 5.47491487 - samples/sec: 134.79 - lr: 0.300000 2021-03-26 05:01:52,616 epoch 23 - iter 14/25 - loss 5.70795202 - samples/sec: 125.53 - lr: 0.300000 2021-03-26 05:01:53,650 epoch 23 - iter 16/25 - loss 5.80719429 - samples/sec: 123.95 - lr: 0.300000 2021-03-26 05:01:54,808 epoch 23 - iter 18/25 - loss 5.87293715 - samples/sec: 110.66 - lr: 0.300000 2021-03-26 05:01:55,896 epoch 23 - iter 20/25 - loss 5.88861842 - samples/sec: 117.84 - lr: 0.300000 2021-03-26 05:01:56,873 epoch 23 - iter 22/25 - loss 5.97894348 - samples/sec: 131.40 - lr: 0.300000 2021-03-26 05:01:57,893 epoch 23 - iter 24/25 - loss 5.95262996 - samples/sec: 125.57 - lr: 0.300000 2021-03-26 05:01:58,309 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:01:58,309 EPOCH 23 done: loss 5.9302 - lr 0.3000000 2021-03-26 05:01:59,065 DEV : loss 6.203141212463379 - score 0.8923 2021-03-26 05:01:59,090 BAD EPOCHS (no improvement): 0 2021-03-26 05:02:08,747 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:02:09,693 epoch 24 - iter 2/25 - loss 5.83838153 - samples/sec: 135.60 - lr: 0.300000 2021-03-26 05:02:10,704 epoch 24 - iter 4/25 - loss 5.43874466 - samples/sec: 126.97 - lr: 0.300000 2021-03-26 05:02:11,898 epoch 24 - iter 6/25 - loss 5.50941141 - samples/sec: 107.31 - lr: 0.300000 2021-03-26 05:02:12,935 epoch 24 - iter 8/25 - loss 5.53578192 - samples/sec: 123.72 - lr: 0.300000 2021-03-26 05:02:13,958 epoch 24 - iter 10/25 - loss 5.54665751 - samples/sec: 125.25 - lr: 0.300000 2021-03-26 05:02:15,095 epoch 24 - iter 12/25 - loss 5.65865084 - samples/sec: 112.75 - lr: 0.300000 2021-03-26 05:02:16,059 epoch 24 - iter 14/25 - loss 5.69921316 - samples/sec: 132.83 - lr: 0.300000 2021-03-26 05:02:17,018 epoch 24 - iter 16/25 - loss 5.58560240 - samples/sec: 133.82 - lr: 0.300000 2021-03-26 05:02:18,107 epoch 24 - iter 18/25 - loss 5.56636564 - samples/sec: 117.65 - lr: 0.300000 2021-03-26 05:02:19,165 epoch 24 - iter 20/25 - loss 5.66292870 - samples/sec: 121.21 - lr: 0.300000 2021-03-26 05:02:20,101 epoch 24 - iter 22/25 - loss 5.70732947 - samples/sec: 137.00 - lr: 0.300000 2021-03-26 05:02:21,104 epoch 24 - iter 24/25 - loss 5.70914570 - samples/sec: 127.80 - lr: 0.300000 2021-03-26 05:02:21,512 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:02:21,513 EPOCH 24 done: loss 5.6692 - lr 0.3000000 2021-03-26 05:02:22,271 DEV : loss 6.174157619476318 - score 0.8896 2021-03-26 05:02:22,295 BAD EPOCHS (no improvement): 1 2021-03-26 05:02:22,296 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:02:23,296 epoch 25 - iter 2/25 - loss 5.88089275 - samples/sec: 128.16 - lr: 0.300000 2021-03-26 05:02:24,282 epoch 25 - iter 4/25 - loss 5.79316306 - samples/sec: 130.07 - lr: 0.300000 2021-03-26 05:02:25,277 epoch 25 - iter 6/25 - loss 5.90174588 - samples/sec: 128.83 - lr: 0.300000 2021-03-26 05:02:26,328 epoch 25 - iter 8/25 - loss 5.62725395 - samples/sec: 121.91 - lr: 0.300000 2021-03-26 05:02:27,533 epoch 25 - iter 10/25 - loss 5.43041258 - samples/sec: 106.42 - lr: 0.300000 2021-03-26 05:02:28,504 epoch 25 - iter 12/25 - loss 5.60077206 - samples/sec: 131.97 - lr: 0.300000 2021-03-26 05:02:29,625 epoch 25 - iter 14/25 - loss 5.79898020 - samples/sec: 114.35 - lr: 0.300000 2021-03-26 05:02:30,630 epoch 25 - iter 16/25 - loss 5.70678404 - samples/sec: 127.65 - lr: 0.300000 2021-03-26 05:02:31,573 epoch 25 - iter 18/25 - loss 5.63503326 - samples/sec: 135.94 - lr: 0.300000 2021-03-26 05:02:32,604 epoch 25 - iter 20/25 - loss 5.63442178 - samples/sec: 124.30 - lr: 0.300000 2021-03-26 05:02:33,687 epoch 25 - iter 22/25 - loss 5.58001575 - samples/sec: 118.33 - lr: 0.300000 2021-03-26 05:02:34,799 epoch 25 - iter 24/25 - loss 5.57944733 - samples/sec: 115.26 - lr: 0.300000 2021-03-26 05:02:35,209 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:02:35,210 EPOCH 25 done: loss 5.6108 - lr 0.3000000 2021-03-26 05:02:35,984 DEV : loss 6.509549617767334 - score 0.8944 2021-03-26 05:02:36,023 BAD EPOCHS (no improvement): 0 2021-03-26 05:02:45,731 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:02:46,977 epoch 26 - iter 2/25 - loss 4.82402134 - samples/sec: 102.98 - lr: 0.300000 2021-03-26 05:02:47,925 epoch 26 - iter 4/25 - loss 4.93847728 - samples/sec: 135.20 - lr: 0.300000 2021-03-26 05:02:48,867 epoch 26 - iter 6/25 - loss 4.81588984 - samples/sec: 136.11 - lr: 0.300000 2021-03-26 05:02:49,851 epoch 26 - iter 8/25 - loss 4.91123033 - samples/sec: 130.28 - lr: 0.300000 2021-03-26 05:02:50,956 epoch 26 - iter 10/25 - loss 5.07624397 - samples/sec: 115.98 - lr: 0.300000 2021-03-26 05:02:51,992 epoch 26 - iter 12/25 - loss 5.22970748 - samples/sec: 123.72 - lr: 0.300000 2021-03-26 05:02:53,125 epoch 26 - iter 14/25 - loss 5.27702308 - samples/sec: 113.17 - lr: 0.300000 2021-03-26 05:02:54,070 epoch 26 - iter 16/25 - loss 5.36155292 - samples/sec: 135.67 - lr: 0.300000 2021-03-26 05:02:55,220 epoch 26 - iter 18/25 - loss 5.42107211 - samples/sec: 111.37 - lr: 0.300000 2021-03-26 05:02:56,224 epoch 26 - iter 20/25 - loss 5.44072261 - samples/sec: 128.42 - lr: 0.300000 2021-03-26 05:02:57,386 epoch 26 - iter 22/25 - loss 5.39594821 - samples/sec: 110.36 - lr: 0.300000 2021-03-26 05:02:58,489 epoch 26 - iter 24/25 - loss 5.33946991 - samples/sec: 116.16 - lr: 0.300000 2021-03-26 05:02:58,856 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:02:58,856 EPOCH 26 done: loss 5.3082 - lr 0.3000000 2021-03-26 05:03:00,949 DEV : loss 6.412842750549316 - score 0.8935 2021-03-26 05:03:00,966 BAD EPOCHS (no improvement): 1 2021-03-26 05:03:00,967 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:03:01,938 epoch 27 - iter 2/25 - loss 5.76803780 - samples/sec: 132.05 - lr: 0.300000 2021-03-26 05:03:02,874 epoch 27 - iter 4/25 - loss 5.79881656 - samples/sec: 136.92 - lr: 0.300000 2021-03-26 05:03:03,970 epoch 27 - iter 6/25 - loss 5.63473813 - samples/sec: 116.95 - lr: 0.300000 2021-03-26 05:03:05,029 epoch 27 - iter 8/25 - loss 5.36606169 - samples/sec: 121.11 - lr: 0.300000 2021-03-26 05:03:06,019 epoch 27 - iter 10/25 - loss 5.34177275 - samples/sec: 129.56 - lr: 0.300000 2021-03-26 05:03:07,115 epoch 27 - iter 12/25 - loss 5.12930922 - samples/sec: 116.96 - lr: 0.300000 2021-03-26 05:03:08,042 epoch 27 - iter 14/25 - loss 5.18538295 - samples/sec: 138.48 - lr: 0.300000 2021-03-26 05:03:09,057 epoch 27 - iter 16/25 - loss 5.25303575 - samples/sec: 126.31 - lr: 0.300000 2021-03-26 05:03:10,077 epoch 27 - iter 18/25 - loss 5.25039638 - samples/sec: 125.59 - lr: 0.300000 2021-03-26 05:03:11,030 epoch 27 - iter 20/25 - loss 5.24911377 - samples/sec: 134.58 - lr: 0.300000 2021-03-26 05:03:11,958 epoch 27 - iter 22/25 - loss 5.26660614 - samples/sec: 138.14 - lr: 0.300000 2021-03-26 05:03:12,919 epoch 27 - iter 24/25 - loss 5.24369309 - samples/sec: 133.49 - lr: 0.300000 2021-03-26 05:03:13,378 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:03:13,379 EPOCH 27 done: loss 5.2047 - lr 0.3000000 2021-03-26 05:03:14,128 DEV : loss 6.1164445877075195 - score 0.8931 2021-03-26 05:03:14,153 BAD EPOCHS (no improvement): 2 2021-03-26 05:03:14,153 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:03:15,200 epoch 28 - iter 2/25 - loss 4.54348111 - samples/sec: 122.45 - lr: 0.300000 2021-03-26 05:03:16,213 epoch 28 - iter 4/25 - loss 4.49753290 - samples/sec: 126.69 - lr: 0.300000 2021-03-26 05:03:17,257 epoch 28 - iter 6/25 - loss 4.49302026 - samples/sec: 122.77 - lr: 0.300000 2021-03-26 05:03:18,217 epoch 28 - iter 8/25 - loss 4.60087708 - samples/sec: 133.58 - lr: 0.300000 2021-03-26 05:03:19,189 epoch 28 - iter 10/25 - loss 4.63158011 - samples/sec: 131.77 - lr: 0.300000 2021-03-26 05:03:20,103 epoch 28 - iter 12/25 - loss 4.91706479 - samples/sec: 140.25 - lr: 0.300000 2021-03-26 05:03:21,115 epoch 28 - iter 14/25 - loss 5.02842866 - samples/sec: 126.71 - lr: 0.300000 2021-03-26 05:03:22,361 epoch 28 - iter 16/25 - loss 5.06549254 - samples/sec: 102.89 - lr: 0.300000 2021-03-26 05:03:23,390 epoch 28 - iter 18/25 - loss 5.10844946 - samples/sec: 124.53 - lr: 0.300000 2021-03-26 05:03:24,395 epoch 28 - iter 20/25 - loss 5.04149504 - samples/sec: 127.52 - lr: 0.300000 2021-03-26 05:03:25,527 epoch 28 - iter 22/25 - loss 5.04856537 - samples/sec: 113.23 - lr: 0.300000 2021-03-26 05:03:26,533 epoch 28 - iter 24/25 - loss 5.06248623 - samples/sec: 127.57 - lr: 0.300000 2021-03-26 05:03:26,959 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:03:26,960 EPOCH 28 done: loss 5.0528 - lr 0.3000000 2021-03-26 05:03:27,725 DEV : loss 6.103049278259277 - score 0.8976 2021-03-26 05:03:27,749 BAD EPOCHS (no improvement): 0 2021-03-26 05:03:37,303 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:03:38,451 epoch 29 - iter 2/25 - loss 5.05517960 - samples/sec: 111.65 - lr: 0.300000 2021-03-26 05:03:39,498 epoch 29 - iter 4/25 - loss 4.97563481 - samples/sec: 122.59 - lr: 0.300000 2021-03-26 05:03:40,585 epoch 29 - iter 6/25 - loss 4.55425203 - samples/sec: 117.81 - lr: 0.300000 2021-03-26 05:03:41,632 epoch 29 - iter 8/25 - loss 4.85152861 - samples/sec: 123.08 - lr: 0.300000 2021-03-26 05:03:42,650 epoch 29 - iter 10/25 - loss 4.92433279 - samples/sec: 125.88 - lr: 0.300000 2021-03-26 05:03:43,563 epoch 29 - iter 12/25 - loss 4.80069878 - samples/sec: 140.71 - lr: 0.300000 2021-03-26 05:03:44,568 epoch 29 - iter 14/25 - loss 4.84453751 - samples/sec: 127.38 - lr: 0.300000 2021-03-26 05:03:45,615 epoch 29 - iter 16/25 - loss 4.81823514 - samples/sec: 122.51 - lr: 0.300000 2021-03-26 05:03:46,566 epoch 29 - iter 18/25 - loss 4.79057773 - samples/sec: 134.84 - lr: 0.300000 2021-03-26 05:03:47,589 epoch 29 - iter 20/25 - loss 4.94235287 - samples/sec: 125.26 - lr: 0.300000 2021-03-26 05:03:48,599 epoch 29 - iter 22/25 - loss 5.10795288 - samples/sec: 126.84 - lr: 0.300000 2021-03-26 05:03:49,627 epoch 29 - iter 24/25 - loss 5.05398512 - samples/sec: 124.76 - lr: 0.300000 2021-03-26 05:03:50,017 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:03:50,019 EPOCH 29 done: loss 5.0790 - lr 0.3000000 2021-03-26 05:03:50,770 DEV : loss 6.255495071411133 - score 0.8982 2021-03-26 05:03:50,796 BAD EPOCHS (no improvement): 0 2021-03-26 05:04:00,750 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:04:01,870 epoch 30 - iter 2/25 - loss 3.96568513 - samples/sec: 114.60 - lr: 0.300000 2021-03-26 05:04:02,933 epoch 30 - iter 4/25 - loss 4.38202250 - samples/sec: 120.73 - lr: 0.300000 2021-03-26 05:04:03,852 epoch 30 - iter 6/25 - loss 4.26282032 - samples/sec: 139.72 - lr: 0.300000 2021-03-26 05:04:04,888 epoch 30 - iter 8/25 - loss 4.35874718 - samples/sec: 123.69 - lr: 0.300000 2021-03-26 05:04:05,801 epoch 30 - iter 10/25 - loss 4.36084557 - samples/sec: 140.40 - lr: 0.300000 2021-03-26 05:04:06,790 epoch 30 - iter 12/25 - loss 4.43777708 - samples/sec: 129.54 - lr: 0.300000 2021-03-26 05:04:07,807 epoch 30 - iter 14/25 - loss 4.34066623 - samples/sec: 126.10 - lr: 0.300000 2021-03-26 05:04:08,783 epoch 30 - iter 16/25 - loss 4.34390225 - samples/sec: 131.35 - lr: 0.300000 2021-03-26 05:04:09,779 epoch 30 - iter 18/25 - loss 4.47116148 - samples/sec: 128.85 - lr: 0.300000 2021-03-26 05:04:10,815 epoch 30 - iter 20/25 - loss 4.57056917 - samples/sec: 123.76 - lr: 0.300000 2021-03-26 05:04:11,991 epoch 30 - iter 22/25 - loss 4.66263837 - samples/sec: 109.11 - lr: 0.300000 2021-03-26 05:04:13,097 epoch 30 - iter 24/25 - loss 4.72658858 - samples/sec: 115.89 - lr: 0.300000 2021-03-26 05:04:13,528 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:04:13,529 EPOCH 30 done: loss 4.7034 - lr 0.3000000 2021-03-26 05:04:14,287 DEV : loss 5.959897518157959 - score 0.9017 2021-03-26 05:04:14,312 BAD EPOCHS (no improvement): 0 2021-03-26 05:04:23,959 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:04:25,058 epoch 31 - iter 2/25 - loss 4.17308486 - samples/sec: 116.66 - lr: 0.300000 2021-03-26 05:04:25,998 epoch 31 - iter 4/25 - loss 4.29541880 - samples/sec: 136.45 - lr: 0.300000 2021-03-26 05:04:27,346 epoch 31 - iter 6/25 - loss 4.24963518 - samples/sec: 95.08 - lr: 0.300000 2021-03-26 05:04:28,591 epoch 31 - iter 8/25 - loss 4.32354948 - samples/sec: 102.98 - lr: 0.300000 2021-03-26 05:04:29,657 epoch 31 - iter 10/25 - loss 4.43648040 - samples/sec: 120.20 - lr: 0.300000 2021-03-26 05:04:30,689 epoch 31 - iter 12/25 - loss 4.37655514 - samples/sec: 124.25 - lr: 0.300000 2021-03-26 05:04:31,947 epoch 31 - iter 14/25 - loss 4.42733589 - samples/sec: 101.93 - lr: 0.300000 2021-03-26 05:04:32,934 epoch 31 - iter 16/25 - loss 4.45457639 - samples/sec: 129.87 - lr: 0.300000 2021-03-26 05:04:33,980 epoch 31 - iter 18/25 - loss 4.51251603 - samples/sec: 122.48 - lr: 0.300000 2021-03-26 05:04:35,034 epoch 31 - iter 20/25 - loss 4.54872774 - samples/sec: 121.62 - lr: 0.300000 2021-03-26 05:04:36,119 epoch 31 - iter 22/25 - loss 4.63483950 - samples/sec: 118.36 - lr: 0.300000 2021-03-26 05:04:37,054 epoch 31 - iter 24/25 - loss 4.68269272 - samples/sec: 137.37 - lr: 0.300000 2021-03-26 05:04:37,503 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:04:37,504 EPOCH 31 done: loss 4.6874 - lr 0.3000000 2021-03-26 05:04:38,262 DEV : loss 6.077559471130371 - score 0.9038 2021-03-26 05:04:38,282 BAD EPOCHS (no improvement): 0 2021-03-26 05:04:47,950 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:04:48,988 epoch 32 - iter 2/25 - loss 5.09888327 - samples/sec: 123.58 - lr: 0.300000 2021-03-26 05:04:49,981 epoch 32 - iter 4/25 - loss 5.04619509 - samples/sec: 129.18 - lr: 0.300000 2021-03-26 05:04:51,035 epoch 32 - iter 6/25 - loss 4.74786079 - samples/sec: 121.64 - lr: 0.300000 2021-03-26 05:04:52,048 epoch 32 - iter 8/25 - loss 4.76913765 - samples/sec: 126.53 - lr: 0.300000 2021-03-26 05:04:52,962 epoch 32 - iter 10/25 - loss 4.60072637 - samples/sec: 140.27 - lr: 0.300000 2021-03-26 05:04:53,981 epoch 32 - iter 12/25 - loss 4.57425706 - samples/sec: 125.71 - lr: 0.300000 2021-03-26 05:04:54,919 epoch 32 - iter 14/25 - loss 4.58698058 - samples/sec: 136.65 - lr: 0.300000 2021-03-26 05:04:55,956 epoch 32 - iter 16/25 - loss 4.55232823 - samples/sec: 123.64 - lr: 0.300000 2021-03-26 05:04:56,900 epoch 32 - iter 18/25 - loss 4.48894657 - samples/sec: 135.83 - lr: 0.300000 2021-03-26 05:04:58,000 epoch 32 - iter 20/25 - loss 4.55172141 - samples/sec: 116.53 - lr: 0.300000 2021-03-26 05:04:58,967 epoch 32 - iter 22/25 - loss 4.54604585 - samples/sec: 132.47 - lr: 0.300000 2021-03-26 05:04:59,905 epoch 32 - iter 24/25 - loss 4.52481222 - samples/sec: 136.66 - lr: 0.300000 2021-03-26 05:05:00,313 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:05:00,314 EPOCH 32 done: loss 4.4789 - lr 0.3000000 2021-03-26 05:05:01,059 DEV : loss 6.159491539001465 - score 0.8985 2021-03-26 05:05:01,078 BAD EPOCHS (no improvement): 1 2021-03-26 05:05:01,078 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:05:01,999 epoch 33 - iter 2/25 - loss 3.86642694 - samples/sec: 139.32 - lr: 0.300000 2021-03-26 05:05:03,054 epoch 33 - iter 4/25 - loss 4.42698538 - samples/sec: 121.51 - lr: 0.300000 2021-03-26 05:05:04,068 epoch 33 - iter 6/25 - loss 4.18415781 - samples/sec: 126.40 - lr: 0.300000 2021-03-26 05:05:05,177 epoch 33 - iter 8/25 - loss 4.36480805 - samples/sec: 115.51 - lr: 0.300000 2021-03-26 05:05:06,154 epoch 33 - iter 10/25 - loss 4.28511436 - samples/sec: 131.13 - lr: 0.300000 2021-03-26 05:05:07,122 epoch 33 - iter 12/25 - loss 4.38803158 - samples/sec: 132.48 - lr: 0.300000 2021-03-26 05:05:08,060 epoch 33 - iter 14/25 - loss 4.41996484 - samples/sec: 136.69 - lr: 0.300000 2021-03-26 05:05:09,097 epoch 33 - iter 16/25 - loss 4.32795495 - samples/sec: 123.60 - lr: 0.300000 2021-03-26 05:05:10,130 epoch 33 - iter 18/25 - loss 4.39254861 - samples/sec: 124.11 - lr: 0.300000 2021-03-26 05:05:11,213 epoch 33 - iter 20/25 - loss 4.43446009 - samples/sec: 118.29 - lr: 0.300000 2021-03-26 05:05:12,223 epoch 33 - iter 22/25 - loss 4.50511661 - samples/sec: 126.89 - lr: 0.300000 2021-03-26 05:05:13,226 epoch 33 - iter 24/25 - loss 4.46563493 - samples/sec: 127.92 - lr: 0.300000 2021-03-26 05:05:13,599 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:05:13,600 EPOCH 33 done: loss 4.4243 - lr 0.3000000 2021-03-26 05:05:14,365 DEV : loss 6.339288234710693 - score 0.8939 2021-03-26 05:05:14,387 BAD EPOCHS (no improvement): 2 2021-03-26 05:05:14,388 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:05:15,475 epoch 34 - iter 2/25 - loss 3.79593897 - samples/sec: 117.93 - lr: 0.300000 2021-03-26 05:05:16,628 epoch 34 - iter 4/25 - loss 4.36681736 - samples/sec: 111.14 - lr: 0.300000 2021-03-26 05:05:17,738 epoch 34 - iter 6/25 - loss 4.18173194 - samples/sec: 115.66 - lr: 0.300000 2021-03-26 05:05:18,795 epoch 34 - iter 8/25 - loss 4.21293190 - samples/sec: 121.30 - lr: 0.300000 2021-03-26 05:05:19,875 epoch 34 - iter 10/25 - loss 4.08440013 - samples/sec: 118.62 - lr: 0.300000 2021-03-26 05:05:20,904 epoch 34 - iter 12/25 - loss 4.21117107 - samples/sec: 124.54 - lr: 0.300000 2021-03-26 05:05:21,982 epoch 34 - iter 14/25 - loss 4.21686561 - samples/sec: 118.88 - lr: 0.300000 2021-03-26 05:05:22,980 epoch 34 - iter 16/25 - loss 4.20642594 - samples/sec: 128.56 - lr: 0.300000 2021-03-26 05:05:24,036 epoch 34 - iter 18/25 - loss 4.21764451 - samples/sec: 121.44 - lr: 0.300000 2021-03-26 05:05:25,148 epoch 34 - iter 20/25 - loss 4.20252523 - samples/sec: 115.30 - lr: 0.300000 2021-03-26 05:05:26,171 epoch 34 - iter 22/25 - loss 4.26032110 - samples/sec: 125.37 - lr: 0.300000 2021-03-26 05:05:27,258 epoch 34 - iter 24/25 - loss 4.23303785 - samples/sec: 117.90 - lr: 0.300000 2021-03-26 05:05:27,672 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:05:27,673 EPOCH 34 done: loss 4.2135 - lr 0.3000000 2021-03-26 05:05:28,422 DEV : loss 6.024802207946777 - score 0.9017 2021-03-26 05:05:28,447 BAD EPOCHS (no improvement): 3 2021-03-26 05:05:28,448 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:05:29,518 epoch 35 - iter 2/25 - loss 3.52251232 - samples/sec: 119.74 - lr: 0.300000 2021-03-26 05:05:30,678 epoch 35 - iter 4/25 - loss 3.63564587 - samples/sec: 110.56 - lr: 0.300000 2021-03-26 05:05:31,841 epoch 35 - iter 6/25 - loss 3.83008993 - samples/sec: 110.14 - lr: 0.300000 2021-03-26 05:05:33,198 epoch 35 - iter 8/25 - loss 3.89975199 - samples/sec: 94.44 - lr: 0.300000 2021-03-26 05:05:34,371 epoch 35 - iter 10/25 - loss 3.91242580 - samples/sec: 109.36 - lr: 0.300000 2021-03-26 05:05:35,294 epoch 35 - iter 12/25 - loss 3.84081417 - samples/sec: 138.87 - lr: 0.300000 2021-03-26 05:05:36,202 epoch 35 - iter 14/25 - loss 3.87259611 - samples/sec: 141.21 - lr: 0.300000 2021-03-26 05:05:37,299 epoch 35 - iter 16/25 - loss 3.97355936 - samples/sec: 116.82 - lr: 0.300000 2021-03-26 05:05:38,579 epoch 35 - iter 18/25 - loss 4.02878156 - samples/sec: 100.12 - lr: 0.300000 2021-03-26 05:05:39,728 epoch 35 - iter 20/25 - loss 4.05654567 - samples/sec: 111.58 - lr: 0.300000 2021-03-26 05:05:40,751 epoch 35 - iter 22/25 - loss 4.08147327 - samples/sec: 125.42 - lr: 0.300000 2021-03-26 05:05:41,753 epoch 35 - iter 24/25 - loss 4.07749189 - samples/sec: 127.96 - lr: 0.300000 2021-03-26 05:05:42,202 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:05:42,203 EPOCH 35 done: loss 4.0435 - lr 0.3000000 2021-03-26 05:05:42,955 DEV : loss 6.006406784057617 - score 0.9026 2021-03-26 05:05:42,976 BAD EPOCHS (no improvement): 4 2021-03-26 05:05:42,977 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:05:43,943 epoch 36 - iter 2/25 - loss 3.29583621 - samples/sec: 132.70 - lr: 0.150000 2021-03-26 05:05:45,019 epoch 36 - iter 4/25 - loss 3.62722439 - samples/sec: 119.20 - lr: 0.150000 2021-03-26 05:05:46,059 epoch 36 - iter 6/25 - loss 3.59074521 - samples/sec: 123.38 - lr: 0.150000 2021-03-26 05:05:47,147 epoch 36 - iter 8/25 - loss 3.63234273 - samples/sec: 117.75 - lr: 0.150000 2021-03-26 05:05:48,360 epoch 36 - iter 10/25 - loss 3.72383468 - samples/sec: 105.67 - lr: 0.150000 2021-03-26 05:05:49,397 epoch 36 - iter 12/25 - loss 3.77632584 - samples/sec: 123.76 - lr: 0.150000 2021-03-26 05:05:50,348 epoch 36 - iter 14/25 - loss 3.86353767 - samples/sec: 134.94 - lr: 0.150000 2021-03-26 05:05:51,284 epoch 36 - iter 16/25 - loss 3.91810854 - samples/sec: 137.10 - lr: 0.150000 2021-03-26 05:05:52,391 epoch 36 - iter 18/25 - loss 3.85365452 - samples/sec: 115.85 - lr: 0.150000 2021-03-26 05:05:53,380 epoch 36 - iter 20/25 - loss 3.83702444 - samples/sec: 129.71 - lr: 0.150000 2021-03-26 05:05:54,475 epoch 36 - iter 22/25 - loss 3.82520224 - samples/sec: 117.11 - lr: 0.150000 2021-03-26 05:05:55,555 epoch 36 - iter 24/25 - loss 3.77670812 - samples/sec: 118.77 - lr: 0.150000 2021-03-26 05:05:55,938 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:05:55,938 EPOCH 36 done: loss 3.7885 - lr 0.1500000 2021-03-26 05:05:56,689 DEV : loss 6.025773525238037 - score 0.9013 2021-03-26 05:05:56,714 BAD EPOCHS (no improvement): 1 2021-03-26 05:05:56,714 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:05:57,683 epoch 37 - iter 2/25 - loss 3.20568120 - samples/sec: 132.44 - lr: 0.150000 2021-03-26 05:05:58,701 epoch 37 - iter 4/25 - loss 2.85169643 - samples/sec: 125.95 - lr: 0.150000 2021-03-26 05:05:59,780 epoch 37 - iter 6/25 - loss 3.11694094 - samples/sec: 118.87 - lr: 0.150000 2021-03-26 05:06:00,850 epoch 37 - iter 8/25 - loss 3.43193850 - samples/sec: 119.83 - lr: 0.150000 2021-03-26 05:06:01,955 epoch 37 - iter 10/25 - loss 3.53840156 - samples/sec: 116.04 - lr: 0.150000 2021-03-26 05:06:02,916 epoch 37 - iter 12/25 - loss 3.46605579 - samples/sec: 133.40 - lr: 0.150000 2021-03-26 05:06:03,936 epoch 37 - iter 14/25 - loss 3.47069495 - samples/sec: 125.70 - lr: 0.150000 2021-03-26 05:06:04,923 epoch 37 - iter 16/25 - loss 3.50348461 - samples/sec: 129.89 - lr: 0.150000 2021-03-26 05:06:05,931 epoch 37 - iter 18/25 - loss 3.45909891 - samples/sec: 127.19 - lr: 0.150000 2021-03-26 05:06:06,966 epoch 37 - iter 20/25 - loss 3.50957373 - samples/sec: 123.97 - lr: 0.150000 2021-03-26 05:06:08,015 epoch 37 - iter 22/25 - loss 3.49967953 - samples/sec: 122.23 - lr: 0.150000 2021-03-26 05:06:09,128 epoch 37 - iter 24/25 - loss 3.52551559 - samples/sec: 115.20 - lr: 0.150000 2021-03-26 05:06:09,511 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:06:09,511 EPOCH 37 done: loss 3.5291 - lr 0.1500000 2021-03-26 05:06:10,271 DEV : loss 5.989836692810059 - score 0.9054 2021-03-26 05:06:10,296 BAD EPOCHS (no improvement): 0 2021-03-26 05:06:20,120 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:06:21,127 epoch 38 - iter 2/25 - loss 3.95863259 - samples/sec: 127.34 - lr: 0.150000 2021-03-26 05:06:22,137 epoch 38 - iter 4/25 - loss 3.60193974 - samples/sec: 126.85 - lr: 0.150000 2021-03-26 05:06:23,140 epoch 38 - iter 6/25 - loss 3.31770647 - samples/sec: 127.85 - lr: 0.150000 2021-03-26 05:06:24,141 epoch 38 - iter 8/25 - loss 3.34797689 - samples/sec: 128.04 - lr: 0.150000 2021-03-26 05:06:25,187 epoch 38 - iter 10/25 - loss 3.29749174 - samples/sec: 122.57 - lr: 0.150000 2021-03-26 05:06:26,342 epoch 38 - iter 12/25 - loss 3.26623889 - samples/sec: 111.07 - lr: 0.150000 2021-03-26 05:06:27,446 epoch 38 - iter 14/25 - loss 3.33318295 - samples/sec: 116.26 - lr: 0.150000 2021-03-26 05:06:28,490 epoch 38 - iter 16/25 - loss 3.37746768 - samples/sec: 122.81 - lr: 0.150000 2021-03-26 05:06:29,449 epoch 38 - iter 18/25 - loss 3.37950455 - samples/sec: 133.70 - lr: 0.150000 2021-03-26 05:06:30,652 epoch 38 - iter 20/25 - loss 3.36337690 - samples/sec: 106.49 - lr: 0.150000 2021-03-26 05:06:31,791 epoch 38 - iter 22/25 - loss 3.31866727 - samples/sec: 112.52 - lr: 0.150000 2021-03-26 05:06:32,804 epoch 38 - iter 24/25 - loss 3.36319639 - samples/sec: 126.51 - lr: 0.150000 2021-03-26 05:06:33,246 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:06:33,248 EPOCH 38 done: loss 3.3952 - lr 0.1500000 2021-03-26 05:06:34,037 DEV : loss 5.92034912109375 - score 0.9099 2021-03-26 05:06:34,057 BAD EPOCHS (no improvement): 0 2021-03-26 05:06:43,932 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:06:44,984 epoch 39 - iter 2/25 - loss 3.76524377 - samples/sec: 122.01 - lr: 0.150000 2021-03-26 05:06:46,028 epoch 39 - iter 4/25 - loss 3.60626566 - samples/sec: 122.73 - lr: 0.150000 2021-03-26 05:06:47,252 epoch 39 - iter 6/25 - loss 3.51154840 - samples/sec: 104.70 - lr: 0.150000 2021-03-26 05:06:48,586 epoch 39 - iter 8/25 - loss 3.43775126 - samples/sec: 96.04 - lr: 0.150000 2021-03-26 05:06:49,906 epoch 39 - iter 10/25 - loss 3.43897688 - samples/sec: 97.05 - lr: 0.150000 2021-03-26 05:06:51,086 epoch 39 - iter 12/25 - loss 3.35562142 - samples/sec: 108.60 - lr: 0.150000 2021-03-26 05:06:52,566 epoch 39 - iter 14/25 - loss 3.44103054 - samples/sec: 86.66 - lr: 0.150000 2021-03-26 05:06:53,504 epoch 39 - iter 16/25 - loss 3.37010135 - samples/sec: 136.61 - lr: 0.150000 2021-03-26 05:06:54,547 epoch 39 - iter 18/25 - loss 3.43329452 - samples/sec: 122.89 - lr: 0.150000 2021-03-26 05:06:55,544 epoch 39 - iter 20/25 - loss 3.41735572 - samples/sec: 128.64 - lr: 0.150000 2021-03-26 05:06:56,626 epoch 39 - iter 22/25 - loss 3.38750473 - samples/sec: 118.54 - lr: 0.150000 2021-03-26 05:06:57,596 epoch 39 - iter 24/25 - loss 3.35841246 - samples/sec: 132.05 - lr: 0.150000 2021-03-26 05:06:58,018 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:06:58,018 EPOCH 39 done: loss 3.3819 - lr 0.1500000 2021-03-26 05:06:58,842 DEV : loss 5.770626068115234 - score 0.9104 2021-03-26 05:06:58,865 BAD EPOCHS (no improvement): 0 2021-03-26 05:07:08,812 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:07:09,950 epoch 40 - iter 2/25 - loss 3.43545771 - samples/sec: 112.83 - lr: 0.150000 2021-03-26 05:07:11,157 epoch 40 - iter 4/25 - loss 3.34994274 - samples/sec: 106.27 - lr: 0.150000 2021-03-26 05:07:12,161 epoch 40 - iter 6/25 - loss 3.34000186 - samples/sec: 127.67 - lr: 0.150000 2021-03-26 05:07:13,157 epoch 40 - iter 8/25 - loss 3.21268988 - samples/sec: 128.82 - lr: 0.150000 2021-03-26 05:07:14,151 epoch 40 - iter 10/25 - loss 3.27631202 - samples/sec: 128.79 - lr: 0.150000 2021-03-26 05:07:15,085 epoch 40 - iter 12/25 - loss 3.40312175 - samples/sec: 137.31 - lr: 0.150000 2021-03-26 05:07:16,061 epoch 40 - iter 14/25 - loss 3.38169825 - samples/sec: 131.38 - lr: 0.150000 2021-03-26 05:07:17,056 epoch 40 - iter 16/25 - loss 3.36957914 - samples/sec: 128.78 - lr: 0.150000 2021-03-26 05:07:18,091 epoch 40 - iter 18/25 - loss 3.34364626 - samples/sec: 123.75 - lr: 0.150000 2021-03-26 05:07:19,111 epoch 40 - iter 20/25 - loss 3.28600924 - samples/sec: 125.76 - lr: 0.150000 2021-03-26 05:07:20,175 epoch 40 - iter 22/25 - loss 3.27949734 - samples/sec: 120.46 - lr: 0.150000 2021-03-26 05:07:21,140 epoch 40 - iter 24/25 - loss 3.24670504 - samples/sec: 133.07 - lr: 0.150000 2021-03-26 05:07:21,528 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:07:21,529 EPOCH 40 done: loss 3.2736 - lr 0.1500000 2021-03-26 05:07:22,266 DEV : loss 5.83436393737793 - score 0.9116 2021-03-26 05:07:22,289 BAD EPOCHS (no improvement): 0 2021-03-26 05:07:31,972 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:07:33,047 epoch 41 - iter 2/25 - loss 3.27510595 - samples/sec: 119.23 - lr: 0.150000 2021-03-26 05:07:34,066 epoch 41 - iter 4/25 - loss 3.29064929 - samples/sec: 125.80 - lr: 0.150000 2021-03-26 05:07:35,004 epoch 41 - iter 6/25 - loss 3.27098417 - samples/sec: 136.91 - lr: 0.150000 2021-03-26 05:07:36,059 epoch 41 - iter 8/25 - loss 3.24700919 - samples/sec: 121.42 - lr: 0.150000 2021-03-26 05:07:37,197 epoch 41 - iter 10/25 - loss 3.20525589 - samples/sec: 112.81 - lr: 0.150000 2021-03-26 05:07:38,498 epoch 41 - iter 12/25 - loss 3.20550714 - samples/sec: 98.53 - lr: 0.150000 2021-03-26 05:07:39,991 epoch 41 - iter 14/25 - loss 3.20661204 - samples/sec: 85.82 - lr: 0.150000 2021-03-26 05:07:41,300 epoch 41 - iter 16/25 - loss 3.22897214 - samples/sec: 97.93 - lr: 0.150000 2021-03-26 05:07:42,835 epoch 41 - iter 18/25 - loss 3.26956442 - samples/sec: 83.52 - lr: 0.150000 2021-03-26 05:07:44,192 epoch 41 - iter 20/25 - loss 3.26194243 - samples/sec: 94.54 - lr: 0.150000 2021-03-26 05:07:45,789 epoch 41 - iter 22/25 - loss 3.24391865 - samples/sec: 80.23 - lr: 0.150000 2021-03-26 05:07:46,808 epoch 41 - iter 24/25 - loss 3.29610110 - samples/sec: 125.85 - lr: 0.150000 2021-03-26 05:07:47,187 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:07:47,188 EPOCH 41 done: loss 3.3268 - lr 0.1500000 2021-03-26 05:07:48,069 DEV : loss 5.8652191162109375 - score 0.9042 2021-03-26 05:07:48,104 BAD EPOCHS (no improvement): 1 2021-03-26 05:07:48,104 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:07:49,110 epoch 42 - iter 2/25 - loss 2.96759343 - samples/sec: 127.58 - lr: 0.150000 2021-03-26 05:07:50,155 epoch 42 - iter 4/25 - loss 2.98074931 - samples/sec: 122.72 - lr: 0.150000 2021-03-26 05:07:51,080 epoch 42 - iter 6/25 - loss 3.09812117 - samples/sec: 138.57 - lr: 0.150000 2021-03-26 05:07:51,987 epoch 42 - iter 8/25 - loss 3.03092873 - samples/sec: 141.37 - lr: 0.150000 2021-03-26 05:07:52,996 epoch 42 - iter 10/25 - loss 3.01937761 - samples/sec: 127.09 - lr: 0.150000 2021-03-26 05:07:53,907 epoch 42 - iter 12/25 - loss 3.12206964 - samples/sec: 140.65 - lr: 0.150000 2021-03-26 05:07:54,944 epoch 42 - iter 14/25 - loss 3.05444547 - samples/sec: 123.65 - lr: 0.150000 2021-03-26 05:07:56,077 epoch 42 - iter 16/25 - loss 3.09004906 - samples/sec: 113.25 - lr: 0.150000 2021-03-26 05:07:57,070 epoch 42 - iter 18/25 - loss 3.17406662 - samples/sec: 129.02 - lr: 0.150000 2021-03-26 05:07:58,145 epoch 42 - iter 20/25 - loss 3.12834384 - samples/sec: 119.22 - lr: 0.150000 2021-03-26 05:07:59,282 epoch 42 - iter 22/25 - loss 3.16719803 - samples/sec: 112.72 - lr: 0.150000 2021-03-26 05:08:00,216 epoch 42 - iter 24/25 - loss 3.17727866 - samples/sec: 137.52 - lr: 0.150000 2021-03-26 05:08:00,642 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:08:00,643 EPOCH 42 done: loss 3.2355 - lr 0.1500000 2021-03-26 05:08:01,393 DEV : loss 6.138169765472412 - score 0.9087 2021-03-26 05:08:01,418 BAD EPOCHS (no improvement): 2 2021-03-26 05:08:01,419 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:08:02,314 epoch 43 - iter 2/25 - loss 2.71440363 - samples/sec: 143.30 - lr: 0.150000 2021-03-26 05:08:03,386 epoch 43 - iter 4/25 - loss 2.60436940 - samples/sec: 119.47 - lr: 0.150000 2021-03-26 05:08:04,515 epoch 43 - iter 6/25 - loss 2.90601961 - samples/sec: 113.71 - lr: 0.150000 2021-03-26 05:08:05,603 epoch 43 - iter 8/25 - loss 2.87946507 - samples/sec: 117.88 - lr: 0.150000 2021-03-26 05:08:06,562 epoch 43 - iter 10/25 - loss 2.92280562 - samples/sec: 133.72 - lr: 0.150000 2021-03-26 05:08:07,605 epoch 43 - iter 12/25 - loss 3.02853823 - samples/sec: 122.97 - lr: 0.150000 2021-03-26 05:08:08,542 epoch 43 - iter 14/25 - loss 3.05405590 - samples/sec: 136.82 - lr: 0.150000 2021-03-26 05:08:09,481 epoch 43 - iter 16/25 - loss 3.07377841 - samples/sec: 136.51 - lr: 0.150000 2021-03-26 05:08:10,552 epoch 43 - iter 18/25 - loss 3.16452719 - samples/sec: 119.71 - lr: 0.150000 2021-03-26 05:08:11,590 epoch 43 - iter 20/25 - loss 3.12460047 - samples/sec: 123.47 - lr: 0.150000 2021-03-26 05:08:12,565 epoch 43 - iter 22/25 - loss 3.13019865 - samples/sec: 131.42 - lr: 0.150000 2021-03-26 05:08:13,455 epoch 43 - iter 24/25 - loss 3.10660581 - samples/sec: 144.13 - lr: 0.150000 2021-03-26 05:08:13,917 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:08:13,918 EPOCH 43 done: loss 3.1132 - lr 0.1500000 2021-03-26 05:08:14,652 DEV : loss 5.965920925140381 - score 0.905 2021-03-26 05:08:14,669 BAD EPOCHS (no improvement): 3 2021-03-26 05:08:14,669 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:08:15,501 epoch 44 - iter 2/25 - loss 2.74227560 - samples/sec: 154.12 - lr: 0.150000 2021-03-26 05:08:16,485 epoch 44 - iter 4/25 - loss 2.89587241 - samples/sec: 130.33 - lr: 0.150000 2021-03-26 05:08:17,504 epoch 44 - iter 6/25 - loss 2.87528515 - samples/sec: 125.87 - lr: 0.150000 2021-03-26 05:08:18,483 epoch 44 - iter 8/25 - loss 2.86914650 - samples/sec: 130.94 - lr: 0.150000 2021-03-26 05:08:19,471 epoch 44 - iter 10/25 - loss 2.98499634 - samples/sec: 129.74 - lr: 0.150000 2021-03-26 05:08:20,410 epoch 44 - iter 12/25 - loss 3.04275517 - samples/sec: 136.50 - lr: 0.150000 2021-03-26 05:08:21,499 epoch 44 - iter 14/25 - loss 3.04654218 - samples/sec: 117.85 - lr: 0.150000 2021-03-26 05:08:22,529 epoch 44 - iter 16/25 - loss 3.04176466 - samples/sec: 124.36 - lr: 0.150000 2021-03-26 05:08:23,525 epoch 44 - iter 18/25 - loss 3.02761053 - samples/sec: 128.73 - lr: 0.150000 2021-03-26 05:08:24,552 epoch 44 - iter 20/25 - loss 3.06731745 - samples/sec: 124.84 - lr: 0.150000 2021-03-26 05:08:25,503 epoch 44 - iter 22/25 - loss 3.11688727 - samples/sec: 134.78 - lr: 0.150000 2021-03-26 05:08:26,522 epoch 44 - iter 24/25 - loss 3.14101426 - samples/sec: 125.77 - lr: 0.150000 2021-03-26 05:08:26,918 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:08:26,919 EPOCH 44 done: loss 3.1204 - lr 0.1500000 2021-03-26 05:08:27,670 DEV : loss 5.890631675720215 - score 0.9104 2021-03-26 05:08:27,695 BAD EPOCHS (no improvement): 4 2021-03-26 05:08:27,695 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:08:28,747 epoch 45 - iter 2/25 - loss 2.88748765 - samples/sec: 122.00 - lr: 0.075000 2021-03-26 05:08:29,758 epoch 45 - iter 4/25 - loss 3.12572914 - samples/sec: 126.94 - lr: 0.075000 2021-03-26 05:08:31,051 epoch 45 - iter 6/25 - loss 3.04913143 - samples/sec: 99.11 - lr: 0.075000 2021-03-26 05:08:32,318 epoch 45 - iter 8/25 - loss 3.06582740 - samples/sec: 101.19 - lr: 0.075000 2021-03-26 05:08:33,733 epoch 45 - iter 10/25 - loss 3.13125415 - samples/sec: 90.52 - lr: 0.075000 2021-03-26 05:08:35,027 epoch 45 - iter 12/25 - loss 3.13420904 - samples/sec: 99.12 - lr: 0.075000 2021-03-26 05:08:35,963 epoch 45 - iter 14/25 - loss 3.15804754 - samples/sec: 136.99 - lr: 0.075000 2021-03-26 05:08:36,924 epoch 45 - iter 16/25 - loss 3.18488774 - samples/sec: 133.34 - lr: 0.075000 2021-03-26 05:08:37,992 epoch 45 - iter 18/25 - loss 3.21598185 - samples/sec: 120.19 - lr: 0.075000 2021-03-26 05:08:38,961 epoch 45 - iter 20/25 - loss 3.21181444 - samples/sec: 132.21 - lr: 0.075000 2021-03-26 05:08:40,006 epoch 45 - iter 22/25 - loss 3.19924637 - samples/sec: 122.70 - lr: 0.075000 2021-03-26 05:08:41,079 epoch 45 - iter 24/25 - loss 3.12326492 - samples/sec: 119.41 - lr: 0.075000 2021-03-26 05:08:41,467 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:08:41,468 EPOCH 45 done: loss 3.1075 - lr 0.0750000 2021-03-26 05:08:42,198 DEV : loss 5.875544548034668 - score 0.912 2021-03-26 05:08:42,222 BAD EPOCHS (no improvement): 0 2021-03-26 05:08:51,984 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:08:53,045 epoch 46 - iter 2/25 - loss 2.93449044 - samples/sec: 120.89 - lr: 0.075000 2021-03-26 05:08:54,027 epoch 46 - iter 4/25 - loss 2.99039429 - samples/sec: 130.48 - lr: 0.075000 2021-03-26 05:08:55,046 epoch 46 - iter 6/25 - loss 2.93938939 - samples/sec: 125.82 - lr: 0.075000 2021-03-26 05:08:56,105 epoch 46 - iter 8/25 - loss 2.99548993 - samples/sec: 120.95 - lr: 0.075000 2021-03-26 05:08:57,441 epoch 46 - iter 10/25 - loss 2.97969875 - samples/sec: 95.96 - lr: 0.075000 2021-03-26 05:08:58,762 epoch 46 - iter 12/25 - loss 2.97719487 - samples/sec: 96.97 - lr: 0.075000 2021-03-26 05:09:00,173 epoch 46 - iter 14/25 - loss 2.96926909 - samples/sec: 90.79 - lr: 0.075000 2021-03-26 05:09:01,175 epoch 46 - iter 16/25 - loss 2.86651133 - samples/sec: 127.93 - lr: 0.075000 2021-03-26 05:09:02,147 epoch 46 - iter 18/25 - loss 2.83233232 - samples/sec: 131.95 - lr: 0.075000 2021-03-26 05:09:03,273 epoch 46 - iter 20/25 - loss 2.86771994 - samples/sec: 113.75 - lr: 0.075000 2021-03-26 05:09:04,273 epoch 46 - iter 22/25 - loss 2.93586094 - samples/sec: 128.26 - lr: 0.075000 2021-03-26 05:09:05,280 epoch 46 - iter 24/25 - loss 2.91423157 - samples/sec: 127.24 - lr: 0.075000 2021-03-26 05:09:05,674 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:09:05,674 EPOCH 46 done: loss 2.9104 - lr 0.0750000 2021-03-26 05:09:06,420 DEV : loss 5.901866436004639 - score 0.9128 2021-03-26 05:09:06,444 BAD EPOCHS (no improvement): 0 2021-03-26 05:09:16,208 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:09:17,307 epoch 47 - iter 2/25 - loss 2.89936328 - samples/sec: 116.74 - lr: 0.075000 2021-03-26 05:09:18,420 epoch 47 - iter 4/25 - loss 3.05432254 - samples/sec: 115.20 - lr: 0.075000 2021-03-26 05:09:19,609 epoch 47 - iter 6/25 - loss 3.02896949 - samples/sec: 107.74 - lr: 0.075000 2021-03-26 05:09:20,605 epoch 47 - iter 8/25 - loss 2.99035889 - samples/sec: 128.88 - lr: 0.075000 2021-03-26 05:09:21,620 epoch 47 - iter 10/25 - loss 2.95006714 - samples/sec: 126.38 - lr: 0.075000 2021-03-26 05:09:22,681 epoch 47 - iter 12/25 - loss 2.90737903 - samples/sec: 120.81 - lr: 0.075000 2021-03-26 05:09:23,646 epoch 47 - iter 14/25 - loss 2.91904158 - samples/sec: 133.04 - lr: 0.075000 2021-03-26 05:09:24,556 epoch 47 - iter 16/25 - loss 2.85798267 - samples/sec: 140.88 - lr: 0.075000 2021-03-26 05:09:25,453 epoch 47 - iter 18/25 - loss 2.82409436 - samples/sec: 142.76 - lr: 0.075000 2021-03-26 05:09:26,450 epoch 47 - iter 20/25 - loss 2.83086318 - samples/sec: 128.60 - lr: 0.075000 2021-03-26 05:09:27,466 epoch 47 - iter 22/25 - loss 2.79424383 - samples/sec: 126.07 - lr: 0.075000 2021-03-26 05:09:28,477 epoch 47 - iter 24/25 - loss 2.80670158 - samples/sec: 126.97 - lr: 0.075000 2021-03-26 05:09:29,006 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:09:29,008 EPOCH 47 done: loss 2.8199 - lr 0.0750000 2021-03-26 05:09:29,759 DEV : loss 5.768811225891113 - score 0.9124 2021-03-26 05:09:29,783 BAD EPOCHS (no improvement): 1 2021-03-26 05:09:29,784 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:09:30,841 epoch 48 - iter 2/25 - loss 2.51504290 - samples/sec: 121.26 - lr: 0.075000 2021-03-26 05:09:31,922 epoch 48 - iter 4/25 - loss 2.33881265 - samples/sec: 118.72 - lr: 0.075000 2021-03-26 05:09:32,962 epoch 48 - iter 6/25 - loss 2.54652301 - samples/sec: 123.26 - lr: 0.075000 2021-03-26 05:09:34,069 epoch 48 - iter 8/25 - loss 2.60233888 - samples/sec: 115.85 - lr: 0.075000 2021-03-26 05:09:35,007 epoch 48 - iter 10/25 - loss 2.64935668 - samples/sec: 136.62 - lr: 0.075000 2021-03-26 05:09:35,985 epoch 48 - iter 12/25 - loss 2.67004373 - samples/sec: 131.18 - lr: 0.075000 2021-03-26 05:09:37,010 epoch 48 - iter 14/25 - loss 2.76849834 - samples/sec: 125.05 - lr: 0.075000 2021-03-26 05:09:38,014 epoch 48 - iter 16/25 - loss 2.82034701 - samples/sec: 127.64 - lr: 0.075000 2021-03-26 05:09:38,976 epoch 48 - iter 18/25 - loss 2.83146665 - samples/sec: 133.26 - lr: 0.075000 2021-03-26 05:09:39,936 epoch 48 - iter 20/25 - loss 2.83332582 - samples/sec: 133.66 - lr: 0.075000 2021-03-26 05:09:40,992 epoch 48 - iter 22/25 - loss 2.85925512 - samples/sec: 121.42 - lr: 0.075000 2021-03-26 05:09:42,018 epoch 48 - iter 24/25 - loss 2.82686347 - samples/sec: 125.17 - lr: 0.075000 2021-03-26 05:09:42,375 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:09:42,377 EPOCH 48 done: loss 2.8162 - lr 0.0750000 2021-03-26 05:09:43,143 DEV : loss 5.849610328674316 - score 0.9165 2021-03-26 05:09:43,161 BAD EPOCHS (no improvement): 0 2021-03-26 05:09:53,064 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:09:54,097 epoch 49 - iter 2/25 - loss 3.24604166 - samples/sec: 124.25 - lr: 0.075000 2021-03-26 05:09:55,183 epoch 49 - iter 4/25 - loss 3.21493500 - samples/sec: 118.06 - lr: 0.075000 2021-03-26 05:09:56,179 epoch 49 - iter 6/25 - loss 2.91566018 - samples/sec: 128.78 - lr: 0.075000 2021-03-26 05:09:57,183 epoch 49 - iter 8/25 - loss 2.90339556 - samples/sec: 127.74 - lr: 0.075000 2021-03-26 05:09:58,302 epoch 49 - iter 10/25 - loss 2.77145948 - samples/sec: 114.58 - lr: 0.075000 2021-03-26 05:09:59,481 epoch 49 - iter 12/25 - loss 2.80427039 - samples/sec: 108.89 - lr: 0.075000 2021-03-26 05:10:00,481 epoch 49 - iter 14/25 - loss 2.79233439 - samples/sec: 128.23 - lr: 0.075000 2021-03-26 05:10:01,534 epoch 49 - iter 16/25 - loss 2.72604090 - samples/sec: 121.80 - lr: 0.075000 2021-03-26 05:10:02,728 epoch 49 - iter 18/25 - loss 2.72750786 - samples/sec: 107.31 - lr: 0.075000 2021-03-26 05:10:03,735 epoch 49 - iter 20/25 - loss 2.73197596 - samples/sec: 127.33 - lr: 0.075000 2021-03-26 05:10:04,679 epoch 49 - iter 22/25 - loss 2.74085398 - samples/sec: 135.80 - lr: 0.075000 2021-03-26 05:10:05,680 epoch 49 - iter 24/25 - loss 2.73969566 - samples/sec: 128.00 - lr: 0.075000 2021-03-26 05:10:06,075 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:10:06,075 EPOCH 49 done: loss 2.7766 - lr 0.0750000 2021-03-26 05:10:06,816 DEV : loss 5.883216857910156 - score 0.9157 2021-03-26 05:10:06,840 BAD EPOCHS (no improvement): 1 2021-03-26 05:10:06,841 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:10:07,848 epoch 50 - iter 2/25 - loss 2.59146047 - samples/sec: 127.38 - lr: 0.075000 2021-03-26 05:10:08,849 epoch 50 - iter 4/25 - loss 2.85189342 - samples/sec: 128.01 - lr: 0.075000 2021-03-26 05:10:09,838 epoch 50 - iter 6/25 - loss 2.85971487 - samples/sec: 129.57 - lr: 0.075000 2021-03-26 05:10:10,796 epoch 50 - iter 8/25 - loss 2.81490350 - samples/sec: 133.84 - lr: 0.075000 2021-03-26 05:10:11,810 epoch 50 - iter 10/25 - loss 2.72035291 - samples/sec: 126.40 - lr: 0.075000 2021-03-26 05:10:12,878 epoch 50 - iter 12/25 - loss 2.77295995 - samples/sec: 120.02 - lr: 0.075000 2021-03-26 05:10:13,955 epoch 50 - iter 14/25 - loss 2.76417232 - samples/sec: 119.49 - lr: 0.075000 2021-03-26 05:10:14,902 epoch 50 - iter 16/25 - loss 2.77499604 - samples/sec: 135.33 - lr: 0.075000 2021-03-26 05:10:15,913 epoch 50 - iter 18/25 - loss 2.73042422 - samples/sec: 126.79 - lr: 0.075000 2021-03-26 05:10:16,936 epoch 50 - iter 20/25 - loss 2.72530985 - samples/sec: 125.36 - lr: 0.075000 2021-03-26 05:10:18,015 epoch 50 - iter 22/25 - loss 2.68665273 - samples/sec: 118.91 - lr: 0.075000 2021-03-26 05:10:19,058 epoch 50 - iter 24/25 - loss 2.68635307 - samples/sec: 122.84 - lr: 0.075000 2021-03-26 05:10:19,451 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:10:19,452 EPOCH 50 done: loss 2.6908 - lr 0.0750000 2021-03-26 05:10:20,196 DEV : loss 5.870783805847168 - score 0.9165 2021-03-26 05:10:20,222 BAD EPOCHS (no improvement): 2 2021-03-26 05:10:20,223 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:10:21,245 epoch 51 - iter 2/25 - loss 3.00540733 - samples/sec: 125.46 - lr: 0.075000 2021-03-26 05:10:22,211 epoch 51 - iter 4/25 - loss 2.98089063 - samples/sec: 132.85 - lr: 0.075000 2021-03-26 05:10:23,317 epoch 51 - iter 6/25 - loss 2.87133745 - samples/sec: 115.83 - lr: 0.075000 2021-03-26 05:10:24,245 epoch 51 - iter 8/25 - loss 2.79685122 - samples/sec: 138.23 - lr: 0.075000 2021-03-26 05:10:25,314 epoch 51 - iter 10/25 - loss 2.83986566 - samples/sec: 119.88 - lr: 0.075000 2021-03-26 05:10:26,240 epoch 51 - iter 12/25 - loss 2.70246488 - samples/sec: 138.42 - lr: 0.075000 2021-03-26 05:10:27,172 epoch 51 - iter 14/25 - loss 2.68542244 - samples/sec: 137.51 - lr: 0.075000 2021-03-26 05:10:28,293 epoch 51 - iter 16/25 - loss 2.66649586 - samples/sec: 114.35 - lr: 0.075000 2021-03-26 05:10:29,616 epoch 51 - iter 18/25 - loss 2.69753993 - samples/sec: 96.85 - lr: 0.075000 2021-03-26 05:10:30,640 epoch 51 - iter 20/25 - loss 2.73083348 - samples/sec: 125.16 - lr: 0.075000 2021-03-26 05:10:31,801 epoch 51 - iter 22/25 - loss 2.74151724 - samples/sec: 110.31 - lr: 0.075000 2021-03-26 05:10:32,894 epoch 51 - iter 24/25 - loss 2.78909466 - samples/sec: 117.31 - lr: 0.075000 2021-03-26 05:10:33,297 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:10:33,298 EPOCH 51 done: loss 2.7559 - lr 0.0750000 2021-03-26 05:10:34,050 DEV : loss 5.804508209228516 - score 0.9157 2021-03-26 05:10:34,068 BAD EPOCHS (no improvement): 3 2021-03-26 05:10:34,068 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:10:35,051 epoch 52 - iter 2/25 - loss 2.59395206 - samples/sec: 130.52 - lr: 0.075000 2021-03-26 05:10:36,047 epoch 52 - iter 4/25 - loss 2.64464074 - samples/sec: 128.60 - lr: 0.075000 2021-03-26 05:10:36,977 epoch 52 - iter 6/25 - loss 2.63424472 - samples/sec: 137.92 - lr: 0.075000 2021-03-26 05:10:38,067 epoch 52 - iter 8/25 - loss 2.66879421 - samples/sec: 117.59 - lr: 0.075000 2021-03-26 05:10:39,227 epoch 52 - iter 10/25 - loss 2.63585150 - samples/sec: 110.48 - lr: 0.075000 2021-03-26 05:10:40,216 epoch 52 - iter 12/25 - loss 2.56680995 - samples/sec: 129.67 - lr: 0.075000 2021-03-26 05:10:41,237 epoch 52 - iter 14/25 - loss 2.54881712 - samples/sec: 125.51 - lr: 0.075000 2021-03-26 05:10:42,260 epoch 52 - iter 16/25 - loss 2.60447754 - samples/sec: 125.39 - lr: 0.075000 2021-03-26 05:10:43,231 epoch 52 - iter 18/25 - loss 2.55686861 - samples/sec: 131.86 - lr: 0.075000 2021-03-26 05:10:44,205 epoch 52 - iter 20/25 - loss 2.60568520 - samples/sec: 131.73 - lr: 0.075000 2021-03-26 05:10:45,252 epoch 52 - iter 22/25 - loss 2.67642180 - samples/sec: 122.38 - lr: 0.075000 2021-03-26 05:10:46,209 epoch 52 - iter 24/25 - loss 2.69854917 - samples/sec: 133.98 - lr: 0.075000 2021-03-26 05:10:46,588 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:10:46,589 EPOCH 52 done: loss 2.6946 - lr 0.0750000 2021-03-26 05:10:47,361 DEV : loss 5.859862804412842 - score 0.9128 2021-03-26 05:10:47,389 BAD EPOCHS (no improvement): 4 2021-03-26 05:10:47,390 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:10:48,466 epoch 53 - iter 2/25 - loss 2.85855150 - samples/sec: 119.12 - lr: 0.037500 2021-03-26 05:10:49,499 epoch 53 - iter 4/25 - loss 2.82207429 - samples/sec: 124.04 - lr: 0.037500 2021-03-26 05:10:50,589 epoch 53 - iter 6/25 - loss 2.66309059 - samples/sec: 117.62 - lr: 0.037500 2021-03-26 05:10:51,582 epoch 53 - iter 8/25 - loss 2.65608153 - samples/sec: 129.17 - lr: 0.037500 2021-03-26 05:10:52,583 epoch 53 - iter 10/25 - loss 2.80065906 - samples/sec: 128.28 - lr: 0.037500 2021-03-26 05:10:53,621 epoch 53 - iter 12/25 - loss 2.76256857 - samples/sec: 123.54 - lr: 0.037500 2021-03-26 05:10:54,591 epoch 53 - iter 14/25 - loss 2.71554373 - samples/sec: 132.25 - lr: 0.037500 2021-03-26 05:10:55,563 epoch 53 - iter 16/25 - loss 2.73098251 - samples/sec: 131.95 - lr: 0.037500 2021-03-26 05:10:56,763 epoch 53 - iter 18/25 - loss 2.69363379 - samples/sec: 106.77 - lr: 0.037500 2021-03-26 05:10:57,722 epoch 53 - iter 20/25 - loss 2.69553999 - samples/sec: 133.64 - lr: 0.037500 2021-03-26 05:10:58,774 epoch 53 - iter 22/25 - loss 2.71568731 - samples/sec: 121.87 - lr: 0.037500 2021-03-26 05:10:59,737 epoch 53 - iter 24/25 - loss 2.75015450 - samples/sec: 133.29 - lr: 0.037500 2021-03-26 05:11:00,161 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:11:00,162 EPOCH 53 done: loss 2.7943 - lr 0.0375000 2021-03-26 05:11:00,943 DEV : loss 5.84169864654541 - score 0.9153 2021-03-26 05:11:00,962 BAD EPOCHS (no improvement): 1 2021-03-26 05:11:00,963 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:11:01,985 epoch 54 - iter 2/25 - loss 2.60842669 - samples/sec: 125.45 - lr: 0.037500 2021-03-26 05:11:03,055 epoch 54 - iter 4/25 - loss 2.65413553 - samples/sec: 119.73 - lr: 0.037500 2021-03-26 05:11:03,993 epoch 54 - iter 6/25 - loss 2.61845867 - samples/sec: 136.63 - lr: 0.037500 2021-03-26 05:11:05,132 epoch 54 - iter 8/25 - loss 2.58603658 - samples/sec: 112.68 - lr: 0.037500 2021-03-26 05:11:06,201 epoch 54 - iter 10/25 - loss 2.65403033 - samples/sec: 119.94 - lr: 0.037500 2021-03-26 05:11:07,283 epoch 54 - iter 12/25 - loss 2.73958167 - samples/sec: 118.39 - lr: 0.037500 2021-03-26 05:11:08,408 epoch 54 - iter 14/25 - loss 2.76906408 - samples/sec: 114.00 - lr: 0.037500 2021-03-26 05:11:09,636 epoch 54 - iter 16/25 - loss 2.74741621 - samples/sec: 104.33 - lr: 0.037500 2021-03-26 05:11:10,794 epoch 54 - iter 18/25 - loss 2.73005085 - samples/sec: 110.87 - lr: 0.037500 2021-03-26 05:11:11,738 epoch 54 - iter 20/25 - loss 2.76427827 - samples/sec: 135.69 - lr: 0.037500 2021-03-26 05:11:12,781 epoch 54 - iter 22/25 - loss 2.76628489 - samples/sec: 122.98 - lr: 0.037500 2021-03-26 05:11:13,868 epoch 54 - iter 24/25 - loss 2.78265012 - samples/sec: 117.86 - lr: 0.037500 2021-03-26 05:11:14,243 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:11:14,244 EPOCH 54 done: loss 2.7432 - lr 0.0375000 2021-03-26 05:11:15,075 DEV : loss 5.859203338623047 - score 0.9132 2021-03-26 05:11:15,103 BAD EPOCHS (no improvement): 2 2021-03-26 05:11:15,104 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:11:16,164 epoch 55 - iter 2/25 - loss 2.60760987 - samples/sec: 120.93 - lr: 0.037500 2021-03-26 05:11:17,322 epoch 55 - iter 4/25 - loss 2.61803091 - samples/sec: 110.63 - lr: 0.037500 2021-03-26 05:11:18,328 epoch 55 - iter 6/25 - loss 2.67942242 - samples/sec: 127.42 - lr: 0.037500 2021-03-26 05:11:19,411 epoch 55 - iter 8/25 - loss 2.65748721 - samples/sec: 118.38 - lr: 0.037500 2021-03-26 05:11:20,659 epoch 55 - iter 10/25 - loss 2.59346483 - samples/sec: 102.72 - lr: 0.037500 2021-03-26 05:11:21,786 epoch 55 - iter 12/25 - loss 2.66532530 - samples/sec: 113.73 - lr: 0.037500 2021-03-26 05:11:22,685 epoch 55 - iter 14/25 - loss 2.65365284 - samples/sec: 143.41 - lr: 0.037500 2021-03-26 05:11:23,646 epoch 55 - iter 16/25 - loss 2.70386864 - samples/sec: 133.36 - lr: 0.037500 2021-03-26 05:11:24,700 epoch 55 - iter 18/25 - loss 2.70395010 - samples/sec: 121.62 - lr: 0.037500 2021-03-26 05:11:25,598 epoch 55 - iter 20/25 - loss 2.66016315 - samples/sec: 142.75 - lr: 0.037500 2021-03-26 05:11:26,611 epoch 55 - iter 22/25 - loss 2.69519200 - samples/sec: 126.71 - lr: 0.037500 2021-03-26 05:11:27,617 epoch 55 - iter 24/25 - loss 2.66808570 - samples/sec: 127.43 - lr: 0.037500 2021-03-26 05:11:28,000 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:11:28,001 EPOCH 55 done: loss 2.6832 - lr 0.0375000 2021-03-26 05:11:28,754 DEV : loss 5.843713283538818 - score 0.9141 2021-03-26 05:11:28,772 BAD EPOCHS (no improvement): 3 2021-03-26 05:11:28,772 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:11:29,808 epoch 56 - iter 2/25 - loss 2.50143147 - samples/sec: 123.76 - lr: 0.037500 2021-03-26 05:11:30,889 epoch 56 - iter 4/25 - loss 2.65481919 - samples/sec: 118.62 - lr: 0.037500 2021-03-26 05:11:32,017 epoch 56 - iter 6/25 - loss 2.60150150 - samples/sec: 113.60 - lr: 0.037500 2021-03-26 05:11:33,057 epoch 56 - iter 8/25 - loss 2.60987782 - samples/sec: 123.20 - lr: 0.037500 2021-03-26 05:11:33,940 epoch 56 - iter 10/25 - loss 2.58933198 - samples/sec: 145.08 - lr: 0.037500 2021-03-26 05:11:34,885 epoch 56 - iter 12/25 - loss 2.58792406 - samples/sec: 135.73 - lr: 0.037500 2021-03-26 05:11:35,833 epoch 56 - iter 14/25 - loss 2.56192599 - samples/sec: 135.15 - lr: 0.037500 2021-03-26 05:11:36,891 epoch 56 - iter 16/25 - loss 2.60430720 - samples/sec: 121.24 - lr: 0.037500 2021-03-26 05:11:37,917 epoch 56 - iter 18/25 - loss 2.60791723 - samples/sec: 124.84 - lr: 0.037500 2021-03-26 05:11:38,932 epoch 56 - iter 20/25 - loss 2.63114707 - samples/sec: 126.37 - lr: 0.037500 2021-03-26 05:11:39,958 epoch 56 - iter 22/25 - loss 2.62431546 - samples/sec: 124.85 - lr: 0.037500 2021-03-26 05:11:40,929 epoch 56 - iter 24/25 - loss 2.63437254 - samples/sec: 132.04 - lr: 0.037500 2021-03-26 05:11:41,364 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:11:41,365 EPOCH 56 done: loss 2.6203 - lr 0.0375000 2021-03-26 05:11:42,109 DEV : loss 5.871529579162598 - score 0.9149 2021-03-26 05:11:42,134 BAD EPOCHS (no improvement): 4 2021-03-26 05:11:42,135 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:11:43,263 epoch 57 - iter 2/25 - loss 2.50472903 - samples/sec: 113.60 - lr: 0.018750 2021-03-26 05:11:44,255 epoch 57 - iter 4/25 - loss 2.49861062 - samples/sec: 129.41 - lr: 0.018750 2021-03-26 05:11:45,263 epoch 57 - iter 6/25 - loss 2.46727121 - samples/sec: 127.14 - lr: 0.018750 2021-03-26 05:11:46,321 epoch 57 - iter 8/25 - loss 2.36317959 - samples/sec: 121.23 - lr: 0.018750 2021-03-26 05:11:47,310 epoch 57 - iter 10/25 - loss 2.46769011 - samples/sec: 129.66 - lr: 0.018750 2021-03-26 05:11:48,323 epoch 57 - iter 12/25 - loss 2.48527586 - samples/sec: 126.71 - lr: 0.018750 2021-03-26 05:11:49,358 epoch 57 - iter 14/25 - loss 2.50677931 - samples/sec: 123.84 - lr: 0.018750 2021-03-26 05:11:50,425 epoch 57 - iter 16/25 - loss 2.46721734 - samples/sec: 120.02 - lr: 0.018750 2021-03-26 05:11:51,317 epoch 57 - iter 18/25 - loss 2.47471072 - samples/sec: 143.74 - lr: 0.018750 2021-03-26 05:11:52,350 epoch 57 - iter 20/25 - loss 2.49596874 - samples/sec: 124.22 - lr: 0.018750 2021-03-26 05:11:53,357 epoch 57 - iter 22/25 - loss 2.52411051 - samples/sec: 127.30 - lr: 0.018750 2021-03-26 05:11:54,282 epoch 57 - iter 24/25 - loss 2.46456073 - samples/sec: 138.49 - lr: 0.018750 2021-03-26 05:11:54,675 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:11:54,675 EPOCH 57 done: loss 2.4844 - lr 0.0187500 2021-03-26 05:11:55,418 DEV : loss 5.8934326171875 - score 0.9141 2021-03-26 05:11:55,443 BAD EPOCHS (no improvement): 1 2021-03-26 05:11:55,443 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:11:56,448 epoch 58 - iter 2/25 - loss 2.22876573 - samples/sec: 127.67 - lr: 0.018750 2021-03-26 05:11:57,451 epoch 58 - iter 4/25 - loss 2.52895451 - samples/sec: 127.87 - lr: 0.018750 2021-03-26 05:11:58,483 epoch 58 - iter 6/25 - loss 2.48068396 - samples/sec: 124.18 - lr: 0.018750 2021-03-26 05:11:59,489 epoch 58 - iter 8/25 - loss 2.41449891 - samples/sec: 127.56 - lr: 0.018750 2021-03-26 05:12:00,499 epoch 58 - iter 10/25 - loss 2.40381507 - samples/sec: 127.00 - lr: 0.018750 2021-03-26 05:12:01,522 epoch 58 - iter 12/25 - loss 2.42167565 - samples/sec: 125.30 - lr: 0.018750 2021-03-26 05:12:02,549 epoch 58 - iter 14/25 - loss 2.52895261 - samples/sec: 124.93 - lr: 0.018750 2021-03-26 05:12:03,595 epoch 58 - iter 16/25 - loss 2.56519403 - samples/sec: 122.76 - lr: 0.018750 2021-03-26 05:12:04,581 epoch 58 - iter 18/25 - loss 2.54275488 - samples/sec: 129.98 - lr: 0.018750 2021-03-26 05:12:05,694 epoch 58 - iter 20/25 - loss 2.51648801 - samples/sec: 115.12 - lr: 0.018750 2021-03-26 05:12:06,939 epoch 58 - iter 22/25 - loss 2.52750042 - samples/sec: 102.93 - lr: 0.018750 2021-03-26 05:12:07,900 epoch 58 - iter 24/25 - loss 2.53430993 - samples/sec: 133.41 - lr: 0.018750 2021-03-26 05:12:08,329 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:12:08,330 EPOCH 58 done: loss 2.5247 - lr 0.0187500 2021-03-26 05:12:09,139 DEV : loss 5.869925498962402 - score 0.9153 2021-03-26 05:12:09,163 BAD EPOCHS (no improvement): 2 2021-03-26 05:12:09,164 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:12:10,200 epoch 59 - iter 2/25 - loss 2.34672618 - samples/sec: 123.70 - lr: 0.018750 2021-03-26 05:12:11,324 epoch 59 - iter 4/25 - loss 2.17811352 - samples/sec: 114.08 - lr: 0.018750 2021-03-26 05:12:12,332 epoch 59 - iter 6/25 - loss 2.23459947 - samples/sec: 127.22 - lr: 0.018750 2021-03-26 05:12:13,388 epoch 59 - iter 8/25 - loss 2.42494446 - samples/sec: 121.54 - lr: 0.018750 2021-03-26 05:12:14,448 epoch 59 - iter 10/25 - loss 2.44249034 - samples/sec: 120.92 - lr: 0.018750 2021-03-26 05:12:15,374 epoch 59 - iter 12/25 - loss 2.42561487 - samples/sec: 138.50 - lr: 0.018750 2021-03-26 05:12:16,325 epoch 59 - iter 14/25 - loss 2.37253753 - samples/sec: 134.70 - lr: 0.018750 2021-03-26 05:12:17,346 epoch 59 - iter 16/25 - loss 2.36989839 - samples/sec: 125.65 - lr: 0.018750 2021-03-26 05:12:18,355 epoch 59 - iter 18/25 - loss 2.37962128 - samples/sec: 127.11 - lr: 0.018750 2021-03-26 05:12:19,338 epoch 59 - iter 20/25 - loss 2.39560186 - samples/sec: 130.44 - lr: 0.018750 2021-03-26 05:12:20,283 epoch 59 - iter 22/25 - loss 2.39514251 - samples/sec: 135.67 - lr: 0.018750 2021-03-26 05:12:21,443 epoch 59 - iter 24/25 - loss 2.44506241 - samples/sec: 110.53 - lr: 0.018750 2021-03-26 05:12:21,938 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:12:21,939 EPOCH 59 done: loss 2.4598 - lr 0.0187500 2021-03-26 05:12:22,694 DEV : loss 5.848148345947266 - score 0.9153 2021-03-26 05:12:22,718 BAD EPOCHS (no improvement): 3 2021-03-26 05:12:22,719 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:12:23,910 epoch 60 - iter 2/25 - loss 3.09506989 - samples/sec: 107.58 - lr: 0.018750 2021-03-26 05:12:26,192 epoch 60 - iter 4/25 - loss 2.75254214 - samples/sec: 56.14 - lr: 0.018750 2021-03-26 05:12:27,279 epoch 60 - iter 6/25 - loss 2.84813265 - samples/sec: 117.88 - lr: 0.018750 2021-03-26 05:12:28,297 epoch 60 - iter 8/25 - loss 2.69154215 - samples/sec: 125.84 - lr: 0.018750 2021-03-26 05:12:29,232 epoch 60 - iter 10/25 - loss 2.61896937 - samples/sec: 137.16 - lr: 0.018750 2021-03-26 05:12:30,252 epoch 60 - iter 12/25 - loss 2.55158060 - samples/sec: 125.64 - lr: 0.018750 2021-03-26 05:12:31,363 epoch 60 - iter 14/25 - loss 2.55699714 - samples/sec: 115.47 - lr: 0.018750 2021-03-26 05:12:32,463 epoch 60 - iter 16/25 - loss 2.57129791 - samples/sec: 116.69 - lr: 0.018750 2021-03-26 05:12:33,580 epoch 60 - iter 18/25 - loss 2.60079477 - samples/sec: 114.69 - lr: 0.018750 2021-03-26 05:12:34,567 epoch 60 - iter 20/25 - loss 2.59394053 - samples/sec: 130.11 - lr: 0.018750 2021-03-26 05:12:35,746 epoch 60 - iter 22/25 - loss 2.63753519 - samples/sec: 108.65 - lr: 0.018750 2021-03-26 05:12:36,862 epoch 60 - iter 24/25 - loss 2.60684501 - samples/sec: 114.85 - lr: 0.018750 2021-03-26 05:12:37,274 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:12:37,274 EPOCH 60 done: loss 2.6101 - lr 0.0187500 2021-03-26 05:12:38,056 DEV : loss 5.879796028137207 - score 0.9165 2021-03-26 05:12:38,074 BAD EPOCHS (no improvement): 4 2021-03-26 05:12:38,075 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:12:39,090 epoch 61 - iter 2/25 - loss 2.78632987 - samples/sec: 126.29 - lr: 0.009375 2021-03-26 05:12:40,059 epoch 61 - iter 4/25 - loss 2.51424420 - samples/sec: 132.54 - lr: 0.009375 2021-03-26 05:12:40,989 epoch 61 - iter 6/25 - loss 2.56003817 - samples/sec: 137.81 - lr: 0.009375 2021-03-26 05:12:41,999 epoch 61 - iter 8/25 - loss 2.39456305 - samples/sec: 127.01 - lr: 0.009375 2021-03-26 05:12:42,951 epoch 61 - iter 10/25 - loss 2.40821805 - samples/sec: 134.69 - lr: 0.009375 2021-03-26 05:12:43,956 epoch 61 - iter 12/25 - loss 2.53350836 - samples/sec: 127.51 - lr: 0.009375 2021-03-26 05:12:44,919 epoch 61 - iter 14/25 - loss 2.53313923 - samples/sec: 133.12 - lr: 0.009375 2021-03-26 05:12:45,990 epoch 61 - iter 16/25 - loss 2.51551811 - samples/sec: 119.72 - lr: 0.009375 2021-03-26 05:12:47,052 epoch 61 - iter 18/25 - loss 2.51872024 - samples/sec: 120.76 - lr: 0.009375 2021-03-26 05:12:48,033 epoch 61 - iter 20/25 - loss 2.50309972 - samples/sec: 130.59 - lr: 0.009375 2021-03-26 05:12:49,082 epoch 61 - iter 22/25 - loss 2.48143146 - samples/sec: 122.17 - lr: 0.009375 2021-03-26 05:12:50,012 epoch 61 - iter 24/25 - loss 2.45347960 - samples/sec: 138.08 - lr: 0.009375 2021-03-26 05:12:50,468 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:12:50,468 EPOCH 61 done: loss 2.4569 - lr 0.0093750 2021-03-26 05:12:51,225 DEV : loss 5.87412166595459 - score 0.9161 2021-03-26 05:12:51,250 BAD EPOCHS (no improvement): 1 2021-03-26 05:12:51,250 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:12:52,234 epoch 62 - iter 2/25 - loss 2.94621849 - samples/sec: 130.36 - lr: 0.009375 2021-03-26 05:12:53,488 epoch 62 - iter 4/25 - loss 2.98037022 - samples/sec: 102.16 - lr: 0.009375 2021-03-26 05:12:54,500 epoch 62 - iter 6/25 - loss 2.73766508 - samples/sec: 126.65 - lr: 0.009375 2021-03-26 05:12:55,659 epoch 62 - iter 8/25 - loss 2.73090996 - samples/sec: 110.57 - lr: 0.009375 2021-03-26 05:12:56,756 epoch 62 - iter 10/25 - loss 2.75914966 - samples/sec: 116.86 - lr: 0.009375 2021-03-26 05:12:57,696 epoch 62 - iter 12/25 - loss 2.66616900 - samples/sec: 136.38 - lr: 0.009375 2021-03-26 05:12:58,648 epoch 62 - iter 14/25 - loss 2.62130229 - samples/sec: 134.58 - lr: 0.009375 2021-03-26 05:12:59,693 epoch 62 - iter 16/25 - loss 2.59688076 - samples/sec: 122.69 - lr: 0.009375 2021-03-26 05:13:00,661 epoch 62 - iter 18/25 - loss 2.63257953 - samples/sec: 132.59 - lr: 0.009375 2021-03-26 05:13:01,796 epoch 62 - iter 20/25 - loss 2.62566335 - samples/sec: 112.96 - lr: 0.009375 2021-03-26 05:13:02,774 epoch 62 - iter 22/25 - loss 2.58263093 - samples/sec: 131.28 - lr: 0.009375 2021-03-26 05:13:03,781 epoch 62 - iter 24/25 - loss 2.53657119 - samples/sec: 127.23 - lr: 0.009375 2021-03-26 05:13:04,209 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:13:04,209 EPOCH 62 done: loss 2.5635 - lr 0.0093750 2021-03-26 05:13:05,058 DEV : loss 5.863091945648193 - score 0.9165 2021-03-26 05:13:05,087 BAD EPOCHS (no improvement): 2 2021-03-26 05:13:05,088 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:13:06,122 epoch 63 - iter 2/25 - loss 3.11381161 - samples/sec: 123.94 - lr: 0.009375 2021-03-26 05:13:07,116 epoch 63 - iter 4/25 - loss 2.85715598 - samples/sec: 129.11 - lr: 0.009375 2021-03-26 05:13:08,228 epoch 63 - iter 6/25 - loss 2.82112515 - samples/sec: 115.23 - lr: 0.009375 2021-03-26 05:13:09,259 epoch 63 - iter 8/25 - loss 2.75335273 - samples/sec: 124.37 - lr: 0.009375 2021-03-26 05:13:10,286 epoch 63 - iter 10/25 - loss 2.71493399 - samples/sec: 124.77 - lr: 0.009375 2021-03-26 05:13:11,242 epoch 63 - iter 12/25 - loss 2.60437027 - samples/sec: 134.13 - lr: 0.009375 2021-03-26 05:13:12,288 epoch 63 - iter 14/25 - loss 2.62287546 - samples/sec: 122.53 - lr: 0.009375 2021-03-26 05:13:13,354 epoch 63 - iter 16/25 - loss 2.64130027 - samples/sec: 120.16 - lr: 0.009375 2021-03-26 05:13:14,451 epoch 63 - iter 18/25 - loss 2.67891765 - samples/sec: 116.81 - lr: 0.009375 2021-03-26 05:13:15,483 epoch 63 - iter 20/25 - loss 2.69270143 - samples/sec: 124.19 - lr: 0.009375 2021-03-26 05:13:16,393 epoch 63 - iter 22/25 - loss 2.65427635 - samples/sec: 141.12 - lr: 0.009375 2021-03-26 05:13:17,485 epoch 63 - iter 24/25 - loss 2.63275237 - samples/sec: 117.50 - lr: 0.009375 2021-03-26 05:13:17,966 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:13:17,967 EPOCH 63 done: loss 2.5749 - lr 0.0093750 2021-03-26 05:13:18,819 DEV : loss 5.869942665100098 - score 0.9161 2021-03-26 05:13:18,855 BAD EPOCHS (no improvement): 3 2021-03-26 05:13:18,855 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:13:19,939 epoch 64 - iter 2/25 - loss 2.58264530 - samples/sec: 118.29 - lr: 0.009375 2021-03-26 05:13:20,939 epoch 64 - iter 4/25 - loss 2.61912549 - samples/sec: 128.18 - lr: 0.009375 2021-03-26 05:13:22,026 epoch 64 - iter 6/25 - loss 2.56370831 - samples/sec: 117.86 - lr: 0.009375 2021-03-26 05:13:23,047 epoch 64 - iter 8/25 - loss 2.60431099 - samples/sec: 125.72 - lr: 0.009375 2021-03-26 05:13:24,057 epoch 64 - iter 10/25 - loss 2.60761056 - samples/sec: 126.93 - lr: 0.009375 2021-03-26 05:13:25,106 epoch 64 - iter 12/25 - loss 2.72470246 - samples/sec: 122.07 - lr: 0.009375 2021-03-26 05:13:26,033 epoch 64 - iter 14/25 - loss 2.66825621 - samples/sec: 138.53 - lr: 0.009375 2021-03-26 05:13:27,101 epoch 64 - iter 16/25 - loss 2.60022439 - samples/sec: 120.00 - lr: 0.009375 2021-03-26 05:13:28,074 epoch 64 - iter 18/25 - loss 2.60289984 - samples/sec: 131.76 - lr: 0.009375 2021-03-26 05:13:29,159 epoch 64 - iter 20/25 - loss 2.58121090 - samples/sec: 118.53 - lr: 0.009375 2021-03-26 05:13:30,114 epoch 64 - iter 22/25 - loss 2.57763591 - samples/sec: 134.19 - lr: 0.009375 2021-03-26 05:13:31,172 epoch 64 - iter 24/25 - loss 2.54952568 - samples/sec: 121.21 - lr: 0.009375 2021-03-26 05:13:31,584 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:13:31,584 EPOCH 64 done: loss 2.5479 - lr 0.0093750 2021-03-26 05:13:32,339 DEV : loss 5.880767822265625 - score 0.9153 2021-03-26 05:13:32,360 BAD EPOCHS (no improvement): 4 2021-03-26 05:13:32,360 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:13:33,437 epoch 65 - iter 2/25 - loss 2.48567939 - samples/sec: 119.00 - lr: 0.004687 2021-03-26 05:13:34,449 epoch 65 - iter 4/25 - loss 2.40336627 - samples/sec: 126.63 - lr: 0.004687 2021-03-26 05:13:35,517 epoch 65 - iter 6/25 - loss 2.40060925 - samples/sec: 120.07 - lr: 0.004687 2021-03-26 05:13:36,655 epoch 65 - iter 8/25 - loss 2.41814449 - samples/sec: 112.69 - lr: 0.004687 2021-03-26 05:13:37,690 epoch 65 - iter 10/25 - loss 2.37873857 - samples/sec: 123.99 - lr: 0.004687 2021-03-26 05:13:38,751 epoch 65 - iter 12/25 - loss 2.45327397 - samples/sec: 120.76 - lr: 0.004687 2021-03-26 05:13:39,797 epoch 65 - iter 14/25 - loss 2.53240495 - samples/sec: 122.67 - lr: 0.004687 2021-03-26 05:13:40,798 epoch 65 - iter 16/25 - loss 2.51361752 - samples/sec: 128.07 - lr: 0.004687 2021-03-26 05:13:41,785 epoch 65 - iter 18/25 - loss 2.53079632 - samples/sec: 129.86 - lr: 0.004687 2021-03-26 05:13:42,803 epoch 65 - iter 20/25 - loss 2.53446673 - samples/sec: 125.99 - lr: 0.004687 2021-03-26 05:13:43,922 epoch 65 - iter 22/25 - loss 2.52693764 - samples/sec: 114.57 - lr: 0.004687 2021-03-26 05:13:44,886 epoch 65 - iter 24/25 - loss 2.52916571 - samples/sec: 133.22 - lr: 0.004687 2021-03-26 05:13:45,382 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:13:45,383 EPOCH 65 done: loss 2.5173 - lr 0.0046875 2021-03-26 05:13:46,142 DEV : loss 5.894221305847168 - score 0.9145 2021-03-26 05:13:46,161 BAD EPOCHS (no improvement): 1 2021-03-26 05:13:46,161 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:13:47,203 epoch 66 - iter 2/25 - loss 2.66583157 - samples/sec: 122.99 - lr: 0.004687 2021-03-26 05:13:48,255 epoch 66 - iter 4/25 - loss 2.67538059 - samples/sec: 122.04 - lr: 0.004687 2021-03-26 05:13:49,417 epoch 66 - iter 6/25 - loss 2.65424959 - samples/sec: 110.23 - lr: 0.004687 2021-03-26 05:13:50,456 epoch 66 - iter 8/25 - loss 2.47670087 - samples/sec: 123.34 - lr: 0.004687 2021-03-26 05:13:51,585 epoch 66 - iter 10/25 - loss 2.41680605 - samples/sec: 113.51 - lr: 0.004687 2021-03-26 05:13:52,562 epoch 66 - iter 12/25 - loss 2.40879808 - samples/sec: 131.25 - lr: 0.004687 2021-03-26 05:13:53,556 epoch 66 - iter 14/25 - loss 2.41184230 - samples/sec: 128.90 - lr: 0.004687 2021-03-26 05:13:54,512 epoch 66 - iter 16/25 - loss 2.43771230 - samples/sec: 134.14 - lr: 0.004687 2021-03-26 05:13:55,522 epoch 66 - iter 18/25 - loss 2.44391036 - samples/sec: 126.81 - lr: 0.004687 2021-03-26 05:13:56,665 epoch 66 - iter 20/25 - loss 2.44685799 - samples/sec: 112.12 - lr: 0.004687 2021-03-26 05:13:57,641 epoch 66 - iter 22/25 - loss 2.45734792 - samples/sec: 131.33 - lr: 0.004687 2021-03-26 05:13:58,658 epoch 66 - iter 24/25 - loss 2.47441368 - samples/sec: 126.08 - lr: 0.004687 2021-03-26 05:13:59,212 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:13:59,213 EPOCH 66 done: loss 2.4660 - lr 0.0046875 2021-03-26 05:13:59,990 DEV : loss 5.8916521072387695 - score 0.9145 2021-03-26 05:14:00,015 BAD EPOCHS (no improvement): 2 2021-03-26 05:14:00,015 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:14:01,057 epoch 67 - iter 2/25 - loss 2.12723714 - samples/sec: 123.10 - lr: 0.004687 2021-03-26 05:14:02,138 epoch 67 - iter 4/25 - loss 2.38154635 - samples/sec: 118.62 - lr: 0.004687 2021-03-26 05:14:03,340 epoch 67 - iter 6/25 - loss 2.29852877 - samples/sec: 106.67 - lr: 0.004687 2021-03-26 05:14:04,341 epoch 67 - iter 8/25 - loss 2.45876713 - samples/sec: 127.97 - lr: 0.004687 2021-03-26 05:14:05,262 epoch 67 - iter 10/25 - loss 2.47288162 - samples/sec: 139.40 - lr: 0.004687 2021-03-26 05:14:06,179 epoch 67 - iter 12/25 - loss 2.52856566 - samples/sec: 139.82 - lr: 0.004687 2021-03-26 05:14:07,310 epoch 67 - iter 14/25 - loss 2.53270457 - samples/sec: 113.46 - lr: 0.004687 2021-03-26 05:14:08,291 epoch 67 - iter 16/25 - loss 2.51299741 - samples/sec: 131.29 - lr: 0.004687 2021-03-26 05:14:09,357 epoch 67 - iter 18/25 - loss 2.54054016 - samples/sec: 120.20 - lr: 0.004687 2021-03-26 05:14:10,623 epoch 67 - iter 20/25 - loss 2.52876206 - samples/sec: 101.22 - lr: 0.004687 2021-03-26 05:14:11,603 epoch 67 - iter 22/25 - loss 2.48196209 - samples/sec: 130.77 - lr: 0.004687 2021-03-26 05:14:12,712 epoch 67 - iter 24/25 - loss 2.50243033 - samples/sec: 115.67 - lr: 0.004687 2021-03-26 05:14:13,202 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:14:13,204 EPOCH 67 done: loss 2.4775 - lr 0.0046875 2021-03-26 05:14:13,970 DEV : loss 5.894336700439453 - score 0.9149 2021-03-26 05:14:13,994 BAD EPOCHS (no improvement): 3 2021-03-26 05:14:13,995 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:14:14,997 epoch 68 - iter 2/25 - loss 2.99291348 - samples/sec: 127.91 - lr: 0.004687 2021-03-26 05:14:16,162 epoch 68 - iter 4/25 - loss 2.62084621 - samples/sec: 110.09 - lr: 0.004687 2021-03-26 05:14:17,201 epoch 68 - iter 6/25 - loss 2.47720150 - samples/sec: 123.29 - lr: 0.004687 2021-03-26 05:14:18,203 epoch 68 - iter 8/25 - loss 2.42659986 - samples/sec: 127.96 - lr: 0.004687 2021-03-26 05:14:19,273 epoch 68 - iter 10/25 - loss 2.46118934 - samples/sec: 119.76 - lr: 0.004687 2021-03-26 05:14:20,279 epoch 68 - iter 12/25 - loss 2.50323576 - samples/sec: 127.52 - lr: 0.004687 2021-03-26 05:14:21,324 epoch 68 - iter 14/25 - loss 2.51408729 - samples/sec: 122.60 - lr: 0.004687 2021-03-26 05:14:22,284 epoch 68 - iter 16/25 - loss 2.47996278 - samples/sec: 133.74 - lr: 0.004687 2021-03-26 05:14:23,295 epoch 68 - iter 18/25 - loss 2.43154002 - samples/sec: 126.92 - lr: 0.004687 2021-03-26 05:14:24,251 epoch 68 - iter 20/25 - loss 2.43471692 - samples/sec: 134.03 - lr: 0.004687 2021-03-26 05:14:25,238 epoch 68 - iter 22/25 - loss 2.39672428 - samples/sec: 129.89 - lr: 0.004687 2021-03-26 05:14:26,290 epoch 68 - iter 24/25 - loss 2.39018219 - samples/sec: 121.87 - lr: 0.004687 2021-03-26 05:14:26,797 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:14:26,798 EPOCH 68 done: loss 2.3901 - lr 0.0046875 2021-03-26 05:14:27,532 DEV : loss 5.884322166442871 - score 0.9149 2021-03-26 05:14:27,557 BAD EPOCHS (no improvement): 4 2021-03-26 05:14:27,558 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:14:28,546 epoch 69 - iter 2/25 - loss 2.71395886 - samples/sec: 129.73 - lr: 0.002344 2021-03-26 05:14:29,499 epoch 69 - iter 4/25 - loss 2.59460217 - samples/sec: 134.51 - lr: 0.002344 2021-03-26 05:14:30,626 epoch 69 - iter 6/25 - loss 2.48051115 - samples/sec: 113.74 - lr: 0.002344 2021-03-26 05:14:31,639 epoch 69 - iter 8/25 - loss 2.50417805 - samples/sec: 126.57 - lr: 0.002344 2021-03-26 05:14:32,660 epoch 69 - iter 10/25 - loss 2.46161354 - samples/sec: 125.52 - lr: 0.002344 2021-03-26 05:14:33,644 epoch 69 - iter 12/25 - loss 2.41734342 - samples/sec: 130.14 - lr: 0.002344 2021-03-26 05:14:34,611 epoch 69 - iter 14/25 - loss 2.47696517 - samples/sec: 132.53 - lr: 0.002344 2021-03-26 05:14:35,670 epoch 69 - iter 16/25 - loss 2.49385166 - samples/sec: 121.04 - lr: 0.002344 2021-03-26 05:14:36,768 epoch 69 - iter 18/25 - loss 2.45438148 - samples/sec: 116.73 - lr: 0.002344 2021-03-26 05:14:37,702 epoch 69 - iter 20/25 - loss 2.42570816 - samples/sec: 137.20 - lr: 0.002344 2021-03-26 05:14:38,752 epoch 69 - iter 22/25 - loss 2.39344043 - samples/sec: 122.11 - lr: 0.002344 2021-03-26 05:14:39,838 epoch 69 - iter 24/25 - loss 2.40441241 - samples/sec: 118.11 - lr: 0.002344 2021-03-26 05:14:40,355 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:14:40,356 EPOCH 69 done: loss 2.3998 - lr 0.0023437 2021-03-26 05:14:41,098 DEV : loss 5.891051769256592 - score 0.9145 2021-03-26 05:14:41,122 BAD EPOCHS (no improvement): 1 2021-03-26 05:14:41,123 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:14:42,187 epoch 70 - iter 2/25 - loss 2.27894652 - samples/sec: 120.46 - lr: 0.002344 2021-03-26 05:14:43,232 epoch 70 - iter 4/25 - loss 2.45882046 - samples/sec: 122.67 - lr: 0.002344 2021-03-26 05:14:44,185 epoch 70 - iter 6/25 - loss 2.50836623 - samples/sec: 134.52 - lr: 0.002344 2021-03-26 05:14:45,178 epoch 70 - iter 8/25 - loss 2.55092400 - samples/sec: 129.08 - lr: 0.002344 2021-03-26 05:14:46,199 epoch 70 - iter 10/25 - loss 2.50842555 - samples/sec: 125.50 - lr: 0.002344 2021-03-26 05:14:47,232 epoch 70 - iter 12/25 - loss 2.42263022 - samples/sec: 124.23 - lr: 0.002344 2021-03-26 05:14:48,402 epoch 70 - iter 14/25 - loss 2.45893973 - samples/sec: 109.54 - lr: 0.002344 2021-03-26 05:14:49,451 epoch 70 - iter 16/25 - loss 2.55384215 - samples/sec: 122.11 - lr: 0.002344 2021-03-26 05:14:50,418 epoch 70 - iter 18/25 - loss 2.51961670 - samples/sec: 132.62 - lr: 0.002344 2021-03-26 05:14:51,354 epoch 70 - iter 20/25 - loss 2.51997625 - samples/sec: 137.17 - lr: 0.002344 2021-03-26 05:14:52,302 epoch 70 - iter 22/25 - loss 2.48214970 - samples/sec: 135.22 - lr: 0.002344 2021-03-26 05:14:53,306 epoch 70 - iter 24/25 - loss 2.48843359 - samples/sec: 127.55 - lr: 0.002344 2021-03-26 05:14:53,738 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:14:53,738 EPOCH 70 done: loss 2.4984 - lr 0.0023437 2021-03-26 05:14:54,464 DEV : loss 5.888722896575928 - score 0.9149 2021-03-26 05:14:54,488 BAD EPOCHS (no improvement): 2 2021-03-26 05:14:54,489 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:14:55,538 epoch 71 - iter 2/25 - loss 3.02102542 - samples/sec: 122.14 - lr: 0.002344 2021-03-26 05:14:56,471 epoch 71 - iter 4/25 - loss 2.67315236 - samples/sec: 137.54 - lr: 0.002344 2021-03-26 05:14:57,539 epoch 71 - iter 6/25 - loss 2.68371644 - samples/sec: 120.02 - lr: 0.002344 2021-03-26 05:14:58,609 epoch 71 - iter 8/25 - loss 2.78057234 - samples/sec: 119.82 - lr: 0.002344 2021-03-26 05:14:59,655 epoch 71 - iter 10/25 - loss 2.71441923 - samples/sec: 122.49 - lr: 0.002344 2021-03-26 05:15:00,624 epoch 71 - iter 12/25 - loss 2.62355559 - samples/sec: 132.38 - lr: 0.002344 2021-03-26 05:15:01,638 epoch 71 - iter 14/25 - loss 2.59343140 - samples/sec: 126.45 - lr: 0.002344 2021-03-26 05:15:02,641 epoch 71 - iter 16/25 - loss 2.58739737 - samples/sec: 127.71 - lr: 0.002344 2021-03-26 05:15:03,735 epoch 71 - iter 18/25 - loss 2.53276908 - samples/sec: 117.21 - lr: 0.002344 2021-03-26 05:15:04,761 epoch 71 - iter 20/25 - loss 2.48124036 - samples/sec: 124.86 - lr: 0.002344 2021-03-26 05:15:05,992 epoch 71 - iter 22/25 - loss 2.48307181 - samples/sec: 104.05 - lr: 0.002344 2021-03-26 05:15:07,021 epoch 71 - iter 24/25 - loss 2.51121307 - samples/sec: 124.58 - lr: 0.002344 2021-03-26 05:15:07,383 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:15:07,383 EPOCH 71 done: loss 2.5028 - lr 0.0023437 2021-03-26 05:15:08,126 DEV : loss 5.893047332763672 - score 0.9149 2021-03-26 05:15:08,150 BAD EPOCHS (no improvement): 3 2021-03-26 05:15:08,151 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:15:09,154 epoch 72 - iter 2/25 - loss 2.53216600 - samples/sec: 127.79 - lr: 0.002344 2021-03-26 05:15:10,199 epoch 72 - iter 4/25 - loss 2.36523569 - samples/sec: 122.60 - lr: 0.002344 2021-03-26 05:15:11,199 epoch 72 - iter 6/25 - loss 2.33067107 - samples/sec: 128.31 - lr: 0.002344 2021-03-26 05:15:12,208 epoch 72 - iter 8/25 - loss 2.29951569 - samples/sec: 127.19 - lr: 0.002344 2021-03-26 05:15:13,256 epoch 72 - iter 10/25 - loss 2.18574475 - samples/sec: 122.38 - lr: 0.002344 2021-03-26 05:15:14,362 epoch 72 - iter 12/25 - loss 2.24115007 - samples/sec: 115.77 - lr: 0.002344 2021-03-26 05:15:15,302 epoch 72 - iter 14/25 - loss 2.28182931 - samples/sec: 136.40 - lr: 0.002344 2021-03-26 05:15:16,284 epoch 72 - iter 16/25 - loss 2.26094202 - samples/sec: 130.60 - lr: 0.002344 2021-03-26 05:15:17,276 epoch 72 - iter 18/25 - loss 2.30384613 - samples/sec: 129.11 - lr: 0.002344 2021-03-26 05:15:18,324 epoch 72 - iter 20/25 - loss 2.31356340 - samples/sec: 122.35 - lr: 0.002344 2021-03-26 05:15:19,379 epoch 72 - iter 22/25 - loss 2.33352863 - samples/sec: 121.49 - lr: 0.002344 2021-03-26 05:15:20,391 epoch 72 - iter 24/25 - loss 2.37726304 - samples/sec: 126.74 - lr: 0.002344 2021-03-26 05:15:20,819 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:15:20,819 EPOCH 72 done: loss 2.3748 - lr 0.0023437 2021-03-26 05:15:21,559 DEV : loss 5.89636754989624 - score 0.9145 2021-03-26 05:15:21,580 BAD EPOCHS (no improvement): 4 2021-03-26 05:15:21,581 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:15:22,608 epoch 73 - iter 2/25 - loss 2.45547676 - samples/sec: 124.87 - lr: 0.001172 2021-03-26 05:15:23,609 epoch 73 - iter 4/25 - loss 2.33488351 - samples/sec: 128.13 - lr: 0.001172 2021-03-26 05:15:24,542 epoch 73 - iter 6/25 - loss 2.34714580 - samples/sec: 137.56 - lr: 0.001172 2021-03-26 05:15:25,591 epoch 73 - iter 8/25 - loss 2.31484377 - samples/sec: 122.14 - lr: 0.001172 2021-03-26 05:15:26,516 epoch 73 - iter 10/25 - loss 2.39485235 - samples/sec: 138.56 - lr: 0.001172 2021-03-26 05:15:27,651 epoch 73 - iter 12/25 - loss 2.43526415 - samples/sec: 113.01 - lr: 0.001172 2021-03-26 05:15:28,614 epoch 73 - iter 14/25 - loss 2.40032681 - samples/sec: 133.36 - lr: 0.001172 2021-03-26 05:15:29,614 epoch 73 - iter 16/25 - loss 2.45623024 - samples/sec: 128.13 - lr: 0.001172 2021-03-26 05:15:30,586 epoch 73 - iter 18/25 - loss 2.48155908 - samples/sec: 131.99 - lr: 0.001172 2021-03-26 05:15:31,609 epoch 73 - iter 20/25 - loss 2.49938724 - samples/sec: 125.33 - lr: 0.001172 2021-03-26 05:15:32,665 epoch 73 - iter 22/25 - loss 2.49379253 - samples/sec: 121.28 - lr: 0.001172 2021-03-26 05:15:33,706 epoch 73 - iter 24/25 - loss 2.46238670 - samples/sec: 123.33 - lr: 0.001172 2021-03-26 05:15:34,108 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:15:34,110 EPOCH 73 done: loss 2.4599 - lr 0.0011719 2021-03-26 05:15:34,864 DEV : loss 5.895561218261719 - score 0.9153 2021-03-26 05:15:34,888 BAD EPOCHS (no improvement): 1 2021-03-26 05:15:34,888 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:15:35,937 epoch 74 - iter 2/25 - loss 2.04502749 - samples/sec: 122.26 - lr: 0.001172 2021-03-26 05:15:36,960 epoch 74 - iter 4/25 - loss 2.12893891 - samples/sec: 125.32 - lr: 0.001172 2021-03-26 05:15:37,977 epoch 74 - iter 6/25 - loss 2.39949350 - samples/sec: 125.98 - lr: 0.001172 2021-03-26 05:15:38,946 epoch 74 - iter 8/25 - loss 2.45375013 - samples/sec: 132.30 - lr: 0.001172 2021-03-26 05:15:39,907 epoch 74 - iter 10/25 - loss 2.48227482 - samples/sec: 133.42 - lr: 0.001172 2021-03-26 05:15:40,863 epoch 74 - iter 12/25 - loss 2.49502762 - samples/sec: 134.06 - lr: 0.001172 2021-03-26 05:15:41,847 epoch 74 - iter 14/25 - loss 2.54132497 - samples/sec: 130.24 - lr: 0.001172 2021-03-26 05:15:42,813 epoch 74 - iter 16/25 - loss 2.54034129 - samples/sec: 132.68 - lr: 0.001172 2021-03-26 05:15:43,850 epoch 74 - iter 18/25 - loss 2.51002742 - samples/sec: 123.57 - lr: 0.001172 2021-03-26 05:15:44,895 epoch 74 - iter 20/25 - loss 2.48838024 - samples/sec: 122.82 - lr: 0.001172 2021-03-26 05:15:45,960 epoch 74 - iter 22/25 - loss 2.52372887 - samples/sec: 120.32 - lr: 0.001172 2021-03-26 05:15:47,085 epoch 74 - iter 24/25 - loss 2.50319789 - samples/sec: 113.89 - lr: 0.001172 2021-03-26 05:15:47,526 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:15:47,527 EPOCH 74 done: loss 2.5035 - lr 0.0011719 2021-03-26 05:15:48,279 DEV : loss 5.892360210418701 - score 0.9145 2021-03-26 05:15:48,304 BAD EPOCHS (no improvement): 2 2021-03-26 05:15:48,305 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:15:49,330 epoch 75 - iter 2/25 - loss 2.31683660 - samples/sec: 125.14 - lr: 0.001172 2021-03-26 05:15:50,392 epoch 75 - iter 4/25 - loss 2.51113093 - samples/sec: 120.76 - lr: 0.001172 2021-03-26 05:15:51,367 epoch 75 - iter 6/25 - loss 2.62654793 - samples/sec: 131.42 - lr: 0.001172 2021-03-26 05:15:52,388 epoch 75 - iter 8/25 - loss 2.57659081 - samples/sec: 125.51 - lr: 0.001172 2021-03-26 05:15:53,485 epoch 75 - iter 10/25 - loss 2.54353738 - samples/sec: 116.86 - lr: 0.001172 2021-03-26 05:15:54,555 epoch 75 - iter 12/25 - loss 2.55309633 - samples/sec: 119.92 - lr: 0.001172 2021-03-26 05:15:55,570 epoch 75 - iter 14/25 - loss 2.56957877 - samples/sec: 126.40 - lr: 0.001172 2021-03-26 05:15:56,552 epoch 75 - iter 16/25 - loss 2.58506122 - samples/sec: 130.60 - lr: 0.001172 2021-03-26 05:15:57,625 epoch 75 - iter 18/25 - loss 2.55468255 - samples/sec: 119.44 - lr: 0.001172 2021-03-26 05:15:58,646 epoch 75 - iter 20/25 - loss 2.54961177 - samples/sec: 125.73 - lr: 0.001172 2021-03-26 05:15:59,622 epoch 75 - iter 22/25 - loss 2.54893047 - samples/sec: 131.40 - lr: 0.001172 2021-03-26 05:16:00,537 epoch 75 - iter 24/25 - loss 2.52259524 - samples/sec: 140.18 - lr: 0.001172 2021-03-26 05:16:00,912 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:16:00,913 EPOCH 75 done: loss 2.5293 - lr 0.0011719 2021-03-26 05:16:01,658 DEV : loss 5.892311096191406 - score 0.9145 2021-03-26 05:16:01,678 BAD EPOCHS (no improvement): 3 2021-03-26 05:16:01,678 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:16:02,688 epoch 76 - iter 2/25 - loss 2.29732823 - samples/sec: 127.04 - lr: 0.001172 2021-03-26 05:16:03,637 epoch 76 - iter 4/25 - loss 2.20136350 - samples/sec: 135.14 - lr: 0.001172 2021-03-26 05:16:04,571 epoch 76 - iter 6/25 - loss 2.35438251 - samples/sec: 137.34 - lr: 0.001172 2021-03-26 05:16:05,574 epoch 76 - iter 8/25 - loss 2.44377515 - samples/sec: 127.92 - lr: 0.001172 2021-03-26 05:16:06,774 epoch 76 - iter 10/25 - loss 2.59143996 - samples/sec: 106.79 - lr: 0.001172 2021-03-26 05:16:07,862 epoch 76 - iter 12/25 - loss 2.51938006 - samples/sec: 117.82 - lr: 0.001172 2021-03-26 05:16:08,889 epoch 76 - iter 14/25 - loss 2.47963315 - samples/sec: 124.86 - lr: 0.001172 2021-03-26 05:16:09,845 epoch 76 - iter 16/25 - loss 2.48009091 - samples/sec: 134.32 - lr: 0.001172 2021-03-26 05:16:10,790 epoch 76 - iter 18/25 - loss 2.46930445 - samples/sec: 135.56 - lr: 0.001172 2021-03-26 05:16:11,735 epoch 76 - iter 20/25 - loss 2.44070104 - samples/sec: 135.72 - lr: 0.001172 2021-03-26 05:16:12,732 epoch 76 - iter 22/25 - loss 2.45098142 - samples/sec: 128.53 - lr: 0.001172 2021-03-26 05:16:13,835 epoch 76 - iter 24/25 - loss 2.46575201 - samples/sec: 116.25 - lr: 0.001172 2021-03-26 05:16:14,244 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:16:14,245 EPOCH 76 done: loss 2.4803 - lr 0.0011719 2021-03-26 05:16:14,978 DEV : loss 5.895107269287109 - score 0.9153 2021-03-26 05:16:15,002 BAD EPOCHS (no improvement): 4 2021-03-26 05:16:15,003 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:16:16,058 epoch 77 - iter 2/25 - loss 2.50079191 - samples/sec: 121.56 - lr: 0.000586 2021-03-26 05:16:17,007 epoch 77 - iter 4/25 - loss 2.34794635 - samples/sec: 135.02 - lr: 0.000586 2021-03-26 05:16:18,040 epoch 77 - iter 6/25 - loss 2.35746503 - samples/sec: 124.12 - lr: 0.000586 2021-03-26 05:16:18,979 epoch 77 - iter 8/25 - loss 2.38873762 - samples/sec: 136.50 - lr: 0.000586 2021-03-26 05:16:20,061 epoch 77 - iter 10/25 - loss 2.47826626 - samples/sec: 118.39 - lr: 0.000586 2021-03-26 05:16:21,140 epoch 77 - iter 12/25 - loss 2.39502103 - samples/sec: 118.86 - lr: 0.000586 2021-03-26 05:16:22,180 epoch 77 - iter 14/25 - loss 2.39116090 - samples/sec: 123.17 - lr: 0.000586 2021-03-26 05:16:23,213 epoch 77 - iter 16/25 - loss 2.40831371 - samples/sec: 124.12 - lr: 0.000586 2021-03-26 05:16:24,201 epoch 77 - iter 18/25 - loss 2.46315281 - samples/sec: 129.72 - lr: 0.000586 2021-03-26 05:16:25,258 epoch 77 - iter 20/25 - loss 2.53796518 - samples/sec: 121.30 - lr: 0.000586 2021-03-26 05:16:26,328 epoch 77 - iter 22/25 - loss 2.50689035 - samples/sec: 119.73 - lr: 0.000586 2021-03-26 05:16:27,297 epoch 77 - iter 24/25 - loss 2.49591765 - samples/sec: 132.42 - lr: 0.000586 2021-03-26 05:16:27,655 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:16:27,656 EPOCH 77 done: loss 2.4828 - lr 0.0005859 2021-03-26 05:16:28,391 DEV : loss 5.895717620849609 - score 0.9153 2021-03-26 05:16:28,416 BAD EPOCHS (no improvement): 1 2021-03-26 05:16:28,416 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:16:29,466 epoch 78 - iter 2/25 - loss 2.57285833 - samples/sec: 122.15 - lr: 0.000586 2021-03-26 05:16:30,481 epoch 78 - iter 4/25 - loss 2.78017128 - samples/sec: 126.44 - lr: 0.000586 2021-03-26 05:16:31,399 epoch 78 - iter 6/25 - loss 2.68961970 - samples/sec: 139.73 - lr: 0.000586 2021-03-26 05:16:32,386 epoch 78 - iter 8/25 - loss 2.59651193 - samples/sec: 129.87 - lr: 0.000586 2021-03-26 05:16:33,413 epoch 78 - iter 10/25 - loss 2.53945725 - samples/sec: 124.83 - lr: 0.000586 2021-03-26 05:16:34,389 epoch 78 - iter 12/25 - loss 2.55630420 - samples/sec: 131.41 - lr: 0.000586 2021-03-26 05:16:35,423 epoch 78 - iter 14/25 - loss 2.51264097 - samples/sec: 124.17 - lr: 0.000586 2021-03-26 05:16:36,391 epoch 78 - iter 16/25 - loss 2.52606347 - samples/sec: 132.54 - lr: 0.000586 2021-03-26 05:16:37,456 epoch 78 - iter 18/25 - loss 2.55921306 - samples/sec: 120.33 - lr: 0.000586 2021-03-26 05:16:38,424 epoch 78 - iter 20/25 - loss 2.57019910 - samples/sec: 132.37 - lr: 0.000586 2021-03-26 05:16:39,406 epoch 78 - iter 22/25 - loss 2.58772639 - samples/sec: 130.54 - lr: 0.000586 2021-03-26 05:16:40,495 epoch 78 - iter 24/25 - loss 2.58014371 - samples/sec: 117.67 - lr: 0.000586 2021-03-26 05:16:40,987 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:16:40,987 EPOCH 78 done: loss 2.5614 - lr 0.0005859 2021-03-26 05:16:41,754 DEV : loss 5.895739555358887 - score 0.9145 2021-03-26 05:16:41,771 BAD EPOCHS (no improvement): 2 2021-03-26 05:16:41,772 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:16:42,890 epoch 79 - iter 2/25 - loss 2.56692624 - samples/sec: 114.65 - lr: 0.000586 2021-03-26 05:16:43,923 epoch 79 - iter 4/25 - loss 2.41068244 - samples/sec: 124.04 - lr: 0.000586 2021-03-26 05:16:44,935 epoch 79 - iter 6/25 - loss 2.32666834 - samples/sec: 126.65 - lr: 0.000586 2021-03-26 05:16:45,878 epoch 79 - iter 8/25 - loss 2.32099563 - samples/sec: 135.88 - lr: 0.000586 2021-03-26 05:16:47,046 epoch 79 - iter 10/25 - loss 2.39023049 - samples/sec: 109.85 - lr: 0.000586 2021-03-26 05:16:48,037 epoch 79 - iter 12/25 - loss 2.40536650 - samples/sec: 129.47 - lr: 0.000586 2021-03-26 05:16:49,053 epoch 79 - iter 14/25 - loss 2.34349969 - samples/sec: 126.15 - lr: 0.000586 2021-03-26 05:16:50,086 epoch 79 - iter 16/25 - loss 2.35110615 - samples/sec: 124.14 - lr: 0.000586 2021-03-26 05:16:51,225 epoch 79 - iter 18/25 - loss 2.35502756 - samples/sec: 112.50 - lr: 0.000586 2021-03-26 05:16:52,310 epoch 79 - iter 20/25 - loss 2.37096562 - samples/sec: 118.04 - lr: 0.000586 2021-03-26 05:16:53,567 epoch 79 - iter 22/25 - loss 2.38549516 - samples/sec: 102.01 - lr: 0.000586 2021-03-26 05:16:54,747 epoch 79 - iter 24/25 - loss 2.43781242 - samples/sec: 108.60 - lr: 0.000586 2021-03-26 05:16:55,188 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:16:55,190 EPOCH 79 done: loss 2.4295 - lr 0.0005859 2021-03-26 05:16:56,051 DEV : loss 5.895864009857178 - score 0.9145 2021-03-26 05:16:56,074 BAD EPOCHS (no improvement): 3 2021-03-26 05:16:56,075 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:16:57,105 epoch 80 - iter 2/25 - loss 2.79264188 - samples/sec: 124.50 - lr: 0.000586 2021-03-26 05:16:58,139 epoch 80 - iter 4/25 - loss 2.76948452 - samples/sec: 123.99 - lr: 0.000586 2021-03-26 05:16:59,172 epoch 80 - iter 6/25 - loss 2.66697685 - samples/sec: 123.99 - lr: 0.000586 2021-03-26 05:17:00,172 epoch 80 - iter 8/25 - loss 2.63417286 - samples/sec: 128.34 - lr: 0.000586 2021-03-26 05:17:01,246 epoch 80 - iter 10/25 - loss 2.61623344 - samples/sec: 119.38 - lr: 0.000586 2021-03-26 05:17:02,219 epoch 80 - iter 12/25 - loss 2.60252472 - samples/sec: 131.81 - lr: 0.000586 2021-03-26 05:17:03,270 epoch 80 - iter 14/25 - loss 2.58438189 - samples/sec: 121.86 - lr: 0.000586 2021-03-26 05:17:04,387 epoch 80 - iter 16/25 - loss 2.54688759 - samples/sec: 114.80 - lr: 0.000586 2021-03-26 05:17:05,376 epoch 80 - iter 18/25 - loss 2.59013311 - samples/sec: 130.15 - lr: 0.000586 2021-03-26 05:17:06,459 epoch 80 - iter 20/25 - loss 2.53017637 - samples/sec: 118.32 - lr: 0.000586 2021-03-26 05:17:07,503 epoch 80 - iter 22/25 - loss 2.49253048 - samples/sec: 122.79 - lr: 0.000586 2021-03-26 05:17:08,425 epoch 80 - iter 24/25 - loss 2.48274991 - samples/sec: 139.14 - lr: 0.000586 2021-03-26 05:17:08,858 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:17:08,859 EPOCH 80 done: loss 2.4664 - lr 0.0005859 2021-03-26 05:17:09,695 DEV : loss 5.894993782043457 - score 0.9149 2021-03-26 05:17:09,730 BAD EPOCHS (no improvement): 4 2021-03-26 05:17:09,731 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:17:10,817 epoch 81 - iter 2/25 - loss 2.55748820 - samples/sec: 118.03 - lr: 0.000293 2021-03-26 05:17:11,991 epoch 81 - iter 4/25 - loss 2.57345551 - samples/sec: 109.24 - lr: 0.000293 2021-03-26 05:17:13,067 epoch 81 - iter 6/25 - loss 2.54840223 - samples/sec: 119.13 - lr: 0.000293 2021-03-26 05:17:14,169 epoch 81 - iter 8/25 - loss 2.46514797 - samples/sec: 116.34 - lr: 0.000293 2021-03-26 05:17:15,292 epoch 81 - iter 10/25 - loss 2.34481575 - samples/sec: 114.29 - lr: 0.000293 2021-03-26 05:17:16,417 epoch 81 - iter 12/25 - loss 2.35811732 - samples/sec: 114.02 - lr: 0.000293 2021-03-26 05:17:17,416 epoch 81 - iter 14/25 - loss 2.29000000 - samples/sec: 128.35 - lr: 0.000293 2021-03-26 05:17:18,363 epoch 81 - iter 16/25 - loss 2.31538082 - samples/sec: 135.39 - lr: 0.000293 2021-03-26 05:17:19,453 epoch 81 - iter 18/25 - loss 2.32631816 - samples/sec: 117.56 - lr: 0.000293 2021-03-26 05:17:20,445 epoch 81 - iter 20/25 - loss 2.39415182 - samples/sec: 129.27 - lr: 0.000293 2021-03-26 05:17:21,415 epoch 81 - iter 22/25 - loss 2.41178614 - samples/sec: 132.20 - lr: 0.000293 2021-03-26 05:17:22,378 epoch 81 - iter 24/25 - loss 2.42476410 - samples/sec: 133.28 - lr: 0.000293 2021-03-26 05:17:22,787 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:17:22,788 EPOCH 81 done: loss 2.4483 - lr 0.0002930 2021-03-26 05:17:23,546 DEV : loss 5.89434814453125 - score 0.9149 2021-03-26 05:17:23,566 BAD EPOCHS (no improvement): 1 2021-03-26 05:17:23,566 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:17:24,574 epoch 82 - iter 2/25 - loss 2.53205180 - samples/sec: 127.22 - lr: 0.000293 2021-03-26 05:17:25,609 epoch 82 - iter 4/25 - loss 2.74800831 - samples/sec: 123.83 - lr: 0.000293 2021-03-26 05:17:26,804 epoch 82 - iter 6/25 - loss 2.64432458 - samples/sec: 107.28 - lr: 0.000293 2021-03-26 05:17:27,892 epoch 82 - iter 8/25 - loss 2.65834966 - samples/sec: 117.75 - lr: 0.000293 2021-03-26 05:17:28,872 epoch 82 - iter 10/25 - loss 2.64527481 - samples/sec: 130.87 - lr: 0.000293 2021-03-26 05:17:29,960 epoch 82 - iter 12/25 - loss 2.59270751 - samples/sec: 117.89 - lr: 0.000293 2021-03-26 05:17:31,073 epoch 82 - iter 14/25 - loss 2.53665956 - samples/sec: 115.22 - lr: 0.000293 2021-03-26 05:17:32,158 epoch 82 - iter 16/25 - loss 2.57464282 - samples/sec: 118.07 - lr: 0.000293 2021-03-26 05:17:33,226 epoch 82 - iter 18/25 - loss 2.56387957 - samples/sec: 120.00 - lr: 0.000293 2021-03-26 05:17:34,313 epoch 82 - iter 20/25 - loss 2.53260517 - samples/sec: 117.98 - lr: 0.000293 2021-03-26 05:17:35,219 epoch 82 - iter 22/25 - loss 2.50258301 - samples/sec: 141.54 - lr: 0.000293 2021-03-26 05:17:36,244 epoch 82 - iter 24/25 - loss 2.49314828 - samples/sec: 124.97 - lr: 0.000293 2021-03-26 05:17:36,689 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:17:36,690 EPOCH 82 done: loss 2.4795 - lr 0.0002930 2021-03-26 05:17:37,457 DEV : loss 5.894433498382568 - score 0.9149 2021-03-26 05:17:37,475 BAD EPOCHS (no improvement): 2 2021-03-26 05:17:37,475 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:17:38,444 epoch 83 - iter 2/25 - loss 2.64499736 - samples/sec: 132.45 - lr: 0.000293 2021-03-26 05:17:39,536 epoch 83 - iter 4/25 - loss 2.47873867 - samples/sec: 117.36 - lr: 0.000293 2021-03-26 05:17:40,528 epoch 83 - iter 6/25 - loss 2.45670923 - samples/sec: 129.37 - lr: 0.000293 2021-03-26 05:17:41,576 epoch 83 - iter 8/25 - loss 2.44804201 - samples/sec: 122.37 - lr: 0.000293 2021-03-26 05:17:42,539 epoch 83 - iter 10/25 - loss 2.41737645 - samples/sec: 133.09 - lr: 0.000293 2021-03-26 05:17:43,639 epoch 83 - iter 12/25 - loss 2.54113217 - samples/sec: 116.60 - lr: 0.000293 2021-03-26 05:17:44,607 epoch 83 - iter 14/25 - loss 2.51169845 - samples/sec: 132.33 - lr: 0.000293 2021-03-26 05:17:45,556 epoch 83 - iter 16/25 - loss 2.45529628 - samples/sec: 135.07 - lr: 0.000293 2021-03-26 05:17:46,709 epoch 83 - iter 18/25 - loss 2.44203492 - samples/sec: 111.10 - lr: 0.000293 2021-03-26 05:17:47,775 epoch 83 - iter 20/25 - loss 2.44877447 - samples/sec: 120.43 - lr: 0.000293 2021-03-26 05:17:48,799 epoch 83 - iter 22/25 - loss 2.43847547 - samples/sec: 125.11 - lr: 0.000293 2021-03-26 05:17:49,865 epoch 83 - iter 24/25 - loss 2.41514853 - samples/sec: 120.24 - lr: 0.000293 2021-03-26 05:17:50,383 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:17:50,384 EPOCH 83 done: loss 2.4312 - lr 0.0002930 2021-03-26 05:17:51,148 DEV : loss 5.894194602966309 - score 0.9149 2021-03-26 05:17:51,173 BAD EPOCHS (no improvement): 3 2021-03-26 05:17:51,173 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:17:52,185 epoch 84 - iter 2/25 - loss 2.68529451 - samples/sec: 126.80 - lr: 0.000293 2021-03-26 05:17:53,304 epoch 84 - iter 4/25 - loss 2.73050523 - samples/sec: 114.46 - lr: 0.000293 2021-03-26 05:17:54,349 epoch 84 - iter 6/25 - loss 2.87833659 - samples/sec: 122.86 - lr: 0.000293 2021-03-26 05:17:55,398 epoch 84 - iter 8/25 - loss 2.77546391 - samples/sec: 122.19 - lr: 0.000293 2021-03-26 05:17:56,412 epoch 84 - iter 10/25 - loss 2.66991322 - samples/sec: 126.45 - lr: 0.000293 2021-03-26 05:17:57,450 epoch 84 - iter 12/25 - loss 2.60607652 - samples/sec: 123.46 - lr: 0.000293 2021-03-26 05:17:58,487 epoch 84 - iter 14/25 - loss 2.59675641 - samples/sec: 123.65 - lr: 0.000293 2021-03-26 05:17:59,547 epoch 84 - iter 16/25 - loss 2.57955612 - samples/sec: 121.00 - lr: 0.000293 2021-03-26 05:18:00,699 epoch 84 - iter 18/25 - loss 2.57765302 - samples/sec: 111.25 - lr: 0.000293 2021-03-26 05:18:01,740 epoch 84 - iter 20/25 - loss 2.54013187 - samples/sec: 123.33 - lr: 0.000293 2021-03-26 05:18:02,736 epoch 84 - iter 22/25 - loss 2.53648597 - samples/sec: 128.65 - lr: 0.000293 2021-03-26 05:18:03,676 epoch 84 - iter 24/25 - loss 2.51152799 - samples/sec: 136.43 - lr: 0.000293 2021-03-26 05:18:04,070 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:04,071 EPOCH 84 done: loss 2.5123 - lr 0.0002930 2021-03-26 05:18:04,829 DEV : loss 5.8933563232421875 - score 0.9153 2021-03-26 05:18:04,853 BAD EPOCHS (no improvement): 4 2021-03-26 05:18:04,854 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:05,828 epoch 85 - iter 2/25 - loss 1.89106965 - samples/sec: 131.72 - lr: 0.000146 2021-03-26 05:18:06,827 epoch 85 - iter 4/25 - loss 2.02825582 - samples/sec: 128.21 - lr: 0.000146 2021-03-26 05:18:07,851 epoch 85 - iter 6/25 - loss 2.21381227 - samples/sec: 125.20 - lr: 0.000146 2021-03-26 05:18:08,779 epoch 85 - iter 8/25 - loss 2.27279621 - samples/sec: 138.20 - lr: 0.000146 2021-03-26 05:18:09,736 epoch 85 - iter 10/25 - loss 2.23479022 - samples/sec: 134.07 - lr: 0.000146 2021-03-26 05:18:10,719 epoch 85 - iter 12/25 - loss 2.35115473 - samples/sec: 130.41 - lr: 0.000146 2021-03-26 05:18:11,775 epoch 85 - iter 14/25 - loss 2.36515174 - samples/sec: 121.38 - lr: 0.000146 2021-03-26 05:18:12,720 epoch 85 - iter 16/25 - loss 2.35759029 - samples/sec: 135.56 - lr: 0.000146 2021-03-26 05:18:13,776 epoch 85 - iter 18/25 - loss 2.45850852 - samples/sec: 121.35 - lr: 0.000146 2021-03-26 05:18:14,894 epoch 85 - iter 20/25 - loss 2.40584831 - samples/sec: 114.71 - lr: 0.000146 2021-03-26 05:18:15,826 epoch 85 - iter 22/25 - loss 2.40154030 - samples/sec: 137.61 - lr: 0.000146 2021-03-26 05:18:16,847 epoch 85 - iter 24/25 - loss 2.39086893 - samples/sec: 125.51 - lr: 0.000146 2021-03-26 05:18:17,362 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:17,362 EPOCH 85 done: loss 2.3930 - lr 0.0001465 2021-03-26 05:18:18,105 DEV : loss 5.893399238586426 - score 0.9157 2021-03-26 05:18:18,129 BAD EPOCHS (no improvement): 1 2021-03-26 05:18:18,130 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:19,065 epoch 86 - iter 2/25 - loss 2.54171717 - samples/sec: 137.16 - lr: 0.000146 2021-03-26 05:18:19,991 epoch 86 - iter 4/25 - loss 2.48442131 - samples/sec: 138.64 - lr: 0.000146 2021-03-26 05:18:21,000 epoch 86 - iter 6/25 - loss 2.51889253 - samples/sec: 127.10 - lr: 0.000146 2021-03-26 05:18:21,992 epoch 86 - iter 8/25 - loss 2.56522778 - samples/sec: 129.29 - lr: 0.000146 2021-03-26 05:18:23,055 epoch 86 - iter 10/25 - loss 2.58503950 - samples/sec: 120.51 - lr: 0.000146 2021-03-26 05:18:24,115 epoch 86 - iter 12/25 - loss 2.57443593 - samples/sec: 120.91 - lr: 0.000146 2021-03-26 05:18:25,045 epoch 86 - iter 14/25 - loss 2.51936642 - samples/sec: 138.08 - lr: 0.000146 2021-03-26 05:18:26,061 epoch 86 - iter 16/25 - loss 2.50782646 - samples/sec: 126.09 - lr: 0.000146 2021-03-26 05:18:27,083 epoch 86 - iter 18/25 - loss 2.51853973 - samples/sec: 125.46 - lr: 0.000146 2021-03-26 05:18:28,032 epoch 86 - iter 20/25 - loss 2.46419308 - samples/sec: 135.59 - lr: 0.000146 2021-03-26 05:18:29,187 epoch 86 - iter 22/25 - loss 2.43465135 - samples/sec: 111.02 - lr: 0.000146 2021-03-26 05:18:30,187 epoch 86 - iter 24/25 - loss 2.45168869 - samples/sec: 128.31 - lr: 0.000146 2021-03-26 05:18:30,669 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:30,669 EPOCH 86 done: loss 2.4679 - lr 0.0001465 2021-03-26 05:18:31,446 DEV : loss 5.893395900726318 - score 0.9153 2021-03-26 05:18:31,463 BAD EPOCHS (no improvement): 2 2021-03-26 05:18:31,464 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:32,535 epoch 87 - iter 2/25 - loss 2.91735208 - samples/sec: 119.70 - lr: 0.000146 2021-03-26 05:18:33,557 epoch 87 - iter 4/25 - loss 2.84264398 - samples/sec: 125.54 - lr: 0.000146 2021-03-26 05:18:34,544 epoch 87 - iter 6/25 - loss 2.56568110 - samples/sec: 129.80 - lr: 0.000146 2021-03-26 05:18:35,558 epoch 87 - iter 8/25 - loss 2.63990408 - samples/sec: 126.46 - lr: 0.000146 2021-03-26 05:18:36,455 epoch 87 - iter 10/25 - loss 2.56502144 - samples/sec: 142.83 - lr: 0.000146 2021-03-26 05:18:37,387 epoch 87 - iter 12/25 - loss 2.53154775 - samples/sec: 137.50 - lr: 0.000146 2021-03-26 05:18:38,397 epoch 87 - iter 14/25 - loss 2.54436823 - samples/sec: 126.86 - lr: 0.000146 2021-03-26 05:18:39,401 epoch 87 - iter 16/25 - loss 2.61185640 - samples/sec: 127.69 - lr: 0.000146 2021-03-26 05:18:40,370 epoch 87 - iter 18/25 - loss 2.54913453 - samples/sec: 132.51 - lr: 0.000146 2021-03-26 05:18:41,463 epoch 87 - iter 20/25 - loss 2.50478936 - samples/sec: 117.20 - lr: 0.000146 2021-03-26 05:18:42,489 epoch 87 - iter 22/25 - loss 2.52266684 - samples/sec: 124.89 - lr: 0.000146 2021-03-26 05:18:43,496 epoch 87 - iter 24/25 - loss 2.56201922 - samples/sec: 127.35 - lr: 0.000146 2021-03-26 05:18:43,856 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:43,856 EPOCH 87 done: loss 2.5356 - lr 0.0001465 2021-03-26 05:18:44,595 DEV : loss 5.893139839172363 - score 0.9153 2021-03-26 05:18:44,620 BAD EPOCHS (no improvement): 3 2021-03-26 05:18:44,621 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:45,646 epoch 88 - iter 2/25 - loss 2.54361033 - samples/sec: 125.02 - lr: 0.000146 2021-03-26 05:18:46,620 epoch 88 - iter 4/25 - loss 2.52765530 - samples/sec: 131.69 - lr: 0.000146 2021-03-26 05:18:47,691 epoch 88 - iter 6/25 - loss 2.64498627 - samples/sec: 119.75 - lr: 0.000146 2021-03-26 05:18:48,676 epoch 88 - iter 8/25 - loss 2.57745424 - samples/sec: 130.13 - lr: 0.000146 2021-03-26 05:18:49,701 epoch 88 - iter 10/25 - loss 2.45820544 - samples/sec: 125.23 - lr: 0.000146 2021-03-26 05:18:50,694 epoch 88 - iter 12/25 - loss 2.43848614 - samples/sec: 129.11 - lr: 0.000146 2021-03-26 05:18:51,837 epoch 88 - iter 14/25 - loss 2.40640150 - samples/sec: 112.61 - lr: 0.000146 2021-03-26 05:18:52,925 epoch 88 - iter 16/25 - loss 2.45308697 - samples/sec: 117.86 - lr: 0.000146 2021-03-26 05:18:53,985 epoch 88 - iter 18/25 - loss 2.43245529 - samples/sec: 120.87 - lr: 0.000146 2021-03-26 05:18:54,902 epoch 88 - iter 20/25 - loss 2.46451479 - samples/sec: 139.94 - lr: 0.000146 2021-03-26 05:18:55,882 epoch 88 - iter 22/25 - loss 2.47260539 - samples/sec: 130.79 - lr: 0.000146 2021-03-26 05:18:57,029 epoch 88 - iter 24/25 - loss 2.48033151 - samples/sec: 111.84 - lr: 0.000146 2021-03-26 05:18:57,437 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:57,438 EPOCH 88 done: loss 2.4450 - lr 0.0001465 2021-03-26 05:18:58,197 DEV : loss 5.893143653869629 - score 0.9153 2021-03-26 05:18:58,222 BAD EPOCHS (no improvement): 4 2021-03-26 05:18:58,223 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:58,223 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:18:58,224 learning rate too small - quitting training! 2021-03-26 05:18:58,224 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:07,624 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:07,624 Testing using best model ... 2021-03-26 05:19:07,625 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260453/best-model.pt 2021-03-26 05:19:15,030 0.9089 2021-03-26 05:19:15,030 Results: - F-score (micro): 0.9058 - F-score (macro): 0.5975 - Accuracy (incl. no class): 0.9089 By class: precision recall f1-score support SCONJ 0.9697 0.9412 0.9552 34 PRON 0.9453 0.9744 0.9596 195 ADP 0.9604 0.9604 0.9604 101 AUX 0.9118 0.9688 0.9394 32 VERB 0.9615 0.9058 0.9328 138 ADV 0.9160 0.8934 0.9046 122 PART 0.9583 0.9684 0.9634 190 NOUN 0.9224 0.9378 0.9300 418 CCONJ 1.0000 0.9438 0.9711 89 ADJ 0.9205 0.8710 0.8950 93 DET 0.9545 0.8936 0.9231 47 NUM 0.9048 0.7917 0.8444 24 PUNCT 1.0000 1.0000 1.0000 30 PROPN 0.8000 0.8000 0.8000 20 INTJ 1.0000 1.0000 1.0000 19 X 1.0000 0.0000 0.0000 1 FOREIGN 1.0000 1.0000 1.0000 3 MENTION 1.0000 1.0000 1.0000 20 NOUN+NSUFF 0.8000 0.9180 0.8550 61 ADJ+NSUFF 0.7500 0.6562 0.7000 32 CONJ+NOUN+NSUFF 1.0000 1.0000 1.0000 1 DET+NOUN 0.9726 0.9861 0.9793 72 V 0.8500 0.8500 0.8500 80 PREP+DET+NOUN 0.6923 0.9000 0.7826 10 CONJ+PART 1.0000 1.0000 1.0000 9 PREP+V 1.0000 0.0000 0.0000 2 CONJ+V 0.7857 0.8462 0.8148 13 PREP 0.9219 0.9833 0.9516 60 DET+NOUN+NSUFF 0.9167 0.9167 0.9167 24 DET+ADJ+NSUFF 0.6667 0.8000 0.7273 5 EOS 1.0000 1.0000 1.0000 70 NOUN+PRON 0.7901 0.8533 0.8205 75 V+PRON 0.7541 0.8070 0.7797 57 CONJ+NOUN+PRON 0.7500 0.5000 0.6000 6 PUNC 0.9919 1.0000 0.9960 123 V+PRON+PRON 0.5455 0.5000 0.5217 12 CONJ+V+PREP+PRON 0.0000 0.0000 0.0000 2 CONJ+V+PRON 0.4444 0.8000 0.5714 5 PROG_PART+V 0.7500 0.8462 0.7952 39 PART+PRON 1.0000 0.9444 0.9714 18 CONJ 1.0000 1.0000 1.0000 44 V+PREP+PRON 0.3333 0.4000 0.3636 5 PART+V+PRON 1.0000 1.0000 1.0000 1 CONJ+PRON 1.0000 1.0000 1.0000 6 PRON+DET+NOUN 1.0000 0.2500 0.4000 4 PRON+DET+NOUN+NSUFF 0.3333 1.0000 0.5000 1 PREP+PART+PRON 1.0000 0.8000 0.8889 5 PROG_PART+V+PRON 0.6000 0.9231 0.7273 13 PREP+DET+ADJ 1.0000 0.0000 0.0000 3 PREP+NOUN 0.7500 0.6000 0.6667 10 PREP+NOUN+PRON 0.5000 0.1667 0.2500 6 PREP+DET+NOUN+NSUFF 1.0000 0.5000 0.6667 2 HASH 0.9688 1.0000 0.9841 31 FUT_PART+V 0.8571 0.5455 0.6667 11 PREP+PRON 0.8947 0.8500 0.8718 20 PREP+PRON+DET 1.0000 0.0000 0.0000 1 CONJ+DET+NOUN 1.0000 1.0000 1.0000 6 NOUN+PREP+PRON 1.0000 0.0000 0.0000 1 V+PRON+PREP+PRON 0.0000 0.0000 0.0000 1 NOUN+NSUFF+PRON 0.5385 0.7000 0.6087 10 FUT_PART 1.0000 1.0000 1.0000 5 CONJ+ADJ 1.0000 1.0000 1.0000 1 PREP+NOUN+NSUFF+PRON 0.6667 0.6667 0.6667 3 PART+V 0.0000 0.0000 0.0000 1 PART+NOUN+PRON 1.0000 0.0000 0.0000 1 PROG_PART+V+PREP+PRON 1.0000 0.0000 0.0000 2 ADJ+PRON 0.6667 0.8000 0.7273 5 EMOT 1.0000 1.0000 1.0000 10 CONJ+FUT_PART+V 1.0000 0.0000 0.0000 3 CONJ+PROG_PART+V 0.2500 1.0000 0.4000 1 PREP+NOUN+NSUFF 1.0000 1.0000 1.0000 1 PART+V+NEG_PART 0.5000 0.5000 0.5000 4 CONJ+PREP+PRON 1.0000 1.0000 1.0000 1 PREP+PART 1.0000 0.0000 0.0000 1 CONJ+NOUN 0.8333 0.8333 0.8333 6 ADJ+NSUFF+PRON 1.0000 0.0000 0.0000 1 PROG_PART+V+NEG_PART 0.0000 1.0000 0.0000 0 NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 ADV+PRON 1.0000 0.0000 0.0000 1 PROG_PART+V+PRON+PRON 0.0000 1.0000 0.0000 0 PART+FUT_PART 1.0000 0.0000 0.0000 1 URL 1.0000 1.0000 1.0000 1 CONJ+PART+V+PRON 1.0000 0.0000 0.0000 1 CONJ+PART+PROG_PART+V 1.0000 0.0000 0.0000 1 PART+NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 NUM+NSUFF 1.0000 0.0000 0.0000 1 CONJ+NOUN+NSUFF+PRON 0.0000 1.0000 0.0000 0 ADV+NSUFF 1.0000 0.6667 0.8000 3 PART+V+PRON+NEG_PART 0.4286 0.7500 0.5455 4 NOUN+CASE 1.0000 0.7500 0.8571 4 DET+ADJ 0.8750 1.0000 0.9333 7 ADJ+CASE 1.0000 1.0000 1.0000 3 PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+PROG_PART+V+NEG_PART 0.5000 1.0000 0.6667 1 PART+NOUN+NEG_PART 1.0000 0.0000 0.0000 2 PART+PREP+NEG_PART 1.0000 1.0000 1.0000 4 DET+NUM 1.0000 0.0000 0.0000 1 PART+NOUN 1.0000 1.0000 1.0000 1 PART+PART 1.0000 0.0000 0.0000 1 CONJ+PROG_PART+V+PRON 1.0000 1.0000 1.0000 1 FUT_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 CONJ+ADV 1.0000 1.0000 1.0000 1 V+NEG_PART 1.0000 0.0000 0.0000 1 micro avg 0.9058 0.9058 0.9058 2623 macro avg 0.8270 0.6506 0.5975 2623 weighted avg 0.9135 0.9058 0.9019 2623 2021-03-26 05:19:15,031 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:15,031 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:21,164 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC 2021-03-26 05:19:21,165 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu 2021-03-26 05:19:21,165 Dev: None 2021-03-26 05:19:21,165 Test: None 2021-03-26 05:19:21,457 Reading data from ../../Datasets_adhoc/UD_MADAR 2021-03-26 05:19:21,458 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu 2021-03-26 05:19:21,458 Dev: None 2021-03-26 05:19:21,458 Test: None 2021-03-26 05:19:22,836 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 05:19:22,836 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt 2021-03-26 05:19:22,837 Dev: None 2021-03-26 05:19:22,837 Test: None 2021-03-26 05:19:23,009 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 05:19:23,009 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt 2021-03-26 05:19:23,009 Dev: None 2021-03-26 05:19:23,010 Test: None 2021-03-26 05:19:23,174 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 05:19:23,175 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt 2021-03-26 05:19:23,175 Dev: None 2021-03-26 05:19:23,175 Test: None 2021-03-26 05:19:23,333 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 05:19:23,333 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt 2021-03-26 05:19:23,333 Dev: None 2021-03-26 05:19:23,333 Test: None 2021-03-26 05:19:23,489 Filtering long sentences 2021-03-26 05:19:23,531 MultiCorpus: 1573 train + 177 dev + 194 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences 2021-03-26 05:19:23,936 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:23,937 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): WordEmbeddings('ar') (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) (list_embedding_2): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=206, bias=True) (beta): 1.0 (weights): None (weight_tensor) None )" 2021-03-26 05:19:23,937 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:23,937 Corpus: "MultiCorpus: 1573 train + 177 dev + 194 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" 2021-03-26 05:19:23,938 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:23,938 Parameters: 2021-03-26 05:19:23,938 - learning_rate: "0.3" 2021-03-26 05:19:23,938 - mini_batch_size: "64" 2021-03-26 05:19:23,938 - patience: "3" 2021-03-26 05:19:23,939 - anneal_factor: "0.5" 2021-03-26 05:19:23,939 - max_epochs: "150" 2021-03-26 05:19:23,939 - shuffle: "True" 2021-03-26 05:19:23,939 - train_with_dev: "False" 2021-03-26 05:19:23,940 - batch_growth_annealing: "False" 2021-03-26 05:19:23,940 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:23,940 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260519" 2021-03-26 05:19:23,940 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:23,941 Device: cuda:0 2021-03-26 05:19:23,941 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:23,941 Embeddings storage mode: cpu 2021-03-26 05:19:23,943 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:25,612 epoch 1 - iter 2/25 - loss 81.95887375 - samples/sec: 76.75 - lr: 0.300000 2021-03-26 05:19:26,880 epoch 1 - iter 4/25 - loss 74.30733109 - samples/sec: 101.12 - lr: 0.300000 2021-03-26 05:19:28,119 epoch 1 - iter 6/25 - loss 70.52834956 - samples/sec: 103.39 - lr: 0.300000 2021-03-26 05:19:29,401 epoch 1 - iter 8/25 - loss 68.74974346 - samples/sec: 99.98 - lr: 0.300000 2021-03-26 05:19:30,716 epoch 1 - iter 10/25 - loss 66.45365906 - samples/sec: 97.44 - lr: 0.300000 2021-03-26 05:19:31,966 epoch 1 - iter 12/25 - loss 65.50715129 - samples/sec: 102.53 - lr: 0.300000 2021-03-26 05:19:33,366 epoch 1 - iter 14/25 - loss 63.79074342 - samples/sec: 91.50 - lr: 0.300000 2021-03-26 05:19:34,641 epoch 1 - iter 16/25 - loss 62.57994103 - samples/sec: 100.49 - lr: 0.300000 2021-03-26 05:19:36,005 epoch 1 - iter 18/25 - loss 61.44075076 - samples/sec: 93.93 - lr: 0.300000 2021-03-26 05:19:37,361 epoch 1 - iter 20/25 - loss 60.14129944 - samples/sec: 94.47 - lr: 0.300000 2021-03-26 05:19:38,797 epoch 1 - iter 22/25 - loss 59.64039404 - samples/sec: 89.22 - lr: 0.300000 2021-03-26 05:19:40,124 epoch 1 - iter 24/25 - loss 58.65103499 - samples/sec: 96.54 - lr: 0.300000 2021-03-26 05:19:40,645 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:40,645 EPOCH 1 done: loss 57.9464 - lr 0.3000000 2021-03-26 05:19:41,921 DEV : loss 43.50901412963867 - score 0.3135 2021-03-26 05:19:41,947 BAD EPOCHS (no improvement): 0 2021-03-26 05:19:51,382 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:19:52,367 epoch 2 - iter 2/25 - loss 44.36688232 - samples/sec: 130.47 - lr: 0.300000 2021-03-26 05:19:53,356 epoch 2 - iter 4/25 - loss 42.73228741 - samples/sec: 129.72 - lr: 0.300000 2021-03-26 05:19:54,393 epoch 2 - iter 6/25 - loss 40.84166209 - samples/sec: 123.59 - lr: 0.300000 2021-03-26 05:19:55,413 epoch 2 - iter 8/25 - loss 40.15511179 - samples/sec: 125.69 - lr: 0.300000 2021-03-26 05:19:56,447 epoch 2 - iter 10/25 - loss 40.64278603 - samples/sec: 124.04 - lr: 0.300000 2021-03-26 05:19:57,458 epoch 2 - iter 12/25 - loss 40.30080064 - samples/sec: 126.89 - lr: 0.300000 2021-03-26 05:19:58,498 epoch 2 - iter 14/25 - loss 40.01132883 - samples/sec: 123.24 - lr: 0.300000 2021-03-26 05:19:59,430 epoch 2 - iter 16/25 - loss 39.21363187 - samples/sec: 137.60 - lr: 0.300000 2021-03-26 05:20:00,477 epoch 2 - iter 18/25 - loss 38.71213330 - samples/sec: 122.37 - lr: 0.300000 2021-03-26 05:20:01,407 epoch 2 - iter 20/25 - loss 37.77274303 - samples/sec: 137.85 - lr: 0.300000 2021-03-26 05:20:02,361 epoch 2 - iter 22/25 - loss 37.60560018 - samples/sec: 134.37 - lr: 0.300000 2021-03-26 05:20:03,321 epoch 2 - iter 24/25 - loss 37.28476222 - samples/sec: 133.44 - lr: 0.300000 2021-03-26 05:20:03,656 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:20:03,656 EPOCH 2 done: loss 36.7479 - lr 0.3000000 2021-03-26 05:20:04,447 DEV : loss 33.068904876708984 - score 0.4714 2021-03-26 05:20:04,465 BAD EPOCHS (no improvement): 0 2021-03-26 05:20:14,204 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:20:15,255 epoch 3 - iter 2/25 - loss 31.90020370 - samples/sec: 122.13 - lr: 0.300000 2021-03-26 05:20:16,285 epoch 3 - iter 4/25 - loss 31.23775816 - samples/sec: 124.44 - lr: 0.300000 2021-03-26 05:20:17,265 epoch 3 - iter 6/25 - loss 30.26333078 - samples/sec: 131.63 - lr: 0.300000 2021-03-26 05:20:18,213 epoch 3 - iter 8/25 - loss 29.47907281 - samples/sec: 135.10 - lr: 0.300000 2021-03-26 05:20:19,165 epoch 3 - iter 10/25 - loss 29.23647823 - samples/sec: 134.77 - lr: 0.300000 2021-03-26 05:20:20,207 epoch 3 - iter 12/25 - loss 29.02191750 - samples/sec: 123.00 - lr: 0.300000 2021-03-26 05:20:21,277 epoch 3 - iter 14/25 - loss 29.22410951 - samples/sec: 119.75 - lr: 0.300000 2021-03-26 05:20:22,232 epoch 3 - iter 16/25 - loss 29.01444256 - samples/sec: 134.22 - lr: 0.300000 2021-03-26 05:20:23,303 epoch 3 - iter 18/25 - loss 28.52765550 - samples/sec: 119.69 - lr: 0.300000 2021-03-26 05:20:24,415 epoch 3 - iter 20/25 - loss 28.52102280 - samples/sec: 115.28 - lr: 0.300000 2021-03-26 05:20:25,377 epoch 3 - iter 22/25 - loss 28.27422359 - samples/sec: 133.24 - lr: 0.300000 2021-03-26 05:20:26,336 epoch 3 - iter 24/25 - loss 27.83582926 - samples/sec: 133.70 - lr: 0.300000 2021-03-26 05:20:26,824 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:20:26,825 EPOCH 3 done: loss 27.8058 - lr 0.3000000 2021-03-26 05:20:27,615 DEV : loss 22.344778060913086 - score 0.6352 2021-03-26 05:20:27,642 BAD EPOCHS (no improvement): 0 2021-03-26 05:20:37,530 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:20:38,578 epoch 4 - iter 2/25 - loss 23.20929146 - samples/sec: 122.51 - lr: 0.300000 2021-03-26 05:20:39,519 epoch 4 - iter 4/25 - loss 23.56956482 - samples/sec: 136.19 - lr: 0.300000 2021-03-26 05:20:40,446 epoch 4 - iter 6/25 - loss 22.66243076 - samples/sec: 138.36 - lr: 0.300000 2021-03-26 05:20:41,449 epoch 4 - iter 8/25 - loss 23.12808657 - samples/sec: 127.94 - lr: 0.300000 2021-03-26 05:20:42,463 epoch 4 - iter 10/25 - loss 23.03308296 - samples/sec: 126.45 - lr: 0.300000 2021-03-26 05:20:43,457 epoch 4 - iter 12/25 - loss 23.30867656 - samples/sec: 128.93 - lr: 0.300000 2021-03-26 05:20:44,382 epoch 4 - iter 14/25 - loss 22.81305817 - samples/sec: 138.56 - lr: 0.300000 2021-03-26 05:20:45,381 epoch 4 - iter 16/25 - loss 22.66594744 - samples/sec: 128.33 - lr: 0.300000 2021-03-26 05:20:46,427 epoch 4 - iter 18/25 - loss 22.44728237 - samples/sec: 122.62 - lr: 0.300000 2021-03-26 05:20:47,545 epoch 4 - iter 20/25 - loss 22.31258392 - samples/sec: 115.19 - lr: 0.300000 2021-03-26 05:20:48,552 epoch 4 - iter 22/25 - loss 22.11111285 - samples/sec: 127.39 - lr: 0.300000 2021-03-26 05:20:49,610 epoch 4 - iter 24/25 - loss 22.03010790 - samples/sec: 121.30 - lr: 0.300000 2021-03-26 05:20:50,058 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:20:50,058 EPOCH 4 done: loss 22.0868 - lr 0.3000000 2021-03-26 05:20:50,854 DEV : loss 17.412494659423828 - score 0.7033 2021-03-26 05:20:50,873 BAD EPOCHS (no improvement): 0 2021-03-26 05:21:00,674 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:21:01,811 epoch 5 - iter 2/25 - loss 22.17486286 - samples/sec: 112.84 - lr: 0.300000 2021-03-26 05:21:02,788 epoch 5 - iter 4/25 - loss 21.01916075 - samples/sec: 131.36 - lr: 0.300000 2021-03-26 05:21:03,715 epoch 5 - iter 6/25 - loss 20.32295863 - samples/sec: 138.47 - lr: 0.300000 2021-03-26 05:21:04,677 epoch 5 - iter 8/25 - loss 19.95706081 - samples/sec: 133.45 - lr: 0.300000 2021-03-26 05:21:05,654 epoch 5 - iter 10/25 - loss 19.42259941 - samples/sec: 131.30 - lr: 0.300000 2021-03-26 05:21:06,592 epoch 5 - iter 12/25 - loss 19.28429031 - samples/sec: 136.74 - lr: 0.300000 2021-03-26 05:21:07,655 epoch 5 - iter 14/25 - loss 19.41569465 - samples/sec: 120.63 - lr: 0.300000 2021-03-26 05:21:08,657 epoch 5 - iter 16/25 - loss 19.41432917 - samples/sec: 127.88 - lr: 0.300000 2021-03-26 05:21:09,640 epoch 5 - iter 18/25 - loss 19.03300916 - samples/sec: 130.35 - lr: 0.300000 2021-03-26 05:21:10,581 epoch 5 - iter 20/25 - loss 18.95893478 - samples/sec: 136.20 - lr: 0.300000 2021-03-26 05:21:11,511 epoch 5 - iter 22/25 - loss 18.87176644 - samples/sec: 137.83 - lr: 0.300000 2021-03-26 05:21:12,484 epoch 5 - iter 24/25 - loss 18.72764615 - samples/sec: 131.73 - lr: 0.300000 2021-03-26 05:21:12,918 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:21:12,918 EPOCH 5 done: loss 18.6854 - lr 0.3000000 2021-03-26 05:21:13,709 DEV : loss 14.548635482788086 - score 0.7653 2021-03-26 05:21:13,736 BAD EPOCHS (no improvement): 0 2021-03-26 05:21:23,422 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:21:24,410 epoch 6 - iter 2/25 - loss 15.13766766 - samples/sec: 129.88 - lr: 0.300000 2021-03-26 05:21:25,573 epoch 6 - iter 4/25 - loss 16.41607046 - samples/sec: 110.14 - lr: 0.300000 2021-03-26 05:21:26,539 epoch 6 - iter 6/25 - loss 16.33872000 - samples/sec: 132.86 - lr: 0.300000 2021-03-26 05:21:27,528 epoch 6 - iter 8/25 - loss 16.65881395 - samples/sec: 129.65 - lr: 0.300000 2021-03-26 05:21:28,566 epoch 6 - iter 10/25 - loss 16.38934278 - samples/sec: 123.48 - lr: 0.300000 2021-03-26 05:21:29,472 epoch 6 - iter 12/25 - loss 15.87123386 - samples/sec: 141.46 - lr: 0.300000 2021-03-26 05:21:30,478 epoch 6 - iter 14/25 - loss 15.92985732 - samples/sec: 127.35 - lr: 0.300000 2021-03-26 05:21:31,514 epoch 6 - iter 16/25 - loss 15.99678320 - samples/sec: 123.90 - lr: 0.300000 2021-03-26 05:21:32,521 epoch 6 - iter 18/25 - loss 15.88874594 - samples/sec: 127.46 - lr: 0.300000 2021-03-26 05:21:33,613 epoch 6 - iter 20/25 - loss 16.16925859 - samples/sec: 117.39 - lr: 0.300000 2021-03-26 05:21:34,611 epoch 6 - iter 22/25 - loss 16.03849519 - samples/sec: 128.51 - lr: 0.300000 2021-03-26 05:21:35,683 epoch 6 - iter 24/25 - loss 16.23500764 - samples/sec: 119.66 - lr: 0.300000 2021-03-26 05:21:36,060 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:21:36,060 EPOCH 6 done: loss 16.4187 - lr 0.3000000 2021-03-26 05:21:36,879 DEV : loss 13.509325981140137 - score 0.7897 2021-03-26 05:21:36,909 BAD EPOCHS (no improvement): 0 2021-03-26 05:21:46,893 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:21:47,945 epoch 7 - iter 2/25 - loss 17.45014191 - samples/sec: 122.04 - lr: 0.300000 2021-03-26 05:21:49,160 epoch 7 - iter 4/25 - loss 16.16570139 - samples/sec: 105.47 - lr: 0.300000 2021-03-26 05:21:50,573 epoch 7 - iter 6/25 - loss 15.32639710 - samples/sec: 90.65 - lr: 0.300000 2021-03-26 05:21:51,939 epoch 7 - iter 8/25 - loss 14.89120066 - samples/sec: 93.85 - lr: 0.300000 2021-03-26 05:21:53,445 epoch 7 - iter 10/25 - loss 14.58858881 - samples/sec: 85.07 - lr: 0.300000 2021-03-26 05:21:54,772 epoch 7 - iter 12/25 - loss 14.68719117 - samples/sec: 96.69 - lr: 0.300000 2021-03-26 05:21:55,866 epoch 7 - iter 14/25 - loss 14.78044850 - samples/sec: 117.17 - lr: 0.300000 2021-03-26 05:21:57,147 epoch 7 - iter 16/25 - loss 14.58556020 - samples/sec: 99.99 - lr: 0.300000 2021-03-26 05:21:58,149 epoch 7 - iter 18/25 - loss 14.70696529 - samples/sec: 127.97 - lr: 0.300000 2021-03-26 05:21:59,192 epoch 7 - iter 20/25 - loss 14.69041018 - samples/sec: 122.89 - lr: 0.300000 2021-03-26 05:22:00,259 epoch 7 - iter 22/25 - loss 14.80504721 - samples/sec: 120.07 - lr: 0.300000 2021-03-26 05:22:01,233 epoch 7 - iter 24/25 - loss 14.69321915 - samples/sec: 131.79 - lr: 0.300000 2021-03-26 05:22:01,714 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:22:01,714 EPOCH 7 done: loss 14.6265 - lr 0.3000000 2021-03-26 05:22:02,536 DEV : loss 10.955870628356934 - score 0.8189 2021-03-26 05:22:02,559 BAD EPOCHS (no improvement): 0 2021-03-26 05:22:12,290 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:22:13,430 epoch 8 - iter 2/25 - loss 12.19628286 - samples/sec: 112.50 - lr: 0.300000 2021-03-26 05:22:14,425 epoch 8 - iter 4/25 - loss 13.13171005 - samples/sec: 128.97 - lr: 0.300000 2021-03-26 05:22:15,458 epoch 8 - iter 6/25 - loss 13.02878348 - samples/sec: 124.17 - lr: 0.300000 2021-03-26 05:22:16,442 epoch 8 - iter 8/25 - loss 12.53637004 - samples/sec: 130.31 - lr: 0.300000 2021-03-26 05:22:17,454 epoch 8 - iter 10/25 - loss 12.59346609 - samples/sec: 126.79 - lr: 0.300000 2021-03-26 05:22:18,601 epoch 8 - iter 12/25 - loss 12.86952901 - samples/sec: 111.75 - lr: 0.300000 2021-03-26 05:22:19,582 epoch 8 - iter 14/25 - loss 13.26604230 - samples/sec: 130.67 - lr: 0.300000 2021-03-26 05:22:20,571 epoch 8 - iter 16/25 - loss 13.08011448 - samples/sec: 129.67 - lr: 0.300000 2021-03-26 05:22:21,576 epoch 8 - iter 18/25 - loss 12.98361148 - samples/sec: 127.56 - lr: 0.300000 2021-03-26 05:22:22,700 epoch 8 - iter 20/25 - loss 12.99734645 - samples/sec: 114.08 - lr: 0.300000 2021-03-26 05:22:23,728 epoch 8 - iter 22/25 - loss 13.00599228 - samples/sec: 124.65 - lr: 0.300000 2021-03-26 05:22:24,735 epoch 8 - iter 24/25 - loss 12.97749150 - samples/sec: 127.30 - lr: 0.300000 2021-03-26 05:22:25,180 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:22:25,181 EPOCH 8 done: loss 12.9387 - lr 0.3000000 2021-03-26 05:22:26,010 DEV : loss 9.875782012939453 - score 0.836 2021-03-26 05:22:26,038 BAD EPOCHS (no improvement): 0 2021-03-26 05:22:35,930 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:22:37,002 epoch 9 - iter 2/25 - loss 13.14775276 - samples/sec: 119.88 - lr: 0.300000 2021-03-26 05:22:38,044 epoch 9 - iter 4/25 - loss 12.90201044 - samples/sec: 123.00 - lr: 0.300000 2021-03-26 05:22:39,092 epoch 9 - iter 6/25 - loss 12.12569332 - samples/sec: 122.33 - lr: 0.300000 2021-03-26 05:22:40,101 epoch 9 - iter 8/25 - loss 11.99610913 - samples/sec: 127.05 - lr: 0.300000 2021-03-26 05:22:41,177 epoch 9 - iter 10/25 - loss 12.22638769 - samples/sec: 119.09 - lr: 0.300000 2021-03-26 05:22:42,153 epoch 9 - iter 12/25 - loss 12.00737731 - samples/sec: 131.48 - lr: 0.300000 2021-03-26 05:22:43,172 epoch 9 - iter 14/25 - loss 12.14194461 - samples/sec: 125.80 - lr: 0.300000 2021-03-26 05:22:44,022 epoch 9 - iter 16/25 - loss 11.95955336 - samples/sec: 150.96 - lr: 0.300000 2021-03-26 05:22:45,047 epoch 9 - iter 18/25 - loss 12.03510804 - samples/sec: 125.01 - lr: 0.300000 2021-03-26 05:22:46,007 epoch 9 - iter 20/25 - loss 12.04640160 - samples/sec: 133.67 - lr: 0.300000 2021-03-26 05:22:46,961 epoch 9 - iter 22/25 - loss 12.06278606 - samples/sec: 134.33 - lr: 0.300000 2021-03-26 05:22:47,959 epoch 9 - iter 24/25 - loss 12.02169251 - samples/sec: 128.46 - lr: 0.300000 2021-03-26 05:22:48,357 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:22:48,358 EPOCH 9 done: loss 11.9918 - lr 0.3000000 2021-03-26 05:22:49,153 DEV : loss 9.951400756835938 - score 0.8373 2021-03-26 05:22:49,173 BAD EPOCHS (no improvement): 0 2021-03-26 05:22:59,122 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:23:00,224 epoch 10 - iter 2/25 - loss 12.44782639 - samples/sec: 116.48 - lr: 0.300000 2021-03-26 05:23:01,238 epoch 10 - iter 4/25 - loss 11.81677580 - samples/sec: 126.41 - lr: 0.300000 2021-03-26 05:23:02,329 epoch 10 - iter 6/25 - loss 11.34556882 - samples/sec: 117.52 - lr: 0.300000 2021-03-26 05:23:03,619 epoch 10 - iter 8/25 - loss 11.62646890 - samples/sec: 99.33 - lr: 0.300000 2021-03-26 05:23:04,555 epoch 10 - iter 10/25 - loss 11.20175314 - samples/sec: 137.26 - lr: 0.300000 2021-03-26 05:23:05,546 epoch 10 - iter 12/25 - loss 11.30701804 - samples/sec: 129.31 - lr: 0.300000 2021-03-26 05:23:06,587 epoch 10 - iter 14/25 - loss 11.24839946 - samples/sec: 123.13 - lr: 0.300000 2021-03-26 05:23:07,580 epoch 10 - iter 16/25 - loss 11.15712929 - samples/sec: 129.04 - lr: 0.300000 2021-03-26 05:23:08,536 epoch 10 - iter 18/25 - loss 10.91551023 - samples/sec: 134.13 - lr: 0.300000 2021-03-26 05:23:09,584 epoch 10 - iter 20/25 - loss 11.12621126 - samples/sec: 122.41 - lr: 0.300000 2021-03-26 05:23:10,656 epoch 10 - iter 22/25 - loss 11.30736299 - samples/sec: 119.70 - lr: 0.300000 2021-03-26 05:23:11,635 epoch 10 - iter 24/25 - loss 11.32537091 - samples/sec: 130.96 - lr: 0.300000 2021-03-26 05:23:12,084 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:23:12,089 EPOCH 10 done: loss 11.1850 - lr 0.3000000 2021-03-26 05:23:12,892 DEV : loss 9.316814422607422 - score 0.85 2021-03-26 05:23:12,918 BAD EPOCHS (no improvement): 0 2021-03-26 05:23:22,610 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:23:23,675 epoch 11 - iter 2/25 - loss 11.53073597 - samples/sec: 120.46 - lr: 0.300000 2021-03-26 05:23:24,974 epoch 11 - iter 4/25 - loss 11.28462696 - samples/sec: 98.57 - lr: 0.300000 2021-03-26 05:23:25,985 epoch 11 - iter 6/25 - loss 10.85450268 - samples/sec: 126.86 - lr: 0.300000 2021-03-26 05:23:26,991 epoch 11 - iter 8/25 - loss 11.06903195 - samples/sec: 127.47 - lr: 0.300000 2021-03-26 05:23:27,961 epoch 11 - iter 10/25 - loss 10.65050640 - samples/sec: 132.10 - lr: 0.300000 2021-03-26 05:23:28,893 epoch 11 - iter 12/25 - loss 10.32414166 - samples/sec: 137.71 - lr: 0.300000 2021-03-26 05:23:29,793 epoch 11 - iter 14/25 - loss 10.17382424 - samples/sec: 142.40 - lr: 0.300000 2021-03-26 05:23:30,854 epoch 11 - iter 16/25 - loss 10.42914373 - samples/sec: 120.84 - lr: 0.300000 2021-03-26 05:23:31,796 epoch 11 - iter 18/25 - loss 10.50087696 - samples/sec: 136.03 - lr: 0.300000 2021-03-26 05:23:32,865 epoch 11 - iter 20/25 - loss 10.54544296 - samples/sec: 119.85 - lr: 0.300000 2021-03-26 05:23:33,806 epoch 11 - iter 22/25 - loss 10.47581499 - samples/sec: 136.31 - lr: 0.300000 2021-03-26 05:23:34,916 epoch 11 - iter 24/25 - loss 10.45924973 - samples/sec: 115.39 - lr: 0.300000 2021-03-26 05:23:35,288 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:23:35,289 EPOCH 11 done: loss 10.5797 - lr 0.3000000 2021-03-26 05:23:36,124 DEV : loss 8.551046371459961 - score 0.8569 2021-03-26 05:23:36,151 BAD EPOCHS (no improvement): 0 2021-03-26 05:23:45,942 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:23:46,915 epoch 12 - iter 2/25 - loss 10.74679136 - samples/sec: 131.89 - lr: 0.300000 2021-03-26 05:23:47,927 epoch 12 - iter 4/25 - loss 10.23639774 - samples/sec: 126.62 - lr: 0.300000 2021-03-26 05:23:48,890 epoch 12 - iter 6/25 - loss 10.10948531 - samples/sec: 133.25 - lr: 0.300000 2021-03-26 05:23:49,902 epoch 12 - iter 8/25 - loss 10.19528675 - samples/sec: 126.61 - lr: 0.300000 2021-03-26 05:23:50,901 epoch 12 - iter 10/25 - loss 9.82142744 - samples/sec: 128.32 - lr: 0.300000 2021-03-26 05:23:51,943 epoch 12 - iter 12/25 - loss 9.58679080 - samples/sec: 123.03 - lr: 0.300000 2021-03-26 05:23:52,946 epoch 12 - iter 14/25 - loss 9.66052955 - samples/sec: 127.91 - lr: 0.300000 2021-03-26 05:23:54,045 epoch 12 - iter 16/25 - loss 9.79544091 - samples/sec: 116.59 - lr: 0.300000 2021-03-26 05:23:55,070 epoch 12 - iter 18/25 - loss 9.79722193 - samples/sec: 125.09 - lr: 0.300000 2021-03-26 05:23:56,102 epoch 12 - iter 20/25 - loss 9.64639330 - samples/sec: 124.27 - lr: 0.300000 2021-03-26 05:23:57,259 epoch 12 - iter 22/25 - loss 9.72506645 - samples/sec: 110.77 - lr: 0.300000 2021-03-26 05:23:58,241 epoch 12 - iter 24/25 - loss 9.83624764 - samples/sec: 130.41 - lr: 0.300000 2021-03-26 05:23:58,657 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:23:58,657 EPOCH 12 done: loss 9.9765 - lr 0.3000000 2021-03-26 05:23:59,449 DEV : loss 8.387750625610352 - score 0.8586 2021-03-26 05:23:59,467 BAD EPOCHS (no improvement): 0 2021-03-26 05:24:09,166 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:24:10,298 epoch 13 - iter 2/25 - loss 8.35482121 - samples/sec: 113.22 - lr: 0.300000 2021-03-26 05:24:11,330 epoch 13 - iter 4/25 - loss 8.62327766 - samples/sec: 124.28 - lr: 0.300000 2021-03-26 05:24:12,379 epoch 13 - iter 6/25 - loss 8.79010773 - samples/sec: 122.13 - lr: 0.300000 2021-03-26 05:24:13,375 epoch 13 - iter 8/25 - loss 9.15598595 - samples/sec: 128.79 - lr: 0.300000 2021-03-26 05:24:14,375 epoch 13 - iter 10/25 - loss 9.02062783 - samples/sec: 128.09 - lr: 0.300000 2021-03-26 05:24:15,440 epoch 13 - iter 12/25 - loss 9.00885872 - samples/sec: 120.33 - lr: 0.300000 2021-03-26 05:24:16,420 epoch 13 - iter 14/25 - loss 9.12177137 - samples/sec: 130.81 - lr: 0.300000 2021-03-26 05:24:17,316 epoch 13 - iter 16/25 - loss 9.05517802 - samples/sec: 143.08 - lr: 0.300000 2021-03-26 05:24:18,340 epoch 13 - iter 18/25 - loss 9.30420470 - samples/sec: 125.33 - lr: 0.300000 2021-03-26 05:24:19,351 epoch 13 - iter 20/25 - loss 9.40496576 - samples/sec: 126.84 - lr: 0.300000 2021-03-26 05:24:20,275 epoch 13 - iter 22/25 - loss 9.29032285 - samples/sec: 138.84 - lr: 0.300000 2021-03-26 05:24:21,216 epoch 13 - iter 24/25 - loss 9.41522477 - samples/sec: 136.34 - lr: 0.300000 2021-03-26 05:24:21,581 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:24:21,583 EPOCH 13 done: loss 9.3761 - lr 0.3000000 2021-03-26 05:24:22,387 DEV : loss 8.103069305419922 - score 0.8616 2021-03-26 05:24:22,417 BAD EPOCHS (no improvement): 0 2021-03-26 05:24:32,178 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:24:33,201 epoch 14 - iter 2/25 - loss 8.71276283 - samples/sec: 125.42 - lr: 0.300000 2021-03-26 05:24:34,223 epoch 14 - iter 4/25 - loss 8.67824745 - samples/sec: 125.44 - lr: 0.300000 2021-03-26 05:24:35,347 epoch 14 - iter 6/25 - loss 9.10136700 - samples/sec: 114.10 - lr: 0.300000 2021-03-26 05:24:36,303 epoch 14 - iter 8/25 - loss 9.12157381 - samples/sec: 134.01 - lr: 0.300000 2021-03-26 05:24:37,296 epoch 14 - iter 10/25 - loss 8.77482038 - samples/sec: 129.18 - lr: 0.300000 2021-03-26 05:24:38,248 epoch 14 - iter 12/25 - loss 8.90701234 - samples/sec: 134.67 - lr: 0.300000 2021-03-26 05:24:39,248 epoch 14 - iter 14/25 - loss 8.99114633 - samples/sec: 128.19 - lr: 0.300000 2021-03-26 05:24:40,225 epoch 14 - iter 16/25 - loss 9.13066396 - samples/sec: 131.33 - lr: 0.300000 2021-03-26 05:24:41,134 epoch 14 - iter 18/25 - loss 9.10450154 - samples/sec: 140.92 - lr: 0.300000 2021-03-26 05:24:42,083 epoch 14 - iter 20/25 - loss 9.14051502 - samples/sec: 135.24 - lr: 0.300000 2021-03-26 05:24:43,230 epoch 14 - iter 22/25 - loss 9.12343365 - samples/sec: 111.78 - lr: 0.300000 2021-03-26 05:24:44,225 epoch 14 - iter 24/25 - loss 9.09766744 - samples/sec: 128.73 - lr: 0.300000 2021-03-26 05:24:44,585 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:24:44,586 EPOCH 14 done: loss 9.0100 - lr 0.3000000 2021-03-26 05:24:45,383 DEV : loss 7.8762006759643555 - score 0.8742 2021-03-26 05:24:45,408 BAD EPOCHS (no improvement): 0 2021-03-26 05:24:55,253 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:24:56,534 epoch 15 - iter 2/25 - loss 7.36967874 - samples/sec: 100.07 - lr: 0.300000 2021-03-26 05:24:57,584 epoch 15 - iter 4/25 - loss 8.21823275 - samples/sec: 122.07 - lr: 0.300000 2021-03-26 05:24:58,702 epoch 15 - iter 6/25 - loss 8.26005371 - samples/sec: 114.61 - lr: 0.300000 2021-03-26 05:24:59,681 epoch 15 - iter 8/25 - loss 8.40027934 - samples/sec: 131.16 - lr: 0.300000 2021-03-26 05:25:00,589 epoch 15 - iter 10/25 - loss 8.38377128 - samples/sec: 141.26 - lr: 0.300000 2021-03-26 05:25:01,554 epoch 15 - iter 12/25 - loss 8.43352679 - samples/sec: 132.81 - lr: 0.300000 2021-03-26 05:25:02,543 epoch 15 - iter 14/25 - loss 8.60027528 - samples/sec: 129.54 - lr: 0.300000 2021-03-26 05:25:03,456 epoch 15 - iter 16/25 - loss 8.50458527 - samples/sec: 140.39 - lr: 0.300000 2021-03-26 05:25:04,441 epoch 15 - iter 18/25 - loss 8.59310924 - samples/sec: 130.13 - lr: 0.300000 2021-03-26 05:25:05,374 epoch 15 - iter 20/25 - loss 8.45520668 - samples/sec: 137.44 - lr: 0.300000 2021-03-26 05:25:06,528 epoch 15 - iter 22/25 - loss 8.46085405 - samples/sec: 111.06 - lr: 0.300000 2021-03-26 05:25:07,471 epoch 15 - iter 24/25 - loss 8.52216919 - samples/sec: 135.86 - lr: 0.300000 2021-03-26 05:25:07,959 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:25:07,960 EPOCH 15 done: loss 8.5789 - lr 0.3000000 2021-03-26 05:25:08,749 DEV : loss 7.669604301452637 - score 0.8778 2021-03-26 05:25:08,767 BAD EPOCHS (no improvement): 0 2021-03-26 05:25:18,338 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:25:19,395 epoch 16 - iter 2/25 - loss 8.13550901 - samples/sec: 121.28 - lr: 0.300000 2021-03-26 05:25:20,334 epoch 16 - iter 4/25 - loss 7.03732085 - samples/sec: 136.58 - lr: 0.300000 2021-03-26 05:25:21,549 epoch 16 - iter 6/25 - loss 7.44733779 - samples/sec: 105.45 - lr: 0.300000 2021-03-26 05:25:22,598 epoch 16 - iter 8/25 - loss 7.37813056 - samples/sec: 122.42 - lr: 0.300000 2021-03-26 05:25:23,678 epoch 16 - iter 10/25 - loss 7.22389278 - samples/sec: 118.64 - lr: 0.300000 2021-03-26 05:25:24,947 epoch 16 - iter 12/25 - loss 7.47639052 - samples/sec: 100.94 - lr: 0.300000 2021-03-26 05:25:26,035 epoch 16 - iter 14/25 - loss 7.56412601 - samples/sec: 117.79 - lr: 0.300000 2021-03-26 05:25:27,062 epoch 16 - iter 16/25 - loss 7.49645185 - samples/sec: 125.03 - lr: 0.300000 2021-03-26 05:25:27,988 epoch 16 - iter 18/25 - loss 7.50306392 - samples/sec: 138.30 - lr: 0.300000 2021-03-26 05:25:28,967 epoch 16 - iter 20/25 - loss 7.53772652 - samples/sec: 131.12 - lr: 0.300000 2021-03-26 05:25:29,986 epoch 16 - iter 22/25 - loss 7.72518433 - samples/sec: 125.90 - lr: 0.300000 2021-03-26 05:25:31,108 epoch 16 - iter 24/25 - loss 7.87033107 - samples/sec: 114.28 - lr: 0.300000 2021-03-26 05:25:31,507 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:25:31,509 EPOCH 16 done: loss 7.9388 - lr 0.3000000 2021-03-26 05:25:32,308 DEV : loss 7.1785101890563965 - score 0.8821 2021-03-26 05:25:32,334 BAD EPOCHS (no improvement): 0 2021-03-26 05:25:41,929 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:25:42,993 epoch 17 - iter 2/25 - loss 7.94741535 - samples/sec: 120.59 - lr: 0.300000 2021-03-26 05:25:44,103 epoch 17 - iter 4/25 - loss 7.50709152 - samples/sec: 115.58 - lr: 0.300000 2021-03-26 05:25:45,093 epoch 17 - iter 6/25 - loss 7.20844897 - samples/sec: 129.49 - lr: 0.300000 2021-03-26 05:25:45,997 epoch 17 - iter 8/25 - loss 7.34367311 - samples/sec: 141.68 - lr: 0.300000 2021-03-26 05:25:46,963 epoch 17 - iter 10/25 - loss 7.38157673 - samples/sec: 132.71 - lr: 0.300000 2021-03-26 05:25:48,126 epoch 17 - iter 12/25 - loss 7.26675844 - samples/sec: 110.15 - lr: 0.300000 2021-03-26 05:25:49,145 epoch 17 - iter 14/25 - loss 7.44016184 - samples/sec: 125.87 - lr: 0.300000 2021-03-26 05:25:50,131 epoch 17 - iter 16/25 - loss 7.60327804 - samples/sec: 130.01 - lr: 0.300000 2021-03-26 05:25:51,126 epoch 17 - iter 18/25 - loss 7.59039590 - samples/sec: 128.86 - lr: 0.300000 2021-03-26 05:25:52,082 epoch 17 - iter 20/25 - loss 7.60469155 - samples/sec: 134.06 - lr: 0.300000 2021-03-26 05:25:53,058 epoch 17 - iter 22/25 - loss 7.59717865 - samples/sec: 131.42 - lr: 0.300000 2021-03-26 05:25:54,143 epoch 17 - iter 24/25 - loss 7.81563658 - samples/sec: 118.16 - lr: 0.300000 2021-03-26 05:25:54,547 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:25:54,548 EPOCH 17 done: loss 7.8554 - lr 0.3000000 2021-03-26 05:25:55,364 DEV : loss 7.210777282714844 - score 0.8833 2021-03-26 05:25:55,401 BAD EPOCHS (no improvement): 0 2021-03-26 05:26:05,034 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:26:06,147 epoch 18 - iter 2/25 - loss 6.63195515 - samples/sec: 115.19 - lr: 0.300000 2021-03-26 05:26:07,231 epoch 18 - iter 4/25 - loss 7.14458799 - samples/sec: 118.29 - lr: 0.300000 2021-03-26 05:26:08,413 epoch 18 - iter 6/25 - loss 7.36422459 - samples/sec: 108.38 - lr: 0.300000 2021-03-26 05:26:09,472 epoch 18 - iter 8/25 - loss 7.26226878 - samples/sec: 121.10 - lr: 0.300000 2021-03-26 05:26:10,477 epoch 18 - iter 10/25 - loss 7.17983127 - samples/sec: 127.57 - lr: 0.300000 2021-03-26 05:26:11,527 epoch 18 - iter 12/25 - loss 7.28797615 - samples/sec: 121.98 - lr: 0.300000 2021-03-26 05:26:12,576 epoch 18 - iter 14/25 - loss 7.35998808 - samples/sec: 122.43 - lr: 0.300000 2021-03-26 05:26:13,580 epoch 18 - iter 16/25 - loss 7.37090909 - samples/sec: 127.61 - lr: 0.300000 2021-03-26 05:26:14,574 epoch 18 - iter 18/25 - loss 7.48112631 - samples/sec: 128.90 - lr: 0.300000 2021-03-26 05:26:15,576 epoch 18 - iter 20/25 - loss 7.40971677 - samples/sec: 127.94 - lr: 0.300000 2021-03-26 05:26:16,544 epoch 18 - iter 22/25 - loss 7.27288305 - samples/sec: 132.49 - lr: 0.300000 2021-03-26 05:26:17,487 epoch 18 - iter 24/25 - loss 7.28376927 - samples/sec: 135.93 - lr: 0.300000 2021-03-26 05:26:17,878 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:26:17,878 EPOCH 18 done: loss 7.3653 - lr 0.3000000 2021-03-26 05:26:18,660 DEV : loss 7.1419501304626465 - score 0.8825 2021-03-26 05:26:18,681 BAD EPOCHS (no improvement): 1 2021-03-26 05:26:18,682 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:26:19,665 epoch 19 - iter 2/25 - loss 6.25295830 - samples/sec: 130.48 - lr: 0.300000 2021-03-26 05:26:20,672 epoch 19 - iter 4/25 - loss 6.57761550 - samples/sec: 127.28 - lr: 0.300000 2021-03-26 05:26:21,720 epoch 19 - iter 6/25 - loss 6.53581572 - samples/sec: 122.31 - lr: 0.300000 2021-03-26 05:26:22,894 epoch 19 - iter 8/25 - loss 6.69568771 - samples/sec: 109.14 - lr: 0.300000 2021-03-26 05:26:24,062 epoch 19 - iter 10/25 - loss 6.88163967 - samples/sec: 109.81 - lr: 0.300000 2021-03-26 05:26:25,069 epoch 19 - iter 12/25 - loss 7.00405916 - samples/sec: 127.20 - lr: 0.300000 2021-03-26 05:26:26,000 epoch 19 - iter 14/25 - loss 7.03675880 - samples/sec: 137.68 - lr: 0.300000 2021-03-26 05:26:26,956 epoch 19 - iter 16/25 - loss 7.00491729 - samples/sec: 134.10 - lr: 0.300000 2021-03-26 05:26:27,982 epoch 19 - iter 18/25 - loss 6.89046571 - samples/sec: 124.92 - lr: 0.300000 2021-03-26 05:26:28,945 epoch 19 - iter 20/25 - loss 6.84953601 - samples/sec: 133.14 - lr: 0.300000 2021-03-26 05:26:29,932 epoch 19 - iter 22/25 - loss 6.87707803 - samples/sec: 130.02 - lr: 0.300000 2021-03-26 05:26:30,914 epoch 19 - iter 24/25 - loss 6.94888598 - samples/sec: 130.60 - lr: 0.300000 2021-03-26 05:26:31,419 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:26:31,420 EPOCH 19 done: loss 6.9400 - lr 0.3000000 2021-03-26 05:26:32,208 DEV : loss 7.064170837402344 - score 0.8805 2021-03-26 05:26:32,234 BAD EPOCHS (no improvement): 2 2021-03-26 05:26:32,235 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:26:33,210 epoch 20 - iter 2/25 - loss 6.28246975 - samples/sec: 131.61 - lr: 0.300000 2021-03-26 05:26:34,385 epoch 20 - iter 4/25 - loss 6.50505972 - samples/sec: 109.18 - lr: 0.300000 2021-03-26 05:26:35,386 epoch 20 - iter 6/25 - loss 6.39607708 - samples/sec: 128.61 - lr: 0.300000 2021-03-26 05:26:36,443 epoch 20 - iter 8/25 - loss 6.44121069 - samples/sec: 121.30 - lr: 0.300000 2021-03-26 05:26:37,390 epoch 20 - iter 10/25 - loss 6.51437483 - samples/sec: 135.54 - lr: 0.300000 2021-03-26 05:26:38,382 epoch 20 - iter 12/25 - loss 6.48822912 - samples/sec: 129.24 - lr: 0.300000 2021-03-26 05:26:39,321 epoch 20 - iter 14/25 - loss 6.52995566 - samples/sec: 136.39 - lr: 0.300000 2021-03-26 05:26:40,374 epoch 20 - iter 16/25 - loss 6.48401153 - samples/sec: 121.82 - lr: 0.300000 2021-03-26 05:26:41,452 epoch 20 - iter 18/25 - loss 6.68182256 - samples/sec: 118.89 - lr: 0.300000 2021-03-26 05:26:42,403 epoch 20 - iter 20/25 - loss 6.63910098 - samples/sec: 134.85 - lr: 0.300000 2021-03-26 05:26:43,378 epoch 20 - iter 22/25 - loss 6.61011570 - samples/sec: 131.52 - lr: 0.300000 2021-03-26 05:26:44,399 epoch 20 - iter 24/25 - loss 6.74762255 - samples/sec: 125.65 - lr: 0.300000 2021-03-26 05:26:44,788 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:26:44,788 EPOCH 20 done: loss 6.7392 - lr 0.3000000 2021-03-26 05:26:45,587 DEV : loss 7.2709784507751465 - score 0.878 2021-03-26 05:26:45,613 BAD EPOCHS (no improvement): 3 2021-03-26 05:26:45,613 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:26:46,601 epoch 21 - iter 2/25 - loss 6.23770928 - samples/sec: 129.85 - lr: 0.300000 2021-03-26 05:26:47,602 epoch 21 - iter 4/25 - loss 6.37881196 - samples/sec: 128.15 - lr: 0.300000 2021-03-26 05:26:48,649 epoch 21 - iter 6/25 - loss 6.54572686 - samples/sec: 122.42 - lr: 0.300000 2021-03-26 05:26:49,646 epoch 21 - iter 8/25 - loss 6.36524707 - samples/sec: 128.57 - lr: 0.300000 2021-03-26 05:26:50,621 epoch 21 - iter 10/25 - loss 6.47547174 - samples/sec: 131.58 - lr: 0.300000 2021-03-26 05:26:51,622 epoch 21 - iter 12/25 - loss 6.50450758 - samples/sec: 128.00 - lr: 0.300000 2021-03-26 05:26:52,728 epoch 21 - iter 14/25 - loss 6.59004596 - samples/sec: 115.91 - lr: 0.300000 2021-03-26 05:26:53,747 epoch 21 - iter 16/25 - loss 6.56259722 - samples/sec: 125.80 - lr: 0.300000 2021-03-26 05:26:54,718 epoch 21 - iter 18/25 - loss 6.59617737 - samples/sec: 131.99 - lr: 0.300000 2021-03-26 05:26:55,717 epoch 21 - iter 20/25 - loss 6.62009044 - samples/sec: 128.23 - lr: 0.300000 2021-03-26 05:26:56,831 epoch 21 - iter 22/25 - loss 6.60993984 - samples/sec: 115.11 - lr: 0.300000 2021-03-26 05:26:57,793 epoch 21 - iter 24/25 - loss 6.62797530 - samples/sec: 133.20 - lr: 0.300000 2021-03-26 05:26:58,192 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:26:58,193 EPOCH 21 done: loss 6.5851 - lr 0.3000000 2021-03-26 05:26:59,025 DEV : loss 6.789755821228027 - score 0.8945 2021-03-26 05:26:59,051 BAD EPOCHS (no improvement): 0 2021-03-26 05:27:08,721 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:27:09,779 epoch 22 - iter 2/25 - loss 5.53058386 - samples/sec: 121.32 - lr: 0.300000 2021-03-26 05:27:10,809 epoch 22 - iter 4/25 - loss 5.39357162 - samples/sec: 124.36 - lr: 0.300000 2021-03-26 05:27:11,833 epoch 22 - iter 6/25 - loss 5.63118982 - samples/sec: 125.28 - lr: 0.300000 2021-03-26 05:27:12,780 epoch 22 - iter 8/25 - loss 6.02222788 - samples/sec: 135.52 - lr: 0.300000 2021-03-26 05:27:13,839 epoch 22 - iter 10/25 - loss 6.07552347 - samples/sec: 121.04 - lr: 0.300000 2021-03-26 05:27:14,765 epoch 22 - iter 12/25 - loss 6.22429490 - samples/sec: 138.41 - lr: 0.300000 2021-03-26 05:27:15,759 epoch 22 - iter 14/25 - loss 6.22745047 - samples/sec: 129.08 - lr: 0.300000 2021-03-26 05:27:16,689 epoch 22 - iter 16/25 - loss 6.30747709 - samples/sec: 137.81 - lr: 0.300000 2021-03-26 05:27:17,708 epoch 22 - iter 18/25 - loss 6.26303742 - samples/sec: 125.81 - lr: 0.300000 2021-03-26 05:27:18,708 epoch 22 - iter 20/25 - loss 6.22910388 - samples/sec: 128.20 - lr: 0.300000 2021-03-26 05:27:19,780 epoch 22 - iter 22/25 - loss 6.24298755 - samples/sec: 119.67 - lr: 0.300000 2021-03-26 05:27:20,741 epoch 22 - iter 24/25 - loss 6.24154566 - samples/sec: 133.33 - lr: 0.300000 2021-03-26 05:27:21,120 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:27:21,121 EPOCH 22 done: loss 6.2946 - lr 0.3000000 2021-03-26 05:27:21,895 DEV : loss 6.656912803649902 - score 0.8945 2021-03-26 05:27:21,921 BAD EPOCHS (no improvement): 0 2021-03-26 05:27:31,685 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:27:32,736 epoch 23 - iter 2/25 - loss 4.46555138 - samples/sec: 122.08 - lr: 0.300000 2021-03-26 05:27:33,969 epoch 23 - iter 4/25 - loss 4.78493762 - samples/sec: 103.93 - lr: 0.300000 2021-03-26 05:27:35,147 epoch 23 - iter 6/25 - loss 5.48626884 - samples/sec: 108.79 - lr: 0.300000 2021-03-26 05:27:36,228 epoch 23 - iter 8/25 - loss 5.68426472 - samples/sec: 118.54 - lr: 0.300000 2021-03-26 05:27:37,297 epoch 23 - iter 10/25 - loss 5.91127291 - samples/sec: 119.97 - lr: 0.300000 2021-03-26 05:27:38,310 epoch 23 - iter 12/25 - loss 6.00124419 - samples/sec: 126.64 - lr: 0.300000 2021-03-26 05:27:39,237 epoch 23 - iter 14/25 - loss 5.97189999 - samples/sec: 138.59 - lr: 0.300000 2021-03-26 05:27:40,249 epoch 23 - iter 16/25 - loss 6.07553238 - samples/sec: 126.71 - lr: 0.300000 2021-03-26 05:27:41,284 epoch 23 - iter 18/25 - loss 6.01765153 - samples/sec: 123.81 - lr: 0.300000 2021-03-26 05:27:42,351 epoch 23 - iter 20/25 - loss 6.02717052 - samples/sec: 120.24 - lr: 0.300000 2021-03-26 05:27:43,362 epoch 23 - iter 22/25 - loss 5.97675254 - samples/sec: 126.76 - lr: 0.300000 2021-03-26 05:27:44,345 epoch 23 - iter 24/25 - loss 6.03377473 - samples/sec: 130.46 - lr: 0.300000 2021-03-26 05:27:44,802 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:27:44,804 EPOCH 23 done: loss 6.0530 - lr 0.3000000 2021-03-26 05:27:45,614 DEV : loss 6.680357933044434 - score 0.8937 2021-03-26 05:27:45,644 BAD EPOCHS (no improvement): 1 2021-03-26 05:27:45,645 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:27:47,974 epoch 24 - iter 2/25 - loss 6.29046893 - samples/sec: 55.00 - lr: 0.300000 2021-03-26 05:27:48,979 epoch 24 - iter 4/25 - loss 5.58529961 - samples/sec: 127.72 - lr: 0.300000 2021-03-26 05:27:49,976 epoch 24 - iter 6/25 - loss 5.39165346 - samples/sec: 128.73 - lr: 0.300000 2021-03-26 05:27:50,908 epoch 24 - iter 8/25 - loss 5.54855865 - samples/sec: 137.57 - lr: 0.300000 2021-03-26 05:27:51,986 epoch 24 - iter 10/25 - loss 5.66706057 - samples/sec: 118.97 - lr: 0.300000 2021-03-26 05:27:52,966 epoch 24 - iter 12/25 - loss 5.52589273 - samples/sec: 130.86 - lr: 0.300000 2021-03-26 05:27:53,939 epoch 24 - iter 14/25 - loss 5.49458109 - samples/sec: 131.71 - lr: 0.300000 2021-03-26 05:27:54,993 epoch 24 - iter 16/25 - loss 5.55637434 - samples/sec: 121.63 - lr: 0.300000 2021-03-26 05:27:56,061 epoch 24 - iter 18/25 - loss 5.59738755 - samples/sec: 120.01 - lr: 0.300000 2021-03-26 05:27:57,045 epoch 24 - iter 20/25 - loss 5.65361750 - samples/sec: 130.29 - lr: 0.300000 2021-03-26 05:27:58,039 epoch 24 - iter 22/25 - loss 5.59108576 - samples/sec: 128.97 - lr: 0.300000 2021-03-26 05:27:59,097 epoch 24 - iter 24/25 - loss 5.64387566 - samples/sec: 121.36 - lr: 0.300000 2021-03-26 05:27:59,523 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:27:59,525 EPOCH 24 done: loss 5.6586 - lr 0.3000000 2021-03-26 05:28:00,334 DEV : loss 6.7325544357299805 - score 0.8902 2021-03-26 05:28:00,359 BAD EPOCHS (no improvement): 2 2021-03-26 05:28:00,360 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:28:01,283 epoch 25 - iter 2/25 - loss 5.46722913 - samples/sec: 138.87 - lr: 0.300000 2021-03-26 05:28:02,392 epoch 25 - iter 4/25 - loss 5.36274683 - samples/sec: 115.62 - lr: 0.300000 2021-03-26 05:28:03,433 epoch 25 - iter 6/25 - loss 5.46451076 - samples/sec: 123.11 - lr: 0.300000 2021-03-26 05:28:04,419 epoch 25 - iter 8/25 - loss 5.23664898 - samples/sec: 129.97 - lr: 0.300000 2021-03-26 05:28:05,482 epoch 25 - iter 10/25 - loss 5.23238530 - samples/sec: 120.68 - lr: 0.300000 2021-03-26 05:28:06,497 epoch 25 - iter 12/25 - loss 5.37240080 - samples/sec: 126.34 - lr: 0.300000 2021-03-26 05:28:07,519 epoch 25 - iter 14/25 - loss 5.52948417 - samples/sec: 125.40 - lr: 0.300000 2021-03-26 05:28:08,558 epoch 25 - iter 16/25 - loss 5.39657971 - samples/sec: 123.42 - lr: 0.300000 2021-03-26 05:28:09,553 epoch 25 - iter 18/25 - loss 5.50258454 - samples/sec: 128.79 - lr: 0.300000 2021-03-26 05:28:10,536 epoch 25 - iter 20/25 - loss 5.53387783 - samples/sec: 130.55 - lr: 0.300000 2021-03-26 05:28:11,567 epoch 25 - iter 22/25 - loss 5.66683754 - samples/sec: 124.40 - lr: 0.300000 2021-03-26 05:28:12,579 epoch 25 - iter 24/25 - loss 5.67239670 - samples/sec: 126.73 - lr: 0.300000 2021-03-26 05:28:13,003 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:28:13,004 EPOCH 25 done: loss 5.6747 - lr 0.3000000 2021-03-26 05:28:13,806 DEV : loss 6.763164043426514 - score 0.8958 2021-03-26 05:28:13,832 BAD EPOCHS (no improvement): 0 2021-03-26 05:28:23,462 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:28:24,555 epoch 26 - iter 2/25 - loss 6.63353133 - samples/sec: 117.27 - lr: 0.300000 2021-03-26 05:28:25,475 epoch 26 - iter 4/25 - loss 5.68939912 - samples/sec: 139.37 - lr: 0.300000 2021-03-26 05:28:26,410 epoch 26 - iter 6/25 - loss 5.65168484 - samples/sec: 137.19 - lr: 0.300000 2021-03-26 05:28:27,395 epoch 26 - iter 8/25 - loss 5.62042195 - samples/sec: 130.19 - lr: 0.300000 2021-03-26 05:28:28,351 epoch 26 - iter 10/25 - loss 5.50891285 - samples/sec: 134.16 - lr: 0.300000 2021-03-26 05:28:29,378 epoch 26 - iter 12/25 - loss 5.33578368 - samples/sec: 124.93 - lr: 0.300000 2021-03-26 05:28:30,697 epoch 26 - iter 14/25 - loss 5.28669374 - samples/sec: 97.14 - lr: 0.300000 2021-03-26 05:28:31,976 epoch 26 - iter 16/25 - loss 5.39646566 - samples/sec: 100.15 - lr: 0.300000 2021-03-26 05:28:33,171 epoch 26 - iter 18/25 - loss 5.44764969 - samples/sec: 107.27 - lr: 0.300000 2021-03-26 05:28:34,436 epoch 26 - iter 20/25 - loss 5.51364100 - samples/sec: 101.31 - lr: 0.300000 2021-03-26 05:28:35,782 epoch 26 - iter 22/25 - loss 5.52997396 - samples/sec: 95.16 - lr: 0.300000 2021-03-26 05:28:36,856 epoch 26 - iter 24/25 - loss 5.52822081 - samples/sec: 119.42 - lr: 0.300000 2021-03-26 05:28:37,296 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:28:37,297 EPOCH 26 done: loss 5.5572 - lr 0.3000000 2021-03-26 05:28:38,110 DEV : loss 6.5847883224487305 - score 0.9026 2021-03-26 05:28:38,136 BAD EPOCHS (no improvement): 0 2021-03-26 05:28:47,954 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:28:48,908 epoch 27 - iter 2/25 - loss 4.54578447 - samples/sec: 134.48 - lr: 0.300000 2021-03-26 05:28:49,852 epoch 27 - iter 4/25 - loss 5.14534843 - samples/sec: 135.88 - lr: 0.300000 2021-03-26 05:28:50,817 epoch 27 - iter 6/25 - loss 5.24506561 - samples/sec: 132.91 - lr: 0.300000 2021-03-26 05:28:51,775 epoch 27 - iter 8/25 - loss 5.30547184 - samples/sec: 133.91 - lr: 0.300000 2021-03-26 05:28:52,740 epoch 27 - iter 10/25 - loss 5.18562655 - samples/sec: 132.79 - lr: 0.300000 2021-03-26 05:28:53,790 epoch 27 - iter 12/25 - loss 5.12651292 - samples/sec: 122.02 - lr: 0.300000 2021-03-26 05:28:54,822 epoch 27 - iter 14/25 - loss 5.12403495 - samples/sec: 124.34 - lr: 0.300000 2021-03-26 05:28:55,773 epoch 27 - iter 16/25 - loss 5.09699774 - samples/sec: 134.84 - lr: 0.300000 2021-03-26 05:28:56,885 epoch 27 - iter 18/25 - loss 5.10937105 - samples/sec: 115.29 - lr: 0.300000 2021-03-26 05:28:57,943 epoch 27 - iter 20/25 - loss 5.11198812 - samples/sec: 121.82 - lr: 0.300000 2021-03-26 05:28:58,995 epoch 27 - iter 22/25 - loss 5.11410750 - samples/sec: 121.95 - lr: 0.300000 2021-03-26 05:29:00,035 epoch 27 - iter 24/25 - loss 5.11160312 - samples/sec: 123.37 - lr: 0.300000 2021-03-26 05:29:00,435 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:29:00,436 EPOCH 27 done: loss 5.1355 - lr 0.3000000 2021-03-26 05:29:01,239 DEV : loss 6.560229301452637 - score 0.8943 2021-03-26 05:29:01,264 BAD EPOCHS (no improvement): 1 2021-03-26 05:29:01,265 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:29:02,186 epoch 28 - iter 2/25 - loss 4.66952753 - samples/sec: 139.12 - lr: 0.300000 2021-03-26 05:29:03,296 epoch 28 - iter 4/25 - loss 4.94452524 - samples/sec: 115.61 - lr: 0.300000 2021-03-26 05:29:04,307 epoch 28 - iter 6/25 - loss 5.28915143 - samples/sec: 126.82 - lr: 0.300000 2021-03-26 05:29:05,387 epoch 28 - iter 8/25 - loss 5.23088485 - samples/sec: 118.70 - lr: 0.300000 2021-03-26 05:29:06,361 epoch 28 - iter 10/25 - loss 5.18750887 - samples/sec: 132.29 - lr: 0.300000 2021-03-26 05:29:07,301 epoch 28 - iter 12/25 - loss 5.27375034 - samples/sec: 136.41 - lr: 0.300000 2021-03-26 05:29:08,326 epoch 28 - iter 14/25 - loss 5.17865321 - samples/sec: 125.17 - lr: 0.300000 2021-03-26 05:29:09,257 epoch 28 - iter 16/25 - loss 5.02847534 - samples/sec: 137.74 - lr: 0.300000 2021-03-26 05:29:10,228 epoch 28 - iter 18/25 - loss 4.98988385 - samples/sec: 132.00 - lr: 0.300000 2021-03-26 05:29:11,255 epoch 28 - iter 20/25 - loss 5.05867081 - samples/sec: 124.96 - lr: 0.300000 2021-03-26 05:29:12,241 epoch 28 - iter 22/25 - loss 5.07502324 - samples/sec: 129.99 - lr: 0.300000 2021-03-26 05:29:13,203 epoch 28 - iter 24/25 - loss 5.07763763 - samples/sec: 133.29 - lr: 0.300000 2021-03-26 05:29:13,619 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:29:13,620 EPOCH 28 done: loss 5.0847 - lr 0.3000000 2021-03-26 05:29:14,425 DEV : loss 6.717233180999756 - score 0.8993 2021-03-26 05:29:14,451 BAD EPOCHS (no improvement): 2 2021-03-26 05:29:14,452 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:29:15,416 epoch 29 - iter 2/25 - loss 4.26012993 - samples/sec: 132.99 - lr: 0.300000 2021-03-26 05:29:16,349 epoch 29 - iter 4/25 - loss 4.56135571 - samples/sec: 137.37 - lr: 0.300000 2021-03-26 05:29:17,389 epoch 29 - iter 6/25 - loss 4.93187809 - samples/sec: 123.23 - lr: 0.300000 2021-03-26 05:29:18,366 epoch 29 - iter 8/25 - loss 4.83429003 - samples/sec: 131.28 - lr: 0.300000 2021-03-26 05:29:19,296 epoch 29 - iter 10/25 - loss 4.67303157 - samples/sec: 137.83 - lr: 0.300000 2021-03-26 05:29:20,392 epoch 29 - iter 12/25 - loss 4.82254970 - samples/sec: 116.98 - lr: 0.300000 2021-03-26 05:29:21,387 epoch 29 - iter 14/25 - loss 4.81970096 - samples/sec: 128.74 - lr: 0.300000 2021-03-26 05:29:22,338 epoch 29 - iter 16/25 - loss 4.91178760 - samples/sec: 134.92 - lr: 0.300000 2021-03-26 05:29:23,320 epoch 29 - iter 18/25 - loss 4.96595081 - samples/sec: 130.51 - lr: 0.300000 2021-03-26 05:29:24,274 epoch 29 - iter 20/25 - loss 4.99328244 - samples/sec: 134.36 - lr: 0.300000 2021-03-26 05:29:25,226 epoch 29 - iter 22/25 - loss 4.95126521 - samples/sec: 134.75 - lr: 0.300000 2021-03-26 05:29:26,292 epoch 29 - iter 24/25 - loss 5.03944759 - samples/sec: 120.22 - lr: 0.300000 2021-03-26 05:29:26,696 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:29:26,697 EPOCH 29 done: loss 5.0159 - lr 0.3000000 2021-03-26 05:29:27,485 DEV : loss 6.611323356628418 - score 0.8941 2021-03-26 05:29:27,510 BAD EPOCHS (no improvement): 3 2021-03-26 05:29:27,511 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:29:28,445 epoch 30 - iter 2/25 - loss 4.52283883 - samples/sec: 137.36 - lr: 0.300000 2021-03-26 05:29:29,467 epoch 30 - iter 4/25 - loss 4.83641267 - samples/sec: 125.41 - lr: 0.300000 2021-03-26 05:29:30,417 epoch 30 - iter 6/25 - loss 4.50342270 - samples/sec: 135.10 - lr: 0.300000 2021-03-26 05:29:31,465 epoch 30 - iter 8/25 - loss 4.70275000 - samples/sec: 122.34 - lr: 0.300000 2021-03-26 05:29:32,435 epoch 30 - iter 10/25 - loss 4.65241845 - samples/sec: 132.29 - lr: 0.300000 2021-03-26 05:29:33,378 epoch 30 - iter 12/25 - loss 4.55542413 - samples/sec: 135.97 - lr: 0.300000 2021-03-26 05:29:34,269 epoch 30 - iter 14/25 - loss 4.53860392 - samples/sec: 143.94 - lr: 0.300000 2021-03-26 05:29:35,282 epoch 30 - iter 16/25 - loss 4.59294629 - samples/sec: 126.47 - lr: 0.300000 2021-03-26 05:29:36,315 epoch 30 - iter 18/25 - loss 4.61952384 - samples/sec: 124.11 - lr: 0.300000 2021-03-26 05:29:37,454 epoch 30 - iter 20/25 - loss 4.71150489 - samples/sec: 112.55 - lr: 0.300000 2021-03-26 05:29:38,557 epoch 30 - iter 22/25 - loss 4.75680065 - samples/sec: 116.20 - lr: 0.300000 2021-03-26 05:29:39,645 epoch 30 - iter 24/25 - loss 4.75982225 - samples/sec: 117.87 - lr: 0.300000 2021-03-26 05:29:40,043 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:29:40,045 EPOCH 30 done: loss 4.7312 - lr 0.3000000 2021-03-26 05:29:40,842 DEV : loss 6.537320613861084 - score 0.8995 2021-03-26 05:29:40,863 BAD EPOCHS (no improvement): 4 2021-03-26 05:29:40,864 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:29:41,845 epoch 31 - iter 2/25 - loss 4.45332122 - samples/sec: 130.55 - lr: 0.150000 2021-03-26 05:29:42,845 epoch 31 - iter 4/25 - loss 4.77456248 - samples/sec: 128.23 - lr: 0.150000 2021-03-26 05:29:43,911 epoch 31 - iter 6/25 - loss 4.98453426 - samples/sec: 120.27 - lr: 0.150000 2021-03-26 05:29:44,926 epoch 31 - iter 8/25 - loss 4.82937908 - samples/sec: 126.26 - lr: 0.150000 2021-03-26 05:29:46,072 epoch 31 - iter 10/25 - loss 4.81652617 - samples/sec: 111.93 - lr: 0.150000 2021-03-26 05:29:47,143 epoch 31 - iter 12/25 - loss 4.75675062 - samples/sec: 119.70 - lr: 0.150000 2021-03-26 05:29:48,116 epoch 31 - iter 14/25 - loss 4.63152540 - samples/sec: 131.80 - lr: 0.150000 2021-03-26 05:29:49,085 epoch 31 - iter 16/25 - loss 4.68138213 - samples/sec: 132.39 - lr: 0.150000 2021-03-26 05:29:50,143 epoch 31 - iter 18/25 - loss 4.68593766 - samples/sec: 121.12 - lr: 0.150000 2021-03-26 05:29:51,145 epoch 31 - iter 20/25 - loss 4.57285674 - samples/sec: 127.93 - lr: 0.150000 2021-03-26 05:29:52,061 epoch 31 - iter 22/25 - loss 4.52167766 - samples/sec: 140.15 - lr: 0.150000 2021-03-26 05:29:53,091 epoch 31 - iter 24/25 - loss 4.50825019 - samples/sec: 124.33 - lr: 0.150000 2021-03-26 05:29:53,517 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:29:53,518 EPOCH 31 done: loss 4.5345 - lr 0.1500000 2021-03-26 05:29:54,316 DEV : loss 6.355429649353027 - score 0.9032 2021-03-26 05:29:54,342 BAD EPOCHS (no improvement): 0 2021-03-26 05:30:04,001 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:30:04,983 epoch 32 - iter 2/25 - loss 3.71819592 - samples/sec: 130.68 - lr: 0.150000 2021-03-26 05:30:06,056 epoch 32 - iter 4/25 - loss 3.12705719 - samples/sec: 119.66 - lr: 0.150000 2021-03-26 05:30:07,109 epoch 32 - iter 6/25 - loss 3.75584714 - samples/sec: 121.68 - lr: 0.150000 2021-03-26 05:30:08,066 epoch 32 - iter 8/25 - loss 3.80353063 - samples/sec: 134.08 - lr: 0.150000 2021-03-26 05:30:09,040 epoch 32 - iter 10/25 - loss 3.88620362 - samples/sec: 131.58 - lr: 0.150000 2021-03-26 05:30:10,029 epoch 32 - iter 12/25 - loss 3.96990573 - samples/sec: 129.68 - lr: 0.150000 2021-03-26 05:30:11,052 epoch 32 - iter 14/25 - loss 3.94366251 - samples/sec: 125.40 - lr: 0.150000 2021-03-26 05:30:12,113 epoch 32 - iter 16/25 - loss 3.95921826 - samples/sec: 120.83 - lr: 0.150000 2021-03-26 05:30:13,076 epoch 32 - iter 18/25 - loss 4.00487158 - samples/sec: 133.16 - lr: 0.150000 2021-03-26 05:30:14,024 epoch 32 - iter 20/25 - loss 3.98131732 - samples/sec: 135.31 - lr: 0.150000 2021-03-26 05:30:15,062 epoch 32 - iter 22/25 - loss 3.95869911 - samples/sec: 123.46 - lr: 0.150000 2021-03-26 05:30:16,072 epoch 32 - iter 24/25 - loss 3.95320159 - samples/sec: 126.86 - lr: 0.150000 2021-03-26 05:30:16,475 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:30:16,476 EPOCH 32 done: loss 4.0076 - lr 0.1500000 2021-03-26 05:30:17,282 DEV : loss 6.234824180603027 - score 0.9098 2021-03-26 05:30:17,308 BAD EPOCHS (no improvement): 0 2021-03-26 05:30:27,096 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:30:28,289 epoch 33 - iter 2/25 - loss 3.86456370 - samples/sec: 107.51 - lr: 0.150000 2021-03-26 05:30:29,477 epoch 33 - iter 4/25 - loss 3.84213394 - samples/sec: 107.94 - lr: 0.150000 2021-03-26 05:30:30,473 epoch 33 - iter 6/25 - loss 3.88618358 - samples/sec: 128.64 - lr: 0.150000 2021-03-26 05:30:31,506 epoch 33 - iter 8/25 - loss 3.77276155 - samples/sec: 124.09 - lr: 0.150000 2021-03-26 05:30:32,470 epoch 33 - iter 10/25 - loss 3.84732611 - samples/sec: 132.98 - lr: 0.150000 2021-03-26 05:30:33,478 epoch 33 - iter 12/25 - loss 3.89968775 - samples/sec: 127.17 - lr: 0.150000 2021-03-26 05:30:34,514 epoch 33 - iter 14/25 - loss 3.83863291 - samples/sec: 123.71 - lr: 0.150000 2021-03-26 05:30:35,475 epoch 33 - iter 16/25 - loss 3.93329898 - samples/sec: 133.42 - lr: 0.150000 2021-03-26 05:30:36,449 epoch 33 - iter 18/25 - loss 3.88469691 - samples/sec: 131.60 - lr: 0.150000 2021-03-26 05:30:37,630 epoch 33 - iter 20/25 - loss 3.87576485 - samples/sec: 108.50 - lr: 0.150000 2021-03-26 05:30:39,099 epoch 33 - iter 22/25 - loss 3.88772985 - samples/sec: 87.29 - lr: 0.150000 2021-03-26 05:30:40,419 epoch 33 - iter 24/25 - loss 3.87317472 - samples/sec: 97.19 - lr: 0.150000 2021-03-26 05:30:40,976 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:30:40,977 EPOCH 33 done: loss 3.9395 - lr 0.1500000 2021-03-26 05:30:41,789 DEV : loss 6.255607604980469 - score 0.9077 2021-03-26 05:30:41,816 BAD EPOCHS (no improvement): 1 2021-03-26 05:30:41,817 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:30:42,792 epoch 34 - iter 2/25 - loss 4.43583751 - samples/sec: 131.50 - lr: 0.150000 2021-03-26 05:30:43,852 epoch 34 - iter 4/25 - loss 4.17272305 - samples/sec: 120.94 - lr: 0.150000 2021-03-26 05:30:44,826 epoch 34 - iter 6/25 - loss 4.04421759 - samples/sec: 131.67 - lr: 0.150000 2021-03-26 05:30:45,798 epoch 34 - iter 8/25 - loss 3.94364154 - samples/sec: 131.89 - lr: 0.150000 2021-03-26 05:30:46,941 epoch 34 - iter 10/25 - loss 4.09087896 - samples/sec: 112.17 - lr: 0.150000 2021-03-26 05:30:48,254 epoch 34 - iter 12/25 - loss 4.05837268 - samples/sec: 97.58 - lr: 0.150000 2021-03-26 05:30:49,198 epoch 34 - iter 14/25 - loss 4.10885179 - samples/sec: 135.76 - lr: 0.150000 2021-03-26 05:30:50,242 epoch 34 - iter 16/25 - loss 4.08000012 - samples/sec: 122.93 - lr: 0.150000 2021-03-26 05:30:51,204 epoch 34 - iter 18/25 - loss 4.03305531 - samples/sec: 133.30 - lr: 0.150000 2021-03-26 05:30:52,261 epoch 34 - iter 20/25 - loss 4.00350456 - samples/sec: 121.28 - lr: 0.150000 2021-03-26 05:30:53,384 epoch 34 - iter 22/25 - loss 4.03260352 - samples/sec: 114.20 - lr: 0.150000 2021-03-26 05:30:54,485 epoch 34 - iter 24/25 - loss 4.04323124 - samples/sec: 116.79 - lr: 0.150000 2021-03-26 05:30:54,899 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:30:54,900 EPOCH 34 done: loss 4.0360 - lr 0.1500000 2021-03-26 05:30:55,696 DEV : loss 6.293361186981201 - score 0.9056 2021-03-26 05:30:55,723 BAD EPOCHS (no improvement): 2 2021-03-26 05:30:55,723 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:30:56,785 epoch 35 - iter 2/25 - loss 4.15070677 - samples/sec: 120.74 - lr: 0.150000 2021-03-26 05:30:57,731 epoch 35 - iter 4/25 - loss 3.72079283 - samples/sec: 135.67 - lr: 0.150000 2021-03-26 05:30:58,683 epoch 35 - iter 6/25 - loss 3.54775671 - samples/sec: 134.58 - lr: 0.150000 2021-03-26 05:30:59,699 epoch 35 - iter 8/25 - loss 3.53297698 - samples/sec: 126.23 - lr: 0.150000 2021-03-26 05:31:00,601 epoch 35 - iter 10/25 - loss 3.63503337 - samples/sec: 142.13 - lr: 0.150000 2021-03-26 05:31:01,469 epoch 35 - iter 12/25 - loss 3.80509043 - samples/sec: 147.60 - lr: 0.150000 2021-03-26 05:31:02,457 epoch 35 - iter 14/25 - loss 3.84039981 - samples/sec: 129.73 - lr: 0.150000 2021-03-26 05:31:03,372 epoch 35 - iter 16/25 - loss 3.75528948 - samples/sec: 140.06 - lr: 0.150000 2021-03-26 05:31:04,357 epoch 35 - iter 18/25 - loss 3.78421675 - samples/sec: 130.23 - lr: 0.150000 2021-03-26 05:31:05,294 epoch 35 - iter 20/25 - loss 3.84083500 - samples/sec: 136.68 - lr: 0.150000 2021-03-26 05:31:06,327 epoch 35 - iter 22/25 - loss 3.80932782 - samples/sec: 124.13 - lr: 0.150000 2021-03-26 05:31:07,391 epoch 35 - iter 24/25 - loss 3.83606430 - samples/sec: 120.46 - lr: 0.150000 2021-03-26 05:31:07,829 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:31:07,830 EPOCH 35 done: loss 3.8055 - lr 0.1500000 2021-03-26 05:31:08,645 DEV : loss 6.475340843200684 - score 0.9042 2021-03-26 05:31:08,664 BAD EPOCHS (no improvement): 3 2021-03-26 05:31:08,664 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:31:09,654 epoch 36 - iter 2/25 - loss 3.07113969 - samples/sec: 129.55 - lr: 0.150000 2021-03-26 05:31:10,606 epoch 36 - iter 4/25 - loss 3.30959159 - samples/sec: 134.75 - lr: 0.150000 2021-03-26 05:31:11,589 epoch 36 - iter 6/25 - loss 3.37420321 - samples/sec: 130.38 - lr: 0.150000 2021-03-26 05:31:12,610 epoch 36 - iter 8/25 - loss 3.33646402 - samples/sec: 125.47 - lr: 0.150000 2021-03-26 05:31:13,666 epoch 36 - iter 10/25 - loss 3.33366504 - samples/sec: 121.47 - lr: 0.150000 2021-03-26 05:31:14,639 epoch 36 - iter 12/25 - loss 3.28806067 - samples/sec: 131.85 - lr: 0.150000 2021-03-26 05:31:15,681 epoch 36 - iter 14/25 - loss 3.42399103 - samples/sec: 123.02 - lr: 0.150000 2021-03-26 05:31:16,717 epoch 36 - iter 16/25 - loss 3.52277085 - samples/sec: 123.67 - lr: 0.150000 2021-03-26 05:31:17,703 epoch 36 - iter 18/25 - loss 3.48878141 - samples/sec: 130.17 - lr: 0.150000 2021-03-26 05:31:18,687 epoch 36 - iter 20/25 - loss 3.50925676 - samples/sec: 130.27 - lr: 0.150000 2021-03-26 05:31:19,707 epoch 36 - iter 22/25 - loss 3.53277724 - samples/sec: 125.79 - lr: 0.150000 2021-03-26 05:31:20,915 epoch 36 - iter 24/25 - loss 3.55849650 - samples/sec: 106.08 - lr: 0.150000 2021-03-26 05:31:21,477 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:31:21,479 EPOCH 36 done: loss 3.6749 - lr 0.1500000 2021-03-26 05:31:22,280 DEV : loss 6.371415615081787 - score 0.904 2021-03-26 05:31:22,311 BAD EPOCHS (no improvement): 4 2021-03-26 05:31:22,312 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:31:23,281 epoch 37 - iter 2/25 - loss 3.39781427 - samples/sec: 132.23 - lr: 0.075000 2021-03-26 05:31:24,245 epoch 37 - iter 4/25 - loss 3.25910103 - samples/sec: 132.95 - lr: 0.075000 2021-03-26 05:31:25,398 epoch 37 - iter 6/25 - loss 3.40158709 - samples/sec: 111.26 - lr: 0.075000 2021-03-26 05:31:26,477 epoch 37 - iter 8/25 - loss 3.44064540 - samples/sec: 118.84 - lr: 0.075000 2021-03-26 05:31:27,459 epoch 37 - iter 10/25 - loss 3.39384527 - samples/sec: 130.54 - lr: 0.075000 2021-03-26 05:31:28,466 epoch 37 - iter 12/25 - loss 3.36764000 - samples/sec: 127.31 - lr: 0.075000 2021-03-26 05:31:29,547 epoch 37 - iter 14/25 - loss 3.42956323 - samples/sec: 118.53 - lr: 0.075000 2021-03-26 05:31:30,658 epoch 37 - iter 16/25 - loss 3.52977823 - samples/sec: 115.41 - lr: 0.075000 2021-03-26 05:31:31,864 epoch 37 - iter 18/25 - loss 3.54157498 - samples/sec: 106.20 - lr: 0.075000 2021-03-26 05:31:32,845 epoch 37 - iter 20/25 - loss 3.55722978 - samples/sec: 130.65 - lr: 0.075000 2021-03-26 05:31:33,849 epoch 37 - iter 22/25 - loss 3.58427523 - samples/sec: 127.62 - lr: 0.075000 2021-03-26 05:31:34,859 epoch 37 - iter 24/25 - loss 3.56133931 - samples/sec: 126.97 - lr: 0.075000 2021-03-26 05:31:35,250 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:31:35,251 EPOCH 37 done: loss 3.5676 - lr 0.0750000 2021-03-26 05:31:36,082 DEV : loss 6.338122367858887 - score 0.9034 2021-03-26 05:31:36,112 BAD EPOCHS (no improvement): 1 2021-03-26 05:31:36,113 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:31:37,043 epoch 38 - iter 2/25 - loss 3.31011808 - samples/sec: 137.87 - lr: 0.075000 2021-03-26 05:31:38,021 epoch 38 - iter 4/25 - loss 3.29740214 - samples/sec: 131.14 - lr: 0.075000 2021-03-26 05:31:39,028 epoch 38 - iter 6/25 - loss 3.31071031 - samples/sec: 127.24 - lr: 0.075000 2021-03-26 05:31:40,026 epoch 38 - iter 8/25 - loss 3.45570368 - samples/sec: 128.49 - lr: 0.075000 2021-03-26 05:31:41,035 epoch 38 - iter 10/25 - loss 3.50104599 - samples/sec: 126.94 - lr: 0.075000 2021-03-26 05:31:41,963 epoch 38 - iter 12/25 - loss 3.52571265 - samples/sec: 138.43 - lr: 0.075000 2021-03-26 05:31:42,920 epoch 38 - iter 14/25 - loss 3.42115290 - samples/sec: 133.98 - lr: 0.075000 2021-03-26 05:31:43,892 epoch 38 - iter 16/25 - loss 3.41037132 - samples/sec: 131.84 - lr: 0.075000 2021-03-26 05:31:44,868 epoch 38 - iter 18/25 - loss 3.32819162 - samples/sec: 131.31 - lr: 0.075000 2021-03-26 05:31:45,977 epoch 38 - iter 20/25 - loss 3.37640109 - samples/sec: 115.61 - lr: 0.075000 2021-03-26 05:31:46,938 epoch 38 - iter 22/25 - loss 3.35057510 - samples/sec: 133.40 - lr: 0.075000 2021-03-26 05:31:47,950 epoch 38 - iter 24/25 - loss 3.31927971 - samples/sec: 126.65 - lr: 0.075000 2021-03-26 05:31:48,426 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:31:48,428 EPOCH 38 done: loss 3.3323 - lr 0.0750000 2021-03-26 05:31:49,240 DEV : loss 6.319576263427734 - score 0.9028 2021-03-26 05:31:49,262 BAD EPOCHS (no improvement): 2 2021-03-26 05:31:49,262 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:31:50,330 epoch 39 - iter 2/25 - loss 3.29510450 - samples/sec: 120.01 - lr: 0.075000 2021-03-26 05:31:51,365 epoch 39 - iter 4/25 - loss 3.25340539 - samples/sec: 123.87 - lr: 0.075000 2021-03-26 05:31:52,482 epoch 39 - iter 6/25 - loss 3.37839731 - samples/sec: 114.79 - lr: 0.075000 2021-03-26 05:31:53,461 epoch 39 - iter 8/25 - loss 3.27371565 - samples/sec: 131.10 - lr: 0.075000 2021-03-26 05:31:54,453 epoch 39 - iter 10/25 - loss 3.29483247 - samples/sec: 129.14 - lr: 0.075000 2021-03-26 05:31:55,474 epoch 39 - iter 12/25 - loss 3.41713019 - samples/sec: 125.51 - lr: 0.075000 2021-03-26 05:31:56,485 epoch 39 - iter 14/25 - loss 3.50181120 - samples/sec: 126.82 - lr: 0.075000 2021-03-26 05:31:57,551 epoch 39 - iter 16/25 - loss 3.52467915 - samples/sec: 120.23 - lr: 0.075000 2021-03-26 05:31:58,575 epoch 39 - iter 18/25 - loss 3.49890432 - samples/sec: 125.11 - lr: 0.075000 2021-03-26 05:31:59,549 epoch 39 - iter 20/25 - loss 3.44117440 - samples/sec: 131.83 - lr: 0.075000 2021-03-26 05:32:00,602 epoch 39 - iter 22/25 - loss 3.44448699 - samples/sec: 121.74 - lr: 0.075000 2021-03-26 05:32:01,693 epoch 39 - iter 24/25 - loss 3.42074833 - samples/sec: 117.51 - lr: 0.075000 2021-03-26 05:32:02,109 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:32:02,109 EPOCH 39 done: loss 3.4132 - lr 0.0750000 2021-03-26 05:32:03,022 DEV : loss 6.410008907318115 - score 0.9026 2021-03-26 05:32:03,059 BAD EPOCHS (no improvement): 3 2021-03-26 05:32:03,060 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:32:04,419 epoch 40 - iter 2/25 - loss 3.88798189 - samples/sec: 94.24 - lr: 0.075000 2021-03-26 05:32:05,455 epoch 40 - iter 4/25 - loss 3.43252563 - samples/sec: 123.75 - lr: 0.075000 2021-03-26 05:32:06,402 epoch 40 - iter 6/25 - loss 3.40401618 - samples/sec: 135.34 - lr: 0.075000 2021-03-26 05:32:07,373 epoch 40 - iter 8/25 - loss 3.47805119 - samples/sec: 132.02 - lr: 0.075000 2021-03-26 05:32:08,406 epoch 40 - iter 10/25 - loss 3.35810285 - samples/sec: 124.20 - lr: 0.075000 2021-03-26 05:32:09,540 epoch 40 - iter 12/25 - loss 3.38514515 - samples/sec: 113.06 - lr: 0.075000 2021-03-26 05:32:10,601 epoch 40 - iter 14/25 - loss 3.45678117 - samples/sec: 120.87 - lr: 0.075000 2021-03-26 05:32:11,651 epoch 40 - iter 16/25 - loss 3.44114582 - samples/sec: 122.06 - lr: 0.075000 2021-03-26 05:32:12,652 epoch 40 - iter 18/25 - loss 3.41769059 - samples/sec: 128.04 - lr: 0.075000 2021-03-26 05:32:13,656 epoch 40 - iter 20/25 - loss 3.35850424 - samples/sec: 127.65 - lr: 0.075000 2021-03-26 05:32:14,629 epoch 40 - iter 22/25 - loss 3.37367962 - samples/sec: 131.95 - lr: 0.075000 2021-03-26 05:32:15,607 epoch 40 - iter 24/25 - loss 3.38378793 - samples/sec: 131.13 - lr: 0.075000 2021-03-26 05:32:16,021 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:32:16,022 EPOCH 40 done: loss 3.4051 - lr 0.0750000 2021-03-26 05:32:16,848 DEV : loss 6.3716535568237305 - score 0.905 2021-03-26 05:32:16,869 BAD EPOCHS (no improvement): 4 2021-03-26 05:32:16,870 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:32:17,957 epoch 41 - iter 2/25 - loss 3.34791684 - samples/sec: 117.92 - lr: 0.037500 2021-03-26 05:32:18,933 epoch 41 - iter 4/25 - loss 3.20045227 - samples/sec: 131.34 - lr: 0.037500 2021-03-26 05:32:19,933 epoch 41 - iter 6/25 - loss 3.11513686 - samples/sec: 128.19 - lr: 0.037500 2021-03-26 05:32:20,901 epoch 41 - iter 8/25 - loss 3.14670616 - samples/sec: 132.54 - lr: 0.037500 2021-03-26 05:32:21,826 epoch 41 - iter 10/25 - loss 3.06280284 - samples/sec: 138.61 - lr: 0.037500 2021-03-26 05:32:22,758 epoch 41 - iter 12/25 - loss 2.96115875 - samples/sec: 137.70 - lr: 0.037500 2021-03-26 05:32:23,710 epoch 41 - iter 14/25 - loss 3.05098459 - samples/sec: 134.59 - lr: 0.037500 2021-03-26 05:32:24,724 epoch 41 - iter 16/25 - loss 3.09193727 - samples/sec: 126.46 - lr: 0.037500 2021-03-26 05:32:25,707 epoch 41 - iter 18/25 - loss 3.13746330 - samples/sec: 130.49 - lr: 0.037500 2021-03-26 05:32:26,736 epoch 41 - iter 20/25 - loss 3.17778745 - samples/sec: 124.68 - lr: 0.037500 2021-03-26 05:32:27,756 epoch 41 - iter 22/25 - loss 3.17188667 - samples/sec: 125.86 - lr: 0.037500 2021-03-26 05:32:28,751 epoch 41 - iter 24/25 - loss 3.18510689 - samples/sec: 128.90 - lr: 0.037500 2021-03-26 05:32:29,223 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:32:29,223 EPOCH 41 done: loss 3.1504 - lr 0.0375000 2021-03-26 05:32:30,005 DEV : loss 6.398674011230469 - score 0.9053 2021-03-26 05:32:30,030 BAD EPOCHS (no improvement): 1 2021-03-26 05:32:30,031 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:32:30,985 epoch 42 - iter 2/25 - loss 3.15925717 - samples/sec: 134.36 - lr: 0.037500 2021-03-26 05:32:32,147 epoch 42 - iter 4/25 - loss 3.45493281 - samples/sec: 110.30 - lr: 0.037500 2021-03-26 05:32:33,177 epoch 42 - iter 6/25 - loss 3.47178149 - samples/sec: 124.61 - lr: 0.037500 2021-03-26 05:32:34,163 epoch 42 - iter 8/25 - loss 3.59958142 - samples/sec: 129.93 - lr: 0.037500 2021-03-26 05:32:35,197 epoch 42 - iter 10/25 - loss 3.63480940 - samples/sec: 123.95 - lr: 0.037500 2021-03-26 05:32:36,247 epoch 42 - iter 12/25 - loss 3.63019852 - samples/sec: 122.12 - lr: 0.037500 2021-03-26 05:32:37,260 epoch 42 - iter 14/25 - loss 3.54821042 - samples/sec: 126.46 - lr: 0.037500 2021-03-26 05:32:38,251 epoch 42 - iter 16/25 - loss 3.53470874 - samples/sec: 129.42 - lr: 0.037500 2021-03-26 05:32:39,251 epoch 42 - iter 18/25 - loss 3.45202567 - samples/sec: 128.12 - lr: 0.037500 2021-03-26 05:32:40,278 epoch 42 - iter 20/25 - loss 3.44232910 - samples/sec: 124.83 - lr: 0.037500 2021-03-26 05:32:41,627 epoch 42 - iter 22/25 - loss 3.45118979 - samples/sec: 95.01 - lr: 0.037500 2021-03-26 05:32:42,888 epoch 42 - iter 24/25 - loss 3.40880919 - samples/sec: 101.62 - lr: 0.037500 2021-03-26 05:32:43,434 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:32:43,435 EPOCH 42 done: loss 3.3765 - lr 0.0375000 2021-03-26 05:32:44,352 DEV : loss 6.4176788330078125 - score 0.9057 2021-03-26 05:32:44,377 BAD EPOCHS (no improvement): 2 2021-03-26 05:32:44,378 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:32:45,335 epoch 43 - iter 2/25 - loss 2.82083404 - samples/sec: 133.97 - lr: 0.037500 2021-03-26 05:32:46,395 epoch 43 - iter 4/25 - loss 3.20271707 - samples/sec: 120.95 - lr: 0.037500 2021-03-26 05:32:47,336 epoch 43 - iter 6/25 - loss 3.25204869 - samples/sec: 136.25 - lr: 0.037500 2021-03-26 05:32:48,299 epoch 43 - iter 8/25 - loss 3.03663874 - samples/sec: 133.12 - lr: 0.037500 2021-03-26 05:32:49,351 epoch 43 - iter 10/25 - loss 3.14305215 - samples/sec: 121.88 - lr: 0.037500 2021-03-26 05:32:50,326 epoch 43 - iter 12/25 - loss 3.22221923 - samples/sec: 131.41 - lr: 0.037500 2021-03-26 05:32:51,400 epoch 43 - iter 14/25 - loss 3.29125517 - samples/sec: 119.39 - lr: 0.037500 2021-03-26 05:32:52,432 epoch 43 - iter 16/25 - loss 3.22110757 - samples/sec: 124.14 - lr: 0.037500 2021-03-26 05:32:53,486 epoch 43 - iter 18/25 - loss 3.21015777 - samples/sec: 121.63 - lr: 0.037500 2021-03-26 05:32:54,487 epoch 43 - iter 20/25 - loss 3.20252607 - samples/sec: 128.80 - lr: 0.037500 2021-03-26 05:32:55,439 epoch 43 - iter 22/25 - loss 3.22109760 - samples/sec: 134.82 - lr: 0.037500 2021-03-26 05:32:56,690 epoch 43 - iter 24/25 - loss 3.17892468 - samples/sec: 102.49 - lr: 0.037500 2021-03-26 05:32:57,175 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:32:57,177 EPOCH 43 done: loss 3.1621 - lr 0.0375000 2021-03-26 05:32:57,997 DEV : loss 6.35455322265625 - score 0.9098 2021-03-26 05:32:58,023 BAD EPOCHS (no improvement): 3 2021-03-26 05:32:58,024 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:32:59,031 epoch 44 - iter 2/25 - loss 3.62210631 - samples/sec: 127.41 - lr: 0.037500 2021-03-26 05:33:00,050 epoch 44 - iter 4/25 - loss 3.45865715 - samples/sec: 125.70 - lr: 0.037500 2021-03-26 05:33:01,058 epoch 44 - iter 6/25 - loss 3.61776721 - samples/sec: 127.25 - lr: 0.037500 2021-03-26 05:33:02,125 epoch 44 - iter 8/25 - loss 3.50572792 - samples/sec: 120.06 - lr: 0.037500 2021-03-26 05:33:03,192 epoch 44 - iter 10/25 - loss 3.52592924 - samples/sec: 120.09 - lr: 0.037500 2021-03-26 05:33:04,135 epoch 44 - iter 12/25 - loss 3.39799915 - samples/sec: 136.00 - lr: 0.037500 2021-03-26 05:33:05,091 epoch 44 - iter 14/25 - loss 3.30765348 - samples/sec: 134.25 - lr: 0.037500 2021-03-26 05:33:06,141 epoch 44 - iter 16/25 - loss 3.21087818 - samples/sec: 122.22 - lr: 0.037500 2021-03-26 05:33:07,140 epoch 44 - iter 18/25 - loss 3.22574906 - samples/sec: 128.36 - lr: 0.037500 2021-03-26 05:33:08,100 epoch 44 - iter 20/25 - loss 3.22437223 - samples/sec: 133.61 - lr: 0.037500 2021-03-26 05:33:09,127 epoch 44 - iter 22/25 - loss 3.21107636 - samples/sec: 124.97 - lr: 0.037500 2021-03-26 05:33:10,079 epoch 44 - iter 24/25 - loss 3.21195141 - samples/sec: 134.66 - lr: 0.037500 2021-03-26 05:33:10,502 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:33:10,503 EPOCH 44 done: loss 3.2208 - lr 0.0375000 2021-03-26 05:33:11,338 DEV : loss 6.370487213134766 - score 0.9091 2021-03-26 05:33:11,361 BAD EPOCHS (no improvement): 4 2021-03-26 05:33:11,361 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:33:12,317 epoch 45 - iter 2/25 - loss 3.09825575 - samples/sec: 134.09 - lr: 0.018750 2021-03-26 05:33:13,357 epoch 45 - iter 4/25 - loss 3.01253098 - samples/sec: 123.29 - lr: 0.018750 2021-03-26 05:33:14,358 epoch 45 - iter 6/25 - loss 3.01518444 - samples/sec: 128.01 - lr: 0.018750 2021-03-26 05:33:15,372 epoch 45 - iter 8/25 - loss 2.96483403 - samples/sec: 126.48 - lr: 0.018750 2021-03-26 05:33:16,288 epoch 45 - iter 10/25 - loss 3.02756095 - samples/sec: 139.85 - lr: 0.018750 2021-03-26 05:33:17,306 epoch 45 - iter 12/25 - loss 3.05424559 - samples/sec: 125.89 - lr: 0.018750 2021-03-26 05:33:18,188 epoch 45 - iter 14/25 - loss 2.99477315 - samples/sec: 145.41 - lr: 0.018750 2021-03-26 05:33:19,206 epoch 45 - iter 16/25 - loss 3.03542289 - samples/sec: 125.96 - lr: 0.018750 2021-03-26 05:33:20,261 epoch 45 - iter 18/25 - loss 2.99328383 - samples/sec: 121.49 - lr: 0.018750 2021-03-26 05:33:21,242 epoch 45 - iter 20/25 - loss 2.96314116 - samples/sec: 130.65 - lr: 0.018750 2021-03-26 05:33:22,238 epoch 45 - iter 22/25 - loss 2.94665932 - samples/sec: 128.64 - lr: 0.018750 2021-03-26 05:33:23,322 epoch 45 - iter 24/25 - loss 2.94883043 - samples/sec: 118.29 - lr: 0.018750 2021-03-26 05:33:23,816 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:33:23,817 EPOCH 45 done: loss 3.0238 - lr 0.0187500 2021-03-26 05:33:24,632 DEV : loss 6.327853679656982 - score 0.9091 2021-03-26 05:33:24,657 BAD EPOCHS (no improvement): 1 2021-03-26 05:33:24,658 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:33:25,639 epoch 46 - iter 2/25 - loss 3.14161575 - samples/sec: 130.62 - lr: 0.018750 2021-03-26 05:33:26,581 epoch 46 - iter 4/25 - loss 3.40499622 - samples/sec: 136.11 - lr: 0.018750 2021-03-26 05:33:27,609 epoch 46 - iter 6/25 - loss 3.28151731 - samples/sec: 124.88 - lr: 0.018750 2021-03-26 05:33:28,675 epoch 46 - iter 8/25 - loss 3.20727524 - samples/sec: 120.27 - lr: 0.018750 2021-03-26 05:33:29,690 epoch 46 - iter 10/25 - loss 3.12398851 - samples/sec: 126.18 - lr: 0.018750 2021-03-26 05:33:30,707 epoch 46 - iter 12/25 - loss 3.18230096 - samples/sec: 126.08 - lr: 0.018750 2021-03-26 05:33:31,716 epoch 46 - iter 14/25 - loss 3.17806830 - samples/sec: 127.03 - lr: 0.018750 2021-03-26 05:33:32,697 epoch 46 - iter 16/25 - loss 3.15068266 - samples/sec: 130.74 - lr: 0.018750 2021-03-26 05:33:33,684 epoch 46 - iter 18/25 - loss 3.09301374 - samples/sec: 129.92 - lr: 0.018750 2021-03-26 05:33:34,735 epoch 46 - iter 20/25 - loss 3.11505752 - samples/sec: 122.02 - lr: 0.018750 2021-03-26 05:33:35,761 epoch 46 - iter 22/25 - loss 3.13960059 - samples/sec: 124.91 - lr: 0.018750 2021-03-26 05:33:36,756 epoch 46 - iter 24/25 - loss 3.14284589 - samples/sec: 128.82 - lr: 0.018750 2021-03-26 05:33:37,172 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:33:37,173 EPOCH 46 done: loss 3.1034 - lr 0.0187500 2021-03-26 05:33:37,957 DEV : loss 6.337489128112793 - score 0.9087 2021-03-26 05:33:37,976 BAD EPOCHS (no improvement): 2 2021-03-26 05:33:37,977 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:33:38,964 epoch 47 - iter 2/25 - loss 2.61264515 - samples/sec: 129.86 - lr: 0.018750 2021-03-26 05:33:40,083 epoch 47 - iter 4/25 - loss 3.06716627 - samples/sec: 114.66 - lr: 0.018750 2021-03-26 05:33:41,147 epoch 47 - iter 6/25 - loss 3.07863943 - samples/sec: 120.99 - lr: 0.018750 2021-03-26 05:33:42,200 epoch 47 - iter 8/25 - loss 3.02197772 - samples/sec: 121.70 - lr: 0.018750 2021-03-26 05:33:43,263 epoch 47 - iter 10/25 - loss 2.94771438 - samples/sec: 120.58 - lr: 0.018750 2021-03-26 05:33:44,381 epoch 47 - iter 12/25 - loss 3.08820546 - samples/sec: 114.68 - lr: 0.018750 2021-03-26 05:33:45,348 epoch 47 - iter 14/25 - loss 3.06957422 - samples/sec: 132.64 - lr: 0.018750 2021-03-26 05:33:46,439 epoch 47 - iter 16/25 - loss 3.05880760 - samples/sec: 117.44 - lr: 0.018750 2021-03-26 05:33:47,395 epoch 47 - iter 18/25 - loss 3.07288737 - samples/sec: 134.16 - lr: 0.018750 2021-03-26 05:33:48,386 epoch 47 - iter 20/25 - loss 3.03714904 - samples/sec: 129.25 - lr: 0.018750 2021-03-26 05:33:49,410 epoch 47 - iter 22/25 - loss 3.10431696 - samples/sec: 125.20 - lr: 0.018750 2021-03-26 05:33:50,409 epoch 47 - iter 24/25 - loss 3.18330712 - samples/sec: 128.37 - lr: 0.018750 2021-03-26 05:33:50,815 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:33:50,815 EPOCH 47 done: loss 3.2111 - lr 0.0187500 2021-03-26 05:33:51,603 DEV : loss 6.363578796386719 - score 0.9063 2021-03-26 05:33:51,628 BAD EPOCHS (no improvement): 3 2021-03-26 05:33:51,629 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:33:52,737 epoch 48 - iter 2/25 - loss 3.24833369 - samples/sec: 115.76 - lr: 0.018750 2021-03-26 05:33:53,781 epoch 48 - iter 4/25 - loss 3.13858742 - samples/sec: 122.69 - lr: 0.018750 2021-03-26 05:33:54,843 epoch 48 - iter 6/25 - loss 3.05964986 - samples/sec: 120.77 - lr: 0.018750 2021-03-26 05:33:55,835 epoch 48 - iter 8/25 - loss 3.11314756 - samples/sec: 129.26 - lr: 0.018750 2021-03-26 05:33:56,761 epoch 48 - iter 10/25 - loss 3.24263086 - samples/sec: 138.52 - lr: 0.018750 2021-03-26 05:33:57,660 epoch 48 - iter 12/25 - loss 3.24108704 - samples/sec: 142.56 - lr: 0.018750 2021-03-26 05:33:58,584 epoch 48 - iter 14/25 - loss 3.15899248 - samples/sec: 138.89 - lr: 0.018750 2021-03-26 05:33:59,661 epoch 48 - iter 16/25 - loss 3.08050616 - samples/sec: 119.03 - lr: 0.018750 2021-03-26 05:34:00,609 epoch 48 - iter 18/25 - loss 3.03743966 - samples/sec: 135.31 - lr: 0.018750 2021-03-26 05:34:01,608 epoch 48 - iter 20/25 - loss 3.04794432 - samples/sec: 128.33 - lr: 0.018750 2021-03-26 05:34:02,693 epoch 48 - iter 22/25 - loss 3.06857406 - samples/sec: 118.19 - lr: 0.018750 2021-03-26 05:34:03,674 epoch 48 - iter 24/25 - loss 3.06631840 - samples/sec: 130.70 - lr: 0.018750 2021-03-26 05:34:04,129 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:34:04,130 EPOCH 48 done: loss 3.0756 - lr 0.0187500 2021-03-26 05:34:04,920 DEV : loss 6.344338893890381 - score 0.9079 2021-03-26 05:34:04,946 BAD EPOCHS (no improvement): 4 2021-03-26 05:34:04,947 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:34:05,956 epoch 49 - iter 2/25 - loss 2.90295172 - samples/sec: 127.06 - lr: 0.009375 2021-03-26 05:34:07,057 epoch 49 - iter 4/25 - loss 2.80841428 - samples/sec: 116.39 - lr: 0.009375 2021-03-26 05:34:08,051 epoch 49 - iter 6/25 - loss 3.03871755 - samples/sec: 128.94 - lr: 0.009375 2021-03-26 05:34:09,033 epoch 49 - iter 8/25 - loss 2.99644694 - samples/sec: 130.55 - lr: 0.009375 2021-03-26 05:34:10,126 epoch 49 - iter 10/25 - loss 3.09761269 - samples/sec: 117.32 - lr: 0.009375 2021-03-26 05:34:11,289 epoch 49 - iter 12/25 - loss 3.13177150 - samples/sec: 110.28 - lr: 0.009375 2021-03-26 05:34:12,285 epoch 49 - iter 14/25 - loss 3.14276028 - samples/sec: 128.80 - lr: 0.009375 2021-03-26 05:34:13,293 epoch 49 - iter 16/25 - loss 3.15640897 - samples/sec: 127.17 - lr: 0.009375 2021-03-26 05:34:14,359 epoch 49 - iter 18/25 - loss 3.10915577 - samples/sec: 120.22 - lr: 0.009375 2021-03-26 05:34:15,401 epoch 49 - iter 20/25 - loss 3.11137698 - samples/sec: 122.95 - lr: 0.009375 2021-03-26 05:34:16,557 epoch 49 - iter 22/25 - loss 3.11102456 - samples/sec: 110.91 - lr: 0.009375 2021-03-26 05:34:17,577 epoch 49 - iter 24/25 - loss 3.09136966 - samples/sec: 125.73 - lr: 0.009375 2021-03-26 05:34:17,991 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:34:17,992 EPOCH 49 done: loss 3.0979 - lr 0.0093750 2021-03-26 05:34:18,835 DEV : loss 6.343489646911621 - score 0.9083 2021-03-26 05:34:18,863 BAD EPOCHS (no improvement): 1 2021-03-26 05:34:18,863 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:34:20,004 epoch 50 - iter 2/25 - loss 3.47275829 - samples/sec: 112.39 - lr: 0.009375 2021-03-26 05:34:20,952 epoch 50 - iter 4/25 - loss 3.08802807 - samples/sec: 135.18 - lr: 0.009375 2021-03-26 05:34:21,939 epoch 50 - iter 6/25 - loss 3.08415246 - samples/sec: 129.81 - lr: 0.009375 2021-03-26 05:34:22,827 epoch 50 - iter 8/25 - loss 2.93821096 - samples/sec: 144.47 - lr: 0.009375 2021-03-26 05:34:23,795 epoch 50 - iter 10/25 - loss 2.97539239 - samples/sec: 132.39 - lr: 0.009375 2021-03-26 05:34:24,733 epoch 50 - iter 12/25 - loss 3.06500099 - samples/sec: 136.72 - lr: 0.009375 2021-03-26 05:34:25,685 epoch 50 - iter 14/25 - loss 3.03774973 - samples/sec: 134.60 - lr: 0.009375 2021-03-26 05:34:26,743 epoch 50 - iter 16/25 - loss 3.02531400 - samples/sec: 121.21 - lr: 0.009375 2021-03-26 05:34:27,751 epoch 50 - iter 18/25 - loss 3.06442942 - samples/sec: 127.10 - lr: 0.009375 2021-03-26 05:34:28,821 epoch 50 - iter 20/25 - loss 3.06219168 - samples/sec: 119.87 - lr: 0.009375 2021-03-26 05:34:29,823 epoch 50 - iter 22/25 - loss 3.11562943 - samples/sec: 127.91 - lr: 0.009375 2021-03-26 05:34:30,851 epoch 50 - iter 24/25 - loss 3.09196840 - samples/sec: 124.64 - lr: 0.009375 2021-03-26 05:34:31,287 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:34:31,288 EPOCH 50 done: loss 3.0812 - lr 0.0093750 2021-03-26 05:34:32,108 DEV : loss 6.350255489349365 - score 0.9083 2021-03-26 05:34:32,134 BAD EPOCHS (no improvement): 2 2021-03-26 05:34:32,135 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:34:33,177 epoch 51 - iter 2/25 - loss 2.86416495 - samples/sec: 123.12 - lr: 0.009375 2021-03-26 05:34:34,203 epoch 51 - iter 4/25 - loss 2.97483855 - samples/sec: 125.12 - lr: 0.009375 2021-03-26 05:34:35,163 epoch 51 - iter 6/25 - loss 3.07030086 - samples/sec: 133.64 - lr: 0.009375 2021-03-26 05:34:36,245 epoch 51 - iter 8/25 - loss 3.02221566 - samples/sec: 118.49 - lr: 0.009375 2021-03-26 05:34:37,260 epoch 51 - iter 10/25 - loss 3.09667757 - samples/sec: 126.23 - lr: 0.009375 2021-03-26 05:34:38,207 epoch 51 - iter 12/25 - loss 2.98070341 - samples/sec: 135.28 - lr: 0.009375 2021-03-26 05:34:39,182 epoch 51 - iter 14/25 - loss 3.00901944 - samples/sec: 131.61 - lr: 0.009375 2021-03-26 05:34:40,156 epoch 51 - iter 16/25 - loss 3.00842324 - samples/sec: 131.62 - lr: 0.009375 2021-03-26 05:34:41,214 epoch 51 - iter 18/25 - loss 3.00116184 - samples/sec: 121.14 - lr: 0.009375 2021-03-26 05:34:42,194 epoch 51 - iter 20/25 - loss 2.96676878 - samples/sec: 130.76 - lr: 0.009375 2021-03-26 05:34:43,226 epoch 51 - iter 22/25 - loss 2.99044778 - samples/sec: 124.21 - lr: 0.009375 2021-03-26 05:34:44,183 epoch 51 - iter 24/25 - loss 2.96513258 - samples/sec: 133.94 - lr: 0.009375 2021-03-26 05:34:44,625 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:34:44,626 EPOCH 51 done: loss 2.9734 - lr 0.0093750 2021-03-26 05:34:45,445 DEV : loss 6.363671779632568 - score 0.9087 2021-03-26 05:34:45,464 BAD EPOCHS (no improvement): 3 2021-03-26 05:34:45,464 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:34:46,437 epoch 52 - iter 2/25 - loss 2.26139700 - samples/sec: 131.88 - lr: 0.009375 2021-03-26 05:34:47,526 epoch 52 - iter 4/25 - loss 3.09052759 - samples/sec: 117.63 - lr: 0.009375 2021-03-26 05:34:48,565 epoch 52 - iter 6/25 - loss 3.00616634 - samples/sec: 123.55 - lr: 0.009375 2021-03-26 05:34:49,646 epoch 52 - iter 8/25 - loss 3.15239090 - samples/sec: 118.57 - lr: 0.009375 2021-03-26 05:34:50,583 epoch 52 - iter 10/25 - loss 3.08683448 - samples/sec: 136.88 - lr: 0.009375 2021-03-26 05:34:51,666 epoch 52 - iter 12/25 - loss 3.13589352 - samples/sec: 118.42 - lr: 0.009375 2021-03-26 05:34:52,599 epoch 52 - iter 14/25 - loss 3.12352077 - samples/sec: 137.38 - lr: 0.009375 2021-03-26 05:34:53,593 epoch 52 - iter 16/25 - loss 3.15578875 - samples/sec: 128.99 - lr: 0.009375 2021-03-26 05:34:54,508 epoch 52 - iter 18/25 - loss 3.09990822 - samples/sec: 139.98 - lr: 0.009375 2021-03-26 05:34:55,486 epoch 52 - iter 20/25 - loss 3.11360563 - samples/sec: 131.33 - lr: 0.009375 2021-03-26 05:34:56,500 epoch 52 - iter 22/25 - loss 3.08807242 - samples/sec: 126.45 - lr: 0.009375 2021-03-26 05:34:57,670 epoch 52 - iter 24/25 - loss 3.12041364 - samples/sec: 109.59 - lr: 0.009375 2021-03-26 05:34:58,163 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:34:58,164 EPOCH 52 done: loss 3.1056 - lr 0.0093750 2021-03-26 05:34:58,969 DEV : loss 6.35579252243042 - score 0.9071 2021-03-26 05:34:58,995 BAD EPOCHS (no improvement): 4 2021-03-26 05:34:58,996 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:34:59,929 epoch 53 - iter 2/25 - loss 3.55681157 - samples/sec: 137.52 - lr: 0.004687 2021-03-26 05:35:01,034 epoch 53 - iter 4/25 - loss 3.48824114 - samples/sec: 116.18 - lr: 0.004687 2021-03-26 05:35:01,980 epoch 53 - iter 6/25 - loss 3.35435299 - samples/sec: 135.43 - lr: 0.004687 2021-03-26 05:35:02,911 epoch 53 - iter 8/25 - loss 3.39199522 - samples/sec: 137.81 - lr: 0.004687 2021-03-26 05:35:03,926 epoch 53 - iter 10/25 - loss 3.30036414 - samples/sec: 126.29 - lr: 0.004687 2021-03-26 05:35:04,928 epoch 53 - iter 12/25 - loss 3.37839397 - samples/sec: 127.93 - lr: 0.004687 2021-03-26 05:35:05,957 epoch 53 - iter 14/25 - loss 3.29179309 - samples/sec: 124.68 - lr: 0.004687 2021-03-26 05:35:06,908 epoch 53 - iter 16/25 - loss 3.22510788 - samples/sec: 134.90 - lr: 0.004687 2021-03-26 05:35:07,859 epoch 53 - iter 18/25 - loss 3.10731896 - samples/sec: 134.79 - lr: 0.004687 2021-03-26 05:35:08,904 epoch 53 - iter 20/25 - loss 3.15653430 - samples/sec: 122.68 - lr: 0.004687 2021-03-26 05:35:09,953 epoch 53 - iter 22/25 - loss 3.10727158 - samples/sec: 122.14 - lr: 0.004687 2021-03-26 05:35:10,992 epoch 53 - iter 24/25 - loss 3.12946189 - samples/sec: 123.56 - lr: 0.004687 2021-03-26 05:35:11,388 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:35:11,389 EPOCH 53 done: loss 3.1814 - lr 0.0046875 2021-03-26 05:35:12,227 DEV : loss 6.365192413330078 - score 0.9063 2021-03-26 05:35:12,253 BAD EPOCHS (no improvement): 1 2021-03-26 05:35:12,254 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:35:13,283 epoch 54 - iter 2/25 - loss 3.76396322 - samples/sec: 124.63 - lr: 0.004687 2021-03-26 05:35:14,337 epoch 54 - iter 4/25 - loss 3.37461126 - samples/sec: 121.51 - lr: 0.004687 2021-03-26 05:35:15,308 epoch 54 - iter 6/25 - loss 3.34941999 - samples/sec: 132.10 - lr: 0.004687 2021-03-26 05:35:16,349 epoch 54 - iter 8/25 - loss 3.31672096 - samples/sec: 123.21 - lr: 0.004687 2021-03-26 05:35:17,361 epoch 54 - iter 10/25 - loss 3.20109630 - samples/sec: 126.83 - lr: 0.004687 2021-03-26 05:35:18,394 epoch 54 - iter 12/25 - loss 3.12487588 - samples/sec: 124.05 - lr: 0.004687 2021-03-26 05:35:19,355 epoch 54 - iter 14/25 - loss 3.03142987 - samples/sec: 133.38 - lr: 0.004687 2021-03-26 05:35:20,262 epoch 54 - iter 16/25 - loss 3.00063097 - samples/sec: 141.34 - lr: 0.004687 2021-03-26 05:35:21,245 epoch 54 - iter 18/25 - loss 2.98818886 - samples/sec: 130.38 - lr: 0.004687 2021-03-26 05:35:22,153 epoch 54 - iter 20/25 - loss 2.98626262 - samples/sec: 141.15 - lr: 0.004687 2021-03-26 05:35:23,284 epoch 54 - iter 22/25 - loss 3.06153673 - samples/sec: 113.38 - lr: 0.004687 2021-03-26 05:35:24,313 epoch 54 - iter 24/25 - loss 3.05973284 - samples/sec: 124.95 - lr: 0.004687 2021-03-26 05:35:24,650 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:35:24,651 EPOCH 54 done: loss 3.0402 - lr 0.0046875 2021-03-26 05:35:25,433 DEV : loss 6.367043972015381 - score 0.9083 2021-03-26 05:35:25,458 BAD EPOCHS (no improvement): 2 2021-03-26 05:35:25,458 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:35:26,443 epoch 55 - iter 2/25 - loss 3.18132973 - samples/sec: 130.12 - lr: 0.004687 2021-03-26 05:35:27,402 epoch 55 - iter 4/25 - loss 3.01191485 - samples/sec: 133.68 - lr: 0.004687 2021-03-26 05:35:28,349 epoch 55 - iter 6/25 - loss 3.20665197 - samples/sec: 135.35 - lr: 0.004687 2021-03-26 05:35:29,357 epoch 55 - iter 8/25 - loss 3.06920093 - samples/sec: 127.17 - lr: 0.004687 2021-03-26 05:35:30,308 epoch 55 - iter 10/25 - loss 3.01357651 - samples/sec: 134.84 - lr: 0.004687 2021-03-26 05:35:31,295 epoch 55 - iter 12/25 - loss 2.96591622 - samples/sec: 130.00 - lr: 0.004687 2021-03-26 05:35:32,279 epoch 55 - iter 14/25 - loss 3.02729602 - samples/sec: 130.45 - lr: 0.004687 2021-03-26 05:35:33,331 epoch 55 - iter 16/25 - loss 3.05965899 - samples/sec: 121.77 - lr: 0.004687 2021-03-26 05:35:34,362 epoch 55 - iter 18/25 - loss 3.08807895 - samples/sec: 124.51 - lr: 0.004687 2021-03-26 05:35:35,329 epoch 55 - iter 20/25 - loss 3.05238564 - samples/sec: 132.51 - lr: 0.004687 2021-03-26 05:35:36,240 epoch 55 - iter 22/25 - loss 3.09957942 - samples/sec: 140.64 - lr: 0.004687 2021-03-26 05:35:37,262 epoch 55 - iter 24/25 - loss 3.08548250 - samples/sec: 125.64 - lr: 0.004687 2021-03-26 05:35:37,675 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:35:37,676 EPOCH 55 done: loss 3.0630 - lr 0.0046875 2021-03-26 05:35:38,494 DEV : loss 6.367519378662109 - score 0.9083 2021-03-26 05:35:38,519 BAD EPOCHS (no improvement): 3 2021-03-26 05:35:38,520 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:35:40,903 epoch 56 - iter 2/25 - loss 3.04370713 - samples/sec: 53.74 - lr: 0.004687 2021-03-26 05:35:41,945 epoch 56 - iter 4/25 - loss 2.89844847 - samples/sec: 123.12 - lr: 0.004687 2021-03-26 05:35:42,927 epoch 56 - iter 6/25 - loss 3.06276659 - samples/sec: 130.57 - lr: 0.004687 2021-03-26 05:35:43,962 epoch 56 - iter 8/25 - loss 2.99346858 - samples/sec: 123.83 - lr: 0.004687 2021-03-26 05:35:45,041 epoch 56 - iter 10/25 - loss 3.06103053 - samples/sec: 118.82 - lr: 0.004687 2021-03-26 05:35:46,109 epoch 56 - iter 12/25 - loss 3.04417761 - samples/sec: 119.97 - lr: 0.004687 2021-03-26 05:35:47,080 epoch 56 - iter 14/25 - loss 2.95337142 - samples/sec: 132.06 - lr: 0.004687 2021-03-26 05:35:48,062 epoch 56 - iter 16/25 - loss 2.97934601 - samples/sec: 130.64 - lr: 0.004687 2021-03-26 05:35:49,057 epoch 56 - iter 18/25 - loss 3.03061761 - samples/sec: 128.70 - lr: 0.004687 2021-03-26 05:35:50,112 epoch 56 - iter 20/25 - loss 2.99428663 - samples/sec: 121.59 - lr: 0.004687 2021-03-26 05:35:51,098 epoch 56 - iter 22/25 - loss 2.97807759 - samples/sec: 129.87 - lr: 0.004687 2021-03-26 05:35:52,007 epoch 56 - iter 24/25 - loss 3.01112747 - samples/sec: 141.03 - lr: 0.004687 2021-03-26 05:35:52,426 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:35:52,426 EPOCH 56 done: loss 3.0223 - lr 0.0046875 2021-03-26 05:35:53,229 DEV : loss 6.379022121429443 - score 0.9079 2021-03-26 05:35:53,255 BAD EPOCHS (no improvement): 4 2021-03-26 05:35:53,256 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:35:54,324 epoch 57 - iter 2/25 - loss 2.88998890 - samples/sec: 119.97 - lr: 0.002344 2021-03-26 05:35:55,245 epoch 57 - iter 4/25 - loss 2.84257925 - samples/sec: 139.19 - lr: 0.002344 2021-03-26 05:35:56,317 epoch 57 - iter 6/25 - loss 2.82792898 - samples/sec: 119.56 - lr: 0.002344 2021-03-26 05:35:57,254 epoch 57 - iter 8/25 - loss 2.98162171 - samples/sec: 136.75 - lr: 0.002344 2021-03-26 05:35:58,260 epoch 57 - iter 10/25 - loss 2.88686635 - samples/sec: 127.45 - lr: 0.002344 2021-03-26 05:35:59,248 epoch 57 - iter 12/25 - loss 2.93057438 - samples/sec: 129.76 - lr: 0.002344 2021-03-26 05:36:00,294 epoch 57 - iter 14/25 - loss 3.00041650 - samples/sec: 122.51 - lr: 0.002344 2021-03-26 05:36:01,305 epoch 57 - iter 16/25 - loss 2.98985383 - samples/sec: 126.77 - lr: 0.002344 2021-03-26 05:36:02,389 epoch 57 - iter 18/25 - loss 3.00885790 - samples/sec: 118.16 - lr: 0.002344 2021-03-26 05:36:03,359 epoch 57 - iter 20/25 - loss 3.08822603 - samples/sec: 132.10 - lr: 0.002344 2021-03-26 05:36:04,328 epoch 57 - iter 22/25 - loss 3.06954510 - samples/sec: 132.25 - lr: 0.002344 2021-03-26 05:36:05,306 epoch 57 - iter 24/25 - loss 3.01040073 - samples/sec: 131.05 - lr: 0.002344 2021-03-26 05:36:05,745 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:36:05,746 EPOCH 57 done: loss 3.0233 - lr 0.0023437 2021-03-26 05:36:06,549 DEV : loss 6.381903648376465 - score 0.9079 2021-03-26 05:36:06,568 BAD EPOCHS (no improvement): 1 2021-03-26 05:36:06,569 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:36:07,556 epoch 58 - iter 2/25 - loss 3.42313194 - samples/sec: 129.79 - lr: 0.002344 2021-03-26 05:36:08,494 epoch 58 - iter 4/25 - loss 2.94785899 - samples/sec: 136.95 - lr: 0.002344 2021-03-26 05:36:09,451 epoch 58 - iter 6/25 - loss 3.20075154 - samples/sec: 133.97 - lr: 0.002344 2021-03-26 05:36:10,430 epoch 58 - iter 8/25 - loss 3.01168898 - samples/sec: 130.85 - lr: 0.002344 2021-03-26 05:36:11,344 epoch 58 - iter 10/25 - loss 3.00945287 - samples/sec: 140.48 - lr: 0.002344 2021-03-26 05:36:12,317 epoch 58 - iter 12/25 - loss 2.96118955 - samples/sec: 131.81 - lr: 0.002344 2021-03-26 05:36:13,387 epoch 58 - iter 14/25 - loss 3.06854313 - samples/sec: 119.75 - lr: 0.002344 2021-03-26 05:36:14,458 epoch 58 - iter 16/25 - loss 3.09105952 - samples/sec: 119.70 - lr: 0.002344 2021-03-26 05:36:15,444 epoch 58 - iter 18/25 - loss 3.07459023 - samples/sec: 129.99 - lr: 0.002344 2021-03-26 05:36:16,498 epoch 58 - iter 20/25 - loss 3.08885949 - samples/sec: 121.59 - lr: 0.002344 2021-03-26 05:36:17,478 epoch 58 - iter 22/25 - loss 3.01594584 - samples/sec: 130.82 - lr: 0.002344 2021-03-26 05:36:18,463 epoch 58 - iter 24/25 - loss 3.01048505 - samples/sec: 130.16 - lr: 0.002344 2021-03-26 05:36:18,853 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:36:18,854 EPOCH 58 done: loss 3.0107 - lr 0.0023437 2021-03-26 05:36:19,644 DEV : loss 6.3849005699157715 - score 0.9075 2021-03-26 05:36:19,665 BAD EPOCHS (no improvement): 2 2021-03-26 05:36:19,665 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:36:20,693 epoch 59 - iter 2/25 - loss 3.14258122 - samples/sec: 124.68 - lr: 0.002344 2021-03-26 05:36:21,687 epoch 59 - iter 4/25 - loss 3.02675045 - samples/sec: 128.98 - lr: 0.002344 2021-03-26 05:36:22,686 epoch 59 - iter 6/25 - loss 3.07165686 - samples/sec: 128.31 - lr: 0.002344 2021-03-26 05:36:23,795 epoch 59 - iter 8/25 - loss 3.22081539 - samples/sec: 115.57 - lr: 0.002344 2021-03-26 05:36:24,772 epoch 59 - iter 10/25 - loss 3.13599381 - samples/sec: 131.19 - lr: 0.002344 2021-03-26 05:36:25,738 epoch 59 - iter 12/25 - loss 3.14673634 - samples/sec: 132.74 - lr: 0.002344 2021-03-26 05:36:26,705 epoch 59 - iter 14/25 - loss 3.11754748 - samples/sec: 132.56 - lr: 0.002344 2021-03-26 05:36:27,768 epoch 59 - iter 16/25 - loss 3.15714070 - samples/sec: 120.55 - lr: 0.002344 2021-03-26 05:36:28,789 epoch 59 - iter 18/25 - loss 3.15854804 - samples/sec: 125.54 - lr: 0.002344 2021-03-26 05:36:29,748 epoch 59 - iter 20/25 - loss 3.11217291 - samples/sec: 133.71 - lr: 0.002344 2021-03-26 05:36:30,762 epoch 59 - iter 22/25 - loss 3.09361850 - samples/sec: 126.35 - lr: 0.002344 2021-03-26 05:36:31,769 epoch 59 - iter 24/25 - loss 3.06539866 - samples/sec: 127.36 - lr: 0.002344 2021-03-26 05:36:32,160 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:36:32,161 EPOCH 59 done: loss 3.0495 - lr 0.0023437 2021-03-26 05:36:32,961 DEV : loss 6.381946563720703 - score 0.9071 2021-03-26 05:36:32,981 BAD EPOCHS (no improvement): 3 2021-03-26 05:36:32,982 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:36:33,952 epoch 60 - iter 2/25 - loss 3.10077047 - samples/sec: 132.13 - lr: 0.002344 2021-03-26 05:36:34,962 epoch 60 - iter 4/25 - loss 2.88989782 - samples/sec: 126.93 - lr: 0.002344 2021-03-26 05:36:35,873 epoch 60 - iter 6/25 - loss 2.90555080 - samples/sec: 140.67 - lr: 0.002344 2021-03-26 05:36:36,838 epoch 60 - iter 8/25 - loss 2.95553589 - samples/sec: 132.88 - lr: 0.002344 2021-03-26 05:36:37,849 epoch 60 - iter 10/25 - loss 2.89822590 - samples/sec: 126.71 - lr: 0.002344 2021-03-26 05:36:38,844 epoch 60 - iter 12/25 - loss 2.84939009 - samples/sec: 128.89 - lr: 0.002344 2021-03-26 05:36:39,908 epoch 60 - iter 14/25 - loss 2.96647346 - samples/sec: 120.48 - lr: 0.002344 2021-03-26 05:36:40,890 epoch 60 - iter 16/25 - loss 2.92440678 - samples/sec: 130.50 - lr: 0.002344 2021-03-26 05:36:41,859 epoch 60 - iter 18/25 - loss 2.91018026 - samples/sec: 132.35 - lr: 0.002344 2021-03-26 05:36:42,913 epoch 60 - iter 20/25 - loss 2.87115874 - samples/sec: 121.70 - lr: 0.002344 2021-03-26 05:36:43,925 epoch 60 - iter 22/25 - loss 2.86456972 - samples/sec: 126.67 - lr: 0.002344 2021-03-26 05:36:44,911 epoch 60 - iter 24/25 - loss 2.85872015 - samples/sec: 130.03 - lr: 0.002344 2021-03-26 05:36:45,329 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:36:45,331 EPOCH 60 done: loss 2.8299 - lr 0.0023437 2021-03-26 05:36:46,121 DEV : loss 6.377230644226074 - score 0.9075 2021-03-26 05:36:46,148 BAD EPOCHS (no improvement): 4 2021-03-26 05:36:46,149 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:36:47,118 epoch 61 - iter 2/25 - loss 2.99023068 - samples/sec: 132.24 - lr: 0.001172 2021-03-26 05:36:48,077 epoch 61 - iter 4/25 - loss 3.16693401 - samples/sec: 133.73 - lr: 0.001172 2021-03-26 05:36:49,123 epoch 61 - iter 6/25 - loss 3.01899978 - samples/sec: 122.59 - lr: 0.001172 2021-03-26 05:36:50,130 epoch 61 - iter 8/25 - loss 2.93510966 - samples/sec: 127.22 - lr: 0.001172 2021-03-26 05:36:51,117 epoch 61 - iter 10/25 - loss 2.98047098 - samples/sec: 129.84 - lr: 0.001172 2021-03-26 05:36:52,018 epoch 61 - iter 12/25 - loss 2.97246026 - samples/sec: 142.56 - lr: 0.001172 2021-03-26 05:36:52,953 epoch 61 - iter 14/25 - loss 2.95344056 - samples/sec: 137.22 - lr: 0.001172 2021-03-26 05:36:53,988 epoch 61 - iter 16/25 - loss 2.95693909 - samples/sec: 123.85 - lr: 0.001172 2021-03-26 05:36:54,997 epoch 61 - iter 18/25 - loss 2.91543525 - samples/sec: 127.00 - lr: 0.001172 2021-03-26 05:36:55,949 epoch 61 - iter 20/25 - loss 2.89163441 - samples/sec: 134.74 - lr: 0.001172 2021-03-26 05:36:56,862 epoch 61 - iter 22/25 - loss 2.89356511 - samples/sec: 140.45 - lr: 0.001172 2021-03-26 05:36:57,976 epoch 61 - iter 24/25 - loss 2.88996286 - samples/sec: 115.07 - lr: 0.001172 2021-03-26 05:36:58,341 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:36:58,343 EPOCH 61 done: loss 2.8845 - lr 0.0011719 2021-03-26 05:36:59,160 DEV : loss 6.3796706199646 - score 0.9059 2021-03-26 05:36:59,178 BAD EPOCHS (no improvement): 1 2021-03-26 05:36:59,179 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:37:00,132 epoch 62 - iter 2/25 - loss 3.14777243 - samples/sec: 134.59 - lr: 0.001172 2021-03-26 05:37:01,114 epoch 62 - iter 4/25 - loss 3.04239559 - samples/sec: 130.54 - lr: 0.001172 2021-03-26 05:37:02,109 epoch 62 - iter 6/25 - loss 2.89608514 - samples/sec: 128.83 - lr: 0.001172 2021-03-26 05:37:03,144 epoch 62 - iter 8/25 - loss 3.01286557 - samples/sec: 123.77 - lr: 0.001172 2021-03-26 05:37:04,403 epoch 62 - iter 10/25 - loss 3.09971406 - samples/sec: 101.85 - lr: 0.001172 2021-03-26 05:37:05,541 epoch 62 - iter 12/25 - loss 3.12393322 - samples/sec: 112.61 - lr: 0.001172 2021-03-26 05:37:06,497 epoch 62 - iter 14/25 - loss 3.08095258 - samples/sec: 134.34 - lr: 0.001172 2021-03-26 05:37:07,512 epoch 62 - iter 16/25 - loss 3.00411677 - samples/sec: 126.24 - lr: 0.001172 2021-03-26 05:37:08,586 epoch 62 - iter 18/25 - loss 2.98847085 - samples/sec: 119.31 - lr: 0.001172 2021-03-26 05:37:09,476 epoch 62 - iter 20/25 - loss 2.92107501 - samples/sec: 144.82 - lr: 0.001172 2021-03-26 05:37:10,445 epoch 62 - iter 22/25 - loss 2.91261944 - samples/sec: 132.26 - lr: 0.001172 2021-03-26 05:37:11,455 epoch 62 - iter 24/25 - loss 2.91488413 - samples/sec: 126.93 - lr: 0.001172 2021-03-26 05:37:11,925 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:37:11,925 EPOCH 62 done: loss 2.9002 - lr 0.0011719 2021-03-26 05:37:12,718 DEV : loss 6.378934383392334 - score 0.9071 2021-03-26 05:37:12,744 BAD EPOCHS (no improvement): 2 2021-03-26 05:37:12,745 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:37:13,690 epoch 63 - iter 2/25 - loss 3.09829128 - samples/sec: 135.69 - lr: 0.001172 2021-03-26 05:37:14,650 epoch 63 - iter 4/25 - loss 3.14043283 - samples/sec: 133.44 - lr: 0.001172 2021-03-26 05:37:15,746 epoch 63 - iter 6/25 - loss 3.22749770 - samples/sec: 117.13 - lr: 0.001172 2021-03-26 05:37:16,890 epoch 63 - iter 8/25 - loss 3.31378135 - samples/sec: 111.97 - lr: 0.001172 2021-03-26 05:37:17,947 epoch 63 - iter 10/25 - loss 3.31359937 - samples/sec: 121.74 - lr: 0.001172 2021-03-26 05:37:19,012 epoch 63 - iter 12/25 - loss 3.16441872 - samples/sec: 120.37 - lr: 0.001172 2021-03-26 05:37:20,024 epoch 63 - iter 14/25 - loss 3.16780548 - samples/sec: 126.63 - lr: 0.001172 2021-03-26 05:37:20,964 epoch 63 - iter 16/25 - loss 3.11357681 - samples/sec: 136.39 - lr: 0.001172 2021-03-26 05:37:21,908 epoch 63 - iter 18/25 - loss 3.13789663 - samples/sec: 135.82 - lr: 0.001172 2021-03-26 05:37:22,882 epoch 63 - iter 20/25 - loss 3.10503579 - samples/sec: 131.60 - lr: 0.001172 2021-03-26 05:37:23,869 epoch 63 - iter 22/25 - loss 3.10525139 - samples/sec: 129.83 - lr: 0.001172 2021-03-26 05:37:24,891 epoch 63 - iter 24/25 - loss 3.08538382 - samples/sec: 125.54 - lr: 0.001172 2021-03-26 05:37:25,262 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:37:25,263 EPOCH 63 done: loss 3.0456 - lr 0.0011719 2021-03-26 05:37:26,081 DEV : loss 6.38121223449707 - score 0.9067 2021-03-26 05:37:26,111 BAD EPOCHS (no improvement): 3 2021-03-26 05:37:26,112 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:37:27,011 epoch 64 - iter 2/25 - loss 2.78713417 - samples/sec: 142.63 - lr: 0.001172 2021-03-26 05:37:28,020 epoch 64 - iter 4/25 - loss 3.04669195 - samples/sec: 126.99 - lr: 0.001172 2021-03-26 05:37:29,108 epoch 64 - iter 6/25 - loss 3.15520930 - samples/sec: 117.81 - lr: 0.001172 2021-03-26 05:37:30,040 epoch 64 - iter 8/25 - loss 3.03692526 - samples/sec: 137.48 - lr: 0.001172 2021-03-26 05:37:31,237 epoch 64 - iter 10/25 - loss 3.20279713 - samples/sec: 107.03 - lr: 0.001172 2021-03-26 05:37:32,351 epoch 64 - iter 12/25 - loss 3.16470673 - samples/sec: 115.08 - lr: 0.001172 2021-03-26 05:37:33,328 epoch 64 - iter 14/25 - loss 3.18147813 - samples/sec: 131.12 - lr: 0.001172 2021-03-26 05:37:34,319 epoch 64 - iter 16/25 - loss 3.21489021 - samples/sec: 129.42 - lr: 0.001172 2021-03-26 05:37:35,387 epoch 64 - iter 18/25 - loss 3.15895859 - samples/sec: 120.14 - lr: 0.001172 2021-03-26 05:37:36,331 epoch 64 - iter 20/25 - loss 3.12339705 - samples/sec: 135.77 - lr: 0.001172 2021-03-26 05:37:37,332 epoch 64 - iter 22/25 - loss 3.08132965 - samples/sec: 127.99 - lr: 0.001172 2021-03-26 05:37:38,330 epoch 64 - iter 24/25 - loss 3.05778714 - samples/sec: 128.62 - lr: 0.001172 2021-03-26 05:37:38,785 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:37:38,785 EPOCH 64 done: loss 3.0483 - lr 0.0011719 2021-03-26 05:37:39,569 DEV : loss 6.384737968444824 - score 0.9071 2021-03-26 05:37:39,594 BAD EPOCHS (no improvement): 4 2021-03-26 05:37:39,595 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:37:40,592 epoch 65 - iter 2/25 - loss 2.51159430 - samples/sec: 128.67 - lr: 0.000586 2021-03-26 05:37:41,546 epoch 65 - iter 4/25 - loss 2.72224629 - samples/sec: 134.31 - lr: 0.000586 2021-03-26 05:37:42,488 epoch 65 - iter 6/25 - loss 2.85944200 - samples/sec: 136.11 - lr: 0.000586 2021-03-26 05:37:43,417 epoch 65 - iter 8/25 - loss 2.79438365 - samples/sec: 138.13 - lr: 0.000586 2021-03-26 05:37:44,388 epoch 65 - iter 10/25 - loss 2.92464945 - samples/sec: 131.98 - lr: 0.000586 2021-03-26 05:37:45,512 epoch 65 - iter 12/25 - loss 2.93784885 - samples/sec: 114.03 - lr: 0.000586 2021-03-26 05:37:46,508 epoch 65 - iter 14/25 - loss 2.94183019 - samples/sec: 128.66 - lr: 0.000586 2021-03-26 05:37:47,508 epoch 65 - iter 16/25 - loss 2.93601930 - samples/sec: 128.24 - lr: 0.000586 2021-03-26 05:37:48,508 epoch 65 - iter 18/25 - loss 2.96305468 - samples/sec: 128.09 - lr: 0.000586 2021-03-26 05:37:49,589 epoch 65 - iter 20/25 - loss 2.99386072 - samples/sec: 118.71 - lr: 0.000586 2021-03-26 05:37:50,851 epoch 65 - iter 22/25 - loss 2.97353937 - samples/sec: 101.97 - lr: 0.000586 2021-03-26 05:37:51,905 epoch 65 - iter 24/25 - loss 2.96730961 - samples/sec: 121.63 - lr: 0.000586 2021-03-26 05:37:52,346 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:37:52,346 EPOCH 65 done: loss 2.9653 - lr 0.0005859 2021-03-26 05:37:53,121 DEV : loss 6.384479522705078 - score 0.9071 2021-03-26 05:37:53,147 BAD EPOCHS (no improvement): 1 2021-03-26 05:37:53,148 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:37:54,108 epoch 66 - iter 2/25 - loss 2.86535895 - samples/sec: 133.54 - lr: 0.000586 2021-03-26 05:37:55,153 epoch 66 - iter 4/25 - loss 3.12844360 - samples/sec: 122.85 - lr: 0.000586 2021-03-26 05:37:56,028 epoch 66 - iter 6/25 - loss 3.19749145 - samples/sec: 146.66 - lr: 0.000586 2021-03-26 05:37:57,019 epoch 66 - iter 8/25 - loss 3.20166433 - samples/sec: 129.42 - lr: 0.000586 2021-03-26 05:37:58,045 epoch 66 - iter 10/25 - loss 3.20313184 - samples/sec: 124.89 - lr: 0.000586 2021-03-26 05:37:59,123 epoch 66 - iter 12/25 - loss 3.24322170 - samples/sec: 118.85 - lr: 0.000586 2021-03-26 05:38:00,118 epoch 66 - iter 14/25 - loss 3.24902667 - samples/sec: 128.97 - lr: 0.000586 2021-03-26 05:38:01,092 epoch 66 - iter 16/25 - loss 3.24626854 - samples/sec: 131.65 - lr: 0.000586 2021-03-26 05:38:02,049 epoch 66 - iter 18/25 - loss 3.15644164 - samples/sec: 134.00 - lr: 0.000586 2021-03-26 05:38:02,984 epoch 66 - iter 20/25 - loss 3.15955758 - samples/sec: 137.25 - lr: 0.000586 2021-03-26 05:38:03,937 epoch 66 - iter 22/25 - loss 3.14094888 - samples/sec: 134.56 - lr: 0.000586 2021-03-26 05:38:04,959 epoch 66 - iter 24/25 - loss 3.12368099 - samples/sec: 125.60 - lr: 0.000586 2021-03-26 05:38:05,393 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:38:05,394 EPOCH 66 done: loss 3.1324 - lr 0.0005859 2021-03-26 05:38:06,193 DEV : loss 6.383347988128662 - score 0.9071 2021-03-26 05:38:06,219 BAD EPOCHS (no improvement): 2 2021-03-26 05:38:06,220 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:38:07,222 epoch 67 - iter 2/25 - loss 2.55367327 - samples/sec: 127.93 - lr: 0.000586 2021-03-26 05:38:08,192 epoch 67 - iter 4/25 - loss 2.99387980 - samples/sec: 132.23 - lr: 0.000586 2021-03-26 05:38:09,163 epoch 67 - iter 6/25 - loss 2.84334783 - samples/sec: 132.02 - lr: 0.000586 2021-03-26 05:38:10,224 epoch 67 - iter 8/25 - loss 2.86086762 - samples/sec: 120.83 - lr: 0.000586 2021-03-26 05:38:11,237 epoch 67 - iter 10/25 - loss 2.88043385 - samples/sec: 126.68 - lr: 0.000586 2021-03-26 05:38:12,189 epoch 67 - iter 12/25 - loss 2.88910687 - samples/sec: 134.72 - lr: 0.000586 2021-03-26 05:38:13,192 epoch 67 - iter 14/25 - loss 2.86610917 - samples/sec: 127.81 - lr: 0.000586 2021-03-26 05:38:14,202 epoch 67 - iter 16/25 - loss 2.89737588 - samples/sec: 126.87 - lr: 0.000586 2021-03-26 05:38:15,159 epoch 67 - iter 18/25 - loss 2.94335104 - samples/sec: 133.91 - lr: 0.000586 2021-03-26 05:38:16,344 epoch 67 - iter 20/25 - loss 2.94453681 - samples/sec: 108.16 - lr: 0.000586 2021-03-26 05:38:17,347 epoch 67 - iter 22/25 - loss 2.95571147 - samples/sec: 127.73 - lr: 0.000586 2021-03-26 05:38:18,284 epoch 67 - iter 24/25 - loss 2.99705405 - samples/sec: 136.80 - lr: 0.000586 2021-03-26 05:38:18,689 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:38:18,690 EPOCH 67 done: loss 3.0371 - lr 0.0005859 2021-03-26 05:38:19,480 DEV : loss 6.382637977600098 - score 0.9071 2021-03-26 05:38:19,506 BAD EPOCHS (no improvement): 3 2021-03-26 05:38:19,507 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:38:20,492 epoch 68 - iter 2/25 - loss 3.10730994 - samples/sec: 130.21 - lr: 0.000586 2021-03-26 05:38:21,431 epoch 68 - iter 4/25 - loss 2.95590889 - samples/sec: 136.60 - lr: 0.000586 2021-03-26 05:38:22,477 epoch 68 - iter 6/25 - loss 3.17937148 - samples/sec: 122.65 - lr: 0.000586 2021-03-26 05:38:23,406 epoch 68 - iter 8/25 - loss 3.15134969 - samples/sec: 137.97 - lr: 0.000586 2021-03-26 05:38:24,325 epoch 68 - iter 10/25 - loss 3.12095699 - samples/sec: 139.40 - lr: 0.000586 2021-03-26 05:38:25,360 epoch 68 - iter 12/25 - loss 3.15177401 - samples/sec: 123.89 - lr: 0.000586 2021-03-26 05:38:26,362 epoch 68 - iter 14/25 - loss 3.06146678 - samples/sec: 127.90 - lr: 0.000586 2021-03-26 05:38:27,372 epoch 68 - iter 16/25 - loss 3.05155736 - samples/sec: 126.82 - lr: 0.000586 2021-03-26 05:38:28,280 epoch 68 - iter 18/25 - loss 3.01436926 - samples/sec: 141.24 - lr: 0.000586 2021-03-26 05:38:29,272 epoch 68 - iter 20/25 - loss 2.95574481 - samples/sec: 129.20 - lr: 0.000586 2021-03-26 05:38:30,317 epoch 68 - iter 22/25 - loss 2.91949245 - samples/sec: 122.69 - lr: 0.000586 2021-03-26 05:38:31,374 epoch 68 - iter 24/25 - loss 2.93033016 - samples/sec: 121.20 - lr: 0.000586 2021-03-26 05:38:31,740 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:38:31,740 EPOCH 68 done: loss 2.9355 - lr 0.0005859 2021-03-26 05:38:32,512 DEV : loss 6.382974147796631 - score 0.9075 2021-03-26 05:38:32,538 BAD EPOCHS (no improvement): 4 2021-03-26 05:38:32,538 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:38:33,505 epoch 69 - iter 2/25 - loss 3.36692905 - samples/sec: 132.58 - lr: 0.000293 2021-03-26 05:38:34,582 epoch 69 - iter 4/25 - loss 3.05667657 - samples/sec: 119.03 - lr: 0.000293 2021-03-26 05:38:35,605 epoch 69 - iter 6/25 - loss 2.85133298 - samples/sec: 125.44 - lr: 0.000293 2021-03-26 05:38:36,648 epoch 69 - iter 8/25 - loss 3.03246859 - samples/sec: 122.90 - lr: 0.000293 2021-03-26 05:38:37,639 epoch 69 - iter 10/25 - loss 3.05118198 - samples/sec: 129.33 - lr: 0.000293 2021-03-26 05:38:38,621 epoch 69 - iter 12/25 - loss 3.07936551 - samples/sec: 130.59 - lr: 0.000293 2021-03-26 05:38:39,694 epoch 69 - iter 14/25 - loss 3.08012353 - samples/sec: 119.44 - lr: 0.000293 2021-03-26 05:38:40,723 epoch 69 - iter 16/25 - loss 3.05436873 - samples/sec: 124.73 - lr: 0.000293 2021-03-26 05:38:41,691 epoch 69 - iter 18/25 - loss 3.02919669 - samples/sec: 132.74 - lr: 0.000293 2021-03-26 05:38:42,782 epoch 69 - iter 20/25 - loss 3.06230100 - samples/sec: 117.41 - lr: 0.000293 2021-03-26 05:38:43,804 epoch 69 - iter 22/25 - loss 3.05611721 - samples/sec: 125.67 - lr: 0.000293 2021-03-26 05:38:44,805 epoch 69 - iter 24/25 - loss 3.02325180 - samples/sec: 128.11 - lr: 0.000293 2021-03-26 05:38:45,197 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:38:45,198 EPOCH 69 done: loss 2.9970 - lr 0.0002930 2021-03-26 05:38:46,013 DEV : loss 6.38275671005249 - score 0.9079 2021-03-26 05:38:46,039 BAD EPOCHS (no improvement): 1 2021-03-26 05:38:46,040 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:38:47,094 epoch 70 - iter 2/25 - loss 3.02799535 - samples/sec: 121.76 - lr: 0.000293 2021-03-26 05:38:48,094 epoch 70 - iter 4/25 - loss 2.87969232 - samples/sec: 128.27 - lr: 0.000293 2021-03-26 05:38:49,006 epoch 70 - iter 6/25 - loss 2.79067183 - samples/sec: 140.62 - lr: 0.000293 2021-03-26 05:38:49,987 epoch 70 - iter 8/25 - loss 2.86862090 - samples/sec: 130.89 - lr: 0.000293 2021-03-26 05:38:50,945 epoch 70 - iter 10/25 - loss 2.88230951 - samples/sec: 133.74 - lr: 0.000293 2021-03-26 05:38:51,994 epoch 70 - iter 12/25 - loss 2.93184088 - samples/sec: 122.29 - lr: 0.000293 2021-03-26 05:38:52,990 epoch 70 - iter 14/25 - loss 2.92722591 - samples/sec: 128.83 - lr: 0.000293 2021-03-26 05:38:53,917 epoch 70 - iter 16/25 - loss 2.90687205 - samples/sec: 138.33 - lr: 0.000293 2021-03-26 05:38:54,874 epoch 70 - iter 18/25 - loss 2.88737437 - samples/sec: 133.88 - lr: 0.000293 2021-03-26 05:38:55,812 epoch 70 - iter 20/25 - loss 2.90604259 - samples/sec: 136.69 - lr: 0.000293 2021-03-26 05:38:56,785 epoch 70 - iter 22/25 - loss 2.91495455 - samples/sec: 131.71 - lr: 0.000293 2021-03-26 05:38:57,872 epoch 70 - iter 24/25 - loss 2.93928660 - samples/sec: 117.95 - lr: 0.000293 2021-03-26 05:38:58,335 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:38:58,336 EPOCH 70 done: loss 2.9512 - lr 0.0002930 2021-03-26 05:38:59,159 DEV : loss 6.38282585144043 - score 0.9075 2021-03-26 05:38:59,179 BAD EPOCHS (no improvement): 2 2021-03-26 05:38:59,179 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:39:00,199 epoch 71 - iter 2/25 - loss 3.33338666 - samples/sec: 125.78 - lr: 0.000293 2021-03-26 05:39:01,226 epoch 71 - iter 4/25 - loss 3.29438347 - samples/sec: 125.03 - lr: 0.000293 2021-03-26 05:39:02,194 epoch 71 - iter 6/25 - loss 3.51837246 - samples/sec: 132.36 - lr: 0.000293 2021-03-26 05:39:03,224 epoch 71 - iter 8/25 - loss 3.15652186 - samples/sec: 124.46 - lr: 0.000293 2021-03-26 05:39:04,315 epoch 71 - iter 10/25 - loss 3.04163837 - samples/sec: 117.53 - lr: 0.000293 2021-03-26 05:39:05,358 epoch 71 - iter 12/25 - loss 3.07537399 - samples/sec: 123.09 - lr: 0.000293 2021-03-26 05:39:06,447 epoch 71 - iter 14/25 - loss 3.09981923 - samples/sec: 117.62 - lr: 0.000293 2021-03-26 05:39:07,478 epoch 71 - iter 16/25 - loss 3.06394473 - samples/sec: 124.39 - lr: 0.000293 2021-03-26 05:39:08,442 epoch 71 - iter 18/25 - loss 2.99298355 - samples/sec: 132.98 - lr: 0.000293 2021-03-26 05:39:09,463 epoch 71 - iter 20/25 - loss 3.02383423 - samples/sec: 125.50 - lr: 0.000293 2021-03-26 05:39:10,448 epoch 71 - iter 22/25 - loss 2.97648482 - samples/sec: 130.27 - lr: 0.000293 2021-03-26 05:39:11,363 epoch 71 - iter 24/25 - loss 2.90388579 - samples/sec: 140.01 - lr: 0.000293 2021-03-26 05:39:11,716 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:39:11,718 EPOCH 71 done: loss 2.8741 - lr 0.0002930 2021-03-26 05:39:12,534 DEV : loss 6.382222652435303 - score 0.9079 2021-03-26 05:39:12,560 BAD EPOCHS (no improvement): 3 2021-03-26 05:39:12,560 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:39:13,492 epoch 72 - iter 2/25 - loss 2.51642132 - samples/sec: 137.58 - lr: 0.000293 2021-03-26 05:39:14,583 epoch 72 - iter 4/25 - loss 2.58781946 - samples/sec: 117.48 - lr: 0.000293 2021-03-26 05:39:15,685 epoch 72 - iter 6/25 - loss 2.80074112 - samples/sec: 116.31 - lr: 0.000293 2021-03-26 05:39:16,671 epoch 72 - iter 8/25 - loss 2.86493906 - samples/sec: 129.94 - lr: 0.000293 2021-03-26 05:39:17,638 epoch 72 - iter 10/25 - loss 2.81937277 - samples/sec: 132.72 - lr: 0.000293 2021-03-26 05:39:18,640 epoch 72 - iter 12/25 - loss 2.77459627 - samples/sec: 128.01 - lr: 0.000293 2021-03-26 05:39:19,736 epoch 72 - iter 14/25 - loss 2.78194279 - samples/sec: 116.93 - lr: 0.000293 2021-03-26 05:39:20,696 epoch 72 - iter 16/25 - loss 2.89242551 - samples/sec: 133.58 - lr: 0.000293 2021-03-26 05:39:21,722 epoch 72 - iter 18/25 - loss 2.91030233 - samples/sec: 124.85 - lr: 0.000293 2021-03-26 05:39:22,669 epoch 72 - iter 20/25 - loss 2.86490841 - samples/sec: 135.49 - lr: 0.000293 2021-03-26 05:39:23,705 epoch 72 - iter 22/25 - loss 2.91277636 - samples/sec: 123.73 - lr: 0.000293 2021-03-26 05:39:24,667 epoch 72 - iter 24/25 - loss 2.93769855 - samples/sec: 133.35 - lr: 0.000293 2021-03-26 05:39:25,085 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:39:25,086 EPOCH 72 done: loss 2.9390 - lr 0.0002930 2021-03-26 05:39:25,886 DEV : loss 6.382140159606934 - score 0.9079 2021-03-26 05:39:25,912 BAD EPOCHS (no improvement): 4 2021-03-26 05:39:25,913 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:39:26,871 epoch 73 - iter 2/25 - loss 3.32987905 - samples/sec: 133.92 - lr: 0.000146 2021-03-26 05:39:27,962 epoch 73 - iter 4/25 - loss 3.21548235 - samples/sec: 117.60 - lr: 0.000146 2021-03-26 05:39:29,020 epoch 73 - iter 6/25 - loss 3.13740551 - samples/sec: 121.14 - lr: 0.000146 2021-03-26 05:39:30,091 epoch 73 - iter 8/25 - loss 3.15512255 - samples/sec: 119.77 - lr: 0.000146 2021-03-26 05:39:31,171 epoch 73 - iter 10/25 - loss 3.23293400 - samples/sec: 118.88 - lr: 0.000146 2021-03-26 05:39:32,197 epoch 73 - iter 12/25 - loss 3.15044584 - samples/sec: 125.06 - lr: 0.000146 2021-03-26 05:39:33,139 epoch 73 - iter 14/25 - loss 3.13014708 - samples/sec: 136.04 - lr: 0.000146 2021-03-26 05:39:34,193 epoch 73 - iter 16/25 - loss 3.11161730 - samples/sec: 121.54 - lr: 0.000146 2021-03-26 05:39:35,187 epoch 73 - iter 18/25 - loss 3.10345639 - samples/sec: 129.02 - lr: 0.000146 2021-03-26 05:39:36,135 epoch 73 - iter 20/25 - loss 3.00746253 - samples/sec: 135.28 - lr: 0.000146 2021-03-26 05:39:37,089 epoch 73 - iter 22/25 - loss 3.05165423 - samples/sec: 134.50 - lr: 0.000146 2021-03-26 05:39:38,073 epoch 73 - iter 24/25 - loss 3.08920673 - samples/sec: 130.18 - lr: 0.000146 2021-03-26 05:39:38,504 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:39:38,505 EPOCH 73 done: loss 3.0432 - lr 0.0001465 2021-03-26 05:39:39,286 DEV : loss 6.382096290588379 - score 0.9079 2021-03-26 05:39:39,311 BAD EPOCHS (no improvement): 1 2021-03-26 05:39:39,312 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:39:40,292 epoch 74 - iter 2/25 - loss 2.67405450 - samples/sec: 130.92 - lr: 0.000146 2021-03-26 05:39:41,236 epoch 74 - iter 4/25 - loss 2.68950474 - samples/sec: 135.77 - lr: 0.000146 2021-03-26 05:39:42,206 epoch 74 - iter 6/25 - loss 2.83899264 - samples/sec: 132.15 - lr: 0.000146 2021-03-26 05:39:43,203 epoch 74 - iter 8/25 - loss 3.01522473 - samples/sec: 128.67 - lr: 0.000146 2021-03-26 05:39:44,201 epoch 74 - iter 10/25 - loss 3.02439599 - samples/sec: 128.58 - lr: 0.000146 2021-03-26 05:39:45,258 epoch 74 - iter 12/25 - loss 3.01054655 - samples/sec: 121.44 - lr: 0.000146 2021-03-26 05:39:46,218 epoch 74 - iter 14/25 - loss 2.94204186 - samples/sec: 133.48 - lr: 0.000146 2021-03-26 05:39:47,151 epoch 74 - iter 16/25 - loss 2.91268761 - samples/sec: 137.34 - lr: 0.000146 2021-03-26 05:39:48,101 epoch 74 - iter 18/25 - loss 2.94408124 - samples/sec: 135.23 - lr: 0.000146 2021-03-26 05:39:49,190 epoch 74 - iter 20/25 - loss 2.96560382 - samples/sec: 117.66 - lr: 0.000146 2021-03-26 05:39:50,123 epoch 74 - iter 22/25 - loss 3.01023171 - samples/sec: 137.41 - lr: 0.000146 2021-03-26 05:39:51,093 epoch 74 - iter 24/25 - loss 3.04150907 - samples/sec: 132.27 - lr: 0.000146 2021-03-26 05:39:51,529 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:39:51,531 EPOCH 74 done: loss 3.0545 - lr 0.0001465 2021-03-26 05:39:52,333 DEV : loss 6.38180685043335 - score 0.9079 2021-03-26 05:39:52,362 BAD EPOCHS (no improvement): 2 2021-03-26 05:39:52,362 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:39:53,381 epoch 75 - iter 2/25 - loss 2.16002882 - samples/sec: 125.80 - lr: 0.000146 2021-03-26 05:39:54,398 epoch 75 - iter 4/25 - loss 2.68587881 - samples/sec: 126.03 - lr: 0.000146 2021-03-26 05:39:55,475 epoch 75 - iter 6/25 - loss 2.60181061 - samples/sec: 119.08 - lr: 0.000146 2021-03-26 05:39:56,405 epoch 75 - iter 8/25 - loss 2.68238357 - samples/sec: 137.75 - lr: 0.000146 2021-03-26 05:39:57,382 epoch 75 - iter 10/25 - loss 2.71803734 - samples/sec: 131.21 - lr: 0.000146 2021-03-26 05:39:58,316 epoch 75 - iter 12/25 - loss 2.79610946 - samples/sec: 137.31 - lr: 0.000146 2021-03-26 05:39:59,316 epoch 75 - iter 14/25 - loss 2.81926460 - samples/sec: 128.11 - lr: 0.000146 2021-03-26 05:40:00,277 epoch 75 - iter 16/25 - loss 2.81095479 - samples/sec: 133.54 - lr: 0.000146 2021-03-26 05:40:01,370 epoch 75 - iter 18/25 - loss 2.88618902 - samples/sec: 117.26 - lr: 0.000146 2021-03-26 05:40:02,382 epoch 75 - iter 20/25 - loss 2.91740313 - samples/sec: 126.69 - lr: 0.000146 2021-03-26 05:40:03,315 epoch 75 - iter 22/25 - loss 2.92163591 - samples/sec: 137.48 - lr: 0.000146 2021-03-26 05:40:04,340 epoch 75 - iter 24/25 - loss 2.91205307 - samples/sec: 125.07 - lr: 0.000146 2021-03-26 05:40:04,764 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:04,765 EPOCH 75 done: loss 2.9227 - lr 0.0001465 2021-03-26 05:40:05,574 DEV : loss 6.382327079772949 - score 0.9079 2021-03-26 05:40:05,600 BAD EPOCHS (no improvement): 3 2021-03-26 05:40:05,601 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:06,600 epoch 76 - iter 2/25 - loss 3.22646475 - samples/sec: 128.42 - lr: 0.000146 2021-03-26 05:40:07,624 epoch 76 - iter 4/25 - loss 2.85521507 - samples/sec: 125.13 - lr: 0.000146 2021-03-26 05:40:08,773 epoch 76 - iter 6/25 - loss 2.90632153 - samples/sec: 111.54 - lr: 0.000146 2021-03-26 05:40:09,816 epoch 76 - iter 8/25 - loss 2.89671373 - samples/sec: 122.89 - lr: 0.000146 2021-03-26 05:40:10,797 epoch 76 - iter 10/25 - loss 2.87506793 - samples/sec: 130.77 - lr: 0.000146 2021-03-26 05:40:11,720 epoch 76 - iter 12/25 - loss 2.86794132 - samples/sec: 138.91 - lr: 0.000146 2021-03-26 05:40:12,841 epoch 76 - iter 14/25 - loss 2.85794585 - samples/sec: 114.38 - lr: 0.000146 2021-03-26 05:40:13,865 epoch 76 - iter 16/25 - loss 2.98309749 - samples/sec: 125.23 - lr: 0.000146 2021-03-26 05:40:14,763 epoch 76 - iter 18/25 - loss 2.96132292 - samples/sec: 142.73 - lr: 0.000146 2021-03-26 05:40:15,792 epoch 76 - iter 20/25 - loss 2.96512048 - samples/sec: 124.50 - lr: 0.000146 2021-03-26 05:40:16,785 epoch 76 - iter 22/25 - loss 2.92676656 - samples/sec: 129.11 - lr: 0.000146 2021-03-26 05:40:17,722 epoch 76 - iter 24/25 - loss 2.92380586 - samples/sec: 136.79 - lr: 0.000146 2021-03-26 05:40:18,106 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:18,107 EPOCH 76 done: loss 2.9335 - lr 0.0001465 2021-03-26 05:40:18,890 DEV : loss 6.382133483886719 - score 0.9079 2021-03-26 05:40:18,917 BAD EPOCHS (no improvement): 4 2021-03-26 05:40:18,917 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:18,918 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:18,918 learning rate too small - quitting training! 2021-03-26 05:40:18,918 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:28,321 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:28,321 Testing using best model ... 2021-03-26 05:40:28,322 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260519/best-model.pt 2021-03-26 05:40:35,778 0.9155 2021-03-26 05:40:35,779 Results: - F-score (micro): 0.9114 - F-score (macro): 0.5279 - Accuracy (incl. no class): 0.9155 By class: precision recall f1-score support NOUN 0.9394 0.9482 0.9438 425 NUM 1.0000 0.9091 0.9524 22 PRON 0.9856 0.9856 0.9856 209 VERB 0.9532 0.9477 0.9504 172 ADP 0.9444 0.9341 0.9392 91 CCONJ 0.9762 1.0000 0.9880 82 PROPN 0.9000 0.8182 0.8571 22 ADV 0.9310 0.8504 0.8889 127 DET 0.9574 0.9574 0.9574 47 PUNCT 1.0000 1.0000 1.0000 25 ADJ 0.8938 0.8783 0.8860 115 AUX 0.9111 0.9318 0.9213 44 PART 0.9242 0.9848 0.9535 198 SCONJ 0.9762 1.0000 0.9880 41 INTJ 0.8571 1.0000 0.9231 18 PREP 0.9531 0.9683 0.9606 63 NOUN+NSUFF 0.8644 0.8095 0.8361 63 V+PRON 0.8421 0.7619 0.8000 42 V+PRON+PRON 0.5000 0.3636 0.4211 11 ADJ+NSUFF 0.6061 0.7407 0.6667 27 PREP+V 1.0000 0.5000 0.6667 2 NOUN+PRON 0.8082 0.9516 0.8741 62 PREP+PART 0.5000 1.0000 0.6667 1 V 0.9140 0.9043 0.9091 94 PROG_PART+V 0.7750 0.9394 0.8493 33 EOS 1.0000 1.0000 1.0000 70 MENTION 0.9130 1.0000 0.9545 21 PART+V 1.0000 0.0000 0.0000 3 PREP+NOUN 0.7857 0.6111 0.6875 18 PREP+PRON 0.9091 0.9524 0.9302 21 PREP+DET+NOUN 0.9000 0.9000 0.9000 10 URL 1.0000 1.0000 1.0000 3 FOREIGN 1.0000 0.7500 0.8571 4 CONJ+NOUN+NSUFF 0.7500 0.7500 0.7500 4 CONJ+NOUN 0.8333 1.0000 0.9091 10 DET+NOUN 0.9670 0.9565 0.9617 92 CONJ+PART 0.9091 0.9091 0.9091 11 CONJ+V 0.7143 0.6250 0.6667 8 DET+NOUN+NSUFF 0.8148 0.9565 0.8800 23 HASH 1.0000 0.8667 0.9286 15 PUNC 1.0000 1.0000 1.0000 135 CONJ 0.9459 0.9722 0.9589 36 NOUN+CASE 1.0000 1.0000 1.0000 4 CONJ+DET+NOUN 0.7143 1.0000 0.8333 5 PREP+NOUN+NSUFF+PRON 0.0000 0.0000 0.0000 1 PREP+NOUN+PRON 0.4286 0.5000 0.4615 6 FUT_PART+V 1.0000 0.5000 0.6667 6 CONJ+NOUN+PRON 0.8000 0.6667 0.7273 6 ADJ+PREP+PRON 0.0000 1.0000 0.0000 0 CONJ+V+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+V+PRON 0.8333 1.0000 0.9091 5 DET+ADJ 0.8333 0.9091 0.8696 11 ADJ+PRON 0.5000 0.2500 0.3333 4 PROG_PART+V+PREP+PRON 0.0000 0.0000 0.0000 3 NOUN+NSUFF+PRON 0.7778 0.7000 0.7368 10 PREP+PART+PRON 1.0000 0.6000 0.7500 5 PREP+PRON+DET+NOUN 1.0000 0.0000 0.0000 1 CONJ+NUM+NSUFF 1.0000 0.0000 0.0000 1 CONJ+NUM 0.0000 1.0000 0.0000 0 PROG_PART+V+PRON 0.6000 0.7500 0.6667 12 PART+PRON 0.9333 0.8750 0.9032 16 CONJ+PREP+NOUN+PRON 1.0000 0.0000 0.0000 1 PART+ADJ 1.0000 0.0000 0.0000 1 PART+NOUN 0.8000 1.0000 0.8889 4 CONJ+PREP+NOUN 1.0000 0.0000 0.0000 1 CONJ+PRON 1.0000 0.7500 0.8571 4 CONJ+FUT_PART 1.0000 0.0000 0.0000 1 PART+NOUN+PRON 0.0000 1.0000 0.0000 0 PART+PREP 1.0000 1.0000 1.0000 2 PREP+NOUN+NSUFF 0.6000 0.7500 0.6667 4 PREP+DET+NOUN+NSUFF 0.0000 0.0000 0.0000 1 V+PRON+PREP+PRON 0.5000 0.5000 0.5000 4 EMOT 1.0000 0.9333 0.9655 15 V+PREP+PRON 0.2000 0.3333 0.2500 3 PART+V+NEG_PART 0.1667 0.3333 0.2222 3 PROG_PART+V+PRON+PRON 0.0000 0.0000 0.0000 3 DET+ADJ+NSUFF 1.0000 0.5000 0.6667 4 FUT_PART 1.0000 1.0000 1.0000 4 CONJ+DET+ADJ+NSUFF 1.0000 0.0000 0.0000 1 CONJ+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 CONJ+ADV 1.0000 1.0000 1.0000 1 PART+PART 1.0000 0.0000 0.0000 2 CONJ+PREP 1.0000 1.0000 1.0000 3 PART+PROG_PART 1.0000 0.0000 0.0000 1 ADJ+CASE 0.5000 1.0000 0.6667 1 CONJ+PART+ADJ 1.0000 0.0000 0.0000 1 PROG_PART+V+NEG_PART 1.0000 0.0000 0.0000 1 PART+V+NOUN 1.0000 0.0000 0.0000 1 PART+V+PRON 1.0000 1.0000 1.0000 1 CONJ+PART+PROG_PART+V 1.0000 0.0000 0.0000 1 CONJ+PROG_PART+V 0.0000 1.0000 0.0000 0 CONJ+PART+V+NEG_PART 1.0000 0.0000 0.0000 1 CONJ+FUT_PART+V 0.0000 1.0000 0.0000 0 PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+PROG_PART+V+NEG_PART 0.0000 0.0000 0.0000 3 PART+V+PRON+NEG_PART 0.5000 0.4000 0.4444 5 PART+PREP+NEG_PART 1.0000 1.0000 1.0000 3 NOUN+CASE+PRON 0.0000 1.0000 0.0000 0 PART+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 DET+NUM 1.0000 0.0000 0.0000 1 FUT_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 FUT_PART+V+PRON 0.0000 1.0000 0.0000 0 PART+PROG_PART+V+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 CONJ+ADJ+NSUFF 0.0000 0.0000 0.0000 1 PREP+ADJ 1.0000 0.0000 0.0000 1 NOUN+CASE+NSUFF+NOUN+PRON 1.0000 0.0000 0.0000 1 V+NEG_PART 1.0000 0.0000 0.0000 1 ADJ+NSUFF+PRON 1.0000 0.0000 0.0000 1 micro avg 0.9114 0.9114 0.9114 2710 macro avg 0.7731 0.6053 0.5279 2710 weighted avg 0.9186 0.9114 0.9080 2710 2021-03-26 05:40:35,779 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:35,779 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:41,356 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC 2021-03-26 05:40:41,357 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu 2021-03-26 05:40:41,357 Dev: None 2021-03-26 05:40:41,357 Test: None 2021-03-26 05:40:41,619 Reading data from ../../Datasets_adhoc/UD_MADAR 2021-03-26 05:40:41,620 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu 2021-03-26 05:40:41,620 Dev: None 2021-03-26 05:40:41,620 Test: None 2021-03-26 05:40:41,660 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 05:40:41,661 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt 2021-03-26 05:40:41,661 Dev: None 2021-03-26 05:40:41,661 Test: None 2021-03-26 05:40:41,813 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 05:40:41,813 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt 2021-03-26 05:40:41,813 Dev: None 2021-03-26 05:40:41,814 Test: None 2021-03-26 05:40:41,985 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 05:40:41,985 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt 2021-03-26 05:40:41,986 Dev: None 2021-03-26 05:40:41,986 Test: None 2021-03-26 05:40:42,149 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 05:40:42,149 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt 2021-03-26 05:40:42,150 Dev: None 2021-03-26 05:40:42,150 Test: None 2021-03-26 05:40:42,309 Filtering long sentences 2021-03-26 05:40:42,352 MultiCorpus: 1574 train + 176 dev + 194 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences 2021-03-26 05:40:42,752 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:42,753 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): WordEmbeddings('ar') (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) (list_embedding_2): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=206, bias=True) (beta): 1.0 (weights): None (weight_tensor) None )" 2021-03-26 05:40:42,753 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:42,754 Corpus: "MultiCorpus: 1574 train + 176 dev + 194 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" 2021-03-26 05:40:42,754 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:42,754 Parameters: 2021-03-26 05:40:42,754 - learning_rate: "0.3" 2021-03-26 05:40:42,755 - mini_batch_size: "64" 2021-03-26 05:40:42,755 - patience: "3" 2021-03-26 05:40:42,755 - anneal_factor: "0.5" 2021-03-26 05:40:42,755 - max_epochs: "150" 2021-03-26 05:40:42,756 - shuffle: "True" 2021-03-26 05:40:42,756 - train_with_dev: "False" 2021-03-26 05:40:42,756 - batch_growth_annealing: "False" 2021-03-26 05:40:42,757 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:42,757 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260540" 2021-03-26 05:40:42,757 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:42,757 Device: cuda:0 2021-03-26 05:40:42,758 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:42,758 Embeddings storage mode: cpu 2021-03-26 05:40:42,759 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:40:44,574 epoch 1 - iter 2/25 - loss 88.64369583 - samples/sec: 70.58 - lr: 0.300000 2021-03-26 05:40:45,955 epoch 1 - iter 4/25 - loss 78.67724133 - samples/sec: 92.82 - lr: 0.300000 2021-03-26 05:40:47,222 epoch 1 - iter 6/25 - loss 75.12040520 - samples/sec: 101.11 - lr: 0.300000 2021-03-26 05:40:48,480 epoch 1 - iter 8/25 - loss 71.83301973 - samples/sec: 101.86 - lr: 0.300000 2021-03-26 05:40:49,869 epoch 1 - iter 10/25 - loss 70.04709473 - samples/sec: 92.21 - lr: 0.300000 2021-03-26 05:40:52,337 epoch 1 - iter 12/25 - loss 67.97456614 - samples/sec: 51.91 - lr: 0.300000 2021-03-26 05:40:53,686 epoch 1 - iter 14/25 - loss 66.16142763 - samples/sec: 94.99 - lr: 0.300000 2021-03-26 05:40:55,088 epoch 1 - iter 16/25 - loss 64.42413473 - samples/sec: 91.40 - lr: 0.300000 2021-03-26 05:40:56,535 epoch 1 - iter 18/25 - loss 62.82821062 - samples/sec: 88.56 - lr: 0.300000 2021-03-26 05:40:57,937 epoch 1 - iter 20/25 - loss 61.43530502 - samples/sec: 91.40 - lr: 0.300000 2021-03-26 05:40:59,282 epoch 1 - iter 22/25 - loss 59.67912865 - samples/sec: 95.27 - lr: 0.300000 2021-03-26 05:41:00,535 epoch 1 - iter 24/25 - loss 58.65658077 - samples/sec: 102.22 - lr: 0.300000 2021-03-26 05:41:01,124 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:41:01,125 EPOCH 1 done: loss 58.1316 - lr 0.3000000 2021-03-26 05:41:02,400 DEV : loss 44.08968734741211 - score 0.319 2021-03-26 05:41:02,426 BAD EPOCHS (no improvement): 0 2021-03-26 05:41:12,015 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:41:13,293 epoch 2 - iter 2/25 - loss 45.49032211 - samples/sec: 100.36 - lr: 0.300000 2021-03-26 05:41:14,221 epoch 2 - iter 4/25 - loss 46.81260490 - samples/sec: 138.31 - lr: 0.300000 2021-03-26 05:41:15,195 epoch 2 - iter 6/25 - loss 43.56631343 - samples/sec: 131.59 - lr: 0.300000 2021-03-26 05:41:16,116 epoch 2 - iter 8/25 - loss 42.27197218 - samples/sec: 139.35 - lr: 0.300000 2021-03-26 05:41:17,051 epoch 2 - iter 10/25 - loss 40.86771469 - samples/sec: 137.18 - lr: 0.300000 2021-03-26 05:41:18,114 epoch 2 - iter 12/25 - loss 39.77363968 - samples/sec: 120.63 - lr: 0.300000 2021-03-26 05:41:19,130 epoch 2 - iter 14/25 - loss 39.80917222 - samples/sec: 126.25 - lr: 0.300000 2021-03-26 05:41:20,100 epoch 2 - iter 16/25 - loss 39.00103319 - samples/sec: 132.27 - lr: 0.300000 2021-03-26 05:41:21,055 epoch 2 - iter 18/25 - loss 38.09381591 - samples/sec: 134.15 - lr: 0.300000 2021-03-26 05:41:21,991 epoch 2 - iter 20/25 - loss 37.36236010 - samples/sec: 137.06 - lr: 0.300000 2021-03-26 05:41:23,050 epoch 2 - iter 22/25 - loss 36.81540619 - samples/sec: 121.01 - lr: 0.300000 2021-03-26 05:41:24,160 epoch 2 - iter 24/25 - loss 36.91029286 - samples/sec: 115.48 - lr: 0.300000 2021-03-26 05:41:24,581 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:41:24,582 EPOCH 2 done: loss 36.7035 - lr 0.3000000 2021-03-26 05:41:25,367 DEV : loss 31.3277645111084 - score 0.5139 2021-03-26 05:41:25,390 BAD EPOCHS (no improvement): 0 2021-03-26 05:41:35,160 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:41:36,210 epoch 3 - iter 2/25 - loss 33.64617920 - samples/sec: 122.03 - lr: 0.300000 2021-03-26 05:41:37,266 epoch 3 - iter 4/25 - loss 29.12564039 - samples/sec: 121.46 - lr: 0.300000 2021-03-26 05:41:38,298 epoch 3 - iter 6/25 - loss 29.82938671 - samples/sec: 124.21 - lr: 0.300000 2021-03-26 05:41:39,277 epoch 3 - iter 8/25 - loss 29.37328362 - samples/sec: 130.98 - lr: 0.300000 2021-03-26 05:41:40,286 epoch 3 - iter 10/25 - loss 28.70174255 - samples/sec: 126.98 - lr: 0.300000 2021-03-26 05:41:41,223 epoch 3 - iter 12/25 - loss 28.44004679 - samples/sec: 136.95 - lr: 0.300000 2021-03-26 05:41:42,218 epoch 3 - iter 14/25 - loss 28.44743238 - samples/sec: 128.82 - lr: 0.300000 2021-03-26 05:41:43,188 epoch 3 - iter 16/25 - loss 27.92235672 - samples/sec: 132.22 - lr: 0.300000 2021-03-26 05:41:44,221 epoch 3 - iter 18/25 - loss 27.86411593 - samples/sec: 124.04 - lr: 0.300000 2021-03-26 05:41:45,210 epoch 3 - iter 20/25 - loss 27.75742226 - samples/sec: 129.62 - lr: 0.300000 2021-03-26 05:41:46,376 epoch 3 - iter 22/25 - loss 27.58030839 - samples/sec: 109.93 - lr: 0.300000 2021-03-26 05:41:47,380 epoch 3 - iter 24/25 - loss 27.43895706 - samples/sec: 127.80 - lr: 0.300000 2021-03-26 05:41:47,785 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:41:47,786 EPOCH 3 done: loss 27.1920 - lr 0.3000000 2021-03-26 05:41:48,593 DEV : loss 22.79669189453125 - score 0.6489 2021-03-26 05:41:48,619 BAD EPOCHS (no improvement): 0 2021-03-26 05:41:58,360 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:41:59,359 epoch 4 - iter 2/25 - loss 22.24100590 - samples/sec: 128.43 - lr: 0.300000 2021-03-26 05:42:00,348 epoch 4 - iter 4/25 - loss 21.90831089 - samples/sec: 129.61 - lr: 0.300000 2021-03-26 05:42:01,456 epoch 4 - iter 6/25 - loss 23.29270585 - samples/sec: 115.69 - lr: 0.300000 2021-03-26 05:42:02,415 epoch 4 - iter 8/25 - loss 23.13200498 - samples/sec: 133.65 - lr: 0.300000 2021-03-26 05:42:03,346 epoch 4 - iter 10/25 - loss 23.02010689 - samples/sec: 137.73 - lr: 0.300000 2021-03-26 05:42:04,364 epoch 4 - iter 12/25 - loss 22.89023797 - samples/sec: 125.93 - lr: 0.300000 2021-03-26 05:42:05,402 epoch 4 - iter 14/25 - loss 23.05532360 - samples/sec: 123.46 - lr: 0.300000 2021-03-26 05:42:06,479 epoch 4 - iter 16/25 - loss 23.22665524 - samples/sec: 119.00 - lr: 0.300000 2021-03-26 05:42:07,364 epoch 4 - iter 18/25 - loss 22.73062568 - samples/sec: 144.81 - lr: 0.300000 2021-03-26 05:42:08,340 epoch 4 - iter 20/25 - loss 22.55190754 - samples/sec: 131.33 - lr: 0.300000 2021-03-26 05:42:09,495 epoch 4 - iter 22/25 - loss 22.42480538 - samples/sec: 110.97 - lr: 0.300000 2021-03-26 05:42:10,548 epoch 4 - iter 24/25 - loss 22.38524278 - samples/sec: 121.73 - lr: 0.300000 2021-03-26 05:42:11,025 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:42:11,025 EPOCH 4 done: loss 22.1926 - lr 0.3000000 2021-03-26 05:42:11,839 DEV : loss 17.91794204711914 - score 0.7292 2021-03-26 05:42:11,862 BAD EPOCHS (no improvement): 0 2021-03-26 05:42:21,724 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:42:22,788 epoch 5 - iter 2/25 - loss 18.36693954 - samples/sec: 120.56 - lr: 0.300000 2021-03-26 05:42:23,768 epoch 5 - iter 4/25 - loss 18.09573936 - samples/sec: 131.00 - lr: 0.300000 2021-03-26 05:42:24,762 epoch 5 - iter 6/25 - loss 17.68604310 - samples/sec: 128.88 - lr: 0.300000 2021-03-26 05:42:25,791 epoch 5 - iter 8/25 - loss 18.63525987 - samples/sec: 124.45 - lr: 0.300000 2021-03-26 05:42:26,695 epoch 5 - iter 10/25 - loss 18.71813297 - samples/sec: 142.48 - lr: 0.300000 2021-03-26 05:42:27,701 epoch 5 - iter 12/25 - loss 18.54522006 - samples/sec: 127.54 - lr: 0.300000 2021-03-26 05:42:28,692 epoch 5 - iter 14/25 - loss 18.73616600 - samples/sec: 129.53 - lr: 0.300000 2021-03-26 05:42:29,767 epoch 5 - iter 16/25 - loss 18.44975418 - samples/sec: 119.20 - lr: 0.300000 2021-03-26 05:42:30,710 epoch 5 - iter 18/25 - loss 18.59771268 - samples/sec: 135.96 - lr: 0.300000 2021-03-26 05:42:31,781 epoch 5 - iter 20/25 - loss 18.54842658 - samples/sec: 119.61 - lr: 0.300000 2021-03-26 05:42:32,744 epoch 5 - iter 22/25 - loss 18.40371821 - samples/sec: 133.15 - lr: 0.300000 2021-03-26 05:42:33,699 epoch 5 - iter 24/25 - loss 18.12898660 - samples/sec: 134.20 - lr: 0.300000 2021-03-26 05:42:34,145 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:42:34,146 EPOCH 5 done: loss 18.2116 - lr 0.3000000 2021-03-26 05:42:34,937 DEV : loss 15.187065124511719 - score 0.7554 2021-03-26 05:42:34,962 BAD EPOCHS (no improvement): 0 2021-03-26 05:42:44,840 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:42:45,857 epoch 6 - iter 2/25 - loss 16.13490629 - samples/sec: 126.10 - lr: 0.300000 2021-03-26 05:42:46,823 epoch 6 - iter 4/25 - loss 15.87294555 - samples/sec: 132.95 - lr: 0.300000 2021-03-26 05:42:47,793 epoch 6 - iter 6/25 - loss 16.28924131 - samples/sec: 132.32 - lr: 0.300000 2021-03-26 05:42:48,742 epoch 6 - iter 8/25 - loss 15.93548644 - samples/sec: 134.96 - lr: 0.300000 2021-03-26 05:42:49,689 epoch 6 - iter 10/25 - loss 16.17079515 - samples/sec: 135.52 - lr: 0.300000 2021-03-26 05:42:50,735 epoch 6 - iter 12/25 - loss 15.89895153 - samples/sec: 122.60 - lr: 0.300000 2021-03-26 05:42:51,805 epoch 6 - iter 14/25 - loss 15.59099872 - samples/sec: 119.76 - lr: 0.300000 2021-03-26 05:42:52,797 epoch 6 - iter 16/25 - loss 15.44279909 - samples/sec: 129.43 - lr: 0.300000 2021-03-26 05:42:53,825 epoch 6 - iter 18/25 - loss 15.70377202 - samples/sec: 124.70 - lr: 0.300000 2021-03-26 05:42:54,955 epoch 6 - iter 20/25 - loss 15.55591607 - samples/sec: 113.46 - lr: 0.300000 2021-03-26 05:42:55,851 epoch 6 - iter 22/25 - loss 15.54194147 - samples/sec: 143.21 - lr: 0.300000 2021-03-26 05:42:56,945 epoch 6 - iter 24/25 - loss 15.66215499 - samples/sec: 117.09 - lr: 0.300000 2021-03-26 05:42:57,368 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:42:57,369 EPOCH 6 done: loss 15.8092 - lr 0.3000000 2021-03-26 05:42:58,145 DEV : loss 14.743255615234375 - score 0.7597 2021-03-26 05:42:58,165 BAD EPOCHS (no improvement): 0 2021-03-26 05:43:08,028 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:43:09,057 epoch 7 - iter 2/25 - loss 18.11933041 - samples/sec: 124.73 - lr: 0.300000 2021-03-26 05:43:09,997 epoch 7 - iter 4/25 - loss 16.30117321 - samples/sec: 136.42 - lr: 0.300000 2021-03-26 05:43:11,305 epoch 7 - iter 6/25 - loss 15.62002357 - samples/sec: 97.93 - lr: 0.300000 2021-03-26 05:43:12,397 epoch 7 - iter 8/25 - loss 15.28432643 - samples/sec: 117.40 - lr: 0.300000 2021-03-26 05:43:13,491 epoch 7 - iter 10/25 - loss 15.35674391 - samples/sec: 117.19 - lr: 0.300000 2021-03-26 05:43:14,526 epoch 7 - iter 12/25 - loss 15.29835645 - samples/sec: 123.82 - lr: 0.300000 2021-03-26 05:43:15,600 epoch 7 - iter 14/25 - loss 15.13747549 - samples/sec: 119.34 - lr: 0.300000 2021-03-26 05:43:16,588 epoch 7 - iter 16/25 - loss 15.07328975 - samples/sec: 130.04 - lr: 0.300000 2021-03-26 05:43:17,676 epoch 7 - iter 18/25 - loss 14.88522440 - samples/sec: 117.79 - lr: 0.300000 2021-03-26 05:43:18,673 epoch 7 - iter 20/25 - loss 14.82129803 - samples/sec: 128.68 - lr: 0.300000 2021-03-26 05:43:19,643 epoch 7 - iter 22/25 - loss 14.80583009 - samples/sec: 132.12 - lr: 0.300000 2021-03-26 05:43:20,617 epoch 7 - iter 24/25 - loss 14.60828722 - samples/sec: 131.68 - lr: 0.300000 2021-03-26 05:43:21,020 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:43:21,021 EPOCH 7 done: loss 14.5242 - lr 0.3000000 2021-03-26 05:43:21,793 DEV : loss 12.157855033874512 - score 0.8015 2021-03-26 05:43:21,818 BAD EPOCHS (no improvement): 0 2021-03-26 05:43:31,696 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:43:32,747 epoch 8 - iter 2/25 - loss 11.99634266 - samples/sec: 122.05 - lr: 0.300000 2021-03-26 05:43:33,751 epoch 8 - iter 4/25 - loss 12.01598549 - samples/sec: 127.66 - lr: 0.300000 2021-03-26 05:43:34,778 epoch 8 - iter 6/25 - loss 12.23479764 - samples/sec: 124.88 - lr: 0.300000 2021-03-26 05:43:35,847 epoch 8 - iter 8/25 - loss 12.58600891 - samples/sec: 119.95 - lr: 0.300000 2021-03-26 05:43:36,884 epoch 8 - iter 10/25 - loss 12.72332258 - samples/sec: 123.62 - lr: 0.300000 2021-03-26 05:43:38,058 epoch 8 - iter 12/25 - loss 12.66574216 - samples/sec: 109.13 - lr: 0.300000 2021-03-26 05:43:39,024 epoch 8 - iter 14/25 - loss 12.68500212 - samples/sec: 132.88 - lr: 0.300000 2021-03-26 05:43:40,057 epoch 8 - iter 16/25 - loss 12.81532699 - samples/sec: 124.04 - lr: 0.300000 2021-03-26 05:43:41,106 epoch 8 - iter 18/25 - loss 12.93446302 - samples/sec: 122.14 - lr: 0.300000 2021-03-26 05:43:42,093 epoch 8 - iter 20/25 - loss 12.94684072 - samples/sec: 130.16 - lr: 0.300000 2021-03-26 05:43:43,106 epoch 8 - iter 22/25 - loss 13.04992381 - samples/sec: 126.44 - lr: 0.300000 2021-03-26 05:43:44,042 epoch 8 - iter 24/25 - loss 12.85443902 - samples/sec: 137.07 - lr: 0.300000 2021-03-26 05:43:44,491 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:43:44,492 EPOCH 8 done: loss 12.8589 - lr 0.3000000 2021-03-26 05:43:45,261 DEV : loss 11.195834159851074 - score 0.8192 2021-03-26 05:43:45,281 BAD EPOCHS (no improvement): 0 2021-03-26 05:43:54,888 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:43:55,945 epoch 9 - iter 2/25 - loss 12.01339865 - samples/sec: 121.34 - lr: 0.300000 2021-03-26 05:43:57,004 epoch 9 - iter 4/25 - loss 12.19045806 - samples/sec: 121.05 - lr: 0.300000 2021-03-26 05:43:58,061 epoch 9 - iter 6/25 - loss 11.92286936 - samples/sec: 121.42 - lr: 0.300000 2021-03-26 05:43:58,999 epoch 9 - iter 8/25 - loss 12.26701045 - samples/sec: 136.70 - lr: 0.300000 2021-03-26 05:44:00,040 epoch 9 - iter 10/25 - loss 12.15806046 - samples/sec: 123.15 - lr: 0.300000 2021-03-26 05:44:01,017 epoch 9 - iter 12/25 - loss 12.16244268 - samples/sec: 131.21 - lr: 0.300000 2021-03-26 05:44:01,965 epoch 9 - iter 14/25 - loss 12.17545857 - samples/sec: 135.17 - lr: 0.300000 2021-03-26 05:44:02,965 epoch 9 - iter 16/25 - loss 12.02299416 - samples/sec: 128.17 - lr: 0.300000 2021-03-26 05:44:04,124 epoch 9 - iter 18/25 - loss 12.25711123 - samples/sec: 110.49 - lr: 0.300000 2021-03-26 05:44:05,086 epoch 9 - iter 20/25 - loss 12.06897812 - samples/sec: 133.46 - lr: 0.300000 2021-03-26 05:44:06,318 epoch 9 - iter 22/25 - loss 11.92144212 - samples/sec: 104.08 - lr: 0.300000 2021-03-26 05:44:07,365 epoch 9 - iter 24/25 - loss 11.91666170 - samples/sec: 122.50 - lr: 0.300000 2021-03-26 05:44:07,806 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:44:07,806 EPOCH 9 done: loss 11.8426 - lr 0.3000000 2021-03-26 05:44:08,597 DEV : loss 11.0265531539917 - score 0.8206 2021-03-26 05:44:08,626 BAD EPOCHS (no improvement): 0 2021-03-26 05:44:18,228 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:44:19,477 epoch 10 - iter 2/25 - loss 11.34327888 - samples/sec: 102.68 - lr: 0.300000 2021-03-26 05:44:20,455 epoch 10 - iter 4/25 - loss 11.71344209 - samples/sec: 131.17 - lr: 0.300000 2021-03-26 05:44:21,429 epoch 10 - iter 6/25 - loss 11.20282396 - samples/sec: 131.54 - lr: 0.300000 2021-03-26 05:44:22,468 epoch 10 - iter 8/25 - loss 11.10128748 - samples/sec: 123.46 - lr: 0.300000 2021-03-26 05:44:23,485 epoch 10 - iter 10/25 - loss 10.91363373 - samples/sec: 126.05 - lr: 0.300000 2021-03-26 05:44:24,582 epoch 10 - iter 12/25 - loss 10.88611531 - samples/sec: 116.79 - lr: 0.300000 2021-03-26 05:44:25,566 epoch 10 - iter 14/25 - loss 10.59839494 - samples/sec: 130.37 - lr: 0.300000 2021-03-26 05:44:26,641 epoch 10 - iter 16/25 - loss 10.86341327 - samples/sec: 119.11 - lr: 0.300000 2021-03-26 05:44:27,623 epoch 10 - iter 18/25 - loss 10.94241174 - samples/sec: 130.58 - lr: 0.300000 2021-03-26 05:44:28,671 epoch 10 - iter 20/25 - loss 10.91952567 - samples/sec: 122.37 - lr: 0.300000 2021-03-26 05:44:29,678 epoch 10 - iter 22/25 - loss 10.93466590 - samples/sec: 127.25 - lr: 0.300000 2021-03-26 05:44:30,656 epoch 10 - iter 24/25 - loss 11.03978912 - samples/sec: 131.27 - lr: 0.300000 2021-03-26 05:44:31,060 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:44:31,061 EPOCH 10 done: loss 10.9988 - lr 0.3000000 2021-03-26 05:44:31,840 DEV : loss 10.144160270690918 - score 0.8356 2021-03-26 05:44:31,863 BAD EPOCHS (no improvement): 0 2021-03-26 05:44:41,520 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:44:42,617 epoch 11 - iter 2/25 - loss 10.94504309 - samples/sec: 116.96 - lr: 0.300000 2021-03-26 05:44:43,667 epoch 11 - iter 4/25 - loss 11.22785974 - samples/sec: 122.18 - lr: 0.300000 2021-03-26 05:44:44,765 epoch 11 - iter 6/25 - loss 11.41904799 - samples/sec: 116.63 - lr: 0.300000 2021-03-26 05:44:45,811 epoch 11 - iter 8/25 - loss 11.12639284 - samples/sec: 122.71 - lr: 0.300000 2021-03-26 05:44:46,856 epoch 11 - iter 10/25 - loss 11.28100567 - samples/sec: 122.59 - lr: 0.300000 2021-03-26 05:44:47,977 epoch 11 - iter 12/25 - loss 11.00330440 - samples/sec: 114.46 - lr: 0.300000 2021-03-26 05:44:49,008 epoch 11 - iter 14/25 - loss 10.97292021 - samples/sec: 124.53 - lr: 0.300000 2021-03-26 05:44:50,077 epoch 11 - iter 16/25 - loss 10.63795060 - samples/sec: 119.83 - lr: 0.300000 2021-03-26 05:44:51,066 epoch 11 - iter 18/25 - loss 10.61853668 - samples/sec: 129.69 - lr: 0.300000 2021-03-26 05:44:51,989 epoch 11 - iter 20/25 - loss 10.53863492 - samples/sec: 138.88 - lr: 0.300000 2021-03-26 05:44:52,966 epoch 11 - iter 22/25 - loss 10.60792611 - samples/sec: 131.20 - lr: 0.300000 2021-03-26 05:44:53,908 epoch 11 - iter 24/25 - loss 10.64770297 - samples/sec: 135.97 - lr: 0.300000 2021-03-26 05:44:54,259 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:44:54,260 EPOCH 11 done: loss 10.5094 - lr 0.3000000 2021-03-26 05:44:55,041 DEV : loss 9.662548065185547 - score 0.842 2021-03-26 05:44:55,063 BAD EPOCHS (no improvement): 0 2021-03-26 05:45:04,767 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:45:05,837 epoch 12 - iter 2/25 - loss 10.17411470 - samples/sec: 119.89 - lr: 0.300000 2021-03-26 05:45:06,893 epoch 12 - iter 4/25 - loss 9.86866117 - samples/sec: 121.47 - lr: 0.300000 2021-03-26 05:45:07,879 epoch 12 - iter 6/25 - loss 10.10727294 - samples/sec: 130.06 - lr: 0.300000 2021-03-26 05:45:08,896 epoch 12 - iter 8/25 - loss 9.61317658 - samples/sec: 125.98 - lr: 0.300000 2021-03-26 05:45:09,979 epoch 12 - iter 10/25 - loss 9.45750113 - samples/sec: 118.34 - lr: 0.300000 2021-03-26 05:45:11,065 epoch 12 - iter 12/25 - loss 9.53312612 - samples/sec: 118.03 - lr: 0.300000 2021-03-26 05:45:12,024 epoch 12 - iter 14/25 - loss 9.49966478 - samples/sec: 133.62 - lr: 0.300000 2021-03-26 05:45:13,058 epoch 12 - iter 16/25 - loss 9.55181575 - samples/sec: 124.01 - lr: 0.300000 2021-03-26 05:45:14,023 epoch 12 - iter 18/25 - loss 9.56000879 - samples/sec: 132.73 - lr: 0.300000 2021-03-26 05:45:14,936 epoch 12 - iter 20/25 - loss 9.61673174 - samples/sec: 140.71 - lr: 0.300000 2021-03-26 05:45:16,021 epoch 12 - iter 22/25 - loss 9.69639319 - samples/sec: 118.08 - lr: 0.300000 2021-03-26 05:45:17,042 epoch 12 - iter 24/25 - loss 9.75539422 - samples/sec: 125.71 - lr: 0.300000 2021-03-26 05:45:17,450 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:45:17,450 EPOCH 12 done: loss 9.7850 - lr 0.3000000 2021-03-26 05:45:18,218 DEV : loss 8.691134452819824 - score 0.8552 2021-03-26 05:45:18,244 BAD EPOCHS (no improvement): 0 2021-03-26 05:45:27,952 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:45:29,023 epoch 13 - iter 2/25 - loss 9.57857800 - samples/sec: 119.78 - lr: 0.300000 2021-03-26 05:45:30,085 epoch 13 - iter 4/25 - loss 9.67772269 - samples/sec: 120.75 - lr: 0.300000 2021-03-26 05:45:31,154 epoch 13 - iter 6/25 - loss 9.08809503 - samples/sec: 119.94 - lr: 0.300000 2021-03-26 05:45:32,351 epoch 13 - iter 8/25 - loss 8.77041090 - samples/sec: 107.04 - lr: 0.300000 2021-03-26 05:45:33,301 epoch 13 - iter 10/25 - loss 8.77164540 - samples/sec: 134.94 - lr: 0.300000 2021-03-26 05:45:34,298 epoch 13 - iter 12/25 - loss 8.93865446 - samples/sec: 128.70 - lr: 0.300000 2021-03-26 05:45:35,393 epoch 13 - iter 14/25 - loss 9.02500653 - samples/sec: 117.03 - lr: 0.300000 2021-03-26 05:45:36,370 epoch 13 - iter 16/25 - loss 8.95800772 - samples/sec: 131.27 - lr: 0.300000 2021-03-26 05:45:37,341 epoch 13 - iter 18/25 - loss 8.78139753 - samples/sec: 131.90 - lr: 0.300000 2021-03-26 05:45:38,390 epoch 13 - iter 20/25 - loss 8.79425094 - samples/sec: 122.27 - lr: 0.300000 2021-03-26 05:45:39,547 epoch 13 - iter 22/25 - loss 9.00863030 - samples/sec: 110.74 - lr: 0.300000 2021-03-26 05:45:40,547 epoch 13 - iter 24/25 - loss 9.03554883 - samples/sec: 128.22 - lr: 0.300000 2021-03-26 05:45:40,918 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:45:40,918 EPOCH 13 done: loss 9.1318 - lr 0.3000000 2021-03-26 05:45:41,743 DEV : loss 9.024928092956543 - score 0.8556 2021-03-26 05:45:41,765 BAD EPOCHS (no improvement): 0 2021-03-26 05:45:51,573 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:45:52,696 epoch 14 - iter 2/25 - loss 8.50551748 - samples/sec: 114.32 - lr: 0.300000 2021-03-26 05:45:53,691 epoch 14 - iter 4/25 - loss 8.25183105 - samples/sec: 129.07 - lr: 0.300000 2021-03-26 05:45:54,698 epoch 14 - iter 6/25 - loss 8.82747555 - samples/sec: 127.24 - lr: 0.300000 2021-03-26 05:45:55,723 epoch 14 - iter 8/25 - loss 8.81481767 - samples/sec: 125.19 - lr: 0.300000 2021-03-26 05:45:56,715 epoch 14 - iter 10/25 - loss 8.86008492 - samples/sec: 129.27 - lr: 0.300000 2021-03-26 05:45:57,773 epoch 14 - iter 12/25 - loss 8.99228557 - samples/sec: 121.11 - lr: 0.300000 2021-03-26 05:45:58,924 epoch 14 - iter 14/25 - loss 8.92841278 - samples/sec: 111.53 - lr: 0.300000 2021-03-26 05:45:59,921 epoch 14 - iter 16/25 - loss 8.81908402 - samples/sec: 128.65 - lr: 0.300000 2021-03-26 05:46:00,917 epoch 14 - iter 18/25 - loss 8.85951858 - samples/sec: 128.72 - lr: 0.300000 2021-03-26 05:46:01,962 epoch 14 - iter 20/25 - loss 8.76512609 - samples/sec: 122.65 - lr: 0.300000 2021-03-26 05:46:02,926 epoch 14 - iter 22/25 - loss 8.77875579 - samples/sec: 132.88 - lr: 0.300000 2021-03-26 05:46:03,941 epoch 14 - iter 24/25 - loss 8.72450026 - samples/sec: 126.29 - lr: 0.300000 2021-03-26 05:46:04,361 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:46:04,362 EPOCH 14 done: loss 8.8084 - lr 0.3000000 2021-03-26 05:46:05,158 DEV : loss 8.242410659790039 - score 0.8596 2021-03-26 05:46:05,176 BAD EPOCHS (no improvement): 0 2021-03-26 05:46:14,853 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:46:16,012 epoch 15 - iter 2/25 - loss 8.07281971 - samples/sec: 110.99 - lr: 0.300000 2021-03-26 05:46:17,051 epoch 15 - iter 4/25 - loss 8.91634297 - samples/sec: 123.53 - lr: 0.300000 2021-03-26 05:46:17,999 epoch 15 - iter 6/25 - loss 8.43717909 - samples/sec: 135.21 - lr: 0.300000 2021-03-26 05:46:18,994 epoch 15 - iter 8/25 - loss 8.13764864 - samples/sec: 128.95 - lr: 0.300000 2021-03-26 05:46:19,991 epoch 15 - iter 10/25 - loss 8.22757897 - samples/sec: 128.62 - lr: 0.300000 2021-03-26 05:46:20,963 epoch 15 - iter 12/25 - loss 8.27018185 - samples/sec: 132.56 - lr: 0.300000 2021-03-26 05:46:21,938 epoch 15 - iter 14/25 - loss 8.31585772 - samples/sec: 131.44 - lr: 0.300000 2021-03-26 05:46:22,995 epoch 15 - iter 16/25 - loss 8.29172522 - samples/sec: 121.31 - lr: 0.300000 2021-03-26 05:46:24,040 epoch 15 - iter 18/25 - loss 8.23328315 - samples/sec: 122.62 - lr: 0.300000 2021-03-26 05:46:24,981 epoch 15 - iter 20/25 - loss 8.09761107 - samples/sec: 136.19 - lr: 0.300000 2021-03-26 05:46:26,065 epoch 15 - iter 22/25 - loss 8.19500925 - samples/sec: 118.27 - lr: 0.300000 2021-03-26 05:46:27,191 epoch 15 - iter 24/25 - loss 8.17269633 - samples/sec: 113.79 - lr: 0.300000 2021-03-26 05:46:27,663 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:46:27,663 EPOCH 15 done: loss 8.2421 - lr 0.3000000 2021-03-26 05:46:28,522 DEV : loss 8.071908950805664 - score 0.8716 2021-03-26 05:46:28,548 BAD EPOCHS (no improvement): 0 2021-03-26 05:46:38,366 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:46:39,495 epoch 16 - iter 2/25 - loss 6.56948996 - samples/sec: 113.55 - lr: 0.300000 2021-03-26 05:46:40,582 epoch 16 - iter 4/25 - loss 7.08489966 - samples/sec: 117.99 - lr: 0.300000 2021-03-26 05:46:41,569 epoch 16 - iter 6/25 - loss 7.12076028 - samples/sec: 129.85 - lr: 0.300000 2021-03-26 05:46:42,539 epoch 16 - iter 8/25 - loss 7.15210068 - samples/sec: 132.06 - lr: 0.300000 2021-03-26 05:46:43,591 epoch 16 - iter 10/25 - loss 7.48657627 - samples/sec: 121.91 - lr: 0.300000 2021-03-26 05:46:44,556 epoch 16 - iter 12/25 - loss 7.65123530 - samples/sec: 133.57 - lr: 0.300000 2021-03-26 05:46:45,508 epoch 16 - iter 14/25 - loss 7.54185397 - samples/sec: 134.63 - lr: 0.300000 2021-03-26 05:46:46,493 epoch 16 - iter 16/25 - loss 7.41987136 - samples/sec: 130.13 - lr: 0.300000 2021-03-26 05:46:47,513 epoch 16 - iter 18/25 - loss 7.63816534 - samples/sec: 125.67 - lr: 0.300000 2021-03-26 05:46:48,487 epoch 16 - iter 20/25 - loss 7.70393589 - samples/sec: 131.57 - lr: 0.300000 2021-03-26 05:46:49,485 epoch 16 - iter 22/25 - loss 7.68824291 - samples/sec: 128.48 - lr: 0.300000 2021-03-26 05:46:50,458 epoch 16 - iter 24/25 - loss 7.76888394 - samples/sec: 131.66 - lr: 0.300000 2021-03-26 05:46:50,926 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:46:50,926 EPOCH 16 done: loss 7.7148 - lr 0.3000000 2021-03-26 05:46:51,699 DEV : loss 7.759528636932373 - score 0.8764 2021-03-26 05:46:51,726 BAD EPOCHS (no improvement): 0 2021-03-26 05:47:01,505 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:47:02,648 epoch 17 - iter 2/25 - loss 7.21234059 - samples/sec: 112.30 - lr: 0.300000 2021-03-26 05:47:03,629 epoch 17 - iter 4/25 - loss 7.31978428 - samples/sec: 130.64 - lr: 0.300000 2021-03-26 05:47:04,689 epoch 17 - iter 6/25 - loss 7.75275191 - samples/sec: 120.98 - lr: 0.300000 2021-03-26 05:47:05,784 epoch 17 - iter 8/25 - loss 7.63426083 - samples/sec: 116.98 - lr: 0.300000 2021-03-26 05:47:06,829 epoch 17 - iter 10/25 - loss 7.70618315 - samples/sec: 122.75 - lr: 0.300000 2021-03-26 05:47:07,839 epoch 17 - iter 12/25 - loss 7.80201284 - samples/sec: 126.90 - lr: 0.300000 2021-03-26 05:47:08,941 epoch 17 - iter 14/25 - loss 7.86180087 - samples/sec: 116.28 - lr: 0.300000 2021-03-26 05:47:09,987 epoch 17 - iter 16/25 - loss 7.67045468 - samples/sec: 122.61 - lr: 0.300000 2021-03-26 05:47:11,023 epoch 17 - iter 18/25 - loss 7.80948228 - samples/sec: 123.82 - lr: 0.300000 2021-03-26 05:47:11,955 epoch 17 - iter 20/25 - loss 7.69441493 - samples/sec: 137.57 - lr: 0.300000 2021-03-26 05:47:12,997 epoch 17 - iter 22/25 - loss 7.67484760 - samples/sec: 122.99 - lr: 0.300000 2021-03-26 05:47:14,164 epoch 17 - iter 24/25 - loss 7.75634086 - samples/sec: 109.77 - lr: 0.300000 2021-03-26 05:47:14,562 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:47:14,563 EPOCH 17 done: loss 7.7065 - lr 0.3000000 2021-03-26 05:47:15,365 DEV : loss 7.736227989196777 - score 0.8774 2021-03-26 05:47:15,387 BAD EPOCHS (no improvement): 0 2021-03-26 05:47:25,233 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:47:26,346 epoch 18 - iter 2/25 - loss 7.17800355 - samples/sec: 115.22 - lr: 0.300000 2021-03-26 05:47:27,483 epoch 18 - iter 4/25 - loss 7.56220233 - samples/sec: 112.90 - lr: 0.300000 2021-03-26 05:47:28,563 epoch 18 - iter 6/25 - loss 7.32986403 - samples/sec: 118.70 - lr: 0.300000 2021-03-26 05:47:29,565 epoch 18 - iter 8/25 - loss 7.14382195 - samples/sec: 127.95 - lr: 0.300000 2021-03-26 05:47:30,537 epoch 18 - iter 10/25 - loss 7.13372774 - samples/sec: 131.85 - lr: 0.300000 2021-03-26 05:47:31,619 epoch 18 - iter 12/25 - loss 7.36776002 - samples/sec: 118.43 - lr: 0.300000 2021-03-26 05:47:32,635 epoch 18 - iter 14/25 - loss 7.26921167 - samples/sec: 126.40 - lr: 0.300000 2021-03-26 05:47:33,691 epoch 18 - iter 16/25 - loss 7.35486242 - samples/sec: 121.31 - lr: 0.300000 2021-03-26 05:47:34,902 epoch 18 - iter 18/25 - loss 7.30507800 - samples/sec: 105.82 - lr: 0.300000 2021-03-26 05:47:35,916 epoch 18 - iter 20/25 - loss 7.24085248 - samples/sec: 126.64 - lr: 0.300000 2021-03-26 05:47:36,889 epoch 18 - iter 22/25 - loss 7.21271409 - samples/sec: 131.65 - lr: 0.300000 2021-03-26 05:47:37,888 epoch 18 - iter 24/25 - loss 7.22516036 - samples/sec: 128.35 - lr: 0.300000 2021-03-26 05:47:38,322 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:47:38,323 EPOCH 18 done: loss 7.2983 - lr 0.3000000 2021-03-26 05:47:39,106 DEV : loss 7.408334732055664 - score 0.8796 2021-03-26 05:47:39,132 BAD EPOCHS (no improvement): 0 2021-03-26 05:47:48,970 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:47:49,936 epoch 19 - iter 2/25 - loss 7.55202436 - samples/sec: 132.73 - lr: 0.300000 2021-03-26 05:47:50,894 epoch 19 - iter 4/25 - loss 6.82127225 - samples/sec: 133.93 - lr: 0.300000 2021-03-26 05:47:51,881 epoch 19 - iter 6/25 - loss 6.78057869 - samples/sec: 130.03 - lr: 0.300000 2021-03-26 05:47:53,154 epoch 19 - iter 8/25 - loss 6.69285744 - samples/sec: 100.75 - lr: 0.300000 2021-03-26 05:47:54,148 epoch 19 - iter 10/25 - loss 6.59116917 - samples/sec: 128.99 - lr: 0.300000 2021-03-26 05:47:55,029 epoch 19 - iter 12/25 - loss 6.69682733 - samples/sec: 145.48 - lr: 0.300000 2021-03-26 05:47:56,012 epoch 19 - iter 14/25 - loss 6.73391271 - samples/sec: 130.40 - lr: 0.300000 2021-03-26 05:47:57,053 epoch 19 - iter 16/25 - loss 6.77963328 - samples/sec: 123.15 - lr: 0.300000 2021-03-26 05:47:58,029 epoch 19 - iter 18/25 - loss 6.72666068 - samples/sec: 131.55 - lr: 0.300000 2021-03-26 05:47:59,114 epoch 19 - iter 20/25 - loss 6.93562422 - samples/sec: 118.20 - lr: 0.300000 2021-03-26 05:48:00,115 epoch 19 - iter 22/25 - loss 6.89227057 - samples/sec: 128.10 - lr: 0.300000 2021-03-26 05:48:01,107 epoch 19 - iter 24/25 - loss 6.89240589 - samples/sec: 129.35 - lr: 0.300000 2021-03-26 05:48:01,518 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:48:01,520 EPOCH 19 done: loss 6.8797 - lr 0.3000000 2021-03-26 05:48:02,335 DEV : loss 7.592113018035889 - score 0.8746 2021-03-26 05:48:02,365 BAD EPOCHS (no improvement): 1 2021-03-26 05:48:02,365 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:48:03,396 epoch 20 - iter 2/25 - loss 6.29856849 - samples/sec: 124.38 - lr: 0.300000 2021-03-26 05:48:04,360 epoch 20 - iter 4/25 - loss 6.76293743 - samples/sec: 133.10 - lr: 0.300000 2021-03-26 05:48:05,396 epoch 20 - iter 6/25 - loss 6.62074788 - samples/sec: 123.75 - lr: 0.300000 2021-03-26 05:48:06,555 epoch 20 - iter 8/25 - loss 6.44405246 - samples/sec: 110.54 - lr: 0.300000 2021-03-26 05:48:07,562 epoch 20 - iter 10/25 - loss 6.45065737 - samples/sec: 127.35 - lr: 0.300000 2021-03-26 05:48:08,536 epoch 20 - iter 12/25 - loss 6.51046475 - samples/sec: 131.57 - lr: 0.300000 2021-03-26 05:48:09,484 epoch 20 - iter 14/25 - loss 6.43319787 - samples/sec: 135.39 - lr: 0.300000 2021-03-26 05:48:10,474 epoch 20 - iter 16/25 - loss 6.43069494 - samples/sec: 129.43 - lr: 0.300000 2021-03-26 05:48:11,561 epoch 20 - iter 18/25 - loss 6.40164778 - samples/sec: 118.00 - lr: 0.300000 2021-03-26 05:48:12,743 epoch 20 - iter 20/25 - loss 6.54254618 - samples/sec: 108.39 - lr: 0.300000 2021-03-26 05:48:13,764 epoch 20 - iter 22/25 - loss 6.58177922 - samples/sec: 125.65 - lr: 0.300000 2021-03-26 05:48:14,738 epoch 20 - iter 24/25 - loss 6.61301217 - samples/sec: 131.54 - lr: 0.300000 2021-03-26 05:48:15,139 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:48:15,140 EPOCH 20 done: loss 6.5882 - lr 0.3000000 2021-03-26 05:48:15,916 DEV : loss 7.316654205322266 - score 0.8808 2021-03-26 05:48:15,942 BAD EPOCHS (no improvement): 0 2021-03-26 05:48:25,531 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:48:26,515 epoch 21 - iter 2/25 - loss 6.55948019 - samples/sec: 130.53 - lr: 0.300000 2021-03-26 05:48:27,655 epoch 21 - iter 4/25 - loss 5.92072499 - samples/sec: 112.40 - lr: 0.300000 2021-03-26 05:48:28,666 epoch 21 - iter 6/25 - loss 5.60158626 - samples/sec: 126.75 - lr: 0.300000 2021-03-26 05:48:29,621 epoch 21 - iter 8/25 - loss 5.78064823 - samples/sec: 134.21 - lr: 0.300000 2021-03-26 05:48:30,595 epoch 21 - iter 10/25 - loss 5.86341887 - samples/sec: 131.69 - lr: 0.300000 2021-03-26 05:48:31,640 epoch 21 - iter 12/25 - loss 5.97549959 - samples/sec: 122.62 - lr: 0.300000 2021-03-26 05:48:32,697 epoch 21 - iter 14/25 - loss 6.21892132 - samples/sec: 121.24 - lr: 0.300000 2021-03-26 05:48:33,902 epoch 21 - iter 16/25 - loss 6.26286507 - samples/sec: 106.35 - lr: 0.300000 2021-03-26 05:48:35,248 epoch 21 - iter 18/25 - loss 6.13798594 - samples/sec: 95.30 - lr: 0.300000 2021-03-26 05:48:36,248 epoch 21 - iter 20/25 - loss 6.16988015 - samples/sec: 128.15 - lr: 0.300000 2021-03-26 05:48:37,287 epoch 21 - iter 22/25 - loss 6.20285286 - samples/sec: 123.43 - lr: 0.300000 2021-03-26 05:48:38,356 epoch 21 - iter 24/25 - loss 6.24573507 - samples/sec: 119.83 - lr: 0.300000 2021-03-26 05:48:38,960 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:48:38,961 EPOCH 21 done: loss 6.2932 - lr 0.3000000 2021-03-26 05:48:39,802 DEV : loss 7.598691940307617 - score 0.8726 2021-03-26 05:48:39,837 BAD EPOCHS (no improvement): 1 2021-03-26 05:48:39,838 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:48:41,253 epoch 22 - iter 2/25 - loss 5.48124957 - samples/sec: 90.58 - lr: 0.300000 2021-03-26 05:48:42,563 epoch 22 - iter 4/25 - loss 6.06442177 - samples/sec: 97.80 - lr: 0.300000 2021-03-26 05:48:43,863 epoch 22 - iter 6/25 - loss 5.93726675 - samples/sec: 98.52 - lr: 0.300000 2021-03-26 05:48:45,364 epoch 22 - iter 8/25 - loss 6.18622816 - samples/sec: 85.58 - lr: 0.300000 2021-03-26 05:48:46,774 epoch 22 - iter 10/25 - loss 6.23740492 - samples/sec: 90.87 - lr: 0.300000 2021-03-26 05:48:47,923 epoch 22 - iter 12/25 - loss 6.31357996 - samples/sec: 111.53 - lr: 0.300000 2021-03-26 05:48:48,873 epoch 22 - iter 14/25 - loss 6.24927248 - samples/sec: 134.88 - lr: 0.300000 2021-03-26 05:48:49,861 epoch 22 - iter 16/25 - loss 6.27198809 - samples/sec: 129.93 - lr: 0.300000 2021-03-26 05:48:50,877 epoch 22 - iter 18/25 - loss 6.30164162 - samples/sec: 126.04 - lr: 0.300000 2021-03-26 05:48:51,869 epoch 22 - iter 20/25 - loss 6.28241801 - samples/sec: 129.22 - lr: 0.300000 2021-03-26 05:48:52,862 epoch 22 - iter 22/25 - loss 6.31069586 - samples/sec: 129.24 - lr: 0.300000 2021-03-26 05:48:53,961 epoch 22 - iter 24/25 - loss 6.28890721 - samples/sec: 116.59 - lr: 0.300000 2021-03-26 05:48:54,462 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:48:54,463 EPOCH 22 done: loss 6.3392 - lr 0.3000000 2021-03-26 05:48:55,284 DEV : loss 7.424391746520996 - score 0.8802 2021-03-26 05:48:55,315 BAD EPOCHS (no improvement): 2 2021-03-26 05:48:55,315 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:48:56,413 epoch 23 - iter 2/25 - loss 7.48641682 - samples/sec: 116.85 - lr: 0.300000 2021-03-26 05:48:57,395 epoch 23 - iter 4/25 - loss 6.89555621 - samples/sec: 130.61 - lr: 0.300000 2021-03-26 05:48:58,460 epoch 23 - iter 6/25 - loss 6.57134056 - samples/sec: 120.40 - lr: 0.300000 2021-03-26 05:48:59,660 epoch 23 - iter 8/25 - loss 6.16241872 - samples/sec: 106.79 - lr: 0.300000 2021-03-26 05:49:00,733 epoch 23 - iter 10/25 - loss 6.01301031 - samples/sec: 119.43 - lr: 0.300000 2021-03-26 05:49:01,727 epoch 23 - iter 12/25 - loss 5.96942111 - samples/sec: 129.08 - lr: 0.300000 2021-03-26 05:49:02,695 epoch 23 - iter 14/25 - loss 5.98979133 - samples/sec: 132.53 - lr: 0.300000 2021-03-26 05:49:03,756 epoch 23 - iter 16/25 - loss 5.93200183 - samples/sec: 120.81 - lr: 0.300000 2021-03-26 05:49:04,735 epoch 23 - iter 18/25 - loss 5.90244452 - samples/sec: 130.93 - lr: 0.300000 2021-03-26 05:49:05,761 epoch 23 - iter 20/25 - loss 5.99406066 - samples/sec: 124.90 - lr: 0.300000 2021-03-26 05:49:06,817 epoch 23 - iter 22/25 - loss 6.01188855 - samples/sec: 121.40 - lr: 0.300000 2021-03-26 05:49:07,786 epoch 23 - iter 24/25 - loss 6.00390444 - samples/sec: 132.41 - lr: 0.300000 2021-03-26 05:49:08,255 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:49:08,256 EPOCH 23 done: loss 6.0797 - lr 0.3000000 2021-03-26 05:49:09,056 DEV : loss 7.356023788452148 - score 0.883 2021-03-26 05:49:09,075 BAD EPOCHS (no improvement): 0 2021-03-26 05:49:18,746 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:49:19,866 epoch 24 - iter 2/25 - loss 5.56664753 - samples/sec: 114.59 - lr: 0.300000 2021-03-26 05:49:20,854 epoch 24 - iter 4/25 - loss 5.72063935 - samples/sec: 129.66 - lr: 0.300000 2021-03-26 05:49:21,812 epoch 24 - iter 6/25 - loss 5.76620873 - samples/sec: 133.94 - lr: 0.300000 2021-03-26 05:49:23,020 epoch 24 - iter 8/25 - loss 5.84735155 - samples/sec: 106.04 - lr: 0.300000 2021-03-26 05:49:24,401 epoch 24 - iter 10/25 - loss 5.96158013 - samples/sec: 92.83 - lr: 0.300000 2021-03-26 05:49:25,393 epoch 24 - iter 12/25 - loss 5.80328266 - samples/sec: 129.21 - lr: 0.300000 2021-03-26 05:49:26,386 epoch 24 - iter 14/25 - loss 5.73408553 - samples/sec: 129.11 - lr: 0.300000 2021-03-26 05:49:27,388 epoch 24 - iter 16/25 - loss 5.67245895 - samples/sec: 127.98 - lr: 0.300000 2021-03-26 05:49:28,392 epoch 24 - iter 18/25 - loss 5.72128481 - samples/sec: 127.61 - lr: 0.300000 2021-03-26 05:49:29,443 epoch 24 - iter 20/25 - loss 5.75462203 - samples/sec: 121.95 - lr: 0.300000 2021-03-26 05:49:30,462 epoch 24 - iter 22/25 - loss 5.76823217 - samples/sec: 125.81 - lr: 0.300000 2021-03-26 05:49:31,465 epoch 24 - iter 24/25 - loss 5.79069185 - samples/sec: 127.82 - lr: 0.300000 2021-03-26 05:49:31,910 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:49:31,911 EPOCH 24 done: loss 5.8127 - lr 0.3000000 2021-03-26 05:49:32,690 DEV : loss 7.419638633728027 - score 0.8864 2021-03-26 05:49:32,715 BAD EPOCHS (no improvement): 0 2021-03-26 05:49:42,741 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:49:43,746 epoch 25 - iter 2/25 - loss 6.35212374 - samples/sec: 127.64 - lr: 0.300000 2021-03-26 05:49:44,824 epoch 25 - iter 4/25 - loss 5.96967781 - samples/sec: 118.94 - lr: 0.300000 2021-03-26 05:49:45,725 epoch 25 - iter 6/25 - loss 6.14810308 - samples/sec: 142.53 - lr: 0.300000 2021-03-26 05:49:46,767 epoch 25 - iter 8/25 - loss 6.18986714 - samples/sec: 122.95 - lr: 0.300000 2021-03-26 05:49:47,734 epoch 25 - iter 10/25 - loss 6.18832707 - samples/sec: 132.51 - lr: 0.300000 2021-03-26 05:49:48,665 epoch 25 - iter 12/25 - loss 6.09619081 - samples/sec: 137.72 - lr: 0.300000 2021-03-26 05:49:49,794 epoch 25 - iter 14/25 - loss 5.98943094 - samples/sec: 113.51 - lr: 0.300000 2021-03-26 05:49:50,774 epoch 25 - iter 16/25 - loss 5.93080184 - samples/sec: 130.90 - lr: 0.300000 2021-03-26 05:49:51,814 epoch 25 - iter 18/25 - loss 5.93358392 - samples/sec: 123.27 - lr: 0.300000 2021-03-26 05:49:52,891 epoch 25 - iter 20/25 - loss 5.89371572 - samples/sec: 118.95 - lr: 0.300000 2021-03-26 05:49:53,987 epoch 25 - iter 22/25 - loss 5.87030755 - samples/sec: 116.98 - lr: 0.300000 2021-03-26 05:49:55,301 epoch 25 - iter 24/25 - loss 5.85457329 - samples/sec: 97.58 - lr: 0.300000 2021-03-26 05:49:55,677 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:49:55,678 EPOCH 25 done: loss 5.7638 - lr 0.3000000 2021-03-26 05:49:56,482 DEV : loss 7.5117692947387695 - score 0.8825 2021-03-26 05:49:56,508 BAD EPOCHS (no improvement): 1 2021-03-26 05:49:56,508 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:49:57,657 epoch 26 - iter 2/25 - loss 5.19982338 - samples/sec: 111.59 - lr: 0.300000 2021-03-26 05:49:58,815 epoch 26 - iter 4/25 - loss 4.92103577 - samples/sec: 110.68 - lr: 0.300000 2021-03-26 05:49:59,991 epoch 26 - iter 6/25 - loss 5.07783246 - samples/sec: 109.06 - lr: 0.300000 2021-03-26 05:50:01,047 epoch 26 - iter 8/25 - loss 5.18258464 - samples/sec: 122.00 - lr: 0.300000 2021-03-26 05:50:02,294 epoch 26 - iter 10/25 - loss 5.27087030 - samples/sec: 102.81 - lr: 0.300000 2021-03-26 05:50:03,302 epoch 26 - iter 12/25 - loss 5.23783867 - samples/sec: 127.23 - lr: 0.300000 2021-03-26 05:50:04,382 epoch 26 - iter 14/25 - loss 5.33069127 - samples/sec: 118.57 - lr: 0.300000 2021-03-26 05:50:05,358 epoch 26 - iter 16/25 - loss 5.30787414 - samples/sec: 131.42 - lr: 0.300000 2021-03-26 05:50:06,347 epoch 26 - iter 18/25 - loss 5.30061195 - samples/sec: 129.55 - lr: 0.300000 2021-03-26 05:50:07,295 epoch 26 - iter 20/25 - loss 5.31607125 - samples/sec: 135.22 - lr: 0.300000 2021-03-26 05:50:08,469 epoch 26 - iter 22/25 - loss 5.24197604 - samples/sec: 109.23 - lr: 0.300000 2021-03-26 05:50:09,498 epoch 26 - iter 24/25 - loss 5.32497180 - samples/sec: 124.55 - lr: 0.300000 2021-03-26 05:50:09,926 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:50:09,927 EPOCH 26 done: loss 5.2962 - lr 0.3000000 2021-03-26 05:50:10,686 DEV : loss 7.333011150360107 - score 0.8881 2021-03-26 05:50:10,710 BAD EPOCHS (no improvement): 0 2021-03-26 05:50:20,333 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:50:21,431 epoch 27 - iter 2/25 - loss 5.16186333 - samples/sec: 116.78 - lr: 0.300000 2021-03-26 05:50:22,464 epoch 27 - iter 4/25 - loss 5.12837744 - samples/sec: 124.20 - lr: 0.300000 2021-03-26 05:50:23,572 epoch 27 - iter 6/25 - loss 5.31019672 - samples/sec: 115.61 - lr: 0.300000 2021-03-26 05:50:24,600 epoch 27 - iter 8/25 - loss 5.32734174 - samples/sec: 124.69 - lr: 0.300000 2021-03-26 05:50:25,613 epoch 27 - iter 10/25 - loss 5.22059751 - samples/sec: 126.63 - lr: 0.300000 2021-03-26 05:50:26,594 epoch 27 - iter 12/25 - loss 5.27950998 - samples/sec: 130.75 - lr: 0.300000 2021-03-26 05:50:27,597 epoch 27 - iter 14/25 - loss 5.41089095 - samples/sec: 127.69 - lr: 0.300000 2021-03-26 05:50:28,609 epoch 27 - iter 16/25 - loss 5.40781000 - samples/sec: 126.67 - lr: 0.300000 2021-03-26 05:50:29,656 epoch 27 - iter 18/25 - loss 5.39746579 - samples/sec: 122.47 - lr: 0.300000 2021-03-26 05:50:30,589 epoch 27 - iter 20/25 - loss 5.43662884 - samples/sec: 137.43 - lr: 0.300000 2021-03-26 05:50:31,638 epoch 27 - iter 22/25 - loss 5.37713861 - samples/sec: 122.16 - lr: 0.300000 2021-03-26 05:50:32,594 epoch 27 - iter 24/25 - loss 5.36295988 - samples/sec: 134.30 - lr: 0.300000 2021-03-26 05:50:32,996 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:50:32,996 EPOCH 27 done: loss 5.3616 - lr 0.3000000 2021-03-26 05:50:33,787 DEV : loss 7.019782066345215 - score 0.8849 2021-03-26 05:50:33,817 BAD EPOCHS (no improvement): 1 2021-03-26 05:50:33,817 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:50:34,879 epoch 28 - iter 2/25 - loss 5.07017589 - samples/sec: 120.77 - lr: 0.300000 2021-03-26 05:50:36,041 epoch 28 - iter 4/25 - loss 5.02670932 - samples/sec: 110.30 - lr: 0.300000 2021-03-26 05:50:37,066 epoch 28 - iter 6/25 - loss 4.80902608 - samples/sec: 125.02 - lr: 0.300000 2021-03-26 05:50:38,124 epoch 28 - iter 8/25 - loss 4.62447375 - samples/sec: 121.11 - lr: 0.300000 2021-03-26 05:50:39,335 epoch 28 - iter 10/25 - loss 4.67827401 - samples/sec: 105.87 - lr: 0.300000 2021-03-26 05:50:40,588 epoch 28 - iter 12/25 - loss 4.77806664 - samples/sec: 102.26 - lr: 0.300000 2021-03-26 05:50:41,721 epoch 28 - iter 14/25 - loss 4.72924319 - samples/sec: 113.66 - lr: 0.300000 2021-03-26 05:50:42,844 epoch 28 - iter 16/25 - loss 4.71131511 - samples/sec: 114.18 - lr: 0.300000 2021-03-26 05:50:43,847 epoch 28 - iter 18/25 - loss 4.68717831 - samples/sec: 127.83 - lr: 0.300000 2021-03-26 05:50:45,010 epoch 28 - iter 20/25 - loss 4.77009903 - samples/sec: 110.20 - lr: 0.300000 2021-03-26 05:50:46,075 epoch 28 - iter 22/25 - loss 4.80764932 - samples/sec: 120.32 - lr: 0.300000 2021-03-26 05:50:47,113 epoch 28 - iter 24/25 - loss 4.90641921 - samples/sec: 123.50 - lr: 0.300000 2021-03-26 05:50:47,557 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:50:47,557 EPOCH 28 done: loss 4.9966 - lr 0.3000000 2021-03-26 05:50:48,339 DEV : loss 7.182887077331543 - score 0.8918 2021-03-26 05:50:48,364 BAD EPOCHS (no improvement): 0 2021-03-26 05:50:58,235 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:50:59,334 epoch 29 - iter 2/25 - loss 4.95724106 - samples/sec: 116.78 - lr: 0.300000 2021-03-26 05:51:00,317 epoch 29 - iter 4/25 - loss 4.68853933 - samples/sec: 130.36 - lr: 0.300000 2021-03-26 05:51:01,279 epoch 29 - iter 6/25 - loss 4.66039519 - samples/sec: 133.25 - lr: 0.300000 2021-03-26 05:51:02,244 epoch 29 - iter 8/25 - loss 4.51463768 - samples/sec: 132.87 - lr: 0.300000 2021-03-26 05:51:03,327 epoch 29 - iter 10/25 - loss 4.73710887 - samples/sec: 118.37 - lr: 0.300000 2021-03-26 05:51:04,504 epoch 29 - iter 12/25 - loss 4.68225382 - samples/sec: 108.91 - lr: 0.300000 2021-03-26 05:51:05,503 epoch 29 - iter 14/25 - loss 4.56697776 - samples/sec: 128.41 - lr: 0.300000 2021-03-26 05:51:06,493 epoch 29 - iter 16/25 - loss 4.53189199 - samples/sec: 129.70 - lr: 0.300000 2021-03-26 05:51:07,506 epoch 29 - iter 18/25 - loss 4.68264200 - samples/sec: 127.20 - lr: 0.300000 2021-03-26 05:51:08,637 epoch 29 - iter 20/25 - loss 4.78868047 - samples/sec: 113.21 - lr: 0.300000 2021-03-26 05:51:09,667 epoch 29 - iter 22/25 - loss 4.92078643 - samples/sec: 124.51 - lr: 0.300000 2021-03-26 05:51:10,900 epoch 29 - iter 24/25 - loss 4.98505587 - samples/sec: 103.91 - lr: 0.300000 2021-03-26 05:51:11,284 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:51:11,285 EPOCH 29 done: loss 5.0067 - lr 0.3000000 2021-03-26 05:51:12,098 DEV : loss 7.243838310241699 - score 0.8918 2021-03-26 05:51:12,128 BAD EPOCHS (no improvement): 1 2021-03-26 05:51:12,129 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:51:13,162 epoch 30 - iter 2/25 - loss 4.69986725 - samples/sec: 124.15 - lr: 0.300000 2021-03-26 05:51:14,296 epoch 30 - iter 4/25 - loss 4.84692991 - samples/sec: 112.95 - lr: 0.300000 2021-03-26 05:51:15,280 epoch 30 - iter 6/25 - loss 4.66442370 - samples/sec: 130.35 - lr: 0.300000 2021-03-26 05:51:16,392 epoch 30 - iter 8/25 - loss 4.60726658 - samples/sec: 115.31 - lr: 0.300000 2021-03-26 05:51:17,411 epoch 30 - iter 10/25 - loss 4.60311129 - samples/sec: 125.83 - lr: 0.300000 2021-03-26 05:51:18,448 epoch 30 - iter 12/25 - loss 4.72527681 - samples/sec: 123.73 - lr: 0.300000 2021-03-26 05:51:19,475 epoch 30 - iter 14/25 - loss 4.62729422 - samples/sec: 124.77 - lr: 0.300000 2021-03-26 05:51:21,828 epoch 30 - iter 16/25 - loss 4.68846346 - samples/sec: 54.46 - lr: 0.300000 2021-03-26 05:51:22,903 epoch 30 - iter 18/25 - loss 4.74469176 - samples/sec: 119.27 - lr: 0.300000 2021-03-26 05:51:23,885 epoch 30 - iter 20/25 - loss 4.70663766 - samples/sec: 130.50 - lr: 0.300000 2021-03-26 05:51:25,067 epoch 30 - iter 22/25 - loss 4.72499332 - samples/sec: 108.44 - lr: 0.300000 2021-03-26 05:51:26,152 epoch 30 - iter 24/25 - loss 4.69499653 - samples/sec: 118.15 - lr: 0.300000 2021-03-26 05:51:26,532 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:51:26,532 EPOCH 30 done: loss 4.6847 - lr 0.3000000 2021-03-26 05:51:27,342 DEV : loss 7.096852779388428 - score 0.8954 2021-03-26 05:51:27,365 BAD EPOCHS (no improvement): 0 2021-03-26 05:51:37,234 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:51:38,257 epoch 31 - iter 2/25 - loss 4.87494373 - samples/sec: 125.43 - lr: 0.300000 2021-03-26 05:51:39,293 epoch 31 - iter 4/25 - loss 4.54467654 - samples/sec: 123.83 - lr: 0.300000 2021-03-26 05:51:40,261 epoch 31 - iter 6/25 - loss 4.44595257 - samples/sec: 132.41 - lr: 0.300000 2021-03-26 05:51:41,212 epoch 31 - iter 8/25 - loss 4.43987417 - samples/sec: 134.97 - lr: 0.300000 2021-03-26 05:51:42,199 epoch 31 - iter 10/25 - loss 4.30324354 - samples/sec: 129.78 - lr: 0.300000 2021-03-26 05:51:43,255 epoch 31 - iter 12/25 - loss 4.50878437 - samples/sec: 121.38 - lr: 0.300000 2021-03-26 05:51:44,323 epoch 31 - iter 14/25 - loss 4.55510497 - samples/sec: 120.11 - lr: 0.300000 2021-03-26 05:51:45,461 epoch 31 - iter 16/25 - loss 4.60535118 - samples/sec: 112.62 - lr: 0.300000 2021-03-26 05:51:46,501 epoch 31 - iter 18/25 - loss 4.64036245 - samples/sec: 123.17 - lr: 0.300000 2021-03-26 05:51:47,513 epoch 31 - iter 20/25 - loss 4.69187005 - samples/sec: 126.69 - lr: 0.300000 2021-03-26 05:51:48,481 epoch 31 - iter 22/25 - loss 4.67161454 - samples/sec: 132.40 - lr: 0.300000 2021-03-26 05:51:49,496 epoch 31 - iter 24/25 - loss 4.69764324 - samples/sec: 126.34 - lr: 0.300000 2021-03-26 05:51:49,914 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:51:49,915 EPOCH 31 done: loss 4.7288 - lr 0.3000000 2021-03-26 05:51:50,690 DEV : loss 6.818641662597656 - score 0.8986 2021-03-26 05:51:50,716 BAD EPOCHS (no improvement): 0 2021-03-26 05:52:00,339 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:52:01,491 epoch 32 - iter 2/25 - loss 4.50873399 - samples/sec: 111.29 - lr: 0.300000 2021-03-26 05:52:02,534 epoch 32 - iter 4/25 - loss 4.65008414 - samples/sec: 123.03 - lr: 0.300000 2021-03-26 05:52:03,521 epoch 32 - iter 6/25 - loss 4.54596925 - samples/sec: 129.96 - lr: 0.300000 2021-03-26 05:52:04,630 epoch 32 - iter 8/25 - loss 4.67705244 - samples/sec: 115.53 - lr: 0.300000 2021-03-26 05:52:05,581 epoch 32 - iter 10/25 - loss 4.58810802 - samples/sec: 134.90 - lr: 0.300000 2021-03-26 05:52:06,535 epoch 32 - iter 12/25 - loss 4.48243205 - samples/sec: 134.26 - lr: 0.300000 2021-03-26 05:52:07,522 epoch 32 - iter 14/25 - loss 4.34981622 - samples/sec: 130.01 - lr: 0.300000 2021-03-26 05:52:08,685 epoch 32 - iter 16/25 - loss 4.30454047 - samples/sec: 110.29 - lr: 0.300000 2021-03-26 05:52:09,695 epoch 32 - iter 18/25 - loss 4.40030144 - samples/sec: 127.12 - lr: 0.300000 2021-03-26 05:52:10,720 epoch 32 - iter 20/25 - loss 4.42493467 - samples/sec: 125.11 - lr: 0.300000 2021-03-26 05:52:11,691 epoch 32 - iter 22/25 - loss 4.41666954 - samples/sec: 132.00 - lr: 0.300000 2021-03-26 05:52:12,622 epoch 32 - iter 24/25 - loss 4.46280225 - samples/sec: 137.75 - lr: 0.300000 2021-03-26 05:52:13,004 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:52:13,005 EPOCH 32 done: loss 4.4738 - lr 0.3000000 2021-03-26 05:52:13,773 DEV : loss 6.889939308166504 - score 0.8994 2021-03-26 05:52:13,798 BAD EPOCHS (no improvement): 0 2021-03-26 05:52:23,499 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:52:24,515 epoch 33 - iter 2/25 - loss 4.49444866 - samples/sec: 126.26 - lr: 0.300000 2021-03-26 05:52:25,520 epoch 33 - iter 4/25 - loss 4.15728247 - samples/sec: 127.56 - lr: 0.300000 2021-03-26 05:52:26,490 epoch 33 - iter 6/25 - loss 4.49758712 - samples/sec: 132.10 - lr: 0.300000 2021-03-26 05:52:27,420 epoch 33 - iter 8/25 - loss 4.29979959 - samples/sec: 137.95 - lr: 0.300000 2021-03-26 05:52:28,443 epoch 33 - iter 10/25 - loss 4.28199461 - samples/sec: 125.34 - lr: 0.300000 2021-03-26 05:52:29,503 epoch 33 - iter 12/25 - loss 4.25882107 - samples/sec: 120.95 - lr: 0.300000 2021-03-26 05:52:30,482 epoch 33 - iter 14/25 - loss 4.39515034 - samples/sec: 131.00 - lr: 0.300000 2021-03-26 05:52:31,428 epoch 33 - iter 16/25 - loss 4.38297199 - samples/sec: 135.72 - lr: 0.300000 2021-03-26 05:52:32,521 epoch 33 - iter 18/25 - loss 4.39067970 - samples/sec: 117.24 - lr: 0.300000 2021-03-26 05:52:33,692 epoch 33 - iter 20/25 - loss 4.40039575 - samples/sec: 109.58 - lr: 0.300000 2021-03-26 05:52:34,733 epoch 33 - iter 22/25 - loss 4.41531073 - samples/sec: 123.12 - lr: 0.300000 2021-03-26 05:52:35,734 epoch 33 - iter 24/25 - loss 4.43675790 - samples/sec: 128.11 - lr: 0.300000 2021-03-26 05:52:36,210 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:52:36,212 EPOCH 33 done: loss 4.4681 - lr 0.3000000 2021-03-26 05:52:36,990 DEV : loss 6.704212188720703 - score 0.899 2021-03-26 05:52:37,016 BAD EPOCHS (no improvement): 1 2021-03-26 05:52:37,017 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:52:38,049 epoch 34 - iter 2/25 - loss 4.85452342 - samples/sec: 124.26 - lr: 0.300000 2021-03-26 05:52:39,022 epoch 34 - iter 4/25 - loss 4.34141463 - samples/sec: 131.71 - lr: 0.300000 2021-03-26 05:52:40,061 epoch 34 - iter 6/25 - loss 4.44528925 - samples/sec: 123.40 - lr: 0.300000 2021-03-26 05:52:41,019 epoch 34 - iter 8/25 - loss 4.39014325 - samples/sec: 133.80 - lr: 0.300000 2021-03-26 05:52:42,082 epoch 34 - iter 10/25 - loss 4.40972173 - samples/sec: 120.60 - lr: 0.300000 2021-03-26 05:52:43,066 epoch 34 - iter 12/25 - loss 4.44770990 - samples/sec: 130.30 - lr: 0.300000 2021-03-26 05:52:44,164 epoch 34 - iter 14/25 - loss 4.42394963 - samples/sec: 116.79 - lr: 0.300000 2021-03-26 05:52:45,102 epoch 34 - iter 16/25 - loss 4.28590170 - samples/sec: 136.59 - lr: 0.300000 2021-03-26 05:52:46,081 epoch 34 - iter 18/25 - loss 4.33693907 - samples/sec: 130.91 - lr: 0.300000 2021-03-26 05:52:47,060 epoch 34 - iter 20/25 - loss 4.30833759 - samples/sec: 131.04 - lr: 0.300000 2021-03-26 05:52:48,102 epoch 34 - iter 22/25 - loss 4.32592613 - samples/sec: 123.03 - lr: 0.300000 2021-03-26 05:52:49,156 epoch 34 - iter 24/25 - loss 4.35851695 - samples/sec: 121.67 - lr: 0.300000 2021-03-26 05:52:49,577 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:52:49,577 EPOCH 34 done: loss 4.3686 - lr 0.3000000 2021-03-26 05:52:50,360 DEV : loss 6.8909149169921875 - score 0.9 2021-03-26 05:52:50,378 BAD EPOCHS (no improvement): 0 2021-03-26 05:52:59,953 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:53:00,923 epoch 35 - iter 2/25 - loss 4.03293800 - samples/sec: 132.31 - lr: 0.300000 2021-03-26 05:53:01,966 epoch 35 - iter 4/25 - loss 4.25911993 - samples/sec: 123.01 - lr: 0.300000 2021-03-26 05:53:02,931 epoch 35 - iter 6/25 - loss 4.52007488 - samples/sec: 132.70 - lr: 0.300000 2021-03-26 05:53:03,865 epoch 35 - iter 8/25 - loss 4.61368671 - samples/sec: 137.34 - lr: 0.300000 2021-03-26 05:53:04,867 epoch 35 - iter 10/25 - loss 4.53169215 - samples/sec: 127.88 - lr: 0.300000 2021-03-26 05:53:05,865 epoch 35 - iter 12/25 - loss 4.48455348 - samples/sec: 128.38 - lr: 0.300000 2021-03-26 05:53:06,842 epoch 35 - iter 14/25 - loss 4.45766437 - samples/sec: 131.18 - lr: 0.300000 2021-03-26 05:53:07,998 epoch 35 - iter 16/25 - loss 4.30519691 - samples/sec: 110.88 - lr: 0.300000 2021-03-26 05:53:09,055 epoch 35 - iter 18/25 - loss 4.27180097 - samples/sec: 121.38 - lr: 0.300000 2021-03-26 05:53:10,090 epoch 35 - iter 20/25 - loss 4.19139869 - samples/sec: 123.76 - lr: 0.300000 2021-03-26 05:53:11,007 epoch 35 - iter 22/25 - loss 4.17696692 - samples/sec: 139.89 - lr: 0.300000 2021-03-26 05:53:11,997 epoch 35 - iter 24/25 - loss 4.14253498 - samples/sec: 129.48 - lr: 0.300000 2021-03-26 05:53:12,410 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:53:12,410 EPOCH 35 done: loss 4.1496 - lr 0.3000000 2021-03-26 05:53:13,190 DEV : loss 6.916918754577637 - score 0.8928 2021-03-26 05:53:13,215 BAD EPOCHS (no improvement): 1 2021-03-26 05:53:13,216 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:53:14,245 epoch 36 - iter 2/25 - loss 3.75446880 - samples/sec: 124.71 - lr: 0.300000 2021-03-26 05:53:15,430 epoch 36 - iter 4/25 - loss 4.07209581 - samples/sec: 108.10 - lr: 0.300000 2021-03-26 05:53:16,350 epoch 36 - iter 6/25 - loss 4.02450681 - samples/sec: 139.43 - lr: 0.300000 2021-03-26 05:53:17,365 epoch 36 - iter 8/25 - loss 3.84632128 - samples/sec: 126.28 - lr: 0.300000 2021-03-26 05:53:18,361 epoch 36 - iter 10/25 - loss 3.93511500 - samples/sec: 128.67 - lr: 0.300000 2021-03-26 05:53:19,307 epoch 36 - iter 12/25 - loss 3.98764678 - samples/sec: 135.51 - lr: 0.300000 2021-03-26 05:53:20,246 epoch 36 - iter 14/25 - loss 4.08523498 - samples/sec: 136.38 - lr: 0.300000 2021-03-26 05:53:21,269 epoch 36 - iter 16/25 - loss 4.14107890 - samples/sec: 125.35 - lr: 0.300000 2021-03-26 05:53:22,221 epoch 36 - iter 18/25 - loss 4.20975999 - samples/sec: 134.58 - lr: 0.300000 2021-03-26 05:53:23,149 epoch 36 - iter 20/25 - loss 4.16134720 - samples/sec: 138.34 - lr: 0.300000 2021-03-26 05:53:24,232 epoch 36 - iter 22/25 - loss 4.14724630 - samples/sec: 118.33 - lr: 0.300000 2021-03-26 05:53:25,199 epoch 36 - iter 24/25 - loss 4.13445437 - samples/sec: 132.58 - lr: 0.300000 2021-03-26 05:53:25,669 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:53:25,670 EPOCH 36 done: loss 4.1594 - lr 0.3000000 2021-03-26 05:53:26,453 DEV : loss 6.872126579284668 - score 0.9006 2021-03-26 05:53:26,471 BAD EPOCHS (no improvement): 0 2021-03-26 05:53:35,962 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:53:37,012 epoch 37 - iter 2/25 - loss 4.32914829 - samples/sec: 122.13 - lr: 0.300000 2021-03-26 05:53:38,126 epoch 37 - iter 4/25 - loss 3.83761263 - samples/sec: 115.03 - lr: 0.300000 2021-03-26 05:53:39,136 epoch 37 - iter 6/25 - loss 3.83161823 - samples/sec: 126.89 - lr: 0.300000 2021-03-26 05:53:40,095 epoch 37 - iter 8/25 - loss 3.91476762 - samples/sec: 133.59 - lr: 0.300000 2021-03-26 05:53:41,096 epoch 37 - iter 10/25 - loss 4.02498407 - samples/sec: 128.16 - lr: 0.300000 2021-03-26 05:53:42,295 epoch 37 - iter 12/25 - loss 3.99303458 - samples/sec: 106.84 - lr: 0.300000 2021-03-26 05:53:43,421 epoch 37 - iter 14/25 - loss 4.16374413 - samples/sec: 113.83 - lr: 0.300000 2021-03-26 05:53:44,493 epoch 37 - iter 16/25 - loss 4.11268929 - samples/sec: 119.59 - lr: 0.300000 2021-03-26 05:53:45,482 epoch 37 - iter 18/25 - loss 4.08254796 - samples/sec: 130.27 - lr: 0.300000 2021-03-26 05:53:46,607 epoch 37 - iter 20/25 - loss 4.03820162 - samples/sec: 114.08 - lr: 0.300000 2021-03-26 05:53:47,599 epoch 37 - iter 22/25 - loss 3.96956437 - samples/sec: 129.42 - lr: 0.300000 2021-03-26 05:53:48,570 epoch 37 - iter 24/25 - loss 3.88680551 - samples/sec: 132.00 - lr: 0.300000 2021-03-26 05:53:49,025 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:53:49,026 EPOCH 37 done: loss 3.9496 - lr 0.3000000 2021-03-26 05:53:49,826 DEV : loss 6.98248815536499 - score 0.8924 2021-03-26 05:53:49,851 BAD EPOCHS (no improvement): 1 2021-03-26 05:53:49,852 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:53:50,995 epoch 38 - iter 2/25 - loss 4.03832924 - samples/sec: 112.15 - lr: 0.300000 2021-03-26 05:53:51,988 epoch 38 - iter 4/25 - loss 3.38519096 - samples/sec: 129.16 - lr: 0.300000 2021-03-26 05:53:52,950 epoch 38 - iter 6/25 - loss 3.72530087 - samples/sec: 133.17 - lr: 0.300000 2021-03-26 05:53:54,155 epoch 38 - iter 8/25 - loss 3.67750609 - samples/sec: 106.36 - lr: 0.300000 2021-03-26 05:53:55,130 epoch 38 - iter 10/25 - loss 3.67888119 - samples/sec: 131.49 - lr: 0.300000 2021-03-26 05:53:56,153 epoch 38 - iter 12/25 - loss 3.70259313 - samples/sec: 125.41 - lr: 0.300000 2021-03-26 05:53:57,202 epoch 38 - iter 14/25 - loss 3.85171386 - samples/sec: 122.12 - lr: 0.300000 2021-03-26 05:53:58,218 epoch 38 - iter 16/25 - loss 3.86089347 - samples/sec: 126.26 - lr: 0.300000 2021-03-26 05:53:59,198 epoch 38 - iter 18/25 - loss 3.88625879 - samples/sec: 130.94 - lr: 0.300000 2021-03-26 05:54:00,240 epoch 38 - iter 20/25 - loss 3.87472203 - samples/sec: 123.05 - lr: 0.300000 2021-03-26 05:54:01,183 epoch 38 - iter 22/25 - loss 3.76388594 - samples/sec: 135.94 - lr: 0.300000 2021-03-26 05:54:02,138 epoch 38 - iter 24/25 - loss 3.86877202 - samples/sec: 134.32 - lr: 0.300000 2021-03-26 05:54:02,580 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:54:02,580 EPOCH 38 done: loss 3.8287 - lr 0.3000000 2021-03-26 05:54:03,366 DEV : loss 7.446681976318359 - score 0.8976 2021-03-26 05:54:03,387 BAD EPOCHS (no improvement): 2 2021-03-26 05:54:03,388 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:54:04,454 epoch 39 - iter 2/25 - loss 3.38113582 - samples/sec: 120.22 - lr: 0.300000 2021-03-26 05:54:05,495 epoch 39 - iter 4/25 - loss 3.34563071 - samples/sec: 123.35 - lr: 0.300000 2021-03-26 05:54:06,533 epoch 39 - iter 6/25 - loss 3.38525800 - samples/sec: 123.45 - lr: 0.300000 2021-03-26 05:54:07,545 epoch 39 - iter 8/25 - loss 3.45749402 - samples/sec: 126.65 - lr: 0.300000 2021-03-26 05:54:08,568 epoch 39 - iter 10/25 - loss 3.58208604 - samples/sec: 125.35 - lr: 0.300000 2021-03-26 05:54:09,585 epoch 39 - iter 12/25 - loss 3.58137528 - samples/sec: 126.12 - lr: 0.300000 2021-03-26 05:54:10,726 epoch 39 - iter 14/25 - loss 3.57692300 - samples/sec: 112.27 - lr: 0.300000 2021-03-26 05:54:11,666 epoch 39 - iter 16/25 - loss 3.63784450 - samples/sec: 136.50 - lr: 0.300000 2021-03-26 05:54:12,659 epoch 39 - iter 18/25 - loss 3.71754699 - samples/sec: 129.00 - lr: 0.300000 2021-03-26 05:54:13,624 epoch 39 - iter 20/25 - loss 3.71790321 - samples/sec: 132.81 - lr: 0.300000 2021-03-26 05:54:14,604 epoch 39 - iter 22/25 - loss 3.71617423 - samples/sec: 130.85 - lr: 0.300000 2021-03-26 05:54:15,692 epoch 39 - iter 24/25 - loss 3.82966572 - samples/sec: 117.87 - lr: 0.300000 2021-03-26 05:54:16,136 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:54:16,136 EPOCH 39 done: loss 3.7891 - lr 0.3000000 2021-03-26 05:54:16,942 DEV : loss 6.701484680175781 - score 0.9033 2021-03-26 05:54:16,966 BAD EPOCHS (no improvement): 0 2021-03-26 05:54:26,975 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:54:28,066 epoch 40 - iter 2/25 - loss 3.69630694 - samples/sec: 117.74 - lr: 0.300000 2021-03-26 05:54:29,004 epoch 40 - iter 4/25 - loss 3.42667872 - samples/sec: 136.54 - lr: 0.300000 2021-03-26 05:54:30,129 epoch 40 - iter 6/25 - loss 3.64207637 - samples/sec: 113.97 - lr: 0.300000 2021-03-26 05:54:31,179 epoch 40 - iter 8/25 - loss 3.56710127 - samples/sec: 122.08 - lr: 0.300000 2021-03-26 05:54:32,170 epoch 40 - iter 10/25 - loss 3.51173098 - samples/sec: 129.30 - lr: 0.300000 2021-03-26 05:54:33,167 epoch 40 - iter 12/25 - loss 3.52370836 - samples/sec: 128.57 - lr: 0.300000 2021-03-26 05:54:34,201 epoch 40 - iter 14/25 - loss 3.64333616 - samples/sec: 124.09 - lr: 0.300000 2021-03-26 05:54:35,280 epoch 40 - iter 16/25 - loss 3.65618315 - samples/sec: 118.80 - lr: 0.300000 2021-03-26 05:54:36,367 epoch 40 - iter 18/25 - loss 3.66785228 - samples/sec: 118.29 - lr: 0.300000 2021-03-26 05:54:37,362 epoch 40 - iter 20/25 - loss 3.67650574 - samples/sec: 128.92 - lr: 0.300000 2021-03-26 05:54:38,307 epoch 40 - iter 22/25 - loss 3.73330291 - samples/sec: 135.55 - lr: 0.300000 2021-03-26 05:54:39,254 epoch 40 - iter 24/25 - loss 3.69615401 - samples/sec: 135.49 - lr: 0.300000 2021-03-26 05:54:39,667 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:54:39,667 EPOCH 40 done: loss 3.7016 - lr 0.3000000 2021-03-26 05:54:40,457 DEV : loss 6.90902853012085 - score 0.8994 2021-03-26 05:54:40,475 BAD EPOCHS (no improvement): 1 2021-03-26 05:54:40,476 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:54:41,494 epoch 41 - iter 2/25 - loss 3.64263535 - samples/sec: 125.98 - lr: 0.300000 2021-03-26 05:54:42,432 epoch 41 - iter 4/25 - loss 3.54664224 - samples/sec: 136.59 - lr: 0.300000 2021-03-26 05:54:43,418 epoch 41 - iter 6/25 - loss 3.35265764 - samples/sec: 130.06 - lr: 0.300000 2021-03-26 05:54:44,393 epoch 41 - iter 8/25 - loss 3.50885856 - samples/sec: 131.48 - lr: 0.300000 2021-03-26 05:54:45,393 epoch 41 - iter 10/25 - loss 3.50825651 - samples/sec: 128.29 - lr: 0.300000 2021-03-26 05:54:46,391 epoch 41 - iter 12/25 - loss 3.50486132 - samples/sec: 128.37 - lr: 0.300000 2021-03-26 05:54:47,467 epoch 41 - iter 14/25 - loss 3.58481785 - samples/sec: 119.20 - lr: 0.300000 2021-03-26 05:54:48,393 epoch 41 - iter 16/25 - loss 3.61369254 - samples/sec: 138.41 - lr: 0.300000 2021-03-26 05:54:49,453 epoch 41 - iter 18/25 - loss 3.66834854 - samples/sec: 120.89 - lr: 0.300000 2021-03-26 05:54:50,430 epoch 41 - iter 20/25 - loss 3.67402879 - samples/sec: 131.42 - lr: 0.300000 2021-03-26 05:54:51,403 epoch 41 - iter 22/25 - loss 3.65755617 - samples/sec: 131.61 - lr: 0.300000 2021-03-26 05:54:52,552 epoch 41 - iter 24/25 - loss 3.67933555 - samples/sec: 111.63 - lr: 0.300000 2021-03-26 05:54:52,980 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:54:52,981 EPOCH 41 done: loss 3.6777 - lr 0.3000000 2021-03-26 05:54:53,759 DEV : loss 7.189461708068848 - score 0.8944 2021-03-26 05:54:53,777 BAD EPOCHS (no improvement): 2 2021-03-26 05:54:53,778 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:54:54,731 epoch 42 - iter 2/25 - loss 3.30671835 - samples/sec: 134.50 - lr: 0.300000 2021-03-26 05:54:55,799 epoch 42 - iter 4/25 - loss 3.40699106 - samples/sec: 120.18 - lr: 0.300000 2021-03-26 05:54:56,863 epoch 42 - iter 6/25 - loss 3.40852686 - samples/sec: 120.49 - lr: 0.300000 2021-03-26 05:54:57,894 epoch 42 - iter 8/25 - loss 3.28201953 - samples/sec: 124.31 - lr: 0.300000 2021-03-26 05:54:58,942 epoch 42 - iter 10/25 - loss 3.39735591 - samples/sec: 122.35 - lr: 0.300000 2021-03-26 05:55:00,003 epoch 42 - iter 12/25 - loss 3.49690189 - samples/sec: 120.76 - lr: 0.300000 2021-03-26 05:55:01,012 epoch 42 - iter 14/25 - loss 3.53133660 - samples/sec: 127.05 - lr: 0.300000 2021-03-26 05:55:02,057 epoch 42 - iter 16/25 - loss 3.58082850 - samples/sec: 122.79 - lr: 0.300000 2021-03-26 05:55:03,084 epoch 42 - iter 18/25 - loss 3.54918623 - samples/sec: 124.72 - lr: 0.300000 2021-03-26 05:55:03,990 epoch 42 - iter 20/25 - loss 3.49929521 - samples/sec: 141.54 - lr: 0.300000 2021-03-26 05:55:04,965 epoch 42 - iter 22/25 - loss 3.52338445 - samples/sec: 131.47 - lr: 0.300000 2021-03-26 05:55:06,106 epoch 42 - iter 24/25 - loss 3.55230616 - samples/sec: 112.34 - lr: 0.300000 2021-03-26 05:55:06,499 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:55:06,500 EPOCH 42 done: loss 3.5816 - lr 0.3000000 2021-03-26 05:55:07,294 DEV : loss 7.039179801940918 - score 0.8952 2021-03-26 05:55:07,320 BAD EPOCHS (no improvement): 3 2021-03-26 05:55:07,320 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:55:08,321 epoch 43 - iter 2/25 - loss 4.29221988 - samples/sec: 128.18 - lr: 0.300000 2021-03-26 05:55:09,370 epoch 43 - iter 4/25 - loss 3.51265657 - samples/sec: 122.15 - lr: 0.300000 2021-03-26 05:55:10,306 epoch 43 - iter 6/25 - loss 3.35663565 - samples/sec: 137.22 - lr: 0.300000 2021-03-26 05:55:11,343 epoch 43 - iter 8/25 - loss 3.41256848 - samples/sec: 123.64 - lr: 0.300000 2021-03-26 05:55:12,365 epoch 43 - iter 10/25 - loss 3.50655358 - samples/sec: 125.49 - lr: 0.300000 2021-03-26 05:55:13,370 epoch 43 - iter 12/25 - loss 3.55152710 - samples/sec: 127.64 - lr: 0.300000 2021-03-26 05:55:14,378 epoch 43 - iter 14/25 - loss 3.63799204 - samples/sec: 127.13 - lr: 0.300000 2021-03-26 05:55:15,412 epoch 43 - iter 16/25 - loss 3.52342936 - samples/sec: 123.97 - lr: 0.300000 2021-03-26 05:55:16,394 epoch 43 - iter 18/25 - loss 3.52740237 - samples/sec: 130.74 - lr: 0.300000 2021-03-26 05:55:17,507 epoch 43 - iter 20/25 - loss 3.50906000 - samples/sec: 115.22 - lr: 0.300000 2021-03-26 05:55:18,504 epoch 43 - iter 22/25 - loss 3.45362682 - samples/sec: 128.72 - lr: 0.300000 2021-03-26 05:55:19,522 epoch 43 - iter 24/25 - loss 3.39437827 - samples/sec: 125.84 - lr: 0.300000 2021-03-26 05:55:19,977 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:55:19,978 EPOCH 43 done: loss 3.4366 - lr 0.3000000 2021-03-26 05:55:20,754 DEV : loss 6.94519567489624 - score 0.8942 2021-03-26 05:55:20,777 BAD EPOCHS (no improvement): 4 2021-03-26 05:55:20,778 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:55:21,913 epoch 44 - iter 2/25 - loss 3.44202733 - samples/sec: 112.97 - lr: 0.150000 2021-03-26 05:55:22,894 epoch 44 - iter 4/25 - loss 3.22575986 - samples/sec: 130.56 - lr: 0.150000 2021-03-26 05:55:23,893 epoch 44 - iter 6/25 - loss 3.08975744 - samples/sec: 128.30 - lr: 0.150000 2021-03-26 05:55:24,957 epoch 44 - iter 8/25 - loss 3.07908553 - samples/sec: 120.50 - lr: 0.150000 2021-03-26 05:55:25,926 epoch 44 - iter 10/25 - loss 3.09396088 - samples/sec: 132.28 - lr: 0.150000 2021-03-26 05:55:26,981 epoch 44 - iter 12/25 - loss 3.13797561 - samples/sec: 121.52 - lr: 0.150000 2021-03-26 05:55:28,053 epoch 44 - iter 14/25 - loss 3.04971560 - samples/sec: 119.52 - lr: 0.150000 2021-03-26 05:55:29,089 epoch 44 - iter 16/25 - loss 3.02506705 - samples/sec: 123.69 - lr: 0.150000 2021-03-26 05:55:30,105 epoch 44 - iter 18/25 - loss 3.03164853 - samples/sec: 126.20 - lr: 0.150000 2021-03-26 05:55:31,151 epoch 44 - iter 20/25 - loss 2.96418314 - samples/sec: 122.63 - lr: 0.150000 2021-03-26 05:55:32,211 epoch 44 - iter 22/25 - loss 2.97567326 - samples/sec: 121.02 - lr: 0.150000 2021-03-26 05:55:33,208 epoch 44 - iter 24/25 - loss 3.00339601 - samples/sec: 128.47 - lr: 0.150000 2021-03-26 05:55:33,678 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:55:33,679 EPOCH 44 done: loss 3.0213 - lr 0.1500000 2021-03-26 05:55:34,556 DEV : loss 6.775360107421875 - score 0.9004 2021-03-26 05:55:34,580 BAD EPOCHS (no improvement): 1 2021-03-26 05:55:34,581 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:55:35,661 epoch 45 - iter 2/25 - loss 2.80927062 - samples/sec: 118.79 - lr: 0.150000 2021-03-26 05:55:36,710 epoch 45 - iter 4/25 - loss 2.83806175 - samples/sec: 122.24 - lr: 0.150000 2021-03-26 05:55:37,665 epoch 45 - iter 6/25 - loss 3.05464844 - samples/sec: 134.23 - lr: 0.150000 2021-03-26 05:55:38,787 epoch 45 - iter 8/25 - loss 3.02497697 - samples/sec: 114.24 - lr: 0.150000 2021-03-26 05:55:39,785 epoch 45 - iter 10/25 - loss 3.01038718 - samples/sec: 129.14 - lr: 0.150000 2021-03-26 05:55:40,810 epoch 45 - iter 12/25 - loss 3.01171037 - samples/sec: 125.01 - lr: 0.150000 2021-03-26 05:55:41,864 epoch 45 - iter 14/25 - loss 3.17433233 - samples/sec: 121.65 - lr: 0.150000 2021-03-26 05:55:42,834 epoch 45 - iter 16/25 - loss 3.11452146 - samples/sec: 132.06 - lr: 0.150000 2021-03-26 05:55:43,966 epoch 45 - iter 18/25 - loss 3.12591218 - samples/sec: 113.19 - lr: 0.150000 2021-03-26 05:55:45,011 epoch 45 - iter 20/25 - loss 3.17873971 - samples/sec: 122.68 - lr: 0.150000 2021-03-26 05:55:46,026 epoch 45 - iter 22/25 - loss 3.17179457 - samples/sec: 126.29 - lr: 0.150000 2021-03-26 05:55:47,002 epoch 45 - iter 24/25 - loss 3.18488333 - samples/sec: 131.43 - lr: 0.150000 2021-03-26 05:55:47,407 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:55:47,408 EPOCH 45 done: loss 3.1822 - lr 0.1500000 2021-03-26 05:55:48,211 DEV : loss 6.951686859130859 - score 0.9 2021-03-26 05:55:48,250 BAD EPOCHS (no improvement): 2 2021-03-26 05:55:48,251 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:55:49,256 epoch 46 - iter 2/25 - loss 3.24467790 - samples/sec: 127.61 - lr: 0.150000 2021-03-26 05:55:50,189 epoch 46 - iter 4/25 - loss 3.12835592 - samples/sec: 137.50 - lr: 0.150000 2021-03-26 05:55:51,315 epoch 46 - iter 6/25 - loss 3.09644337 - samples/sec: 113.96 - lr: 0.150000 2021-03-26 05:55:52,403 epoch 46 - iter 8/25 - loss 3.07437801 - samples/sec: 117.79 - lr: 0.150000 2021-03-26 05:55:53,467 epoch 46 - iter 10/25 - loss 3.15177259 - samples/sec: 120.44 - lr: 0.150000 2021-03-26 05:55:54,547 epoch 46 - iter 12/25 - loss 3.13121780 - samples/sec: 118.66 - lr: 0.150000 2021-03-26 05:55:55,598 epoch 46 - iter 14/25 - loss 3.08818018 - samples/sec: 121.96 - lr: 0.150000 2021-03-26 05:55:56,626 epoch 46 - iter 16/25 - loss 3.08784597 - samples/sec: 124.73 - lr: 0.150000 2021-03-26 05:55:57,688 epoch 46 - iter 18/25 - loss 3.09030705 - samples/sec: 120.84 - lr: 0.150000 2021-03-26 05:55:58,777 epoch 46 - iter 20/25 - loss 3.11507770 - samples/sec: 117.72 - lr: 0.150000 2021-03-26 05:55:59,779 epoch 46 - iter 22/25 - loss 3.08774398 - samples/sec: 128.02 - lr: 0.150000 2021-03-26 05:56:00,811 epoch 46 - iter 24/25 - loss 3.04504895 - samples/sec: 124.23 - lr: 0.150000 2021-03-26 05:56:01,215 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:56:01,216 EPOCH 46 done: loss 3.0049 - lr 0.1500000 2021-03-26 05:56:02,006 DEV : loss 6.751981258392334 - score 0.9 2021-03-26 05:56:02,031 BAD EPOCHS (no improvement): 3 2021-03-26 05:56:02,032 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:56:02,968 epoch 47 - iter 2/25 - loss 2.74569619 - samples/sec: 137.08 - lr: 0.150000 2021-03-26 05:56:04,009 epoch 47 - iter 4/25 - loss 3.09366006 - samples/sec: 123.16 - lr: 0.150000 2021-03-26 05:56:05,056 epoch 47 - iter 6/25 - loss 2.99806523 - samples/sec: 122.60 - lr: 0.150000 2021-03-26 05:56:06,063 epoch 47 - iter 8/25 - loss 2.96191627 - samples/sec: 127.32 - lr: 0.150000 2021-03-26 05:56:07,134 epoch 47 - iter 10/25 - loss 2.90949333 - samples/sec: 119.73 - lr: 0.150000 2021-03-26 05:56:08,134 epoch 47 - iter 12/25 - loss 2.87643186 - samples/sec: 128.13 - lr: 0.150000 2021-03-26 05:56:09,313 epoch 47 - iter 14/25 - loss 2.81540270 - samples/sec: 108.75 - lr: 0.150000 2021-03-26 05:56:10,362 epoch 47 - iter 16/25 - loss 2.88302286 - samples/sec: 122.20 - lr: 0.150000 2021-03-26 05:56:11,369 epoch 47 - iter 18/25 - loss 2.93328830 - samples/sec: 127.35 - lr: 0.150000 2021-03-26 05:56:12,420 epoch 47 - iter 20/25 - loss 2.91642751 - samples/sec: 121.90 - lr: 0.150000 2021-03-26 05:56:13,327 epoch 47 - iter 22/25 - loss 2.92303073 - samples/sec: 141.27 - lr: 0.150000 2021-03-26 05:56:14,309 epoch 47 - iter 24/25 - loss 2.90322950 - samples/sec: 130.61 - lr: 0.150000 2021-03-26 05:56:14,758 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:56:14,759 EPOCH 47 done: loss 2.9369 - lr 0.1500000 2021-03-26 05:56:15,537 DEV : loss 6.756464958190918 - score 0.902 2021-03-26 05:56:15,562 BAD EPOCHS (no improvement): 4 2021-03-26 05:56:15,563 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:56:16,562 epoch 48 - iter 2/25 - loss 2.99353659 - samples/sec: 128.20 - lr: 0.075000 2021-03-26 05:56:17,661 epoch 48 - iter 4/25 - loss 2.75419962 - samples/sec: 116.69 - lr: 0.075000 2021-03-26 05:56:18,679 epoch 48 - iter 6/25 - loss 2.74369395 - samples/sec: 125.97 - lr: 0.075000 2021-03-26 05:56:19,754 epoch 48 - iter 8/25 - loss 2.73128772 - samples/sec: 119.34 - lr: 0.075000 2021-03-26 05:56:20,688 epoch 48 - iter 10/25 - loss 2.61472061 - samples/sec: 137.29 - lr: 0.075000 2021-03-26 05:56:21,723 epoch 48 - iter 12/25 - loss 2.70608862 - samples/sec: 123.87 - lr: 0.075000 2021-03-26 05:56:22,649 epoch 48 - iter 14/25 - loss 2.71813272 - samples/sec: 138.55 - lr: 0.075000 2021-03-26 05:56:23,595 epoch 48 - iter 16/25 - loss 2.72055332 - samples/sec: 135.69 - lr: 0.075000 2021-03-26 05:56:24,628 epoch 48 - iter 18/25 - loss 2.67474678 - samples/sec: 124.10 - lr: 0.075000 2021-03-26 05:56:25,615 epoch 48 - iter 20/25 - loss 2.65116678 - samples/sec: 129.82 - lr: 0.075000 2021-03-26 05:56:26,612 epoch 48 - iter 22/25 - loss 2.65450315 - samples/sec: 128.66 - lr: 0.075000 2021-03-26 05:56:27,641 epoch 48 - iter 24/25 - loss 2.64664620 - samples/sec: 124.63 - lr: 0.075000 2021-03-26 05:56:28,043 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:56:28,044 EPOCH 48 done: loss 2.6319 - lr 0.0750000 2021-03-26 05:56:28,873 DEV : loss 6.665907859802246 - score 0.9054 2021-03-26 05:56:28,901 BAD EPOCHS (no improvement): 0 2021-03-26 05:56:38,673 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:56:39,835 epoch 49 - iter 2/25 - loss 2.66986585 - samples/sec: 110.39 - lr: 0.075000 2021-03-26 05:56:40,851 epoch 49 - iter 4/25 - loss 2.79160547 - samples/sec: 126.40 - lr: 0.075000 2021-03-26 05:56:41,909 epoch 49 - iter 6/25 - loss 2.87581944 - samples/sec: 121.07 - lr: 0.075000 2021-03-26 05:56:42,900 epoch 49 - iter 8/25 - loss 2.73760222 - samples/sec: 129.40 - lr: 0.075000 2021-03-26 05:56:43,859 epoch 49 - iter 10/25 - loss 2.65547196 - samples/sec: 133.67 - lr: 0.075000 2021-03-26 05:56:44,835 epoch 49 - iter 12/25 - loss 2.72983141 - samples/sec: 131.25 - lr: 0.075000 2021-03-26 05:56:45,777 epoch 49 - iter 14/25 - loss 2.76542152 - samples/sec: 136.19 - lr: 0.075000 2021-03-26 05:56:46,686 epoch 49 - iter 16/25 - loss 2.71359808 - samples/sec: 141.01 - lr: 0.075000 2021-03-26 05:56:47,654 epoch 49 - iter 18/25 - loss 2.68785766 - samples/sec: 132.43 - lr: 0.075000 2021-03-26 05:56:48,665 epoch 49 - iter 20/25 - loss 2.70319249 - samples/sec: 126.81 - lr: 0.075000 2021-03-26 05:56:49,680 epoch 49 - iter 22/25 - loss 2.72296827 - samples/sec: 126.19 - lr: 0.075000 2021-03-26 05:56:50,751 epoch 49 - iter 24/25 - loss 2.70314044 - samples/sec: 119.85 - lr: 0.075000 2021-03-26 05:56:51,173 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:56:51,174 EPOCH 49 done: loss 2.7032 - lr 0.0750000 2021-03-26 05:56:51,962 DEV : loss 6.667111396789551 - score 0.9042 2021-03-26 05:56:51,980 BAD EPOCHS (no improvement): 1 2021-03-26 05:56:51,981 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:56:53,059 epoch 50 - iter 2/25 - loss 2.86935806 - samples/sec: 118.93 - lr: 0.075000 2021-03-26 05:56:54,103 epoch 50 - iter 4/25 - loss 2.68313444 - samples/sec: 122.80 - lr: 0.075000 2021-03-26 05:56:55,167 epoch 50 - iter 6/25 - loss 2.58666865 - samples/sec: 120.48 - lr: 0.075000 2021-03-26 05:56:56,130 epoch 50 - iter 8/25 - loss 2.52996308 - samples/sec: 132.96 - lr: 0.075000 2021-03-26 05:56:57,128 epoch 50 - iter 10/25 - loss 2.59376135 - samples/sec: 128.46 - lr: 0.075000 2021-03-26 05:56:58,198 epoch 50 - iter 12/25 - loss 2.67243918 - samples/sec: 119.83 - lr: 0.075000 2021-03-26 05:56:59,246 epoch 50 - iter 14/25 - loss 2.64159279 - samples/sec: 122.25 - lr: 0.075000 2021-03-26 05:57:00,280 epoch 50 - iter 16/25 - loss 2.61747177 - samples/sec: 124.03 - lr: 0.075000 2021-03-26 05:57:01,333 epoch 50 - iter 18/25 - loss 2.63498076 - samples/sec: 121.65 - lr: 0.075000 2021-03-26 05:57:02,449 epoch 50 - iter 20/25 - loss 2.62919823 - samples/sec: 114.84 - lr: 0.075000 2021-03-26 05:57:03,553 epoch 50 - iter 22/25 - loss 2.61773355 - samples/sec: 116.14 - lr: 0.075000 2021-03-26 05:57:04,578 epoch 50 - iter 24/25 - loss 2.62837592 - samples/sec: 125.06 - lr: 0.075000 2021-03-26 05:57:05,003 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:57:05,004 EPOCH 50 done: loss 2.5942 - lr 0.0750000 2021-03-26 05:57:05,774 DEV : loss 6.875411033630371 - score 0.9 2021-03-26 05:57:05,799 BAD EPOCHS (no improvement): 2 2021-03-26 05:57:05,800 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:57:06,859 epoch 51 - iter 2/25 - loss 2.71137524 - samples/sec: 121.06 - lr: 0.075000 2021-03-26 05:57:07,886 epoch 51 - iter 4/25 - loss 2.77748650 - samples/sec: 124.85 - lr: 0.075000 2021-03-26 05:57:09,027 epoch 51 - iter 6/25 - loss 2.73082197 - samples/sec: 112.36 - lr: 0.075000 2021-03-26 05:57:10,070 epoch 51 - iter 8/25 - loss 2.65448046 - samples/sec: 122.80 - lr: 0.075000 2021-03-26 05:57:11,067 epoch 51 - iter 10/25 - loss 2.59957895 - samples/sec: 128.71 - lr: 0.075000 2021-03-26 05:57:12,223 epoch 51 - iter 12/25 - loss 2.62106667 - samples/sec: 110.86 - lr: 0.075000 2021-03-26 05:57:13,274 epoch 51 - iter 14/25 - loss 2.60977428 - samples/sec: 121.94 - lr: 0.075000 2021-03-26 05:57:14,214 epoch 51 - iter 16/25 - loss 2.58111915 - samples/sec: 136.37 - lr: 0.075000 2021-03-26 05:57:15,317 epoch 51 - iter 18/25 - loss 2.58968723 - samples/sec: 116.18 - lr: 0.075000 2021-03-26 05:57:16,302 epoch 51 - iter 20/25 - loss 2.55809421 - samples/sec: 130.33 - lr: 0.075000 2021-03-26 05:57:17,395 epoch 51 - iter 22/25 - loss 2.54893119 - samples/sec: 117.22 - lr: 0.075000 2021-03-26 05:57:18,324 epoch 51 - iter 24/25 - loss 2.52646407 - samples/sec: 138.03 - lr: 0.075000 2021-03-26 05:57:18,765 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:57:18,766 EPOCH 51 done: loss 2.5377 - lr 0.0750000 2021-03-26 05:57:19,530 DEV : loss 6.827261924743652 - score 0.9048 2021-03-26 05:57:19,555 BAD EPOCHS (no improvement): 3 2021-03-26 05:57:19,556 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:57:20,486 epoch 52 - iter 2/25 - loss 2.62309206 - samples/sec: 137.95 - lr: 0.075000 2021-03-26 05:57:21,560 epoch 52 - iter 4/25 - loss 2.72515994 - samples/sec: 119.25 - lr: 0.075000 2021-03-26 05:57:22,545 epoch 52 - iter 6/25 - loss 2.54840565 - samples/sec: 130.31 - lr: 0.075000 2021-03-26 05:57:23,689 epoch 52 - iter 8/25 - loss 2.53041878 - samples/sec: 112.02 - lr: 0.075000 2021-03-26 05:57:24,729 epoch 52 - iter 10/25 - loss 2.59141297 - samples/sec: 123.23 - lr: 0.075000 2021-03-26 05:57:25,787 epoch 52 - iter 12/25 - loss 2.63222436 - samples/sec: 121.13 - lr: 0.075000 2021-03-26 05:57:26,746 epoch 52 - iter 14/25 - loss 2.52890756 - samples/sec: 133.69 - lr: 0.075000 2021-03-26 05:57:27,744 epoch 52 - iter 16/25 - loss 2.55082352 - samples/sec: 128.52 - lr: 0.075000 2021-03-26 05:57:28,683 epoch 52 - iter 18/25 - loss 2.52916779 - samples/sec: 136.66 - lr: 0.075000 2021-03-26 05:57:29,610 epoch 52 - iter 20/25 - loss 2.53436933 - samples/sec: 138.29 - lr: 0.075000 2021-03-26 05:57:30,691 epoch 52 - iter 22/25 - loss 2.54360505 - samples/sec: 118.58 - lr: 0.075000 2021-03-26 05:57:31,760 epoch 52 - iter 24/25 - loss 2.58355175 - samples/sec: 120.47 - lr: 0.075000 2021-03-26 05:57:32,163 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:57:32,163 EPOCH 52 done: loss 2.6029 - lr 0.0750000 2021-03-26 05:57:32,968 DEV : loss 6.790990829467773 - score 0.9012 2021-03-26 05:57:32,997 BAD EPOCHS (no improvement): 4 2021-03-26 05:57:32,998 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:57:34,101 epoch 53 - iter 2/25 - loss 2.89371443 - samples/sec: 116.19 - lr: 0.037500 2021-03-26 05:57:35,161 epoch 53 - iter 4/25 - loss 2.57764858 - samples/sec: 120.96 - lr: 0.037500 2021-03-26 05:57:36,153 epoch 53 - iter 6/25 - loss 2.53048742 - samples/sec: 129.16 - lr: 0.037500 2021-03-26 05:57:37,191 epoch 53 - iter 8/25 - loss 2.55369267 - samples/sec: 123.51 - lr: 0.037500 2021-03-26 05:57:38,300 epoch 53 - iter 10/25 - loss 2.53974638 - samples/sec: 116.08 - lr: 0.037500 2021-03-26 05:57:39,345 epoch 53 - iter 12/25 - loss 2.58391945 - samples/sec: 122.86 - lr: 0.037500 2021-03-26 05:57:40,439 epoch 53 - iter 14/25 - loss 2.63310596 - samples/sec: 117.18 - lr: 0.037500 2021-03-26 05:57:41,538 epoch 53 - iter 16/25 - loss 2.67614898 - samples/sec: 116.64 - lr: 0.037500 2021-03-26 05:57:42,489 epoch 53 - iter 18/25 - loss 2.66936096 - samples/sec: 135.00 - lr: 0.037500 2021-03-26 05:57:43,426 epoch 53 - iter 20/25 - loss 2.61346560 - samples/sec: 136.88 - lr: 0.037500 2021-03-26 05:57:44,415 epoch 53 - iter 22/25 - loss 2.58636801 - samples/sec: 129.55 - lr: 0.037500 2021-03-26 05:57:45,403 epoch 53 - iter 24/25 - loss 2.63187471 - samples/sec: 129.77 - lr: 0.037500 2021-03-26 05:57:45,784 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:57:45,785 EPOCH 53 done: loss 2.6194 - lr 0.0375000 2021-03-26 05:57:46,555 DEV : loss 6.770951747894287 - score 0.9048 2021-03-26 05:57:46,574 BAD EPOCHS (no improvement): 1 2021-03-26 05:57:46,575 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:57:47,623 epoch 54 - iter 2/25 - loss 2.28589094 - samples/sec: 122.26 - lr: 0.037500 2021-03-26 05:57:48,591 epoch 54 - iter 4/25 - loss 2.56535906 - samples/sec: 132.52 - lr: 0.037500 2021-03-26 05:57:49,602 epoch 54 - iter 6/25 - loss 2.55666661 - samples/sec: 126.74 - lr: 0.037500 2021-03-26 05:57:50,590 epoch 54 - iter 8/25 - loss 2.60720205 - samples/sec: 129.76 - lr: 0.037500 2021-03-26 05:57:51,707 epoch 54 - iter 10/25 - loss 2.68733101 - samples/sec: 114.79 - lr: 0.037500 2021-03-26 05:57:52,840 epoch 54 - iter 12/25 - loss 2.67932904 - samples/sec: 113.16 - lr: 0.037500 2021-03-26 05:57:53,903 epoch 54 - iter 14/25 - loss 2.65863521 - samples/sec: 120.56 - lr: 0.037500 2021-03-26 05:57:54,898 epoch 54 - iter 16/25 - loss 2.67555900 - samples/sec: 128.76 - lr: 0.037500 2021-03-26 05:57:55,990 epoch 54 - iter 18/25 - loss 2.66907661 - samples/sec: 117.46 - lr: 0.037500 2021-03-26 05:57:56,986 epoch 54 - iter 20/25 - loss 2.65760111 - samples/sec: 128.68 - lr: 0.037500 2021-03-26 05:57:58,209 epoch 54 - iter 22/25 - loss 2.64885169 - samples/sec: 104.75 - lr: 0.037500 2021-03-26 05:57:59,237 epoch 54 - iter 24/25 - loss 2.65786774 - samples/sec: 124.80 - lr: 0.037500 2021-03-26 05:57:59,634 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:57:59,635 EPOCH 54 done: loss 2.6281 - lr 0.0375000 2021-03-26 05:58:00,394 DEV : loss 6.769601821899414 - score 0.9036 2021-03-26 05:58:00,419 BAD EPOCHS (no improvement): 2 2021-03-26 05:58:00,420 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:58:01,385 epoch 55 - iter 2/25 - loss 2.57250500 - samples/sec: 132.96 - lr: 0.037500 2021-03-26 05:58:02,363 epoch 55 - iter 4/25 - loss 2.56209105 - samples/sec: 131.00 - lr: 0.037500 2021-03-26 05:58:03,519 epoch 55 - iter 6/25 - loss 2.55206974 - samples/sec: 110.90 - lr: 0.037500 2021-03-26 05:58:04,451 epoch 55 - iter 8/25 - loss 2.63297400 - samples/sec: 137.55 - lr: 0.037500 2021-03-26 05:58:05,438 epoch 55 - iter 10/25 - loss 2.64202976 - samples/sec: 129.83 - lr: 0.037500 2021-03-26 05:58:06,455 epoch 55 - iter 12/25 - loss 2.58580096 - samples/sec: 126.06 - lr: 0.037500 2021-03-26 05:58:07,536 epoch 55 - iter 14/25 - loss 2.55834146 - samples/sec: 118.54 - lr: 0.037500 2021-03-26 05:58:08,675 epoch 55 - iter 16/25 - loss 2.57096677 - samples/sec: 112.55 - lr: 0.037500 2021-03-26 05:58:09,650 epoch 55 - iter 18/25 - loss 2.56309658 - samples/sec: 131.40 - lr: 0.037500 2021-03-26 05:58:10,805 epoch 55 - iter 20/25 - loss 2.53701287 - samples/sec: 111.09 - lr: 0.037500 2021-03-26 05:58:12,006 epoch 55 - iter 22/25 - loss 2.56861101 - samples/sec: 106.71 - lr: 0.037500 2021-03-26 05:58:13,012 epoch 55 - iter 24/25 - loss 2.53881068 - samples/sec: 127.36 - lr: 0.037500 2021-03-26 05:58:13,431 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:58:13,433 EPOCH 55 done: loss 2.5274 - lr 0.0375000 2021-03-26 05:58:14,198 DEV : loss 6.722831726074219 - score 0.9034 2021-03-26 05:58:14,222 BAD EPOCHS (no improvement): 3 2021-03-26 05:58:14,223 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:58:15,235 epoch 56 - iter 2/25 - loss 3.06336522 - samples/sec: 126.75 - lr: 0.037500 2021-03-26 05:58:16,235 epoch 56 - iter 4/25 - loss 2.66866070 - samples/sec: 128.31 - lr: 0.037500 2021-03-26 05:58:17,254 epoch 56 - iter 6/25 - loss 2.59741449 - samples/sec: 125.72 - lr: 0.037500 2021-03-26 05:58:18,230 epoch 56 - iter 8/25 - loss 2.50272012 - samples/sec: 131.39 - lr: 0.037500 2021-03-26 05:58:19,459 epoch 56 - iter 10/25 - loss 2.50687180 - samples/sec: 104.40 - lr: 0.037500 2021-03-26 05:58:20,490 epoch 56 - iter 12/25 - loss 2.49787994 - samples/sec: 124.27 - lr: 0.037500 2021-03-26 05:58:21,569 epoch 56 - iter 14/25 - loss 2.48121374 - samples/sec: 118.80 - lr: 0.037500 2021-03-26 05:58:22,634 epoch 56 - iter 16/25 - loss 2.48644322 - samples/sec: 120.45 - lr: 0.037500 2021-03-26 05:58:23,633 epoch 56 - iter 18/25 - loss 2.49157370 - samples/sec: 128.30 - lr: 0.037500 2021-03-26 05:58:24,723 epoch 56 - iter 20/25 - loss 2.48946971 - samples/sec: 117.56 - lr: 0.037500 2021-03-26 05:58:25,809 epoch 56 - iter 22/25 - loss 2.49483615 - samples/sec: 118.10 - lr: 0.037500 2021-03-26 05:58:26,827 epoch 56 - iter 24/25 - loss 2.50274724 - samples/sec: 125.92 - lr: 0.037500 2021-03-26 05:58:27,323 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:58:27,324 EPOCH 56 done: loss 2.4933 - lr 0.0375000 2021-03-26 05:58:28,121 DEV : loss 6.749120235443115 - score 0.9046 2021-03-26 05:58:28,147 BAD EPOCHS (no improvement): 4 2021-03-26 05:58:28,148 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:58:29,243 epoch 57 - iter 2/25 - loss 2.20577013 - samples/sec: 117.11 - lr: 0.018750 2021-03-26 05:58:30,440 epoch 57 - iter 4/25 - loss 2.51310360 - samples/sec: 107.03 - lr: 0.018750 2021-03-26 05:58:31,442 epoch 57 - iter 6/25 - loss 2.57224830 - samples/sec: 127.91 - lr: 0.018750 2021-03-26 05:58:32,495 epoch 57 - iter 8/25 - loss 2.51049601 - samples/sec: 121.98 - lr: 0.018750 2021-03-26 05:58:33,803 epoch 57 - iter 10/25 - loss 2.53363069 - samples/sec: 97.94 - lr: 0.018750 2021-03-26 05:58:34,897 epoch 57 - iter 12/25 - loss 2.46666911 - samples/sec: 117.34 - lr: 0.018750 2021-03-26 05:58:35,940 epoch 57 - iter 14/25 - loss 2.48624198 - samples/sec: 122.90 - lr: 0.018750 2021-03-26 05:58:37,023 epoch 57 - iter 16/25 - loss 2.53736014 - samples/sec: 118.44 - lr: 0.018750 2021-03-26 05:58:37,958 epoch 57 - iter 18/25 - loss 2.54605499 - samples/sec: 137.17 - lr: 0.018750 2021-03-26 05:58:38,985 epoch 57 - iter 20/25 - loss 2.56636576 - samples/sec: 124.85 - lr: 0.018750 2021-03-26 05:58:40,040 epoch 57 - iter 22/25 - loss 2.59158968 - samples/sec: 121.54 - lr: 0.018750 2021-03-26 05:58:41,065 epoch 57 - iter 24/25 - loss 2.56697516 - samples/sec: 125.03 - lr: 0.018750 2021-03-26 05:58:41,446 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:58:41,447 EPOCH 57 done: loss 2.5322 - lr 0.0187500 2021-03-26 05:58:42,214 DEV : loss 6.725451946258545 - score 0.9042 2021-03-26 05:58:42,240 BAD EPOCHS (no improvement): 1 2021-03-26 05:58:42,241 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:58:43,198 epoch 58 - iter 2/25 - loss 2.39105058 - samples/sec: 133.89 - lr: 0.018750 2021-03-26 05:58:44,218 epoch 58 - iter 4/25 - loss 2.44910604 - samples/sec: 125.74 - lr: 0.018750 2021-03-26 05:58:45,178 epoch 58 - iter 6/25 - loss 2.47911966 - samples/sec: 133.55 - lr: 0.018750 2021-03-26 05:58:46,280 epoch 58 - iter 8/25 - loss 2.48556039 - samples/sec: 116.26 - lr: 0.018750 2021-03-26 05:58:47,264 epoch 58 - iter 10/25 - loss 2.55912685 - samples/sec: 130.36 - lr: 0.018750 2021-03-26 05:58:48,195 epoch 58 - iter 12/25 - loss 2.43912268 - samples/sec: 137.76 - lr: 0.018750 2021-03-26 05:58:49,124 epoch 58 - iter 14/25 - loss 2.48512311 - samples/sec: 138.01 - lr: 0.018750 2021-03-26 05:58:50,078 epoch 58 - iter 16/25 - loss 2.50301301 - samples/sec: 134.37 - lr: 0.018750 2021-03-26 05:58:51,198 epoch 58 - iter 18/25 - loss 2.48813513 - samples/sec: 114.42 - lr: 0.018750 2021-03-26 05:58:52,204 epoch 58 - iter 20/25 - loss 2.48385862 - samples/sec: 127.48 - lr: 0.018750 2021-03-26 05:58:53,224 epoch 58 - iter 22/25 - loss 2.44755840 - samples/sec: 125.61 - lr: 0.018750 2021-03-26 05:58:54,287 epoch 58 - iter 24/25 - loss 2.44767216 - samples/sec: 120.70 - lr: 0.018750 2021-03-26 05:58:54,755 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:58:54,755 EPOCH 58 done: loss 2.4693 - lr 0.0187500 2021-03-26 05:58:55,549 DEV : loss 6.709056854248047 - score 0.9034 2021-03-26 05:58:55,568 BAD EPOCHS (no improvement): 2 2021-03-26 05:58:55,568 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:58:56,501 epoch 59 - iter 2/25 - loss 2.32147455 - samples/sec: 137.39 - lr: 0.018750 2021-03-26 05:58:57,557 epoch 59 - iter 4/25 - loss 2.47985613 - samples/sec: 121.48 - lr: 0.018750 2021-03-26 05:58:58,499 epoch 59 - iter 6/25 - loss 2.58878783 - samples/sec: 136.14 - lr: 0.018750 2021-03-26 05:58:59,479 epoch 59 - iter 8/25 - loss 2.46861984 - samples/sec: 130.79 - lr: 0.018750 2021-03-26 05:59:00,465 epoch 59 - iter 10/25 - loss 2.48236761 - samples/sec: 130.03 - lr: 0.018750 2021-03-26 05:59:01,412 epoch 59 - iter 12/25 - loss 2.56132956 - samples/sec: 135.33 - lr: 0.018750 2021-03-26 05:59:02,467 epoch 59 - iter 14/25 - loss 2.55461029 - samples/sec: 121.54 - lr: 0.018750 2021-03-26 05:59:03,520 epoch 59 - iter 16/25 - loss 2.49694520 - samples/sec: 121.79 - lr: 0.018750 2021-03-26 05:59:04,445 epoch 59 - iter 18/25 - loss 2.47332254 - samples/sec: 138.50 - lr: 0.018750 2021-03-26 05:59:05,536 epoch 59 - iter 20/25 - loss 2.46696305 - samples/sec: 117.49 - lr: 0.018750 2021-03-26 05:59:06,508 epoch 59 - iter 22/25 - loss 2.48456865 - samples/sec: 132.12 - lr: 0.018750 2021-03-26 05:59:07,548 epoch 59 - iter 24/25 - loss 2.47716364 - samples/sec: 123.35 - lr: 0.018750 2021-03-26 05:59:07,990 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:59:07,991 EPOCH 59 done: loss 2.4872 - lr 0.0187500 2021-03-26 05:59:08,798 DEV : loss 6.735278129577637 - score 0.905 2021-03-26 05:59:08,823 BAD EPOCHS (no improvement): 3 2021-03-26 05:59:08,824 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:59:09,809 epoch 60 - iter 2/25 - loss 1.87354106 - samples/sec: 130.31 - lr: 0.018750 2021-03-26 05:59:10,874 epoch 60 - iter 4/25 - loss 1.79272857 - samples/sec: 120.32 - lr: 0.018750 2021-03-26 05:59:12,177 epoch 60 - iter 6/25 - loss 2.07879815 - samples/sec: 98.42 - lr: 0.018750 2021-03-26 05:59:13,083 epoch 60 - iter 8/25 - loss 2.28630097 - samples/sec: 141.74 - lr: 0.018750 2021-03-26 05:59:14,132 epoch 60 - iter 10/25 - loss 2.30058919 - samples/sec: 122.18 - lr: 0.018750 2021-03-26 05:59:15,020 epoch 60 - iter 12/25 - loss 2.32238834 - samples/sec: 144.28 - lr: 0.018750 2021-03-26 05:59:16,011 epoch 60 - iter 14/25 - loss 2.34343993 - samples/sec: 129.45 - lr: 0.018750 2021-03-26 05:59:16,958 epoch 60 - iter 16/25 - loss 2.39775931 - samples/sec: 135.37 - lr: 0.018750 2021-03-26 05:59:18,001 epoch 60 - iter 18/25 - loss 2.39503040 - samples/sec: 122.85 - lr: 0.018750 2021-03-26 05:59:19,016 epoch 60 - iter 20/25 - loss 2.40990252 - samples/sec: 126.40 - lr: 0.018750 2021-03-26 05:59:20,003 epoch 60 - iter 22/25 - loss 2.41649341 - samples/sec: 129.95 - lr: 0.018750 2021-03-26 05:59:20,937 epoch 60 - iter 24/25 - loss 2.42599278 - samples/sec: 137.34 - lr: 0.018750 2021-03-26 05:59:21,408 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:59:21,408 EPOCH 60 done: loss 2.4178 - lr 0.0187500 2021-03-26 05:59:22,184 DEV : loss 6.739635467529297 - score 0.9058 2021-03-26 05:59:22,209 BAD EPOCHS (no improvement): 0 2021-03-26 05:59:31,905 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:59:33,017 epoch 61 - iter 2/25 - loss 2.60011214 - samples/sec: 115.35 - lr: 0.018750 2021-03-26 05:59:34,162 epoch 61 - iter 4/25 - loss 2.52214578 - samples/sec: 112.02 - lr: 0.018750 2021-03-26 05:59:35,247 epoch 61 - iter 6/25 - loss 2.38890078 - samples/sec: 118.01 - lr: 0.018750 2021-03-26 05:59:36,290 epoch 61 - iter 8/25 - loss 2.56866135 - samples/sec: 122.99 - lr: 0.018750 2021-03-26 05:59:37,174 epoch 61 - iter 10/25 - loss 2.39370874 - samples/sec: 144.91 - lr: 0.018750 2021-03-26 05:59:38,153 epoch 61 - iter 12/25 - loss 2.40410967 - samples/sec: 131.03 - lr: 0.018750 2021-03-26 05:59:39,170 epoch 61 - iter 14/25 - loss 2.44239428 - samples/sec: 126.04 - lr: 0.018750 2021-03-26 05:59:40,165 epoch 61 - iter 16/25 - loss 2.39614346 - samples/sec: 128.80 - lr: 0.018750 2021-03-26 05:59:41,158 epoch 61 - iter 18/25 - loss 2.44974503 - samples/sec: 129.07 - lr: 0.018750 2021-03-26 05:59:42,118 epoch 61 - iter 20/25 - loss 2.46324108 - samples/sec: 133.63 - lr: 0.018750 2021-03-26 05:59:43,074 epoch 61 - iter 22/25 - loss 2.44603983 - samples/sec: 134.01 - lr: 0.018750 2021-03-26 05:59:44,176 epoch 61 - iter 24/25 - loss 2.48440110 - samples/sec: 116.33 - lr: 0.018750 2021-03-26 05:59:44,584 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:59:44,585 EPOCH 61 done: loss 2.4975 - lr 0.0187500 2021-03-26 05:59:45,362 DEV : loss 6.744481563568115 - score 0.9054 2021-03-26 05:59:45,384 BAD EPOCHS (no improvement): 1 2021-03-26 05:59:45,385 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:59:46,348 epoch 62 - iter 2/25 - loss 2.30863678 - samples/sec: 133.17 - lr: 0.018750 2021-03-26 05:59:47,293 epoch 62 - iter 4/25 - loss 2.49650627 - samples/sec: 135.68 - lr: 0.018750 2021-03-26 05:59:48,201 epoch 62 - iter 6/25 - loss 2.44710239 - samples/sec: 141.16 - lr: 0.018750 2021-03-26 05:59:49,260 epoch 62 - iter 8/25 - loss 2.52125069 - samples/sec: 121.07 - lr: 0.018750 2021-03-26 05:59:50,292 epoch 62 - iter 10/25 - loss 2.56077802 - samples/sec: 124.27 - lr: 0.018750 2021-03-26 05:59:51,382 epoch 62 - iter 12/25 - loss 2.61480961 - samples/sec: 117.56 - lr: 0.018750 2021-03-26 05:59:52,553 epoch 62 - iter 14/25 - loss 2.61970052 - samples/sec: 109.63 - lr: 0.018750 2021-03-26 05:59:53,515 epoch 62 - iter 16/25 - loss 2.57987718 - samples/sec: 133.27 - lr: 0.018750 2021-03-26 05:59:54,569 epoch 62 - iter 18/25 - loss 2.53430665 - samples/sec: 121.62 - lr: 0.018750 2021-03-26 05:59:55,656 epoch 62 - iter 20/25 - loss 2.52618066 - samples/sec: 117.95 - lr: 0.018750 2021-03-26 05:59:56,860 epoch 62 - iter 22/25 - loss 2.53868995 - samples/sec: 106.47 - lr: 0.018750 2021-03-26 05:59:58,000 epoch 62 - iter 24/25 - loss 2.54573082 - samples/sec: 112.49 - lr: 0.018750 2021-03-26 05:59:58,443 ---------------------------------------------------------------------------------------------------- 2021-03-26 05:59:58,445 EPOCH 62 done: loss 2.5331 - lr 0.0187500 2021-03-26 05:59:59,231 DEV : loss 6.735997676849365 - score 0.9042 2021-03-26 05:59:59,257 BAD EPOCHS (no improvement): 2 2021-03-26 05:59:59,257 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:00:00,271 epoch 63 - iter 2/25 - loss 2.81712377 - samples/sec: 126.50 - lr: 0.018750 2021-03-26 06:00:01,333 epoch 63 - iter 4/25 - loss 2.43051547 - samples/sec: 120.77 - lr: 0.018750 2021-03-26 06:00:02,362 epoch 63 - iter 6/25 - loss 2.45722274 - samples/sec: 124.62 - lr: 0.018750 2021-03-26 06:00:03,332 epoch 63 - iter 8/25 - loss 2.45625201 - samples/sec: 132.27 - lr: 0.018750 2021-03-26 06:00:04,390 epoch 63 - iter 10/25 - loss 2.40530462 - samples/sec: 121.10 - lr: 0.018750 2021-03-26 06:00:05,365 epoch 63 - iter 12/25 - loss 2.45525324 - samples/sec: 131.63 - lr: 0.018750 2021-03-26 06:00:06,373 epoch 63 - iter 14/25 - loss 2.35274683 - samples/sec: 127.06 - lr: 0.018750 2021-03-26 06:00:07,455 epoch 63 - iter 16/25 - loss 2.37795563 - samples/sec: 118.49 - lr: 0.018750 2021-03-26 06:00:08,433 epoch 63 - iter 18/25 - loss 2.35921160 - samples/sec: 131.10 - lr: 0.018750 2021-03-26 06:00:09,563 epoch 63 - iter 20/25 - loss 2.44812641 - samples/sec: 113.42 - lr: 0.018750 2021-03-26 06:00:10,507 epoch 63 - iter 22/25 - loss 2.43963941 - samples/sec: 135.85 - lr: 0.018750 2021-03-26 06:00:11,492 epoch 63 - iter 24/25 - loss 2.40271173 - samples/sec: 130.19 - lr: 0.018750 2021-03-26 06:00:11,884 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:00:11,886 EPOCH 63 done: loss 2.4120 - lr 0.0187500 2021-03-26 06:00:12,680 DEV : loss 6.751418113708496 - score 0.9046 2021-03-26 06:00:12,710 BAD EPOCHS (no improvement): 3 2021-03-26 06:00:12,711 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:00:15,095 epoch 64 - iter 2/25 - loss 2.59013653 - samples/sec: 53.73 - lr: 0.018750 2021-03-26 06:00:16,160 epoch 64 - iter 4/25 - loss 2.39185470 - samples/sec: 120.47 - lr: 0.018750 2021-03-26 06:00:17,149 epoch 64 - iter 6/25 - loss 2.35445166 - samples/sec: 129.73 - lr: 0.018750 2021-03-26 06:00:18,163 epoch 64 - iter 8/25 - loss 2.37706771 - samples/sec: 126.32 - lr: 0.018750 2021-03-26 06:00:19,163 epoch 64 - iter 10/25 - loss 2.35224319 - samples/sec: 128.21 - lr: 0.018750 2021-03-26 06:00:20,164 epoch 64 - iter 12/25 - loss 2.30195101 - samples/sec: 128.08 - lr: 0.018750 2021-03-26 06:00:21,419 epoch 64 - iter 14/25 - loss 2.37853982 - samples/sec: 102.04 - lr: 0.018750 2021-03-26 06:00:22,547 epoch 64 - iter 16/25 - loss 2.38729794 - samples/sec: 113.68 - lr: 0.018750 2021-03-26 06:00:23,553 epoch 64 - iter 18/25 - loss 2.38070179 - samples/sec: 127.44 - lr: 0.018750 2021-03-26 06:00:24,569 epoch 64 - iter 20/25 - loss 2.38173290 - samples/sec: 126.15 - lr: 0.018750 2021-03-26 06:00:25,591 epoch 64 - iter 22/25 - loss 2.42094920 - samples/sec: 125.41 - lr: 0.018750 2021-03-26 06:00:26,649 epoch 64 - iter 24/25 - loss 2.47213012 - samples/sec: 121.22 - lr: 0.018750 2021-03-26 06:00:27,065 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:00:27,065 EPOCH 64 done: loss 2.4368 - lr 0.0187500 2021-03-26 06:00:27,871 DEV : loss 6.746072292327881 - score 0.9046 2021-03-26 06:00:27,895 BAD EPOCHS (no improvement): 4 2021-03-26 06:00:27,896 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:00:28,950 epoch 65 - iter 2/25 - loss 2.09257662 - samples/sec: 121.68 - lr: 0.009375 2021-03-26 06:00:29,918 epoch 65 - iter 4/25 - loss 2.38884521 - samples/sec: 132.42 - lr: 0.009375 2021-03-26 06:00:30,886 epoch 65 - iter 6/25 - loss 2.30170250 - samples/sec: 132.44 - lr: 0.009375 2021-03-26 06:00:31,959 epoch 65 - iter 8/25 - loss 2.26360065 - samples/sec: 119.53 - lr: 0.009375 2021-03-26 06:00:32,959 epoch 65 - iter 10/25 - loss 2.35580001 - samples/sec: 128.25 - lr: 0.009375 2021-03-26 06:00:33,896 epoch 65 - iter 12/25 - loss 2.32859574 - samples/sec: 136.83 - lr: 0.009375 2021-03-26 06:00:34,950 epoch 65 - iter 14/25 - loss 2.38266451 - samples/sec: 121.70 - lr: 0.009375 2021-03-26 06:00:35,865 epoch 65 - iter 16/25 - loss 2.34344883 - samples/sec: 140.21 - lr: 0.009375 2021-03-26 06:00:36,890 epoch 65 - iter 18/25 - loss 2.32179760 - samples/sec: 125.11 - lr: 0.009375 2021-03-26 06:00:37,933 epoch 65 - iter 20/25 - loss 2.34824256 - samples/sec: 123.40 - lr: 0.009375 2021-03-26 06:00:38,990 epoch 65 - iter 22/25 - loss 2.35746112 - samples/sec: 121.18 - lr: 0.009375 2021-03-26 06:00:40,052 epoch 65 - iter 24/25 - loss 2.37810714 - samples/sec: 120.75 - lr: 0.009375 2021-03-26 06:00:40,446 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:00:40,448 EPOCH 65 done: loss 2.3485 - lr 0.0093750 2021-03-26 06:00:41,222 DEV : loss 6.7548370361328125 - score 0.9054 2021-03-26 06:00:41,248 BAD EPOCHS (no improvement): 1 2021-03-26 06:00:41,249 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:00:42,273 epoch 66 - iter 2/25 - loss 2.60710657 - samples/sec: 125.14 - lr: 0.009375 2021-03-26 06:00:43,227 epoch 66 - iter 4/25 - loss 2.56015295 - samples/sec: 134.33 - lr: 0.009375 2021-03-26 06:00:44,116 epoch 66 - iter 6/25 - loss 2.44887527 - samples/sec: 144.16 - lr: 0.009375 2021-03-26 06:00:45,162 epoch 66 - iter 8/25 - loss 2.41221850 - samples/sec: 122.61 - lr: 0.009375 2021-03-26 06:00:46,120 epoch 66 - iter 10/25 - loss 2.33587884 - samples/sec: 133.87 - lr: 0.009375 2021-03-26 06:00:47,126 epoch 66 - iter 12/25 - loss 2.41186844 - samples/sec: 127.46 - lr: 0.009375 2021-03-26 06:00:48,053 epoch 66 - iter 14/25 - loss 2.38344255 - samples/sec: 138.38 - lr: 0.009375 2021-03-26 06:00:48,997 epoch 66 - iter 16/25 - loss 2.40268778 - samples/sec: 135.89 - lr: 0.009375 2021-03-26 06:00:50,225 epoch 66 - iter 18/25 - loss 2.46428358 - samples/sec: 104.37 - lr: 0.009375 2021-03-26 06:00:51,411 epoch 66 - iter 20/25 - loss 2.52800439 - samples/sec: 108.15 - lr: 0.009375 2021-03-26 06:00:52,363 epoch 66 - iter 22/25 - loss 2.48158739 - samples/sec: 134.83 - lr: 0.009375 2021-03-26 06:00:53,364 epoch 66 - iter 24/25 - loss 2.51746005 - samples/sec: 127.92 - lr: 0.009375 2021-03-26 06:00:53,785 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:00:53,786 EPOCH 66 done: loss 2.5028 - lr 0.0093750 2021-03-26 06:00:54,567 DEV : loss 6.752009868621826 - score 0.9058 2021-03-26 06:00:54,590 BAD EPOCHS (no improvement): 2 2021-03-26 06:00:54,591 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:00:55,558 epoch 67 - iter 2/25 - loss 2.90654492 - samples/sec: 132.50 - lr: 0.009375 2021-03-26 06:00:56,624 epoch 67 - iter 4/25 - loss 2.68288243 - samples/sec: 120.25 - lr: 0.009375 2021-03-26 06:00:57,733 epoch 67 - iter 6/25 - loss 2.66686583 - samples/sec: 115.71 - lr: 0.009375 2021-03-26 06:00:58,699 epoch 67 - iter 8/25 - loss 2.66637003 - samples/sec: 132.89 - lr: 0.009375 2021-03-26 06:00:59,626 epoch 67 - iter 10/25 - loss 2.71289964 - samples/sec: 138.25 - lr: 0.009375 2021-03-26 06:01:00,729 epoch 67 - iter 12/25 - loss 2.61172573 - samples/sec: 116.19 - lr: 0.009375 2021-03-26 06:01:01,698 epoch 67 - iter 14/25 - loss 2.55450325 - samples/sec: 132.57 - lr: 0.009375 2021-03-26 06:01:02,709 epoch 67 - iter 16/25 - loss 2.55706222 - samples/sec: 126.75 - lr: 0.009375 2021-03-26 06:01:03,692 epoch 67 - iter 18/25 - loss 2.54990624 - samples/sec: 130.46 - lr: 0.009375 2021-03-26 06:01:04,784 epoch 67 - iter 20/25 - loss 2.58042824 - samples/sec: 117.36 - lr: 0.009375 2021-03-26 06:01:05,847 epoch 67 - iter 22/25 - loss 2.54599817 - samples/sec: 120.56 - lr: 0.009375 2021-03-26 06:01:06,920 epoch 67 - iter 24/25 - loss 2.53072828 - samples/sec: 119.51 - lr: 0.009375 2021-03-26 06:01:07,321 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:01:07,321 EPOCH 67 done: loss 2.5156 - lr 0.0093750 2021-03-26 06:01:08,148 DEV : loss 6.734310150146484 - score 0.9054 2021-03-26 06:01:08,168 BAD EPOCHS (no improvement): 3 2021-03-26 06:01:08,169 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:01:09,213 epoch 68 - iter 2/25 - loss 2.69127226 - samples/sec: 122.66 - lr: 0.009375 2021-03-26 06:01:10,181 epoch 68 - iter 4/25 - loss 2.73949641 - samples/sec: 132.55 - lr: 0.009375 2021-03-26 06:01:11,141 epoch 68 - iter 6/25 - loss 2.49159026 - samples/sec: 133.70 - lr: 0.009375 2021-03-26 06:01:12,088 epoch 68 - iter 8/25 - loss 2.57020289 - samples/sec: 135.36 - lr: 0.009375 2021-03-26 06:01:13,095 epoch 68 - iter 10/25 - loss 2.51954174 - samples/sec: 127.28 - lr: 0.009375 2021-03-26 06:01:14,181 epoch 68 - iter 12/25 - loss 2.44195040 - samples/sec: 118.04 - lr: 0.009375 2021-03-26 06:01:15,190 epoch 68 - iter 14/25 - loss 2.47290715 - samples/sec: 127.08 - lr: 0.009375 2021-03-26 06:01:16,358 epoch 68 - iter 16/25 - loss 2.44247594 - samples/sec: 109.69 - lr: 0.009375 2021-03-26 06:01:17,437 epoch 68 - iter 18/25 - loss 2.41137531 - samples/sec: 118.88 - lr: 0.009375 2021-03-26 06:01:18,445 epoch 68 - iter 20/25 - loss 2.37544742 - samples/sec: 127.30 - lr: 0.009375 2021-03-26 06:01:19,648 epoch 68 - iter 22/25 - loss 2.36865735 - samples/sec: 106.46 - lr: 0.009375 2021-03-26 06:01:20,609 epoch 68 - iter 24/25 - loss 2.39470330 - samples/sec: 133.59 - lr: 0.009375 2021-03-26 06:01:21,010 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:01:21,011 EPOCH 68 done: loss 2.3852 - lr 0.0093750 2021-03-26 06:01:21,771 DEV : loss 6.732729911804199 - score 0.9054 2021-03-26 06:01:21,795 BAD EPOCHS (no improvement): 4 2021-03-26 06:01:21,795 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:01:22,731 epoch 69 - iter 2/25 - loss 2.14961201 - samples/sec: 137.02 - lr: 0.004687 2021-03-26 06:01:23,622 epoch 69 - iter 4/25 - loss 2.15720448 - samples/sec: 144.16 - lr: 0.004687 2021-03-26 06:01:24,580 epoch 69 - iter 6/25 - loss 2.13730657 - samples/sec: 133.78 - lr: 0.004687 2021-03-26 06:01:25,690 epoch 69 - iter 8/25 - loss 2.14602265 - samples/sec: 115.47 - lr: 0.004687 2021-03-26 06:01:26,672 epoch 69 - iter 10/25 - loss 2.24792657 - samples/sec: 130.74 - lr: 0.004687 2021-03-26 06:01:27,758 epoch 69 - iter 12/25 - loss 2.22681612 - samples/sec: 118.12 - lr: 0.004687 2021-03-26 06:01:28,719 epoch 69 - iter 14/25 - loss 2.17326404 - samples/sec: 133.59 - lr: 0.004687 2021-03-26 06:01:29,765 epoch 69 - iter 16/25 - loss 2.18757197 - samples/sec: 122.65 - lr: 0.004687 2021-03-26 06:01:30,818 epoch 69 - iter 18/25 - loss 2.28958360 - samples/sec: 121.86 - lr: 0.004687 2021-03-26 06:01:31,843 epoch 69 - iter 20/25 - loss 2.28681493 - samples/sec: 125.19 - lr: 0.004687 2021-03-26 06:01:32,902 epoch 69 - iter 22/25 - loss 2.31756993 - samples/sec: 121.07 - lr: 0.004687 2021-03-26 06:01:33,937 epoch 69 - iter 24/25 - loss 2.33851089 - samples/sec: 123.91 - lr: 0.004687 2021-03-26 06:01:34,336 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:01:34,338 EPOCH 69 done: loss 2.3385 - lr 0.0046875 2021-03-26 06:01:35,144 DEV : loss 6.749580383300781 - score 0.9066 2021-03-26 06:01:35,169 BAD EPOCHS (no improvement): 0 2021-03-26 06:01:44,939 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:01:46,012 epoch 70 - iter 2/25 - loss 2.96315742 - samples/sec: 119.62 - lr: 0.004687 2021-03-26 06:01:47,104 epoch 70 - iter 4/25 - loss 2.70809591 - samples/sec: 117.51 - lr: 0.004687 2021-03-26 06:01:48,137 epoch 70 - iter 6/25 - loss 2.69718925 - samples/sec: 124.12 - lr: 0.004687 2021-03-26 06:01:49,195 epoch 70 - iter 8/25 - loss 2.65873078 - samples/sec: 121.17 - lr: 0.004687 2021-03-26 06:01:50,229 epoch 70 - iter 10/25 - loss 2.57560856 - samples/sec: 123.94 - lr: 0.004687 2021-03-26 06:01:51,311 epoch 70 - iter 12/25 - loss 2.63722142 - samples/sec: 118.57 - lr: 0.004687 2021-03-26 06:01:52,425 epoch 70 - iter 14/25 - loss 2.60139622 - samples/sec: 115.20 - lr: 0.004687 2021-03-26 06:01:53,531 epoch 70 - iter 16/25 - loss 2.61185963 - samples/sec: 115.89 - lr: 0.004687 2021-03-26 06:01:54,471 epoch 70 - iter 18/25 - loss 2.58912293 - samples/sec: 136.41 - lr: 0.004687 2021-03-26 06:01:55,460 epoch 70 - iter 20/25 - loss 2.60415198 - samples/sec: 129.59 - lr: 0.004687 2021-03-26 06:01:56,472 epoch 70 - iter 22/25 - loss 2.56834055 - samples/sec: 126.68 - lr: 0.004687 2021-03-26 06:01:57,547 epoch 70 - iter 24/25 - loss 2.56006884 - samples/sec: 119.35 - lr: 0.004687 2021-03-26 06:01:57,992 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:01:57,992 EPOCH 70 done: loss 2.5346 - lr 0.0046875 2021-03-26 06:01:58,765 DEV : loss 6.751224517822266 - score 0.9062 2021-03-26 06:01:58,785 BAD EPOCHS (no improvement): 1 2021-03-26 06:01:58,786 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:01:59,808 epoch 71 - iter 2/25 - loss 2.36810112 - samples/sec: 125.51 - lr: 0.004687 2021-03-26 06:02:00,804 epoch 71 - iter 4/25 - loss 2.16877392 - samples/sec: 128.64 - lr: 0.004687 2021-03-26 06:02:01,893 epoch 71 - iter 6/25 - loss 2.23906277 - samples/sec: 117.72 - lr: 0.004687 2021-03-26 06:02:03,393 epoch 71 - iter 8/25 - loss 2.35320215 - samples/sec: 85.41 - lr: 0.004687 2021-03-26 06:02:04,442 epoch 71 - iter 10/25 - loss 2.17347422 - samples/sec: 122.18 - lr: 0.004687 2021-03-26 06:02:05,460 epoch 71 - iter 12/25 - loss 2.27554323 - samples/sec: 125.95 - lr: 0.004687 2021-03-26 06:02:06,360 epoch 71 - iter 14/25 - loss 2.27385560 - samples/sec: 142.69 - lr: 0.004687 2021-03-26 06:02:07,479 epoch 71 - iter 16/25 - loss 2.30233501 - samples/sec: 114.52 - lr: 0.004687 2021-03-26 06:02:08,511 epoch 71 - iter 18/25 - loss 2.34775718 - samples/sec: 124.20 - lr: 0.004687 2021-03-26 06:02:09,502 epoch 71 - iter 20/25 - loss 2.37440296 - samples/sec: 129.38 - lr: 0.004687 2021-03-26 06:02:10,527 epoch 71 - iter 22/25 - loss 2.34015569 - samples/sec: 125.15 - lr: 0.004687 2021-03-26 06:02:11,548 epoch 71 - iter 24/25 - loss 2.31703675 - samples/sec: 125.47 - lr: 0.004687 2021-03-26 06:02:11,985 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:02:11,987 EPOCH 71 done: loss 2.3079 - lr 0.0046875 2021-03-26 06:02:12,798 DEV : loss 6.757469177246094 - score 0.907 2021-03-26 06:02:12,838 BAD EPOCHS (no improvement): 0 2021-03-26 06:02:22,994 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:02:24,090 epoch 72 - iter 2/25 - loss 1.88978660 - samples/sec: 117.15 - lr: 0.004687 2021-03-26 06:02:25,107 epoch 72 - iter 4/25 - loss 2.08690861 - samples/sec: 126.25 - lr: 0.004687 2021-03-26 06:02:26,251 epoch 72 - iter 6/25 - loss 2.20860885 - samples/sec: 112.06 - lr: 0.004687 2021-03-26 06:02:27,414 epoch 72 - iter 8/25 - loss 2.27037697 - samples/sec: 110.19 - lr: 0.004687 2021-03-26 06:02:28,636 epoch 72 - iter 10/25 - loss 2.46181704 - samples/sec: 104.91 - lr: 0.004687 2021-03-26 06:02:29,795 epoch 72 - iter 12/25 - loss 2.40252721 - samples/sec: 110.61 - lr: 0.004687 2021-03-26 06:02:30,794 epoch 72 - iter 14/25 - loss 2.46973358 - samples/sec: 128.27 - lr: 0.004687 2021-03-26 06:02:31,747 epoch 72 - iter 16/25 - loss 2.51864427 - samples/sec: 134.47 - lr: 0.004687 2021-03-26 06:02:32,766 epoch 72 - iter 18/25 - loss 2.51311964 - samples/sec: 125.83 - lr: 0.004687 2021-03-26 06:02:33,781 epoch 72 - iter 20/25 - loss 2.48758618 - samples/sec: 126.28 - lr: 0.004687 2021-03-26 06:02:34,687 epoch 72 - iter 22/25 - loss 2.43633575 - samples/sec: 141.56 - lr: 0.004687 2021-03-26 06:02:35,710 epoch 72 - iter 24/25 - loss 2.41600278 - samples/sec: 125.20 - lr: 0.004687 2021-03-26 06:02:36,128 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:02:36,129 EPOCH 72 done: loss 2.4187 - lr 0.0046875 2021-03-26 06:02:36,909 DEV : loss 6.750691890716553 - score 0.9062 2021-03-26 06:02:36,935 BAD EPOCHS (no improvement): 1 2021-03-26 06:02:36,936 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:02:37,935 epoch 73 - iter 2/25 - loss 2.43253404 - samples/sec: 128.30 - lr: 0.004687 2021-03-26 06:02:39,029 epoch 73 - iter 4/25 - loss 2.42423633 - samples/sec: 117.15 - lr: 0.004687 2021-03-26 06:02:40,048 epoch 73 - iter 6/25 - loss 2.41635050 - samples/sec: 125.78 - lr: 0.004687 2021-03-26 06:02:41,083 epoch 73 - iter 8/25 - loss 2.35884486 - samples/sec: 123.78 - lr: 0.004687 2021-03-26 06:02:42,091 epoch 73 - iter 10/25 - loss 2.36682016 - samples/sec: 127.16 - lr: 0.004687 2021-03-26 06:02:43,234 epoch 73 - iter 12/25 - loss 2.33669632 - samples/sec: 112.15 - lr: 0.004687 2021-03-26 06:02:44,171 epoch 73 - iter 14/25 - loss 2.29896097 - samples/sec: 136.88 - lr: 0.004687 2021-03-26 06:02:45,190 epoch 73 - iter 16/25 - loss 2.35040139 - samples/sec: 125.73 - lr: 0.004687 2021-03-26 06:02:46,184 epoch 73 - iter 18/25 - loss 2.40921725 - samples/sec: 129.00 - lr: 0.004687 2021-03-26 06:02:47,238 epoch 73 - iter 20/25 - loss 2.44946590 - samples/sec: 121.50 - lr: 0.004687 2021-03-26 06:02:48,285 epoch 73 - iter 22/25 - loss 2.42929926 - samples/sec: 122.49 - lr: 0.004687 2021-03-26 06:02:49,298 epoch 73 - iter 24/25 - loss 2.42566876 - samples/sec: 126.67 - lr: 0.004687 2021-03-26 06:02:49,682 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:02:49,682 EPOCH 73 done: loss 2.4041 - lr 0.0046875 2021-03-26 06:02:50,473 DEV : loss 6.749738693237305 - score 0.9058 2021-03-26 06:02:50,498 BAD EPOCHS (no improvement): 2 2021-03-26 06:02:50,499 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:02:51,521 epoch 74 - iter 2/25 - loss 2.52992201 - samples/sec: 125.48 - lr: 0.004687 2021-03-26 06:02:52,495 epoch 74 - iter 4/25 - loss 2.31653494 - samples/sec: 131.58 - lr: 0.004687 2021-03-26 06:02:53,454 epoch 74 - iter 6/25 - loss 2.40114681 - samples/sec: 133.68 - lr: 0.004687 2021-03-26 06:02:54,628 epoch 74 - iter 8/25 - loss 2.56239355 - samples/sec: 109.21 - lr: 0.004687 2021-03-26 06:02:55,790 epoch 74 - iter 10/25 - loss 2.53644116 - samples/sec: 110.26 - lr: 0.004687 2021-03-26 06:02:56,800 epoch 74 - iter 12/25 - loss 2.53346517 - samples/sec: 126.97 - lr: 0.004687 2021-03-26 06:02:57,694 epoch 74 - iter 14/25 - loss 2.49255146 - samples/sec: 143.46 - lr: 0.004687 2021-03-26 06:02:58,680 epoch 74 - iter 16/25 - loss 2.54234672 - samples/sec: 129.96 - lr: 0.004687 2021-03-26 06:02:59,637 epoch 74 - iter 18/25 - loss 2.44926276 - samples/sec: 133.94 - lr: 0.004687 2021-03-26 06:03:00,644 epoch 74 - iter 20/25 - loss 2.44066932 - samples/sec: 127.36 - lr: 0.004687 2021-03-26 06:03:01,663 epoch 74 - iter 22/25 - loss 2.48960504 - samples/sec: 125.95 - lr: 0.004687 2021-03-26 06:03:02,738 epoch 74 - iter 24/25 - loss 2.49861135 - samples/sec: 119.29 - lr: 0.004687 2021-03-26 06:03:03,138 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:03:03,139 EPOCH 74 done: loss 2.4818 - lr 0.0046875 2021-03-26 06:03:03,915 DEV : loss 6.755931854248047 - score 0.9062 2021-03-26 06:03:03,941 BAD EPOCHS (no improvement): 3 2021-03-26 06:03:03,942 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:03:04,893 epoch 75 - iter 2/25 - loss 2.53918266 - samples/sec: 134.96 - lr: 0.004687 2021-03-26 06:03:05,906 epoch 75 - iter 4/25 - loss 2.29154232 - samples/sec: 126.53 - lr: 0.004687 2021-03-26 06:03:06,923 epoch 75 - iter 6/25 - loss 2.21423191 - samples/sec: 126.16 - lr: 0.004687 2021-03-26 06:03:08,088 epoch 75 - iter 8/25 - loss 2.15976694 - samples/sec: 110.02 - lr: 0.004687 2021-03-26 06:03:09,092 epoch 75 - iter 10/25 - loss 2.17650853 - samples/sec: 127.66 - lr: 0.004687 2021-03-26 06:03:10,091 epoch 75 - iter 12/25 - loss 2.24466210 - samples/sec: 128.36 - lr: 0.004687 2021-03-26 06:03:11,241 epoch 75 - iter 14/25 - loss 2.26832461 - samples/sec: 111.53 - lr: 0.004687 2021-03-26 06:03:12,515 epoch 75 - iter 16/25 - loss 2.31134140 - samples/sec: 100.63 - lr: 0.004687 2021-03-26 06:03:13,511 epoch 75 - iter 18/25 - loss 2.32876862 - samples/sec: 128.86 - lr: 0.004687 2021-03-26 06:03:14,546 epoch 75 - iter 20/25 - loss 2.32084221 - samples/sec: 123.96 - lr: 0.004687 2021-03-26 06:03:15,564 epoch 75 - iter 22/25 - loss 2.30626246 - samples/sec: 126.04 - lr: 0.004687 2021-03-26 06:03:16,665 epoch 75 - iter 24/25 - loss 2.32581308 - samples/sec: 116.42 - lr: 0.004687 2021-03-26 06:03:17,029 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:03:17,030 EPOCH 75 done: loss 2.3309 - lr 0.0046875 2021-03-26 06:03:17,793 DEV : loss 6.7594757080078125 - score 0.9058 2021-03-26 06:03:17,819 BAD EPOCHS (no improvement): 4 2021-03-26 06:03:17,820 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:03:18,848 epoch 76 - iter 2/25 - loss 2.15280581 - samples/sec: 124.71 - lr: 0.002344 2021-03-26 06:03:19,792 epoch 76 - iter 4/25 - loss 2.35897440 - samples/sec: 135.76 - lr: 0.002344 2021-03-26 06:03:20,733 epoch 76 - iter 6/25 - loss 2.32432910 - samples/sec: 136.21 - lr: 0.002344 2021-03-26 06:03:21,692 epoch 76 - iter 8/25 - loss 2.36514843 - samples/sec: 133.77 - lr: 0.002344 2021-03-26 06:03:22,782 epoch 76 - iter 10/25 - loss 2.41345239 - samples/sec: 117.65 - lr: 0.002344 2021-03-26 06:03:23,884 epoch 76 - iter 12/25 - loss 2.36118354 - samples/sec: 116.41 - lr: 0.002344 2021-03-26 06:03:24,894 epoch 76 - iter 14/25 - loss 2.37700932 - samples/sec: 127.10 - lr: 0.002344 2021-03-26 06:03:25,917 epoch 76 - iter 16/25 - loss 2.37870736 - samples/sec: 125.32 - lr: 0.002344 2021-03-26 06:03:26,964 epoch 76 - iter 18/25 - loss 2.34423612 - samples/sec: 122.33 - lr: 0.002344 2021-03-26 06:03:27,945 epoch 76 - iter 20/25 - loss 2.35013263 - samples/sec: 130.63 - lr: 0.002344 2021-03-26 06:03:29,025 epoch 76 - iter 22/25 - loss 2.34488303 - samples/sec: 118.68 - lr: 0.002344 2021-03-26 06:03:30,051 epoch 76 - iter 24/25 - loss 2.38089673 - samples/sec: 124.92 - lr: 0.002344 2021-03-26 06:03:30,461 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:03:30,462 EPOCH 76 done: loss 2.4297 - lr 0.0023437 2021-03-26 06:03:31,220 DEV : loss 6.752521514892578 - score 0.907 2021-03-26 06:03:31,246 BAD EPOCHS (no improvement): 0 2021-03-26 06:03:40,746 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:03:41,763 epoch 77 - iter 2/25 - loss 2.50017357 - samples/sec: 126.27 - lr: 0.002344 2021-03-26 06:03:42,741 epoch 77 - iter 4/25 - loss 2.22791696 - samples/sec: 130.96 - lr: 0.002344 2021-03-26 06:03:43,758 epoch 77 - iter 6/25 - loss 2.39182313 - samples/sec: 126.02 - lr: 0.002344 2021-03-26 06:03:44,691 epoch 77 - iter 8/25 - loss 2.38184157 - samples/sec: 137.71 - lr: 0.002344 2021-03-26 06:03:45,694 epoch 77 - iter 10/25 - loss 2.44655206 - samples/sec: 127.74 - lr: 0.002344 2021-03-26 06:03:46,789 epoch 77 - iter 12/25 - loss 2.40551968 - samples/sec: 117.06 - lr: 0.002344 2021-03-26 06:03:47,802 epoch 77 - iter 14/25 - loss 2.38007069 - samples/sec: 126.62 - lr: 0.002344 2021-03-26 06:03:48,909 epoch 77 - iter 16/25 - loss 2.43085665 - samples/sec: 115.78 - lr: 0.002344 2021-03-26 06:03:49,932 epoch 77 - iter 18/25 - loss 2.45488136 - samples/sec: 125.38 - lr: 0.002344 2021-03-26 06:03:50,914 epoch 77 - iter 20/25 - loss 2.43090633 - samples/sec: 130.50 - lr: 0.002344 2021-03-26 06:03:51,955 epoch 77 - iter 22/25 - loss 2.38712107 - samples/sec: 123.25 - lr: 0.002344 2021-03-26 06:03:52,997 epoch 77 - iter 24/25 - loss 2.41249888 - samples/sec: 123.06 - lr: 0.002344 2021-03-26 06:03:53,411 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:03:53,412 EPOCH 77 done: loss 2.4017 - lr 0.0023437 2021-03-26 06:03:54,214 DEV : loss 6.756526470184326 - score 0.9062 2021-03-26 06:03:54,240 BAD EPOCHS (no improvement): 1 2021-03-26 06:03:54,241 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:03:55,315 epoch 78 - iter 2/25 - loss 2.16830981 - samples/sec: 119.40 - lr: 0.002344 2021-03-26 06:03:56,398 epoch 78 - iter 4/25 - loss 2.32413882 - samples/sec: 118.28 - lr: 0.002344 2021-03-26 06:03:57,365 epoch 78 - iter 6/25 - loss 2.29382400 - samples/sec: 132.76 - lr: 0.002344 2021-03-26 06:03:58,310 epoch 78 - iter 8/25 - loss 2.32180464 - samples/sec: 135.65 - lr: 0.002344 2021-03-26 06:03:59,297 epoch 78 - iter 10/25 - loss 2.41712666 - samples/sec: 129.89 - lr: 0.002344 2021-03-26 06:04:00,251 epoch 78 - iter 12/25 - loss 2.34759426 - samples/sec: 134.32 - lr: 0.002344 2021-03-26 06:04:01,282 epoch 78 - iter 14/25 - loss 2.37938215 - samples/sec: 124.38 - lr: 0.002344 2021-03-26 06:04:02,324 epoch 78 - iter 16/25 - loss 2.36263736 - samples/sec: 123.04 - lr: 0.002344 2021-03-26 06:04:03,311 epoch 78 - iter 18/25 - loss 2.35715266 - samples/sec: 129.94 - lr: 0.002344 2021-03-26 06:04:04,388 epoch 78 - iter 20/25 - loss 2.36736240 - samples/sec: 118.97 - lr: 0.002344 2021-03-26 06:04:05,463 epoch 78 - iter 22/25 - loss 2.38804259 - samples/sec: 119.23 - lr: 0.002344 2021-03-26 06:04:06,517 epoch 78 - iter 24/25 - loss 2.38424336 - samples/sec: 121.65 - lr: 0.002344 2021-03-26 06:04:07,017 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:04:07,018 EPOCH 78 done: loss 2.3911 - lr 0.0023437 2021-03-26 06:04:07,791 DEV : loss 6.753111839294434 - score 0.9066 2021-03-26 06:04:07,817 BAD EPOCHS (no improvement): 2 2021-03-26 06:04:07,818 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:04:08,882 epoch 79 - iter 2/25 - loss 2.51307201 - samples/sec: 120.52 - lr: 0.002344 2021-03-26 06:04:09,937 epoch 79 - iter 4/25 - loss 2.67237234 - samples/sec: 121.44 - lr: 0.002344 2021-03-26 06:04:10,994 epoch 79 - iter 6/25 - loss 2.48488834 - samples/sec: 121.36 - lr: 0.002344 2021-03-26 06:04:12,163 epoch 79 - iter 8/25 - loss 2.51524417 - samples/sec: 109.72 - lr: 0.002344 2021-03-26 06:04:13,153 epoch 79 - iter 10/25 - loss 2.38670518 - samples/sec: 129.48 - lr: 0.002344 2021-03-26 06:04:14,192 epoch 79 - iter 12/25 - loss 2.34001416 - samples/sec: 123.40 - lr: 0.002344 2021-03-26 06:04:15,247 epoch 79 - iter 14/25 - loss 2.27449070 - samples/sec: 121.54 - lr: 0.002344 2021-03-26 06:04:16,275 epoch 79 - iter 16/25 - loss 2.31747055 - samples/sec: 124.71 - lr: 0.002344 2021-03-26 06:04:17,248 epoch 79 - iter 18/25 - loss 2.27182851 - samples/sec: 131.83 - lr: 0.002344 2021-03-26 06:04:18,200 epoch 79 - iter 20/25 - loss 2.28024085 - samples/sec: 134.52 - lr: 0.002344 2021-03-26 06:04:19,199 epoch 79 - iter 22/25 - loss 2.29187663 - samples/sec: 128.35 - lr: 0.002344 2021-03-26 06:04:20,251 epoch 79 - iter 24/25 - loss 2.32610110 - samples/sec: 121.97 - lr: 0.002344 2021-03-26 06:04:20,656 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:04:20,658 EPOCH 79 done: loss 2.3529 - lr 0.0023437 2021-03-26 06:04:21,461 DEV : loss 6.750301361083984 - score 0.9058 2021-03-26 06:04:21,479 BAD EPOCHS (no improvement): 3 2021-03-26 06:04:21,480 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:04:22,530 epoch 80 - iter 2/25 - loss 2.18168497 - samples/sec: 122.11 - lr: 0.002344 2021-03-26 06:04:23,531 epoch 80 - iter 4/25 - loss 2.26245016 - samples/sec: 128.10 - lr: 0.002344 2021-03-26 06:04:24,522 epoch 80 - iter 6/25 - loss 2.26572899 - samples/sec: 129.44 - lr: 0.002344 2021-03-26 06:04:25,496 epoch 80 - iter 8/25 - loss 2.22850430 - samples/sec: 131.55 - lr: 0.002344 2021-03-26 06:04:26,602 epoch 80 - iter 10/25 - loss 2.22804742 - samples/sec: 115.94 - lr: 0.002344 2021-03-26 06:04:27,737 epoch 80 - iter 12/25 - loss 2.25041121 - samples/sec: 112.91 - lr: 0.002344 2021-03-26 06:04:28,740 epoch 80 - iter 14/25 - loss 2.35656748 - samples/sec: 127.88 - lr: 0.002344 2021-03-26 06:04:29,859 epoch 80 - iter 16/25 - loss 2.41935366 - samples/sec: 114.61 - lr: 0.002344 2021-03-26 06:04:30,861 epoch 80 - iter 18/25 - loss 2.41030420 - samples/sec: 127.86 - lr: 0.002344 2021-03-26 06:04:31,862 epoch 80 - iter 20/25 - loss 2.41881890 - samples/sec: 128.21 - lr: 0.002344 2021-03-26 06:04:32,852 epoch 80 - iter 22/25 - loss 2.43735432 - samples/sec: 129.48 - lr: 0.002344 2021-03-26 06:04:33,803 epoch 80 - iter 24/25 - loss 2.37674877 - samples/sec: 134.96 - lr: 0.002344 2021-03-26 06:04:34,286 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:04:34,287 EPOCH 80 done: loss 2.4121 - lr 0.0023437 2021-03-26 06:04:35,059 DEV : loss 6.750476837158203 - score 0.9062 2021-03-26 06:04:35,076 BAD EPOCHS (no improvement): 4 2021-03-26 06:04:35,077 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:04:36,051 epoch 81 - iter 2/25 - loss 2.33307743 - samples/sec: 131.62 - lr: 0.001172 2021-03-26 06:04:37,034 epoch 81 - iter 4/25 - loss 2.36750239 - samples/sec: 130.42 - lr: 0.001172 2021-03-26 06:04:38,019 epoch 81 - iter 6/25 - loss 2.15509367 - samples/sec: 130.15 - lr: 0.001172 2021-03-26 06:04:39,004 epoch 81 - iter 8/25 - loss 2.14618689 - samples/sec: 130.31 - lr: 0.001172 2021-03-26 06:04:39,985 epoch 81 - iter 10/25 - loss 2.29240470 - samples/sec: 130.81 - lr: 0.001172 2021-03-26 06:04:41,226 epoch 81 - iter 12/25 - loss 2.39395547 - samples/sec: 103.67 - lr: 0.001172 2021-03-26 06:04:42,262 epoch 81 - iter 14/25 - loss 2.49634773 - samples/sec: 123.76 - lr: 0.001172 2021-03-26 06:04:43,200 epoch 81 - iter 16/25 - loss 2.48210439 - samples/sec: 136.78 - lr: 0.001172 2021-03-26 06:04:44,216 epoch 81 - iter 18/25 - loss 2.53619586 - samples/sec: 126.17 - lr: 0.001172 2021-03-26 06:04:45,281 epoch 81 - iter 20/25 - loss 2.53158684 - samples/sec: 120.34 - lr: 0.001172 2021-03-26 06:04:46,511 epoch 81 - iter 22/25 - loss 2.54447044 - samples/sec: 104.27 - lr: 0.001172 2021-03-26 06:04:47,627 epoch 81 - iter 24/25 - loss 2.54063163 - samples/sec: 115.01 - lr: 0.001172 2021-03-26 06:04:48,035 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:04:48,036 EPOCH 81 done: loss 2.5130 - lr 0.0011719 2021-03-26 06:04:48,863 DEV : loss 6.748684883117676 - score 0.9062 2021-03-26 06:04:48,882 BAD EPOCHS (no improvement): 1 2021-03-26 06:04:48,883 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:04:50,061 epoch 82 - iter 2/25 - loss 2.38473833 - samples/sec: 108.86 - lr: 0.001172 2021-03-26 06:04:51,204 epoch 82 - iter 4/25 - loss 2.48814780 - samples/sec: 112.13 - lr: 0.001172 2021-03-26 06:04:52,241 epoch 82 - iter 6/25 - loss 2.38537379 - samples/sec: 123.63 - lr: 0.001172 2021-03-26 06:04:53,341 epoch 82 - iter 8/25 - loss 2.33337769 - samples/sec: 116.55 - lr: 0.001172 2021-03-26 06:04:54,469 epoch 82 - iter 10/25 - loss 2.37745783 - samples/sec: 113.66 - lr: 0.001172 2021-03-26 06:04:55,486 epoch 82 - iter 12/25 - loss 2.35676177 - samples/sec: 126.05 - lr: 0.001172 2021-03-26 06:04:56,435 epoch 82 - iter 14/25 - loss 2.34542345 - samples/sec: 135.12 - lr: 0.001172 2021-03-26 06:04:57,400 epoch 82 - iter 16/25 - loss 2.44239170 - samples/sec: 132.83 - lr: 0.001172 2021-03-26 06:04:58,376 epoch 82 - iter 18/25 - loss 2.41721334 - samples/sec: 131.38 - lr: 0.001172 2021-03-26 06:04:59,343 epoch 82 - iter 20/25 - loss 2.38730320 - samples/sec: 132.61 - lr: 0.001172 2021-03-26 06:05:00,370 epoch 82 - iter 22/25 - loss 2.36518301 - samples/sec: 124.76 - lr: 0.001172 2021-03-26 06:05:01,399 epoch 82 - iter 24/25 - loss 2.36671859 - samples/sec: 124.62 - lr: 0.001172 2021-03-26 06:05:01,784 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:05:01,785 EPOCH 82 done: loss 2.3515 - lr 0.0011719 2021-03-26 06:05:02,558 DEV : loss 6.749919891357422 - score 0.9054 2021-03-26 06:05:02,576 BAD EPOCHS (no improvement): 2 2021-03-26 06:05:02,577 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:05:03,579 epoch 83 - iter 2/25 - loss 2.50160813 - samples/sec: 127.96 - lr: 0.001172 2021-03-26 06:05:04,592 epoch 83 - iter 4/25 - loss 2.49640393 - samples/sec: 126.56 - lr: 0.001172 2021-03-26 06:05:05,683 epoch 83 - iter 6/25 - loss 2.44112500 - samples/sec: 117.45 - lr: 0.001172 2021-03-26 06:05:06,603 epoch 83 - iter 8/25 - loss 2.29882008 - samples/sec: 139.55 - lr: 0.001172 2021-03-26 06:05:07,521 epoch 83 - iter 10/25 - loss 2.32238986 - samples/sec: 139.72 - lr: 0.001172 2021-03-26 06:05:08,533 epoch 83 - iter 12/25 - loss 2.32237995 - samples/sec: 126.68 - lr: 0.001172 2021-03-26 06:05:09,491 epoch 83 - iter 14/25 - loss 2.42954922 - samples/sec: 133.91 - lr: 0.001172 2021-03-26 06:05:10,504 epoch 83 - iter 16/25 - loss 2.42919302 - samples/sec: 126.52 - lr: 0.001172 2021-03-26 06:05:11,528 epoch 83 - iter 18/25 - loss 2.42619081 - samples/sec: 125.13 - lr: 0.001172 2021-03-26 06:05:12,633 epoch 83 - iter 20/25 - loss 2.42287891 - samples/sec: 115.97 - lr: 0.001172 2021-03-26 06:05:13,594 epoch 83 - iter 22/25 - loss 2.42632379 - samples/sec: 133.48 - lr: 0.001172 2021-03-26 06:05:14,561 epoch 83 - iter 24/25 - loss 2.42381104 - samples/sec: 132.57 - lr: 0.001172 2021-03-26 06:05:15,115 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:05:15,117 EPOCH 83 done: loss 2.4270 - lr 0.0011719 2021-03-26 06:05:15,905 DEV : loss 6.750192165374756 - score 0.9058 2021-03-26 06:05:15,930 BAD EPOCHS (no improvement): 3 2021-03-26 06:05:15,931 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:05:17,094 epoch 84 - iter 2/25 - loss 2.93648970 - samples/sec: 110.24 - lr: 0.001172 2021-03-26 06:05:18,206 epoch 84 - iter 4/25 - loss 2.73000449 - samples/sec: 115.40 - lr: 0.001172 2021-03-26 06:05:19,245 epoch 84 - iter 6/25 - loss 2.49694339 - samples/sec: 123.50 - lr: 0.001172 2021-03-26 06:05:20,367 epoch 84 - iter 8/25 - loss 2.38217634 - samples/sec: 114.35 - lr: 0.001172 2021-03-26 06:05:21,393 epoch 84 - iter 10/25 - loss 2.38356829 - samples/sec: 124.90 - lr: 0.001172 2021-03-26 06:05:22,454 epoch 84 - iter 12/25 - loss 2.39194481 - samples/sec: 120.76 - lr: 0.001172 2021-03-26 06:05:23,530 epoch 84 - iter 14/25 - loss 2.37829908 - samples/sec: 119.18 - lr: 0.001172 2021-03-26 06:05:24,408 epoch 84 - iter 16/25 - loss 2.33745650 - samples/sec: 146.02 - lr: 0.001172 2021-03-26 06:05:25,357 epoch 84 - iter 18/25 - loss 2.37947904 - samples/sec: 135.05 - lr: 0.001172 2021-03-26 06:05:26,338 epoch 84 - iter 20/25 - loss 2.38875579 - samples/sec: 130.75 - lr: 0.001172 2021-03-26 06:05:27,409 epoch 84 - iter 22/25 - loss 2.39732352 - samples/sec: 119.67 - lr: 0.001172 2021-03-26 06:05:28,397 epoch 84 - iter 24/25 - loss 2.43036128 - samples/sec: 129.78 - lr: 0.001172 2021-03-26 06:05:28,943 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:05:28,945 EPOCH 84 done: loss 2.4095 - lr 0.0011719 2021-03-26 06:05:29,765 DEV : loss 6.74921989440918 - score 0.9054 2021-03-26 06:05:29,796 BAD EPOCHS (no improvement): 4 2021-03-26 06:05:29,796 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:05:30,862 epoch 85 - iter 2/25 - loss 2.46741247 - samples/sec: 120.43 - lr: 0.000586 2021-03-26 06:05:31,847 epoch 85 - iter 4/25 - loss 2.50471973 - samples/sec: 130.04 - lr: 0.000586 2021-03-26 06:05:32,920 epoch 85 - iter 6/25 - loss 2.49037719 - samples/sec: 119.44 - lr: 0.000586 2021-03-26 06:05:34,014 epoch 85 - iter 8/25 - loss 2.46623042 - samples/sec: 117.19 - lr: 0.000586 2021-03-26 06:05:35,145 epoch 85 - iter 10/25 - loss 2.53815739 - samples/sec: 113.39 - lr: 0.000586 2021-03-26 06:05:36,090 epoch 85 - iter 12/25 - loss 2.43846828 - samples/sec: 135.55 - lr: 0.000586 2021-03-26 06:05:37,054 epoch 85 - iter 14/25 - loss 2.50122946 - samples/sec: 133.03 - lr: 0.000586 2021-03-26 06:05:38,042 epoch 85 - iter 16/25 - loss 2.51318491 - samples/sec: 129.71 - lr: 0.000586 2021-03-26 06:05:39,062 epoch 85 - iter 18/25 - loss 2.51794383 - samples/sec: 125.81 - lr: 0.000586 2021-03-26 06:05:40,066 epoch 85 - iter 20/25 - loss 2.51388612 - samples/sec: 127.59 - lr: 0.000586 2021-03-26 06:05:41,068 epoch 85 - iter 22/25 - loss 2.50162070 - samples/sec: 127.99 - lr: 0.000586 2021-03-26 06:05:42,209 epoch 85 - iter 24/25 - loss 2.50571299 - samples/sec: 112.36 - lr: 0.000586 2021-03-26 06:05:42,646 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:05:42,647 EPOCH 85 done: loss 2.4964 - lr 0.0005859 2021-03-26 06:05:43,416 DEV : loss 6.748666763305664 - score 0.9058 2021-03-26 06:05:43,442 BAD EPOCHS (no improvement): 1 2021-03-26 06:05:43,443 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:05:44,603 epoch 86 - iter 2/25 - loss 2.20638573 - samples/sec: 110.50 - lr: 0.000586 2021-03-26 06:05:45,676 epoch 86 - iter 4/25 - loss 2.36967194 - samples/sec: 119.49 - lr: 0.000586 2021-03-26 06:05:46,743 epoch 86 - iter 6/25 - loss 2.34573106 - samples/sec: 120.13 - lr: 0.000586 2021-03-26 06:05:47,757 epoch 86 - iter 8/25 - loss 2.33546901 - samples/sec: 126.46 - lr: 0.000586 2021-03-26 06:05:48,822 epoch 86 - iter 10/25 - loss 2.33539317 - samples/sec: 120.32 - lr: 0.000586 2021-03-26 06:05:49,966 epoch 86 - iter 12/25 - loss 2.31430407 - samples/sec: 112.07 - lr: 0.000586 2021-03-26 06:05:51,001 epoch 86 - iter 14/25 - loss 2.36735562 - samples/sec: 123.82 - lr: 0.000586 2021-03-26 06:05:51,962 epoch 86 - iter 16/25 - loss 2.38468708 - samples/sec: 133.36 - lr: 0.000586 2021-03-26 06:05:52,917 epoch 86 - iter 18/25 - loss 2.36988348 - samples/sec: 134.25 - lr: 0.000586 2021-03-26 06:05:53,927 epoch 86 - iter 20/25 - loss 2.41210339 - samples/sec: 126.95 - lr: 0.000586 2021-03-26 06:05:54,892 epoch 86 - iter 22/25 - loss 2.42297436 - samples/sec: 132.83 - lr: 0.000586 2021-03-26 06:05:55,905 epoch 86 - iter 24/25 - loss 2.45856727 - samples/sec: 126.59 - lr: 0.000586 2021-03-26 06:05:56,363 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:05:56,363 EPOCH 86 done: loss 2.4923 - lr 0.0005859 2021-03-26 06:05:57,179 DEV : loss 6.748099327087402 - score 0.9054 2021-03-26 06:05:57,214 BAD EPOCHS (no improvement): 2 2021-03-26 06:05:57,214 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:05:58,279 epoch 87 - iter 2/25 - loss 2.76640713 - samples/sec: 120.45 - lr: 0.000586 2021-03-26 06:05:59,409 epoch 87 - iter 4/25 - loss 2.25503093 - samples/sec: 113.45 - lr: 0.000586 2021-03-26 06:06:00,409 epoch 87 - iter 6/25 - loss 2.23643498 - samples/sec: 128.20 - lr: 0.000586 2021-03-26 06:06:01,516 epoch 87 - iter 8/25 - loss 2.42684510 - samples/sec: 115.74 - lr: 0.000586 2021-03-26 06:06:02,479 epoch 87 - iter 10/25 - loss 2.44421749 - samples/sec: 133.10 - lr: 0.000586 2021-03-26 06:06:03,457 epoch 87 - iter 12/25 - loss 2.56141780 - samples/sec: 131.10 - lr: 0.000586 2021-03-26 06:06:04,589 epoch 87 - iter 14/25 - loss 2.52008237 - samples/sec: 113.18 - lr: 0.000586 2021-03-26 06:06:05,551 epoch 87 - iter 16/25 - loss 2.52018423 - samples/sec: 133.33 - lr: 0.000586 2021-03-26 06:06:06,588 epoch 87 - iter 18/25 - loss 2.54743120 - samples/sec: 123.54 - lr: 0.000586 2021-03-26 06:06:07,548 epoch 87 - iter 20/25 - loss 2.50725095 - samples/sec: 133.75 - lr: 0.000586 2021-03-26 06:06:08,610 epoch 87 - iter 22/25 - loss 2.54458821 - samples/sec: 120.70 - lr: 0.000586 2021-03-26 06:06:09,664 epoch 87 - iter 24/25 - loss 2.53648801 - samples/sec: 121.68 - lr: 0.000586 2021-03-26 06:06:10,095 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:06:10,096 EPOCH 87 done: loss 2.5160 - lr 0.0005859 2021-03-26 06:06:10,915 DEV : loss 6.7480010986328125 - score 0.9054 2021-03-26 06:06:10,942 BAD EPOCHS (no improvement): 3 2021-03-26 06:06:10,942 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:06:11,965 epoch 88 - iter 2/25 - loss 2.35491943 - samples/sec: 125.32 - lr: 0.000586 2021-03-26 06:06:13,099 epoch 88 - iter 4/25 - loss 2.46524775 - samples/sec: 112.98 - lr: 0.000586 2021-03-26 06:06:14,226 epoch 88 - iter 6/25 - loss 2.32718579 - samples/sec: 113.69 - lr: 0.000586 2021-03-26 06:06:15,238 epoch 88 - iter 8/25 - loss 2.20188485 - samples/sec: 126.78 - lr: 0.000586 2021-03-26 06:06:16,358 epoch 88 - iter 10/25 - loss 2.20019561 - samples/sec: 114.55 - lr: 0.000586 2021-03-26 06:06:17,428 epoch 88 - iter 12/25 - loss 2.24452155 - samples/sec: 119.80 - lr: 0.000586 2021-03-26 06:06:18,464 epoch 88 - iter 14/25 - loss 2.23918286 - samples/sec: 123.74 - lr: 0.000586 2021-03-26 06:06:19,371 epoch 88 - iter 16/25 - loss 2.28264054 - samples/sec: 141.38 - lr: 0.000586 2021-03-26 06:06:20,329 epoch 88 - iter 18/25 - loss 2.28697127 - samples/sec: 133.84 - lr: 0.000586 2021-03-26 06:06:21,244 epoch 88 - iter 20/25 - loss 2.33569470 - samples/sec: 140.01 - lr: 0.000586 2021-03-26 06:06:22,231 epoch 88 - iter 22/25 - loss 2.35366069 - samples/sec: 129.84 - lr: 0.000586 2021-03-26 06:06:23,213 epoch 88 - iter 24/25 - loss 2.34399285 - samples/sec: 130.59 - lr: 0.000586 2021-03-26 06:06:23,692 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:06:23,694 EPOCH 88 done: loss 2.3643 - lr 0.0005859 2021-03-26 06:06:24,466 DEV : loss 6.7463812828063965 - score 0.9058 2021-03-26 06:06:24,488 BAD EPOCHS (no improvement): 4 2021-03-26 06:06:24,489 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:06:25,540 epoch 89 - iter 2/25 - loss 2.73587209 - samples/sec: 121.92 - lr: 0.000293 2021-03-26 06:06:26,501 epoch 89 - iter 4/25 - loss 2.68548825 - samples/sec: 133.47 - lr: 0.000293 2021-03-26 06:06:27,498 epoch 89 - iter 6/25 - loss 2.69872004 - samples/sec: 128.66 - lr: 0.000293 2021-03-26 06:06:28,456 epoch 89 - iter 8/25 - loss 2.63567401 - samples/sec: 133.91 - lr: 0.000293 2021-03-26 06:06:29,545 epoch 89 - iter 10/25 - loss 2.61476969 - samples/sec: 117.69 - lr: 0.000293 2021-03-26 06:06:30,674 epoch 89 - iter 12/25 - loss 2.61334603 - samples/sec: 113.54 - lr: 0.000293 2021-03-26 06:06:31,763 epoch 89 - iter 14/25 - loss 2.59073320 - samples/sec: 117.78 - lr: 0.000293 2021-03-26 06:06:32,784 epoch 89 - iter 16/25 - loss 2.52647568 - samples/sec: 125.50 - lr: 0.000293 2021-03-26 06:06:33,805 epoch 89 - iter 18/25 - loss 2.51834848 - samples/sec: 125.62 - lr: 0.000293 2021-03-26 06:06:34,896 epoch 89 - iter 20/25 - loss 2.46717817 - samples/sec: 117.44 - lr: 0.000293 2021-03-26 06:06:35,884 epoch 89 - iter 22/25 - loss 2.45679012 - samples/sec: 129.77 - lr: 0.000293 2021-03-26 06:06:36,865 epoch 89 - iter 24/25 - loss 2.39095766 - samples/sec: 130.53 - lr: 0.000293 2021-03-26 06:06:37,303 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:06:37,304 EPOCH 89 done: loss 2.3895 - lr 0.0002930 2021-03-26 06:06:38,066 DEV : loss 6.746824264526367 - score 0.9058 2021-03-26 06:06:38,087 BAD EPOCHS (no improvement): 1 2021-03-26 06:06:38,088 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:06:39,103 epoch 90 - iter 2/25 - loss 2.29537117 - samples/sec: 126.45 - lr: 0.000293 2021-03-26 06:06:40,283 epoch 90 - iter 4/25 - loss 2.12782717 - samples/sec: 108.60 - lr: 0.000293 2021-03-26 06:06:41,405 epoch 90 - iter 6/25 - loss 2.28929679 - samples/sec: 114.16 - lr: 0.000293 2021-03-26 06:06:42,392 epoch 90 - iter 8/25 - loss 2.35938233 - samples/sec: 129.89 - lr: 0.000293 2021-03-26 06:06:43,303 epoch 90 - iter 10/25 - loss 2.32694597 - samples/sec: 140.65 - lr: 0.000293 2021-03-26 06:06:44,302 epoch 90 - iter 12/25 - loss 2.37408519 - samples/sec: 128.31 - lr: 0.000293 2021-03-26 06:06:45,326 epoch 90 - iter 14/25 - loss 2.40497916 - samples/sec: 125.28 - lr: 0.000293 2021-03-26 06:06:46,292 epoch 90 - iter 16/25 - loss 2.43432885 - samples/sec: 132.67 - lr: 0.000293 2021-03-26 06:06:47,297 epoch 90 - iter 18/25 - loss 2.41117813 - samples/sec: 127.55 - lr: 0.000293 2021-03-26 06:06:48,362 epoch 90 - iter 20/25 - loss 2.34449456 - samples/sec: 120.35 - lr: 0.000293 2021-03-26 06:06:49,500 epoch 90 - iter 22/25 - loss 2.32467386 - samples/sec: 112.72 - lr: 0.000293 2021-03-26 06:06:50,504 epoch 90 - iter 24/25 - loss 2.35763361 - samples/sec: 127.72 - lr: 0.000293 2021-03-26 06:06:51,075 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:06:51,075 EPOCH 90 done: loss 2.3738 - lr 0.0002930 2021-03-26 06:06:51,866 DEV : loss 6.747221946716309 - score 0.9058 2021-03-26 06:06:51,889 BAD EPOCHS (no improvement): 2 2021-03-26 06:06:51,889 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:06:52,864 epoch 91 - iter 2/25 - loss 2.09060097 - samples/sec: 131.51 - lr: 0.000293 2021-03-26 06:06:53,871 epoch 91 - iter 4/25 - loss 2.77089983 - samples/sec: 127.22 - lr: 0.000293 2021-03-26 06:06:54,886 epoch 91 - iter 6/25 - loss 2.58453842 - samples/sec: 126.39 - lr: 0.000293 2021-03-26 06:06:56,033 epoch 91 - iter 8/25 - loss 2.52161238 - samples/sec: 111.74 - lr: 0.000293 2021-03-26 06:06:57,466 epoch 91 - iter 10/25 - loss 2.41345448 - samples/sec: 89.46 - lr: 0.000293 2021-03-26 06:06:58,844 epoch 91 - iter 12/25 - loss 2.37983878 - samples/sec: 92.99 - lr: 0.000293 2021-03-26 06:07:00,250 epoch 91 - iter 14/25 - loss 2.36699120 - samples/sec: 91.14 - lr: 0.000293 2021-03-26 06:07:01,595 epoch 91 - iter 16/25 - loss 2.36627452 - samples/sec: 95.30 - lr: 0.000293 2021-03-26 06:07:02,860 epoch 91 - iter 18/25 - loss 2.38054042 - samples/sec: 101.27 - lr: 0.000293 2021-03-26 06:07:04,215 epoch 91 - iter 20/25 - loss 2.38775070 - samples/sec: 94.62 - lr: 0.000293 2021-03-26 06:07:05,525 epoch 91 - iter 22/25 - loss 2.42452406 - samples/sec: 97.77 - lr: 0.000293 2021-03-26 06:07:06,861 epoch 91 - iter 24/25 - loss 2.43479525 - samples/sec: 95.92 - lr: 0.000293 2021-03-26 06:07:07,426 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:07:07,427 EPOCH 91 done: loss 2.4289 - lr 0.0002930 2021-03-26 06:07:08,309 DEV : loss 6.747243404388428 - score 0.9058 2021-03-26 06:07:08,345 BAD EPOCHS (no improvement): 3 2021-03-26 06:07:08,346 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:07:09,714 epoch 92 - iter 2/25 - loss 2.56387246 - samples/sec: 93.67 - lr: 0.000293 2021-03-26 06:07:11,135 epoch 92 - iter 4/25 - loss 2.47984356 - samples/sec: 90.18 - lr: 0.000293 2021-03-26 06:07:12,472 epoch 92 - iter 6/25 - loss 2.34325659 - samples/sec: 95.81 - lr: 0.000293 2021-03-26 06:07:13,501 epoch 92 - iter 8/25 - loss 2.34978437 - samples/sec: 124.61 - lr: 0.000293 2021-03-26 06:07:14,436 epoch 92 - iter 10/25 - loss 2.38845448 - samples/sec: 137.32 - lr: 0.000293 2021-03-26 06:07:15,382 epoch 92 - iter 12/25 - loss 2.38777759 - samples/sec: 135.50 - lr: 0.000293 2021-03-26 06:07:16,411 epoch 92 - iter 14/25 - loss 2.46206682 - samples/sec: 124.52 - lr: 0.000293 2021-03-26 06:07:17,485 epoch 92 - iter 16/25 - loss 2.48083726 - samples/sec: 119.50 - lr: 0.000293 2021-03-26 06:07:18,507 epoch 92 - iter 18/25 - loss 2.50218729 - samples/sec: 125.45 - lr: 0.000293 2021-03-26 06:07:19,597 epoch 92 - iter 20/25 - loss 2.49390016 - samples/sec: 117.58 - lr: 0.000293 2021-03-26 06:07:20,652 epoch 92 - iter 22/25 - loss 2.48165541 - samples/sec: 121.53 - lr: 0.000293 2021-03-26 06:07:21,588 epoch 92 - iter 24/25 - loss 2.45996833 - samples/sec: 137.24 - lr: 0.000293 2021-03-26 06:07:22,022 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:07:22,023 EPOCH 92 done: loss 2.4582 - lr 0.0002930 2021-03-26 06:07:22,843 DEV : loss 6.7463531494140625 - score 0.9058 2021-03-26 06:07:22,865 BAD EPOCHS (no improvement): 4 2021-03-26 06:07:22,866 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:07:23,916 epoch 93 - iter 2/25 - loss 2.83326006 - samples/sec: 122.07 - lr: 0.000146 2021-03-26 06:07:24,990 epoch 93 - iter 4/25 - loss 2.57940161 - samples/sec: 119.39 - lr: 0.000146 2021-03-26 06:07:26,116 epoch 93 - iter 6/25 - loss 2.51093678 - samples/sec: 113.84 - lr: 0.000146 2021-03-26 06:07:27,157 epoch 93 - iter 8/25 - loss 2.54783881 - samples/sec: 123.21 - lr: 0.000146 2021-03-26 06:07:28,087 epoch 93 - iter 10/25 - loss 2.52907882 - samples/sec: 137.85 - lr: 0.000146 2021-03-26 06:07:29,180 epoch 93 - iter 12/25 - loss 2.53294051 - samples/sec: 117.34 - lr: 0.000146 2021-03-26 06:07:30,152 epoch 93 - iter 14/25 - loss 2.58373092 - samples/sec: 131.92 - lr: 0.000146 2021-03-26 06:07:31,101 epoch 93 - iter 16/25 - loss 2.49791358 - samples/sec: 135.12 - lr: 0.000146 2021-03-26 06:07:32,036 epoch 93 - iter 18/25 - loss 2.52266951 - samples/sec: 137.17 - lr: 0.000146 2021-03-26 06:07:33,002 epoch 93 - iter 20/25 - loss 2.55563478 - samples/sec: 132.64 - lr: 0.000146 2021-03-26 06:07:33,961 epoch 93 - iter 22/25 - loss 2.51628619 - samples/sec: 133.82 - lr: 0.000146 2021-03-26 06:07:35,016 epoch 93 - iter 24/25 - loss 2.50995046 - samples/sec: 121.45 - lr: 0.000146 2021-03-26 06:07:35,421 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:07:35,422 EPOCH 93 done: loss 2.5024 - lr 0.0001465 2021-03-26 06:07:36,224 DEV : loss 6.746026515960693 - score 0.9058 2021-03-26 06:07:36,251 BAD EPOCHS (no improvement): 1 2021-03-26 06:07:36,252 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:07:37,310 epoch 94 - iter 2/25 - loss 2.69705701 - samples/sec: 121.25 - lr: 0.000146 2021-03-26 06:07:38,263 epoch 94 - iter 4/25 - loss 2.63027239 - samples/sec: 134.47 - lr: 0.000146 2021-03-26 06:07:39,516 epoch 94 - iter 6/25 - loss 2.61560170 - samples/sec: 102.22 - lr: 0.000146 2021-03-26 06:07:40,577 epoch 94 - iter 8/25 - loss 2.48729473 - samples/sec: 120.79 - lr: 0.000146 2021-03-26 06:07:41,696 epoch 94 - iter 10/25 - loss 2.48122675 - samples/sec: 114.59 - lr: 0.000146 2021-03-26 06:07:42,723 epoch 94 - iter 12/25 - loss 2.45633954 - samples/sec: 124.84 - lr: 0.000146 2021-03-26 06:07:43,694 epoch 94 - iter 14/25 - loss 2.50454484 - samples/sec: 131.93 - lr: 0.000146 2021-03-26 06:07:44,786 epoch 94 - iter 16/25 - loss 2.55948451 - samples/sec: 117.41 - lr: 0.000146 2021-03-26 06:07:45,859 epoch 94 - iter 18/25 - loss 2.50821441 - samples/sec: 119.48 - lr: 0.000146 2021-03-26 06:07:46,975 epoch 94 - iter 20/25 - loss 2.54369184 - samples/sec: 114.87 - lr: 0.000146 2021-03-26 06:07:47,924 epoch 94 - iter 22/25 - loss 2.56217494 - samples/sec: 135.23 - lr: 0.000146 2021-03-26 06:07:48,912 epoch 94 - iter 24/25 - loss 2.52295050 - samples/sec: 129.79 - lr: 0.000146 2021-03-26 06:07:49,312 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:07:49,314 EPOCH 94 done: loss 2.5068 - lr 0.0001465 2021-03-26 06:07:50,089 DEV : loss 6.7457427978515625 - score 0.9058 2021-03-26 06:07:50,115 BAD EPOCHS (no improvement): 2 2021-03-26 06:07:50,116 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:07:51,198 epoch 95 - iter 2/25 - loss 2.21429241 - samples/sec: 118.45 - lr: 0.000146 2021-03-26 06:07:52,201 epoch 95 - iter 4/25 - loss 2.23874307 - samples/sec: 127.73 - lr: 0.000146 2021-03-26 06:07:53,273 epoch 95 - iter 6/25 - loss 2.25965214 - samples/sec: 119.73 - lr: 0.000146 2021-03-26 06:07:54,322 epoch 95 - iter 8/25 - loss 2.37573701 - samples/sec: 122.20 - lr: 0.000146 2021-03-26 06:07:55,351 epoch 95 - iter 10/25 - loss 2.34428158 - samples/sec: 124.53 - lr: 0.000146 2021-03-26 06:07:56,347 epoch 95 - iter 12/25 - loss 2.31626995 - samples/sec: 128.90 - lr: 0.000146 2021-03-26 06:07:57,367 epoch 95 - iter 14/25 - loss 2.34982332 - samples/sec: 125.55 - lr: 0.000146 2021-03-26 06:07:58,551 epoch 95 - iter 16/25 - loss 2.37993184 - samples/sec: 108.24 - lr: 0.000146 2021-03-26 06:07:59,564 epoch 95 - iter 18/25 - loss 2.36954919 - samples/sec: 126.58 - lr: 0.000146 2021-03-26 06:08:00,535 epoch 95 - iter 20/25 - loss 2.35135136 - samples/sec: 132.01 - lr: 0.000146 2021-03-26 06:08:01,488 epoch 95 - iter 22/25 - loss 2.30894905 - samples/sec: 134.72 - lr: 0.000146 2021-03-26 06:08:02,480 epoch 95 - iter 24/25 - loss 2.33361017 - samples/sec: 129.32 - lr: 0.000146 2021-03-26 06:08:02,882 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:02,883 EPOCH 95 done: loss 2.3231 - lr 0.0001465 2021-03-26 06:08:03,669 DEV : loss 6.745711326599121 - score 0.9058 2021-03-26 06:08:03,694 BAD EPOCHS (no improvement): 3 2021-03-26 06:08:03,694 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:04,616 epoch 96 - iter 2/25 - loss 2.30606425 - samples/sec: 139.22 - lr: 0.000146 2021-03-26 06:08:05,646 epoch 96 - iter 4/25 - loss 2.47422612 - samples/sec: 124.52 - lr: 0.000146 2021-03-26 06:08:06,612 epoch 96 - iter 6/25 - loss 2.45720597 - samples/sec: 132.77 - lr: 0.000146 2021-03-26 06:08:07,633 epoch 96 - iter 8/25 - loss 2.44069329 - samples/sec: 125.46 - lr: 0.000146 2021-03-26 06:08:08,753 epoch 96 - iter 10/25 - loss 2.38360977 - samples/sec: 114.49 - lr: 0.000146 2021-03-26 06:08:09,991 epoch 96 - iter 12/25 - loss 2.33929481 - samples/sec: 103.49 - lr: 0.000146 2021-03-26 06:08:11,153 epoch 96 - iter 14/25 - loss 2.29825859 - samples/sec: 110.32 - lr: 0.000146 2021-03-26 06:08:12,130 epoch 96 - iter 16/25 - loss 2.28283183 - samples/sec: 131.24 - lr: 0.000146 2021-03-26 06:08:13,162 epoch 96 - iter 18/25 - loss 2.28030713 - samples/sec: 124.31 - lr: 0.000146 2021-03-26 06:08:14,189 epoch 96 - iter 20/25 - loss 2.30450888 - samples/sec: 124.70 - lr: 0.000146 2021-03-26 06:08:15,193 epoch 96 - iter 22/25 - loss 2.33098056 - samples/sec: 127.78 - lr: 0.000146 2021-03-26 06:08:16,322 epoch 96 - iter 24/25 - loss 2.31253702 - samples/sec: 113.43 - lr: 0.000146 2021-03-26 06:08:16,783 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:16,784 EPOCH 96 done: loss 2.2834 - lr 0.0001465 2021-03-26 06:08:17,592 DEV : loss 6.745593070983887 - score 0.9058 2021-03-26 06:08:17,618 BAD EPOCHS (no improvement): 4 2021-03-26 06:08:17,619 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:17,619 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:17,620 learning rate too small - quitting training! 2021-03-26 06:08:17,620 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:27,028 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:27,029 Testing using best model ... 2021-03-26 06:08:27,029 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.3_202103260540/best-model.pt 2021-03-26 06:08:34,565 0.9157 2021-03-26 06:08:34,565 Results: - F-score (micro): 0.912 - F-score (macro): 0.5618 - Accuracy (incl. no class): 0.9157 By class: precision recall f1-score support NOUN 0.9314 0.9359 0.9336 421 PRON 0.9732 0.9909 0.9820 220 VERB 0.9359 0.9359 0.9359 156 AUX 0.9200 0.9787 0.9485 47 PROPN 0.9333 1.0000 0.9655 14 ADV 0.9423 0.8909 0.9159 110 ADP 0.9730 0.9643 0.9686 112 ADJ 0.9174 0.8333 0.8734 120 DET 0.9649 0.9016 0.9322 61 CCONJ 1.0000 0.9620 0.9806 79 NUM 0.8000 0.8889 0.8421 18 PART 0.9241 0.9605 0.9419 228 SCONJ 0.8684 0.9706 0.9167 34 PUNCT 1.0000 1.0000 1.0000 22 INTJ 1.0000 0.8750 0.9333 8 MENTION 1.0000 1.0000 1.0000 21 V 0.8990 0.9082 0.9036 98 FUT_PART+V+PREP+PRON 1.0000 0.5000 0.6667 2 PREP+ADV 0.0000 1.0000 0.0000 0 ADJ+NSUFF 0.6897 0.9091 0.7843 22 PREP+PRON 0.9259 0.8621 0.8929 29 PUNC 0.9936 1.0000 0.9968 156 EOS 1.0000 1.0000 1.0000 70 DET+NOUN 0.9390 0.9872 0.9625 78 PREP 0.9867 0.9610 0.9737 77 HASH 0.9600 0.8889 0.9231 27 PROG_PART+V 0.8919 0.9429 0.9167 35 CONJ 1.0000 0.9762 0.9880 42 NOUN+CASE 0.2500 0.5000 0.3333 2 NOUN+PRON 0.8701 0.8816 0.8758 76 V+PRON 0.6714 0.8103 0.7344 58 PART+PRON 0.9167 0.8462 0.8800 13 NOUN+NSUFF+PRON 0.7895 0.8333 0.8108 18 PROG_PART+V+PRON 0.7500 0.7500 0.7500 16 PREP+NOUN+NSUFF 0.3333 0.3333 0.3333 3 NOUN+NSUFF 0.9167 0.8684 0.8919 38 DET+NOUN+NSUFF 0.8400 0.8400 0.8400 25 FOREIGN 1.0000 1.0000 1.0000 2 CONJ+NOUN 0.7273 0.8889 0.8000 9 CONJ+PART 0.8462 1.0000 0.9167 11 PART+NSUFF 0.0000 1.0000 0.0000 0 PREP+PART 1.0000 0.0000 0.0000 2 ADJ+NSUFF+PRON 0.5000 0.5000 0.5000 2 PREP+NOUN 0.6923 0.8182 0.7500 11 PART+NOUN 0.6667 0.3333 0.4444 6 PREP+PART+PRON 1.0000 1.0000 1.0000 3 PREP+DET+ADJ 1.0000 0.0000 0.0000 1 PREP+DET+NOUN 0.7500 0.6667 0.7059 9 PREP+DET+NOUN+NSUFF 0.5000 0.6667 0.5714 3 CONJ+V+PRON 0.2857 0.5000 0.3636 4 CONJ+V 0.3333 0.6667 0.4444 3 CONJ+DET+NOUN 0.3333 0.6667 0.4444 3 CONJ+NOUN+NSUFF+PRON 1.0000 0.0000 0.0000 1 CONJ+NOUN+PRON 0.7143 0.8333 0.7692 6 PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 ADJ+PRON 0.3333 0.2000 0.2500 5 V+PRON+PRON 0.6000 0.3000 0.4000 10 CONJ+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 DET+ADJ+NSUFF 0.5000 0.5000 0.5000 6 EMOT 0.8571 1.0000 0.9231 6 CONJ+PREP 1.0000 0.5000 0.6667 2 PRON+DET+NOUN 0.0000 1.0000 0.0000 0 CONJ+PRON 1.0000 1.0000 1.0000 4 DET+ADJ 1.0000 0.8182 0.9000 11 PART+V+PRON+NEG_PART 0.5000 1.0000 0.6667 2 V+PRON+PREP+PRON 0.5000 0.5000 0.5000 2 PROG_PART+V+PREP+PRON 0.0000 0.0000 0.0000 1 CONJ+PROG_PART+V 1.0000 0.5000 0.6667 2 URL 1.0000 1.0000 1.0000 3 NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+PROG_PART+V+NEG_PART 0.5000 0.5000 0.5000 2 FUT_PART+V 0.9167 0.9167 0.9167 12 PRON+NOUN 1.0000 0.0000 0.0000 1 NUM+NSUFF 0.0000 0.0000 0.0000 3 CONJ+PREP+PRON 1.0000 1.0000 1.0000 1 FUT_PART 1.0000 1.0000 1.0000 3 PREP+NOUN+PRON 0.6667 0.5714 0.6154 7 PART+V 1.0000 0.0000 0.0000 2 V+PREP+PRON 0.6667 0.4000 0.5000 5 CONJ+V+PRON+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+DET+ADJ 0.0000 1.0000 0.0000 0 PART+NOUN+PRON 1.0000 0.0000 0.0000 1 ADJ+NSUFF+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+NOUN+NSUFF 0.6667 1.0000 0.8000 2 CONJ+PART+PROG_PART+V 1.0000 0.0000 0.0000 1 ADV+NSUFF 1.0000 1.0000 1.0000 1 PART+V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+V+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 PART+PREP+NEG_PART 1.0000 1.0000 1.0000 2 ADJ+CASE 1.0000 0.5000 0.6667 2 CONJ+ADV+NSUFF 1.0000 0.0000 0.0000 1 DET+NUM 1.0000 1.0000 1.0000 1 CONJ+PROG_PART+V+PRON 1.0000 0.0000 0.0000 1 PREP+DET+NOUN+NSUFF+PART 1.0000 0.0000 0.0000 1 CONJ+PREP+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 FUT_PART+V+PRON 1.0000 0.2500 0.4000 4 CONJ+ADJ+NSUFF 1.0000 0.0000 0.0000 2 NOUN+CASE+PRON 1.0000 1.0000 1.0000 1 CONJ+NOUN+CASE 1.0000 0.0000 0.0000 1 CONJ+PART+V 0.0000 1.0000 0.0000 0 PART+PREP+PRON 1.0000 0.0000 0.0000 1 V+NSUFF 1.0000 0.0000 0.0000 1 PREP+NOUN+NSUFF+PRON 1.0000 0.0000 0.0000 1 PROG_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 PART+NOUN+NSUFF 1.0000 0.0000 0.0000 1 PART+V+NEG_PART 1.0000 1.0000 1.0000 1 micro avg 0.9125 0.9115 0.9120 2757 macro avg 0.8015 0.6232 0.5618 2757 weighted avg 0.9195 0.9115 0.9086 2757 2021-03-26 06:08:34,566 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:34,566 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:41,131 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC 2021-03-26 06:08:41,132 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu 2021-03-26 06:08:41,132 Dev: None 2021-03-26 06:08:41,133 Test: None 2021-03-26 06:08:41,417 Reading data from ../../Datasets_adhoc/UD_MADAR 2021-03-26 06:08:41,417 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu 2021-03-26 06:08:41,418 Dev: None 2021-03-26 06:08:41,418 Test: None 2021-03-26 06:08:41,462 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 06:08:41,462 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt 2021-03-26 06:08:41,462 Dev: None 2021-03-26 06:08:41,462 Test: None 2021-03-26 06:08:41,610 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 06:08:41,611 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt 2021-03-26 06:08:41,611 Dev: None 2021-03-26 06:08:41,611 Test: None 2021-03-26 06:08:41,782 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 06:08:41,783 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt 2021-03-26 06:08:41,783 Dev: None 2021-03-26 06:08:41,783 Test: None 2021-03-26 06:08:41,954 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 06:08:41,955 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt 2021-03-26 06:08:41,955 Dev: None 2021-03-26 06:08:41,956 Test: None 2021-03-26 06:08:42,103 Filtering long sentences 2021-03-26 06:08:42,144 MultiCorpus: 1573 train + 176 dev + 195 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences 2021-03-26 06:08:42,545 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:42,546 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): WordEmbeddings('ar') (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) (list_embedding_2): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=206, bias=True) (beta): 1.0 (weights): None (weight_tensor) None )" 2021-03-26 06:08:42,546 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:42,546 Corpus: "MultiCorpus: 1573 train + 176 dev + 195 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" 2021-03-26 06:08:42,547 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:42,547 Parameters: 2021-03-26 06:08:42,547 - learning_rate: "0.5" 2021-03-26 06:08:42,548 - mini_batch_size: "32" 2021-03-26 06:08:42,548 - patience: "3" 2021-03-26 06:08:42,548 - anneal_factor: "0.5" 2021-03-26 06:08:42,548 - max_epochs: "150" 2021-03-26 06:08:42,549 - shuffle: "True" 2021-03-26 06:08:42,549 - train_with_dev: "False" 2021-03-26 06:08:42,549 - batch_growth_annealing: "False" 2021-03-26 06:08:42,549 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:42,550 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260608" 2021-03-26 06:08:42,550 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:42,550 Device: cuda:0 2021-03-26 06:08:42,551 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:42,551 Embeddings storage mode: cpu 2021-03-26 06:08:42,552 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:08:45,412 epoch 1 - iter 5/50 - loss 82.38076324 - samples/sec: 56.00 - lr: 0.500000 2021-03-26 06:08:47,767 epoch 1 - iter 10/50 - loss 72.45342216 - samples/sec: 67.97 - lr: 0.500000 2021-03-26 06:08:50,426 epoch 1 - iter 15/50 - loss 69.72581075 - samples/sec: 60.21 - lr: 0.500000 2021-03-26 06:08:52,798 epoch 1 - iter 20/50 - loss 65.73783169 - samples/sec: 67.50 - lr: 0.500000 2021-03-26 06:08:55,293 epoch 1 - iter 25/50 - loss 62.81619507 - samples/sec: 64.18 - lr: 0.500000 2021-03-26 06:08:57,724 epoch 1 - iter 30/50 - loss 59.64540342 - samples/sec: 65.87 - lr: 0.500000 2021-03-26 06:09:00,117 epoch 1 - iter 35/50 - loss 56.33768899 - samples/sec: 66.90 - lr: 0.500000 2021-03-26 06:09:02,584 epoch 1 - iter 40/50 - loss 53.98569193 - samples/sec: 64.92 - lr: 0.500000 2021-03-26 06:09:05,056 epoch 1 - iter 45/50 - loss 51.83909531 - samples/sec: 64.76 - lr: 0.500000 2021-03-26 06:09:07,437 epoch 1 - iter 50/50 - loss 49.78158417 - samples/sec: 67.27 - lr: 0.500000 2021-03-26 06:09:07,438 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:09:07,438 EPOCH 1 done: loss 49.7816 - lr 0.5000000 2021-03-26 06:09:08,778 DEV : loss 32.66966247558594 - score 0.5248 2021-03-26 06:09:08,803 BAD EPOCHS (no improvement): 0 2021-03-26 06:09:18,311 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:09:20,362 epoch 2 - iter 5/50 - loss 29.21817436 - samples/sec: 78.13 - lr: 0.500000 2021-03-26 06:09:22,218 epoch 2 - iter 10/50 - loss 27.90745525 - samples/sec: 86.29 - lr: 0.500000 2021-03-26 06:09:24,250 epoch 2 - iter 15/50 - loss 27.92045682 - samples/sec: 78.85 - lr: 0.500000 2021-03-26 06:09:26,263 epoch 2 - iter 20/50 - loss 27.60578327 - samples/sec: 79.54 - lr: 0.500000 2021-03-26 06:09:28,349 epoch 2 - iter 25/50 - loss 27.37625938 - samples/sec: 76.80 - lr: 0.500000 2021-03-26 06:09:30,286 epoch 2 - iter 30/50 - loss 26.46395251 - samples/sec: 82.74 - lr: 0.500000 2021-03-26 06:09:32,155 epoch 2 - iter 35/50 - loss 25.86999474 - samples/sec: 85.71 - lr: 0.500000 2021-03-26 06:09:33,980 epoch 2 - iter 40/50 - loss 25.79278846 - samples/sec: 87.74 - lr: 0.500000 2021-03-26 06:09:36,027 epoch 2 - iter 45/50 - loss 25.15292172 - samples/sec: 78.25 - lr: 0.500000 2021-03-26 06:09:37,966 epoch 2 - iter 50/50 - loss 24.61890293 - samples/sec: 82.63 - lr: 0.500000 2021-03-26 06:09:37,966 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:09:37,966 EPOCH 2 done: loss 24.6189 - lr 0.5000000 2021-03-26 06:09:38,823 DEV : loss 18.001943588256836 - score 0.6871 2021-03-26 06:09:38,852 BAD EPOCHS (no improvement): 0 2021-03-26 06:09:48,858 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:09:50,846 epoch 3 - iter 5/50 - loss 20.17376251 - samples/sec: 80.59 - lr: 0.500000 2021-03-26 06:09:52,846 epoch 3 - iter 10/50 - loss 19.30381107 - samples/sec: 80.10 - lr: 0.500000 2021-03-26 06:09:54,888 epoch 3 - iter 15/50 - loss 18.62234275 - samples/sec: 78.44 - lr: 0.500000 2021-03-26 06:09:56,880 epoch 3 - iter 20/50 - loss 18.49893241 - samples/sec: 80.53 - lr: 0.500000 2021-03-26 06:09:58,848 epoch 3 - iter 25/50 - loss 18.25910244 - samples/sec: 81.38 - lr: 0.500000 2021-03-26 06:10:00,846 epoch 3 - iter 30/50 - loss 17.74629262 - samples/sec: 80.16 - lr: 0.500000 2021-03-26 06:10:02,721 epoch 3 - iter 35/50 - loss 17.59613062 - samples/sec: 85.41 - lr: 0.500000 2021-03-26 06:10:04,734 epoch 3 - iter 40/50 - loss 17.44145768 - samples/sec: 79.58 - lr: 0.500000 2021-03-26 06:10:06,725 epoch 3 - iter 45/50 - loss 17.46167198 - samples/sec: 80.44 - lr: 0.500000 2021-03-26 06:10:08,382 epoch 3 - iter 50/50 - loss 17.12887486 - samples/sec: 96.65 - lr: 0.500000 2021-03-26 06:10:08,383 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:10:08,383 EPOCH 3 done: loss 17.1289 - lr 0.5000000 2021-03-26 06:10:09,199 DEV : loss 15.706961631774902 - score 0.732 2021-03-26 06:10:09,225 BAD EPOCHS (no improvement): 0 2021-03-26 06:10:18,978 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:10:21,066 epoch 4 - iter 5/50 - loss 13.97613335 - samples/sec: 76.77 - lr: 0.500000 2021-03-26 06:10:23,081 epoch 4 - iter 10/50 - loss 14.95872822 - samples/sec: 79.48 - lr: 0.500000 2021-03-26 06:10:25,143 epoch 4 - iter 15/50 - loss 14.66935984 - samples/sec: 77.66 - lr: 0.500000 2021-03-26 06:10:27,077 epoch 4 - iter 20/50 - loss 14.39494796 - samples/sec: 82.77 - lr: 0.500000 2021-03-26 06:10:29,005 epoch 4 - iter 25/50 - loss 14.47061756 - samples/sec: 83.12 - lr: 0.500000 2021-03-26 06:10:30,820 epoch 4 - iter 30/50 - loss 14.52058875 - samples/sec: 88.27 - lr: 0.500000 2021-03-26 06:10:32,733 epoch 4 - iter 35/50 - loss 14.39601618 - samples/sec: 83.71 - lr: 0.500000 2021-03-26 06:10:34,640 epoch 4 - iter 40/50 - loss 13.98630881 - samples/sec: 83.98 - lr: 0.500000 2021-03-26 06:10:36,501 epoch 4 - iter 45/50 - loss 14.07221792 - samples/sec: 86.04 - lr: 0.500000 2021-03-26 06:10:38,324 epoch 4 - iter 50/50 - loss 14.00682487 - samples/sec: 87.87 - lr: 0.500000 2021-03-26 06:10:38,325 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:10:38,325 EPOCH 4 done: loss 14.0068 - lr 0.5000000 2021-03-26 06:10:39,130 DEV : loss 12.180521965026855 - score 0.7796 2021-03-26 06:10:39,155 BAD EPOCHS (no improvement): 0 2021-03-26 06:10:48,923 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:10:51,016 epoch 5 - iter 5/50 - loss 11.56535034 - samples/sec: 76.55 - lr: 0.500000 2021-03-26 06:10:53,638 epoch 5 - iter 10/50 - loss 11.53233051 - samples/sec: 61.08 - lr: 0.500000 2021-03-26 06:10:55,804 epoch 5 - iter 15/50 - loss 12.04551531 - samples/sec: 73.94 - lr: 0.500000 2021-03-26 06:10:57,689 epoch 5 - iter 20/50 - loss 12.20313921 - samples/sec: 84.96 - lr: 0.500000 2021-03-26 06:10:59,583 epoch 5 - iter 25/50 - loss 12.66698563 - samples/sec: 84.56 - lr: 0.500000 2021-03-26 06:11:01,684 epoch 5 - iter 30/50 - loss 12.56654412 - samples/sec: 76.27 - lr: 0.500000 2021-03-26 06:11:03,679 epoch 5 - iter 35/50 - loss 12.49077797 - samples/sec: 80.27 - lr: 0.500000 2021-03-26 06:11:05,908 epoch 5 - iter 40/50 - loss 12.50401361 - samples/sec: 71.87 - lr: 0.500000 2021-03-26 06:11:07,803 epoch 5 - iter 45/50 - loss 12.35786915 - samples/sec: 84.52 - lr: 0.500000 2021-03-26 06:11:09,566 epoch 5 - iter 50/50 - loss 12.31663839 - samples/sec: 90.85 - lr: 0.500000 2021-03-26 06:11:09,567 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:11:09,567 EPOCH 5 done: loss 12.3166 - lr 0.5000000 2021-03-26 06:11:10,380 DEV : loss 9.9617280960083 - score 0.8285 2021-03-26 06:11:10,406 BAD EPOCHS (no improvement): 0 2021-03-26 06:11:19,847 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:11:21,763 epoch 6 - iter 5/50 - loss 9.79197845 - samples/sec: 83.62 - lr: 0.500000 2021-03-26 06:11:23,801 epoch 6 - iter 10/50 - loss 9.94693480 - samples/sec: 78.55 - lr: 0.500000 2021-03-26 06:11:25,926 epoch 6 - iter 15/50 - loss 10.89867058 - samples/sec: 75.38 - lr: 0.500000 2021-03-26 06:11:27,780 epoch 6 - iter 20/50 - loss 11.04457297 - samples/sec: 86.41 - lr: 0.500000 2021-03-26 06:11:29,718 epoch 6 - iter 25/50 - loss 10.92623383 - samples/sec: 82.63 - lr: 0.500000 2021-03-26 06:11:31,983 epoch 6 - iter 30/50 - loss 11.11003348 - samples/sec: 70.68 - lr: 0.500000 2021-03-26 06:11:33,943 epoch 6 - iter 35/50 - loss 11.11525487 - samples/sec: 81.71 - lr: 0.500000 2021-03-26 06:11:35,953 epoch 6 - iter 40/50 - loss 11.06258175 - samples/sec: 79.69 - lr: 0.500000 2021-03-26 06:11:37,890 epoch 6 - iter 45/50 - loss 10.88173576 - samples/sec: 82.69 - lr: 0.500000 2021-03-26 06:11:39,859 epoch 6 - iter 50/50 - loss 10.87174243 - samples/sec: 81.33 - lr: 0.500000 2021-03-26 06:11:39,860 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:11:39,861 EPOCH 6 done: loss 10.8717 - lr 0.5000000 2021-03-26 06:11:40,690 DEV : loss 9.194941520690918 - score 0.8365 2021-03-26 06:11:40,718 BAD EPOCHS (no improvement): 0 2021-03-26 06:11:50,455 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:11:52,589 epoch 7 - iter 5/50 - loss 9.42767963 - samples/sec: 75.11 - lr: 0.500000 2021-03-26 06:11:54,539 epoch 7 - iter 10/50 - loss 10.54737329 - samples/sec: 82.16 - lr: 0.500000 2021-03-26 06:11:56,590 epoch 7 - iter 15/50 - loss 10.36283264 - samples/sec: 78.09 - lr: 0.500000 2021-03-26 06:11:58,549 epoch 7 - iter 20/50 - loss 9.77013283 - samples/sec: 81.77 - lr: 0.500000 2021-03-26 06:12:00,711 epoch 7 - iter 25/50 - loss 9.82616146 - samples/sec: 74.04 - lr: 0.500000 2021-03-26 06:12:02,534 epoch 7 - iter 30/50 - loss 9.88486067 - samples/sec: 87.88 - lr: 0.500000 2021-03-26 06:12:04,348 epoch 7 - iter 35/50 - loss 9.92839804 - samples/sec: 88.30 - lr: 0.500000 2021-03-26 06:12:06,383 epoch 7 - iter 40/50 - loss 9.83319703 - samples/sec: 78.71 - lr: 0.500000 2021-03-26 06:12:08,432 epoch 7 - iter 45/50 - loss 9.77937012 - samples/sec: 78.18 - lr: 0.500000 2021-03-26 06:12:10,308 epoch 7 - iter 50/50 - loss 9.99064981 - samples/sec: 85.37 - lr: 0.500000 2021-03-26 06:12:10,309 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:12:10,310 EPOCH 7 done: loss 9.9906 - lr 0.5000000 2021-03-26 06:12:11,157 DEV : loss 9.499085426330566 - score 0.8304 2021-03-26 06:12:11,175 BAD EPOCHS (no improvement): 1 2021-03-26 06:12:11,176 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:12:13,192 epoch 8 - iter 5/50 - loss 9.51839771 - samples/sec: 79.42 - lr: 0.500000 2021-03-26 06:12:15,302 epoch 8 - iter 10/50 - loss 9.78024168 - samples/sec: 75.92 - lr: 0.500000 2021-03-26 06:12:17,159 epoch 8 - iter 15/50 - loss 9.80983674 - samples/sec: 86.26 - lr: 0.500000 2021-03-26 06:12:19,321 epoch 8 - iter 20/50 - loss 9.67828443 - samples/sec: 74.08 - lr: 0.500000 2021-03-26 06:12:21,384 epoch 8 - iter 25/50 - loss 9.43407003 - samples/sec: 77.64 - lr: 0.500000 2021-03-26 06:12:23,454 epoch 8 - iter 30/50 - loss 9.60205196 - samples/sec: 77.35 - lr: 0.500000 2021-03-26 06:12:25,465 epoch 8 - iter 35/50 - loss 9.51624957 - samples/sec: 79.61 - lr: 0.500000 2021-03-26 06:12:27,402 epoch 8 - iter 40/50 - loss 9.43358691 - samples/sec: 82.70 - lr: 0.500000 2021-03-26 06:12:29,527 epoch 8 - iter 45/50 - loss 9.54377569 - samples/sec: 75.34 - lr: 0.500000 2021-03-26 06:12:31,460 epoch 8 - iter 50/50 - loss 9.43047031 - samples/sec: 82.89 - lr: 0.500000 2021-03-26 06:12:31,461 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:12:31,461 EPOCH 8 done: loss 9.4305 - lr 0.5000000 2021-03-26 06:12:32,288 DEV : loss 8.57904052734375 - score 0.8468 2021-03-26 06:12:32,315 BAD EPOCHS (no improvement): 0 2021-03-26 06:12:42,103 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:12:44,199 epoch 9 - iter 5/50 - loss 7.69350290 - samples/sec: 76.48 - lr: 0.500000 2021-03-26 06:12:46,324 epoch 9 - iter 10/50 - loss 7.68171325 - samples/sec: 75.38 - lr: 0.500000 2021-03-26 06:12:48,318 epoch 9 - iter 15/50 - loss 8.52010759 - samples/sec: 80.31 - lr: 0.500000 2021-03-26 06:12:50,344 epoch 9 - iter 20/50 - loss 8.30756495 - samples/sec: 79.07 - lr: 0.500000 2021-03-26 06:12:52,326 epoch 9 - iter 25/50 - loss 8.40493088 - samples/sec: 80.82 - lr: 0.500000 2021-03-26 06:12:54,388 epoch 9 - iter 30/50 - loss 8.46827960 - samples/sec: 77.68 - lr: 0.500000 2021-03-26 06:12:56,481 epoch 9 - iter 35/50 - loss 8.47599471 - samples/sec: 76.50 - lr: 0.500000 2021-03-26 06:12:58,432 epoch 9 - iter 40/50 - loss 8.47426549 - samples/sec: 82.10 - lr: 0.500000 2021-03-26 06:13:00,478 epoch 9 - iter 45/50 - loss 8.47557190 - samples/sec: 78.27 - lr: 0.500000 2021-03-26 06:13:02,428 epoch 9 - iter 50/50 - loss 8.44233681 - samples/sec: 82.13 - lr: 0.500000 2021-03-26 06:13:02,429 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:13:02,429 EPOCH 9 done: loss 8.4423 - lr 0.5000000 2021-03-26 06:13:03,234 DEV : loss 7.8837103843688965 - score 0.8662 2021-03-26 06:13:03,260 BAD EPOCHS (no improvement): 0 2021-03-26 06:13:12,899 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:13:15,004 epoch 10 - iter 5/50 - loss 7.26523943 - samples/sec: 76.13 - lr: 0.500000 2021-03-26 06:13:16,811 epoch 10 - iter 10/50 - loss 7.22758307 - samples/sec: 88.64 - lr: 0.500000 2021-03-26 06:13:18,871 epoch 10 - iter 15/50 - loss 7.90553220 - samples/sec: 77.74 - lr: 0.500000 2021-03-26 06:13:20,825 epoch 10 - iter 20/50 - loss 8.07978196 - samples/sec: 81.93 - lr: 0.500000 2021-03-26 06:13:22,788 epoch 10 - iter 25/50 - loss 8.07027122 - samples/sec: 81.62 - lr: 0.500000 2021-03-26 06:13:24,679 epoch 10 - iter 30/50 - loss 7.96194596 - samples/sec: 84.71 - lr: 0.500000 2021-03-26 06:13:26,850 epoch 10 - iter 35/50 - loss 8.07621027 - samples/sec: 73.80 - lr: 0.500000 2021-03-26 06:13:28,847 epoch 10 - iter 40/50 - loss 8.07766933 - samples/sec: 80.18 - lr: 0.500000 2021-03-26 06:13:30,754 epoch 10 - iter 45/50 - loss 8.08958091 - samples/sec: 83.95 - lr: 0.500000 2021-03-26 06:13:32,690 epoch 10 - iter 50/50 - loss 8.25567496 - samples/sec: 82.80 - lr: 0.500000 2021-03-26 06:13:32,690 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:13:32,691 EPOCH 10 done: loss 8.2557 - lr 0.5000000 2021-03-26 06:13:33,492 DEV : loss 7.9223737716674805 - score 0.8646 2021-03-26 06:13:33,517 BAD EPOCHS (no improvement): 1 2021-03-26 06:13:33,518 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:13:35,584 epoch 11 - iter 5/50 - loss 6.95426130 - samples/sec: 77.54 - lr: 0.500000 2021-03-26 06:13:37,640 epoch 11 - iter 10/50 - loss 7.45029273 - samples/sec: 77.89 - lr: 0.500000 2021-03-26 06:13:39,666 epoch 11 - iter 15/50 - loss 7.58958505 - samples/sec: 79.07 - lr: 0.500000 2021-03-26 06:13:41,612 epoch 11 - iter 20/50 - loss 7.46826644 - samples/sec: 82.29 - lr: 0.500000 2021-03-26 06:13:43,535 epoch 11 - iter 25/50 - loss 7.58064199 - samples/sec: 83.26 - lr: 0.500000 2021-03-26 06:13:45,470 epoch 11 - iter 30/50 - loss 7.82060990 - samples/sec: 82.79 - lr: 0.500000 2021-03-26 06:13:47,670 epoch 11 - iter 35/50 - loss 7.85852415 - samples/sec: 72.74 - lr: 0.500000 2021-03-26 06:13:49,684 epoch 11 - iter 40/50 - loss 7.85756209 - samples/sec: 79.57 - lr: 0.500000 2021-03-26 06:13:51,764 epoch 11 - iter 45/50 - loss 7.93629023 - samples/sec: 76.99 - lr: 0.500000 2021-03-26 06:13:53,692 epoch 11 - iter 50/50 - loss 7.82427173 - samples/sec: 83.10 - lr: 0.500000 2021-03-26 06:13:53,692 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:13:53,693 EPOCH 11 done: loss 7.8243 - lr 0.5000000 2021-03-26 06:13:54,500 DEV : loss 8.221084594726562 - score 0.8649 2021-03-26 06:13:54,526 BAD EPOCHS (no improvement): 2 2021-03-26 06:13:54,527 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:13:56,488 epoch 12 - iter 5/50 - loss 6.72069206 - samples/sec: 81.66 - lr: 0.500000 2021-03-26 06:13:58,466 epoch 12 - iter 10/50 - loss 6.91607099 - samples/sec: 81.02 - lr: 0.500000 2021-03-26 06:14:00,469 epoch 12 - iter 15/50 - loss 7.14914818 - samples/sec: 79.93 - lr: 0.500000 2021-03-26 06:14:02,477 epoch 12 - iter 20/50 - loss 7.04800429 - samples/sec: 79.77 - lr: 0.500000 2021-03-26 06:14:04,583 epoch 12 - iter 25/50 - loss 6.98862305 - samples/sec: 76.06 - lr: 0.500000 2021-03-26 06:14:06,535 epoch 12 - iter 30/50 - loss 7.23589487 - samples/sec: 82.02 - lr: 0.500000 2021-03-26 06:14:08,669 epoch 12 - iter 35/50 - loss 7.31580941 - samples/sec: 75.06 - lr: 0.500000 2021-03-26 06:14:10,699 epoch 12 - iter 40/50 - loss 7.40670675 - samples/sec: 78.87 - lr: 0.500000 2021-03-26 06:14:12,860 epoch 12 - iter 45/50 - loss 7.41732713 - samples/sec: 74.12 - lr: 0.500000 2021-03-26 06:14:14,847 epoch 12 - iter 50/50 - loss 7.43627445 - samples/sec: 80.58 - lr: 0.500000 2021-03-26 06:14:14,848 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:14:14,849 EPOCH 12 done: loss 7.4363 - lr 0.5000000 2021-03-26 06:14:15,689 DEV : loss 7.278804779052734 - score 0.8755 2021-03-26 06:14:15,719 BAD EPOCHS (no improvement): 0 2021-03-26 06:14:25,575 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:14:27,591 epoch 13 - iter 5/50 - loss 6.20711269 - samples/sec: 79.44 - lr: 0.500000 2021-03-26 06:14:29,702 epoch 13 - iter 10/50 - loss 6.75995908 - samples/sec: 75.89 - lr: 0.500000 2021-03-26 06:14:31,766 epoch 13 - iter 15/50 - loss 6.80636247 - samples/sec: 77.59 - lr: 0.500000 2021-03-26 06:14:33,680 epoch 13 - iter 20/50 - loss 6.94637887 - samples/sec: 83.64 - lr: 0.500000 2021-03-26 06:14:35,674 epoch 13 - iter 25/50 - loss 6.90052332 - samples/sec: 80.31 - lr: 0.500000 2021-03-26 06:14:37,568 epoch 13 - iter 30/50 - loss 6.98368498 - samples/sec: 84.58 - lr: 0.500000 2021-03-26 06:14:39,824 epoch 13 - iter 35/50 - loss 6.98485334 - samples/sec: 71.01 - lr: 0.500000 2021-03-26 06:14:41,879 epoch 13 - iter 40/50 - loss 7.06695175 - samples/sec: 77.93 - lr: 0.500000 2021-03-26 06:14:43,867 epoch 13 - iter 45/50 - loss 6.98606850 - samples/sec: 80.57 - lr: 0.500000 2021-03-26 06:14:45,701 epoch 13 - iter 50/50 - loss 7.08044469 - samples/sec: 87.32 - lr: 0.500000 2021-03-26 06:14:45,701 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:14:45,702 EPOCH 13 done: loss 7.0804 - lr 0.5000000 2021-03-26 06:14:46,504 DEV : loss 7.4897284507751465 - score 0.8719 2021-03-26 06:14:46,529 BAD EPOCHS (no improvement): 1 2021-03-26 06:14:46,530 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:14:48,405 epoch 14 - iter 5/50 - loss 7.57633114 - samples/sec: 85.46 - lr: 0.500000 2021-03-26 06:14:50,632 epoch 14 - iter 10/50 - loss 7.42076797 - samples/sec: 71.90 - lr: 0.500000 2021-03-26 06:14:52,687 epoch 14 - iter 15/50 - loss 7.21374400 - samples/sec: 77.95 - lr: 0.500000 2021-03-26 06:14:54,498 epoch 14 - iter 20/50 - loss 7.15166688 - samples/sec: 88.41 - lr: 0.500000 2021-03-26 06:14:56,346 epoch 14 - iter 25/50 - loss 6.90149778 - samples/sec: 86.67 - lr: 0.500000 2021-03-26 06:14:58,407 epoch 14 - iter 30/50 - loss 6.92999091 - samples/sec: 77.72 - lr: 0.500000 2021-03-26 06:15:00,390 epoch 14 - iter 35/50 - loss 6.82046195 - samples/sec: 80.78 - lr: 0.500000 2021-03-26 06:15:02,279 epoch 14 - iter 40/50 - loss 6.83094221 - samples/sec: 84.77 - lr: 0.500000 2021-03-26 06:15:04,448 epoch 14 - iter 45/50 - loss 6.72926092 - samples/sec: 73.84 - lr: 0.500000 2021-03-26 06:15:06,312 epoch 14 - iter 50/50 - loss 6.71817581 - samples/sec: 85.93 - lr: 0.500000 2021-03-26 06:15:06,312 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:15:06,313 EPOCH 14 done: loss 6.7182 - lr 0.5000000 2021-03-26 06:15:07,135 DEV : loss 7.169527053833008 - score 0.8864 2021-03-26 06:15:07,167 BAD EPOCHS (no improvement): 0 2021-03-26 06:15:16,787 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:15:18,732 epoch 15 - iter 5/50 - loss 5.80208635 - samples/sec: 82.38 - lr: 0.500000 2021-03-26 06:15:20,703 epoch 15 - iter 10/50 - loss 5.96044936 - samples/sec: 81.26 - lr: 0.500000 2021-03-26 06:15:22,749 epoch 15 - iter 15/50 - loss 6.08839035 - samples/sec: 78.27 - lr: 0.500000 2021-03-26 06:15:24,806 epoch 15 - iter 20/50 - loss 5.84570023 - samples/sec: 77.86 - lr: 0.500000 2021-03-26 06:15:26,917 epoch 15 - iter 25/50 - loss 5.87174901 - samples/sec: 75.89 - lr: 0.500000 2021-03-26 06:15:28,997 epoch 15 - iter 30/50 - loss 5.93718410 - samples/sec: 76.97 - lr: 0.500000 2021-03-26 06:15:30,898 epoch 15 - iter 35/50 - loss 5.89442012 - samples/sec: 84.27 - lr: 0.500000 2021-03-26 06:15:33,130 epoch 15 - iter 40/50 - loss 6.00830467 - samples/sec: 71.72 - lr: 0.500000 2021-03-26 06:15:35,087 epoch 15 - iter 45/50 - loss 6.02102583 - samples/sec: 81.85 - lr: 0.500000 2021-03-26 06:15:37,009 epoch 15 - iter 50/50 - loss 6.10250228 - samples/sec: 83.35 - lr: 0.500000 2021-03-26 06:15:37,010 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:15:37,010 EPOCH 15 done: loss 6.1025 - lr 0.5000000 2021-03-26 06:15:37,834 DEV : loss 7.358739852905273 - score 0.8742 2021-03-26 06:15:37,860 BAD EPOCHS (no improvement): 1 2021-03-26 06:15:37,861 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:15:39,834 epoch 16 - iter 5/50 - loss 6.96504974 - samples/sec: 81.15 - lr: 0.500000 2021-03-26 06:15:41,884 epoch 16 - iter 10/50 - loss 6.91026793 - samples/sec: 78.16 - lr: 0.500000 2021-03-26 06:15:44,050 epoch 16 - iter 15/50 - loss 6.68998731 - samples/sec: 73.95 - lr: 0.500000 2021-03-26 06:15:45,978 epoch 16 - iter 20/50 - loss 6.60481675 - samples/sec: 83.05 - lr: 0.500000 2021-03-26 06:15:47,988 epoch 16 - iter 25/50 - loss 6.48665789 - samples/sec: 79.68 - lr: 0.500000 2021-03-26 06:15:50,019 epoch 16 - iter 30/50 - loss 6.29562639 - samples/sec: 78.88 - lr: 0.500000 2021-03-26 06:15:51,866 epoch 16 - iter 35/50 - loss 6.23206079 - samples/sec: 86.71 - lr: 0.500000 2021-03-26 06:15:53,828 epoch 16 - iter 40/50 - loss 6.17947595 - samples/sec: 81.64 - lr: 0.500000 2021-03-26 06:15:55,798 epoch 16 - iter 45/50 - loss 6.15443833 - samples/sec: 81.29 - lr: 0.500000 2021-03-26 06:15:57,465 epoch 16 - iter 50/50 - loss 6.11608263 - samples/sec: 96.17 - lr: 0.500000 2021-03-26 06:15:57,465 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:15:57,466 EPOCH 16 done: loss 6.1161 - lr 0.5000000 2021-03-26 06:15:58,289 DEV : loss 7.640191555023193 - score 0.875 2021-03-26 06:15:58,318 BAD EPOCHS (no improvement): 2 2021-03-26 06:15:58,319 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:16:00,374 epoch 17 - iter 5/50 - loss 5.76472216 - samples/sec: 77.91 - lr: 0.500000 2021-03-26 06:16:02,165 epoch 17 - iter 10/50 - loss 5.60648484 - samples/sec: 89.45 - lr: 0.500000 2021-03-26 06:16:04,243 epoch 17 - iter 15/50 - loss 5.77117564 - samples/sec: 77.05 - lr: 0.500000 2021-03-26 06:16:06,119 epoch 17 - iter 20/50 - loss 5.80412705 - samples/sec: 85.35 - lr: 0.500000 2021-03-26 06:16:08,090 epoch 17 - iter 25/50 - loss 5.77925716 - samples/sec: 81.25 - lr: 0.500000 2021-03-26 06:16:10,089 epoch 17 - iter 30/50 - loss 5.70927278 - samples/sec: 80.13 - lr: 0.500000 2021-03-26 06:16:12,107 epoch 17 - iter 35/50 - loss 5.70613817 - samples/sec: 79.39 - lr: 0.500000 2021-03-26 06:16:14,074 epoch 17 - iter 40/50 - loss 5.77921520 - samples/sec: 81.41 - lr: 0.500000 2021-03-26 06:16:16,031 epoch 17 - iter 45/50 - loss 5.70467767 - samples/sec: 81.84 - lr: 0.500000 2021-03-26 06:16:18,138 epoch 17 - iter 50/50 - loss 5.87843167 - samples/sec: 76.00 - lr: 0.500000 2021-03-26 06:16:18,139 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:16:18,140 EPOCH 17 done: loss 5.8784 - lr 0.5000000 2021-03-26 06:16:19,033 DEV : loss 7.120694637298584 - score 0.8817 2021-03-26 06:16:19,061 BAD EPOCHS (no improvement): 3 2021-03-26 06:16:19,062 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:16:21,362 epoch 18 - iter 5/50 - loss 5.80058298 - samples/sec: 69.63 - lr: 0.500000 2021-03-26 06:16:23,438 epoch 18 - iter 10/50 - loss 6.00460644 - samples/sec: 77.15 - lr: 0.500000 2021-03-26 06:16:25,358 epoch 18 - iter 15/50 - loss 5.90615212 - samples/sec: 83.43 - lr: 0.500000 2021-03-26 06:16:27,357 epoch 18 - iter 20/50 - loss 5.72014546 - samples/sec: 80.12 - lr: 0.500000 2021-03-26 06:16:29,346 epoch 18 - iter 25/50 - loss 5.80794409 - samples/sec: 80.52 - lr: 0.500000 2021-03-26 06:16:31,315 epoch 18 - iter 30/50 - loss 5.80667328 - samples/sec: 81.37 - lr: 0.500000 2021-03-26 06:16:33,298 epoch 18 - iter 35/50 - loss 5.76956917 - samples/sec: 80.78 - lr: 0.500000 2021-03-26 06:16:35,522 epoch 18 - iter 40/50 - loss 5.68624325 - samples/sec: 72.06 - lr: 0.500000 2021-03-26 06:16:37,455 epoch 18 - iter 45/50 - loss 5.70470218 - samples/sec: 82.87 - lr: 0.500000 2021-03-26 06:16:39,290 epoch 18 - iter 50/50 - loss 5.82466750 - samples/sec: 87.27 - lr: 0.500000 2021-03-26 06:16:39,292 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:16:39,292 EPOCH 18 done: loss 5.8247 - lr 0.5000000 2021-03-26 06:16:40,114 DEV : loss 7.067171096801758 - score 0.8886 2021-03-26 06:16:40,141 BAD EPOCHS (no improvement): 0 2021-03-26 06:16:49,832 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:16:51,943 epoch 19 - iter 5/50 - loss 5.32156200 - samples/sec: 75.90 - lr: 0.500000 2021-03-26 06:16:53,868 epoch 19 - iter 10/50 - loss 4.92078037 - samples/sec: 83.24 - lr: 0.500000 2021-03-26 06:16:55,873 epoch 19 - iter 15/50 - loss 5.07983332 - samples/sec: 79.84 - lr: 0.500000 2021-03-26 06:16:57,998 epoch 19 - iter 20/50 - loss 5.24572079 - samples/sec: 75.36 - lr: 0.500000 2021-03-26 06:17:00,367 epoch 19 - iter 25/50 - loss 5.34888216 - samples/sec: 67.61 - lr: 0.500000 2021-03-26 06:17:02,223 epoch 19 - iter 30/50 - loss 5.33492244 - samples/sec: 86.28 - lr: 0.500000 2021-03-26 06:17:04,089 epoch 19 - iter 35/50 - loss 5.38144553 - samples/sec: 85.85 - lr: 0.500000 2021-03-26 06:17:06,062 epoch 19 - iter 40/50 - loss 5.36061793 - samples/sec: 81.15 - lr: 0.500000 2021-03-26 06:17:08,112 epoch 19 - iter 45/50 - loss 5.48110933 - samples/sec: 78.14 - lr: 0.500000 2021-03-26 06:17:09,926 epoch 19 - iter 50/50 - loss 5.53508369 - samples/sec: 88.30 - lr: 0.500000 2021-03-26 06:17:09,927 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:17:09,927 EPOCH 19 done: loss 5.5351 - lr 0.5000000 2021-03-26 06:17:10,741 DEV : loss 7.041009902954102 - score 0.8852 2021-03-26 06:17:10,766 BAD EPOCHS (no improvement): 1 2021-03-26 06:17:10,767 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:17:12,781 epoch 20 - iter 5/50 - loss 5.35584059 - samples/sec: 79.52 - lr: 0.500000 2021-03-26 06:17:14,751 epoch 20 - iter 10/50 - loss 4.90886288 - samples/sec: 81.29 - lr: 0.500000 2021-03-26 06:17:16,668 epoch 20 - iter 15/50 - loss 4.89809780 - samples/sec: 83.53 - lr: 0.500000 2021-03-26 06:17:18,533 epoch 20 - iter 20/50 - loss 5.13916326 - samples/sec: 85.85 - lr: 0.500000 2021-03-26 06:17:20,464 epoch 20 - iter 25/50 - loss 5.10323879 - samples/sec: 82.96 - lr: 0.500000 2021-03-26 06:17:22,463 epoch 20 - iter 30/50 - loss 5.10014350 - samples/sec: 80.09 - lr: 0.500000 2021-03-26 06:17:24,533 epoch 20 - iter 35/50 - loss 5.20402196 - samples/sec: 77.37 - lr: 0.500000 2021-03-26 06:17:26,636 epoch 20 - iter 40/50 - loss 5.37771060 - samples/sec: 76.18 - lr: 0.500000 2021-03-26 06:17:28,694 epoch 20 - iter 45/50 - loss 5.33990641 - samples/sec: 77.84 - lr: 0.500000 2021-03-26 06:17:30,776 epoch 20 - iter 50/50 - loss 5.52597651 - samples/sec: 76.91 - lr: 0.500000 2021-03-26 06:17:30,777 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:17:30,777 EPOCH 20 done: loss 5.5260 - lr 0.5000000 2021-03-26 06:17:31,645 DEV : loss 7.155633926391602 - score 0.8897 2021-03-26 06:17:31,672 BAD EPOCHS (no improvement): 0 2021-03-26 06:17:41,442 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:17:43,438 epoch 21 - iter 5/50 - loss 4.66815495 - samples/sec: 80.24 - lr: 0.500000 2021-03-26 06:17:45,394 epoch 21 - iter 10/50 - loss 5.02459919 - samples/sec: 81.88 - lr: 0.500000 2021-03-26 06:17:47,435 epoch 21 - iter 15/50 - loss 5.26974597 - samples/sec: 78.50 - lr: 0.500000 2021-03-26 06:17:49,541 epoch 21 - iter 20/50 - loss 5.35654365 - samples/sec: 76.02 - lr: 0.500000 2021-03-26 06:17:51,710 epoch 21 - iter 25/50 - loss 5.32445769 - samples/sec: 73.84 - lr: 0.500000 2021-03-26 06:17:53,601 epoch 21 - iter 30/50 - loss 5.29939301 - samples/sec: 84.68 - lr: 0.500000 2021-03-26 06:17:55,610 epoch 21 - iter 35/50 - loss 5.35172689 - samples/sec: 79.74 - lr: 0.500000 2021-03-26 06:17:57,641 epoch 21 - iter 40/50 - loss 5.39133795 - samples/sec: 78.84 - lr: 0.500000 2021-03-26 06:17:59,650 epoch 21 - iter 45/50 - loss 5.44440748 - samples/sec: 79.74 - lr: 0.500000 2021-03-26 06:18:01,498 epoch 21 - iter 50/50 - loss 5.35541382 - samples/sec: 86.73 - lr: 0.500000 2021-03-26 06:18:01,499 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:18:01,499 EPOCH 21 done: loss 5.3554 - lr 0.5000000 2021-03-26 06:18:02,315 DEV : loss 7.129164695739746 - score 0.8845 2021-03-26 06:18:02,341 BAD EPOCHS (no improvement): 1 2021-03-26 06:18:02,342 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:18:04,458 epoch 22 - iter 5/50 - loss 4.22467690 - samples/sec: 75.67 - lr: 0.500000 2021-03-26 06:18:06,354 epoch 22 - iter 10/50 - loss 4.66893404 - samples/sec: 84.46 - lr: 0.500000 2021-03-26 06:18:08,340 epoch 22 - iter 15/50 - loss 4.67403258 - samples/sec: 80.68 - lr: 0.500000 2021-03-26 06:18:10,377 epoch 22 - iter 20/50 - loss 4.74228729 - samples/sec: 78.59 - lr: 0.500000 2021-03-26 06:18:12,309 epoch 22 - iter 25/50 - loss 4.84663138 - samples/sec: 82.90 - lr: 0.500000 2021-03-26 06:18:14,283 epoch 22 - iter 30/50 - loss 4.83845309 - samples/sec: 81.16 - lr: 0.500000 2021-03-26 06:18:16,352 epoch 22 - iter 35/50 - loss 4.90364776 - samples/sec: 77.40 - lr: 0.500000 2021-03-26 06:18:18,535 epoch 22 - iter 40/50 - loss 4.99079722 - samples/sec: 73.39 - lr: 0.500000 2021-03-26 06:18:20,558 epoch 22 - iter 45/50 - loss 5.00144645 - samples/sec: 79.18 - lr: 0.500000 2021-03-26 06:18:22,538 epoch 22 - iter 50/50 - loss 5.00026049 - samples/sec: 80.89 - lr: 0.500000 2021-03-26 06:18:22,539 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:18:22,539 EPOCH 22 done: loss 5.0003 - lr 0.5000000 2021-03-26 06:18:23,358 DEV : loss 7.209229946136475 - score 0.892 2021-03-26 06:18:23,376 BAD EPOCHS (no improvement): 0 2021-03-26 06:18:33,094 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:18:35,157 epoch 23 - iter 5/50 - loss 4.42421436 - samples/sec: 77.67 - lr: 0.500000 2021-03-26 06:18:37,257 epoch 23 - iter 10/50 - loss 4.58858795 - samples/sec: 76.24 - lr: 0.500000 2021-03-26 06:18:39,192 epoch 23 - iter 15/50 - loss 4.68085804 - samples/sec: 82.79 - lr: 0.500000 2021-03-26 06:18:41,191 epoch 23 - iter 20/50 - loss 4.48246378 - samples/sec: 80.15 - lr: 0.500000 2021-03-26 06:18:43,197 epoch 23 - iter 25/50 - loss 4.60831195 - samples/sec: 79.84 - lr: 0.500000 2021-03-26 06:18:45,338 epoch 23 - iter 30/50 - loss 4.53582498 - samples/sec: 74.82 - lr: 0.500000 2021-03-26 06:18:47,158 epoch 23 - iter 35/50 - loss 4.60754177 - samples/sec: 87.95 - lr: 0.500000 2021-03-26 06:18:49,133 epoch 23 - iter 40/50 - loss 4.75192796 - samples/sec: 81.14 - lr: 0.500000 2021-03-26 06:18:51,214 epoch 23 - iter 45/50 - loss 4.73687989 - samples/sec: 76.95 - lr: 0.500000 2021-03-26 06:18:53,076 epoch 23 - iter 50/50 - loss 4.76302664 - samples/sec: 85.98 - lr: 0.500000 2021-03-26 06:18:53,077 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:18:53,077 EPOCH 23 done: loss 4.7630 - lr 0.5000000 2021-03-26 06:18:53,928 DEV : loss 7.368309020996094 - score 0.8877 2021-03-26 06:18:53,958 BAD EPOCHS (no improvement): 1 2021-03-26 06:18:53,959 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:18:55,819 epoch 24 - iter 5/50 - loss 4.53367014 - samples/sec: 86.10 - lr: 0.500000 2021-03-26 06:18:57,857 epoch 24 - iter 10/50 - loss 4.77556491 - samples/sec: 78.59 - lr: 0.500000 2021-03-26 06:18:59,726 epoch 24 - iter 15/50 - loss 4.68768574 - samples/sec: 85.68 - lr: 0.500000 2021-03-26 06:19:01,703 epoch 24 - iter 20/50 - loss 4.69659667 - samples/sec: 81.03 - lr: 0.500000 2021-03-26 06:19:03,714 epoch 24 - iter 25/50 - loss 4.78619684 - samples/sec: 79.62 - lr: 0.500000 2021-03-26 06:19:05,530 epoch 24 - iter 30/50 - loss 4.71550113 - samples/sec: 88.19 - lr: 0.500000 2021-03-26 06:19:07,538 epoch 24 - iter 35/50 - loss 4.63759924 - samples/sec: 79.79 - lr: 0.500000 2021-03-26 06:19:09,731 epoch 24 - iter 40/50 - loss 4.67768221 - samples/sec: 73.00 - lr: 0.500000 2021-03-26 06:19:11,687 epoch 24 - iter 45/50 - loss 4.77335002 - samples/sec: 81.89 - lr: 0.500000 2021-03-26 06:19:13,645 epoch 24 - iter 50/50 - loss 4.70509425 - samples/sec: 81.77 - lr: 0.500000 2021-03-26 06:19:13,646 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:19:13,646 EPOCH 24 done: loss 4.7051 - lr 0.5000000 2021-03-26 06:19:14,450 DEV : loss 7.073756217956543 - score 0.8827 2021-03-26 06:19:14,471 BAD EPOCHS (no improvement): 2 2021-03-26 06:19:14,471 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:19:16,465 epoch 25 - iter 5/50 - loss 5.02869673 - samples/sec: 80.30 - lr: 0.500000 2021-03-26 06:19:18,430 epoch 25 - iter 10/50 - loss 4.77017746 - samples/sec: 81.52 - lr: 0.500000 2021-03-26 06:19:20,331 epoch 25 - iter 15/50 - loss 4.87391117 - samples/sec: 84.29 - lr: 0.500000 2021-03-26 06:19:22,391 epoch 25 - iter 20/50 - loss 4.88199726 - samples/sec: 77.76 - lr: 0.500000 2021-03-26 06:19:24,358 epoch 25 - iter 25/50 - loss 4.92657302 - samples/sec: 81.38 - lr: 0.500000 2021-03-26 06:19:26,455 epoch 25 - iter 30/50 - loss 4.89457783 - samples/sec: 76.42 - lr: 0.500000 2021-03-26 06:19:28,517 epoch 25 - iter 35/50 - loss 4.78275283 - samples/sec: 77.64 - lr: 0.500000 2021-03-26 06:19:30,658 epoch 25 - iter 40/50 - loss 4.82162649 - samples/sec: 74.81 - lr: 0.500000 2021-03-26 06:19:32,627 epoch 25 - iter 45/50 - loss 4.90526964 - samples/sec: 81.31 - lr: 0.500000 2021-03-26 06:19:34,235 epoch 25 - iter 50/50 - loss 4.78762905 - samples/sec: 99.59 - lr: 0.500000 2021-03-26 06:19:34,236 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:19:34,236 EPOCH 25 done: loss 4.7876 - lr 0.5000000 2021-03-26 06:19:35,034 DEV : loss 7.848437786102295 - score 0.8751 2021-03-26 06:19:35,059 BAD EPOCHS (no improvement): 3 2021-03-26 06:19:35,060 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:19:37,013 epoch 26 - iter 5/50 - loss 4.39827414 - samples/sec: 82.02 - lr: 0.500000 2021-03-26 06:19:38,978 epoch 26 - iter 10/50 - loss 4.59327972 - samples/sec: 81.55 - lr: 0.500000 2021-03-26 06:19:40,939 epoch 26 - iter 15/50 - loss 4.79339280 - samples/sec: 81.68 - lr: 0.500000 2021-03-26 06:19:42,973 epoch 26 - iter 20/50 - loss 4.70258976 - samples/sec: 78.72 - lr: 0.500000 2021-03-26 06:19:45,190 epoch 26 - iter 25/50 - loss 4.63147681 - samples/sec: 72.22 - lr: 0.500000 2021-03-26 06:19:47,322 epoch 26 - iter 30/50 - loss 4.65599515 - samples/sec: 75.09 - lr: 0.500000 2021-03-26 06:19:49,139 epoch 26 - iter 35/50 - loss 4.64802615 - samples/sec: 88.17 - lr: 0.500000 2021-03-26 06:19:50,954 epoch 26 - iter 40/50 - loss 4.52358636 - samples/sec: 88.23 - lr: 0.500000 2021-03-26 06:19:53,038 epoch 26 - iter 45/50 - loss 4.53593655 - samples/sec: 76.85 - lr: 0.500000 2021-03-26 06:19:54,966 epoch 26 - iter 50/50 - loss 4.52930590 - samples/sec: 83.07 - lr: 0.500000 2021-03-26 06:19:54,966 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:19:54,966 EPOCH 26 done: loss 4.5293 - lr 0.5000000 2021-03-26 06:19:55,779 DEV : loss 7.53148078918457 - score 0.8884 2021-03-26 06:19:55,804 BAD EPOCHS (no improvement): 4 2021-03-26 06:19:55,805 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:19:57,707 epoch 27 - iter 5/50 - loss 3.82686830 - samples/sec: 84.19 - lr: 0.250000 2021-03-26 06:19:59,679 epoch 27 - iter 10/50 - loss 3.90787556 - samples/sec: 81.26 - lr: 0.250000 2021-03-26 06:20:01,702 epoch 27 - iter 15/50 - loss 3.89724426 - samples/sec: 79.15 - lr: 0.250000 2021-03-26 06:20:03,651 epoch 27 - iter 20/50 - loss 4.00114959 - samples/sec: 82.24 - lr: 0.250000 2021-03-26 06:20:05,654 epoch 27 - iter 25/50 - loss 3.96433598 - samples/sec: 79.97 - lr: 0.250000 2021-03-26 06:20:07,642 epoch 27 - iter 30/50 - loss 4.10539275 - samples/sec: 80.54 - lr: 0.250000 2021-03-26 06:20:09,525 epoch 27 - iter 35/50 - loss 4.07188780 - samples/sec: 85.05 - lr: 0.250000 2021-03-26 06:20:11,589 epoch 27 - iter 40/50 - loss 4.01781631 - samples/sec: 77.60 - lr: 0.250000 2021-03-26 06:20:13,728 epoch 27 - iter 45/50 - loss 3.93384314 - samples/sec: 74.89 - lr: 0.250000 2021-03-26 06:20:15,676 epoch 27 - iter 50/50 - loss 3.94948688 - samples/sec: 82.18 - lr: 0.250000 2021-03-26 06:20:15,677 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:20:15,677 EPOCH 27 done: loss 3.9495 - lr 0.2500000 2021-03-26 06:20:16,497 DEV : loss 6.849296569824219 - score 0.8994 2021-03-26 06:20:16,524 BAD EPOCHS (no improvement): 0 2021-03-26 06:20:26,002 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:20:29,219 epoch 28 - iter 5/50 - loss 3.81995850 - samples/sec: 49.78 - lr: 0.250000 2021-03-26 06:20:31,150 epoch 28 - iter 10/50 - loss 4.01414056 - samples/sec: 82.96 - lr: 0.250000 2021-03-26 06:20:33,235 epoch 28 - iter 15/50 - loss 4.01229688 - samples/sec: 76.80 - lr: 0.250000 2021-03-26 06:20:35,220 epoch 28 - iter 20/50 - loss 3.84462682 - samples/sec: 80.66 - lr: 0.250000 2021-03-26 06:20:37,200 epoch 28 - iter 25/50 - loss 3.89083829 - samples/sec: 80.96 - lr: 0.250000 2021-03-26 06:20:39,270 epoch 28 - iter 30/50 - loss 3.85976992 - samples/sec: 77.36 - lr: 0.250000 2021-03-26 06:20:41,560 epoch 28 - iter 35/50 - loss 3.85642435 - samples/sec: 69.94 - lr: 0.250000 2021-03-26 06:20:43,739 epoch 28 - iter 40/50 - loss 3.84121451 - samples/sec: 73.48 - lr: 0.250000 2021-03-26 06:20:45,542 epoch 28 - iter 45/50 - loss 3.83978069 - samples/sec: 88.87 - lr: 0.250000 2021-03-26 06:20:47,391 epoch 28 - iter 50/50 - loss 3.77387645 - samples/sec: 86.65 - lr: 0.250000 2021-03-26 06:20:47,391 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:20:47,392 EPOCH 28 done: loss 3.7739 - lr 0.2500000 2021-03-26 06:20:48,184 DEV : loss 6.964501857757568 - score 0.9041 2021-03-26 06:20:48,209 BAD EPOCHS (no improvement): 0 2021-03-26 06:20:57,673 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:21:00,127 epoch 29 - iter 5/50 - loss 4.25758047 - samples/sec: 65.28 - lr: 0.250000 2021-03-26 06:21:01,940 epoch 29 - iter 10/50 - loss 4.10903742 - samples/sec: 88.34 - lr: 0.250000 2021-03-26 06:21:03,888 epoch 29 - iter 15/50 - loss 4.05873850 - samples/sec: 82.23 - lr: 0.250000 2021-03-26 06:21:05,746 epoch 29 - iter 20/50 - loss 4.01524862 - samples/sec: 86.22 - lr: 0.250000 2021-03-26 06:21:07,887 epoch 29 - iter 25/50 - loss 3.92797098 - samples/sec: 74.81 - lr: 0.250000 2021-03-26 06:21:09,833 epoch 29 - iter 30/50 - loss 3.90643357 - samples/sec: 82.32 - lr: 0.250000 2021-03-26 06:21:11,857 epoch 29 - iter 35/50 - loss 3.89231799 - samples/sec: 79.14 - lr: 0.250000 2021-03-26 06:21:13,967 epoch 29 - iter 40/50 - loss 3.86935532 - samples/sec: 75.93 - lr: 0.250000 2021-03-26 06:21:15,892 epoch 29 - iter 45/50 - loss 3.83151262 - samples/sec: 83.17 - lr: 0.250000 2021-03-26 06:21:17,638 epoch 29 - iter 50/50 - loss 3.83023209 - samples/sec: 91.72 - lr: 0.250000 2021-03-26 06:21:17,639 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:21:17,640 EPOCH 29 done: loss 3.8302 - lr 0.2500000 2021-03-26 06:21:18,435 DEV : loss 6.683016777038574 - score 0.9014 2021-03-26 06:21:18,460 BAD EPOCHS (no improvement): 1 2021-03-26 06:21:18,461 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:21:20,384 epoch 30 - iter 5/50 - loss 3.60974512 - samples/sec: 83.31 - lr: 0.250000 2021-03-26 06:21:22,259 epoch 30 - iter 10/50 - loss 3.61541648 - samples/sec: 85.42 - lr: 0.250000 2021-03-26 06:21:24,185 epoch 30 - iter 15/50 - loss 3.51423572 - samples/sec: 83.19 - lr: 0.250000 2021-03-26 06:21:26,347 epoch 30 - iter 20/50 - loss 3.53319275 - samples/sec: 74.10 - lr: 0.250000 2021-03-26 06:21:28,304 epoch 30 - iter 25/50 - loss 3.55055931 - samples/sec: 81.87 - lr: 0.250000 2021-03-26 06:21:30,267 epoch 30 - iter 30/50 - loss 3.54994337 - samples/sec: 81.60 - lr: 0.250000 2021-03-26 06:21:32,073 epoch 30 - iter 35/50 - loss 3.45288790 - samples/sec: 88.66 - lr: 0.250000 2021-03-26 06:21:33,966 epoch 30 - iter 40/50 - loss 3.45501449 - samples/sec: 84.60 - lr: 0.250000 2021-03-26 06:21:36,166 epoch 30 - iter 45/50 - loss 3.48634175 - samples/sec: 72.78 - lr: 0.250000 2021-03-26 06:21:38,099 epoch 30 - iter 50/50 - loss 3.59221848 - samples/sec: 82.87 - lr: 0.250000 2021-03-26 06:21:38,100 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:21:38,100 EPOCH 30 done: loss 3.5922 - lr 0.2500000 2021-03-26 06:21:38,892 DEV : loss 6.731375694274902 - score 0.9015 2021-03-26 06:21:38,917 BAD EPOCHS (no improvement): 2 2021-03-26 06:21:38,918 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:21:40,960 epoch 31 - iter 5/50 - loss 4.21938186 - samples/sec: 78.45 - lr: 0.250000 2021-03-26 06:21:42,822 epoch 31 - iter 10/50 - loss 3.82266951 - samples/sec: 86.03 - lr: 0.250000 2021-03-26 06:21:44,824 epoch 31 - iter 15/50 - loss 3.58426552 - samples/sec: 80.01 - lr: 0.250000 2021-03-26 06:21:46,855 epoch 31 - iter 20/50 - loss 3.58201323 - samples/sec: 78.85 - lr: 0.250000 2021-03-26 06:21:48,698 epoch 31 - iter 25/50 - loss 3.57560792 - samples/sec: 86.89 - lr: 0.250000 2021-03-26 06:21:50,670 epoch 31 - iter 30/50 - loss 3.42569039 - samples/sec: 81.23 - lr: 0.250000 2021-03-26 06:21:52,689 epoch 31 - iter 35/50 - loss 3.49048048 - samples/sec: 79.30 - lr: 0.250000 2021-03-26 06:21:54,700 epoch 31 - iter 40/50 - loss 3.50268674 - samples/sec: 79.63 - lr: 0.250000 2021-03-26 06:21:56,692 epoch 31 - iter 45/50 - loss 3.46837054 - samples/sec: 80.38 - lr: 0.250000 2021-03-26 06:21:58,607 epoch 31 - iter 50/50 - loss 3.57005912 - samples/sec: 83.62 - lr: 0.250000 2021-03-26 06:21:58,608 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:21:58,608 EPOCH 31 done: loss 3.5701 - lr 0.2500000 2021-03-26 06:21:59,398 DEV : loss 6.836461067199707 - score 0.9019 2021-03-26 06:21:59,424 BAD EPOCHS (no improvement): 3 2021-03-26 06:21:59,425 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:22:01,484 epoch 32 - iter 5/50 - loss 2.85547342 - samples/sec: 77.75 - lr: 0.250000 2021-03-26 06:22:03,524 epoch 32 - iter 10/50 - loss 3.03122098 - samples/sec: 78.51 - lr: 0.250000 2021-03-26 06:22:05,581 epoch 32 - iter 15/50 - loss 3.22923800 - samples/sec: 77.86 - lr: 0.250000 2021-03-26 06:22:07,481 epoch 32 - iter 20/50 - loss 3.31757702 - samples/sec: 84.29 - lr: 0.250000 2021-03-26 06:22:09,504 epoch 32 - iter 25/50 - loss 3.27683636 - samples/sec: 79.16 - lr: 0.250000 2021-03-26 06:22:11,483 epoch 32 - iter 30/50 - loss 3.24276355 - samples/sec: 80.96 - lr: 0.250000 2021-03-26 06:22:13,444 epoch 32 - iter 35/50 - loss 3.23634225 - samples/sec: 81.68 - lr: 0.250000 2021-03-26 06:22:15,387 epoch 32 - iter 40/50 - loss 3.20656458 - samples/sec: 82.46 - lr: 0.250000 2021-03-26 06:22:17,206 epoch 32 - iter 45/50 - loss 3.18769871 - samples/sec: 88.05 - lr: 0.250000 2021-03-26 06:22:19,018 epoch 32 - iter 50/50 - loss 3.25717090 - samples/sec: 88.34 - lr: 0.250000 2021-03-26 06:22:19,019 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:22:19,020 EPOCH 32 done: loss 3.2572 - lr 0.2500000 2021-03-26 06:22:19,845 DEV : loss 7.020652770996094 - score 0.8992 2021-03-26 06:22:19,869 BAD EPOCHS (no improvement): 4 2021-03-26 06:22:19,869 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:22:21,939 epoch 33 - iter 5/50 - loss 3.13701382 - samples/sec: 77.39 - lr: 0.125000 2021-03-26 06:22:23,942 epoch 33 - iter 10/50 - loss 2.86191452 - samples/sec: 79.96 - lr: 0.125000 2021-03-26 06:22:25,922 epoch 33 - iter 15/50 - loss 3.00424566 - samples/sec: 80.87 - lr: 0.125000 2021-03-26 06:22:27,923 epoch 33 - iter 20/50 - loss 3.03444778 - samples/sec: 80.06 - lr: 0.125000 2021-03-26 06:22:29,911 epoch 33 - iter 25/50 - loss 3.12197756 - samples/sec: 80.55 - lr: 0.125000 2021-03-26 06:22:31,898 epoch 33 - iter 30/50 - loss 3.03927702 - samples/sec: 80.61 - lr: 0.125000 2021-03-26 06:22:33,980 epoch 33 - iter 35/50 - loss 3.08446097 - samples/sec: 76.89 - lr: 0.125000 2021-03-26 06:22:36,147 epoch 33 - iter 40/50 - loss 3.05413010 - samples/sec: 73.91 - lr: 0.125000 2021-03-26 06:22:38,285 epoch 33 - iter 45/50 - loss 3.09317961 - samples/sec: 74.93 - lr: 0.125000 2021-03-26 06:22:40,047 epoch 33 - iter 50/50 - loss 3.13532100 - samples/sec: 90.97 - lr: 0.125000 2021-03-26 06:22:40,048 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:22:40,048 EPOCH 33 done: loss 3.1353 - lr 0.1250000 2021-03-26 06:22:40,857 DEV : loss 6.7104997634887695 - score 0.9059 2021-03-26 06:22:40,875 BAD EPOCHS (no improvement): 0 2021-03-26 06:22:50,423 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:22:52,500 epoch 34 - iter 5/50 - loss 2.97224755 - samples/sec: 77.16 - lr: 0.125000 2021-03-26 06:22:54,279 epoch 34 - iter 10/50 - loss 2.79403913 - samples/sec: 90.02 - lr: 0.125000 2021-03-26 06:22:56,702 epoch 34 - iter 15/50 - loss 2.96103163 - samples/sec: 66.08 - lr: 0.125000 2021-03-26 06:22:58,558 epoch 34 - iter 20/50 - loss 3.00038437 - samples/sec: 86.35 - lr: 0.125000 2021-03-26 06:23:00,439 epoch 34 - iter 25/50 - loss 3.00155699 - samples/sec: 85.17 - lr: 0.125000 2021-03-26 06:23:02,343 epoch 34 - iter 30/50 - loss 3.01357543 - samples/sec: 84.10 - lr: 0.125000 2021-03-26 06:23:04,427 epoch 34 - iter 35/50 - loss 3.05171795 - samples/sec: 76.86 - lr: 0.125000 2021-03-26 06:23:06,363 epoch 34 - iter 40/50 - loss 3.04145568 - samples/sec: 82.73 - lr: 0.125000 2021-03-26 06:23:08,553 epoch 34 - iter 45/50 - loss 3.06644186 - samples/sec: 73.15 - lr: 0.125000 2021-03-26 06:23:10,411 epoch 34 - iter 50/50 - loss 3.18904578 - samples/sec: 86.19 - lr: 0.125000 2021-03-26 06:23:10,412 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:23:10,412 EPOCH 34 done: loss 3.1890 - lr 0.1250000 2021-03-26 06:23:11,228 DEV : loss 6.743964195251465 - score 0.9039 2021-03-26 06:23:11,254 BAD EPOCHS (no improvement): 1 2021-03-26 06:23:11,255 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:23:13,323 epoch 35 - iter 5/50 - loss 3.39564099 - samples/sec: 77.45 - lr: 0.125000 2021-03-26 06:23:15,123 epoch 35 - iter 10/50 - loss 3.06941562 - samples/sec: 89.01 - lr: 0.125000 2021-03-26 06:23:17,156 epoch 35 - iter 15/50 - loss 3.02297851 - samples/sec: 78.77 - lr: 0.125000 2021-03-26 06:23:19,140 epoch 35 - iter 20/50 - loss 3.13186671 - samples/sec: 80.74 - lr: 0.125000 2021-03-26 06:23:21,073 epoch 35 - iter 25/50 - loss 3.11243931 - samples/sec: 82.83 - lr: 0.125000 2021-03-26 06:23:23,244 epoch 35 - iter 30/50 - loss 3.08870709 - samples/sec: 73.78 - lr: 0.125000 2021-03-26 06:23:25,385 epoch 35 - iter 35/50 - loss 3.09268135 - samples/sec: 74.78 - lr: 0.125000 2021-03-26 06:23:27,392 epoch 35 - iter 40/50 - loss 3.08425107 - samples/sec: 79.80 - lr: 0.125000 2021-03-26 06:23:29,379 epoch 35 - iter 45/50 - loss 3.03528121 - samples/sec: 80.58 - lr: 0.125000 2021-03-26 06:23:31,335 epoch 35 - iter 50/50 - loss 3.02395340 - samples/sec: 81.91 - lr: 0.125000 2021-03-26 06:23:31,336 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:23:31,336 EPOCH 35 done: loss 3.0240 - lr 0.1250000 2021-03-26 06:23:32,181 DEV : loss 6.659381866455078 - score 0.9053 2021-03-26 06:23:32,209 BAD EPOCHS (no improvement): 2 2021-03-26 06:23:32,210 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:23:34,186 epoch 36 - iter 5/50 - loss 2.70919876 - samples/sec: 81.04 - lr: 0.125000 2021-03-26 06:23:36,188 epoch 36 - iter 10/50 - loss 2.72161946 - samples/sec: 80.02 - lr: 0.125000 2021-03-26 06:23:38,140 epoch 36 - iter 15/50 - loss 2.75374413 - samples/sec: 82.04 - lr: 0.125000 2021-03-26 06:23:40,098 epoch 36 - iter 20/50 - loss 2.82946196 - samples/sec: 81.84 - lr: 0.125000 2021-03-26 06:23:42,028 epoch 36 - iter 25/50 - loss 2.80491947 - samples/sec: 83.00 - lr: 0.125000 2021-03-26 06:23:43,997 epoch 36 - iter 30/50 - loss 2.75406922 - samples/sec: 81.31 - lr: 0.125000 2021-03-26 06:23:46,008 epoch 36 - iter 35/50 - loss 2.81473930 - samples/sec: 79.66 - lr: 0.125000 2021-03-26 06:23:47,999 epoch 36 - iter 40/50 - loss 2.88393461 - samples/sec: 80.43 - lr: 0.125000 2021-03-26 06:23:50,307 epoch 36 - iter 45/50 - loss 2.86291863 - samples/sec: 69.38 - lr: 0.125000 2021-03-26 06:23:52,128 epoch 36 - iter 50/50 - loss 2.95692794 - samples/sec: 87.99 - lr: 0.125000 2021-03-26 06:23:52,129 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:23:52,129 EPOCH 36 done: loss 2.9569 - lr 0.1250000 2021-03-26 06:23:52,924 DEV : loss 6.722310543060303 - score 0.9073 2021-03-26 06:23:52,950 BAD EPOCHS (no improvement): 0 2021-03-26 06:24:02,551 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:24:04,482 epoch 37 - iter 5/50 - loss 2.98329856 - samples/sec: 82.97 - lr: 0.125000 2021-03-26 06:24:06,335 epoch 37 - iter 10/50 - loss 2.84934751 - samples/sec: 86.45 - lr: 0.125000 2021-03-26 06:24:08,383 epoch 37 - iter 15/50 - loss 2.94346907 - samples/sec: 78.21 - lr: 0.125000 2021-03-26 06:24:10,496 epoch 37 - iter 20/50 - loss 2.94718229 - samples/sec: 75.78 - lr: 0.125000 2021-03-26 06:24:12,662 epoch 37 - iter 25/50 - loss 2.97769873 - samples/sec: 73.99 - lr: 0.125000 2021-03-26 06:24:14,789 epoch 37 - iter 30/50 - loss 3.06307853 - samples/sec: 75.31 - lr: 0.125000 2021-03-26 06:24:16,705 epoch 37 - iter 35/50 - loss 3.06100252 - samples/sec: 83.55 - lr: 0.125000 2021-03-26 06:24:18,806 epoch 37 - iter 40/50 - loss 3.07404596 - samples/sec: 76.26 - lr: 0.125000 2021-03-26 06:24:20,702 epoch 37 - iter 45/50 - loss 3.04176707 - samples/sec: 84.47 - lr: 0.125000 2021-03-26 06:24:22,559 epoch 37 - iter 50/50 - loss 3.05250724 - samples/sec: 86.26 - lr: 0.125000 2021-03-26 06:24:22,560 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:24:22,560 EPOCH 37 done: loss 3.0525 - lr 0.1250000 2021-03-26 06:24:23,430 DEV : loss 6.977239608764648 - score 0.903 2021-03-26 06:24:23,456 BAD EPOCHS (no improvement): 1 2021-03-26 06:24:23,457 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:24:25,331 epoch 38 - iter 5/50 - loss 2.45115697 - samples/sec: 85.48 - lr: 0.125000 2021-03-26 06:24:27,448 epoch 38 - iter 10/50 - loss 2.73839563 - samples/sec: 75.63 - lr: 0.125000 2021-03-26 06:24:29,559 epoch 38 - iter 15/50 - loss 2.77460554 - samples/sec: 75.86 - lr: 0.125000 2021-03-26 06:24:31,510 epoch 38 - iter 20/50 - loss 2.69243065 - samples/sec: 82.08 - lr: 0.125000 2021-03-26 06:24:33,407 epoch 38 - iter 25/50 - loss 2.70140969 - samples/sec: 84.43 - lr: 0.125000 2021-03-26 06:24:35,323 epoch 38 - iter 30/50 - loss 2.66803439 - samples/sec: 83.58 - lr: 0.125000 2021-03-26 06:24:37,261 epoch 38 - iter 35/50 - loss 2.63252071 - samples/sec: 82.66 - lr: 0.125000 2021-03-26 06:24:39,443 epoch 38 - iter 40/50 - loss 2.69146514 - samples/sec: 73.44 - lr: 0.125000 2021-03-26 06:24:41,734 epoch 38 - iter 45/50 - loss 2.65392441 - samples/sec: 69.89 - lr: 0.125000 2021-03-26 06:24:43,774 epoch 38 - iter 50/50 - loss 2.71148792 - samples/sec: 78.54 - lr: 0.125000 2021-03-26 06:24:43,774 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:24:43,774 EPOCH 38 done: loss 2.7115 - lr 0.1250000 2021-03-26 06:24:44,591 DEV : loss 6.88974666595459 - score 0.9069 2021-03-26 06:24:44,620 BAD EPOCHS (no improvement): 2 2021-03-26 06:24:44,620 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:24:46,729 epoch 39 - iter 5/50 - loss 2.89031749 - samples/sec: 75.95 - lr: 0.125000 2021-03-26 06:24:48,776 epoch 39 - iter 10/50 - loss 2.72867922 - samples/sec: 78.26 - lr: 0.125000 2021-03-26 06:24:50,761 epoch 39 - iter 15/50 - loss 2.58544503 - samples/sec: 80.68 - lr: 0.125000 2021-03-26 06:24:52,832 epoch 39 - iter 20/50 - loss 2.64498028 - samples/sec: 77.36 - lr: 0.125000 2021-03-26 06:24:54,870 epoch 39 - iter 25/50 - loss 2.71735142 - samples/sec: 78.61 - lr: 0.125000 2021-03-26 06:24:56,906 epoch 39 - iter 30/50 - loss 2.69654935 - samples/sec: 78.66 - lr: 0.125000 2021-03-26 06:24:58,840 epoch 39 - iter 35/50 - loss 2.77980161 - samples/sec: 82.82 - lr: 0.125000 2021-03-26 06:25:00,885 epoch 39 - iter 40/50 - loss 2.74732492 - samples/sec: 78.30 - lr: 0.125000 2021-03-26 06:25:02,936 epoch 39 - iter 45/50 - loss 2.75970213 - samples/sec: 78.09 - lr: 0.125000 2021-03-26 06:25:04,820 epoch 39 - iter 50/50 - loss 2.73851163 - samples/sec: 85.00 - lr: 0.125000 2021-03-26 06:25:04,821 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:25:04,821 EPOCH 39 done: loss 2.7385 - lr 0.1250000 2021-03-26 06:25:05,634 DEV : loss 6.86600399017334 - score 0.9061 2021-03-26 06:25:05,661 BAD EPOCHS (no improvement): 3 2021-03-26 06:25:05,661 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:25:07,492 epoch 40 - iter 5/50 - loss 2.93947897 - samples/sec: 87.51 - lr: 0.125000 2021-03-26 06:25:09,591 epoch 40 - iter 10/50 - loss 2.59734519 - samples/sec: 76.30 - lr: 0.125000 2021-03-26 06:25:11,603 epoch 40 - iter 15/50 - loss 2.59909554 - samples/sec: 79.60 - lr: 0.125000 2021-03-26 06:25:13,531 epoch 40 - iter 20/50 - loss 2.56581937 - samples/sec: 83.06 - lr: 0.125000 2021-03-26 06:25:15,644 epoch 40 - iter 25/50 - loss 2.63403885 - samples/sec: 75.80 - lr: 0.125000 2021-03-26 06:25:17,734 epoch 40 - iter 30/50 - loss 2.66231444 - samples/sec: 76.59 - lr: 0.125000 2021-03-26 06:25:19,742 epoch 40 - iter 35/50 - loss 2.60746949 - samples/sec: 79.77 - lr: 0.125000 2021-03-26 06:25:21,846 epoch 40 - iter 40/50 - loss 2.65097475 - samples/sec: 76.10 - lr: 0.125000 2021-03-26 06:25:23,899 epoch 40 - iter 45/50 - loss 2.70838504 - samples/sec: 78.06 - lr: 0.125000 2021-03-26 06:25:26,172 epoch 40 - iter 50/50 - loss 2.72248378 - samples/sec: 70.43 - lr: 0.125000 2021-03-26 06:25:26,173 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:25:26,173 EPOCH 40 done: loss 2.7225 - lr 0.1250000 2021-03-26 06:25:26,974 DEV : loss 7.042795181274414 - score 0.9061 2021-03-26 06:25:26,998 BAD EPOCHS (no improvement): 4 2021-03-26 06:25:26,999 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:25:28,891 epoch 41 - iter 5/50 - loss 3.03020132 - samples/sec: 84.66 - lr: 0.062500 2021-03-26 06:25:30,996 epoch 41 - iter 10/50 - loss 2.92019740 - samples/sec: 76.07 - lr: 0.062500 2021-03-26 06:25:33,009 epoch 41 - iter 15/50 - loss 2.80438588 - samples/sec: 79.56 - lr: 0.062500 2021-03-26 06:25:34,912 epoch 41 - iter 20/50 - loss 2.79609125 - samples/sec: 84.20 - lr: 0.062500 2021-03-26 06:25:36,831 epoch 41 - iter 25/50 - loss 2.66950620 - samples/sec: 83.48 - lr: 0.062500 2021-03-26 06:25:38,766 epoch 41 - iter 30/50 - loss 2.64369078 - samples/sec: 82.75 - lr: 0.062500 2021-03-26 06:25:40,669 epoch 41 - iter 35/50 - loss 2.71000017 - samples/sec: 84.16 - lr: 0.062500 2021-03-26 06:25:42,605 epoch 41 - iter 40/50 - loss 2.67410662 - samples/sec: 82.70 - lr: 0.062500 2021-03-26 06:25:44,568 epoch 41 - iter 45/50 - loss 2.72351690 - samples/sec: 81.61 - lr: 0.062500 2021-03-26 06:25:46,331 epoch 41 - iter 50/50 - loss 2.68049579 - samples/sec: 90.80 - lr: 0.062500 2021-03-26 06:25:46,332 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:25:46,333 EPOCH 41 done: loss 2.6805 - lr 0.0625000 2021-03-26 06:25:47,172 DEV : loss 6.836457252502441 - score 0.9069 2021-03-26 06:25:47,195 BAD EPOCHS (no improvement): 1 2021-03-26 06:25:47,195 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:25:49,081 epoch 42 - iter 5/50 - loss 2.83079109 - samples/sec: 84.93 - lr: 0.062500 2021-03-26 06:25:51,170 epoch 42 - iter 10/50 - loss 2.60190370 - samples/sec: 76.68 - lr: 0.062500 2021-03-26 06:25:53,425 epoch 42 - iter 15/50 - loss 2.63134141 - samples/sec: 70.98 - lr: 0.062500 2021-03-26 06:25:55,509 epoch 42 - iter 20/50 - loss 2.64654483 - samples/sec: 76.84 - lr: 0.062500 2021-03-26 06:25:57,489 epoch 42 - iter 25/50 - loss 2.63535637 - samples/sec: 80.90 - lr: 0.062500 2021-03-26 06:25:59,512 epoch 42 - iter 30/50 - loss 2.61309492 - samples/sec: 79.13 - lr: 0.062500 2021-03-26 06:26:01,568 epoch 42 - iter 35/50 - loss 2.62725921 - samples/sec: 77.92 - lr: 0.062500 2021-03-26 06:26:03,567 epoch 42 - iter 40/50 - loss 2.62116627 - samples/sec: 80.07 - lr: 0.062500 2021-03-26 06:26:05,470 epoch 42 - iter 45/50 - loss 2.61511114 - samples/sec: 84.20 - lr: 0.062500 2021-03-26 06:26:07,470 epoch 42 - iter 50/50 - loss 2.65905147 - samples/sec: 80.07 - lr: 0.062500 2021-03-26 06:26:07,471 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:26:07,471 EPOCH 42 done: loss 2.6591 - lr 0.0625000 2021-03-26 06:26:08,289 DEV : loss 6.842776298522949 - score 0.9073 2021-03-26 06:26:08,315 BAD EPOCHS (no improvement): 2 2021-03-26 06:26:08,316 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:26:10,383 epoch 43 - iter 5/50 - loss 2.99454298 - samples/sec: 77.48 - lr: 0.062500 2021-03-26 06:26:12,295 epoch 43 - iter 10/50 - loss 2.78095438 - samples/sec: 83.76 - lr: 0.062500 2021-03-26 06:26:14,251 epoch 43 - iter 15/50 - loss 2.72735570 - samples/sec: 81.87 - lr: 0.062500 2021-03-26 06:26:16,157 epoch 43 - iter 20/50 - loss 2.76881919 - samples/sec: 84.02 - lr: 0.062500 2021-03-26 06:26:18,023 epoch 43 - iter 25/50 - loss 2.70888021 - samples/sec: 85.83 - lr: 0.062500 2021-03-26 06:26:20,126 epoch 43 - iter 30/50 - loss 2.77247720 - samples/sec: 76.15 - lr: 0.062500 2021-03-26 06:26:21,995 epoch 43 - iter 35/50 - loss 2.79106678 - samples/sec: 85.74 - lr: 0.062500 2021-03-26 06:26:23,894 epoch 43 - iter 40/50 - loss 2.72925793 - samples/sec: 84.33 - lr: 0.062500 2021-03-26 06:26:25,728 epoch 43 - iter 45/50 - loss 2.62710329 - samples/sec: 87.37 - lr: 0.062500 2021-03-26 06:26:27,771 epoch 43 - iter 50/50 - loss 2.69412341 - samples/sec: 78.35 - lr: 0.062500 2021-03-26 06:26:27,772 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:26:27,772 EPOCH 43 done: loss 2.6941 - lr 0.0625000 2021-03-26 06:26:28,606 DEV : loss 6.853123664855957 - score 0.9077 2021-03-26 06:26:28,632 BAD EPOCHS (no improvement): 0 2021-03-26 06:26:38,399 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:26:40,417 epoch 44 - iter 5/50 - loss 2.92129889 - samples/sec: 79.39 - lr: 0.062500 2021-03-26 06:26:42,392 epoch 44 - iter 10/50 - loss 2.71936698 - samples/sec: 81.11 - lr: 0.062500 2021-03-26 06:26:44,365 epoch 44 - iter 15/50 - loss 2.64585795 - samples/sec: 81.18 - lr: 0.062500 2021-03-26 06:26:46,328 epoch 44 - iter 20/50 - loss 2.64387424 - samples/sec: 81.57 - lr: 0.062500 2021-03-26 06:26:48,311 epoch 44 - iter 25/50 - loss 2.71903181 - samples/sec: 80.74 - lr: 0.062500 2021-03-26 06:26:50,314 epoch 44 - iter 30/50 - loss 2.72144562 - samples/sec: 80.00 - lr: 0.062500 2021-03-26 06:26:52,203 epoch 44 - iter 35/50 - loss 2.68308628 - samples/sec: 84.77 - lr: 0.062500 2021-03-26 06:26:53,970 epoch 44 - iter 40/50 - loss 2.65461990 - samples/sec: 90.67 - lr: 0.062500 2021-03-26 06:26:55,801 epoch 44 - iter 45/50 - loss 2.61197187 - samples/sec: 87.43 - lr: 0.062500 2021-03-26 06:26:57,800 epoch 44 - iter 50/50 - loss 2.62100311 - samples/sec: 80.13 - lr: 0.062500 2021-03-26 06:26:57,801 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:26:57,801 EPOCH 44 done: loss 2.6210 - lr 0.0625000 2021-03-26 06:26:58,632 DEV : loss 6.854997158050537 - score 0.9049 2021-03-26 06:26:58,658 BAD EPOCHS (no improvement): 1 2021-03-26 06:26:58,659 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:27:00,764 epoch 45 - iter 5/50 - loss 3.28177967 - samples/sec: 76.10 - lr: 0.062500 2021-03-26 06:27:02,821 epoch 45 - iter 10/50 - loss 3.06228626 - samples/sec: 77.88 - lr: 0.062500 2021-03-26 06:27:04,672 epoch 45 - iter 15/50 - loss 2.91840841 - samples/sec: 86.60 - lr: 0.062500 2021-03-26 06:27:06,564 epoch 45 - iter 20/50 - loss 2.84936029 - samples/sec: 84.61 - lr: 0.062500 2021-03-26 06:27:08,635 epoch 45 - iter 25/50 - loss 2.74072369 - samples/sec: 77.34 - lr: 0.062500 2021-03-26 06:27:10,756 epoch 45 - iter 30/50 - loss 2.75789352 - samples/sec: 75.48 - lr: 0.062500 2021-03-26 06:27:12,729 epoch 45 - iter 35/50 - loss 2.69706963 - samples/sec: 81.19 - lr: 0.062500 2021-03-26 06:27:14,716 epoch 45 - iter 40/50 - loss 2.65504144 - samples/sec: 80.57 - lr: 0.062500 2021-03-26 06:27:16,848 epoch 45 - iter 45/50 - loss 2.69478651 - samples/sec: 75.14 - lr: 0.062500 2021-03-26 06:27:18,660 epoch 45 - iter 50/50 - loss 2.65418336 - samples/sec: 88.40 - lr: 0.062500 2021-03-26 06:27:18,661 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:27:18,661 EPOCH 45 done: loss 2.6542 - lr 0.0625000 2021-03-26 06:27:19,458 DEV : loss 6.984309196472168 - score 0.9061 2021-03-26 06:27:19,477 BAD EPOCHS (no improvement): 2 2021-03-26 06:27:19,478 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:27:21,492 epoch 46 - iter 5/50 - loss 2.30996280 - samples/sec: 79.50 - lr: 0.062500 2021-03-26 06:27:23,499 epoch 46 - iter 10/50 - loss 2.30306711 - samples/sec: 79.82 - lr: 0.062500 2021-03-26 06:27:25,486 epoch 46 - iter 15/50 - loss 2.26478287 - samples/sec: 80.63 - lr: 0.062500 2021-03-26 06:27:27,582 epoch 46 - iter 20/50 - loss 2.30942514 - samples/sec: 76.41 - lr: 0.062500 2021-03-26 06:27:29,516 epoch 46 - iter 25/50 - loss 2.29877130 - samples/sec: 82.80 - lr: 0.062500 2021-03-26 06:27:31,389 epoch 46 - iter 30/50 - loss 2.34397881 - samples/sec: 85.53 - lr: 0.062500 2021-03-26 06:27:33,547 epoch 46 - iter 35/50 - loss 2.41658501 - samples/sec: 74.21 - lr: 0.062500 2021-03-26 06:27:35,528 epoch 46 - iter 40/50 - loss 2.38164246 - samples/sec: 80.82 - lr: 0.062500 2021-03-26 06:27:37,533 epoch 46 - iter 45/50 - loss 2.40390159 - samples/sec: 79.93 - lr: 0.062500 2021-03-26 06:27:39,300 epoch 46 - iter 50/50 - loss 2.46796207 - samples/sec: 90.61 - lr: 0.062500 2021-03-26 06:27:39,301 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:27:39,301 EPOCH 46 done: loss 2.4680 - lr 0.0625000 2021-03-26 06:27:40,109 DEV : loss 6.913209438323975 - score 0.9063 2021-03-26 06:27:40,135 BAD EPOCHS (no improvement): 3 2021-03-26 06:27:40,136 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:27:42,137 epoch 47 - iter 5/50 - loss 2.18956802 - samples/sec: 80.03 - lr: 0.062500 2021-03-26 06:27:44,077 epoch 47 - iter 10/50 - loss 2.32679708 - samples/sec: 82.55 - lr: 0.062500 2021-03-26 06:27:46,002 epoch 47 - iter 15/50 - loss 2.30111314 - samples/sec: 83.20 - lr: 0.062500 2021-03-26 06:27:47,814 epoch 47 - iter 20/50 - loss 2.36776996 - samples/sec: 88.42 - lr: 0.062500 2021-03-26 06:27:49,736 epoch 47 - iter 25/50 - loss 2.41560647 - samples/sec: 83.32 - lr: 0.062500 2021-03-26 06:27:51,867 epoch 47 - iter 30/50 - loss 2.51549866 - samples/sec: 75.16 - lr: 0.062500 2021-03-26 06:27:53,813 epoch 47 - iter 35/50 - loss 2.52135927 - samples/sec: 82.32 - lr: 0.062500 2021-03-26 06:27:56,368 epoch 47 - iter 40/50 - loss 2.52845867 - samples/sec: 62.66 - lr: 0.062500 2021-03-26 06:27:58,419 epoch 47 - iter 45/50 - loss 2.54538176 - samples/sec: 78.09 - lr: 0.062500 2021-03-26 06:28:00,367 epoch 47 - iter 50/50 - loss 2.54789333 - samples/sec: 82.20 - lr: 0.062500 2021-03-26 06:28:00,368 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:28:00,368 EPOCH 47 done: loss 2.5479 - lr 0.0625000 2021-03-26 06:28:01,161 DEV : loss 6.987135887145996 - score 0.9045 2021-03-26 06:28:01,180 BAD EPOCHS (no improvement): 4 2021-03-26 06:28:01,180 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:28:03,192 epoch 48 - iter 5/50 - loss 2.81061130 - samples/sec: 79.65 - lr: 0.031250 2021-03-26 06:28:05,368 epoch 48 - iter 10/50 - loss 2.62682199 - samples/sec: 73.58 - lr: 0.031250 2021-03-26 06:28:07,466 epoch 48 - iter 15/50 - loss 2.61593165 - samples/sec: 76.32 - lr: 0.031250 2021-03-26 06:28:09,554 epoch 48 - iter 20/50 - loss 2.65244003 - samples/sec: 76.75 - lr: 0.031250 2021-03-26 06:28:11,657 epoch 48 - iter 25/50 - loss 2.63186843 - samples/sec: 76.12 - lr: 0.031250 2021-03-26 06:28:13,596 epoch 48 - iter 30/50 - loss 2.64713775 - samples/sec: 82.63 - lr: 0.031250 2021-03-26 06:28:15,574 epoch 48 - iter 35/50 - loss 2.62877602 - samples/sec: 80.94 - lr: 0.031250 2021-03-26 06:28:17,448 epoch 48 - iter 40/50 - loss 2.60681930 - samples/sec: 85.49 - lr: 0.031250 2021-03-26 06:28:19,385 epoch 48 - iter 45/50 - loss 2.60738970 - samples/sec: 82.65 - lr: 0.031250 2021-03-26 06:28:21,336 epoch 48 - iter 50/50 - loss 2.59641813 - samples/sec: 82.08 - lr: 0.031250 2021-03-26 06:28:21,337 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:28:21,337 EPOCH 48 done: loss 2.5964 - lr 0.0312500 2021-03-26 06:28:22,159 DEV : loss 6.932497978210449 - score 0.9067 2021-03-26 06:28:22,178 BAD EPOCHS (no improvement): 1 2021-03-26 06:28:22,178 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:28:24,056 epoch 49 - iter 5/50 - loss 1.88422883 - samples/sec: 85.30 - lr: 0.031250 2021-03-26 06:28:26,127 epoch 49 - iter 10/50 - loss 2.10428255 - samples/sec: 77.33 - lr: 0.031250 2021-03-26 06:28:28,343 epoch 49 - iter 15/50 - loss 2.11673807 - samples/sec: 72.28 - lr: 0.031250 2021-03-26 06:28:30,435 epoch 49 - iter 20/50 - loss 2.26653606 - samples/sec: 76.52 - lr: 0.031250 2021-03-26 06:28:32,392 epoch 49 - iter 25/50 - loss 2.33341152 - samples/sec: 81.86 - lr: 0.031250 2021-03-26 06:28:34,420 epoch 49 - iter 30/50 - loss 2.31826424 - samples/sec: 78.94 - lr: 0.031250 2021-03-26 06:28:36,367 epoch 49 - iter 35/50 - loss 2.28970445 - samples/sec: 82.25 - lr: 0.031250 2021-03-26 06:28:38,483 epoch 49 - iter 40/50 - loss 2.29986542 - samples/sec: 75.66 - lr: 0.031250 2021-03-26 06:28:40,619 epoch 49 - iter 45/50 - loss 2.30139024 - samples/sec: 74.98 - lr: 0.031250 2021-03-26 06:28:42,612 epoch 49 - iter 50/50 - loss 2.28942521 - samples/sec: 80.38 - lr: 0.031250 2021-03-26 06:28:42,613 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:28:42,613 EPOCH 49 done: loss 2.2894 - lr 0.0312500 2021-03-26 06:28:43,404 DEV : loss 6.93155574798584 - score 0.9073 2021-03-26 06:28:43,429 BAD EPOCHS (no improvement): 2 2021-03-26 06:28:43,430 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:28:45,259 epoch 50 - iter 5/50 - loss 2.34129095 - samples/sec: 87.59 - lr: 0.031250 2021-03-26 06:28:47,334 epoch 50 - iter 10/50 - loss 2.65424306 - samples/sec: 77.19 - lr: 0.031250 2021-03-26 06:28:49,442 epoch 50 - iter 15/50 - loss 2.70122841 - samples/sec: 75.98 - lr: 0.031250 2021-03-26 06:28:51,798 epoch 50 - iter 20/50 - loss 2.61473920 - samples/sec: 67.98 - lr: 0.031250 2021-03-26 06:28:53,857 epoch 50 - iter 25/50 - loss 2.58245116 - samples/sec: 77.82 - lr: 0.031250 2021-03-26 06:28:55,886 epoch 50 - iter 30/50 - loss 2.49980253 - samples/sec: 78.94 - lr: 0.031250 2021-03-26 06:28:57,844 epoch 50 - iter 35/50 - loss 2.42350192 - samples/sec: 81.98 - lr: 0.031250 2021-03-26 06:28:59,900 epoch 50 - iter 40/50 - loss 2.43286690 - samples/sec: 77.87 - lr: 0.031250 2021-03-26 06:29:01,807 epoch 50 - iter 45/50 - loss 2.45776600 - samples/sec: 83.97 - lr: 0.031250 2021-03-26 06:29:03,680 epoch 50 - iter 50/50 - loss 2.47108955 - samples/sec: 85.54 - lr: 0.031250 2021-03-26 06:29:03,680 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:29:03,681 EPOCH 50 done: loss 2.4711 - lr 0.0312500 2021-03-26 06:29:04,483 DEV : loss 6.923022270202637 - score 0.9079 2021-03-26 06:29:04,513 BAD EPOCHS (no improvement): 0 2021-03-26 06:29:14,331 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:29:16,637 epoch 51 - iter 5/50 - loss 2.19648669 - samples/sec: 69.48 - lr: 0.031250 2021-03-26 06:29:18,647 epoch 51 - iter 10/50 - loss 2.40439309 - samples/sec: 79.64 - lr: 0.031250 2021-03-26 06:29:20,792 epoch 51 - iter 15/50 - loss 2.41015805 - samples/sec: 74.67 - lr: 0.031250 2021-03-26 06:29:22,920 epoch 51 - iter 20/50 - loss 2.38679830 - samples/sec: 75.24 - lr: 0.031250 2021-03-26 06:29:24,740 epoch 51 - iter 25/50 - loss 2.36478817 - samples/sec: 88.00 - lr: 0.031250 2021-03-26 06:29:26,747 epoch 51 - iter 30/50 - loss 2.38509798 - samples/sec: 79.80 - lr: 0.031250 2021-03-26 06:29:28,748 epoch 51 - iter 35/50 - loss 2.38499307 - samples/sec: 80.05 - lr: 0.031250 2021-03-26 06:29:30,781 epoch 51 - iter 40/50 - loss 2.38311165 - samples/sec: 78.76 - lr: 0.031250 2021-03-26 06:29:32,956 epoch 51 - iter 45/50 - loss 2.38785090 - samples/sec: 73.65 - lr: 0.031250 2021-03-26 06:29:34,543 epoch 51 - iter 50/50 - loss 2.33555826 - samples/sec: 100.92 - lr: 0.031250 2021-03-26 06:29:34,544 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:29:34,544 EPOCH 51 done: loss 2.3356 - lr 0.0312500 2021-03-26 06:29:35,465 DEV : loss 6.930795192718506 - score 0.9085 2021-03-26 06:29:35,489 BAD EPOCHS (no improvement): 0 2021-03-26 06:29:45,316 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:29:47,331 epoch 52 - iter 5/50 - loss 2.31803503 - samples/sec: 79.53 - lr: 0.031250 2021-03-26 06:29:49,380 epoch 52 - iter 10/50 - loss 2.48303349 - samples/sec: 78.17 - lr: 0.031250 2021-03-26 06:29:51,368 epoch 52 - iter 15/50 - loss 2.52508163 - samples/sec: 80.55 - lr: 0.031250 2021-03-26 06:29:53,344 epoch 52 - iter 20/50 - loss 2.48334989 - samples/sec: 81.04 - lr: 0.031250 2021-03-26 06:29:55,432 epoch 52 - iter 25/50 - loss 2.55548737 - samples/sec: 76.71 - lr: 0.031250 2021-03-26 06:29:57,291 epoch 52 - iter 30/50 - loss 2.52494575 - samples/sec: 86.15 - lr: 0.031250 2021-03-26 06:29:59,287 epoch 52 - iter 35/50 - loss 2.56608360 - samples/sec: 80.23 - lr: 0.031250 2021-03-26 06:30:01,387 epoch 52 - iter 40/50 - loss 2.55950036 - samples/sec: 76.25 - lr: 0.031250 2021-03-26 06:30:03,315 epoch 52 - iter 45/50 - loss 2.50635450 - samples/sec: 83.06 - lr: 0.031250 2021-03-26 06:30:05,174 epoch 52 - iter 50/50 - loss 2.49100937 - samples/sec: 86.16 - lr: 0.031250 2021-03-26 06:30:05,174 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:30:05,174 EPOCH 52 done: loss 2.4910 - lr 0.0312500 2021-03-26 06:30:06,005 DEV : loss 6.951305389404297 - score 0.9065 2021-03-26 06:30:06,030 BAD EPOCHS (no improvement): 1 2021-03-26 06:30:06,031 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:30:07,924 epoch 53 - iter 5/50 - loss 2.52924616 - samples/sec: 84.61 - lr: 0.031250 2021-03-26 06:30:10,031 epoch 53 - iter 10/50 - loss 2.44083222 - samples/sec: 76.18 - lr: 0.031250 2021-03-26 06:30:12,320 epoch 53 - iter 15/50 - loss 2.48448149 - samples/sec: 69.97 - lr: 0.031250 2021-03-26 06:30:14,239 epoch 53 - iter 20/50 - loss 2.39575719 - samples/sec: 83.42 - lr: 0.031250 2021-03-26 06:30:16,356 epoch 53 - iter 25/50 - loss 2.44080576 - samples/sec: 75.66 - lr: 0.031250 2021-03-26 06:30:18,319 epoch 53 - iter 30/50 - loss 2.45035690 - samples/sec: 81.60 - lr: 0.031250 2021-03-26 06:30:20,224 epoch 53 - iter 35/50 - loss 2.47454193 - samples/sec: 84.05 - lr: 0.031250 2021-03-26 06:30:22,368 epoch 53 - iter 40/50 - loss 2.51555236 - samples/sec: 74.72 - lr: 0.031250 2021-03-26 06:30:24,355 epoch 53 - iter 45/50 - loss 2.46132856 - samples/sec: 80.60 - lr: 0.031250 2021-03-26 06:30:25,991 epoch 53 - iter 50/50 - loss 2.44961992 - samples/sec: 97.93 - lr: 0.031250 2021-03-26 06:30:25,992 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:30:25,992 EPOCH 53 done: loss 2.4496 - lr 0.0312500 2021-03-26 06:30:26,800 DEV : loss 6.995821952819824 - score 0.9069 2021-03-26 06:30:26,826 BAD EPOCHS (no improvement): 2 2021-03-26 06:30:26,827 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:30:28,642 epoch 54 - iter 5/50 - loss 1.79619451 - samples/sec: 88.23 - lr: 0.031250 2021-03-26 06:30:30,643 epoch 54 - iter 10/50 - loss 2.11190095 - samples/sec: 80.05 - lr: 0.031250 2021-03-26 06:30:32,587 epoch 54 - iter 15/50 - loss 2.20107423 - samples/sec: 82.38 - lr: 0.031250 2021-03-26 06:30:34,720 epoch 54 - iter 20/50 - loss 2.28467238 - samples/sec: 75.08 - lr: 0.031250 2021-03-26 06:30:36,636 epoch 54 - iter 25/50 - loss 2.27544066 - samples/sec: 83.63 - lr: 0.031250 2021-03-26 06:30:38,920 epoch 54 - iter 30/50 - loss 2.28766701 - samples/sec: 70.11 - lr: 0.031250 2021-03-26 06:30:40,846 epoch 54 - iter 35/50 - loss 2.34247447 - samples/sec: 83.17 - lr: 0.031250 2021-03-26 06:30:43,019 epoch 54 - iter 40/50 - loss 2.37213463 - samples/sec: 73.68 - lr: 0.031250 2021-03-26 06:30:45,034 epoch 54 - iter 45/50 - loss 2.38454656 - samples/sec: 79.47 - lr: 0.031250 2021-03-26 06:30:46,956 epoch 54 - iter 50/50 - loss 2.32289241 - samples/sec: 83.39 - lr: 0.031250 2021-03-26 06:30:46,957 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:30:46,958 EPOCH 54 done: loss 2.3229 - lr 0.0312500 2021-03-26 06:30:47,808 DEV : loss 6.978719711303711 - score 0.9081 2021-03-26 06:30:47,833 BAD EPOCHS (no improvement): 3 2021-03-26 06:30:47,834 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:30:49,840 epoch 55 - iter 5/50 - loss 2.17873693 - samples/sec: 79.88 - lr: 0.031250 2021-03-26 06:30:51,816 epoch 55 - iter 10/50 - loss 2.29883653 - samples/sec: 81.08 - lr: 0.031250 2021-03-26 06:30:53,847 epoch 55 - iter 15/50 - loss 2.39913157 - samples/sec: 78.84 - lr: 0.031250 2021-03-26 06:30:55,812 epoch 55 - iter 20/50 - loss 2.48835702 - samples/sec: 81.48 - lr: 0.031250 2021-03-26 06:30:57,836 epoch 55 - iter 25/50 - loss 2.56765687 - samples/sec: 79.18 - lr: 0.031250 2021-03-26 06:30:59,832 epoch 55 - iter 30/50 - loss 2.46267944 - samples/sec: 80.21 - lr: 0.031250 2021-03-26 06:31:01,858 epoch 55 - iter 35/50 - loss 2.45796065 - samples/sec: 79.03 - lr: 0.031250 2021-03-26 06:31:03,845 epoch 55 - iter 40/50 - loss 2.41670229 - samples/sec: 80.64 - lr: 0.031250 2021-03-26 06:31:05,833 epoch 55 - iter 45/50 - loss 2.39211874 - samples/sec: 80.56 - lr: 0.031250 2021-03-26 06:31:07,734 epoch 55 - iter 50/50 - loss 2.39450463 - samples/sec: 84.25 - lr: 0.031250 2021-03-26 06:31:07,735 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:31:07,735 EPOCH 55 done: loss 2.3945 - lr 0.0312500 2021-03-26 06:31:08,516 DEV : loss 6.9821391105651855 - score 0.9085 2021-03-26 06:31:08,542 BAD EPOCHS (no improvement): 4 2021-03-26 06:31:08,543 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:31:10,595 epoch 56 - iter 5/50 - loss 2.45446777 - samples/sec: 78.07 - lr: 0.015625 2021-03-26 06:31:12,510 epoch 56 - iter 10/50 - loss 2.22199507 - samples/sec: 83.59 - lr: 0.015625 2021-03-26 06:31:14,363 epoch 56 - iter 15/50 - loss 2.10502376 - samples/sec: 86.51 - lr: 0.015625 2021-03-26 06:31:16,485 epoch 56 - iter 20/50 - loss 2.24321173 - samples/sec: 75.47 - lr: 0.015625 2021-03-26 06:31:18,768 epoch 56 - iter 25/50 - loss 2.23664840 - samples/sec: 70.16 - lr: 0.015625 2021-03-26 06:31:20,821 epoch 56 - iter 30/50 - loss 2.30482445 - samples/sec: 77.98 - lr: 0.015625 2021-03-26 06:31:22,870 epoch 56 - iter 35/50 - loss 2.29642651 - samples/sec: 78.22 - lr: 0.015625 2021-03-26 06:31:25,184 epoch 56 - iter 40/50 - loss 2.26805057 - samples/sec: 69.19 - lr: 0.015625 2021-03-26 06:31:27,337 epoch 56 - iter 45/50 - loss 2.22978039 - samples/sec: 74.40 - lr: 0.015625 2021-03-26 06:31:29,212 epoch 56 - iter 50/50 - loss 2.24846376 - samples/sec: 85.42 - lr: 0.015625 2021-03-26 06:31:29,213 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:31:29,213 EPOCH 56 done: loss 2.2485 - lr 0.0156250 2021-03-26 06:31:30,036 DEV : loss 6.974067687988281 - score 0.9085 2021-03-26 06:31:30,062 BAD EPOCHS (no improvement): 1 2021-03-26 06:31:30,063 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:31:32,073 epoch 57 - iter 5/50 - loss 2.64384212 - samples/sec: 79.65 - lr: 0.015625 2021-03-26 06:31:34,058 epoch 57 - iter 10/50 - loss 2.46076876 - samples/sec: 80.66 - lr: 0.015625 2021-03-26 06:31:35,971 epoch 57 - iter 15/50 - loss 2.51936740 - samples/sec: 83.73 - lr: 0.015625 2021-03-26 06:31:37,840 epoch 57 - iter 20/50 - loss 2.41057147 - samples/sec: 85.70 - lr: 0.015625 2021-03-26 06:31:39,880 epoch 57 - iter 25/50 - loss 2.46316812 - samples/sec: 78.53 - lr: 0.015625 2021-03-26 06:31:41,926 epoch 57 - iter 30/50 - loss 2.50473585 - samples/sec: 78.26 - lr: 0.015625 2021-03-26 06:31:43,993 epoch 57 - iter 35/50 - loss 2.50068537 - samples/sec: 77.53 - lr: 0.015625 2021-03-26 06:31:46,047 epoch 57 - iter 40/50 - loss 2.52348277 - samples/sec: 77.97 - lr: 0.015625 2021-03-26 06:31:47,885 epoch 57 - iter 45/50 - loss 2.52473041 - samples/sec: 87.19 - lr: 0.015625 2021-03-26 06:31:49,749 epoch 57 - iter 50/50 - loss 2.51703615 - samples/sec: 85.91 - lr: 0.015625 2021-03-26 06:31:49,750 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:31:49,750 EPOCH 57 done: loss 2.5170 - lr 0.0156250 2021-03-26 06:31:50,525 DEV : loss 6.9953436851501465 - score 0.9085 2021-03-26 06:31:50,551 BAD EPOCHS (no improvement): 2 2021-03-26 06:31:50,552 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:31:52,377 epoch 58 - iter 5/50 - loss 1.99235198 - samples/sec: 87.79 - lr: 0.015625 2021-03-26 06:31:54,379 epoch 58 - iter 10/50 - loss 2.16711535 - samples/sec: 79.98 - lr: 0.015625 2021-03-26 06:31:56,416 epoch 58 - iter 15/50 - loss 2.26408356 - samples/sec: 78.63 - lr: 0.015625 2021-03-26 06:31:58,382 epoch 58 - iter 20/50 - loss 2.20503994 - samples/sec: 81.45 - lr: 0.015625 2021-03-26 06:32:00,325 epoch 58 - iter 25/50 - loss 2.27958061 - samples/sec: 82.63 - lr: 0.015625 2021-03-26 06:32:02,186 epoch 58 - iter 30/50 - loss 2.28277320 - samples/sec: 86.03 - lr: 0.015625 2021-03-26 06:32:04,143 epoch 58 - iter 35/50 - loss 2.27501016 - samples/sec: 82.02 - lr: 0.015625 2021-03-26 06:32:06,054 epoch 58 - iter 40/50 - loss 2.27238119 - samples/sec: 83.82 - lr: 0.015625 2021-03-26 06:32:08,064 epoch 58 - iter 45/50 - loss 2.24532957 - samples/sec: 79.72 - lr: 0.015625 2021-03-26 06:32:09,825 epoch 58 - iter 50/50 - loss 2.30386667 - samples/sec: 90.94 - lr: 0.015625 2021-03-26 06:32:09,825 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:32:09,826 EPOCH 58 done: loss 2.3039 - lr 0.0156250 2021-03-26 06:32:10,603 DEV : loss 6.978977203369141 - score 0.9085 2021-03-26 06:32:10,628 BAD EPOCHS (no improvement): 3 2021-03-26 06:32:10,629 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:32:12,571 epoch 59 - iter 5/50 - loss 2.65150757 - samples/sec: 82.47 - lr: 0.015625 2021-03-26 06:32:14,555 epoch 59 - iter 10/50 - loss 2.28247037 - samples/sec: 80.68 - lr: 0.015625 2021-03-26 06:32:16,560 epoch 59 - iter 15/50 - loss 2.24065713 - samples/sec: 79.88 - lr: 0.015625 2021-03-26 06:32:18,386 epoch 59 - iter 20/50 - loss 2.24658821 - samples/sec: 87.77 - lr: 0.015625 2021-03-26 06:32:20,261 epoch 59 - iter 25/50 - loss 2.21917336 - samples/sec: 85.44 - lr: 0.015625 2021-03-26 06:32:22,271 epoch 59 - iter 30/50 - loss 2.27102419 - samples/sec: 79.68 - lr: 0.015625 2021-03-26 06:32:24,218 epoch 59 - iter 35/50 - loss 2.29636217 - samples/sec: 82.24 - lr: 0.015625 2021-03-26 06:32:26,292 epoch 59 - iter 40/50 - loss 2.35502929 - samples/sec: 77.23 - lr: 0.015625 2021-03-26 06:32:28,437 epoch 59 - iter 45/50 - loss 2.34205148 - samples/sec: 74.70 - lr: 0.015625 2021-03-26 06:32:30,252 epoch 59 - iter 50/50 - loss 2.32947894 - samples/sec: 88.26 - lr: 0.015625 2021-03-26 06:32:30,254 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:32:30,254 EPOCH 59 done: loss 2.3295 - lr 0.0156250 2021-03-26 06:32:31,029 DEV : loss 6.991868495941162 - score 0.9077 2021-03-26 06:32:31,055 BAD EPOCHS (no improvement): 4 2021-03-26 06:32:31,055 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:32:32,924 epoch 60 - iter 5/50 - loss 2.06255875 - samples/sec: 85.70 - lr: 0.007812 2021-03-26 06:32:35,056 epoch 60 - iter 10/50 - loss 2.46557306 - samples/sec: 75.15 - lr: 0.007812 2021-03-26 06:32:37,085 epoch 60 - iter 15/50 - loss 2.41977864 - samples/sec: 78.95 - lr: 0.007812 2021-03-26 06:32:39,435 epoch 60 - iter 20/50 - loss 2.33271241 - samples/sec: 68.18 - lr: 0.007812 2021-03-26 06:32:41,548 epoch 60 - iter 25/50 - loss 2.40608020 - samples/sec: 75.81 - lr: 0.007812 2021-03-26 06:32:43,612 epoch 60 - iter 30/50 - loss 2.41698582 - samples/sec: 77.60 - lr: 0.007812 2021-03-26 06:32:45,488 epoch 60 - iter 35/50 - loss 2.41312816 - samples/sec: 85.38 - lr: 0.007812 2021-03-26 06:32:47,628 epoch 60 - iter 40/50 - loss 2.44367003 - samples/sec: 74.83 - lr: 0.007812 2021-03-26 06:32:49,645 epoch 60 - iter 45/50 - loss 2.46628921 - samples/sec: 79.39 - lr: 0.007812 2021-03-26 06:32:51,504 epoch 60 - iter 50/50 - loss 2.49696096 - samples/sec: 86.20 - lr: 0.007812 2021-03-26 06:32:51,505 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:32:51,505 EPOCH 60 done: loss 2.4970 - lr 0.0078125 2021-03-26 06:32:52,343 DEV : loss 6.983875274658203 - score 0.9085 2021-03-26 06:32:52,368 BAD EPOCHS (no improvement): 1 2021-03-26 06:32:52,369 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:32:54,449 epoch 61 - iter 5/50 - loss 2.79651160 - samples/sec: 76.99 - lr: 0.007812 2021-03-26 06:32:56,568 epoch 61 - iter 10/50 - loss 2.62580528 - samples/sec: 75.57 - lr: 0.007812 2021-03-26 06:32:58,704 epoch 61 - iter 15/50 - loss 2.37854838 - samples/sec: 74.97 - lr: 0.007812 2021-03-26 06:33:00,696 epoch 61 - iter 20/50 - loss 2.32473083 - samples/sec: 80.40 - lr: 0.007812 2021-03-26 06:33:02,726 epoch 61 - iter 25/50 - loss 2.32159848 - samples/sec: 78.89 - lr: 0.007812 2021-03-26 06:33:04,813 epoch 61 - iter 30/50 - loss 2.37875148 - samples/sec: 76.72 - lr: 0.007812 2021-03-26 06:33:06,687 epoch 61 - iter 35/50 - loss 2.38597493 - samples/sec: 85.53 - lr: 0.007812 2021-03-26 06:33:08,626 epoch 61 - iter 40/50 - loss 2.39198539 - samples/sec: 82.63 - lr: 0.007812 2021-03-26 06:33:10,685 epoch 61 - iter 45/50 - loss 2.36188733 - samples/sec: 77.77 - lr: 0.007812 2021-03-26 06:33:12,661 epoch 61 - iter 50/50 - loss 2.39228940 - samples/sec: 81.06 - lr: 0.007812 2021-03-26 06:33:12,662 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:33:12,662 EPOCH 61 done: loss 2.3923 - lr 0.0078125 2021-03-26 06:33:13,474 DEV : loss 6.977319240570068 - score 0.9081 2021-03-26 06:33:13,498 BAD EPOCHS (no improvement): 2 2021-03-26 06:33:13,498 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:33:15,563 epoch 62 - iter 5/50 - loss 2.31099248 - samples/sec: 77.58 - lr: 0.007812 2021-03-26 06:33:17,406 epoch 62 - iter 10/50 - loss 2.62232625 - samples/sec: 86.91 - lr: 0.007812 2021-03-26 06:33:19,558 epoch 62 - iter 15/50 - loss 2.64010038 - samples/sec: 74.41 - lr: 0.007812 2021-03-26 06:33:21,592 epoch 62 - iter 20/50 - loss 2.55420481 - samples/sec: 78.75 - lr: 0.007812 2021-03-26 06:33:23,697 epoch 62 - iter 25/50 - loss 2.54057485 - samples/sec: 76.06 - lr: 0.007812 2021-03-26 06:33:25,633 epoch 62 - iter 30/50 - loss 2.48879879 - samples/sec: 82.80 - lr: 0.007812 2021-03-26 06:33:27,457 epoch 62 - iter 35/50 - loss 2.39246745 - samples/sec: 87.83 - lr: 0.007812 2021-03-26 06:33:29,382 epoch 62 - iter 40/50 - loss 2.38044807 - samples/sec: 83.24 - lr: 0.007812 2021-03-26 06:33:31,243 epoch 62 - iter 45/50 - loss 2.39029401 - samples/sec: 86.12 - lr: 0.007812 2021-03-26 06:33:33,048 epoch 62 - iter 50/50 - loss 2.34383795 - samples/sec: 88.71 - lr: 0.007812 2021-03-26 06:33:33,050 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:33:33,050 EPOCH 62 done: loss 2.3438 - lr 0.0078125 2021-03-26 06:33:33,867 DEV : loss 6.974912166595459 - score 0.9081 2021-03-26 06:33:33,886 BAD EPOCHS (no improvement): 3 2021-03-26 06:33:33,887 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:33:35,800 epoch 63 - iter 5/50 - loss 2.44417329 - samples/sec: 83.72 - lr: 0.007812 2021-03-26 06:33:37,832 epoch 63 - iter 10/50 - loss 2.33602285 - samples/sec: 78.83 - lr: 0.007812 2021-03-26 06:33:39,903 epoch 63 - iter 15/50 - loss 2.30568608 - samples/sec: 77.32 - lr: 0.007812 2021-03-26 06:33:41,871 epoch 63 - iter 20/50 - loss 2.34029179 - samples/sec: 81.38 - lr: 0.007812 2021-03-26 06:33:43,840 epoch 63 - iter 25/50 - loss 2.40513277 - samples/sec: 81.33 - lr: 0.007812 2021-03-26 06:33:45,749 epoch 63 - iter 30/50 - loss 2.46392405 - samples/sec: 83.88 - lr: 0.007812 2021-03-26 06:33:47,690 epoch 63 - iter 35/50 - loss 2.45260457 - samples/sec: 82.53 - lr: 0.007812 2021-03-26 06:33:49,832 epoch 63 - iter 40/50 - loss 2.47064377 - samples/sec: 74.75 - lr: 0.007812 2021-03-26 06:33:51,744 epoch 63 - iter 45/50 - loss 2.47501290 - samples/sec: 83.76 - lr: 0.007812 2021-03-26 06:33:53,476 epoch 63 - iter 50/50 - loss 2.43155539 - samples/sec: 92.54 - lr: 0.007812 2021-03-26 06:33:53,477 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:33:53,477 EPOCH 63 done: loss 2.4316 - lr 0.0078125 2021-03-26 06:33:54,263 DEV : loss 6.969865322113037 - score 0.9069 2021-03-26 06:33:54,281 BAD EPOCHS (no improvement): 4 2021-03-26 06:33:54,281 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:33:56,270 epoch 64 - iter 5/50 - loss 2.39939275 - samples/sec: 80.54 - lr: 0.003906 2021-03-26 06:33:58,286 epoch 64 - iter 10/50 - loss 2.33425676 - samples/sec: 79.44 - lr: 0.003906 2021-03-26 06:34:00,204 epoch 64 - iter 15/50 - loss 2.25310161 - samples/sec: 83.49 - lr: 0.003906 2021-03-26 06:34:02,188 epoch 64 - iter 20/50 - loss 2.31001709 - samples/sec: 80.76 - lr: 0.003906 2021-03-26 06:34:04,360 epoch 64 - iter 25/50 - loss 2.33261591 - samples/sec: 73.76 - lr: 0.003906 2021-03-26 06:34:06,230 epoch 64 - iter 30/50 - loss 2.31331635 - samples/sec: 85.62 - lr: 0.003906 2021-03-26 06:34:08,207 epoch 64 - iter 35/50 - loss 2.21878336 - samples/sec: 81.01 - lr: 0.003906 2021-03-26 06:34:10,376 epoch 64 - iter 40/50 - loss 2.24540700 - samples/sec: 73.88 - lr: 0.003906 2021-03-26 06:34:12,411 epoch 64 - iter 45/50 - loss 2.26711696 - samples/sec: 78.71 - lr: 0.003906 2021-03-26 06:34:14,178 epoch 64 - iter 50/50 - loss 2.25436100 - samples/sec: 90.64 - lr: 0.003906 2021-03-26 06:34:14,179 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:34:14,179 EPOCH 64 done: loss 2.2544 - lr 0.0039062 2021-03-26 06:34:14,985 DEV : loss 6.970977783203125 - score 0.9069 2021-03-26 06:34:15,008 BAD EPOCHS (no improvement): 1 2021-03-26 06:34:15,009 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:34:17,126 epoch 65 - iter 5/50 - loss 2.40217991 - samples/sec: 75.67 - lr: 0.003906 2021-03-26 06:34:19,195 epoch 65 - iter 10/50 - loss 2.63361117 - samples/sec: 77.41 - lr: 0.003906 2021-03-26 06:34:21,188 epoch 65 - iter 15/50 - loss 2.50125097 - samples/sec: 80.35 - lr: 0.003906 2021-03-26 06:34:23,109 epoch 65 - iter 20/50 - loss 2.52610157 - samples/sec: 83.38 - lr: 0.003906 2021-03-26 06:34:25,070 epoch 65 - iter 25/50 - loss 2.49457950 - samples/sec: 81.69 - lr: 0.003906 2021-03-26 06:34:26,939 epoch 65 - iter 30/50 - loss 2.41889030 - samples/sec: 85.65 - lr: 0.003906 2021-03-26 06:34:28,862 epoch 65 - iter 35/50 - loss 2.39602560 - samples/sec: 83.31 - lr: 0.003906 2021-03-26 06:34:31,097 epoch 65 - iter 40/50 - loss 2.44050950 - samples/sec: 71.67 - lr: 0.003906 2021-03-26 06:34:33,128 epoch 65 - iter 45/50 - loss 2.48830043 - samples/sec: 78.83 - lr: 0.003906 2021-03-26 06:34:35,244 epoch 65 - iter 50/50 - loss 2.49684168 - samples/sec: 75.70 - lr: 0.003906 2021-03-26 06:34:35,245 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:34:35,245 EPOCH 65 done: loss 2.4968 - lr 0.0039062 2021-03-26 06:34:36,057 DEV : loss 6.972274303436279 - score 0.9073 2021-03-26 06:34:36,078 BAD EPOCHS (no improvement): 2 2021-03-26 06:34:36,079 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:34:38,081 epoch 66 - iter 5/50 - loss 2.64790382 - samples/sec: 79.98 - lr: 0.003906 2021-03-26 06:34:40,053 epoch 66 - iter 10/50 - loss 2.31422681 - samples/sec: 81.27 - lr: 0.003906 2021-03-26 06:34:42,108 epoch 66 - iter 15/50 - loss 2.18160001 - samples/sec: 77.93 - lr: 0.003906 2021-03-26 06:34:44,010 epoch 66 - iter 20/50 - loss 2.20333160 - samples/sec: 84.24 - lr: 0.003906 2021-03-26 06:34:46,138 epoch 66 - iter 25/50 - loss 2.22043241 - samples/sec: 75.27 - lr: 0.003906 2021-03-26 06:34:48,024 epoch 66 - iter 30/50 - loss 2.23266133 - samples/sec: 84.94 - lr: 0.003906 2021-03-26 06:34:50,202 epoch 66 - iter 35/50 - loss 2.22326636 - samples/sec: 73.52 - lr: 0.003906 2021-03-26 06:34:52,060 epoch 66 - iter 40/50 - loss 2.22595744 - samples/sec: 86.24 - lr: 0.003906 2021-03-26 06:34:54,013 epoch 66 - iter 45/50 - loss 2.27869542 - samples/sec: 82.05 - lr: 0.003906 2021-03-26 06:34:55,845 epoch 66 - iter 50/50 - loss 2.30708797 - samples/sec: 87.42 - lr: 0.003906 2021-03-26 06:34:55,846 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:34:55,847 EPOCH 66 done: loss 2.3071 - lr 0.0039062 2021-03-26 06:34:56,675 DEV : loss 6.974557399749756 - score 0.9077 2021-03-26 06:34:56,714 BAD EPOCHS (no improvement): 3 2021-03-26 06:34:56,715 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:34:58,841 epoch 67 - iter 5/50 - loss 2.96658783 - samples/sec: 75.32 - lr: 0.003906 2021-03-26 06:35:00,812 epoch 67 - iter 10/50 - loss 2.71965330 - samples/sec: 81.26 - lr: 0.003906 2021-03-26 06:35:02,970 epoch 67 - iter 15/50 - loss 2.48100974 - samples/sec: 74.21 - lr: 0.003906 2021-03-26 06:35:04,864 epoch 67 - iter 20/50 - loss 2.41596460 - samples/sec: 84.56 - lr: 0.003906 2021-03-26 06:35:06,998 epoch 67 - iter 25/50 - loss 2.46593875 - samples/sec: 75.08 - lr: 0.003906 2021-03-26 06:35:08,909 epoch 67 - iter 30/50 - loss 2.43296328 - samples/sec: 83.79 - lr: 0.003906 2021-03-26 06:35:10,769 epoch 67 - iter 35/50 - loss 2.38436657 - samples/sec: 86.10 - lr: 0.003906 2021-03-26 06:35:12,947 epoch 67 - iter 40/50 - loss 2.38160687 - samples/sec: 73.52 - lr: 0.003906 2021-03-26 06:35:14,827 epoch 67 - iter 45/50 - loss 2.41164478 - samples/sec: 85.23 - lr: 0.003906 2021-03-26 06:35:16,677 epoch 67 - iter 50/50 - loss 2.36796960 - samples/sec: 86.61 - lr: 0.003906 2021-03-26 06:35:16,677 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:35:16,678 EPOCH 67 done: loss 2.3680 - lr 0.0039062 2021-03-26 06:35:17,495 DEV : loss 6.978951930999756 - score 0.9081 2021-03-26 06:35:17,520 BAD EPOCHS (no improvement): 4 2021-03-26 06:35:17,520 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:35:19,624 epoch 68 - iter 5/50 - loss 2.00108428 - samples/sec: 76.15 - lr: 0.001953 2021-03-26 06:35:21,586 epoch 68 - iter 10/50 - loss 2.05748752 - samples/sec: 81.61 - lr: 0.001953 2021-03-26 06:35:23,543 epoch 68 - iter 15/50 - loss 2.20883069 - samples/sec: 81.81 - lr: 0.001953 2021-03-26 06:35:25,506 epoch 68 - iter 20/50 - loss 2.31636751 - samples/sec: 81.62 - lr: 0.001953 2021-03-26 06:35:27,585 epoch 68 - iter 25/50 - loss 2.26701699 - samples/sec: 77.00 - lr: 0.001953 2021-03-26 06:35:29,632 epoch 68 - iter 30/50 - loss 2.25838938 - samples/sec: 78.25 - lr: 0.001953 2021-03-26 06:35:31,617 epoch 68 - iter 35/50 - loss 2.29719248 - samples/sec: 80.74 - lr: 0.001953 2021-03-26 06:35:33,673 epoch 68 - iter 40/50 - loss 2.31215978 - samples/sec: 77.88 - lr: 0.001953 2021-03-26 06:35:35,496 epoch 68 - iter 45/50 - loss 2.28480958 - samples/sec: 87.82 - lr: 0.001953 2021-03-26 06:35:37,365 epoch 68 - iter 50/50 - loss 2.36084390 - samples/sec: 85.70 - lr: 0.001953 2021-03-26 06:35:37,366 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:35:37,366 EPOCH 68 done: loss 2.3608 - lr 0.0019531 2021-03-26 06:35:38,197 DEV : loss 6.982030391693115 - score 0.9085 2021-03-26 06:35:38,225 BAD EPOCHS (no improvement): 1 2021-03-26 06:35:38,226 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:35:40,234 epoch 69 - iter 5/50 - loss 2.34777708 - samples/sec: 79.76 - lr: 0.001953 2021-03-26 06:35:42,268 epoch 69 - iter 10/50 - loss 2.57082732 - samples/sec: 78.72 - lr: 0.001953 2021-03-26 06:35:44,194 epoch 69 - iter 15/50 - loss 2.47520032 - samples/sec: 83.17 - lr: 0.001953 2021-03-26 06:35:46,157 epoch 69 - iter 20/50 - loss 2.37645018 - samples/sec: 81.61 - lr: 0.001953 2021-03-26 06:35:48,206 epoch 69 - iter 25/50 - loss 2.29699348 - samples/sec: 78.14 - lr: 0.001953 2021-03-26 06:35:50,224 epoch 69 - iter 30/50 - loss 2.33158594 - samples/sec: 79.37 - lr: 0.001953 2021-03-26 06:35:52,198 epoch 69 - iter 35/50 - loss 2.37844305 - samples/sec: 81.14 - lr: 0.001953 2021-03-26 06:35:54,057 epoch 69 - iter 40/50 - loss 2.34696364 - samples/sec: 86.16 - lr: 0.001953 2021-03-26 06:35:56,198 epoch 69 - iter 45/50 - loss 2.36180254 - samples/sec: 74.80 - lr: 0.001953 2021-03-26 06:35:58,097 epoch 69 - iter 50/50 - loss 2.36717457 - samples/sec: 84.33 - lr: 0.001953 2021-03-26 06:35:58,098 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:35:58,098 EPOCH 69 done: loss 2.3672 - lr 0.0019531 2021-03-26 06:35:58,905 DEV : loss 6.981610298156738 - score 0.9085 2021-03-26 06:35:58,931 BAD EPOCHS (no improvement): 2 2021-03-26 06:35:58,932 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:36:00,950 epoch 70 - iter 5/50 - loss 2.39901481 - samples/sec: 79.38 - lr: 0.001953 2021-03-26 06:36:02,836 epoch 70 - iter 10/50 - loss 2.35888321 - samples/sec: 84.88 - lr: 0.001953 2021-03-26 06:36:04,860 epoch 70 - iter 15/50 - loss 2.43844715 - samples/sec: 79.15 - lr: 0.001953 2021-03-26 06:36:06,786 epoch 70 - iter 20/50 - loss 2.33588195 - samples/sec: 83.12 - lr: 0.001953 2021-03-26 06:36:08,782 epoch 70 - iter 25/50 - loss 2.25162241 - samples/sec: 80.30 - lr: 0.001953 2021-03-26 06:36:10,721 epoch 70 - iter 30/50 - loss 2.21778770 - samples/sec: 82.58 - lr: 0.001953 2021-03-26 06:36:12,888 epoch 70 - iter 35/50 - loss 2.20874165 - samples/sec: 73.90 - lr: 0.001953 2021-03-26 06:36:14,860 epoch 70 - iter 40/50 - loss 2.23795464 - samples/sec: 81.20 - lr: 0.001953 2021-03-26 06:36:16,889 epoch 70 - iter 45/50 - loss 2.28399056 - samples/sec: 78.91 - lr: 0.001953 2021-03-26 06:36:18,675 epoch 70 - iter 50/50 - loss 2.28482699 - samples/sec: 89.68 - lr: 0.001953 2021-03-26 06:36:18,676 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:36:18,676 EPOCH 70 done: loss 2.2848 - lr 0.0019531 2021-03-26 06:36:19,499 DEV : loss 6.978541851043701 - score 0.9085 2021-03-26 06:36:19,526 BAD EPOCHS (no improvement): 3 2021-03-26 06:36:19,526 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:36:21,448 epoch 71 - iter 5/50 - loss 2.11014709 - samples/sec: 83.38 - lr: 0.001953 2021-03-26 06:36:23,579 epoch 71 - iter 10/50 - loss 2.29674499 - samples/sec: 75.16 - lr: 0.001953 2021-03-26 06:36:25,884 epoch 71 - iter 15/50 - loss 2.46122870 - samples/sec: 69.46 - lr: 0.001953 2021-03-26 06:36:27,768 epoch 71 - iter 20/50 - loss 2.38656909 - samples/sec: 85.01 - lr: 0.001953 2021-03-26 06:36:29,708 epoch 71 - iter 25/50 - loss 2.40159395 - samples/sec: 82.54 - lr: 0.001953 2021-03-26 06:36:31,656 epoch 71 - iter 30/50 - loss 2.42725286 - samples/sec: 82.26 - lr: 0.001953 2021-03-26 06:36:33,384 epoch 71 - iter 35/50 - loss 2.40592557 - samples/sec: 92.71 - lr: 0.001953 2021-03-26 06:36:35,346 epoch 71 - iter 40/50 - loss 2.38390470 - samples/sec: 81.63 - lr: 0.001953 2021-03-26 06:36:37,375 epoch 71 - iter 45/50 - loss 2.38397009 - samples/sec: 78.91 - lr: 0.001953 2021-03-26 06:36:39,274 epoch 71 - iter 50/50 - loss 2.38416300 - samples/sec: 84.33 - lr: 0.001953 2021-03-26 06:36:39,275 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:36:39,275 EPOCH 71 done: loss 2.3842 - lr 0.0019531 2021-03-26 06:36:40,111 DEV : loss 6.977227687835693 - score 0.9081 2021-03-26 06:36:40,138 BAD EPOCHS (no improvement): 4 2021-03-26 06:36:40,138 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:36:42,121 epoch 72 - iter 5/50 - loss 2.51898463 - samples/sec: 80.79 - lr: 0.000977 2021-03-26 06:36:44,016 epoch 72 - iter 10/50 - loss 2.57655363 - samples/sec: 84.49 - lr: 0.000977 2021-03-26 06:36:45,969 epoch 72 - iter 15/50 - loss 2.44915295 - samples/sec: 82.02 - lr: 0.000977 2021-03-26 06:36:47,892 epoch 72 - iter 20/50 - loss 2.42828819 - samples/sec: 83.27 - lr: 0.000977 2021-03-26 06:36:49,708 epoch 72 - iter 25/50 - loss 2.40627950 - samples/sec: 88.18 - lr: 0.000977 2021-03-26 06:36:51,922 epoch 72 - iter 30/50 - loss 2.45578847 - samples/sec: 72.32 - lr: 0.000977 2021-03-26 06:36:54,237 epoch 72 - iter 35/50 - loss 2.44811320 - samples/sec: 69.20 - lr: 0.000977 2021-03-26 06:36:56,060 epoch 72 - iter 40/50 - loss 2.42384168 - samples/sec: 87.83 - lr: 0.000977 2021-03-26 06:36:58,065 epoch 72 - iter 45/50 - loss 2.46919827 - samples/sec: 79.86 - lr: 0.000977 2021-03-26 06:36:59,973 epoch 72 - iter 50/50 - loss 2.42740743 - samples/sec: 83.94 - lr: 0.000977 2021-03-26 06:36:59,974 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:36:59,974 EPOCH 72 done: loss 2.4274 - lr 0.0009766 2021-03-26 06:37:00,798 DEV : loss 6.975755214691162 - score 0.9077 2021-03-26 06:37:00,824 BAD EPOCHS (no improvement): 1 2021-03-26 06:37:00,825 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:37:02,909 epoch 73 - iter 5/50 - loss 2.65864558 - samples/sec: 76.82 - lr: 0.000977 2021-03-26 06:37:04,841 epoch 73 - iter 10/50 - loss 2.38106189 - samples/sec: 82.91 - lr: 0.000977 2021-03-26 06:37:06,808 epoch 73 - iter 15/50 - loss 2.48718756 - samples/sec: 81.41 - lr: 0.000977 2021-03-26 06:37:08,729 epoch 73 - iter 20/50 - loss 2.44999477 - samples/sec: 83.40 - lr: 0.000977 2021-03-26 06:37:10,749 epoch 73 - iter 25/50 - loss 2.38722777 - samples/sec: 79.28 - lr: 0.000977 2021-03-26 06:37:12,831 epoch 73 - iter 30/50 - loss 2.47670416 - samples/sec: 76.93 - lr: 0.000977 2021-03-26 06:37:14,732 epoch 73 - iter 35/50 - loss 2.47073682 - samples/sec: 84.24 - lr: 0.000977 2021-03-26 06:37:16,732 epoch 73 - iter 40/50 - loss 2.47696608 - samples/sec: 80.08 - lr: 0.000977 2021-03-26 06:37:18,700 epoch 73 - iter 45/50 - loss 2.46970473 - samples/sec: 81.40 - lr: 0.000977 2021-03-26 06:37:20,574 epoch 73 - iter 50/50 - loss 2.43881308 - samples/sec: 85.43 - lr: 0.000977 2021-03-26 06:37:20,575 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:37:20,575 EPOCH 73 done: loss 2.4388 - lr 0.0009766 2021-03-26 06:37:21,431 DEV : loss 6.9776506423950195 - score 0.9077 2021-03-26 06:37:21,457 BAD EPOCHS (no improvement): 2 2021-03-26 06:37:21,458 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:37:23,596 epoch 74 - iter 5/50 - loss 2.47849181 - samples/sec: 74.89 - lr: 0.000977 2021-03-26 06:37:25,583 epoch 74 - iter 10/50 - loss 2.28436197 - samples/sec: 80.65 - lr: 0.000977 2021-03-26 06:37:27,686 epoch 74 - iter 15/50 - loss 2.22879147 - samples/sec: 76.14 - lr: 0.000977 2021-03-26 06:37:29,797 epoch 74 - iter 20/50 - loss 2.20650732 - samples/sec: 75.85 - lr: 0.000977 2021-03-26 06:37:31,671 epoch 74 - iter 25/50 - loss 2.30046971 - samples/sec: 85.46 - lr: 0.000977 2021-03-26 06:37:33,840 epoch 74 - iter 30/50 - loss 2.26771376 - samples/sec: 73.84 - lr: 0.000977 2021-03-26 06:37:36,030 epoch 74 - iter 35/50 - loss 2.31302182 - samples/sec: 73.12 - lr: 0.000977 2021-03-26 06:37:38,171 epoch 74 - iter 40/50 - loss 2.27059739 - samples/sec: 74.85 - lr: 0.000977 2021-03-26 06:37:40,057 epoch 74 - iter 45/50 - loss 2.24902633 - samples/sec: 84.93 - lr: 0.000977 2021-03-26 06:37:41,941 epoch 74 - iter 50/50 - loss 2.22677505 - samples/sec: 85.03 - lr: 0.000977 2021-03-26 06:37:41,942 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:37:41,942 EPOCH 74 done: loss 2.2268 - lr 0.0009766 2021-03-26 06:37:42,762 DEV : loss 6.978555202484131 - score 0.9085 2021-03-26 06:37:42,784 BAD EPOCHS (no improvement): 3 2021-03-26 06:37:42,785 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:37:44,741 epoch 75 - iter 5/50 - loss 2.61107960 - samples/sec: 81.92 - lr: 0.000977 2021-03-26 06:37:46,814 epoch 75 - iter 10/50 - loss 2.43473128 - samples/sec: 77.27 - lr: 0.000977 2021-03-26 06:37:48,770 epoch 75 - iter 15/50 - loss 2.42162393 - samples/sec: 81.89 - lr: 0.000977 2021-03-26 06:37:51,045 epoch 75 - iter 20/50 - loss 2.41610397 - samples/sec: 70.39 - lr: 0.000977 2021-03-26 06:37:53,177 epoch 75 - iter 25/50 - loss 2.37377398 - samples/sec: 75.12 - lr: 0.000977 2021-03-26 06:37:55,129 epoch 75 - iter 30/50 - loss 2.37280536 - samples/sec: 82.05 - lr: 0.000977 2021-03-26 06:37:57,302 epoch 75 - iter 35/50 - loss 2.35321695 - samples/sec: 73.71 - lr: 0.000977 2021-03-26 06:37:59,206 epoch 75 - iter 40/50 - loss 2.37838320 - samples/sec: 84.10 - lr: 0.000977 2021-03-26 06:38:01,143 epoch 75 - iter 45/50 - loss 2.36447463 - samples/sec: 82.70 - lr: 0.000977 2021-03-26 06:38:02,955 epoch 75 - iter 50/50 - loss 2.33910566 - samples/sec: 88.43 - lr: 0.000977 2021-03-26 06:38:02,956 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:38:02,956 EPOCH 75 done: loss 2.3391 - lr 0.0009766 2021-03-26 06:38:03,760 DEV : loss 6.977889060974121 - score 0.9081 2021-03-26 06:38:03,779 BAD EPOCHS (no improvement): 4 2021-03-26 06:38:03,780 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:38:05,974 epoch 76 - iter 5/50 - loss 2.18173971 - samples/sec: 72.99 - lr: 0.000488 2021-03-26 06:38:08,091 epoch 76 - iter 10/50 - loss 2.49060502 - samples/sec: 75.63 - lr: 0.000488 2021-03-26 06:38:10,140 epoch 76 - iter 15/50 - loss 2.63107382 - samples/sec: 78.19 - lr: 0.000488 2021-03-26 06:38:12,034 epoch 76 - iter 20/50 - loss 2.49251807 - samples/sec: 84.56 - lr: 0.000488 2021-03-26 06:38:14,076 epoch 76 - iter 25/50 - loss 2.46709409 - samples/sec: 78.41 - lr: 0.000488 2021-03-26 06:38:16,021 epoch 76 - iter 30/50 - loss 2.41153224 - samples/sec: 82.33 - lr: 0.000488 2021-03-26 06:38:18,042 epoch 76 - iter 35/50 - loss 2.44753593 - samples/sec: 79.30 - lr: 0.000488 2021-03-26 06:38:20,055 epoch 76 - iter 40/50 - loss 2.40904220 - samples/sec: 79.55 - lr: 0.000488 2021-03-26 06:38:22,198 epoch 76 - iter 45/50 - loss 2.36160962 - samples/sec: 74.75 - lr: 0.000488 2021-03-26 06:38:23,977 epoch 76 - iter 50/50 - loss 2.33519658 - samples/sec: 90.02 - lr: 0.000488 2021-03-26 06:38:23,978 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:38:23,979 EPOCH 76 done: loss 2.3352 - lr 0.0004883 2021-03-26 06:38:24,807 DEV : loss 6.977874755859375 - score 0.9085 2021-03-26 06:38:24,832 BAD EPOCHS (no improvement): 1 2021-03-26 06:38:24,833 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:38:26,749 epoch 77 - iter 5/50 - loss 2.34179015 - samples/sec: 83.62 - lr: 0.000488 2021-03-26 06:38:28,651 epoch 77 - iter 10/50 - loss 2.40300212 - samples/sec: 84.21 - lr: 0.000488 2021-03-26 06:38:30,673 epoch 77 - iter 15/50 - loss 2.56726098 - samples/sec: 79.18 - lr: 0.000488 2021-03-26 06:38:32,700 epoch 77 - iter 20/50 - loss 2.45115488 - samples/sec: 78.98 - lr: 0.000488 2021-03-26 06:38:34,697 epoch 77 - iter 25/50 - loss 2.38101733 - samples/sec: 80.25 - lr: 0.000488 2021-03-26 06:38:36,544 epoch 77 - iter 30/50 - loss 2.32366014 - samples/sec: 86.75 - lr: 0.000488 2021-03-26 06:38:38,512 epoch 77 - iter 35/50 - loss 2.29412194 - samples/sec: 81.38 - lr: 0.000488 2021-03-26 06:38:40,441 epoch 77 - iter 40/50 - loss 2.28109241 - samples/sec: 83.00 - lr: 0.000488 2021-03-26 06:38:42,316 epoch 77 - iter 45/50 - loss 2.24014996 - samples/sec: 85.40 - lr: 0.000488 2021-03-26 06:38:44,230 epoch 77 - iter 50/50 - loss 2.22467905 - samples/sec: 83.71 - lr: 0.000488 2021-03-26 06:38:44,231 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:38:44,231 EPOCH 77 done: loss 2.2247 - lr 0.0004883 2021-03-26 06:38:45,039 DEV : loss 6.978282928466797 - score 0.9085 2021-03-26 06:38:45,065 BAD EPOCHS (no improvement): 2 2021-03-26 06:38:45,066 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:38:47,132 epoch 78 - iter 5/50 - loss 2.36803710 - samples/sec: 77.53 - lr: 0.000488 2021-03-26 06:38:49,245 epoch 78 - iter 10/50 - loss 2.36767075 - samples/sec: 75.80 - lr: 0.000488 2021-03-26 06:38:51,321 epoch 78 - iter 15/50 - loss 2.36822032 - samples/sec: 77.12 - lr: 0.000488 2021-03-26 06:38:53,205 epoch 78 - iter 20/50 - loss 2.43670136 - samples/sec: 85.02 - lr: 0.000488 2021-03-26 06:38:56,392 epoch 78 - iter 25/50 - loss 2.54821656 - samples/sec: 50.23 - lr: 0.000488 2021-03-26 06:38:58,274 epoch 78 - iter 30/50 - loss 2.50956246 - samples/sec: 85.14 - lr: 0.000488 2021-03-26 06:39:00,269 epoch 78 - iter 35/50 - loss 2.50315506 - samples/sec: 80.26 - lr: 0.000488 2021-03-26 06:39:02,195 epoch 78 - iter 40/50 - loss 2.48417808 - samples/sec: 83.11 - lr: 0.000488 2021-03-26 06:39:04,387 epoch 78 - iter 45/50 - loss 2.48333181 - samples/sec: 73.10 - lr: 0.000488 2021-03-26 06:39:06,265 epoch 78 - iter 50/50 - loss 2.51921896 - samples/sec: 85.24 - lr: 0.000488 2021-03-26 06:39:06,266 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:39:06,266 EPOCH 78 done: loss 2.5192 - lr 0.0004883 2021-03-26 06:39:07,058 DEV : loss 6.978475093841553 - score 0.9089 2021-03-26 06:39:07,077 BAD EPOCHS (no improvement): 0 2021-03-26 06:39:16,637 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:39:18,659 epoch 79 - iter 5/50 - loss 2.17209723 - samples/sec: 79.21 - lr: 0.000488 2021-03-26 06:39:20,697 epoch 79 - iter 10/50 - loss 2.41715573 - samples/sec: 78.56 - lr: 0.000488 2021-03-26 06:39:22,792 epoch 79 - iter 15/50 - loss 2.41948086 - samples/sec: 76.43 - lr: 0.000488 2021-03-26 06:39:24,817 epoch 79 - iter 20/50 - loss 2.34051616 - samples/sec: 79.12 - lr: 0.000488 2021-03-26 06:39:26,806 epoch 79 - iter 25/50 - loss 2.37092519 - samples/sec: 80.54 - lr: 0.000488 2021-03-26 06:39:28,681 epoch 79 - iter 30/50 - loss 2.37991382 - samples/sec: 85.39 - lr: 0.000488 2021-03-26 06:39:30,604 epoch 79 - iter 35/50 - loss 2.33880022 - samples/sec: 83.30 - lr: 0.000488 2021-03-26 06:39:32,575 epoch 79 - iter 40/50 - loss 2.31781658 - samples/sec: 81.27 - lr: 0.000488 2021-03-26 06:39:34,484 epoch 79 - iter 45/50 - loss 2.31230679 - samples/sec: 83.87 - lr: 0.000488 2021-03-26 06:39:36,253 epoch 79 - iter 50/50 - loss 2.32847360 - samples/sec: 90.55 - lr: 0.000488 2021-03-26 06:39:36,253 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:39:36,254 EPOCH 79 done: loss 2.3285 - lr 0.0004883 2021-03-26 06:39:37,043 DEV : loss 6.978264808654785 - score 0.9085 2021-03-26 06:39:37,068 BAD EPOCHS (no improvement): 1 2021-03-26 06:39:37,069 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:39:38,917 epoch 80 - iter 5/50 - loss 2.10100920 - samples/sec: 86.66 - lr: 0.000488 2021-03-26 06:39:40,905 epoch 80 - iter 10/50 - loss 2.34779080 - samples/sec: 80.60 - lr: 0.000488 2021-03-26 06:39:42,961 epoch 80 - iter 15/50 - loss 2.31027282 - samples/sec: 77.91 - lr: 0.000488 2021-03-26 06:39:45,191 epoch 80 - iter 20/50 - loss 2.34272481 - samples/sec: 71.79 - lr: 0.000488 2021-03-26 06:39:47,142 epoch 80 - iter 25/50 - loss 2.42053922 - samples/sec: 82.11 - lr: 0.000488 2021-03-26 06:39:48,938 epoch 80 - iter 30/50 - loss 2.38158702 - samples/sec: 89.16 - lr: 0.000488 2021-03-26 06:39:50,971 epoch 80 - iter 35/50 - loss 2.35023596 - samples/sec: 78.75 - lr: 0.000488 2021-03-26 06:39:52,937 epoch 80 - iter 40/50 - loss 2.35125192 - samples/sec: 81.50 - lr: 0.000488 2021-03-26 06:39:54,859 epoch 80 - iter 45/50 - loss 2.32870909 - samples/sec: 83.29 - lr: 0.000488 2021-03-26 06:39:56,680 epoch 80 - iter 50/50 - loss 2.33931771 - samples/sec: 88.00 - lr: 0.000488 2021-03-26 06:39:56,681 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:39:56,681 EPOCH 80 done: loss 2.3393 - lr 0.0004883 2021-03-26 06:39:57,472 DEV : loss 6.979015350341797 - score 0.9085 2021-03-26 06:39:57,492 BAD EPOCHS (no improvement): 2 2021-03-26 06:39:57,493 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:39:59,386 epoch 81 - iter 5/50 - loss 2.24958613 - samples/sec: 84.62 - lr: 0.000488 2021-03-26 06:40:01,445 epoch 81 - iter 10/50 - loss 2.38275572 - samples/sec: 77.79 - lr: 0.000488 2021-03-26 06:40:03,325 epoch 81 - iter 15/50 - loss 2.29164292 - samples/sec: 85.19 - lr: 0.000488 2021-03-26 06:40:05,495 epoch 81 - iter 20/50 - loss 2.23068131 - samples/sec: 73.83 - lr: 0.000488 2021-03-26 06:40:07,548 epoch 81 - iter 25/50 - loss 2.28351954 - samples/sec: 78.03 - lr: 0.000488 2021-03-26 06:40:09,528 epoch 81 - iter 30/50 - loss 2.28917814 - samples/sec: 80.88 - lr: 0.000488 2021-03-26 06:40:11,488 epoch 81 - iter 35/50 - loss 2.27807065 - samples/sec: 81.74 - lr: 0.000488 2021-03-26 06:40:13,472 epoch 81 - iter 40/50 - loss 2.29849924 - samples/sec: 80.76 - lr: 0.000488 2021-03-26 06:40:15,586 epoch 81 - iter 45/50 - loss 2.32857788 - samples/sec: 75.76 - lr: 0.000488 2021-03-26 06:40:17,260 epoch 81 - iter 50/50 - loss 2.28520381 - samples/sec: 95.68 - lr: 0.000488 2021-03-26 06:40:17,261 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:40:17,261 EPOCH 81 done: loss 2.2852 - lr 0.0004883 2021-03-26 06:40:18,038 DEV : loss 6.977649211883545 - score 0.9085 2021-03-26 06:40:18,063 BAD EPOCHS (no improvement): 3 2021-03-26 06:40:18,064 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:40:20,200 epoch 82 - iter 5/50 - loss 2.10976236 - samples/sec: 74.98 - lr: 0.000488 2021-03-26 06:40:22,140 epoch 82 - iter 10/50 - loss 2.34995564 - samples/sec: 82.61 - lr: 0.000488 2021-03-26 06:40:24,016 epoch 82 - iter 15/50 - loss 2.16507681 - samples/sec: 85.34 - lr: 0.000488 2021-03-26 06:40:25,816 epoch 82 - iter 20/50 - loss 2.16873180 - samples/sec: 89.04 - lr: 0.000488 2021-03-26 06:40:27,697 epoch 82 - iter 25/50 - loss 2.16649104 - samples/sec: 85.15 - lr: 0.000488 2021-03-26 06:40:29,542 epoch 82 - iter 30/50 - loss 2.19325154 - samples/sec: 86.85 - lr: 0.000488 2021-03-26 06:40:31,484 epoch 82 - iter 35/50 - loss 2.21187508 - samples/sec: 82.44 - lr: 0.000488 2021-03-26 06:40:33,431 epoch 82 - iter 40/50 - loss 2.25956178 - samples/sec: 82.27 - lr: 0.000488 2021-03-26 06:40:35,325 epoch 82 - iter 45/50 - loss 2.28050901 - samples/sec: 84.53 - lr: 0.000488 2021-03-26 06:40:37,077 epoch 82 - iter 50/50 - loss 2.32061182 - samples/sec: 91.44 - lr: 0.000488 2021-03-26 06:40:37,078 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:40:37,078 EPOCH 82 done: loss 2.3206 - lr 0.0004883 2021-03-26 06:40:37,850 DEV : loss 6.976663112640381 - score 0.9081 2021-03-26 06:40:37,871 BAD EPOCHS (no improvement): 4 2021-03-26 06:40:37,871 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:40:39,804 epoch 83 - iter 5/50 - loss 2.51074567 - samples/sec: 82.87 - lr: 0.000244 2021-03-26 06:40:41,789 epoch 83 - iter 10/50 - loss 2.25548234 - samples/sec: 80.69 - lr: 0.000244 2021-03-26 06:40:43,924 epoch 83 - iter 15/50 - loss 2.50467954 - samples/sec: 75.03 - lr: 0.000244 2021-03-26 06:40:45,824 epoch 83 - iter 20/50 - loss 2.39716090 - samples/sec: 84.29 - lr: 0.000244 2021-03-26 06:40:47,599 epoch 83 - iter 25/50 - loss 2.38792141 - samples/sec: 90.23 - lr: 0.000244 2021-03-26 06:40:49,486 epoch 83 - iter 30/50 - loss 2.36569843 - samples/sec: 84.89 - lr: 0.000244 2021-03-26 06:40:51,382 epoch 83 - iter 35/50 - loss 2.37671602 - samples/sec: 84.45 - lr: 0.000244 2021-03-26 06:40:53,214 epoch 83 - iter 40/50 - loss 2.37701699 - samples/sec: 87.46 - lr: 0.000244 2021-03-26 06:40:55,165 epoch 83 - iter 45/50 - loss 2.35641030 - samples/sec: 82.05 - lr: 0.000244 2021-03-26 06:40:57,019 epoch 83 - iter 50/50 - loss 2.40662164 - samples/sec: 86.38 - lr: 0.000244 2021-03-26 06:40:57,020 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:40:57,020 EPOCH 83 done: loss 2.4066 - lr 0.0002441 2021-03-26 06:40:57,859 DEV : loss 6.976783752441406 - score 0.9081 2021-03-26 06:40:57,882 BAD EPOCHS (no improvement): 1 2021-03-26 06:40:57,883 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:40:59,880 epoch 84 - iter 5/50 - loss 2.11736424 - samples/sec: 80.18 - lr: 0.000244 2021-03-26 06:41:01,725 epoch 84 - iter 10/50 - loss 2.27809557 - samples/sec: 86.82 - lr: 0.000244 2021-03-26 06:41:03,986 epoch 84 - iter 15/50 - loss 2.30093797 - samples/sec: 70.82 - lr: 0.000244 2021-03-26 06:41:06,068 epoch 84 - iter 20/50 - loss 2.30980914 - samples/sec: 76.93 - lr: 0.000244 2021-03-26 06:41:07,898 epoch 84 - iter 25/50 - loss 2.37576721 - samples/sec: 87.50 - lr: 0.000244 2021-03-26 06:41:09,867 epoch 84 - iter 30/50 - loss 2.36038343 - samples/sec: 81.32 - lr: 0.000244 2021-03-26 06:41:11,715 epoch 84 - iter 35/50 - loss 2.37259437 - samples/sec: 86.69 - lr: 0.000244 2021-03-26 06:41:13,635 epoch 84 - iter 40/50 - loss 2.32505912 - samples/sec: 83.41 - lr: 0.000244 2021-03-26 06:41:15,493 epoch 84 - iter 45/50 - loss 2.35598942 - samples/sec: 86.21 - lr: 0.000244 2021-03-26 06:41:17,527 epoch 84 - iter 50/50 - loss 2.36311971 - samples/sec: 78.72 - lr: 0.000244 2021-03-26 06:41:17,528 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:41:17,528 EPOCH 84 done: loss 2.3631 - lr 0.0002441 2021-03-26 06:41:18,385 DEV : loss 6.976834297180176 - score 0.9081 2021-03-26 06:41:18,419 BAD EPOCHS (no improvement): 2 2021-03-26 06:41:18,420 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:41:20,424 epoch 85 - iter 5/50 - loss 2.38491588 - samples/sec: 79.94 - lr: 0.000244 2021-03-26 06:41:22,145 epoch 85 - iter 10/50 - loss 2.31585956 - samples/sec: 93.08 - lr: 0.000244 2021-03-26 06:41:24,066 epoch 85 - iter 15/50 - loss 2.32960722 - samples/sec: 83.37 - lr: 0.000244 2021-03-26 06:41:25,888 epoch 85 - iter 20/50 - loss 2.32371256 - samples/sec: 87.90 - lr: 0.000244 2021-03-26 06:41:27,846 epoch 85 - iter 25/50 - loss 2.27565221 - samples/sec: 81.78 - lr: 0.000244 2021-03-26 06:41:29,649 epoch 85 - iter 30/50 - loss 2.28047132 - samples/sec: 88.82 - lr: 0.000244 2021-03-26 06:41:31,515 epoch 85 - iter 35/50 - loss 2.26845495 - samples/sec: 85.83 - lr: 0.000244 2021-03-26 06:41:33,374 epoch 85 - iter 40/50 - loss 2.25583084 - samples/sec: 86.16 - lr: 0.000244 2021-03-26 06:41:35,260 epoch 85 - iter 45/50 - loss 2.22698094 - samples/sec: 84.92 - lr: 0.000244 2021-03-26 06:41:36,857 epoch 85 - iter 50/50 - loss 2.19129711 - samples/sec: 100.31 - lr: 0.000244 2021-03-26 06:41:36,857 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:41:36,858 EPOCH 85 done: loss 2.1913 - lr 0.0002441 2021-03-26 06:41:37,613 DEV : loss 6.975969314575195 - score 0.9081 2021-03-26 06:41:37,638 BAD EPOCHS (no improvement): 3 2021-03-26 06:41:37,639 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:41:39,553 epoch 86 - iter 5/50 - loss 2.32959135 - samples/sec: 83.68 - lr: 0.000244 2021-03-26 06:41:41,499 epoch 86 - iter 10/50 - loss 2.47228733 - samples/sec: 82.29 - lr: 0.000244 2021-03-26 06:41:43,278 epoch 86 - iter 15/50 - loss 2.50017960 - samples/sec: 90.06 - lr: 0.000244 2021-03-26 06:41:45,235 epoch 86 - iter 20/50 - loss 2.46814180 - samples/sec: 81.81 - lr: 0.000244 2021-03-26 06:41:47,290 epoch 86 - iter 25/50 - loss 2.41104710 - samples/sec: 77.90 - lr: 0.000244 2021-03-26 06:41:49,184 epoch 86 - iter 30/50 - loss 2.36386334 - samples/sec: 84.59 - lr: 0.000244 2021-03-26 06:41:50,852 epoch 86 - iter 35/50 - loss 2.33780671 - samples/sec: 96.02 - lr: 0.000244 2021-03-26 06:41:52,832 epoch 86 - iter 40/50 - loss 2.33240881 - samples/sec: 80.89 - lr: 0.000244 2021-03-26 06:41:54,827 epoch 86 - iter 45/50 - loss 2.32977507 - samples/sec: 80.30 - lr: 0.000244 2021-03-26 06:41:56,643 epoch 86 - iter 50/50 - loss 2.30426145 - samples/sec: 88.19 - lr: 0.000244 2021-03-26 06:41:56,643 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:41:56,644 EPOCH 86 done: loss 2.3043 - lr 0.0002441 2021-03-26 06:41:57,442 DEV : loss 6.975675106048584 - score 0.9081 2021-03-26 06:41:57,467 BAD EPOCHS (no improvement): 4 2021-03-26 06:41:57,467 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:41:59,463 epoch 87 - iter 5/50 - loss 2.02026136 - samples/sec: 80.25 - lr: 0.000122 2021-03-26 06:42:01,359 epoch 87 - iter 10/50 - loss 2.17304178 - samples/sec: 84.49 - lr: 0.000122 2021-03-26 06:42:03,282 epoch 87 - iter 15/50 - loss 2.38836434 - samples/sec: 83.31 - lr: 0.000122 2021-03-26 06:42:05,124 epoch 87 - iter 20/50 - loss 2.35105944 - samples/sec: 87.13 - lr: 0.000122 2021-03-26 06:42:07,194 epoch 87 - iter 25/50 - loss 2.37925279 - samples/sec: 77.36 - lr: 0.000122 2021-03-26 06:42:09,259 epoch 87 - iter 30/50 - loss 2.36004749 - samples/sec: 77.55 - lr: 0.000122 2021-03-26 06:42:11,066 epoch 87 - iter 35/50 - loss 2.35606966 - samples/sec: 88.64 - lr: 0.000122 2021-03-26 06:42:13,291 epoch 87 - iter 40/50 - loss 2.37431050 - samples/sec: 71.97 - lr: 0.000122 2021-03-26 06:42:15,124 epoch 87 - iter 45/50 - loss 2.33771084 - samples/sec: 87.39 - lr: 0.000122 2021-03-26 06:42:16,850 epoch 87 - iter 50/50 - loss 2.25152152 - samples/sec: 92.81 - lr: 0.000122 2021-03-26 06:42:16,851 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:42:16,851 EPOCH 87 done: loss 2.2515 - lr 0.0001221 2021-03-26 06:42:17,616 DEV : loss 6.9754133224487305 - score 0.9081 2021-03-26 06:42:17,641 BAD EPOCHS (no improvement): 1 2021-03-26 06:42:17,642 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:42:19,618 epoch 88 - iter 5/50 - loss 2.34914608 - samples/sec: 81.03 - lr: 0.000122 2021-03-26 06:42:21,605 epoch 88 - iter 10/50 - loss 2.30592394 - samples/sec: 80.62 - lr: 0.000122 2021-03-26 06:42:23,575 epoch 88 - iter 15/50 - loss 2.33506842 - samples/sec: 81.27 - lr: 0.000122 2021-03-26 06:42:25,397 epoch 88 - iter 20/50 - loss 2.24439114 - samples/sec: 87.89 - lr: 0.000122 2021-03-26 06:42:27,390 epoch 88 - iter 25/50 - loss 2.22855584 - samples/sec: 80.34 - lr: 0.000122 2021-03-26 06:42:29,345 epoch 88 - iter 30/50 - loss 2.26887201 - samples/sec: 81.92 - lr: 0.000122 2021-03-26 06:42:31,099 epoch 88 - iter 35/50 - loss 2.28389955 - samples/sec: 91.31 - lr: 0.000122 2021-03-26 06:42:33,054 epoch 88 - iter 40/50 - loss 2.30694303 - samples/sec: 81.91 - lr: 0.000122 2021-03-26 06:42:34,922 epoch 88 - iter 45/50 - loss 2.35164838 - samples/sec: 85.74 - lr: 0.000122 2021-03-26 06:42:36,791 epoch 88 - iter 50/50 - loss 2.29030984 - samples/sec: 85.72 - lr: 0.000122 2021-03-26 06:42:36,791 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:42:36,792 EPOCH 88 done: loss 2.2903 - lr 0.0001221 2021-03-26 06:42:37,578 DEV : loss 6.975350856781006 - score 0.9081 2021-03-26 06:42:37,602 BAD EPOCHS (no improvement): 2 2021-03-26 06:42:37,603 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:42:39,785 epoch 89 - iter 5/50 - loss 2.01891148 - samples/sec: 73.41 - lr: 0.000122 2021-03-26 06:42:41,805 epoch 89 - iter 10/50 - loss 2.10284538 - samples/sec: 79.30 - lr: 0.000122 2021-03-26 06:42:43,746 epoch 89 - iter 15/50 - loss 2.15385324 - samples/sec: 82.51 - lr: 0.000122 2021-03-26 06:42:45,688 epoch 89 - iter 20/50 - loss 2.26479766 - samples/sec: 82.47 - lr: 0.000122 2021-03-26 06:42:47,868 epoch 89 - iter 25/50 - loss 2.41565577 - samples/sec: 73.47 - lr: 0.000122 2021-03-26 06:42:49,782 epoch 89 - iter 30/50 - loss 2.43289930 - samples/sec: 83.67 - lr: 0.000122 2021-03-26 06:42:51,658 epoch 89 - iter 35/50 - loss 2.38985022 - samples/sec: 85.36 - lr: 0.000122 2021-03-26 06:42:53,928 epoch 89 - iter 40/50 - loss 2.40961840 - samples/sec: 70.52 - lr: 0.000122 2021-03-26 06:42:55,765 epoch 89 - iter 45/50 - loss 2.40256732 - samples/sec: 87.19 - lr: 0.000122 2021-03-26 06:42:57,628 epoch 89 - iter 50/50 - loss 2.43918465 - samples/sec: 86.00 - lr: 0.000122 2021-03-26 06:42:57,628 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:42:57,629 EPOCH 89 done: loss 2.4392 - lr 0.0001221 2021-03-26 06:42:58,423 DEV : loss 6.9754157066345215 - score 0.9081 2021-03-26 06:42:58,448 BAD EPOCHS (no improvement): 3 2021-03-26 06:42:58,449 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:00,420 epoch 90 - iter 5/50 - loss 2.78787105 - samples/sec: 81.24 - lr: 0.000122 2021-03-26 06:43:02,331 epoch 90 - iter 10/50 - loss 2.66487483 - samples/sec: 83.84 - lr: 0.000122 2021-03-26 06:43:04,225 epoch 90 - iter 15/50 - loss 2.54435790 - samples/sec: 84.53 - lr: 0.000122 2021-03-26 06:43:06,276 epoch 90 - iter 20/50 - loss 2.62745005 - samples/sec: 78.09 - lr: 0.000122 2021-03-26 06:43:08,139 epoch 90 - iter 25/50 - loss 2.48686338 - samples/sec: 85.97 - lr: 0.000122 2021-03-26 06:43:10,136 epoch 90 - iter 30/50 - loss 2.46287971 - samples/sec: 80.18 - lr: 0.000122 2021-03-26 06:43:12,156 epoch 90 - iter 35/50 - loss 2.49387327 - samples/sec: 79.29 - lr: 0.000122 2021-03-26 06:43:13,992 epoch 90 - iter 40/50 - loss 2.46719781 - samples/sec: 87.22 - lr: 0.000122 2021-03-26 06:43:15,985 epoch 90 - iter 45/50 - loss 2.43868358 - samples/sec: 80.37 - lr: 0.000122 2021-03-26 06:43:17,851 epoch 90 - iter 50/50 - loss 2.48019275 - samples/sec: 85.82 - lr: 0.000122 2021-03-26 06:43:17,851 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:17,852 EPOCH 90 done: loss 2.4802 - lr 0.0001221 2021-03-26 06:43:18,601 DEV : loss 6.975543022155762 - score 0.9081 2021-03-26 06:43:18,627 BAD EPOCHS (no improvement): 4 2021-03-26 06:43:18,627 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:18,628 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:18,628 learning rate too small - quitting training! 2021-03-26 06:43:18,628 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:27,954 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:27,955 Testing using best model ... 2021-03-26 06:43:27,956 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260608/best-model.pt 2021-03-26 06:43:35,839 0.9172 2021-03-26 06:43:35,840 Results: - F-score (micro): 0.9134 - F-score (macro): 0.5333 - Accuracy (incl. no class): 0.9172 By class: precision recall f1-score support INTJ 1.0000 1.0000 1.0000 16 CCONJ 0.9892 1.0000 0.9946 92 ADP 0.9640 0.9640 0.9640 111 PRON 0.9945 0.9837 0.9891 184 NOUN 0.9180 0.9527 0.9350 423 PART 0.9513 0.9513 0.9513 226 ADJ 0.8559 0.8347 0.8452 121 ADV 0.9189 0.8793 0.8987 116 PROPN 0.9500 0.7917 0.8636 24 NUM 1.0000 0.8889 0.9412 27 VERB 0.9529 0.9529 0.9529 170 DET 0.9583 1.0000 0.9787 46 PUNCT 1.0000 1.0000 1.0000 36 SCONJ 0.9143 0.8889 0.9014 36 AUX 0.9062 0.9355 0.9206 31 PROG_PART+V 0.7812 0.9615 0.8621 26 PROG_PART+V+PRON 0.7826 0.7500 0.7660 24 PROG_PART+V+PREP+PRON 0.0000 0.0000 0.0000 3 V+PRON 0.7778 0.8596 0.8167 57 PART+PRON 1.0000 0.8333 0.9091 18 DET+NOUN 0.9200 1.0000 0.9583 69 CONJ+DET+NOUN 0.5556 1.0000 0.7143 5 EOS 1.0000 1.0000 1.0000 70 NOUN+CASE 1.0000 0.2500 0.4000 4 ADJ+CASE 0.0000 1.0000 0.0000 0 DET+NOUN+NSUFF 0.9310 0.9000 0.9153 30 PREP+DET+NOUN 1.0000 1.0000 1.0000 10 V 0.9091 0.8974 0.9032 78 PREP 0.9853 0.9571 0.9710 70 V+PRON+PRON 0.7500 0.6000 0.6667 5 CONJ 0.9231 1.0000 0.9600 24 PREP+V 1.0000 0.5000 0.6667 2 PREP+PRON 0.8750 0.9545 0.9130 22 PUNC 0.9933 1.0000 0.9967 149 DET+ADJ 1.0000 0.7500 0.8571 16 NOUN+NSUFF 0.8158 0.8857 0.8493 35 CONJ+PART 0.7222 0.9286 0.8125 14 NOUN+PRON 0.8571 0.8571 0.8571 56 PREP+NOUN 0.9286 0.8667 0.8966 15 PROG_PART 1.0000 1.0000 1.0000 1 PREP+NOUN+PRON 1.0000 0.2500 0.4000 4 PREP+NOUN+NSUFF+PRON 0.0000 0.0000 0.0000 1 HASH 1.0000 1.0000 1.0000 13 CONJ+NOUN 0.8333 0.7692 0.8000 13 CONJ+V+PRON 0.6667 0.5000 0.5714 4 CONJ+NOUN+PRON 0.0000 0.0000 0.0000 4 MENTION 1.0000 1.0000 1.0000 32 NOUN+NSUFF+PRON 0.5385 0.5833 0.5600 12 CONJ+PRON 1.0000 1.0000 1.0000 5 V+PREP+PRON 0.6667 0.3636 0.4706 11 FUT_PART 1.0000 1.0000 1.0000 3 DET+NUM 0.0000 1.0000 0.0000 0 V+PRON+PREP+PRON 0.5000 0.3333 0.4000 3 EMOT 1.0000 0.9545 0.9767 22 PREP+PART+PRON 1.0000 1.0000 1.0000 3 ADJ+NSUFF 0.8788 0.8788 0.8788 33 PREP+DET+ADJ 1.0000 1.0000 1.0000 1 CONJ+V 0.4286 1.0000 0.6000 3 FUT_PART+V 0.8000 0.8000 0.8000 5 PART+NOUN 0.7143 0.5556 0.6250 9 PART+ADJ+PRON+PRON 0.0000 1.0000 0.0000 0 CONJ+DET+ADJ 1.0000 0.0000 0.0000 3 PREP+DET+NOUN+NSUFF 0.6667 1.0000 0.8000 2 ADJ+PREP+PRON 1.0000 1.0000 1.0000 1 FOREIGN 1.0000 1.0000 1.0000 3 PROG_PART+V+PRON+PRON 0.0000 0.0000 0.0000 2 URL 1.0000 1.0000 1.0000 3 CONJ+PROG_PART+V 0.7500 1.0000 0.8571 3 CONJ+ADJ 0.3333 0.2500 0.2857 4 PREP+NOUN+NSUFF 1.0000 0.5000 0.6667 4 CONJ+ADV 0.0000 1.0000 0.0000 0 PREP+DET+PART 1.0000 0.0000 0.0000 1 PART+NOUN+PRON 1.0000 0.3333 0.5000 3 CONJ+PREP+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 2 CONJ+PART+V 0.0000 1.0000 0.0000 0 DET+ADJ+NSUFF 0.6667 0.6667 0.6667 3 PART+V+PRON 0.5000 0.5000 0.5000 2 CONJ+ADJ+NSUFF+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+PART+PREP 0.0000 1.0000 0.0000 0 NOUN+PREP+PRON 1.0000 0.0000 0.0000 1 ADJ+PRON 0.4286 0.6000 0.5000 5 CONJ+FUT_PART+V+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+PROG_PART+V+PRON 0.0000 1.0000 0.0000 0 PART+PREP 1.0000 0.0000 0.0000 1 PART+ADV 0.0000 1.0000 0.0000 0 PROG_PART+V+PREP 0.0000 1.0000 0.0000 0 PART+V 0.0000 0.0000 0.0000 2 V+NOUN 1.0000 0.0000 0.0000 2 ADJ+NSUFF+PRON 0.0000 1.0000 0.0000 0 PRON+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 PRON+DET+NOUN 0.0000 1.0000 0.0000 0 CONJ+PREP 1.0000 1.0000 1.0000 1 FUT_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 FUT_PART+V+PRON 0.6667 1.0000 0.8000 2 PART+PROG_PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 CONJ+PART+V+NEG_PART 1.0000 0.0000 0.0000 1 PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+V+NEG_PART 0.3333 1.0000 0.5000 1 V+NEG_PART 1.0000 0.0000 0.0000 1 CONJ+FUT_PART+V+PRON 1.0000 0.0000 0.0000 1 PART+NSUFF 0.0000 0.0000 0.0000 1 PART+PREP+NEG_PART 1.0000 1.0000 1.0000 4 PART+V+PRON+NEG_PART 0.5000 0.6667 0.5714 3 NOUN+CASE+PRON 0.0000 1.0000 0.0000 0 CONJ+PART+PREP+NEG_PART 1.0000 0.0000 0.0000 2 PART+PART 1.0000 0.0000 0.0000 1 PART+PROG_PART+V+NEG_PART 1.0000 0.3333 0.5000 3 PROG_PART+V+PRON+PREP+PRON 1.0000 0.0000 0.0000 1 NOUN+CASE+NSUFF+NOUN+PRON 1.0000 0.0000 0.0000 1 CONJ+NOUN+NSUFF+PRON 0.0000 1.0000 0.0000 0 CONJ+NOUN+NSUFF 0.6667 1.0000 0.8000 2 NUM+NSUFF 1.0000 1.0000 1.0000 1 ADV+NSUFF 1.0000 1.0000 1.0000 1 PART+V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 CONJ+V+PRON+PREP+PRON 0.0000 1.0000 0.0000 0 micro avg 0.9134 0.9134 0.9134 2724 macro avg 0.7269 0.6691 0.5333 2724 weighted avg 0.9221 0.9134 0.9105 2724 2021-03-26 06:43:35,840 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:35,841 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:41,464 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC 2021-03-26 06:43:41,464 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu 2021-03-26 06:43:41,465 Dev: None 2021-03-26 06:43:41,465 Test: None 2021-03-26 06:43:41,742 Reading data from ../../Datasets_adhoc/UD_MADAR 2021-03-26 06:43:41,743 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu 2021-03-26 06:43:41,743 Dev: None 2021-03-26 06:43:41,743 Test: None 2021-03-26 06:43:41,775 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 06:43:41,776 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt 2021-03-26 06:43:41,776 Dev: None 2021-03-26 06:43:41,777 Test: None 2021-03-26 06:43:41,956 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 06:43:41,956 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt 2021-03-26 06:43:41,957 Dev: None 2021-03-26 06:43:41,957 Test: None 2021-03-26 06:43:42,128 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 06:43:42,128 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt 2021-03-26 06:43:42,129 Dev: None 2021-03-26 06:43:42,129 Test: None 2021-03-26 06:43:42,284 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 06:43:42,284 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt 2021-03-26 06:43:42,285 Dev: None 2021-03-26 06:43:42,285 Test: None 2021-03-26 06:43:42,444 Filtering long sentences 2021-03-26 06:43:42,480 MultiCorpus: 1575 train + 176 dev + 193 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences 2021-03-26 06:43:42,903 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:42,904 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): WordEmbeddings('ar') (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) (list_embedding_2): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=206, bias=True) (beta): 1.0 (weights): None (weight_tensor) None )" 2021-03-26 06:43:42,904 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:42,904 Corpus: "MultiCorpus: 1575 train + 176 dev + 193 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" 2021-03-26 06:43:42,905 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:42,905 Parameters: 2021-03-26 06:43:42,905 - learning_rate: "0.5" 2021-03-26 06:43:42,905 - mini_batch_size: "32" 2021-03-26 06:43:42,906 - patience: "3" 2021-03-26 06:43:42,906 - anneal_factor: "0.5" 2021-03-26 06:43:42,906 - max_epochs: "150" 2021-03-26 06:43:42,906 - shuffle: "True" 2021-03-26 06:43:42,907 - train_with_dev: "False" 2021-03-26 06:43:42,907 - batch_growth_annealing: "False" 2021-03-26 06:43:42,907 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:42,908 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260643" 2021-03-26 06:43:42,908 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:42,908 Device: cuda:0 2021-03-26 06:43:42,908 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:42,909 Embeddings storage mode: cpu 2021-03-26 06:43:42,910 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:43:45,750 epoch 1 - iter 5/50 - loss 75.85967865 - samples/sec: 56.37 - lr: 0.500000 2021-03-26 06:43:48,460 epoch 1 - iter 10/50 - loss 72.91885376 - samples/sec: 59.10 - lr: 0.500000 2021-03-26 06:43:50,928 epoch 1 - iter 15/50 - loss 68.27410405 - samples/sec: 64.85 - lr: 0.500000 2021-03-26 06:43:53,456 epoch 1 - iter 20/50 - loss 66.77132072 - samples/sec: 63.35 - lr: 0.500000 2021-03-26 06:43:55,893 epoch 1 - iter 25/50 - loss 62.94671341 - samples/sec: 65.69 - lr: 0.500000 2021-03-26 06:43:58,295 epoch 1 - iter 30/50 - loss 59.68897349 - samples/sec: 66.68 - lr: 0.500000 2021-03-26 06:44:00,773 epoch 1 - iter 35/50 - loss 56.34382128 - samples/sec: 64.62 - lr: 0.500000 2021-03-26 06:44:03,237 epoch 1 - iter 40/50 - loss 54.21474304 - samples/sec: 64.97 - lr: 0.500000 2021-03-26 06:44:05,549 epoch 1 - iter 45/50 - loss 51.69305013 - samples/sec: 69.23 - lr: 0.500000 2021-03-26 06:44:07,771 epoch 1 - iter 50/50 - loss 49.45876930 - samples/sec: 72.08 - lr: 0.500000 2021-03-26 06:44:07,771 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:44:07,771 EPOCH 1 done: loss 49.4588 - lr 0.5000000 2021-03-26 06:44:09,122 DEV : loss 28.84688377380371 - score 0.5287 2021-03-26 06:44:09,151 BAD EPOCHS (no improvement): 0 2021-03-26 06:44:18,447 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:44:20,548 epoch 2 - iter 5/50 - loss 29.69394226 - samples/sec: 76.23 - lr: 0.500000 2021-03-26 06:44:22,528 epoch 2 - iter 10/50 - loss 29.02742157 - samples/sec: 80.86 - lr: 0.500000 2021-03-26 06:44:24,434 epoch 2 - iter 15/50 - loss 28.63661626 - samples/sec: 84.02 - lr: 0.500000 2021-03-26 06:44:26,392 epoch 2 - iter 20/50 - loss 28.82985716 - samples/sec: 81.81 - lr: 0.500000 2021-03-26 06:44:28,342 epoch 2 - iter 25/50 - loss 27.57712044 - samples/sec: 82.14 - lr: 0.500000 2021-03-26 06:44:30,190 epoch 2 - iter 30/50 - loss 26.53857161 - samples/sec: 86.65 - lr: 0.500000 2021-03-26 06:44:32,111 epoch 2 - iter 35/50 - loss 25.69812393 - samples/sec: 83.38 - lr: 0.500000 2021-03-26 06:44:34,036 epoch 2 - iter 40/50 - loss 25.05725961 - samples/sec: 83.17 - lr: 0.500000 2021-03-26 06:44:35,814 epoch 2 - iter 45/50 - loss 24.72802349 - samples/sec: 90.09 - lr: 0.500000 2021-03-26 06:44:37,773 epoch 2 - iter 50/50 - loss 24.46823097 - samples/sec: 81.77 - lr: 0.500000 2021-03-26 06:44:37,774 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:44:37,774 EPOCH 2 done: loss 24.4682 - lr 0.5000000 2021-03-26 06:44:38,574 DEV : loss 18.737186431884766 - score 0.6859 2021-03-26 06:44:38,597 BAD EPOCHS (no improvement): 0 2021-03-26 06:44:48,335 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:44:50,250 epoch 3 - iter 5/50 - loss 20.64498234 - samples/sec: 83.67 - lr: 0.500000 2021-03-26 06:44:52,270 epoch 3 - iter 10/50 - loss 19.51538610 - samples/sec: 79.28 - lr: 0.500000 2021-03-26 06:44:54,281 epoch 3 - iter 15/50 - loss 19.19895941 - samples/sec: 79.62 - lr: 0.500000 2021-03-26 06:44:56,226 epoch 3 - iter 20/50 - loss 18.74911275 - samples/sec: 82.34 - lr: 0.500000 2021-03-26 06:44:58,202 epoch 3 - iter 25/50 - loss 18.68923565 - samples/sec: 81.09 - lr: 0.500000 2021-03-26 06:45:00,559 epoch 3 - iter 30/50 - loss 18.81346890 - samples/sec: 67.93 - lr: 0.500000 2021-03-26 06:45:02,525 epoch 3 - iter 35/50 - loss 18.32439662 - samples/sec: 81.48 - lr: 0.500000 2021-03-26 06:45:04,334 epoch 3 - iter 40/50 - loss 17.78430147 - samples/sec: 88.53 - lr: 0.500000 2021-03-26 06:45:06,254 epoch 3 - iter 45/50 - loss 17.57249622 - samples/sec: 83.38 - lr: 0.500000 2021-03-26 06:45:08,092 epoch 3 - iter 50/50 - loss 17.63071257 - samples/sec: 87.17 - lr: 0.500000 2021-03-26 06:45:08,092 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:45:08,093 EPOCH 3 done: loss 17.6307 - lr 0.5000000 2021-03-26 06:45:08,864 DEV : loss 12.816518783569336 - score 0.7775 2021-03-26 06:45:08,882 BAD EPOCHS (no improvement): 0 2021-03-26 06:45:18,098 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:45:19,934 epoch 4 - iter 5/50 - loss 13.45718422 - samples/sec: 87.25 - lr: 0.500000 2021-03-26 06:45:21,852 epoch 4 - iter 10/50 - loss 13.95723553 - samples/sec: 83.54 - lr: 0.500000 2021-03-26 06:45:23,589 epoch 4 - iter 15/50 - loss 14.16512273 - samples/sec: 92.21 - lr: 0.500000 2021-03-26 06:45:25,528 epoch 4 - iter 20/50 - loss 14.24711242 - samples/sec: 82.61 - lr: 0.500000 2021-03-26 06:45:27,528 epoch 4 - iter 25/50 - loss 14.34868843 - samples/sec: 80.08 - lr: 0.500000 2021-03-26 06:45:29,367 epoch 4 - iter 30/50 - loss 14.10320505 - samples/sec: 87.10 - lr: 0.500000 2021-03-26 06:45:31,207 epoch 4 - iter 35/50 - loss 14.21685265 - samples/sec: 87.03 - lr: 0.500000 2021-03-26 06:45:33,402 epoch 4 - iter 40/50 - loss 14.32871656 - samples/sec: 72.95 - lr: 0.500000 2021-03-26 06:45:35,550 epoch 4 - iter 45/50 - loss 14.40581788 - samples/sec: 74.58 - lr: 0.500000 2021-03-26 06:45:37,423 epoch 4 - iter 50/50 - loss 14.52564522 - samples/sec: 85.52 - lr: 0.500000 2021-03-26 06:45:37,424 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:45:37,425 EPOCH 4 done: loss 14.5256 - lr 0.5000000 2021-03-26 06:45:38,232 DEV : loss 10.77255630493164 - score 0.8089 2021-03-26 06:45:38,256 BAD EPOCHS (no improvement): 0 2021-03-26 06:45:48,051 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:45:50,089 epoch 5 - iter 5/50 - loss 13.15967064 - samples/sec: 78.60 - lr: 0.500000 2021-03-26 06:45:51,982 epoch 5 - iter 10/50 - loss 12.23003578 - samples/sec: 84.64 - lr: 0.500000 2021-03-26 06:45:53,973 epoch 5 - iter 15/50 - loss 11.92497025 - samples/sec: 80.43 - lr: 0.500000 2021-03-26 06:45:55,948 epoch 5 - iter 20/50 - loss 11.61746993 - samples/sec: 81.11 - lr: 0.500000 2021-03-26 06:45:57,947 epoch 5 - iter 25/50 - loss 11.85876007 - samples/sec: 80.12 - lr: 0.500000 2021-03-26 06:45:59,818 epoch 5 - iter 30/50 - loss 11.88168329 - samples/sec: 85.57 - lr: 0.500000 2021-03-26 06:46:01,790 epoch 5 - iter 35/50 - loss 12.09282556 - samples/sec: 81.23 - lr: 0.500000 2021-03-26 06:46:03,808 epoch 5 - iter 40/50 - loss 12.18096049 - samples/sec: 79.51 - lr: 0.500000 2021-03-26 06:46:05,740 epoch 5 - iter 45/50 - loss 12.18161873 - samples/sec: 82.88 - lr: 0.500000 2021-03-26 06:46:07,736 epoch 5 - iter 50/50 - loss 12.94767559 - samples/sec: 80.23 - lr: 0.500000 2021-03-26 06:46:07,737 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:46:07,737 EPOCH 5 done: loss 12.9477 - lr 0.5000000 2021-03-26 06:46:08,529 DEV : loss 10.317449569702148 - score 0.8217 2021-03-26 06:46:08,554 BAD EPOCHS (no improvement): 0 2021-03-26 06:46:18,018 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:46:19,983 epoch 6 - iter 5/50 - loss 12.50278988 - samples/sec: 81.55 - lr: 0.500000 2021-03-26 06:46:22,231 epoch 6 - iter 10/50 - loss 11.03866415 - samples/sec: 71.24 - lr: 0.500000 2021-03-26 06:46:24,657 epoch 6 - iter 15/50 - loss 10.50042890 - samples/sec: 66.02 - lr: 0.500000 2021-03-26 06:46:26,706 epoch 6 - iter 20/50 - loss 10.49348035 - samples/sec: 78.15 - lr: 0.500000 2021-03-26 06:46:28,699 epoch 6 - iter 25/50 - loss 10.91942196 - samples/sec: 80.34 - lr: 0.500000 2021-03-26 06:46:30,795 epoch 6 - iter 30/50 - loss 11.12543367 - samples/sec: 76.40 - lr: 0.500000 2021-03-26 06:46:32,643 epoch 6 - iter 35/50 - loss 11.14839009 - samples/sec: 86.70 - lr: 0.500000 2021-03-26 06:46:34,602 epoch 6 - iter 40/50 - loss 11.11419473 - samples/sec: 81.76 - lr: 0.500000 2021-03-26 06:46:36,637 epoch 6 - iter 45/50 - loss 10.98261443 - samples/sec: 78.68 - lr: 0.500000 2021-03-26 06:46:38,625 epoch 6 - iter 50/50 - loss 11.21462681 - samples/sec: 80.55 - lr: 0.500000 2021-03-26 06:46:38,626 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:46:38,626 EPOCH 6 done: loss 11.2146 - lr 0.5000000 2021-03-26 06:46:39,400 DEV : loss 8.676898002624512 - score 0.8514 2021-03-26 06:46:39,425 BAD EPOCHS (no improvement): 0 2021-03-26 06:46:48,983 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:46:51,093 epoch 7 - iter 5/50 - loss 8.98820639 - samples/sec: 75.93 - lr: 0.500000 2021-03-26 06:46:53,091 epoch 7 - iter 10/50 - loss 9.30682864 - samples/sec: 80.12 - lr: 0.500000 2021-03-26 06:46:55,331 epoch 7 - iter 15/50 - loss 9.97786722 - samples/sec: 71.49 - lr: 0.500000 2021-03-26 06:46:57,801 epoch 7 - iter 20/50 - loss 10.29002182 - samples/sec: 64.82 - lr: 0.500000 2021-03-26 06:47:00,198 epoch 7 - iter 25/50 - loss 10.03100792 - samples/sec: 66.82 - lr: 0.500000 2021-03-26 06:47:02,050 epoch 7 - iter 30/50 - loss 10.03738489 - samples/sec: 86.44 - lr: 0.500000 2021-03-26 06:47:04,092 epoch 7 - iter 35/50 - loss 9.97502296 - samples/sec: 78.43 - lr: 0.500000 2021-03-26 06:47:06,103 epoch 7 - iter 40/50 - loss 9.96401833 - samples/sec: 79.67 - lr: 0.500000 2021-03-26 06:47:08,005 epoch 7 - iter 45/50 - loss 10.01933646 - samples/sec: 84.21 - lr: 0.500000 2021-03-26 06:47:10,007 epoch 7 - iter 50/50 - loss 9.98463810 - samples/sec: 80.01 - lr: 0.500000 2021-03-26 06:47:10,008 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:47:10,008 EPOCH 7 done: loss 9.9846 - lr 0.5000000 2021-03-26 06:47:10,791 DEV : loss 8.427385330200195 - score 0.8597 2021-03-26 06:47:10,815 BAD EPOCHS (no improvement): 0 2021-03-26 06:47:20,332 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:47:22,127 epoch 8 - iter 5/50 - loss 8.90783539 - samples/sec: 89.29 - lr: 0.500000 2021-03-26 06:47:23,991 epoch 8 - iter 10/50 - loss 9.33205786 - samples/sec: 85.93 - lr: 0.500000 2021-03-26 06:47:25,895 epoch 8 - iter 15/50 - loss 9.32208843 - samples/sec: 84.11 - lr: 0.500000 2021-03-26 06:47:27,863 epoch 8 - iter 20/50 - loss 9.05227771 - samples/sec: 81.33 - lr: 0.500000 2021-03-26 06:47:29,891 epoch 8 - iter 25/50 - loss 9.22336864 - samples/sec: 78.98 - lr: 0.500000 2021-03-26 06:47:31,740 epoch 8 - iter 30/50 - loss 9.01553138 - samples/sec: 86.64 - lr: 0.500000 2021-03-26 06:47:33,679 epoch 8 - iter 35/50 - loss 9.00514094 - samples/sec: 82.61 - lr: 0.500000 2021-03-26 06:47:35,753 epoch 8 - iter 40/50 - loss 9.07460406 - samples/sec: 77.23 - lr: 0.500000 2021-03-26 06:47:37,789 epoch 8 - iter 45/50 - loss 9.12630397 - samples/sec: 78.63 - lr: 0.500000 2021-03-26 06:47:39,659 epoch 8 - iter 50/50 - loss 9.14114470 - samples/sec: 85.65 - lr: 0.500000 2021-03-26 06:47:39,660 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:47:39,660 EPOCH 8 done: loss 9.1411 - lr 0.5000000 2021-03-26 06:47:40,404 DEV : loss 7.99708890914917 - score 0.8648 2021-03-26 06:47:40,429 BAD EPOCHS (no improvement): 0 2021-03-26 06:47:50,043 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:47:52,084 epoch 9 - iter 5/50 - loss 9.45290432 - samples/sec: 78.48 - lr: 0.500000 2021-03-26 06:47:54,091 epoch 9 - iter 10/50 - loss 8.74829707 - samples/sec: 79.82 - lr: 0.500000 2021-03-26 06:47:56,046 epoch 9 - iter 15/50 - loss 8.56488609 - samples/sec: 81.90 - lr: 0.500000 2021-03-26 06:47:58,411 epoch 9 - iter 20/50 - loss 8.78860641 - samples/sec: 67.72 - lr: 0.500000 2021-03-26 06:48:00,279 epoch 9 - iter 25/50 - loss 8.74139832 - samples/sec: 85.76 - lr: 0.500000 2021-03-26 06:48:02,319 epoch 9 - iter 30/50 - loss 8.76814893 - samples/sec: 78.51 - lr: 0.500000 2021-03-26 06:48:04,117 epoch 9 - iter 35/50 - loss 8.67930719 - samples/sec: 89.07 - lr: 0.500000 2021-03-26 06:48:06,072 epoch 9 - iter 40/50 - loss 8.66237577 - samples/sec: 81.97 - lr: 0.500000 2021-03-26 06:48:07,905 epoch 9 - iter 45/50 - loss 8.61329811 - samples/sec: 87.33 - lr: 0.500000 2021-03-26 06:48:09,671 epoch 9 - iter 50/50 - loss 8.58023993 - samples/sec: 90.74 - lr: 0.500000 2021-03-26 06:48:09,672 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:48:09,672 EPOCH 9 done: loss 8.5802 - lr 0.5000000 2021-03-26 06:48:10,497 DEV : loss 7.6555399894714355 - score 0.8727 2021-03-26 06:48:10,527 BAD EPOCHS (no improvement): 0 2021-03-26 06:48:20,047 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:48:21,871 epoch 10 - iter 5/50 - loss 8.95939608 - samples/sec: 87.86 - lr: 0.500000 2021-03-26 06:48:23,800 epoch 10 - iter 10/50 - loss 8.59934626 - samples/sec: 83.01 - lr: 0.500000 2021-03-26 06:48:25,650 epoch 10 - iter 15/50 - loss 8.84599787 - samples/sec: 86.58 - lr: 0.500000 2021-03-26 06:48:27,448 epoch 10 - iter 20/50 - loss 8.35713410 - samples/sec: 89.09 - lr: 0.500000 2021-03-26 06:48:29,400 epoch 10 - iter 25/50 - loss 8.28440033 - samples/sec: 82.04 - lr: 0.500000 2021-03-26 06:48:31,451 epoch 10 - iter 30/50 - loss 8.36959931 - samples/sec: 78.07 - lr: 0.500000 2021-03-26 06:48:33,492 epoch 10 - iter 35/50 - loss 8.35715817 - samples/sec: 78.48 - lr: 0.500000 2021-03-26 06:48:35,355 epoch 10 - iter 40/50 - loss 8.29464904 - samples/sec: 85.95 - lr: 0.500000 2021-03-26 06:48:37,289 epoch 10 - iter 45/50 - loss 8.22910577 - samples/sec: 82.80 - lr: 0.500000 2021-03-26 06:48:39,187 epoch 10 - iter 50/50 - loss 8.30617043 - samples/sec: 84.38 - lr: 0.500000 2021-03-26 06:48:39,188 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:48:39,188 EPOCH 10 done: loss 8.3062 - lr 0.5000000 2021-03-26 06:48:39,977 DEV : loss 7.696311950683594 - score 0.8707 2021-03-26 06:48:40,000 BAD EPOCHS (no improvement): 1 2021-03-26 06:48:40,001 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:48:42,095 epoch 11 - iter 5/50 - loss 7.56332026 - samples/sec: 76.46 - lr: 0.500000 2021-03-26 06:48:44,126 epoch 11 - iter 10/50 - loss 8.00237746 - samples/sec: 78.86 - lr: 0.500000 2021-03-26 06:48:46,114 epoch 11 - iter 15/50 - loss 7.75435925 - samples/sec: 80.55 - lr: 0.500000 2021-03-26 06:48:48,167 epoch 11 - iter 20/50 - loss 7.90142670 - samples/sec: 78.02 - lr: 0.500000 2021-03-26 06:48:50,158 epoch 11 - iter 25/50 - loss 7.71243017 - samples/sec: 80.45 - lr: 0.500000 2021-03-26 06:48:52,086 epoch 11 - iter 30/50 - loss 7.78967923 - samples/sec: 83.08 - lr: 0.500000 2021-03-26 06:48:54,002 epoch 11 - iter 35/50 - loss 7.78273422 - samples/sec: 83.57 - lr: 0.500000 2021-03-26 06:48:55,890 epoch 11 - iter 40/50 - loss 7.72826921 - samples/sec: 84.83 - lr: 0.500000 2021-03-26 06:48:57,931 epoch 11 - iter 45/50 - loss 7.82425939 - samples/sec: 78.46 - lr: 0.500000 2021-03-26 06:48:59,678 epoch 11 - iter 50/50 - loss 7.68138412 - samples/sec: 91.73 - lr: 0.500000 2021-03-26 06:48:59,679 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:48:59,679 EPOCH 11 done: loss 7.6814 - lr 0.5000000 2021-03-26 06:49:00,450 DEV : loss 7.157756805419922 - score 0.8807 2021-03-26 06:49:00,473 BAD EPOCHS (no improvement): 0 2021-03-26 06:49:10,115 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:49:12,177 epoch 12 - iter 5/50 - loss 7.84549236 - samples/sec: 77.68 - lr: 0.500000 2021-03-26 06:49:14,071 epoch 12 - iter 10/50 - loss 7.50635629 - samples/sec: 84.58 - lr: 0.500000 2021-03-26 06:49:16,092 epoch 12 - iter 15/50 - loss 7.29152123 - samples/sec: 79.19 - lr: 0.500000 2021-03-26 06:49:18,041 epoch 12 - iter 20/50 - loss 7.36902924 - samples/sec: 82.19 - lr: 0.500000 2021-03-26 06:49:19,999 epoch 12 - iter 25/50 - loss 7.43501221 - samples/sec: 81.82 - lr: 0.500000 2021-03-26 06:49:21,872 epoch 12 - iter 30/50 - loss 7.38531418 - samples/sec: 85.49 - lr: 0.500000 2021-03-26 06:49:23,871 epoch 12 - iter 35/50 - loss 7.36810846 - samples/sec: 80.11 - lr: 0.500000 2021-03-26 06:49:25,918 epoch 12 - iter 40/50 - loss 7.36435213 - samples/sec: 78.21 - lr: 0.500000 2021-03-26 06:49:27,970 epoch 12 - iter 45/50 - loss 7.35478680 - samples/sec: 78.03 - lr: 0.500000 2021-03-26 06:49:29,796 epoch 12 - iter 50/50 - loss 7.20360278 - samples/sec: 87.74 - lr: 0.500000 2021-03-26 06:49:29,796 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:49:29,797 EPOCH 12 done: loss 7.2036 - lr 0.5000000 2021-03-26 06:49:30,670 DEV : loss 7.709672451019287 - score 0.865 2021-03-26 06:49:30,687 BAD EPOCHS (no improvement): 1 2021-03-26 06:49:30,688 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:49:32,483 epoch 13 - iter 5/50 - loss 6.63320417 - samples/sec: 89.24 - lr: 0.500000 2021-03-26 06:49:34,652 epoch 13 - iter 10/50 - loss 6.21271782 - samples/sec: 73.83 - lr: 0.500000 2021-03-26 06:49:36,669 epoch 13 - iter 15/50 - loss 6.43153175 - samples/sec: 79.41 - lr: 0.500000 2021-03-26 06:49:38,621 epoch 13 - iter 20/50 - loss 6.54766500 - samples/sec: 82.03 - lr: 0.500000 2021-03-26 06:49:40,449 epoch 13 - iter 25/50 - loss 6.75074554 - samples/sec: 87.65 - lr: 0.500000 2021-03-26 06:49:42,286 epoch 13 - iter 30/50 - loss 6.62655284 - samples/sec: 87.21 - lr: 0.500000 2021-03-26 06:49:44,322 epoch 13 - iter 35/50 - loss 6.64460591 - samples/sec: 78.65 - lr: 0.500000 2021-03-26 06:49:46,308 epoch 13 - iter 40/50 - loss 6.84230437 - samples/sec: 80.70 - lr: 0.500000 2021-03-26 06:49:48,240 epoch 13 - iter 45/50 - loss 6.82575455 - samples/sec: 82.86 - lr: 0.500000 2021-03-26 06:49:50,119 epoch 13 - iter 50/50 - loss 6.94763111 - samples/sec: 85.25 - lr: 0.500000 2021-03-26 06:49:50,120 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:49:50,120 EPOCH 13 done: loss 6.9476 - lr 0.5000000 2021-03-26 06:49:50,944 DEV : loss 6.670480728149414 - score 0.8864 2021-03-26 06:49:50,968 BAD EPOCHS (no improvement): 0 2021-03-26 06:50:00,535 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:50:02,378 epoch 14 - iter 5/50 - loss 6.48247395 - samples/sec: 86.92 - lr: 0.500000 2021-03-26 06:50:04,288 epoch 14 - iter 10/50 - loss 6.92508960 - samples/sec: 83.88 - lr: 0.500000 2021-03-26 06:50:06,091 epoch 14 - iter 15/50 - loss 6.34038912 - samples/sec: 88.81 - lr: 0.500000 2021-03-26 06:50:07,901 epoch 14 - iter 20/50 - loss 6.43623948 - samples/sec: 88.48 - lr: 0.500000 2021-03-26 06:50:09,764 epoch 14 - iter 25/50 - loss 6.37658873 - samples/sec: 85.99 - lr: 0.500000 2021-03-26 06:50:11,878 epoch 14 - iter 30/50 - loss 6.38693593 - samples/sec: 75.71 - lr: 0.500000 2021-03-26 06:50:13,785 epoch 14 - iter 35/50 - loss 6.41976604 - samples/sec: 83.97 - lr: 0.500000 2021-03-26 06:50:15,822 epoch 14 - iter 40/50 - loss 6.63060038 - samples/sec: 78.63 - lr: 0.500000 2021-03-26 06:50:17,789 epoch 14 - iter 45/50 - loss 6.71217939 - samples/sec: 81.41 - lr: 0.500000 2021-03-26 06:50:19,565 epoch 14 - iter 50/50 - loss 6.62694312 - samples/sec: 90.22 - lr: 0.500000 2021-03-26 06:50:19,566 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:50:19,566 EPOCH 14 done: loss 6.6269 - lr 0.5000000 2021-03-26 06:50:20,308 DEV : loss 6.7144269943237305 - score 0.8922 2021-03-26 06:50:20,333 BAD EPOCHS (no improvement): 0 2021-03-26 06:50:29,818 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:50:31,598 epoch 15 - iter 5/50 - loss 6.74751863 - samples/sec: 90.04 - lr: 0.500000 2021-03-26 06:50:33,500 epoch 15 - iter 10/50 - loss 6.18601270 - samples/sec: 84.22 - lr: 0.500000 2021-03-26 06:50:35,390 epoch 15 - iter 15/50 - loss 6.23807548 - samples/sec: 84.73 - lr: 0.500000 2021-03-26 06:50:37,241 epoch 15 - iter 20/50 - loss 6.25633945 - samples/sec: 86.55 - lr: 0.500000 2021-03-26 06:50:39,233 epoch 15 - iter 25/50 - loss 6.15654793 - samples/sec: 80.42 - lr: 0.500000 2021-03-26 06:50:41,173 epoch 15 - iter 30/50 - loss 6.16274056 - samples/sec: 82.55 - lr: 0.500000 2021-03-26 06:50:43,216 epoch 15 - iter 35/50 - loss 6.34539483 - samples/sec: 78.39 - lr: 0.500000 2021-03-26 06:50:45,216 epoch 15 - iter 40/50 - loss 6.36484003 - samples/sec: 80.09 - lr: 0.500000 2021-03-26 06:50:47,143 epoch 15 - iter 45/50 - loss 6.34316796 - samples/sec: 83.10 - lr: 0.500000 2021-03-26 06:50:49,115 epoch 15 - iter 50/50 - loss 6.36780796 - samples/sec: 81.21 - lr: 0.500000 2021-03-26 06:50:49,116 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:50:49,116 EPOCH 15 done: loss 6.3678 - lr 0.5000000 2021-03-26 06:50:49,900 DEV : loss 7.053887367248535 - score 0.8894 2021-03-26 06:50:49,929 BAD EPOCHS (no improvement): 1 2021-03-26 06:50:49,930 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:50:51,939 epoch 16 - iter 5/50 - loss 6.18410025 - samples/sec: 79.73 - lr: 0.500000 2021-03-26 06:50:53,856 epoch 16 - iter 10/50 - loss 6.26023331 - samples/sec: 83.53 - lr: 0.500000 2021-03-26 06:50:55,840 epoch 16 - iter 15/50 - loss 5.68440321 - samples/sec: 80.70 - lr: 0.500000 2021-03-26 06:50:57,899 epoch 16 - iter 20/50 - loss 5.82707292 - samples/sec: 77.77 - lr: 0.500000 2021-03-26 06:50:59,847 epoch 16 - iter 25/50 - loss 6.05982093 - samples/sec: 82.24 - lr: 0.500000 2021-03-26 06:51:01,744 epoch 16 - iter 30/50 - loss 5.95639232 - samples/sec: 84.44 - lr: 0.500000 2021-03-26 06:51:03,599 epoch 16 - iter 35/50 - loss 5.98666367 - samples/sec: 86.33 - lr: 0.500000 2021-03-26 06:51:05,575 epoch 16 - iter 40/50 - loss 5.99230914 - samples/sec: 81.05 - lr: 0.500000 2021-03-26 06:51:07,615 epoch 16 - iter 45/50 - loss 6.01807009 - samples/sec: 78.50 - lr: 0.500000 2021-03-26 06:51:09,435 epoch 16 - iter 50/50 - loss 6.09721210 - samples/sec: 87.97 - lr: 0.500000 2021-03-26 06:51:09,436 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:51:09,436 EPOCH 16 done: loss 6.0972 - lr 0.5000000 2021-03-26 06:51:10,229 DEV : loss 6.481603145599365 - score 0.8997 2021-03-26 06:51:10,255 BAD EPOCHS (no improvement): 0 2021-03-26 06:51:19,805 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:51:21,992 epoch 17 - iter 5/50 - loss 5.81198111 - samples/sec: 73.24 - lr: 0.500000 2021-03-26 06:51:23,985 epoch 17 - iter 10/50 - loss 5.60492916 - samples/sec: 80.39 - lr: 0.500000 2021-03-26 06:51:25,926 epoch 17 - iter 15/50 - loss 5.65811930 - samples/sec: 82.51 - lr: 0.500000 2021-03-26 06:51:27,982 epoch 17 - iter 20/50 - loss 5.58428044 - samples/sec: 77.92 - lr: 0.500000 2021-03-26 06:51:30,310 epoch 17 - iter 25/50 - loss 5.61901649 - samples/sec: 68.79 - lr: 0.500000 2021-03-26 06:51:32,473 epoch 17 - iter 30/50 - loss 5.74076101 - samples/sec: 74.03 - lr: 0.500000 2021-03-26 06:51:34,300 epoch 17 - iter 35/50 - loss 5.79365714 - samples/sec: 87.69 - lr: 0.500000 2021-03-26 06:51:36,214 epoch 17 - iter 40/50 - loss 5.76530095 - samples/sec: 83.67 - lr: 0.500000 2021-03-26 06:51:38,208 epoch 17 - iter 45/50 - loss 5.71586393 - samples/sec: 80.31 - lr: 0.500000 2021-03-26 06:51:40,087 epoch 17 - iter 50/50 - loss 5.81929769 - samples/sec: 85.23 - lr: 0.500000 2021-03-26 06:51:40,088 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:51:40,089 EPOCH 17 done: loss 5.8193 - lr 0.5000000 2021-03-26 06:51:40,855 DEV : loss 6.675870418548584 - score 0.8979 2021-03-26 06:51:40,876 BAD EPOCHS (no improvement): 1 2021-03-26 06:51:40,877 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:51:42,688 epoch 18 - iter 5/50 - loss 4.84788232 - samples/sec: 88.44 - lr: 0.500000 2021-03-26 06:51:44,686 epoch 18 - iter 10/50 - loss 5.07229483 - samples/sec: 80.12 - lr: 0.500000 2021-03-26 06:51:46,638 epoch 18 - iter 15/50 - loss 5.39524067 - samples/sec: 82.06 - lr: 0.500000 2021-03-26 06:51:48,557 epoch 18 - iter 20/50 - loss 5.47071220 - samples/sec: 83.45 - lr: 0.500000 2021-03-26 06:51:50,552 epoch 18 - iter 25/50 - loss 5.63781402 - samples/sec: 80.30 - lr: 0.500000 2021-03-26 06:51:52,695 epoch 18 - iter 30/50 - loss 5.65345579 - samples/sec: 74.70 - lr: 0.500000 2021-03-26 06:51:54,590 epoch 18 - iter 35/50 - loss 5.74147837 - samples/sec: 84.53 - lr: 0.500000 2021-03-26 06:51:56,651 epoch 18 - iter 40/50 - loss 5.74021587 - samples/sec: 77.71 - lr: 0.500000 2021-03-26 06:51:58,886 epoch 18 - iter 45/50 - loss 5.64183188 - samples/sec: 71.65 - lr: 0.500000 2021-03-26 06:52:00,848 epoch 18 - iter 50/50 - loss 5.63788223 - samples/sec: 81.61 - lr: 0.500000 2021-03-26 06:52:00,849 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:52:00,849 EPOCH 18 done: loss 5.6379 - lr 0.5000000 2021-03-26 06:52:01,630 DEV : loss 6.928553581237793 - score 0.8888 2021-03-26 06:52:01,655 BAD EPOCHS (no improvement): 2 2021-03-26 06:52:01,656 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:52:03,675 epoch 19 - iter 5/50 - loss 6.10535774 - samples/sec: 79.36 - lr: 0.500000 2021-03-26 06:52:05,646 epoch 19 - iter 10/50 - loss 5.71634462 - samples/sec: 81.23 - lr: 0.500000 2021-03-26 06:52:07,554 epoch 19 - iter 15/50 - loss 5.74245432 - samples/sec: 83.90 - lr: 0.500000 2021-03-26 06:52:09,566 epoch 19 - iter 20/50 - loss 5.63408655 - samples/sec: 79.61 - lr: 0.500000 2021-03-26 06:52:11,580 epoch 19 - iter 25/50 - loss 5.59964517 - samples/sec: 79.52 - lr: 0.500000 2021-03-26 06:52:13,424 epoch 19 - iter 30/50 - loss 5.48802720 - samples/sec: 86.89 - lr: 0.500000 2021-03-26 06:52:15,428 epoch 19 - iter 35/50 - loss 5.43323846 - samples/sec: 79.91 - lr: 0.500000 2021-03-26 06:52:17,222 epoch 19 - iter 40/50 - loss 5.37883350 - samples/sec: 89.29 - lr: 0.500000 2021-03-26 06:52:19,185 epoch 19 - iter 45/50 - loss 5.49508470 - samples/sec: 81.58 - lr: 0.500000 2021-03-26 06:52:21,043 epoch 19 - iter 50/50 - loss 5.50686664 - samples/sec: 86.21 - lr: 0.500000 2021-03-26 06:52:21,044 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:52:21,045 EPOCH 19 done: loss 5.5069 - lr 0.5000000 2021-03-26 06:52:21,840 DEV : loss 6.811120510101318 - score 0.8956 2021-03-26 06:52:21,864 BAD EPOCHS (no improvement): 3 2021-03-26 06:52:21,865 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:52:23,719 epoch 20 - iter 5/50 - loss 5.16742334 - samples/sec: 86.43 - lr: 0.500000 2021-03-26 06:52:25,647 epoch 20 - iter 10/50 - loss 5.16280818 - samples/sec: 83.07 - lr: 0.500000 2021-03-26 06:52:27,586 epoch 20 - iter 15/50 - loss 5.03337404 - samples/sec: 82.58 - lr: 0.500000 2021-03-26 06:52:29,544 epoch 20 - iter 20/50 - loss 4.95772830 - samples/sec: 81.79 - lr: 0.500000 2021-03-26 06:52:31,663 epoch 20 - iter 25/50 - loss 5.16223987 - samples/sec: 75.59 - lr: 0.500000 2021-03-26 06:52:33,671 epoch 20 - iter 30/50 - loss 5.12701021 - samples/sec: 79.72 - lr: 0.500000 2021-03-26 06:52:35,726 epoch 20 - iter 35/50 - loss 5.08359601 - samples/sec: 77.94 - lr: 0.500000 2021-03-26 06:52:37,891 epoch 20 - iter 40/50 - loss 5.22774757 - samples/sec: 73.97 - lr: 0.500000 2021-03-26 06:52:39,850 epoch 20 - iter 45/50 - loss 5.34838710 - samples/sec: 81.75 - lr: 0.500000 2021-03-26 06:52:41,877 epoch 20 - iter 50/50 - loss 5.35895552 - samples/sec: 78.97 - lr: 0.500000 2021-03-26 06:52:41,878 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:52:41,878 EPOCH 20 done: loss 5.3590 - lr 0.5000000 2021-03-26 06:52:42,646 DEV : loss 6.63265323638916 - score 0.8984 2021-03-26 06:52:42,670 BAD EPOCHS (no improvement): 4 2021-03-26 06:52:42,671 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:52:44,695 epoch 21 - iter 5/50 - loss 4.11205826 - samples/sec: 79.12 - lr: 0.250000 2021-03-26 06:52:46,598 epoch 21 - iter 10/50 - loss 4.39735847 - samples/sec: 84.14 - lr: 0.250000 2021-03-26 06:52:48,690 epoch 21 - iter 15/50 - loss 4.48490636 - samples/sec: 76.57 - lr: 0.250000 2021-03-26 06:52:50,870 epoch 21 - iter 20/50 - loss 4.59468763 - samples/sec: 73.44 - lr: 0.250000 2021-03-26 06:52:52,921 epoch 21 - iter 25/50 - loss 4.42129483 - samples/sec: 78.08 - lr: 0.250000 2021-03-26 06:52:55,056 epoch 21 - iter 30/50 - loss 4.40010382 - samples/sec: 75.02 - lr: 0.250000 2021-03-26 06:52:57,027 epoch 21 - iter 35/50 - loss 4.41746838 - samples/sec: 81.27 - lr: 0.250000 2021-03-26 06:52:59,240 epoch 21 - iter 40/50 - loss 4.45754075 - samples/sec: 72.35 - lr: 0.250000 2021-03-26 06:53:01,148 epoch 21 - iter 45/50 - loss 4.47268793 - samples/sec: 83.94 - lr: 0.250000 2021-03-26 06:53:03,087 epoch 21 - iter 50/50 - loss 4.48687317 - samples/sec: 82.61 - lr: 0.250000 2021-03-26 06:53:03,088 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:53:03,088 EPOCH 21 done: loss 4.4869 - lr 0.2500000 2021-03-26 06:53:03,846 DEV : loss 6.153275966644287 - score 0.9111 2021-03-26 06:53:03,870 BAD EPOCHS (no improvement): 0 2021-03-26 06:53:13,292 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:53:15,249 epoch 22 - iter 5/50 - loss 4.61283131 - samples/sec: 81.87 - lr: 0.250000 2021-03-26 06:53:17,388 epoch 22 - iter 10/50 - loss 4.34403934 - samples/sec: 74.88 - lr: 0.250000 2021-03-26 06:53:19,318 epoch 22 - iter 15/50 - loss 4.35565983 - samples/sec: 82.97 - lr: 0.250000 2021-03-26 06:53:21,313 epoch 22 - iter 20/50 - loss 4.36567014 - samples/sec: 80.28 - lr: 0.250000 2021-03-26 06:53:23,246 epoch 22 - iter 25/50 - loss 4.31828299 - samples/sec: 82.83 - lr: 0.250000 2021-03-26 06:53:25,194 epoch 22 - iter 30/50 - loss 4.21510568 - samples/sec: 82.22 - lr: 0.250000 2021-03-26 06:53:27,255 epoch 22 - iter 35/50 - loss 4.25309601 - samples/sec: 77.73 - lr: 0.250000 2021-03-26 06:53:29,444 epoch 22 - iter 40/50 - loss 4.32016158 - samples/sec: 73.13 - lr: 0.250000 2021-03-26 06:53:31,403 epoch 22 - iter 45/50 - loss 4.35405540 - samples/sec: 81.75 - lr: 0.250000 2021-03-26 06:53:33,192 epoch 22 - iter 50/50 - loss 4.37461680 - samples/sec: 89.57 - lr: 0.250000 2021-03-26 06:53:33,192 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:53:33,193 EPOCH 22 done: loss 4.3746 - lr 0.2500000 2021-03-26 06:53:34,000 DEV : loss 6.201013565063477 - score 0.9111 2021-03-26 06:53:34,026 BAD EPOCHS (no improvement): 1 2021-03-26 06:53:34,026 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:53:35,966 epoch 23 - iter 5/50 - loss 3.95178127 - samples/sec: 82.75 - lr: 0.250000 2021-03-26 06:53:37,791 epoch 23 - iter 10/50 - loss 3.83079326 - samples/sec: 87.78 - lr: 0.250000 2021-03-26 06:53:39,907 epoch 23 - iter 15/50 - loss 3.91210388 - samples/sec: 75.69 - lr: 0.250000 2021-03-26 06:53:41,936 epoch 23 - iter 20/50 - loss 3.97783825 - samples/sec: 78.94 - lr: 0.250000 2021-03-26 06:53:43,957 epoch 23 - iter 25/50 - loss 4.08968901 - samples/sec: 79.27 - lr: 0.250000 2021-03-26 06:53:46,037 epoch 23 - iter 30/50 - loss 4.01160220 - samples/sec: 77.01 - lr: 0.250000 2021-03-26 06:53:47,974 epoch 23 - iter 35/50 - loss 3.99960697 - samples/sec: 82.66 - lr: 0.250000 2021-03-26 06:53:50,038 epoch 23 - iter 40/50 - loss 4.04271683 - samples/sec: 77.59 - lr: 0.250000 2021-03-26 06:53:52,097 epoch 23 - iter 45/50 - loss 4.03870177 - samples/sec: 77.78 - lr: 0.250000 2021-03-26 06:53:53,915 epoch 23 - iter 50/50 - loss 4.17525750 - samples/sec: 88.07 - lr: 0.250000 2021-03-26 06:53:53,916 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:53:53,917 EPOCH 23 done: loss 4.1753 - lr 0.2500000 2021-03-26 06:53:54,707 DEV : loss 6.3719329833984375 - score 0.9071 2021-03-26 06:53:54,731 BAD EPOCHS (no improvement): 2 2021-03-26 06:53:54,732 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:53:56,617 epoch 24 - iter 5/50 - loss 3.20824995 - samples/sec: 84.99 - lr: 0.250000 2021-03-26 06:53:58,559 epoch 24 - iter 10/50 - loss 3.63779755 - samples/sec: 82.49 - lr: 0.250000 2021-03-26 06:54:00,565 epoch 24 - iter 15/50 - loss 3.66257393 - samples/sec: 79.83 - lr: 0.250000 2021-03-26 06:54:02,510 epoch 24 - iter 20/50 - loss 3.57795765 - samples/sec: 82.35 - lr: 0.250000 2021-03-26 06:54:04,570 epoch 24 - iter 25/50 - loss 3.67160597 - samples/sec: 77.74 - lr: 0.250000 2021-03-26 06:54:06,748 epoch 24 - iter 30/50 - loss 3.83814465 - samples/sec: 73.53 - lr: 0.250000 2021-03-26 06:54:08,685 epoch 24 - iter 35/50 - loss 3.87231894 - samples/sec: 82.70 - lr: 0.250000 2021-03-26 06:54:10,597 epoch 24 - iter 40/50 - loss 3.93506041 - samples/sec: 83.75 - lr: 0.250000 2021-03-26 06:54:12,489 epoch 24 - iter 45/50 - loss 3.92431094 - samples/sec: 84.62 - lr: 0.250000 2021-03-26 06:54:14,299 epoch 24 - iter 50/50 - loss 3.97243820 - samples/sec: 88.53 - lr: 0.250000 2021-03-26 06:54:14,300 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:54:14,300 EPOCH 24 done: loss 3.9724 - lr 0.2500000 2021-03-26 06:54:15,067 DEV : loss 6.205985069274902 - score 0.9077 2021-03-26 06:54:15,084 BAD EPOCHS (no improvement): 3 2021-03-26 06:54:15,085 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:54:16,911 epoch 25 - iter 5/50 - loss 3.35670197 - samples/sec: 87.75 - lr: 0.250000 2021-03-26 06:54:18,825 epoch 25 - iter 10/50 - loss 3.88174726 - samples/sec: 83.65 - lr: 0.250000 2021-03-26 06:54:20,851 epoch 25 - iter 15/50 - loss 3.94504937 - samples/sec: 79.06 - lr: 0.250000 2021-03-26 06:54:22,718 epoch 25 - iter 20/50 - loss 3.86929995 - samples/sec: 85.81 - lr: 0.250000 2021-03-26 06:54:24,663 epoch 25 - iter 25/50 - loss 3.85905517 - samples/sec: 82.31 - lr: 0.250000 2021-03-26 06:54:26,772 epoch 25 - iter 30/50 - loss 3.91150273 - samples/sec: 75.92 - lr: 0.250000 2021-03-26 06:54:28,774 epoch 25 - iter 35/50 - loss 3.95813450 - samples/sec: 79.99 - lr: 0.250000 2021-03-26 06:54:30,700 epoch 25 - iter 40/50 - loss 3.95463861 - samples/sec: 83.15 - lr: 0.250000 2021-03-26 06:54:32,766 epoch 25 - iter 45/50 - loss 3.94753084 - samples/sec: 77.53 - lr: 0.250000 2021-03-26 06:54:34,540 epoch 25 - iter 50/50 - loss 3.90389370 - samples/sec: 90.32 - lr: 0.250000 2021-03-26 06:54:34,541 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:54:34,541 EPOCH 25 done: loss 3.9039 - lr 0.2500000 2021-03-26 06:54:35,316 DEV : loss 6.084229946136475 - score 0.9119 2021-03-26 06:54:35,342 BAD EPOCHS (no improvement): 0 2021-03-26 06:54:44,799 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:54:46,667 epoch 26 - iter 5/50 - loss 3.07638960 - samples/sec: 85.82 - lr: 0.250000 2021-03-26 06:54:48,604 epoch 26 - iter 10/50 - loss 3.45474722 - samples/sec: 82.70 - lr: 0.250000 2021-03-26 06:54:50,592 epoch 26 - iter 15/50 - loss 3.80810453 - samples/sec: 80.55 - lr: 0.250000 2021-03-26 06:54:52,617 epoch 26 - iter 20/50 - loss 3.68002245 - samples/sec: 79.07 - lr: 0.250000 2021-03-26 06:54:54,533 epoch 26 - iter 25/50 - loss 3.78536674 - samples/sec: 83.60 - lr: 0.250000 2021-03-26 06:54:56,643 epoch 26 - iter 30/50 - loss 3.72170803 - samples/sec: 75.88 - lr: 0.250000 2021-03-26 06:54:59,727 epoch 26 - iter 35/50 - loss 3.70023379 - samples/sec: 51.92 - lr: 0.250000 2021-03-26 06:55:01,693 epoch 26 - iter 40/50 - loss 3.70014390 - samples/sec: 81.46 - lr: 0.250000 2021-03-26 06:55:03,743 epoch 26 - iter 45/50 - loss 3.71391151 - samples/sec: 78.10 - lr: 0.250000 2021-03-26 06:55:05,534 epoch 26 - iter 50/50 - loss 3.66467248 - samples/sec: 89.39 - lr: 0.250000 2021-03-26 06:55:05,535 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:55:05,536 EPOCH 26 done: loss 3.6647 - lr 0.2500000 2021-03-26 06:55:06,325 DEV : loss 6.3462018966674805 - score 0.9103 2021-03-26 06:55:06,349 BAD EPOCHS (no improvement): 1 2021-03-26 06:55:06,350 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:55:08,437 epoch 27 - iter 5/50 - loss 4.35266724 - samples/sec: 76.75 - lr: 0.250000 2021-03-26 06:55:10,217 epoch 27 - iter 10/50 - loss 3.97600597 - samples/sec: 89.99 - lr: 0.250000 2021-03-26 06:55:12,205 epoch 27 - iter 15/50 - loss 3.96660554 - samples/sec: 80.57 - lr: 0.250000 2021-03-26 06:55:14,359 epoch 27 - iter 20/50 - loss 3.75834839 - samples/sec: 74.32 - lr: 0.250000 2021-03-26 06:55:16,446 epoch 27 - iter 25/50 - loss 3.81192220 - samples/sec: 76.75 - lr: 0.250000 2021-03-26 06:55:18,343 epoch 27 - iter 30/50 - loss 3.70215019 - samples/sec: 84.40 - lr: 0.250000 2021-03-26 06:55:20,359 epoch 27 - iter 35/50 - loss 3.66418975 - samples/sec: 79.47 - lr: 0.250000 2021-03-26 06:55:22,303 epoch 27 - iter 40/50 - loss 3.71305492 - samples/sec: 82.37 - lr: 0.250000 2021-03-26 06:55:24,188 epoch 27 - iter 45/50 - loss 3.75636695 - samples/sec: 84.99 - lr: 0.250000 2021-03-26 06:55:26,120 epoch 27 - iter 50/50 - loss 3.79689440 - samples/sec: 82.91 - lr: 0.250000 2021-03-26 06:55:26,120 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:55:26,121 EPOCH 27 done: loss 3.7969 - lr 0.2500000 2021-03-26 06:55:26,899 DEV : loss 6.2635931968688965 - score 0.9075 2021-03-26 06:55:26,924 BAD EPOCHS (no improvement): 2 2021-03-26 06:55:26,925 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:55:28,807 epoch 28 - iter 5/50 - loss 3.31051455 - samples/sec: 85.09 - lr: 0.250000 2021-03-26 06:55:30,804 epoch 28 - iter 10/50 - loss 3.48266208 - samples/sec: 80.23 - lr: 0.250000 2021-03-26 06:55:32,674 epoch 28 - iter 15/50 - loss 3.33151848 - samples/sec: 85.69 - lr: 0.250000 2021-03-26 06:55:34,657 epoch 28 - iter 20/50 - loss 3.58922390 - samples/sec: 80.72 - lr: 0.250000 2021-03-26 06:55:36,628 epoch 28 - iter 25/50 - loss 3.52472383 - samples/sec: 81.27 - lr: 0.250000 2021-03-26 06:55:38,604 epoch 28 - iter 30/50 - loss 3.60845091 - samples/sec: 81.07 - lr: 0.250000 2021-03-26 06:55:40,671 epoch 28 - iter 35/50 - loss 3.65303677 - samples/sec: 77.51 - lr: 0.250000 2021-03-26 06:55:42,851 epoch 28 - iter 40/50 - loss 3.65428785 - samples/sec: 73.44 - lr: 0.250000 2021-03-26 06:55:44,715 epoch 28 - iter 45/50 - loss 3.69842546 - samples/sec: 85.97 - lr: 0.250000 2021-03-26 06:55:46,756 epoch 28 - iter 50/50 - loss 3.68927022 - samples/sec: 78.46 - lr: 0.250000 2021-03-26 06:55:46,757 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:55:46,757 EPOCH 28 done: loss 3.6893 - lr 0.2500000 2021-03-26 06:55:47,558 DEV : loss 6.469142913818359 - score 0.9119 2021-03-26 06:55:47,578 BAD EPOCHS (no improvement): 3 2021-03-26 06:55:47,578 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:55:49,695 epoch 29 - iter 5/50 - loss 3.68538837 - samples/sec: 75.63 - lr: 0.250000 2021-03-26 06:55:51,593 epoch 29 - iter 10/50 - loss 3.49097276 - samples/sec: 84.41 - lr: 0.250000 2021-03-26 06:55:53,500 epoch 29 - iter 15/50 - loss 3.51349881 - samples/sec: 83.97 - lr: 0.250000 2021-03-26 06:55:55,462 epoch 29 - iter 20/50 - loss 3.47587315 - samples/sec: 81.63 - lr: 0.250000 2021-03-26 06:55:57,701 epoch 29 - iter 25/50 - loss 3.52188293 - samples/sec: 71.52 - lr: 0.250000 2021-03-26 06:55:59,674 epoch 29 - iter 30/50 - loss 3.52639737 - samples/sec: 81.19 - lr: 0.250000 2021-03-26 06:56:01,531 epoch 29 - iter 35/50 - loss 3.54751299 - samples/sec: 86.20 - lr: 0.250000 2021-03-26 06:56:03,585 epoch 29 - iter 40/50 - loss 3.59073097 - samples/sec: 77.98 - lr: 0.250000 2021-03-26 06:56:05,983 epoch 29 - iter 45/50 - loss 3.64443576 - samples/sec: 66.76 - lr: 0.250000 2021-03-26 06:56:07,729 epoch 29 - iter 50/50 - loss 3.62032253 - samples/sec: 91.72 - lr: 0.250000 2021-03-26 06:56:07,730 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:56:07,730 EPOCH 29 done: loss 3.6203 - lr 0.2500000 2021-03-26 06:56:08,499 DEV : loss 6.374938488006592 - score 0.9095 2021-03-26 06:56:08,524 BAD EPOCHS (no improvement): 4 2021-03-26 06:56:08,525 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:56:10,462 epoch 30 - iter 5/50 - loss 3.23985057 - samples/sec: 82.68 - lr: 0.125000 2021-03-26 06:56:12,372 epoch 30 - iter 10/50 - loss 3.15554574 - samples/sec: 83.90 - lr: 0.125000 2021-03-26 06:56:14,327 epoch 30 - iter 15/50 - loss 3.30265152 - samples/sec: 81.89 - lr: 0.125000 2021-03-26 06:56:16,296 epoch 30 - iter 20/50 - loss 3.36633397 - samples/sec: 81.35 - lr: 0.125000 2021-03-26 06:56:18,228 epoch 30 - iter 25/50 - loss 3.36648575 - samples/sec: 82.89 - lr: 0.125000 2021-03-26 06:56:20,251 epoch 30 - iter 30/50 - loss 3.46024215 - samples/sec: 79.19 - lr: 0.125000 2021-03-26 06:56:22,427 epoch 30 - iter 35/50 - loss 3.40659271 - samples/sec: 73.62 - lr: 0.125000 2021-03-26 06:56:24,481 epoch 30 - iter 40/50 - loss 3.42356200 - samples/sec: 77.99 - lr: 0.125000 2021-03-26 06:56:26,488 epoch 30 - iter 45/50 - loss 3.38766970 - samples/sec: 79.78 - lr: 0.125000 2021-03-26 06:56:28,404 epoch 30 - iter 50/50 - loss 3.30463873 - samples/sec: 83.57 - lr: 0.125000 2021-03-26 06:56:28,405 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:56:28,406 EPOCH 30 done: loss 3.3046 - lr 0.1250000 2021-03-26 06:56:29,163 DEV : loss 6.366065979003906 - score 0.9118 2021-03-26 06:56:29,180 BAD EPOCHS (no improvement): 1 2021-03-26 06:56:29,181 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:56:31,012 epoch 31 - iter 5/50 - loss 3.19004385 - samples/sec: 87.48 - lr: 0.125000 2021-03-26 06:56:32,754 epoch 31 - iter 10/50 - loss 2.96236542 - samples/sec: 91.96 - lr: 0.125000 2021-03-26 06:56:34,698 epoch 31 - iter 15/50 - loss 3.00070031 - samples/sec: 82.38 - lr: 0.125000 2021-03-26 06:56:36,535 epoch 31 - iter 20/50 - loss 3.12568991 - samples/sec: 87.15 - lr: 0.125000 2021-03-26 06:56:38,533 epoch 31 - iter 25/50 - loss 3.18731164 - samples/sec: 80.20 - lr: 0.125000 2021-03-26 06:56:40,448 epoch 31 - iter 30/50 - loss 3.14024767 - samples/sec: 83.60 - lr: 0.125000 2021-03-26 06:56:42,443 epoch 31 - iter 35/50 - loss 3.16921918 - samples/sec: 80.28 - lr: 0.125000 2021-03-26 06:56:44,223 epoch 31 - iter 40/50 - loss 3.11370867 - samples/sec: 90.01 - lr: 0.125000 2021-03-26 06:56:46,127 epoch 31 - iter 45/50 - loss 3.06799367 - samples/sec: 84.11 - lr: 0.125000 2021-03-26 06:56:47,960 epoch 31 - iter 50/50 - loss 3.13836832 - samples/sec: 87.41 - lr: 0.125000 2021-03-26 06:56:47,961 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:56:47,961 EPOCH 31 done: loss 3.1384 - lr 0.1250000 2021-03-26 06:56:48,724 DEV : loss 6.220050811767578 - score 0.916 2021-03-26 06:56:48,749 BAD EPOCHS (no improvement): 0 2021-03-26 06:56:58,306 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:57:00,061 epoch 32 - iter 5/50 - loss 2.85820818 - samples/sec: 91.32 - lr: 0.125000 2021-03-26 06:57:02,183 epoch 32 - iter 10/50 - loss 3.07046874 - samples/sec: 75.47 - lr: 0.125000 2021-03-26 06:57:04,132 epoch 32 - iter 15/50 - loss 2.93962463 - samples/sec: 82.19 - lr: 0.125000 2021-03-26 06:57:05,988 epoch 32 - iter 20/50 - loss 3.03024350 - samples/sec: 86.31 - lr: 0.125000 2021-03-26 06:57:07,979 epoch 32 - iter 25/50 - loss 3.07685027 - samples/sec: 80.44 - lr: 0.125000 2021-03-26 06:57:09,770 epoch 32 - iter 30/50 - loss 3.12704571 - samples/sec: 89.43 - lr: 0.125000 2021-03-26 06:57:11,865 epoch 32 - iter 35/50 - loss 3.12945336 - samples/sec: 76.44 - lr: 0.125000 2021-03-26 06:57:13,974 epoch 32 - iter 40/50 - loss 3.21813447 - samples/sec: 75.92 - lr: 0.125000 2021-03-26 06:57:16,025 epoch 32 - iter 45/50 - loss 3.22640948 - samples/sec: 78.08 - lr: 0.125000 2021-03-26 06:57:17,793 epoch 32 - iter 50/50 - loss 3.19685128 - samples/sec: 90.64 - lr: 0.125000 2021-03-26 06:57:17,794 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:57:17,794 EPOCH 32 done: loss 3.1969 - lr 0.1250000 2021-03-26 06:57:18,620 DEV : loss 6.164405822753906 - score 0.918 2021-03-26 06:57:18,654 BAD EPOCHS (no improvement): 0 2021-03-26 06:57:28,508 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:57:30,475 epoch 33 - iter 5/50 - loss 2.54982162 - samples/sec: 81.44 - lr: 0.125000 2021-03-26 06:57:32,367 epoch 33 - iter 10/50 - loss 2.87348752 - samples/sec: 84.61 - lr: 0.125000 2021-03-26 06:57:34,396 epoch 33 - iter 15/50 - loss 2.85248617 - samples/sec: 78.95 - lr: 0.125000 2021-03-26 06:57:36,316 epoch 33 - iter 20/50 - loss 2.77764983 - samples/sec: 83.42 - lr: 0.125000 2021-03-26 06:57:38,474 epoch 33 - iter 25/50 - loss 2.80861367 - samples/sec: 74.17 - lr: 0.125000 2021-03-26 06:57:40,469 epoch 33 - iter 30/50 - loss 2.89748362 - samples/sec: 80.29 - lr: 0.125000 2021-03-26 06:57:42,439 epoch 33 - iter 35/50 - loss 2.89348922 - samples/sec: 81.29 - lr: 0.125000 2021-03-26 06:57:44,478 epoch 33 - iter 40/50 - loss 2.92309944 - samples/sec: 78.55 - lr: 0.125000 2021-03-26 06:57:46,640 epoch 33 - iter 45/50 - loss 2.87363632 - samples/sec: 74.07 - lr: 0.125000 2021-03-26 06:57:48,648 epoch 33 - iter 50/50 - loss 2.93893275 - samples/sec: 79.75 - lr: 0.125000 2021-03-26 06:57:48,649 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:57:48,650 EPOCH 33 done: loss 2.9389 - lr 0.1250000 2021-03-26 06:57:49,453 DEV : loss 6.186984062194824 - score 0.917 2021-03-26 06:57:49,475 BAD EPOCHS (no improvement): 1 2021-03-26 06:57:49,476 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:57:51,745 epoch 34 - iter 5/50 - loss 3.01934357 - samples/sec: 70.54 - lr: 0.125000 2021-03-26 06:57:53,900 epoch 34 - iter 10/50 - loss 2.74939032 - samples/sec: 74.32 - lr: 0.125000 2021-03-26 06:57:55,908 epoch 34 - iter 15/50 - loss 2.77931755 - samples/sec: 79.76 - lr: 0.125000 2021-03-26 06:57:57,846 epoch 34 - iter 20/50 - loss 2.89119451 - samples/sec: 82.65 - lr: 0.125000 2021-03-26 06:57:59,694 epoch 34 - iter 25/50 - loss 2.82531148 - samples/sec: 86.70 - lr: 0.125000 2021-03-26 06:58:01,704 epoch 34 - iter 30/50 - loss 2.80726229 - samples/sec: 79.68 - lr: 0.125000 2021-03-26 06:58:03,539 epoch 34 - iter 35/50 - loss 2.83326924 - samples/sec: 87.27 - lr: 0.125000 2021-03-26 06:58:05,540 epoch 34 - iter 40/50 - loss 2.87008840 - samples/sec: 80.07 - lr: 0.125000 2021-03-26 06:58:07,448 epoch 34 - iter 45/50 - loss 2.94134395 - samples/sec: 83.94 - lr: 0.125000 2021-03-26 06:58:09,333 epoch 34 - iter 50/50 - loss 3.07199399 - samples/sec: 84.96 - lr: 0.125000 2021-03-26 06:58:09,334 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:58:09,334 EPOCH 34 done: loss 3.0720 - lr 0.1250000 2021-03-26 06:58:10,124 DEV : loss 6.177824974060059 - score 0.9192 2021-03-26 06:58:10,148 BAD EPOCHS (no improvement): 0 2021-03-26 06:58:19,752 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:58:21,539 epoch 35 - iter 5/50 - loss 3.17444773 - samples/sec: 89.67 - lr: 0.125000 2021-03-26 06:58:23,452 epoch 35 - iter 10/50 - loss 3.35550301 - samples/sec: 83.71 - lr: 0.125000 2021-03-26 06:58:25,379 epoch 35 - iter 15/50 - loss 3.17661854 - samples/sec: 83.08 - lr: 0.125000 2021-03-26 06:58:27,562 epoch 35 - iter 20/50 - loss 3.04253340 - samples/sec: 73.37 - lr: 0.125000 2021-03-26 06:58:29,476 epoch 35 - iter 25/50 - loss 3.03141157 - samples/sec: 83.68 - lr: 0.125000 2021-03-26 06:58:31,266 epoch 35 - iter 30/50 - loss 2.99109411 - samples/sec: 89.43 - lr: 0.125000 2021-03-26 06:58:33,409 epoch 35 - iter 35/50 - loss 3.02970564 - samples/sec: 74.75 - lr: 0.125000 2021-03-26 06:58:35,343 epoch 35 - iter 40/50 - loss 2.96278723 - samples/sec: 82.79 - lr: 0.125000 2021-03-26 06:58:37,293 epoch 35 - iter 45/50 - loss 2.96387931 - samples/sec: 82.14 - lr: 0.125000 2021-03-26 06:58:39,079 epoch 35 - iter 50/50 - loss 2.94057366 - samples/sec: 89.68 - lr: 0.125000 2021-03-26 06:58:39,080 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:58:39,080 EPOCH 35 done: loss 2.9406 - lr 0.1250000 2021-03-26 06:58:39,869 DEV : loss 6.2086615562438965 - score 0.9176 2021-03-26 06:58:39,887 BAD EPOCHS (no improvement): 1 2021-03-26 06:58:39,888 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:58:41,841 epoch 36 - iter 5/50 - loss 2.91071506 - samples/sec: 82.00 - lr: 0.125000 2021-03-26 06:58:43,859 epoch 36 - iter 10/50 - loss 2.79274061 - samples/sec: 79.36 - lr: 0.125000 2021-03-26 06:58:45,663 epoch 36 - iter 15/50 - loss 2.83839884 - samples/sec: 88.79 - lr: 0.125000 2021-03-26 06:58:47,575 epoch 36 - iter 20/50 - loss 2.74050820 - samples/sec: 83.77 - lr: 0.125000 2021-03-26 06:58:49,517 epoch 36 - iter 25/50 - loss 2.80581013 - samples/sec: 82.47 - lr: 0.125000 2021-03-26 06:58:51,403 epoch 36 - iter 30/50 - loss 2.79912020 - samples/sec: 84.91 - lr: 0.125000 2021-03-26 06:58:53,307 epoch 36 - iter 35/50 - loss 2.82536783 - samples/sec: 84.10 - lr: 0.125000 2021-03-26 06:58:55,291 epoch 36 - iter 40/50 - loss 2.78613535 - samples/sec: 80.73 - lr: 0.125000 2021-03-26 06:58:57,144 epoch 36 - iter 45/50 - loss 2.75108689 - samples/sec: 86.65 - lr: 0.125000 2021-03-26 06:58:59,082 epoch 36 - iter 50/50 - loss 2.76818178 - samples/sec: 82.61 - lr: 0.125000 2021-03-26 06:58:59,083 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:58:59,083 EPOCH 36 done: loss 2.7682 - lr 0.1250000 2021-03-26 06:58:59,869 DEV : loss 6.127795696258545 - score 0.9174 2021-03-26 06:58:59,886 BAD EPOCHS (no improvement): 2 2021-03-26 06:58:59,886 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:59:01,782 epoch 37 - iter 5/50 - loss 2.71080043 - samples/sec: 84.48 - lr: 0.125000 2021-03-26 06:59:03,754 epoch 37 - iter 10/50 - loss 2.78810500 - samples/sec: 81.22 - lr: 0.125000 2021-03-26 06:59:05,672 epoch 37 - iter 15/50 - loss 2.70681345 - samples/sec: 83.50 - lr: 0.125000 2021-03-26 06:59:07,555 epoch 37 - iter 20/50 - loss 2.85075440 - samples/sec: 85.05 - lr: 0.125000 2021-03-26 06:59:09,645 epoch 37 - iter 25/50 - loss 2.78323064 - samples/sec: 76.63 - lr: 0.125000 2021-03-26 06:59:11,596 epoch 37 - iter 30/50 - loss 2.73008518 - samples/sec: 82.09 - lr: 0.125000 2021-03-26 06:59:13,941 epoch 37 - iter 35/50 - loss 2.70971954 - samples/sec: 68.27 - lr: 0.125000 2021-03-26 06:59:15,808 epoch 37 - iter 40/50 - loss 2.74770900 - samples/sec: 85.79 - lr: 0.125000 2021-03-26 06:59:17,907 epoch 37 - iter 45/50 - loss 2.71176132 - samples/sec: 76.28 - lr: 0.125000 2021-03-26 06:59:19,929 epoch 37 - iter 50/50 - loss 2.73245367 - samples/sec: 79.21 - lr: 0.125000 2021-03-26 06:59:19,930 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:59:19,930 EPOCH 37 done: loss 2.7325 - lr 0.1250000 2021-03-26 06:59:20,707 DEV : loss 6.2698516845703125 - score 0.9136 2021-03-26 06:59:20,732 BAD EPOCHS (no improvement): 3 2021-03-26 06:59:20,733 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:59:22,756 epoch 38 - iter 5/50 - loss 2.90655355 - samples/sec: 79.18 - lr: 0.125000 2021-03-26 06:59:24,734 epoch 38 - iter 10/50 - loss 2.93488872 - samples/sec: 81.01 - lr: 0.125000 2021-03-26 06:59:26,744 epoch 38 - iter 15/50 - loss 2.86677569 - samples/sec: 79.68 - lr: 0.125000 2021-03-26 06:59:28,976 epoch 38 - iter 20/50 - loss 2.91202615 - samples/sec: 71.74 - lr: 0.125000 2021-03-26 06:59:30,891 epoch 38 - iter 25/50 - loss 2.86307866 - samples/sec: 83.65 - lr: 0.125000 2021-03-26 06:59:33,072 epoch 38 - iter 30/50 - loss 2.95075897 - samples/sec: 73.44 - lr: 0.125000 2021-03-26 06:59:34,952 epoch 38 - iter 35/50 - loss 2.90607083 - samples/sec: 85.16 - lr: 0.125000 2021-03-26 06:59:36,765 epoch 38 - iter 40/50 - loss 2.84376124 - samples/sec: 88.34 - lr: 0.125000 2021-03-26 06:59:38,899 epoch 38 - iter 45/50 - loss 2.79523445 - samples/sec: 75.02 - lr: 0.125000 2021-03-26 06:59:40,948 epoch 38 - iter 50/50 - loss 2.86676959 - samples/sec: 78.18 - lr: 0.125000 2021-03-26 06:59:40,949 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:59:40,949 EPOCH 38 done: loss 2.8668 - lr 0.1250000 2021-03-26 06:59:41,715 DEV : loss 6.26828145980835 - score 0.9162 2021-03-26 06:59:41,740 BAD EPOCHS (no improvement): 4 2021-03-26 06:59:41,741 ---------------------------------------------------------------------------------------------------- 2021-03-26 06:59:43,849 epoch 39 - iter 5/50 - loss 2.46924899 - samples/sec: 75.96 - lr: 0.062500 2021-03-26 06:59:46,083 epoch 39 - iter 10/50 - loss 2.86362134 - samples/sec: 71.68 - lr: 0.062500 2021-03-26 06:59:48,163 epoch 39 - iter 15/50 - loss 2.78012599 - samples/sec: 77.00 - lr: 0.062500 2021-03-26 06:59:49,991 epoch 39 - iter 20/50 - loss 2.71098604 - samples/sec: 87.64 - lr: 0.062500 2021-03-26 06:59:52,191 epoch 39 - iter 25/50 - loss 2.65226630 - samples/sec: 72.76 - lr: 0.062500 2021-03-26 06:59:54,168 epoch 39 - iter 30/50 - loss 2.74109042 - samples/sec: 81.03 - lr: 0.062500 2021-03-26 06:59:56,261 epoch 39 - iter 35/50 - loss 2.78488238 - samples/sec: 76.50 - lr: 0.062500 2021-03-26 06:59:58,198 epoch 39 - iter 40/50 - loss 2.74619337 - samples/sec: 82.68 - lr: 0.062500 2021-03-26 06:59:59,987 epoch 39 - iter 45/50 - loss 2.72437237 - samples/sec: 89.50 - lr: 0.062500 2021-03-26 07:00:01,796 epoch 39 - iter 50/50 - loss 2.68863890 - samples/sec: 88.52 - lr: 0.062500 2021-03-26 07:00:01,797 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:00:01,797 EPOCH 39 done: loss 2.6886 - lr 0.0625000 2021-03-26 07:00:02,589 DEV : loss 6.284448146820068 - score 0.917 2021-03-26 07:00:02,614 BAD EPOCHS (no improvement): 1 2021-03-26 07:00:02,614 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:00:04,580 epoch 40 - iter 5/50 - loss 2.98166246 - samples/sec: 81.48 - lr: 0.062500 2021-03-26 07:00:06,572 epoch 40 - iter 10/50 - loss 2.72216311 - samples/sec: 80.40 - lr: 0.062500 2021-03-26 07:00:08,496 epoch 40 - iter 15/50 - loss 2.75526333 - samples/sec: 83.23 - lr: 0.062500 2021-03-26 07:00:10,561 epoch 40 - iter 20/50 - loss 2.78459678 - samples/sec: 77.58 - lr: 0.062500 2021-03-26 07:00:13,042 epoch 40 - iter 25/50 - loss 2.79560489 - samples/sec: 64.55 - lr: 0.062500 2021-03-26 07:00:14,886 epoch 40 - iter 30/50 - loss 2.71052564 - samples/sec: 86.84 - lr: 0.062500 2021-03-26 07:00:16,780 epoch 40 - iter 35/50 - loss 2.66883559 - samples/sec: 84.57 - lr: 0.062500 2021-03-26 07:00:18,875 epoch 40 - iter 40/50 - loss 2.69405829 - samples/sec: 76.44 - lr: 0.062500 2021-03-26 07:00:20,935 epoch 40 - iter 45/50 - loss 2.69044249 - samples/sec: 77.72 - lr: 0.062500 2021-03-26 07:00:22,728 epoch 40 - iter 50/50 - loss 2.72203983 - samples/sec: 89.33 - lr: 0.062500 2021-03-26 07:00:22,729 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:00:22,730 EPOCH 40 done: loss 2.7220 - lr 0.0625000 2021-03-26 07:00:23,521 DEV : loss 6.248233795166016 - score 0.9182 2021-03-26 07:00:23,546 BAD EPOCHS (no improvement): 2 2021-03-26 07:00:23,547 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:00:25,680 epoch 41 - iter 5/50 - loss 2.46897027 - samples/sec: 75.08 - lr: 0.062500 2021-03-26 07:00:27,579 epoch 41 - iter 10/50 - loss 2.66373817 - samples/sec: 84.32 - lr: 0.062500 2021-03-26 07:00:29,420 epoch 41 - iter 15/50 - loss 2.72111547 - samples/sec: 87.02 - lr: 0.062500 2021-03-26 07:00:31,279 epoch 41 - iter 20/50 - loss 2.65721455 - samples/sec: 86.12 - lr: 0.062500 2021-03-26 07:00:33,235 epoch 41 - iter 25/50 - loss 2.60619400 - samples/sec: 81.86 - lr: 0.062500 2021-03-26 07:00:35,120 epoch 41 - iter 30/50 - loss 2.56057245 - samples/sec: 84.99 - lr: 0.062500 2021-03-26 07:00:37,214 epoch 41 - iter 35/50 - loss 2.58174192 - samples/sec: 76.45 - lr: 0.062500 2021-03-26 07:00:39,073 epoch 41 - iter 40/50 - loss 2.62893503 - samples/sec: 86.18 - lr: 0.062500 2021-03-26 07:00:41,160 epoch 41 - iter 45/50 - loss 2.67516147 - samples/sec: 76.72 - lr: 0.062500 2021-03-26 07:00:42,937 epoch 41 - iter 50/50 - loss 2.65079946 - samples/sec: 90.12 - lr: 0.062500 2021-03-26 07:00:42,937 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:00:42,938 EPOCH 41 done: loss 2.6508 - lr 0.0625000 2021-03-26 07:00:43,724 DEV : loss 6.362326622009277 - score 0.9166 2021-03-26 07:00:43,748 BAD EPOCHS (no improvement): 3 2021-03-26 07:00:43,749 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:00:45,638 epoch 42 - iter 5/50 - loss 2.71794162 - samples/sec: 84.83 - lr: 0.062500 2021-03-26 07:00:47,540 epoch 42 - iter 10/50 - loss 2.52051284 - samples/sec: 84.19 - lr: 0.062500 2021-03-26 07:00:49,791 epoch 42 - iter 15/50 - loss 2.58148971 - samples/sec: 71.13 - lr: 0.062500 2021-03-26 07:00:51,879 epoch 42 - iter 20/50 - loss 2.66884552 - samples/sec: 76.72 - lr: 0.062500 2021-03-26 07:00:53,852 epoch 42 - iter 25/50 - loss 2.65866809 - samples/sec: 81.15 - lr: 0.062500 2021-03-26 07:00:55,757 epoch 42 - iter 30/50 - loss 2.65247952 - samples/sec: 84.11 - lr: 0.062500 2021-03-26 07:00:57,712 epoch 42 - iter 35/50 - loss 2.66691832 - samples/sec: 81.89 - lr: 0.062500 2021-03-26 07:00:59,609 epoch 42 - iter 40/50 - loss 2.69011677 - samples/sec: 84.47 - lr: 0.062500 2021-03-26 07:01:01,585 epoch 42 - iter 45/50 - loss 2.70725158 - samples/sec: 81.06 - lr: 0.062500 2021-03-26 07:01:03,305 epoch 42 - iter 50/50 - loss 2.74755251 - samples/sec: 93.14 - lr: 0.062500 2021-03-26 07:01:03,306 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:01:03,306 EPOCH 42 done: loss 2.7476 - lr 0.0625000 2021-03-26 07:01:04,111 DEV : loss 6.264382839202881 - score 0.917 2021-03-26 07:01:04,135 BAD EPOCHS (no improvement): 4 2021-03-26 07:01:04,136 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:01:06,096 epoch 43 - iter 5/50 - loss 2.62440243 - samples/sec: 81.72 - lr: 0.031250 2021-03-26 07:01:08,236 epoch 43 - iter 10/50 - loss 2.86344099 - samples/sec: 74.87 - lr: 0.031250 2021-03-26 07:01:10,153 epoch 43 - iter 15/50 - loss 2.81902952 - samples/sec: 83.53 - lr: 0.031250 2021-03-26 07:01:12,006 epoch 43 - iter 20/50 - loss 2.73979611 - samples/sec: 86.45 - lr: 0.031250 2021-03-26 07:01:13,986 epoch 43 - iter 25/50 - loss 2.61896025 - samples/sec: 80.89 - lr: 0.031250 2021-03-26 07:01:15,972 epoch 43 - iter 30/50 - loss 2.68594955 - samples/sec: 80.61 - lr: 0.031250 2021-03-26 07:01:18,047 epoch 43 - iter 35/50 - loss 2.70499782 - samples/sec: 77.19 - lr: 0.031250 2021-03-26 07:01:20,037 epoch 43 - iter 40/50 - loss 2.66383172 - samples/sec: 80.45 - lr: 0.031250 2021-03-26 07:01:21,974 epoch 43 - iter 45/50 - loss 2.66746508 - samples/sec: 82.67 - lr: 0.031250 2021-03-26 07:01:23,966 epoch 43 - iter 50/50 - loss 2.67940281 - samples/sec: 80.39 - lr: 0.031250 2021-03-26 07:01:23,967 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:01:23,968 EPOCH 43 done: loss 2.6794 - lr 0.0312500 2021-03-26 07:01:24,820 DEV : loss 6.249972820281982 - score 0.9186 2021-03-26 07:01:24,855 BAD EPOCHS (no improvement): 1 2021-03-26 07:01:24,855 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:01:27,155 epoch 44 - iter 5/50 - loss 2.88372693 - samples/sec: 69.63 - lr: 0.031250 2021-03-26 07:01:29,087 epoch 44 - iter 10/50 - loss 2.74055066 - samples/sec: 82.92 - lr: 0.031250 2021-03-26 07:01:31,005 epoch 44 - iter 15/50 - loss 2.69428252 - samples/sec: 83.48 - lr: 0.031250 2021-03-26 07:01:33,184 epoch 44 - iter 20/50 - loss 2.59563210 - samples/sec: 73.47 - lr: 0.031250 2021-03-26 07:01:35,087 epoch 44 - iter 25/50 - loss 2.51464007 - samples/sec: 84.19 - lr: 0.031250 2021-03-26 07:01:36,900 epoch 44 - iter 30/50 - loss 2.48493527 - samples/sec: 88.38 - lr: 0.031250 2021-03-26 07:01:38,948 epoch 44 - iter 35/50 - loss 2.50545301 - samples/sec: 78.18 - lr: 0.031250 2021-03-26 07:01:41,152 epoch 44 - iter 40/50 - loss 2.50784069 - samples/sec: 72.66 - lr: 0.031250 2021-03-26 07:01:43,004 epoch 44 - iter 45/50 - loss 2.53496263 - samples/sec: 86.51 - lr: 0.031250 2021-03-26 07:01:44,704 epoch 44 - iter 50/50 - loss 2.51157553 - samples/sec: 94.19 - lr: 0.031250 2021-03-26 07:01:44,705 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:01:44,706 EPOCH 44 done: loss 2.5116 - lr 0.0312500 2021-03-26 07:01:45,546 DEV : loss 6.207180023193359 - score 0.9186 2021-03-26 07:01:45,578 BAD EPOCHS (no improvement): 2 2021-03-26 07:01:45,578 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:01:47,291 epoch 45 - iter 5/50 - loss 2.37124238 - samples/sec: 93.49 - lr: 0.031250 2021-03-26 07:01:49,119 epoch 45 - iter 10/50 - loss 2.76336849 - samples/sec: 87.65 - lr: 0.031250 2021-03-26 07:01:51,108 epoch 45 - iter 15/50 - loss 2.62719330 - samples/sec: 80.52 - lr: 0.031250 2021-03-26 07:01:53,185 epoch 45 - iter 20/50 - loss 2.60141521 - samples/sec: 77.08 - lr: 0.031250 2021-03-26 07:01:55,152 epoch 45 - iter 25/50 - loss 2.53654681 - samples/sec: 81.41 - lr: 0.031250 2021-03-26 07:01:57,178 epoch 45 - iter 30/50 - loss 2.52064539 - samples/sec: 79.05 - lr: 0.031250 2021-03-26 07:01:59,147 epoch 45 - iter 35/50 - loss 2.49631712 - samples/sec: 81.32 - lr: 0.031250 2021-03-26 07:02:01,260 epoch 45 - iter 40/50 - loss 2.51912695 - samples/sec: 75.83 - lr: 0.031250 2021-03-26 07:02:03,368 epoch 45 - iter 45/50 - loss 2.55996369 - samples/sec: 75.93 - lr: 0.031250 2021-03-26 07:02:05,176 epoch 45 - iter 50/50 - loss 2.54872012 - samples/sec: 88.59 - lr: 0.031250 2021-03-26 07:02:05,176 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:02:05,177 EPOCH 45 done: loss 2.5487 - lr 0.0312500 2021-03-26 07:02:05,990 DEV : loss 6.239845275878906 - score 0.9178 2021-03-26 07:02:06,018 BAD EPOCHS (no improvement): 3 2021-03-26 07:02:06,019 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:02:07,934 epoch 46 - iter 5/50 - loss 2.08215249 - samples/sec: 83.63 - lr: 0.031250 2021-03-26 07:02:09,916 epoch 46 - iter 10/50 - loss 2.39230992 - samples/sec: 80.81 - lr: 0.031250 2021-03-26 07:02:11,888 epoch 46 - iter 15/50 - loss 2.51131487 - samples/sec: 81.21 - lr: 0.031250 2021-03-26 07:02:13,878 epoch 46 - iter 20/50 - loss 2.56332803 - samples/sec: 80.47 - lr: 0.031250 2021-03-26 07:02:15,898 epoch 46 - iter 25/50 - loss 2.58750481 - samples/sec: 79.24 - lr: 0.031250 2021-03-26 07:02:18,080 epoch 46 - iter 30/50 - loss 2.56808805 - samples/sec: 73.39 - lr: 0.031250 2021-03-26 07:02:20,000 epoch 46 - iter 35/50 - loss 2.59616448 - samples/sec: 83.43 - lr: 0.031250 2021-03-26 07:02:21,798 epoch 46 - iter 40/50 - loss 2.54724448 - samples/sec: 89.11 - lr: 0.031250 2021-03-26 07:02:23,840 epoch 46 - iter 45/50 - loss 2.60226175 - samples/sec: 78.42 - lr: 0.031250 2021-03-26 07:02:25,524 epoch 46 - iter 50/50 - loss 2.62334806 - samples/sec: 95.10 - lr: 0.031250 2021-03-26 07:02:25,525 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:02:25,526 EPOCH 46 done: loss 2.6233 - lr 0.0312500 2021-03-26 07:02:26,305 DEV : loss 6.2371320724487305 - score 0.9186 2021-03-26 07:02:26,330 BAD EPOCHS (no improvement): 4 2021-03-26 07:02:26,331 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:02:28,374 epoch 47 - iter 5/50 - loss 2.41000776 - samples/sec: 78.39 - lr: 0.015625 2021-03-26 07:02:30,413 epoch 47 - iter 10/50 - loss 2.40392948 - samples/sec: 78.56 - lr: 0.015625 2021-03-26 07:02:32,296 epoch 47 - iter 15/50 - loss 2.49342973 - samples/sec: 85.04 - lr: 0.015625 2021-03-26 07:02:34,113 epoch 47 - iter 20/50 - loss 2.41740447 - samples/sec: 88.16 - lr: 0.015625 2021-03-26 07:02:35,975 epoch 47 - iter 25/50 - loss 2.43529265 - samples/sec: 86.01 - lr: 0.015625 2021-03-26 07:02:38,034 epoch 47 - iter 30/50 - loss 2.50073948 - samples/sec: 77.74 - lr: 0.015625 2021-03-26 07:02:40,083 epoch 47 - iter 35/50 - loss 2.48987853 - samples/sec: 78.17 - lr: 0.015625 2021-03-26 07:02:41,987 epoch 47 - iter 40/50 - loss 2.52052880 - samples/sec: 84.12 - lr: 0.015625 2021-03-26 07:02:43,994 epoch 47 - iter 45/50 - loss 2.52663625 - samples/sec: 79.79 - lr: 0.015625 2021-03-26 07:02:45,779 epoch 47 - iter 50/50 - loss 2.56214527 - samples/sec: 89.70 - lr: 0.015625 2021-03-26 07:02:45,780 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:02:45,780 EPOCH 47 done: loss 2.5621 - lr 0.0156250 2021-03-26 07:02:46,552 DEV : loss 6.216527462005615 - score 0.9182 2021-03-26 07:02:46,577 BAD EPOCHS (no improvement): 1 2021-03-26 07:02:46,577 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:02:48,736 epoch 48 - iter 5/50 - loss 2.62342062 - samples/sec: 74.16 - lr: 0.015625 2021-03-26 07:02:50,661 epoch 48 - iter 10/50 - loss 2.50259314 - samples/sec: 83.22 - lr: 0.015625 2021-03-26 07:02:52,640 epoch 48 - iter 15/50 - loss 2.41186643 - samples/sec: 80.92 - lr: 0.015625 2021-03-26 07:02:54,835 epoch 48 - iter 20/50 - loss 2.50129139 - samples/sec: 72.96 - lr: 0.015625 2021-03-26 07:02:56,807 epoch 48 - iter 25/50 - loss 2.53100896 - samples/sec: 81.22 - lr: 0.015625 2021-03-26 07:02:58,817 epoch 48 - iter 30/50 - loss 2.56003957 - samples/sec: 79.65 - lr: 0.015625 2021-03-26 07:03:00,548 epoch 48 - iter 35/50 - loss 2.57986932 - samples/sec: 92.56 - lr: 0.015625 2021-03-26 07:03:02,387 epoch 48 - iter 40/50 - loss 2.54486949 - samples/sec: 87.06 - lr: 0.015625 2021-03-26 07:03:04,379 epoch 48 - iter 45/50 - loss 2.50048207 - samples/sec: 80.40 - lr: 0.015625 2021-03-26 07:03:06,137 epoch 48 - iter 50/50 - loss 2.47580104 - samples/sec: 91.09 - lr: 0.015625 2021-03-26 07:03:06,138 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:03:06,138 EPOCH 48 done: loss 2.4758 - lr 0.0156250 2021-03-26 07:03:06,958 DEV : loss 6.240680694580078 - score 0.9178 2021-03-26 07:03:06,980 BAD EPOCHS (no improvement): 2 2021-03-26 07:03:06,980 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:03:09,284 epoch 49 - iter 5/50 - loss 2.28204064 - samples/sec: 69.50 - lr: 0.015625 2021-03-26 07:03:11,353 epoch 49 - iter 10/50 - loss 2.35861111 - samples/sec: 77.45 - lr: 0.015625 2021-03-26 07:03:13,375 epoch 49 - iter 15/50 - loss 2.40004193 - samples/sec: 79.18 - lr: 0.015625 2021-03-26 07:03:15,352 epoch 49 - iter 20/50 - loss 2.59549707 - samples/sec: 81.01 - lr: 0.015625 2021-03-26 07:03:17,290 epoch 49 - iter 25/50 - loss 2.60552569 - samples/sec: 82.64 - lr: 0.015625 2021-03-26 07:03:19,361 epoch 49 - iter 30/50 - loss 2.58463446 - samples/sec: 77.34 - lr: 0.015625 2021-03-26 07:03:21,146 epoch 49 - iter 35/50 - loss 2.56579349 - samples/sec: 89.72 - lr: 0.015625 2021-03-26 07:03:23,055 epoch 49 - iter 40/50 - loss 2.57229652 - samples/sec: 83.90 - lr: 0.015625 2021-03-26 07:03:25,090 epoch 49 - iter 45/50 - loss 2.51541369 - samples/sec: 78.69 - lr: 0.015625 2021-03-26 07:03:26,932 epoch 49 - iter 50/50 - loss 2.54103005 - samples/sec: 86.93 - lr: 0.015625 2021-03-26 07:03:26,933 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:03:26,933 EPOCH 49 done: loss 2.5410 - lr 0.0156250 2021-03-26 07:03:27,705 DEV : loss 6.239748954772949 - score 0.917 2021-03-26 07:03:27,730 BAD EPOCHS (no improvement): 3 2021-03-26 07:03:27,731 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:03:29,696 epoch 50 - iter 5/50 - loss 2.28539340 - samples/sec: 81.51 - lr: 0.015625 2021-03-26 07:03:31,503 epoch 50 - iter 10/50 - loss 2.48737829 - samples/sec: 88.64 - lr: 0.015625 2021-03-26 07:03:33,520 epoch 50 - iter 15/50 - loss 2.59285909 - samples/sec: 79.41 - lr: 0.015625 2021-03-26 07:03:35,439 epoch 50 - iter 20/50 - loss 2.56907600 - samples/sec: 83.43 - lr: 0.015625 2021-03-26 07:03:37,519 epoch 50 - iter 25/50 - loss 2.49343590 - samples/sec: 77.02 - lr: 0.015625 2021-03-26 07:03:39,594 epoch 50 - iter 30/50 - loss 2.54235819 - samples/sec: 77.17 - lr: 0.015625 2021-03-26 07:03:41,490 epoch 50 - iter 35/50 - loss 2.49685275 - samples/sec: 84.47 - lr: 0.015625 2021-03-26 07:03:43,392 epoch 50 - iter 40/50 - loss 2.48172221 - samples/sec: 84.19 - lr: 0.015625 2021-03-26 07:03:45,403 epoch 50 - iter 45/50 - loss 2.52431675 - samples/sec: 79.63 - lr: 0.015625 2021-03-26 07:03:47,179 epoch 50 - iter 50/50 - loss 2.48997502 - samples/sec: 90.19 - lr: 0.015625 2021-03-26 07:03:47,179 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:03:47,179 EPOCH 50 done: loss 2.4900 - lr 0.0156250 2021-03-26 07:03:47,947 DEV : loss 6.231075286865234 - score 0.917 2021-03-26 07:03:47,971 BAD EPOCHS (no improvement): 4 2021-03-26 07:03:47,972 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:03:49,943 epoch 51 - iter 5/50 - loss 2.58739772 - samples/sec: 81.26 - lr: 0.007812 2021-03-26 07:03:51,900 epoch 51 - iter 10/50 - loss 2.75475857 - samples/sec: 81.82 - lr: 0.007812 2021-03-26 07:03:53,847 epoch 51 - iter 15/50 - loss 2.64987794 - samples/sec: 82.25 - lr: 0.007812 2021-03-26 07:03:55,996 epoch 51 - iter 20/50 - loss 2.58137577 - samples/sec: 74.53 - lr: 0.007812 2021-03-26 07:03:58,032 epoch 51 - iter 25/50 - loss 2.63140382 - samples/sec: 78.68 - lr: 0.007812 2021-03-26 07:03:59,964 epoch 51 - iter 30/50 - loss 2.57393547 - samples/sec: 82.87 - lr: 0.007812 2021-03-26 07:04:01,820 epoch 51 - iter 35/50 - loss 2.55677542 - samples/sec: 86.33 - lr: 0.007812 2021-03-26 07:04:03,802 epoch 51 - iter 40/50 - loss 2.55924032 - samples/sec: 80.81 - lr: 0.007812 2021-03-26 07:04:05,813 epoch 51 - iter 45/50 - loss 2.55842340 - samples/sec: 79.66 - lr: 0.007812 2021-03-26 07:04:07,493 epoch 51 - iter 50/50 - loss 2.51336263 - samples/sec: 95.37 - lr: 0.007812 2021-03-26 07:04:07,494 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:04:07,494 EPOCH 51 done: loss 2.5134 - lr 0.0078125 2021-03-26 07:04:08,281 DEV : loss 6.22976016998291 - score 0.9174 2021-03-26 07:04:08,306 BAD EPOCHS (no improvement): 1 2021-03-26 07:04:08,307 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:04:10,185 epoch 52 - iter 5/50 - loss 2.12111568 - samples/sec: 85.30 - lr: 0.007812 2021-03-26 07:04:12,259 epoch 52 - iter 10/50 - loss 2.28783898 - samples/sec: 77.20 - lr: 0.007812 2021-03-26 07:04:14,173 epoch 52 - iter 15/50 - loss 2.27034943 - samples/sec: 83.65 - lr: 0.007812 2021-03-26 07:04:15,945 epoch 52 - iter 20/50 - loss 2.35835313 - samples/sec: 90.39 - lr: 0.007812 2021-03-26 07:04:17,875 epoch 52 - iter 25/50 - loss 2.32353603 - samples/sec: 82.97 - lr: 0.007812 2021-03-26 07:04:19,849 epoch 52 - iter 30/50 - loss 2.36913104 - samples/sec: 81.15 - lr: 0.007812 2021-03-26 07:04:21,828 epoch 52 - iter 35/50 - loss 2.35468242 - samples/sec: 80.91 - lr: 0.007812 2021-03-26 07:04:23,653 epoch 52 - iter 40/50 - loss 2.32059786 - samples/sec: 87.77 - lr: 0.007812 2021-03-26 07:04:25,634 epoch 52 - iter 45/50 - loss 2.38650590 - samples/sec: 80.83 - lr: 0.007812 2021-03-26 07:04:27,572 epoch 52 - iter 50/50 - loss 2.41335446 - samples/sec: 82.62 - lr: 0.007812 2021-03-26 07:04:27,573 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:04:27,573 EPOCH 52 done: loss 2.4134 - lr 0.0078125 2021-03-26 07:04:28,309 DEV : loss 6.234460353851318 - score 0.9168 2021-03-26 07:04:28,333 BAD EPOCHS (no improvement): 2 2021-03-26 07:04:28,334 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:04:30,365 epoch 53 - iter 5/50 - loss 2.29304843 - samples/sec: 78.86 - lr: 0.007812 2021-03-26 07:04:32,302 epoch 53 - iter 10/50 - loss 2.36527128 - samples/sec: 82.68 - lr: 0.007812 2021-03-26 07:04:34,216 epoch 53 - iter 15/50 - loss 2.42961562 - samples/sec: 84.15 - lr: 0.007812 2021-03-26 07:04:36,211 epoch 53 - iter 20/50 - loss 2.49262825 - samples/sec: 82.64 - lr: 0.007812 2021-03-26 07:04:38,041 epoch 53 - iter 25/50 - loss 2.42180495 - samples/sec: 87.49 - lr: 0.007812 2021-03-26 07:04:39,948 epoch 53 - iter 30/50 - loss 2.48365995 - samples/sec: 83.98 - lr: 0.007812 2021-03-26 07:04:41,756 epoch 53 - iter 35/50 - loss 2.49418128 - samples/sec: 88.59 - lr: 0.007812 2021-03-26 07:04:43,746 epoch 53 - iter 40/50 - loss 2.52012618 - samples/sec: 80.48 - lr: 0.007812 2021-03-26 07:04:45,773 epoch 53 - iter 45/50 - loss 2.51719693 - samples/sec: 79.01 - lr: 0.007812 2021-03-26 07:04:47,665 epoch 53 - iter 50/50 - loss 2.44318517 - samples/sec: 84.65 - lr: 0.007812 2021-03-26 07:04:47,666 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:04:47,667 EPOCH 53 done: loss 2.4432 - lr 0.0078125 2021-03-26 07:04:48,427 DEV : loss 6.224849700927734 - score 0.9174 2021-03-26 07:04:48,452 BAD EPOCHS (no improvement): 3 2021-03-26 07:04:48,453 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:04:50,426 epoch 54 - iter 5/50 - loss 2.27110686 - samples/sec: 81.16 - lr: 0.007812 2021-03-26 07:04:52,346 epoch 54 - iter 10/50 - loss 2.36324615 - samples/sec: 83.42 - lr: 0.007812 2021-03-26 07:04:54,189 epoch 54 - iter 15/50 - loss 2.39531080 - samples/sec: 86.89 - lr: 0.007812 2021-03-26 07:04:56,261 epoch 54 - iter 20/50 - loss 2.34100553 - samples/sec: 77.30 - lr: 0.007812 2021-03-26 07:04:58,123 epoch 54 - iter 25/50 - loss 2.32279901 - samples/sec: 86.03 - lr: 0.007812 2021-03-26 07:04:59,964 epoch 54 - iter 30/50 - loss 2.30034845 - samples/sec: 86.99 - lr: 0.007812 2021-03-26 07:05:01,816 epoch 54 - iter 35/50 - loss 2.32203776 - samples/sec: 86.51 - lr: 0.007812 2021-03-26 07:05:03,781 epoch 54 - iter 40/50 - loss 2.35916250 - samples/sec: 81.48 - lr: 0.007812 2021-03-26 07:05:05,783 epoch 54 - iter 45/50 - loss 2.31716502 - samples/sec: 80.00 - lr: 0.007812 2021-03-26 07:05:07,576 epoch 54 - iter 50/50 - loss 2.39397667 - samples/sec: 89.33 - lr: 0.007812 2021-03-26 07:05:07,576 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:05:07,577 EPOCH 54 done: loss 2.3940 - lr 0.0078125 2021-03-26 07:05:08,342 DEV : loss 6.229462623596191 - score 0.9174 2021-03-26 07:05:08,368 BAD EPOCHS (no improvement): 4 2021-03-26 07:05:08,368 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:05:10,228 epoch 55 - iter 5/50 - loss 2.43591893 - samples/sec: 86.12 - lr: 0.003906 2021-03-26 07:05:12,297 epoch 55 - iter 10/50 - loss 2.29127170 - samples/sec: 77.40 - lr: 0.003906 2021-03-26 07:05:14,303 epoch 55 - iter 15/50 - loss 2.29653165 - samples/sec: 79.82 - lr: 0.003906 2021-03-26 07:05:16,052 epoch 55 - iter 20/50 - loss 2.30765195 - samples/sec: 91.57 - lr: 0.003906 2021-03-26 07:05:17,788 epoch 55 - iter 25/50 - loss 2.35338634 - samples/sec: 92.31 - lr: 0.003906 2021-03-26 07:05:19,830 epoch 55 - iter 30/50 - loss 2.39339325 - samples/sec: 78.42 - lr: 0.003906 2021-03-26 07:05:21,748 epoch 55 - iter 35/50 - loss 2.35602188 - samples/sec: 83.50 - lr: 0.003906 2021-03-26 07:05:23,679 epoch 55 - iter 40/50 - loss 2.36002087 - samples/sec: 82.94 - lr: 0.003906 2021-03-26 07:05:25,625 epoch 55 - iter 45/50 - loss 2.38230093 - samples/sec: 82.29 - lr: 0.003906 2021-03-26 07:05:27,669 epoch 55 - iter 50/50 - loss 2.42242070 - samples/sec: 78.35 - lr: 0.003906 2021-03-26 07:05:27,670 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:05:27,670 EPOCH 55 done: loss 2.4224 - lr 0.0039062 2021-03-26 07:05:28,484 DEV : loss 6.228795528411865 - score 0.9174 2021-03-26 07:05:28,508 BAD EPOCHS (no improvement): 1 2021-03-26 07:05:28,509 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:05:30,536 epoch 56 - iter 5/50 - loss 2.19912152 - samples/sec: 79.06 - lr: 0.003906 2021-03-26 07:05:32,433 epoch 56 - iter 10/50 - loss 2.17522593 - samples/sec: 84.42 - lr: 0.003906 2021-03-26 07:05:34,525 epoch 56 - iter 15/50 - loss 2.18695093 - samples/sec: 76.55 - lr: 0.003906 2021-03-26 07:05:36,694 epoch 56 - iter 20/50 - loss 2.24990177 - samples/sec: 73.82 - lr: 0.003906 2021-03-26 07:05:38,712 epoch 56 - iter 25/50 - loss 2.33317802 - samples/sec: 79.36 - lr: 0.003906 2021-03-26 07:05:40,630 epoch 56 - iter 30/50 - loss 2.36618576 - samples/sec: 83.54 - lr: 0.003906 2021-03-26 07:05:42,494 epoch 56 - iter 35/50 - loss 2.34973713 - samples/sec: 85.89 - lr: 0.003906 2021-03-26 07:05:44,488 epoch 56 - iter 40/50 - loss 2.37181876 - samples/sec: 80.34 - lr: 0.003906 2021-03-26 07:05:46,501 epoch 56 - iter 45/50 - loss 2.38742908 - samples/sec: 79.55 - lr: 0.003906 2021-03-26 07:05:48,183 epoch 56 - iter 50/50 - loss 2.36818949 - samples/sec: 95.22 - lr: 0.003906 2021-03-26 07:05:48,184 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:05:48,185 EPOCH 56 done: loss 2.3682 - lr 0.0039062 2021-03-26 07:05:49,075 DEV : loss 6.230513572692871 - score 0.9174 2021-03-26 07:05:49,102 BAD EPOCHS (no improvement): 2 2021-03-26 07:05:49,103 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:05:51,070 epoch 57 - iter 5/50 - loss 2.74660211 - samples/sec: 81.45 - lr: 0.003906 2021-03-26 07:05:53,166 epoch 57 - iter 10/50 - loss 2.75735655 - samples/sec: 76.42 - lr: 0.003906 2021-03-26 07:05:55,067 epoch 57 - iter 15/50 - loss 2.63150340 - samples/sec: 84.25 - lr: 0.003906 2021-03-26 07:05:56,888 epoch 57 - iter 20/50 - loss 2.65938060 - samples/sec: 87.96 - lr: 0.003906 2021-03-26 07:05:58,773 epoch 57 - iter 25/50 - loss 2.65044247 - samples/sec: 84.96 - lr: 0.003906 2021-03-26 07:06:00,679 epoch 57 - iter 30/50 - loss 2.63459703 - samples/sec: 84.03 - lr: 0.003906 2021-03-26 07:06:02,698 epoch 57 - iter 35/50 - loss 2.60287085 - samples/sec: 79.28 - lr: 0.003906 2021-03-26 07:06:04,687 epoch 57 - iter 40/50 - loss 2.59769873 - samples/sec: 80.54 - lr: 0.003906 2021-03-26 07:06:06,605 epoch 57 - iter 45/50 - loss 2.61791177 - samples/sec: 83.51 - lr: 0.003906 2021-03-26 07:06:08,472 epoch 57 - iter 50/50 - loss 2.57899422 - samples/sec: 85.76 - lr: 0.003906 2021-03-26 07:06:08,473 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:06:08,473 EPOCH 57 done: loss 2.5790 - lr 0.0039062 2021-03-26 07:06:09,227 DEV : loss 6.234955787658691 - score 0.9174 2021-03-26 07:06:09,252 BAD EPOCHS (no improvement): 3 2021-03-26 07:06:09,252 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:06:11,192 epoch 58 - iter 5/50 - loss 2.11452603 - samples/sec: 82.58 - lr: 0.003906 2021-03-26 07:06:13,175 epoch 58 - iter 10/50 - loss 2.48117228 - samples/sec: 80.77 - lr: 0.003906 2021-03-26 07:06:14,938 epoch 58 - iter 15/50 - loss 2.51002020 - samples/sec: 90.85 - lr: 0.003906 2021-03-26 07:06:16,856 epoch 58 - iter 20/50 - loss 2.51059042 - samples/sec: 83.48 - lr: 0.003906 2021-03-26 07:06:18,958 epoch 58 - iter 25/50 - loss 2.51444347 - samples/sec: 76.18 - lr: 0.003906 2021-03-26 07:06:20,793 epoch 58 - iter 30/50 - loss 2.46463509 - samples/sec: 87.30 - lr: 0.003906 2021-03-26 07:06:22,700 epoch 58 - iter 35/50 - loss 2.54111838 - samples/sec: 84.01 - lr: 0.003906 2021-03-26 07:06:24,620 epoch 58 - iter 40/50 - loss 2.48917502 - samples/sec: 83.44 - lr: 0.003906 2021-03-26 07:06:26,621 epoch 58 - iter 45/50 - loss 2.50563401 - samples/sec: 80.04 - lr: 0.003906 2021-03-26 07:06:28,525 epoch 58 - iter 50/50 - loss 2.51603883 - samples/sec: 84.14 - lr: 0.003906 2021-03-26 07:06:28,525 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:06:28,526 EPOCH 58 done: loss 2.5160 - lr 0.0039062 2021-03-26 07:06:29,281 DEV : loss 6.235045909881592 - score 0.9174 2021-03-26 07:06:29,304 BAD EPOCHS (no improvement): 4 2021-03-26 07:06:29,305 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:06:31,310 epoch 59 - iter 5/50 - loss 2.58779755 - samples/sec: 79.89 - lr: 0.001953 2021-03-26 07:06:33,249 epoch 59 - iter 10/50 - loss 2.31887306 - samples/sec: 82.58 - lr: 0.001953 2021-03-26 07:06:35,243 epoch 59 - iter 15/50 - loss 2.39448693 - samples/sec: 80.34 - lr: 0.001953 2021-03-26 07:06:37,895 epoch 59 - iter 20/50 - loss 2.40186156 - samples/sec: 60.37 - lr: 0.001953 2021-03-26 07:06:40,190 epoch 59 - iter 25/50 - loss 2.38810690 - samples/sec: 69.76 - lr: 0.001953 2021-03-26 07:06:41,923 epoch 59 - iter 30/50 - loss 2.42196012 - samples/sec: 92.44 - lr: 0.001953 2021-03-26 07:06:43,954 epoch 59 - iter 35/50 - loss 2.43881528 - samples/sec: 78.85 - lr: 0.001953 2021-03-26 07:06:46,001 epoch 59 - iter 40/50 - loss 2.42435353 - samples/sec: 78.23 - lr: 0.001953 2021-03-26 07:06:48,009 epoch 59 - iter 45/50 - loss 2.41908374 - samples/sec: 79.76 - lr: 0.001953 2021-03-26 07:06:49,660 epoch 59 - iter 50/50 - loss 2.39769382 - samples/sec: 96.98 - lr: 0.001953 2021-03-26 07:06:49,661 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:06:49,661 EPOCH 59 done: loss 2.3977 - lr 0.0019531 2021-03-26 07:06:50,422 DEV : loss 6.2370758056640625 - score 0.917 2021-03-26 07:06:50,446 BAD EPOCHS (no improvement): 1 2021-03-26 07:06:50,447 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:06:52,362 epoch 60 - iter 5/50 - loss 2.21496704 - samples/sec: 83.65 - lr: 0.001953 2021-03-26 07:06:54,187 epoch 60 - iter 10/50 - loss 2.42700636 - samples/sec: 87.78 - lr: 0.001953 2021-03-26 07:06:56,086 epoch 60 - iter 15/50 - loss 2.53753303 - samples/sec: 84.35 - lr: 0.001953 2021-03-26 07:06:58,182 epoch 60 - iter 20/50 - loss 2.60033162 - samples/sec: 76.40 - lr: 0.001953 2021-03-26 07:07:00,224 epoch 60 - iter 25/50 - loss 2.51529428 - samples/sec: 78.42 - lr: 0.001953 2021-03-26 07:07:02,296 epoch 60 - iter 30/50 - loss 2.49559573 - samples/sec: 77.28 - lr: 0.001953 2021-03-26 07:07:04,332 epoch 60 - iter 35/50 - loss 2.51090207 - samples/sec: 78.66 - lr: 0.001953 2021-03-26 07:07:06,404 epoch 60 - iter 40/50 - loss 2.51477104 - samples/sec: 77.30 - lr: 0.001953 2021-03-26 07:07:08,233 epoch 60 - iter 45/50 - loss 2.50222202 - samples/sec: 87.58 - lr: 0.001953 2021-03-26 07:07:09,957 epoch 60 - iter 50/50 - loss 2.44377190 - samples/sec: 92.90 - lr: 0.001953 2021-03-26 07:07:09,957 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:07:09,958 EPOCH 60 done: loss 2.4438 - lr 0.0019531 2021-03-26 07:07:10,699 DEV : loss 6.238881587982178 - score 0.9172 2021-03-26 07:07:10,723 BAD EPOCHS (no improvement): 2 2021-03-26 07:07:10,724 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:07:12,525 epoch 61 - iter 5/50 - loss 2.12126102 - samples/sec: 88.95 - lr: 0.001953 2021-03-26 07:07:14,428 epoch 61 - iter 10/50 - loss 2.54890535 - samples/sec: 84.16 - lr: 0.001953 2021-03-26 07:07:16,408 epoch 61 - iter 15/50 - loss 2.51401180 - samples/sec: 80.91 - lr: 0.001953 2021-03-26 07:07:18,503 epoch 61 - iter 20/50 - loss 2.49502721 - samples/sec: 76.41 - lr: 0.001953 2021-03-26 07:07:20,446 epoch 61 - iter 25/50 - loss 2.57242790 - samples/sec: 82.49 - lr: 0.001953 2021-03-26 07:07:22,349 epoch 61 - iter 30/50 - loss 2.60423574 - samples/sec: 84.13 - lr: 0.001953 2021-03-26 07:07:24,326 epoch 61 - iter 35/50 - loss 2.54501983 - samples/sec: 81.05 - lr: 0.001953 2021-03-26 07:07:26,422 epoch 61 - iter 40/50 - loss 2.52295736 - samples/sec: 76.37 - lr: 0.001953 2021-03-26 07:07:28,231 epoch 61 - iter 45/50 - loss 2.53937517 - samples/sec: 88.56 - lr: 0.001953 2021-03-26 07:07:30,169 epoch 61 - iter 50/50 - loss 2.51024843 - samples/sec: 82.65 - lr: 0.001953 2021-03-26 07:07:30,170 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:07:30,170 EPOCH 61 done: loss 2.5102 - lr 0.0019531 2021-03-26 07:07:30,921 DEV : loss 6.2310614585876465 - score 0.917 2021-03-26 07:07:30,945 BAD EPOCHS (no improvement): 3 2021-03-26 07:07:30,946 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:07:33,212 epoch 62 - iter 5/50 - loss 2.69974794 - samples/sec: 70.66 - lr: 0.001953 2021-03-26 07:07:35,175 epoch 62 - iter 10/50 - loss 2.58760433 - samples/sec: 81.58 - lr: 0.001953 2021-03-26 07:07:37,300 epoch 62 - iter 15/50 - loss 2.45208565 - samples/sec: 75.38 - lr: 0.001953 2021-03-26 07:07:39,445 epoch 62 - iter 20/50 - loss 2.47846313 - samples/sec: 74.67 - lr: 0.001953 2021-03-26 07:07:41,524 epoch 62 - iter 25/50 - loss 2.44997159 - samples/sec: 76.99 - lr: 0.001953 2021-03-26 07:07:43,774 epoch 62 - iter 30/50 - loss 2.38889544 - samples/sec: 71.20 - lr: 0.001953 2021-03-26 07:07:45,679 epoch 62 - iter 35/50 - loss 2.36844786 - samples/sec: 84.05 - lr: 0.001953 2021-03-26 07:07:47,775 epoch 62 - iter 40/50 - loss 2.38948717 - samples/sec: 76.41 - lr: 0.001953 2021-03-26 07:07:49,904 epoch 62 - iter 45/50 - loss 2.37542345 - samples/sec: 75.19 - lr: 0.001953 2021-03-26 07:07:51,797 epoch 62 - iter 50/50 - loss 2.37077458 - samples/sec: 84.61 - lr: 0.001953 2021-03-26 07:07:51,798 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:07:51,799 EPOCH 62 done: loss 2.3708 - lr 0.0019531 2021-03-26 07:07:52,608 DEV : loss 6.230165004730225 - score 0.9182 2021-03-26 07:07:52,634 BAD EPOCHS (no improvement): 4 2021-03-26 07:07:52,635 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:07:54,592 epoch 63 - iter 5/50 - loss 2.13501656 - samples/sec: 81.85 - lr: 0.000977 2021-03-26 07:07:56,438 epoch 63 - iter 10/50 - loss 2.24644828 - samples/sec: 86.76 - lr: 0.000977 2021-03-26 07:07:58,325 epoch 63 - iter 15/50 - loss 2.22856548 - samples/sec: 84.85 - lr: 0.000977 2021-03-26 07:08:00,141 epoch 63 - iter 20/50 - loss 2.17708767 - samples/sec: 88.23 - lr: 0.000977 2021-03-26 07:08:02,279 epoch 63 - iter 25/50 - loss 2.24079601 - samples/sec: 74.87 - lr: 0.000977 2021-03-26 07:08:04,266 epoch 63 - iter 30/50 - loss 2.28632747 - samples/sec: 80.59 - lr: 0.000977 2021-03-26 07:08:06,002 epoch 63 - iter 35/50 - loss 2.30275180 - samples/sec: 92.26 - lr: 0.000977 2021-03-26 07:08:07,813 epoch 63 - iter 40/50 - loss 2.37092484 - samples/sec: 88.44 - lr: 0.000977 2021-03-26 07:08:09,693 epoch 63 - iter 45/50 - loss 2.41254362 - samples/sec: 85.23 - lr: 0.000977 2021-03-26 07:08:11,485 epoch 63 - iter 50/50 - loss 2.44104299 - samples/sec: 89.36 - lr: 0.000977 2021-03-26 07:08:11,485 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:08:11,486 EPOCH 63 done: loss 2.4410 - lr 0.0009766 2021-03-26 07:08:12,242 DEV : loss 6.228281021118164 - score 0.9186 2021-03-26 07:08:12,267 BAD EPOCHS (no improvement): 1 2021-03-26 07:08:12,268 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:08:14,289 epoch 64 - iter 5/50 - loss 2.34046373 - samples/sec: 79.26 - lr: 0.000977 2021-03-26 07:08:16,307 epoch 64 - iter 10/50 - loss 2.34239235 - samples/sec: 79.38 - lr: 0.000977 2021-03-26 07:08:18,078 epoch 64 - iter 15/50 - loss 2.39339416 - samples/sec: 90.42 - lr: 0.000977 2021-03-26 07:08:20,052 epoch 64 - iter 20/50 - loss 2.39595767 - samples/sec: 81.10 - lr: 0.000977 2021-03-26 07:08:22,033 epoch 64 - iter 25/50 - loss 2.36385040 - samples/sec: 80.87 - lr: 0.000977 2021-03-26 07:08:23,979 epoch 64 - iter 30/50 - loss 2.38750821 - samples/sec: 82.31 - lr: 0.000977 2021-03-26 07:08:25,919 epoch 64 - iter 35/50 - loss 2.35466352 - samples/sec: 82.54 - lr: 0.000977 2021-03-26 07:08:27,796 epoch 64 - iter 40/50 - loss 2.36880653 - samples/sec: 85.35 - lr: 0.000977 2021-03-26 07:08:29,596 epoch 64 - iter 45/50 - loss 2.35453144 - samples/sec: 88.94 - lr: 0.000977 2021-03-26 07:08:31,425 epoch 64 - iter 50/50 - loss 2.41410687 - samples/sec: 87.55 - lr: 0.000977 2021-03-26 07:08:31,426 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:08:31,427 EPOCH 64 done: loss 2.4141 - lr 0.0009766 2021-03-26 07:08:32,197 DEV : loss 6.226396560668945 - score 0.9186 2021-03-26 07:08:32,221 BAD EPOCHS (no improvement): 2 2021-03-26 07:08:32,222 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:08:34,197 epoch 65 - iter 5/50 - loss 2.37803488 - samples/sec: 81.13 - lr: 0.000977 2021-03-26 07:08:36,191 epoch 65 - iter 10/50 - loss 2.32778922 - samples/sec: 80.32 - lr: 0.000977 2021-03-26 07:08:38,083 epoch 65 - iter 15/50 - loss 2.34324029 - samples/sec: 84.66 - lr: 0.000977 2021-03-26 07:08:40,155 epoch 65 - iter 20/50 - loss 2.35554613 - samples/sec: 77.26 - lr: 0.000977 2021-03-26 07:08:42,065 epoch 65 - iter 25/50 - loss 2.43224439 - samples/sec: 83.88 - lr: 0.000977 2021-03-26 07:08:43,994 epoch 65 - iter 30/50 - loss 2.49575868 - samples/sec: 83.04 - lr: 0.000977 2021-03-26 07:08:45,917 epoch 65 - iter 35/50 - loss 2.46905589 - samples/sec: 83.30 - lr: 0.000977 2021-03-26 07:08:48,010 epoch 65 - iter 40/50 - loss 2.46037631 - samples/sec: 76.53 - lr: 0.000977 2021-03-26 07:08:49,954 epoch 65 - iter 45/50 - loss 2.51430530 - samples/sec: 82.34 - lr: 0.000977 2021-03-26 07:08:51,587 epoch 65 - iter 50/50 - loss 2.51490304 - samples/sec: 98.08 - lr: 0.000977 2021-03-26 07:08:51,588 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:08:51,589 EPOCH 65 done: loss 2.5149 - lr 0.0009766 2021-03-26 07:08:52,356 DEV : loss 6.223830223083496 - score 0.9182 2021-03-26 07:08:52,377 BAD EPOCHS (no improvement): 3 2021-03-26 07:08:52,378 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:08:54,353 epoch 66 - iter 5/50 - loss 2.32253141 - samples/sec: 81.07 - lr: 0.000977 2021-03-26 07:08:56,536 epoch 66 - iter 10/50 - loss 2.18922327 - samples/sec: 73.34 - lr: 0.000977 2021-03-26 07:08:58,433 epoch 66 - iter 15/50 - loss 2.25993318 - samples/sec: 84.45 - lr: 0.000977 2021-03-26 07:09:00,259 epoch 66 - iter 20/50 - loss 2.23364434 - samples/sec: 87.70 - lr: 0.000977 2021-03-26 07:09:02,355 epoch 66 - iter 25/50 - loss 2.26889848 - samples/sec: 76.41 - lr: 0.000977 2021-03-26 07:09:04,329 epoch 66 - iter 30/50 - loss 2.27924853 - samples/sec: 81.15 - lr: 0.000977 2021-03-26 07:09:06,233 epoch 66 - iter 35/50 - loss 2.26974100 - samples/sec: 84.09 - lr: 0.000977 2021-03-26 07:09:08,058 epoch 66 - iter 40/50 - loss 2.27447847 - samples/sec: 87.76 - lr: 0.000977 2021-03-26 07:09:09,847 epoch 66 - iter 45/50 - loss 2.25675712 - samples/sec: 89.55 - lr: 0.000977 2021-03-26 07:09:11,649 epoch 66 - iter 50/50 - loss 2.29599657 - samples/sec: 88.85 - lr: 0.000977 2021-03-26 07:09:11,650 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:09:11,650 EPOCH 66 done: loss 2.2960 - lr 0.0009766 2021-03-26 07:09:12,413 DEV : loss 6.223401069641113 - score 0.9182 2021-03-26 07:09:12,438 BAD EPOCHS (no improvement): 4 2021-03-26 07:09:12,439 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:09:14,400 epoch 67 - iter 5/50 - loss 2.42672076 - samples/sec: 81.68 - lr: 0.000488 2021-03-26 07:09:16,353 epoch 67 - iter 10/50 - loss 2.32563019 - samples/sec: 82.02 - lr: 0.000488 2021-03-26 07:09:18,545 epoch 67 - iter 15/50 - loss 2.35495213 - samples/sec: 73.06 - lr: 0.000488 2021-03-26 07:09:20,412 epoch 67 - iter 20/50 - loss 2.39149110 - samples/sec: 85.75 - lr: 0.000488 2021-03-26 07:09:22,504 epoch 67 - iter 25/50 - loss 2.39185100 - samples/sec: 76.56 - lr: 0.000488 2021-03-26 07:09:24,438 epoch 67 - iter 30/50 - loss 2.41217314 - samples/sec: 82.80 - lr: 0.000488 2021-03-26 07:09:26,420 epoch 67 - iter 35/50 - loss 2.40400888 - samples/sec: 80.81 - lr: 0.000488 2021-03-26 07:09:28,313 epoch 67 - iter 40/50 - loss 2.38190263 - samples/sec: 84.60 - lr: 0.000488 2021-03-26 07:09:30,201 epoch 67 - iter 45/50 - loss 2.38458229 - samples/sec: 84.82 - lr: 0.000488 2021-03-26 07:09:32,145 epoch 67 - iter 50/50 - loss 2.40125116 - samples/sec: 82.40 - lr: 0.000488 2021-03-26 07:09:32,145 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:09:32,146 EPOCH 67 done: loss 2.4013 - lr 0.0004883 2021-03-26 07:09:32,925 DEV : loss 6.223155498504639 - score 0.9182 2021-03-26 07:09:32,951 BAD EPOCHS (no improvement): 1 2021-03-26 07:09:32,951 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:09:35,054 epoch 68 - iter 5/50 - loss 2.63773403 - samples/sec: 76.19 - lr: 0.000488 2021-03-26 07:09:36,951 epoch 68 - iter 10/50 - loss 2.35623841 - samples/sec: 84.40 - lr: 0.000488 2021-03-26 07:09:38,783 epoch 68 - iter 15/50 - loss 2.23468730 - samples/sec: 87.43 - lr: 0.000488 2021-03-26 07:09:40,653 epoch 68 - iter 20/50 - loss 2.27483031 - samples/sec: 85.62 - lr: 0.000488 2021-03-26 07:09:42,595 epoch 68 - iter 25/50 - loss 2.26047467 - samples/sec: 82.49 - lr: 0.000488 2021-03-26 07:09:44,477 epoch 68 - iter 30/50 - loss 2.24738997 - samples/sec: 85.11 - lr: 0.000488 2021-03-26 07:09:46,274 epoch 68 - iter 35/50 - loss 2.27638966 - samples/sec: 89.09 - lr: 0.000488 2021-03-26 07:09:48,185 epoch 68 - iter 40/50 - loss 2.37632107 - samples/sec: 83.80 - lr: 0.000488 2021-03-26 07:09:50,100 epoch 68 - iter 45/50 - loss 2.42216489 - samples/sec: 83.62 - lr: 0.000488 2021-03-26 07:09:51,873 epoch 68 - iter 50/50 - loss 2.41903520 - samples/sec: 90.36 - lr: 0.000488 2021-03-26 07:09:51,873 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:09:51,873 EPOCH 68 done: loss 2.4190 - lr 0.0004883 2021-03-26 07:09:52,634 DEV : loss 6.222766876220703 - score 0.9182 2021-03-26 07:09:52,659 BAD EPOCHS (no improvement): 2 2021-03-26 07:09:52,659 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:09:54,578 epoch 69 - iter 5/50 - loss 2.19501238 - samples/sec: 83.46 - lr: 0.000488 2021-03-26 07:09:56,609 epoch 69 - iter 10/50 - loss 2.28023877 - samples/sec: 78.86 - lr: 0.000488 2021-03-26 07:09:58,571 epoch 69 - iter 15/50 - loss 2.18687286 - samples/sec: 81.64 - lr: 0.000488 2021-03-26 07:10:00,551 epoch 69 - iter 20/50 - loss 2.21458565 - samples/sec: 80.88 - lr: 0.000488 2021-03-26 07:10:02,397 epoch 69 - iter 25/50 - loss 2.32697690 - samples/sec: 86.77 - lr: 0.000488 2021-03-26 07:10:04,341 epoch 69 - iter 30/50 - loss 2.25094236 - samples/sec: 82.37 - lr: 0.000488 2021-03-26 07:10:06,141 epoch 69 - iter 35/50 - loss 2.25566019 - samples/sec: 88.97 - lr: 0.000488 2021-03-26 07:10:07,971 epoch 69 - iter 40/50 - loss 2.31639599 - samples/sec: 87.50 - lr: 0.000488 2021-03-26 07:10:09,862 epoch 69 - iter 45/50 - loss 2.33205634 - samples/sec: 84.72 - lr: 0.000488 2021-03-26 07:10:11,657 epoch 69 - iter 50/50 - loss 2.35981682 - samples/sec: 89.25 - lr: 0.000488 2021-03-26 07:10:11,657 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:10:11,658 EPOCH 69 done: loss 2.3598 - lr 0.0004883 2021-03-26 07:10:12,476 DEV : loss 6.222655296325684 - score 0.9178 2021-03-26 07:10:12,507 BAD EPOCHS (no improvement): 3 2021-03-26 07:10:12,508 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:10:14,565 epoch 70 - iter 5/50 - loss 2.74699225 - samples/sec: 77.88 - lr: 0.000488 2021-03-26 07:10:16,709 epoch 70 - iter 10/50 - loss 2.38880478 - samples/sec: 74.68 - lr: 0.000488 2021-03-26 07:10:18,691 epoch 70 - iter 15/50 - loss 2.40105462 - samples/sec: 80.81 - lr: 0.000488 2021-03-26 07:10:20,770 epoch 70 - iter 20/50 - loss 2.48934642 - samples/sec: 77.01 - lr: 0.000488 2021-03-26 07:10:22,632 epoch 70 - iter 25/50 - loss 2.50072574 - samples/sec: 86.01 - lr: 0.000488 2021-03-26 07:10:24,441 epoch 70 - iter 30/50 - loss 2.47702555 - samples/sec: 88.52 - lr: 0.000488 2021-03-26 07:10:26,167 epoch 70 - iter 35/50 - loss 2.47407045 - samples/sec: 92.78 - lr: 0.000488 2021-03-26 07:10:28,018 epoch 70 - iter 40/50 - loss 2.45811075 - samples/sec: 86.51 - lr: 0.000488 2021-03-26 07:10:29,873 epoch 70 - iter 45/50 - loss 2.47587674 - samples/sec: 86.34 - lr: 0.000488 2021-03-26 07:10:31,727 epoch 70 - iter 50/50 - loss 2.49858140 - samples/sec: 86.41 - lr: 0.000488 2021-03-26 07:10:31,728 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:10:31,728 EPOCH 70 done: loss 2.4986 - lr 0.0004883 2021-03-26 07:10:32,504 DEV : loss 6.222859859466553 - score 0.9178 2021-03-26 07:10:32,528 BAD EPOCHS (no improvement): 4 2021-03-26 07:10:32,529 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:10:34,406 epoch 71 - iter 5/50 - loss 2.86681695 - samples/sec: 85.36 - lr: 0.000244 2021-03-26 07:10:36,252 epoch 71 - iter 10/50 - loss 2.64170929 - samples/sec: 86.77 - lr: 0.000244 2021-03-26 07:10:38,417 epoch 71 - iter 15/50 - loss 2.50964533 - samples/sec: 73.98 - lr: 0.000244 2021-03-26 07:10:40,297 epoch 71 - iter 20/50 - loss 2.50780855 - samples/sec: 85.19 - lr: 0.000244 2021-03-26 07:10:42,640 epoch 71 - iter 25/50 - loss 2.55021320 - samples/sec: 68.33 - lr: 0.000244 2021-03-26 07:10:44,470 epoch 71 - iter 30/50 - loss 2.58371531 - samples/sec: 87.56 - lr: 0.000244 2021-03-26 07:10:46,490 epoch 71 - iter 35/50 - loss 2.58910989 - samples/sec: 79.27 - lr: 0.000244 2021-03-26 07:10:48,399 epoch 71 - iter 40/50 - loss 2.56937031 - samples/sec: 84.06 - lr: 0.000244 2021-03-26 07:10:50,358 epoch 71 - iter 45/50 - loss 2.55463270 - samples/sec: 81.73 - lr: 0.000244 2021-03-26 07:10:52,191 epoch 71 - iter 50/50 - loss 2.50627966 - samples/sec: 87.38 - lr: 0.000244 2021-03-26 07:10:52,192 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:10:52,193 EPOCH 71 done: loss 2.5063 - lr 0.0002441 2021-03-26 07:10:52,969 DEV : loss 6.223049163818359 - score 0.9178 2021-03-26 07:10:52,986 BAD EPOCHS (no improvement): 1 2021-03-26 07:10:52,986 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:10:54,835 epoch 72 - iter 5/50 - loss 2.39624953 - samples/sec: 86.62 - lr: 0.000244 2021-03-26 07:10:56,682 epoch 72 - iter 10/50 - loss 2.36824905 - samples/sec: 86.74 - lr: 0.000244 2021-03-26 07:10:58,365 epoch 72 - iter 15/50 - loss 2.43008339 - samples/sec: 95.10 - lr: 0.000244 2021-03-26 07:11:00,466 epoch 72 - iter 20/50 - loss 2.38753402 - samples/sec: 76.22 - lr: 0.000244 2021-03-26 07:11:02,453 epoch 72 - iter 25/50 - loss 2.36955923 - samples/sec: 80.62 - lr: 0.000244 2021-03-26 07:11:04,570 epoch 72 - iter 30/50 - loss 2.41096191 - samples/sec: 75.66 - lr: 0.000244 2021-03-26 07:11:06,661 epoch 72 - iter 35/50 - loss 2.52839747 - samples/sec: 76.70 - lr: 0.000244 2021-03-26 07:11:08,590 epoch 72 - iter 40/50 - loss 2.53110349 - samples/sec: 83.06 - lr: 0.000244 2021-03-26 07:11:10,380 epoch 72 - iter 45/50 - loss 2.48378272 - samples/sec: 89.48 - lr: 0.000244 2021-03-26 07:11:12,240 epoch 72 - iter 50/50 - loss 2.47316787 - samples/sec: 86.09 - lr: 0.000244 2021-03-26 07:11:12,241 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:11:12,241 EPOCH 72 done: loss 2.4732 - lr 0.0002441 2021-03-26 07:11:13,002 DEV : loss 6.223275661468506 - score 0.9178 2021-03-26 07:11:13,027 BAD EPOCHS (no improvement): 2 2021-03-26 07:11:13,028 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:11:14,920 epoch 73 - iter 5/50 - loss 2.70434468 - samples/sec: 84.67 - lr: 0.000244 2021-03-26 07:11:16,812 epoch 73 - iter 10/50 - loss 2.45531214 - samples/sec: 84.65 - lr: 0.000244 2021-03-26 07:11:18,591 epoch 73 - iter 15/50 - loss 2.30294273 - samples/sec: 90.03 - lr: 0.000244 2021-03-26 07:11:20,686 epoch 73 - iter 20/50 - loss 2.35064186 - samples/sec: 76.42 - lr: 0.000244 2021-03-26 07:11:22,626 epoch 73 - iter 25/50 - loss 2.28706394 - samples/sec: 82.57 - lr: 0.000244 2021-03-26 07:11:24,619 epoch 73 - iter 30/50 - loss 2.27880787 - samples/sec: 80.36 - lr: 0.000244 2021-03-26 07:11:26,533 epoch 73 - iter 35/50 - loss 2.33573643 - samples/sec: 83.68 - lr: 0.000244 2021-03-26 07:11:28,426 epoch 73 - iter 40/50 - loss 2.39343770 - samples/sec: 84.60 - lr: 0.000244 2021-03-26 07:11:30,385 epoch 73 - iter 45/50 - loss 2.37945244 - samples/sec: 81.77 - lr: 0.000244 2021-03-26 07:11:32,151 epoch 73 - iter 50/50 - loss 2.40987703 - samples/sec: 90.69 - lr: 0.000244 2021-03-26 07:11:32,152 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:11:32,152 EPOCH 73 done: loss 2.4099 - lr 0.0002441 2021-03-26 07:11:32,982 DEV : loss 6.222825527191162 - score 0.9178 2021-03-26 07:11:33,014 BAD EPOCHS (no improvement): 3 2021-03-26 07:11:33,015 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:11:35,001 epoch 74 - iter 5/50 - loss 2.38879180 - samples/sec: 80.67 - lr: 0.000244 2021-03-26 07:11:37,166 epoch 74 - iter 10/50 - loss 2.39860876 - samples/sec: 73.95 - lr: 0.000244 2021-03-26 07:11:39,146 epoch 74 - iter 15/50 - loss 2.40750088 - samples/sec: 80.90 - lr: 0.000244 2021-03-26 07:11:41,415 epoch 74 - iter 20/50 - loss 2.48535473 - samples/sec: 70.56 - lr: 0.000244 2021-03-26 07:11:43,546 epoch 74 - iter 25/50 - loss 2.52484987 - samples/sec: 75.14 - lr: 0.000244 2021-03-26 07:11:45,422 epoch 74 - iter 30/50 - loss 2.46785964 - samples/sec: 85.38 - lr: 0.000244 2021-03-26 07:11:47,381 epoch 74 - iter 35/50 - loss 2.49764286 - samples/sec: 81.75 - lr: 0.000244 2021-03-26 07:11:49,942 epoch 74 - iter 40/50 - loss 2.50870748 - samples/sec: 62.51 - lr: 0.000244 2021-03-26 07:11:52,122 epoch 74 - iter 45/50 - loss 2.45298528 - samples/sec: 73.46 - lr: 0.000244 2021-03-26 07:11:54,078 epoch 74 - iter 50/50 - loss 2.48597799 - samples/sec: 81.88 - lr: 0.000244 2021-03-26 07:11:54,079 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:11:54,079 EPOCH 74 done: loss 2.4860 - lr 0.0002441 2021-03-26 07:11:54,879 DEV : loss 6.222982406616211 - score 0.9174 2021-03-26 07:11:54,899 BAD EPOCHS (no improvement): 4 2021-03-26 07:11:54,900 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:11:56,828 epoch 75 - iter 5/50 - loss 2.88134527 - samples/sec: 83.06 - lr: 0.000122 2021-03-26 07:11:58,775 epoch 75 - iter 10/50 - loss 2.43894054 - samples/sec: 82.27 - lr: 0.000122 2021-03-26 07:12:00,731 epoch 75 - iter 15/50 - loss 2.42260317 - samples/sec: 81.87 - lr: 0.000122 2021-03-26 07:12:02,762 epoch 75 - iter 20/50 - loss 2.35718715 - samples/sec: 78.84 - lr: 0.000122 2021-03-26 07:12:04,664 epoch 75 - iter 25/50 - loss 2.36030079 - samples/sec: 84.24 - lr: 0.000122 2021-03-26 07:12:06,571 epoch 75 - iter 30/50 - loss 2.34997718 - samples/sec: 83.97 - lr: 0.000122 2021-03-26 07:12:08,758 epoch 75 - iter 35/50 - loss 2.34546312 - samples/sec: 73.21 - lr: 0.000122 2021-03-26 07:12:10,806 epoch 75 - iter 40/50 - loss 2.33144889 - samples/sec: 78.21 - lr: 0.000122 2021-03-26 07:12:12,805 epoch 75 - iter 45/50 - loss 2.33342309 - samples/sec: 80.11 - lr: 0.000122 2021-03-26 07:12:15,445 epoch 75 - iter 50/50 - loss 2.34604343 - samples/sec: 60.65 - lr: 0.000122 2021-03-26 07:12:15,446 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:12:15,446 EPOCH 75 done: loss 2.3460 - lr 0.0001221 2021-03-26 07:12:16,193 DEV : loss 6.222970962524414 - score 0.9174 2021-03-26 07:12:16,215 BAD EPOCHS (no improvement): 1 2021-03-26 07:12:16,216 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:12:17,898 epoch 76 - iter 5/50 - loss 2.58834620 - samples/sec: 95.27 - lr: 0.000122 2021-03-26 07:12:19,610 epoch 76 - iter 10/50 - loss 2.46520443 - samples/sec: 93.58 - lr: 0.000122 2021-03-26 07:12:21,781 epoch 76 - iter 15/50 - loss 2.60440030 - samples/sec: 73.76 - lr: 0.000122 2021-03-26 07:12:23,690 epoch 76 - iter 20/50 - loss 2.48082805 - samples/sec: 83.89 - lr: 0.000122 2021-03-26 07:12:25,820 epoch 76 - iter 25/50 - loss 2.49374959 - samples/sec: 75.17 - lr: 0.000122 2021-03-26 07:12:28,936 epoch 76 - iter 30/50 - loss 2.49200382 - samples/sec: 51.38 - lr: 0.000122 2021-03-26 07:12:31,047 epoch 76 - iter 35/50 - loss 2.49267069 - samples/sec: 75.84 - lr: 0.000122 2021-03-26 07:12:32,928 epoch 76 - iter 40/50 - loss 2.51199569 - samples/sec: 85.16 - lr: 0.000122 2021-03-26 07:12:34,799 epoch 76 - iter 45/50 - loss 2.55731874 - samples/sec: 85.61 - lr: 0.000122 2021-03-26 07:12:36,678 epoch 76 - iter 50/50 - loss 2.53093982 - samples/sec: 85.21 - lr: 0.000122 2021-03-26 07:12:36,679 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:12:36,679 EPOCH 76 done: loss 2.5309 - lr 0.0001221 2021-03-26 07:12:37,482 DEV : loss 6.222939968109131 - score 0.9178 2021-03-26 07:12:37,503 BAD EPOCHS (no improvement): 2 2021-03-26 07:12:37,504 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:12:39,530 epoch 77 - iter 5/50 - loss 3.01930332 - samples/sec: 79.08 - lr: 0.000122 2021-03-26 07:12:41,490 epoch 77 - iter 10/50 - loss 2.82609091 - samples/sec: 81.71 - lr: 0.000122 2021-03-26 07:12:43,424 epoch 77 - iter 15/50 - loss 2.54770860 - samples/sec: 82.78 - lr: 0.000122 2021-03-26 07:12:45,203 epoch 77 - iter 20/50 - loss 2.54026788 - samples/sec: 90.03 - lr: 0.000122 2021-03-26 07:12:47,071 epoch 77 - iter 25/50 - loss 2.49590103 - samples/sec: 85.77 - lr: 0.000122 2021-03-26 07:12:49,215 epoch 77 - iter 30/50 - loss 2.51122611 - samples/sec: 74.68 - lr: 0.000122 2021-03-26 07:12:51,031 epoch 77 - iter 35/50 - loss 2.51107687 - samples/sec: 88.21 - lr: 0.000122 2021-03-26 07:12:52,890 epoch 77 - iter 40/50 - loss 2.45614245 - samples/sec: 86.11 - lr: 0.000122 2021-03-26 07:12:54,806 epoch 77 - iter 45/50 - loss 2.47441014 - samples/sec: 83.59 - lr: 0.000122 2021-03-26 07:12:56,636 epoch 77 - iter 50/50 - loss 2.48978061 - samples/sec: 87.54 - lr: 0.000122 2021-03-26 07:12:56,637 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:12:56,637 EPOCH 77 done: loss 2.4898 - lr 0.0001221 2021-03-26 07:12:57,396 DEV : loss 6.222973823547363 - score 0.9178 2021-03-26 07:12:57,419 BAD EPOCHS (no improvement): 3 2021-03-26 07:12:57,420 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:12:59,638 epoch 78 - iter 5/50 - loss 2.42860546 - samples/sec: 72.22 - lr: 0.000122 2021-03-26 07:13:01,613 epoch 78 - iter 10/50 - loss 2.37586279 - samples/sec: 81.09 - lr: 0.000122 2021-03-26 07:13:03,592 epoch 78 - iter 15/50 - loss 2.51110738 - samples/sec: 80.97 - lr: 0.000122 2021-03-26 07:13:05,713 epoch 78 - iter 20/50 - loss 2.53387340 - samples/sec: 75.52 - lr: 0.000122 2021-03-26 07:13:07,510 epoch 78 - iter 25/50 - loss 2.56651618 - samples/sec: 89.11 - lr: 0.000122 2021-03-26 07:13:09,289 epoch 78 - iter 30/50 - loss 2.50188101 - samples/sec: 90.08 - lr: 0.000122 2021-03-26 07:13:11,360 epoch 78 - iter 35/50 - loss 2.51097650 - samples/sec: 77.30 - lr: 0.000122 2021-03-26 07:13:13,312 epoch 78 - iter 40/50 - loss 2.52860793 - samples/sec: 82.06 - lr: 0.000122 2021-03-26 07:13:15,210 epoch 78 - iter 45/50 - loss 2.50384797 - samples/sec: 84.44 - lr: 0.000122 2021-03-26 07:13:16,980 epoch 78 - iter 50/50 - loss 2.56408679 - samples/sec: 90.44 - lr: 0.000122 2021-03-26 07:13:16,981 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:16,981 EPOCH 78 done: loss 2.5641 - lr 0.0001221 2021-03-26 07:13:17,758 DEV : loss 6.22302770614624 - score 0.9178 2021-03-26 07:13:17,779 BAD EPOCHS (no improvement): 4 2021-03-26 07:13:17,779 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:17,780 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:17,780 learning rate too small - quitting training! 2021-03-26 07:13:17,780 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:26,869 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:26,870 Testing using best model ... 2021-03-26 07:13:26,870 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260643/best-model.pt 2021-03-26 07:13:34,253 0.9029 2021-03-26 07:13:34,254 Results: - F-score (micro): 0.8996 - F-score (macro): 0.576 - Accuracy (incl. no class): 0.9029 By class: precision recall f1-score support INTJ 0.9375 0.9375 0.9375 16 NOUN 0.9203 0.9245 0.9224 437 NUM 0.8000 0.8000 0.8000 15 ADJ 0.6983 0.8265 0.7570 98 PRON 0.9751 0.9800 0.9776 200 VERB 0.9500 0.9301 0.9399 143 AUX 0.9535 0.9535 0.9535 43 PROPN 0.8400 0.8750 0.8571 24 ADV 0.9083 0.8390 0.8722 118 DET 0.9623 0.9444 0.9533 54 ADP 0.9909 0.9478 0.9689 115 SCONJ 0.8387 0.9630 0.8966 27 PART 0.9333 0.9655 0.9492 174 CCONJ 0.9545 0.9545 0.9545 88 PUNCT 1.0000 1.0000 1.0000 30 V 0.9294 0.8229 0.8729 96 PREP+PRON 0.8500 0.8947 0.8718 19 V+PRON+PRON 0.3636 0.3333 0.3478 12 NOUN+PRON 0.8644 0.8226 0.8430 62 NOUN+NSUFF 0.9259 0.8621 0.8929 58 PREP+NOUN+PRON 0.6667 1.0000 0.8000 2 PUNC 0.9805 1.0000 0.9902 151 V+PRON 0.7963 0.8113 0.8037 53 V+PRON+PREP+PRON 1.0000 0.0000 0.0000 2 V+PREP+PRON 0.6000 0.6000 0.6000 5 DET+NOUN+NSUFF 0.7619 0.8889 0.8205 18 NOUN+NSUFF+PRON 0.6957 0.8421 0.7619 19 CONJ 0.9714 0.9714 0.9714 35 FOREIGN 1.0000 0.8333 0.9091 6 MENTION 0.8800 1.0000 0.9362 22 EOS 1.0000 1.0000 1.0000 70 PREP 1.0000 0.9714 0.9855 70 DET+NOUN 0.9315 0.9577 0.9444 71 PROG_PART+V 0.8529 0.7632 0.8056 38 CONJ+NOUN+PRON 0.5000 0.3333 0.4000 3 CONJ+NOUN 0.7500 0.9000 0.8182 10 ADJ+NSUFF 0.7879 0.8387 0.8125 31 PART+PRON 1.0000 1.0000 1.0000 15 PREP+DET+NOUN 0.9412 1.0000 0.9697 16 CONJ+V+PRON 0.7500 0.8571 0.8000 7 PROG_PART+V+PRON 0.6471 0.9167 0.7586 12 CONJ+V 0.7500 0.6667 0.7059 9 ADJ+PRON 0.2500 0.2500 0.2500 4 FUT_PART 1.0000 1.0000 1.0000 5 PRON+DET+NOUN+NSUFF 1.0000 1.0000 1.0000 1 DET+ADJ+NSUFF 0.6667 0.4000 0.5000 5 PREP+DET+NUM+NSUFF 1.0000 0.0000 0.0000 1 PREP+NOUN+NSUFF 0.7500 1.0000 0.8571 3 NOUN+CASE 1.0000 1.0000 1.0000 3 PREP+PART 1.0000 1.0000 1.0000 1 FUT_PART+V 0.8571 0.5455 0.6667 11 PREP+NOUN 0.2727 0.5000 0.3529 6 CONJ+DET+NOUN 1.0000 1.0000 1.0000 5 PREP+PRON+DET 1.0000 0.0000 0.0000 1 HASH 0.9375 0.8333 0.8824 18 PART+V+PRON+PRON 0.0000 1.0000 0.0000 0 PREP+DET+NOUN+NSUFF 1.0000 0.5000 0.6667 4 EMOT 1.0000 0.9062 0.9508 32 CONJ+FUT_PART+V 1.0000 0.2500 0.4000 4 CONJ+PROG_PART+V 0.1667 1.0000 0.2857 1 CONJ+NOUN+NSUFF 1.0000 0.3333 0.5000 3 CONJ+ADJ 0.0000 1.0000 0.0000 0 DET+ADJ 0.5000 0.6000 0.5455 5 CONJ+PRON 1.0000 1.0000 1.0000 6 PREP+V 1.0000 0.5000 0.6667 2 CONJ+PREP+V 1.0000 0.0000 0.0000 1 PREP+V+PRON 1.0000 0.0000 0.0000 1 URL 1.0000 1.0000 1.0000 3 PRON+DET+NOUN 1.0000 1.0000 1.0000 1 CONJ+PREP 1.0000 0.6667 0.8000 3 CONJ+PART 0.7500 0.8571 0.8000 7 CONJ+PREP+NOUN 1.0000 0.0000 0.0000 1 PREP+NOUN+NSUFF+PRON 1.0000 1.0000 1.0000 3 FUT_PART+V+PREP+PRON 0.0000 1.0000 0.0000 0 V+NEG_PART 1.0000 0.0000 0.0000 1 PROG_PART+V+NEG_PART 1.0000 1.0000 1.0000 1 ADJ+CASE 0.3333 1.0000 0.5000 1 PREP+PART+PRON 1.0000 1.0000 1.0000 2 ADJ+PREP+PRON 1.0000 0.0000 0.0000 1 PART+NOUN+PRON 0.0000 1.0000 0.0000 0 PART+V 1.0000 0.0000 0.0000 1 PART+ADJ+PRON+PRON 1.0000 0.0000 0.0000 1 PART+NOUN 0.8333 0.8333 0.8333 6 CONJ+ADV 1.0000 0.5000 0.6667 2 CONJ+ADJ+NSUFF 0.0000 1.0000 0.0000 0 PREP+PROG_PART+V+PRON 1.0000 0.0000 0.0000 1 PART+PROG_PART+V 1.0000 0.0000 0.0000 1 PART+V+NEG_PART 0.0000 0.0000 0.0000 6 PART+V+PRON+NEG_PART 0.2000 1.0000 0.3333 2 PROG_PART+V+PREP+PRON 1.0000 0.0000 0.0000 1 PART+NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 NUM+NSUFF 1.0000 0.0000 0.0000 2 PART+PREP+NEG_PART 0.7500 1.0000 0.8571 3 FUT_PART+V+PRON 0.5000 1.0000 0.6667 1 PART+PROG_PART+V+NEG_PART 0.3333 0.3333 0.3333 3 DET+NUM 1.0000 0.0000 0.0000 2 PART+NOUN+NEG_PART 1.0000 0.0000 0.0000 1 ADJ+NSUFF+PRON 0.0000 1.0000 0.0000 0 PART+V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+PROG_PART+V+PRON+NEG_PART 0.0000 1.0000 0.0000 0 CONJ+PART+PROG_PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 CONJ+PART+V+PRON+NEG_PART 0.0000 1.0000 0.0000 0 ADV+NSUFF 1.0000 1.0000 1.0000 1 PART+V+PRON 1.0000 0.0000 0.0000 1 PREP+PRON+DET+NOUN 0.0000 1.0000 0.0000 0 micro avg 0.8996 0.8996 0.8996 2679 macro avg 0.7806 0.6813 0.5760 2679 weighted avg 0.9102 0.8996 0.8981 2679 2021-03-26 07:13:34,254 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:34,254 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:38,055 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC 2021-03-26 07:13:38,055 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu 2021-03-26 07:13:38,056 Dev: None 2021-03-26 07:13:38,056 Test: None 2021-03-26 07:13:38,351 Reading data from ../../Datasets_adhoc/UD_MADAR 2021-03-26 07:13:38,351 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu 2021-03-26 07:13:38,351 Dev: None 2021-03-26 07:13:38,352 Test: None 2021-03-26 07:13:38,384 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 07:13:38,384 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt 2021-03-26 07:13:38,384 Dev: None 2021-03-26 07:13:38,385 Test: None 2021-03-26 07:13:38,564 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 07:13:38,565 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt 2021-03-26 07:13:38,565 Dev: None 2021-03-26 07:13:38,566 Test: None 2021-03-26 07:13:40,620 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 07:13:40,621 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt 2021-03-26 07:13:40,621 Dev: None 2021-03-26 07:13:40,621 Test: None 2021-03-26 07:13:40,776 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 07:13:40,776 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt 2021-03-26 07:13:40,777 Dev: None 2021-03-26 07:13:40,777 Test: None 2021-03-26 07:13:40,916 Filtering long sentences 2021-03-26 07:13:40,958 MultiCorpus: 1574 train + 177 dev + 193 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences 2021-03-26 07:13:41,340 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:41,340 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): WordEmbeddings('ar') (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) (list_embedding_2): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=206, bias=True) (beta): 1.0 (weights): None (weight_tensor) None )" 2021-03-26 07:13:41,341 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:41,341 Corpus: "MultiCorpus: 1574 train + 177 dev + 193 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" 2021-03-26 07:13:41,341 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:41,342 Parameters: 2021-03-26 07:13:41,342 - learning_rate: "0.5" 2021-03-26 07:13:41,342 - mini_batch_size: "32" 2021-03-26 07:13:41,343 - patience: "3" 2021-03-26 07:13:41,343 - anneal_factor: "0.5" 2021-03-26 07:13:41,343 - max_epochs: "150" 2021-03-26 07:13:41,344 - shuffle: "True" 2021-03-26 07:13:41,344 - train_with_dev: "False" 2021-03-26 07:13:41,344 - batch_growth_annealing: "False" 2021-03-26 07:13:41,344 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:41,345 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260713" 2021-03-26 07:13:41,345 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:41,345 Device: cuda:0 2021-03-26 07:13:41,346 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:41,346 Embeddings storage mode: cpu 2021-03-26 07:13:41,347 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:13:44,081 epoch 1 - iter 5/50 - loss 85.61149902 - samples/sec: 58.57 - lr: 0.500000 2021-03-26 07:13:46,470 epoch 1 - iter 10/50 - loss 77.91009293 - samples/sec: 67.01 - lr: 0.500000 2021-03-26 07:13:48,885 epoch 1 - iter 15/50 - loss 71.52193502 - samples/sec: 66.30 - lr: 0.500000 2021-03-26 07:13:51,430 epoch 1 - iter 20/50 - loss 67.45240936 - samples/sec: 62.91 - lr: 0.500000 2021-03-26 07:13:53,859 epoch 1 - iter 25/50 - loss 63.35490433 - samples/sec: 65.91 - lr: 0.500000 2021-03-26 07:13:56,148 epoch 1 - iter 30/50 - loss 59.82105052 - samples/sec: 69.96 - lr: 0.500000 2021-03-26 07:13:58,638 epoch 1 - iter 35/50 - loss 57.17276971 - samples/sec: 64.30 - lr: 0.500000 2021-03-26 07:14:01,151 epoch 1 - iter 40/50 - loss 54.94282775 - samples/sec: 63.73 - lr: 0.500000 2021-03-26 07:14:03,378 epoch 1 - iter 45/50 - loss 52.87651193 - samples/sec: 71.90 - lr: 0.500000 2021-03-26 07:14:05,763 epoch 1 - iter 50/50 - loss 50.76319237 - samples/sec: 67.11 - lr: 0.500000 2021-03-26 07:14:05,764 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:14:05,765 EPOCH 1 done: loss 50.7632 - lr 0.5000000 2021-03-26 07:14:07,067 DEV : loss 29.096771240234375 - score 0.5047 2021-03-26 07:14:07,088 BAD EPOCHS (no improvement): 0 2021-03-26 07:14:16,389 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:14:18,245 epoch 2 - iter 5/50 - loss 28.80629158 - samples/sec: 86.31 - lr: 0.500000 2021-03-26 07:14:20,218 epoch 2 - iter 10/50 - loss 28.44908276 - samples/sec: 81.18 - lr: 0.500000 2021-03-26 07:14:22,159 epoch 2 - iter 15/50 - loss 28.31970825 - samples/sec: 82.53 - lr: 0.500000 2021-03-26 07:14:24,044 epoch 2 - iter 20/50 - loss 27.87325191 - samples/sec: 84.93 - lr: 0.500000 2021-03-26 07:14:26,137 epoch 2 - iter 25/50 - loss 27.86790703 - samples/sec: 76.55 - lr: 0.500000 2021-03-26 07:14:27,876 epoch 2 - iter 30/50 - loss 27.33799852 - samples/sec: 92.06 - lr: 0.500000 2021-03-26 07:14:29,563 epoch 2 - iter 35/50 - loss 26.45283541 - samples/sec: 94.96 - lr: 0.500000 2021-03-26 07:14:31,480 epoch 2 - iter 40/50 - loss 26.06411510 - samples/sec: 83.53 - lr: 0.500000 2021-03-26 07:14:33,605 epoch 2 - iter 45/50 - loss 25.72350714 - samples/sec: 75.35 - lr: 0.500000 2021-03-26 07:14:35,452 epoch 2 - iter 50/50 - loss 25.18292456 - samples/sec: 86.72 - lr: 0.500000 2021-03-26 07:14:35,453 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:14:35,453 EPOCH 2 done: loss 25.1829 - lr 0.5000000 2021-03-26 07:14:36,255 DEV : loss 17.214523315429688 - score 0.7089 2021-03-26 07:14:36,278 BAD EPOCHS (no improvement): 0 2021-03-26 07:14:45,899 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:14:47,996 epoch 3 - iter 5/50 - loss 21.90074081 - samples/sec: 76.41 - lr: 0.500000 2021-03-26 07:14:49,788 epoch 3 - iter 10/50 - loss 20.25820999 - samples/sec: 89.43 - lr: 0.500000 2021-03-26 07:14:51,799 epoch 3 - iter 15/50 - loss 20.09863516 - samples/sec: 79.61 - lr: 0.500000 2021-03-26 07:14:53,730 epoch 3 - iter 20/50 - loss 19.78721623 - samples/sec: 82.94 - lr: 0.500000 2021-03-26 07:14:55,675 epoch 3 - iter 25/50 - loss 19.39301216 - samples/sec: 82.37 - lr: 0.500000 2021-03-26 07:14:57,547 epoch 3 - iter 30/50 - loss 18.84088933 - samples/sec: 85.52 - lr: 0.500000 2021-03-26 07:14:59,587 epoch 3 - iter 35/50 - loss 18.47238004 - samples/sec: 78.51 - lr: 0.500000 2021-03-26 07:15:01,545 epoch 3 - iter 40/50 - loss 18.23260474 - samples/sec: 81.80 - lr: 0.500000 2021-03-26 07:15:03,478 epoch 3 - iter 45/50 - loss 17.99070036 - samples/sec: 82.85 - lr: 0.500000 2021-03-26 07:15:05,314 epoch 3 - iter 50/50 - loss 17.81347260 - samples/sec: 87.38 - lr: 0.500000 2021-03-26 07:15:05,315 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:15:05,315 EPOCH 3 done: loss 17.8135 - lr 0.5000000 2021-03-26 07:15:06,085 DEV : loss 15.27471923828125 - score 0.7544 2021-03-26 07:15:06,110 BAD EPOCHS (no improvement): 0 2021-03-26 07:15:15,597 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:15:17,570 epoch 4 - iter 5/50 - loss 14.33682823 - samples/sec: 81.25 - lr: 0.500000 2021-03-26 07:15:19,459 epoch 4 - iter 10/50 - loss 14.34253550 - samples/sec: 84.75 - lr: 0.500000 2021-03-26 07:15:21,335 epoch 4 - iter 15/50 - loss 15.17600791 - samples/sec: 85.40 - lr: 0.500000 2021-03-26 07:15:23,263 epoch 4 - iter 20/50 - loss 14.81236367 - samples/sec: 83.07 - lr: 0.500000 2021-03-26 07:15:25,123 epoch 4 - iter 25/50 - loss 14.28828472 - samples/sec: 86.14 - lr: 0.500000 2021-03-26 07:15:27,060 epoch 4 - iter 30/50 - loss 14.58110215 - samples/sec: 82.66 - lr: 0.500000 2021-03-26 07:15:29,032 epoch 4 - iter 35/50 - loss 14.54754723 - samples/sec: 81.20 - lr: 0.500000 2021-03-26 07:15:30,904 epoch 4 - iter 40/50 - loss 14.51193528 - samples/sec: 85.57 - lr: 0.500000 2021-03-26 07:15:32,835 epoch 4 - iter 45/50 - loss 14.46083404 - samples/sec: 82.91 - lr: 0.500000 2021-03-26 07:15:34,540 epoch 4 - iter 50/50 - loss 14.43629816 - samples/sec: 94.01 - lr: 0.500000 2021-03-26 07:15:34,541 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:15:34,542 EPOCH 4 done: loss 14.4363 - lr 0.5000000 2021-03-26 07:15:35,319 DEV : loss 11.61365795135498 - score 0.7946 2021-03-26 07:15:35,343 BAD EPOCHS (no improvement): 0 2021-03-26 07:15:44,962 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:15:46,876 epoch 5 - iter 5/50 - loss 11.90889206 - samples/sec: 83.74 - lr: 0.500000 2021-03-26 07:15:48,752 epoch 5 - iter 10/50 - loss 11.91375709 - samples/sec: 85.33 - lr: 0.500000 2021-03-26 07:15:50,891 epoch 5 - iter 15/50 - loss 12.07819735 - samples/sec: 74.86 - lr: 0.500000 2021-03-26 07:15:52,813 epoch 5 - iter 20/50 - loss 12.28275967 - samples/sec: 83.32 - lr: 0.500000 2021-03-26 07:15:54,664 epoch 5 - iter 25/50 - loss 12.37690090 - samples/sec: 86.50 - lr: 0.500000 2021-03-26 07:15:56,634 epoch 5 - iter 30/50 - loss 12.64451520 - samples/sec: 81.32 - lr: 0.500000 2021-03-26 07:15:58,420 epoch 5 - iter 35/50 - loss 12.54120459 - samples/sec: 89.70 - lr: 0.500000 2021-03-26 07:16:00,227 epoch 5 - iter 40/50 - loss 12.50868628 - samples/sec: 88.64 - lr: 0.500000 2021-03-26 07:16:02,407 epoch 5 - iter 45/50 - loss 12.45480209 - samples/sec: 73.47 - lr: 0.500000 2021-03-26 07:16:04,282 epoch 5 - iter 50/50 - loss 12.34371593 - samples/sec: 85.40 - lr: 0.500000 2021-03-26 07:16:04,283 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:16:04,284 EPOCH 5 done: loss 12.3437 - lr 0.5000000 2021-03-26 07:16:05,070 DEV : loss 10.551742553710938 - score 0.8147 2021-03-26 07:16:05,089 BAD EPOCHS (no improvement): 0 2021-03-26 07:16:14,569 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:16:16,407 epoch 6 - iter 5/50 - loss 10.71872311 - samples/sec: 87.23 - lr: 0.500000 2021-03-26 07:16:18,174 epoch 6 - iter 10/50 - loss 10.88694544 - samples/sec: 90.60 - lr: 0.500000 2021-03-26 07:16:20,041 epoch 6 - iter 15/50 - loss 10.37633387 - samples/sec: 85.80 - lr: 0.500000 2021-03-26 07:16:22,004 epoch 6 - iter 20/50 - loss 10.93236151 - samples/sec: 81.59 - lr: 0.500000 2021-03-26 07:16:24,159 epoch 6 - iter 25/50 - loss 11.00346493 - samples/sec: 74.31 - lr: 0.500000 2021-03-26 07:16:26,102 epoch 6 - iter 30/50 - loss 11.11314979 - samples/sec: 82.41 - lr: 0.500000 2021-03-26 07:16:27,821 epoch 6 - iter 35/50 - loss 11.19503303 - samples/sec: 93.18 - lr: 0.500000 2021-03-26 07:16:29,628 epoch 6 - iter 40/50 - loss 11.21847228 - samples/sec: 88.64 - lr: 0.500000 2021-03-26 07:16:31,576 epoch 6 - iter 45/50 - loss 11.18473713 - samples/sec: 82.23 - lr: 0.500000 2021-03-26 07:16:33,331 epoch 6 - iter 50/50 - loss 11.14374526 - samples/sec: 91.25 - lr: 0.500000 2021-03-26 07:16:33,332 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:16:33,332 EPOCH 6 done: loss 11.1437 - lr 0.5000000 2021-03-26 07:16:34,111 DEV : loss 9.204329490661621 - score 0.8331 2021-03-26 07:16:34,135 BAD EPOCHS (no improvement): 0 2021-03-26 07:16:43,722 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:16:45,606 epoch 7 - iter 5/50 - loss 9.51087551 - samples/sec: 85.07 - lr: 0.500000 2021-03-26 07:16:47,488 epoch 7 - iter 10/50 - loss 9.76788254 - samples/sec: 85.10 - lr: 0.500000 2021-03-26 07:16:49,407 epoch 7 - iter 15/50 - loss 10.30387751 - samples/sec: 83.47 - lr: 0.500000 2021-03-26 07:16:51,248 epoch 7 - iter 20/50 - loss 10.07285285 - samples/sec: 86.98 - lr: 0.500000 2021-03-26 07:16:53,197 epoch 7 - iter 25/50 - loss 9.87292212 - samples/sec: 82.17 - lr: 0.500000 2021-03-26 07:16:55,852 epoch 7 - iter 30/50 - loss 9.99077923 - samples/sec: 60.30 - lr: 0.500000 2021-03-26 07:16:58,247 epoch 7 - iter 35/50 - loss 10.18544400 - samples/sec: 66.88 - lr: 0.500000 2021-03-26 07:17:00,211 epoch 7 - iter 40/50 - loss 10.22825085 - samples/sec: 81.54 - lr: 0.500000 2021-03-26 07:17:02,108 epoch 7 - iter 45/50 - loss 10.17230478 - samples/sec: 84.45 - lr: 0.500000 2021-03-26 07:17:03,681 epoch 7 - iter 50/50 - loss 9.96814963 - samples/sec: 101.79 - lr: 0.500000 2021-03-26 07:17:03,682 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:17:03,682 EPOCH 7 done: loss 9.9681 - lr 0.5000000 2021-03-26 07:17:04,465 DEV : loss 8.45942211151123 - score 0.8574 2021-03-26 07:17:04,483 BAD EPOCHS (no improvement): 0 2021-03-26 07:17:13,790 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:17:15,695 epoch 8 - iter 5/50 - loss 9.15951490 - samples/sec: 84.10 - lr: 0.500000 2021-03-26 07:17:17,390 epoch 8 - iter 10/50 - loss 8.63786511 - samples/sec: 94.54 - lr: 0.500000 2021-03-26 07:17:19,199 epoch 8 - iter 15/50 - loss 8.81899722 - samples/sec: 88.52 - lr: 0.500000 2021-03-26 07:17:20,948 epoch 8 - iter 20/50 - loss 8.66737792 - samples/sec: 91.61 - lr: 0.500000 2021-03-26 07:17:22,752 epoch 8 - iter 25/50 - loss 8.90464464 - samples/sec: 88.75 - lr: 0.500000 2021-03-26 07:17:24,692 epoch 8 - iter 30/50 - loss 8.98467898 - samples/sec: 82.60 - lr: 0.500000 2021-03-26 07:17:26,695 epoch 8 - iter 35/50 - loss 9.11239944 - samples/sec: 79.95 - lr: 0.500000 2021-03-26 07:17:28,507 epoch 8 - iter 40/50 - loss 9.09702935 - samples/sec: 88.40 - lr: 0.500000 2021-03-26 07:17:30,483 epoch 8 - iter 45/50 - loss 9.25233818 - samples/sec: 81.03 - lr: 0.500000 2021-03-26 07:17:32,190 epoch 8 - iter 50/50 - loss 9.19621006 - samples/sec: 93.82 - lr: 0.500000 2021-03-26 07:17:32,191 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:17:32,192 EPOCH 8 done: loss 9.1962 - lr 0.5000000 2021-03-26 07:17:33,000 DEV : loss 8.076202392578125 - score 0.8576 2021-03-26 07:17:33,025 BAD EPOCHS (no improvement): 0 2021-03-26 07:17:42,590 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:17:44,748 epoch 9 - iter 5/50 - loss 9.24322968 - samples/sec: 74.24 - lr: 0.500000 2021-03-26 07:17:46,621 epoch 9 - iter 10/50 - loss 9.26248417 - samples/sec: 85.55 - lr: 0.500000 2021-03-26 07:17:48,630 epoch 9 - iter 15/50 - loss 8.89867210 - samples/sec: 79.69 - lr: 0.500000 2021-03-26 07:17:50,482 epoch 9 - iter 20/50 - loss 8.52454841 - samples/sec: 86.50 - lr: 0.500000 2021-03-26 07:17:52,639 epoch 9 - iter 25/50 - loss 8.52478699 - samples/sec: 74.23 - lr: 0.500000 2021-03-26 07:17:54,490 epoch 9 - iter 30/50 - loss 8.67076422 - samples/sec: 86.55 - lr: 0.500000 2021-03-26 07:17:56,353 epoch 9 - iter 35/50 - loss 8.56702987 - samples/sec: 85.98 - lr: 0.500000 2021-03-26 07:17:58,147 epoch 9 - iter 40/50 - loss 8.77756805 - samples/sec: 89.26 - lr: 0.500000 2021-03-26 07:18:00,078 epoch 9 - iter 45/50 - loss 8.78899221 - samples/sec: 82.94 - lr: 0.500000 2021-03-26 07:18:01,952 epoch 9 - iter 50/50 - loss 8.89824164 - samples/sec: 85.49 - lr: 0.500000 2021-03-26 07:18:01,953 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:18:01,953 EPOCH 9 done: loss 8.8982 - lr 0.5000000 2021-03-26 07:18:02,706 DEV : loss 8.47732162475586 - score 0.8572 2021-03-26 07:18:02,729 BAD EPOCHS (no improvement): 1 2021-03-26 07:18:02,730 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:18:04,654 epoch 10 - iter 5/50 - loss 7.55866394 - samples/sec: 83.23 - lr: 0.500000 2021-03-26 07:18:06,436 epoch 10 - iter 10/50 - loss 7.83560810 - samples/sec: 89.94 - lr: 0.500000 2021-03-26 07:18:08,507 epoch 10 - iter 15/50 - loss 8.03101371 - samples/sec: 77.30 - lr: 0.500000 2021-03-26 07:18:10,516 epoch 10 - iter 20/50 - loss 8.23381851 - samples/sec: 79.72 - lr: 0.500000 2021-03-26 07:18:12,494 epoch 10 - iter 25/50 - loss 8.40272551 - samples/sec: 80.95 - lr: 0.500000 2021-03-26 07:18:14,446 epoch 10 - iter 30/50 - loss 8.41102055 - samples/sec: 82.06 - lr: 0.500000 2021-03-26 07:18:16,363 epoch 10 - iter 35/50 - loss 8.35400145 - samples/sec: 83.54 - lr: 0.500000 2021-03-26 07:18:18,414 epoch 10 - iter 40/50 - loss 8.27732370 - samples/sec: 78.06 - lr: 0.500000 2021-03-26 07:18:20,306 epoch 10 - iter 45/50 - loss 8.28684916 - samples/sec: 84.63 - lr: 0.500000 2021-03-26 07:18:22,175 epoch 10 - iter 50/50 - loss 8.46044763 - samples/sec: 85.70 - lr: 0.500000 2021-03-26 07:18:22,175 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:18:22,176 EPOCH 10 done: loss 8.4604 - lr 0.5000000 2021-03-26 07:18:22,990 DEV : loss 7.925993919372559 - score 0.8629 2021-03-26 07:18:23,023 BAD EPOCHS (no improvement): 0 2021-03-26 07:18:32,404 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:18:34,243 epoch 11 - iter 5/50 - loss 6.82128010 - samples/sec: 87.14 - lr: 0.500000 2021-03-26 07:18:36,094 epoch 11 - iter 10/50 - loss 6.75115950 - samples/sec: 86.53 - lr: 0.500000 2021-03-26 07:18:37,971 epoch 11 - iter 15/50 - loss 7.84271013 - samples/sec: 85.33 - lr: 0.500000 2021-03-26 07:18:39,877 epoch 11 - iter 20/50 - loss 8.39340039 - samples/sec: 84.03 - lr: 0.500000 2021-03-26 07:18:41,753 epoch 11 - iter 25/50 - loss 8.26747842 - samples/sec: 85.35 - lr: 0.500000 2021-03-26 07:18:43,611 epoch 11 - iter 30/50 - loss 8.06985706 - samples/sec: 86.23 - lr: 0.500000 2021-03-26 07:18:45,418 epoch 11 - iter 35/50 - loss 8.02684945 - samples/sec: 88.63 - lr: 0.500000 2021-03-26 07:18:47,267 epoch 11 - iter 40/50 - loss 7.87398350 - samples/sec: 86.60 - lr: 0.500000 2021-03-26 07:18:49,091 epoch 11 - iter 45/50 - loss 7.81394832 - samples/sec: 87.83 - lr: 0.500000 2021-03-26 07:18:50,969 epoch 11 - iter 50/50 - loss 7.85806360 - samples/sec: 85.24 - lr: 0.500000 2021-03-26 07:18:50,970 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:18:50,970 EPOCH 11 done: loss 7.8581 - lr 0.5000000 2021-03-26 07:18:51,702 DEV : loss 8.213061332702637 - score 0.8627 2021-03-26 07:18:51,725 BAD EPOCHS (no improvement): 1 2021-03-26 07:18:51,726 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:18:53,648 epoch 12 - iter 5/50 - loss 8.13135443 - samples/sec: 83.32 - lr: 0.500000 2021-03-26 07:18:55,463 epoch 12 - iter 10/50 - loss 7.90176258 - samples/sec: 88.28 - lr: 0.500000 2021-03-26 07:18:57,262 epoch 12 - iter 15/50 - loss 7.70501353 - samples/sec: 89.01 - lr: 0.500000 2021-03-26 07:18:59,057 epoch 12 - iter 20/50 - loss 7.43266449 - samples/sec: 89.23 - lr: 0.500000 2021-03-26 07:19:00,976 epoch 12 - iter 25/50 - loss 7.29033449 - samples/sec: 83.45 - lr: 0.500000 2021-03-26 07:19:02,949 epoch 12 - iter 30/50 - loss 7.39221203 - samples/sec: 81.14 - lr: 0.500000 2021-03-26 07:19:04,752 epoch 12 - iter 35/50 - loss 7.36656352 - samples/sec: 88.83 - lr: 0.500000 2021-03-26 07:19:06,694 epoch 12 - iter 40/50 - loss 7.25957453 - samples/sec: 82.48 - lr: 0.500000 2021-03-26 07:19:08,666 epoch 12 - iter 45/50 - loss 7.20568540 - samples/sec: 81.22 - lr: 0.500000 2021-03-26 07:19:10,519 epoch 12 - iter 50/50 - loss 7.38290733 - samples/sec: 86.40 - lr: 0.500000 2021-03-26 07:19:10,520 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:19:10,521 EPOCH 12 done: loss 7.3829 - lr 0.5000000 2021-03-26 07:19:11,299 DEV : loss 7.321202754974365 - score 0.8794 2021-03-26 07:19:11,328 BAD EPOCHS (no improvement): 0 2021-03-26 07:19:20,919 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:19:22,829 epoch 13 - iter 5/50 - loss 6.82536354 - samples/sec: 83.90 - lr: 0.500000 2021-03-26 07:19:24,718 epoch 13 - iter 10/50 - loss 6.86277728 - samples/sec: 84.80 - lr: 0.500000 2021-03-26 07:19:26,660 epoch 13 - iter 15/50 - loss 6.89155194 - samples/sec: 82.45 - lr: 0.500000 2021-03-26 07:19:28,433 epoch 13 - iter 20/50 - loss 6.78971924 - samples/sec: 90.29 - lr: 0.500000 2021-03-26 07:19:30,261 epoch 13 - iter 25/50 - loss 6.97617368 - samples/sec: 87.62 - lr: 0.500000 2021-03-26 07:19:32,017 epoch 13 - iter 30/50 - loss 6.93531903 - samples/sec: 91.21 - lr: 0.500000 2021-03-26 07:19:33,717 epoch 13 - iter 35/50 - loss 6.90266201 - samples/sec: 94.22 - lr: 0.500000 2021-03-26 07:19:35,568 epoch 13 - iter 40/50 - loss 6.96388810 - samples/sec: 86.53 - lr: 0.500000 2021-03-26 07:19:37,534 epoch 13 - iter 45/50 - loss 6.98509439 - samples/sec: 81.45 - lr: 0.500000 2021-03-26 07:19:39,373 epoch 13 - iter 50/50 - loss 7.03323493 - samples/sec: 87.09 - lr: 0.500000 2021-03-26 07:19:39,374 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:19:39,374 EPOCH 13 done: loss 7.0332 - lr 0.5000000 2021-03-26 07:19:40,117 DEV : loss 7.746276378631592 - score 0.8679 2021-03-26 07:19:40,141 BAD EPOCHS (no improvement): 1 2021-03-26 07:19:40,142 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:19:42,097 epoch 14 - iter 5/50 - loss 8.29018412 - samples/sec: 81.96 - lr: 0.500000 2021-03-26 07:19:43,923 epoch 14 - iter 10/50 - loss 7.02287879 - samples/sec: 87.72 - lr: 0.500000 2021-03-26 07:19:45,874 epoch 14 - iter 15/50 - loss 6.64712122 - samples/sec: 82.06 - lr: 0.500000 2021-03-26 07:19:47,724 epoch 14 - iter 20/50 - loss 6.63326148 - samples/sec: 86.57 - lr: 0.500000 2021-03-26 07:19:49,762 epoch 14 - iter 25/50 - loss 6.82262151 - samples/sec: 78.58 - lr: 0.500000 2021-03-26 07:19:51,767 epoch 14 - iter 30/50 - loss 6.73005801 - samples/sec: 79.86 - lr: 0.500000 2021-03-26 07:19:53,622 epoch 14 - iter 35/50 - loss 6.68655214 - samples/sec: 86.36 - lr: 0.500000 2021-03-26 07:19:55,413 epoch 14 - iter 40/50 - loss 6.68068996 - samples/sec: 89.45 - lr: 0.500000 2021-03-26 07:19:57,250 epoch 14 - iter 45/50 - loss 6.69404273 - samples/sec: 87.18 - lr: 0.500000 2021-03-26 07:19:59,327 epoch 14 - iter 50/50 - loss 6.89570993 - samples/sec: 77.09 - lr: 0.500000 2021-03-26 07:19:59,328 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:19:59,328 EPOCH 14 done: loss 6.8957 - lr 0.5000000 2021-03-26 07:20:00,100 DEV : loss 7.810580253601074 - score 0.8754 2021-03-26 07:20:00,124 BAD EPOCHS (no improvement): 2 2021-03-26 07:20:00,125 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:20:02,456 epoch 15 - iter 5/50 - loss 5.86052933 - samples/sec: 68.73 - lr: 0.500000 2021-03-26 07:20:04,317 epoch 15 - iter 10/50 - loss 5.63950758 - samples/sec: 86.02 - lr: 0.500000 2021-03-26 07:20:06,292 epoch 15 - iter 15/50 - loss 5.85248785 - samples/sec: 81.10 - lr: 0.500000 2021-03-26 07:20:08,170 epoch 15 - iter 20/50 - loss 6.19100833 - samples/sec: 85.29 - lr: 0.500000 2021-03-26 07:20:09,969 epoch 15 - iter 25/50 - loss 6.32947805 - samples/sec: 89.01 - lr: 0.500000 2021-03-26 07:20:11,895 epoch 15 - iter 30/50 - loss 6.30526145 - samples/sec: 83.20 - lr: 0.500000 2021-03-26 07:20:14,361 epoch 15 - iter 35/50 - loss 6.28907063 - samples/sec: 64.93 - lr: 0.500000 2021-03-26 07:20:16,378 epoch 15 - iter 40/50 - loss 6.33453875 - samples/sec: 79.40 - lr: 0.500000 2021-03-26 07:20:18,273 epoch 15 - iter 45/50 - loss 6.43837033 - samples/sec: 84.46 - lr: 0.500000 2021-03-26 07:20:19,958 epoch 15 - iter 50/50 - loss 6.35599067 - samples/sec: 95.07 - lr: 0.500000 2021-03-26 07:20:19,959 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:20:19,960 EPOCH 15 done: loss 6.3560 - lr 0.5000000 2021-03-26 07:20:20,732 DEV : loss 7.69682502746582 - score 0.8702 2021-03-26 07:20:20,756 BAD EPOCHS (no improvement): 3 2021-03-26 07:20:20,757 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:20:22,535 epoch 16 - iter 5/50 - loss 5.94938555 - samples/sec: 90.08 - lr: 0.500000 2021-03-26 07:20:24,532 epoch 16 - iter 10/50 - loss 5.70883789 - samples/sec: 80.19 - lr: 0.500000 2021-03-26 07:20:26,548 epoch 16 - iter 15/50 - loss 5.92653176 - samples/sec: 79.43 - lr: 0.500000 2021-03-26 07:20:28,477 epoch 16 - iter 20/50 - loss 6.07018437 - samples/sec: 83.04 - lr: 0.500000 2021-03-26 07:20:30,356 epoch 16 - iter 25/50 - loss 5.94514559 - samples/sec: 85.19 - lr: 0.500000 2021-03-26 07:20:32,259 epoch 16 - iter 30/50 - loss 6.15654551 - samples/sec: 84.17 - lr: 0.500000 2021-03-26 07:20:34,121 epoch 16 - iter 35/50 - loss 6.13194075 - samples/sec: 86.02 - lr: 0.500000 2021-03-26 07:20:35,925 epoch 16 - iter 40/50 - loss 6.17352308 - samples/sec: 88.76 - lr: 0.500000 2021-03-26 07:20:37,712 epoch 16 - iter 45/50 - loss 6.20834807 - samples/sec: 89.65 - lr: 0.500000 2021-03-26 07:20:39,459 epoch 16 - iter 50/50 - loss 6.29687656 - samples/sec: 91.73 - lr: 0.500000 2021-03-26 07:20:39,460 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:20:39,460 EPOCH 16 done: loss 6.2969 - lr 0.5000000 2021-03-26 07:20:40,223 DEV : loss 7.245139122009277 - score 0.8862 2021-03-26 07:20:40,247 BAD EPOCHS (no improvement): 0 2021-03-26 07:20:49,704 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:20:51,540 epoch 17 - iter 5/50 - loss 5.49544411 - samples/sec: 87.28 - lr: 0.500000 2021-03-26 07:20:53,435 epoch 17 - iter 10/50 - loss 5.53276272 - samples/sec: 84.53 - lr: 0.500000 2021-03-26 07:20:55,254 epoch 17 - iter 15/50 - loss 5.87554884 - samples/sec: 88.05 - lr: 0.500000 2021-03-26 07:20:57,513 epoch 17 - iter 20/50 - loss 5.90366483 - samples/sec: 70.88 - lr: 0.500000 2021-03-26 07:21:00,040 epoch 17 - iter 25/50 - loss 5.88849455 - samples/sec: 63.38 - lr: 0.500000 2021-03-26 07:21:01,932 epoch 17 - iter 30/50 - loss 5.99359461 - samples/sec: 84.60 - lr: 0.500000 2021-03-26 07:21:03,962 epoch 17 - iter 35/50 - loss 6.00984791 - samples/sec: 78.89 - lr: 0.500000 2021-03-26 07:21:05,843 epoch 17 - iter 40/50 - loss 6.00084951 - samples/sec: 85.14 - lr: 0.500000 2021-03-26 07:21:07,641 epoch 17 - iter 45/50 - loss 5.98434572 - samples/sec: 89.10 - lr: 0.500000 2021-03-26 07:21:09,476 epoch 17 - iter 50/50 - loss 5.94449944 - samples/sec: 87.28 - lr: 0.500000 2021-03-26 07:21:09,477 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:21:09,478 EPOCH 17 done: loss 5.9445 - lr 0.5000000 2021-03-26 07:21:10,240 DEV : loss 7.041107177734375 - score 0.8888 2021-03-26 07:21:10,263 BAD EPOCHS (no improvement): 0 2021-03-26 07:21:19,640 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:21:21,345 epoch 18 - iter 5/50 - loss 5.46243830 - samples/sec: 93.98 - lr: 0.500000 2021-03-26 07:21:23,270 epoch 18 - iter 10/50 - loss 5.42810898 - samples/sec: 83.23 - lr: 0.500000 2021-03-26 07:21:25,252 epoch 18 - iter 15/50 - loss 5.50631167 - samples/sec: 80.78 - lr: 0.500000 2021-03-26 07:21:27,184 epoch 18 - iter 20/50 - loss 5.40645225 - samples/sec: 82.90 - lr: 0.500000 2021-03-26 07:21:29,157 epoch 18 - iter 25/50 - loss 5.53953705 - samples/sec: 81.15 - lr: 0.500000 2021-03-26 07:21:31,044 epoch 18 - iter 30/50 - loss 5.67473930 - samples/sec: 84.85 - lr: 0.500000 2021-03-26 07:21:33,113 epoch 18 - iter 35/50 - loss 5.69194521 - samples/sec: 77.40 - lr: 0.500000 2021-03-26 07:21:35,063 epoch 18 - iter 40/50 - loss 5.79437474 - samples/sec: 82.13 - lr: 0.500000 2021-03-26 07:21:36,880 epoch 18 - iter 45/50 - loss 5.81149050 - samples/sec: 88.11 - lr: 0.500000 2021-03-26 07:21:38,666 epoch 18 - iter 50/50 - loss 5.94897350 - samples/sec: 89.70 - lr: 0.500000 2021-03-26 07:21:38,666 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:21:38,667 EPOCH 18 done: loss 5.9490 - lr 0.5000000 2021-03-26 07:21:39,405 DEV : loss 7.531728267669678 - score 0.8795 2021-03-26 07:21:39,429 BAD EPOCHS (no improvement): 1 2021-03-26 07:21:39,429 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:21:41,309 epoch 19 - iter 5/50 - loss 6.17610245 - samples/sec: 85.22 - lr: 0.500000 2021-03-26 07:21:43,200 epoch 19 - iter 10/50 - loss 5.49422398 - samples/sec: 84.68 - lr: 0.500000 2021-03-26 07:21:45,021 epoch 19 - iter 15/50 - loss 5.62345748 - samples/sec: 87.99 - lr: 0.500000 2021-03-26 07:21:46,927 epoch 19 - iter 20/50 - loss 5.61114949 - samples/sec: 84.00 - lr: 0.500000 2021-03-26 07:21:48,793 epoch 19 - iter 25/50 - loss 5.58432162 - samples/sec: 85.85 - lr: 0.500000 2021-03-26 07:21:50,738 epoch 19 - iter 30/50 - loss 5.60729048 - samples/sec: 82.30 - lr: 0.500000 2021-03-26 07:21:52,627 epoch 19 - iter 35/50 - loss 5.65039468 - samples/sec: 84.80 - lr: 0.500000 2021-03-26 07:21:54,402 epoch 19 - iter 40/50 - loss 5.60913602 - samples/sec: 90.27 - lr: 0.500000 2021-03-26 07:21:56,310 epoch 19 - iter 45/50 - loss 5.58604383 - samples/sec: 83.97 - lr: 0.500000 2021-03-26 07:21:58,069 epoch 19 - iter 50/50 - loss 5.60817661 - samples/sec: 91.07 - lr: 0.500000 2021-03-26 07:21:58,070 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:21:58,070 EPOCH 19 done: loss 5.6082 - lr 0.5000000 2021-03-26 07:21:58,818 DEV : loss 7.617198467254639 - score 0.8725 2021-03-26 07:21:58,842 BAD EPOCHS (no improvement): 2 2021-03-26 07:21:58,842 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:22:01,088 epoch 20 - iter 5/50 - loss 4.85406322 - samples/sec: 71.30 - lr: 0.500000 2021-03-26 07:22:02,889 epoch 20 - iter 10/50 - loss 5.31827521 - samples/sec: 88.93 - lr: 0.500000 2021-03-26 07:22:04,776 epoch 20 - iter 15/50 - loss 5.31004019 - samples/sec: 84.86 - lr: 0.500000 2021-03-26 07:22:06,555 epoch 20 - iter 20/50 - loss 5.45654291 - samples/sec: 90.04 - lr: 0.500000 2021-03-26 07:22:08,464 epoch 20 - iter 25/50 - loss 5.38042480 - samples/sec: 83.86 - lr: 0.500000 2021-03-26 07:22:10,340 epoch 20 - iter 30/50 - loss 5.55804290 - samples/sec: 85.39 - lr: 0.500000 2021-03-26 07:22:12,087 epoch 20 - iter 35/50 - loss 5.47529270 - samples/sec: 91.63 - lr: 0.500000 2021-03-26 07:22:13,934 epoch 20 - iter 40/50 - loss 5.46405497 - samples/sec: 86.72 - lr: 0.500000 2021-03-26 07:22:16,049 epoch 20 - iter 45/50 - loss 5.52897902 - samples/sec: 75.70 - lr: 0.500000 2021-03-26 07:22:17,768 epoch 20 - iter 50/50 - loss 5.54975753 - samples/sec: 93.19 - lr: 0.500000 2021-03-26 07:22:17,769 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:22:17,769 EPOCH 20 done: loss 5.5498 - lr 0.5000000 2021-03-26 07:22:18,532 DEV : loss 7.108177185058594 - score 0.8819 2021-03-26 07:22:18,556 BAD EPOCHS (no improvement): 3 2021-03-26 07:22:18,557 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:22:20,513 epoch 21 - iter 5/50 - loss 5.16098585 - samples/sec: 81.87 - lr: 0.500000 2021-03-26 07:22:22,461 epoch 21 - iter 10/50 - loss 5.18648055 - samples/sec: 82.22 - lr: 0.500000 2021-03-26 07:22:24,278 epoch 21 - iter 15/50 - loss 4.95983901 - samples/sec: 88.16 - lr: 0.500000 2021-03-26 07:22:26,051 epoch 21 - iter 20/50 - loss 4.78192036 - samples/sec: 90.29 - lr: 0.500000 2021-03-26 07:22:27,861 epoch 21 - iter 25/50 - loss 4.84919130 - samples/sec: 88.48 - lr: 0.500000 2021-03-26 07:22:29,806 epoch 21 - iter 30/50 - loss 4.90508842 - samples/sec: 82.35 - lr: 0.500000 2021-03-26 07:22:31,748 epoch 21 - iter 35/50 - loss 4.91921381 - samples/sec: 82.45 - lr: 0.500000 2021-03-26 07:22:33,575 epoch 21 - iter 40/50 - loss 4.94709951 - samples/sec: 87.67 - lr: 0.500000 2021-03-26 07:22:35,497 epoch 21 - iter 45/50 - loss 5.04448750 - samples/sec: 83.36 - lr: 0.500000 2021-03-26 07:22:37,206 epoch 21 - iter 50/50 - loss 5.09773973 - samples/sec: 93.72 - lr: 0.500000 2021-03-26 07:22:37,207 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:22:37,207 EPOCH 21 done: loss 5.0977 - lr 0.5000000 2021-03-26 07:22:37,955 DEV : loss 7.056064128875732 - score 0.8867 2021-03-26 07:22:37,978 BAD EPOCHS (no improvement): 4 2021-03-26 07:22:37,978 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:22:40,485 epoch 22 - iter 5/50 - loss 4.97422800 - samples/sec: 63.88 - lr: 0.250000 2021-03-26 07:22:42,424 epoch 22 - iter 10/50 - loss 5.04048371 - samples/sec: 82.56 - lr: 0.250000 2021-03-26 07:22:44,332 epoch 22 - iter 15/50 - loss 5.08245630 - samples/sec: 83.96 - lr: 0.250000 2021-03-26 07:22:46,359 epoch 22 - iter 20/50 - loss 4.87549429 - samples/sec: 78.98 - lr: 0.250000 2021-03-26 07:22:48,277 epoch 22 - iter 25/50 - loss 4.77376683 - samples/sec: 83.53 - lr: 0.250000 2021-03-26 07:22:49,957 epoch 22 - iter 30/50 - loss 4.53706872 - samples/sec: 95.33 - lr: 0.250000 2021-03-26 07:22:51,797 epoch 22 - iter 35/50 - loss 4.60300268 - samples/sec: 87.03 - lr: 0.250000 2021-03-26 07:22:54,177 epoch 22 - iter 40/50 - loss 4.57214971 - samples/sec: 67.27 - lr: 0.250000 2021-03-26 07:22:55,995 epoch 22 - iter 45/50 - loss 4.56367335 - samples/sec: 88.10 - lr: 0.250000 2021-03-26 07:22:57,760 epoch 22 - iter 50/50 - loss 4.46984960 - samples/sec: 90.70 - lr: 0.250000 2021-03-26 07:22:57,761 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:22:57,762 EPOCH 22 done: loss 4.4698 - lr 0.2500000 2021-03-26 07:22:58,540 DEV : loss 6.589443206787109 - score 0.8909 2021-03-26 07:22:58,563 BAD EPOCHS (no improvement): 0 2021-03-26 07:23:07,888 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:23:10,013 epoch 23 - iter 5/50 - loss 4.35152192 - samples/sec: 75.40 - lr: 0.250000 2021-03-26 07:23:12,196 epoch 23 - iter 10/50 - loss 4.12232957 - samples/sec: 73.35 - lr: 0.250000 2021-03-26 07:23:14,091 epoch 23 - iter 15/50 - loss 4.29600732 - samples/sec: 84.52 - lr: 0.250000 2021-03-26 07:23:16,476 epoch 23 - iter 20/50 - loss 4.32097278 - samples/sec: 67.15 - lr: 0.250000 2021-03-26 07:23:19,105 epoch 23 - iter 25/50 - loss 4.47653557 - samples/sec: 60.90 - lr: 0.250000 2021-03-26 07:23:21,677 epoch 23 - iter 30/50 - loss 4.30746055 - samples/sec: 62.25 - lr: 0.250000 2021-03-26 07:23:23,532 epoch 23 - iter 35/50 - loss 4.25856673 - samples/sec: 86.33 - lr: 0.250000 2021-03-26 07:23:25,373 epoch 23 - iter 40/50 - loss 4.24913441 - samples/sec: 87.04 - lr: 0.250000 2021-03-26 07:23:27,457 epoch 23 - iter 45/50 - loss 4.21190177 - samples/sec: 76.81 - lr: 0.250000 2021-03-26 07:23:29,194 epoch 23 - iter 50/50 - loss 4.23462809 - samples/sec: 92.24 - lr: 0.250000 2021-03-26 07:23:29,195 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:23:29,195 EPOCH 23 done: loss 4.2346 - lr 0.2500000 2021-03-26 07:23:29,932 DEV : loss 6.665990352630615 - score 0.8944 2021-03-26 07:23:29,955 BAD EPOCHS (no improvement): 0 2021-03-26 07:23:39,427 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:23:42,038 epoch 24 - iter 5/50 - loss 3.49912410 - samples/sec: 61.35 - lr: 0.250000 2021-03-26 07:23:44,305 epoch 24 - iter 10/50 - loss 3.82160933 - samples/sec: 70.64 - lr: 0.250000 2021-03-26 07:23:46,197 epoch 24 - iter 15/50 - loss 4.00115436 - samples/sec: 84.65 - lr: 0.250000 2021-03-26 07:23:48,026 epoch 24 - iter 20/50 - loss 3.97462704 - samples/sec: 87.56 - lr: 0.250000 2021-03-26 07:23:49,968 epoch 24 - iter 25/50 - loss 4.03846642 - samples/sec: 82.48 - lr: 0.250000 2021-03-26 07:23:51,808 epoch 24 - iter 30/50 - loss 4.01450043 - samples/sec: 87.06 - lr: 0.250000 2021-03-26 07:23:53,705 epoch 24 - iter 35/50 - loss 4.03496842 - samples/sec: 84.44 - lr: 0.250000 2021-03-26 07:23:55,830 epoch 24 - iter 40/50 - loss 4.01507310 - samples/sec: 75.35 - lr: 0.250000 2021-03-26 07:23:57,902 epoch 24 - iter 45/50 - loss 4.05896441 - samples/sec: 77.29 - lr: 0.250000 2021-03-26 07:23:59,648 epoch 24 - iter 50/50 - loss 4.00071608 - samples/sec: 91.72 - lr: 0.250000 2021-03-26 07:23:59,649 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:23:59,649 EPOCH 24 done: loss 4.0007 - lr 0.2500000 2021-03-26 07:24:00,406 DEV : loss 6.711074352264404 - score 0.8923 2021-03-26 07:24:00,429 BAD EPOCHS (no improvement): 1 2021-03-26 07:24:00,430 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:24:02,724 epoch 25 - iter 5/50 - loss 4.26815948 - samples/sec: 69.81 - lr: 0.250000 2021-03-26 07:24:04,567 epoch 25 - iter 10/50 - loss 4.09609385 - samples/sec: 86.90 - lr: 0.250000 2021-03-26 07:24:06,381 epoch 25 - iter 15/50 - loss 3.88939308 - samples/sec: 88.31 - lr: 0.250000 2021-03-26 07:24:08,402 epoch 25 - iter 20/50 - loss 3.84197943 - samples/sec: 79.21 - lr: 0.250000 2021-03-26 07:24:10,315 epoch 25 - iter 25/50 - loss 3.85339655 - samples/sec: 83.79 - lr: 0.250000 2021-03-26 07:24:12,193 epoch 25 - iter 30/50 - loss 4.03538799 - samples/sec: 85.27 - lr: 0.250000 2021-03-26 07:24:13,909 epoch 25 - iter 35/50 - loss 4.00416953 - samples/sec: 93.33 - lr: 0.250000 2021-03-26 07:24:15,730 epoch 25 - iter 40/50 - loss 3.97750949 - samples/sec: 87.96 - lr: 0.250000 2021-03-26 07:24:17,652 epoch 25 - iter 45/50 - loss 4.02689092 - samples/sec: 83.33 - lr: 0.250000 2021-03-26 07:24:19,436 epoch 25 - iter 50/50 - loss 4.01936452 - samples/sec: 89.76 - lr: 0.250000 2021-03-26 07:24:19,437 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:24:19,437 EPOCH 25 done: loss 4.0194 - lr 0.2500000 2021-03-26 07:24:20,202 DEV : loss 6.78806209564209 - score 0.8905 2021-03-26 07:24:20,226 BAD EPOCHS (no improvement): 2 2021-03-26 07:24:20,227 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:24:22,040 epoch 26 - iter 5/50 - loss 3.97351761 - samples/sec: 88.37 - lr: 0.250000 2021-03-26 07:24:24,191 epoch 26 - iter 10/50 - loss 3.95412531 - samples/sec: 74.41 - lr: 0.250000 2021-03-26 07:24:26,350 epoch 26 - iter 15/50 - loss 3.84625311 - samples/sec: 74.16 - lr: 0.250000 2021-03-26 07:24:28,195 epoch 26 - iter 20/50 - loss 3.71876276 - samples/sec: 86.83 - lr: 0.250000 2021-03-26 07:24:29,948 epoch 26 - iter 25/50 - loss 3.80736031 - samples/sec: 91.35 - lr: 0.250000 2021-03-26 07:24:31,884 epoch 26 - iter 30/50 - loss 3.73687125 - samples/sec: 82.70 - lr: 0.250000 2021-03-26 07:24:33,803 epoch 26 - iter 35/50 - loss 3.74322898 - samples/sec: 83.44 - lr: 0.250000 2021-03-26 07:24:35,661 epoch 26 - iter 40/50 - loss 3.73008016 - samples/sec: 86.21 - lr: 0.250000 2021-03-26 07:24:37,550 epoch 26 - iter 45/50 - loss 3.72723302 - samples/sec: 84.78 - lr: 0.250000 2021-03-26 07:24:39,196 epoch 26 - iter 50/50 - loss 3.73614890 - samples/sec: 97.34 - lr: 0.250000 2021-03-26 07:24:39,197 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:24:39,197 EPOCH 26 done: loss 3.7361 - lr 0.2500000 2021-03-26 07:24:39,926 DEV : loss 6.780250549316406 - score 0.8948 2021-03-26 07:24:39,949 BAD EPOCHS (no improvement): 0 2021-03-26 07:24:49,179 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:24:51,143 epoch 27 - iter 5/50 - loss 3.09613357 - samples/sec: 81.57 - lr: 0.250000 2021-03-26 07:24:53,307 epoch 27 - iter 10/50 - loss 3.32715802 - samples/sec: 74.02 - lr: 0.250000 2021-03-26 07:24:55,195 epoch 27 - iter 15/50 - loss 3.44630082 - samples/sec: 84.85 - lr: 0.250000 2021-03-26 07:24:57,059 epoch 27 - iter 20/50 - loss 3.56317025 - samples/sec: 85.90 - lr: 0.250000 2021-03-26 07:24:59,006 epoch 27 - iter 25/50 - loss 3.52153577 - samples/sec: 82.26 - lr: 0.250000 2021-03-26 07:25:00,851 epoch 27 - iter 30/50 - loss 3.51912872 - samples/sec: 86.79 - lr: 0.250000 2021-03-26 07:25:02,735 epoch 27 - iter 35/50 - loss 3.56115228 - samples/sec: 85.03 - lr: 0.250000 2021-03-26 07:25:04,512 epoch 27 - iter 40/50 - loss 3.63291188 - samples/sec: 90.13 - lr: 0.250000 2021-03-26 07:25:06,374 epoch 27 - iter 45/50 - loss 3.60862633 - samples/sec: 86.03 - lr: 0.250000 2021-03-26 07:25:08,332 epoch 27 - iter 50/50 - loss 3.70079799 - samples/sec: 81.77 - lr: 0.250000 2021-03-26 07:25:08,333 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:25:08,334 EPOCH 27 done: loss 3.7008 - lr 0.2500000 2021-03-26 07:25:09,093 DEV : loss 6.70261287689209 - score 0.8979 2021-03-26 07:25:09,116 BAD EPOCHS (no improvement): 0 2021-03-26 07:25:18,299 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:25:20,119 epoch 28 - iter 5/50 - loss 3.45658116 - samples/sec: 88.05 - lr: 0.250000 2021-03-26 07:25:21,814 epoch 28 - iter 10/50 - loss 3.54790840 - samples/sec: 94.48 - lr: 0.250000 2021-03-26 07:25:23,590 epoch 28 - iter 15/50 - loss 3.70695179 - samples/sec: 90.16 - lr: 0.250000 2021-03-26 07:25:25,547 epoch 28 - iter 20/50 - loss 3.64068614 - samples/sec: 81.83 - lr: 0.250000 2021-03-26 07:25:27,362 epoch 28 - iter 25/50 - loss 3.64049382 - samples/sec: 88.26 - lr: 0.250000 2021-03-26 07:25:29,297 epoch 28 - iter 30/50 - loss 3.62528403 - samples/sec: 82.75 - lr: 0.250000 2021-03-26 07:25:31,147 epoch 28 - iter 35/50 - loss 3.69814000 - samples/sec: 86.61 - lr: 0.250000 2021-03-26 07:25:32,902 epoch 28 - iter 40/50 - loss 3.70481655 - samples/sec: 91.25 - lr: 0.250000 2021-03-26 07:25:34,654 epoch 28 - iter 45/50 - loss 3.72054102 - samples/sec: 91.42 - lr: 0.250000 2021-03-26 07:25:36,296 epoch 28 - iter 50/50 - loss 3.62931919 - samples/sec: 97.51 - lr: 0.250000 2021-03-26 07:25:36,297 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:25:36,297 EPOCH 28 done: loss 3.6293 - lr 0.2500000 2021-03-26 07:25:37,035 DEV : loss 6.870364189147949 - score 0.8938 2021-03-26 07:25:37,058 BAD EPOCHS (no improvement): 1 2021-03-26 07:25:37,059 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:25:38,852 epoch 29 - iter 5/50 - loss 3.84682484 - samples/sec: 89.36 - lr: 0.250000 2021-03-26 07:25:40,763 epoch 29 - iter 10/50 - loss 3.43832564 - samples/sec: 83.80 - lr: 0.250000 2021-03-26 07:25:42,686 epoch 29 - iter 15/50 - loss 3.30578225 - samples/sec: 83.29 - lr: 0.250000 2021-03-26 07:25:44,467 epoch 29 - iter 20/50 - loss 3.17273824 - samples/sec: 89.91 - lr: 0.250000 2021-03-26 07:25:46,512 epoch 29 - iter 25/50 - loss 3.26868323 - samples/sec: 78.30 - lr: 0.250000 2021-03-26 07:25:48,416 epoch 29 - iter 30/50 - loss 3.36351695 - samples/sec: 84.08 - lr: 0.250000 2021-03-26 07:25:50,494 epoch 29 - iter 35/50 - loss 3.44216044 - samples/sec: 77.09 - lr: 0.250000 2021-03-26 07:25:52,408 epoch 29 - iter 40/50 - loss 3.45896108 - samples/sec: 83.64 - lr: 0.250000 2021-03-26 07:25:54,167 epoch 29 - iter 45/50 - loss 3.45697110 - samples/sec: 91.05 - lr: 0.250000 2021-03-26 07:25:56,013 epoch 29 - iter 50/50 - loss 3.46739976 - samples/sec: 86.79 - lr: 0.250000 2021-03-26 07:25:56,014 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:25:56,014 EPOCH 29 done: loss 3.4674 - lr 0.2500000 2021-03-26 07:25:56,785 DEV : loss 6.876599311828613 - score 0.8942 2021-03-26 07:25:56,808 BAD EPOCHS (no improvement): 2 2021-03-26 07:25:56,813 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:25:58,709 epoch 30 - iter 5/50 - loss 3.38032241 - samples/sec: 84.46 - lr: 0.250000 2021-03-26 07:26:00,609 epoch 30 - iter 10/50 - loss 3.35932336 - samples/sec: 84.33 - lr: 0.250000 2021-03-26 07:26:02,705 epoch 30 - iter 15/50 - loss 3.36644726 - samples/sec: 76.38 - lr: 0.250000 2021-03-26 07:26:04,522 epoch 30 - iter 20/50 - loss 3.44605324 - samples/sec: 88.18 - lr: 0.250000 2021-03-26 07:26:06,409 epoch 30 - iter 25/50 - loss 3.31322743 - samples/sec: 84.86 - lr: 0.250000 2021-03-26 07:26:08,375 epoch 30 - iter 30/50 - loss 3.30118415 - samples/sec: 81.46 - lr: 0.250000 2021-03-26 07:26:10,268 epoch 30 - iter 35/50 - loss 3.35088294 - samples/sec: 84.60 - lr: 0.250000 2021-03-26 07:26:12,346 epoch 30 - iter 40/50 - loss 3.35634041 - samples/sec: 77.06 - lr: 0.250000 2021-03-26 07:26:14,373 epoch 30 - iter 45/50 - loss 3.32806790 - samples/sec: 79.05 - lr: 0.250000 2021-03-26 07:26:16,372 epoch 30 - iter 50/50 - loss 3.35867829 - samples/sec: 80.10 - lr: 0.250000 2021-03-26 07:26:16,373 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:26:16,374 EPOCH 30 done: loss 3.3587 - lr 0.2500000 2021-03-26 07:26:17,119 DEV : loss 6.797982692718506 - score 0.8985 2021-03-26 07:26:17,142 BAD EPOCHS (no improvement): 0 2021-03-26 07:26:26,559 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:26:28,668 epoch 31 - iter 5/50 - loss 3.23683105 - samples/sec: 75.97 - lr: 0.250000 2021-03-26 07:26:30,578 epoch 31 - iter 10/50 - loss 3.41368830 - samples/sec: 83.83 - lr: 0.250000 2021-03-26 07:26:32,372 epoch 31 - iter 15/50 - loss 3.27620610 - samples/sec: 89.30 - lr: 0.250000 2021-03-26 07:26:34,304 epoch 31 - iter 20/50 - loss 3.27124422 - samples/sec: 82.89 - lr: 0.250000 2021-03-26 07:26:36,180 epoch 31 - iter 25/50 - loss 3.34331117 - samples/sec: 85.38 - lr: 0.250000 2021-03-26 07:26:38,171 epoch 31 - iter 30/50 - loss 3.37468915 - samples/sec: 80.41 - lr: 0.250000 2021-03-26 07:26:40,132 epoch 31 - iter 35/50 - loss 3.39782465 - samples/sec: 81.66 - lr: 0.250000 2021-03-26 07:26:41,928 epoch 31 - iter 40/50 - loss 3.44018836 - samples/sec: 89.17 - lr: 0.250000 2021-03-26 07:26:43,705 epoch 31 - iter 45/50 - loss 3.43161379 - samples/sec: 90.12 - lr: 0.250000 2021-03-26 07:26:45,676 epoch 31 - iter 50/50 - loss 3.38064914 - samples/sec: 81.24 - lr: 0.250000 2021-03-26 07:26:45,677 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:26:45,677 EPOCH 31 done: loss 3.3806 - lr 0.2500000 2021-03-26 07:26:46,427 DEV : loss 6.567054748535156 - score 0.8995 2021-03-26 07:26:46,451 BAD EPOCHS (no improvement): 0 2021-03-26 07:26:55,784 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:26:57,725 epoch 32 - iter 5/50 - loss 3.01723185 - samples/sec: 82.52 - lr: 0.250000 2021-03-26 07:26:59,668 epoch 32 - iter 10/50 - loss 2.86918635 - samples/sec: 82.41 - lr: 0.250000 2021-03-26 07:27:01,524 epoch 32 - iter 15/50 - loss 2.91769945 - samples/sec: 86.28 - lr: 0.250000 2021-03-26 07:27:03,487 epoch 32 - iter 20/50 - loss 3.17702067 - samples/sec: 81.61 - lr: 0.250000 2021-03-26 07:27:05,403 epoch 32 - iter 25/50 - loss 3.25882262 - samples/sec: 83.56 - lr: 0.250000 2021-03-26 07:27:07,277 epoch 32 - iter 30/50 - loss 3.21344744 - samples/sec: 85.48 - lr: 0.250000 2021-03-26 07:27:09,152 epoch 32 - iter 35/50 - loss 3.31935992 - samples/sec: 85.39 - lr: 0.250000 2021-03-26 07:27:10,972 epoch 32 - iter 40/50 - loss 3.29181318 - samples/sec: 87.99 - lr: 0.250000 2021-03-26 07:27:12,972 epoch 32 - iter 45/50 - loss 3.27006182 - samples/sec: 80.11 - lr: 0.250000 2021-03-26 07:27:14,708 epoch 32 - iter 50/50 - loss 3.22836397 - samples/sec: 92.22 - lr: 0.250000 2021-03-26 07:27:14,709 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:27:14,709 EPOCH 32 done: loss 3.2284 - lr 0.2500000 2021-03-26 07:27:15,478 DEV : loss 6.905445098876953 - score 0.8964 2021-03-26 07:27:15,495 BAD EPOCHS (no improvement): 1 2021-03-26 07:27:15,496 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:27:17,361 epoch 33 - iter 5/50 - loss 2.92829876 - samples/sec: 85.90 - lr: 0.250000 2021-03-26 07:27:19,260 epoch 33 - iter 10/50 - loss 3.38583150 - samples/sec: 84.31 - lr: 0.250000 2021-03-26 07:27:20,997 epoch 33 - iter 15/50 - loss 3.30913531 - samples/sec: 92.19 - lr: 0.250000 2021-03-26 07:27:22,878 epoch 33 - iter 20/50 - loss 3.23969779 - samples/sec: 85.15 - lr: 0.250000 2021-03-26 07:27:24,908 epoch 33 - iter 25/50 - loss 3.33625289 - samples/sec: 78.86 - lr: 0.250000 2021-03-26 07:27:26,825 epoch 33 - iter 30/50 - loss 3.32055628 - samples/sec: 83.54 - lr: 0.250000 2021-03-26 07:27:28,752 epoch 33 - iter 35/50 - loss 3.35262194 - samples/sec: 83.14 - lr: 0.250000 2021-03-26 07:27:30,695 epoch 33 - iter 40/50 - loss 3.32879865 - samples/sec: 82.45 - lr: 0.250000 2021-03-26 07:27:32,552 epoch 33 - iter 45/50 - loss 3.29292493 - samples/sec: 86.22 - lr: 0.250000 2021-03-26 07:27:34,441 epoch 33 - iter 50/50 - loss 3.29516871 - samples/sec: 84.80 - lr: 0.250000 2021-03-26 07:27:34,442 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:27:34,442 EPOCH 33 done: loss 3.2952 - lr 0.2500000 2021-03-26 07:27:35,209 DEV : loss 7.081844329833984 - score 0.8922 2021-03-26 07:27:35,232 BAD EPOCHS (no improvement): 2 2021-03-26 07:27:35,233 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:27:37,298 epoch 34 - iter 5/50 - loss 3.31536870 - samples/sec: 77.57 - lr: 0.250000 2021-03-26 07:27:39,187 epoch 34 - iter 10/50 - loss 3.41087134 - samples/sec: 84.78 - lr: 0.250000 2021-03-26 07:27:40,988 epoch 34 - iter 15/50 - loss 3.22168013 - samples/sec: 88.92 - lr: 0.250000 2021-03-26 07:27:43,080 epoch 34 - iter 20/50 - loss 3.27023110 - samples/sec: 76.54 - lr: 0.250000 2021-03-26 07:27:44,884 epoch 34 - iter 25/50 - loss 3.25223615 - samples/sec: 88.78 - lr: 0.250000 2021-03-26 07:27:46,779 epoch 34 - iter 30/50 - loss 3.19871631 - samples/sec: 84.50 - lr: 0.250000 2021-03-26 07:27:48,725 epoch 34 - iter 35/50 - loss 3.21700798 - samples/sec: 82.27 - lr: 0.250000 2021-03-26 07:27:50,559 epoch 34 - iter 40/50 - loss 3.25193202 - samples/sec: 87.36 - lr: 0.250000 2021-03-26 07:27:52,276 epoch 34 - iter 45/50 - loss 3.17302873 - samples/sec: 93.24 - lr: 0.250000 2021-03-26 07:27:54,008 epoch 34 - iter 50/50 - loss 3.21401441 - samples/sec: 92.47 - lr: 0.250000 2021-03-26 07:27:54,009 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:27:54,009 EPOCH 34 done: loss 3.2140 - lr 0.2500000 2021-03-26 07:27:54,749 DEV : loss 6.83712911605835 - score 0.8979 2021-03-26 07:27:54,773 BAD EPOCHS (no improvement): 3 2021-03-26 07:27:54,773 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:27:56,701 epoch 35 - iter 5/50 - loss 3.56807122 - samples/sec: 83.11 - lr: 0.250000 2021-03-26 07:27:58,584 epoch 35 - iter 10/50 - loss 3.23114133 - samples/sec: 85.02 - lr: 0.250000 2021-03-26 07:28:00,492 epoch 35 - iter 15/50 - loss 3.04977350 - samples/sec: 83.96 - lr: 0.250000 2021-03-26 07:28:02,232 epoch 35 - iter 20/50 - loss 3.10344130 - samples/sec: 92.01 - lr: 0.250000 2021-03-26 07:28:04,194 epoch 35 - iter 25/50 - loss 3.07100470 - samples/sec: 81.63 - lr: 0.250000 2021-03-26 07:28:06,060 epoch 35 - iter 30/50 - loss 3.07075768 - samples/sec: 85.82 - lr: 0.250000 2021-03-26 07:28:07,840 epoch 35 - iter 35/50 - loss 3.09712845 - samples/sec: 89.97 - lr: 0.250000 2021-03-26 07:28:09,991 epoch 35 - iter 40/50 - loss 3.11089132 - samples/sec: 74.44 - lr: 0.250000 2021-03-26 07:28:11,928 epoch 35 - iter 45/50 - loss 3.07642374 - samples/sec: 82.69 - lr: 0.250000 2021-03-26 07:28:13,727 epoch 35 - iter 50/50 - loss 3.12534660 - samples/sec: 89.01 - lr: 0.250000 2021-03-26 07:28:13,728 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:28:13,728 EPOCH 35 done: loss 3.1253 - lr 0.2500000 2021-03-26 07:28:14,471 DEV : loss 7.0124664306640625 - score 0.9001 2021-03-26 07:28:14,487 BAD EPOCHS (no improvement): 0 2021-03-26 07:28:23,704 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:28:25,601 epoch 36 - iter 5/50 - loss 3.38841424 - samples/sec: 84.46 - lr: 0.250000 2021-03-26 07:28:27,763 epoch 36 - iter 10/50 - loss 3.24558442 - samples/sec: 74.06 - lr: 0.250000 2021-03-26 07:28:29,859 epoch 36 - iter 15/50 - loss 3.03712088 - samples/sec: 76.40 - lr: 0.250000 2021-03-26 07:28:31,779 epoch 36 - iter 20/50 - loss 2.96634012 - samples/sec: 83.44 - lr: 0.250000 2021-03-26 07:28:33,656 epoch 36 - iter 25/50 - loss 3.01913111 - samples/sec: 85.29 - lr: 0.250000 2021-03-26 07:28:35,623 epoch 36 - iter 30/50 - loss 3.01572754 - samples/sec: 81.41 - lr: 0.250000 2021-03-26 07:28:37,505 epoch 36 - iter 35/50 - loss 2.99192525 - samples/sec: 85.09 - lr: 0.250000 2021-03-26 07:28:39,644 epoch 36 - iter 40/50 - loss 2.98679020 - samples/sec: 74.87 - lr: 0.250000 2021-03-26 07:28:41,608 epoch 36 - iter 45/50 - loss 3.01203941 - samples/sec: 81.54 - lr: 0.250000 2021-03-26 07:28:43,396 epoch 36 - iter 50/50 - loss 3.01612416 - samples/sec: 89.57 - lr: 0.250000 2021-03-26 07:28:43,397 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:28:43,397 EPOCH 36 done: loss 3.0161 - lr 0.2500000 2021-03-26 07:28:44,140 DEV : loss 6.748878479003906 - score 0.9009 2021-03-26 07:28:44,163 BAD EPOCHS (no improvement): 0 2021-03-26 07:28:53,279 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:28:55,230 epoch 37 - iter 5/50 - loss 3.26688495 - samples/sec: 82.06 - lr: 0.250000 2021-03-26 07:28:57,277 epoch 37 - iter 10/50 - loss 3.48513596 - samples/sec: 78.25 - lr: 0.250000 2021-03-26 07:28:59,254 epoch 37 - iter 15/50 - loss 3.37037697 - samples/sec: 80.98 - lr: 0.250000 2021-03-26 07:29:01,033 epoch 37 - iter 20/50 - loss 3.33579125 - samples/sec: 90.00 - lr: 0.250000 2021-03-26 07:29:03,090 epoch 37 - iter 25/50 - loss 3.29973496 - samples/sec: 77.85 - lr: 0.250000 2021-03-26 07:29:05,378 epoch 37 - iter 30/50 - loss 3.26574783 - samples/sec: 69.98 - lr: 0.250000 2021-03-26 07:29:07,272 epoch 37 - iter 35/50 - loss 3.18129197 - samples/sec: 84.55 - lr: 0.250000 2021-03-26 07:29:09,191 epoch 37 - iter 40/50 - loss 3.16513637 - samples/sec: 83.47 - lr: 0.250000 2021-03-26 07:29:11,029 epoch 37 - iter 45/50 - loss 3.19896053 - samples/sec: 87.12 - lr: 0.250000 2021-03-26 07:29:12,624 epoch 37 - iter 50/50 - loss 3.09600536 - samples/sec: 100.38 - lr: 0.250000 2021-03-26 07:29:12,625 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:29:12,625 EPOCH 37 done: loss 3.0960 - lr 0.2500000 2021-03-26 07:29:13,380 DEV : loss 7.075430870056152 - score 0.894 2021-03-26 07:29:13,399 BAD EPOCHS (no improvement): 1 2021-03-26 07:29:13,399 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:29:15,379 epoch 38 - iter 5/50 - loss 2.92954760 - samples/sec: 80.92 - lr: 0.250000 2021-03-26 07:29:17,270 epoch 38 - iter 10/50 - loss 3.19841943 - samples/sec: 84.68 - lr: 0.250000 2021-03-26 07:29:19,069 epoch 38 - iter 15/50 - loss 3.19953322 - samples/sec: 89.00 - lr: 0.250000 2021-03-26 07:29:21,011 epoch 38 - iter 20/50 - loss 3.19118745 - samples/sec: 82.46 - lr: 0.250000 2021-03-26 07:29:23,093 epoch 38 - iter 25/50 - loss 3.15121227 - samples/sec: 76.91 - lr: 0.250000 2021-03-26 07:29:24,945 epoch 38 - iter 30/50 - loss 3.11521289 - samples/sec: 86.47 - lr: 0.250000 2021-03-26 07:29:26,874 epoch 38 - iter 35/50 - loss 3.17518789 - samples/sec: 83.04 - lr: 0.250000 2021-03-26 07:29:28,737 epoch 38 - iter 40/50 - loss 3.10305169 - samples/sec: 85.96 - lr: 0.250000 2021-03-26 07:29:30,716 epoch 38 - iter 45/50 - loss 3.10097717 - samples/sec: 80.92 - lr: 0.250000 2021-03-26 07:29:32,369 epoch 38 - iter 50/50 - loss 3.10808053 - samples/sec: 96.92 - lr: 0.250000 2021-03-26 07:29:32,369 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:29:32,370 EPOCH 38 done: loss 3.1081 - lr 0.2500000 2021-03-26 07:29:33,114 DEV : loss 7.046388149261475 - score 0.8964 2021-03-26 07:29:33,137 BAD EPOCHS (no improvement): 2 2021-03-26 07:29:33,138 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:29:34,929 epoch 39 - iter 5/50 - loss 2.50283210 - samples/sec: 89.45 - lr: 0.250000 2021-03-26 07:29:36,718 epoch 39 - iter 10/50 - loss 2.79771951 - samples/sec: 89.50 - lr: 0.250000 2021-03-26 07:29:38,533 epoch 39 - iter 15/50 - loss 2.75515049 - samples/sec: 88.24 - lr: 0.250000 2021-03-26 07:29:40,377 epoch 39 - iter 20/50 - loss 2.72164462 - samples/sec: 86.86 - lr: 0.250000 2021-03-26 07:29:42,317 epoch 39 - iter 25/50 - loss 2.82088650 - samples/sec: 82.52 - lr: 0.250000 2021-03-26 07:29:44,130 epoch 39 - iter 30/50 - loss 2.86356262 - samples/sec: 88.37 - lr: 0.250000 2021-03-26 07:29:46,074 epoch 39 - iter 35/50 - loss 2.78411827 - samples/sec: 82.34 - lr: 0.250000 2021-03-26 07:29:48,047 epoch 39 - iter 40/50 - loss 2.78368353 - samples/sec: 81.18 - lr: 0.250000 2021-03-26 07:29:50,249 epoch 39 - iter 45/50 - loss 2.78644994 - samples/sec: 72.71 - lr: 0.250000 2021-03-26 07:29:52,012 epoch 39 - iter 50/50 - loss 2.91670645 - samples/sec: 90.85 - lr: 0.250000 2021-03-26 07:29:52,013 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:29:52,014 EPOCH 39 done: loss 2.9167 - lr 0.2500000 2021-03-26 07:29:52,784 DEV : loss 6.61793851852417 - score 0.8997 2021-03-26 07:29:52,806 BAD EPOCHS (no improvement): 3 2021-03-26 07:29:52,807 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:29:54,705 epoch 40 - iter 5/50 - loss 2.90698042 - samples/sec: 84.39 - lr: 0.250000 2021-03-26 07:29:56,712 epoch 40 - iter 10/50 - loss 3.07795951 - samples/sec: 79.80 - lr: 0.250000 2021-03-26 07:29:58,621 epoch 40 - iter 15/50 - loss 3.00809290 - samples/sec: 83.88 - lr: 0.250000 2021-03-26 07:30:00,450 epoch 40 - iter 20/50 - loss 2.98251244 - samples/sec: 87.58 - lr: 0.250000 2021-03-26 07:30:02,370 epoch 40 - iter 25/50 - loss 3.01671128 - samples/sec: 83.44 - lr: 0.250000 2021-03-26 07:30:04,182 epoch 40 - iter 30/50 - loss 2.96408661 - samples/sec: 88.39 - lr: 0.250000 2021-03-26 07:30:06,157 epoch 40 - iter 35/50 - loss 2.98403021 - samples/sec: 81.05 - lr: 0.250000 2021-03-26 07:30:08,207 epoch 40 - iter 40/50 - loss 2.96908256 - samples/sec: 78.14 - lr: 0.250000 2021-03-26 07:30:10,113 epoch 40 - iter 45/50 - loss 2.96535072 - samples/sec: 84.02 - lr: 0.250000 2021-03-26 07:30:11,796 epoch 40 - iter 50/50 - loss 2.92278719 - samples/sec: 95.15 - lr: 0.250000 2021-03-26 07:30:11,797 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:30:11,797 EPOCH 40 done: loss 2.9228 - lr 0.2500000 2021-03-26 07:30:12,573 DEV : loss 6.909717559814453 - score 0.8993 2021-03-26 07:30:12,589 BAD EPOCHS (no improvement): 4 2021-03-26 07:30:12,589 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:30:14,528 epoch 41 - iter 5/50 - loss 2.69951725 - samples/sec: 82.61 - lr: 0.125000 2021-03-26 07:30:16,345 epoch 41 - iter 10/50 - loss 2.91924675 - samples/sec: 88.13 - lr: 0.125000 2021-03-26 07:30:18,195 epoch 41 - iter 15/50 - loss 2.98765427 - samples/sec: 86.59 - lr: 0.125000 2021-03-26 07:30:20,152 epoch 41 - iter 20/50 - loss 2.92043756 - samples/sec: 81.82 - lr: 0.125000 2021-03-26 07:30:22,045 epoch 41 - iter 25/50 - loss 2.89524738 - samples/sec: 84.59 - lr: 0.125000 2021-03-26 07:30:23,910 epoch 41 - iter 30/50 - loss 2.76787079 - samples/sec: 85.93 - lr: 0.125000 2021-03-26 07:30:25,981 epoch 41 - iter 35/50 - loss 2.83275717 - samples/sec: 77.31 - lr: 0.125000 2021-03-26 07:30:27,874 epoch 41 - iter 40/50 - loss 2.84688303 - samples/sec: 84.62 - lr: 0.125000 2021-03-26 07:30:29,934 epoch 41 - iter 45/50 - loss 2.83739944 - samples/sec: 77.73 - lr: 0.125000 2021-03-26 07:30:31,677 epoch 41 - iter 50/50 - loss 2.75893023 - samples/sec: 91.90 - lr: 0.125000 2021-03-26 07:30:31,678 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:30:31,678 EPOCH 41 done: loss 2.7589 - lr 0.1250000 2021-03-26 07:30:32,451 DEV : loss 7.06007194519043 - score 0.9005 2021-03-26 07:30:32,474 BAD EPOCHS (no improvement): 1 2021-03-26 07:30:32,475 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:30:34,232 epoch 42 - iter 5/50 - loss 2.20300517 - samples/sec: 91.14 - lr: 0.125000 2021-03-26 07:30:36,209 epoch 42 - iter 10/50 - loss 2.60058794 - samples/sec: 81.01 - lr: 0.125000 2021-03-26 07:30:38,096 epoch 42 - iter 15/50 - loss 2.56362259 - samples/sec: 84.89 - lr: 0.125000 2021-03-26 07:30:40,114 epoch 42 - iter 20/50 - loss 2.60823840 - samples/sec: 79.32 - lr: 0.125000 2021-03-26 07:30:41,981 epoch 42 - iter 25/50 - loss 2.54176937 - samples/sec: 85.79 - lr: 0.125000 2021-03-26 07:30:43,878 epoch 42 - iter 30/50 - loss 2.59154145 - samples/sec: 84.39 - lr: 0.125000 2021-03-26 07:30:45,792 epoch 42 - iter 35/50 - loss 2.60671992 - samples/sec: 83.68 - lr: 0.125000 2021-03-26 07:30:47,760 epoch 42 - iter 40/50 - loss 2.60145665 - samples/sec: 81.36 - lr: 0.125000 2021-03-26 07:30:49,630 epoch 42 - iter 45/50 - loss 2.62242535 - samples/sec: 85.65 - lr: 0.125000 2021-03-26 07:30:51,461 epoch 42 - iter 50/50 - loss 2.64225022 - samples/sec: 87.44 - lr: 0.125000 2021-03-26 07:30:51,462 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:30:51,462 EPOCH 42 done: loss 2.6423 - lr 0.1250000 2021-03-26 07:30:52,213 DEV : loss 6.793901443481445 - score 0.8985 2021-03-26 07:30:52,237 BAD EPOCHS (no improvement): 2 2021-03-26 07:30:52,237 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:30:54,094 epoch 43 - iter 5/50 - loss 2.99211526 - samples/sec: 86.26 - lr: 0.125000 2021-03-26 07:30:55,996 epoch 43 - iter 10/50 - loss 3.00851283 - samples/sec: 84.22 - lr: 0.125000 2021-03-26 07:30:57,841 epoch 43 - iter 15/50 - loss 2.80074444 - samples/sec: 86.83 - lr: 0.125000 2021-03-26 07:30:59,641 epoch 43 - iter 20/50 - loss 2.79492873 - samples/sec: 88.95 - lr: 0.125000 2021-03-26 07:31:01,367 epoch 43 - iter 25/50 - loss 2.69082284 - samples/sec: 92.77 - lr: 0.125000 2021-03-26 07:31:03,222 epoch 43 - iter 30/50 - loss 2.71694736 - samples/sec: 86.36 - lr: 0.125000 2021-03-26 07:31:04,991 epoch 43 - iter 35/50 - loss 2.74828994 - samples/sec: 90.51 - lr: 0.125000 2021-03-26 07:31:06,992 epoch 43 - iter 40/50 - loss 2.68706407 - samples/sec: 80.06 - lr: 0.125000 2021-03-26 07:31:08,941 epoch 43 - iter 45/50 - loss 2.67314792 - samples/sec: 82.15 - lr: 0.125000 2021-03-26 07:31:10,759 epoch 43 - iter 50/50 - loss 2.70004877 - samples/sec: 88.09 - lr: 0.125000 2021-03-26 07:31:10,760 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:31:10,760 EPOCH 43 done: loss 2.7000 - lr 0.1250000 2021-03-26 07:31:12,603 DEV : loss 6.907846450805664 - score 0.8958 2021-03-26 07:31:12,627 BAD EPOCHS (no improvement): 3 2021-03-26 07:31:12,628 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:31:14,573 epoch 44 - iter 5/50 - loss 2.65794680 - samples/sec: 82.31 - lr: 0.125000 2021-03-26 07:31:16,429 epoch 44 - iter 10/50 - loss 2.65035695 - samples/sec: 86.29 - lr: 0.125000 2021-03-26 07:31:18,413 epoch 44 - iter 15/50 - loss 2.43190594 - samples/sec: 80.73 - lr: 0.125000 2021-03-26 07:31:20,506 epoch 44 - iter 20/50 - loss 2.35712910 - samples/sec: 76.49 - lr: 0.125000 2021-03-26 07:31:22,407 epoch 44 - iter 25/50 - loss 2.42408473 - samples/sec: 84.25 - lr: 0.125000 2021-03-26 07:31:24,360 epoch 44 - iter 30/50 - loss 2.43956635 - samples/sec: 82.00 - lr: 0.125000 2021-03-26 07:31:26,108 epoch 44 - iter 35/50 - loss 2.39717156 - samples/sec: 91.59 - lr: 0.125000 2021-03-26 07:31:27,983 epoch 44 - iter 40/50 - loss 2.40287006 - samples/sec: 85.42 - lr: 0.125000 2021-03-26 07:31:29,855 epoch 44 - iter 45/50 - loss 2.48455736 - samples/sec: 85.52 - lr: 0.125000 2021-03-26 07:31:31,687 epoch 44 - iter 50/50 - loss 2.45833510 - samples/sec: 87.43 - lr: 0.125000 2021-03-26 07:31:31,687 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:31:31,688 EPOCH 44 done: loss 2.4583 - lr 0.1250000 2021-03-26 07:31:32,430 DEV : loss 6.795278072357178 - score 0.9013 2021-03-26 07:31:32,453 BAD EPOCHS (no improvement): 0 2021-03-26 07:31:41,582 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:31:43,686 epoch 45 - iter 5/50 - loss 2.66865928 - samples/sec: 76.12 - lr: 0.125000 2021-03-26 07:31:45,682 epoch 45 - iter 10/50 - loss 2.62356399 - samples/sec: 80.21 - lr: 0.125000 2021-03-26 07:31:47,632 epoch 45 - iter 15/50 - loss 2.62658451 - samples/sec: 82.13 - lr: 0.125000 2021-03-26 07:31:49,495 epoch 45 - iter 20/50 - loss 2.51406005 - samples/sec: 85.93 - lr: 0.125000 2021-03-26 07:31:51,298 epoch 45 - iter 25/50 - loss 2.43753824 - samples/sec: 88.83 - lr: 0.125000 2021-03-26 07:31:53,130 epoch 45 - iter 30/50 - loss 2.41095030 - samples/sec: 87.46 - lr: 0.125000 2021-03-26 07:31:55,089 epoch 45 - iter 35/50 - loss 2.47412571 - samples/sec: 81.72 - lr: 0.125000 2021-03-26 07:31:57,064 epoch 45 - iter 40/50 - loss 2.51294028 - samples/sec: 81.11 - lr: 0.125000 2021-03-26 07:31:59,012 epoch 45 - iter 45/50 - loss 2.51155817 - samples/sec: 82.21 - lr: 0.125000 2021-03-26 07:32:00,785 epoch 45 - iter 50/50 - loss 2.48287714 - samples/sec: 90.32 - lr: 0.125000 2021-03-26 07:32:00,786 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:32:00,786 EPOCH 45 done: loss 2.4829 - lr 0.1250000 2021-03-26 07:32:01,572 DEV : loss 6.7710466384887695 - score 0.9001 2021-03-26 07:32:01,587 BAD EPOCHS (no improvement): 1 2021-03-26 07:32:01,588 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:32:03,478 epoch 46 - iter 5/50 - loss 2.60788822 - samples/sec: 84.73 - lr: 0.125000 2021-03-26 07:32:05,434 epoch 46 - iter 10/50 - loss 2.64636929 - samples/sec: 81.88 - lr: 0.125000 2021-03-26 07:32:07,406 epoch 46 - iter 15/50 - loss 2.56494276 - samples/sec: 81.19 - lr: 0.125000 2021-03-26 07:32:09,479 epoch 46 - iter 20/50 - loss 2.48922122 - samples/sec: 77.25 - lr: 0.125000 2021-03-26 07:32:11,354 epoch 46 - iter 25/50 - loss 2.39800469 - samples/sec: 85.41 - lr: 0.125000 2021-03-26 07:32:13,496 epoch 46 - iter 30/50 - loss 2.46986194 - samples/sec: 74.73 - lr: 0.125000 2021-03-26 07:32:15,438 epoch 46 - iter 35/50 - loss 2.52575457 - samples/sec: 82.46 - lr: 0.125000 2021-03-26 07:32:17,230 epoch 46 - iter 40/50 - loss 2.51320742 - samples/sec: 89.42 - lr: 0.125000 2021-03-26 07:32:19,111 epoch 46 - iter 45/50 - loss 2.50932122 - samples/sec: 85.10 - lr: 0.125000 2021-03-26 07:32:20,931 epoch 46 - iter 50/50 - loss 2.51372438 - samples/sec: 87.99 - lr: 0.125000 2021-03-26 07:32:20,932 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:32:20,933 EPOCH 46 done: loss 2.5137 - lr 0.1250000 2021-03-26 07:32:21,761 DEV : loss 6.900432109832764 - score 0.9009 2021-03-26 07:32:21,784 BAD EPOCHS (no improvement): 2 2021-03-26 07:32:21,785 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:32:23,828 epoch 47 - iter 5/50 - loss 2.17711518 - samples/sec: 78.39 - lr: 0.125000 2021-03-26 07:32:25,751 epoch 47 - iter 10/50 - loss 2.16294295 - samples/sec: 83.29 - lr: 0.125000 2021-03-26 07:32:27,646 epoch 47 - iter 15/50 - loss 2.39153829 - samples/sec: 84.51 - lr: 0.125000 2021-03-26 07:32:29,583 epoch 47 - iter 20/50 - loss 2.47488337 - samples/sec: 82.68 - lr: 0.125000 2021-03-26 07:32:31,383 epoch 47 - iter 25/50 - loss 2.43727724 - samples/sec: 88.96 - lr: 0.125000 2021-03-26 07:32:33,312 epoch 47 - iter 30/50 - loss 2.39282555 - samples/sec: 83.04 - lr: 0.125000 2021-03-26 07:32:35,297 epoch 47 - iter 35/50 - loss 2.37608892 - samples/sec: 80.67 - lr: 0.125000 2021-03-26 07:32:37,316 epoch 47 - iter 40/50 - loss 2.43819055 - samples/sec: 79.31 - lr: 0.125000 2021-03-26 07:32:39,249 epoch 47 - iter 45/50 - loss 2.43623858 - samples/sec: 82.85 - lr: 0.125000 2021-03-26 07:32:41,043 epoch 47 - iter 50/50 - loss 2.43931659 - samples/sec: 89.28 - lr: 0.125000 2021-03-26 07:32:41,044 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:32:41,044 EPOCH 47 done: loss 2.4393 - lr 0.1250000 2021-03-26 07:32:41,829 DEV : loss 6.888872146606445 - score 0.9058 2021-03-26 07:32:41,853 BAD EPOCHS (no improvement): 0 2021-03-26 07:32:51,326 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:32:53,272 epoch 48 - iter 5/50 - loss 2.19716325 - samples/sec: 82.35 - lr: 0.125000 2021-03-26 07:32:55,250 epoch 48 - iter 10/50 - loss 2.12402178 - samples/sec: 80.97 - lr: 0.125000 2021-03-26 07:32:57,196 epoch 48 - iter 15/50 - loss 2.37712937 - samples/sec: 82.29 - lr: 0.125000 2021-03-26 07:32:59,158 epoch 48 - iter 20/50 - loss 2.46164612 - samples/sec: 81.59 - lr: 0.125000 2021-03-26 07:33:01,030 epoch 48 - iter 25/50 - loss 2.40853381 - samples/sec: 85.58 - lr: 0.125000 2021-03-26 07:33:02,830 epoch 48 - iter 30/50 - loss 2.40115452 - samples/sec: 88.98 - lr: 0.125000 2021-03-26 07:33:04,636 epoch 48 - iter 35/50 - loss 2.41172790 - samples/sec: 88.72 - lr: 0.125000 2021-03-26 07:33:06,417 epoch 48 - iter 40/50 - loss 2.43344751 - samples/sec: 89.90 - lr: 0.125000 2021-03-26 07:33:08,403 epoch 48 - iter 45/50 - loss 2.36635066 - samples/sec: 80.63 - lr: 0.125000 2021-03-26 07:33:10,071 epoch 48 - iter 50/50 - loss 2.31805849 - samples/sec: 96.03 - lr: 0.125000 2021-03-26 07:33:10,072 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:33:10,072 EPOCH 48 done: loss 2.3181 - lr 0.1250000 2021-03-26 07:33:10,825 DEV : loss 6.771639823913574 - score 0.9038 2021-03-26 07:33:10,849 BAD EPOCHS (no improvement): 1 2021-03-26 07:33:10,850 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:33:12,763 epoch 49 - iter 5/50 - loss 2.02018476 - samples/sec: 83.75 - lr: 0.125000 2021-03-26 07:33:14,858 epoch 49 - iter 10/50 - loss 2.49009266 - samples/sec: 76.47 - lr: 0.125000 2021-03-26 07:33:16,829 epoch 49 - iter 15/50 - loss 2.47853305 - samples/sec: 81.24 - lr: 0.125000 2021-03-26 07:33:18,545 epoch 49 - iter 20/50 - loss 2.35995474 - samples/sec: 93.34 - lr: 0.125000 2021-03-26 07:33:20,647 epoch 49 - iter 25/50 - loss 2.44599000 - samples/sec: 76.17 - lr: 0.125000 2021-03-26 07:33:22,585 epoch 49 - iter 30/50 - loss 2.42486334 - samples/sec: 82.63 - lr: 0.125000 2021-03-26 07:33:24,695 epoch 49 - iter 35/50 - loss 2.40869852 - samples/sec: 75.91 - lr: 0.125000 2021-03-26 07:33:26,614 epoch 49 - iter 40/50 - loss 2.40710014 - samples/sec: 83.44 - lr: 0.125000 2021-03-26 07:33:28,519 epoch 49 - iter 45/50 - loss 2.39904715 - samples/sec: 84.05 - lr: 0.125000 2021-03-26 07:33:30,267 epoch 49 - iter 50/50 - loss 2.49603957 - samples/sec: 91.62 - lr: 0.125000 2021-03-26 07:33:30,268 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:33:30,269 EPOCH 49 done: loss 2.4960 - lr 0.1250000 2021-03-26 07:33:31,042 DEV : loss 6.907740592956543 - score 0.8997 2021-03-26 07:33:31,066 BAD EPOCHS (no improvement): 2 2021-03-26 07:33:31,067 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:33:33,085 epoch 50 - iter 5/50 - loss 2.59914744 - samples/sec: 79.35 - lr: 0.125000 2021-03-26 07:33:35,070 epoch 50 - iter 10/50 - loss 2.43452330 - samples/sec: 80.65 - lr: 0.125000 2021-03-26 07:33:37,278 epoch 50 - iter 15/50 - loss 2.46958216 - samples/sec: 72.53 - lr: 0.125000 2021-03-26 07:33:39,115 epoch 50 - iter 20/50 - loss 2.50137873 - samples/sec: 87.18 - lr: 0.125000 2021-03-26 07:33:40,963 epoch 50 - iter 25/50 - loss 2.44457360 - samples/sec: 86.64 - lr: 0.125000 2021-03-26 07:33:42,959 epoch 50 - iter 30/50 - loss 2.38336607 - samples/sec: 80.24 - lr: 0.125000 2021-03-26 07:33:44,884 epoch 50 - iter 35/50 - loss 2.34478426 - samples/sec: 83.19 - lr: 0.125000 2021-03-26 07:33:46,766 epoch 50 - iter 40/50 - loss 2.36545067 - samples/sec: 85.06 - lr: 0.125000 2021-03-26 07:33:48,718 epoch 50 - iter 45/50 - loss 2.32171084 - samples/sec: 82.04 - lr: 0.125000 2021-03-26 07:33:50,552 epoch 50 - iter 50/50 - loss 2.32172669 - samples/sec: 87.32 - lr: 0.125000 2021-03-26 07:33:50,553 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:33:50,554 EPOCH 50 done: loss 2.3217 - lr 0.1250000 2021-03-26 07:33:51,325 DEV : loss 6.9584059715271 - score 0.9001 2021-03-26 07:33:51,348 BAD EPOCHS (no improvement): 3 2021-03-26 07:33:51,349 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:33:53,245 epoch 51 - iter 5/50 - loss 2.67340035 - samples/sec: 84.46 - lr: 0.125000 2021-03-26 07:33:55,107 epoch 51 - iter 10/50 - loss 2.55235028 - samples/sec: 86.03 - lr: 0.125000 2021-03-26 07:33:57,007 epoch 51 - iter 15/50 - loss 2.41178315 - samples/sec: 84.26 - lr: 0.125000 2021-03-26 07:33:59,093 epoch 51 - iter 20/50 - loss 2.43315674 - samples/sec: 76.78 - lr: 0.125000 2021-03-26 07:34:00,902 epoch 51 - iter 25/50 - loss 2.43270576 - samples/sec: 88.55 - lr: 0.125000 2021-03-26 07:34:02,906 epoch 51 - iter 30/50 - loss 2.46247502 - samples/sec: 79.89 - lr: 0.125000 2021-03-26 07:34:04,810 epoch 51 - iter 35/50 - loss 2.42736442 - samples/sec: 84.15 - lr: 0.125000 2021-03-26 07:34:06,616 epoch 51 - iter 40/50 - loss 2.43767639 - samples/sec: 88.66 - lr: 0.125000 2021-03-26 07:34:08,503 epoch 51 - iter 45/50 - loss 2.43689898 - samples/sec: 84.89 - lr: 0.125000 2021-03-26 07:34:10,323 epoch 51 - iter 50/50 - loss 2.46275803 - samples/sec: 88.00 - lr: 0.125000 2021-03-26 07:34:10,324 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:34:10,325 EPOCH 51 done: loss 2.4628 - lr 0.1250000 2021-03-26 07:34:11,097 DEV : loss 7.135655879974365 - score 0.8993 2021-03-26 07:34:11,120 BAD EPOCHS (no improvement): 4 2021-03-26 07:34:11,121 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:34:13,126 epoch 52 - iter 5/50 - loss 2.37440939 - samples/sec: 79.89 - lr: 0.062500 2021-03-26 07:34:15,203 epoch 52 - iter 10/50 - loss 2.19602715 - samples/sec: 77.09 - lr: 0.062500 2021-03-26 07:34:17,264 epoch 52 - iter 15/50 - loss 2.14743586 - samples/sec: 77.68 - lr: 0.062500 2021-03-26 07:34:19,074 epoch 52 - iter 20/50 - loss 2.16298716 - samples/sec: 88.50 - lr: 0.062500 2021-03-26 07:34:20,918 epoch 52 - iter 25/50 - loss 2.23282931 - samples/sec: 86.83 - lr: 0.062500 2021-03-26 07:34:22,807 epoch 52 - iter 30/50 - loss 2.24415377 - samples/sec: 84.77 - lr: 0.062500 2021-03-26 07:34:24,701 epoch 52 - iter 35/50 - loss 2.22840363 - samples/sec: 84.57 - lr: 0.062500 2021-03-26 07:34:26,556 epoch 52 - iter 40/50 - loss 2.27091244 - samples/sec: 86.31 - lr: 0.062500 2021-03-26 07:34:28,460 epoch 52 - iter 45/50 - loss 2.29454314 - samples/sec: 84.17 - lr: 0.062500 2021-03-26 07:34:30,182 epoch 52 - iter 50/50 - loss 2.28486887 - samples/sec: 92.97 - lr: 0.062500 2021-03-26 07:34:30,183 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:34:30,183 EPOCH 52 done: loss 2.2849 - lr 0.0625000 2021-03-26 07:34:31,000 DEV : loss 6.901858329772949 - score 0.9021 2021-03-26 07:34:31,024 BAD EPOCHS (no improvement): 1 2021-03-26 07:34:31,025 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:34:32,894 epoch 53 - iter 5/50 - loss 2.22966437 - samples/sec: 85.69 - lr: 0.062500 2021-03-26 07:34:34,851 epoch 53 - iter 10/50 - loss 2.42526587 - samples/sec: 81.82 - lr: 0.062500 2021-03-26 07:34:36,650 epoch 53 - iter 15/50 - loss 2.38672217 - samples/sec: 89.04 - lr: 0.062500 2021-03-26 07:34:38,679 epoch 53 - iter 20/50 - loss 2.36199059 - samples/sec: 78.92 - lr: 0.062500 2021-03-26 07:34:40,667 epoch 53 - iter 25/50 - loss 2.39168876 - samples/sec: 80.53 - lr: 0.062500 2021-03-26 07:34:42,463 epoch 53 - iter 30/50 - loss 2.40719418 - samples/sec: 89.20 - lr: 0.062500 2021-03-26 07:34:44,433 epoch 53 - iter 35/50 - loss 2.35297212 - samples/sec: 81.29 - lr: 0.062500 2021-03-26 07:34:46,681 epoch 53 - iter 40/50 - loss 2.35524164 - samples/sec: 71.21 - lr: 0.062500 2021-03-26 07:34:48,626 epoch 53 - iter 45/50 - loss 2.35598461 - samples/sec: 82.32 - lr: 0.062500 2021-03-26 07:34:50,347 epoch 53 - iter 50/50 - loss 2.37433032 - samples/sec: 93.04 - lr: 0.062500 2021-03-26 07:34:50,349 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:34:50,349 EPOCH 53 done: loss 2.3743 - lr 0.0625000 2021-03-26 07:34:51,113 DEV : loss 6.839196681976318 - score 0.9046 2021-03-26 07:34:51,136 BAD EPOCHS (no improvement): 2 2021-03-26 07:34:51,137 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:34:53,007 epoch 54 - iter 5/50 - loss 2.27673917 - samples/sec: 85.66 - lr: 0.062500 2021-03-26 07:34:55,023 epoch 54 - iter 10/50 - loss 2.14428871 - samples/sec: 79.43 - lr: 0.062500 2021-03-26 07:34:56,831 epoch 54 - iter 15/50 - loss 2.28928282 - samples/sec: 88.58 - lr: 0.062500 2021-03-26 07:34:58,829 epoch 54 - iter 20/50 - loss 2.22522359 - samples/sec: 80.16 - lr: 0.062500 2021-03-26 07:35:00,861 epoch 54 - iter 25/50 - loss 2.26239754 - samples/sec: 78.81 - lr: 0.062500 2021-03-26 07:35:02,671 epoch 54 - iter 30/50 - loss 2.30489551 - samples/sec: 88.46 - lr: 0.062500 2021-03-26 07:35:04,581 epoch 54 - iter 35/50 - loss 2.31927896 - samples/sec: 83.85 - lr: 0.062500 2021-03-26 07:35:06,708 epoch 54 - iter 40/50 - loss 2.34227904 - samples/sec: 75.29 - lr: 0.062500 2021-03-26 07:35:08,487 epoch 54 - iter 45/50 - loss 2.33702100 - samples/sec: 90.05 - lr: 0.062500 2021-03-26 07:35:10,331 epoch 54 - iter 50/50 - loss 2.34119186 - samples/sec: 86.83 - lr: 0.062500 2021-03-26 07:35:10,332 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:35:10,332 EPOCH 54 done: loss 2.3412 - lr 0.0625000 2021-03-26 07:35:11,131 DEV : loss 6.923906326293945 - score 0.9042 2021-03-26 07:35:11,164 BAD EPOCHS (no improvement): 3 2021-03-26 07:35:11,165 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:35:12,986 epoch 55 - iter 5/50 - loss 2.08973429 - samples/sec: 87.94 - lr: 0.062500 2021-03-26 07:35:14,894 epoch 55 - iter 10/50 - loss 2.20089608 - samples/sec: 83.92 - lr: 0.062500 2021-03-26 07:35:16,799 epoch 55 - iter 15/50 - loss 2.15231233 - samples/sec: 84.08 - lr: 0.062500 2021-03-26 07:35:18,687 epoch 55 - iter 20/50 - loss 2.22952471 - samples/sec: 84.85 - lr: 0.062500 2021-03-26 07:35:20,659 epoch 55 - iter 25/50 - loss 2.24563358 - samples/sec: 81.17 - lr: 0.062500 2021-03-26 07:35:22,479 epoch 55 - iter 30/50 - loss 2.27382298 - samples/sec: 88.04 - lr: 0.062500 2021-03-26 07:35:24,381 epoch 55 - iter 35/50 - loss 2.24468018 - samples/sec: 84.19 - lr: 0.062500 2021-03-26 07:35:26,116 epoch 55 - iter 40/50 - loss 2.24657573 - samples/sec: 92.28 - lr: 0.062500 2021-03-26 07:35:27,975 epoch 55 - iter 45/50 - loss 2.27305697 - samples/sec: 86.13 - lr: 0.062500 2021-03-26 07:35:29,662 epoch 55 - iter 50/50 - loss 2.28477174 - samples/sec: 94.96 - lr: 0.062500 2021-03-26 07:35:29,663 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:35:29,663 EPOCH 55 done: loss 2.2848 - lr 0.0625000 2021-03-26 07:35:30,400 DEV : loss 6.859457969665527 - score 0.903 2021-03-26 07:35:30,423 BAD EPOCHS (no improvement): 4 2021-03-26 07:35:30,423 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:35:32,314 epoch 56 - iter 5/50 - loss 2.43909552 - samples/sec: 84.73 - lr: 0.031250 2021-03-26 07:35:34,214 epoch 56 - iter 10/50 - loss 2.14584787 - samples/sec: 84.30 - lr: 0.031250 2021-03-26 07:35:36,107 epoch 56 - iter 15/50 - loss 2.06615899 - samples/sec: 84.62 - lr: 0.031250 2021-03-26 07:35:37,919 epoch 56 - iter 20/50 - loss 2.14122699 - samples/sec: 88.35 - lr: 0.031250 2021-03-26 07:35:39,958 epoch 56 - iter 25/50 - loss 2.30763709 - samples/sec: 78.56 - lr: 0.031250 2021-03-26 07:35:41,878 epoch 56 - iter 30/50 - loss 2.30141586 - samples/sec: 83.42 - lr: 0.031250 2021-03-26 07:35:43,886 epoch 56 - iter 35/50 - loss 2.27650926 - samples/sec: 79.73 - lr: 0.031250 2021-03-26 07:35:45,623 epoch 56 - iter 40/50 - loss 2.20432305 - samples/sec: 92.22 - lr: 0.031250 2021-03-26 07:35:47,470 epoch 56 - iter 45/50 - loss 2.18452919 - samples/sec: 86.69 - lr: 0.031250 2021-03-26 07:35:49,279 epoch 56 - iter 50/50 - loss 2.20786890 - samples/sec: 88.54 - lr: 0.031250 2021-03-26 07:35:49,279 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:35:49,280 EPOCH 56 done: loss 2.2079 - lr 0.0312500 2021-03-26 07:35:50,032 DEV : loss 6.856510162353516 - score 0.9034 2021-03-26 07:35:50,056 BAD EPOCHS (no improvement): 1 2021-03-26 07:35:50,057 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:35:51,895 epoch 57 - iter 5/50 - loss 2.20711362 - samples/sec: 87.11 - lr: 0.031250 2021-03-26 07:35:53,859 epoch 57 - iter 10/50 - loss 2.09820828 - samples/sec: 81.53 - lr: 0.031250 2021-03-26 07:35:55,852 epoch 57 - iter 15/50 - loss 2.17774463 - samples/sec: 80.39 - lr: 0.031250 2021-03-26 07:35:57,847 epoch 57 - iter 20/50 - loss 2.19380504 - samples/sec: 80.25 - lr: 0.031250 2021-03-26 07:35:59,793 epoch 57 - iter 25/50 - loss 2.25433336 - samples/sec: 82.30 - lr: 0.031250 2021-03-26 07:36:01,485 epoch 57 - iter 30/50 - loss 2.18909953 - samples/sec: 94.70 - lr: 0.031250 2021-03-26 07:36:03,414 epoch 57 - iter 35/50 - loss 2.21699893 - samples/sec: 83.00 - lr: 0.031250 2021-03-26 07:36:05,258 epoch 57 - iter 40/50 - loss 2.20707081 - samples/sec: 86.82 - lr: 0.031250 2021-03-26 07:36:07,154 epoch 57 - iter 45/50 - loss 2.18183962 - samples/sec: 84.49 - lr: 0.031250 2021-03-26 07:36:08,967 epoch 57 - iter 50/50 - loss 2.14463658 - samples/sec: 88.33 - lr: 0.031250 2021-03-26 07:36:08,968 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:36:08,968 EPOCH 57 done: loss 2.1446 - lr 0.0312500 2021-03-26 07:36:09,739 DEV : loss 6.898594856262207 - score 0.9021 2021-03-26 07:36:09,762 BAD EPOCHS (no improvement): 2 2021-03-26 07:36:09,763 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:36:11,604 epoch 58 - iter 5/50 - loss 2.11004088 - samples/sec: 86.98 - lr: 0.031250 2021-03-26 07:36:13,716 epoch 58 - iter 10/50 - loss 2.17785354 - samples/sec: 75.84 - lr: 0.031250 2021-03-26 07:36:15,752 epoch 58 - iter 15/50 - loss 2.09577750 - samples/sec: 78.66 - lr: 0.031250 2021-03-26 07:36:17,695 epoch 58 - iter 20/50 - loss 2.15892507 - samples/sec: 82.43 - lr: 0.031250 2021-03-26 07:36:19,600 epoch 58 - iter 25/50 - loss 2.18243889 - samples/sec: 84.09 - lr: 0.031250 2021-03-26 07:36:21,597 epoch 58 - iter 30/50 - loss 2.20849555 - samples/sec: 80.18 - lr: 0.031250 2021-03-26 07:36:23,455 epoch 58 - iter 35/50 - loss 2.19579933 - samples/sec: 86.18 - lr: 0.031250 2021-03-26 07:36:25,829 epoch 58 - iter 40/50 - loss 2.20347180 - samples/sec: 67.46 - lr: 0.031250 2021-03-26 07:36:27,922 epoch 58 - iter 45/50 - loss 2.14917173 - samples/sec: 76.51 - lr: 0.031250 2021-03-26 07:36:29,699 epoch 58 - iter 50/50 - loss 2.12325423 - samples/sec: 90.12 - lr: 0.031250 2021-03-26 07:36:29,700 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:36:29,700 EPOCH 58 done: loss 2.1233 - lr 0.0312500 2021-03-26 07:36:30,470 DEV : loss 6.878035545349121 - score 0.9021 2021-03-26 07:36:30,486 BAD EPOCHS (no improvement): 3 2021-03-26 07:36:30,487 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:36:32,199 epoch 59 - iter 5/50 - loss 1.79576149 - samples/sec: 93.55 - lr: 0.031250 2021-03-26 07:36:34,316 epoch 59 - iter 10/50 - loss 1.99188758 - samples/sec: 75.64 - lr: 0.031250 2021-03-26 07:36:36,503 epoch 59 - iter 15/50 - loss 1.93114869 - samples/sec: 73.23 - lr: 0.031250 2021-03-26 07:36:38,470 epoch 59 - iter 20/50 - loss 2.04898582 - samples/sec: 81.41 - lr: 0.031250 2021-03-26 07:36:40,451 epoch 59 - iter 25/50 - loss 2.05024611 - samples/sec: 80.83 - lr: 0.031250 2021-03-26 07:36:42,341 epoch 59 - iter 30/50 - loss 2.11055830 - samples/sec: 84.71 - lr: 0.031250 2021-03-26 07:36:44,205 epoch 59 - iter 35/50 - loss 2.07869013 - samples/sec: 85.90 - lr: 0.031250 2021-03-26 07:36:46,054 epoch 59 - iter 40/50 - loss 2.07881030 - samples/sec: 86.63 - lr: 0.031250 2021-03-26 07:36:47,989 epoch 59 - iter 45/50 - loss 2.10570982 - samples/sec: 82.75 - lr: 0.031250 2021-03-26 07:36:49,789 epoch 59 - iter 50/50 - loss 2.15876318 - samples/sec: 88.98 - lr: 0.031250 2021-03-26 07:36:49,790 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:36:49,790 EPOCH 59 done: loss 2.1588 - lr 0.0312500 2021-03-26 07:36:50,538 DEV : loss 6.879229545593262 - score 0.9034 2021-03-26 07:36:50,561 BAD EPOCHS (no improvement): 4 2021-03-26 07:36:50,562 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:36:52,470 epoch 60 - iter 5/50 - loss 2.24569612 - samples/sec: 83.94 - lr: 0.015625 2021-03-26 07:36:54,424 epoch 60 - iter 10/50 - loss 2.16484799 - samples/sec: 81.97 - lr: 0.015625 2021-03-26 07:36:56,284 epoch 60 - iter 15/50 - loss 2.07707895 - samples/sec: 86.09 - lr: 0.015625 2021-03-26 07:36:58,097 epoch 60 - iter 20/50 - loss 2.07254428 - samples/sec: 88.34 - lr: 0.015625 2021-03-26 07:37:00,098 epoch 60 - iter 25/50 - loss 2.11141718 - samples/sec: 80.04 - lr: 0.015625 2021-03-26 07:37:01,970 epoch 60 - iter 30/50 - loss 2.16300511 - samples/sec: 85.56 - lr: 0.015625 2021-03-26 07:37:04,210 epoch 60 - iter 35/50 - loss 2.17970347 - samples/sec: 71.49 - lr: 0.015625 2021-03-26 07:37:06,194 epoch 60 - iter 40/50 - loss 2.14677085 - samples/sec: 80.71 - lr: 0.015625 2021-03-26 07:37:08,151 epoch 60 - iter 45/50 - loss 2.19213055 - samples/sec: 81.82 - lr: 0.015625 2021-03-26 07:37:09,798 epoch 60 - iter 50/50 - loss 2.19411132 - samples/sec: 97.22 - lr: 0.015625 2021-03-26 07:37:09,799 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:37:09,800 EPOCH 60 done: loss 2.1941 - lr 0.0156250 2021-03-26 07:37:10,560 DEV : loss 6.902478218078613 - score 0.9013 2021-03-26 07:37:10,583 BAD EPOCHS (no improvement): 1 2021-03-26 07:37:10,584 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:37:12,558 epoch 61 - iter 5/50 - loss 1.71156173 - samples/sec: 81.11 - lr: 0.015625 2021-03-26 07:37:14,379 epoch 61 - iter 10/50 - loss 2.01954519 - samples/sec: 87.97 - lr: 0.015625 2021-03-26 07:37:16,305 epoch 61 - iter 15/50 - loss 2.12591212 - samples/sec: 83.17 - lr: 0.015625 2021-03-26 07:37:18,326 epoch 61 - iter 20/50 - loss 2.02409774 - samples/sec: 79.24 - lr: 0.015625 2021-03-26 07:37:20,245 epoch 61 - iter 25/50 - loss 2.01767517 - samples/sec: 83.43 - lr: 0.015625 2021-03-26 07:37:22,231 epoch 61 - iter 30/50 - loss 2.13288812 - samples/sec: 80.66 - lr: 0.015625 2021-03-26 07:37:24,110 epoch 61 - iter 35/50 - loss 2.08045435 - samples/sec: 85.21 - lr: 0.015625 2021-03-26 07:37:26,075 epoch 61 - iter 40/50 - loss 2.11307848 - samples/sec: 81.51 - lr: 0.015625 2021-03-26 07:37:28,083 epoch 61 - iter 45/50 - loss 2.10234797 - samples/sec: 79.74 - lr: 0.015625 2021-03-26 07:37:29,939 epoch 61 - iter 50/50 - loss 2.15973810 - samples/sec: 86.28 - lr: 0.015625 2021-03-26 07:37:29,940 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:37:29,940 EPOCH 61 done: loss 2.1597 - lr 0.0156250 2021-03-26 07:37:30,696 DEV : loss 6.937427520751953 - score 0.9034 2021-03-26 07:37:30,719 BAD EPOCHS (no improvement): 2 2021-03-26 07:37:30,720 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:37:32,540 epoch 62 - iter 5/50 - loss 2.36723418 - samples/sec: 88.03 - lr: 0.015625 2021-03-26 07:37:34,377 epoch 62 - iter 10/50 - loss 2.27173837 - samples/sec: 87.16 - lr: 0.015625 2021-03-26 07:37:36,305 epoch 62 - iter 15/50 - loss 2.23157530 - samples/sec: 83.09 - lr: 0.015625 2021-03-26 07:37:38,239 epoch 62 - iter 20/50 - loss 2.18523218 - samples/sec: 82.85 - lr: 0.015625 2021-03-26 07:37:40,194 epoch 62 - iter 25/50 - loss 2.12496989 - samples/sec: 81.88 - lr: 0.015625 2021-03-26 07:37:42,082 epoch 62 - iter 30/50 - loss 2.15814011 - samples/sec: 84.81 - lr: 0.015625 2021-03-26 07:37:43,911 epoch 62 - iter 35/50 - loss 2.14331431 - samples/sec: 87.54 - lr: 0.015625 2021-03-26 07:37:45,790 epoch 62 - iter 40/50 - loss 2.16459944 - samples/sec: 85.25 - lr: 0.015625 2021-03-26 07:37:47,614 epoch 62 - iter 45/50 - loss 2.14495964 - samples/sec: 87.80 - lr: 0.015625 2021-03-26 07:37:49,495 epoch 62 - iter 50/50 - loss 2.15630883 - samples/sec: 85.14 - lr: 0.015625 2021-03-26 07:37:49,495 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:37:49,496 EPOCH 62 done: loss 2.1563 - lr 0.0156250 2021-03-26 07:37:50,270 DEV : loss 6.966155052185059 - score 0.9026 2021-03-26 07:37:50,289 BAD EPOCHS (no improvement): 3 2021-03-26 07:37:50,289 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:37:52,372 epoch 63 - iter 5/50 - loss 2.65498366 - samples/sec: 76.90 - lr: 0.015625 2021-03-26 07:37:54,246 epoch 63 - iter 10/50 - loss 2.41633086 - samples/sec: 85.45 - lr: 0.015625 2021-03-26 07:37:56,141 epoch 63 - iter 15/50 - loss 2.24042091 - samples/sec: 84.55 - lr: 0.015625 2021-03-26 07:37:57,922 epoch 63 - iter 20/50 - loss 2.24434867 - samples/sec: 89.89 - lr: 0.015625 2021-03-26 07:38:00,010 epoch 63 - iter 25/50 - loss 2.21254596 - samples/sec: 76.71 - lr: 0.015625 2021-03-26 07:38:01,846 epoch 63 - iter 30/50 - loss 2.23309576 - samples/sec: 87.21 - lr: 0.015625 2021-03-26 07:38:03,664 epoch 63 - iter 35/50 - loss 2.25727123 - samples/sec: 88.13 - lr: 0.015625 2021-03-26 07:38:05,727 epoch 63 - iter 40/50 - loss 2.29153742 - samples/sec: 77.60 - lr: 0.015625 2021-03-26 07:38:07,670 epoch 63 - iter 45/50 - loss 2.27542347 - samples/sec: 82.43 - lr: 0.015625 2021-03-26 07:38:09,298 epoch 63 - iter 50/50 - loss 2.20794976 - samples/sec: 98.42 - lr: 0.015625 2021-03-26 07:38:09,298 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:38:09,299 EPOCH 63 done: loss 2.2079 - lr 0.0156250 2021-03-26 07:38:10,041 DEV : loss 6.970630645751953 - score 0.9034 2021-03-26 07:38:10,064 BAD EPOCHS (no improvement): 4 2021-03-26 07:38:10,065 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:38:11,867 epoch 64 - iter 5/50 - loss 2.33771241 - samples/sec: 88.87 - lr: 0.007812 2021-03-26 07:38:13,818 epoch 64 - iter 10/50 - loss 2.31490641 - samples/sec: 82.11 - lr: 0.007812 2021-03-26 07:38:15,604 epoch 64 - iter 15/50 - loss 2.26052773 - samples/sec: 89.71 - lr: 0.007812 2021-03-26 07:38:17,435 epoch 64 - iter 20/50 - loss 2.25423276 - samples/sec: 87.44 - lr: 0.007812 2021-03-26 07:38:19,224 epoch 64 - iter 25/50 - loss 2.17361112 - samples/sec: 89.55 - lr: 0.007812 2021-03-26 07:38:21,099 epoch 64 - iter 30/50 - loss 2.22481107 - samples/sec: 85.42 - lr: 0.007812 2021-03-26 07:38:22,964 epoch 64 - iter 35/50 - loss 2.21949503 - samples/sec: 85.83 - lr: 0.007812 2021-03-26 07:38:24,980 epoch 64 - iter 40/50 - loss 2.25171840 - samples/sec: 79.43 - lr: 0.007812 2021-03-26 07:38:26,948 epoch 64 - iter 45/50 - loss 2.24363305 - samples/sec: 81.39 - lr: 0.007812 2021-03-26 07:38:28,687 epoch 64 - iter 50/50 - loss 2.19226521 - samples/sec: 92.10 - lr: 0.007812 2021-03-26 07:38:28,687 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:38:28,688 EPOCH 64 done: loss 2.1923 - lr 0.0078125 2021-03-26 07:38:29,423 DEV : loss 6.981413841247559 - score 0.9038 2021-03-26 07:38:29,445 BAD EPOCHS (no improvement): 1 2021-03-26 07:38:29,446 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:38:31,416 epoch 65 - iter 5/50 - loss 2.65382171 - samples/sec: 81.32 - lr: 0.007812 2021-03-26 07:38:33,343 epoch 65 - iter 10/50 - loss 2.49962165 - samples/sec: 83.10 - lr: 0.007812 2021-03-26 07:38:35,096 epoch 65 - iter 15/50 - loss 2.24140796 - samples/sec: 91.39 - lr: 0.007812 2021-03-26 07:38:37,073 epoch 65 - iter 20/50 - loss 2.36456291 - samples/sec: 80.97 - lr: 0.007812 2021-03-26 07:38:38,881 epoch 65 - iter 25/50 - loss 2.31400333 - samples/sec: 88.57 - lr: 0.007812 2021-03-26 07:38:40,631 epoch 65 - iter 30/50 - loss 2.31376214 - samples/sec: 91.55 - lr: 0.007812 2021-03-26 07:38:42,559 epoch 65 - iter 35/50 - loss 2.36776146 - samples/sec: 83.05 - lr: 0.007812 2021-03-26 07:38:44,738 epoch 65 - iter 40/50 - loss 2.31570167 - samples/sec: 73.49 - lr: 0.007812 2021-03-26 07:38:46,633 epoch 65 - iter 45/50 - loss 2.27233456 - samples/sec: 84.51 - lr: 0.007812 2021-03-26 07:38:48,435 epoch 65 - iter 50/50 - loss 2.19037542 - samples/sec: 88.87 - lr: 0.007812 2021-03-26 07:38:48,435 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:38:48,436 EPOCH 65 done: loss 2.1904 - lr 0.0078125 2021-03-26 07:38:49,231 DEV : loss 6.993993759155273 - score 0.903 2021-03-26 07:38:49,254 BAD EPOCHS (no improvement): 2 2021-03-26 07:38:49,255 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:38:51,251 epoch 66 - iter 5/50 - loss 2.00561562 - samples/sec: 80.23 - lr: 0.007812 2021-03-26 07:38:53,199 epoch 66 - iter 10/50 - loss 2.08727456 - samples/sec: 82.21 - lr: 0.007812 2021-03-26 07:38:54,977 epoch 66 - iter 15/50 - loss 2.09013340 - samples/sec: 90.07 - lr: 0.007812 2021-03-26 07:38:56,739 epoch 66 - iter 20/50 - loss 2.02487870 - samples/sec: 90.88 - lr: 0.007812 2021-03-26 07:38:58,921 epoch 66 - iter 25/50 - loss 2.04558960 - samples/sec: 73.39 - lr: 0.007812 2021-03-26 07:39:00,804 epoch 66 - iter 30/50 - loss 2.07894075 - samples/sec: 85.07 - lr: 0.007812 2021-03-26 07:39:02,528 epoch 66 - iter 35/50 - loss 2.08804049 - samples/sec: 92.88 - lr: 0.007812 2021-03-26 07:39:04,480 epoch 66 - iter 40/50 - loss 2.09458185 - samples/sec: 82.03 - lr: 0.007812 2021-03-26 07:39:06,413 epoch 66 - iter 45/50 - loss 2.11317086 - samples/sec: 82.83 - lr: 0.007812 2021-03-26 07:39:08,271 epoch 66 - iter 50/50 - loss 2.07886559 - samples/sec: 86.20 - lr: 0.007812 2021-03-26 07:39:08,272 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:39:08,272 EPOCH 66 done: loss 2.0789 - lr 0.0078125 2021-03-26 07:39:09,033 DEV : loss 6.994531154632568 - score 0.9034 2021-03-26 07:39:09,056 BAD EPOCHS (no improvement): 3 2021-03-26 07:39:09,057 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:39:10,930 epoch 67 - iter 5/50 - loss 2.03548367 - samples/sec: 85.49 - lr: 0.007812 2021-03-26 07:39:12,657 epoch 67 - iter 10/50 - loss 2.00109907 - samples/sec: 92.76 - lr: 0.007812 2021-03-26 07:39:14,540 epoch 67 - iter 15/50 - loss 2.15618571 - samples/sec: 85.05 - lr: 0.007812 2021-03-26 07:39:16,452 epoch 67 - iter 20/50 - loss 2.18579333 - samples/sec: 83.76 - lr: 0.007812 2021-03-26 07:39:18,346 epoch 67 - iter 25/50 - loss 2.19617332 - samples/sec: 84.57 - lr: 0.007812 2021-03-26 07:39:20,270 epoch 67 - iter 30/50 - loss 2.17452607 - samples/sec: 83.22 - lr: 0.007812 2021-03-26 07:39:22,070 epoch 67 - iter 35/50 - loss 2.18120226 - samples/sec: 88.98 - lr: 0.007812 2021-03-26 07:39:24,166 epoch 67 - iter 40/50 - loss 2.15019251 - samples/sec: 76.39 - lr: 0.007812 2021-03-26 07:39:26,149 epoch 67 - iter 45/50 - loss 2.11730300 - samples/sec: 80.78 - lr: 0.007812 2021-03-26 07:39:27,831 epoch 67 - iter 50/50 - loss 2.13063770 - samples/sec: 95.24 - lr: 0.007812 2021-03-26 07:39:27,831 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:39:27,832 EPOCH 67 done: loss 2.1306 - lr 0.0078125 2021-03-26 07:39:28,606 DEV : loss 6.979950904846191 - score 0.9026 2021-03-26 07:39:28,630 BAD EPOCHS (no improvement): 4 2021-03-26 07:39:28,631 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:39:30,486 epoch 68 - iter 5/50 - loss 2.37680974 - samples/sec: 86.37 - lr: 0.003906 2021-03-26 07:39:32,411 epoch 68 - iter 10/50 - loss 2.26592950 - samples/sec: 83.17 - lr: 0.003906 2021-03-26 07:39:34,282 epoch 68 - iter 15/50 - loss 2.11205655 - samples/sec: 85.57 - lr: 0.003906 2021-03-26 07:39:36,162 epoch 68 - iter 20/50 - loss 2.03751066 - samples/sec: 85.19 - lr: 0.003906 2021-03-26 07:39:37,970 epoch 68 - iter 25/50 - loss 2.08158858 - samples/sec: 88.58 - lr: 0.003906 2021-03-26 07:39:39,700 epoch 68 - iter 30/50 - loss 2.10940475 - samples/sec: 92.57 - lr: 0.003906 2021-03-26 07:39:41,620 epoch 68 - iter 35/50 - loss 2.15269453 - samples/sec: 83.43 - lr: 0.003906 2021-03-26 07:39:43,378 epoch 68 - iter 40/50 - loss 2.13817808 - samples/sec: 91.07 - lr: 0.003906 2021-03-26 07:39:45,278 epoch 68 - iter 45/50 - loss 2.13679235 - samples/sec: 84.29 - lr: 0.003906 2021-03-26 07:39:47,019 epoch 68 - iter 50/50 - loss 2.10934273 - samples/sec: 92.02 - lr: 0.003906 2021-03-26 07:39:47,019 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:39:47,020 EPOCH 68 done: loss 2.1093 - lr 0.0039062 2021-03-26 07:39:47,820 DEV : loss 6.979579925537109 - score 0.903 2021-03-26 07:39:47,843 BAD EPOCHS (no improvement): 1 2021-03-26 07:39:47,844 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:39:49,797 epoch 69 - iter 5/50 - loss 2.19237757 - samples/sec: 82.01 - lr: 0.003906 2021-03-26 07:39:51,672 epoch 69 - iter 10/50 - loss 2.21753824 - samples/sec: 85.41 - lr: 0.003906 2021-03-26 07:39:53,603 epoch 69 - iter 15/50 - loss 2.31954001 - samples/sec: 82.93 - lr: 0.003906 2021-03-26 07:39:55,499 epoch 69 - iter 20/50 - loss 2.24179201 - samples/sec: 84.45 - lr: 0.003906 2021-03-26 07:39:57,354 epoch 69 - iter 25/50 - loss 2.20438880 - samples/sec: 86.31 - lr: 0.003906 2021-03-26 07:39:59,096 epoch 69 - iter 30/50 - loss 2.17145384 - samples/sec: 91.95 - lr: 0.003906 2021-03-26 07:40:00,874 epoch 69 - iter 35/50 - loss 2.11442671 - samples/sec: 90.09 - lr: 0.003906 2021-03-26 07:40:02,829 epoch 69 - iter 40/50 - loss 2.15811507 - samples/sec: 81.90 - lr: 0.003906 2021-03-26 07:40:04,641 epoch 69 - iter 45/50 - loss 2.16106107 - samples/sec: 88.38 - lr: 0.003906 2021-03-26 07:40:06,419 epoch 69 - iter 50/50 - loss 2.23029785 - samples/sec: 90.08 - lr: 0.003906 2021-03-26 07:40:06,420 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:40:06,420 EPOCH 69 done: loss 2.2303 - lr 0.0039062 2021-03-26 07:40:07,189 DEV : loss 6.972920894622803 - score 0.9038 2021-03-26 07:40:07,212 BAD EPOCHS (no improvement): 2 2021-03-26 07:40:07,212 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:40:08,933 epoch 70 - iter 5/50 - loss 2.01184175 - samples/sec: 93.09 - lr: 0.003906 2021-03-26 07:40:10,930 epoch 70 - iter 10/50 - loss 2.08696496 - samples/sec: 80.22 - lr: 0.003906 2021-03-26 07:40:12,654 epoch 70 - iter 15/50 - loss 1.95998446 - samples/sec: 92.88 - lr: 0.003906 2021-03-26 07:40:14,568 epoch 70 - iter 20/50 - loss 2.11174902 - samples/sec: 83.69 - lr: 0.003906 2021-03-26 07:40:16,406 epoch 70 - iter 25/50 - loss 2.08903484 - samples/sec: 87.14 - lr: 0.003906 2021-03-26 07:40:18,401 epoch 70 - iter 30/50 - loss 2.13611888 - samples/sec: 80.25 - lr: 0.003906 2021-03-26 07:40:20,342 epoch 70 - iter 35/50 - loss 2.13178201 - samples/sec: 82.53 - lr: 0.003906 2021-03-26 07:40:22,363 epoch 70 - iter 40/50 - loss 2.13080277 - samples/sec: 79.25 - lr: 0.003906 2021-03-26 07:40:24,326 epoch 70 - iter 45/50 - loss 2.15558514 - samples/sec: 81.55 - lr: 0.003906 2021-03-26 07:40:26,018 epoch 70 - iter 50/50 - loss 2.11314784 - samples/sec: 94.69 - lr: 0.003906 2021-03-26 07:40:26,019 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:40:26,019 EPOCH 70 done: loss 2.1131 - lr 0.0039062 2021-03-26 07:40:26,761 DEV : loss 6.973997116088867 - score 0.903 2021-03-26 07:40:26,781 BAD EPOCHS (no improvement): 3 2021-03-26 07:40:26,782 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:40:28,575 epoch 71 - iter 5/50 - loss 1.67595360 - samples/sec: 89.32 - lr: 0.003906 2021-03-26 07:40:30,399 epoch 71 - iter 10/50 - loss 1.95769655 - samples/sec: 87.80 - lr: 0.003906 2021-03-26 07:40:32,351 epoch 71 - iter 15/50 - loss 2.07777414 - samples/sec: 82.02 - lr: 0.003906 2021-03-26 07:40:34,190 epoch 71 - iter 20/50 - loss 2.15321826 - samples/sec: 87.07 - lr: 0.003906 2021-03-26 07:40:36,014 epoch 71 - iter 25/50 - loss 2.10259217 - samples/sec: 87.82 - lr: 0.003906 2021-03-26 07:40:37,842 epoch 71 - iter 30/50 - loss 2.16118869 - samples/sec: 87.62 - lr: 0.003906 2021-03-26 07:40:39,714 epoch 71 - iter 35/50 - loss 2.17599435 - samples/sec: 85.53 - lr: 0.003906 2021-03-26 07:40:41,431 epoch 71 - iter 40/50 - loss 2.16178396 - samples/sec: 93.31 - lr: 0.003906 2021-03-26 07:40:43,301 epoch 71 - iter 45/50 - loss 2.17204736 - samples/sec: 85.63 - lr: 0.003906 2021-03-26 07:40:45,200 epoch 71 - iter 50/50 - loss 2.12972616 - samples/sec: 84.33 - lr: 0.003906 2021-03-26 07:40:45,201 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:40:45,201 EPOCH 71 done: loss 2.1297 - lr 0.0039062 2021-03-26 07:40:45,940 DEV : loss 6.972382545471191 - score 0.9034 2021-03-26 07:40:45,963 BAD EPOCHS (no improvement): 4 2021-03-26 07:40:45,963 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:40:47,754 epoch 72 - iter 5/50 - loss 1.90588937 - samples/sec: 89.42 - lr: 0.001953 2021-03-26 07:40:49,579 epoch 72 - iter 10/50 - loss 2.24016843 - samples/sec: 87.75 - lr: 0.001953 2021-03-26 07:40:51,509 epoch 72 - iter 15/50 - loss 2.16828419 - samples/sec: 82.98 - lr: 0.001953 2021-03-26 07:40:53,601 epoch 72 - iter 20/50 - loss 2.19448624 - samples/sec: 76.53 - lr: 0.001953 2021-03-26 07:40:55,455 epoch 72 - iter 25/50 - loss 2.19809166 - samples/sec: 86.44 - lr: 0.001953 2021-03-26 07:40:57,508 epoch 72 - iter 30/50 - loss 2.20144189 - samples/sec: 77.98 - lr: 0.001953 2021-03-26 07:40:59,316 epoch 72 - iter 35/50 - loss 2.22671560 - samples/sec: 88.59 - lr: 0.001953 2021-03-26 07:41:01,173 epoch 72 - iter 40/50 - loss 2.17772023 - samples/sec: 86.24 - lr: 0.001953 2021-03-26 07:41:03,015 epoch 72 - iter 45/50 - loss 2.18650738 - samples/sec: 86.95 - lr: 0.001953 2021-03-26 07:41:04,680 epoch 72 - iter 50/50 - loss 2.16092991 - samples/sec: 96.16 - lr: 0.001953 2021-03-26 07:41:04,681 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:41:04,681 EPOCH 72 done: loss 2.1609 - lr 0.0019531 2021-03-26 07:41:05,422 DEV : loss 6.972001552581787 - score 0.9026 2021-03-26 07:41:05,445 BAD EPOCHS (no improvement): 1 2021-03-26 07:41:05,446 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:41:07,154 epoch 73 - iter 5/50 - loss 2.01508794 - samples/sec: 93.75 - lr: 0.001953 2021-03-26 07:41:09,018 epoch 73 - iter 10/50 - loss 2.01582296 - samples/sec: 85.96 - lr: 0.001953 2021-03-26 07:41:10,979 epoch 73 - iter 15/50 - loss 2.08683247 - samples/sec: 81.65 - lr: 0.001953 2021-03-26 07:41:12,987 epoch 73 - iter 20/50 - loss 2.14391717 - samples/sec: 79.75 - lr: 0.001953 2021-03-26 07:41:14,784 epoch 73 - iter 25/50 - loss 2.16491332 - samples/sec: 89.08 - lr: 0.001953 2021-03-26 07:41:16,612 epoch 73 - iter 30/50 - loss 2.10842774 - samples/sec: 87.61 - lr: 0.001953 2021-03-26 07:41:18,415 epoch 73 - iter 35/50 - loss 2.14298798 - samples/sec: 88.81 - lr: 0.001953 2021-03-26 07:41:20,346 epoch 73 - iter 40/50 - loss 2.13127421 - samples/sec: 82.96 - lr: 0.001953 2021-03-26 07:41:22,330 epoch 73 - iter 45/50 - loss 2.14531492 - samples/sec: 80.72 - lr: 0.001953 2021-03-26 07:41:24,260 epoch 73 - iter 50/50 - loss 2.21428915 - samples/sec: 82.97 - lr: 0.001953 2021-03-26 07:41:24,261 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:41:24,261 EPOCH 73 done: loss 2.2143 - lr 0.0019531 2021-03-26 07:41:25,044 DEV : loss 6.968757152557373 - score 0.9021 2021-03-26 07:41:25,068 BAD EPOCHS (no improvement): 2 2021-03-26 07:41:25,069 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:41:26,904 epoch 74 - iter 5/50 - loss 1.73311095 - samples/sec: 87.31 - lr: 0.001953 2021-03-26 07:41:28,809 epoch 74 - iter 10/50 - loss 1.89269283 - samples/sec: 84.05 - lr: 0.001953 2021-03-26 07:41:30,945 epoch 74 - iter 15/50 - loss 2.01824257 - samples/sec: 74.96 - lr: 0.001953 2021-03-26 07:41:32,798 epoch 74 - iter 20/50 - loss 2.00726241 - samples/sec: 86.47 - lr: 0.001953 2021-03-26 07:41:34,757 epoch 74 - iter 25/50 - loss 2.07169030 - samples/sec: 81.73 - lr: 0.001953 2021-03-26 07:41:36,715 epoch 74 - iter 30/50 - loss 2.04404934 - samples/sec: 81.80 - lr: 0.001953 2021-03-26 07:41:38,708 epoch 74 - iter 35/50 - loss 2.02334532 - samples/sec: 80.32 - lr: 0.001953 2021-03-26 07:41:40,620 epoch 74 - iter 40/50 - loss 2.01096104 - samples/sec: 83.78 - lr: 0.001953 2021-03-26 07:41:42,494 epoch 74 - iter 45/50 - loss 2.04188972 - samples/sec: 85.45 - lr: 0.001953 2021-03-26 07:41:44,434 epoch 74 - iter 50/50 - loss 2.05837031 - samples/sec: 82.53 - lr: 0.001953 2021-03-26 07:41:44,435 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:41:44,436 EPOCH 74 done: loss 2.0584 - lr 0.0019531 2021-03-26 07:41:45,180 DEV : loss 6.965629577636719 - score 0.9026 2021-03-26 07:41:45,196 BAD EPOCHS (no improvement): 3 2021-03-26 07:41:45,197 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:41:46,992 epoch 75 - iter 5/50 - loss 1.92352896 - samples/sec: 89.25 - lr: 0.001953 2021-03-26 07:41:48,810 epoch 75 - iter 10/50 - loss 1.83091727 - samples/sec: 88.09 - lr: 0.001953 2021-03-26 07:41:50,914 epoch 75 - iter 15/50 - loss 2.02670843 - samples/sec: 76.11 - lr: 0.001953 2021-03-26 07:41:52,900 epoch 75 - iter 20/50 - loss 2.15639005 - samples/sec: 80.64 - lr: 0.001953 2021-03-26 07:41:54,641 epoch 75 - iter 25/50 - loss 2.10837126 - samples/sec: 91.94 - lr: 0.001953 2021-03-26 07:41:56,614 epoch 75 - iter 30/50 - loss 2.08132513 - samples/sec: 81.15 - lr: 0.001953 2021-03-26 07:41:58,507 epoch 75 - iter 35/50 - loss 2.08502996 - samples/sec: 84.64 - lr: 0.001953 2021-03-26 07:42:00,352 epoch 75 - iter 40/50 - loss 2.09589115 - samples/sec: 86.78 - lr: 0.001953 2021-03-26 07:42:02,167 epoch 75 - iter 45/50 - loss 2.08379084 - samples/sec: 88.25 - lr: 0.001953 2021-03-26 07:42:04,252 epoch 75 - iter 50/50 - loss 2.13769540 - samples/sec: 76.80 - lr: 0.001953 2021-03-26 07:42:04,253 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:42:04,253 EPOCH 75 done: loss 2.1377 - lr 0.0019531 2021-03-26 07:42:04,999 DEV : loss 6.962711334228516 - score 0.903 2021-03-26 07:42:05,023 BAD EPOCHS (no improvement): 4 2021-03-26 07:42:05,024 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:42:06,880 epoch 76 - iter 5/50 - loss 2.25840490 - samples/sec: 86.29 - lr: 0.000977 2021-03-26 07:42:08,742 epoch 76 - iter 10/50 - loss 2.21133794 - samples/sec: 85.99 - lr: 0.000977 2021-03-26 07:42:10,596 epoch 76 - iter 15/50 - loss 2.18264640 - samples/sec: 86.39 - lr: 0.000977 2021-03-26 07:42:12,651 epoch 76 - iter 20/50 - loss 2.15330370 - samples/sec: 77.93 - lr: 0.000977 2021-03-26 07:42:14,499 epoch 76 - iter 25/50 - loss 2.13097959 - samples/sec: 86.64 - lr: 0.000977 2021-03-26 07:42:16,428 epoch 76 - iter 30/50 - loss 2.11351199 - samples/sec: 83.03 - lr: 0.000977 2021-03-26 07:42:18,222 epoch 76 - iter 35/50 - loss 2.08954168 - samples/sec: 89.29 - lr: 0.000977 2021-03-26 07:42:20,143 epoch 76 - iter 40/50 - loss 2.09097285 - samples/sec: 83.35 - lr: 0.000977 2021-03-26 07:42:21,935 epoch 76 - iter 45/50 - loss 2.11737519 - samples/sec: 89.38 - lr: 0.000977 2021-03-26 07:42:23,632 epoch 76 - iter 50/50 - loss 2.09018610 - samples/sec: 94.35 - lr: 0.000977 2021-03-26 07:42:23,633 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:42:23,633 EPOCH 76 done: loss 2.0902 - lr 0.0009766 2021-03-26 07:42:24,378 DEV : loss 6.960842132568359 - score 0.9026 2021-03-26 07:42:24,393 BAD EPOCHS (no improvement): 1 2021-03-26 07:42:24,394 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:42:26,331 epoch 77 - iter 5/50 - loss 2.89146829 - samples/sec: 82.66 - lr: 0.000977 2021-03-26 07:42:28,101 epoch 77 - iter 10/50 - loss 2.54311110 - samples/sec: 90.51 - lr: 0.000977 2021-03-26 07:42:30,008 epoch 77 - iter 15/50 - loss 2.33039279 - samples/sec: 83.98 - lr: 0.000977 2021-03-26 07:42:31,779 epoch 77 - iter 20/50 - loss 2.27280054 - samples/sec: 90.40 - lr: 0.000977 2021-03-26 07:42:33,568 epoch 77 - iter 25/50 - loss 2.19065623 - samples/sec: 89.55 - lr: 0.000977 2021-03-26 07:42:35,402 epoch 77 - iter 30/50 - loss 2.19404272 - samples/sec: 87.34 - lr: 0.000977 2021-03-26 07:42:37,251 epoch 77 - iter 35/50 - loss 2.22139244 - samples/sec: 86.59 - lr: 0.000977 2021-03-26 07:42:39,080 epoch 77 - iter 40/50 - loss 2.23841817 - samples/sec: 87.55 - lr: 0.000977 2021-03-26 07:42:40,981 epoch 77 - iter 45/50 - loss 2.23272769 - samples/sec: 84.24 - lr: 0.000977 2021-03-26 07:42:42,748 epoch 77 - iter 50/50 - loss 2.25151328 - samples/sec: 90.64 - lr: 0.000977 2021-03-26 07:42:42,749 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:42:42,749 EPOCH 77 done: loss 2.2515 - lr 0.0009766 2021-03-26 07:42:43,578 DEV : loss 6.961365699768066 - score 0.9021 2021-03-26 07:42:43,595 BAD EPOCHS (no improvement): 2 2021-03-26 07:42:43,595 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:42:45,489 epoch 78 - iter 5/50 - loss 2.24054193 - samples/sec: 84.58 - lr: 0.000977 2021-03-26 07:42:47,366 epoch 78 - iter 10/50 - loss 2.30875487 - samples/sec: 85.33 - lr: 0.000977 2021-03-26 07:42:49,315 epoch 78 - iter 15/50 - loss 2.24832230 - samples/sec: 82.19 - lr: 0.000977 2021-03-26 07:42:51,239 epoch 78 - iter 20/50 - loss 2.18487847 - samples/sec: 83.26 - lr: 0.000977 2021-03-26 07:42:53,163 epoch 78 - iter 25/50 - loss 2.11373229 - samples/sec: 83.21 - lr: 0.000977 2021-03-26 07:42:55,036 epoch 78 - iter 30/50 - loss 2.11837125 - samples/sec: 85.50 - lr: 0.000977 2021-03-26 07:42:56,865 epoch 78 - iter 35/50 - loss 2.13645056 - samples/sec: 87.57 - lr: 0.000977 2021-03-26 07:42:58,632 epoch 78 - iter 40/50 - loss 2.15983583 - samples/sec: 90.65 - lr: 0.000977 2021-03-26 07:43:00,568 epoch 78 - iter 45/50 - loss 2.12658242 - samples/sec: 82.72 - lr: 0.000977 2021-03-26 07:43:02,286 epoch 78 - iter 50/50 - loss 2.15449122 - samples/sec: 93.23 - lr: 0.000977 2021-03-26 07:43:02,287 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:43:02,287 EPOCH 78 done: loss 2.1545 - lr 0.0009766 2021-03-26 07:43:03,044 DEV : loss 6.962085247039795 - score 0.903 2021-03-26 07:43:03,068 BAD EPOCHS (no improvement): 3 2021-03-26 07:43:03,069 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:43:04,873 epoch 79 - iter 5/50 - loss 2.21793807 - samples/sec: 88.78 - lr: 0.000977 2021-03-26 07:43:06,839 epoch 79 - iter 10/50 - loss 2.29863290 - samples/sec: 81.46 - lr: 0.000977 2021-03-26 07:43:08,782 epoch 79 - iter 15/50 - loss 2.32131858 - samples/sec: 82.44 - lr: 0.000977 2021-03-26 07:43:10,800 epoch 79 - iter 20/50 - loss 2.24925628 - samples/sec: 79.34 - lr: 0.000977 2021-03-26 07:43:12,761 epoch 79 - iter 25/50 - loss 2.27966526 - samples/sec: 81.66 - lr: 0.000977 2021-03-26 07:43:14,581 epoch 79 - iter 30/50 - loss 2.22692318 - samples/sec: 87.99 - lr: 0.000977 2021-03-26 07:43:16,503 epoch 79 - iter 35/50 - loss 2.24432735 - samples/sec: 83.30 - lr: 0.000977 2021-03-26 07:43:18,380 epoch 79 - iter 40/50 - loss 2.16589115 - samples/sec: 85.33 - lr: 0.000977 2021-03-26 07:43:20,339 epoch 79 - iter 45/50 - loss 2.21727477 - samples/sec: 81.75 - lr: 0.000977 2021-03-26 07:43:22,056 epoch 79 - iter 50/50 - loss 2.18660015 - samples/sec: 93.30 - lr: 0.000977 2021-03-26 07:43:22,056 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:43:22,057 EPOCH 79 done: loss 2.1866 - lr 0.0009766 2021-03-26 07:43:22,825 DEV : loss 6.9618353843688965 - score 0.9034 2021-03-26 07:43:22,849 BAD EPOCHS (no improvement): 4 2021-03-26 07:43:22,850 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:43:24,903 epoch 80 - iter 5/50 - loss 2.36558053 - samples/sec: 78.01 - lr: 0.000488 2021-03-26 07:43:26,783 epoch 80 - iter 10/50 - loss 2.11772819 - samples/sec: 85.21 - lr: 0.000488 2021-03-26 07:43:28,660 epoch 80 - iter 15/50 - loss 2.01982580 - samples/sec: 85.36 - lr: 0.000488 2021-03-26 07:43:30,435 epoch 80 - iter 20/50 - loss 1.97181435 - samples/sec: 90.22 - lr: 0.000488 2021-03-26 07:43:32,457 epoch 80 - iter 25/50 - loss 1.94486945 - samples/sec: 79.18 - lr: 0.000488 2021-03-26 07:43:34,381 epoch 80 - iter 30/50 - loss 2.03408866 - samples/sec: 83.27 - lr: 0.000488 2021-03-26 07:43:36,194 epoch 80 - iter 35/50 - loss 2.01605985 - samples/sec: 88.31 - lr: 0.000488 2021-03-26 07:43:38,146 epoch 80 - iter 40/50 - loss 1.98962872 - samples/sec: 82.07 - lr: 0.000488 2021-03-26 07:43:40,176 epoch 80 - iter 45/50 - loss 2.01850783 - samples/sec: 78.90 - lr: 0.000488 2021-03-26 07:43:41,884 epoch 80 - iter 50/50 - loss 2.00040341 - samples/sec: 93.77 - lr: 0.000488 2021-03-26 07:43:41,884 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:43:41,884 EPOCH 80 done: loss 2.0004 - lr 0.0004883 2021-03-26 07:43:42,657 DEV : loss 6.960967063903809 - score 0.9034 2021-03-26 07:43:42,680 BAD EPOCHS (no improvement): 1 2021-03-26 07:43:42,681 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:43:44,534 epoch 81 - iter 5/50 - loss 1.82899351 - samples/sec: 86.40 - lr: 0.000488 2021-03-26 07:43:46,503 epoch 81 - iter 10/50 - loss 2.06492004 - samples/sec: 81.36 - lr: 0.000488 2021-03-26 07:43:48,577 epoch 81 - iter 15/50 - loss 2.03007421 - samples/sec: 77.20 - lr: 0.000488 2021-03-26 07:43:50,461 epoch 81 - iter 20/50 - loss 2.05423158 - samples/sec: 84.97 - lr: 0.000488 2021-03-26 07:43:52,368 epoch 81 - iter 25/50 - loss 2.15495116 - samples/sec: 84.01 - lr: 0.000488 2021-03-26 07:43:54,207 epoch 81 - iter 30/50 - loss 2.11696693 - samples/sec: 87.07 - lr: 0.000488 2021-03-26 07:43:56,144 epoch 81 - iter 35/50 - loss 2.10432662 - samples/sec: 82.70 - lr: 0.000488 2021-03-26 07:43:58,144 epoch 81 - iter 40/50 - loss 2.13423683 - samples/sec: 80.08 - lr: 0.000488 2021-03-26 07:43:59,931 epoch 81 - iter 45/50 - loss 2.13024547 - samples/sec: 89.61 - lr: 0.000488 2021-03-26 07:44:01,891 epoch 81 - iter 50/50 - loss 2.16672405 - samples/sec: 81.71 - lr: 0.000488 2021-03-26 07:44:01,891 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:44:01,892 EPOCH 81 done: loss 2.1667 - lr 0.0004883 2021-03-26 07:44:02,680 DEV : loss 6.962381362915039 - score 0.903 2021-03-26 07:44:02,696 BAD EPOCHS (no improvement): 2 2021-03-26 07:44:02,697 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:44:04,854 epoch 82 - iter 5/50 - loss 2.00777359 - samples/sec: 74.25 - lr: 0.000488 2021-03-26 07:44:06,801 epoch 82 - iter 10/50 - loss 2.22989006 - samples/sec: 82.27 - lr: 0.000488 2021-03-26 07:44:08,627 epoch 82 - iter 15/50 - loss 2.09961801 - samples/sec: 87.68 - lr: 0.000488 2021-03-26 07:44:10,477 epoch 82 - iter 20/50 - loss 2.15289982 - samples/sec: 86.59 - lr: 0.000488 2021-03-26 07:44:12,330 epoch 82 - iter 25/50 - loss 2.18809028 - samples/sec: 86.45 - lr: 0.000488 2021-03-26 07:44:14,319 epoch 82 - iter 30/50 - loss 2.18357460 - samples/sec: 80.52 - lr: 0.000488 2021-03-26 07:44:16,347 epoch 82 - iter 35/50 - loss 2.13511537 - samples/sec: 78.97 - lr: 0.000488 2021-03-26 07:44:18,339 epoch 82 - iter 40/50 - loss 2.12040598 - samples/sec: 80.40 - lr: 0.000488 2021-03-26 07:44:20,167 epoch 82 - iter 45/50 - loss 2.11619196 - samples/sec: 87.63 - lr: 0.000488 2021-03-26 07:44:22,028 epoch 82 - iter 50/50 - loss 2.11520464 - samples/sec: 86.02 - lr: 0.000488 2021-03-26 07:44:22,029 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:44:22,029 EPOCH 82 done: loss 2.1152 - lr 0.0004883 2021-03-26 07:44:22,828 DEV : loss 6.961461544036865 - score 0.9034 2021-03-26 07:44:22,851 BAD EPOCHS (no improvement): 3 2021-03-26 07:44:22,852 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:44:24,831 epoch 83 - iter 5/50 - loss 2.41702788 - samples/sec: 80.95 - lr: 0.000488 2021-03-26 07:44:26,699 epoch 83 - iter 10/50 - loss 2.16960576 - samples/sec: 85.74 - lr: 0.000488 2021-03-26 07:44:28,456 epoch 83 - iter 15/50 - loss 2.26841859 - samples/sec: 91.15 - lr: 0.000488 2021-03-26 07:44:30,467 epoch 83 - iter 20/50 - loss 2.25694011 - samples/sec: 79.64 - lr: 0.000488 2021-03-26 07:44:32,382 epoch 83 - iter 25/50 - loss 2.28131059 - samples/sec: 83.64 - lr: 0.000488 2021-03-26 07:44:34,348 epoch 83 - iter 30/50 - loss 2.28328307 - samples/sec: 81.42 - lr: 0.000488 2021-03-26 07:44:36,313 epoch 83 - iter 35/50 - loss 2.26969606 - samples/sec: 81.49 - lr: 0.000488 2021-03-26 07:44:38,324 epoch 83 - iter 40/50 - loss 2.24495017 - samples/sec: 79.65 - lr: 0.000488 2021-03-26 07:44:40,213 epoch 83 - iter 45/50 - loss 2.25571009 - samples/sec: 84.78 - lr: 0.000488 2021-03-26 07:44:41,904 epoch 83 - iter 50/50 - loss 2.26575992 - samples/sec: 94.67 - lr: 0.000488 2021-03-26 07:44:41,905 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:44:41,905 EPOCH 83 done: loss 2.2658 - lr 0.0004883 2021-03-26 07:44:42,684 DEV : loss 6.96260404586792 - score 0.9034 2021-03-26 07:44:42,703 BAD EPOCHS (no improvement): 4 2021-03-26 07:44:42,704 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:44:44,763 epoch 84 - iter 5/50 - loss 2.29062424 - samples/sec: 77.77 - lr: 0.000244 2021-03-26 07:44:46,685 epoch 84 - iter 10/50 - loss 2.16970339 - samples/sec: 83.34 - lr: 0.000244 2021-03-26 07:44:48,469 epoch 84 - iter 15/50 - loss 2.22968817 - samples/sec: 89.76 - lr: 0.000244 2021-03-26 07:44:50,497 epoch 84 - iter 20/50 - loss 2.26354269 - samples/sec: 78.95 - lr: 0.000244 2021-03-26 07:44:52,408 epoch 84 - iter 25/50 - loss 2.23960634 - samples/sec: 83.81 - lr: 0.000244 2021-03-26 07:44:54,257 epoch 84 - iter 30/50 - loss 2.20255948 - samples/sec: 86.63 - lr: 0.000244 2021-03-26 07:44:56,223 epoch 84 - iter 35/50 - loss 2.21711638 - samples/sec: 81.44 - lr: 0.000244 2021-03-26 07:44:58,059 epoch 84 - iter 40/50 - loss 2.22642836 - samples/sec: 87.22 - lr: 0.000244 2021-03-26 07:44:59,900 epoch 84 - iter 45/50 - loss 2.24058124 - samples/sec: 87.03 - lr: 0.000244 2021-03-26 07:45:01,711 epoch 84 - iter 50/50 - loss 2.21161734 - samples/sec: 88.39 - lr: 0.000244 2021-03-26 07:45:01,712 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:45:01,712 EPOCH 84 done: loss 2.2116 - lr 0.0002441 2021-03-26 07:45:02,459 DEV : loss 6.962458610534668 - score 0.903 2021-03-26 07:45:02,481 BAD EPOCHS (no improvement): 1 2021-03-26 07:45:02,482 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:45:04,349 epoch 85 - iter 5/50 - loss 1.92033653 - samples/sec: 85.77 - lr: 0.000244 2021-03-26 07:45:06,066 epoch 85 - iter 10/50 - loss 1.91954648 - samples/sec: 93.28 - lr: 0.000244 2021-03-26 07:45:08,037 epoch 85 - iter 15/50 - loss 2.06335115 - samples/sec: 81.21 - lr: 0.000244 2021-03-26 07:45:09,907 epoch 85 - iter 20/50 - loss 2.08999578 - samples/sec: 85.66 - lr: 0.000244 2021-03-26 07:45:11,884 epoch 85 - iter 25/50 - loss 2.09058705 - samples/sec: 80.99 - lr: 0.000244 2021-03-26 07:45:13,731 epoch 85 - iter 30/50 - loss 2.07775679 - samples/sec: 86.72 - lr: 0.000244 2021-03-26 07:45:15,624 epoch 85 - iter 35/50 - loss 2.07566880 - samples/sec: 84.56 - lr: 0.000244 2021-03-26 07:45:17,543 epoch 85 - iter 40/50 - loss 2.07374095 - samples/sec: 83.46 - lr: 0.000244 2021-03-26 07:45:19,480 epoch 85 - iter 45/50 - loss 2.06040113 - samples/sec: 82.71 - lr: 0.000244 2021-03-26 07:45:21,332 epoch 85 - iter 50/50 - loss 2.11410238 - samples/sec: 86.45 - lr: 0.000244 2021-03-26 07:45:21,333 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:45:21,333 EPOCH 85 done: loss 2.1141 - lr 0.0002441 2021-03-26 07:45:22,084 DEV : loss 6.961507320404053 - score 0.903 2021-03-26 07:45:22,103 BAD EPOCHS (no improvement): 2 2021-03-26 07:45:22,104 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:45:24,176 epoch 86 - iter 5/50 - loss 2.08228850 - samples/sec: 77.30 - lr: 0.000244 2021-03-26 07:45:26,138 epoch 86 - iter 10/50 - loss 2.17302984 - samples/sec: 81.63 - lr: 0.000244 2021-03-26 07:45:28,015 epoch 86 - iter 15/50 - loss 2.14811590 - samples/sec: 85.32 - lr: 0.000244 2021-03-26 07:45:29,892 epoch 86 - iter 20/50 - loss 2.11968042 - samples/sec: 85.31 - lr: 0.000244 2021-03-26 07:45:31,674 epoch 86 - iter 25/50 - loss 2.03037696 - samples/sec: 89.86 - lr: 0.000244 2021-03-26 07:45:33,639 epoch 86 - iter 30/50 - loss 2.10106587 - samples/sec: 81.51 - lr: 0.000244 2021-03-26 07:45:35,595 epoch 86 - iter 35/50 - loss 2.07236118 - samples/sec: 81.88 - lr: 0.000244 2021-03-26 07:45:37,637 epoch 86 - iter 40/50 - loss 2.11832686 - samples/sec: 78.44 - lr: 0.000244 2021-03-26 07:45:39,532 epoch 86 - iter 45/50 - loss 2.13117549 - samples/sec: 84.53 - lr: 0.000244 2021-03-26 07:45:41,400 epoch 86 - iter 50/50 - loss 2.14113355 - samples/sec: 85.74 - lr: 0.000244 2021-03-26 07:45:41,400 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:45:41,401 EPOCH 86 done: loss 2.1411 - lr 0.0002441 2021-03-26 07:45:42,164 DEV : loss 6.961318016052246 - score 0.903 2021-03-26 07:45:42,184 BAD EPOCHS (no improvement): 3 2021-03-26 07:45:42,184 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:45:44,109 epoch 87 - iter 5/50 - loss 2.50831332 - samples/sec: 83.19 - lr: 0.000244 2021-03-26 07:45:45,999 epoch 87 - iter 10/50 - loss 2.19329880 - samples/sec: 84.75 - lr: 0.000244 2021-03-26 07:45:47,854 epoch 87 - iter 15/50 - loss 2.13173851 - samples/sec: 86.31 - lr: 0.000244 2021-03-26 07:45:49,763 epoch 87 - iter 20/50 - loss 2.13360924 - samples/sec: 83.88 - lr: 0.000244 2021-03-26 07:45:51,596 epoch 87 - iter 25/50 - loss 2.10489856 - samples/sec: 87.39 - lr: 0.000244 2021-03-26 07:45:53,621 epoch 87 - iter 30/50 - loss 2.07814337 - samples/sec: 79.06 - lr: 0.000244 2021-03-26 07:45:55,520 epoch 87 - iter 35/50 - loss 2.11120173 - samples/sec: 84.37 - lr: 0.000244 2021-03-26 07:45:57,411 epoch 87 - iter 40/50 - loss 2.06504587 - samples/sec: 84.68 - lr: 0.000244 2021-03-26 07:45:59,961 epoch 87 - iter 45/50 - loss 2.06413985 - samples/sec: 62.79 - lr: 0.000244 2021-03-26 07:46:01,714 epoch 87 - iter 50/50 - loss 2.10936777 - samples/sec: 91.35 - lr: 0.000244 2021-03-26 07:46:01,715 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:46:01,716 EPOCH 87 done: loss 2.1094 - lr 0.0002441 2021-03-26 07:46:02,459 DEV : loss 6.961304664611816 - score 0.903 2021-03-26 07:46:02,482 BAD EPOCHS (no improvement): 4 2021-03-26 07:46:02,482 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:46:04,294 epoch 88 - iter 5/50 - loss 1.52949798 - samples/sec: 88.37 - lr: 0.000122 2021-03-26 07:46:06,213 epoch 88 - iter 10/50 - loss 1.76798514 - samples/sec: 83.47 - lr: 0.000122 2021-03-26 07:46:08,009 epoch 88 - iter 15/50 - loss 1.96657056 - samples/sec: 89.19 - lr: 0.000122 2021-03-26 07:46:09,863 epoch 88 - iter 20/50 - loss 2.16982436 - samples/sec: 86.35 - lr: 0.000122 2021-03-26 07:46:11,814 epoch 88 - iter 25/50 - loss 2.20400687 - samples/sec: 82.12 - lr: 0.000122 2021-03-26 07:46:13,717 epoch 88 - iter 30/50 - loss 2.09522241 - samples/sec: 84.12 - lr: 0.000122 2021-03-26 07:46:15,620 epoch 88 - iter 35/50 - loss 2.09541041 - samples/sec: 84.14 - lr: 0.000122 2021-03-26 07:46:17,503 epoch 88 - iter 40/50 - loss 2.10176090 - samples/sec: 85.10 - lr: 0.000122 2021-03-26 07:46:19,396 epoch 88 - iter 45/50 - loss 2.09184035 - samples/sec: 84.58 - lr: 0.000122 2021-03-26 07:46:21,335 epoch 88 - iter 50/50 - loss 2.08811824 - samples/sec: 82.60 - lr: 0.000122 2021-03-26 07:46:21,336 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:46:21,336 EPOCH 88 done: loss 2.0881 - lr 0.0001221 2021-03-26 07:46:22,099 DEV : loss 6.961170196533203 - score 0.903 2021-03-26 07:46:22,123 BAD EPOCHS (no improvement): 1 2021-03-26 07:46:22,124 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:46:24,152 epoch 89 - iter 5/50 - loss 1.76743813 - samples/sec: 78.98 - lr: 0.000122 2021-03-26 07:46:26,045 epoch 89 - iter 10/50 - loss 1.99386131 - samples/sec: 84.64 - lr: 0.000122 2021-03-26 07:46:27,924 epoch 89 - iter 15/50 - loss 1.92271976 - samples/sec: 85.26 - lr: 0.000122 2021-03-26 07:46:29,738 epoch 89 - iter 20/50 - loss 2.00147249 - samples/sec: 88.33 - lr: 0.000122 2021-03-26 07:46:31,611 epoch 89 - iter 25/50 - loss 2.12277925 - samples/sec: 85.50 - lr: 0.000122 2021-03-26 07:46:33,492 epoch 89 - iter 30/50 - loss 2.13457530 - samples/sec: 85.14 - lr: 0.000122 2021-03-26 07:46:35,399 epoch 89 - iter 35/50 - loss 2.16680305 - samples/sec: 83.96 - lr: 0.000122 2021-03-26 07:46:37,298 epoch 89 - iter 40/50 - loss 2.15806007 - samples/sec: 84.36 - lr: 0.000122 2021-03-26 07:46:39,166 epoch 89 - iter 45/50 - loss 2.18511029 - samples/sec: 85.70 - lr: 0.000122 2021-03-26 07:46:40,945 epoch 89 - iter 50/50 - loss 2.26934085 - samples/sec: 90.05 - lr: 0.000122 2021-03-26 07:46:40,946 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:46:40,946 EPOCH 89 done: loss 2.2693 - lr 0.0001221 2021-03-26 07:46:41,712 DEV : loss 6.960937023162842 - score 0.903 2021-03-26 07:46:41,739 BAD EPOCHS (no improvement): 2 2021-03-26 07:46:41,740 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:46:43,582 epoch 90 - iter 5/50 - loss 2.45033231 - samples/sec: 86.95 - lr: 0.000122 2021-03-26 07:46:45,564 epoch 90 - iter 10/50 - loss 2.30596037 - samples/sec: 80.79 - lr: 0.000122 2021-03-26 07:46:47,486 epoch 90 - iter 15/50 - loss 2.30664878 - samples/sec: 83.36 - lr: 0.000122 2021-03-26 07:46:49,472 epoch 90 - iter 20/50 - loss 2.22472465 - samples/sec: 80.61 - lr: 0.000122 2021-03-26 07:46:51,344 epoch 90 - iter 25/50 - loss 2.25325889 - samples/sec: 85.55 - lr: 0.000122 2021-03-26 07:46:53,309 epoch 90 - iter 30/50 - loss 2.24244933 - samples/sec: 81.52 - lr: 0.000122 2021-03-26 07:46:55,212 epoch 90 - iter 35/50 - loss 2.22077815 - samples/sec: 84.16 - lr: 0.000122 2021-03-26 07:46:56,960 epoch 90 - iter 40/50 - loss 2.15720444 - samples/sec: 91.59 - lr: 0.000122 2021-03-26 07:46:58,769 epoch 90 - iter 45/50 - loss 2.16676752 - samples/sec: 88.53 - lr: 0.000122 2021-03-26 07:47:00,485 epoch 90 - iter 50/50 - loss 2.14653236 - samples/sec: 93.33 - lr: 0.000122 2021-03-26 07:47:00,486 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:00,486 EPOCH 90 done: loss 2.1465 - lr 0.0001221 2021-03-26 07:47:01,218 DEV : loss 6.960792064666748 - score 0.903 2021-03-26 07:47:01,241 BAD EPOCHS (no improvement): 3 2021-03-26 07:47:01,242 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:03,074 epoch 91 - iter 5/50 - loss 2.31844633 - samples/sec: 87.42 - lr: 0.000122 2021-03-26 07:47:04,983 epoch 91 - iter 10/50 - loss 2.14653064 - samples/sec: 83.88 - lr: 0.000122 2021-03-26 07:47:06,843 epoch 91 - iter 15/50 - loss 2.03093856 - samples/sec: 86.09 - lr: 0.000122 2021-03-26 07:47:08,666 epoch 91 - iter 20/50 - loss 2.11692855 - samples/sec: 87.88 - lr: 0.000122 2021-03-26 07:47:10,518 epoch 91 - iter 25/50 - loss 2.11030499 - samples/sec: 86.44 - lr: 0.000122 2021-03-26 07:47:12,443 epoch 91 - iter 30/50 - loss 2.12978576 - samples/sec: 83.20 - lr: 0.000122 2021-03-26 07:47:14,267 epoch 91 - iter 35/50 - loss 2.13929472 - samples/sec: 87.82 - lr: 0.000122 2021-03-26 07:47:16,110 epoch 91 - iter 40/50 - loss 2.13259790 - samples/sec: 86.85 - lr: 0.000122 2021-03-26 07:47:18,008 epoch 91 - iter 45/50 - loss 2.16532231 - samples/sec: 84.43 - lr: 0.000122 2021-03-26 07:47:19,685 epoch 91 - iter 50/50 - loss 2.11413988 - samples/sec: 95.48 - lr: 0.000122 2021-03-26 07:47:19,685 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:19,686 EPOCH 91 done: loss 2.1141 - lr 0.0001221 2021-03-26 07:47:20,415 DEV : loss 6.9608001708984375 - score 0.903 2021-03-26 07:47:20,438 BAD EPOCHS (no improvement): 4 2021-03-26 07:47:20,439 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:20,439 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:20,439 learning rate too small - quitting training! 2021-03-26 07:47:20,440 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:29,508 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:29,509 Testing using best model ... 2021-03-26 07:47:29,510 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__32__0.5_202103260713/best-model.pt 2021-03-26 07:47:36,788 0.9055 2021-03-26 07:47:36,788 Results: - F-score (micro): 0.9012 - F-score (macro): 0.4811 - Accuracy (incl. no class): 0.9055 By class: precision recall f1-score support CCONJ 0.9865 0.9865 0.9865 74 PRON 0.9725 0.9833 0.9779 180 PART 0.9617 0.9757 0.9687 206 VERB 0.9549 0.8819 0.9170 144 ADJ 0.8947 0.7589 0.8213 112 NOUN 0.8897 0.9322 0.9104 398 ADP 0.9263 0.9565 0.9412 92 DET 0.9821 0.9649 0.9735 57 SCONJ 1.0000 0.9697 0.9846 33 ADV 0.9074 0.8829 0.8950 111 AUX 0.8387 0.9286 0.8814 28 PROPN 1.0000 1.0000 1.0000 10 NUM 0.9333 0.8750 0.9032 16 INTJ 1.0000 1.0000 1.0000 17 PUNCT 1.0000 1.0000 1.0000 20 V 0.9651 0.8300 0.8925 100 NOUN+PRON 0.7975 0.8873 0.8400 71 PREP+DET+ADJ 1.0000 0.0000 0.0000 1 PREP+DET+NOUN 0.5385 0.8750 0.6667 8 PROG_PART 1.0000 0.0000 0.0000 1 PREP+PRON 0.8056 0.9667 0.8788 30 PRON+DET+NOUN 0.5000 1.0000 0.6667 1 HASH 1.0000 0.9524 0.9756 21 EOS 1.0000 1.0000 1.0000 70 V+PRON+PRON 0.6000 0.3000 0.4000 10 NOUN+NSUFF 0.8261 0.8837 0.8539 43 PREP+NOUN+PRON 0.6667 0.4000 0.5000 5 PUNC 1.0000 1.0000 1.0000 127 V+PRON 0.7164 0.8727 0.7869 55 V+PRON+PREP+PRON 0.0000 0.0000 0.0000 4 V+PREP+PRON 0.2000 0.2500 0.2222 4 DET+NOUN+NSUFF 0.9310 0.8710 0.9000 31 NOUN+NSUFF+PRON 0.9375 0.7143 0.8108 21 CONJ 0.9667 1.0000 0.9831 29 FOREIGN 1.0000 0.0000 0.0000 3 MENTION 0.9412 1.0000 0.9697 16 DET+NOUN 0.9615 0.9804 0.9709 51 PROG_PART+V 0.9000 0.9730 0.9351 37 PREP+NOUN+PRON+PRON 1.0000 0.0000 0.0000 1 PREP+NOUN+NSUFF+PRON 0.0000 1.0000 0.0000 0 PART+PRON 1.0000 0.8636 0.9268 22 PREP+V+PRON 1.0000 0.0000 0.0000 1 CONJ+V+PRON 0.5556 0.8333 0.6667 6 PREP 0.9464 0.9815 0.9636 54 ADJ+NSUFF 0.7632 0.9667 0.8529 30 PREP+V 0.5000 0.5000 0.5000 2 PREP+NOUN 0.8947 0.8500 0.8718 20 PREP+NOUN+NSUFF 1.0000 0.7500 0.8571 4 PROG_PART+V+PRON 0.7143 1.0000 0.8333 10 DET+ADJ+NSUFF 0.5000 0.5000 0.5000 4 CONJ+DET+NOUN 1.0000 1.0000 1.0000 2 FUT_PART+V+PRON 1.0000 0.3333 0.5000 6 CONJ+V 0.8889 0.8000 0.8421 10 CONJ+PROG_PART+V 0.5000 1.0000 0.6667 2 NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 CONJ+NOUN+PRON 0.0000 0.0000 0.0000 2 CONJ+DET+NOUN+NSUFF 0.0000 1.0000 0.0000 0 CONJ+PREP+DET+NOUN 0.0000 1.0000 0.0000 0 ADJ+PRON 0.2500 0.2500 0.2500 4 CONJ+PRON 1.0000 1.0000 1.0000 9 URL 1.0000 1.0000 1.0000 3 ADJ+PREP+PRON 1.0000 0.0000 0.0000 5 PART+NOUN 0.7500 1.0000 0.8571 3 CONJ+NOUN 0.8333 0.7143 0.7692 7 PREP+ADJ+NSUFF 0.0000 1.0000 0.0000 0 EMOT 1.0000 1.0000 1.0000 15 PART+NOUN+PRON 0.0000 1.0000 0.0000 0 CONJ+PREP 1.0000 1.0000 1.0000 1 PREP+PRON+DET+NOUN 1.0000 0.0000 0.0000 1 PREP+DET+NOUN+NSUFF+PREP+PRON 0.0000 1.0000 0.0000 0 DET+ADJ 0.5000 1.0000 0.6667 2 PREP+DET+NOUN+NSUFF 0.7500 0.7500 0.7500 4 PREP+DET+ADV 1.0000 0.0000 0.0000 1 CONJ+NOUN+NSUFF 0.7500 1.0000 0.8571 3 CONJ+PART+V+NOUN 1.0000 0.0000 0.0000 1 CONJ+ADJ 1.0000 0.0000 0.0000 2 V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+V+PRON+NEG_PART 0.5556 0.7143 0.6250 7 FUT_PART+V 0.9000 0.9000 0.9000 10 CONJ+PART 1.0000 1.0000 1.0000 5 PART+V+NEG_PART 1.0000 1.0000 1.0000 3 FUT_PART 1.0000 1.0000 1.0000 1 PART+V 0.0000 0.0000 0.0000 1 CONJ+V+PREP+PRON 0.0000 1.0000 0.0000 0 NOUN+PREP+PRON 1.0000 0.0000 0.0000 1 PREP+PART 1.0000 1.0000 1.0000 3 CONJ+PART+V+PRON 1.0000 0.0000 0.0000 2 PRON+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 PREP+PART+PRON 1.0000 1.0000 1.0000 1 V+NOUN 1.0000 0.0000 0.0000 1 CONJ+PART+PROG_PART+V 1.0000 0.0000 0.0000 1 NOUN+CASE 0.6000 0.7500 0.6667 4 CONJ+PART+V+NEG_PART 1.0000 0.0000 0.0000 2 CONJ+FUT_PART+V 0.0000 0.0000 0.0000 1 PART+PROG_PART+V+NEG_PART 0.0000 0.0000 0.0000 1 PART+PREP+NEG_PART 1.0000 1.0000 1.0000 1 PREP+DET 1.0000 0.0000 0.0000 1 PROG_PART+V+PREP+PRON 1.0000 0.0000 0.0000 1 FUT_PART+V+PREP+PRON 0.0000 0.0000 0.0000 1 PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 PART+V+PRON 1.0000 0.0000 0.0000 1 PROG_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 PART+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 DET+NUM 1.0000 0.0000 0.0000 1 PART+NSUFF 1.0000 0.0000 0.0000 1 ADJ+CASE 0.0000 1.0000 0.0000 0 FUT_PART+V+PRON+PRON 0.0000 1.0000 0.0000 0 V+NEG_PART 0.0000 0.0000 0.0000 2 PART+NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 NOUN+PRON+NEG_PART 0.0000 1.0000 0.0000 0 NOUN+CASE+PRON 1.0000 0.0000 0.0000 1 CONJ+PROG_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 CONJ+PROG_PART+V+PRON 0.0000 1.0000 0.0000 0 ADV+NSUFF 1.0000 1.0000 1.0000 1 PART+V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+V+PREP+PRON+NEG_PART 0.0000 0.0000 0.0000 1 PART+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+ADV+NSUFF 1.0000 0.0000 0.0000 1 CONJ+FUT_PART+V+PREP+PRON 0.0000 1.0000 0.0000 0 NUM+CASE 1.0000 0.0000 0.0000 2 micro avg 0.9016 0.9009 0.9012 2542 macro avg 0.7388 0.6026 0.4811 2542 weighted avg 0.9136 0.9009 0.8966 2542 2021-03-26 07:47:36,789 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:36,789 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:42,946 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC 2021-03-26 07:47:42,947 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu 2021-03-26 07:47:42,947 Dev: None 2021-03-26 07:47:42,947 Test: None 2021-03-26 07:47:43,237 Reading data from ../../Datasets_adhoc/UD_MADAR 2021-03-26 07:47:43,237 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu 2021-03-26 07:47:43,238 Dev: None 2021-03-26 07:47:43,238 Test: None 2021-03-26 07:47:43,282 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 07:47:43,282 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt 2021-03-26 07:47:43,282 Dev: None 2021-03-26 07:47:43,282 Test: None 2021-03-26 07:47:43,434 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 07:47:43,435 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt 2021-03-26 07:47:43,435 Dev: None 2021-03-26 07:47:43,436 Test: None 2021-03-26 07:47:43,590 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 07:47:43,591 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt 2021-03-26 07:47:43,591 Dev: None 2021-03-26 07:47:43,591 Test: None 2021-03-26 07:47:43,758 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 07:47:43,759 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt 2021-03-26 07:47:43,759 Dev: None 2021-03-26 07:47:43,760 Test: None 2021-03-26 07:47:43,923 Filtering long sentences 2021-03-26 07:47:43,962 MultiCorpus: 1573 train + 176 dev + 195 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences 2021-03-26 07:47:44,365 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:44,366 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): WordEmbeddings('ar') (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) (list_embedding_2): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=206, bias=True) (beta): 1.0 (weights): None (weight_tensor) None )" 2021-03-26 07:47:44,367 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:44,367 Corpus: "MultiCorpus: 1573 train + 176 dev + 195 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" 2021-03-26 07:47:44,368 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:44,368 Parameters: 2021-03-26 07:47:44,368 - learning_rate: "0.5" 2021-03-26 07:47:44,369 - mini_batch_size: "64" 2021-03-26 07:47:44,369 - patience: "3" 2021-03-26 07:47:44,370 - anneal_factor: "0.5" 2021-03-26 07:47:44,370 - max_epochs: "150" 2021-03-26 07:47:44,370 - shuffle: "True" 2021-03-26 07:47:44,371 - train_with_dev: "False" 2021-03-26 07:47:44,371 - batch_growth_annealing: "False" 2021-03-26 07:47:44,371 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:44,372 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260747" 2021-03-26 07:47:44,372 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:44,373 Device: cuda:0 2021-03-26 07:47:44,373 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:44,373 Embeddings storage mode: cpu 2021-03-26 07:47:44,375 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:47:46,063 epoch 1 - iter 2/25 - loss 82.29800034 - samples/sec: 75.91 - lr: 0.500000 2021-03-26 07:47:47,596 epoch 1 - iter 4/25 - loss 86.63934135 - samples/sec: 83.61 - lr: 0.500000 2021-03-26 07:47:48,959 epoch 1 - iter 6/25 - loss 79.94940567 - samples/sec: 93.96 - lr: 0.500000 2021-03-26 07:47:50,361 epoch 1 - iter 8/25 - loss 77.86869812 - samples/sec: 91.37 - lr: 0.500000 2021-03-26 07:47:51,702 epoch 1 - iter 10/25 - loss 74.69780655 - samples/sec: 95.57 - lr: 0.500000 2021-03-26 07:47:52,924 epoch 1 - iter 12/25 - loss 72.09690285 - samples/sec: 104.85 - lr: 0.500000 2021-03-26 07:47:54,260 epoch 1 - iter 14/25 - loss 70.73670796 - samples/sec: 95.90 - lr: 0.500000 2021-03-26 07:47:55,589 epoch 1 - iter 16/25 - loss 68.22431612 - samples/sec: 96.47 - lr: 0.500000 2021-03-26 07:47:57,001 epoch 1 - iter 18/25 - loss 66.37756750 - samples/sec: 90.72 - lr: 0.500000 2021-03-26 07:47:58,261 epoch 1 - iter 20/25 - loss 64.80691071 - samples/sec: 101.65 - lr: 0.500000 2021-03-26 07:47:59,516 epoch 1 - iter 22/25 - loss 63.18399325 - samples/sec: 102.11 - lr: 0.500000 2021-03-26 07:48:00,785 epoch 1 - iter 24/25 - loss 62.12906313 - samples/sec: 101.03 - lr: 0.500000 2021-03-26 07:48:01,328 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:48:01,328 EPOCH 1 done: loss 62.0460 - lr 0.5000000 2021-03-26 07:48:02,503 DEV : loss 41.141868591308594 - score 0.3298 2021-03-26 07:48:02,525 BAD EPOCHS (no improvement): 0 2021-03-26 07:48:11,783 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:48:12,737 epoch 2 - iter 2/25 - loss 40.23099422 - samples/sec: 134.34 - lr: 0.500000 2021-03-26 07:48:13,743 epoch 2 - iter 4/25 - loss 43.27464151 - samples/sec: 127.49 - lr: 0.500000 2021-03-26 07:48:14,783 epoch 2 - iter 6/25 - loss 42.34753831 - samples/sec: 123.20 - lr: 0.500000 2021-03-26 07:48:15,725 epoch 2 - iter 8/25 - loss 41.78717351 - samples/sec: 136.09 - lr: 0.500000 2021-03-26 07:48:16,657 epoch 2 - iter 10/25 - loss 41.55979214 - samples/sec: 137.55 - lr: 0.500000 2021-03-26 07:48:17,587 epoch 2 - iter 12/25 - loss 40.78030396 - samples/sec: 137.84 - lr: 0.500000 2021-03-26 07:48:18,534 epoch 2 - iter 14/25 - loss 39.94518444 - samples/sec: 135.34 - lr: 0.500000 2021-03-26 07:48:19,497 epoch 2 - iter 16/25 - loss 39.79153347 - samples/sec: 133.14 - lr: 0.500000 2021-03-26 07:48:20,461 epoch 2 - iter 18/25 - loss 39.26791233 - samples/sec: 133.07 - lr: 0.500000 2021-03-26 07:48:21,479 epoch 2 - iter 20/25 - loss 38.48154440 - samples/sec: 125.87 - lr: 0.500000 2021-03-26 07:48:22,407 epoch 2 - iter 22/25 - loss 37.72989420 - samples/sec: 138.28 - lr: 0.500000 2021-03-26 07:48:23,413 epoch 2 - iter 24/25 - loss 37.17000763 - samples/sec: 127.35 - lr: 0.500000 2021-03-26 07:48:23,756 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:48:23,757 EPOCH 2 done: loss 36.6704 - lr 0.5000000 2021-03-26 07:48:24,470 DEV : loss 27.52168846130371 - score 0.5767 2021-03-26 07:48:24,488 BAD EPOCHS (no improvement): 0 2021-03-26 07:48:33,906 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:48:34,930 epoch 3 - iter 2/25 - loss 31.42631340 - samples/sec: 125.23 - lr: 0.500000 2021-03-26 07:48:35,829 epoch 3 - iter 4/25 - loss 30.74935913 - samples/sec: 142.67 - lr: 0.500000 2021-03-26 07:48:36,887 epoch 3 - iter 6/25 - loss 30.67000516 - samples/sec: 121.16 - lr: 0.500000 2021-03-26 07:48:37,905 epoch 3 - iter 8/25 - loss 30.57007957 - samples/sec: 125.87 - lr: 0.500000 2021-03-26 07:48:38,826 epoch 3 - iter 10/25 - loss 29.24179325 - samples/sec: 139.08 - lr: 0.500000 2021-03-26 07:48:39,726 epoch 3 - iter 12/25 - loss 28.71167231 - samples/sec: 142.44 - lr: 0.500000 2021-03-26 07:48:40,635 epoch 3 - iter 14/25 - loss 28.50176103 - samples/sec: 141.10 - lr: 0.500000 2021-03-26 07:48:41,535 epoch 3 - iter 16/25 - loss 27.73931015 - samples/sec: 142.40 - lr: 0.500000 2021-03-26 07:48:42,535 epoch 3 - iter 18/25 - loss 27.32102193 - samples/sec: 128.23 - lr: 0.500000 2021-03-26 07:48:43,648 epoch 3 - iter 20/25 - loss 26.90655651 - samples/sec: 115.15 - lr: 0.500000 2021-03-26 07:48:44,618 epoch 3 - iter 22/25 - loss 26.83808604 - samples/sec: 132.17 - lr: 0.500000 2021-03-26 07:48:45,525 epoch 3 - iter 24/25 - loss 26.56665818 - samples/sec: 141.39 - lr: 0.500000 2021-03-26 07:48:46,009 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:48:46,009 EPOCH 3 done: loss 26.6577 - lr 0.5000000 2021-03-26 07:48:46,736 DEV : loss 19.457054138183594 - score 0.6646 2021-03-26 07:48:46,759 BAD EPOCHS (no improvement): 0 2021-03-26 07:48:56,214 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:48:57,224 epoch 4 - iter 2/25 - loss 22.99335194 - samples/sec: 126.92 - lr: 0.500000 2021-03-26 07:48:58,204 epoch 4 - iter 4/25 - loss 21.46911478 - samples/sec: 130.90 - lr: 0.500000 2021-03-26 07:48:59,167 epoch 4 - iter 6/25 - loss 21.33844217 - samples/sec: 133.15 - lr: 0.500000 2021-03-26 07:49:00,171 epoch 4 - iter 8/25 - loss 21.12010169 - samples/sec: 127.80 - lr: 0.500000 2021-03-26 07:49:01,216 epoch 4 - iter 10/25 - loss 20.65613098 - samples/sec: 122.57 - lr: 0.500000 2021-03-26 07:49:02,258 epoch 4 - iter 12/25 - loss 20.85910098 - samples/sec: 123.21 - lr: 0.500000 2021-03-26 07:49:03,221 epoch 4 - iter 14/25 - loss 20.61655971 - samples/sec: 133.16 - lr: 0.500000 2021-03-26 07:49:04,185 epoch 4 - iter 16/25 - loss 20.45642447 - samples/sec: 133.02 - lr: 0.500000 2021-03-26 07:49:05,120 epoch 4 - iter 18/25 - loss 20.19972791 - samples/sec: 137.10 - lr: 0.500000 2021-03-26 07:49:06,146 epoch 4 - iter 20/25 - loss 20.46301136 - samples/sec: 124.92 - lr: 0.500000 2021-03-26 07:49:07,218 epoch 4 - iter 22/25 - loss 20.59231134 - samples/sec: 119.61 - lr: 0.500000 2021-03-26 07:49:08,211 epoch 4 - iter 24/25 - loss 20.43321848 - samples/sec: 129.24 - lr: 0.500000 2021-03-26 07:49:08,686 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:49:08,686 EPOCH 4 done: loss 20.3970 - lr 0.5000000 2021-03-26 07:49:09,409 DEV : loss 14.515106201171875 - score 0.7436 2021-03-26 07:49:09,432 BAD EPOCHS (no improvement): 0 2021-03-26 07:49:18,910 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:49:19,938 epoch 5 - iter 2/25 - loss 16.30500841 - samples/sec: 124.87 - lr: 0.500000 2021-03-26 07:49:21,073 epoch 5 - iter 4/25 - loss 16.66907477 - samples/sec: 113.01 - lr: 0.500000 2021-03-26 07:49:22,070 epoch 5 - iter 6/25 - loss 16.89636453 - samples/sec: 128.51 - lr: 0.500000 2021-03-26 07:49:23,008 epoch 5 - iter 8/25 - loss 17.29816079 - samples/sec: 136.71 - lr: 0.500000 2021-03-26 07:49:24,077 epoch 5 - iter 10/25 - loss 17.58455524 - samples/sec: 120.01 - lr: 0.500000 2021-03-26 07:49:25,115 epoch 5 - iter 12/25 - loss 17.42553425 - samples/sec: 123.42 - lr: 0.500000 2021-03-26 07:49:26,059 epoch 5 - iter 14/25 - loss 17.41548838 - samples/sec: 135.86 - lr: 0.500000 2021-03-26 07:49:27,146 epoch 5 - iter 16/25 - loss 17.00069827 - samples/sec: 117.90 - lr: 0.500000 2021-03-26 07:49:28,079 epoch 5 - iter 18/25 - loss 17.08402385 - samples/sec: 137.47 - lr: 0.500000 2021-03-26 07:49:29,011 epoch 5 - iter 20/25 - loss 17.05419016 - samples/sec: 137.58 - lr: 0.500000 2021-03-26 07:49:30,011 epoch 5 - iter 22/25 - loss 16.79865685 - samples/sec: 128.23 - lr: 0.500000 2021-03-26 07:49:30,955 epoch 5 - iter 24/25 - loss 16.75121248 - samples/sec: 135.78 - lr: 0.500000 2021-03-26 07:49:31,389 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:49:31,389 EPOCH 5 done: loss 16.7338 - lr 0.5000000 2021-03-26 07:49:32,101 DEV : loss 11.223102569580078 - score 0.7999 2021-03-26 07:49:32,125 BAD EPOCHS (no improvement): 0 2021-03-26 07:49:41,554 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:49:42,543 epoch 6 - iter 2/25 - loss 13.59014797 - samples/sec: 129.75 - lr: 0.500000 2021-03-26 07:49:43,617 epoch 6 - iter 4/25 - loss 15.00557327 - samples/sec: 119.32 - lr: 0.500000 2021-03-26 07:49:44,655 epoch 6 - iter 6/25 - loss 15.50979487 - samples/sec: 123.50 - lr: 0.500000 2021-03-26 07:49:45,650 epoch 6 - iter 8/25 - loss 15.80515766 - samples/sec: 128.84 - lr: 0.500000 2021-03-26 07:49:46,584 epoch 6 - iter 10/25 - loss 15.28509512 - samples/sec: 137.26 - lr: 0.500000 2021-03-26 07:49:47,567 epoch 6 - iter 12/25 - loss 14.93954698 - samples/sec: 130.42 - lr: 0.500000 2021-03-26 07:49:48,566 epoch 6 - iter 14/25 - loss 15.16099828 - samples/sec: 128.34 - lr: 0.500000 2021-03-26 07:49:49,564 epoch 6 - iter 16/25 - loss 14.71998441 - samples/sec: 128.47 - lr: 0.500000 2021-03-26 07:49:50,587 epoch 6 - iter 18/25 - loss 14.49083164 - samples/sec: 125.45 - lr: 0.500000 2021-03-26 07:49:51,542 epoch 6 - iter 20/25 - loss 14.44817972 - samples/sec: 134.06 - lr: 0.500000 2021-03-26 07:49:52,538 epoch 6 - iter 22/25 - loss 14.51069000 - samples/sec: 128.74 - lr: 0.500000 2021-03-26 07:49:53,434 epoch 6 - iter 24/25 - loss 14.39883967 - samples/sec: 143.11 - lr: 0.500000 2021-03-26 07:49:53,875 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:49:53,875 EPOCH 6 done: loss 14.4636 - lr 0.5000000 2021-03-26 07:49:54,605 DEV : loss 9.920324325561523 - score 0.8191 2021-03-26 07:49:54,628 BAD EPOCHS (no improvement): 0 2021-03-26 07:50:04,174 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:50:05,175 epoch 7 - iter 2/25 - loss 13.02674007 - samples/sec: 128.18 - lr: 0.500000 2021-03-26 07:50:06,133 epoch 7 - iter 4/25 - loss 12.42072916 - samples/sec: 133.74 - lr: 0.500000 2021-03-26 07:50:07,111 epoch 7 - iter 6/25 - loss 13.04151154 - samples/sec: 131.06 - lr: 0.500000 2021-03-26 07:50:08,098 epoch 7 - iter 8/25 - loss 13.00413275 - samples/sec: 130.06 - lr: 0.500000 2021-03-26 07:50:09,096 epoch 7 - iter 10/25 - loss 13.29990978 - samples/sec: 128.32 - lr: 0.500000 2021-03-26 07:50:10,077 epoch 7 - iter 12/25 - loss 13.34977380 - samples/sec: 130.78 - lr: 0.500000 2021-03-26 07:50:10,983 epoch 7 - iter 14/25 - loss 13.37637084 - samples/sec: 141.41 - lr: 0.500000 2021-03-26 07:50:11,973 epoch 7 - iter 16/25 - loss 13.16614377 - samples/sec: 129.41 - lr: 0.500000 2021-03-26 07:50:12,880 epoch 7 - iter 18/25 - loss 13.04947027 - samples/sec: 141.42 - lr: 0.500000 2021-03-26 07:50:13,845 epoch 7 - iter 20/25 - loss 13.18624725 - samples/sec: 132.82 - lr: 0.500000 2021-03-26 07:50:14,782 epoch 7 - iter 22/25 - loss 13.21761140 - samples/sec: 136.79 - lr: 0.500000 2021-03-26 07:50:15,775 epoch 7 - iter 24/25 - loss 13.07190700 - samples/sec: 129.11 - lr: 0.500000 2021-03-26 07:50:16,101 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:50:16,102 EPOCH 7 done: loss 12.9792 - lr 0.5000000 2021-03-26 07:50:16,808 DEV : loss 8.733400344848633 - score 0.8492 2021-03-26 07:50:16,831 BAD EPOCHS (no improvement): 0 2021-03-26 07:50:26,329 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:50:27,316 epoch 8 - iter 2/25 - loss 11.35763407 - samples/sec: 130.12 - lr: 0.500000 2021-03-26 07:50:28,217 epoch 8 - iter 4/25 - loss 12.17101026 - samples/sec: 142.29 - lr: 0.500000 2021-03-26 07:50:29,203 epoch 8 - iter 6/25 - loss 12.12305895 - samples/sec: 129.94 - lr: 0.500000 2021-03-26 07:50:30,141 epoch 8 - iter 8/25 - loss 11.69760966 - samples/sec: 136.60 - lr: 0.500000 2021-03-26 07:50:31,170 epoch 8 - iter 10/25 - loss 11.54711952 - samples/sec: 124.59 - lr: 0.500000 2021-03-26 07:50:32,163 epoch 8 - iter 12/25 - loss 11.50161934 - samples/sec: 129.13 - lr: 0.500000 2021-03-26 07:50:33,209 epoch 8 - iter 14/25 - loss 11.45421546 - samples/sec: 122.52 - lr: 0.500000 2021-03-26 07:50:34,148 epoch 8 - iter 16/25 - loss 11.58866638 - samples/sec: 136.51 - lr: 0.500000 2021-03-26 07:50:35,143 epoch 8 - iter 18/25 - loss 11.90951268 - samples/sec: 128.91 - lr: 0.500000 2021-03-26 07:50:36,152 epoch 8 - iter 20/25 - loss 11.85078907 - samples/sec: 126.97 - lr: 0.500000 2021-03-26 07:50:37,289 epoch 8 - iter 22/25 - loss 11.94904106 - samples/sec: 112.76 - lr: 0.500000 2021-03-26 07:50:38,344 epoch 8 - iter 24/25 - loss 11.85964342 - samples/sec: 121.51 - lr: 0.500000 2021-03-26 07:50:38,732 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:50:38,734 EPOCH 8 done: loss 11.8554 - lr 0.5000000 2021-03-26 07:50:39,477 DEV : loss 8.442928314208984 - score 0.8514 2021-03-26 07:50:39,492 BAD EPOCHS (no improvement): 0 2021-03-26 07:50:49,122 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:50:50,141 epoch 9 - iter 2/25 - loss 9.26947498 - samples/sec: 125.97 - lr: 0.500000 2021-03-26 07:50:51,119 epoch 9 - iter 4/25 - loss 9.99433255 - samples/sec: 131.15 - lr: 0.500000 2021-03-26 07:50:52,141 epoch 9 - iter 6/25 - loss 10.41451152 - samples/sec: 125.36 - lr: 0.500000 2021-03-26 07:50:53,079 epoch 9 - iter 8/25 - loss 10.95319188 - samples/sec: 136.71 - lr: 0.500000 2021-03-26 07:50:54,140 epoch 9 - iter 10/25 - loss 11.07087307 - samples/sec: 120.88 - lr: 0.500000 2021-03-26 07:50:55,277 epoch 9 - iter 12/25 - loss 11.12073040 - samples/sec: 112.75 - lr: 0.500000 2021-03-26 07:50:56,251 epoch 9 - iter 14/25 - loss 10.87324211 - samples/sec: 131.64 - lr: 0.500000 2021-03-26 07:50:57,192 epoch 9 - iter 16/25 - loss 10.93167228 - samples/sec: 136.26 - lr: 0.500000 2021-03-26 07:50:58,211 epoch 9 - iter 18/25 - loss 10.78684590 - samples/sec: 125.85 - lr: 0.500000 2021-03-26 07:50:59,305 epoch 9 - iter 20/25 - loss 10.82854271 - samples/sec: 117.21 - lr: 0.500000 2021-03-26 07:51:00,267 epoch 9 - iter 22/25 - loss 10.73710918 - samples/sec: 133.28 - lr: 0.500000 2021-03-26 07:51:01,216 epoch 9 - iter 24/25 - loss 10.62689821 - samples/sec: 135.07 - lr: 0.500000 2021-03-26 07:51:01,562 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:51:01,562 EPOCH 9 done: loss 10.6405 - lr 0.5000000 2021-03-26 07:51:02,273 DEV : loss 7.894908905029297 - score 0.8567 2021-03-26 07:51:02,289 BAD EPOCHS (no improvement): 0 2021-03-26 07:51:11,862 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:51:12,885 epoch 10 - iter 2/25 - loss 9.14244080 - samples/sec: 125.42 - lr: 0.500000 2021-03-26 07:51:13,987 epoch 10 - iter 4/25 - loss 9.32414103 - samples/sec: 116.23 - lr: 0.500000 2021-03-26 07:51:14,926 epoch 10 - iter 6/25 - loss 9.80224291 - samples/sec: 136.64 - lr: 0.500000 2021-03-26 07:51:15,828 epoch 10 - iter 8/25 - loss 9.85451603 - samples/sec: 142.12 - lr: 0.500000 2021-03-26 07:51:16,856 epoch 10 - iter 10/25 - loss 10.12377825 - samples/sec: 124.72 - lr: 0.500000 2021-03-26 07:51:17,805 epoch 10 - iter 12/25 - loss 10.22787682 - samples/sec: 135.21 - lr: 0.500000 2021-03-26 07:51:18,782 epoch 10 - iter 14/25 - loss 10.23994528 - samples/sec: 131.21 - lr: 0.500000 2021-03-26 07:51:19,750 epoch 10 - iter 16/25 - loss 10.26584750 - samples/sec: 132.35 - lr: 0.500000 2021-03-26 07:51:20,800 epoch 10 - iter 18/25 - loss 10.03869134 - samples/sec: 122.03 - lr: 0.500000 2021-03-26 07:51:21,765 epoch 10 - iter 20/25 - loss 10.07074564 - samples/sec: 132.86 - lr: 0.500000 2021-03-26 07:51:22,791 epoch 10 - iter 22/25 - loss 10.11766666 - samples/sec: 124.93 - lr: 0.500000 2021-03-26 07:51:23,761 epoch 10 - iter 24/25 - loss 10.09356664 - samples/sec: 132.20 - lr: 0.500000 2021-03-26 07:51:24,210 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:51:24,211 EPOCH 10 done: loss 10.0166 - lr 0.5000000 2021-03-26 07:51:24,909 DEV : loss 7.626852989196777 - score 0.8632 2021-03-26 07:51:24,931 BAD EPOCHS (no improvement): 0 2021-03-26 07:51:34,414 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:51:35,444 epoch 11 - iter 2/25 - loss 8.91252089 - samples/sec: 124.50 - lr: 0.500000 2021-03-26 07:51:36,482 epoch 11 - iter 4/25 - loss 10.49854732 - samples/sec: 123.46 - lr: 0.500000 2021-03-26 07:51:37,509 epoch 11 - iter 6/25 - loss 10.22471849 - samples/sec: 124.78 - lr: 0.500000 2021-03-26 07:51:38,507 epoch 11 - iter 8/25 - loss 10.09014302 - samples/sec: 128.50 - lr: 0.500000 2021-03-26 07:51:39,488 epoch 11 - iter 10/25 - loss 9.73106799 - samples/sec: 130.67 - lr: 0.500000 2021-03-26 07:51:40,420 epoch 11 - iter 12/25 - loss 9.70978502 - samples/sec: 137.52 - lr: 0.500000 2021-03-26 07:51:41,410 epoch 11 - iter 14/25 - loss 9.73665486 - samples/sec: 129.50 - lr: 0.500000 2021-03-26 07:51:42,513 epoch 11 - iter 16/25 - loss 9.72871694 - samples/sec: 116.19 - lr: 0.500000 2021-03-26 07:51:43,538 epoch 11 - iter 18/25 - loss 9.64404469 - samples/sec: 124.99 - lr: 0.500000 2021-03-26 07:51:44,531 epoch 11 - iter 20/25 - loss 9.57440164 - samples/sec: 129.19 - lr: 0.500000 2021-03-26 07:51:45,455 epoch 11 - iter 22/25 - loss 9.50417950 - samples/sec: 138.90 - lr: 0.500000 2021-03-26 07:51:46,481 epoch 11 - iter 24/25 - loss 9.46735579 - samples/sec: 125.01 - lr: 0.500000 2021-03-26 07:51:46,855 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:51:46,855 EPOCH 11 done: loss 9.4820 - lr 0.5000000 2021-03-26 07:51:47,566 DEV : loss 7.620308876037598 - score 0.8716 2021-03-26 07:51:47,585 BAD EPOCHS (no improvement): 0 2021-03-26 07:51:57,050 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:51:58,083 epoch 12 - iter 2/25 - loss 7.67201424 - samples/sec: 124.09 - lr: 0.500000 2021-03-26 07:51:59,256 epoch 12 - iter 4/25 - loss 8.82815576 - samples/sec: 109.28 - lr: 0.500000 2021-03-26 07:52:00,158 epoch 12 - iter 6/25 - loss 8.54262582 - samples/sec: 142.13 - lr: 0.500000 2021-03-26 07:52:01,363 epoch 12 - iter 8/25 - loss 8.54254794 - samples/sec: 106.30 - lr: 0.500000 2021-03-26 07:52:02,511 epoch 12 - iter 10/25 - loss 8.47853055 - samples/sec: 111.64 - lr: 0.500000 2021-03-26 07:52:03,550 epoch 12 - iter 12/25 - loss 8.54671808 - samples/sec: 123.45 - lr: 0.500000 2021-03-26 07:52:04,530 epoch 12 - iter 14/25 - loss 8.79490154 - samples/sec: 130.86 - lr: 0.500000 2021-03-26 07:52:05,419 epoch 12 - iter 16/25 - loss 8.82608131 - samples/sec: 144.29 - lr: 0.500000 2021-03-26 07:52:06,437 epoch 12 - iter 18/25 - loss 8.86837135 - samples/sec: 125.84 - lr: 0.500000 2021-03-26 07:52:07,425 epoch 12 - iter 20/25 - loss 8.90788066 - samples/sec: 129.71 - lr: 0.500000 2021-03-26 07:52:08,398 epoch 12 - iter 22/25 - loss 8.91862988 - samples/sec: 131.79 - lr: 0.500000 2021-03-26 07:52:09,330 epoch 12 - iter 24/25 - loss 8.84400580 - samples/sec: 137.58 - lr: 0.500000 2021-03-26 07:52:09,699 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:52:09,700 EPOCH 12 done: loss 8.9432 - lr 0.5000000 2021-03-26 07:52:10,423 DEV : loss 6.980446815490723 - score 0.8768 2021-03-26 07:52:10,446 BAD EPOCHS (no improvement): 0 2021-03-26 07:52:20,022 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:52:20,956 epoch 13 - iter 2/25 - loss 7.05646849 - samples/sec: 137.35 - lr: 0.500000 2021-03-26 07:52:21,982 epoch 13 - iter 4/25 - loss 7.92554772 - samples/sec: 124.97 - lr: 0.500000 2021-03-26 07:52:22,942 epoch 13 - iter 6/25 - loss 7.98379747 - samples/sec: 133.41 - lr: 0.500000 2021-03-26 07:52:23,958 epoch 13 - iter 8/25 - loss 8.20600587 - samples/sec: 126.20 - lr: 0.500000 2021-03-26 07:52:24,950 epoch 13 - iter 10/25 - loss 8.51461835 - samples/sec: 129.37 - lr: 0.500000 2021-03-26 07:52:25,909 epoch 13 - iter 12/25 - loss 8.30212975 - samples/sec: 133.64 - lr: 0.500000 2021-03-26 07:52:26,958 epoch 13 - iter 14/25 - loss 8.24377768 - samples/sec: 122.13 - lr: 0.500000 2021-03-26 07:52:27,908 epoch 13 - iter 16/25 - loss 8.16531828 - samples/sec: 135.01 - lr: 0.500000 2021-03-26 07:52:29,409 epoch 13 - iter 18/25 - loss 8.26274286 - samples/sec: 85.57 - lr: 0.500000 2021-03-26 07:52:30,333 epoch 13 - iter 20/25 - loss 8.25441029 - samples/sec: 138.93 - lr: 0.500000 2021-03-26 07:52:31,310 epoch 13 - iter 22/25 - loss 8.27757309 - samples/sec: 131.11 - lr: 0.500000 2021-03-26 07:52:32,235 epoch 13 - iter 24/25 - loss 8.19572711 - samples/sec: 138.71 - lr: 0.500000 2021-03-26 07:52:32,605 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:52:32,606 EPOCH 13 done: loss 8.1866 - lr 0.5000000 2021-03-26 07:52:33,404 DEV : loss 7.257625579833984 - score 0.8733 2021-03-26 07:52:33,435 BAD EPOCHS (no improvement): 1 2021-03-26 07:52:33,436 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:52:34,424 epoch 14 - iter 2/25 - loss 6.78682852 - samples/sec: 129.80 - lr: 0.500000 2021-03-26 07:52:35,467 epoch 14 - iter 4/25 - loss 7.39494026 - samples/sec: 122.86 - lr: 0.500000 2021-03-26 07:52:36,561 epoch 14 - iter 6/25 - loss 7.72300220 - samples/sec: 117.14 - lr: 0.500000 2021-03-26 07:52:37,529 epoch 14 - iter 8/25 - loss 7.89962667 - samples/sec: 132.48 - lr: 0.500000 2021-03-26 07:52:38,382 epoch 14 - iter 10/25 - loss 7.97575660 - samples/sec: 150.40 - lr: 0.500000 2021-03-26 07:52:39,356 epoch 14 - iter 12/25 - loss 8.17424468 - samples/sec: 131.59 - lr: 0.500000 2021-03-26 07:52:40,322 epoch 14 - iter 14/25 - loss 8.15351108 - samples/sec: 132.69 - lr: 0.500000 2021-03-26 07:52:41,273 epoch 14 - iter 16/25 - loss 8.02801412 - samples/sec: 134.77 - lr: 0.500000 2021-03-26 07:52:42,181 epoch 14 - iter 18/25 - loss 8.01207256 - samples/sec: 141.41 - lr: 0.500000 2021-03-26 07:52:43,190 epoch 14 - iter 20/25 - loss 8.08496556 - samples/sec: 126.91 - lr: 0.500000 2021-03-26 07:52:44,144 epoch 14 - iter 22/25 - loss 8.06667874 - samples/sec: 134.46 - lr: 0.500000 2021-03-26 07:52:45,052 epoch 14 - iter 24/25 - loss 7.95059959 - samples/sec: 141.03 - lr: 0.500000 2021-03-26 07:52:45,397 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:52:45,398 EPOCH 14 done: loss 7.9863 - lr 0.5000000 2021-03-26 07:52:46,098 DEV : loss 6.4345903396606445 - score 0.8932 2021-03-26 07:52:46,120 BAD EPOCHS (no improvement): 0 2021-03-26 07:52:55,331 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:52:56,273 epoch 15 - iter 2/25 - loss 6.71810126 - samples/sec: 136.21 - lr: 0.500000 2021-03-26 07:52:57,302 epoch 15 - iter 4/25 - loss 7.19238472 - samples/sec: 124.59 - lr: 0.500000 2021-03-26 07:52:58,313 epoch 15 - iter 6/25 - loss 7.44798557 - samples/sec: 126.80 - lr: 0.500000 2021-03-26 07:52:59,309 epoch 15 - iter 8/25 - loss 7.45711040 - samples/sec: 128.56 - lr: 0.500000 2021-03-26 07:53:00,285 epoch 15 - iter 10/25 - loss 7.55697489 - samples/sec: 131.53 - lr: 0.500000 2021-03-26 07:53:01,241 epoch 15 - iter 12/25 - loss 7.40633933 - samples/sec: 134.05 - lr: 0.500000 2021-03-26 07:53:02,215 epoch 15 - iter 14/25 - loss 7.48868448 - samples/sec: 131.48 - lr: 0.500000 2021-03-26 07:53:03,189 epoch 15 - iter 16/25 - loss 7.32153139 - samples/sec: 131.63 - lr: 0.500000 2021-03-26 07:53:04,157 epoch 15 - iter 18/25 - loss 7.27229791 - samples/sec: 132.38 - lr: 0.500000 2021-03-26 07:53:05,160 epoch 15 - iter 20/25 - loss 7.24484162 - samples/sec: 127.90 - lr: 0.500000 2021-03-26 07:53:06,214 epoch 15 - iter 22/25 - loss 7.36682300 - samples/sec: 121.52 - lr: 0.500000 2021-03-26 07:53:07,226 epoch 15 - iter 24/25 - loss 7.40492298 - samples/sec: 126.72 - lr: 0.500000 2021-03-26 07:53:07,634 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:53:07,634 EPOCH 15 done: loss 7.3385 - lr 0.5000000 2021-03-26 07:53:08,344 DEV : loss 6.836724281311035 - score 0.8777 2021-03-26 07:53:08,367 BAD EPOCHS (no improvement): 1 2021-03-26 07:53:08,367 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:53:09,413 epoch 16 - iter 2/25 - loss 6.85027146 - samples/sec: 122.62 - lr: 0.500000 2021-03-26 07:53:10,360 epoch 16 - iter 4/25 - loss 7.33330190 - samples/sec: 135.55 - lr: 0.500000 2021-03-26 07:53:11,350 epoch 16 - iter 6/25 - loss 7.08793155 - samples/sec: 129.51 - lr: 0.500000 2021-03-26 07:53:12,379 epoch 16 - iter 8/25 - loss 6.93696636 - samples/sec: 124.58 - lr: 0.500000 2021-03-26 07:53:13,315 epoch 16 - iter 10/25 - loss 7.01335969 - samples/sec: 136.89 - lr: 0.500000 2021-03-26 07:53:14,336 epoch 16 - iter 12/25 - loss 7.08117270 - samples/sec: 125.55 - lr: 0.500000 2021-03-26 07:53:15,324 epoch 16 - iter 14/25 - loss 7.18411749 - samples/sec: 129.81 - lr: 0.500000 2021-03-26 07:53:16,324 epoch 16 - iter 16/25 - loss 7.09306252 - samples/sec: 128.08 - lr: 0.500000 2021-03-26 07:53:17,201 epoch 16 - iter 18/25 - loss 7.03924876 - samples/sec: 146.25 - lr: 0.500000 2021-03-26 07:53:18,261 epoch 16 - iter 20/25 - loss 7.06018779 - samples/sec: 120.94 - lr: 0.500000 2021-03-26 07:53:19,187 epoch 16 - iter 22/25 - loss 6.94654627 - samples/sec: 138.52 - lr: 0.500000 2021-03-26 07:53:20,187 epoch 16 - iter 24/25 - loss 6.94220535 - samples/sec: 128.15 - lr: 0.500000 2021-03-26 07:53:20,553 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:53:20,554 EPOCH 16 done: loss 6.9130 - lr 0.5000000 2021-03-26 07:53:21,271 DEV : loss 6.996776580810547 - score 0.8729 2021-03-26 07:53:21,288 BAD EPOCHS (no improvement): 2 2021-03-26 07:53:21,288 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:53:22,350 epoch 17 - iter 2/25 - loss 7.18355846 - samples/sec: 120.72 - lr: 0.500000 2021-03-26 07:53:23,351 epoch 17 - iter 4/25 - loss 6.90322018 - samples/sec: 128.19 - lr: 0.500000 2021-03-26 07:53:24,380 epoch 17 - iter 6/25 - loss 6.44195573 - samples/sec: 124.54 - lr: 0.500000 2021-03-26 07:53:25,429 epoch 17 - iter 8/25 - loss 6.49139535 - samples/sec: 122.26 - lr: 0.500000 2021-03-26 07:53:26,473 epoch 17 - iter 10/25 - loss 6.46172991 - samples/sec: 122.68 - lr: 0.500000 2021-03-26 07:53:27,439 epoch 17 - iter 12/25 - loss 6.50268042 - samples/sec: 132.81 - lr: 0.500000 2021-03-26 07:53:28,396 epoch 17 - iter 14/25 - loss 6.74807654 - samples/sec: 133.84 - lr: 0.500000 2021-03-26 07:53:29,381 epoch 17 - iter 16/25 - loss 6.66306704 - samples/sec: 130.21 - lr: 0.500000 2021-03-26 07:53:30,329 epoch 17 - iter 18/25 - loss 6.71620478 - samples/sec: 135.14 - lr: 0.500000 2021-03-26 07:53:31,283 epoch 17 - iter 20/25 - loss 6.61219385 - samples/sec: 134.35 - lr: 0.500000 2021-03-26 07:53:32,313 epoch 17 - iter 22/25 - loss 6.64762094 - samples/sec: 124.53 - lr: 0.500000 2021-03-26 07:53:33,351 epoch 17 - iter 24/25 - loss 6.66213008 - samples/sec: 123.49 - lr: 0.500000 2021-03-26 07:53:33,764 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:53:33,765 EPOCH 17 done: loss 6.7078 - lr 0.5000000 2021-03-26 07:53:34,470 DEV : loss 7.163369655609131 - score 0.8774 2021-03-26 07:53:34,487 BAD EPOCHS (no improvement): 3 2021-03-26 07:53:34,487 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:53:35,424 epoch 18 - iter 2/25 - loss 5.66614342 - samples/sec: 136.85 - lr: 0.500000 2021-03-26 07:53:36,422 epoch 18 - iter 4/25 - loss 6.22169757 - samples/sec: 128.51 - lr: 0.500000 2021-03-26 07:53:37,479 epoch 18 - iter 6/25 - loss 6.44018420 - samples/sec: 121.25 - lr: 0.500000 2021-03-26 07:53:38,479 epoch 18 - iter 8/25 - loss 6.35334659 - samples/sec: 128.20 - lr: 0.500000 2021-03-26 07:53:39,441 epoch 18 - iter 10/25 - loss 6.60762820 - samples/sec: 133.28 - lr: 0.500000 2021-03-26 07:53:40,318 epoch 18 - iter 12/25 - loss 6.52668659 - samples/sec: 146.07 - lr: 0.500000 2021-03-26 07:53:41,289 epoch 18 - iter 14/25 - loss 6.49653762 - samples/sec: 132.09 - lr: 0.500000 2021-03-26 07:53:42,288 epoch 18 - iter 16/25 - loss 6.45660698 - samples/sec: 128.25 - lr: 0.500000 2021-03-26 07:53:43,313 epoch 18 - iter 18/25 - loss 6.49343742 - samples/sec: 125.05 - lr: 0.500000 2021-03-26 07:53:44,321 epoch 18 - iter 20/25 - loss 6.41346722 - samples/sec: 127.13 - lr: 0.500000 2021-03-26 07:53:45,243 epoch 18 - iter 22/25 - loss 6.40939960 - samples/sec: 139.04 - lr: 0.500000 2021-03-26 07:53:46,247 epoch 18 - iter 24/25 - loss 6.44130540 - samples/sec: 127.68 - lr: 0.500000 2021-03-26 07:53:46,713 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:53:46,713 EPOCH 18 done: loss 6.4110 - lr 0.5000000 2021-03-26 07:53:47,407 DEV : loss 6.419670104980469 - score 0.8917 2021-03-26 07:53:47,430 BAD EPOCHS (no improvement): 4 2021-03-26 07:53:47,431 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:53:48,485 epoch 19 - iter 2/25 - loss 6.66521072 - samples/sec: 121.70 - lr: 0.250000 2021-03-26 07:53:49,509 epoch 19 - iter 4/25 - loss 6.19065154 - samples/sec: 125.08 - lr: 0.250000 2021-03-26 07:53:50,400 epoch 19 - iter 6/25 - loss 5.78149088 - samples/sec: 143.87 - lr: 0.250000 2021-03-26 07:53:51,319 epoch 19 - iter 8/25 - loss 5.54224241 - samples/sec: 139.50 - lr: 0.250000 2021-03-26 07:53:52,359 epoch 19 - iter 10/25 - loss 5.49967384 - samples/sec: 123.35 - lr: 0.250000 2021-03-26 07:53:53,332 epoch 19 - iter 12/25 - loss 5.58384212 - samples/sec: 131.92 - lr: 0.250000 2021-03-26 07:53:54,260 epoch 19 - iter 14/25 - loss 5.44728538 - samples/sec: 138.16 - lr: 0.250000 2021-03-26 07:53:55,227 epoch 19 - iter 16/25 - loss 5.31798606 - samples/sec: 132.47 - lr: 0.250000 2021-03-26 07:53:56,159 epoch 19 - iter 18/25 - loss 5.34107133 - samples/sec: 137.57 - lr: 0.250000 2021-03-26 07:53:57,098 epoch 19 - iter 20/25 - loss 5.36848167 - samples/sec: 136.56 - lr: 0.250000 2021-03-26 07:53:58,164 epoch 19 - iter 22/25 - loss 5.34087520 - samples/sec: 120.18 - lr: 0.250000 2021-03-26 07:54:00,147 epoch 19 - iter 24/25 - loss 5.38866775 - samples/sec: 64.60 - lr: 0.250000 2021-03-26 07:54:00,529 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:54:00,530 EPOCH 19 done: loss 5.3676 - lr 0.2500000 2021-03-26 07:54:01,219 DEV : loss 6.131153106689453 - score 0.9014 2021-03-26 07:54:01,242 BAD EPOCHS (no improvement): 0 2021-03-26 07:54:10,431 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:54:11,321 epoch 20 - iter 2/25 - loss 5.13135290 - samples/sec: 144.08 - lr: 0.250000 2021-03-26 07:54:12,240 epoch 20 - iter 4/25 - loss 4.89559913 - samples/sec: 139.45 - lr: 0.250000 2021-03-26 07:54:13,232 epoch 20 - iter 6/25 - loss 4.89744592 - samples/sec: 129.28 - lr: 0.250000 2021-03-26 07:54:14,291 epoch 20 - iter 8/25 - loss 5.01821065 - samples/sec: 121.03 - lr: 0.250000 2021-03-26 07:54:15,261 epoch 20 - iter 10/25 - loss 5.12131805 - samples/sec: 132.08 - lr: 0.250000 2021-03-26 07:54:16,311 epoch 20 - iter 12/25 - loss 5.06408310 - samples/sec: 122.06 - lr: 0.250000 2021-03-26 07:54:17,336 epoch 20 - iter 14/25 - loss 5.08166361 - samples/sec: 125.08 - lr: 0.250000 2021-03-26 07:54:18,357 epoch 20 - iter 16/25 - loss 5.11939466 - samples/sec: 125.51 - lr: 0.250000 2021-03-26 07:54:19,365 epoch 20 - iter 18/25 - loss 5.16788875 - samples/sec: 127.25 - lr: 0.250000 2021-03-26 07:54:20,331 epoch 20 - iter 20/25 - loss 5.13450365 - samples/sec: 132.66 - lr: 0.250000 2021-03-26 07:54:21,269 epoch 20 - iter 22/25 - loss 5.11057932 - samples/sec: 136.79 - lr: 0.250000 2021-03-26 07:54:22,210 epoch 20 - iter 24/25 - loss 5.13755359 - samples/sec: 136.13 - lr: 0.250000 2021-03-26 07:54:22,603 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:54:22,604 EPOCH 20 done: loss 5.1303 - lr 0.2500000 2021-03-26 07:54:23,308 DEV : loss 6.083251476287842 - score 0.9051 2021-03-26 07:54:23,332 BAD EPOCHS (no improvement): 0 2021-03-26 07:54:32,902 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:54:33,912 epoch 21 - iter 2/25 - loss 5.07513094 - samples/sec: 126.92 - lr: 0.250000 2021-03-26 07:54:34,865 epoch 21 - iter 4/25 - loss 4.59668767 - samples/sec: 134.46 - lr: 0.250000 2021-03-26 07:54:35,851 epoch 21 - iter 6/25 - loss 5.09568977 - samples/sec: 129.98 - lr: 0.250000 2021-03-26 07:54:36,825 epoch 21 - iter 8/25 - loss 5.05758089 - samples/sec: 131.64 - lr: 0.250000 2021-03-26 07:54:37,841 epoch 21 - iter 10/25 - loss 5.14766736 - samples/sec: 126.22 - lr: 0.250000 2021-03-26 07:54:38,836 epoch 21 - iter 12/25 - loss 5.02646943 - samples/sec: 128.90 - lr: 0.250000 2021-03-26 07:54:39,832 epoch 21 - iter 14/25 - loss 5.08124334 - samples/sec: 128.62 - lr: 0.250000 2021-03-26 07:54:40,856 epoch 21 - iter 16/25 - loss 5.03595272 - samples/sec: 125.25 - lr: 0.250000 2021-03-26 07:54:41,819 epoch 21 - iter 18/25 - loss 4.96463317 - samples/sec: 133.00 - lr: 0.250000 2021-03-26 07:54:42,750 epoch 21 - iter 20/25 - loss 5.01546376 - samples/sec: 137.84 - lr: 0.250000 2021-03-26 07:54:43,756 epoch 21 - iter 22/25 - loss 5.00643774 - samples/sec: 127.34 - lr: 0.250000 2021-03-26 07:54:44,773 epoch 21 - iter 24/25 - loss 4.98594165 - samples/sec: 126.09 - lr: 0.250000 2021-03-26 07:54:45,267 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:54:45,268 EPOCH 21 done: loss 4.9665 - lr 0.2500000 2021-03-26 07:54:45,997 DEV : loss 5.991503715515137 - score 0.9039 2021-03-26 07:54:46,023 BAD EPOCHS (no improvement): 1 2021-03-26 07:54:46,023 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:54:47,082 epoch 22 - iter 2/25 - loss 4.61021471 - samples/sec: 121.14 - lr: 0.250000 2021-03-26 07:54:48,179 epoch 22 - iter 4/25 - loss 4.90505421 - samples/sec: 116.73 - lr: 0.250000 2021-03-26 07:54:49,127 epoch 22 - iter 6/25 - loss 4.72144151 - samples/sec: 135.31 - lr: 0.250000 2021-03-26 07:54:50,087 epoch 22 - iter 8/25 - loss 4.85369271 - samples/sec: 133.47 - lr: 0.250000 2021-03-26 07:54:51,112 epoch 22 - iter 10/25 - loss 4.81175332 - samples/sec: 124.94 - lr: 0.250000 2021-03-26 07:54:52,276 epoch 22 - iter 12/25 - loss 4.84444952 - samples/sec: 110.16 - lr: 0.250000 2021-03-26 07:54:53,381 epoch 22 - iter 14/25 - loss 4.80926984 - samples/sec: 115.97 - lr: 0.250000 2021-03-26 07:54:54,307 epoch 22 - iter 16/25 - loss 4.71817878 - samples/sec: 138.44 - lr: 0.250000 2021-03-26 07:54:55,316 epoch 22 - iter 18/25 - loss 4.80634297 - samples/sec: 127.00 - lr: 0.250000 2021-03-26 07:54:56,435 epoch 22 - iter 20/25 - loss 4.80697734 - samples/sec: 114.55 - lr: 0.250000 2021-03-26 07:54:57,396 epoch 22 - iter 22/25 - loss 4.84137470 - samples/sec: 133.46 - lr: 0.250000 2021-03-26 07:54:58,321 epoch 22 - iter 24/25 - loss 4.78170185 - samples/sec: 138.50 - lr: 0.250000 2021-03-26 07:54:58,681 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:54:58,682 EPOCH 22 done: loss 4.7751 - lr 0.2500000 2021-03-26 07:54:59,410 DEV : loss 6.314764976501465 - score 0.9001 2021-03-26 07:54:59,437 BAD EPOCHS (no improvement): 2 2021-03-26 07:54:59,438 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:55:00,397 epoch 23 - iter 2/25 - loss 4.72261405 - samples/sec: 133.65 - lr: 0.250000 2021-03-26 07:55:01,388 epoch 23 - iter 4/25 - loss 4.50521022 - samples/sec: 129.32 - lr: 0.250000 2021-03-26 07:55:02,391 epoch 23 - iter 6/25 - loss 4.55807165 - samples/sec: 127.91 - lr: 0.250000 2021-03-26 07:55:03,469 epoch 23 - iter 8/25 - loss 4.49088663 - samples/sec: 118.83 - lr: 0.250000 2021-03-26 07:55:04,517 epoch 23 - iter 10/25 - loss 4.44302006 - samples/sec: 122.35 - lr: 0.250000 2021-03-26 07:55:05,511 epoch 23 - iter 12/25 - loss 4.43786772 - samples/sec: 128.97 - lr: 0.250000 2021-03-26 07:55:06,439 epoch 23 - iter 14/25 - loss 4.38659229 - samples/sec: 138.04 - lr: 0.250000 2021-03-26 07:55:07,430 epoch 23 - iter 16/25 - loss 4.30455354 - samples/sec: 129.39 - lr: 0.250000 2021-03-26 07:55:08,486 epoch 23 - iter 18/25 - loss 4.44754314 - samples/sec: 121.42 - lr: 0.250000 2021-03-26 07:55:09,624 epoch 23 - iter 20/25 - loss 4.51354938 - samples/sec: 112.55 - lr: 0.250000 2021-03-26 07:55:10,623 epoch 23 - iter 22/25 - loss 4.57599319 - samples/sec: 128.33 - lr: 0.250000 2021-03-26 07:55:11,578 epoch 23 - iter 24/25 - loss 4.56353696 - samples/sec: 134.26 - lr: 0.250000 2021-03-26 07:55:11,972 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:55:11,973 EPOCH 23 done: loss 4.6286 - lr 0.2500000 2021-03-26 07:55:12,670 DEV : loss 6.0209550857543945 - score 0.9043 2021-03-26 07:55:12,687 BAD EPOCHS (no improvement): 3 2021-03-26 07:55:12,688 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:55:13,617 epoch 24 - iter 2/25 - loss 4.41330647 - samples/sec: 137.88 - lr: 0.250000 2021-03-26 07:55:14,613 epoch 24 - iter 4/25 - loss 4.45420444 - samples/sec: 128.80 - lr: 0.250000 2021-03-26 07:55:15,576 epoch 24 - iter 6/25 - loss 4.73466698 - samples/sec: 133.14 - lr: 0.250000 2021-03-26 07:55:16,675 epoch 24 - iter 8/25 - loss 4.60940790 - samples/sec: 116.64 - lr: 0.250000 2021-03-26 07:55:17,679 epoch 24 - iter 10/25 - loss 4.70087466 - samples/sec: 127.74 - lr: 0.250000 2021-03-26 07:55:18,752 epoch 24 - iter 12/25 - loss 4.71539728 - samples/sec: 119.48 - lr: 0.250000 2021-03-26 07:55:19,818 epoch 24 - iter 14/25 - loss 4.94809369 - samples/sec: 120.33 - lr: 0.250000 2021-03-26 07:55:20,707 epoch 24 - iter 16/25 - loss 4.83176947 - samples/sec: 144.17 - lr: 0.250000 2021-03-26 07:55:21,710 epoch 24 - iter 18/25 - loss 4.76679060 - samples/sec: 127.78 - lr: 0.250000 2021-03-26 07:55:22,612 epoch 24 - iter 20/25 - loss 4.67498168 - samples/sec: 142.17 - lr: 0.250000 2021-03-26 07:55:23,582 epoch 24 - iter 22/25 - loss 4.66213375 - samples/sec: 132.25 - lr: 0.250000 2021-03-26 07:55:24,509 epoch 24 - iter 24/25 - loss 4.64017315 - samples/sec: 138.16 - lr: 0.250000 2021-03-26 07:55:25,050 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:55:25,051 EPOCH 24 done: loss 4.6423 - lr 0.2500000 2021-03-26 07:55:25,770 DEV : loss 6.313882827758789 - score 0.8984 2021-03-26 07:55:25,787 BAD EPOCHS (no improvement): 4 2021-03-26 07:55:25,788 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:55:26,720 epoch 25 - iter 2/25 - loss 4.32253599 - samples/sec: 137.42 - lr: 0.125000 2021-03-26 07:55:27,739 epoch 25 - iter 4/25 - loss 4.18250024 - samples/sec: 125.83 - lr: 0.125000 2021-03-26 07:55:28,672 epoch 25 - iter 6/25 - loss 4.49590453 - samples/sec: 137.51 - lr: 0.125000 2021-03-26 07:55:29,685 epoch 25 - iter 8/25 - loss 4.32094258 - samples/sec: 126.65 - lr: 0.125000 2021-03-26 07:55:30,652 epoch 25 - iter 10/25 - loss 4.35360093 - samples/sec: 132.42 - lr: 0.125000 2021-03-26 07:55:31,693 epoch 25 - iter 12/25 - loss 4.36569381 - samples/sec: 123.19 - lr: 0.125000 2021-03-26 07:55:32,614 epoch 25 - iter 14/25 - loss 4.32159005 - samples/sec: 139.21 - lr: 0.125000 2021-03-26 07:55:33,653 epoch 25 - iter 16/25 - loss 4.24558356 - samples/sec: 123.38 - lr: 0.125000 2021-03-26 07:55:34,617 epoch 25 - iter 18/25 - loss 4.22551627 - samples/sec: 133.01 - lr: 0.125000 2021-03-26 07:55:35,577 epoch 25 - iter 20/25 - loss 4.23848104 - samples/sec: 133.47 - lr: 0.125000 2021-03-26 07:55:36,581 epoch 25 - iter 22/25 - loss 4.18101357 - samples/sec: 127.68 - lr: 0.125000 2021-03-26 07:55:37,655 epoch 25 - iter 24/25 - loss 4.18645438 - samples/sec: 119.40 - lr: 0.125000 2021-03-26 07:55:38,209 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:55:38,209 EPOCH 25 done: loss 4.1730 - lr 0.1250000 2021-03-26 07:55:38,921 DEV : loss 5.988753318786621 - score 0.9076 2021-03-26 07:55:38,944 BAD EPOCHS (no improvement): 0 2021-03-26 07:55:48,363 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:55:49,285 epoch 26 - iter 2/25 - loss 4.35821223 - samples/sec: 139.18 - lr: 0.125000 2021-03-26 07:55:50,217 epoch 26 - iter 4/25 - loss 4.52768040 - samples/sec: 137.51 - lr: 0.125000 2021-03-26 07:55:51,212 epoch 26 - iter 6/25 - loss 4.56613183 - samples/sec: 128.75 - lr: 0.125000 2021-03-26 07:55:52,148 epoch 26 - iter 8/25 - loss 4.34084165 - samples/sec: 137.01 - lr: 0.125000 2021-03-26 07:55:53,102 epoch 26 - iter 10/25 - loss 4.26465180 - samples/sec: 134.35 - lr: 0.125000 2021-03-26 07:55:54,004 epoch 26 - iter 12/25 - loss 4.19501319 - samples/sec: 142.17 - lr: 0.125000 2021-03-26 07:55:55,014 epoch 26 - iter 14/25 - loss 4.12131996 - samples/sec: 126.85 - lr: 0.125000 2021-03-26 07:55:56,021 epoch 26 - iter 16/25 - loss 4.20322640 - samples/sec: 127.34 - lr: 0.125000 2021-03-26 07:55:57,307 epoch 26 - iter 18/25 - loss 4.15836038 - samples/sec: 99.62 - lr: 0.125000 2021-03-26 07:55:58,309 epoch 26 - iter 20/25 - loss 4.14088476 - samples/sec: 127.91 - lr: 0.125000 2021-03-26 07:55:59,211 epoch 26 - iter 22/25 - loss 4.06868502 - samples/sec: 142.16 - lr: 0.125000 2021-03-26 07:56:00,131 epoch 26 - iter 24/25 - loss 4.07539245 - samples/sec: 139.38 - lr: 0.125000 2021-03-26 07:56:00,505 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:56:00,506 EPOCH 26 done: loss 4.0593 - lr 0.1250000 2021-03-26 07:56:01,210 DEV : loss 5.971909999847412 - score 0.9089 2021-03-26 07:56:01,232 BAD EPOCHS (no improvement): 0 2021-03-26 07:56:10,845 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:56:11,860 epoch 27 - iter 2/25 - loss 4.68327546 - samples/sec: 126.33 - lr: 0.125000 2021-03-26 07:56:12,875 epoch 27 - iter 4/25 - loss 4.54550594 - samples/sec: 126.33 - lr: 0.125000 2021-03-26 07:56:13,771 epoch 27 - iter 6/25 - loss 4.35505827 - samples/sec: 143.85 - lr: 0.125000 2021-03-26 07:56:14,750 epoch 27 - iter 8/25 - loss 4.39272493 - samples/sec: 130.93 - lr: 0.125000 2021-03-26 07:56:16,153 epoch 27 - iter 10/25 - loss 4.28880415 - samples/sec: 91.35 - lr: 0.125000 2021-03-26 07:56:17,447 epoch 27 - iter 12/25 - loss 4.25273832 - samples/sec: 99.00 - lr: 0.125000 2021-03-26 07:56:18,522 epoch 27 - iter 14/25 - loss 4.23168690 - samples/sec: 119.20 - lr: 0.125000 2021-03-26 07:56:19,857 epoch 27 - iter 16/25 - loss 4.15360086 - samples/sec: 96.01 - lr: 0.125000 2021-03-26 07:56:21,082 epoch 27 - iter 18/25 - loss 4.17100709 - samples/sec: 104.65 - lr: 0.125000 2021-03-26 07:56:22,382 epoch 27 - iter 20/25 - loss 4.19213790 - samples/sec: 98.51 - lr: 0.125000 2021-03-26 07:56:23,650 epoch 27 - iter 22/25 - loss 4.14735520 - samples/sec: 101.07 - lr: 0.125000 2021-03-26 07:56:24,807 epoch 27 - iter 24/25 - loss 4.09487817 - samples/sec: 110.81 - lr: 0.125000 2021-03-26 07:56:25,255 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:56:25,256 EPOCH 27 done: loss 4.0652 - lr 0.1250000 2021-03-26 07:56:25,971 DEV : loss 5.983981132507324 - score 0.9106 2021-03-26 07:56:25,987 BAD EPOCHS (no improvement): 0 2021-03-26 07:56:35,553 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:56:36,823 epoch 28 - iter 2/25 - loss 3.55781388 - samples/sec: 101.03 - lr: 0.125000 2021-03-26 07:56:38,258 epoch 28 - iter 4/25 - loss 4.03270161 - samples/sec: 89.30 - lr: 0.125000 2021-03-26 07:56:39,398 epoch 28 - iter 6/25 - loss 4.05528371 - samples/sec: 112.44 - lr: 0.125000 2021-03-26 07:56:40,275 epoch 28 - iter 8/25 - loss 3.89411348 - samples/sec: 146.25 - lr: 0.125000 2021-03-26 07:56:41,225 epoch 28 - iter 10/25 - loss 3.93902600 - samples/sec: 134.86 - lr: 0.125000 2021-03-26 07:56:42,204 epoch 28 - iter 12/25 - loss 3.89163711 - samples/sec: 130.96 - lr: 0.125000 2021-03-26 07:56:43,268 epoch 28 - iter 14/25 - loss 3.83746786 - samples/sec: 120.52 - lr: 0.125000 2021-03-26 07:56:44,240 epoch 28 - iter 16/25 - loss 3.78483669 - samples/sec: 131.87 - lr: 0.125000 2021-03-26 07:56:45,195 epoch 28 - iter 18/25 - loss 3.86538806 - samples/sec: 134.37 - lr: 0.125000 2021-03-26 07:56:46,164 epoch 28 - iter 20/25 - loss 3.83518734 - samples/sec: 132.42 - lr: 0.125000 2021-03-26 07:56:47,233 epoch 28 - iter 22/25 - loss 3.83221832 - samples/sec: 119.91 - lr: 0.125000 2021-03-26 07:56:48,466 epoch 28 - iter 24/25 - loss 3.86373717 - samples/sec: 103.89 - lr: 0.125000 2021-03-26 07:56:48,977 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:56:48,977 EPOCH 28 done: loss 3.8487 - lr 0.1250000 2021-03-26 07:56:49,695 DEV : loss 6.044693946838379 - score 0.9064 2021-03-26 07:56:49,717 BAD EPOCHS (no improvement): 1 2021-03-26 07:56:49,717 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:56:50,936 epoch 29 - iter 2/25 - loss 2.99994659 - samples/sec: 105.17 - lr: 0.125000 2021-03-26 07:56:52,113 epoch 29 - iter 4/25 - loss 3.65278989 - samples/sec: 108.94 - lr: 0.125000 2021-03-26 07:56:53,079 epoch 29 - iter 6/25 - loss 3.79836094 - samples/sec: 132.63 - lr: 0.125000 2021-03-26 07:56:54,131 epoch 29 - iter 8/25 - loss 3.85753009 - samples/sec: 121.81 - lr: 0.125000 2021-03-26 07:56:55,074 epoch 29 - iter 10/25 - loss 3.81739969 - samples/sec: 135.93 - lr: 0.125000 2021-03-26 07:56:56,008 epoch 29 - iter 12/25 - loss 3.90435483 - samples/sec: 137.43 - lr: 0.125000 2021-03-26 07:56:57,121 epoch 29 - iter 14/25 - loss 3.80697027 - samples/sec: 115.14 - lr: 0.125000 2021-03-26 07:56:58,281 epoch 29 - iter 16/25 - loss 3.77527751 - samples/sec: 110.49 - lr: 0.125000 2021-03-26 07:56:59,406 epoch 29 - iter 18/25 - loss 3.78040032 - samples/sec: 113.91 - lr: 0.125000 2021-03-26 07:57:00,543 epoch 29 - iter 20/25 - loss 3.73319964 - samples/sec: 112.71 - lr: 0.125000 2021-03-26 07:57:01,681 epoch 29 - iter 22/25 - loss 3.72866162 - samples/sec: 112.74 - lr: 0.125000 2021-03-26 07:57:02,612 epoch 29 - iter 24/25 - loss 3.70865181 - samples/sec: 137.60 - lr: 0.125000 2021-03-26 07:57:03,003 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:57:03,004 EPOCH 29 done: loss 3.7015 - lr 0.1250000 2021-03-26 07:57:03,779 DEV : loss 6.004291534423828 - score 0.9089 2021-03-26 07:57:03,799 BAD EPOCHS (no improvement): 2 2021-03-26 07:57:03,800 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:57:04,894 epoch 30 - iter 2/25 - loss 3.83538103 - samples/sec: 117.24 - lr: 0.125000 2021-03-26 07:57:05,923 epoch 30 - iter 4/25 - loss 3.71162516 - samples/sec: 124.56 - lr: 0.125000 2021-03-26 07:57:06,884 epoch 30 - iter 6/25 - loss 3.87396975 - samples/sec: 133.32 - lr: 0.125000 2021-03-26 07:57:07,869 epoch 30 - iter 8/25 - loss 3.91214129 - samples/sec: 130.05 - lr: 0.125000 2021-03-26 07:57:08,862 epoch 30 - iter 10/25 - loss 3.95587745 - samples/sec: 129.22 - lr: 0.125000 2021-03-26 07:57:09,862 epoch 30 - iter 12/25 - loss 3.95001904 - samples/sec: 128.22 - lr: 0.125000 2021-03-26 07:57:10,820 epoch 30 - iter 14/25 - loss 3.84484034 - samples/sec: 133.84 - lr: 0.125000 2021-03-26 07:57:11,801 epoch 30 - iter 16/25 - loss 3.82006696 - samples/sec: 130.73 - lr: 0.125000 2021-03-26 07:57:12,815 epoch 30 - iter 18/25 - loss 3.82373320 - samples/sec: 126.50 - lr: 0.125000 2021-03-26 07:57:13,873 epoch 30 - iter 20/25 - loss 3.71675273 - samples/sec: 121.20 - lr: 0.125000 2021-03-26 07:57:14,950 epoch 30 - iter 22/25 - loss 3.74051334 - samples/sec: 119.05 - lr: 0.125000 2021-03-26 07:57:15,987 epoch 30 - iter 24/25 - loss 3.75257254 - samples/sec: 123.69 - lr: 0.125000 2021-03-26 07:57:16,422 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:57:16,423 EPOCH 30 done: loss 3.7078 - lr 0.1250000 2021-03-26 07:57:17,145 DEV : loss 5.97253942489624 - score 0.9102 2021-03-26 07:57:17,168 BAD EPOCHS (no improvement): 3 2021-03-26 07:57:17,168 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:57:18,306 epoch 31 - iter 2/25 - loss 3.89688802 - samples/sec: 112.68 - lr: 0.125000 2021-03-26 07:57:19,187 epoch 31 - iter 4/25 - loss 3.70891809 - samples/sec: 145.60 - lr: 0.125000 2021-03-26 07:57:20,129 epoch 31 - iter 6/25 - loss 3.78291845 - samples/sec: 136.03 - lr: 0.125000 2021-03-26 07:57:21,123 epoch 31 - iter 8/25 - loss 3.78016075 - samples/sec: 129.10 - lr: 0.125000 2021-03-26 07:57:22,101 epoch 31 - iter 10/25 - loss 3.74238210 - samples/sec: 131.21 - lr: 0.125000 2021-03-26 07:57:23,084 epoch 31 - iter 12/25 - loss 3.75834699 - samples/sec: 130.42 - lr: 0.125000 2021-03-26 07:57:24,184 epoch 31 - iter 14/25 - loss 3.67023890 - samples/sec: 116.44 - lr: 0.125000 2021-03-26 07:57:25,137 epoch 31 - iter 16/25 - loss 3.59937003 - samples/sec: 134.60 - lr: 0.125000 2021-03-26 07:57:26,051 epoch 31 - iter 18/25 - loss 3.61951150 - samples/sec: 140.16 - lr: 0.125000 2021-03-26 07:57:27,041 epoch 31 - iter 20/25 - loss 3.66483914 - samples/sec: 129.52 - lr: 0.125000 2021-03-26 07:57:28,030 epoch 31 - iter 22/25 - loss 3.61880018 - samples/sec: 129.63 - lr: 0.125000 2021-03-26 07:57:28,950 epoch 31 - iter 24/25 - loss 3.60269628 - samples/sec: 139.32 - lr: 0.125000 2021-03-26 07:57:29,419 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:57:29,419 EPOCH 31 done: loss 3.6264 - lr 0.1250000 2021-03-26 07:57:30,161 DEV : loss 5.973457336425781 - score 0.9139 2021-03-26 07:57:30,182 BAD EPOCHS (no improvement): 0 2021-03-26 07:57:39,545 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:57:40,610 epoch 32 - iter 2/25 - loss 3.11975098 - samples/sec: 120.45 - lr: 0.125000 2021-03-26 07:57:41,581 epoch 32 - iter 4/25 - loss 3.29605865 - samples/sec: 132.13 - lr: 0.125000 2021-03-26 07:57:42,541 epoch 32 - iter 6/25 - loss 3.50240274 - samples/sec: 133.40 - lr: 0.125000 2021-03-26 07:57:43,498 epoch 32 - iter 8/25 - loss 3.41950604 - samples/sec: 134.00 - lr: 0.125000 2021-03-26 07:57:44,451 epoch 32 - iter 10/25 - loss 3.45227201 - samples/sec: 134.48 - lr: 0.125000 2021-03-26 07:57:45,596 epoch 32 - iter 12/25 - loss 3.46193991 - samples/sec: 111.94 - lr: 0.125000 2021-03-26 07:57:46,620 epoch 32 - iter 14/25 - loss 3.43003614 - samples/sec: 125.13 - lr: 0.125000 2021-03-26 07:57:47,548 epoch 32 - iter 16/25 - loss 3.40545425 - samples/sec: 138.31 - lr: 0.125000 2021-03-26 07:57:48,546 epoch 32 - iter 18/25 - loss 3.41395897 - samples/sec: 128.40 - lr: 0.125000 2021-03-26 07:57:49,505 epoch 32 - iter 20/25 - loss 3.36117598 - samples/sec: 133.62 - lr: 0.125000 2021-03-26 07:57:50,544 epoch 32 - iter 22/25 - loss 3.38892034 - samples/sec: 123.47 - lr: 0.125000 2021-03-26 07:57:51,502 epoch 32 - iter 24/25 - loss 3.40688891 - samples/sec: 133.87 - lr: 0.125000 2021-03-26 07:57:51,961 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:57:51,962 EPOCH 32 done: loss 3.4289 - lr 0.1250000 2021-03-26 07:57:52,668 DEV : loss 6.120488166809082 - score 0.9131 2021-03-26 07:57:52,686 BAD EPOCHS (no improvement): 1 2021-03-26 07:57:52,686 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:57:53,583 epoch 33 - iter 2/25 - loss 3.59765363 - samples/sec: 142.92 - lr: 0.125000 2021-03-26 07:57:54,539 epoch 33 - iter 4/25 - loss 3.69549191 - samples/sec: 133.96 - lr: 0.125000 2021-03-26 07:57:55,547 epoch 33 - iter 6/25 - loss 3.97329044 - samples/sec: 127.28 - lr: 0.125000 2021-03-26 07:57:56,547 epoch 33 - iter 8/25 - loss 3.83983880 - samples/sec: 128.23 - lr: 0.125000 2021-03-26 07:57:57,617 epoch 33 - iter 10/25 - loss 3.84227064 - samples/sec: 119.77 - lr: 0.125000 2021-03-26 07:57:58,659 epoch 33 - iter 12/25 - loss 3.78336207 - samples/sec: 123.00 - lr: 0.125000 2021-03-26 07:57:59,608 epoch 33 - iter 14/25 - loss 3.68971201 - samples/sec: 135.00 - lr: 0.125000 2021-03-26 07:58:00,559 epoch 33 - iter 16/25 - loss 3.55988289 - samples/sec: 134.91 - lr: 0.125000 2021-03-26 07:58:01,609 epoch 33 - iter 18/25 - loss 3.52473515 - samples/sec: 122.04 - lr: 0.125000 2021-03-26 07:58:02,540 epoch 33 - iter 20/25 - loss 3.49631225 - samples/sec: 137.71 - lr: 0.125000 2021-03-26 07:58:03,496 epoch 33 - iter 22/25 - loss 3.54496320 - samples/sec: 134.09 - lr: 0.125000 2021-03-26 07:58:04,537 epoch 33 - iter 24/25 - loss 3.56121511 - samples/sec: 123.13 - lr: 0.125000 2021-03-26 07:58:04,910 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:58:04,911 EPOCH 33 done: loss 3.5053 - lr 0.1250000 2021-03-26 07:58:05,626 DEV : loss 6.25508451461792 - score 0.9093 2021-03-26 07:58:05,649 BAD EPOCHS (no improvement): 2 2021-03-26 07:58:05,650 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:58:06,672 epoch 34 - iter 2/25 - loss 3.48426843 - samples/sec: 125.49 - lr: 0.125000 2021-03-26 07:58:07,726 epoch 34 - iter 4/25 - loss 3.28659660 - samples/sec: 121.57 - lr: 0.125000 2021-03-26 07:58:08,750 epoch 34 - iter 6/25 - loss 3.18423168 - samples/sec: 125.25 - lr: 0.125000 2021-03-26 07:58:09,805 epoch 34 - iter 8/25 - loss 3.21228367 - samples/sec: 121.47 - lr: 0.125000 2021-03-26 07:58:10,858 epoch 34 - iter 10/25 - loss 3.12810292 - samples/sec: 121.67 - lr: 0.125000 2021-03-26 07:58:11,879 epoch 34 - iter 12/25 - loss 3.27248840 - samples/sec: 125.73 - lr: 0.125000 2021-03-26 07:58:13,291 epoch 34 - iter 14/25 - loss 3.33075367 - samples/sec: 90.74 - lr: 0.125000 2021-03-26 07:58:14,675 epoch 34 - iter 16/25 - loss 3.39736582 - samples/sec: 92.58 - lr: 0.125000 2021-03-26 07:58:16,055 epoch 34 - iter 18/25 - loss 3.37778645 - samples/sec: 92.87 - lr: 0.125000 2021-03-26 07:58:17,327 epoch 34 - iter 20/25 - loss 3.35320867 - samples/sec: 100.73 - lr: 0.125000 2021-03-26 07:58:18,337 epoch 34 - iter 22/25 - loss 3.38617272 - samples/sec: 126.90 - lr: 0.125000 2021-03-26 07:58:19,281 epoch 34 - iter 24/25 - loss 3.36547829 - samples/sec: 135.78 - lr: 0.125000 2021-03-26 07:58:19,735 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:58:19,735 EPOCH 34 done: loss 3.3455 - lr 0.1250000 2021-03-26 07:58:20,451 DEV : loss 6.16842794418335 - score 0.9148 2021-03-26 07:58:20,474 BAD EPOCHS (no improvement): 0 2021-03-26 07:58:29,897 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:58:30,954 epoch 35 - iter 2/25 - loss 2.87888587 - samples/sec: 121.35 - lr: 0.125000 2021-03-26 07:58:31,884 epoch 35 - iter 4/25 - loss 3.05180985 - samples/sec: 137.88 - lr: 0.125000 2021-03-26 07:58:32,911 epoch 35 - iter 6/25 - loss 2.96273021 - samples/sec: 124.74 - lr: 0.125000 2021-03-26 07:58:33,942 epoch 35 - iter 8/25 - loss 3.08268613 - samples/sec: 124.41 - lr: 0.125000 2021-03-26 07:58:34,926 epoch 35 - iter 10/25 - loss 3.14363208 - samples/sec: 130.30 - lr: 0.125000 2021-03-26 07:58:35,927 epoch 35 - iter 12/25 - loss 3.29011472 - samples/sec: 127.96 - lr: 0.125000 2021-03-26 07:58:36,831 epoch 35 - iter 14/25 - loss 3.34185306 - samples/sec: 141.86 - lr: 0.125000 2021-03-26 07:58:37,949 epoch 35 - iter 16/25 - loss 3.37906143 - samples/sec: 114.59 - lr: 0.125000 2021-03-26 07:58:38,958 epoch 35 - iter 18/25 - loss 3.36027298 - samples/sec: 127.15 - lr: 0.125000 2021-03-26 07:58:39,892 epoch 35 - iter 20/25 - loss 3.36290877 - samples/sec: 137.43 - lr: 0.125000 2021-03-26 07:58:40,836 epoch 35 - iter 22/25 - loss 3.39653666 - samples/sec: 135.74 - lr: 0.125000 2021-03-26 07:58:41,828 epoch 35 - iter 24/25 - loss 3.39750968 - samples/sec: 129.22 - lr: 0.125000 2021-03-26 07:58:42,263 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:58:42,264 EPOCH 35 done: loss 3.3719 - lr 0.1250000 2021-03-26 07:58:42,981 DEV : loss 6.130885124206543 - score 0.9135 2021-03-26 07:58:42,997 BAD EPOCHS (no improvement): 1 2021-03-26 07:58:42,997 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:58:44,006 epoch 36 - iter 2/25 - loss 3.68344629 - samples/sec: 127.05 - lr: 0.125000 2021-03-26 07:58:45,076 epoch 36 - iter 4/25 - loss 3.82729638 - samples/sec: 119.80 - lr: 0.125000 2021-03-26 07:58:45,950 epoch 36 - iter 6/25 - loss 3.52623641 - samples/sec: 146.70 - lr: 0.125000 2021-03-26 07:58:46,895 epoch 36 - iter 8/25 - loss 3.51207724 - samples/sec: 135.66 - lr: 0.125000 2021-03-26 07:58:47,779 epoch 36 - iter 10/25 - loss 3.54672844 - samples/sec: 145.14 - lr: 0.125000 2021-03-26 07:58:48,749 epoch 36 - iter 12/25 - loss 3.48546157 - samples/sec: 132.10 - lr: 0.125000 2021-03-26 07:58:49,619 epoch 36 - iter 14/25 - loss 3.43780621 - samples/sec: 147.49 - lr: 0.125000 2021-03-26 07:58:50,709 epoch 36 - iter 16/25 - loss 3.35306793 - samples/sec: 117.58 - lr: 0.125000 2021-03-26 07:58:51,670 epoch 36 - iter 18/25 - loss 3.29722373 - samples/sec: 133.35 - lr: 0.125000 2021-03-26 07:58:52,684 epoch 36 - iter 20/25 - loss 3.34835799 - samples/sec: 126.47 - lr: 0.125000 2021-03-26 07:58:53,682 epoch 36 - iter 22/25 - loss 3.32949518 - samples/sec: 128.42 - lr: 0.125000 2021-03-26 07:58:54,728 epoch 36 - iter 24/25 - loss 3.32508539 - samples/sec: 122.54 - lr: 0.125000 2021-03-26 07:58:55,161 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:58:55,162 EPOCH 36 done: loss 3.3480 - lr 0.1250000 2021-03-26 07:58:55,881 DEV : loss 6.107944488525391 - score 0.9118 2021-03-26 07:58:55,897 BAD EPOCHS (no improvement): 2 2021-03-26 07:58:55,898 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:58:56,914 epoch 37 - iter 2/25 - loss 3.42133987 - samples/sec: 126.21 - lr: 0.125000 2021-03-26 07:58:57,950 epoch 37 - iter 4/25 - loss 3.14635503 - samples/sec: 123.69 - lr: 0.125000 2021-03-26 07:58:58,940 epoch 37 - iter 6/25 - loss 3.18325603 - samples/sec: 129.51 - lr: 0.125000 2021-03-26 07:58:59,906 epoch 37 - iter 8/25 - loss 3.35971868 - samples/sec: 132.78 - lr: 0.125000 2021-03-26 07:59:00,893 epoch 37 - iter 10/25 - loss 3.33887649 - samples/sec: 129.78 - lr: 0.125000 2021-03-26 07:59:01,841 epoch 37 - iter 12/25 - loss 3.28747088 - samples/sec: 135.28 - lr: 0.125000 2021-03-26 07:59:02,774 epoch 37 - iter 14/25 - loss 3.31001389 - samples/sec: 137.48 - lr: 0.125000 2021-03-26 07:59:03,825 epoch 37 - iter 16/25 - loss 3.27298586 - samples/sec: 122.00 - lr: 0.125000 2021-03-26 07:59:04,997 epoch 37 - iter 18/25 - loss 3.28302748 - samples/sec: 109.26 - lr: 0.125000 2021-03-26 07:59:05,943 epoch 37 - iter 20/25 - loss 3.32614937 - samples/sec: 135.59 - lr: 0.125000 2021-03-26 07:59:06,910 epoch 37 - iter 22/25 - loss 3.35880077 - samples/sec: 132.50 - lr: 0.125000 2021-03-26 07:59:07,803 epoch 37 - iter 24/25 - loss 3.34755198 - samples/sec: 143.58 - lr: 0.125000 2021-03-26 07:59:08,232 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:59:08,232 EPOCH 37 done: loss 3.3532 - lr 0.1250000 2021-03-26 07:59:08,941 DEV : loss 5.940009117126465 - score 0.916 2021-03-26 07:59:08,964 BAD EPOCHS (no improvement): 0 2021-03-26 07:59:18,132 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:59:18,997 epoch 38 - iter 2/25 - loss 3.08662271 - samples/sec: 148.43 - lr: 0.125000 2021-03-26 07:59:19,947 epoch 38 - iter 4/25 - loss 3.46704584 - samples/sec: 135.02 - lr: 0.125000 2021-03-26 07:59:20,870 epoch 38 - iter 6/25 - loss 3.45128655 - samples/sec: 138.92 - lr: 0.125000 2021-03-26 07:59:21,902 epoch 38 - iter 8/25 - loss 3.49598333 - samples/sec: 124.21 - lr: 0.125000 2021-03-26 07:59:22,925 epoch 38 - iter 10/25 - loss 3.31913579 - samples/sec: 125.35 - lr: 0.125000 2021-03-26 07:59:24,020 epoch 38 - iter 12/25 - loss 3.36305465 - samples/sec: 117.11 - lr: 0.125000 2021-03-26 07:59:25,021 epoch 38 - iter 14/25 - loss 3.27468027 - samples/sec: 128.02 - lr: 0.125000 2021-03-26 07:59:25,956 epoch 38 - iter 16/25 - loss 3.22047535 - samples/sec: 137.09 - lr: 0.125000 2021-03-26 07:59:27,007 epoch 38 - iter 18/25 - loss 3.27085086 - samples/sec: 122.08 - lr: 0.125000 2021-03-26 07:59:28,007 epoch 38 - iter 20/25 - loss 3.20065688 - samples/sec: 128.18 - lr: 0.125000 2021-03-26 07:59:29,013 epoch 38 - iter 22/25 - loss 3.20793955 - samples/sec: 127.45 - lr: 0.125000 2021-03-26 07:59:30,112 epoch 38 - iter 24/25 - loss 3.22620339 - samples/sec: 116.61 - lr: 0.125000 2021-03-26 07:59:30,574 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:59:30,574 EPOCH 38 done: loss 3.2033 - lr 0.1250000 2021-03-26 07:59:31,315 DEV : loss 6.086957931518555 - score 0.9135 2021-03-26 07:59:31,341 BAD EPOCHS (no improvement): 1 2021-03-26 07:59:31,342 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:59:32,431 epoch 39 - iter 2/25 - loss 3.51137233 - samples/sec: 117.79 - lr: 0.125000 2021-03-26 07:59:33,483 epoch 39 - iter 4/25 - loss 3.32509232 - samples/sec: 121.80 - lr: 0.125000 2021-03-26 07:59:34,524 epoch 39 - iter 6/25 - loss 3.28555572 - samples/sec: 123.17 - lr: 0.125000 2021-03-26 07:59:35,573 epoch 39 - iter 8/25 - loss 3.28893805 - samples/sec: 122.32 - lr: 0.125000 2021-03-26 07:59:36,536 epoch 39 - iter 10/25 - loss 3.32282088 - samples/sec: 133.08 - lr: 0.125000 2021-03-26 07:59:37,530 epoch 39 - iter 12/25 - loss 3.31090313 - samples/sec: 128.97 - lr: 0.125000 2021-03-26 07:59:38,529 epoch 39 - iter 14/25 - loss 3.28282419 - samples/sec: 128.38 - lr: 0.125000 2021-03-26 07:59:39,567 epoch 39 - iter 16/25 - loss 3.28705211 - samples/sec: 123.40 - lr: 0.125000 2021-03-26 07:59:40,584 epoch 39 - iter 18/25 - loss 3.35787143 - samples/sec: 126.16 - lr: 0.125000 2021-03-26 07:59:41,554 epoch 39 - iter 20/25 - loss 3.34916631 - samples/sec: 132.06 - lr: 0.125000 2021-03-26 07:59:42,576 epoch 39 - iter 22/25 - loss 3.33695148 - samples/sec: 125.43 - lr: 0.125000 2021-03-26 07:59:43,521 epoch 39 - iter 24/25 - loss 3.33044088 - samples/sec: 135.67 - lr: 0.125000 2021-03-26 07:59:44,035 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:59:44,035 EPOCH 39 done: loss 3.3141 - lr 0.1250000 2021-03-26 07:59:44,769 DEV : loss 6.12675666809082 - score 0.9118 2021-03-26 07:59:44,788 BAD EPOCHS (no improvement): 2 2021-03-26 07:59:44,789 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:59:45,848 epoch 40 - iter 2/25 - loss 3.39053595 - samples/sec: 121.01 - lr: 0.125000 2021-03-26 07:59:46,885 epoch 40 - iter 4/25 - loss 3.01093823 - samples/sec: 123.67 - lr: 0.125000 2021-03-26 07:59:47,829 epoch 40 - iter 6/25 - loss 3.07951188 - samples/sec: 135.76 - lr: 0.125000 2021-03-26 07:59:48,870 epoch 40 - iter 8/25 - loss 2.95052671 - samples/sec: 123.12 - lr: 0.125000 2021-03-26 07:59:49,803 epoch 40 - iter 10/25 - loss 2.97352629 - samples/sec: 137.51 - lr: 0.125000 2021-03-26 07:59:50,717 epoch 40 - iter 12/25 - loss 3.00901592 - samples/sec: 140.26 - lr: 0.125000 2021-03-26 07:59:51,599 epoch 40 - iter 14/25 - loss 3.02175573 - samples/sec: 145.35 - lr: 0.125000 2021-03-26 07:59:52,584 epoch 40 - iter 16/25 - loss 3.04483742 - samples/sec: 130.12 - lr: 0.125000 2021-03-26 07:59:53,524 epoch 40 - iter 18/25 - loss 3.16045962 - samples/sec: 136.38 - lr: 0.125000 2021-03-26 07:59:54,635 epoch 40 - iter 20/25 - loss 3.21983838 - samples/sec: 115.39 - lr: 0.125000 2021-03-26 07:59:55,686 epoch 40 - iter 22/25 - loss 3.30566324 - samples/sec: 122.03 - lr: 0.125000 2021-03-26 07:59:56,596 epoch 40 - iter 24/25 - loss 3.26253485 - samples/sec: 140.78 - lr: 0.125000 2021-03-26 07:59:56,945 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:59:56,945 EPOCH 40 done: loss 3.2575 - lr 0.1250000 2021-03-26 07:59:57,658 DEV : loss 6.137569427490234 - score 0.9135 2021-03-26 07:59:57,680 BAD EPOCHS (no improvement): 3 2021-03-26 07:59:57,681 ---------------------------------------------------------------------------------------------------- 2021-03-26 07:59:58,718 epoch 41 - iter 2/25 - loss 3.11338151 - samples/sec: 123.60 - lr: 0.125000 2021-03-26 07:59:59,662 epoch 41 - iter 4/25 - loss 2.86183566 - samples/sec: 135.82 - lr: 0.125000 2021-03-26 08:00:00,674 epoch 41 - iter 6/25 - loss 3.09293191 - samples/sec: 126.66 - lr: 0.125000 2021-03-26 08:00:01,587 epoch 41 - iter 8/25 - loss 3.08784479 - samples/sec: 140.40 - lr: 0.125000 2021-03-26 08:00:02,515 epoch 41 - iter 10/25 - loss 3.09713953 - samples/sec: 138.11 - lr: 0.125000 2021-03-26 08:00:03,456 epoch 41 - iter 12/25 - loss 3.03154357 - samples/sec: 136.29 - lr: 0.125000 2021-03-26 08:00:04,461 epoch 41 - iter 14/25 - loss 3.16953162 - samples/sec: 127.60 - lr: 0.125000 2021-03-26 08:00:05,458 epoch 41 - iter 16/25 - loss 3.24205971 - samples/sec: 128.47 - lr: 0.125000 2021-03-26 08:00:06,382 epoch 41 - iter 18/25 - loss 3.22677610 - samples/sec: 138.77 - lr: 0.125000 2021-03-26 08:00:07,273 epoch 41 - iter 20/25 - loss 3.19914433 - samples/sec: 143.88 - lr: 0.125000 2021-03-26 08:00:08,260 epoch 41 - iter 22/25 - loss 3.25783063 - samples/sec: 129.85 - lr: 0.125000 2021-03-26 08:00:09,198 epoch 41 - iter 24/25 - loss 3.25372606 - samples/sec: 136.70 - lr: 0.125000 2021-03-26 08:00:09,540 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:00:09,540 EPOCH 41 done: loss 3.2544 - lr 0.1250000 2021-03-26 08:00:10,220 DEV : loss 6.269647598266602 - score 0.9102 2021-03-26 08:00:10,243 BAD EPOCHS (no improvement): 4 2021-03-26 08:00:10,243 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:00:11,223 epoch 42 - iter 2/25 - loss 3.20560980 - samples/sec: 130.83 - lr: 0.062500 2021-03-26 08:00:12,201 epoch 42 - iter 4/25 - loss 3.05283481 - samples/sec: 131.28 - lr: 0.062500 2021-03-26 08:00:13,126 epoch 42 - iter 6/25 - loss 3.11910899 - samples/sec: 138.65 - lr: 0.062500 2021-03-26 08:00:14,062 epoch 42 - iter 8/25 - loss 3.23691389 - samples/sec: 136.90 - lr: 0.062500 2021-03-26 08:00:14,985 epoch 42 - iter 10/25 - loss 3.31064866 - samples/sec: 138.91 - lr: 0.062500 2021-03-26 08:00:15,963 epoch 42 - iter 12/25 - loss 3.20677197 - samples/sec: 131.07 - lr: 0.062500 2021-03-26 08:00:17,113 epoch 42 - iter 14/25 - loss 3.16082122 - samples/sec: 111.48 - lr: 0.062500 2021-03-26 08:00:18,181 epoch 42 - iter 16/25 - loss 3.12908374 - samples/sec: 119.98 - lr: 0.062500 2021-03-26 08:00:19,111 epoch 42 - iter 18/25 - loss 3.12907228 - samples/sec: 138.01 - lr: 0.062500 2021-03-26 08:00:20,110 epoch 42 - iter 20/25 - loss 3.13966805 - samples/sec: 128.26 - lr: 0.062500 2021-03-26 08:00:21,155 epoch 42 - iter 22/25 - loss 3.12461065 - samples/sec: 122.65 - lr: 0.062500 2021-03-26 08:00:22,126 epoch 42 - iter 24/25 - loss 3.09273655 - samples/sec: 132.07 - lr: 0.062500 2021-03-26 08:00:22,535 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:00:22,536 EPOCH 42 done: loss 3.1158 - lr 0.0625000 2021-03-26 08:00:23,241 DEV : loss 6.14277982711792 - score 0.9118 2021-03-26 08:00:23,264 BAD EPOCHS (no improvement): 1 2021-03-26 08:00:23,265 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:00:24,222 epoch 43 - iter 2/25 - loss 3.33047795 - samples/sec: 134.00 - lr: 0.062500 2021-03-26 08:00:25,191 epoch 43 - iter 4/25 - loss 3.21108902 - samples/sec: 132.35 - lr: 0.062500 2021-03-26 08:00:26,208 epoch 43 - iter 6/25 - loss 3.19124313 - samples/sec: 126.03 - lr: 0.062500 2021-03-26 08:00:27,152 epoch 43 - iter 8/25 - loss 3.31731814 - samples/sec: 135.76 - lr: 0.062500 2021-03-26 08:00:28,168 epoch 43 - iter 10/25 - loss 3.35041995 - samples/sec: 126.20 - lr: 0.062500 2021-03-26 08:00:29,203 epoch 43 - iter 12/25 - loss 3.29511603 - samples/sec: 123.81 - lr: 0.062500 2021-03-26 08:00:30,155 epoch 43 - iter 14/25 - loss 3.16224260 - samples/sec: 134.72 - lr: 0.062500 2021-03-26 08:00:31,046 epoch 43 - iter 16/25 - loss 3.12122820 - samples/sec: 143.92 - lr: 0.062500 2021-03-26 08:00:32,078 epoch 43 - iter 18/25 - loss 3.08958398 - samples/sec: 124.21 - lr: 0.062500 2021-03-26 08:00:33,229 epoch 43 - iter 20/25 - loss 3.06157826 - samples/sec: 111.36 - lr: 0.062500 2021-03-26 08:00:34,129 epoch 43 - iter 22/25 - loss 3.06261855 - samples/sec: 142.44 - lr: 0.062500 2021-03-26 08:00:35,160 epoch 43 - iter 24/25 - loss 3.03469709 - samples/sec: 124.45 - lr: 0.062500 2021-03-26 08:00:35,526 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:00:35,526 EPOCH 43 done: loss 3.0090 - lr 0.0625000 2021-03-26 08:00:36,251 DEV : loss 6.123751640319824 - score 0.9131 2021-03-26 08:00:36,275 BAD EPOCHS (no improvement): 2 2021-03-26 08:00:36,276 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:00:37,279 epoch 44 - iter 2/25 - loss 2.47428727 - samples/sec: 127.77 - lr: 0.062500 2021-03-26 08:00:38,298 epoch 44 - iter 4/25 - loss 2.93294007 - samples/sec: 125.85 - lr: 0.062500 2021-03-26 08:00:39,301 epoch 44 - iter 6/25 - loss 2.94476159 - samples/sec: 127.78 - lr: 0.062500 2021-03-26 08:00:40,377 epoch 44 - iter 8/25 - loss 2.87063783 - samples/sec: 119.07 - lr: 0.062500 2021-03-26 08:00:41,428 epoch 44 - iter 10/25 - loss 2.81950252 - samples/sec: 122.07 - lr: 0.062500 2021-03-26 08:00:42,400 epoch 44 - iter 12/25 - loss 2.83043108 - samples/sec: 131.87 - lr: 0.062500 2021-03-26 08:00:43,405 epoch 44 - iter 14/25 - loss 2.80339539 - samples/sec: 127.63 - lr: 0.062500 2021-03-26 08:00:44,385 epoch 44 - iter 16/25 - loss 2.83290577 - samples/sec: 130.85 - lr: 0.062500 2021-03-26 08:00:45,276 epoch 44 - iter 18/25 - loss 2.81380676 - samples/sec: 143.81 - lr: 0.062500 2021-03-26 08:00:46,270 epoch 44 - iter 20/25 - loss 2.86733302 - samples/sec: 128.89 - lr: 0.062500 2021-03-26 08:00:47,213 epoch 44 - iter 22/25 - loss 2.89061084 - samples/sec: 136.20 - lr: 0.062500 2021-03-26 08:00:48,134 epoch 44 - iter 24/25 - loss 2.88428577 - samples/sec: 139.19 - lr: 0.062500 2021-03-26 08:00:48,531 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:00:48,531 EPOCH 44 done: loss 2.9018 - lr 0.0625000 2021-03-26 08:00:49,238 DEV : loss 6.177193641662598 - score 0.9131 2021-03-26 08:00:49,261 BAD EPOCHS (no improvement): 3 2021-03-26 08:00:49,262 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:00:50,201 epoch 45 - iter 2/25 - loss 3.11781955 - samples/sec: 136.46 - lr: 0.062500 2021-03-26 08:00:51,263 epoch 45 - iter 4/25 - loss 3.27651954 - samples/sec: 120.78 - lr: 0.062500 2021-03-26 08:00:52,184 epoch 45 - iter 6/25 - loss 3.09253220 - samples/sec: 139.15 - lr: 0.062500 2021-03-26 08:00:53,203 epoch 45 - iter 8/25 - loss 3.09594426 - samples/sec: 125.75 - lr: 0.062500 2021-03-26 08:00:54,148 epoch 45 - iter 10/25 - loss 3.06181715 - samples/sec: 135.70 - lr: 0.062500 2021-03-26 08:00:55,128 epoch 45 - iter 12/25 - loss 3.08044499 - samples/sec: 130.72 - lr: 0.062500 2021-03-26 08:00:56,165 epoch 45 - iter 14/25 - loss 3.07888399 - samples/sec: 123.64 - lr: 0.062500 2021-03-26 08:00:57,333 epoch 45 - iter 16/25 - loss 3.10207799 - samples/sec: 109.79 - lr: 0.062500 2021-03-26 08:00:58,353 epoch 45 - iter 18/25 - loss 3.08716683 - samples/sec: 125.76 - lr: 0.062500 2021-03-26 08:00:59,401 epoch 45 - iter 20/25 - loss 3.07444774 - samples/sec: 122.40 - lr: 0.062500 2021-03-26 08:01:00,443 epoch 45 - iter 22/25 - loss 3.08478190 - samples/sec: 122.96 - lr: 0.062500 2021-03-26 08:01:01,516 epoch 45 - iter 24/25 - loss 3.18055204 - samples/sec: 119.48 - lr: 0.062500 2021-03-26 08:01:01,929 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:01:01,930 EPOCH 45 done: loss 3.1790 - lr 0.0625000 2021-03-26 08:01:02,641 DEV : loss 6.166326522827148 - score 0.9135 2021-03-26 08:01:02,664 BAD EPOCHS (no improvement): 4 2021-03-26 08:01:02,664 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:01:03,613 epoch 46 - iter 2/25 - loss 2.76693845 - samples/sec: 135.28 - lr: 0.031250 2021-03-26 08:01:04,550 epoch 46 - iter 4/25 - loss 2.73128182 - samples/sec: 136.83 - lr: 0.031250 2021-03-26 08:01:05,521 epoch 46 - iter 6/25 - loss 2.70524808 - samples/sec: 131.96 - lr: 0.031250 2021-03-26 08:01:06,528 epoch 46 - iter 8/25 - loss 2.72330138 - samples/sec: 127.29 - lr: 0.031250 2021-03-26 08:01:07,538 epoch 46 - iter 10/25 - loss 2.83866251 - samples/sec: 126.93 - lr: 0.031250 2021-03-26 08:01:08,607 epoch 46 - iter 12/25 - loss 2.86050973 - samples/sec: 119.98 - lr: 0.031250 2021-03-26 08:01:09,582 epoch 46 - iter 14/25 - loss 2.90982878 - samples/sec: 131.45 - lr: 0.031250 2021-03-26 08:01:10,611 epoch 46 - iter 16/25 - loss 2.95056996 - samples/sec: 124.60 - lr: 0.031250 2021-03-26 08:01:11,612 epoch 46 - iter 18/25 - loss 2.92731520 - samples/sec: 128.01 - lr: 0.031250 2021-03-26 08:01:12,586 epoch 46 - iter 20/25 - loss 2.92244887 - samples/sec: 131.69 - lr: 0.031250 2021-03-26 08:01:13,513 epoch 46 - iter 22/25 - loss 2.93590753 - samples/sec: 138.27 - lr: 0.031250 2021-03-26 08:01:14,560 epoch 46 - iter 24/25 - loss 2.95312613 - samples/sec: 122.46 - lr: 0.031250 2021-03-26 08:01:14,950 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:01:14,951 EPOCH 46 done: loss 2.9404 - lr 0.0312500 2021-03-26 08:01:15,671 DEV : loss 6.119317054748535 - score 0.9156 2021-03-26 08:01:15,687 BAD EPOCHS (no improvement): 1 2021-03-26 08:01:15,687 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:01:16,674 epoch 47 - iter 2/25 - loss 3.48621786 - samples/sec: 129.82 - lr: 0.031250 2021-03-26 08:01:17,701 epoch 47 - iter 4/25 - loss 3.34163475 - samples/sec: 124.98 - lr: 0.031250 2021-03-26 08:01:18,782 epoch 47 - iter 6/25 - loss 3.01573094 - samples/sec: 118.47 - lr: 0.031250 2021-03-26 08:01:19,807 epoch 47 - iter 8/25 - loss 2.88064113 - samples/sec: 125.08 - lr: 0.031250 2021-03-26 08:01:20,787 epoch 47 - iter 10/25 - loss 2.93670418 - samples/sec: 131.25 - lr: 0.031250 2021-03-26 08:01:21,677 epoch 47 - iter 12/25 - loss 2.78884999 - samples/sec: 144.15 - lr: 0.031250 2021-03-26 08:01:22,824 epoch 47 - iter 14/25 - loss 2.83959506 - samples/sec: 111.78 - lr: 0.031250 2021-03-26 08:01:23,840 epoch 47 - iter 16/25 - loss 2.85051008 - samples/sec: 126.15 - lr: 0.031250 2021-03-26 08:01:24,859 epoch 47 - iter 18/25 - loss 2.89104986 - samples/sec: 125.78 - lr: 0.031250 2021-03-26 08:01:25,912 epoch 47 - iter 20/25 - loss 2.84013094 - samples/sec: 121.77 - lr: 0.031250 2021-03-26 08:01:26,946 epoch 47 - iter 22/25 - loss 2.81927793 - samples/sec: 123.89 - lr: 0.031250 2021-03-26 08:01:27,855 epoch 47 - iter 24/25 - loss 2.78637945 - samples/sec: 141.13 - lr: 0.031250 2021-03-26 08:01:28,295 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:01:28,295 EPOCH 47 done: loss 2.8195 - lr 0.0312500 2021-03-26 08:01:29,019 DEV : loss 6.151057720184326 - score 0.9152 2021-03-26 08:01:29,042 BAD EPOCHS (no improvement): 2 2021-03-26 08:01:29,042 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:01:30,167 epoch 48 - iter 2/25 - loss 2.96697593 - samples/sec: 114.02 - lr: 0.031250 2021-03-26 08:01:31,246 epoch 48 - iter 4/25 - loss 2.93493122 - samples/sec: 118.78 - lr: 0.031250 2021-03-26 08:01:32,282 epoch 48 - iter 6/25 - loss 2.72235334 - samples/sec: 123.77 - lr: 0.031250 2021-03-26 08:01:33,287 epoch 48 - iter 8/25 - loss 2.87266427 - samples/sec: 127.50 - lr: 0.031250 2021-03-26 08:01:34,244 epoch 48 - iter 10/25 - loss 2.86259272 - samples/sec: 133.84 - lr: 0.031250 2021-03-26 08:01:35,324 epoch 48 - iter 12/25 - loss 2.92353100 - samples/sec: 118.71 - lr: 0.031250 2021-03-26 08:01:36,311 epoch 48 - iter 14/25 - loss 2.92250403 - samples/sec: 129.78 - lr: 0.031250 2021-03-26 08:01:37,175 epoch 48 - iter 16/25 - loss 2.90623675 - samples/sec: 148.52 - lr: 0.031250 2021-03-26 08:01:38,215 epoch 48 - iter 18/25 - loss 2.89371492 - samples/sec: 123.35 - lr: 0.031250 2021-03-26 08:01:39,261 epoch 48 - iter 20/25 - loss 2.87912372 - samples/sec: 122.54 - lr: 0.031250 2021-03-26 08:01:40,338 epoch 48 - iter 22/25 - loss 2.87841503 - samples/sec: 119.11 - lr: 0.031250 2021-03-26 08:01:41,359 epoch 48 - iter 24/25 - loss 2.88370716 - samples/sec: 125.56 - lr: 0.031250 2021-03-26 08:01:41,780 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:01:41,781 EPOCH 48 done: loss 2.9046 - lr 0.0312500 2021-03-26 08:01:42,496 DEV : loss 6.098921775817871 - score 0.9169 2021-03-26 08:01:42,519 BAD EPOCHS (no improvement): 0 2021-03-26 08:01:51,972 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:01:52,991 epoch 49 - iter 2/25 - loss 2.45004141 - samples/sec: 125.84 - lr: 0.031250 2021-03-26 08:01:53,911 epoch 49 - iter 4/25 - loss 2.69668263 - samples/sec: 139.29 - lr: 0.031250 2021-03-26 08:01:54,987 epoch 49 - iter 6/25 - loss 2.57327108 - samples/sec: 119.17 - lr: 0.031250 2021-03-26 08:01:55,986 epoch 49 - iter 8/25 - loss 2.64431378 - samples/sec: 128.25 - lr: 0.031250 2021-03-26 08:01:57,033 epoch 49 - iter 10/25 - loss 2.68631110 - samples/sec: 122.35 - lr: 0.031250 2021-03-26 08:01:57,998 epoch 49 - iter 12/25 - loss 2.64004058 - samples/sec: 132.92 - lr: 0.031250 2021-03-26 08:01:58,999 epoch 49 - iter 14/25 - loss 2.69672627 - samples/sec: 127.97 - lr: 0.031250 2021-03-26 08:01:59,938 epoch 49 - iter 16/25 - loss 2.72755112 - samples/sec: 136.53 - lr: 0.031250 2021-03-26 08:02:00,883 epoch 49 - iter 18/25 - loss 2.75644163 - samples/sec: 135.63 - lr: 0.031250 2021-03-26 08:02:01,870 epoch 49 - iter 20/25 - loss 2.72888288 - samples/sec: 129.82 - lr: 0.031250 2021-03-26 08:02:02,864 epoch 49 - iter 22/25 - loss 2.73804910 - samples/sec: 128.98 - lr: 0.031250 2021-03-26 08:02:03,872 epoch 49 - iter 24/25 - loss 2.72713549 - samples/sec: 127.16 - lr: 0.031250 2021-03-26 08:02:04,248 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:02:04,249 EPOCH 49 done: loss 2.7030 - lr 0.0312500 2021-03-26 08:02:04,966 DEV : loss 6.120596885681152 - score 0.9156 2021-03-26 08:02:04,984 BAD EPOCHS (no improvement): 1 2021-03-26 08:02:04,985 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:02:06,072 epoch 50 - iter 2/25 - loss 2.36916256 - samples/sec: 117.87 - lr: 0.031250 2021-03-26 08:02:07,116 epoch 50 - iter 4/25 - loss 2.65234518 - samples/sec: 122.84 - lr: 0.031250 2021-03-26 08:02:08,202 epoch 50 - iter 6/25 - loss 2.63142109 - samples/sec: 118.08 - lr: 0.031250 2021-03-26 08:02:09,176 epoch 50 - iter 8/25 - loss 2.78051311 - samples/sec: 131.60 - lr: 0.031250 2021-03-26 08:02:10,136 epoch 50 - iter 10/25 - loss 2.68362582 - samples/sec: 133.59 - lr: 0.031250 2021-03-26 08:02:11,067 epoch 50 - iter 12/25 - loss 2.75772727 - samples/sec: 137.74 - lr: 0.031250 2021-03-26 08:02:12,088 epoch 50 - iter 14/25 - loss 2.77727463 - samples/sec: 125.51 - lr: 0.031250 2021-03-26 08:02:13,021 epoch 50 - iter 16/25 - loss 2.79809412 - samples/sec: 137.46 - lr: 0.031250 2021-03-26 08:02:13,965 epoch 50 - iter 18/25 - loss 2.80761631 - samples/sec: 135.84 - lr: 0.031250 2021-03-26 08:02:15,013 epoch 50 - iter 20/25 - loss 2.76133984 - samples/sec: 122.29 - lr: 0.031250 2021-03-26 08:02:16,007 epoch 50 - iter 22/25 - loss 2.76965055 - samples/sec: 128.94 - lr: 0.031250 2021-03-26 08:02:16,999 epoch 50 - iter 24/25 - loss 2.75284729 - samples/sec: 129.24 - lr: 0.031250 2021-03-26 08:02:17,409 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:02:17,410 EPOCH 50 done: loss 2.7666 - lr 0.0312500 2021-03-26 08:02:18,172 DEV : loss 6.149482727050781 - score 0.916 2021-03-26 08:02:18,187 BAD EPOCHS (no improvement): 2 2021-03-26 08:02:18,188 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:02:19,205 epoch 51 - iter 2/25 - loss 2.66018462 - samples/sec: 126.03 - lr: 0.031250 2021-03-26 08:02:20,302 epoch 51 - iter 4/25 - loss 2.74374110 - samples/sec: 116.79 - lr: 0.031250 2021-03-26 08:02:21,422 epoch 51 - iter 6/25 - loss 2.78392152 - samples/sec: 114.48 - lr: 0.031250 2021-03-26 08:02:22,698 epoch 51 - iter 8/25 - loss 2.76540592 - samples/sec: 100.38 - lr: 0.031250 2021-03-26 08:02:23,749 epoch 51 - iter 10/25 - loss 2.92016602 - samples/sec: 122.00 - lr: 0.031250 2021-03-26 08:02:24,735 epoch 51 - iter 12/25 - loss 3.01396398 - samples/sec: 130.09 - lr: 0.031250 2021-03-26 08:02:25,611 epoch 51 - iter 14/25 - loss 2.88940683 - samples/sec: 146.28 - lr: 0.031250 2021-03-26 08:02:26,691 epoch 51 - iter 16/25 - loss 2.86734806 - samples/sec: 118.71 - lr: 0.031250 2021-03-26 08:02:27,586 epoch 51 - iter 18/25 - loss 2.84448752 - samples/sec: 143.35 - lr: 0.031250 2021-03-26 08:02:28,457 epoch 51 - iter 20/25 - loss 2.81145627 - samples/sec: 147.07 - lr: 0.031250 2021-03-26 08:02:29,367 epoch 51 - iter 22/25 - loss 2.78330393 - samples/sec: 140.93 - lr: 0.031250 2021-03-26 08:02:30,321 epoch 51 - iter 24/25 - loss 2.80717117 - samples/sec: 134.35 - lr: 0.031250 2021-03-26 08:02:30,680 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:02:30,681 EPOCH 51 done: loss 2.8318 - lr 0.0312500 2021-03-26 08:02:31,374 DEV : loss 6.155113220214844 - score 0.9152 2021-03-26 08:02:31,388 BAD EPOCHS (no improvement): 3 2021-03-26 08:02:31,389 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:02:32,238 epoch 52 - iter 2/25 - loss 2.47030675 - samples/sec: 151.04 - lr: 0.031250 2021-03-26 08:02:34,188 epoch 52 - iter 4/25 - loss 2.87434727 - samples/sec: 65.69 - lr: 0.031250 2021-03-26 08:02:35,209 epoch 52 - iter 6/25 - loss 2.79234389 - samples/sec: 125.59 - lr: 0.031250 2021-03-26 08:02:36,172 epoch 52 - iter 8/25 - loss 2.85479537 - samples/sec: 133.17 - lr: 0.031250 2021-03-26 08:02:37,181 epoch 52 - iter 10/25 - loss 2.77208908 - samples/sec: 127.03 - lr: 0.031250 2021-03-26 08:02:38,167 epoch 52 - iter 12/25 - loss 2.76445206 - samples/sec: 130.04 - lr: 0.031250 2021-03-26 08:02:39,119 epoch 52 - iter 14/25 - loss 2.73319456 - samples/sec: 134.67 - lr: 0.031250 2021-03-26 08:02:40,021 epoch 52 - iter 16/25 - loss 2.71554077 - samples/sec: 142.15 - lr: 0.031250 2021-03-26 08:02:41,002 epoch 52 - iter 18/25 - loss 2.74959408 - samples/sec: 130.58 - lr: 0.031250 2021-03-26 08:02:42,042 epoch 52 - iter 20/25 - loss 2.73781605 - samples/sec: 123.32 - lr: 0.031250 2021-03-26 08:02:43,116 epoch 52 - iter 22/25 - loss 2.72883092 - samples/sec: 119.23 - lr: 0.031250 2021-03-26 08:02:44,163 epoch 52 - iter 24/25 - loss 2.74334789 - samples/sec: 122.47 - lr: 0.031250 2021-03-26 08:02:44,598 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:02:44,598 EPOCH 52 done: loss 2.7554 - lr 0.0312500 2021-03-26 08:02:45,314 DEV : loss 6.191333770751953 - score 0.9143 2021-03-26 08:02:45,337 BAD EPOCHS (no improvement): 4 2021-03-26 08:02:45,338 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:02:46,323 epoch 53 - iter 2/25 - loss 3.07710552 - samples/sec: 130.14 - lr: 0.015625 2021-03-26 08:02:47,307 epoch 53 - iter 4/25 - loss 2.85961819 - samples/sec: 130.30 - lr: 0.015625 2021-03-26 08:02:48,234 epoch 53 - iter 6/25 - loss 2.77078088 - samples/sec: 138.32 - lr: 0.015625 2021-03-26 08:02:49,175 epoch 53 - iter 8/25 - loss 2.94273353 - samples/sec: 136.19 - lr: 0.015625 2021-03-26 08:02:50,153 epoch 53 - iter 10/25 - loss 2.90624659 - samples/sec: 131.13 - lr: 0.015625 2021-03-26 08:02:51,187 epoch 53 - iter 12/25 - loss 2.89880838 - samples/sec: 123.88 - lr: 0.015625 2021-03-26 08:02:52,097 epoch 53 - iter 14/25 - loss 2.93234507 - samples/sec: 140.85 - lr: 0.015625 2021-03-26 08:02:53,001 epoch 53 - iter 16/25 - loss 2.95822586 - samples/sec: 141.86 - lr: 0.015625 2021-03-26 08:02:54,014 epoch 53 - iter 18/25 - loss 2.93325014 - samples/sec: 126.50 - lr: 0.015625 2021-03-26 08:02:54,941 epoch 53 - iter 20/25 - loss 2.86297564 - samples/sec: 138.37 - lr: 0.015625 2021-03-26 08:02:56,011 epoch 53 - iter 22/25 - loss 2.81225182 - samples/sec: 119.79 - lr: 0.015625 2021-03-26 08:02:57,026 epoch 53 - iter 24/25 - loss 2.79774497 - samples/sec: 126.31 - lr: 0.015625 2021-03-26 08:02:57,437 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:02:57,438 EPOCH 53 done: loss 2.8176 - lr 0.0156250 2021-03-26 08:02:58,158 DEV : loss 6.180680751800537 - score 0.9139 2021-03-26 08:02:58,182 BAD EPOCHS (no improvement): 1 2021-03-26 08:02:58,183 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:02:59,191 epoch 54 - iter 2/25 - loss 2.82024026 - samples/sec: 127.11 - lr: 0.015625 2021-03-26 08:03:00,137 epoch 54 - iter 4/25 - loss 2.84587026 - samples/sec: 135.55 - lr: 0.015625 2021-03-26 08:03:01,214 epoch 54 - iter 6/25 - loss 3.08639145 - samples/sec: 119.01 - lr: 0.015625 2021-03-26 08:03:02,171 epoch 54 - iter 8/25 - loss 3.04659450 - samples/sec: 133.97 - lr: 0.015625 2021-03-26 08:03:03,099 epoch 54 - iter 10/25 - loss 2.94338441 - samples/sec: 138.21 - lr: 0.015625 2021-03-26 08:03:04,121 epoch 54 - iter 12/25 - loss 2.86436790 - samples/sec: 125.34 - lr: 0.015625 2021-03-26 08:03:05,144 epoch 54 - iter 14/25 - loss 2.85726164 - samples/sec: 125.38 - lr: 0.015625 2021-03-26 08:03:06,139 epoch 54 - iter 16/25 - loss 2.84729083 - samples/sec: 128.86 - lr: 0.015625 2021-03-26 08:03:07,054 epoch 54 - iter 18/25 - loss 2.87125883 - samples/sec: 140.14 - lr: 0.015625 2021-03-26 08:03:07,957 epoch 54 - iter 20/25 - loss 2.86455238 - samples/sec: 142.02 - lr: 0.015625 2021-03-26 08:03:08,839 epoch 54 - iter 22/25 - loss 2.85972881 - samples/sec: 145.24 - lr: 0.015625 2021-03-26 08:03:09,997 epoch 54 - iter 24/25 - loss 2.88444576 - samples/sec: 110.71 - lr: 0.015625 2021-03-26 08:03:10,551 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:03:10,552 EPOCH 54 done: loss 2.8656 - lr 0.0156250 2021-03-26 08:03:11,251 DEV : loss 6.184042930603027 - score 0.9148 2021-03-26 08:03:11,273 BAD EPOCHS (no improvement): 2 2021-03-26 08:03:11,274 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:03:12,567 epoch 55 - iter 2/25 - loss 2.71724856 - samples/sec: 99.09 - lr: 0.015625 2021-03-26 08:03:13,572 epoch 55 - iter 4/25 - loss 2.92992836 - samples/sec: 127.53 - lr: 0.015625 2021-03-26 08:03:14,521 epoch 55 - iter 6/25 - loss 2.92324722 - samples/sec: 135.12 - lr: 0.015625 2021-03-26 08:03:15,552 epoch 55 - iter 8/25 - loss 3.01632318 - samples/sec: 124.29 - lr: 0.015625 2021-03-26 08:03:16,469 epoch 55 - iter 10/25 - loss 2.85993338 - samples/sec: 139.77 - lr: 0.015625 2021-03-26 08:03:17,506 epoch 55 - iter 12/25 - loss 2.88023615 - samples/sec: 123.60 - lr: 0.015625 2021-03-26 08:03:18,448 epoch 55 - iter 14/25 - loss 2.78017191 - samples/sec: 136.85 - lr: 0.015625 2021-03-26 08:03:19,322 epoch 55 - iter 16/25 - loss 2.75286925 - samples/sec: 146.64 - lr: 0.015625 2021-03-26 08:03:20,441 epoch 55 - iter 18/25 - loss 2.78123773 - samples/sec: 114.54 - lr: 0.015625 2021-03-26 08:03:21,305 epoch 55 - iter 20/25 - loss 2.82595617 - samples/sec: 148.62 - lr: 0.015625 2021-03-26 08:03:22,283 epoch 55 - iter 22/25 - loss 2.82012420 - samples/sec: 131.04 - lr: 0.015625 2021-03-26 08:03:23,237 epoch 55 - iter 24/25 - loss 2.82870817 - samples/sec: 134.39 - lr: 0.015625 2021-03-26 08:03:23,607 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:03:23,608 EPOCH 55 done: loss 2.8143 - lr 0.0156250 2021-03-26 08:03:24,314 DEV : loss 6.195590972900391 - score 0.9152 2021-03-26 08:03:24,337 BAD EPOCHS (no improvement): 3 2021-03-26 08:03:24,338 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:03:25,307 epoch 56 - iter 2/25 - loss 2.88957417 - samples/sec: 132.41 - lr: 0.015625 2021-03-26 08:03:26,310 epoch 56 - iter 4/25 - loss 2.73559302 - samples/sec: 127.86 - lr: 0.015625 2021-03-26 08:03:27,334 epoch 56 - iter 6/25 - loss 2.70232844 - samples/sec: 125.13 - lr: 0.015625 2021-03-26 08:03:28,243 epoch 56 - iter 8/25 - loss 2.69687900 - samples/sec: 141.08 - lr: 0.015625 2021-03-26 08:03:29,265 epoch 56 - iter 10/25 - loss 2.80344183 - samples/sec: 125.43 - lr: 0.015625 2021-03-26 08:03:30,362 epoch 56 - iter 12/25 - loss 2.83803695 - samples/sec: 116.77 - lr: 0.015625 2021-03-26 08:03:31,314 epoch 56 - iter 14/25 - loss 2.82412973 - samples/sec: 134.73 - lr: 0.015625 2021-03-26 08:03:32,293 epoch 56 - iter 16/25 - loss 2.85887678 - samples/sec: 130.91 - lr: 0.015625 2021-03-26 08:03:33,367 epoch 56 - iter 18/25 - loss 2.81283089 - samples/sec: 119.29 - lr: 0.015625 2021-03-26 08:03:34,344 epoch 56 - iter 20/25 - loss 2.76934352 - samples/sec: 131.22 - lr: 0.015625 2021-03-26 08:03:35,274 epoch 56 - iter 22/25 - loss 2.78952987 - samples/sec: 137.91 - lr: 0.015625 2021-03-26 08:03:36,304 epoch 56 - iter 24/25 - loss 2.85014056 - samples/sec: 124.53 - lr: 0.015625 2021-03-26 08:03:36,703 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:03:36,704 EPOCH 56 done: loss 2.8169 - lr 0.0156250 2021-03-26 08:03:37,402 DEV : loss 6.179908752441406 - score 0.916 2021-03-26 08:03:37,425 BAD EPOCHS (no improvement): 4 2021-03-26 08:03:37,426 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:03:38,402 epoch 57 - iter 2/25 - loss 2.89332473 - samples/sec: 131.40 - lr: 0.007812 2021-03-26 08:03:39,368 epoch 57 - iter 4/25 - loss 2.77982998 - samples/sec: 132.66 - lr: 0.007812 2021-03-26 08:03:40,401 epoch 57 - iter 6/25 - loss 2.71375469 - samples/sec: 124.09 - lr: 0.007812 2021-03-26 08:03:41,394 epoch 57 - iter 8/25 - loss 2.68291500 - samples/sec: 129.03 - lr: 0.007812 2021-03-26 08:03:42,516 epoch 57 - iter 10/25 - loss 2.70066304 - samples/sec: 114.27 - lr: 0.007812 2021-03-26 08:03:43,523 epoch 57 - iter 12/25 - loss 2.73475238 - samples/sec: 127.31 - lr: 0.007812 2021-03-26 08:03:44,454 epoch 57 - iter 14/25 - loss 2.73579529 - samples/sec: 137.72 - lr: 0.007812 2021-03-26 08:03:45,423 epoch 57 - iter 16/25 - loss 2.73032914 - samples/sec: 132.24 - lr: 0.007812 2021-03-26 08:03:46,456 epoch 57 - iter 18/25 - loss 2.72013121 - samples/sec: 124.20 - lr: 0.007812 2021-03-26 08:03:47,330 epoch 57 - iter 20/25 - loss 2.70019326 - samples/sec: 146.77 - lr: 0.007812 2021-03-26 08:03:48,316 epoch 57 - iter 22/25 - loss 2.70320966 - samples/sec: 129.97 - lr: 0.007812 2021-03-26 08:03:49,287 epoch 57 - iter 24/25 - loss 2.71015591 - samples/sec: 132.05 - lr: 0.007812 2021-03-26 08:03:49,677 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:03:49,677 EPOCH 57 done: loss 2.6899 - lr 0.0078125 2021-03-26 08:03:50,422 DEV : loss 6.175540447235107 - score 0.9177 2021-03-26 08:03:50,448 BAD EPOCHS (no improvement): 0 2021-03-26 08:04:00,281 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:04:01,264 epoch 58 - iter 2/25 - loss 2.83420646 - samples/sec: 130.48 - lr: 0.007812 2021-03-26 08:04:02,184 epoch 58 - iter 4/25 - loss 2.72172236 - samples/sec: 139.39 - lr: 0.007812 2021-03-26 08:04:03,100 epoch 58 - iter 6/25 - loss 2.92447762 - samples/sec: 139.93 - lr: 0.007812 2021-03-26 08:04:04,046 epoch 58 - iter 8/25 - loss 2.98371637 - samples/sec: 135.51 - lr: 0.007812 2021-03-26 08:04:05,063 epoch 58 - iter 10/25 - loss 2.98575375 - samples/sec: 126.08 - lr: 0.007812 2021-03-26 08:04:06,094 epoch 58 - iter 12/25 - loss 2.91445754 - samples/sec: 124.33 - lr: 0.007812 2021-03-26 08:04:07,082 epoch 58 - iter 14/25 - loss 2.87455147 - samples/sec: 129.64 - lr: 0.007812 2021-03-26 08:04:08,014 epoch 58 - iter 16/25 - loss 2.85320154 - samples/sec: 137.54 - lr: 0.007812 2021-03-26 08:04:08,985 epoch 58 - iter 18/25 - loss 2.81956890 - samples/sec: 132.11 - lr: 0.007812 2021-03-26 08:04:10,017 epoch 58 - iter 20/25 - loss 2.76832836 - samples/sec: 124.16 - lr: 0.007812 2021-03-26 08:04:11,068 epoch 58 - iter 22/25 - loss 2.76364282 - samples/sec: 121.88 - lr: 0.007812 2021-03-26 08:04:12,067 epoch 58 - iter 24/25 - loss 2.77961952 - samples/sec: 128.42 - lr: 0.007812 2021-03-26 08:04:12,445 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:04:12,446 EPOCH 58 done: loss 2.7634 - lr 0.0078125 2021-03-26 08:04:13,186 DEV : loss 6.17774772644043 - score 0.9173 2021-03-26 08:04:13,209 BAD EPOCHS (no improvement): 1 2021-03-26 08:04:13,211 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:04:14,205 epoch 59 - iter 2/25 - loss 2.55935884 - samples/sec: 128.95 - lr: 0.007812 2021-03-26 08:04:15,168 epoch 59 - iter 4/25 - loss 2.47871357 - samples/sec: 133.11 - lr: 0.007812 2021-03-26 08:04:16,134 epoch 59 - iter 6/25 - loss 2.69761129 - samples/sec: 132.82 - lr: 0.007812 2021-03-26 08:04:17,298 epoch 59 - iter 8/25 - loss 2.87344724 - samples/sec: 110.09 - lr: 0.007812 2021-03-26 08:04:18,418 epoch 59 - iter 10/25 - loss 2.79624567 - samples/sec: 114.41 - lr: 0.007812 2021-03-26 08:04:19,577 epoch 59 - iter 12/25 - loss 2.82552556 - samples/sec: 111.41 - lr: 0.007812 2021-03-26 08:04:20,704 epoch 59 - iter 14/25 - loss 2.79570598 - samples/sec: 114.87 - lr: 0.007812 2021-03-26 08:04:21,623 epoch 59 - iter 16/25 - loss 2.77979054 - samples/sec: 140.65 - lr: 0.007812 2021-03-26 08:04:22,655 epoch 59 - iter 18/25 - loss 2.79154040 - samples/sec: 124.19 - lr: 0.007812 2021-03-26 08:04:23,594 epoch 59 - iter 20/25 - loss 2.77637383 - samples/sec: 136.43 - lr: 0.007812 2021-03-26 08:04:24,473 epoch 59 - iter 22/25 - loss 2.73464512 - samples/sec: 145.86 - lr: 0.007812 2021-03-26 08:04:25,341 epoch 59 - iter 24/25 - loss 2.72856198 - samples/sec: 147.73 - lr: 0.007812 2021-03-26 08:04:25,719 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:04:25,721 EPOCH 59 done: loss 2.7133 - lr 0.0078125 2021-03-26 08:04:26,418 DEV : loss 6.174241542816162 - score 0.9181 2021-03-26 08:04:26,440 BAD EPOCHS (no improvement): 0 2021-03-26 08:04:35,908 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:04:36,881 epoch 60 - iter 2/25 - loss 2.56185210 - samples/sec: 131.81 - lr: 0.007812 2021-03-26 08:04:37,874 epoch 60 - iter 4/25 - loss 2.69779313 - samples/sec: 129.08 - lr: 0.007812 2021-03-26 08:04:38,881 epoch 60 - iter 6/25 - loss 2.75501200 - samples/sec: 127.33 - lr: 0.007812 2021-03-26 08:04:39,931 epoch 60 - iter 8/25 - loss 2.73785749 - samples/sec: 122.05 - lr: 0.007812 2021-03-26 08:04:40,897 epoch 60 - iter 10/25 - loss 2.72844467 - samples/sec: 132.60 - lr: 0.007812 2021-03-26 08:04:41,948 epoch 60 - iter 12/25 - loss 2.71255245 - samples/sec: 121.99 - lr: 0.007812 2021-03-26 08:04:42,874 epoch 60 - iter 14/25 - loss 2.69912761 - samples/sec: 138.62 - lr: 0.007812 2021-03-26 08:04:43,868 epoch 60 - iter 16/25 - loss 2.73890451 - samples/sec: 128.92 - lr: 0.007812 2021-03-26 08:04:44,820 epoch 60 - iter 18/25 - loss 2.72212456 - samples/sec: 134.59 - lr: 0.007812 2021-03-26 08:04:45,783 epoch 60 - iter 20/25 - loss 2.72613677 - samples/sec: 133.27 - lr: 0.007812 2021-03-26 08:04:46,781 epoch 60 - iter 22/25 - loss 2.74840914 - samples/sec: 128.40 - lr: 0.007812 2021-03-26 08:04:47,710 epoch 60 - iter 24/25 - loss 2.78334759 - samples/sec: 138.02 - lr: 0.007812 2021-03-26 08:04:48,101 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:04:48,102 EPOCH 60 done: loss 2.7870 - lr 0.0078125 2021-03-26 08:04:48,803 DEV : loss 6.180110454559326 - score 0.9177 2021-03-26 08:04:48,825 BAD EPOCHS (no improvement): 1 2021-03-26 08:04:48,826 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:04:49,736 epoch 61 - iter 2/25 - loss 2.79055202 - samples/sec: 141.03 - lr: 0.007812 2021-03-26 08:04:50,681 epoch 61 - iter 4/25 - loss 2.93640441 - samples/sec: 135.57 - lr: 0.007812 2021-03-26 08:04:51,615 epoch 61 - iter 6/25 - loss 2.99876702 - samples/sec: 137.22 - lr: 0.007812 2021-03-26 08:04:52,511 epoch 61 - iter 8/25 - loss 2.97727868 - samples/sec: 143.18 - lr: 0.007812 2021-03-26 08:04:53,917 epoch 61 - iter 10/25 - loss 2.98115492 - samples/sec: 91.08 - lr: 0.007812 2021-03-26 08:04:54,899 epoch 61 - iter 12/25 - loss 2.97442589 - samples/sec: 130.56 - lr: 0.007812 2021-03-26 08:04:55,853 epoch 61 - iter 14/25 - loss 2.91914528 - samples/sec: 134.43 - lr: 0.007812 2021-03-26 08:04:56,908 epoch 61 - iter 16/25 - loss 2.90313707 - samples/sec: 121.50 - lr: 0.007812 2021-03-26 08:04:57,821 epoch 61 - iter 18/25 - loss 2.86542445 - samples/sec: 140.31 - lr: 0.007812 2021-03-26 08:04:58,881 epoch 61 - iter 20/25 - loss 2.82913233 - samples/sec: 121.06 - lr: 0.007812 2021-03-26 08:04:59,827 epoch 61 - iter 22/25 - loss 2.79248063 - samples/sec: 135.50 - lr: 0.007812 2021-03-26 08:05:00,801 epoch 61 - iter 24/25 - loss 2.83465045 - samples/sec: 131.53 - lr: 0.007812 2021-03-26 08:05:01,418 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:05:01,419 EPOCH 61 done: loss 2.8220 - lr 0.0078125 2021-03-26 08:05:02,124 DEV : loss 6.179595470428467 - score 0.9173 2021-03-26 08:05:02,147 BAD EPOCHS (no improvement): 2 2021-03-26 08:05:02,147 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:05:03,143 epoch 62 - iter 2/25 - loss 2.05444831 - samples/sec: 128.83 - lr: 0.007812 2021-03-26 08:05:04,275 epoch 62 - iter 4/25 - loss 2.62164161 - samples/sec: 113.15 - lr: 0.007812 2021-03-26 08:05:05,260 epoch 62 - iter 6/25 - loss 2.69296275 - samples/sec: 130.27 - lr: 0.007812 2021-03-26 08:05:06,405 epoch 62 - iter 8/25 - loss 2.83039902 - samples/sec: 111.95 - lr: 0.007812 2021-03-26 08:05:07,497 epoch 62 - iter 10/25 - loss 2.78501257 - samples/sec: 117.43 - lr: 0.007812 2021-03-26 08:05:08,463 epoch 62 - iter 12/25 - loss 2.81995478 - samples/sec: 132.75 - lr: 0.007812 2021-03-26 08:05:09,451 epoch 62 - iter 14/25 - loss 2.88364232 - samples/sec: 129.73 - lr: 0.007812 2021-03-26 08:05:10,516 epoch 62 - iter 16/25 - loss 2.83740675 - samples/sec: 120.32 - lr: 0.007812 2021-03-26 08:05:11,445 epoch 62 - iter 18/25 - loss 2.83892163 - samples/sec: 138.15 - lr: 0.007812 2021-03-26 08:05:12,365 epoch 62 - iter 20/25 - loss 2.78212218 - samples/sec: 139.29 - lr: 0.007812 2021-03-26 08:05:13,385 epoch 62 - iter 22/25 - loss 2.78157476 - samples/sec: 125.78 - lr: 0.007812 2021-03-26 08:05:14,403 epoch 62 - iter 24/25 - loss 2.81738104 - samples/sec: 125.90 - lr: 0.007812 2021-03-26 08:05:14,776 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:05:14,777 EPOCH 62 done: loss 2.8339 - lr 0.0078125 2021-03-26 08:05:15,491 DEV : loss 6.171009063720703 - score 0.9181 2021-03-26 08:05:15,514 BAD EPOCHS (no improvement): 0 2021-03-26 08:05:24,932 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:05:25,826 epoch 63 - iter 2/25 - loss 2.51848078 - samples/sec: 143.50 - lr: 0.007812 2021-03-26 08:05:26,829 epoch 63 - iter 4/25 - loss 2.60838878 - samples/sec: 127.81 - lr: 0.007812 2021-03-26 08:05:27,777 epoch 63 - iter 6/25 - loss 2.60025974 - samples/sec: 135.22 - lr: 0.007812 2021-03-26 08:05:28,670 epoch 63 - iter 8/25 - loss 2.63403827 - samples/sec: 143.40 - lr: 0.007812 2021-03-26 08:05:29,666 epoch 63 - iter 10/25 - loss 2.80634141 - samples/sec: 128.72 - lr: 0.007812 2021-03-26 08:05:30,645 epoch 63 - iter 12/25 - loss 2.77708697 - samples/sec: 131.04 - lr: 0.007812 2021-03-26 08:05:31,602 epoch 63 - iter 14/25 - loss 2.71201353 - samples/sec: 134.04 - lr: 0.007812 2021-03-26 08:05:32,581 epoch 63 - iter 16/25 - loss 2.72835907 - samples/sec: 130.96 - lr: 0.007812 2021-03-26 08:05:33,721 epoch 63 - iter 18/25 - loss 2.72997316 - samples/sec: 112.34 - lr: 0.007812 2021-03-26 08:05:34,621 epoch 63 - iter 20/25 - loss 2.71590858 - samples/sec: 142.39 - lr: 0.007812 2021-03-26 08:05:35,778 epoch 63 - iter 22/25 - loss 2.73422451 - samples/sec: 110.81 - lr: 0.007812 2021-03-26 08:05:36,865 epoch 63 - iter 24/25 - loss 2.73682607 - samples/sec: 117.96 - lr: 0.007812 2021-03-26 08:05:37,286 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:05:37,287 EPOCH 63 done: loss 2.7471 - lr 0.0078125 2021-03-26 08:05:38,067 DEV : loss 6.168983459472656 - score 0.9173 2021-03-26 08:05:38,085 BAD EPOCHS (no improvement): 1 2021-03-26 08:05:38,085 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:05:39,141 epoch 64 - iter 2/25 - loss 3.04874635 - samples/sec: 121.39 - lr: 0.007812 2021-03-26 08:05:40,160 epoch 64 - iter 4/25 - loss 3.02451402 - samples/sec: 125.84 - lr: 0.007812 2021-03-26 08:05:41,141 epoch 64 - iter 6/25 - loss 3.02157851 - samples/sec: 130.54 - lr: 0.007812 2021-03-26 08:05:42,156 epoch 64 - iter 8/25 - loss 3.11041513 - samples/sec: 126.40 - lr: 0.007812 2021-03-26 08:05:43,227 epoch 64 - iter 10/25 - loss 3.03794565 - samples/sec: 119.70 - lr: 0.007812 2021-03-26 08:05:44,173 epoch 64 - iter 12/25 - loss 2.95237716 - samples/sec: 135.52 - lr: 0.007812 2021-03-26 08:05:45,163 epoch 64 - iter 14/25 - loss 2.82234809 - samples/sec: 129.54 - lr: 0.007812 2021-03-26 08:05:46,188 epoch 64 - iter 16/25 - loss 2.79835985 - samples/sec: 124.99 - lr: 0.007812 2021-03-26 08:05:47,164 epoch 64 - iter 18/25 - loss 2.83342325 - samples/sec: 131.35 - lr: 0.007812 2021-03-26 08:05:48,081 epoch 64 - iter 20/25 - loss 2.77793549 - samples/sec: 139.74 - lr: 0.007812 2021-03-26 08:05:48,993 epoch 64 - iter 22/25 - loss 2.76137200 - samples/sec: 140.61 - lr: 0.007812 2021-03-26 08:05:50,009 epoch 64 - iter 24/25 - loss 2.81117615 - samples/sec: 126.14 - lr: 0.007812 2021-03-26 08:05:50,467 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:05:50,468 EPOCH 64 done: loss 2.8223 - lr 0.0078125 2021-03-26 08:05:51,173 DEV : loss 6.175807952880859 - score 0.9173 2021-03-26 08:05:51,189 BAD EPOCHS (no improvement): 2 2021-03-26 08:05:51,190 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:05:52,179 epoch 65 - iter 2/25 - loss 2.65639818 - samples/sec: 129.64 - lr: 0.007812 2021-03-26 08:05:53,178 epoch 65 - iter 4/25 - loss 2.82343382 - samples/sec: 128.24 - lr: 0.007812 2021-03-26 08:05:54,222 epoch 65 - iter 6/25 - loss 2.72004970 - samples/sec: 122.73 - lr: 0.007812 2021-03-26 08:05:55,260 epoch 65 - iter 8/25 - loss 2.67372179 - samples/sec: 123.52 - lr: 0.007812 2021-03-26 08:05:56,283 epoch 65 - iter 10/25 - loss 2.58629749 - samples/sec: 125.36 - lr: 0.007812 2021-03-26 08:05:57,308 epoch 65 - iter 12/25 - loss 2.52548907 - samples/sec: 124.96 - lr: 0.007812 2021-03-26 08:05:58,310 epoch 65 - iter 14/25 - loss 2.55922290 - samples/sec: 127.94 - lr: 0.007812 2021-03-26 08:05:59,198 epoch 65 - iter 16/25 - loss 2.51116529 - samples/sec: 144.34 - lr: 0.007812 2021-03-26 08:06:00,151 epoch 65 - iter 18/25 - loss 2.54416679 - samples/sec: 134.58 - lr: 0.007812 2021-03-26 08:06:01,081 epoch 65 - iter 20/25 - loss 2.59224055 - samples/sec: 137.88 - lr: 0.007812 2021-03-26 08:06:02,018 epoch 65 - iter 22/25 - loss 2.63312927 - samples/sec: 136.79 - lr: 0.007812 2021-03-26 08:06:03,002 epoch 65 - iter 24/25 - loss 2.68356836 - samples/sec: 130.23 - lr: 0.007812 2021-03-26 08:06:03,393 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:06:03,393 EPOCH 65 done: loss 2.6666 - lr 0.0078125 2021-03-26 08:06:04,119 DEV : loss 6.184266090393066 - score 0.9177 2021-03-26 08:06:04,141 BAD EPOCHS (no improvement): 3 2021-03-26 08:06:04,142 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:06:05,166 epoch 66 - iter 2/25 - loss 3.00271451 - samples/sec: 125.29 - lr: 0.007812 2021-03-26 08:06:06,190 epoch 66 - iter 4/25 - loss 2.93223822 - samples/sec: 125.16 - lr: 0.007812 2021-03-26 08:06:07,112 epoch 66 - iter 6/25 - loss 2.83356301 - samples/sec: 138.97 - lr: 0.007812 2021-03-26 08:06:08,284 epoch 66 - iter 8/25 - loss 2.68446249 - samples/sec: 109.43 - lr: 0.007812 2021-03-26 08:06:09,288 epoch 66 - iter 10/25 - loss 2.81322050 - samples/sec: 127.55 - lr: 0.007812 2021-03-26 08:06:10,237 epoch 66 - iter 12/25 - loss 2.79783382 - samples/sec: 135.02 - lr: 0.007812 2021-03-26 08:06:11,231 epoch 66 - iter 14/25 - loss 2.82855047 - samples/sec: 129.01 - lr: 0.007812 2021-03-26 08:06:12,211 epoch 66 - iter 16/25 - loss 2.82480106 - samples/sec: 130.92 - lr: 0.007812 2021-03-26 08:06:13,238 epoch 66 - iter 18/25 - loss 2.79299916 - samples/sec: 124.73 - lr: 0.007812 2021-03-26 08:06:14,184 epoch 66 - iter 20/25 - loss 2.84182227 - samples/sec: 135.54 - lr: 0.007812 2021-03-26 08:06:15,108 epoch 66 - iter 22/25 - loss 2.86088503 - samples/sec: 138.71 - lr: 0.007812 2021-03-26 08:06:16,117 epoch 66 - iter 24/25 - loss 2.83759779 - samples/sec: 127.06 - lr: 0.007812 2021-03-26 08:06:16,548 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:06:16,548 EPOCH 66 done: loss 2.8567 - lr 0.0078125 2021-03-26 08:06:17,275 DEV : loss 6.17871618270874 - score 0.9173 2021-03-26 08:06:17,291 BAD EPOCHS (no improvement): 4 2021-03-26 08:06:17,291 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:06:18,335 epoch 67 - iter 2/25 - loss 2.24417460 - samples/sec: 122.80 - lr: 0.003906 2021-03-26 08:06:19,355 epoch 67 - iter 4/25 - loss 2.76672024 - samples/sec: 125.73 - lr: 0.003906 2021-03-26 08:06:20,368 epoch 67 - iter 6/25 - loss 2.68564645 - samples/sec: 126.53 - lr: 0.003906 2021-03-26 08:06:21,420 epoch 67 - iter 8/25 - loss 2.79277220 - samples/sec: 121.91 - lr: 0.003906 2021-03-26 08:06:22,371 epoch 67 - iter 10/25 - loss 2.75309818 - samples/sec: 134.78 - lr: 0.003906 2021-03-26 08:06:23,376 epoch 67 - iter 12/25 - loss 2.65836643 - samples/sec: 127.70 - lr: 0.003906 2021-03-26 08:06:24,344 epoch 67 - iter 14/25 - loss 2.62789102 - samples/sec: 132.35 - lr: 0.003906 2021-03-26 08:06:25,422 epoch 67 - iter 16/25 - loss 2.64878156 - samples/sec: 118.85 - lr: 0.003906 2021-03-26 08:06:26,483 epoch 67 - iter 18/25 - loss 2.72514617 - samples/sec: 120.85 - lr: 0.003906 2021-03-26 08:06:27,484 epoch 67 - iter 20/25 - loss 2.79397482 - samples/sec: 127.98 - lr: 0.003906 2021-03-26 08:06:28,437 epoch 67 - iter 22/25 - loss 2.76995759 - samples/sec: 134.53 - lr: 0.003906 2021-03-26 08:06:29,334 epoch 67 - iter 24/25 - loss 2.79965747 - samples/sec: 142.84 - lr: 0.003906 2021-03-26 08:06:29,777 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:06:29,778 EPOCH 67 done: loss 2.7726 - lr 0.0039062 2021-03-26 08:06:30,502 DEV : loss 6.179577827453613 - score 0.9173 2021-03-26 08:06:30,523 BAD EPOCHS (no improvement): 1 2021-03-26 08:06:30,524 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:06:31,504 epoch 68 - iter 2/25 - loss 3.11455142 - samples/sec: 131.01 - lr: 0.003906 2021-03-26 08:06:32,431 epoch 68 - iter 4/25 - loss 2.82809067 - samples/sec: 138.15 - lr: 0.003906 2021-03-26 08:06:33,366 epoch 68 - iter 6/25 - loss 2.72769392 - samples/sec: 137.20 - lr: 0.003906 2021-03-26 08:06:34,286 epoch 68 - iter 8/25 - loss 2.65765330 - samples/sec: 139.28 - lr: 0.003906 2021-03-26 08:06:35,218 epoch 68 - iter 10/25 - loss 2.73607638 - samples/sec: 137.49 - lr: 0.003906 2021-03-26 08:06:36,254 epoch 68 - iter 12/25 - loss 2.71418903 - samples/sec: 123.80 - lr: 0.003906 2021-03-26 08:06:37,189 epoch 68 - iter 14/25 - loss 2.77073138 - samples/sec: 137.11 - lr: 0.003906 2021-03-26 08:06:38,260 epoch 68 - iter 16/25 - loss 2.78171870 - samples/sec: 119.64 - lr: 0.003906 2021-03-26 08:06:39,270 epoch 68 - iter 18/25 - loss 2.77806143 - samples/sec: 126.91 - lr: 0.003906 2021-03-26 08:06:40,250 epoch 68 - iter 20/25 - loss 2.75635051 - samples/sec: 130.87 - lr: 0.003906 2021-03-26 08:06:41,216 epoch 68 - iter 22/25 - loss 2.78475171 - samples/sec: 132.71 - lr: 0.003906 2021-03-26 08:06:42,157 epoch 68 - iter 24/25 - loss 2.78028412 - samples/sec: 136.27 - lr: 0.003906 2021-03-26 08:06:42,559 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:06:42,560 EPOCH 68 done: loss 2.8145 - lr 0.0039062 2021-03-26 08:06:43,266 DEV : loss 6.189845085144043 - score 0.9177 2021-03-26 08:06:43,286 BAD EPOCHS (no improvement): 2 2021-03-26 08:06:43,286 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:06:44,309 epoch 69 - iter 2/25 - loss 2.54150748 - samples/sec: 125.31 - lr: 0.003906 2021-03-26 08:06:45,311 epoch 69 - iter 4/25 - loss 2.69912976 - samples/sec: 127.93 - lr: 0.003906 2021-03-26 08:06:46,286 epoch 69 - iter 6/25 - loss 2.71940128 - samples/sec: 131.57 - lr: 0.003906 2021-03-26 08:06:47,311 epoch 69 - iter 8/25 - loss 2.73294532 - samples/sec: 125.02 - lr: 0.003906 2021-03-26 08:06:48,291 epoch 69 - iter 10/25 - loss 2.77171984 - samples/sec: 130.84 - lr: 0.003906 2021-03-26 08:06:49,182 epoch 69 - iter 12/25 - loss 2.85729210 - samples/sec: 143.80 - lr: 0.003906 2021-03-26 08:06:50,113 epoch 69 - iter 14/25 - loss 2.84100824 - samples/sec: 137.66 - lr: 0.003906 2021-03-26 08:06:51,160 epoch 69 - iter 16/25 - loss 2.81065214 - samples/sec: 122.42 - lr: 0.003906 2021-03-26 08:06:52,056 epoch 69 - iter 18/25 - loss 2.79097626 - samples/sec: 143.06 - lr: 0.003906 2021-03-26 08:06:53,085 epoch 69 - iter 20/25 - loss 2.80775127 - samples/sec: 124.57 - lr: 0.003906 2021-03-26 08:06:53,990 epoch 69 - iter 22/25 - loss 2.77170913 - samples/sec: 141.60 - lr: 0.003906 2021-03-26 08:06:55,044 epoch 69 - iter 24/25 - loss 2.81093141 - samples/sec: 121.61 - lr: 0.003906 2021-03-26 08:06:55,481 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:06:55,483 EPOCH 69 done: loss 2.8364 - lr 0.0039062 2021-03-26 08:06:56,219 DEV : loss 6.186550140380859 - score 0.9177 2021-03-26 08:06:56,242 BAD EPOCHS (no improvement): 3 2021-03-26 08:06:56,243 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:06:57,225 epoch 70 - iter 2/25 - loss 2.53426838 - samples/sec: 130.52 - lr: 0.003906 2021-03-26 08:06:58,243 epoch 70 - iter 4/25 - loss 2.63942099 - samples/sec: 125.89 - lr: 0.003906 2021-03-26 08:06:59,312 epoch 70 - iter 6/25 - loss 2.77118782 - samples/sec: 119.87 - lr: 0.003906 2021-03-26 08:07:00,331 epoch 70 - iter 8/25 - loss 2.83780730 - samples/sec: 125.88 - lr: 0.003906 2021-03-26 08:07:01,269 epoch 70 - iter 10/25 - loss 2.77150600 - samples/sec: 136.62 - lr: 0.003906 2021-03-26 08:07:02,208 epoch 70 - iter 12/25 - loss 2.68753244 - samples/sec: 136.58 - lr: 0.003906 2021-03-26 08:07:03,134 epoch 70 - iter 14/25 - loss 2.68498324 - samples/sec: 138.39 - lr: 0.003906 2021-03-26 08:07:04,030 epoch 70 - iter 16/25 - loss 2.81244020 - samples/sec: 143.03 - lr: 0.003906 2021-03-26 08:07:05,001 epoch 70 - iter 18/25 - loss 2.85790327 - samples/sec: 132.05 - lr: 0.003906 2021-03-26 08:07:05,958 epoch 70 - iter 20/25 - loss 2.80446895 - samples/sec: 134.12 - lr: 0.003906 2021-03-26 08:07:06,852 epoch 70 - iter 22/25 - loss 2.78271670 - samples/sec: 143.38 - lr: 0.003906 2021-03-26 08:07:07,821 epoch 70 - iter 24/25 - loss 2.80982100 - samples/sec: 132.17 - lr: 0.003906 2021-03-26 08:07:08,261 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:07:08,262 EPOCH 70 done: loss 2.8286 - lr 0.0039062 2021-03-26 08:07:09,004 DEV : loss 6.186135768890381 - score 0.9177 2021-03-26 08:07:09,027 BAD EPOCHS (no improvement): 4 2021-03-26 08:07:09,028 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:07:10,094 epoch 71 - iter 2/25 - loss 2.33365417 - samples/sec: 120.27 - lr: 0.001953 2021-03-26 08:07:11,122 epoch 71 - iter 4/25 - loss 2.40433973 - samples/sec: 124.67 - lr: 0.001953 2021-03-26 08:07:12,127 epoch 71 - iter 6/25 - loss 2.42733010 - samples/sec: 127.63 - lr: 0.001953 2021-03-26 08:07:13,177 epoch 71 - iter 8/25 - loss 2.58082166 - samples/sec: 122.06 - lr: 0.001953 2021-03-26 08:07:14,222 epoch 71 - iter 10/25 - loss 2.59674971 - samples/sec: 122.72 - lr: 0.001953 2021-03-26 08:07:15,215 epoch 71 - iter 12/25 - loss 2.59080458 - samples/sec: 129.06 - lr: 0.001953 2021-03-26 08:07:16,220 epoch 71 - iter 14/25 - loss 2.58698351 - samples/sec: 127.62 - lr: 0.001953 2021-03-26 08:07:17,080 epoch 71 - iter 16/25 - loss 2.55493875 - samples/sec: 149.07 - lr: 0.001953 2021-03-26 08:07:18,053 epoch 71 - iter 18/25 - loss 2.53936093 - samples/sec: 131.78 - lr: 0.001953 2021-03-26 08:07:19,048 epoch 71 - iter 20/25 - loss 2.60882049 - samples/sec: 128.90 - lr: 0.001953 2021-03-26 08:07:20,112 epoch 71 - iter 22/25 - loss 2.62530265 - samples/sec: 120.51 - lr: 0.001953 2021-03-26 08:07:21,064 epoch 71 - iter 24/25 - loss 2.63414593 - samples/sec: 134.58 - lr: 0.001953 2021-03-26 08:07:21,479 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:07:21,480 EPOCH 71 done: loss 2.6570 - lr 0.0019531 2021-03-26 08:07:22,193 DEV : loss 6.187526226043701 - score 0.9177 2021-03-26 08:07:22,215 BAD EPOCHS (no improvement): 1 2021-03-26 08:07:22,215 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:07:23,173 epoch 72 - iter 2/25 - loss 2.70109022 - samples/sec: 133.91 - lr: 0.001953 2021-03-26 08:07:24,108 epoch 72 - iter 4/25 - loss 2.65733844 - samples/sec: 137.20 - lr: 0.001953 2021-03-26 08:07:25,165 epoch 72 - iter 6/25 - loss 2.72642616 - samples/sec: 121.24 - lr: 0.001953 2021-03-26 08:07:26,224 epoch 72 - iter 8/25 - loss 2.78701332 - samples/sec: 121.09 - lr: 0.001953 2021-03-26 08:07:27,335 epoch 72 - iter 10/25 - loss 2.74920893 - samples/sec: 115.34 - lr: 0.001953 2021-03-26 08:07:28,286 epoch 72 - iter 12/25 - loss 2.75116277 - samples/sec: 134.73 - lr: 0.001953 2021-03-26 08:07:29,419 epoch 72 - iter 14/25 - loss 2.75593237 - samples/sec: 113.16 - lr: 0.001953 2021-03-26 08:07:30,383 epoch 72 - iter 16/25 - loss 2.71642062 - samples/sec: 132.93 - lr: 0.001953 2021-03-26 08:07:31,441 epoch 72 - iter 18/25 - loss 2.77816828 - samples/sec: 121.12 - lr: 0.001953 2021-03-26 08:07:32,400 epoch 72 - iter 20/25 - loss 2.77651918 - samples/sec: 133.69 - lr: 0.001953 2021-03-26 08:07:33,476 epoch 72 - iter 22/25 - loss 2.75537030 - samples/sec: 119.09 - lr: 0.001953 2021-03-26 08:07:34,482 epoch 72 - iter 24/25 - loss 2.74449572 - samples/sec: 127.49 - lr: 0.001953 2021-03-26 08:07:34,909 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:07:34,910 EPOCH 72 done: loss 2.6977 - lr 0.0019531 2021-03-26 08:07:35,639 DEV : loss 6.187828063964844 - score 0.9177 2021-03-26 08:07:35,662 BAD EPOCHS (no improvement): 2 2021-03-26 08:07:35,663 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:07:36,711 epoch 73 - iter 2/25 - loss 2.32378674 - samples/sec: 122.32 - lr: 0.001953 2021-03-26 08:07:37,684 epoch 73 - iter 4/25 - loss 2.64039457 - samples/sec: 131.82 - lr: 0.001953 2021-03-26 08:07:38,673 epoch 73 - iter 6/25 - loss 2.71906400 - samples/sec: 129.55 - lr: 0.001953 2021-03-26 08:07:39,577 epoch 73 - iter 8/25 - loss 2.76447648 - samples/sec: 141.91 - lr: 0.001953 2021-03-26 08:07:40,511 epoch 73 - iter 10/25 - loss 2.75839403 - samples/sec: 137.37 - lr: 0.001953 2021-03-26 08:07:41,443 epoch 73 - iter 12/25 - loss 2.68266495 - samples/sec: 137.44 - lr: 0.001953 2021-03-26 08:07:42,503 epoch 73 - iter 14/25 - loss 2.70562822 - samples/sec: 120.99 - lr: 0.001953 2021-03-26 08:07:43,378 epoch 73 - iter 16/25 - loss 2.63112544 - samples/sec: 146.44 - lr: 0.001953 2021-03-26 08:07:44,439 epoch 73 - iter 18/25 - loss 2.65143849 - samples/sec: 120.82 - lr: 0.001953 2021-03-26 08:07:45,419 epoch 73 - iter 20/25 - loss 2.64402987 - samples/sec: 130.89 - lr: 0.001953 2021-03-26 08:07:46,403 epoch 73 - iter 22/25 - loss 2.63108562 - samples/sec: 130.37 - lr: 0.001953 2021-03-26 08:07:47,380 epoch 73 - iter 24/25 - loss 2.65545537 - samples/sec: 131.20 - lr: 0.001953 2021-03-26 08:07:47,805 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:07:47,806 EPOCH 73 done: loss 2.6644 - lr 0.0019531 2021-03-26 08:07:48,505 DEV : loss 6.188020706176758 - score 0.9177 2021-03-26 08:07:48,528 BAD EPOCHS (no improvement): 3 2021-03-26 08:07:48,528 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:07:49,569 epoch 74 - iter 2/25 - loss 2.36066926 - samples/sec: 123.27 - lr: 0.001953 2021-03-26 08:07:50,563 epoch 74 - iter 4/25 - loss 2.84606272 - samples/sec: 128.86 - lr: 0.001953 2021-03-26 08:07:51,476 epoch 74 - iter 6/25 - loss 2.78872546 - samples/sec: 140.41 - lr: 0.001953 2021-03-26 08:07:52,518 epoch 74 - iter 8/25 - loss 2.83456469 - samples/sec: 123.13 - lr: 0.001953 2021-03-26 08:07:53,486 epoch 74 - iter 10/25 - loss 2.81370616 - samples/sec: 132.30 - lr: 0.001953 2021-03-26 08:07:54,524 epoch 74 - iter 12/25 - loss 2.74629303 - samples/sec: 123.59 - lr: 0.001953 2021-03-26 08:07:55,514 epoch 74 - iter 14/25 - loss 2.76375324 - samples/sec: 129.46 - lr: 0.001953 2021-03-26 08:07:56,490 epoch 74 - iter 16/25 - loss 2.70198634 - samples/sec: 131.30 - lr: 0.001953 2021-03-26 08:07:57,435 epoch 74 - iter 18/25 - loss 2.71382519 - samples/sec: 135.74 - lr: 0.001953 2021-03-26 08:07:58,519 epoch 74 - iter 20/25 - loss 2.77547271 - samples/sec: 118.26 - lr: 0.001953 2021-03-26 08:07:59,485 epoch 74 - iter 22/25 - loss 2.77600351 - samples/sec: 132.73 - lr: 0.001953 2021-03-26 08:08:00,557 epoch 74 - iter 24/25 - loss 2.77583278 - samples/sec: 119.49 - lr: 0.001953 2021-03-26 08:08:00,907 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:08:00,908 EPOCH 74 done: loss 2.7614 - lr 0.0019531 2021-03-26 08:08:01,613 DEV : loss 6.188040733337402 - score 0.9173 2021-03-26 08:08:01,636 BAD EPOCHS (no improvement): 4 2021-03-26 08:08:01,637 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:08:02,603 epoch 75 - iter 2/25 - loss 2.41651249 - samples/sec: 132.80 - lr: 0.000977 2021-03-26 08:08:03,626 epoch 75 - iter 4/25 - loss 2.59921521 - samples/sec: 125.28 - lr: 0.000977 2021-03-26 08:08:04,561 epoch 75 - iter 6/25 - loss 2.52756298 - samples/sec: 137.11 - lr: 0.000977 2021-03-26 08:08:05,673 epoch 75 - iter 8/25 - loss 2.70598119 - samples/sec: 115.34 - lr: 0.000977 2021-03-26 08:08:06,639 epoch 75 - iter 10/25 - loss 2.77697103 - samples/sec: 132.71 - lr: 0.000977 2021-03-26 08:08:07,572 epoch 75 - iter 12/25 - loss 2.78852485 - samples/sec: 137.43 - lr: 0.000977 2021-03-26 08:08:08,576 epoch 75 - iter 14/25 - loss 2.77042481 - samples/sec: 127.69 - lr: 0.000977 2021-03-26 08:08:09,662 epoch 75 - iter 16/25 - loss 2.73992635 - samples/sec: 118.18 - lr: 0.000977 2021-03-26 08:08:10,681 epoch 75 - iter 18/25 - loss 2.74360679 - samples/sec: 125.79 - lr: 0.000977 2021-03-26 08:08:11,594 epoch 75 - iter 20/25 - loss 2.77423202 - samples/sec: 140.46 - lr: 0.000977 2021-03-26 08:08:12,533 epoch 75 - iter 22/25 - loss 2.75198400 - samples/sec: 136.51 - lr: 0.000977 2021-03-26 08:08:13,464 epoch 75 - iter 24/25 - loss 2.76625696 - samples/sec: 137.79 - lr: 0.000977 2021-03-26 08:08:13,907 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:08:13,907 EPOCH 75 done: loss 2.7788 - lr 0.0009766 2021-03-26 08:08:14,631 DEV : loss 6.188265800476074 - score 0.9173 2021-03-26 08:08:14,654 BAD EPOCHS (no improvement): 1 2021-03-26 08:08:14,655 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:08:15,657 epoch 76 - iter 2/25 - loss 3.10712671 - samples/sec: 127.94 - lr: 0.000977 2021-03-26 08:08:16,666 epoch 76 - iter 4/25 - loss 3.13716650 - samples/sec: 127.06 - lr: 0.000977 2021-03-26 08:08:17,714 epoch 76 - iter 6/25 - loss 3.12287060 - samples/sec: 122.41 - lr: 0.000977 2021-03-26 08:08:18,718 epoch 76 - iter 8/25 - loss 2.90880463 - samples/sec: 127.72 - lr: 0.000977 2021-03-26 08:08:19,676 epoch 76 - iter 10/25 - loss 2.83946962 - samples/sec: 133.82 - lr: 0.000977 2021-03-26 08:08:20,712 epoch 76 - iter 12/25 - loss 2.73176597 - samples/sec: 123.82 - lr: 0.000977 2021-03-26 08:08:21,641 epoch 76 - iter 14/25 - loss 2.76485353 - samples/sec: 138.04 - lr: 0.000977 2021-03-26 08:08:22,618 epoch 76 - iter 16/25 - loss 2.73733263 - samples/sec: 131.22 - lr: 0.000977 2021-03-26 08:08:23,532 epoch 76 - iter 18/25 - loss 2.75554327 - samples/sec: 140.35 - lr: 0.000977 2021-03-26 08:08:24,485 epoch 76 - iter 20/25 - loss 2.71294504 - samples/sec: 134.43 - lr: 0.000977 2021-03-26 08:08:25,534 epoch 76 - iter 22/25 - loss 2.72319514 - samples/sec: 122.20 - lr: 0.000977 2021-03-26 08:08:26,518 epoch 76 - iter 24/25 - loss 2.73717165 - samples/sec: 130.24 - lr: 0.000977 2021-03-26 08:08:26,921 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:08:26,922 EPOCH 76 done: loss 2.7566 - lr 0.0009766 2021-03-26 08:08:27,667 DEV : loss 6.188060760498047 - score 0.9173 2021-03-26 08:08:27,689 BAD EPOCHS (no improvement): 2 2021-03-26 08:08:27,690 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:08:28,633 epoch 77 - iter 2/25 - loss 2.09625757 - samples/sec: 135.96 - lr: 0.000977 2021-03-26 08:08:29,545 epoch 77 - iter 4/25 - loss 2.58931965 - samples/sec: 140.55 - lr: 0.000977 2021-03-26 08:08:30,490 epoch 77 - iter 6/25 - loss 2.62068017 - samples/sec: 135.74 - lr: 0.000977 2021-03-26 08:08:31,398 epoch 77 - iter 8/25 - loss 2.60470769 - samples/sec: 141.00 - lr: 0.000977 2021-03-26 08:08:32,408 epoch 77 - iter 10/25 - loss 2.61880281 - samples/sec: 126.95 - lr: 0.000977 2021-03-26 08:08:33,335 epoch 77 - iter 12/25 - loss 2.62026229 - samples/sec: 138.23 - lr: 0.000977 2021-03-26 08:08:34,261 epoch 77 - iter 14/25 - loss 2.58505779 - samples/sec: 138.47 - lr: 0.000977 2021-03-26 08:08:35,268 epoch 77 - iter 16/25 - loss 2.56365578 - samples/sec: 127.37 - lr: 0.000977 2021-03-26 08:08:36,264 epoch 77 - iter 18/25 - loss 2.59711338 - samples/sec: 128.71 - lr: 0.000977 2021-03-26 08:08:37,287 epoch 77 - iter 20/25 - loss 2.63529403 - samples/sec: 125.35 - lr: 0.000977 2021-03-26 08:08:38,359 epoch 77 - iter 22/25 - loss 2.62909592 - samples/sec: 119.59 - lr: 0.000977 2021-03-26 08:08:39,330 epoch 77 - iter 24/25 - loss 2.66551463 - samples/sec: 132.00 - lr: 0.000977 2021-03-26 08:08:39,717 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:08:39,717 EPOCH 77 done: loss 2.6587 - lr 0.0009766 2021-03-26 08:08:40,429 DEV : loss 6.187152862548828 - score 0.9173 2021-03-26 08:08:40,452 BAD EPOCHS (no improvement): 3 2021-03-26 08:08:40,452 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:08:41,430 epoch 78 - iter 2/25 - loss 3.36305547 - samples/sec: 131.20 - lr: 0.000977 2021-03-26 08:08:42,365 epoch 78 - iter 4/25 - loss 2.86613524 - samples/sec: 137.06 - lr: 0.000977 2021-03-26 08:08:43,425 epoch 78 - iter 6/25 - loss 2.85387758 - samples/sec: 121.04 - lr: 0.000977 2021-03-26 08:08:44,449 epoch 78 - iter 8/25 - loss 2.87975633 - samples/sec: 125.30 - lr: 0.000977 2021-03-26 08:08:45,368 epoch 78 - iter 10/25 - loss 2.78941748 - samples/sec: 139.38 - lr: 0.000977 2021-03-26 08:08:46,296 epoch 78 - iter 12/25 - loss 2.83068611 - samples/sec: 138.15 - lr: 0.000977 2021-03-26 08:08:47,323 epoch 78 - iter 14/25 - loss 2.80439939 - samples/sec: 124.87 - lr: 0.000977 2021-03-26 08:08:48,368 epoch 78 - iter 16/25 - loss 2.82471086 - samples/sec: 122.62 - lr: 0.000977 2021-03-26 08:08:49,325 epoch 78 - iter 18/25 - loss 2.83826003 - samples/sec: 134.05 - lr: 0.000977 2021-03-26 08:08:50,269 epoch 78 - iter 20/25 - loss 2.84359343 - samples/sec: 135.87 - lr: 0.000977 2021-03-26 08:08:51,337 epoch 78 - iter 22/25 - loss 2.87817531 - samples/sec: 120.06 - lr: 0.000977 2021-03-26 08:08:52,373 epoch 78 - iter 24/25 - loss 2.82889307 - samples/sec: 123.81 - lr: 0.000977 2021-03-26 08:08:52,754 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:08:52,755 EPOCH 78 done: loss 2.8359 - lr 0.0009766 2021-03-26 08:08:53,457 DEV : loss 6.186553001403809 - score 0.9173 2021-03-26 08:08:53,481 BAD EPOCHS (no improvement): 4 2021-03-26 08:08:53,481 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:08:54,362 epoch 79 - iter 2/25 - loss 2.95909679 - samples/sec: 145.63 - lr: 0.000488 2021-03-26 08:08:55,289 epoch 79 - iter 4/25 - loss 2.81746089 - samples/sec: 138.31 - lr: 0.000488 2021-03-26 08:08:56,320 epoch 79 - iter 6/25 - loss 2.89012504 - samples/sec: 124.34 - lr: 0.000488 2021-03-26 08:08:57,322 epoch 79 - iter 8/25 - loss 2.91810098 - samples/sec: 127.94 - lr: 0.000488 2021-03-26 08:08:58,204 epoch 79 - iter 10/25 - loss 2.91408601 - samples/sec: 145.42 - lr: 0.000488 2021-03-26 08:08:59,138 epoch 79 - iter 12/25 - loss 2.82381584 - samples/sec: 137.23 - lr: 0.000488 2021-03-26 08:09:00,065 epoch 79 - iter 14/25 - loss 2.79630446 - samples/sec: 138.33 - lr: 0.000488 2021-03-26 08:09:01,051 epoch 79 - iter 16/25 - loss 2.77844489 - samples/sec: 129.98 - lr: 0.000488 2021-03-26 08:09:02,047 epoch 79 - iter 18/25 - loss 2.81718817 - samples/sec: 128.72 - lr: 0.000488 2021-03-26 08:09:03,127 epoch 79 - iter 20/25 - loss 2.85397427 - samples/sec: 118.83 - lr: 0.000488 2021-03-26 08:09:04,159 epoch 79 - iter 22/25 - loss 2.82800953 - samples/sec: 124.13 - lr: 0.000488 2021-03-26 08:09:05,137 epoch 79 - iter 24/25 - loss 2.81754422 - samples/sec: 131.10 - lr: 0.000488 2021-03-26 08:09:05,617 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:09:05,618 EPOCH 79 done: loss 2.8396 - lr 0.0004883 2021-03-26 08:09:06,327 DEV : loss 6.186826229095459 - score 0.9173 2021-03-26 08:09:06,350 BAD EPOCHS (no improvement): 1 2021-03-26 08:09:06,351 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:09:07,377 epoch 80 - iter 2/25 - loss 2.64280188 - samples/sec: 125.05 - lr: 0.000488 2021-03-26 08:09:08,318 epoch 80 - iter 4/25 - loss 2.53505731 - samples/sec: 136.12 - lr: 0.000488 2021-03-26 08:09:09,308 epoch 80 - iter 6/25 - loss 2.48866999 - samples/sec: 129.48 - lr: 0.000488 2021-03-26 08:09:10,281 epoch 80 - iter 8/25 - loss 2.47880760 - samples/sec: 131.83 - lr: 0.000488 2021-03-26 08:09:11,304 epoch 80 - iter 10/25 - loss 2.54190221 - samples/sec: 125.28 - lr: 0.000488 2021-03-26 08:09:12,611 epoch 80 - iter 12/25 - loss 2.59663198 - samples/sec: 97.98 - lr: 0.000488 2021-03-26 08:09:13,537 epoch 80 - iter 14/25 - loss 2.53316179 - samples/sec: 138.56 - lr: 0.000488 2021-03-26 08:09:14,619 epoch 80 - iter 16/25 - loss 2.58981419 - samples/sec: 118.51 - lr: 0.000488 2021-03-26 08:09:15,665 epoch 80 - iter 18/25 - loss 2.62501939 - samples/sec: 122.58 - lr: 0.000488 2021-03-26 08:09:16,645 epoch 80 - iter 20/25 - loss 2.58662164 - samples/sec: 130.87 - lr: 0.000488 2021-03-26 08:09:17,617 epoch 80 - iter 22/25 - loss 2.58739001 - samples/sec: 131.94 - lr: 0.000488 2021-03-26 08:09:18,684 epoch 80 - iter 24/25 - loss 2.58220779 - samples/sec: 120.15 - lr: 0.000488 2021-03-26 08:09:19,086 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:09:19,087 EPOCH 80 done: loss 2.6174 - lr 0.0004883 2021-03-26 08:09:19,790 DEV : loss 6.187230110168457 - score 0.9173 2021-03-26 08:09:19,812 BAD EPOCHS (no improvement): 2 2021-03-26 08:09:19,813 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:09:20,786 epoch 81 - iter 2/25 - loss 2.75670075 - samples/sec: 131.78 - lr: 0.000488 2021-03-26 08:09:21,781 epoch 81 - iter 4/25 - loss 3.00184679 - samples/sec: 128.75 - lr: 0.000488 2021-03-26 08:09:22,720 epoch 81 - iter 6/25 - loss 2.95591950 - samples/sec: 136.62 - lr: 0.000488 2021-03-26 08:09:23,666 epoch 81 - iter 8/25 - loss 2.84314916 - samples/sec: 135.57 - lr: 0.000488 2021-03-26 08:09:24,686 epoch 81 - iter 10/25 - loss 2.74002171 - samples/sec: 125.75 - lr: 0.000488 2021-03-26 08:09:25,656 epoch 81 - iter 12/25 - loss 2.79702922 - samples/sec: 132.05 - lr: 0.000488 2021-03-26 08:09:26,761 epoch 81 - iter 14/25 - loss 2.86702430 - samples/sec: 115.98 - lr: 0.000488 2021-03-26 08:09:27,712 epoch 81 - iter 16/25 - loss 2.88608465 - samples/sec: 134.76 - lr: 0.000488 2021-03-26 08:09:28,614 epoch 81 - iter 18/25 - loss 2.89608358 - samples/sec: 142.11 - lr: 0.000488 2021-03-26 08:09:29,537 epoch 81 - iter 20/25 - loss 2.86903533 - samples/sec: 138.97 - lr: 0.000488 2021-03-26 08:09:30,571 epoch 81 - iter 22/25 - loss 2.82412986 - samples/sec: 123.98 - lr: 0.000488 2021-03-26 08:09:31,647 epoch 81 - iter 24/25 - loss 2.81362030 - samples/sec: 119.07 - lr: 0.000488 2021-03-26 08:09:32,038 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:09:32,039 EPOCH 81 done: loss 2.8294 - lr 0.0004883 2021-03-26 08:09:32,758 DEV : loss 6.185996055603027 - score 0.9173 2021-03-26 08:09:32,782 BAD EPOCHS (no improvement): 3 2021-03-26 08:09:32,783 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:09:33,724 epoch 82 - iter 2/25 - loss 2.84042430 - samples/sec: 136.20 - lr: 0.000488 2021-03-26 08:09:34,657 epoch 82 - iter 4/25 - loss 2.65296417 - samples/sec: 137.43 - lr: 0.000488 2021-03-26 08:09:35,609 epoch 82 - iter 6/25 - loss 2.70227369 - samples/sec: 134.63 - lr: 0.000488 2021-03-26 08:09:36,600 epoch 82 - iter 8/25 - loss 2.78364819 - samples/sec: 129.28 - lr: 0.000488 2021-03-26 08:09:37,611 epoch 82 - iter 10/25 - loss 2.66723704 - samples/sec: 126.85 - lr: 0.000488 2021-03-26 08:09:38,583 epoch 82 - iter 12/25 - loss 2.66724110 - samples/sec: 131.91 - lr: 0.000488 2021-03-26 08:09:39,528 epoch 82 - iter 14/25 - loss 2.66870518 - samples/sec: 135.71 - lr: 0.000488 2021-03-26 08:09:40,516 epoch 82 - iter 16/25 - loss 2.75237661 - samples/sec: 129.75 - lr: 0.000488 2021-03-26 08:09:41,504 epoch 82 - iter 18/25 - loss 2.71161309 - samples/sec: 129.70 - lr: 0.000488 2021-03-26 08:09:42,489 epoch 82 - iter 20/25 - loss 2.73067624 - samples/sec: 130.12 - lr: 0.000488 2021-03-26 08:09:43,507 epoch 82 - iter 22/25 - loss 2.74962510 - samples/sec: 125.94 - lr: 0.000488 2021-03-26 08:09:44,513 epoch 82 - iter 24/25 - loss 2.71701292 - samples/sec: 127.32 - lr: 0.000488 2021-03-26 08:09:44,967 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:09:44,968 EPOCH 82 done: loss 2.7060 - lr 0.0004883 2021-03-26 08:09:45,706 DEV : loss 6.185783386230469 - score 0.9173 2021-03-26 08:09:45,729 BAD EPOCHS (no improvement): 4 2021-03-26 08:09:45,730 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:09:46,716 epoch 83 - iter 2/25 - loss 2.85962808 - samples/sec: 130.18 - lr: 0.000244 2021-03-26 08:09:47,740 epoch 83 - iter 4/25 - loss 2.75480992 - samples/sec: 125.14 - lr: 0.000244 2021-03-26 08:09:48,780 epoch 83 - iter 6/25 - loss 2.68297470 - samples/sec: 123.23 - lr: 0.000244 2021-03-26 08:09:49,741 epoch 83 - iter 8/25 - loss 2.79082182 - samples/sec: 133.45 - lr: 0.000244 2021-03-26 08:09:50,686 epoch 83 - iter 10/25 - loss 2.74289029 - samples/sec: 135.70 - lr: 0.000244 2021-03-26 08:09:51,599 epoch 83 - iter 12/25 - loss 2.74736812 - samples/sec: 140.49 - lr: 0.000244 2021-03-26 08:09:52,638 epoch 83 - iter 14/25 - loss 2.69184843 - samples/sec: 123.28 - lr: 0.000244 2021-03-26 08:09:53,648 epoch 83 - iter 16/25 - loss 2.75959721 - samples/sec: 126.95 - lr: 0.000244 2021-03-26 08:09:54,608 epoch 83 - iter 18/25 - loss 2.71115977 - samples/sec: 133.53 - lr: 0.000244 2021-03-26 08:09:55,878 epoch 83 - iter 20/25 - loss 2.74285476 - samples/sec: 100.90 - lr: 0.000244 2021-03-26 08:09:56,819 epoch 83 - iter 22/25 - loss 2.72497958 - samples/sec: 136.33 - lr: 0.000244 2021-03-26 08:09:57,716 epoch 83 - iter 24/25 - loss 2.67853123 - samples/sec: 142.90 - lr: 0.000244 2021-03-26 08:09:58,148 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:09:58,150 EPOCH 83 done: loss 2.6657 - lr 0.0002441 2021-03-26 08:09:58,863 DEV : loss 6.185376167297363 - score 0.9173 2021-03-26 08:09:58,884 BAD EPOCHS (no improvement): 1 2021-03-26 08:09:58,885 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:09:59,810 epoch 84 - iter 2/25 - loss 2.58333921 - samples/sec: 138.61 - lr: 0.000244 2021-03-26 08:10:00,749 epoch 84 - iter 4/25 - loss 2.76761711 - samples/sec: 136.47 - lr: 0.000244 2021-03-26 08:10:01,781 epoch 84 - iter 6/25 - loss 2.71056978 - samples/sec: 124.26 - lr: 0.000244 2021-03-26 08:10:02,830 epoch 84 - iter 8/25 - loss 2.72177193 - samples/sec: 122.24 - lr: 0.000244 2021-03-26 08:10:03,801 epoch 84 - iter 10/25 - loss 2.75179241 - samples/sec: 132.03 - lr: 0.000244 2021-03-26 08:10:04,863 epoch 84 - iter 12/25 - loss 2.76484060 - samples/sec: 120.77 - lr: 0.000244 2021-03-26 08:10:05,896 epoch 84 - iter 14/25 - loss 2.70445476 - samples/sec: 124.09 - lr: 0.000244 2021-03-26 08:10:07,880 epoch 84 - iter 16/25 - loss 2.75915471 - samples/sec: 64.55 - lr: 0.000244 2021-03-26 08:10:08,782 epoch 84 - iter 18/25 - loss 2.67348589 - samples/sec: 142.25 - lr: 0.000244 2021-03-26 08:10:09,901 epoch 84 - iter 20/25 - loss 2.65599443 - samples/sec: 114.49 - lr: 0.000244 2021-03-26 08:10:10,908 epoch 84 - iter 22/25 - loss 2.68863658 - samples/sec: 127.38 - lr: 0.000244 2021-03-26 08:10:11,912 epoch 84 - iter 24/25 - loss 2.72180267 - samples/sec: 127.64 - lr: 0.000244 2021-03-26 08:10:12,348 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:10:12,348 EPOCH 84 done: loss 2.7303 - lr 0.0002441 2021-03-26 08:10:13,110 DEV : loss 6.185603618621826 - score 0.9173 2021-03-26 08:10:13,142 BAD EPOCHS (no improvement): 2 2021-03-26 08:10:13,143 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:10:14,276 epoch 85 - iter 2/25 - loss 2.66001129 - samples/sec: 113.17 - lr: 0.000244 2021-03-26 08:10:15,390 epoch 85 - iter 4/25 - loss 2.54920405 - samples/sec: 115.13 - lr: 0.000244 2021-03-26 08:10:16,350 epoch 85 - iter 6/25 - loss 2.38967019 - samples/sec: 133.48 - lr: 0.000244 2021-03-26 08:10:17,285 epoch 85 - iter 8/25 - loss 2.52254377 - samples/sec: 137.09 - lr: 0.000244 2021-03-26 08:10:18,137 epoch 85 - iter 10/25 - loss 2.48643364 - samples/sec: 150.34 - lr: 0.000244 2021-03-26 08:10:19,155 epoch 85 - iter 12/25 - loss 2.50120088 - samples/sec: 125.91 - lr: 0.000244 2021-03-26 08:10:20,085 epoch 85 - iter 14/25 - loss 2.58942009 - samples/sec: 138.04 - lr: 0.000244 2021-03-26 08:10:21,044 epoch 85 - iter 16/25 - loss 2.71086637 - samples/sec: 133.54 - lr: 0.000244 2021-03-26 08:10:22,044 epoch 85 - iter 18/25 - loss 2.73113488 - samples/sec: 128.27 - lr: 0.000244 2021-03-26 08:10:23,144 epoch 85 - iter 20/25 - loss 2.73964195 - samples/sec: 116.51 - lr: 0.000244 2021-03-26 08:10:24,110 epoch 85 - iter 22/25 - loss 2.72373083 - samples/sec: 132.59 - lr: 0.000244 2021-03-26 08:10:25,135 epoch 85 - iter 24/25 - loss 2.71947557 - samples/sec: 125.07 - lr: 0.000244 2021-03-26 08:10:25,558 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:10:25,559 EPOCH 85 done: loss 2.6878 - lr 0.0002441 2021-03-26 08:10:26,268 DEV : loss 6.186032772064209 - score 0.9173 2021-03-26 08:10:26,287 BAD EPOCHS (no improvement): 3 2021-03-26 08:10:26,288 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:10:27,381 epoch 86 - iter 2/25 - loss 2.83442032 - samples/sec: 117.18 - lr: 0.000244 2021-03-26 08:10:28,427 epoch 86 - iter 4/25 - loss 3.07467341 - samples/sec: 122.58 - lr: 0.000244 2021-03-26 08:10:29,412 epoch 86 - iter 6/25 - loss 3.05214135 - samples/sec: 130.16 - lr: 0.000244 2021-03-26 08:10:30,469 epoch 86 - iter 8/25 - loss 2.90568414 - samples/sec: 121.27 - lr: 0.000244 2021-03-26 08:10:31,431 epoch 86 - iter 10/25 - loss 2.91739962 - samples/sec: 133.22 - lr: 0.000244 2021-03-26 08:10:32,462 epoch 86 - iter 12/25 - loss 2.81895500 - samples/sec: 124.39 - lr: 0.000244 2021-03-26 08:10:33,395 epoch 86 - iter 14/25 - loss 2.82476473 - samples/sec: 137.33 - lr: 0.000244 2021-03-26 08:10:34,277 epoch 86 - iter 16/25 - loss 2.78650875 - samples/sec: 145.50 - lr: 0.000244 2021-03-26 08:10:35,278 epoch 86 - iter 18/25 - loss 2.71929197 - samples/sec: 128.04 - lr: 0.000244 2021-03-26 08:10:36,220 epoch 86 - iter 20/25 - loss 2.72559129 - samples/sec: 136.11 - lr: 0.000244 2021-03-26 08:10:37,262 epoch 86 - iter 22/25 - loss 2.68120156 - samples/sec: 123.07 - lr: 0.000244 2021-03-26 08:10:38,257 epoch 86 - iter 24/25 - loss 2.71939340 - samples/sec: 128.85 - lr: 0.000244 2021-03-26 08:10:38,695 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:10:38,696 EPOCH 86 done: loss 2.7590 - lr 0.0002441 2021-03-26 08:10:39,418 DEV : loss 6.185995101928711 - score 0.9173 2021-03-26 08:10:39,441 BAD EPOCHS (no improvement): 4 2021-03-26 08:10:39,441 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:10:40,535 epoch 87 - iter 2/25 - loss 3.12590420 - samples/sec: 117.25 - lr: 0.000122 2021-03-26 08:10:41,574 epoch 87 - iter 4/25 - loss 3.28620195 - samples/sec: 123.43 - lr: 0.000122 2021-03-26 08:10:42,655 epoch 87 - iter 6/25 - loss 3.30520022 - samples/sec: 118.61 - lr: 0.000122 2021-03-26 08:10:43,690 epoch 87 - iter 8/25 - loss 3.32119605 - samples/sec: 123.86 - lr: 0.000122 2021-03-26 08:10:44,668 epoch 87 - iter 10/25 - loss 3.14544759 - samples/sec: 131.02 - lr: 0.000122 2021-03-26 08:10:45,613 epoch 87 - iter 12/25 - loss 3.15682425 - samples/sec: 135.74 - lr: 0.000122 2021-03-26 08:10:46,555 epoch 87 - iter 14/25 - loss 3.06249428 - samples/sec: 136.06 - lr: 0.000122 2021-03-26 08:10:47,534 epoch 87 - iter 16/25 - loss 3.03412677 - samples/sec: 130.96 - lr: 0.000122 2021-03-26 08:10:48,552 epoch 87 - iter 18/25 - loss 2.96904975 - samples/sec: 125.98 - lr: 0.000122 2021-03-26 08:10:49,470 epoch 87 - iter 20/25 - loss 2.94991877 - samples/sec: 139.62 - lr: 0.000122 2021-03-26 08:10:50,427 epoch 87 - iter 22/25 - loss 2.88184339 - samples/sec: 134.01 - lr: 0.000122 2021-03-26 08:10:51,340 epoch 87 - iter 24/25 - loss 2.85626723 - samples/sec: 140.36 - lr: 0.000122 2021-03-26 08:10:51,784 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:10:51,784 EPOCH 87 done: loss 2.8113 - lr 0.0001221 2021-03-26 08:10:52,491 DEV : loss 6.186027526855469 - score 0.9173 2021-03-26 08:10:52,513 BAD EPOCHS (no improvement): 1 2021-03-26 08:10:52,514 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:10:53,566 epoch 88 - iter 2/25 - loss 2.68268955 - samples/sec: 121.82 - lr: 0.000122 2021-03-26 08:10:54,524 epoch 88 - iter 4/25 - loss 2.62284821 - samples/sec: 133.81 - lr: 0.000122 2021-03-26 08:10:55,522 epoch 88 - iter 6/25 - loss 2.53554523 - samples/sec: 128.54 - lr: 0.000122 2021-03-26 08:10:56,537 epoch 88 - iter 8/25 - loss 2.52243441 - samples/sec: 126.30 - lr: 0.000122 2021-03-26 08:10:57,455 epoch 88 - iter 10/25 - loss 2.47092564 - samples/sec: 139.73 - lr: 0.000122 2021-03-26 08:10:58,376 epoch 88 - iter 12/25 - loss 2.52259127 - samples/sec: 139.19 - lr: 0.000122 2021-03-26 08:10:59,424 epoch 88 - iter 14/25 - loss 2.52517985 - samples/sec: 122.39 - lr: 0.000122 2021-03-26 08:11:00,399 epoch 88 - iter 16/25 - loss 2.61265714 - samples/sec: 131.46 - lr: 0.000122 2021-03-26 08:11:01,390 epoch 88 - iter 18/25 - loss 2.59087091 - samples/sec: 129.34 - lr: 0.000122 2021-03-26 08:11:02,398 epoch 88 - iter 20/25 - loss 2.61029449 - samples/sec: 127.06 - lr: 0.000122 2021-03-26 08:11:03,410 epoch 88 - iter 22/25 - loss 2.63463608 - samples/sec: 126.76 - lr: 0.000122 2021-03-26 08:11:04,419 epoch 88 - iter 24/25 - loss 2.66323133 - samples/sec: 127.02 - lr: 0.000122 2021-03-26 08:11:04,836 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:04,837 EPOCH 88 done: loss 2.6686 - lr 0.0001221 2021-03-26 08:11:05,616 DEV : loss 6.185815811157227 - score 0.9173 2021-03-26 08:11:05,639 BAD EPOCHS (no improvement): 2 2021-03-26 08:11:05,640 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:06,667 epoch 89 - iter 2/25 - loss 2.97890735 - samples/sec: 124.79 - lr: 0.000122 2021-03-26 08:11:07,650 epoch 89 - iter 4/25 - loss 2.86776453 - samples/sec: 130.51 - lr: 0.000122 2021-03-26 08:11:08,646 epoch 89 - iter 6/25 - loss 2.75692447 - samples/sec: 128.62 - lr: 0.000122 2021-03-26 08:11:09,654 epoch 89 - iter 8/25 - loss 2.83591112 - samples/sec: 127.24 - lr: 0.000122 2021-03-26 08:11:10,678 epoch 89 - iter 10/25 - loss 2.87064221 - samples/sec: 125.15 - lr: 0.000122 2021-03-26 08:11:11,616 epoch 89 - iter 12/25 - loss 2.92312527 - samples/sec: 136.58 - lr: 0.000122 2021-03-26 08:11:12,512 epoch 89 - iter 14/25 - loss 2.88578798 - samples/sec: 143.21 - lr: 0.000122 2021-03-26 08:11:13,489 epoch 89 - iter 16/25 - loss 2.84937823 - samples/sec: 131.21 - lr: 0.000122 2021-03-26 08:11:14,552 epoch 89 - iter 18/25 - loss 2.79902322 - samples/sec: 120.52 - lr: 0.000122 2021-03-26 08:11:15,521 epoch 89 - iter 20/25 - loss 2.76434165 - samples/sec: 132.20 - lr: 0.000122 2021-03-26 08:11:16,547 epoch 89 - iter 22/25 - loss 2.79289429 - samples/sec: 125.08 - lr: 0.000122 2021-03-26 08:11:17,647 epoch 89 - iter 24/25 - loss 2.79572803 - samples/sec: 117.01 - lr: 0.000122 2021-03-26 08:11:18,005 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:18,006 EPOCH 89 done: loss 2.7775 - lr 0.0001221 2021-03-26 08:11:18,728 DEV : loss 6.185563564300537 - score 0.9173 2021-03-26 08:11:18,750 BAD EPOCHS (no improvement): 3 2021-03-26 08:11:18,751 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:19,745 epoch 90 - iter 2/25 - loss 2.81466031 - samples/sec: 129.08 - lr: 0.000122 2021-03-26 08:11:20,923 epoch 90 - iter 4/25 - loss 2.73348701 - samples/sec: 108.76 - lr: 0.000122 2021-03-26 08:11:21,877 epoch 90 - iter 6/25 - loss 2.85376092 - samples/sec: 134.40 - lr: 0.000122 2021-03-26 08:11:22,931 epoch 90 - iter 8/25 - loss 2.80158153 - samples/sec: 121.54 - lr: 0.000122 2021-03-26 08:11:23,805 epoch 90 - iter 10/25 - loss 2.70515602 - samples/sec: 146.80 - lr: 0.000122 2021-03-26 08:11:24,826 epoch 90 - iter 12/25 - loss 2.72229065 - samples/sec: 125.47 - lr: 0.000122 2021-03-26 08:11:25,868 epoch 90 - iter 14/25 - loss 2.70650598 - samples/sec: 123.07 - lr: 0.000122 2021-03-26 08:11:26,812 epoch 90 - iter 16/25 - loss 2.66336836 - samples/sec: 135.89 - lr: 0.000122 2021-03-26 08:11:27,916 epoch 90 - iter 18/25 - loss 2.60368521 - samples/sec: 115.98 - lr: 0.000122 2021-03-26 08:11:28,785 epoch 90 - iter 20/25 - loss 2.59294172 - samples/sec: 147.67 - lr: 0.000122 2021-03-26 08:11:29,915 epoch 90 - iter 22/25 - loss 2.57639271 - samples/sec: 113.35 - lr: 0.000122 2021-03-26 08:11:30,898 epoch 90 - iter 24/25 - loss 2.62383321 - samples/sec: 130.41 - lr: 0.000122 2021-03-26 08:11:31,369 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:31,369 EPOCH 90 done: loss 2.6541 - lr 0.0001221 2021-03-26 08:11:32,111 DEV : loss 6.185521602630615 - score 0.9173 2021-03-26 08:11:32,134 BAD EPOCHS (no improvement): 4 2021-03-26 08:11:32,135 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:32,135 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:32,135 learning rate too small - quitting training! 2021-03-26 08:11:32,136 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:41,404 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:41,405 Testing using best model ... 2021-03-26 08:11:41,406 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260747/best-model.pt 2021-03-26 08:11:48,677 0.911 2021-03-26 08:11:48,678 Results: - F-score (micro): 0.9077 - F-score (macro): 0.6024 - Accuracy (incl. no class): 0.911 By class: precision recall f1-score support NOUN 0.9227 0.9363 0.9294 408 NUM 0.9333 0.7778 0.8485 18 ADJ 0.8913 0.8367 0.8632 98 CCONJ 1.0000 0.9342 0.9660 76 PRON 0.9609 0.9773 0.9690 176 ADV 0.8776 0.8350 0.8557 103 PART 0.9554 0.9772 0.9661 219 VERB 0.9375 0.9507 0.9441 142 ADP 0.9811 0.9541 0.9674 109 DET 0.9206 0.9667 0.9431 60 PUNCT 1.0000 1.0000 1.0000 30 SCONJ 0.8788 0.9355 0.9062 31 PROPN 0.8077 0.8400 0.8235 25 AUX 0.9250 0.9487 0.9367 39 INTJ 0.9412 0.9412 0.9412 17 DET+NOUN 0.9167 0.9429 0.9296 70 CONJ+DET+NOUN 1.0000 0.8333 0.9091 6 CONJ+DET+ADJ 0.0000 1.0000 0.0000 0 PREP+DET+NOUN+NSUFF 0.3333 0.3333 0.3333 3 PREP+NOUN 0.7500 0.8333 0.7895 18 PREP+DET+NOUN 0.7778 1.0000 0.8750 7 PUNC 1.0000 1.0000 1.0000 148 DET+NOUN+NSUFF 0.8846 0.8846 0.8846 26 PROG_PART+V 0.8140 0.9459 0.8750 37 PART+PRON 1.0000 0.8824 0.9375 17 V+PRON 0.8462 0.7586 0.8000 58 PREP+PART+PRON 1.0000 1.0000 1.0000 4 URL 1.0000 1.0000 1.0000 3 EOS 1.0000 1.0000 1.0000 70 PRON+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 4 PRON+DET+NOUN 0.0000 1.0000 0.0000 0 PROG_PART+V+PREP 1.0000 0.0000 0.0000 1 PREP 0.9545 0.9844 0.9692 64 CONJ+PART 1.0000 0.9333 0.9655 15 CONJ+PROG_PART+V 0.6667 0.6667 0.6667 3 CONJ+NOUN 0.7273 0.8000 0.7619 10 PROG_PART+V+PRON 0.8421 0.8889 0.8649 18 PREP+PRON 0.7931 0.9583 0.8679 24 ADJ+NSUFF 0.8571 0.7317 0.7895 41 NOUN+NSUFF 0.7353 0.8772 0.8000 57 V 0.8532 0.8942 0.8732 104 PART+V+NEG_PART 0.5000 0.6667 0.5714 3 NOUN+PRON 0.8841 0.8243 0.8531 74 PREP+NOUN+NSUFF+PRON 1.0000 0.2500 0.4000 4 PREP+NOUN+PRON 0.0000 0.0000 0.0000 2 FUT_PART+V 1.0000 0.6667 0.8000 12 FOREIGN 1.0000 0.6667 0.8000 3 MENTION 0.9565 1.0000 0.9778 22 CONJ+ADJ 0.0000 0.0000 0.0000 2 CONJ+PRON 1.0000 1.0000 1.0000 7 ADJ+PRON 0.5714 0.8000 0.6667 5 HASH 1.0000 0.9231 0.9600 13 CONJ 0.9722 0.9722 0.9722 36 DET+ADJ 1.0000 0.6250 0.7692 8 DET+NUM 0.0000 1.0000 0.0000 0 ADJ+PREP+PRON 0.0000 0.0000 0.0000 1 PROG_PART+V+PREP+PRON 0.0000 1.0000 0.0000 0 NOUN+NSUFF+PRON 0.5238 0.9167 0.6667 12 PREP+NOUN+NSUFF 0.0000 0.0000 0.0000 4 PART+V 1.0000 0.0000 0.0000 2 EMOT 0.9500 1.0000 0.9744 19 NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 V+PRON+PRON 1.0000 0.4000 0.5714 10 FUT_PART 1.0000 1.0000 1.0000 11 DET+ADJ+NSUFF 0.5000 0.5000 0.5000 2 PREP+DET+NUM+NSUFF 1.0000 0.0000 0.0000 1 NOUN+CASE 0.8000 0.8000 0.8000 5 PREP+PART 1.0000 1.0000 1.0000 1 PART+NOUN 0.5000 0.5000 0.5000 2 NOUN+PRON+PRON 1.0000 0.0000 0.0000 2 PART+NOUN+PRON 1.0000 0.0000 0.0000 2 CONJ+FUT_PART+V 1.0000 0.0000 0.0000 2 FUT_PART+V+PRON 0.4000 1.0000 0.5714 2 CONJ+V 0.7778 1.0000 0.8750 7 CONJ+V+PREP+PRON 0.5000 1.0000 0.6667 1 PROG_PART+V+NEG_PART 1.0000 1.0000 1.0000 1 CONJ+PREP+PRON 1.0000 1.0000 1.0000 1 ADJ+NSUFF+PRON 1.0000 0.0000 0.0000 2 PREP+DET 1.0000 0.0000 0.0000 1 CONJ+NOUN+PRON 0.5000 1.0000 0.6667 2 CONJ+PART+V+PRON 1.0000 0.0000 0.0000 1 V+PREP+PRON 0.3333 0.3333 0.3333 3 PART+V+PRON+NEG_PART 0.3333 1.0000 0.5000 2 V+NOUN 1.0000 0.0000 0.0000 1 CONJ+NOUN+NSUFF 1.0000 1.0000 1.0000 2 ADV+NSUFF 1.0000 1.0000 1.0000 2 PART+V+PRON+PRON+NEG_PART 1.0000 1.0000 1.0000 1 PART+PREP+PRON+NEG_PART 1.0000 0.3333 0.5000 3 CONJ+ADV 1.0000 1.0000 1.0000 2 NOUN+CASE+PRON 1.0000 0.0000 0.0000 1 PART+PREP+NEG_PART 1.0000 1.0000 1.0000 3 PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 CONJ+PROG_PART+V+PRON 1.0000 1.0000 1.0000 1 CONJ+V+PRON 1.0000 1.0000 1.0000 1 PART+PART 1.0000 0.0000 0.0000 1 CONJ+ADJ+NSUFF 1.0000 0.0000 0.0000 1 CONJ+PREP 1.0000 1.0000 1.0000 1 NUM+NSUFF 1.0000 0.0000 0.0000 1 FUT_PART+V+PRON+PRON 1.0000 0.0000 0.0000 2 FUT_PART+V+PREP+PRON 0.0000 1.0000 0.0000 0 ADJ+CASE+PREP 1.0000 0.0000 0.0000 1 PREP+ADV 1.0000 0.0000 0.0000 1 PREP+DET+ADJ 0.0000 1.0000 0.0000 0 NOUN+PRON+NEG_PART 1.0000 0.0000 0.0000 1 micro avg 0.9080 0.9074 0.9077 2677 macro avg 0.8047 0.6744 0.6024 2677 weighted avg 0.9161 0.9074 0.9036 2677 2021-03-26 08:11:48,678 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:48,678 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:52,326 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC 2021-03-26 08:11:52,327 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu 2021-03-26 08:11:52,327 Dev: None 2021-03-26 08:11:52,327 Test: None 2021-03-26 08:11:54,235 Reading data from ../../Datasets_adhoc/UD_MADAR 2021-03-26 08:11:54,235 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu 2021-03-26 08:11:54,236 Dev: None 2021-03-26 08:11:54,236 Test: None 2021-03-26 08:11:54,280 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 08:11:54,281 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt 2021-03-26 08:11:54,281 Dev: None 2021-03-26 08:11:54,282 Test: None 2021-03-26 08:11:54,437 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 08:11:54,437 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt 2021-03-26 08:11:54,438 Dev: None 2021-03-26 08:11:54,438 Test: None 2021-03-26 08:11:54,605 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 08:11:54,605 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt 2021-03-26 08:11:54,606 Dev: None 2021-03-26 08:11:54,606 Test: None 2021-03-26 08:11:54,757 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 08:11:54,757 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt 2021-03-26 08:11:54,758 Dev: None 2021-03-26 08:11:54,758 Test: None 2021-03-26 08:11:54,897 Filtering long sentences 2021-03-26 08:11:54,935 MultiCorpus: 1574 train + 176 dev + 194 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences 2021-03-26 08:11:55,329 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:55,330 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): WordEmbeddings('ar') (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) (list_embedding_2): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=206, bias=True) (beta): 1.0 (weights): None (weight_tensor) None )" 2021-03-26 08:11:55,330 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:55,330 Corpus: "MultiCorpus: 1574 train + 176 dev + 194 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" 2021-03-26 08:11:55,331 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:55,331 Parameters: 2021-03-26 08:11:55,331 - learning_rate: "0.5" 2021-03-26 08:11:55,332 - mini_batch_size: "64" 2021-03-26 08:11:55,332 - patience: "3" 2021-03-26 08:11:55,332 - anneal_factor: "0.5" 2021-03-26 08:11:55,333 - max_epochs: "150" 2021-03-26 08:11:55,333 - shuffle: "True" 2021-03-26 08:11:55,333 - train_with_dev: "False" 2021-03-26 08:11:55,334 - batch_growth_annealing: "False" 2021-03-26 08:11:55,334 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:55,334 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260811" 2021-03-26 08:11:55,335 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:55,335 Device: cuda:0 2021-03-26 08:11:55,335 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:55,336 Embeddings storage mode: cpu 2021-03-26 08:11:55,338 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:11:57,116 epoch 1 - iter 2/25 - loss 82.69540787 - samples/sec: 72.03 - lr: 0.500000 2021-03-26 08:11:58,395 epoch 1 - iter 4/25 - loss 85.90545464 - samples/sec: 100.22 - lr: 0.500000 2021-03-26 08:11:59,684 epoch 1 - iter 6/25 - loss 79.33684158 - samples/sec: 99.42 - lr: 0.500000 2021-03-26 08:12:00,911 epoch 1 - iter 8/25 - loss 74.56877232 - samples/sec: 104.38 - lr: 0.500000 2021-03-26 08:12:02,163 epoch 1 - iter 10/25 - loss 73.09503098 - samples/sec: 102.38 - lr: 0.500000 2021-03-26 08:12:03,374 epoch 1 - iter 12/25 - loss 70.68474229 - samples/sec: 105.82 - lr: 0.500000 2021-03-26 08:12:04,713 epoch 1 - iter 14/25 - loss 69.38999694 - samples/sec: 95.69 - lr: 0.500000 2021-03-26 08:12:05,969 epoch 1 - iter 16/25 - loss 68.14279413 - samples/sec: 102.00 - lr: 0.500000 2021-03-26 08:12:07,290 epoch 1 - iter 18/25 - loss 67.24124633 - samples/sec: 97.01 - lr: 0.500000 2021-03-26 08:12:08,635 epoch 1 - iter 20/25 - loss 65.85308533 - samples/sec: 95.27 - lr: 0.500000 2021-03-26 08:12:10,008 epoch 1 - iter 22/25 - loss 64.25220333 - samples/sec: 93.28 - lr: 0.500000 2021-03-26 08:12:11,403 epoch 1 - iter 24/25 - loss 62.78505087 - samples/sec: 91.84 - lr: 0.500000 2021-03-26 08:12:11,908 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:12:11,909 EPOCH 1 done: loss 61.9633 - lr 0.5000000 2021-03-26 08:12:13,113 DEV : loss 43.57872772216797 - score 0.2937 2021-03-26 08:12:13,135 BAD EPOCHS (no improvement): 0 2021-03-26 08:12:22,390 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:12:23,329 epoch 2 - iter 2/25 - loss 41.18096542 - samples/sec: 136.45 - lr: 0.500000 2021-03-26 08:12:24,274 epoch 2 - iter 4/25 - loss 42.09041786 - samples/sec: 135.70 - lr: 0.500000 2021-03-26 08:12:25,257 epoch 2 - iter 6/25 - loss 41.03157552 - samples/sec: 130.50 - lr: 0.500000 2021-03-26 08:12:26,236 epoch 2 - iter 8/25 - loss 41.45999718 - samples/sec: 130.92 - lr: 0.500000 2021-03-26 08:12:27,220 epoch 2 - iter 10/25 - loss 40.17591343 - samples/sec: 130.21 - lr: 0.500000 2021-03-26 08:12:28,146 epoch 2 - iter 12/25 - loss 39.56078482 - samples/sec: 138.45 - lr: 0.500000 2021-03-26 08:12:29,183 epoch 2 - iter 14/25 - loss 39.04572909 - samples/sec: 123.64 - lr: 0.500000 2021-03-26 08:12:30,176 epoch 2 - iter 16/25 - loss 38.69135320 - samples/sec: 129.02 - lr: 0.500000 2021-03-26 08:12:31,136 epoch 2 - iter 18/25 - loss 38.19763809 - samples/sec: 133.55 - lr: 0.500000 2021-03-26 08:12:32,073 epoch 2 - iter 20/25 - loss 37.77106895 - samples/sec: 136.77 - lr: 0.500000 2021-03-26 08:12:33,096 epoch 2 - iter 22/25 - loss 36.93832788 - samples/sec: 125.43 - lr: 0.500000 2021-03-26 08:12:34,178 epoch 2 - iter 24/25 - loss 36.67819985 - samples/sec: 118.52 - lr: 0.500000 2021-03-26 08:12:34,651 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:12:34,652 EPOCH 2 done: loss 36.9102 - lr 0.5000000 2021-03-26 08:12:35,387 DEV : loss 26.169567108154297 - score 0.5504 2021-03-26 08:12:35,405 BAD EPOCHS (no improvement): 0 2021-03-26 08:12:45,052 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:12:46,079 epoch 3 - iter 2/25 - loss 27.94248581 - samples/sec: 124.85 - lr: 0.500000 2021-03-26 08:12:47,115 epoch 3 - iter 4/25 - loss 29.98732948 - samples/sec: 123.76 - lr: 0.500000 2021-03-26 08:12:48,117 epoch 3 - iter 6/25 - loss 30.17901389 - samples/sec: 127.85 - lr: 0.500000 2021-03-26 08:12:49,017 epoch 3 - iter 8/25 - loss 29.07717657 - samples/sec: 142.45 - lr: 0.500000 2021-03-26 08:12:50,013 epoch 3 - iter 10/25 - loss 28.96063709 - samples/sec: 128.80 - lr: 0.500000 2021-03-26 08:12:50,969 epoch 3 - iter 12/25 - loss 27.79913012 - samples/sec: 134.09 - lr: 0.500000 2021-03-26 08:12:51,853 epoch 3 - iter 14/25 - loss 26.78517042 - samples/sec: 145.13 - lr: 0.500000 2021-03-26 08:12:52,818 epoch 3 - iter 16/25 - loss 26.49672520 - samples/sec: 132.85 - lr: 0.500000 2021-03-26 08:12:53,792 epoch 3 - iter 18/25 - loss 26.31827492 - samples/sec: 131.53 - lr: 0.500000 2021-03-26 08:12:54,721 epoch 3 - iter 20/25 - loss 26.07491808 - samples/sec: 138.08 - lr: 0.500000 2021-03-26 08:12:55,655 epoch 3 - iter 22/25 - loss 25.77279256 - samples/sec: 137.18 - lr: 0.500000 2021-03-26 08:12:56,711 epoch 3 - iter 24/25 - loss 25.54567377 - samples/sec: 121.40 - lr: 0.500000 2021-03-26 08:12:57,090 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:12:57,091 EPOCH 3 done: loss 25.4674 - lr 0.5000000 2021-03-26 08:12:57,824 DEV : loss 17.2449951171875 - score 0.6876 2021-03-26 08:12:57,847 BAD EPOCHS (no improvement): 0 2021-03-26 08:13:07,348 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:13:08,338 epoch 4 - iter 2/25 - loss 22.18989372 - samples/sec: 129.57 - lr: 0.500000 2021-03-26 08:13:09,335 epoch 4 - iter 4/25 - loss 21.36251116 - samples/sec: 128.57 - lr: 0.500000 2021-03-26 08:13:10,370 epoch 4 - iter 6/25 - loss 20.81579622 - samples/sec: 123.77 - lr: 0.500000 2021-03-26 08:13:11,362 epoch 4 - iter 8/25 - loss 20.94697762 - samples/sec: 129.30 - lr: 0.500000 2021-03-26 08:13:12,314 epoch 4 - iter 10/25 - loss 20.32834587 - samples/sec: 134.59 - lr: 0.500000 2021-03-26 08:13:13,332 epoch 4 - iter 12/25 - loss 20.46576881 - samples/sec: 126.00 - lr: 0.500000 2021-03-26 08:13:14,215 epoch 4 - iter 14/25 - loss 20.12338993 - samples/sec: 145.20 - lr: 0.500000 2021-03-26 08:13:15,108 epoch 4 - iter 16/25 - loss 19.82321393 - samples/sec: 143.61 - lr: 0.500000 2021-03-26 08:13:16,155 epoch 4 - iter 18/25 - loss 19.77442031 - samples/sec: 122.47 - lr: 0.500000 2021-03-26 08:13:17,215 epoch 4 - iter 20/25 - loss 19.73856611 - samples/sec: 120.95 - lr: 0.500000 2021-03-26 08:13:18,191 epoch 4 - iter 22/25 - loss 19.75611487 - samples/sec: 131.22 - lr: 0.500000 2021-03-26 08:13:19,153 epoch 4 - iter 24/25 - loss 19.74460936 - samples/sec: 133.20 - lr: 0.500000 2021-03-26 08:13:19,512 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:13:19,513 EPOCH 4 done: loss 19.7620 - lr 0.5000000 2021-03-26 08:13:20,225 DEV : loss 14.096415519714355 - score 0.7539 2021-03-26 08:13:20,247 BAD EPOCHS (no improvement): 0 2021-03-26 08:13:29,660 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:13:30,708 epoch 5 - iter 2/25 - loss 15.49918652 - samples/sec: 122.31 - lr: 0.500000 2021-03-26 08:13:31,707 epoch 5 - iter 4/25 - loss 16.34196806 - samples/sec: 128.31 - lr: 0.500000 2021-03-26 08:13:32,629 epoch 5 - iter 6/25 - loss 16.47365252 - samples/sec: 139.06 - lr: 0.500000 2021-03-26 08:13:33,588 epoch 5 - iter 8/25 - loss 16.72915411 - samples/sec: 133.72 - lr: 0.500000 2021-03-26 08:13:34,628 epoch 5 - iter 10/25 - loss 16.65385647 - samples/sec: 123.11 - lr: 0.500000 2021-03-26 08:13:35,643 epoch 5 - iter 12/25 - loss 16.54397535 - samples/sec: 126.39 - lr: 0.500000 2021-03-26 08:13:36,664 epoch 5 - iter 14/25 - loss 16.43941729 - samples/sec: 125.44 - lr: 0.500000 2021-03-26 08:13:37,627 epoch 5 - iter 16/25 - loss 16.51431048 - samples/sec: 133.18 - lr: 0.500000 2021-03-26 08:13:38,664 epoch 5 - iter 18/25 - loss 16.42917135 - samples/sec: 123.62 - lr: 0.500000 2021-03-26 08:13:39,605 epoch 5 - iter 20/25 - loss 16.32712135 - samples/sec: 136.30 - lr: 0.500000 2021-03-26 08:13:40,631 epoch 5 - iter 22/25 - loss 16.20318218 - samples/sec: 124.99 - lr: 0.500000 2021-03-26 08:13:41,558 epoch 5 - iter 24/25 - loss 16.25766639 - samples/sec: 138.34 - lr: 0.500000 2021-03-26 08:13:41,957 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:13:41,957 EPOCH 5 done: loss 16.3372 - lr 0.5000000 2021-03-26 08:13:42,681 DEV : loss 11.58500862121582 - score 0.7945 2021-03-26 08:13:42,696 BAD EPOCHS (no improvement): 0 2021-03-26 08:13:52,128 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:13:53,106 epoch 6 - iter 2/25 - loss 13.41134262 - samples/sec: 131.11 - lr: 0.500000 2021-03-26 08:13:54,017 epoch 6 - iter 4/25 - loss 14.33909249 - samples/sec: 140.75 - lr: 0.500000 2021-03-26 08:13:54,945 epoch 6 - iter 6/25 - loss 14.27938827 - samples/sec: 138.26 - lr: 0.500000 2021-03-26 08:13:55,888 epoch 6 - iter 8/25 - loss 13.98244059 - samples/sec: 135.93 - lr: 0.500000 2021-03-26 08:13:56,896 epoch 6 - iter 10/25 - loss 13.70569067 - samples/sec: 127.04 - lr: 0.500000 2021-03-26 08:13:57,865 epoch 6 - iter 12/25 - loss 13.65978408 - samples/sec: 132.27 - lr: 0.500000 2021-03-26 08:13:58,841 epoch 6 - iter 14/25 - loss 14.30981084 - samples/sec: 131.43 - lr: 0.500000 2021-03-26 08:13:59,783 epoch 6 - iter 16/25 - loss 13.86719906 - samples/sec: 136.12 - lr: 0.500000 2021-03-26 08:14:00,703 epoch 6 - iter 18/25 - loss 13.85862313 - samples/sec: 139.34 - lr: 0.500000 2021-03-26 08:14:01,630 epoch 6 - iter 20/25 - loss 13.72562017 - samples/sec: 138.32 - lr: 0.500000 2021-03-26 08:14:02,592 epoch 6 - iter 22/25 - loss 14.05992985 - samples/sec: 133.30 - lr: 0.500000 2021-03-26 08:14:03,686 epoch 6 - iter 24/25 - loss 14.23950656 - samples/sec: 117.15 - lr: 0.500000 2021-03-26 08:14:04,051 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:14:04,052 EPOCH 6 done: loss 14.1535 - lr 0.5000000 2021-03-26 08:14:04,768 DEV : loss 10.297845840454102 - score 0.8135 2021-03-26 08:14:04,787 BAD EPOCHS (no improvement): 0 2021-03-26 08:14:14,125 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:14:15,363 epoch 7 - iter 2/25 - loss 12.14385891 - samples/sec: 103.55 - lr: 0.500000 2021-03-26 08:14:16,472 epoch 7 - iter 4/25 - loss 12.05147791 - samples/sec: 115.63 - lr: 0.500000 2021-03-26 08:14:17,559 epoch 7 - iter 6/25 - loss 12.24715741 - samples/sec: 117.85 - lr: 0.500000 2021-03-26 08:14:18,512 epoch 7 - iter 8/25 - loss 12.49173355 - samples/sec: 134.65 - lr: 0.500000 2021-03-26 08:14:19,444 epoch 7 - iter 10/25 - loss 12.60267935 - samples/sec: 137.58 - lr: 0.500000 2021-03-26 08:14:20,450 epoch 7 - iter 12/25 - loss 12.57178235 - samples/sec: 127.45 - lr: 0.500000 2021-03-26 08:14:21,419 epoch 7 - iter 14/25 - loss 12.35317836 - samples/sec: 132.30 - lr: 0.500000 2021-03-26 08:14:22,418 epoch 7 - iter 16/25 - loss 12.54999465 - samples/sec: 128.27 - lr: 0.500000 2021-03-26 08:14:23,487 epoch 7 - iter 18/25 - loss 12.62174733 - samples/sec: 120.03 - lr: 0.500000 2021-03-26 08:14:24,452 epoch 7 - iter 20/25 - loss 12.63937850 - samples/sec: 132.68 - lr: 0.500000 2021-03-26 08:14:25,480 epoch 7 - iter 22/25 - loss 12.64787314 - samples/sec: 124.74 - lr: 0.500000 2021-03-26 08:14:26,472 epoch 7 - iter 24/25 - loss 12.69575051 - samples/sec: 129.18 - lr: 0.500000 2021-03-26 08:14:26,884 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:14:26,885 EPOCH 7 done: loss 12.8097 - lr 0.5000000 2021-03-26 08:14:27,615 DEV : loss 8.942085266113281 - score 0.8416 2021-03-26 08:14:27,639 BAD EPOCHS (no improvement): 0 2021-03-26 08:14:37,096 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:14:38,050 epoch 8 - iter 2/25 - loss 10.87221813 - samples/sec: 134.38 - lr: 0.500000 2021-03-26 08:14:39,084 epoch 8 - iter 4/25 - loss 11.29472542 - samples/sec: 123.98 - lr: 0.500000 2021-03-26 08:14:40,086 epoch 8 - iter 6/25 - loss 12.02312867 - samples/sec: 127.97 - lr: 0.500000 2021-03-26 08:14:41,132 epoch 8 - iter 8/25 - loss 12.35515392 - samples/sec: 122.41 - lr: 0.500000 2021-03-26 08:14:42,096 epoch 8 - iter 10/25 - loss 12.06548958 - samples/sec: 133.20 - lr: 0.500000 2021-03-26 08:14:43,065 epoch 8 - iter 12/25 - loss 11.95635835 - samples/sec: 132.24 - lr: 0.500000 2021-03-26 08:14:44,011 epoch 8 - iter 14/25 - loss 11.88751956 - samples/sec: 135.54 - lr: 0.500000 2021-03-26 08:14:44,973 epoch 8 - iter 16/25 - loss 11.72202009 - samples/sec: 133.15 - lr: 0.500000 2021-03-26 08:14:45,895 epoch 8 - iter 18/25 - loss 11.36020141 - samples/sec: 139.16 - lr: 0.500000 2021-03-26 08:14:46,979 epoch 8 - iter 20/25 - loss 11.38508224 - samples/sec: 118.18 - lr: 0.500000 2021-03-26 08:14:48,005 epoch 8 - iter 22/25 - loss 11.59132914 - samples/sec: 124.91 - lr: 0.500000 2021-03-26 08:14:49,043 epoch 8 - iter 24/25 - loss 11.46632906 - samples/sec: 123.49 - lr: 0.500000 2021-03-26 08:14:49,450 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:14:49,451 EPOCH 8 done: loss 11.4002 - lr 0.5000000 2021-03-26 08:14:50,197 DEV : loss 8.089621543884277 - score 0.8515 2021-03-26 08:14:50,219 BAD EPOCHS (no improvement): 0 2021-03-26 08:14:59,764 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:15:00,849 epoch 9 - iter 2/25 - loss 10.32193851 - samples/sec: 118.26 - lr: 0.500000 2021-03-26 08:15:01,921 epoch 9 - iter 4/25 - loss 10.57744074 - samples/sec: 119.55 - lr: 0.500000 2021-03-26 08:15:02,953 epoch 9 - iter 6/25 - loss 11.03293610 - samples/sec: 124.21 - lr: 0.500000 2021-03-26 08:15:03,914 epoch 9 - iter 8/25 - loss 10.46654230 - samples/sec: 133.41 - lr: 0.500000 2021-03-26 08:15:05,033 epoch 9 - iter 10/25 - loss 10.52596345 - samples/sec: 114.52 - lr: 0.500000 2021-03-26 08:15:06,008 epoch 9 - iter 12/25 - loss 10.55391188 - samples/sec: 131.49 - lr: 0.500000 2021-03-26 08:15:07,056 epoch 9 - iter 14/25 - loss 10.63699726 - samples/sec: 122.31 - lr: 0.500000 2021-03-26 08:15:07,977 epoch 9 - iter 16/25 - loss 10.61607608 - samples/sec: 139.16 - lr: 0.500000 2021-03-26 08:15:08,969 epoch 9 - iter 18/25 - loss 10.64035514 - samples/sec: 129.31 - lr: 0.500000 2021-03-26 08:15:09,888 epoch 9 - iter 20/25 - loss 10.61658571 - samples/sec: 139.45 - lr: 0.500000 2021-03-26 08:15:10,802 epoch 9 - iter 22/25 - loss 10.39599518 - samples/sec: 140.27 - lr: 0.500000 2021-03-26 08:15:11,783 epoch 9 - iter 24/25 - loss 10.44796216 - samples/sec: 130.70 - lr: 0.500000 2021-03-26 08:15:12,143 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:15:12,144 EPOCH 9 done: loss 10.4188 - lr 0.5000000 2021-03-26 08:15:12,914 DEV : loss 7.821622371673584 - score 0.8622 2021-03-26 08:15:12,938 BAD EPOCHS (no improvement): 0 2021-03-26 08:15:22,456 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:15:23,458 epoch 10 - iter 2/25 - loss 7.78988838 - samples/sec: 127.94 - lr: 0.500000 2021-03-26 08:15:24,633 epoch 10 - iter 4/25 - loss 8.20438027 - samples/sec: 109.09 - lr: 0.500000 2021-03-26 08:15:25,654 epoch 10 - iter 6/25 - loss 8.11983109 - samples/sec: 125.65 - lr: 0.500000 2021-03-26 08:15:26,838 epoch 10 - iter 8/25 - loss 8.96964788 - samples/sec: 108.22 - lr: 0.500000 2021-03-26 08:15:28,017 epoch 10 - iter 10/25 - loss 8.98561802 - samples/sec: 108.78 - lr: 0.500000 2021-03-26 08:15:29,215 epoch 10 - iter 12/25 - loss 8.97878774 - samples/sec: 107.01 - lr: 0.500000 2021-03-26 08:15:30,313 epoch 10 - iter 14/25 - loss 9.11639765 - samples/sec: 116.67 - lr: 0.500000 2021-03-26 08:15:31,319 epoch 10 - iter 16/25 - loss 9.25904155 - samples/sec: 127.49 - lr: 0.500000 2021-03-26 08:15:32,356 epoch 10 - iter 18/25 - loss 9.40983295 - samples/sec: 123.64 - lr: 0.500000 2021-03-26 08:15:33,580 epoch 10 - iter 20/25 - loss 9.54233298 - samples/sec: 104.71 - lr: 0.500000 2021-03-26 08:15:34,844 epoch 10 - iter 22/25 - loss 9.56605933 - samples/sec: 101.39 - lr: 0.500000 2021-03-26 08:15:36,143 epoch 10 - iter 24/25 - loss 9.59460024 - samples/sec: 98.64 - lr: 0.500000 2021-03-26 08:15:36,632 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:15:36,633 EPOCH 10 done: loss 9.5178 - lr 0.5000000 2021-03-26 08:15:37,368 DEV : loss 7.125051021575928 - score 0.8689 2021-03-26 08:15:37,390 BAD EPOCHS (no improvement): 0 2021-03-26 08:15:47,068 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:15:48,091 epoch 11 - iter 2/25 - loss 8.83649921 - samples/sec: 125.38 - lr: 0.500000 2021-03-26 08:15:49,052 epoch 11 - iter 4/25 - loss 9.08403540 - samples/sec: 133.36 - lr: 0.500000 2021-03-26 08:15:50,009 epoch 11 - iter 6/25 - loss 8.58666507 - samples/sec: 133.97 - lr: 0.500000 2021-03-26 08:15:50,960 epoch 11 - iter 8/25 - loss 9.06016153 - samples/sec: 134.83 - lr: 0.500000 2021-03-26 08:15:52,063 epoch 11 - iter 10/25 - loss 8.88002706 - samples/sec: 116.22 - lr: 0.500000 2021-03-26 08:15:53,103 epoch 11 - iter 12/25 - loss 8.69153821 - samples/sec: 123.25 - lr: 0.500000 2021-03-26 08:15:54,112 epoch 11 - iter 14/25 - loss 8.90236327 - samples/sec: 127.00 - lr: 0.500000 2021-03-26 08:15:55,164 epoch 11 - iter 16/25 - loss 8.69934401 - samples/sec: 121.84 - lr: 0.500000 2021-03-26 08:15:56,431 epoch 11 - iter 18/25 - loss 8.75323354 - samples/sec: 101.24 - lr: 0.500000 2021-03-26 08:15:57,514 epoch 11 - iter 20/25 - loss 8.71253555 - samples/sec: 118.28 - lr: 0.500000 2021-03-26 08:15:58,470 epoch 11 - iter 22/25 - loss 8.70876809 - samples/sec: 134.25 - lr: 0.500000 2021-03-26 08:15:59,486 epoch 11 - iter 24/25 - loss 8.68266873 - samples/sec: 126.18 - lr: 0.500000 2021-03-26 08:15:59,912 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:15:59,913 EPOCH 11 done: loss 8.8185 - lr 0.5000000 2021-03-26 08:16:00,629 DEV : loss 6.984427452087402 - score 0.8739 2021-03-26 08:16:00,651 BAD EPOCHS (no improvement): 0 2021-03-26 08:16:10,141 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:16:11,372 epoch 12 - iter 2/25 - loss 7.81654215 - samples/sec: 104.18 - lr: 0.500000 2021-03-26 08:16:12,442 epoch 12 - iter 4/25 - loss 7.47270012 - samples/sec: 119.91 - lr: 0.500000 2021-03-26 08:16:13,384 epoch 12 - iter 6/25 - loss 8.39132531 - samples/sec: 136.03 - lr: 0.500000 2021-03-26 08:16:14,323 epoch 12 - iter 8/25 - loss 8.67520797 - samples/sec: 136.58 - lr: 0.500000 2021-03-26 08:16:15,252 epoch 12 - iter 10/25 - loss 8.53146949 - samples/sec: 137.97 - lr: 0.500000 2021-03-26 08:16:16,288 epoch 12 - iter 12/25 - loss 8.33992930 - samples/sec: 123.67 - lr: 0.500000 2021-03-26 08:16:17,412 epoch 12 - iter 14/25 - loss 8.22010701 - samples/sec: 113.99 - lr: 0.500000 2021-03-26 08:16:18,351 epoch 12 - iter 16/25 - loss 8.26404691 - samples/sec: 136.59 - lr: 0.500000 2021-03-26 08:16:19,321 epoch 12 - iter 18/25 - loss 8.19568605 - samples/sec: 132.22 - lr: 0.500000 2021-03-26 08:16:20,420 epoch 12 - iter 20/25 - loss 8.29496424 - samples/sec: 116.60 - lr: 0.500000 2021-03-26 08:16:21,292 epoch 12 - iter 22/25 - loss 8.28023763 - samples/sec: 147.05 - lr: 0.500000 2021-03-26 08:16:22,376 epoch 12 - iter 24/25 - loss 8.29219141 - samples/sec: 118.19 - lr: 0.500000 2021-03-26 08:16:22,776 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:16:22,777 EPOCH 12 done: loss 8.4431 - lr 0.5000000 2021-03-26 08:16:23,488 DEV : loss 7.6165900230407715 - score 0.868 2021-03-26 08:16:23,505 BAD EPOCHS (no improvement): 1 2021-03-26 08:16:23,506 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:16:24,580 epoch 13 - iter 2/25 - loss 8.72345257 - samples/sec: 119.32 - lr: 0.500000 2021-03-26 08:16:25,643 epoch 13 - iter 4/25 - loss 7.85941982 - samples/sec: 120.66 - lr: 0.500000 2021-03-26 08:16:26,713 epoch 13 - iter 6/25 - loss 7.69082999 - samples/sec: 119.86 - lr: 0.500000 2021-03-26 08:16:27,671 epoch 13 - iter 8/25 - loss 7.97219425 - samples/sec: 134.40 - lr: 0.500000 2021-03-26 08:16:28,664 epoch 13 - iter 10/25 - loss 7.87737551 - samples/sec: 129.00 - lr: 0.500000 2021-03-26 08:16:29,764 epoch 13 - iter 12/25 - loss 7.69054703 - samples/sec: 116.50 - lr: 0.500000 2021-03-26 08:16:30,805 epoch 13 - iter 14/25 - loss 7.75256617 - samples/sec: 123.24 - lr: 0.500000 2021-03-26 08:16:31,828 epoch 13 - iter 16/25 - loss 7.90568480 - samples/sec: 125.28 - lr: 0.500000 2021-03-26 08:16:32,788 epoch 13 - iter 18/25 - loss 7.90655682 - samples/sec: 133.51 - lr: 0.500000 2021-03-26 08:16:33,932 epoch 13 - iter 20/25 - loss 7.88109205 - samples/sec: 111.97 - lr: 0.500000 2021-03-26 08:16:34,959 epoch 13 - iter 22/25 - loss 8.02318389 - samples/sec: 124.85 - lr: 0.500000 2021-03-26 08:16:35,870 epoch 13 - iter 24/25 - loss 7.88861491 - samples/sec: 140.80 - lr: 0.500000 2021-03-26 08:16:36,256 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:16:36,257 EPOCH 13 done: loss 7.8336 - lr 0.5000000 2021-03-26 08:16:36,986 DEV : loss 6.979231834411621 - score 0.871 2021-03-26 08:16:37,001 BAD EPOCHS (no improvement): 2 2021-03-26 08:16:37,002 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:16:37,970 epoch 14 - iter 2/25 - loss 10.96346951 - samples/sec: 132.45 - lr: 0.500000 2021-03-26 08:16:39,003 epoch 14 - iter 4/25 - loss 10.93682742 - samples/sec: 124.15 - lr: 0.500000 2021-03-26 08:16:40,104 epoch 14 - iter 6/25 - loss 9.74375391 - samples/sec: 116.45 - lr: 0.500000 2021-03-26 08:16:41,013 epoch 14 - iter 8/25 - loss 9.09124070 - samples/sec: 140.95 - lr: 0.500000 2021-03-26 08:16:41,977 epoch 14 - iter 10/25 - loss 8.89416289 - samples/sec: 133.06 - lr: 0.500000 2021-03-26 08:16:43,015 epoch 14 - iter 12/25 - loss 8.60138631 - samples/sec: 123.50 - lr: 0.500000 2021-03-26 08:16:43,950 epoch 14 - iter 14/25 - loss 8.30748442 - samples/sec: 137.80 - lr: 0.500000 2021-03-26 08:16:44,987 epoch 14 - iter 16/25 - loss 8.26665404 - samples/sec: 123.64 - lr: 0.500000 2021-03-26 08:16:46,067 epoch 14 - iter 18/25 - loss 8.19610063 - samples/sec: 118.69 - lr: 0.500000 2021-03-26 08:16:46,993 epoch 14 - iter 20/25 - loss 8.22014797 - samples/sec: 138.49 - lr: 0.500000 2021-03-26 08:16:48,144 epoch 14 - iter 22/25 - loss 8.11139623 - samples/sec: 111.27 - lr: 0.500000 2021-03-26 08:16:49,092 epoch 14 - iter 24/25 - loss 8.05588663 - samples/sec: 135.35 - lr: 0.500000 2021-03-26 08:16:49,531 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:16:49,531 EPOCH 14 done: loss 8.0288 - lr 0.5000000 2021-03-26 08:16:50,260 DEV : loss 6.266796112060547 - score 0.8801 2021-03-26 08:16:50,283 BAD EPOCHS (no improvement): 0 2021-03-26 08:16:59,697 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:17:00,687 epoch 15 - iter 2/25 - loss 6.77654982 - samples/sec: 129.65 - lr: 0.500000 2021-03-26 08:17:01,722 epoch 15 - iter 4/25 - loss 6.73212910 - samples/sec: 123.75 - lr: 0.500000 2021-03-26 08:17:02,675 epoch 15 - iter 6/25 - loss 7.18263904 - samples/sec: 134.58 - lr: 0.500000 2021-03-26 08:17:03,684 epoch 15 - iter 8/25 - loss 7.14034599 - samples/sec: 126.98 - lr: 0.500000 2021-03-26 08:17:04,614 epoch 15 - iter 10/25 - loss 7.17819848 - samples/sec: 137.83 - lr: 0.500000 2021-03-26 08:17:05,588 epoch 15 - iter 12/25 - loss 6.99284558 - samples/sec: 131.61 - lr: 0.500000 2021-03-26 08:17:06,602 epoch 15 - iter 14/25 - loss 7.10860590 - samples/sec: 126.46 - lr: 0.500000 2021-03-26 08:17:07,648 epoch 15 - iter 16/25 - loss 7.04990083 - samples/sec: 122.49 - lr: 0.500000 2021-03-26 08:17:08,616 epoch 15 - iter 18/25 - loss 7.14585771 - samples/sec: 132.47 - lr: 0.500000 2021-03-26 08:17:09,651 epoch 15 - iter 20/25 - loss 7.15546744 - samples/sec: 123.76 - lr: 0.500000 2021-03-26 08:17:10,651 epoch 15 - iter 22/25 - loss 7.10891477 - samples/sec: 128.28 - lr: 0.500000 2021-03-26 08:17:11,784 epoch 15 - iter 24/25 - loss 7.15305050 - samples/sec: 113.18 - lr: 0.500000 2021-03-26 08:17:12,158 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:17:12,159 EPOCH 15 done: loss 7.1976 - lr 0.5000000 2021-03-26 08:17:12,887 DEV : loss 6.324016571044922 - score 0.8895 2021-03-26 08:17:12,911 BAD EPOCHS (no improvement): 0 2021-03-26 08:17:22,753 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:17:23,828 epoch 16 - iter 2/25 - loss 6.31022072 - samples/sec: 119.38 - lr: 0.500000 2021-03-26 08:17:24,818 epoch 16 - iter 4/25 - loss 6.49304485 - samples/sec: 129.56 - lr: 0.500000 2021-03-26 08:17:25,780 epoch 16 - iter 6/25 - loss 6.19578926 - samples/sec: 133.36 - lr: 0.500000 2021-03-26 08:17:26,857 epoch 16 - iter 8/25 - loss 5.96798402 - samples/sec: 119.01 - lr: 0.500000 2021-03-26 08:17:27,781 epoch 16 - iter 10/25 - loss 6.17498813 - samples/sec: 138.72 - lr: 0.500000 2021-03-26 08:17:28,737 epoch 16 - iter 12/25 - loss 6.21972493 - samples/sec: 134.08 - lr: 0.500000 2021-03-26 08:17:30,084 epoch 16 - iter 14/25 - loss 6.37837097 - samples/sec: 95.18 - lr: 0.500000 2021-03-26 08:17:31,199 epoch 16 - iter 16/25 - loss 6.60098591 - samples/sec: 114.97 - lr: 0.500000 2021-03-26 08:17:32,145 epoch 16 - iter 18/25 - loss 6.65434315 - samples/sec: 135.50 - lr: 0.500000 2021-03-26 08:17:33,129 epoch 16 - iter 20/25 - loss 6.71629548 - samples/sec: 130.24 - lr: 0.500000 2021-03-26 08:17:34,075 epoch 16 - iter 22/25 - loss 6.70602003 - samples/sec: 135.45 - lr: 0.500000 2021-03-26 08:17:35,083 epoch 16 - iter 24/25 - loss 6.80145276 - samples/sec: 127.22 - lr: 0.500000 2021-03-26 08:17:35,425 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:17:35,426 EPOCH 16 done: loss 6.8080 - lr 0.5000000 2021-03-26 08:17:36,127 DEV : loss 5.873531341552734 - score 0.8974 2021-03-26 08:17:36,149 BAD EPOCHS (no improvement): 0 2021-03-26 08:17:45,651 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:17:46,682 epoch 17 - iter 2/25 - loss 5.51331139 - samples/sec: 124.48 - lr: 0.500000 2021-03-26 08:17:47,676 epoch 17 - iter 4/25 - loss 5.94864118 - samples/sec: 128.92 - lr: 0.500000 2021-03-26 08:17:48,744 epoch 17 - iter 6/25 - loss 5.79273653 - samples/sec: 120.06 - lr: 0.500000 2021-03-26 08:17:49,727 epoch 17 - iter 8/25 - loss 6.06304932 - samples/sec: 130.47 - lr: 0.500000 2021-03-26 08:17:50,846 epoch 17 - iter 10/25 - loss 6.33874021 - samples/sec: 114.54 - lr: 0.500000 2021-03-26 08:17:51,925 epoch 17 - iter 12/25 - loss 6.44700205 - samples/sec: 118.83 - lr: 0.500000 2021-03-26 08:17:52,908 epoch 17 - iter 14/25 - loss 6.53073512 - samples/sec: 130.48 - lr: 0.500000 2021-03-26 08:17:53,839 epoch 17 - iter 16/25 - loss 6.51271585 - samples/sec: 137.65 - lr: 0.500000 2021-03-26 08:17:54,870 epoch 17 - iter 18/25 - loss 6.59723255 - samples/sec: 124.42 - lr: 0.500000 2021-03-26 08:17:55,905 epoch 17 - iter 20/25 - loss 6.51246181 - samples/sec: 123.92 - lr: 0.500000 2021-03-26 08:17:56,879 epoch 17 - iter 22/25 - loss 6.48072837 - samples/sec: 131.48 - lr: 0.500000 2021-03-26 08:17:57,819 epoch 17 - iter 24/25 - loss 6.57046958 - samples/sec: 136.45 - lr: 0.500000 2021-03-26 08:17:58,225 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:17:58,226 EPOCH 17 done: loss 6.5876 - lr 0.5000000 2021-03-26 08:17:58,950 DEV : loss 5.898545265197754 - score 0.8946 2021-03-26 08:17:58,973 BAD EPOCHS (no improvement): 1 2021-03-26 08:17:58,974 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:17:59,948 epoch 18 - iter 2/25 - loss 5.93250608 - samples/sec: 131.67 - lr: 0.500000 2021-03-26 08:18:00,989 epoch 18 - iter 4/25 - loss 6.21452951 - samples/sec: 123.26 - lr: 0.500000 2021-03-26 08:18:01,997 epoch 18 - iter 6/25 - loss 6.38846231 - samples/sec: 127.17 - lr: 0.500000 2021-03-26 08:18:02,914 epoch 18 - iter 8/25 - loss 6.20925385 - samples/sec: 139.71 - lr: 0.500000 2021-03-26 08:18:03,951 epoch 18 - iter 10/25 - loss 6.16746583 - samples/sec: 123.66 - lr: 0.500000 2021-03-26 08:18:04,918 epoch 18 - iter 12/25 - loss 6.01534859 - samples/sec: 132.65 - lr: 0.500000 2021-03-26 08:18:06,023 epoch 18 - iter 14/25 - loss 6.08419527 - samples/sec: 116.01 - lr: 0.500000 2021-03-26 08:18:07,122 epoch 18 - iter 16/25 - loss 6.03223971 - samples/sec: 116.67 - lr: 0.500000 2021-03-26 08:18:08,136 epoch 18 - iter 18/25 - loss 6.11314792 - samples/sec: 126.53 - lr: 0.500000 2021-03-26 08:18:09,157 epoch 18 - iter 20/25 - loss 6.26362159 - samples/sec: 125.59 - lr: 0.500000 2021-03-26 08:18:10,088 epoch 18 - iter 22/25 - loss 6.25091089 - samples/sec: 137.61 - lr: 0.500000 2021-03-26 08:18:11,026 epoch 18 - iter 24/25 - loss 6.25768552 - samples/sec: 136.61 - lr: 0.500000 2021-03-26 08:18:11,418 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:18:11,419 EPOCH 18 done: loss 6.2439 - lr 0.5000000 2021-03-26 08:18:12,122 DEV : loss 5.98818302154541 - score 0.8864 2021-03-26 08:18:12,146 BAD EPOCHS (no improvement): 2 2021-03-26 08:18:12,146 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:18:13,158 epoch 19 - iter 2/25 - loss 5.93176770 - samples/sec: 126.80 - lr: 0.500000 2021-03-26 08:18:14,110 epoch 19 - iter 4/25 - loss 6.19001615 - samples/sec: 134.59 - lr: 0.500000 2021-03-26 08:18:15,191 epoch 19 - iter 6/25 - loss 5.82322733 - samples/sec: 118.56 - lr: 0.500000 2021-03-26 08:18:16,218 epoch 19 - iter 8/25 - loss 5.79541320 - samples/sec: 124.75 - lr: 0.500000 2021-03-26 08:18:17,246 epoch 19 - iter 10/25 - loss 5.93873725 - samples/sec: 124.79 - lr: 0.500000 2021-03-26 08:18:18,401 epoch 19 - iter 12/25 - loss 5.89532459 - samples/sec: 111.03 - lr: 0.500000 2021-03-26 08:18:19,426 epoch 19 - iter 14/25 - loss 5.97058112 - samples/sec: 125.05 - lr: 0.500000 2021-03-26 08:18:20,416 epoch 19 - iter 16/25 - loss 5.94304895 - samples/sec: 129.52 - lr: 0.500000 2021-03-26 08:18:21,406 epoch 19 - iter 18/25 - loss 5.90629564 - samples/sec: 129.46 - lr: 0.500000 2021-03-26 08:18:22,367 epoch 19 - iter 20/25 - loss 5.90371339 - samples/sec: 133.58 - lr: 0.500000 2021-03-26 08:18:23,426 epoch 19 - iter 22/25 - loss 6.05487739 - samples/sec: 121.06 - lr: 0.500000 2021-03-26 08:18:24,422 epoch 19 - iter 24/25 - loss 6.00732064 - samples/sec: 128.59 - lr: 0.500000 2021-03-26 08:18:24,861 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:18:24,862 EPOCH 19 done: loss 6.0482 - lr 0.5000000 2021-03-26 08:18:25,576 DEV : loss 5.909600734710693 - score 0.8913 2021-03-26 08:18:25,594 BAD EPOCHS (no improvement): 3 2021-03-26 08:18:25,595 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:18:26,604 epoch 20 - iter 2/25 - loss 5.59814882 - samples/sec: 126.97 - lr: 0.500000 2021-03-26 08:18:27,785 epoch 20 - iter 4/25 - loss 5.75239575 - samples/sec: 108.51 - lr: 0.500000 2021-03-26 08:18:28,743 epoch 20 - iter 6/25 - loss 5.82471029 - samples/sec: 133.87 - lr: 0.500000 2021-03-26 08:18:29,809 epoch 20 - iter 8/25 - loss 5.66427326 - samples/sec: 120.18 - lr: 0.500000 2021-03-26 08:18:30,758 epoch 20 - iter 10/25 - loss 5.80760927 - samples/sec: 135.17 - lr: 0.500000 2021-03-26 08:18:31,810 epoch 20 - iter 12/25 - loss 5.71123417 - samples/sec: 121.93 - lr: 0.500000 2021-03-26 08:18:32,776 epoch 20 - iter 14/25 - loss 5.66636910 - samples/sec: 132.71 - lr: 0.500000 2021-03-26 08:18:34,805 epoch 20 - iter 16/25 - loss 5.60936448 - samples/sec: 63.13 - lr: 0.500000 2021-03-26 08:18:35,716 epoch 20 - iter 18/25 - loss 5.57157196 - samples/sec: 140.69 - lr: 0.500000 2021-03-26 08:18:36,716 epoch 20 - iter 20/25 - loss 5.68519535 - samples/sec: 128.27 - lr: 0.500000 2021-03-26 08:18:37,716 epoch 20 - iter 22/25 - loss 5.76139723 - samples/sec: 128.10 - lr: 0.500000 2021-03-26 08:18:38,714 epoch 20 - iter 24/25 - loss 5.81321446 - samples/sec: 129.07 - lr: 0.500000 2021-03-26 08:18:39,177 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:18:39,178 EPOCH 20 done: loss 5.8133 - lr 0.5000000 2021-03-26 08:18:39,875 DEV : loss 5.780816555023193 - score 0.8964 2021-03-26 08:18:39,892 BAD EPOCHS (no improvement): 4 2021-03-26 08:18:39,892 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:18:40,893 epoch 21 - iter 2/25 - loss 5.82937956 - samples/sec: 128.05 - lr: 0.250000 2021-03-26 08:18:41,862 epoch 21 - iter 4/25 - loss 5.56850576 - samples/sec: 132.20 - lr: 0.250000 2021-03-26 08:18:42,899 epoch 21 - iter 6/25 - loss 5.31449374 - samples/sec: 123.67 - lr: 0.250000 2021-03-26 08:18:43,865 epoch 21 - iter 8/25 - loss 5.22509336 - samples/sec: 132.62 - lr: 0.250000 2021-03-26 08:18:44,906 epoch 21 - iter 10/25 - loss 5.08971348 - samples/sec: 123.11 - lr: 0.250000 2021-03-26 08:18:45,948 epoch 21 - iter 12/25 - loss 5.15612626 - samples/sec: 123.00 - lr: 0.250000 2021-03-26 08:18:46,941 epoch 21 - iter 14/25 - loss 5.17431855 - samples/sec: 129.13 - lr: 0.250000 2021-03-26 08:18:47,809 epoch 21 - iter 16/25 - loss 4.99487485 - samples/sec: 147.69 - lr: 0.250000 2021-03-26 08:18:48,772 epoch 21 - iter 18/25 - loss 5.04701886 - samples/sec: 133.08 - lr: 0.250000 2021-03-26 08:18:49,714 epoch 21 - iter 20/25 - loss 5.06399451 - samples/sec: 136.13 - lr: 0.250000 2021-03-26 08:18:50,770 epoch 21 - iter 22/25 - loss 5.08102173 - samples/sec: 121.44 - lr: 0.250000 2021-03-26 08:18:51,678 epoch 21 - iter 24/25 - loss 4.95142464 - samples/sec: 141.33 - lr: 0.250000 2021-03-26 08:18:52,070 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:18:52,071 EPOCH 21 done: loss 4.9290 - lr 0.2500000 2021-03-26 08:18:52,801 DEV : loss 5.545863628387451 - score 0.9061 2021-03-26 08:18:52,828 BAD EPOCHS (no improvement): 0 2021-03-26 08:19:02,387 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:19:03,438 epoch 22 - iter 2/25 - loss 4.39438319 - samples/sec: 122.01 - lr: 0.250000 2021-03-26 08:19:04,390 epoch 22 - iter 4/25 - loss 4.97888792 - samples/sec: 134.67 - lr: 0.250000 2021-03-26 08:19:05,308 epoch 22 - iter 6/25 - loss 4.67588596 - samples/sec: 139.59 - lr: 0.250000 2021-03-26 08:19:06,252 epoch 22 - iter 8/25 - loss 4.68256471 - samples/sec: 135.69 - lr: 0.250000 2021-03-26 08:19:07,180 epoch 22 - iter 10/25 - loss 4.66318874 - samples/sec: 138.33 - lr: 0.250000 2021-03-26 08:19:08,165 epoch 22 - iter 12/25 - loss 4.74354013 - samples/sec: 130.11 - lr: 0.250000 2021-03-26 08:19:09,230 epoch 22 - iter 14/25 - loss 4.78080085 - samples/sec: 120.36 - lr: 0.250000 2021-03-26 08:19:10,157 epoch 22 - iter 16/25 - loss 4.75316647 - samples/sec: 138.46 - lr: 0.250000 2021-03-26 08:19:11,249 epoch 22 - iter 18/25 - loss 4.82255493 - samples/sec: 117.31 - lr: 0.250000 2021-03-26 08:19:12,189 epoch 22 - iter 20/25 - loss 4.78287302 - samples/sec: 136.41 - lr: 0.250000 2021-03-26 08:19:13,247 epoch 22 - iter 22/25 - loss 4.78834465 - samples/sec: 121.13 - lr: 0.250000 2021-03-26 08:19:14,583 epoch 22 - iter 24/25 - loss 4.79762895 - samples/sec: 95.93 - lr: 0.250000 2021-03-26 08:19:15,129 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:19:15,130 EPOCH 22 done: loss 4.8014 - lr 0.2500000 2021-03-26 08:19:15,915 DEV : loss 5.468145370483398 - score 0.9082 2021-03-26 08:19:15,947 BAD EPOCHS (no improvement): 0 2021-03-26 08:19:25,505 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:19:26,605 epoch 23 - iter 2/25 - loss 3.57932007 - samples/sec: 116.69 - lr: 0.250000 2021-03-26 08:19:27,637 epoch 23 - iter 4/25 - loss 4.20975953 - samples/sec: 124.18 - lr: 0.250000 2021-03-26 08:19:28,685 epoch 23 - iter 6/25 - loss 4.19612292 - samples/sec: 122.23 - lr: 0.250000 2021-03-26 08:19:29,666 epoch 23 - iter 8/25 - loss 4.45020649 - samples/sec: 130.73 - lr: 0.250000 2021-03-26 08:19:30,721 epoch 23 - iter 10/25 - loss 4.41350362 - samples/sec: 121.48 - lr: 0.250000 2021-03-26 08:19:31,691 epoch 23 - iter 12/25 - loss 4.52760651 - samples/sec: 132.20 - lr: 0.250000 2021-03-26 08:19:32,715 epoch 23 - iter 14/25 - loss 4.49416452 - samples/sec: 125.21 - lr: 0.250000 2021-03-26 08:19:33,687 epoch 23 - iter 16/25 - loss 4.55967647 - samples/sec: 131.85 - lr: 0.250000 2021-03-26 08:19:34,644 epoch 23 - iter 18/25 - loss 4.47532535 - samples/sec: 133.90 - lr: 0.250000 2021-03-26 08:19:35,578 epoch 23 - iter 20/25 - loss 4.42298470 - samples/sec: 137.20 - lr: 0.250000 2021-03-26 08:19:36,601 epoch 23 - iter 22/25 - loss 4.43825457 - samples/sec: 125.35 - lr: 0.250000 2021-03-26 08:19:37,737 epoch 23 - iter 24/25 - loss 4.45593623 - samples/sec: 112.76 - lr: 0.250000 2021-03-26 08:19:38,313 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:19:38,314 EPOCH 23 done: loss 4.4883 - lr 0.2500000 2021-03-26 08:19:39,090 DEV : loss 5.342250823974609 - score 0.9046 2021-03-26 08:19:39,115 BAD EPOCHS (no improvement): 1 2021-03-26 08:19:39,116 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:19:40,413 epoch 24 - iter 2/25 - loss 4.78895926 - samples/sec: 98.89 - lr: 0.250000 2021-03-26 08:19:41,367 epoch 24 - iter 4/25 - loss 4.32054168 - samples/sec: 134.38 - lr: 0.250000 2021-03-26 08:19:42,376 epoch 24 - iter 6/25 - loss 4.13959487 - samples/sec: 127.06 - lr: 0.250000 2021-03-26 08:19:43,311 epoch 24 - iter 8/25 - loss 4.05541480 - samples/sec: 137.20 - lr: 0.250000 2021-03-26 08:19:44,302 epoch 24 - iter 10/25 - loss 4.31926289 - samples/sec: 129.38 - lr: 0.250000 2021-03-26 08:19:45,187 epoch 24 - iter 12/25 - loss 4.26717488 - samples/sec: 145.02 - lr: 0.250000 2021-03-26 08:19:46,212 epoch 24 - iter 14/25 - loss 4.47580140 - samples/sec: 124.99 - lr: 0.250000 2021-03-26 08:19:47,285 epoch 24 - iter 16/25 - loss 4.44438323 - samples/sec: 119.49 - lr: 0.250000 2021-03-26 08:19:48,375 epoch 24 - iter 18/25 - loss 4.43706428 - samples/sec: 117.66 - lr: 0.250000 2021-03-26 08:19:49,363 epoch 24 - iter 20/25 - loss 4.40470082 - samples/sec: 129.78 - lr: 0.250000 2021-03-26 08:19:50,622 epoch 24 - iter 22/25 - loss 4.38354238 - samples/sec: 101.83 - lr: 0.250000 2021-03-26 08:19:51,616 epoch 24 - iter 24/25 - loss 4.36244129 - samples/sec: 128.90 - lr: 0.250000 2021-03-26 08:19:51,988 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:19:51,988 EPOCH 24 done: loss 4.3722 - lr 0.2500000 2021-03-26 08:19:52,696 DEV : loss 5.464868068695068 - score 0.9065 2021-03-26 08:19:52,718 BAD EPOCHS (no improvement): 2 2021-03-26 08:19:52,719 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:19:53,713 epoch 25 - iter 2/25 - loss 4.12841630 - samples/sec: 128.95 - lr: 0.250000 2021-03-26 08:19:54,650 epoch 25 - iter 4/25 - loss 4.23390102 - samples/sec: 136.91 - lr: 0.250000 2021-03-26 08:19:55,631 epoch 25 - iter 6/25 - loss 4.19182324 - samples/sec: 130.70 - lr: 0.250000 2021-03-26 08:19:56,678 epoch 25 - iter 8/25 - loss 4.14069355 - samples/sec: 122.38 - lr: 0.250000 2021-03-26 08:19:57,614 epoch 25 - iter 10/25 - loss 4.11968174 - samples/sec: 137.05 - lr: 0.250000 2021-03-26 08:19:58,695 epoch 25 - iter 12/25 - loss 4.03539999 - samples/sec: 118.61 - lr: 0.250000 2021-03-26 08:19:59,628 epoch 25 - iter 14/25 - loss 4.01701995 - samples/sec: 137.30 - lr: 0.250000 2021-03-26 08:20:00,655 epoch 25 - iter 16/25 - loss 4.00432700 - samples/sec: 124.91 - lr: 0.250000 2021-03-26 08:20:01,587 epoch 25 - iter 18/25 - loss 4.02359321 - samples/sec: 137.46 - lr: 0.250000 2021-03-26 08:20:02,633 epoch 25 - iter 20/25 - loss 4.16669927 - samples/sec: 122.58 - lr: 0.250000 2021-03-26 08:20:03,623 epoch 25 - iter 22/25 - loss 4.17694152 - samples/sec: 129.43 - lr: 0.250000 2021-03-26 08:20:04,606 epoch 25 - iter 24/25 - loss 4.19949826 - samples/sec: 130.35 - lr: 0.250000 2021-03-26 08:20:05,000 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:20:05,001 EPOCH 25 done: loss 4.2124 - lr 0.2500000 2021-03-26 08:20:05,735 DEV : loss 5.251356601715088 - score 0.9088 2021-03-26 08:20:05,767 BAD EPOCHS (no improvement): 0 2021-03-26 08:20:15,151 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:20:16,117 epoch 26 - iter 2/25 - loss 3.64543915 - samples/sec: 132.79 - lr: 0.250000 2021-03-26 08:20:17,159 epoch 26 - iter 4/25 - loss 3.60610306 - samples/sec: 122.93 - lr: 0.250000 2021-03-26 08:20:18,177 epoch 26 - iter 6/25 - loss 3.79775095 - samples/sec: 126.03 - lr: 0.250000 2021-03-26 08:20:19,160 epoch 26 - iter 8/25 - loss 4.02053875 - samples/sec: 130.39 - lr: 0.250000 2021-03-26 08:20:20,130 epoch 26 - iter 10/25 - loss 4.08083537 - samples/sec: 132.16 - lr: 0.250000 2021-03-26 08:20:21,211 epoch 26 - iter 12/25 - loss 4.25827795 - samples/sec: 118.56 - lr: 0.250000 2021-03-26 08:20:22,244 epoch 26 - iter 14/25 - loss 4.30301195 - samples/sec: 124.08 - lr: 0.250000 2021-03-26 08:20:23,185 epoch 26 - iter 16/25 - loss 4.31946926 - samples/sec: 136.43 - lr: 0.250000 2021-03-26 08:20:24,115 epoch 26 - iter 18/25 - loss 4.16990701 - samples/sec: 137.79 - lr: 0.250000 2021-03-26 08:20:25,151 epoch 26 - iter 20/25 - loss 4.20403528 - samples/sec: 123.68 - lr: 0.250000 2021-03-26 08:20:26,090 epoch 26 - iter 22/25 - loss 4.15471683 - samples/sec: 136.55 - lr: 0.250000 2021-03-26 08:20:27,107 epoch 26 - iter 24/25 - loss 4.15259549 - samples/sec: 126.08 - lr: 0.250000 2021-03-26 08:20:27,503 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:20:27,503 EPOCH 26 done: loss 4.1552 - lr 0.2500000 2021-03-26 08:20:28,206 DEV : loss 5.207316875457764 - score 0.9092 2021-03-26 08:20:28,229 BAD EPOCHS (no improvement): 0 2021-03-26 08:20:37,713 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:20:38,743 epoch 27 - iter 2/25 - loss 3.82175541 - samples/sec: 124.54 - lr: 0.250000 2021-03-26 08:20:39,725 epoch 27 - iter 4/25 - loss 4.01610708 - samples/sec: 130.61 - lr: 0.250000 2021-03-26 08:20:40,747 epoch 27 - iter 6/25 - loss 3.97728193 - samples/sec: 125.35 - lr: 0.250000 2021-03-26 08:20:41,683 epoch 27 - iter 8/25 - loss 4.06949121 - samples/sec: 136.92 - lr: 0.250000 2021-03-26 08:20:42,639 epoch 27 - iter 10/25 - loss 4.05400198 - samples/sec: 134.22 - lr: 0.250000 2021-03-26 08:20:43,700 epoch 27 - iter 12/25 - loss 3.98850983 - samples/sec: 120.75 - lr: 0.250000 2021-03-26 08:20:44,594 epoch 27 - iter 14/25 - loss 4.08828683 - samples/sec: 143.38 - lr: 0.250000 2021-03-26 08:20:45,593 epoch 27 - iter 16/25 - loss 4.22165529 - samples/sec: 128.28 - lr: 0.250000 2021-03-26 08:20:46,526 epoch 27 - iter 18/25 - loss 4.11700242 - samples/sec: 137.51 - lr: 0.250000 2021-03-26 08:20:47,568 epoch 27 - iter 20/25 - loss 4.16635238 - samples/sec: 122.95 - lr: 0.250000 2021-03-26 08:20:48,515 epoch 27 - iter 22/25 - loss 4.14988176 - samples/sec: 135.49 - lr: 0.250000 2021-03-26 08:20:49,456 epoch 27 - iter 24/25 - loss 4.15121517 - samples/sec: 136.17 - lr: 0.250000 2021-03-26 08:20:49,927 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:20:49,928 EPOCH 27 done: loss 4.1380 - lr 0.2500000 2021-03-26 08:20:50,661 DEV : loss 5.364542007446289 - score 0.9118 2021-03-26 08:20:50,685 BAD EPOCHS (no improvement): 0 2021-03-26 08:21:00,173 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:21:01,222 epoch 28 - iter 2/25 - loss 3.50919247 - samples/sec: 122.34 - lr: 0.250000 2021-03-26 08:21:02,106 epoch 28 - iter 4/25 - loss 3.65314603 - samples/sec: 145.06 - lr: 0.250000 2021-03-26 08:21:03,134 epoch 28 - iter 6/25 - loss 3.99633940 - samples/sec: 124.64 - lr: 0.250000 2021-03-26 08:21:04,115 epoch 28 - iter 8/25 - loss 3.91186973 - samples/sec: 130.68 - lr: 0.250000 2021-03-26 08:21:05,034 epoch 28 - iter 10/25 - loss 3.91712847 - samples/sec: 139.47 - lr: 0.250000 2021-03-26 08:21:06,086 epoch 28 - iter 12/25 - loss 3.98461171 - samples/sec: 121.88 - lr: 0.250000 2021-03-26 08:21:07,029 epoch 28 - iter 14/25 - loss 3.93786047 - samples/sec: 135.97 - lr: 0.250000 2021-03-26 08:21:08,164 epoch 28 - iter 16/25 - loss 3.96120444 - samples/sec: 112.95 - lr: 0.250000 2021-03-26 08:21:09,231 epoch 28 - iter 18/25 - loss 4.01231313 - samples/sec: 120.07 - lr: 0.250000 2021-03-26 08:21:10,217 epoch 28 - iter 20/25 - loss 4.00550351 - samples/sec: 130.04 - lr: 0.250000 2021-03-26 08:21:11,216 epoch 28 - iter 22/25 - loss 4.02875296 - samples/sec: 128.34 - lr: 0.250000 2021-03-26 08:21:12,171 epoch 28 - iter 24/25 - loss 4.04454490 - samples/sec: 134.28 - lr: 0.250000 2021-03-26 08:21:12,550 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:21:12,551 EPOCH 28 done: loss 4.0430 - lr 0.2500000 2021-03-26 08:21:13,276 DEV : loss 5.4191999435424805 - score 0.9057 2021-03-26 08:21:13,293 BAD EPOCHS (no improvement): 1 2021-03-26 08:21:13,293 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:21:14,274 epoch 29 - iter 2/25 - loss 3.11907506 - samples/sec: 130.75 - lr: 0.250000 2021-03-26 08:21:15,264 epoch 29 - iter 4/25 - loss 3.22725344 - samples/sec: 129.39 - lr: 0.250000 2021-03-26 08:21:16,193 epoch 29 - iter 6/25 - loss 3.43090073 - samples/sec: 137.98 - lr: 0.250000 2021-03-26 08:21:17,157 epoch 29 - iter 8/25 - loss 3.64438808 - samples/sec: 132.98 - lr: 0.250000 2021-03-26 08:21:18,103 epoch 29 - iter 10/25 - loss 3.61872423 - samples/sec: 135.63 - lr: 0.250000 2021-03-26 08:21:19,180 epoch 29 - iter 12/25 - loss 3.71516027 - samples/sec: 119.04 - lr: 0.250000 2021-03-26 08:21:20,179 epoch 29 - iter 14/25 - loss 3.74714305 - samples/sec: 128.19 - lr: 0.250000 2021-03-26 08:21:21,090 epoch 29 - iter 16/25 - loss 3.79307090 - samples/sec: 140.77 - lr: 0.250000 2021-03-26 08:21:22,053 epoch 29 - iter 18/25 - loss 3.75355324 - samples/sec: 132.99 - lr: 0.250000 2021-03-26 08:21:23,062 epoch 29 - iter 20/25 - loss 3.80725433 - samples/sec: 127.15 - lr: 0.250000 2021-03-26 08:21:24,146 epoch 29 - iter 22/25 - loss 3.83380900 - samples/sec: 118.29 - lr: 0.250000 2021-03-26 08:21:25,193 epoch 29 - iter 24/25 - loss 3.79775758 - samples/sec: 122.47 - lr: 0.250000 2021-03-26 08:21:25,636 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:21:25,637 EPOCH 29 done: loss 3.8274 - lr 0.2500000 2021-03-26 08:21:26,376 DEV : loss 5.257043838500977 - score 0.9126 2021-03-26 08:21:26,393 BAD EPOCHS (no improvement): 0 2021-03-26 08:21:36,002 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:21:37,083 epoch 30 - iter 2/25 - loss 3.26837575 - samples/sec: 118.73 - lr: 0.250000 2021-03-26 08:21:38,126 epoch 30 - iter 4/25 - loss 3.55052143 - samples/sec: 122.93 - lr: 0.250000 2021-03-26 08:21:39,218 epoch 30 - iter 6/25 - loss 3.51545898 - samples/sec: 117.34 - lr: 0.250000 2021-03-26 08:21:40,228 epoch 30 - iter 8/25 - loss 3.52578855 - samples/sec: 127.02 - lr: 0.250000 2021-03-26 08:21:41,116 epoch 30 - iter 10/25 - loss 3.35951045 - samples/sec: 144.52 - lr: 0.250000 2021-03-26 08:21:42,067 epoch 30 - iter 12/25 - loss 3.37255822 - samples/sec: 134.75 - lr: 0.250000 2021-03-26 08:21:43,062 epoch 30 - iter 14/25 - loss 3.49844871 - samples/sec: 128.91 - lr: 0.250000 2021-03-26 08:21:43,959 epoch 30 - iter 16/25 - loss 3.42460684 - samples/sec: 143.03 - lr: 0.250000 2021-03-26 08:21:44,987 epoch 30 - iter 18/25 - loss 3.54060519 - samples/sec: 124.75 - lr: 0.250000 2021-03-26 08:21:45,989 epoch 30 - iter 20/25 - loss 3.58279957 - samples/sec: 128.44 - lr: 0.250000 2021-03-26 08:21:46,909 epoch 30 - iter 22/25 - loss 3.62058116 - samples/sec: 139.40 - lr: 0.250000 2021-03-26 08:21:47,921 epoch 30 - iter 24/25 - loss 3.57932650 - samples/sec: 126.62 - lr: 0.250000 2021-03-26 08:21:48,333 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:21:48,334 EPOCH 30 done: loss 3.6379 - lr 0.2500000 2021-03-26 08:21:49,048 DEV : loss 5.350982189178467 - score 0.9067 2021-03-26 08:21:49,071 BAD EPOCHS (no improvement): 1 2021-03-26 08:21:49,071 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:21:50,133 epoch 31 - iter 2/25 - loss 3.56340146 - samples/sec: 120.82 - lr: 0.250000 2021-03-26 08:21:51,178 epoch 31 - iter 4/25 - loss 3.83946228 - samples/sec: 122.65 - lr: 0.250000 2021-03-26 08:21:52,149 epoch 31 - iter 6/25 - loss 3.70126661 - samples/sec: 132.12 - lr: 0.250000 2021-03-26 08:21:53,119 epoch 31 - iter 8/25 - loss 3.72824273 - samples/sec: 132.22 - lr: 0.250000 2021-03-26 08:21:54,120 epoch 31 - iter 10/25 - loss 3.52071278 - samples/sec: 128.09 - lr: 0.250000 2021-03-26 08:21:55,105 epoch 31 - iter 12/25 - loss 3.44530173 - samples/sec: 130.22 - lr: 0.250000 2021-03-26 08:21:56,140 epoch 31 - iter 14/25 - loss 3.47722861 - samples/sec: 123.90 - lr: 0.250000 2021-03-26 08:21:57,066 epoch 31 - iter 16/25 - loss 3.49996176 - samples/sec: 138.48 - lr: 0.250000 2021-03-26 08:21:57,977 epoch 31 - iter 18/25 - loss 3.49459041 - samples/sec: 140.59 - lr: 0.250000 2021-03-26 08:21:58,985 epoch 31 - iter 20/25 - loss 3.48911209 - samples/sec: 127.26 - lr: 0.250000 2021-03-26 08:22:00,036 epoch 31 - iter 22/25 - loss 3.47431731 - samples/sec: 122.05 - lr: 0.250000 2021-03-26 08:22:01,062 epoch 31 - iter 24/25 - loss 3.51283441 - samples/sec: 124.85 - lr: 0.250000 2021-03-26 08:22:01,395 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:22:01,396 EPOCH 31 done: loss 3.4821 - lr 0.2500000 2021-03-26 08:22:02,143 DEV : loss 5.382730007171631 - score 0.9099 2021-03-26 08:22:02,165 BAD EPOCHS (no improvement): 2 2021-03-26 08:22:02,166 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:22:03,083 epoch 32 - iter 2/25 - loss 2.84548354 - samples/sec: 139.94 - lr: 0.250000 2021-03-26 08:22:04,010 epoch 32 - iter 4/25 - loss 3.25069106 - samples/sec: 138.21 - lr: 0.250000 2021-03-26 08:22:05,026 epoch 32 - iter 6/25 - loss 3.52776047 - samples/sec: 126.22 - lr: 0.250000 2021-03-26 08:22:06,089 epoch 32 - iter 8/25 - loss 3.55593899 - samples/sec: 120.54 - lr: 0.250000 2021-03-26 08:22:07,090 epoch 32 - iter 10/25 - loss 3.46752748 - samples/sec: 128.07 - lr: 0.250000 2021-03-26 08:22:08,108 epoch 32 - iter 12/25 - loss 3.50795273 - samples/sec: 125.86 - lr: 0.250000 2021-03-26 08:22:09,054 epoch 32 - iter 14/25 - loss 3.61622231 - samples/sec: 135.40 - lr: 0.250000 2021-03-26 08:22:10,010 epoch 32 - iter 16/25 - loss 3.55014120 - samples/sec: 134.27 - lr: 0.250000 2021-03-26 08:22:11,055 epoch 32 - iter 18/25 - loss 3.56227758 - samples/sec: 122.66 - lr: 0.250000 2021-03-26 08:22:12,012 epoch 32 - iter 20/25 - loss 3.55966748 - samples/sec: 134.00 - lr: 0.250000 2021-03-26 08:22:12,924 epoch 32 - iter 22/25 - loss 3.50907019 - samples/sec: 140.67 - lr: 0.250000 2021-03-26 08:22:13,916 epoch 32 - iter 24/25 - loss 3.52688112 - samples/sec: 129.16 - lr: 0.250000 2021-03-26 08:22:14,305 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:22:14,306 EPOCH 32 done: loss 3.5538 - lr 0.2500000 2021-03-26 08:22:15,046 DEV : loss 5.270177841186523 - score 0.9133 2021-03-26 08:22:15,069 BAD EPOCHS (no improvement): 0 2021-03-26 08:22:24,675 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:22:25,689 epoch 33 - iter 2/25 - loss 3.15487409 - samples/sec: 126.48 - lr: 0.250000 2021-03-26 08:22:26,603 epoch 33 - iter 4/25 - loss 2.86552042 - samples/sec: 140.28 - lr: 0.250000 2021-03-26 08:22:27,589 epoch 33 - iter 6/25 - loss 2.91351219 - samples/sec: 129.92 - lr: 0.250000 2021-03-26 08:22:28,564 epoch 33 - iter 8/25 - loss 3.18267134 - samples/sec: 131.43 - lr: 0.250000 2021-03-26 08:22:29,661 epoch 33 - iter 10/25 - loss 3.10160365 - samples/sec: 116.84 - lr: 0.250000 2021-03-26 08:22:30,697 epoch 33 - iter 12/25 - loss 3.12735937 - samples/sec: 123.76 - lr: 0.250000 2021-03-26 08:22:31,604 epoch 33 - iter 14/25 - loss 3.19382245 - samples/sec: 141.36 - lr: 0.250000 2021-03-26 08:22:32,676 epoch 33 - iter 16/25 - loss 3.23439336 - samples/sec: 119.56 - lr: 0.250000 2021-03-26 08:22:33,697 epoch 33 - iter 18/25 - loss 3.21325998 - samples/sec: 125.67 - lr: 0.250000 2021-03-26 08:22:34,634 epoch 33 - iter 20/25 - loss 3.26282755 - samples/sec: 136.89 - lr: 0.250000 2021-03-26 08:22:35,681 epoch 33 - iter 22/25 - loss 3.34196002 - samples/sec: 122.34 - lr: 0.250000 2021-03-26 08:22:36,630 epoch 33 - iter 24/25 - loss 3.33675192 - samples/sec: 135.72 - lr: 0.250000 2021-03-26 08:22:37,055 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:22:37,056 EPOCH 33 done: loss 3.3575 - lr 0.2500000 2021-03-26 08:22:37,766 DEV : loss 5.367270469665527 - score 0.9096 2021-03-26 08:22:37,788 BAD EPOCHS (no improvement): 1 2021-03-26 08:22:37,788 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:22:38,731 epoch 34 - iter 2/25 - loss 3.58751345 - samples/sec: 135.99 - lr: 0.250000 2021-03-26 08:22:39,717 epoch 34 - iter 4/25 - loss 3.53140342 - samples/sec: 129.99 - lr: 0.250000 2021-03-26 08:22:40,936 epoch 34 - iter 6/25 - loss 3.76132822 - samples/sec: 105.13 - lr: 0.250000 2021-03-26 08:22:42,158 epoch 34 - iter 8/25 - loss 3.62088448 - samples/sec: 104.94 - lr: 0.250000 2021-03-26 08:22:43,156 epoch 34 - iter 10/25 - loss 3.64314551 - samples/sec: 128.41 - lr: 0.250000 2021-03-26 08:22:44,166 epoch 34 - iter 12/25 - loss 3.64726754 - samples/sec: 126.97 - lr: 0.250000 2021-03-26 08:22:45,122 epoch 34 - iter 14/25 - loss 3.60614933 - samples/sec: 134.05 - lr: 0.250000 2021-03-26 08:22:46,180 epoch 34 - iter 16/25 - loss 3.61291116 - samples/sec: 121.17 - lr: 0.250000 2021-03-26 08:22:47,315 epoch 34 - iter 18/25 - loss 3.66392313 - samples/sec: 112.94 - lr: 0.250000 2021-03-26 08:22:48,344 epoch 34 - iter 20/25 - loss 3.59719365 - samples/sec: 124.61 - lr: 0.250000 2021-03-26 08:22:49,384 epoch 34 - iter 22/25 - loss 3.57430128 - samples/sec: 123.23 - lr: 0.250000 2021-03-26 08:22:50,325 epoch 34 - iter 24/25 - loss 3.55224803 - samples/sec: 136.42 - lr: 0.250000 2021-03-26 08:22:50,787 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:22:50,787 EPOCH 34 done: loss 3.5453 - lr 0.2500000 2021-03-26 08:22:51,512 DEV : loss 5.184334754943848 - score 0.9107 2021-03-26 08:22:51,535 BAD EPOCHS (no improvement): 2 2021-03-26 08:22:51,537 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:22:52,422 epoch 35 - iter 2/25 - loss 3.18492174 - samples/sec: 144.84 - lr: 0.250000 2021-03-26 08:22:53,467 epoch 35 - iter 4/25 - loss 3.13623476 - samples/sec: 122.76 - lr: 0.250000 2021-03-26 08:22:54,406 epoch 35 - iter 6/25 - loss 3.33380385 - samples/sec: 136.42 - lr: 0.250000 2021-03-26 08:22:55,431 epoch 35 - iter 8/25 - loss 3.39252409 - samples/sec: 125.11 - lr: 0.250000 2021-03-26 08:22:56,487 epoch 35 - iter 10/25 - loss 3.46284273 - samples/sec: 121.42 - lr: 0.250000 2021-03-26 08:22:57,444 epoch 35 - iter 12/25 - loss 3.38363576 - samples/sec: 133.89 - lr: 0.250000 2021-03-26 08:22:58,488 epoch 35 - iter 14/25 - loss 3.38682234 - samples/sec: 122.86 - lr: 0.250000 2021-03-26 08:22:59,420 epoch 35 - iter 16/25 - loss 3.45151280 - samples/sec: 137.47 - lr: 0.250000 2021-03-26 08:23:00,467 epoch 35 - iter 18/25 - loss 3.41133621 - samples/sec: 122.37 - lr: 0.250000 2021-03-26 08:23:01,445 epoch 35 - iter 20/25 - loss 3.35318577 - samples/sec: 131.19 - lr: 0.250000 2021-03-26 08:23:02,375 epoch 35 - iter 22/25 - loss 3.32299045 - samples/sec: 137.74 - lr: 0.250000 2021-03-26 08:23:03,361 epoch 35 - iter 24/25 - loss 3.32118590 - samples/sec: 130.07 - lr: 0.250000 2021-03-26 08:23:03,791 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:23:03,792 EPOCH 35 done: loss 3.3499 - lr 0.2500000 2021-03-26 08:23:04,524 DEV : loss 5.07149600982666 - score 0.9118 2021-03-26 08:23:04,547 BAD EPOCHS (no improvement): 3 2021-03-26 08:23:04,548 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:23:05,621 epoch 36 - iter 2/25 - loss 2.85354340 - samples/sec: 119.48 - lr: 0.250000 2021-03-26 08:23:06,611 epoch 36 - iter 4/25 - loss 2.93653178 - samples/sec: 129.47 - lr: 0.250000 2021-03-26 08:23:07,630 epoch 36 - iter 6/25 - loss 2.91629008 - samples/sec: 125.87 - lr: 0.250000 2021-03-26 08:23:08,692 epoch 36 - iter 8/25 - loss 3.09478921 - samples/sec: 120.69 - lr: 0.250000 2021-03-26 08:23:09,798 epoch 36 - iter 10/25 - loss 3.04699903 - samples/sec: 115.81 - lr: 0.250000 2021-03-26 08:23:10,807 epoch 36 - iter 12/25 - loss 3.05647022 - samples/sec: 127.10 - lr: 0.250000 2021-03-26 08:23:11,789 epoch 36 - iter 14/25 - loss 2.99877063 - samples/sec: 130.50 - lr: 0.250000 2021-03-26 08:23:12,815 epoch 36 - iter 16/25 - loss 3.09249909 - samples/sec: 124.87 - lr: 0.250000 2021-03-26 08:23:13,775 epoch 36 - iter 18/25 - loss 3.12907799 - samples/sec: 133.47 - lr: 0.250000 2021-03-26 08:23:14,788 epoch 36 - iter 20/25 - loss 3.14141412 - samples/sec: 126.61 - lr: 0.250000 2021-03-26 08:23:15,757 epoch 36 - iter 22/25 - loss 3.18839633 - samples/sec: 132.35 - lr: 0.250000 2021-03-26 08:23:16,740 epoch 36 - iter 24/25 - loss 3.19035632 - samples/sec: 130.44 - lr: 0.250000 2021-03-26 08:23:17,177 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:23:17,178 EPOCH 36 done: loss 3.1967 - lr 0.2500000 2021-03-26 08:23:17,868 DEV : loss 5.2465972900390625 - score 0.9126 2021-03-26 08:23:17,888 BAD EPOCHS (no improvement): 4 2021-03-26 08:23:17,889 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:23:18,843 epoch 37 - iter 2/25 - loss 2.63817966 - samples/sec: 134.29 - lr: 0.125000 2021-03-26 08:23:19,830 epoch 37 - iter 4/25 - loss 3.05239159 - samples/sec: 129.86 - lr: 0.125000 2021-03-26 08:23:20,858 epoch 37 - iter 6/25 - loss 3.11489769 - samples/sec: 124.70 - lr: 0.125000 2021-03-26 08:23:21,785 epoch 37 - iter 8/25 - loss 3.18249029 - samples/sec: 138.38 - lr: 0.125000 2021-03-26 08:23:22,760 epoch 37 - iter 10/25 - loss 3.03278530 - samples/sec: 131.41 - lr: 0.125000 2021-03-26 08:23:23,670 epoch 37 - iter 12/25 - loss 3.05815605 - samples/sec: 140.94 - lr: 0.125000 2021-03-26 08:23:24,612 epoch 37 - iter 14/25 - loss 3.04020321 - samples/sec: 136.11 - lr: 0.125000 2021-03-26 08:23:25,562 epoch 37 - iter 16/25 - loss 3.10066675 - samples/sec: 134.96 - lr: 0.125000 2021-03-26 08:23:26,612 epoch 37 - iter 18/25 - loss 3.06573884 - samples/sec: 122.02 - lr: 0.125000 2021-03-26 08:23:27,624 epoch 37 - iter 20/25 - loss 3.08411411 - samples/sec: 126.79 - lr: 0.125000 2021-03-26 08:23:28,687 epoch 37 - iter 22/25 - loss 3.08923004 - samples/sec: 120.68 - lr: 0.125000 2021-03-26 08:23:29,669 epoch 37 - iter 24/25 - loss 3.10842060 - samples/sec: 130.51 - lr: 0.125000 2021-03-26 08:23:30,109 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:23:30,110 EPOCH 37 done: loss 3.0953 - lr 0.1250000 2021-03-26 08:23:30,839 DEV : loss 5.055671691894531 - score 0.916 2021-03-26 08:23:30,862 BAD EPOCHS (no improvement): 0 2021-03-26 08:23:40,200 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:23:41,278 epoch 38 - iter 2/25 - loss 2.59178650 - samples/sec: 118.89 - lr: 0.125000 2021-03-26 08:23:42,280 epoch 38 - iter 4/25 - loss 2.56540596 - samples/sec: 128.03 - lr: 0.125000 2021-03-26 08:23:43,320 epoch 38 - iter 6/25 - loss 2.72324753 - samples/sec: 123.14 - lr: 0.125000 2021-03-26 08:23:44,333 epoch 38 - iter 8/25 - loss 2.74542779 - samples/sec: 126.62 - lr: 0.125000 2021-03-26 08:23:45,362 epoch 38 - iter 10/25 - loss 2.80854850 - samples/sec: 124.49 - lr: 0.125000 2021-03-26 08:23:46,316 epoch 38 - iter 12/25 - loss 2.78986889 - samples/sec: 134.39 - lr: 0.125000 2021-03-26 08:23:47,337 epoch 38 - iter 14/25 - loss 2.87522815 - samples/sec: 125.58 - lr: 0.125000 2021-03-26 08:23:48,349 epoch 38 - iter 16/25 - loss 2.80133051 - samples/sec: 126.72 - lr: 0.125000 2021-03-26 08:23:49,440 epoch 38 - iter 18/25 - loss 2.84491060 - samples/sec: 117.52 - lr: 0.125000 2021-03-26 08:23:50,451 epoch 38 - iter 20/25 - loss 2.85927484 - samples/sec: 126.83 - lr: 0.125000 2021-03-26 08:23:51,373 epoch 38 - iter 22/25 - loss 2.87918811 - samples/sec: 139.14 - lr: 0.125000 2021-03-26 08:23:52,329 epoch 38 - iter 24/25 - loss 2.89003314 - samples/sec: 134.05 - lr: 0.125000 2021-03-26 08:23:52,707 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:23:52,708 EPOCH 38 done: loss 2.9956 - lr 0.1250000 2021-03-26 08:23:53,446 DEV : loss 4.992068290710449 - score 0.9194 2021-03-26 08:23:53,469 BAD EPOCHS (no improvement): 0 2021-03-26 08:24:03,131 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:24:04,213 epoch 39 - iter 2/25 - loss 3.26761401 - samples/sec: 118.64 - lr: 0.125000 2021-03-26 08:24:05,228 epoch 39 - iter 4/25 - loss 3.34832674 - samples/sec: 126.24 - lr: 0.125000 2021-03-26 08:24:06,237 epoch 39 - iter 6/25 - loss 3.22142450 - samples/sec: 127.10 - lr: 0.125000 2021-03-26 08:24:07,128 epoch 39 - iter 8/25 - loss 3.15339360 - samples/sec: 143.79 - lr: 0.125000 2021-03-26 08:24:07,991 epoch 39 - iter 10/25 - loss 3.05391240 - samples/sec: 148.65 - lr: 0.125000 2021-03-26 08:24:08,889 epoch 39 - iter 12/25 - loss 3.06373362 - samples/sec: 142.63 - lr: 0.125000 2021-03-26 08:24:09,803 epoch 39 - iter 14/25 - loss 2.98908319 - samples/sec: 140.15 - lr: 0.125000 2021-03-26 08:24:10,765 epoch 39 - iter 16/25 - loss 2.94973241 - samples/sec: 133.37 - lr: 0.125000 2021-03-26 08:24:11,716 epoch 39 - iter 18/25 - loss 2.95112207 - samples/sec: 134.86 - lr: 0.125000 2021-03-26 08:24:12,764 epoch 39 - iter 20/25 - loss 2.91271211 - samples/sec: 122.28 - lr: 0.125000 2021-03-26 08:24:13,775 epoch 39 - iter 22/25 - loss 2.89700104 - samples/sec: 126.79 - lr: 0.125000 2021-03-26 08:24:14,756 epoch 39 - iter 24/25 - loss 2.84286279 - samples/sec: 130.68 - lr: 0.125000 2021-03-26 08:24:15,185 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:24:15,186 EPOCH 39 done: loss 2.8496 - lr 0.1250000 2021-03-26 08:24:15,932 DEV : loss 5.133090972900391 - score 0.9143 2021-03-26 08:24:15,969 BAD EPOCHS (no improvement): 1 2021-03-26 08:24:15,970 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:24:16,996 epoch 40 - iter 2/25 - loss 3.13895917 - samples/sec: 124.91 - lr: 0.125000 2021-03-26 08:24:17,947 epoch 40 - iter 4/25 - loss 2.89032578 - samples/sec: 134.86 - lr: 0.125000 2021-03-26 08:24:18,982 epoch 40 - iter 6/25 - loss 2.81307308 - samples/sec: 123.74 - lr: 0.125000 2021-03-26 08:24:19,989 epoch 40 - iter 8/25 - loss 2.90850329 - samples/sec: 127.39 - lr: 0.125000 2021-03-26 08:24:21,007 epoch 40 - iter 10/25 - loss 2.93947659 - samples/sec: 125.86 - lr: 0.125000 2021-03-26 08:24:21,987 epoch 40 - iter 12/25 - loss 2.84735465 - samples/sec: 130.83 - lr: 0.125000 2021-03-26 08:24:23,164 epoch 40 - iter 14/25 - loss 2.84274534 - samples/sec: 108.83 - lr: 0.125000 2021-03-26 08:24:24,275 epoch 40 - iter 16/25 - loss 2.84609564 - samples/sec: 115.32 - lr: 0.125000 2021-03-26 08:24:25,291 epoch 40 - iter 18/25 - loss 2.87500971 - samples/sec: 126.18 - lr: 0.125000 2021-03-26 08:24:26,308 epoch 40 - iter 20/25 - loss 2.83314930 - samples/sec: 125.95 - lr: 0.125000 2021-03-26 08:24:27,419 epoch 40 - iter 22/25 - loss 2.86485132 - samples/sec: 115.38 - lr: 0.125000 2021-03-26 08:24:28,389 epoch 40 - iter 24/25 - loss 2.89132403 - samples/sec: 132.13 - lr: 0.125000 2021-03-26 08:24:28,789 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:24:28,789 EPOCH 40 done: loss 2.9279 - lr 0.1250000 2021-03-26 08:24:29,536 DEV : loss 5.163059234619141 - score 0.916 2021-03-26 08:24:29,559 BAD EPOCHS (no improvement): 2 2021-03-26 08:24:29,560 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:24:30,520 epoch 41 - iter 2/25 - loss 3.34717524 - samples/sec: 133.59 - lr: 0.125000 2021-03-26 08:24:31,558 epoch 41 - iter 4/25 - loss 3.11228681 - samples/sec: 123.52 - lr: 0.125000 2021-03-26 08:24:32,512 epoch 41 - iter 6/25 - loss 2.98143824 - samples/sec: 134.40 - lr: 0.125000 2021-03-26 08:24:33,467 epoch 41 - iter 8/25 - loss 3.10384235 - samples/sec: 134.12 - lr: 0.125000 2021-03-26 08:24:34,426 epoch 41 - iter 10/25 - loss 2.93525398 - samples/sec: 133.85 - lr: 0.125000 2021-03-26 08:24:35,417 epoch 41 - iter 12/25 - loss 2.92955963 - samples/sec: 129.33 - lr: 0.125000 2021-03-26 08:24:36,439 epoch 41 - iter 14/25 - loss 2.94683690 - samples/sec: 125.41 - lr: 0.125000 2021-03-26 08:24:37,464 epoch 41 - iter 16/25 - loss 2.97896834 - samples/sec: 125.13 - lr: 0.125000 2021-03-26 08:24:38,461 epoch 41 - iter 18/25 - loss 2.97161441 - samples/sec: 128.58 - lr: 0.125000 2021-03-26 08:24:39,423 epoch 41 - iter 20/25 - loss 2.96228577 - samples/sec: 133.37 - lr: 0.125000 2021-03-26 08:24:40,368 epoch 41 - iter 22/25 - loss 2.91309004 - samples/sec: 135.67 - lr: 0.125000 2021-03-26 08:24:41,327 epoch 41 - iter 24/25 - loss 2.89624717 - samples/sec: 133.65 - lr: 0.125000 2021-03-26 08:24:41,760 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:24:41,761 EPOCH 41 done: loss 2.9257 - lr 0.1250000 2021-03-26 08:24:42,479 DEV : loss 5.18552303314209 - score 0.9135 2021-03-26 08:24:42,494 BAD EPOCHS (no improvement): 3 2021-03-26 08:24:42,495 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:24:43,540 epoch 42 - iter 2/25 - loss 3.23972845 - samples/sec: 122.69 - lr: 0.125000 2021-03-26 08:24:44,629 epoch 42 - iter 4/25 - loss 3.31910771 - samples/sec: 117.69 - lr: 0.125000 2021-03-26 08:24:45,575 epoch 42 - iter 6/25 - loss 3.12779403 - samples/sec: 135.67 - lr: 0.125000 2021-03-26 08:24:46,580 epoch 42 - iter 8/25 - loss 3.03878088 - samples/sec: 127.50 - lr: 0.125000 2021-03-26 08:24:47,585 epoch 42 - iter 10/25 - loss 2.97357968 - samples/sec: 127.57 - lr: 0.125000 2021-03-26 08:24:48,577 epoch 42 - iter 12/25 - loss 2.94139670 - samples/sec: 129.32 - lr: 0.125000 2021-03-26 08:24:49,464 epoch 42 - iter 14/25 - loss 2.87050148 - samples/sec: 144.50 - lr: 0.125000 2021-03-26 08:24:50,514 epoch 42 - iter 16/25 - loss 2.81032179 - samples/sec: 122.15 - lr: 0.125000 2021-03-26 08:24:51,558 epoch 42 - iter 18/25 - loss 2.79874009 - samples/sec: 122.72 - lr: 0.125000 2021-03-26 08:24:52,550 epoch 42 - iter 20/25 - loss 2.83001460 - samples/sec: 129.39 - lr: 0.125000 2021-03-26 08:24:53,508 epoch 42 - iter 22/25 - loss 2.81053819 - samples/sec: 133.73 - lr: 0.125000 2021-03-26 08:24:54,510 epoch 42 - iter 24/25 - loss 2.81009710 - samples/sec: 128.03 - lr: 0.125000 2021-03-26 08:24:54,943 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:24:54,944 EPOCH 42 done: loss 2.8110 - lr 0.1250000 2021-03-26 08:24:55,686 DEV : loss 5.17966365814209 - score 0.9143 2021-03-26 08:24:55,704 BAD EPOCHS (no improvement): 4 2021-03-26 08:24:55,705 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:24:56,788 epoch 43 - iter 2/25 - loss 2.25671196 - samples/sec: 118.30 - lr: 0.062500 2021-03-26 08:24:57,782 epoch 43 - iter 4/25 - loss 2.48410356 - samples/sec: 129.00 - lr: 0.062500 2021-03-26 08:24:58,870 epoch 43 - iter 6/25 - loss 2.70733134 - samples/sec: 117.79 - lr: 0.062500 2021-03-26 08:24:59,818 epoch 43 - iter 8/25 - loss 2.80536109 - samples/sec: 135.18 - lr: 0.062500 2021-03-26 08:25:00,859 epoch 43 - iter 10/25 - loss 2.89832189 - samples/sec: 123.16 - lr: 0.062500 2021-03-26 08:25:01,864 epoch 43 - iter 12/25 - loss 2.77370048 - samples/sec: 127.56 - lr: 0.062500 2021-03-26 08:25:02,706 epoch 43 - iter 14/25 - loss 2.72336340 - samples/sec: 152.15 - lr: 0.062500 2021-03-26 08:25:03,721 epoch 43 - iter 16/25 - loss 2.76130933 - samples/sec: 126.35 - lr: 0.062500 2021-03-26 08:25:04,708 epoch 43 - iter 18/25 - loss 2.74378959 - samples/sec: 129.93 - lr: 0.062500 2021-03-26 08:25:05,670 epoch 43 - iter 20/25 - loss 2.73068177 - samples/sec: 133.23 - lr: 0.062500 2021-03-26 08:25:06,588 epoch 43 - iter 22/25 - loss 2.75959666 - samples/sec: 139.61 - lr: 0.062500 2021-03-26 08:25:07,534 epoch 43 - iter 24/25 - loss 2.79225791 - samples/sec: 135.51 - lr: 0.062500 2021-03-26 08:25:07,958 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:25:07,959 EPOCH 43 done: loss 2.7573 - lr 0.0625000 2021-03-26 08:25:08,683 DEV : loss 5.191972255706787 - score 0.9143 2021-03-26 08:25:08,697 BAD EPOCHS (no improvement): 1 2021-03-26 08:25:08,698 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:25:09,707 epoch 44 - iter 2/25 - loss 2.51410568 - samples/sec: 127.08 - lr: 0.062500 2021-03-26 08:25:10,828 epoch 44 - iter 4/25 - loss 2.87437314 - samples/sec: 114.42 - lr: 0.062500 2021-03-26 08:25:11,853 epoch 44 - iter 6/25 - loss 2.97266304 - samples/sec: 125.03 - lr: 0.062500 2021-03-26 08:25:12,832 epoch 44 - iter 8/25 - loss 2.98653641 - samples/sec: 131.11 - lr: 0.062500 2021-03-26 08:25:13,815 epoch 44 - iter 10/25 - loss 2.91581733 - samples/sec: 130.50 - lr: 0.062500 2021-03-26 08:25:14,819 epoch 44 - iter 12/25 - loss 2.88142518 - samples/sec: 127.70 - lr: 0.062500 2021-03-26 08:25:15,785 epoch 44 - iter 14/25 - loss 2.80705702 - samples/sec: 132.71 - lr: 0.062500 2021-03-26 08:25:16,714 epoch 44 - iter 16/25 - loss 2.71279316 - samples/sec: 137.94 - lr: 0.062500 2021-03-26 08:25:17,711 epoch 44 - iter 18/25 - loss 2.69600504 - samples/sec: 128.74 - lr: 0.062500 2021-03-26 08:25:18,938 epoch 44 - iter 20/25 - loss 2.71131549 - samples/sec: 104.38 - lr: 0.062500 2021-03-26 08:25:20,177 epoch 44 - iter 22/25 - loss 2.72818206 - samples/sec: 103.46 - lr: 0.062500 2021-03-26 08:25:21,206 epoch 44 - iter 24/25 - loss 2.70353389 - samples/sec: 124.55 - lr: 0.062500 2021-03-26 08:25:21,749 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:25:21,750 EPOCH 44 done: loss 2.6863 - lr 0.0625000 2021-03-26 08:25:22,577 DEV : loss 5.165666103363037 - score 0.916 2021-03-26 08:25:22,607 BAD EPOCHS (no improvement): 2 2021-03-26 08:25:22,608 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:25:24,008 epoch 45 - iter 2/25 - loss 2.86002135 - samples/sec: 91.55 - lr: 0.062500 2021-03-26 08:25:25,184 epoch 45 - iter 4/25 - loss 2.98223376 - samples/sec: 109.01 - lr: 0.062500 2021-03-26 08:25:26,187 epoch 45 - iter 6/25 - loss 2.88732882 - samples/sec: 127.78 - lr: 0.062500 2021-03-26 08:25:27,263 epoch 45 - iter 8/25 - loss 2.85796985 - samples/sec: 119.14 - lr: 0.062500 2021-03-26 08:25:28,165 epoch 45 - iter 10/25 - loss 2.75072234 - samples/sec: 142.08 - lr: 0.062500 2021-03-26 08:25:29,082 epoch 45 - iter 12/25 - loss 2.71562586 - samples/sec: 139.99 - lr: 0.062500 2021-03-26 08:25:30,055 epoch 45 - iter 14/25 - loss 2.65661091 - samples/sec: 131.71 - lr: 0.062500 2021-03-26 08:25:31,024 epoch 45 - iter 16/25 - loss 2.68994868 - samples/sec: 132.37 - lr: 0.062500 2021-03-26 08:25:31,946 epoch 45 - iter 18/25 - loss 2.70727919 - samples/sec: 138.91 - lr: 0.062500 2021-03-26 08:25:33,055 epoch 45 - iter 20/25 - loss 2.72820472 - samples/sec: 115.57 - lr: 0.062500 2021-03-26 08:25:34,015 epoch 45 - iter 22/25 - loss 2.69422464 - samples/sec: 133.64 - lr: 0.062500 2021-03-26 08:25:34,973 epoch 45 - iter 24/25 - loss 2.64335350 - samples/sec: 133.73 - lr: 0.062500 2021-03-26 08:25:35,351 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:25:35,352 EPOCH 45 done: loss 2.7020 - lr 0.0625000 2021-03-26 08:25:36,079 DEV : loss 5.152807235717773 - score 0.9135 2021-03-26 08:25:36,094 BAD EPOCHS (no improvement): 3 2021-03-26 08:25:36,095 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:25:37,133 epoch 46 - iter 2/25 - loss 3.18182039 - samples/sec: 123.48 - lr: 0.062500 2021-03-26 08:25:38,117 epoch 46 - iter 4/25 - loss 2.93405211 - samples/sec: 130.35 - lr: 0.062500 2021-03-26 08:25:39,285 epoch 46 - iter 6/25 - loss 2.85977014 - samples/sec: 109.66 - lr: 0.062500 2021-03-26 08:25:40,307 epoch 46 - iter 8/25 - loss 2.68965745 - samples/sec: 125.43 - lr: 0.062500 2021-03-26 08:25:41,269 epoch 46 - iter 10/25 - loss 2.54557719 - samples/sec: 133.24 - lr: 0.062500 2021-03-26 08:25:42,284 epoch 46 - iter 12/25 - loss 2.54885741 - samples/sec: 126.37 - lr: 0.062500 2021-03-26 08:25:43,231 epoch 46 - iter 14/25 - loss 2.59709317 - samples/sec: 135.37 - lr: 0.062500 2021-03-26 08:25:44,319 epoch 46 - iter 16/25 - loss 2.56221853 - samples/sec: 117.78 - lr: 0.062500 2021-03-26 08:25:45,403 epoch 46 - iter 18/25 - loss 2.59396293 - samples/sec: 118.31 - lr: 0.062500 2021-03-26 08:25:46,429 epoch 46 - iter 20/25 - loss 2.58809073 - samples/sec: 124.90 - lr: 0.062500 2021-03-26 08:25:47,507 epoch 46 - iter 22/25 - loss 2.63570851 - samples/sec: 118.93 - lr: 0.062500 2021-03-26 08:25:48,494 epoch 46 - iter 24/25 - loss 2.62493981 - samples/sec: 129.93 - lr: 0.062500 2021-03-26 08:25:48,846 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:25:48,847 EPOCH 46 done: loss 2.6255 - lr 0.0625000 2021-03-26 08:25:49,580 DEV : loss 5.130050182342529 - score 0.9175 2021-03-26 08:25:49,595 BAD EPOCHS (no improvement): 4 2021-03-26 08:25:49,595 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:25:50,572 epoch 47 - iter 2/25 - loss 2.34120893 - samples/sec: 131.29 - lr: 0.031250 2021-03-26 08:25:51,518 epoch 47 - iter 4/25 - loss 2.56537151 - samples/sec: 135.59 - lr: 0.031250 2021-03-26 08:25:52,459 epoch 47 - iter 6/25 - loss 2.50796159 - samples/sec: 136.28 - lr: 0.031250 2021-03-26 08:25:53,484 epoch 47 - iter 8/25 - loss 2.43816668 - samples/sec: 124.99 - lr: 0.031250 2021-03-26 08:25:54,419 epoch 47 - iter 10/25 - loss 2.44248054 - samples/sec: 137.16 - lr: 0.031250 2021-03-26 08:25:55,379 epoch 47 - iter 12/25 - loss 2.43599850 - samples/sec: 133.59 - lr: 0.031250 2021-03-26 08:25:56,413 epoch 47 - iter 14/25 - loss 2.45250283 - samples/sec: 123.96 - lr: 0.031250 2021-03-26 08:25:57,389 epoch 47 - iter 16/25 - loss 2.49436748 - samples/sec: 131.28 - lr: 0.031250 2021-03-26 08:25:58,493 epoch 47 - iter 18/25 - loss 2.55170678 - samples/sec: 116.04 - lr: 0.031250 2021-03-26 08:25:59,541 epoch 47 - iter 20/25 - loss 2.54269922 - samples/sec: 122.32 - lr: 0.031250 2021-03-26 08:26:00,420 epoch 47 - iter 22/25 - loss 2.54198781 - samples/sec: 145.87 - lr: 0.031250 2021-03-26 08:26:01,372 epoch 47 - iter 24/25 - loss 2.59515851 - samples/sec: 134.72 - lr: 0.031250 2021-03-26 08:26:01,809 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:26:01,809 EPOCH 47 done: loss 2.5737 - lr 0.0312500 2021-03-26 08:26:02,527 DEV : loss 5.120047569274902 - score 0.9166 2021-03-26 08:26:02,550 BAD EPOCHS (no improvement): 1 2021-03-26 08:26:02,551 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:26:03,525 epoch 48 - iter 2/25 - loss 2.52614760 - samples/sec: 131.64 - lr: 0.031250 2021-03-26 08:26:04,490 epoch 48 - iter 4/25 - loss 2.53924924 - samples/sec: 133.02 - lr: 0.031250 2021-03-26 08:26:05,527 epoch 48 - iter 6/25 - loss 2.57990440 - samples/sec: 123.51 - lr: 0.031250 2021-03-26 08:26:06,504 epoch 48 - iter 8/25 - loss 2.52673838 - samples/sec: 131.27 - lr: 0.031250 2021-03-26 08:26:07,600 epoch 48 - iter 10/25 - loss 2.44827111 - samples/sec: 116.95 - lr: 0.031250 2021-03-26 08:26:08,585 epoch 48 - iter 12/25 - loss 2.40130955 - samples/sec: 130.11 - lr: 0.031250 2021-03-26 08:26:09,552 epoch 48 - iter 14/25 - loss 2.42768495 - samples/sec: 132.66 - lr: 0.031250 2021-03-26 08:26:10,464 epoch 48 - iter 16/25 - loss 2.40806086 - samples/sec: 140.60 - lr: 0.031250 2021-03-26 08:26:11,491 epoch 48 - iter 18/25 - loss 2.41822616 - samples/sec: 124.78 - lr: 0.031250 2021-03-26 08:26:12,395 epoch 48 - iter 20/25 - loss 2.46524236 - samples/sec: 141.84 - lr: 0.031250 2021-03-26 08:26:13,342 epoch 48 - iter 22/25 - loss 2.53471144 - samples/sec: 135.26 - lr: 0.031250 2021-03-26 08:26:14,413 epoch 48 - iter 24/25 - loss 2.56991556 - samples/sec: 119.74 - lr: 0.031250 2021-03-26 08:26:14,822 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:26:14,823 EPOCH 48 done: loss 2.5892 - lr 0.0312500 2021-03-26 08:26:15,515 DEV : loss 5.128034591674805 - score 0.9158 2021-03-26 08:26:15,537 BAD EPOCHS (no improvement): 2 2021-03-26 08:26:15,538 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:26:16,442 epoch 49 - iter 2/25 - loss 2.41402364 - samples/sec: 141.76 - lr: 0.031250 2021-03-26 08:26:17,409 epoch 49 - iter 4/25 - loss 2.56433320 - samples/sec: 132.75 - lr: 0.031250 2021-03-26 08:26:18,302 epoch 49 - iter 6/25 - loss 2.50378040 - samples/sec: 143.54 - lr: 0.031250 2021-03-26 08:26:19,275 epoch 49 - iter 8/25 - loss 2.42913982 - samples/sec: 131.70 - lr: 0.031250 2021-03-26 08:26:20,310 epoch 49 - iter 10/25 - loss 2.62541809 - samples/sec: 123.92 - lr: 0.031250 2021-03-26 08:26:21,371 epoch 49 - iter 12/25 - loss 2.64117569 - samples/sec: 120.87 - lr: 0.031250 2021-03-26 08:26:22,456 epoch 49 - iter 14/25 - loss 2.62500211 - samples/sec: 118.20 - lr: 0.031250 2021-03-26 08:26:23,494 epoch 49 - iter 16/25 - loss 2.64282882 - samples/sec: 123.41 - lr: 0.031250 2021-03-26 08:26:24,438 epoch 49 - iter 18/25 - loss 2.60936556 - samples/sec: 135.90 - lr: 0.031250 2021-03-26 08:26:25,389 epoch 49 - iter 20/25 - loss 2.64132748 - samples/sec: 134.74 - lr: 0.031250 2021-03-26 08:26:26,456 epoch 49 - iter 22/25 - loss 2.62381291 - samples/sec: 120.09 - lr: 0.031250 2021-03-26 08:26:27,445 epoch 49 - iter 24/25 - loss 2.61296557 - samples/sec: 129.65 - lr: 0.031250 2021-03-26 08:26:27,895 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:26:27,895 EPOCH 49 done: loss 2.5931 - lr 0.0312500 2021-03-26 08:26:28,643 DEV : loss 5.125513076782227 - score 0.9158 2021-03-26 08:26:28,666 BAD EPOCHS (no improvement): 3 2021-03-26 08:26:28,667 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:26:29,652 epoch 50 - iter 2/25 - loss 2.38679540 - samples/sec: 130.08 - lr: 0.031250 2021-03-26 08:26:30,732 epoch 50 - iter 4/25 - loss 2.58361644 - samples/sec: 118.71 - lr: 0.031250 2021-03-26 08:26:31,603 epoch 50 - iter 6/25 - loss 2.50680178 - samples/sec: 147.16 - lr: 0.031250 2021-03-26 08:26:32,510 epoch 50 - iter 8/25 - loss 2.59342428 - samples/sec: 141.43 - lr: 0.031250 2021-03-26 08:26:33,516 epoch 50 - iter 10/25 - loss 2.53629299 - samples/sec: 127.36 - lr: 0.031250 2021-03-26 08:26:34,560 epoch 50 - iter 12/25 - loss 2.62270388 - samples/sec: 122.94 - lr: 0.031250 2021-03-26 08:26:35,685 epoch 50 - iter 14/25 - loss 2.70660563 - samples/sec: 113.94 - lr: 0.031250 2021-03-26 08:26:36,688 epoch 50 - iter 16/25 - loss 2.69249507 - samples/sec: 127.75 - lr: 0.031250 2021-03-26 08:26:37,739 epoch 50 - iter 18/25 - loss 2.66604349 - samples/sec: 121.97 - lr: 0.031250 2021-03-26 08:26:38,816 epoch 50 - iter 20/25 - loss 2.68338645 - samples/sec: 119.00 - lr: 0.031250 2021-03-26 08:26:39,829 epoch 50 - iter 22/25 - loss 2.65240399 - samples/sec: 126.52 - lr: 0.031250 2021-03-26 08:26:40,821 epoch 50 - iter 24/25 - loss 2.65005972 - samples/sec: 129.30 - lr: 0.031250 2021-03-26 08:26:41,250 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:26:41,251 EPOCH 50 done: loss 2.7365 - lr 0.0312500 2021-03-26 08:26:41,962 DEV : loss 5.108526706695557 - score 0.9192 2021-03-26 08:26:41,985 BAD EPOCHS (no improvement): 4 2021-03-26 08:26:41,986 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:26:43,035 epoch 51 - iter 2/25 - loss 3.02723408 - samples/sec: 122.25 - lr: 0.015625 2021-03-26 08:26:44,142 epoch 51 - iter 4/25 - loss 2.87706113 - samples/sec: 115.76 - lr: 0.015625 2021-03-26 08:26:45,109 epoch 51 - iter 6/25 - loss 2.77404606 - samples/sec: 132.55 - lr: 0.015625 2021-03-26 08:26:46,056 epoch 51 - iter 8/25 - loss 2.63912219 - samples/sec: 135.39 - lr: 0.015625 2021-03-26 08:26:47,044 epoch 51 - iter 10/25 - loss 2.55065244 - samples/sec: 129.81 - lr: 0.015625 2021-03-26 08:26:48,022 epoch 51 - iter 12/25 - loss 2.45815141 - samples/sec: 131.01 - lr: 0.015625 2021-03-26 08:26:48,952 epoch 51 - iter 14/25 - loss 2.44239549 - samples/sec: 137.90 - lr: 0.015625 2021-03-26 08:26:49,983 epoch 51 - iter 16/25 - loss 2.48094446 - samples/sec: 124.32 - lr: 0.015625 2021-03-26 08:26:50,892 epoch 51 - iter 18/25 - loss 2.46754158 - samples/sec: 141.04 - lr: 0.015625 2021-03-26 08:26:51,901 epoch 51 - iter 20/25 - loss 2.51611657 - samples/sec: 127.05 - lr: 0.015625 2021-03-26 08:26:52,866 epoch 51 - iter 22/25 - loss 2.47020188 - samples/sec: 132.90 - lr: 0.015625 2021-03-26 08:26:53,750 epoch 51 - iter 24/25 - loss 2.47272858 - samples/sec: 145.09 - lr: 0.015625 2021-03-26 08:26:54,155 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:26:54,157 EPOCH 51 done: loss 2.4645 - lr 0.0156250 2021-03-26 08:26:54,882 DEV : loss 5.136854648590088 - score 0.9192 2021-03-26 08:26:54,897 BAD EPOCHS (no improvement): 1 2021-03-26 08:26:54,898 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:26:55,863 epoch 52 - iter 2/25 - loss 2.48442078 - samples/sec: 132.85 - lr: 0.015625 2021-03-26 08:26:56,817 epoch 52 - iter 4/25 - loss 2.43368804 - samples/sec: 134.33 - lr: 0.015625 2021-03-26 08:26:57,833 epoch 52 - iter 6/25 - loss 2.57455993 - samples/sec: 126.25 - lr: 0.015625 2021-03-26 08:26:58,839 epoch 52 - iter 8/25 - loss 2.46462786 - samples/sec: 127.35 - lr: 0.015625 2021-03-26 08:26:59,814 epoch 52 - iter 10/25 - loss 2.44867802 - samples/sec: 131.52 - lr: 0.015625 2021-03-26 08:27:00,853 epoch 52 - iter 12/25 - loss 2.39227523 - samples/sec: 123.43 - lr: 0.015625 2021-03-26 08:27:01,798 epoch 52 - iter 14/25 - loss 2.47848124 - samples/sec: 135.69 - lr: 0.015625 2021-03-26 08:27:02,804 epoch 52 - iter 16/25 - loss 2.57754317 - samples/sec: 127.33 - lr: 0.015625 2021-03-26 08:27:03,814 epoch 52 - iter 18/25 - loss 2.58900766 - samples/sec: 126.99 - lr: 0.015625 2021-03-26 08:27:04,879 epoch 52 - iter 20/25 - loss 2.58191739 - samples/sec: 120.40 - lr: 0.015625 2021-03-26 08:27:05,879 epoch 52 - iter 22/25 - loss 2.58063118 - samples/sec: 128.32 - lr: 0.015625 2021-03-26 08:27:06,834 epoch 52 - iter 24/25 - loss 2.57483467 - samples/sec: 134.17 - lr: 0.015625 2021-03-26 08:27:07,393 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:27:07,394 EPOCH 52 done: loss 2.5751 - lr 0.0156250 2021-03-26 08:27:09,180 DEV : loss 5.142533302307129 - score 0.9175 2021-03-26 08:27:09,195 BAD EPOCHS (no improvement): 2 2021-03-26 08:27:09,196 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:27:10,144 epoch 53 - iter 2/25 - loss 2.56813502 - samples/sec: 135.19 - lr: 0.015625 2021-03-26 08:27:11,149 epoch 53 - iter 4/25 - loss 2.46606696 - samples/sec: 127.61 - lr: 0.015625 2021-03-26 08:27:12,157 epoch 53 - iter 6/25 - loss 2.41871005 - samples/sec: 127.15 - lr: 0.015625 2021-03-26 08:27:13,250 epoch 53 - iter 8/25 - loss 2.33384241 - samples/sec: 117.30 - lr: 0.015625 2021-03-26 08:27:14,257 epoch 53 - iter 10/25 - loss 2.29487942 - samples/sec: 127.19 - lr: 0.015625 2021-03-26 08:27:15,252 epoch 53 - iter 12/25 - loss 2.33890817 - samples/sec: 128.85 - lr: 0.015625 2021-03-26 08:27:16,236 epoch 53 - iter 14/25 - loss 2.33417889 - samples/sec: 130.32 - lr: 0.015625 2021-03-26 08:27:17,230 epoch 53 - iter 16/25 - loss 2.37781139 - samples/sec: 128.96 - lr: 0.015625 2021-03-26 08:27:18,275 epoch 53 - iter 18/25 - loss 2.44906245 - samples/sec: 122.64 - lr: 0.015625 2021-03-26 08:27:19,250 epoch 53 - iter 20/25 - loss 2.49812522 - samples/sec: 131.47 - lr: 0.015625 2021-03-26 08:27:20,167 epoch 53 - iter 22/25 - loss 2.46865853 - samples/sec: 139.92 - lr: 0.015625 2021-03-26 08:27:21,113 epoch 53 - iter 24/25 - loss 2.51417203 - samples/sec: 135.49 - lr: 0.015625 2021-03-26 08:27:21,479 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:27:21,480 EPOCH 53 done: loss 2.4904 - lr 0.0156250 2021-03-26 08:27:22,198 DEV : loss 5.138154029846191 - score 0.9173 2021-03-26 08:27:22,223 BAD EPOCHS (no improvement): 3 2021-03-26 08:27:22,224 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:27:23,305 epoch 54 - iter 2/25 - loss 2.55291057 - samples/sec: 118.67 - lr: 0.015625 2021-03-26 08:27:24,306 epoch 54 - iter 4/25 - loss 2.38039738 - samples/sec: 128.05 - lr: 0.015625 2021-03-26 08:27:25,219 epoch 54 - iter 6/25 - loss 2.46816170 - samples/sec: 140.38 - lr: 0.015625 2021-03-26 08:27:26,232 epoch 54 - iter 8/25 - loss 2.57770142 - samples/sec: 126.63 - lr: 0.015625 2021-03-26 08:27:27,285 epoch 54 - iter 10/25 - loss 2.55315492 - samples/sec: 121.71 - lr: 0.015625 2021-03-26 08:27:28,352 epoch 54 - iter 12/25 - loss 2.57296405 - samples/sec: 120.11 - lr: 0.015625 2021-03-26 08:27:29,343 epoch 54 - iter 14/25 - loss 2.53743523 - samples/sec: 129.49 - lr: 0.015625 2021-03-26 08:27:30,254 epoch 54 - iter 16/25 - loss 2.55962446 - samples/sec: 140.64 - lr: 0.015625 2021-03-26 08:27:31,290 epoch 54 - iter 18/25 - loss 2.58272949 - samples/sec: 123.80 - lr: 0.015625 2021-03-26 08:27:32,325 epoch 54 - iter 20/25 - loss 2.56538886 - samples/sec: 123.75 - lr: 0.015625 2021-03-26 08:27:33,459 epoch 54 - iter 22/25 - loss 2.54301760 - samples/sec: 113.08 - lr: 0.015625 2021-03-26 08:27:34,489 epoch 54 - iter 24/25 - loss 2.54989679 - samples/sec: 124.46 - lr: 0.015625 2021-03-26 08:27:34,888 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:27:34,889 EPOCH 54 done: loss 2.5705 - lr 0.0156250 2021-03-26 08:27:35,606 DEV : loss 5.1629109382629395 - score 0.9149 2021-03-26 08:27:35,629 BAD EPOCHS (no improvement): 4 2021-03-26 08:27:35,629 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:27:36,630 epoch 55 - iter 2/25 - loss 3.01063824 - samples/sec: 128.15 - lr: 0.007812 2021-03-26 08:27:37,619 epoch 55 - iter 4/25 - loss 2.87859678 - samples/sec: 129.70 - lr: 0.007812 2021-03-26 08:27:38,620 epoch 55 - iter 6/25 - loss 2.72049872 - samples/sec: 128.02 - lr: 0.007812 2021-03-26 08:27:39,644 epoch 55 - iter 8/25 - loss 2.69411999 - samples/sec: 125.21 - lr: 0.007812 2021-03-26 08:27:40,542 epoch 55 - iter 10/25 - loss 2.62634434 - samples/sec: 142.75 - lr: 0.007812 2021-03-26 08:27:41,576 epoch 55 - iter 12/25 - loss 2.62428525 - samples/sec: 123.99 - lr: 0.007812 2021-03-26 08:27:42,613 epoch 55 - iter 14/25 - loss 2.71168633 - samples/sec: 123.68 - lr: 0.007812 2021-03-26 08:27:43,515 epoch 55 - iter 16/25 - loss 2.64237929 - samples/sec: 142.18 - lr: 0.007812 2021-03-26 08:27:44,608 epoch 55 - iter 18/25 - loss 2.62173416 - samples/sec: 117.26 - lr: 0.007812 2021-03-26 08:27:45,625 epoch 55 - iter 20/25 - loss 2.61856262 - samples/sec: 126.10 - lr: 0.007812 2021-03-26 08:27:46,520 epoch 55 - iter 22/25 - loss 2.58942200 - samples/sec: 143.26 - lr: 0.007812 2021-03-26 08:27:47,575 epoch 55 - iter 24/25 - loss 2.52959455 - samples/sec: 121.56 - lr: 0.007812 2021-03-26 08:27:48,022 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:27:48,023 EPOCH 55 done: loss 2.5290 - lr 0.0078125 2021-03-26 08:27:48,748 DEV : loss 5.161101341247559 - score 0.9162 2021-03-26 08:27:48,771 BAD EPOCHS (no improvement): 1 2021-03-26 08:27:48,772 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:27:49,929 epoch 56 - iter 2/25 - loss 2.89138258 - samples/sec: 110.79 - lr: 0.007812 2021-03-26 08:27:50,953 epoch 56 - iter 4/25 - loss 2.80591446 - samples/sec: 125.10 - lr: 0.007812 2021-03-26 08:27:52,010 epoch 56 - iter 6/25 - loss 2.80518850 - samples/sec: 121.30 - lr: 0.007812 2021-03-26 08:27:53,149 epoch 56 - iter 8/25 - loss 2.69240418 - samples/sec: 112.59 - lr: 0.007812 2021-03-26 08:27:54,275 epoch 56 - iter 10/25 - loss 2.70520034 - samples/sec: 113.91 - lr: 0.007812 2021-03-26 08:27:55,224 epoch 56 - iter 12/25 - loss 2.69055593 - samples/sec: 135.07 - lr: 0.007812 2021-03-26 08:27:56,175 epoch 56 - iter 14/25 - loss 2.69155152 - samples/sec: 134.80 - lr: 0.007812 2021-03-26 08:27:57,197 epoch 56 - iter 16/25 - loss 2.69912042 - samples/sec: 125.48 - lr: 0.007812 2021-03-26 08:27:58,337 epoch 56 - iter 18/25 - loss 2.68279980 - samples/sec: 112.50 - lr: 0.007812 2021-03-26 08:27:59,445 epoch 56 - iter 20/25 - loss 2.70082166 - samples/sec: 115.69 - lr: 0.007812 2021-03-26 08:28:00,441 epoch 56 - iter 22/25 - loss 2.71463389 - samples/sec: 128.77 - lr: 0.007812 2021-03-26 08:28:01,455 epoch 56 - iter 24/25 - loss 2.68750911 - samples/sec: 126.45 - lr: 0.007812 2021-03-26 08:28:01,809 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:28:01,811 EPOCH 56 done: loss 2.6804 - lr 0.0078125 2021-03-26 08:28:02,535 DEV : loss 5.161330223083496 - score 0.916 2021-03-26 08:28:02,558 BAD EPOCHS (no improvement): 2 2021-03-26 08:28:02,559 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:28:03,556 epoch 57 - iter 2/25 - loss 1.81027997 - samples/sec: 128.61 - lr: 0.007812 2021-03-26 08:28:04,586 epoch 57 - iter 4/25 - loss 1.96821702 - samples/sec: 124.52 - lr: 0.007812 2021-03-26 08:28:05,528 epoch 57 - iter 6/25 - loss 2.20572285 - samples/sec: 136.05 - lr: 0.007812 2021-03-26 08:28:06,513 epoch 57 - iter 8/25 - loss 2.21117917 - samples/sec: 130.15 - lr: 0.007812 2021-03-26 08:28:07,565 epoch 57 - iter 10/25 - loss 2.29010668 - samples/sec: 121.87 - lr: 0.007812 2021-03-26 08:28:08,619 epoch 57 - iter 12/25 - loss 2.39001824 - samples/sec: 121.53 - lr: 0.007812 2021-03-26 08:28:09,591 epoch 57 - iter 14/25 - loss 2.41587717 - samples/sec: 131.94 - lr: 0.007812 2021-03-26 08:28:10,608 epoch 57 - iter 16/25 - loss 2.40491781 - samples/sec: 126.06 - lr: 0.007812 2021-03-26 08:28:11,610 epoch 57 - iter 18/25 - loss 2.45139188 - samples/sec: 127.92 - lr: 0.007812 2021-03-26 08:28:12,515 epoch 57 - iter 20/25 - loss 2.46481062 - samples/sec: 141.71 - lr: 0.007812 2021-03-26 08:28:13,446 epoch 57 - iter 22/25 - loss 2.46444411 - samples/sec: 137.76 - lr: 0.007812 2021-03-26 08:28:14,378 epoch 57 - iter 24/25 - loss 2.46720476 - samples/sec: 137.55 - lr: 0.007812 2021-03-26 08:28:14,801 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:28:14,802 EPOCH 57 done: loss 2.4673 - lr 0.0078125 2021-03-26 08:28:15,513 DEV : loss 5.167983055114746 - score 0.9156 2021-03-26 08:28:15,536 BAD EPOCHS (no improvement): 3 2021-03-26 08:28:15,537 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:28:16,584 epoch 58 - iter 2/25 - loss 2.96477020 - samples/sec: 122.47 - lr: 0.007812 2021-03-26 08:28:17,500 epoch 58 - iter 4/25 - loss 2.81111413 - samples/sec: 140.09 - lr: 0.007812 2021-03-26 08:28:18,581 epoch 58 - iter 6/25 - loss 2.69619540 - samples/sec: 118.53 - lr: 0.007812 2021-03-26 08:28:19,589 epoch 58 - iter 8/25 - loss 2.66887558 - samples/sec: 127.16 - lr: 0.007812 2021-03-26 08:28:20,526 epoch 58 - iter 10/25 - loss 2.64879587 - samples/sec: 136.90 - lr: 0.007812 2021-03-26 08:28:21,415 epoch 58 - iter 12/25 - loss 2.59787667 - samples/sec: 144.25 - lr: 0.007812 2021-03-26 08:28:22,428 epoch 58 - iter 14/25 - loss 2.65796592 - samples/sec: 126.51 - lr: 0.007812 2021-03-26 08:28:23,432 epoch 58 - iter 16/25 - loss 2.66392018 - samples/sec: 127.69 - lr: 0.007812 2021-03-26 08:28:24,366 epoch 58 - iter 18/25 - loss 2.61717865 - samples/sec: 137.23 - lr: 0.007812 2021-03-26 08:28:25,447 epoch 58 - iter 20/25 - loss 2.58745240 - samples/sec: 118.49 - lr: 0.007812 2021-03-26 08:28:26,437 epoch 58 - iter 22/25 - loss 2.57686839 - samples/sec: 129.51 - lr: 0.007812 2021-03-26 08:28:27,385 epoch 58 - iter 24/25 - loss 2.61233709 - samples/sec: 135.25 - lr: 0.007812 2021-03-26 08:28:27,794 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:28:27,795 EPOCH 58 done: loss 2.5902 - lr 0.0078125 2021-03-26 08:28:28,528 DEV : loss 5.153556823730469 - score 0.9166 2021-03-26 08:28:28,551 BAD EPOCHS (no improvement): 4 2021-03-26 08:28:28,552 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:28:29,576 epoch 59 - iter 2/25 - loss 2.64156973 - samples/sec: 125.17 - lr: 0.003906 2021-03-26 08:28:30,523 epoch 59 - iter 4/25 - loss 2.81922227 - samples/sec: 135.44 - lr: 0.003906 2021-03-26 08:28:31,506 epoch 59 - iter 6/25 - loss 2.72071330 - samples/sec: 130.46 - lr: 0.003906 2021-03-26 08:28:32,542 epoch 59 - iter 8/25 - loss 2.69526169 - samples/sec: 123.69 - lr: 0.003906 2021-03-26 08:28:33,514 epoch 59 - iter 10/25 - loss 2.74493732 - samples/sec: 131.84 - lr: 0.003906 2021-03-26 08:28:34,443 epoch 59 - iter 12/25 - loss 2.66976053 - samples/sec: 138.06 - lr: 0.003906 2021-03-26 08:28:35,604 epoch 59 - iter 14/25 - loss 2.61479444 - samples/sec: 110.40 - lr: 0.003906 2021-03-26 08:28:36,573 epoch 59 - iter 16/25 - loss 2.61310622 - samples/sec: 132.22 - lr: 0.003906 2021-03-26 08:28:37,494 epoch 59 - iter 18/25 - loss 2.58530426 - samples/sec: 139.19 - lr: 0.003906 2021-03-26 08:28:38,582 epoch 59 - iter 20/25 - loss 2.56393338 - samples/sec: 117.78 - lr: 0.003906 2021-03-26 08:28:39,580 epoch 59 - iter 22/25 - loss 2.59042783 - samples/sec: 128.40 - lr: 0.003906 2021-03-26 08:28:40,582 epoch 59 - iter 24/25 - loss 2.55683567 - samples/sec: 127.95 - lr: 0.003906 2021-03-26 08:28:41,005 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:28:41,006 EPOCH 59 done: loss 2.5670 - lr 0.0039062 2021-03-26 08:28:41,716 DEV : loss 5.155137538909912 - score 0.9166 2021-03-26 08:28:41,738 BAD EPOCHS (no improvement): 1 2021-03-26 08:28:41,739 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:28:42,744 epoch 60 - iter 2/25 - loss 3.35700023 - samples/sec: 127.64 - lr: 0.003906 2021-03-26 08:28:43,651 epoch 60 - iter 4/25 - loss 2.85628122 - samples/sec: 141.54 - lr: 0.003906 2021-03-26 08:28:44,655 epoch 60 - iter 6/25 - loss 2.71600425 - samples/sec: 127.60 - lr: 0.003906 2021-03-26 08:28:45,584 epoch 60 - iter 8/25 - loss 2.66859806 - samples/sec: 138.13 - lr: 0.003906 2021-03-26 08:28:46,620 epoch 60 - iter 10/25 - loss 2.69276667 - samples/sec: 123.65 - lr: 0.003906 2021-03-26 08:28:47,548 epoch 60 - iter 12/25 - loss 2.62539530 - samples/sec: 138.32 - lr: 0.003906 2021-03-26 08:28:48,426 epoch 60 - iter 14/25 - loss 2.65403557 - samples/sec: 146.05 - lr: 0.003906 2021-03-26 08:28:49,579 epoch 60 - iter 16/25 - loss 2.66666107 - samples/sec: 111.10 - lr: 0.003906 2021-03-26 08:28:50,516 epoch 60 - iter 18/25 - loss 2.64792878 - samples/sec: 136.92 - lr: 0.003906 2021-03-26 08:28:51,531 epoch 60 - iter 20/25 - loss 2.60760083 - samples/sec: 126.23 - lr: 0.003906 2021-03-26 08:28:52,518 epoch 60 - iter 22/25 - loss 2.58448346 - samples/sec: 129.86 - lr: 0.003906 2021-03-26 08:28:53,512 epoch 60 - iter 24/25 - loss 2.58142538 - samples/sec: 129.00 - lr: 0.003906 2021-03-26 08:28:53,869 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:28:53,869 EPOCH 60 done: loss 2.5570 - lr 0.0039062 2021-03-26 08:28:54,605 DEV : loss 5.156240463256836 - score 0.9166 2021-03-26 08:28:54,628 BAD EPOCHS (no improvement): 2 2021-03-26 08:28:54,629 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:28:55,572 epoch 61 - iter 2/25 - loss 2.64861023 - samples/sec: 135.91 - lr: 0.003906 2021-03-26 08:28:56,509 epoch 61 - iter 4/25 - loss 2.33936024 - samples/sec: 136.95 - lr: 0.003906 2021-03-26 08:28:57,509 epoch 61 - iter 6/25 - loss 2.32444807 - samples/sec: 128.15 - lr: 0.003906 2021-03-26 08:28:58,539 epoch 61 - iter 8/25 - loss 2.34189090 - samples/sec: 124.46 - lr: 0.003906 2021-03-26 08:28:59,639 epoch 61 - iter 10/25 - loss 2.37957444 - samples/sec: 116.45 - lr: 0.003906 2021-03-26 08:29:00,688 epoch 61 - iter 12/25 - loss 2.39998172 - samples/sec: 122.20 - lr: 0.003906 2021-03-26 08:29:01,645 epoch 61 - iter 14/25 - loss 2.42385345 - samples/sec: 133.87 - lr: 0.003906 2021-03-26 08:29:02,655 epoch 61 - iter 16/25 - loss 2.39101814 - samples/sec: 126.88 - lr: 0.003906 2021-03-26 08:29:03,618 epoch 61 - iter 18/25 - loss 2.43700922 - samples/sec: 133.17 - lr: 0.003906 2021-03-26 08:29:04,541 epoch 61 - iter 20/25 - loss 2.40053543 - samples/sec: 138.82 - lr: 0.003906 2021-03-26 08:29:05,590 epoch 61 - iter 22/25 - loss 2.42045249 - samples/sec: 122.23 - lr: 0.003906 2021-03-26 08:29:06,643 epoch 61 - iter 24/25 - loss 2.46234730 - samples/sec: 121.64 - lr: 0.003906 2021-03-26 08:29:07,077 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:29:07,078 EPOCH 61 done: loss 2.5105 - lr 0.0039062 2021-03-26 08:29:07,818 DEV : loss 5.1566362380981445 - score 0.9164 2021-03-26 08:29:07,842 BAD EPOCHS (no improvement): 3 2021-03-26 08:29:07,843 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:29:08,811 epoch 62 - iter 2/25 - loss 2.52020955 - samples/sec: 132.49 - lr: 0.003906 2021-03-26 08:29:09,859 epoch 62 - iter 4/25 - loss 2.50968593 - samples/sec: 122.28 - lr: 0.003906 2021-03-26 08:29:10,840 epoch 62 - iter 6/25 - loss 2.37634569 - samples/sec: 130.69 - lr: 0.003906 2021-03-26 08:29:11,786 epoch 62 - iter 8/25 - loss 2.36597966 - samples/sec: 135.68 - lr: 0.003906 2021-03-26 08:29:12,737 epoch 62 - iter 10/25 - loss 2.49966995 - samples/sec: 134.79 - lr: 0.003906 2021-03-26 08:29:13,878 epoch 62 - iter 12/25 - loss 2.49869683 - samples/sec: 112.32 - lr: 0.003906 2021-03-26 08:29:14,888 epoch 62 - iter 14/25 - loss 2.52862756 - samples/sec: 126.88 - lr: 0.003906 2021-03-26 08:29:15,992 epoch 62 - iter 16/25 - loss 2.55252969 - samples/sec: 116.03 - lr: 0.003906 2021-03-26 08:29:17,034 epoch 62 - iter 18/25 - loss 2.60047205 - samples/sec: 123.04 - lr: 0.003906 2021-03-26 08:29:17,979 epoch 62 - iter 20/25 - loss 2.59337807 - samples/sec: 135.56 - lr: 0.003906 2021-03-26 08:29:18,979 epoch 62 - iter 22/25 - loss 2.61051676 - samples/sec: 128.26 - lr: 0.003906 2021-03-26 08:29:19,983 epoch 62 - iter 24/25 - loss 2.62917062 - samples/sec: 127.69 - lr: 0.003906 2021-03-26 08:29:20,411 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:29:20,411 EPOCH 62 done: loss 2.6084 - lr 0.0039062 2021-03-26 08:29:21,224 DEV : loss 5.154888153076172 - score 0.9173 2021-03-26 08:29:21,255 BAD EPOCHS (no improvement): 4 2021-03-26 08:29:21,256 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:29:22,282 epoch 63 - iter 2/25 - loss 2.45950496 - samples/sec: 124.96 - lr: 0.001953 2021-03-26 08:29:23,280 epoch 63 - iter 4/25 - loss 2.58118606 - samples/sec: 128.45 - lr: 0.001953 2021-03-26 08:29:24,371 epoch 63 - iter 6/25 - loss 2.55522390 - samples/sec: 117.52 - lr: 0.001953 2021-03-26 08:29:25,436 epoch 63 - iter 8/25 - loss 2.64915141 - samples/sec: 120.42 - lr: 0.001953 2021-03-26 08:29:26,379 epoch 63 - iter 10/25 - loss 2.60279477 - samples/sec: 135.94 - lr: 0.001953 2021-03-26 08:29:27,384 epoch 63 - iter 12/25 - loss 2.62300179 - samples/sec: 127.52 - lr: 0.001953 2021-03-26 08:29:28,390 epoch 63 - iter 14/25 - loss 2.74238583 - samples/sec: 127.43 - lr: 0.001953 2021-03-26 08:29:29,472 epoch 63 - iter 16/25 - loss 2.72329572 - samples/sec: 118.47 - lr: 0.001953 2021-03-26 08:29:30,499 epoch 63 - iter 18/25 - loss 2.68225120 - samples/sec: 124.89 - lr: 0.001953 2021-03-26 08:29:31,480 epoch 63 - iter 20/25 - loss 2.65703490 - samples/sec: 130.69 - lr: 0.001953 2021-03-26 08:29:32,507 epoch 63 - iter 22/25 - loss 2.67516613 - samples/sec: 124.84 - lr: 0.001953 2021-03-26 08:29:33,483 epoch 63 - iter 24/25 - loss 2.68743004 - samples/sec: 131.40 - lr: 0.001953 2021-03-26 08:29:33,891 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:29:33,892 EPOCH 63 done: loss 2.6802 - lr 0.0019531 2021-03-26 08:29:34,623 DEV : loss 5.15378475189209 - score 0.9175 2021-03-26 08:29:34,655 BAD EPOCHS (no improvement): 1 2021-03-26 08:29:34,656 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:29:35,734 epoch 64 - iter 2/25 - loss 3.01414752 - samples/sec: 119.03 - lr: 0.001953 2021-03-26 08:29:36,775 epoch 64 - iter 4/25 - loss 2.53828612 - samples/sec: 123.06 - lr: 0.001953 2021-03-26 08:29:37,735 epoch 64 - iter 6/25 - loss 2.57090575 - samples/sec: 133.51 - lr: 0.001953 2021-03-26 08:29:38,693 epoch 64 - iter 8/25 - loss 2.60916208 - samples/sec: 133.92 - lr: 0.001953 2021-03-26 08:29:39,637 epoch 64 - iter 10/25 - loss 2.62027467 - samples/sec: 135.80 - lr: 0.001953 2021-03-26 08:29:40,653 epoch 64 - iter 12/25 - loss 2.55137331 - samples/sec: 126.13 - lr: 0.001953 2021-03-26 08:29:41,744 epoch 64 - iter 14/25 - loss 2.54606689 - samples/sec: 117.47 - lr: 0.001953 2021-03-26 08:29:42,749 epoch 64 - iter 16/25 - loss 2.63612720 - samples/sec: 127.60 - lr: 0.001953 2021-03-26 08:29:43,739 epoch 64 - iter 18/25 - loss 2.65265855 - samples/sec: 129.55 - lr: 0.001953 2021-03-26 08:29:44,720 epoch 64 - iter 20/25 - loss 2.65310151 - samples/sec: 130.76 - lr: 0.001953 2021-03-26 08:29:45,689 epoch 64 - iter 22/25 - loss 2.61852049 - samples/sec: 132.17 - lr: 0.001953 2021-03-26 08:29:46,721 epoch 64 - iter 24/25 - loss 2.56815796 - samples/sec: 124.21 - lr: 0.001953 2021-03-26 08:29:47,112 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:29:47,113 EPOCH 64 done: loss 2.6146 - lr 0.0019531 2021-03-26 08:29:47,886 DEV : loss 5.153861999511719 - score 0.9171 2021-03-26 08:29:47,901 BAD EPOCHS (no improvement): 2 2021-03-26 08:29:47,901 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:29:48,902 epoch 65 - iter 2/25 - loss 2.75063539 - samples/sec: 128.18 - lr: 0.001953 2021-03-26 08:29:49,966 epoch 65 - iter 4/25 - loss 2.82495999 - samples/sec: 120.45 - lr: 0.001953 2021-03-26 08:29:51,020 epoch 65 - iter 6/25 - loss 2.78207270 - samples/sec: 121.65 - lr: 0.001953 2021-03-26 08:29:52,051 epoch 65 - iter 8/25 - loss 2.78051665 - samples/sec: 124.34 - lr: 0.001953 2021-03-26 08:29:53,054 epoch 65 - iter 10/25 - loss 2.76111586 - samples/sec: 127.88 - lr: 0.001953 2021-03-26 08:29:54,088 epoch 65 - iter 12/25 - loss 2.72665600 - samples/sec: 123.91 - lr: 0.001953 2021-03-26 08:29:55,081 epoch 65 - iter 14/25 - loss 2.69365876 - samples/sec: 129.06 - lr: 0.001953 2021-03-26 08:29:56,018 epoch 65 - iter 16/25 - loss 2.62989879 - samples/sec: 136.80 - lr: 0.001953 2021-03-26 08:29:57,029 epoch 65 - iter 18/25 - loss 2.59235953 - samples/sec: 126.74 - lr: 0.001953 2021-03-26 08:29:57,989 epoch 65 - iter 20/25 - loss 2.57889878 - samples/sec: 133.57 - lr: 0.001953 2021-03-26 08:29:58,875 epoch 65 - iter 22/25 - loss 2.57340151 - samples/sec: 144.83 - lr: 0.001953 2021-03-26 08:29:59,849 epoch 65 - iter 24/25 - loss 2.59013875 - samples/sec: 131.54 - lr: 0.001953 2021-03-26 08:30:00,230 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:30:00,231 EPOCH 65 done: loss 2.5700 - lr 0.0019531 2021-03-26 08:30:00,995 DEV : loss 5.1544013023376465 - score 0.9175 2021-03-26 08:30:01,026 BAD EPOCHS (no improvement): 3 2021-03-26 08:30:01,027 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:30:01,991 epoch 66 - iter 2/25 - loss 2.42480397 - samples/sec: 133.06 - lr: 0.001953 2021-03-26 08:30:03,005 epoch 66 - iter 4/25 - loss 2.47575986 - samples/sec: 126.34 - lr: 0.001953 2021-03-26 08:30:04,018 epoch 66 - iter 6/25 - loss 2.42025113 - samples/sec: 126.51 - lr: 0.001953 2021-03-26 08:30:05,271 epoch 66 - iter 8/25 - loss 2.42866611 - samples/sec: 102.29 - lr: 0.001953 2021-03-26 08:30:06,456 epoch 66 - iter 10/25 - loss 2.36550404 - samples/sec: 108.13 - lr: 0.001953 2021-03-26 08:30:07,496 epoch 66 - iter 12/25 - loss 2.32203812 - samples/sec: 123.30 - lr: 0.001953 2021-03-26 08:30:08,682 epoch 66 - iter 14/25 - loss 2.29170151 - samples/sec: 108.09 - lr: 0.001953 2021-03-26 08:30:09,671 epoch 66 - iter 16/25 - loss 2.31758191 - samples/sec: 129.56 - lr: 0.001953 2021-03-26 08:30:10,646 epoch 66 - iter 18/25 - loss 2.36295978 - samples/sec: 131.56 - lr: 0.001953 2021-03-26 08:30:11,611 epoch 66 - iter 20/25 - loss 2.32609164 - samples/sec: 132.79 - lr: 0.001953 2021-03-26 08:30:12,534 epoch 66 - iter 22/25 - loss 2.36869703 - samples/sec: 138.88 - lr: 0.001953 2021-03-26 08:30:13,541 epoch 66 - iter 24/25 - loss 2.39990243 - samples/sec: 127.23 - lr: 0.001953 2021-03-26 08:30:13,984 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:30:13,986 EPOCH 66 done: loss 2.3938 - lr 0.0019531 2021-03-26 08:30:14,680 DEV : loss 5.154584884643555 - score 0.9179 2021-03-26 08:30:14,695 BAD EPOCHS (no improvement): 4 2021-03-26 08:30:14,696 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:30:15,634 epoch 67 - iter 2/25 - loss 2.01868904 - samples/sec: 136.63 - lr: 0.000977 2021-03-26 08:30:16,662 epoch 67 - iter 4/25 - loss 2.34448314 - samples/sec: 124.80 - lr: 0.000977 2021-03-26 08:30:17,615 epoch 67 - iter 6/25 - loss 2.43418237 - samples/sec: 134.41 - lr: 0.000977 2021-03-26 08:30:18,661 epoch 67 - iter 8/25 - loss 2.47001508 - samples/sec: 122.58 - lr: 0.000977 2021-03-26 08:30:19,690 epoch 67 - iter 10/25 - loss 2.52531345 - samples/sec: 124.62 - lr: 0.000977 2021-03-26 08:30:20,662 epoch 67 - iter 12/25 - loss 2.61063679 - samples/sec: 131.93 - lr: 0.000977 2021-03-26 08:30:21,524 epoch 67 - iter 14/25 - loss 2.57734524 - samples/sec: 148.73 - lr: 0.000977 2021-03-26 08:30:22,493 epoch 67 - iter 16/25 - loss 2.52899256 - samples/sec: 132.34 - lr: 0.000977 2021-03-26 08:30:23,473 epoch 67 - iter 18/25 - loss 2.50297968 - samples/sec: 130.84 - lr: 0.000977 2021-03-26 08:30:24,447 epoch 67 - iter 20/25 - loss 2.49524022 - samples/sec: 131.62 - lr: 0.000977 2021-03-26 08:30:25,466 epoch 67 - iter 22/25 - loss 2.46065530 - samples/sec: 125.72 - lr: 0.000977 2021-03-26 08:30:26,358 epoch 67 - iter 24/25 - loss 2.47536785 - samples/sec: 143.77 - lr: 0.000977 2021-03-26 08:30:26,731 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:30:26,732 EPOCH 67 done: loss 2.4502 - lr 0.0009766 2021-03-26 08:30:27,441 DEV : loss 5.155641555786133 - score 0.9177 2021-03-26 08:30:27,463 BAD EPOCHS (no improvement): 1 2021-03-26 08:30:27,464 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:30:28,450 epoch 68 - iter 2/25 - loss 2.34856331 - samples/sec: 130.14 - lr: 0.000977 2021-03-26 08:30:29,385 epoch 68 - iter 4/25 - loss 2.51698828 - samples/sec: 137.22 - lr: 0.000977 2021-03-26 08:30:30,368 epoch 68 - iter 6/25 - loss 2.46542730 - samples/sec: 130.36 - lr: 0.000977 2021-03-26 08:30:31,309 epoch 68 - iter 8/25 - loss 2.51394932 - samples/sec: 136.28 - lr: 0.000977 2021-03-26 08:30:32,230 epoch 68 - iter 10/25 - loss 2.52544693 - samples/sec: 139.25 - lr: 0.000977 2021-03-26 08:30:33,171 epoch 68 - iter 12/25 - loss 2.41423985 - samples/sec: 136.25 - lr: 0.000977 2021-03-26 08:30:34,115 epoch 68 - iter 14/25 - loss 2.55178968 - samples/sec: 135.74 - lr: 0.000977 2021-03-26 08:30:35,129 epoch 68 - iter 16/25 - loss 2.56635130 - samples/sec: 126.42 - lr: 0.000977 2021-03-26 08:30:36,220 epoch 68 - iter 18/25 - loss 2.52978544 - samples/sec: 117.46 - lr: 0.000977 2021-03-26 08:30:37,361 epoch 68 - iter 20/25 - loss 2.50168654 - samples/sec: 112.36 - lr: 0.000977 2021-03-26 08:30:38,438 epoch 68 - iter 22/25 - loss 2.47874781 - samples/sec: 119.03 - lr: 0.000977 2021-03-26 08:30:39,456 epoch 68 - iter 24/25 - loss 2.49064927 - samples/sec: 125.92 - lr: 0.000977 2021-03-26 08:30:39,857 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:30:39,858 EPOCH 68 done: loss 2.4624 - lr 0.0009766 2021-03-26 08:30:40,580 DEV : loss 5.155262470245361 - score 0.9179 2021-03-26 08:30:40,595 BAD EPOCHS (no improvement): 2 2021-03-26 08:30:40,596 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:30:41,629 epoch 69 - iter 2/25 - loss 3.18188810 - samples/sec: 124.03 - lr: 0.000977 2021-03-26 08:30:42,601 epoch 69 - iter 4/25 - loss 2.74411315 - samples/sec: 131.96 - lr: 0.000977 2021-03-26 08:30:43,523 epoch 69 - iter 6/25 - loss 2.70093520 - samples/sec: 139.06 - lr: 0.000977 2021-03-26 08:30:44,440 epoch 69 - iter 8/25 - loss 2.50844127 - samples/sec: 139.71 - lr: 0.000977 2021-03-26 08:30:45,597 epoch 69 - iter 10/25 - loss 2.57270017 - samples/sec: 110.85 - lr: 0.000977 2021-03-26 08:30:46,508 epoch 69 - iter 12/25 - loss 2.53602714 - samples/sec: 140.61 - lr: 0.000977 2021-03-26 08:30:47,470 epoch 69 - iter 14/25 - loss 2.56787088 - samples/sec: 133.26 - lr: 0.000977 2021-03-26 08:30:48,447 epoch 69 - iter 16/25 - loss 2.55436224 - samples/sec: 131.27 - lr: 0.000977 2021-03-26 08:30:49,456 epoch 69 - iter 18/25 - loss 2.53948792 - samples/sec: 126.96 - lr: 0.000977 2021-03-26 08:30:50,488 epoch 69 - iter 20/25 - loss 2.50969934 - samples/sec: 124.29 - lr: 0.000977 2021-03-26 08:30:51,515 epoch 69 - iter 22/25 - loss 2.54528459 - samples/sec: 124.70 - lr: 0.000977 2021-03-26 08:30:52,466 epoch 69 - iter 24/25 - loss 2.50940784 - samples/sec: 134.92 - lr: 0.000977 2021-03-26 08:30:52,890 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:30:52,890 EPOCH 69 done: loss 2.4829 - lr 0.0009766 2021-03-26 08:30:53,588 DEV : loss 5.154378890991211 - score 0.9179 2021-03-26 08:30:53,604 BAD EPOCHS (no improvement): 3 2021-03-26 08:30:53,605 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:30:54,681 epoch 70 - iter 2/25 - loss 2.56138349 - samples/sec: 119.15 - lr: 0.000977 2021-03-26 08:30:55,692 epoch 70 - iter 4/25 - loss 2.57898003 - samples/sec: 126.92 - lr: 0.000977 2021-03-26 08:30:56,590 epoch 70 - iter 6/25 - loss 2.47086718 - samples/sec: 142.71 - lr: 0.000977 2021-03-26 08:30:57,656 epoch 70 - iter 8/25 - loss 2.35704540 - samples/sec: 120.34 - lr: 0.000977 2021-03-26 08:30:58,601 epoch 70 - iter 10/25 - loss 2.33217787 - samples/sec: 135.71 - lr: 0.000977 2021-03-26 08:30:59,560 epoch 70 - iter 12/25 - loss 2.34995758 - samples/sec: 133.71 - lr: 0.000977 2021-03-26 08:31:00,545 epoch 70 - iter 14/25 - loss 2.37106590 - samples/sec: 130.08 - lr: 0.000977 2021-03-26 08:31:01,544 epoch 70 - iter 16/25 - loss 2.41983237 - samples/sec: 128.41 - lr: 0.000977 2021-03-26 08:31:02,613 epoch 70 - iter 18/25 - loss 2.44145462 - samples/sec: 119.88 - lr: 0.000977 2021-03-26 08:31:03,609 epoch 70 - iter 20/25 - loss 2.45750895 - samples/sec: 128.85 - lr: 0.000977 2021-03-26 08:31:04,670 epoch 70 - iter 22/25 - loss 2.48850318 - samples/sec: 120.84 - lr: 0.000977 2021-03-26 08:31:05,761 epoch 70 - iter 24/25 - loss 2.48575143 - samples/sec: 117.54 - lr: 0.000977 2021-03-26 08:31:06,188 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:31:06,188 EPOCH 70 done: loss 2.5322 - lr 0.0009766 2021-03-26 08:31:06,896 DEV : loss 5.15534782409668 - score 0.9177 2021-03-26 08:31:06,918 BAD EPOCHS (no improvement): 4 2021-03-26 08:31:06,918 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:31:07,908 epoch 71 - iter 2/25 - loss 3.07665741 - samples/sec: 129.56 - lr: 0.000488 2021-03-26 08:31:08,921 epoch 71 - iter 4/25 - loss 2.93741924 - samples/sec: 126.56 - lr: 0.000488 2021-03-26 08:31:09,860 epoch 71 - iter 6/25 - loss 2.79518795 - samples/sec: 136.51 - lr: 0.000488 2021-03-26 08:31:10,819 epoch 71 - iter 8/25 - loss 2.67835298 - samples/sec: 133.65 - lr: 0.000488 2021-03-26 08:31:11,826 epoch 71 - iter 10/25 - loss 2.65618186 - samples/sec: 127.29 - lr: 0.000488 2021-03-26 08:31:12,830 epoch 71 - iter 12/25 - loss 2.59280755 - samples/sec: 127.68 - lr: 0.000488 2021-03-26 08:31:13,994 epoch 71 - iter 14/25 - loss 2.60106427 - samples/sec: 110.11 - lr: 0.000488 2021-03-26 08:31:14,915 epoch 71 - iter 16/25 - loss 2.61127068 - samples/sec: 139.19 - lr: 0.000488 2021-03-26 08:31:15,794 epoch 71 - iter 18/25 - loss 2.57161060 - samples/sec: 145.86 - lr: 0.000488 2021-03-26 08:31:16,874 epoch 71 - iter 20/25 - loss 2.60581434 - samples/sec: 118.63 - lr: 0.000488 2021-03-26 08:31:17,850 epoch 71 - iter 22/25 - loss 2.62448980 - samples/sec: 131.39 - lr: 0.000488 2021-03-26 08:31:19,151 epoch 71 - iter 24/25 - loss 2.64342704 - samples/sec: 98.51 - lr: 0.000488 2021-03-26 08:31:19,549 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:31:19,549 EPOCH 71 done: loss 2.6406 - lr 0.0004883 2021-03-26 08:31:20,282 DEV : loss 5.154711723327637 - score 0.9173 2021-03-26 08:31:20,297 BAD EPOCHS (no improvement): 1 2021-03-26 08:31:20,298 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:31:21,324 epoch 72 - iter 2/25 - loss 1.73620844 - samples/sec: 124.91 - lr: 0.000488 2021-03-26 08:31:22,405 epoch 72 - iter 4/25 - loss 2.02630025 - samples/sec: 118.68 - lr: 0.000488 2021-03-26 08:31:23,505 epoch 72 - iter 6/25 - loss 2.08867693 - samples/sec: 116.50 - lr: 0.000488 2021-03-26 08:31:24,492 epoch 72 - iter 8/25 - loss 2.03881234 - samples/sec: 129.96 - lr: 0.000488 2021-03-26 08:31:25,515 epoch 72 - iter 10/25 - loss 2.06679652 - samples/sec: 125.26 - lr: 0.000488 2021-03-26 08:31:26,514 epoch 72 - iter 12/25 - loss 2.07280229 - samples/sec: 128.40 - lr: 0.000488 2021-03-26 08:31:27,522 epoch 72 - iter 14/25 - loss 2.14705983 - samples/sec: 127.19 - lr: 0.000488 2021-03-26 08:31:28,485 epoch 72 - iter 16/25 - loss 2.14889824 - samples/sec: 133.16 - lr: 0.000488 2021-03-26 08:31:29,417 epoch 72 - iter 18/25 - loss 2.22106131 - samples/sec: 137.54 - lr: 0.000488 2021-03-26 08:31:30,459 epoch 72 - iter 20/25 - loss 2.25088767 - samples/sec: 123.02 - lr: 0.000488 2021-03-26 08:31:31,467 epoch 72 - iter 22/25 - loss 2.30408206 - samples/sec: 127.17 - lr: 0.000488 2021-03-26 08:31:32,410 epoch 72 - iter 24/25 - loss 2.30297814 - samples/sec: 136.00 - lr: 0.000488 2021-03-26 08:31:32,946 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:31:32,947 EPOCH 72 done: loss 2.3326 - lr 0.0004883 2021-03-26 08:31:33,666 DEV : loss 5.154988765716553 - score 0.9175 2021-03-26 08:31:33,688 BAD EPOCHS (no improvement): 2 2021-03-26 08:31:33,689 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:31:34,657 epoch 73 - iter 2/25 - loss 2.83746290 - samples/sec: 132.34 - lr: 0.000488 2021-03-26 08:31:35,618 epoch 73 - iter 4/25 - loss 2.60548675 - samples/sec: 133.51 - lr: 0.000488 2021-03-26 08:31:36,587 epoch 73 - iter 6/25 - loss 2.58915744 - samples/sec: 132.35 - lr: 0.000488 2021-03-26 08:31:37,719 epoch 73 - iter 8/25 - loss 2.56479962 - samples/sec: 113.21 - lr: 0.000488 2021-03-26 08:31:38,667 epoch 73 - iter 10/25 - loss 2.53289689 - samples/sec: 135.30 - lr: 0.000488 2021-03-26 08:31:39,705 epoch 73 - iter 12/25 - loss 2.49248828 - samples/sec: 123.57 - lr: 0.000488 2021-03-26 08:31:40,718 epoch 73 - iter 14/25 - loss 2.49144954 - samples/sec: 126.52 - lr: 0.000488 2021-03-26 08:31:41,708 epoch 73 - iter 16/25 - loss 2.48652732 - samples/sec: 129.41 - lr: 0.000488 2021-03-26 08:31:42,780 epoch 73 - iter 18/25 - loss 2.51980366 - samples/sec: 119.58 - lr: 0.000488 2021-03-26 08:31:43,711 epoch 73 - iter 20/25 - loss 2.47590793 - samples/sec: 137.67 - lr: 0.000488 2021-03-26 08:31:44,699 epoch 73 - iter 22/25 - loss 2.51211202 - samples/sec: 129.74 - lr: 0.000488 2021-03-26 08:31:45,691 epoch 73 - iter 24/25 - loss 2.48965444 - samples/sec: 129.27 - lr: 0.000488 2021-03-26 08:31:46,088 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:31:46,089 EPOCH 73 done: loss 2.4835 - lr 0.0004883 2021-03-26 08:31:46,805 DEV : loss 5.154832363128662 - score 0.9173 2021-03-26 08:31:46,832 BAD EPOCHS (no improvement): 3 2021-03-26 08:31:46,832 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:31:47,814 epoch 74 - iter 2/25 - loss 2.08305854 - samples/sec: 130.62 - lr: 0.000488 2021-03-26 08:31:48,847 epoch 74 - iter 4/25 - loss 2.23228666 - samples/sec: 124.13 - lr: 0.000488 2021-03-26 08:31:49,796 epoch 74 - iter 6/25 - loss 2.21459860 - samples/sec: 135.18 - lr: 0.000488 2021-03-26 08:31:50,763 epoch 74 - iter 8/25 - loss 2.23361726 - samples/sec: 132.49 - lr: 0.000488 2021-03-26 08:31:51,801 epoch 74 - iter 10/25 - loss 2.32135657 - samples/sec: 123.54 - lr: 0.000488 2021-03-26 08:31:52,773 epoch 74 - iter 12/25 - loss 2.39057528 - samples/sec: 131.91 - lr: 0.000488 2021-03-26 08:31:53,747 epoch 74 - iter 14/25 - loss 2.43192735 - samples/sec: 131.63 - lr: 0.000488 2021-03-26 08:31:54,668 epoch 74 - iter 16/25 - loss 2.44178198 - samples/sec: 139.10 - lr: 0.000488 2021-03-26 08:31:55,677 epoch 74 - iter 18/25 - loss 2.45810711 - samples/sec: 127.22 - lr: 0.000488 2021-03-26 08:31:56,646 epoch 74 - iter 20/25 - loss 2.42499514 - samples/sec: 132.22 - lr: 0.000488 2021-03-26 08:31:57,572 epoch 74 - iter 22/25 - loss 2.40459371 - samples/sec: 138.51 - lr: 0.000488 2021-03-26 08:31:58,611 epoch 74 - iter 24/25 - loss 2.45315273 - samples/sec: 123.38 - lr: 0.000488 2021-03-26 08:31:59,096 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:31:59,096 EPOCH 74 done: loss 2.4754 - lr 0.0004883 2021-03-26 08:31:59,826 DEV : loss 5.15543270111084 - score 0.9173 2021-03-26 08:31:59,849 BAD EPOCHS (no improvement): 4 2021-03-26 08:31:59,850 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:32:00,856 epoch 75 - iter 2/25 - loss 2.59288913 - samples/sec: 127.57 - lr: 0.000244 2021-03-26 08:32:01,774 epoch 75 - iter 4/25 - loss 2.49805722 - samples/sec: 139.62 - lr: 0.000244 2021-03-26 08:32:02,764 epoch 75 - iter 6/25 - loss 2.51666794 - samples/sec: 129.57 - lr: 0.000244 2021-03-26 08:32:03,911 epoch 75 - iter 8/25 - loss 2.53042136 - samples/sec: 111.73 - lr: 0.000244 2021-03-26 08:32:04,831 epoch 75 - iter 10/25 - loss 2.46552075 - samples/sec: 139.32 - lr: 0.000244 2021-03-26 08:32:05,839 epoch 75 - iter 12/25 - loss 2.49688201 - samples/sec: 127.12 - lr: 0.000244 2021-03-26 08:32:06,839 epoch 75 - iter 14/25 - loss 2.53646792 - samples/sec: 128.18 - lr: 0.000244 2021-03-26 08:32:07,855 epoch 75 - iter 16/25 - loss 2.60008014 - samples/sec: 126.24 - lr: 0.000244 2021-03-26 08:32:08,720 epoch 75 - iter 18/25 - loss 2.56159735 - samples/sec: 148.19 - lr: 0.000244 2021-03-26 08:32:09,629 epoch 75 - iter 20/25 - loss 2.49216157 - samples/sec: 140.97 - lr: 0.000244 2021-03-26 08:32:10,686 epoch 75 - iter 22/25 - loss 2.45089780 - samples/sec: 121.23 - lr: 0.000244 2021-03-26 08:32:11,686 epoch 75 - iter 24/25 - loss 2.48957231 - samples/sec: 128.26 - lr: 0.000244 2021-03-26 08:32:12,103 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:32:12,104 EPOCH 75 done: loss 2.4927 - lr 0.0002441 2021-03-26 08:32:12,823 DEV : loss 5.1556715965271 - score 0.9173 2021-03-26 08:32:12,845 BAD EPOCHS (no improvement): 1 2021-03-26 08:32:12,846 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:32:13,823 epoch 76 - iter 2/25 - loss 2.92167127 - samples/sec: 131.27 - lr: 0.000244 2021-03-26 08:32:14,764 epoch 76 - iter 4/25 - loss 2.67908305 - samples/sec: 136.30 - lr: 0.000244 2021-03-26 08:32:15,679 epoch 76 - iter 6/25 - loss 2.58029282 - samples/sec: 140.02 - lr: 0.000244 2021-03-26 08:32:16,622 epoch 76 - iter 8/25 - loss 2.72686112 - samples/sec: 135.99 - lr: 0.000244 2021-03-26 08:32:17,684 epoch 76 - iter 10/25 - loss 2.58344152 - samples/sec: 120.68 - lr: 0.000244 2021-03-26 08:32:18,622 epoch 76 - iter 12/25 - loss 2.59247253 - samples/sec: 136.82 - lr: 0.000244 2021-03-26 08:32:19,542 epoch 76 - iter 14/25 - loss 2.57681600 - samples/sec: 139.36 - lr: 0.000244 2021-03-26 08:32:20,608 epoch 76 - iter 16/25 - loss 2.54443355 - samples/sec: 120.23 - lr: 0.000244 2021-03-26 08:32:21,703 epoch 76 - iter 18/25 - loss 2.52020005 - samples/sec: 116.98 - lr: 0.000244 2021-03-26 08:32:22,619 epoch 76 - iter 20/25 - loss 2.50298864 - samples/sec: 140.05 - lr: 0.000244 2021-03-26 08:32:23,586 epoch 76 - iter 22/25 - loss 2.49885844 - samples/sec: 132.50 - lr: 0.000244 2021-03-26 08:32:24,474 epoch 76 - iter 24/25 - loss 2.48496979 - samples/sec: 144.49 - lr: 0.000244 2021-03-26 08:32:24,923 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:32:24,924 EPOCH 76 done: loss 2.5162 - lr 0.0002441 2021-03-26 08:32:25,615 DEV : loss 5.155824661254883 - score 0.9177 2021-03-26 08:32:25,638 BAD EPOCHS (no improvement): 2 2021-03-26 08:32:25,639 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:32:26,561 epoch 77 - iter 2/25 - loss 2.35044253 - samples/sec: 138.98 - lr: 0.000244 2021-03-26 08:32:27,486 epoch 77 - iter 4/25 - loss 2.85872233 - samples/sec: 138.68 - lr: 0.000244 2021-03-26 08:32:28,417 epoch 77 - iter 6/25 - loss 2.92169376 - samples/sec: 137.62 - lr: 0.000244 2021-03-26 08:32:29,416 epoch 77 - iter 8/25 - loss 2.81242767 - samples/sec: 128.31 - lr: 0.000244 2021-03-26 08:32:30,435 epoch 77 - iter 10/25 - loss 2.79586065 - samples/sec: 125.78 - lr: 0.000244 2021-03-26 08:32:31,461 epoch 77 - iter 12/25 - loss 2.73711844 - samples/sec: 125.00 - lr: 0.000244 2021-03-26 08:32:32,630 epoch 77 - iter 14/25 - loss 2.69382988 - samples/sec: 109.67 - lr: 0.000244 2021-03-26 08:32:33,805 epoch 77 - iter 16/25 - loss 2.66051930 - samples/sec: 109.02 - lr: 0.000244 2021-03-26 08:32:34,804 epoch 77 - iter 18/25 - loss 2.65610683 - samples/sec: 128.29 - lr: 0.000244 2021-03-26 08:32:35,816 epoch 77 - iter 20/25 - loss 2.59753153 - samples/sec: 126.66 - lr: 0.000244 2021-03-26 08:32:36,754 epoch 77 - iter 22/25 - loss 2.58311897 - samples/sec: 136.66 - lr: 0.000244 2021-03-26 08:32:37,745 epoch 77 - iter 24/25 - loss 2.57920326 - samples/sec: 129.38 - lr: 0.000244 2021-03-26 08:32:38,188 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:32:38,189 EPOCH 77 done: loss 2.5370 - lr 0.0002441 2021-03-26 08:32:38,917 DEV : loss 5.156017780303955 - score 0.9175 2021-03-26 08:32:38,940 BAD EPOCHS (no improvement): 3 2021-03-26 08:32:38,941 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:32:39,892 epoch 78 - iter 2/25 - loss 2.84578502 - samples/sec: 134.80 - lr: 0.000244 2021-03-26 08:32:40,911 epoch 78 - iter 4/25 - loss 2.86701858 - samples/sec: 125.90 - lr: 0.000244 2021-03-26 08:32:41,923 epoch 78 - iter 6/25 - loss 2.53632077 - samples/sec: 126.58 - lr: 0.000244 2021-03-26 08:32:42,896 epoch 78 - iter 8/25 - loss 2.59809896 - samples/sec: 131.74 - lr: 0.000244 2021-03-26 08:32:43,830 epoch 78 - iter 10/25 - loss 2.44800783 - samples/sec: 137.33 - lr: 0.000244 2021-03-26 08:32:44,759 epoch 78 - iter 12/25 - loss 2.47062728 - samples/sec: 137.99 - lr: 0.000244 2021-03-26 08:32:45,724 epoch 78 - iter 14/25 - loss 2.50483849 - samples/sec: 132.84 - lr: 0.000244 2021-03-26 08:32:46,797 epoch 78 - iter 16/25 - loss 2.52945174 - samples/sec: 119.51 - lr: 0.000244 2021-03-26 08:32:47,790 epoch 78 - iter 18/25 - loss 2.58047466 - samples/sec: 128.96 - lr: 0.000244 2021-03-26 08:32:48,698 epoch 78 - iter 20/25 - loss 2.59792646 - samples/sec: 141.18 - lr: 0.000244 2021-03-26 08:32:49,749 epoch 78 - iter 22/25 - loss 2.58495852 - samples/sec: 121.97 - lr: 0.000244 2021-03-26 08:32:50,717 epoch 78 - iter 24/25 - loss 2.54879833 - samples/sec: 132.52 - lr: 0.000244 2021-03-26 08:32:51,194 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:32:51,194 EPOCH 78 done: loss 2.5141 - lr 0.0002441 2021-03-26 08:32:51,938 DEV : loss 5.1558074951171875 - score 0.9175 2021-03-26 08:32:51,969 BAD EPOCHS (no improvement): 4 2021-03-26 08:32:51,970 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:32:52,992 epoch 79 - iter 2/25 - loss 2.72548425 - samples/sec: 125.42 - lr: 0.000122 2021-03-26 08:32:53,943 epoch 79 - iter 4/25 - loss 2.43002558 - samples/sec: 134.78 - lr: 0.000122 2021-03-26 08:32:54,991 epoch 79 - iter 6/25 - loss 2.39026821 - samples/sec: 122.34 - lr: 0.000122 2021-03-26 08:32:55,965 epoch 79 - iter 8/25 - loss 2.45706752 - samples/sec: 131.58 - lr: 0.000122 2021-03-26 08:32:56,886 epoch 79 - iter 10/25 - loss 2.42801330 - samples/sec: 139.07 - lr: 0.000122 2021-03-26 08:32:57,826 epoch 79 - iter 12/25 - loss 2.43927479 - samples/sec: 136.48 - lr: 0.000122 2021-03-26 08:32:58,830 epoch 79 - iter 14/25 - loss 2.41582305 - samples/sec: 127.68 - lr: 0.000122 2021-03-26 08:32:59,895 epoch 79 - iter 16/25 - loss 2.52669768 - samples/sec: 120.32 - lr: 0.000122 2021-03-26 08:33:00,825 epoch 79 - iter 18/25 - loss 2.50340940 - samples/sec: 137.90 - lr: 0.000122 2021-03-26 08:33:01,861 epoch 79 - iter 20/25 - loss 2.48553985 - samples/sec: 123.78 - lr: 0.000122 2021-03-26 08:33:02,826 epoch 79 - iter 22/25 - loss 2.50086822 - samples/sec: 132.91 - lr: 0.000122 2021-03-26 08:33:03,888 epoch 79 - iter 24/25 - loss 2.50031297 - samples/sec: 120.70 - lr: 0.000122 2021-03-26 08:33:04,279 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:04,280 EPOCH 79 done: loss 2.4812 - lr 0.0001221 2021-03-26 08:33:05,033 DEV : loss 5.15554141998291 - score 0.9175 2021-03-26 08:33:05,065 BAD EPOCHS (no improvement): 1 2021-03-26 08:33:05,065 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:06,129 epoch 80 - iter 2/25 - loss 1.92892826 - samples/sec: 120.59 - lr: 0.000122 2021-03-26 08:33:07,054 epoch 80 - iter 4/25 - loss 1.89410821 - samples/sec: 138.55 - lr: 0.000122 2021-03-26 08:33:08,094 epoch 80 - iter 6/25 - loss 2.19573849 - samples/sec: 123.29 - lr: 0.000122 2021-03-26 08:33:09,128 epoch 80 - iter 8/25 - loss 2.34732698 - samples/sec: 123.90 - lr: 0.000122 2021-03-26 08:33:10,045 epoch 80 - iter 10/25 - loss 2.49000527 - samples/sec: 139.87 - lr: 0.000122 2021-03-26 08:33:11,079 epoch 80 - iter 12/25 - loss 2.46932315 - samples/sec: 123.93 - lr: 0.000122 2021-03-26 08:33:12,139 epoch 80 - iter 14/25 - loss 2.45208756 - samples/sec: 120.96 - lr: 0.000122 2021-03-26 08:33:13,168 epoch 80 - iter 16/25 - loss 2.46849288 - samples/sec: 124.61 - lr: 0.000122 2021-03-26 08:33:14,237 epoch 80 - iter 18/25 - loss 2.46822666 - samples/sec: 119.92 - lr: 0.000122 2021-03-26 08:33:15,231 epoch 80 - iter 20/25 - loss 2.48660938 - samples/sec: 128.99 - lr: 0.000122 2021-03-26 08:33:16,195 epoch 80 - iter 22/25 - loss 2.49127837 - samples/sec: 132.96 - lr: 0.000122 2021-03-26 08:33:17,218 epoch 80 - iter 24/25 - loss 2.49067168 - samples/sec: 125.22 - lr: 0.000122 2021-03-26 08:33:17,654 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:17,654 EPOCH 80 done: loss 2.4916 - lr 0.0001221 2021-03-26 08:33:18,378 DEV : loss 5.155505180358887 - score 0.9175 2021-03-26 08:33:18,394 BAD EPOCHS (no improvement): 2 2021-03-26 08:33:18,395 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:19,480 epoch 81 - iter 2/25 - loss 2.51575708 - samples/sec: 118.07 - lr: 0.000122 2021-03-26 08:33:20,596 epoch 81 - iter 4/25 - loss 2.59854716 - samples/sec: 114.84 - lr: 0.000122 2021-03-26 08:33:21,514 epoch 81 - iter 6/25 - loss 2.59062302 - samples/sec: 139.63 - lr: 0.000122 2021-03-26 08:33:22,572 epoch 81 - iter 8/25 - loss 2.50448807 - samples/sec: 121.15 - lr: 0.000122 2021-03-26 08:33:23,767 epoch 81 - iter 10/25 - loss 2.54231604 - samples/sec: 107.28 - lr: 0.000122 2021-03-26 08:33:24,618 epoch 81 - iter 12/25 - loss 2.47345619 - samples/sec: 150.66 - lr: 0.000122 2021-03-26 08:33:25,531 epoch 81 - iter 14/25 - loss 2.43137515 - samples/sec: 140.44 - lr: 0.000122 2021-03-26 08:33:26,489 epoch 81 - iter 16/25 - loss 2.44498353 - samples/sec: 133.80 - lr: 0.000122 2021-03-26 08:33:27,389 epoch 81 - iter 18/25 - loss 2.42864709 - samples/sec: 142.35 - lr: 0.000122 2021-03-26 08:33:28,390 epoch 81 - iter 20/25 - loss 2.48083928 - samples/sec: 128.02 - lr: 0.000122 2021-03-26 08:33:29,390 epoch 81 - iter 22/25 - loss 2.47235147 - samples/sec: 128.20 - lr: 0.000122 2021-03-26 08:33:30,381 epoch 81 - iter 24/25 - loss 2.49743044 - samples/sec: 129.25 - lr: 0.000122 2021-03-26 08:33:30,811 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:30,812 EPOCH 81 done: loss 2.4886 - lr 0.0001221 2021-03-26 08:33:31,536 DEV : loss 5.15550422668457 - score 0.9175 2021-03-26 08:33:31,559 BAD EPOCHS (no improvement): 3 2021-03-26 08:33:31,559 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:32,573 epoch 82 - iter 2/25 - loss 2.69717109 - samples/sec: 126.55 - lr: 0.000122 2021-03-26 08:33:33,590 epoch 82 - iter 4/25 - loss 2.38267857 - samples/sec: 126.02 - lr: 0.000122 2021-03-26 08:33:34,533 epoch 82 - iter 6/25 - loss 2.37339608 - samples/sec: 136.00 - lr: 0.000122 2021-03-26 08:33:35,502 epoch 82 - iter 8/25 - loss 2.46923965 - samples/sec: 132.37 - lr: 0.000122 2021-03-26 08:33:36,515 epoch 82 - iter 10/25 - loss 2.51084020 - samples/sec: 126.59 - lr: 0.000122 2021-03-26 08:33:37,555 epoch 82 - iter 12/25 - loss 2.60109609 - samples/sec: 123.18 - lr: 0.000122 2021-03-26 08:33:38,522 epoch 82 - iter 14/25 - loss 2.60343652 - samples/sec: 132.67 - lr: 0.000122 2021-03-26 08:33:39,516 epoch 82 - iter 16/25 - loss 2.63582183 - samples/sec: 128.88 - lr: 0.000122 2021-03-26 08:33:40,493 epoch 82 - iter 18/25 - loss 2.60362657 - samples/sec: 131.31 - lr: 0.000122 2021-03-26 08:33:41,508 epoch 82 - iter 20/25 - loss 2.61728383 - samples/sec: 126.16 - lr: 0.000122 2021-03-26 08:33:42,475 epoch 82 - iter 22/25 - loss 2.59926463 - samples/sec: 132.56 - lr: 0.000122 2021-03-26 08:33:43,415 epoch 82 - iter 24/25 - loss 2.59879770 - samples/sec: 136.37 - lr: 0.000122 2021-03-26 08:33:43,790 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:43,790 EPOCH 82 done: loss 2.5751 - lr 0.0001221 2021-03-26 08:33:44,512 DEV : loss 5.1555328369140625 - score 0.9175 2021-03-26 08:33:44,534 BAD EPOCHS (no improvement): 4 2021-03-26 08:33:44,535 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:44,536 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:44,536 learning rate too small - quitting training! 2021-03-26 08:33:44,536 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:53,804 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:33:53,804 Testing using best model ... 2021-03-26 08:33:53,805 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260811/best-model.pt 2021-03-26 08:34:00,860 0.9004 2021-03-26 08:34:00,861 Results: - F-score (micro): 0.897 - F-score (macro): 0.5285 - Accuracy (incl. no class): 0.9004 By class: precision recall f1-score support PRON 0.9706 0.9706 0.9706 204 NOUN 0.8985 0.9356 0.9167 388 ADJ 0.8649 0.8136 0.8384 118 SCONJ 0.8649 1.0000 0.9275 32 VERB 0.9259 0.9191 0.9225 136 DET 0.9167 0.9167 0.9167 48 CCONJ 0.9697 0.8889 0.9275 72 ADP 0.9541 0.9811 0.9674 106 ADV 0.8739 0.8509 0.8622 114 PROPN 0.7619 0.6957 0.7273 23 NUM 0.8571 0.7059 0.7742 17 PART 0.9508 0.9775 0.9640 178 PUNCT 1.0000 1.0000 1.0000 32 AUX 0.9459 0.9459 0.9459 37 INTJ 0.9583 0.9583 0.9583 24 X 1.0000 0.0000 0.0000 1 HASH 1.0000 1.0000 1.0000 12 V+PRON 0.8776 0.8113 0.8431 53 V+PRON+PRON 0.5000 0.5556 0.5263 9 PREP+PRON 0.8333 0.9615 0.8929 26 DET+NOUN+NSUFF 0.8750 0.9130 0.8936 23 PREP 0.8889 0.9552 0.9209 67 CONJ+PART 0.9000 0.9000 0.9000 10 V 0.9481 0.8902 0.9182 82 CONJ+V+PRON 0.5000 0.6250 0.5556 8 EOS 1.0000 1.0000 1.0000 70 NOUN+NSUFF 0.8571 0.9231 0.8889 39 PREP+NOUN+PRON 0.3333 0.5000 0.4000 4 PUNC 1.0000 1.0000 1.0000 152 NOUN+PRON 0.7971 0.8209 0.8088 67 V+PRON+PREP+PRON 0.5000 1.0000 0.6667 1 NOUN+NSUFF+PRON 0.6500 0.7222 0.6842 18 CONJ 1.0000 0.8438 0.9153 32 FOREIGN 1.0000 1.0000 1.0000 3 PROG_PART+V 0.8235 0.9333 0.8750 30 PREP+NOUN 0.9048 0.9500 0.9268 20 PART+PRON 0.9474 0.9000 0.9231 20 PREP+NOUN+NSUFF 0.6667 0.4000 0.5000 5 PROG_PART+V+PRON 0.8571 0.7826 0.8182 23 DET+NOUN 0.9103 0.9595 0.9342 74 DET+ADJ+NSUFF 0.6000 0.7500 0.6667 4 CONJ+DET+NOUN 1.0000 0.5000 0.6667 4 FUT_PART+V+PRON 1.0000 0.7500 0.8571 4 DET+ADJ 0.9091 0.5882 0.7143 17 NOUN+CASE+PRON 0.0000 1.0000 0.0000 0 ADJ+NSUFF 0.7879 0.8966 0.8387 29 CONJ+NOUN+PRON 0.6667 0.7500 0.7059 8 CONJ+NOUN+NSUFF+PRON 0.0000 0.0000 0.0000 1 CONJ+V 0.6000 0.6000 0.6000 5 CONJ+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+NOUN 0.7778 1.0000 0.8750 7 CONJ+DET+ADJ 0.0000 0.0000 0.0000 1 CONJ+PREP+DET+NOUN 0.5000 1.0000 0.6667 1 EMOT 1.0000 1.0000 1.0000 11 MENTION 1.0000 1.0000 1.0000 16 CONJ+FUT_PART+V 1.0000 0.0000 0.0000 2 CONJ+PROG_PART+V 0.0000 1.0000 0.0000 0 CONJ+NOUN+NSUFF 1.0000 0.5000 0.6667 2 FUT_PART+V 0.8000 0.8000 0.8000 15 PRON+DET+NOUN 1.0000 0.0000 0.0000 3 PREP+DET+NOUN 0.8889 1.0000 0.9412 8 CONJ+PART+NOUN 1.0000 0.0000 0.0000 1 V+PREP+PRON 0.3333 0.2500 0.2857 4 ADJ+CASE 1.0000 0.3333 0.5000 3 PREP+NOUN+NSUFF+PRON 0.0000 0.0000 0.0000 1 CONJ+PRON 1.0000 0.6667 0.8000 3 ADJ+PRON 0.0000 0.0000 0.0000 5 CONJ+DET+ADJ+NSUFF 1.0000 0.0000 0.0000 1 CONJ+DET+NOUN+NSUFF 0.0000 0.0000 0.0000 1 URL 1.0000 1.0000 1.0000 3 PRON+NOUN 1.0000 0.0000 0.0000 1 NUM+NSUFF 1.0000 0.0000 0.0000 3 NOUN+NSUFF+NSUFF 0.0000 1.0000 0.0000 0 PREP+PART+PRON 1.0000 1.0000 1.0000 2 NOUN+PREP+PRON 1.0000 0.0000 0.0000 1 PART+PROG_PART+V+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 ADV+PRON 1.0000 0.0000 0.0000 1 FUT_PART+V+PREP+PRON 0.0000 1.0000 0.0000 0 PREP+ADJ+NSUFF 1.0000 0.0000 0.0000 2 NOUN+CASE 0.5000 0.7500 0.6000 4 CONJ+PREP 1.0000 1.0000 1.0000 1 PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 PART+V+PRON 0.0000 0.0000 0.0000 1 PREP+PRON+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 PRON+DET+NOUN+NSUFF 0.0000 1.0000 0.0000 0 PREP+DET 1.0000 0.0000 0.0000 1 PART+V 0.5000 1.0000 0.6667 1 FUT_PART+V+PRON+PRON 1.0000 0.5000 0.6667 2 PART+FUT_PART 1.0000 0.0000 0.0000 1 PROG_PART+V+PRON+PRON 0.0000 1.0000 0.0000 0 CONJ+FUT_PART+V+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+PROG_PART+V+PRON 0.5000 1.0000 0.6667 1 PART+PREP 1.0000 1.0000 1.0000 1 FUT_PART 1.0000 1.0000 1.0000 1 CONJ+PART+V+PRON 1.0000 0.0000 0.0000 2 CONJ+PART+PRON 1.0000 0.0000 0.0000 1 CONJ+ADV 0.0000 1.0000 0.0000 0 CONJ+V+PRON+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+PREP+DET+NOUN+NSUFF 0.0000 1.0000 0.0000 0 PART+PROG_PART+V+NEG_PART 1.0000 1.0000 1.0000 1 PART+NOUN 0.6000 0.7500 0.6667 4 CONJ+PREP+PART 1.0000 0.0000 0.0000 1 CONJ+V+PREP+PRON 1.0000 0.0000 0.0000 1 PART+PREP+NEG_PART 1.0000 1.0000 1.0000 2 PART+V+NEG_PART 0.3333 0.6667 0.4444 3 PART+V+PRON+NEG_PART 0.7500 0.4286 0.5455 7 PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 PREP+DET+ADJ 1.0000 0.0000 0.0000 1 CONJ+ADJ+NSUFF 1.0000 0.0000 0.0000 1 CONJ+PART+V+PRON+NEG_PART 0.0000 1.0000 0.0000 0 CONJ+ADJ 1.0000 1.0000 1.0000 1 ADV+NSUFF 1.0000 1.0000 1.0000 2 PART+V+PRON+PRON+NEG_PART 1.0000 0.0000 0.0000 1 PART+NOUN+NSUFF 1.0000 0.0000 0.0000 1 micro avg 0.8971 0.8968 0.8970 2597 macro avg 0.7573 0.6262 0.5285 2597 weighted avg 0.9056 0.8968 0.8931 2597 2021-03-26 08:34:00,862 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:00,862 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:06,156 Reading data from ../../Datasets_adhoc/CSCS_corpus-GUC 2021-03-26 08:34:06,157 Train: ../../Datasets_adhoc/CSCS_corpus-GUC/all_participants.conllu 2021-03-26 08:34:06,157 Dev: None 2021-03-26 08:34:06,158 Test: None 2021-03-26 08:34:07,496 Reading data from ../../Datasets_adhoc/UD_MADAR 2021-03-26 08:34:07,496 Train: ../../Datasets_adhoc/UD_MADAR/ajp_madar-ud-test-edit.conllu 2021-03-26 08:34:07,497 Dev: None 2021-03-26 08:34:07,497 Test: None 2021-03-26 08:34:07,544 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 08:34:07,545 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_lev.txt 2021-03-26 08:34:07,546 Dev: None 2021-03-26 08:34:07,546 Test: None 2021-03-26 08:34:07,717 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 08:34:07,718 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_egy.txt 2021-03-26 08:34:07,718 Dev: None 2021-03-26 08:34:07,719 Test: None 2021-03-26 08:34:07,889 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 08:34:07,890 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_glf.txt 2021-03-26 08:34:07,890 Dev: None 2021-03-26 08:34:07,890 Test: None 2021-03-26 08:34:08,033 Reading data from ../../Datasets_adhoc/dialectal_arabic_resources-master 2021-03-26 08:34:08,034 Train: ../../Datasets_adhoc/dialectal_arabic_resources-master/seg_plus_pos_mgr.txt 2021-03-26 08:34:08,034 Dev: None 2021-03-26 08:34:08,034 Test: None 2021-03-26 08:34:08,185 Filtering long sentences 2021-03-26 08:34:08,214 MultiCorpus: 1575 train + 175 dev + 194 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences 2021-03-26 08:34:08,592 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:08,593 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): WordEmbeddings('ar') (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) (list_embedding_2): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.1, inplace=False) (encoder): Embedding(7125, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=7125, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4396, out_features=4396, bias=True) (rnn): LSTM(4396, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=206, bias=True) (beta): 1.0 (weights): None (weight_tensor) None )" 2021-03-26 08:34:08,593 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:08,594 Corpus: "MultiCorpus: 1575 train + 175 dev + 194 test sentences - ColumnCorpus Corpus: 934 train + 104 dev + 115 test sentences - ColumnCorpus Corpus: 81 train + 9 dev + 10 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences - ColumnCorpus Corpus: 283 train + 32 dev + 35 test sentences" 2021-03-26 08:34:08,594 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:08,594 Parameters: 2021-03-26 08:34:08,594 - learning_rate: "0.5" 2021-03-26 08:34:08,594 - mini_batch_size: "64" 2021-03-26 08:34:08,594 - patience: "3" 2021-03-26 08:34:08,595 - anneal_factor: "0.5" 2021-03-26 08:34:08,595 - max_epochs: "150" 2021-03-26 08:34:08,595 - shuffle: "True" 2021-03-26 08:34:08,595 - train_with_dev: "False" 2021-03-26 08:34:08,595 - batch_growth_annealing: "False" 2021-03-26 08:34:08,596 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:08,596 Model training base path: "/home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260834" 2021-03-26 08:34:08,596 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:08,596 Device: cuda:0 2021-03-26 08:34:08,597 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:08,597 Embeddings storage mode: cpu 2021-03-26 08:34:08,598 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:10,119 epoch 1 - iter 2/25 - loss 72.56362534 - samples/sec: 84.26 - lr: 0.500000 2021-03-26 08:34:11,473 epoch 1 - iter 4/25 - loss 79.34148407 - samples/sec: 94.93 - lr: 0.500000 2021-03-26 08:34:12,754 epoch 1 - iter 6/25 - loss 74.84145292 - samples/sec: 100.03 - lr: 0.500000 2021-03-26 08:34:14,201 epoch 1 - iter 8/25 - loss 72.96682358 - samples/sec: 88.58 - lr: 0.500000 2021-03-26 08:34:15,689 epoch 1 - iter 10/25 - loss 72.07293091 - samples/sec: 86.08 - lr: 0.500000 2021-03-26 08:34:16,909 epoch 1 - iter 12/25 - loss 68.28134028 - samples/sec: 105.00 - lr: 0.500000 2021-03-26 08:34:18,230 epoch 1 - iter 14/25 - loss 67.68329539 - samples/sec: 97.04 - lr: 0.500000 2021-03-26 08:34:19,587 epoch 1 - iter 16/25 - loss 66.21412325 - samples/sec: 94.45 - lr: 0.500000 2021-03-26 08:34:20,906 epoch 1 - iter 18/25 - loss 64.04796452 - samples/sec: 97.11 - lr: 0.500000 2021-03-26 08:34:22,192 epoch 1 - iter 20/25 - loss 63.26277981 - samples/sec: 99.69 - lr: 0.500000 2021-03-26 08:34:23,553 epoch 1 - iter 22/25 - loss 62.30557043 - samples/sec: 94.07 - lr: 0.500000 2021-03-26 08:34:24,854 epoch 1 - iter 24/25 - loss 60.86900187 - samples/sec: 98.54 - lr: 0.500000 2021-03-26 08:34:25,383 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:25,383 EPOCH 1 done: loss 60.2458 - lr 0.5000000 2021-03-26 08:34:26,609 DEV : loss 47.09407043457031 - score 0.305 2021-03-26 08:34:26,632 BAD EPOCHS (no improvement): 0 2021-03-26 08:34:36,005 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:37,029 epoch 2 - iter 2/25 - loss 45.07046700 - samples/sec: 125.27 - lr: 0.500000 2021-03-26 08:34:38,022 epoch 2 - iter 4/25 - loss 44.52553463 - samples/sec: 129.17 - lr: 0.500000 2021-03-26 08:34:38,979 epoch 2 - iter 6/25 - loss 42.01777840 - samples/sec: 134.09 - lr: 0.500000 2021-03-26 08:34:39,908 epoch 2 - iter 8/25 - loss 40.97041130 - samples/sec: 138.00 - lr: 0.500000 2021-03-26 08:34:40,924 epoch 2 - iter 10/25 - loss 39.93429756 - samples/sec: 126.17 - lr: 0.500000 2021-03-26 08:34:42,012 epoch 2 - iter 12/25 - loss 38.79689900 - samples/sec: 117.86 - lr: 0.500000 2021-03-26 08:34:42,960 epoch 2 - iter 14/25 - loss 38.76790496 - samples/sec: 135.23 - lr: 0.500000 2021-03-26 08:34:43,991 epoch 2 - iter 16/25 - loss 38.38748801 - samples/sec: 124.30 - lr: 0.500000 2021-03-26 08:34:45,022 epoch 2 - iter 18/25 - loss 37.90621662 - samples/sec: 124.28 - lr: 0.500000 2021-03-26 08:34:46,056 epoch 2 - iter 20/25 - loss 37.44555807 - samples/sec: 124.01 - lr: 0.500000 2021-03-26 08:34:46,991 epoch 2 - iter 22/25 - loss 36.58613716 - samples/sec: 137.09 - lr: 0.500000 2021-03-26 08:34:48,099 epoch 2 - iter 24/25 - loss 35.93759561 - samples/sec: 115.56 - lr: 0.500000 2021-03-26 08:34:48,504 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:48,504 EPOCH 2 done: loss 35.7602 - lr 0.5000000 2021-03-26 08:34:49,276 DEV : loss 32.68856430053711 - score 0.5186 2021-03-26 08:34:49,294 BAD EPOCHS (no improvement): 0 2021-03-26 08:34:58,804 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:34:59,920 epoch 3 - iter 2/25 - loss 30.71884441 - samples/sec: 114.93 - lr: 0.500000 2021-03-26 08:35:00,892 epoch 3 - iter 4/25 - loss 31.72236824 - samples/sec: 132.05 - lr: 0.500000 2021-03-26 08:35:01,891 epoch 3 - iter 6/25 - loss 29.34960175 - samples/sec: 128.28 - lr: 0.500000 2021-03-26 08:35:02,983 epoch 3 - iter 8/25 - loss 28.83567262 - samples/sec: 117.29 - lr: 0.500000 2021-03-26 08:35:03,978 epoch 3 - iter 10/25 - loss 28.65415955 - samples/sec: 128.78 - lr: 0.500000 2021-03-26 08:35:05,025 epoch 3 - iter 12/25 - loss 28.65188265 - samples/sec: 122.42 - lr: 0.500000 2021-03-26 08:35:06,054 epoch 3 - iter 14/25 - loss 28.25057289 - samples/sec: 124.63 - lr: 0.500000 2021-03-26 08:35:06,949 epoch 3 - iter 16/25 - loss 27.42237711 - samples/sec: 143.32 - lr: 0.500000 2021-03-26 08:35:07,927 epoch 3 - iter 18/25 - loss 26.69088671 - samples/sec: 131.10 - lr: 0.500000 2021-03-26 08:35:08,866 epoch 3 - iter 20/25 - loss 26.34798946 - samples/sec: 136.50 - lr: 0.500000 2021-03-26 08:35:09,798 epoch 3 - iter 22/25 - loss 25.97796076 - samples/sec: 137.58 - lr: 0.500000 2021-03-26 08:35:10,824 epoch 3 - iter 24/25 - loss 25.55988129 - samples/sec: 124.83 - lr: 0.500000 2021-03-26 08:35:11,192 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:35:11,192 EPOCH 3 done: loss 25.4119 - lr 0.5000000 2021-03-26 08:35:11,916 DEV : loss 19.450502395629883 - score 0.6724 2021-03-26 08:35:11,940 BAD EPOCHS (no improvement): 0 2021-03-26 08:35:21,664 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:35:22,579 epoch 4 - iter 2/25 - loss 20.26455975 - samples/sec: 140.22 - lr: 0.500000 2021-03-26 08:35:23,488 epoch 4 - iter 4/25 - loss 20.28417206 - samples/sec: 141.15 - lr: 0.500000 2021-03-26 08:35:24,512 epoch 4 - iter 6/25 - loss 20.85648982 - samples/sec: 125.22 - lr: 0.500000 2021-03-26 08:35:25,561 epoch 4 - iter 8/25 - loss 20.71336842 - samples/sec: 122.19 - lr: 0.500000 2021-03-26 08:35:26,627 epoch 4 - iter 10/25 - loss 21.16067410 - samples/sec: 120.26 - lr: 0.500000 2021-03-26 08:35:27,621 epoch 4 - iter 12/25 - loss 21.02786001 - samples/sec: 128.99 - lr: 0.500000 2021-03-26 08:35:28,671 epoch 4 - iter 14/25 - loss 20.83588055 - samples/sec: 122.10 - lr: 0.500000 2021-03-26 08:35:29,687 epoch 4 - iter 16/25 - loss 20.59734356 - samples/sec: 126.18 - lr: 0.500000 2021-03-26 08:35:30,603 epoch 4 - iter 18/25 - loss 20.37963645 - samples/sec: 140.06 - lr: 0.500000 2021-03-26 08:35:31,538 epoch 4 - iter 20/25 - loss 19.81514883 - samples/sec: 137.05 - lr: 0.500000 2021-03-26 08:35:32,553 epoch 4 - iter 22/25 - loss 19.77684385 - samples/sec: 126.27 - lr: 0.500000 2021-03-26 08:35:33,630 epoch 4 - iter 24/25 - loss 19.64338191 - samples/sec: 119.09 - lr: 0.500000 2021-03-26 08:35:34,042 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:35:34,042 EPOCH 4 done: loss 19.6755 - lr 0.5000000 2021-03-26 08:35:34,796 DEV : loss 14.270602226257324 - score 0.7716 2021-03-26 08:35:34,818 BAD EPOCHS (no improvement): 0 2021-03-26 08:35:44,420 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:35:45,530 epoch 5 - iter 2/25 - loss 15.16248655 - samples/sec: 115.62 - lr: 0.500000 2021-03-26 08:35:46,490 epoch 5 - iter 4/25 - loss 15.24637246 - samples/sec: 133.58 - lr: 0.500000 2021-03-26 08:35:47,652 epoch 5 - iter 6/25 - loss 16.62559589 - samples/sec: 110.23 - lr: 0.500000 2021-03-26 08:35:48,738 epoch 5 - iter 8/25 - loss 17.10516322 - samples/sec: 118.15 - lr: 0.500000 2021-03-26 08:35:49,918 epoch 5 - iter 10/25 - loss 17.65315447 - samples/sec: 108.55 - lr: 0.500000 2021-03-26 08:35:51,103 epoch 5 - iter 12/25 - loss 17.28618590 - samples/sec: 108.15 - lr: 0.500000 2021-03-26 08:35:51,982 epoch 5 - iter 14/25 - loss 17.00687572 - samples/sec: 145.92 - lr: 0.500000 2021-03-26 08:35:52,910 epoch 5 - iter 16/25 - loss 16.78228796 - samples/sec: 138.14 - lr: 0.500000 2021-03-26 08:35:53,891 epoch 5 - iter 18/25 - loss 16.75672356 - samples/sec: 130.69 - lr: 0.500000 2021-03-26 08:35:54,872 epoch 5 - iter 20/25 - loss 16.76150398 - samples/sec: 130.61 - lr: 0.500000 2021-03-26 08:35:55,883 epoch 5 - iter 22/25 - loss 16.63435515 - samples/sec: 126.81 - lr: 0.500000 2021-03-26 08:35:56,836 epoch 5 - iter 24/25 - loss 16.43824724 - samples/sec: 134.48 - lr: 0.500000 2021-03-26 08:35:57,308 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:35:57,309 EPOCH 5 done: loss 16.2580 - lr 0.5000000 2021-03-26 08:35:58,069 DEV : loss 12.45005989074707 - score 0.7882 2021-03-26 08:35:58,091 BAD EPOCHS (no improvement): 0 2021-03-26 08:36:07,764 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:36:08,767 epoch 6 - iter 2/25 - loss 14.79663944 - samples/sec: 127.90 - lr: 0.500000 2021-03-26 08:36:09,712 epoch 6 - iter 4/25 - loss 14.06922007 - samples/sec: 135.66 - lr: 0.500000 2021-03-26 08:36:10,731 epoch 6 - iter 6/25 - loss 13.96056747 - samples/sec: 125.81 - lr: 0.500000 2021-03-26 08:36:11,665 epoch 6 - iter 8/25 - loss 14.25202882 - samples/sec: 137.14 - lr: 0.500000 2021-03-26 08:36:12,618 epoch 6 - iter 10/25 - loss 14.02916756 - samples/sec: 134.54 - lr: 0.500000 2021-03-26 08:36:13,884 epoch 6 - iter 12/25 - loss 13.93397903 - samples/sec: 101.31 - lr: 0.500000 2021-03-26 08:36:15,122 epoch 6 - iter 14/25 - loss 13.85568230 - samples/sec: 103.48 - lr: 0.500000 2021-03-26 08:36:16,162 epoch 6 - iter 16/25 - loss 14.21910048 - samples/sec: 123.28 - lr: 0.500000 2021-03-26 08:36:17,161 epoch 6 - iter 18/25 - loss 14.39588801 - samples/sec: 128.37 - lr: 0.500000 2021-03-26 08:36:18,257 epoch 6 - iter 20/25 - loss 14.28939829 - samples/sec: 116.94 - lr: 0.500000 2021-03-26 08:36:19,194 epoch 6 - iter 22/25 - loss 14.12584786 - samples/sec: 136.82 - lr: 0.500000 2021-03-26 08:36:20,220 epoch 6 - iter 24/25 - loss 14.23289001 - samples/sec: 124.92 - lr: 0.500000 2021-03-26 08:36:20,553 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:36:20,553 EPOCH 6 done: loss 14.0823 - lr 0.5000000 2021-03-26 08:36:21,283 DEV : loss 11.072782516479492 - score 0.8198 2021-03-26 08:36:21,299 BAD EPOCHS (no improvement): 0 2021-03-26 08:36:30,981 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:36:32,262 epoch 7 - iter 2/25 - loss 12.87572241 - samples/sec: 100.12 - lr: 0.500000 2021-03-26 08:36:33,586 epoch 7 - iter 4/25 - loss 12.51348233 - samples/sec: 96.81 - lr: 0.500000 2021-03-26 08:36:34,826 epoch 7 - iter 6/25 - loss 12.69846837 - samples/sec: 103.30 - lr: 0.500000 2021-03-26 08:36:36,145 epoch 7 - iter 8/25 - loss 12.91704547 - samples/sec: 97.14 - lr: 0.500000 2021-03-26 08:36:37,117 epoch 7 - iter 10/25 - loss 12.93074894 - samples/sec: 132.01 - lr: 0.500000 2021-03-26 08:36:38,164 epoch 7 - iter 12/25 - loss 12.55548334 - samples/sec: 122.49 - lr: 0.500000 2021-03-26 08:36:39,063 epoch 7 - iter 14/25 - loss 12.45593732 - samples/sec: 142.64 - lr: 0.500000 2021-03-26 08:36:39,987 epoch 7 - iter 16/25 - loss 12.29866284 - samples/sec: 138.81 - lr: 0.500000 2021-03-26 08:36:40,989 epoch 7 - iter 18/25 - loss 12.40160184 - samples/sec: 127.91 - lr: 0.500000 2021-03-26 08:36:41,916 epoch 7 - iter 20/25 - loss 12.34104595 - samples/sec: 138.34 - lr: 0.500000 2021-03-26 08:36:42,949 epoch 7 - iter 22/25 - loss 12.41457592 - samples/sec: 124.03 - lr: 0.500000 2021-03-26 08:36:43,950 epoch 7 - iter 24/25 - loss 12.49163620 - samples/sec: 128.02 - lr: 0.500000 2021-03-26 08:36:44,382 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:36:44,383 EPOCH 7 done: loss 12.4664 - lr 0.5000000 2021-03-26 08:36:45,152 DEV : loss 11.440208435058594 - score 0.8104 2021-03-26 08:36:45,177 BAD EPOCHS (no improvement): 1 2021-03-26 08:36:45,178 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:36:46,089 epoch 8 - iter 2/25 - loss 10.71792698 - samples/sec: 140.71 - lr: 0.500000 2021-03-26 08:36:47,165 epoch 8 - iter 4/25 - loss 11.36736917 - samples/sec: 119.13 - lr: 0.500000 2021-03-26 08:36:48,124 epoch 8 - iter 6/25 - loss 11.52289375 - samples/sec: 133.76 - lr: 0.500000 2021-03-26 08:36:49,121 epoch 8 - iter 8/25 - loss 11.59045017 - samples/sec: 128.50 - lr: 0.500000 2021-03-26 08:36:50,155 epoch 8 - iter 10/25 - loss 11.33041563 - samples/sec: 124.07 - lr: 0.500000 2021-03-26 08:36:51,129 epoch 8 - iter 12/25 - loss 11.23035900 - samples/sec: 131.66 - lr: 0.500000 2021-03-26 08:36:52,047 epoch 8 - iter 14/25 - loss 11.34309203 - samples/sec: 139.60 - lr: 0.500000 2021-03-26 08:36:52,990 epoch 8 - iter 16/25 - loss 11.18513727 - samples/sec: 135.98 - lr: 0.500000 2021-03-26 08:36:53,996 epoch 8 - iter 18/25 - loss 11.26731078 - samples/sec: 127.39 - lr: 0.500000 2021-03-26 08:36:54,884 epoch 8 - iter 20/25 - loss 11.19707775 - samples/sec: 144.36 - lr: 0.500000 2021-03-26 08:36:55,841 epoch 8 - iter 22/25 - loss 11.44174082 - samples/sec: 134.01 - lr: 0.500000 2021-03-26 08:36:56,828 epoch 8 - iter 24/25 - loss 11.37479019 - samples/sec: 129.85 - lr: 0.500000 2021-03-26 08:36:57,289 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:36:57,290 EPOCH 8 done: loss 11.3881 - lr 0.5000000 2021-03-26 08:36:58,035 DEV : loss 9.5735502243042 - score 0.8414 2021-03-26 08:36:58,059 BAD EPOCHS (no improvement): 0 2021-03-26 08:37:07,275 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:37:08,220 epoch 9 - iter 2/25 - loss 10.45719576 - samples/sec: 135.78 - lr: 0.500000 2021-03-26 08:37:09,281 epoch 9 - iter 4/25 - loss 10.45271993 - samples/sec: 121.36 - lr: 0.500000 2021-03-26 08:37:10,312 epoch 9 - iter 6/25 - loss 10.19409847 - samples/sec: 124.24 - lr: 0.500000 2021-03-26 08:37:11,327 epoch 9 - iter 8/25 - loss 10.50802386 - samples/sec: 126.36 - lr: 0.500000 2021-03-26 08:37:12,329 epoch 9 - iter 10/25 - loss 10.60312605 - samples/sec: 127.89 - lr: 0.500000 2021-03-26 08:37:13,378 epoch 9 - iter 12/25 - loss 10.58466109 - samples/sec: 122.21 - lr: 0.500000 2021-03-26 08:37:14,361 epoch 9 - iter 14/25 - loss 10.61166177 - samples/sec: 130.49 - lr: 0.500000 2021-03-26 08:37:15,380 epoch 9 - iter 16/25 - loss 10.69750690 - samples/sec: 125.77 - lr: 0.500000 2021-03-26 08:37:16,507 epoch 9 - iter 18/25 - loss 10.68582206 - samples/sec: 113.75 - lr: 0.500000 2021-03-26 08:37:17,456 epoch 9 - iter 20/25 - loss 10.49424787 - samples/sec: 135.16 - lr: 0.500000 2021-03-26 08:37:18,358 epoch 9 - iter 22/25 - loss 10.47264957 - samples/sec: 142.10 - lr: 0.500000 2021-03-26 08:37:19,258 epoch 9 - iter 24/25 - loss 10.37763687 - samples/sec: 142.39 - lr: 0.500000 2021-03-26 08:37:19,632 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:37:19,633 EPOCH 9 done: loss 10.3442 - lr 0.5000000 2021-03-26 08:37:20,381 DEV : loss 8.980548858642578 - score 0.8552 2021-03-26 08:37:20,398 BAD EPOCHS (no improvement): 0 2021-03-26 08:37:30,009 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:37:31,060 epoch 10 - iter 2/25 - loss 9.98690414 - samples/sec: 122.03 - lr: 0.500000 2021-03-26 08:37:32,118 epoch 10 - iter 4/25 - loss 9.77284789 - samples/sec: 121.14 - lr: 0.500000 2021-03-26 08:37:33,151 epoch 10 - iter 6/25 - loss 10.12020858 - samples/sec: 124.13 - lr: 0.500000 2021-03-26 08:37:34,129 epoch 10 - iter 8/25 - loss 9.93451309 - samples/sec: 131.15 - lr: 0.500000 2021-03-26 08:37:35,079 epoch 10 - iter 10/25 - loss 9.50131526 - samples/sec: 135.08 - lr: 0.500000 2021-03-26 08:37:36,152 epoch 10 - iter 12/25 - loss 9.33990649 - samples/sec: 119.46 - lr: 0.500000 2021-03-26 08:37:37,195 epoch 10 - iter 14/25 - loss 9.14538295 - samples/sec: 122.95 - lr: 0.500000 2021-03-26 08:37:38,228 epoch 10 - iter 16/25 - loss 9.13881999 - samples/sec: 124.03 - lr: 0.500000 2021-03-26 08:37:39,194 epoch 10 - iter 18/25 - loss 9.19282744 - samples/sec: 132.71 - lr: 0.500000 2021-03-26 08:37:40,280 epoch 10 - iter 20/25 - loss 9.38936849 - samples/sec: 117.95 - lr: 0.500000 2021-03-26 08:37:41,269 epoch 10 - iter 22/25 - loss 9.55408716 - samples/sec: 129.67 - lr: 0.500000 2021-03-26 08:37:42,196 epoch 10 - iter 24/25 - loss 9.56575978 - samples/sec: 138.31 - lr: 0.500000 2021-03-26 08:37:42,643 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:37:42,644 EPOCH 10 done: loss 9.6068 - lr 0.5000000 2021-03-26 08:37:43,392 DEV : loss 8.683530807495117 - score 0.8596 2021-03-26 08:37:43,412 BAD EPOCHS (no improvement): 0 2021-03-26 08:37:53,041 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:37:54,008 epoch 11 - iter 2/25 - loss 8.94612074 - samples/sec: 132.62 - lr: 0.500000 2021-03-26 08:37:55,051 epoch 11 - iter 4/25 - loss 9.17710257 - samples/sec: 122.89 - lr: 0.500000 2021-03-26 08:37:55,985 epoch 11 - iter 6/25 - loss 8.82817014 - samples/sec: 137.20 - lr: 0.500000 2021-03-26 08:37:56,966 epoch 11 - iter 8/25 - loss 8.97275388 - samples/sec: 130.85 - lr: 0.500000 2021-03-26 08:37:57,933 epoch 11 - iter 10/25 - loss 9.16564417 - samples/sec: 132.59 - lr: 0.500000 2021-03-26 08:37:58,894 epoch 11 - iter 12/25 - loss 8.90157481 - samples/sec: 133.41 - lr: 0.500000 2021-03-26 08:37:59,848 epoch 11 - iter 14/25 - loss 8.76389977 - samples/sec: 134.37 - lr: 0.500000 2021-03-26 08:38:00,834 epoch 11 - iter 16/25 - loss 8.81811097 - samples/sec: 129.96 - lr: 0.500000 2021-03-26 08:38:01,823 epoch 11 - iter 18/25 - loss 8.87183176 - samples/sec: 129.77 - lr: 0.500000 2021-03-26 08:38:02,834 epoch 11 - iter 20/25 - loss 8.87209656 - samples/sec: 126.72 - lr: 0.500000 2021-03-26 08:38:03,733 epoch 11 - iter 22/25 - loss 8.93154610 - samples/sec: 142.58 - lr: 0.500000 2021-03-26 08:38:04,627 epoch 11 - iter 24/25 - loss 9.03405192 - samples/sec: 143.50 - lr: 0.500000 2021-03-26 08:38:04,988 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:38:04,989 EPOCH 11 done: loss 8.9434 - lr 0.5000000 2021-03-26 08:38:05,711 DEV : loss 7.607541561126709 - score 0.8788 2021-03-26 08:38:05,734 BAD EPOCHS (no improvement): 0 2021-03-26 08:38:15,129 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:38:16,130 epoch 12 - iter 2/25 - loss 7.44216204 - samples/sec: 128.18 - lr: 0.500000 2021-03-26 08:38:17,114 epoch 12 - iter 4/25 - loss 7.56906962 - samples/sec: 130.32 - lr: 0.500000 2021-03-26 08:38:18,052 epoch 12 - iter 6/25 - loss 7.80390724 - samples/sec: 136.73 - lr: 0.500000 2021-03-26 08:38:19,138 epoch 12 - iter 8/25 - loss 7.80161536 - samples/sec: 117.99 - lr: 0.500000 2021-03-26 08:38:20,109 epoch 12 - iter 10/25 - loss 7.69097266 - samples/sec: 132.06 - lr: 0.500000 2021-03-26 08:38:21,179 epoch 12 - iter 12/25 - loss 7.66236715 - samples/sec: 119.76 - lr: 0.500000 2021-03-26 08:38:22,187 epoch 12 - iter 14/25 - loss 7.76549241 - samples/sec: 127.18 - lr: 0.500000 2021-03-26 08:38:23,178 epoch 12 - iter 16/25 - loss 7.78144372 - samples/sec: 129.30 - lr: 0.500000 2021-03-26 08:38:24,256 epoch 12 - iter 18/25 - loss 7.74660744 - samples/sec: 119.00 - lr: 0.500000 2021-03-26 08:38:25,177 epoch 12 - iter 20/25 - loss 7.90175493 - samples/sec: 139.07 - lr: 0.500000 2021-03-26 08:38:26,171 epoch 12 - iter 22/25 - loss 8.10105963 - samples/sec: 129.05 - lr: 0.500000 2021-03-26 08:38:27,207 epoch 12 - iter 24/25 - loss 8.17068495 - samples/sec: 123.74 - lr: 0.500000 2021-03-26 08:38:27,625 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:38:27,626 EPOCH 12 done: loss 8.2684 - lr 0.5000000 2021-03-26 08:38:28,392 DEV : loss 7.355037212371826 - score 0.8781 2021-03-26 08:38:28,411 BAD EPOCHS (no improvement): 1 2021-03-26 08:38:28,412 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:38:29,456 epoch 13 - iter 2/25 - loss 6.67936397 - samples/sec: 122.80 - lr: 0.500000 2021-03-26 08:38:30,404 epoch 13 - iter 4/25 - loss 7.57645524 - samples/sec: 135.18 - lr: 0.500000 2021-03-26 08:38:31,396 epoch 13 - iter 6/25 - loss 7.58594402 - samples/sec: 129.29 - lr: 0.500000 2021-03-26 08:38:32,350 epoch 13 - iter 8/25 - loss 7.89992797 - samples/sec: 134.20 - lr: 0.500000 2021-03-26 08:38:33,347 epoch 13 - iter 10/25 - loss 7.78038921 - samples/sec: 128.74 - lr: 0.500000 2021-03-26 08:38:34,356 epoch 13 - iter 12/25 - loss 7.66072885 - samples/sec: 126.96 - lr: 0.500000 2021-03-26 08:38:35,420 epoch 13 - iter 14/25 - loss 7.55210911 - samples/sec: 120.48 - lr: 0.500000 2021-03-26 08:38:36,577 epoch 13 - iter 16/25 - loss 7.69292659 - samples/sec: 111.17 - lr: 0.500000 2021-03-26 08:38:37,593 epoch 13 - iter 18/25 - loss 7.75315873 - samples/sec: 126.23 - lr: 0.500000 2021-03-26 08:38:38,590 epoch 13 - iter 20/25 - loss 7.81584940 - samples/sec: 128.51 - lr: 0.500000 2021-03-26 08:38:39,520 epoch 13 - iter 22/25 - loss 7.80141653 - samples/sec: 137.87 - lr: 0.500000 2021-03-26 08:38:40,478 epoch 13 - iter 24/25 - loss 7.83267780 - samples/sec: 133.82 - lr: 0.500000 2021-03-26 08:38:40,834 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:38:40,835 EPOCH 13 done: loss 7.8342 - lr 0.5000000 2021-03-26 08:38:41,566 DEV : loss 7.358130931854248 - score 0.8841 2021-03-26 08:38:41,590 BAD EPOCHS (no improvement): 0 2021-03-26 08:38:50,806 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:38:51,808 epoch 14 - iter 2/25 - loss 6.54843807 - samples/sec: 128.04 - lr: 0.500000 2021-03-26 08:38:52,728 epoch 14 - iter 4/25 - loss 6.75311780 - samples/sec: 139.33 - lr: 0.500000 2021-03-26 08:38:53,728 epoch 14 - iter 6/25 - loss 7.10001183 - samples/sec: 128.21 - lr: 0.500000 2021-03-26 08:38:54,648 epoch 14 - iter 8/25 - loss 6.90141511 - samples/sec: 139.28 - lr: 0.500000 2021-03-26 08:38:55,531 epoch 14 - iter 10/25 - loss 7.01782126 - samples/sec: 145.20 - lr: 0.500000 2021-03-26 08:38:56,475 epoch 14 - iter 12/25 - loss 7.19454340 - samples/sec: 135.86 - lr: 0.500000 2021-03-26 08:38:57,459 epoch 14 - iter 14/25 - loss 7.26160775 - samples/sec: 130.31 - lr: 0.500000 2021-03-26 08:38:58,458 epoch 14 - iter 16/25 - loss 7.34082398 - samples/sec: 128.23 - lr: 0.500000 2021-03-26 08:38:59,442 epoch 14 - iter 18/25 - loss 7.50727646 - samples/sec: 130.36 - lr: 0.500000 2021-03-26 08:39:00,365 epoch 14 - iter 20/25 - loss 7.40654581 - samples/sec: 138.79 - lr: 0.500000 2021-03-26 08:39:01,364 epoch 14 - iter 22/25 - loss 7.31519435 - samples/sec: 128.39 - lr: 0.500000 2021-03-26 08:39:02,317 epoch 14 - iter 24/25 - loss 7.44761574 - samples/sec: 134.51 - lr: 0.500000 2021-03-26 08:39:02,718 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:39:02,719 EPOCH 14 done: loss 7.4455 - lr 0.5000000 2021-03-26 08:39:03,491 DEV : loss 7.53242826461792 - score 0.8761 2021-03-26 08:39:03,508 BAD EPOCHS (no improvement): 1 2021-03-26 08:39:03,509 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:39:04,559 epoch 15 - iter 2/25 - loss 7.57845902 - samples/sec: 122.11 - lr: 0.500000 2021-03-26 08:39:05,486 epoch 15 - iter 4/25 - loss 6.90868700 - samples/sec: 138.40 - lr: 0.500000 2021-03-26 08:39:06,481 epoch 15 - iter 6/25 - loss 7.40500800 - samples/sec: 128.76 - lr: 0.500000 2021-03-26 08:39:07,504 epoch 15 - iter 8/25 - loss 7.43870729 - samples/sec: 125.43 - lr: 0.500000 2021-03-26 08:39:08,523 epoch 15 - iter 10/25 - loss 7.39640107 - samples/sec: 125.85 - lr: 0.500000 2021-03-26 08:39:09,566 epoch 15 - iter 12/25 - loss 7.33035028 - samples/sec: 122.82 - lr: 0.500000 2021-03-26 08:39:10,417 epoch 15 - iter 14/25 - loss 7.25882561 - samples/sec: 150.82 - lr: 0.500000 2021-03-26 08:39:11,383 epoch 15 - iter 16/25 - loss 7.28578192 - samples/sec: 132.74 - lr: 0.500000 2021-03-26 08:39:12,277 epoch 15 - iter 18/25 - loss 7.23205696 - samples/sec: 143.39 - lr: 0.500000 2021-03-26 08:39:13,271 epoch 15 - iter 20/25 - loss 7.30031002 - samples/sec: 128.93 - lr: 0.500000 2021-03-26 08:39:14,196 epoch 15 - iter 22/25 - loss 7.25597163 - samples/sec: 138.67 - lr: 0.500000 2021-03-26 08:39:15,084 epoch 15 - iter 24/25 - loss 7.16761416 - samples/sec: 144.22 - lr: 0.500000 2021-03-26 08:39:15,503 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:39:15,504 EPOCH 15 done: loss 7.1529 - lr 0.5000000 2021-03-26 08:39:16,263 DEV : loss 7.279156684875488 - score 0.8884 2021-03-26 08:39:16,287 BAD EPOCHS (no improvement): 0 2021-03-26 08:39:25,789 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:39:26,927 epoch 16 - iter 2/25 - loss 6.64370656 - samples/sec: 112.61 - lr: 0.500000 2021-03-26 08:39:27,832 epoch 16 - iter 4/25 - loss 6.81591594 - samples/sec: 141.76 - lr: 0.500000 2021-03-26 08:39:28,778 epoch 16 - iter 6/25 - loss 6.51474301 - samples/sec: 135.60 - lr: 0.500000 2021-03-26 08:39:29,703 epoch 16 - iter 8/25 - loss 6.64946312 - samples/sec: 138.49 - lr: 0.500000 2021-03-26 08:39:30,728 epoch 16 - iter 10/25 - loss 6.83099728 - samples/sec: 125.05 - lr: 0.500000 2021-03-26 08:39:31,795 epoch 16 - iter 12/25 - loss 6.86230055 - samples/sec: 120.21 - lr: 0.500000 2021-03-26 08:39:32,842 epoch 16 - iter 14/25 - loss 6.90279361 - samples/sec: 122.40 - lr: 0.500000 2021-03-26 08:39:33,857 epoch 16 - iter 16/25 - loss 6.95628285 - samples/sec: 126.31 - lr: 0.500000 2021-03-26 08:39:34,779 epoch 16 - iter 18/25 - loss 6.91275793 - samples/sec: 138.99 - lr: 0.500000 2021-03-26 08:39:35,744 epoch 16 - iter 20/25 - loss 6.88684988 - samples/sec: 132.98 - lr: 0.500000 2021-03-26 08:39:36,721 epoch 16 - iter 22/25 - loss 6.92513863 - samples/sec: 131.28 - lr: 0.500000 2021-03-26 08:39:37,649 epoch 16 - iter 24/25 - loss 6.90109104 - samples/sec: 138.11 - lr: 0.500000 2021-03-26 08:39:38,014 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:39:38,015 EPOCH 16 done: loss 6.8474 - lr 0.5000000 2021-03-26 08:39:38,765 DEV : loss 7.319377422332764 - score 0.8802 2021-03-26 08:39:38,788 BAD EPOCHS (no improvement): 1 2021-03-26 08:39:38,789 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:39:39,920 epoch 17 - iter 2/25 - loss 5.65548801 - samples/sec: 113.31 - lr: 0.500000 2021-03-26 08:39:40,848 epoch 17 - iter 4/25 - loss 5.68925560 - samples/sec: 138.80 - lr: 0.500000 2021-03-26 08:39:41,815 epoch 17 - iter 6/25 - loss 6.08442839 - samples/sec: 132.59 - lr: 0.500000 2021-03-26 08:39:42,824 epoch 17 - iter 8/25 - loss 5.97657681 - samples/sec: 127.09 - lr: 0.500000 2021-03-26 08:39:43,832 epoch 17 - iter 10/25 - loss 6.26011467 - samples/sec: 127.30 - lr: 0.500000 2021-03-26 08:39:44,733 epoch 17 - iter 12/25 - loss 6.27742251 - samples/sec: 142.41 - lr: 0.500000 2021-03-26 08:39:45,731 epoch 17 - iter 14/25 - loss 6.18852012 - samples/sec: 128.35 - lr: 0.500000 2021-03-26 08:39:46,713 epoch 17 - iter 16/25 - loss 6.38471323 - samples/sec: 130.60 - lr: 0.500000 2021-03-26 08:39:47,739 epoch 17 - iter 18/25 - loss 6.52620485 - samples/sec: 124.86 - lr: 0.500000 2021-03-26 08:39:48,764 epoch 17 - iter 20/25 - loss 6.55608342 - samples/sec: 125.10 - lr: 0.500000 2021-03-26 08:39:49,818 epoch 17 - iter 22/25 - loss 6.47405910 - samples/sec: 121.67 - lr: 0.500000 2021-03-26 08:39:50,766 epoch 17 - iter 24/25 - loss 6.46988698 - samples/sec: 135.37 - lr: 0.500000 2021-03-26 08:39:51,111 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:39:51,112 EPOCH 17 done: loss 6.4509 - lr 0.5000000 2021-03-26 08:39:51,853 DEV : loss 6.72586727142334 - score 0.8953 2021-03-26 08:39:51,878 BAD EPOCHS (no improvement): 0 2021-03-26 08:40:01,225 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:40:02,219 epoch 18 - iter 2/25 - loss 6.05486703 - samples/sec: 129.05 - lr: 0.500000 2021-03-26 08:40:03,451 epoch 18 - iter 4/25 - loss 6.04667962 - samples/sec: 104.06 - lr: 0.500000 2021-03-26 08:40:04,462 epoch 18 - iter 6/25 - loss 5.93956693 - samples/sec: 126.96 - lr: 0.500000 2021-03-26 08:40:05,487 epoch 18 - iter 8/25 - loss 5.89706308 - samples/sec: 125.07 - lr: 0.500000 2021-03-26 08:40:06,433 epoch 18 - iter 10/25 - loss 5.82905989 - samples/sec: 135.45 - lr: 0.500000 2021-03-26 08:40:07,354 epoch 18 - iter 12/25 - loss 5.86640871 - samples/sec: 139.26 - lr: 0.500000 2021-03-26 08:40:08,333 epoch 18 - iter 14/25 - loss 5.87944187 - samples/sec: 130.84 - lr: 0.500000 2021-03-26 08:40:09,424 epoch 18 - iter 16/25 - loss 5.91088557 - samples/sec: 117.56 - lr: 0.500000 2021-03-26 08:40:10,369 epoch 18 - iter 18/25 - loss 5.97162143 - samples/sec: 135.65 - lr: 0.500000 2021-03-26 08:40:11,404 epoch 18 - iter 20/25 - loss 6.03725386 - samples/sec: 123.85 - lr: 0.500000 2021-03-26 08:40:12,408 epoch 18 - iter 22/25 - loss 6.11541224 - samples/sec: 127.79 - lr: 0.500000 2021-03-26 08:40:13,455 epoch 18 - iter 24/25 - loss 6.06696043 - samples/sec: 122.45 - lr: 0.500000 2021-03-26 08:40:13,913 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:40:13,913 EPOCH 18 done: loss 5.9884 - lr 0.5000000 2021-03-26 08:40:14,674 DEV : loss 6.871837139129639 - score 0.8914 2021-03-26 08:40:14,694 BAD EPOCHS (no improvement): 1 2021-03-26 08:40:14,694 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:40:15,696 epoch 19 - iter 2/25 - loss 5.24985552 - samples/sec: 127.91 - lr: 0.500000 2021-03-26 08:40:16,722 epoch 19 - iter 4/25 - loss 5.72425938 - samples/sec: 124.91 - lr: 0.500000 2021-03-26 08:40:17,651 epoch 19 - iter 6/25 - loss 5.87467360 - samples/sec: 138.16 - lr: 0.500000 2021-03-26 08:40:18,627 epoch 19 - iter 8/25 - loss 5.82207859 - samples/sec: 131.38 - lr: 0.500000 2021-03-26 08:40:19,646 epoch 19 - iter 10/25 - loss 5.81725163 - samples/sec: 125.76 - lr: 0.500000 2021-03-26 08:40:20,586 epoch 19 - iter 12/25 - loss 5.86108053 - samples/sec: 136.35 - lr: 0.500000 2021-03-26 08:40:21,545 epoch 19 - iter 14/25 - loss 5.91433290 - samples/sec: 133.76 - lr: 0.500000 2021-03-26 08:40:22,509 epoch 19 - iter 16/25 - loss 5.95745441 - samples/sec: 133.02 - lr: 0.500000 2021-03-26 08:40:23,495 epoch 19 - iter 18/25 - loss 5.95007020 - samples/sec: 130.00 - lr: 0.500000 2021-03-26 08:40:24,447 epoch 19 - iter 20/25 - loss 5.98423173 - samples/sec: 134.59 - lr: 0.500000 2021-03-26 08:40:25,521 epoch 19 - iter 22/25 - loss 6.01124003 - samples/sec: 119.38 - lr: 0.500000 2021-03-26 08:40:26,593 epoch 19 - iter 24/25 - loss 6.01857322 - samples/sec: 119.53 - lr: 0.500000 2021-03-26 08:40:26,991 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:40:26,992 EPOCH 19 done: loss 6.0306 - lr 0.5000000 2021-03-26 08:40:27,758 DEV : loss 7.084388732910156 - score 0.8885 2021-03-26 08:40:27,782 BAD EPOCHS (no improvement): 2 2021-03-26 08:40:27,783 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:40:28,699 epoch 20 - iter 2/25 - loss 4.97285271 - samples/sec: 139.91 - lr: 0.500000 2021-03-26 08:40:29,613 epoch 20 - iter 4/25 - loss 5.13549459 - samples/sec: 140.30 - lr: 0.500000 2021-03-26 08:40:30,616 epoch 20 - iter 6/25 - loss 5.40857466 - samples/sec: 127.93 - lr: 0.500000 2021-03-26 08:40:31,607 epoch 20 - iter 8/25 - loss 5.32610589 - samples/sec: 129.41 - lr: 0.500000 2021-03-26 08:40:32,533 epoch 20 - iter 10/25 - loss 5.53772874 - samples/sec: 138.40 - lr: 0.500000 2021-03-26 08:40:33,543 epoch 20 - iter 12/25 - loss 5.69724166 - samples/sec: 127.09 - lr: 0.500000 2021-03-26 08:40:34,538 epoch 20 - iter 14/25 - loss 5.63601783 - samples/sec: 128.87 - lr: 0.500000 2021-03-26 08:40:35,589 epoch 20 - iter 16/25 - loss 5.61971328 - samples/sec: 121.98 - lr: 0.500000 2021-03-26 08:40:36,594 epoch 20 - iter 18/25 - loss 5.67689334 - samples/sec: 127.52 - lr: 0.500000 2021-03-26 08:40:39,218 epoch 20 - iter 20/25 - loss 5.62989011 - samples/sec: 48.80 - lr: 0.500000 2021-03-26 08:40:40,191 epoch 20 - iter 22/25 - loss 5.64217880 - samples/sec: 131.86 - lr: 0.500000 2021-03-26 08:40:41,153 epoch 20 - iter 24/25 - loss 5.66027576 - samples/sec: 133.33 - lr: 0.500000 2021-03-26 08:40:41,618 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:40:41,619 EPOCH 20 done: loss 5.6115 - lr 0.5000000 2021-03-26 08:40:42,388 DEV : loss 6.946372032165527 - score 0.9005 2021-03-26 08:40:42,405 BAD EPOCHS (no improvement): 0 2021-03-26 08:40:51,793 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:40:52,833 epoch 21 - iter 2/25 - loss 6.11340022 - samples/sec: 123.25 - lr: 0.500000 2021-03-26 08:40:53,724 epoch 21 - iter 4/25 - loss 5.99155140 - samples/sec: 143.91 - lr: 0.500000 2021-03-26 08:40:54,676 epoch 21 - iter 6/25 - loss 5.56770198 - samples/sec: 134.61 - lr: 0.500000 2021-03-26 08:40:55,651 epoch 21 - iter 8/25 - loss 6.15605402 - samples/sec: 131.62 - lr: 0.500000 2021-03-26 08:40:56,618 epoch 21 - iter 10/25 - loss 6.06293464 - samples/sec: 132.62 - lr: 0.500000 2021-03-26 08:40:57,566 epoch 21 - iter 12/25 - loss 5.87442553 - samples/sec: 135.17 - lr: 0.500000 2021-03-26 08:40:58,667 epoch 21 - iter 14/25 - loss 5.79054798 - samples/sec: 116.49 - lr: 0.500000 2021-03-26 08:40:59,713 epoch 21 - iter 16/25 - loss 5.82898086 - samples/sec: 122.49 - lr: 0.500000 2021-03-26 08:41:00,854 epoch 21 - iter 18/25 - loss 5.75326135 - samples/sec: 112.32 - lr: 0.500000 2021-03-26 08:41:01,798 epoch 21 - iter 20/25 - loss 5.71771746 - samples/sec: 135.88 - lr: 0.500000 2021-03-26 08:41:02,742 epoch 21 - iter 22/25 - loss 5.65753235 - samples/sec: 135.71 - lr: 0.500000 2021-03-26 08:41:03,735 epoch 21 - iter 24/25 - loss 5.69550612 - samples/sec: 129.13 - lr: 0.500000 2021-03-26 08:41:04,153 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:41:04,154 EPOCH 21 done: loss 5.6895 - lr 0.5000000 2021-03-26 08:41:04,899 DEV : loss 6.666974067687988 - score 0.9 2021-03-26 08:41:04,922 BAD EPOCHS (no improvement): 1 2021-03-26 08:41:04,923 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:41:05,879 epoch 22 - iter 2/25 - loss 5.06302881 - samples/sec: 134.18 - lr: 0.500000 2021-03-26 08:41:06,784 epoch 22 - iter 4/25 - loss 4.83719516 - samples/sec: 141.67 - lr: 0.500000 2021-03-26 08:41:07,717 epoch 22 - iter 6/25 - loss 4.85809485 - samples/sec: 137.47 - lr: 0.500000 2021-03-26 08:41:08,704 epoch 22 - iter 8/25 - loss 5.69873339 - samples/sec: 129.93 - lr: 0.500000 2021-03-26 08:41:09,704 epoch 22 - iter 10/25 - loss 5.46689510 - samples/sec: 128.09 - lr: 0.500000 2021-03-26 08:41:10,704 epoch 22 - iter 12/25 - loss 5.50562640 - samples/sec: 128.26 - lr: 0.500000 2021-03-26 08:41:11,666 epoch 22 - iter 14/25 - loss 5.52807069 - samples/sec: 133.21 - lr: 0.500000 2021-03-26 08:41:12,677 epoch 22 - iter 16/25 - loss 5.50435984 - samples/sec: 126.85 - lr: 0.500000 2021-03-26 08:41:13,636 epoch 22 - iter 18/25 - loss 5.54933498 - samples/sec: 133.78 - lr: 0.500000 2021-03-26 08:41:14,618 epoch 22 - iter 20/25 - loss 5.52546136 - samples/sec: 130.45 - lr: 0.500000 2021-03-26 08:41:15,586 epoch 22 - iter 22/25 - loss 5.46390449 - samples/sec: 132.48 - lr: 0.500000 2021-03-26 08:41:16,615 epoch 22 - iter 24/25 - loss 5.45882779 - samples/sec: 124.58 - lr: 0.500000 2021-03-26 08:41:17,010 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:41:17,010 EPOCH 22 done: loss 5.4367 - lr 0.5000000 2021-03-26 08:41:17,793 DEV : loss 6.977891445159912 - score 0.8973 2021-03-26 08:41:17,813 BAD EPOCHS (no improvement): 2 2021-03-26 08:41:17,814 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:41:18,854 epoch 23 - iter 2/25 - loss 4.55232620 - samples/sec: 123.24 - lr: 0.500000 2021-03-26 08:41:19,816 epoch 23 - iter 4/25 - loss 4.89346027 - samples/sec: 133.31 - lr: 0.500000 2021-03-26 08:41:20,769 epoch 23 - iter 6/25 - loss 5.26064793 - samples/sec: 134.43 - lr: 0.500000 2021-03-26 08:41:21,988 epoch 23 - iter 8/25 - loss 5.34301186 - samples/sec: 105.15 - lr: 0.500000 2021-03-26 08:41:23,162 epoch 23 - iter 10/25 - loss 5.08064032 - samples/sec: 109.15 - lr: 0.500000 2021-03-26 08:41:24,175 epoch 23 - iter 12/25 - loss 5.16919645 - samples/sec: 126.65 - lr: 0.500000 2021-03-26 08:41:25,233 epoch 23 - iter 14/25 - loss 5.28559242 - samples/sec: 121.14 - lr: 0.500000 2021-03-26 08:41:26,195 epoch 23 - iter 16/25 - loss 5.19672565 - samples/sec: 133.33 - lr: 0.500000 2021-03-26 08:41:27,107 epoch 23 - iter 18/25 - loss 5.19106085 - samples/sec: 140.47 - lr: 0.500000 2021-03-26 08:41:28,134 epoch 23 - iter 20/25 - loss 5.15162991 - samples/sec: 124.82 - lr: 0.500000 2021-03-26 08:41:29,156 epoch 23 - iter 22/25 - loss 5.25128710 - samples/sec: 125.50 - lr: 0.500000 2021-03-26 08:41:30,026 epoch 23 - iter 24/25 - loss 5.23282271 - samples/sec: 147.53 - lr: 0.500000 2021-03-26 08:41:30,413 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:41:30,413 EPOCH 23 done: loss 5.2229 - lr 0.5000000 2021-03-26 08:41:31,149 DEV : loss 7.015801429748535 - score 0.8919 2021-03-26 08:41:31,174 BAD EPOCHS (no improvement): 3 2021-03-26 08:41:31,175 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:41:32,168 epoch 24 - iter 2/25 - loss 4.33896101 - samples/sec: 129.11 - lr: 0.500000 2021-03-26 08:41:33,190 epoch 24 - iter 4/25 - loss 4.21707565 - samples/sec: 126.02 - lr: 0.500000 2021-03-26 08:41:34,198 epoch 24 - iter 6/25 - loss 4.36630698 - samples/sec: 127.20 - lr: 0.500000 2021-03-26 08:41:35,226 epoch 24 - iter 8/25 - loss 4.65370998 - samples/sec: 125.28 - lr: 0.500000 2021-03-26 08:41:36,198 epoch 24 - iter 10/25 - loss 4.71825922 - samples/sec: 131.79 - lr: 0.500000 2021-03-26 08:41:37,268 epoch 24 - iter 12/25 - loss 4.75398221 - samples/sec: 119.84 - lr: 0.500000 2021-03-26 08:41:38,265 epoch 24 - iter 14/25 - loss 4.70081021 - samples/sec: 128.58 - lr: 0.500000 2021-03-26 08:41:39,267 epoch 24 - iter 16/25 - loss 4.67429534 - samples/sec: 127.91 - lr: 0.500000 2021-03-26 08:41:40,232 epoch 24 - iter 18/25 - loss 4.71875207 - samples/sec: 132.88 - lr: 0.500000 2021-03-26 08:41:41,242 epoch 24 - iter 20/25 - loss 4.69584209 - samples/sec: 126.91 - lr: 0.500000 2021-03-26 08:41:42,166 epoch 24 - iter 22/25 - loss 4.68513228 - samples/sec: 138.89 - lr: 0.500000 2021-03-26 08:41:43,180 epoch 24 - iter 24/25 - loss 4.76182438 - samples/sec: 126.46 - lr: 0.500000 2021-03-26 08:41:43,537 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:41:43,538 EPOCH 24 done: loss 4.7702 - lr 0.5000000 2021-03-26 08:41:44,271 DEV : loss 6.763913154602051 - score 0.8957 2021-03-26 08:41:44,292 BAD EPOCHS (no improvement): 4 2021-03-26 08:41:44,293 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:41:45,271 epoch 25 - iter 2/25 - loss 5.24708247 - samples/sec: 131.03 - lr: 0.250000 2021-03-26 08:41:46,204 epoch 25 - iter 4/25 - loss 4.78944886 - samples/sec: 137.57 - lr: 0.250000 2021-03-26 08:41:47,181 epoch 25 - iter 6/25 - loss 4.57319884 - samples/sec: 131.16 - lr: 0.250000 2021-03-26 08:41:48,176 epoch 25 - iter 8/25 - loss 4.33389711 - samples/sec: 128.94 - lr: 0.250000 2021-03-26 08:41:49,180 epoch 25 - iter 10/25 - loss 4.39515495 - samples/sec: 127.59 - lr: 0.250000 2021-03-26 08:41:50,219 epoch 25 - iter 12/25 - loss 4.48107783 - samples/sec: 123.41 - lr: 0.250000 2021-03-26 08:41:51,291 epoch 25 - iter 14/25 - loss 4.49906680 - samples/sec: 119.64 - lr: 0.250000 2021-03-26 08:41:52,312 epoch 25 - iter 16/25 - loss 4.50435439 - samples/sec: 125.44 - lr: 0.250000 2021-03-26 08:41:53,314 epoch 25 - iter 18/25 - loss 4.42300051 - samples/sec: 127.94 - lr: 0.250000 2021-03-26 08:41:54,340 epoch 25 - iter 20/25 - loss 4.39977831 - samples/sec: 124.96 - lr: 0.250000 2021-03-26 08:41:55,354 epoch 25 - iter 22/25 - loss 4.37190740 - samples/sec: 126.53 - lr: 0.250000 2021-03-26 08:41:56,290 epoch 25 - iter 24/25 - loss 4.33572107 - samples/sec: 136.89 - lr: 0.250000 2021-03-26 08:41:56,643 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:41:56,644 EPOCH 25 done: loss 4.3092 - lr 0.2500000 2021-03-26 08:41:57,396 DEV : loss 6.576738357543945 - score 0.9022 2021-03-26 08:41:57,419 BAD EPOCHS (no improvement): 0 2021-03-26 08:42:06,929 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:42:07,974 epoch 26 - iter 2/25 - loss 4.18935883 - samples/sec: 122.77 - lr: 0.250000 2021-03-26 08:42:09,067 epoch 26 - iter 4/25 - loss 3.63023371 - samples/sec: 117.31 - lr: 0.250000 2021-03-26 08:42:10,081 epoch 26 - iter 6/25 - loss 3.64430161 - samples/sec: 126.52 - lr: 0.250000 2021-03-26 08:42:11,060 epoch 26 - iter 8/25 - loss 3.71407154 - samples/sec: 130.92 - lr: 0.250000 2021-03-26 08:42:12,106 epoch 26 - iter 10/25 - loss 3.79352601 - samples/sec: 122.62 - lr: 0.250000 2021-03-26 08:42:13,032 epoch 26 - iter 12/25 - loss 3.97261920 - samples/sec: 138.35 - lr: 0.250000 2021-03-26 08:42:14,085 epoch 26 - iter 14/25 - loss 4.01314941 - samples/sec: 121.75 - lr: 0.250000 2021-03-26 08:42:15,077 epoch 26 - iter 16/25 - loss 4.07384159 - samples/sec: 129.18 - lr: 0.250000 2021-03-26 08:42:15,993 epoch 26 - iter 18/25 - loss 4.06011808 - samples/sec: 140.02 - lr: 0.250000 2021-03-26 08:42:16,912 epoch 26 - iter 20/25 - loss 4.01558514 - samples/sec: 139.43 - lr: 0.250000 2021-03-26 08:42:18,042 epoch 26 - iter 22/25 - loss 3.97340371 - samples/sec: 113.44 - lr: 0.250000 2021-03-26 08:42:19,058 epoch 26 - iter 24/25 - loss 3.99448071 - samples/sec: 126.18 - lr: 0.250000 2021-03-26 08:42:19,458 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:42:19,459 EPOCH 26 done: loss 3.9627 - lr 0.2500000 2021-03-26 08:42:20,219 DEV : loss 6.506423473358154 - score 0.9044 2021-03-26 08:42:20,248 BAD EPOCHS (no improvement): 0 2021-03-26 08:42:29,762 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:42:30,887 epoch 27 - iter 2/25 - loss 4.31431675 - samples/sec: 114.01 - lr: 0.250000 2021-03-26 08:42:31,931 epoch 27 - iter 4/25 - loss 3.82997966 - samples/sec: 122.86 - lr: 0.250000 2021-03-26 08:42:32,994 epoch 27 - iter 6/25 - loss 3.96800526 - samples/sec: 120.56 - lr: 0.250000 2021-03-26 08:42:33,968 epoch 27 - iter 8/25 - loss 3.94697222 - samples/sec: 131.53 - lr: 0.250000 2021-03-26 08:42:34,913 epoch 27 - iter 10/25 - loss 3.94078667 - samples/sec: 135.60 - lr: 0.250000 2021-03-26 08:42:35,891 epoch 27 - iter 12/25 - loss 3.90649138 - samples/sec: 131.07 - lr: 0.250000 2021-03-26 08:42:36,912 epoch 27 - iter 14/25 - loss 3.93579703 - samples/sec: 125.51 - lr: 0.250000 2021-03-26 08:42:37,992 epoch 27 - iter 16/25 - loss 3.91942632 - samples/sec: 118.64 - lr: 0.250000 2021-03-26 08:42:38,971 epoch 27 - iter 18/25 - loss 3.86642596 - samples/sec: 130.98 - lr: 0.250000 2021-03-26 08:42:39,933 epoch 27 - iter 20/25 - loss 3.94384916 - samples/sec: 133.23 - lr: 0.250000 2021-03-26 08:42:40,980 epoch 27 - iter 22/25 - loss 3.94357267 - samples/sec: 122.54 - lr: 0.250000 2021-03-26 08:42:41,898 epoch 27 - iter 24/25 - loss 3.87114109 - samples/sec: 139.72 - lr: 0.250000 2021-03-26 08:42:42,291 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:42:42,292 EPOCH 27 done: loss 3.8927 - lr 0.2500000 2021-03-26 08:42:43,092 DEV : loss 6.57416296005249 - score 0.9036 2021-03-26 08:42:43,111 BAD EPOCHS (no improvement): 1 2021-03-26 08:42:43,112 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:42:44,137 epoch 28 - iter 2/25 - loss 4.20441771 - samples/sec: 125.10 - lr: 0.250000 2021-03-26 08:42:45,079 epoch 28 - iter 4/25 - loss 4.04117072 - samples/sec: 136.22 - lr: 0.250000 2021-03-26 08:42:46,052 epoch 28 - iter 6/25 - loss 3.90217503 - samples/sec: 131.82 - lr: 0.250000 2021-03-26 08:42:46,957 epoch 28 - iter 8/25 - loss 3.69132367 - samples/sec: 141.73 - lr: 0.250000 2021-03-26 08:42:47,947 epoch 28 - iter 10/25 - loss 3.78331764 - samples/sec: 129.49 - lr: 0.250000 2021-03-26 08:42:49,024 epoch 28 - iter 12/25 - loss 3.77474389 - samples/sec: 119.00 - lr: 0.250000 2021-03-26 08:42:50,025 epoch 28 - iter 14/25 - loss 3.82563947 - samples/sec: 128.03 - lr: 0.250000 2021-03-26 08:42:51,001 epoch 28 - iter 16/25 - loss 3.82414263 - samples/sec: 131.42 - lr: 0.250000 2021-03-26 08:42:51,927 epoch 28 - iter 18/25 - loss 3.78158188 - samples/sec: 138.47 - lr: 0.250000 2021-03-26 08:42:52,916 epoch 28 - iter 20/25 - loss 3.72995106 - samples/sec: 129.54 - lr: 0.250000 2021-03-26 08:42:53,934 epoch 28 - iter 22/25 - loss 3.69838404 - samples/sec: 125.95 - lr: 0.250000 2021-03-26 08:42:54,922 epoch 28 - iter 24/25 - loss 3.72896739 - samples/sec: 129.85 - lr: 0.250000 2021-03-26 08:42:55,362 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:42:55,363 EPOCH 28 done: loss 3.6976 - lr 0.2500000 2021-03-26 08:42:56,121 DEV : loss 6.681036949157715 - score 0.9016 2021-03-26 08:42:56,145 BAD EPOCHS (no improvement): 2 2021-03-26 08:42:56,146 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:42:57,191 epoch 29 - iter 2/25 - loss 3.47235179 - samples/sec: 122.64 - lr: 0.250000 2021-03-26 08:42:58,292 epoch 29 - iter 4/25 - loss 3.73076713 - samples/sec: 116.35 - lr: 0.250000 2021-03-26 08:42:59,334 epoch 29 - iter 6/25 - loss 3.74192850 - samples/sec: 123.00 - lr: 0.250000 2021-03-26 08:43:00,408 epoch 29 - iter 8/25 - loss 3.61097527 - samples/sec: 119.38 - lr: 0.250000 2021-03-26 08:43:01,420 epoch 29 - iter 10/25 - loss 3.63619182 - samples/sec: 126.66 - lr: 0.250000 2021-03-26 08:43:02,308 epoch 29 - iter 12/25 - loss 3.61553208 - samples/sec: 144.40 - lr: 0.250000 2021-03-26 08:43:03,206 epoch 29 - iter 14/25 - loss 3.58807942 - samples/sec: 142.65 - lr: 0.250000 2021-03-26 08:43:04,131 epoch 29 - iter 16/25 - loss 3.63437870 - samples/sec: 138.64 - lr: 0.250000 2021-03-26 08:43:05,215 epoch 29 - iter 18/25 - loss 3.66702800 - samples/sec: 118.30 - lr: 0.250000 2021-03-26 08:43:06,271 epoch 29 - iter 20/25 - loss 3.67497628 - samples/sec: 121.48 - lr: 0.250000 2021-03-26 08:43:07,225 epoch 29 - iter 22/25 - loss 3.65543382 - samples/sec: 134.30 - lr: 0.250000 2021-03-26 08:43:08,153 epoch 29 - iter 24/25 - loss 3.64861295 - samples/sec: 138.21 - lr: 0.250000 2021-03-26 08:43:08,580 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:43:08,581 EPOCH 29 done: loss 3.6659 - lr 0.2500000 2021-03-26 08:43:09,315 DEV : loss 6.766953945159912 - score 0.9048 2021-03-26 08:43:09,353 BAD EPOCHS (no improvement): 0 2021-03-26 08:43:18,858 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:43:19,893 epoch 30 - iter 2/25 - loss 3.69414401 - samples/sec: 123.98 - lr: 0.250000 2021-03-26 08:43:20,827 epoch 30 - iter 4/25 - loss 3.38807523 - samples/sec: 137.21 - lr: 0.250000 2021-03-26 08:43:21,764 epoch 30 - iter 6/25 - loss 3.66507101 - samples/sec: 136.75 - lr: 0.250000 2021-03-26 08:43:22,826 epoch 30 - iter 8/25 - loss 3.57221785 - samples/sec: 120.83 - lr: 0.250000 2021-03-26 08:43:23,799 epoch 30 - iter 10/25 - loss 3.56492391 - samples/sec: 131.64 - lr: 0.250000 2021-03-26 08:43:24,769 epoch 30 - iter 12/25 - loss 3.63968432 - samples/sec: 132.21 - lr: 0.250000 2021-03-26 08:43:25,831 epoch 30 - iter 14/25 - loss 3.68632470 - samples/sec: 120.75 - lr: 0.250000 2021-03-26 08:43:26,878 epoch 30 - iter 16/25 - loss 3.70914277 - samples/sec: 122.54 - lr: 0.250000 2021-03-26 08:43:27,792 epoch 30 - iter 18/25 - loss 3.67375978 - samples/sec: 140.37 - lr: 0.250000 2021-03-26 08:43:28,862 epoch 30 - iter 20/25 - loss 3.65931660 - samples/sec: 119.81 - lr: 0.250000 2021-03-26 08:43:29,812 epoch 30 - iter 22/25 - loss 3.63659420 - samples/sec: 134.81 - lr: 0.250000 2021-03-26 08:43:30,741 epoch 30 - iter 24/25 - loss 3.66693341 - samples/sec: 138.06 - lr: 0.250000 2021-03-26 08:43:31,134 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:43:31,135 EPOCH 30 done: loss 3.6635 - lr 0.2500000 2021-03-26 08:43:31,887 DEV : loss 6.755105495452881 - score 0.9054 2021-03-26 08:43:31,904 BAD EPOCHS (no improvement): 0 2021-03-26 08:43:41,430 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:43:42,332 epoch 31 - iter 2/25 - loss 3.49644756 - samples/sec: 142.28 - lr: 0.250000 2021-03-26 08:43:43,336 epoch 31 - iter 4/25 - loss 3.28100044 - samples/sec: 127.58 - lr: 0.250000 2021-03-26 08:43:44,466 epoch 31 - iter 6/25 - loss 3.24940030 - samples/sec: 113.39 - lr: 0.250000 2021-03-26 08:43:45,417 epoch 31 - iter 8/25 - loss 3.31971303 - samples/sec: 134.89 - lr: 0.250000 2021-03-26 08:43:46,462 epoch 31 - iter 10/25 - loss 3.30865183 - samples/sec: 122.75 - lr: 0.250000 2021-03-26 08:43:47,572 epoch 31 - iter 12/25 - loss 3.39676917 - samples/sec: 115.50 - lr: 0.250000 2021-03-26 08:43:48,662 epoch 31 - iter 14/25 - loss 3.42026200 - samples/sec: 117.61 - lr: 0.250000 2021-03-26 08:43:49,642 epoch 31 - iter 16/25 - loss 3.39093278 - samples/sec: 130.67 - lr: 0.250000 2021-03-26 08:43:50,576 epoch 31 - iter 18/25 - loss 3.48906553 - samples/sec: 137.32 - lr: 0.250000 2021-03-26 08:43:51,526 epoch 31 - iter 20/25 - loss 3.49162394 - samples/sec: 135.07 - lr: 0.250000 2021-03-26 08:43:52,472 epoch 31 - iter 22/25 - loss 3.50332228 - samples/sec: 135.46 - lr: 0.250000 2021-03-26 08:43:53,412 epoch 31 - iter 24/25 - loss 3.52942649 - samples/sec: 136.50 - lr: 0.250000 2021-03-26 08:43:53,818 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:43:53,819 EPOCH 31 done: loss 3.5330 - lr 0.2500000 2021-03-26 08:43:54,564 DEV : loss 6.835216522216797 - score 0.903 2021-03-26 08:43:54,589 BAD EPOCHS (no improvement): 1 2021-03-26 08:43:54,590 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:43:55,572 epoch 32 - iter 2/25 - loss 3.68399692 - samples/sec: 130.45 - lr: 0.250000 2021-03-26 08:43:56,593 epoch 32 - iter 4/25 - loss 3.42861271 - samples/sec: 125.63 - lr: 0.250000 2021-03-26 08:43:57,470 epoch 32 - iter 6/25 - loss 3.22910500 - samples/sec: 146.11 - lr: 0.250000 2021-03-26 08:43:58,473 epoch 32 - iter 8/25 - loss 3.16920099 - samples/sec: 127.81 - lr: 0.250000 2021-03-26 08:43:59,454 epoch 32 - iter 10/25 - loss 3.19274833 - samples/sec: 130.76 - lr: 0.250000 2021-03-26 08:44:00,556 epoch 32 - iter 12/25 - loss 3.26887606 - samples/sec: 116.25 - lr: 0.250000 2021-03-26 08:44:01,556 epoch 32 - iter 14/25 - loss 3.29435839 - samples/sec: 128.21 - lr: 0.250000 2021-03-26 08:44:02,485 epoch 32 - iter 16/25 - loss 3.30416803 - samples/sec: 138.11 - lr: 0.250000 2021-03-26 08:44:03,422 epoch 32 - iter 18/25 - loss 3.31257772 - samples/sec: 136.80 - lr: 0.250000 2021-03-26 08:44:04,377 epoch 32 - iter 20/25 - loss 3.34610075 - samples/sec: 134.33 - lr: 0.250000 2021-03-26 08:44:05,421 epoch 32 - iter 22/25 - loss 3.36202605 - samples/sec: 122.87 - lr: 0.250000 2021-03-26 08:44:06,448 epoch 32 - iter 24/25 - loss 3.36099755 - samples/sec: 124.87 - lr: 0.250000 2021-03-26 08:44:06,875 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:44:06,876 EPOCH 32 done: loss 3.3529 - lr 0.2500000 2021-03-26 08:44:07,626 DEV : loss 6.639145851135254 - score 0.9066 2021-03-26 08:44:07,650 BAD EPOCHS (no improvement): 0 2021-03-26 08:44:16,987 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:44:17,947 epoch 33 - iter 2/25 - loss 3.20583904 - samples/sec: 133.66 - lr: 0.250000 2021-03-26 08:44:19,016 epoch 33 - iter 4/25 - loss 3.36605328 - samples/sec: 119.94 - lr: 0.250000 2021-03-26 08:44:19,959 epoch 33 - iter 6/25 - loss 3.21472077 - samples/sec: 136.06 - lr: 0.250000 2021-03-26 08:44:20,907 epoch 33 - iter 8/25 - loss 3.14647344 - samples/sec: 135.18 - lr: 0.250000 2021-03-26 08:44:22,049 epoch 33 - iter 10/25 - loss 3.24315312 - samples/sec: 112.26 - lr: 0.250000 2021-03-26 08:44:23,046 epoch 33 - iter 12/25 - loss 3.17480514 - samples/sec: 128.67 - lr: 0.250000 2021-03-26 08:44:24,183 epoch 33 - iter 14/25 - loss 3.30357449 - samples/sec: 112.76 - lr: 0.250000 2021-03-26 08:44:25,182 epoch 33 - iter 16/25 - loss 3.28795835 - samples/sec: 128.35 - lr: 0.250000 2021-03-26 08:44:26,164 epoch 33 - iter 18/25 - loss 3.24029554 - samples/sec: 130.45 - lr: 0.250000 2021-03-26 08:44:27,271 epoch 33 - iter 20/25 - loss 3.30793083 - samples/sec: 115.84 - lr: 0.250000 2021-03-26 08:44:28,326 epoch 33 - iter 22/25 - loss 3.33434795 - samples/sec: 121.51 - lr: 0.250000 2021-03-26 08:44:29,317 epoch 33 - iter 24/25 - loss 3.32521027 - samples/sec: 129.35 - lr: 0.250000 2021-03-26 08:44:29,715 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:44:29,716 EPOCH 33 done: loss 3.3160 - lr 0.2500000 2021-03-26 08:44:30,494 DEV : loss 6.662256240844727 - score 0.9074 2021-03-26 08:44:30,514 BAD EPOCHS (no improvement): 0 2021-03-26 08:44:40,210 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:44:41,391 epoch 34 - iter 2/25 - loss 3.49009538 - samples/sec: 108.61 - lr: 0.250000 2021-03-26 08:44:42,497 epoch 34 - iter 4/25 - loss 3.26404929 - samples/sec: 115.79 - lr: 0.250000 2021-03-26 08:44:43,500 epoch 34 - iter 6/25 - loss 3.10120742 - samples/sec: 127.74 - lr: 0.250000 2021-03-26 08:44:44,518 epoch 34 - iter 8/25 - loss 3.02353513 - samples/sec: 126.04 - lr: 0.250000 2021-03-26 08:44:45,624 epoch 34 - iter 10/25 - loss 3.07420588 - samples/sec: 115.91 - lr: 0.250000 2021-03-26 08:44:46,712 epoch 34 - iter 12/25 - loss 3.15896785 - samples/sec: 117.81 - lr: 0.250000 2021-03-26 08:44:47,674 epoch 34 - iter 14/25 - loss 3.18685291 - samples/sec: 133.24 - lr: 0.250000 2021-03-26 08:44:48,636 epoch 34 - iter 16/25 - loss 3.23354343 - samples/sec: 133.16 - lr: 0.250000 2021-03-26 08:44:49,626 epoch 34 - iter 18/25 - loss 3.27925718 - samples/sec: 129.59 - lr: 0.250000 2021-03-26 08:44:50,736 epoch 34 - iter 20/25 - loss 3.25261000 - samples/sec: 115.49 - lr: 0.250000 2021-03-26 08:44:51,782 epoch 34 - iter 22/25 - loss 3.26143661 - samples/sec: 122.53 - lr: 0.250000 2021-03-26 08:44:52,785 epoch 34 - iter 24/25 - loss 3.26214166 - samples/sec: 127.80 - lr: 0.250000 2021-03-26 08:44:53,190 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:44:53,191 EPOCH 34 done: loss 3.2520 - lr 0.2500000 2021-03-26 08:44:53,926 DEV : loss 6.694727897644043 - score 0.9102 2021-03-26 08:44:53,950 BAD EPOCHS (no improvement): 0 2021-03-26 08:45:03,526 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:45:04,592 epoch 35 - iter 2/25 - loss 3.33741724 - samples/sec: 120.37 - lr: 0.250000 2021-03-26 08:45:05,596 epoch 35 - iter 4/25 - loss 3.01593387 - samples/sec: 127.60 - lr: 0.250000 2021-03-26 08:45:06,650 epoch 35 - iter 6/25 - loss 3.08290581 - samples/sec: 121.55 - lr: 0.250000 2021-03-26 08:45:07,707 epoch 35 - iter 8/25 - loss 3.21430799 - samples/sec: 121.29 - lr: 0.250000 2021-03-26 08:45:08,768 epoch 35 - iter 10/25 - loss 3.20909221 - samples/sec: 120.87 - lr: 0.250000 2021-03-26 08:45:09,839 epoch 35 - iter 12/25 - loss 3.20400945 - samples/sec: 119.64 - lr: 0.250000 2021-03-26 08:45:10,877 epoch 35 - iter 14/25 - loss 3.25257616 - samples/sec: 123.50 - lr: 0.250000 2021-03-26 08:45:11,873 epoch 35 - iter 16/25 - loss 3.26921830 - samples/sec: 128.72 - lr: 0.250000 2021-03-26 08:45:12,860 epoch 35 - iter 18/25 - loss 3.23871719 - samples/sec: 129.93 - lr: 0.250000 2021-03-26 08:45:13,846 epoch 35 - iter 20/25 - loss 3.22748883 - samples/sec: 130.03 - lr: 0.250000 2021-03-26 08:45:14,923 epoch 35 - iter 22/25 - loss 3.21357602 - samples/sec: 119.01 - lr: 0.250000 2021-03-26 08:45:15,865 epoch 35 - iter 24/25 - loss 3.24208915 - samples/sec: 136.13 - lr: 0.250000 2021-03-26 08:45:16,274 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:45:16,275 EPOCH 35 done: loss 3.2753 - lr 0.2500000 2021-03-26 08:45:17,067 DEV : loss 6.7822699546813965 - score 0.9058 2021-03-26 08:45:17,092 BAD EPOCHS (no improvement): 1 2021-03-26 08:45:17,092 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:45:18,105 epoch 36 - iter 2/25 - loss 3.46526396 - samples/sec: 126.58 - lr: 0.250000 2021-03-26 08:45:19,095 epoch 36 - iter 4/25 - loss 3.33731157 - samples/sec: 129.49 - lr: 0.250000 2021-03-26 08:45:20,066 epoch 36 - iter 6/25 - loss 3.45556358 - samples/sec: 131.92 - lr: 0.250000 2021-03-26 08:45:21,042 epoch 36 - iter 8/25 - loss 3.26688248 - samples/sec: 131.39 - lr: 0.250000 2021-03-26 08:45:21,958 epoch 36 - iter 10/25 - loss 3.25652921 - samples/sec: 139.96 - lr: 0.250000 2021-03-26 08:45:22,991 epoch 36 - iter 12/25 - loss 3.21865408 - samples/sec: 124.14 - lr: 0.250000 2021-03-26 08:45:23,951 epoch 36 - iter 14/25 - loss 3.16231186 - samples/sec: 133.34 - lr: 0.250000 2021-03-26 08:45:24,997 epoch 36 - iter 16/25 - loss 3.22655720 - samples/sec: 122.71 - lr: 0.250000 2021-03-26 08:45:25,978 epoch 36 - iter 18/25 - loss 3.20245018 - samples/sec: 130.57 - lr: 0.250000 2021-03-26 08:45:26,943 epoch 36 - iter 20/25 - loss 3.18311380 - samples/sec: 132.80 - lr: 0.250000 2021-03-26 08:45:27,963 epoch 36 - iter 22/25 - loss 3.12007433 - samples/sec: 125.67 - lr: 0.250000 2021-03-26 08:45:28,930 epoch 36 - iter 24/25 - loss 3.11229009 - samples/sec: 132.60 - lr: 0.250000 2021-03-26 08:45:29,331 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:45:29,332 EPOCH 36 done: loss 3.1368 - lr 0.2500000 2021-03-26 08:45:30,069 DEV : loss 6.718545436859131 - score 0.9066 2021-03-26 08:45:30,092 BAD EPOCHS (no improvement): 2 2021-03-26 08:45:30,092 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:45:31,019 epoch 37 - iter 2/25 - loss 2.93264663 - samples/sec: 138.34 - lr: 0.250000 2021-03-26 08:45:32,020 epoch 37 - iter 4/25 - loss 2.95200199 - samples/sec: 127.98 - lr: 0.250000 2021-03-26 08:45:33,034 epoch 37 - iter 6/25 - loss 3.01997805 - samples/sec: 127.13 - lr: 0.250000 2021-03-26 08:45:34,028 epoch 37 - iter 8/25 - loss 3.02635363 - samples/sec: 128.90 - lr: 0.250000 2021-03-26 08:45:35,004 epoch 37 - iter 10/25 - loss 3.00781195 - samples/sec: 131.34 - lr: 0.250000 2021-03-26 08:45:36,065 epoch 37 - iter 12/25 - loss 2.94086071 - samples/sec: 120.86 - lr: 0.250000 2021-03-26 08:45:37,011 epoch 37 - iter 14/25 - loss 2.98774382 - samples/sec: 135.50 - lr: 0.250000 2021-03-26 08:45:38,064 epoch 37 - iter 16/25 - loss 3.00074844 - samples/sec: 121.68 - lr: 0.250000 2021-03-26 08:45:39,024 epoch 37 - iter 18/25 - loss 2.99614977 - samples/sec: 133.64 - lr: 0.250000 2021-03-26 08:45:40,053 epoch 37 - iter 20/25 - loss 3.02320306 - samples/sec: 124.50 - lr: 0.250000 2021-03-26 08:45:41,101 epoch 37 - iter 22/25 - loss 3.06846061 - samples/sec: 122.33 - lr: 0.250000 2021-03-26 08:45:42,079 epoch 37 - iter 24/25 - loss 3.08361191 - samples/sec: 131.12 - lr: 0.250000 2021-03-26 08:45:42,504 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:45:42,505 EPOCH 37 done: loss 3.0804 - lr 0.2500000 2021-03-26 08:45:43,272 DEV : loss 6.583403587341309 - score 0.9052 2021-03-26 08:45:43,293 BAD EPOCHS (no improvement): 3 2021-03-26 08:45:43,293 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:45:44,243 epoch 38 - iter 2/25 - loss 3.14396656 - samples/sec: 134.92 - lr: 0.250000 2021-03-26 08:45:45,198 epoch 38 - iter 4/25 - loss 2.75033486 - samples/sec: 134.22 - lr: 0.250000 2021-03-26 08:45:46,123 epoch 38 - iter 6/25 - loss 2.67477520 - samples/sec: 138.80 - lr: 0.250000 2021-03-26 08:45:47,131 epoch 38 - iter 8/25 - loss 2.75512740 - samples/sec: 127.13 - lr: 0.250000 2021-03-26 08:45:48,154 epoch 38 - iter 10/25 - loss 2.81929321 - samples/sec: 125.38 - lr: 0.250000 2021-03-26 08:45:49,108 epoch 38 - iter 12/25 - loss 2.80150716 - samples/sec: 134.46 - lr: 0.250000 2021-03-26 08:45:50,072 epoch 38 - iter 14/25 - loss 2.77966118 - samples/sec: 132.95 - lr: 0.250000 2021-03-26 08:45:51,035 epoch 38 - iter 16/25 - loss 2.80229428 - samples/sec: 133.17 - lr: 0.250000 2021-03-26 08:45:52,024 epoch 38 - iter 18/25 - loss 2.84507292 - samples/sec: 129.76 - lr: 0.250000 2021-03-26 08:45:53,159 epoch 38 - iter 20/25 - loss 2.81335837 - samples/sec: 112.93 - lr: 0.250000 2021-03-26 08:45:54,243 epoch 38 - iter 22/25 - loss 2.86274156 - samples/sec: 118.16 - lr: 0.250000 2021-03-26 08:45:55,333 epoch 38 - iter 24/25 - loss 2.85997577 - samples/sec: 117.71 - lr: 0.250000 2021-03-26 08:45:55,806 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:45:55,807 EPOCH 38 done: loss 2.8757 - lr 0.2500000 2021-03-26 08:45:56,583 DEV : loss 6.802188396453857 - score 0.9088 2021-03-26 08:45:56,599 BAD EPOCHS (no improvement): 4 2021-03-26 08:45:56,600 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:45:57,722 epoch 39 - iter 2/25 - loss 2.83308125 - samples/sec: 114.33 - lr: 0.125000 2021-03-26 08:45:58,820 epoch 39 - iter 4/25 - loss 2.94828141 - samples/sec: 116.81 - lr: 0.125000 2021-03-26 08:45:59,832 epoch 39 - iter 6/25 - loss 2.99452492 - samples/sec: 126.68 - lr: 0.125000 2021-03-26 08:46:00,874 epoch 39 - iter 8/25 - loss 2.94919509 - samples/sec: 122.90 - lr: 0.125000 2021-03-26 08:46:01,829 epoch 39 - iter 10/25 - loss 2.93447261 - samples/sec: 134.33 - lr: 0.125000 2021-03-26 08:46:02,818 epoch 39 - iter 12/25 - loss 2.97544058 - samples/sec: 129.57 - lr: 0.125000 2021-03-26 08:46:03,779 epoch 39 - iter 14/25 - loss 2.88676938 - samples/sec: 133.40 - lr: 0.125000 2021-03-26 08:46:04,733 epoch 39 - iter 16/25 - loss 2.84791669 - samples/sec: 134.48 - lr: 0.125000 2021-03-26 08:46:05,713 epoch 39 - iter 18/25 - loss 2.85739271 - samples/sec: 130.81 - lr: 0.125000 2021-03-26 08:46:06,696 epoch 39 - iter 20/25 - loss 2.85741124 - samples/sec: 130.42 - lr: 0.125000 2021-03-26 08:46:07,690 epoch 39 - iter 22/25 - loss 2.86451571 - samples/sec: 128.90 - lr: 0.125000 2021-03-26 08:46:08,722 epoch 39 - iter 24/25 - loss 2.87007837 - samples/sec: 124.15 - lr: 0.125000 2021-03-26 08:46:09,126 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:46:09,127 EPOCH 39 done: loss 2.8566 - lr 0.1250000 2021-03-26 08:46:09,880 DEV : loss 6.693580627441406 - score 0.9088 2021-03-26 08:46:09,897 BAD EPOCHS (no improvement): 1 2021-03-26 08:46:09,898 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:46:10,909 epoch 40 - iter 2/25 - loss 2.41403103 - samples/sec: 126.79 - lr: 0.125000 2021-03-26 08:46:11,820 epoch 40 - iter 4/25 - loss 2.40011775 - samples/sec: 140.76 - lr: 0.125000 2021-03-26 08:46:12,888 epoch 40 - iter 6/25 - loss 2.49695289 - samples/sec: 120.01 - lr: 0.125000 2021-03-26 08:46:13,834 epoch 40 - iter 8/25 - loss 2.58933467 - samples/sec: 135.58 - lr: 0.125000 2021-03-26 08:46:14,709 epoch 40 - iter 10/25 - loss 2.58164473 - samples/sec: 146.48 - lr: 0.125000 2021-03-26 08:46:15,700 epoch 40 - iter 12/25 - loss 2.68910889 - samples/sec: 129.30 - lr: 0.125000 2021-03-26 08:46:16,653 epoch 40 - iter 14/25 - loss 2.66335935 - samples/sec: 134.51 - lr: 0.125000 2021-03-26 08:46:17,672 epoch 40 - iter 16/25 - loss 2.73592710 - samples/sec: 125.95 - lr: 0.125000 2021-03-26 08:46:18,705 epoch 40 - iter 18/25 - loss 2.75818470 - samples/sec: 124.09 - lr: 0.125000 2021-03-26 08:46:19,674 epoch 40 - iter 20/25 - loss 2.74944264 - samples/sec: 132.24 - lr: 0.125000 2021-03-26 08:46:20,646 epoch 40 - iter 22/25 - loss 2.75382914 - samples/sec: 131.99 - lr: 0.125000 2021-03-26 08:46:21,770 epoch 40 - iter 24/25 - loss 2.74564613 - samples/sec: 114.09 - lr: 0.125000 2021-03-26 08:46:22,241 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:46:22,242 EPOCH 40 done: loss 2.7428 - lr 0.1250000 2021-03-26 08:46:23,001 DEV : loss 6.799490928649902 - score 0.9076 2021-03-26 08:46:23,028 BAD EPOCHS (no improvement): 2 2021-03-26 08:46:23,029 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:46:23,908 epoch 41 - iter 2/25 - loss 2.25075608 - samples/sec: 146.07 - lr: 0.125000 2021-03-26 08:46:24,887 epoch 41 - iter 4/25 - loss 2.45685551 - samples/sec: 130.86 - lr: 0.125000 2021-03-26 08:46:25,810 epoch 41 - iter 6/25 - loss 2.61635317 - samples/sec: 139.11 - lr: 0.125000 2021-03-26 08:46:26,765 epoch 41 - iter 8/25 - loss 2.69645248 - samples/sec: 134.10 - lr: 0.125000 2021-03-26 08:46:27,914 epoch 41 - iter 10/25 - loss 2.67517320 - samples/sec: 111.59 - lr: 0.125000 2021-03-26 08:46:28,939 epoch 41 - iter 12/25 - loss 2.65610324 - samples/sec: 125.05 - lr: 0.125000 2021-03-26 08:46:30,026 epoch 41 - iter 14/25 - loss 2.65179953 - samples/sec: 117.92 - lr: 0.125000 2021-03-26 08:46:30,989 epoch 41 - iter 16/25 - loss 2.65190751 - samples/sec: 133.10 - lr: 0.125000 2021-03-26 08:46:31,896 epoch 41 - iter 18/25 - loss 2.66618386 - samples/sec: 141.29 - lr: 0.125000 2021-03-26 08:46:32,914 epoch 41 - iter 20/25 - loss 2.61516859 - samples/sec: 125.90 - lr: 0.125000 2021-03-26 08:46:33,812 epoch 41 - iter 22/25 - loss 2.61662399 - samples/sec: 142.79 - lr: 0.125000 2021-03-26 08:46:34,740 epoch 41 - iter 24/25 - loss 2.66955232 - samples/sec: 138.23 - lr: 0.125000 2021-03-26 08:46:35,170 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:46:35,171 EPOCH 41 done: loss 2.6905 - lr 0.1250000 2021-03-26 08:46:35,915 DEV : loss 6.693910121917725 - score 0.9054 2021-03-26 08:46:35,940 BAD EPOCHS (no improvement): 3 2021-03-26 08:46:35,940 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:46:36,906 epoch 42 - iter 2/25 - loss 2.75737107 - samples/sec: 132.81 - lr: 0.125000 2021-03-26 08:46:37,831 epoch 42 - iter 4/25 - loss 2.93559647 - samples/sec: 138.60 - lr: 0.125000 2021-03-26 08:46:38,895 epoch 42 - iter 6/25 - loss 2.81965216 - samples/sec: 120.46 - lr: 0.125000 2021-03-26 08:46:39,871 epoch 42 - iter 8/25 - loss 2.80153769 - samples/sec: 131.31 - lr: 0.125000 2021-03-26 08:46:40,879 epoch 42 - iter 10/25 - loss 2.78234267 - samples/sec: 127.23 - lr: 0.125000 2021-03-26 08:46:41,917 epoch 42 - iter 12/25 - loss 2.72432474 - samples/sec: 123.45 - lr: 0.125000 2021-03-26 08:46:42,898 epoch 42 - iter 14/25 - loss 2.62786084 - samples/sec: 130.71 - lr: 0.125000 2021-03-26 08:46:43,995 epoch 42 - iter 16/25 - loss 2.63186632 - samples/sec: 116.86 - lr: 0.125000 2021-03-26 08:46:44,915 epoch 42 - iter 18/25 - loss 2.65961860 - samples/sec: 139.22 - lr: 0.125000 2021-03-26 08:46:45,855 epoch 42 - iter 20/25 - loss 2.66174264 - samples/sec: 136.53 - lr: 0.125000 2021-03-26 08:46:46,866 epoch 42 - iter 22/25 - loss 2.61873937 - samples/sec: 126.80 - lr: 0.125000 2021-03-26 08:46:47,855 epoch 42 - iter 24/25 - loss 2.62547451 - samples/sec: 129.64 - lr: 0.125000 2021-03-26 08:46:48,363 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:46:48,364 EPOCH 42 done: loss 2.6608 - lr 0.1250000 2021-03-26 08:46:49,132 DEV : loss 6.850523948669434 - score 0.91 2021-03-26 08:46:49,165 BAD EPOCHS (no improvement): 4 2021-03-26 08:46:49,166 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:46:50,220 epoch 43 - iter 2/25 - loss 2.49043608 - samples/sec: 121.62 - lr: 0.062500 2021-03-26 08:46:51,260 epoch 43 - iter 4/25 - loss 2.86309117 - samples/sec: 123.32 - lr: 0.062500 2021-03-26 08:46:52,181 epoch 43 - iter 6/25 - loss 2.72066963 - samples/sec: 139.08 - lr: 0.062500 2021-03-26 08:46:53,208 epoch 43 - iter 8/25 - loss 2.77377698 - samples/sec: 124.80 - lr: 0.062500 2021-03-26 08:46:54,182 epoch 43 - iter 10/25 - loss 2.66388309 - samples/sec: 131.72 - lr: 0.062500 2021-03-26 08:46:55,219 epoch 43 - iter 12/25 - loss 2.73270496 - samples/sec: 123.65 - lr: 0.062500 2021-03-26 08:46:56,229 epoch 43 - iter 14/25 - loss 2.76990683 - samples/sec: 127.04 - lr: 0.062500 2021-03-26 08:46:57,268 epoch 43 - iter 16/25 - loss 2.69852231 - samples/sec: 123.35 - lr: 0.062500 2021-03-26 08:46:58,290 epoch 43 - iter 18/25 - loss 2.64845743 - samples/sec: 125.34 - lr: 0.062500 2021-03-26 08:46:59,225 epoch 43 - iter 20/25 - loss 2.66699877 - samples/sec: 137.08 - lr: 0.062500 2021-03-26 08:47:00,182 epoch 43 - iter 22/25 - loss 2.68268208 - samples/sec: 134.04 - lr: 0.062500 2021-03-26 08:47:01,230 epoch 43 - iter 24/25 - loss 2.70284759 - samples/sec: 122.36 - lr: 0.062500 2021-03-26 08:47:01,612 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:47:01,613 EPOCH 43 done: loss 2.7074 - lr 0.0625000 2021-03-26 08:47:02,369 DEV : loss 6.793069362640381 - score 0.9094 2021-03-26 08:47:02,393 BAD EPOCHS (no improvement): 1 2021-03-26 08:47:02,393 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:47:03,265 epoch 44 - iter 2/25 - loss 2.39724141 - samples/sec: 147.13 - lr: 0.062500 2021-03-26 08:47:04,189 epoch 44 - iter 4/25 - loss 2.43123588 - samples/sec: 138.76 - lr: 0.062500 2021-03-26 08:47:05,251 epoch 44 - iter 6/25 - loss 2.56454744 - samples/sec: 120.71 - lr: 0.062500 2021-03-26 08:47:06,418 epoch 44 - iter 8/25 - loss 2.74957548 - samples/sec: 109.77 - lr: 0.062500 2021-03-26 08:47:07,411 epoch 44 - iter 10/25 - loss 2.68237354 - samples/sec: 129.16 - lr: 0.062500 2021-03-26 08:47:08,465 epoch 44 - iter 12/25 - loss 2.71276640 - samples/sec: 121.54 - lr: 0.062500 2021-03-26 08:47:09,413 epoch 44 - iter 14/25 - loss 2.66092078 - samples/sec: 135.29 - lr: 0.062500 2021-03-26 08:47:10,426 epoch 44 - iter 16/25 - loss 2.64549053 - samples/sec: 126.61 - lr: 0.062500 2021-03-26 08:47:11,349 epoch 44 - iter 18/25 - loss 2.62857394 - samples/sec: 138.93 - lr: 0.062500 2021-03-26 08:47:12,361 epoch 44 - iter 20/25 - loss 2.63828043 - samples/sec: 126.59 - lr: 0.062500 2021-03-26 08:47:13,360 epoch 44 - iter 22/25 - loss 2.67504076 - samples/sec: 128.40 - lr: 0.062500 2021-03-26 08:47:14,421 epoch 44 - iter 24/25 - loss 2.75515004 - samples/sec: 120.75 - lr: 0.062500 2021-03-26 08:47:14,809 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:47:14,809 EPOCH 44 done: loss 2.7510 - lr 0.0625000 2021-03-26 08:47:15,586 DEV : loss 6.782561302185059 - score 0.9082 2021-03-26 08:47:15,602 BAD EPOCHS (no improvement): 2 2021-03-26 08:47:15,603 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:47:16,633 epoch 45 - iter 2/25 - loss 2.96598649 - samples/sec: 124.51 - lr: 0.062500 2021-03-26 08:47:17,596 epoch 45 - iter 4/25 - loss 2.79108897 - samples/sec: 133.20 - lr: 0.062500 2021-03-26 08:47:18,627 epoch 45 - iter 6/25 - loss 2.74379843 - samples/sec: 124.28 - lr: 0.062500 2021-03-26 08:47:19,648 epoch 45 - iter 8/25 - loss 2.79540862 - samples/sec: 125.51 - lr: 0.062500 2021-03-26 08:47:20,662 epoch 45 - iter 10/25 - loss 2.71001374 - samples/sec: 126.40 - lr: 0.062500 2021-03-26 08:47:21,649 epoch 45 - iter 12/25 - loss 2.65032153 - samples/sec: 129.83 - lr: 0.062500 2021-03-26 08:47:22,676 epoch 45 - iter 14/25 - loss 2.61791879 - samples/sec: 124.91 - lr: 0.062500 2021-03-26 08:47:23,624 epoch 45 - iter 16/25 - loss 2.56420042 - samples/sec: 135.43 - lr: 0.062500 2021-03-26 08:47:24,634 epoch 45 - iter 18/25 - loss 2.52401664 - samples/sec: 126.91 - lr: 0.062500 2021-03-26 08:47:25,664 epoch 45 - iter 20/25 - loss 2.52811976 - samples/sec: 124.42 - lr: 0.062500 2021-03-26 08:47:26,637 epoch 45 - iter 22/25 - loss 2.54474491 - samples/sec: 131.81 - lr: 0.062500 2021-03-26 08:47:27,589 epoch 45 - iter 24/25 - loss 2.54330914 - samples/sec: 134.69 - lr: 0.062500 2021-03-26 08:47:28,030 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:47:28,031 EPOCH 45 done: loss 2.5855 - lr 0.0625000 2021-03-26 08:47:28,791 DEV : loss 6.721575736999512 - score 0.9072 2021-03-26 08:47:28,809 BAD EPOCHS (no improvement): 3 2021-03-26 08:47:28,810 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:47:29,702 epoch 46 - iter 2/25 - loss 2.40093923 - samples/sec: 143.82 - lr: 0.062500 2021-03-26 08:47:30,714 epoch 46 - iter 4/25 - loss 2.47187972 - samples/sec: 126.63 - lr: 0.062500 2021-03-26 08:47:31,763 epoch 46 - iter 6/25 - loss 2.49867698 - samples/sec: 122.12 - lr: 0.062500 2021-03-26 08:47:32,865 epoch 46 - iter 8/25 - loss 2.45476428 - samples/sec: 116.42 - lr: 0.062500 2021-03-26 08:47:33,891 epoch 46 - iter 10/25 - loss 2.33877997 - samples/sec: 124.97 - lr: 0.062500 2021-03-26 08:47:34,873 epoch 46 - iter 12/25 - loss 2.40645109 - samples/sec: 130.41 - lr: 0.062500 2021-03-26 08:47:35,832 epoch 46 - iter 14/25 - loss 2.41505643 - samples/sec: 133.77 - lr: 0.062500 2021-03-26 08:47:36,813 epoch 46 - iter 16/25 - loss 2.48525703 - samples/sec: 130.68 - lr: 0.062500 2021-03-26 08:47:37,962 epoch 46 - iter 18/25 - loss 2.48812330 - samples/sec: 111.48 - lr: 0.062500 2021-03-26 08:47:38,938 epoch 46 - iter 20/25 - loss 2.46523433 - samples/sec: 131.45 - lr: 0.062500 2021-03-26 08:47:39,942 epoch 46 - iter 22/25 - loss 2.48067673 - samples/sec: 127.67 - lr: 0.062500 2021-03-26 08:47:41,269 epoch 46 - iter 24/25 - loss 2.50672932 - samples/sec: 96.60 - lr: 0.062500 2021-03-26 08:47:41,713 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:47:41,714 EPOCH 46 done: loss 2.5113 - lr 0.0625000 2021-03-26 08:47:42,595 DEV : loss 6.7464094161987305 - score 0.9072 2021-03-26 08:47:42,627 BAD EPOCHS (no improvement): 4 2021-03-26 08:47:42,628 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:47:43,630 epoch 47 - iter 2/25 - loss 2.07991660 - samples/sec: 128.09 - lr: 0.031250 2021-03-26 08:47:44,593 epoch 47 - iter 4/25 - loss 2.25694829 - samples/sec: 133.15 - lr: 0.031250 2021-03-26 08:47:45,688 epoch 47 - iter 6/25 - loss 2.24701436 - samples/sec: 116.90 - lr: 0.031250 2021-03-26 08:47:46,771 epoch 47 - iter 8/25 - loss 2.25035590 - samples/sec: 118.39 - lr: 0.031250 2021-03-26 08:47:47,743 epoch 47 - iter 10/25 - loss 2.41273284 - samples/sec: 131.90 - lr: 0.031250 2021-03-26 08:47:48,713 epoch 47 - iter 12/25 - loss 2.48591616 - samples/sec: 132.08 - lr: 0.031250 2021-03-26 08:47:49,691 epoch 47 - iter 14/25 - loss 2.50090785 - samples/sec: 131.23 - lr: 0.031250 2021-03-26 08:47:50,781 epoch 47 - iter 16/25 - loss 2.50587636 - samples/sec: 117.51 - lr: 0.031250 2021-03-26 08:47:51,824 epoch 47 - iter 18/25 - loss 2.53428050 - samples/sec: 122.88 - lr: 0.031250 2021-03-26 08:47:52,853 epoch 47 - iter 20/25 - loss 2.57194322 - samples/sec: 124.66 - lr: 0.031250 2021-03-26 08:47:53,977 epoch 47 - iter 22/25 - loss 2.58494453 - samples/sec: 114.04 - lr: 0.031250 2021-03-26 08:47:55,119 epoch 47 - iter 24/25 - loss 2.59840622 - samples/sec: 112.20 - lr: 0.031250 2021-03-26 08:47:55,490 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:47:55,490 EPOCH 47 done: loss 2.5958 - lr 0.0312500 2021-03-26 08:47:56,241 DEV : loss 6.7468581199646 - score 0.9068 2021-03-26 08:47:56,265 BAD EPOCHS (no improvement): 1 2021-03-26 08:47:56,266 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:47:57,179 epoch 48 - iter 2/25 - loss 2.16672790 - samples/sec: 140.57 - lr: 0.031250 2021-03-26 08:47:58,075 epoch 48 - iter 4/25 - loss 2.28760791 - samples/sec: 143.19 - lr: 0.031250 2021-03-26 08:47:59,117 epoch 48 - iter 6/25 - loss 2.36543711 - samples/sec: 123.08 - lr: 0.031250 2021-03-26 08:48:00,091 epoch 48 - iter 8/25 - loss 2.53750896 - samples/sec: 131.64 - lr: 0.031250 2021-03-26 08:48:01,264 epoch 48 - iter 10/25 - loss 2.52344959 - samples/sec: 109.29 - lr: 0.031250 2021-03-26 08:48:02,245 epoch 48 - iter 12/25 - loss 2.54485043 - samples/sec: 130.63 - lr: 0.031250 2021-03-26 08:48:03,192 epoch 48 - iter 14/25 - loss 2.59198126 - samples/sec: 135.37 - lr: 0.031250 2021-03-26 08:48:04,341 epoch 48 - iter 16/25 - loss 2.58411257 - samples/sec: 111.56 - lr: 0.031250 2021-03-26 08:48:05,302 epoch 48 - iter 18/25 - loss 2.56820940 - samples/sec: 133.31 - lr: 0.031250 2021-03-26 08:48:06,359 epoch 48 - iter 20/25 - loss 2.56453640 - samples/sec: 121.25 - lr: 0.031250 2021-03-26 08:48:07,342 epoch 48 - iter 22/25 - loss 2.58118105 - samples/sec: 130.59 - lr: 0.031250 2021-03-26 08:48:08,366 epoch 48 - iter 24/25 - loss 2.53787378 - samples/sec: 125.17 - lr: 0.031250 2021-03-26 08:48:08,770 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:48:08,771 EPOCH 48 done: loss 2.5485 - lr 0.0312500 2021-03-26 08:48:09,505 DEV : loss 6.730559349060059 - score 0.9076 2021-03-26 08:48:09,529 BAD EPOCHS (no improvement): 2 2021-03-26 08:48:09,530 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:48:10,517 epoch 49 - iter 2/25 - loss 2.36573863 - samples/sec: 129.86 - lr: 0.031250 2021-03-26 08:48:11,512 epoch 49 - iter 4/25 - loss 2.27939576 - samples/sec: 128.82 - lr: 0.031250 2021-03-26 08:48:12,490 epoch 49 - iter 6/25 - loss 2.51892324 - samples/sec: 131.15 - lr: 0.031250 2021-03-26 08:48:13,565 epoch 49 - iter 8/25 - loss 2.37164113 - samples/sec: 119.16 - lr: 0.031250 2021-03-26 08:48:14,605 epoch 49 - iter 10/25 - loss 2.39122179 - samples/sec: 123.27 - lr: 0.031250 2021-03-26 08:48:15,732 epoch 49 - iter 12/25 - loss 2.49826862 - samples/sec: 113.71 - lr: 0.031250 2021-03-26 08:48:16,856 epoch 49 - iter 14/25 - loss 2.58189925 - samples/sec: 114.06 - lr: 0.031250 2021-03-26 08:48:17,929 epoch 49 - iter 16/25 - loss 2.60247803 - samples/sec: 119.58 - lr: 0.031250 2021-03-26 08:48:18,989 epoch 49 - iter 18/25 - loss 2.54768838 - samples/sec: 120.91 - lr: 0.031250 2021-03-26 08:48:19,981 epoch 49 - iter 20/25 - loss 2.51007439 - samples/sec: 129.09 - lr: 0.031250 2021-03-26 08:48:20,902 epoch 49 - iter 22/25 - loss 2.48375236 - samples/sec: 139.24 - lr: 0.031250 2021-03-26 08:48:21,977 epoch 49 - iter 24/25 - loss 2.44927419 - samples/sec: 119.22 - lr: 0.031250 2021-03-26 08:48:22,382 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:48:22,382 EPOCH 49 done: loss 2.4858 - lr 0.0312500 2021-03-26 08:48:23,133 DEV : loss 6.6969757080078125 - score 0.9078 2021-03-26 08:48:23,157 BAD EPOCHS (no improvement): 3 2021-03-26 08:48:23,158 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:48:24,151 epoch 50 - iter 2/25 - loss 2.61817443 - samples/sec: 129.17 - lr: 0.031250 2021-03-26 08:48:25,130 epoch 50 - iter 4/25 - loss 2.31582826 - samples/sec: 130.94 - lr: 0.031250 2021-03-26 08:48:26,083 epoch 50 - iter 6/25 - loss 2.39882831 - samples/sec: 134.41 - lr: 0.031250 2021-03-26 08:48:27,099 epoch 50 - iter 8/25 - loss 2.64799169 - samples/sec: 126.25 - lr: 0.031250 2021-03-26 08:48:28,151 epoch 50 - iter 10/25 - loss 2.57482643 - samples/sec: 121.97 - lr: 0.031250 2021-03-26 08:48:29,159 epoch 50 - iter 12/25 - loss 2.58212765 - samples/sec: 127.22 - lr: 0.031250 2021-03-26 08:48:30,091 epoch 50 - iter 14/25 - loss 2.61432253 - samples/sec: 137.47 - lr: 0.031250 2021-03-26 08:48:31,109 epoch 50 - iter 16/25 - loss 2.60016759 - samples/sec: 125.93 - lr: 0.031250 2021-03-26 08:48:32,090 epoch 50 - iter 18/25 - loss 2.58133325 - samples/sec: 130.66 - lr: 0.031250 2021-03-26 08:48:33,114 epoch 50 - iter 20/25 - loss 2.60757553 - samples/sec: 125.08 - lr: 0.031250 2021-03-26 08:48:34,136 epoch 50 - iter 22/25 - loss 2.61094578 - samples/sec: 125.49 - lr: 0.031250 2021-03-26 08:48:35,100 epoch 50 - iter 24/25 - loss 2.65030560 - samples/sec: 132.91 - lr: 0.031250 2021-03-26 08:48:35,586 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:48:35,587 EPOCH 50 done: loss 2.6470 - lr 0.0312500 2021-03-26 08:48:36,342 DEV : loss 6.714729309082031 - score 0.91 2021-03-26 08:48:36,366 BAD EPOCHS (no improvement): 4 2021-03-26 08:48:36,367 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:48:37,375 epoch 51 - iter 2/25 - loss 2.76345909 - samples/sec: 127.17 - lr: 0.015625 2021-03-26 08:48:38,347 epoch 51 - iter 4/25 - loss 2.76501840 - samples/sec: 131.99 - lr: 0.015625 2021-03-26 08:48:39,522 epoch 51 - iter 6/25 - loss 2.60267341 - samples/sec: 109.16 - lr: 0.015625 2021-03-26 08:48:40,671 epoch 51 - iter 8/25 - loss 2.78652990 - samples/sec: 111.51 - lr: 0.015625 2021-03-26 08:48:41,772 epoch 51 - iter 10/25 - loss 2.75791469 - samples/sec: 116.40 - lr: 0.015625 2021-03-26 08:48:42,724 epoch 51 - iter 12/25 - loss 2.66620205 - samples/sec: 134.69 - lr: 0.015625 2021-03-26 08:48:43,780 epoch 51 - iter 14/25 - loss 2.66174328 - samples/sec: 121.36 - lr: 0.015625 2021-03-26 08:48:44,838 epoch 51 - iter 16/25 - loss 2.58969487 - samples/sec: 121.18 - lr: 0.015625 2021-03-26 08:48:45,873 epoch 51 - iter 18/25 - loss 2.53882750 - samples/sec: 124.01 - lr: 0.015625 2021-03-26 08:48:46,806 epoch 51 - iter 20/25 - loss 2.53399588 - samples/sec: 137.43 - lr: 0.015625 2021-03-26 08:48:47,868 epoch 51 - iter 22/25 - loss 2.51981955 - samples/sec: 120.60 - lr: 0.015625 2021-03-26 08:48:48,863 epoch 51 - iter 24/25 - loss 2.51983155 - samples/sec: 128.95 - lr: 0.015625 2021-03-26 08:48:49,259 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:48:49,259 EPOCH 51 done: loss 2.5411 - lr 0.0156250 2021-03-26 08:48:50,025 DEV : loss 6.709239959716797 - score 0.908 2021-03-26 08:48:50,049 BAD EPOCHS (no improvement): 1 2021-03-26 08:48:50,050 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:48:51,077 epoch 52 - iter 2/25 - loss 2.54436791 - samples/sec: 124.85 - lr: 0.015625 2021-03-26 08:48:52,131 epoch 52 - iter 4/25 - loss 2.69134307 - samples/sec: 121.64 - lr: 0.015625 2021-03-26 08:48:53,119 epoch 52 - iter 6/25 - loss 2.61419793 - samples/sec: 129.66 - lr: 0.015625 2021-03-26 08:48:54,174 epoch 52 - iter 8/25 - loss 2.58930629 - samples/sec: 121.62 - lr: 0.015625 2021-03-26 08:48:55,175 epoch 52 - iter 10/25 - loss 2.53519647 - samples/sec: 128.08 - lr: 0.015625 2021-03-26 08:48:56,147 epoch 52 - iter 12/25 - loss 2.49006718 - samples/sec: 131.99 - lr: 0.015625 2021-03-26 08:48:57,092 epoch 52 - iter 14/25 - loss 2.45591760 - samples/sec: 135.62 - lr: 0.015625 2021-03-26 08:48:58,012 epoch 52 - iter 16/25 - loss 2.49299569 - samples/sec: 139.27 - lr: 0.015625 2021-03-26 08:48:58,930 epoch 52 - iter 18/25 - loss 2.45735917 - samples/sec: 139.65 - lr: 0.015625 2021-03-26 08:48:59,851 epoch 52 - iter 20/25 - loss 2.55742084 - samples/sec: 139.20 - lr: 0.015625 2021-03-26 08:49:00,835 epoch 52 - iter 22/25 - loss 2.54598535 - samples/sec: 130.34 - lr: 0.015625 2021-03-26 08:49:01,797 epoch 52 - iter 24/25 - loss 2.50375176 - samples/sec: 133.30 - lr: 0.015625 2021-03-26 08:49:02,235 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:49:02,236 EPOCH 52 done: loss 2.5224 - lr 0.0156250 2021-03-26 08:49:02,989 DEV : loss 6.722785949707031 - score 0.9084 2021-03-26 08:49:03,006 BAD EPOCHS (no improvement): 2 2021-03-26 08:49:03,007 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:49:04,049 epoch 53 - iter 2/25 - loss 3.04602754 - samples/sec: 123.05 - lr: 0.015625 2021-03-26 08:49:05,091 epoch 53 - iter 4/25 - loss 2.71588653 - samples/sec: 122.99 - lr: 0.015625 2021-03-26 08:49:06,081 epoch 53 - iter 6/25 - loss 2.78629927 - samples/sec: 129.50 - lr: 0.015625 2021-03-26 08:49:07,034 epoch 53 - iter 8/25 - loss 2.62624462 - samples/sec: 134.40 - lr: 0.015625 2021-03-26 08:49:07,922 epoch 53 - iter 10/25 - loss 2.63786384 - samples/sec: 144.55 - lr: 0.015625 2021-03-26 08:49:08,890 epoch 53 - iter 12/25 - loss 2.64222335 - samples/sec: 132.34 - lr: 0.015625 2021-03-26 08:49:09,854 epoch 53 - iter 14/25 - loss 2.52385678 - samples/sec: 132.92 - lr: 0.015625 2021-03-26 08:49:10,766 epoch 53 - iter 16/25 - loss 2.48844558 - samples/sec: 140.64 - lr: 0.015625 2021-03-26 08:49:11,721 epoch 53 - iter 18/25 - loss 2.53972616 - samples/sec: 134.27 - lr: 0.015625 2021-03-26 08:49:12,706 epoch 53 - iter 20/25 - loss 2.49504396 - samples/sec: 130.12 - lr: 0.015625 2021-03-26 08:49:13,751 epoch 53 - iter 22/25 - loss 2.45131991 - samples/sec: 122.71 - lr: 0.015625 2021-03-26 08:49:14,714 epoch 53 - iter 24/25 - loss 2.45370945 - samples/sec: 133.00 - lr: 0.015625 2021-03-26 08:49:15,148 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:49:15,150 EPOCH 53 done: loss 2.4649 - lr 0.0156250 2021-03-26 08:49:15,923 DEV : loss 6.713016033172607 - score 0.909 2021-03-26 08:49:15,947 BAD EPOCHS (no improvement): 3 2021-03-26 08:49:15,948 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:49:18,013 epoch 54 - iter 2/25 - loss 2.33827412 - samples/sec: 62.05 - lr: 0.015625 2021-03-26 08:49:18,981 epoch 54 - iter 4/25 - loss 2.44314307 - samples/sec: 132.48 - lr: 0.015625 2021-03-26 08:49:19,927 epoch 54 - iter 6/25 - loss 2.50450575 - samples/sec: 135.44 - lr: 0.015625 2021-03-26 08:49:20,884 epoch 54 - iter 8/25 - loss 2.38388234 - samples/sec: 134.06 - lr: 0.015625 2021-03-26 08:49:21,837 epoch 54 - iter 10/25 - loss 2.39710271 - samples/sec: 134.53 - lr: 0.015625 2021-03-26 08:49:22,752 epoch 54 - iter 12/25 - loss 2.38589450 - samples/sec: 140.15 - lr: 0.015625 2021-03-26 08:49:23,626 epoch 54 - iter 14/25 - loss 2.45272618 - samples/sec: 146.57 - lr: 0.015625 2021-03-26 08:49:24,625 epoch 54 - iter 16/25 - loss 2.43205299 - samples/sec: 128.45 - lr: 0.015625 2021-03-26 08:49:25,648 epoch 54 - iter 18/25 - loss 2.44015885 - samples/sec: 125.34 - lr: 0.015625 2021-03-26 08:49:26,657 epoch 54 - iter 20/25 - loss 2.40970163 - samples/sec: 127.10 - lr: 0.015625 2021-03-26 08:49:27,652 epoch 54 - iter 22/25 - loss 2.42787213 - samples/sec: 128.88 - lr: 0.015625 2021-03-26 08:49:28,634 epoch 54 - iter 24/25 - loss 2.45196098 - samples/sec: 130.42 - lr: 0.015625 2021-03-26 08:49:29,053 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:49:29,054 EPOCH 54 done: loss 2.4558 - lr 0.0156250 2021-03-26 08:49:29,825 DEV : loss 6.716770172119141 - score 0.9088 2021-03-26 08:49:29,849 BAD EPOCHS (no improvement): 4 2021-03-26 08:49:29,850 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:49:30,871 epoch 55 - iter 2/25 - loss 2.50495231 - samples/sec: 125.59 - lr: 0.007812 2021-03-26 08:49:31,880 epoch 55 - iter 4/25 - loss 2.56151509 - samples/sec: 127.10 - lr: 0.007812 2021-03-26 08:49:32,794 epoch 55 - iter 6/25 - loss 2.51272758 - samples/sec: 140.33 - lr: 0.007812 2021-03-26 08:49:33,786 epoch 55 - iter 8/25 - loss 2.48402715 - samples/sec: 129.16 - lr: 0.007812 2021-03-26 08:49:34,691 epoch 55 - iter 10/25 - loss 2.43414333 - samples/sec: 141.76 - lr: 0.007812 2021-03-26 08:49:35,643 epoch 55 - iter 12/25 - loss 2.41734570 - samples/sec: 134.57 - lr: 0.007812 2021-03-26 08:49:36,656 epoch 55 - iter 14/25 - loss 2.44474280 - samples/sec: 126.55 - lr: 0.007812 2021-03-26 08:49:37,660 epoch 55 - iter 16/25 - loss 2.44347511 - samples/sec: 127.73 - lr: 0.007812 2021-03-26 08:49:38,662 epoch 55 - iter 18/25 - loss 2.40479814 - samples/sec: 128.05 - lr: 0.007812 2021-03-26 08:49:39,680 epoch 55 - iter 20/25 - loss 2.41784536 - samples/sec: 125.90 - lr: 0.007812 2021-03-26 08:49:40,728 epoch 55 - iter 22/25 - loss 2.42849638 - samples/sec: 122.33 - lr: 0.007812 2021-03-26 08:49:41,808 epoch 55 - iter 24/25 - loss 2.42535428 - samples/sec: 118.68 - lr: 0.007812 2021-03-26 08:49:42,208 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:49:42,209 EPOCH 55 done: loss 2.4129 - lr 0.0078125 2021-03-26 08:49:42,994 DEV : loss 6.715353965759277 - score 0.9082 2021-03-26 08:49:43,014 BAD EPOCHS (no improvement): 1 2021-03-26 08:49:43,015 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:49:44,063 epoch 56 - iter 2/25 - loss 2.41294694 - samples/sec: 122.81 - lr: 0.007812 2021-03-26 08:49:45,012 epoch 56 - iter 4/25 - loss 2.40887398 - samples/sec: 135.11 - lr: 0.007812 2021-03-26 08:49:45,883 epoch 56 - iter 6/25 - loss 2.53534293 - samples/sec: 147.28 - lr: 0.007812 2021-03-26 08:49:46,844 epoch 56 - iter 8/25 - loss 2.63588372 - samples/sec: 133.35 - lr: 0.007812 2021-03-26 08:49:47,785 epoch 56 - iter 10/25 - loss 2.57245722 - samples/sec: 136.30 - lr: 0.007812 2021-03-26 08:49:48,681 epoch 56 - iter 12/25 - loss 2.50227737 - samples/sec: 142.97 - lr: 0.007812 2021-03-26 08:49:49,645 epoch 56 - iter 14/25 - loss 2.51777281 - samples/sec: 133.06 - lr: 0.007812 2021-03-26 08:49:50,669 epoch 56 - iter 16/25 - loss 2.53424157 - samples/sec: 125.18 - lr: 0.007812 2021-03-26 08:49:51,625 epoch 56 - iter 18/25 - loss 2.51135084 - samples/sec: 134.01 - lr: 0.007812 2021-03-26 08:49:52,655 epoch 56 - iter 20/25 - loss 2.54296572 - samples/sec: 124.62 - lr: 0.007812 2021-03-26 08:49:53,643 epoch 56 - iter 22/25 - loss 2.59556309 - samples/sec: 129.69 - lr: 0.007812 2021-03-26 08:49:54,662 epoch 56 - iter 24/25 - loss 2.62635016 - samples/sec: 125.76 - lr: 0.007812 2021-03-26 08:49:55,033 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:49:55,033 EPOCH 56 done: loss 2.5899 - lr 0.0078125 2021-03-26 08:49:55,798 DEV : loss 6.716400146484375 - score 0.9086 2021-03-26 08:49:55,826 BAD EPOCHS (no improvement): 2 2021-03-26 08:49:55,827 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:49:56,862 epoch 57 - iter 2/25 - loss 2.28633606 - samples/sec: 123.85 - lr: 0.007812 2021-03-26 08:49:57,800 epoch 57 - iter 4/25 - loss 2.49695295 - samples/sec: 136.61 - lr: 0.007812 2021-03-26 08:49:59,067 epoch 57 - iter 6/25 - loss 2.40712937 - samples/sec: 101.19 - lr: 0.007812 2021-03-26 08:50:00,242 epoch 57 - iter 8/25 - loss 2.43323019 - samples/sec: 109.06 - lr: 0.007812 2021-03-26 08:50:01,234 epoch 57 - iter 10/25 - loss 2.32638550 - samples/sec: 129.28 - lr: 0.007812 2021-03-26 08:50:02,196 epoch 57 - iter 12/25 - loss 2.33791218 - samples/sec: 133.35 - lr: 0.007812 2021-03-26 08:50:03,223 epoch 57 - iter 14/25 - loss 2.34867431 - samples/sec: 124.76 - lr: 0.007812 2021-03-26 08:50:04,196 epoch 57 - iter 16/25 - loss 2.39169888 - samples/sec: 131.79 - lr: 0.007812 2021-03-26 08:50:05,207 epoch 57 - iter 18/25 - loss 2.39465144 - samples/sec: 126.77 - lr: 0.007812 2021-03-26 08:50:06,121 epoch 57 - iter 20/25 - loss 2.40665802 - samples/sec: 140.32 - lr: 0.007812 2021-03-26 08:50:07,032 epoch 57 - iter 22/25 - loss 2.43683068 - samples/sec: 140.71 - lr: 0.007812 2021-03-26 08:50:08,018 epoch 57 - iter 24/25 - loss 2.47992485 - samples/sec: 130.02 - lr: 0.007812 2021-03-26 08:50:08,401 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:50:08,401 EPOCH 57 done: loss 2.4940 - lr 0.0078125 2021-03-26 08:50:09,127 DEV : loss 6.717229843139648 - score 0.909 2021-03-26 08:50:09,151 BAD EPOCHS (no improvement): 3 2021-03-26 08:50:09,152 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:50:10,028 epoch 58 - iter 2/25 - loss 2.40251827 - samples/sec: 146.45 - lr: 0.007812 2021-03-26 08:50:10,997 epoch 58 - iter 4/25 - loss 2.45009959 - samples/sec: 132.24 - lr: 0.007812 2021-03-26 08:50:12,049 epoch 58 - iter 6/25 - loss 2.36056856 - samples/sec: 121.79 - lr: 0.007812 2021-03-26 08:50:12,988 epoch 58 - iter 8/25 - loss 2.26037887 - samples/sec: 136.63 - lr: 0.007812 2021-03-26 08:50:13,963 epoch 58 - iter 10/25 - loss 2.39980597 - samples/sec: 131.41 - lr: 0.007812 2021-03-26 08:50:14,931 epoch 58 - iter 12/25 - loss 2.34398366 - samples/sec: 132.49 - lr: 0.007812 2021-03-26 08:50:15,908 epoch 58 - iter 14/25 - loss 2.33637424 - samples/sec: 131.18 - lr: 0.007812 2021-03-26 08:50:17,012 epoch 58 - iter 16/25 - loss 2.41296620 - samples/sec: 116.08 - lr: 0.007812 2021-03-26 08:50:17,980 epoch 58 - iter 18/25 - loss 2.45545569 - samples/sec: 132.46 - lr: 0.007812 2021-03-26 08:50:18,835 epoch 58 - iter 20/25 - loss 2.47555558 - samples/sec: 149.95 - lr: 0.007812 2021-03-26 08:50:19,849 epoch 58 - iter 22/25 - loss 2.50793289 - samples/sec: 126.51 - lr: 0.007812 2021-03-26 08:50:20,889 epoch 58 - iter 24/25 - loss 2.48611053 - samples/sec: 123.29 - lr: 0.007812 2021-03-26 08:50:21,350 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:50:21,350 EPOCH 58 done: loss 2.5119 - lr 0.0078125 2021-03-26 08:50:22,114 DEV : loss 6.7159423828125 - score 0.9094 2021-03-26 08:50:22,138 BAD EPOCHS (no improvement): 4 2021-03-26 08:50:22,138 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:50:23,080 epoch 59 - iter 2/25 - loss 2.81640446 - samples/sec: 136.21 - lr: 0.003906 2021-03-26 08:50:24,095 epoch 59 - iter 4/25 - loss 2.52743006 - samples/sec: 126.39 - lr: 0.003906 2021-03-26 08:50:25,130 epoch 59 - iter 6/25 - loss 2.78575460 - samples/sec: 123.80 - lr: 0.003906 2021-03-26 08:50:26,130 epoch 59 - iter 8/25 - loss 2.84091517 - samples/sec: 128.19 - lr: 0.003906 2021-03-26 08:50:27,019 epoch 59 - iter 10/25 - loss 2.71048800 - samples/sec: 144.21 - lr: 0.003906 2021-03-26 08:50:28,014 epoch 59 - iter 12/25 - loss 2.75860839 - samples/sec: 128.85 - lr: 0.003906 2021-03-26 08:50:29,113 epoch 59 - iter 14/25 - loss 2.67045104 - samples/sec: 116.69 - lr: 0.003906 2021-03-26 08:50:30,088 epoch 59 - iter 16/25 - loss 2.63434067 - samples/sec: 131.50 - lr: 0.003906 2021-03-26 08:50:31,289 epoch 59 - iter 18/25 - loss 2.56707877 - samples/sec: 106.68 - lr: 0.003906 2021-03-26 08:50:32,303 epoch 59 - iter 20/25 - loss 2.47917590 - samples/sec: 126.46 - lr: 0.003906 2021-03-26 08:50:33,245 epoch 59 - iter 22/25 - loss 2.48207686 - samples/sec: 136.08 - lr: 0.003906 2021-03-26 08:50:34,161 epoch 59 - iter 24/25 - loss 2.48821341 - samples/sec: 139.86 - lr: 0.003906 2021-03-26 08:50:34,563 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:50:34,563 EPOCH 59 done: loss 2.5272 - lr 0.0039062 2021-03-26 08:50:35,305 DEV : loss 6.716526985168457 - score 0.9098 2021-03-26 08:50:35,328 BAD EPOCHS (no improvement): 1 2021-03-26 08:50:35,329 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:50:36,185 epoch 60 - iter 2/25 - loss 2.27116865 - samples/sec: 149.83 - lr: 0.003906 2021-03-26 08:50:37,114 epoch 60 - iter 4/25 - loss 2.25008342 - samples/sec: 138.07 - lr: 0.003906 2021-03-26 08:50:38,045 epoch 60 - iter 6/25 - loss 2.29013628 - samples/sec: 137.75 - lr: 0.003906 2021-03-26 08:50:39,027 epoch 60 - iter 8/25 - loss 2.38840453 - samples/sec: 130.65 - lr: 0.003906 2021-03-26 08:50:39,969 epoch 60 - iter 10/25 - loss 2.30501388 - samples/sec: 136.09 - lr: 0.003906 2021-03-26 08:50:41,052 epoch 60 - iter 12/25 - loss 2.43222015 - samples/sec: 118.32 - lr: 0.003906 2021-03-26 08:50:42,075 epoch 60 - iter 14/25 - loss 2.48389046 - samples/sec: 125.40 - lr: 0.003906 2021-03-26 08:50:43,007 epoch 60 - iter 16/25 - loss 2.47347818 - samples/sec: 137.56 - lr: 0.003906 2021-03-26 08:50:44,112 epoch 60 - iter 18/25 - loss 2.44074412 - samples/sec: 115.92 - lr: 0.003906 2021-03-26 08:50:45,019 epoch 60 - iter 20/25 - loss 2.43466492 - samples/sec: 141.49 - lr: 0.003906 2021-03-26 08:50:46,059 epoch 60 - iter 22/25 - loss 2.43043199 - samples/sec: 123.26 - lr: 0.003906 2021-03-26 08:50:47,074 epoch 60 - iter 24/25 - loss 2.41566441 - samples/sec: 126.19 - lr: 0.003906 2021-03-26 08:50:47,530 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:50:47,531 EPOCH 60 done: loss 2.4161 - lr 0.0039062 2021-03-26 08:50:48,287 DEV : loss 6.71820592880249 - score 0.9098 2021-03-26 08:50:48,303 BAD EPOCHS (no improvement): 2 2021-03-26 08:50:48,304 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:50:49,290 epoch 61 - iter 2/25 - loss 2.58153760 - samples/sec: 130.06 - lr: 0.003906 2021-03-26 08:50:50,258 epoch 61 - iter 4/25 - loss 2.38109404 - samples/sec: 132.41 - lr: 0.003906 2021-03-26 08:50:51,233 epoch 61 - iter 6/25 - loss 2.34258546 - samples/sec: 131.48 - lr: 0.003906 2021-03-26 08:50:52,241 epoch 61 - iter 8/25 - loss 2.38017832 - samples/sec: 127.10 - lr: 0.003906 2021-03-26 08:50:53,260 epoch 61 - iter 10/25 - loss 2.41248423 - samples/sec: 125.82 - lr: 0.003906 2021-03-26 08:50:54,172 epoch 61 - iter 12/25 - loss 2.40937188 - samples/sec: 140.53 - lr: 0.003906 2021-03-26 08:50:55,172 epoch 61 - iter 14/25 - loss 2.38282467 - samples/sec: 128.25 - lr: 0.003906 2021-03-26 08:50:56,381 epoch 61 - iter 16/25 - loss 2.39176605 - samples/sec: 106.04 - lr: 0.003906 2021-03-26 08:50:57,724 epoch 61 - iter 18/25 - loss 2.32950047 - samples/sec: 95.41 - lr: 0.003906 2021-03-26 08:50:58,859 epoch 61 - iter 20/25 - loss 2.30976757 - samples/sec: 112.95 - lr: 0.003906 2021-03-26 08:50:59,864 epoch 61 - iter 22/25 - loss 2.32149645 - samples/sec: 127.64 - lr: 0.003906 2021-03-26 08:51:00,767 epoch 61 - iter 24/25 - loss 2.33463271 - samples/sec: 142.03 - lr: 0.003906 2021-03-26 08:51:01,175 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:51:01,176 EPOCH 61 done: loss 2.3059 - lr 0.0039062 2021-03-26 08:51:01,930 DEV : loss 6.716128349304199 - score 0.9098 2021-03-26 08:51:01,954 BAD EPOCHS (no improvement): 3 2021-03-26 08:51:01,954 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:51:02,905 epoch 62 - iter 2/25 - loss 2.21142137 - samples/sec: 134.98 - lr: 0.003906 2021-03-26 08:51:04,081 epoch 62 - iter 4/25 - loss 2.45809269 - samples/sec: 108.99 - lr: 0.003906 2021-03-26 08:51:05,191 epoch 62 - iter 6/25 - loss 2.34228293 - samples/sec: 115.47 - lr: 0.003906 2021-03-26 08:51:06,278 epoch 62 - iter 8/25 - loss 2.29517706 - samples/sec: 117.93 - lr: 0.003906 2021-03-26 08:51:07,228 epoch 62 - iter 10/25 - loss 2.27721502 - samples/sec: 135.01 - lr: 0.003906 2021-03-26 08:51:08,291 epoch 62 - iter 12/25 - loss 2.28681889 - samples/sec: 120.55 - lr: 0.003906 2021-03-26 08:51:09,235 epoch 62 - iter 14/25 - loss 2.37437183 - samples/sec: 135.69 - lr: 0.003906 2021-03-26 08:51:10,159 epoch 62 - iter 16/25 - loss 2.32531175 - samples/sec: 138.77 - lr: 0.003906 2021-03-26 08:51:11,041 epoch 62 - iter 18/25 - loss 2.32343723 - samples/sec: 145.35 - lr: 0.003906 2021-03-26 08:51:12,045 epoch 62 - iter 20/25 - loss 2.34342856 - samples/sec: 127.75 - lr: 0.003906 2021-03-26 08:51:13,083 epoch 62 - iter 22/25 - loss 2.34453206 - samples/sec: 123.48 - lr: 0.003906 2021-03-26 08:51:14,064 epoch 62 - iter 24/25 - loss 2.34612409 - samples/sec: 130.70 - lr: 0.003906 2021-03-26 08:51:14,469 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:51:14,470 EPOCH 62 done: loss 2.3247 - lr 0.0039062 2021-03-26 08:51:15,197 DEV : loss 6.719080924987793 - score 0.9098 2021-03-26 08:51:15,219 BAD EPOCHS (no improvement): 4 2021-03-26 08:51:15,220 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:51:16,238 epoch 63 - iter 2/25 - loss 3.00047922 - samples/sec: 125.89 - lr: 0.001953 2021-03-26 08:51:17,209 epoch 63 - iter 4/25 - loss 2.73255146 - samples/sec: 132.08 - lr: 0.001953 2021-03-26 08:51:18,118 epoch 63 - iter 6/25 - loss 2.49330461 - samples/sec: 141.03 - lr: 0.001953 2021-03-26 08:51:19,051 epoch 63 - iter 8/25 - loss 2.45386675 - samples/sec: 137.39 - lr: 0.001953 2021-03-26 08:51:20,050 epoch 63 - iter 10/25 - loss 2.39355280 - samples/sec: 128.39 - lr: 0.001953 2021-03-26 08:51:21,009 epoch 63 - iter 12/25 - loss 2.44529299 - samples/sec: 133.62 - lr: 0.001953 2021-03-26 08:51:22,162 epoch 63 - iter 14/25 - loss 2.46328975 - samples/sec: 111.17 - lr: 0.001953 2021-03-26 08:51:23,235 epoch 63 - iter 16/25 - loss 2.51815675 - samples/sec: 119.52 - lr: 0.001953 2021-03-26 08:51:24,479 epoch 63 - iter 18/25 - loss 2.49993791 - samples/sec: 103.00 - lr: 0.001953 2021-03-26 08:51:25,463 epoch 63 - iter 20/25 - loss 2.52069933 - samples/sec: 130.32 - lr: 0.001953 2021-03-26 08:51:26,424 epoch 63 - iter 22/25 - loss 2.48497365 - samples/sec: 133.53 - lr: 0.001953 2021-03-26 08:51:27,384 epoch 63 - iter 24/25 - loss 2.48816618 - samples/sec: 133.46 - lr: 0.001953 2021-03-26 08:51:27,774 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:51:27,775 EPOCH 63 done: loss 2.5114 - lr 0.0019531 2021-03-26 08:51:28,546 DEV : loss 6.721194267272949 - score 0.9098 2021-03-26 08:51:28,570 BAD EPOCHS (no improvement): 1 2021-03-26 08:51:28,571 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:51:29,485 epoch 64 - iter 2/25 - loss 2.62489450 - samples/sec: 140.44 - lr: 0.001953 2021-03-26 08:51:30,494 epoch 64 - iter 4/25 - loss 2.66914499 - samples/sec: 127.01 - lr: 0.001953 2021-03-26 08:51:31,437 epoch 64 - iter 6/25 - loss 2.67395210 - samples/sec: 135.90 - lr: 0.001953 2021-03-26 08:51:32,494 epoch 64 - iter 8/25 - loss 2.64659414 - samples/sec: 121.28 - lr: 0.001953 2021-03-26 08:51:33,489 epoch 64 - iter 10/25 - loss 2.68895016 - samples/sec: 128.80 - lr: 0.001953 2021-03-26 08:51:34,379 epoch 64 - iter 12/25 - loss 2.62112544 - samples/sec: 144.18 - lr: 0.001953 2021-03-26 08:51:35,342 epoch 64 - iter 14/25 - loss 2.57101262 - samples/sec: 133.02 - lr: 0.001953 2021-03-26 08:51:36,297 epoch 64 - iter 16/25 - loss 2.49899342 - samples/sec: 134.28 - lr: 0.001953 2021-03-26 08:51:37,310 epoch 64 - iter 18/25 - loss 2.52965344 - samples/sec: 126.56 - lr: 0.001953 2021-03-26 08:51:38,334 epoch 64 - iter 20/25 - loss 2.55107920 - samples/sec: 125.16 - lr: 0.001953 2021-03-26 08:51:39,345 epoch 64 - iter 22/25 - loss 2.56382647 - samples/sec: 126.83 - lr: 0.001953 2021-03-26 08:51:40,347 epoch 64 - iter 24/25 - loss 2.57456375 - samples/sec: 127.89 - lr: 0.001953 2021-03-26 08:51:40,832 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:51:40,833 EPOCH 64 done: loss 2.5673 - lr 0.0019531 2021-03-26 08:51:41,594 DEV : loss 6.721752166748047 - score 0.909 2021-03-26 08:51:41,613 BAD EPOCHS (no improvement): 2 2021-03-26 08:51:41,614 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:51:42,619 epoch 65 - iter 2/25 - loss 2.50386107 - samples/sec: 127.54 - lr: 0.001953 2021-03-26 08:51:43,773 epoch 65 - iter 4/25 - loss 2.74910355 - samples/sec: 111.07 - lr: 0.001953 2021-03-26 08:51:44,784 epoch 65 - iter 6/25 - loss 2.68734121 - samples/sec: 126.74 - lr: 0.001953 2021-03-26 08:51:45,784 epoch 65 - iter 8/25 - loss 2.60665485 - samples/sec: 128.36 - lr: 0.001953 2021-03-26 08:51:46,777 epoch 65 - iter 10/25 - loss 2.51850870 - samples/sec: 129.11 - lr: 0.001953 2021-03-26 08:51:47,798 epoch 65 - iter 12/25 - loss 2.43202922 - samples/sec: 125.61 - lr: 0.001953 2021-03-26 08:51:48,717 epoch 65 - iter 14/25 - loss 2.41641241 - samples/sec: 139.37 - lr: 0.001953 2021-03-26 08:51:49,681 epoch 65 - iter 16/25 - loss 2.39029852 - samples/sec: 133.10 - lr: 0.001953 2021-03-26 08:51:50,621 epoch 65 - iter 18/25 - loss 2.37561301 - samples/sec: 136.34 - lr: 0.001953 2021-03-26 08:51:51,591 epoch 65 - iter 20/25 - loss 2.38306705 - samples/sec: 132.05 - lr: 0.001953 2021-03-26 08:51:52,450 epoch 65 - iter 22/25 - loss 2.36917746 - samples/sec: 149.20 - lr: 0.001953 2021-03-26 08:51:53,395 epoch 65 - iter 24/25 - loss 2.41181322 - samples/sec: 135.74 - lr: 0.001953 2021-03-26 08:51:53,816 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:51:53,817 EPOCH 65 done: loss 2.3659 - lr 0.0019531 2021-03-26 08:51:54,576 DEV : loss 6.72037410736084 - score 0.9102 2021-03-26 08:51:54,596 BAD EPOCHS (no improvement): 3 2021-03-26 08:51:54,596 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:51:55,588 epoch 66 - iter 2/25 - loss 2.60044074 - samples/sec: 129.23 - lr: 0.001953 2021-03-26 08:51:56,727 epoch 66 - iter 4/25 - loss 2.51240951 - samples/sec: 112.51 - lr: 0.001953 2021-03-26 08:51:57,929 epoch 66 - iter 6/25 - loss 2.61460606 - samples/sec: 106.68 - lr: 0.001953 2021-03-26 08:51:58,813 epoch 66 - iter 8/25 - loss 2.61795145 - samples/sec: 145.14 - lr: 0.001953 2021-03-26 08:51:59,833 epoch 66 - iter 10/25 - loss 2.59454665 - samples/sec: 125.66 - lr: 0.001953 2021-03-26 08:52:00,802 epoch 66 - iter 12/25 - loss 2.51743805 - samples/sec: 132.31 - lr: 0.001953 2021-03-26 08:52:01,815 epoch 66 - iter 14/25 - loss 2.43958904 - samples/sec: 126.52 - lr: 0.001953 2021-03-26 08:52:02,774 epoch 66 - iter 16/25 - loss 2.48947333 - samples/sec: 133.79 - lr: 0.001953 2021-03-26 08:52:03,746 epoch 66 - iter 18/25 - loss 2.50290778 - samples/sec: 131.85 - lr: 0.001953 2021-03-26 08:52:04,716 epoch 66 - iter 20/25 - loss 2.47735422 - samples/sec: 132.29 - lr: 0.001953 2021-03-26 08:52:05,793 epoch 66 - iter 22/25 - loss 2.46050443 - samples/sec: 118.94 - lr: 0.001953 2021-03-26 08:52:06,819 epoch 66 - iter 24/25 - loss 2.50004270 - samples/sec: 125.06 - lr: 0.001953 2021-03-26 08:52:07,162 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:52:07,163 EPOCH 66 done: loss 2.4471 - lr 0.0019531 2021-03-26 08:52:07,918 DEV : loss 6.720994472503662 - score 0.9098 2021-03-26 08:52:07,946 BAD EPOCHS (no improvement): 4 2021-03-26 08:52:07,946 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:52:08,972 epoch 67 - iter 2/25 - loss 2.64988303 - samples/sec: 125.06 - lr: 0.000977 2021-03-26 08:52:09,913 epoch 67 - iter 4/25 - loss 2.85384834 - samples/sec: 136.19 - lr: 0.000977 2021-03-26 08:52:10,838 epoch 67 - iter 6/25 - loss 2.66090802 - samples/sec: 138.57 - lr: 0.000977 2021-03-26 08:52:11,772 epoch 67 - iter 8/25 - loss 2.55720612 - samples/sec: 137.20 - lr: 0.000977 2021-03-26 08:52:12,793 epoch 67 - iter 10/25 - loss 2.50371974 - samples/sec: 125.63 - lr: 0.000977 2021-03-26 08:52:13,719 epoch 67 - iter 12/25 - loss 2.53245260 - samples/sec: 138.44 - lr: 0.000977 2021-03-26 08:52:14,723 epoch 67 - iter 14/25 - loss 2.57271474 - samples/sec: 127.82 - lr: 0.000977 2021-03-26 08:52:15,701 epoch 67 - iter 16/25 - loss 2.54835680 - samples/sec: 131.10 - lr: 0.000977 2021-03-26 08:52:16,666 epoch 67 - iter 18/25 - loss 2.52905520 - samples/sec: 132.85 - lr: 0.000977 2021-03-26 08:52:17,592 epoch 67 - iter 20/25 - loss 2.48178655 - samples/sec: 138.58 - lr: 0.000977 2021-03-26 08:52:18,570 epoch 67 - iter 22/25 - loss 2.43028183 - samples/sec: 130.98 - lr: 0.000977 2021-03-26 08:52:19,685 epoch 67 - iter 24/25 - loss 2.45312720 - samples/sec: 115.01 - lr: 0.000977 2021-03-26 08:52:20,159 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:52:20,160 EPOCH 67 done: loss 2.4251 - lr 0.0009766 2021-03-26 08:52:20,941 DEV : loss 6.7206525802612305 - score 0.9098 2021-03-26 08:52:20,966 BAD EPOCHS (no improvement): 1 2021-03-26 08:52:20,967 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:52:21,929 epoch 68 - iter 2/25 - loss 2.50237751 - samples/sec: 133.25 - lr: 0.000977 2021-03-26 08:52:22,910 epoch 68 - iter 4/25 - loss 2.11894467 - samples/sec: 130.79 - lr: 0.000977 2021-03-26 08:52:23,931 epoch 68 - iter 6/25 - loss 2.22465249 - samples/sec: 125.95 - lr: 0.000977 2021-03-26 08:52:24,939 epoch 68 - iter 8/25 - loss 2.28452024 - samples/sec: 127.10 - lr: 0.000977 2021-03-26 08:52:25,963 epoch 68 - iter 10/25 - loss 2.31511095 - samples/sec: 125.17 - lr: 0.000977 2021-03-26 08:52:26,908 epoch 68 - iter 12/25 - loss 2.33202116 - samples/sec: 135.67 - lr: 0.000977 2021-03-26 08:52:27,909 epoch 68 - iter 14/25 - loss 2.34010000 - samples/sec: 128.20 - lr: 0.000977 2021-03-26 08:52:28,933 epoch 68 - iter 16/25 - loss 2.38829292 - samples/sec: 125.21 - lr: 0.000977 2021-03-26 08:52:29,959 epoch 68 - iter 18/25 - loss 2.34564919 - samples/sec: 124.92 - lr: 0.000977 2021-03-26 08:52:30,970 epoch 68 - iter 20/25 - loss 2.32275848 - samples/sec: 126.74 - lr: 0.000977 2021-03-26 08:52:31,947 epoch 68 - iter 22/25 - loss 2.33297563 - samples/sec: 131.31 - lr: 0.000977 2021-03-26 08:52:32,813 epoch 68 - iter 24/25 - loss 2.30395630 - samples/sec: 148.08 - lr: 0.000977 2021-03-26 08:52:33,185 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:52:33,185 EPOCH 68 done: loss 2.3152 - lr 0.0009766 2021-03-26 08:52:33,926 DEV : loss 6.721766471862793 - score 0.9094 2021-03-26 08:52:33,949 BAD EPOCHS (no improvement): 2 2021-03-26 08:52:33,950 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:52:34,923 epoch 69 - iter 2/25 - loss 2.32102466 - samples/sec: 131.82 - lr: 0.000977 2021-03-26 08:52:35,944 epoch 69 - iter 4/25 - loss 2.19986767 - samples/sec: 125.53 - lr: 0.000977 2021-03-26 08:52:37,137 epoch 69 - iter 6/25 - loss 2.22192717 - samples/sec: 107.50 - lr: 0.000977 2021-03-26 08:52:38,157 epoch 69 - iter 8/25 - loss 2.37492293 - samples/sec: 125.69 - lr: 0.000977 2021-03-26 08:52:39,150 epoch 69 - iter 10/25 - loss 2.43023714 - samples/sec: 129.12 - lr: 0.000977 2021-03-26 08:52:40,154 epoch 69 - iter 12/25 - loss 2.35289692 - samples/sec: 127.75 - lr: 0.000977 2021-03-26 08:52:41,286 epoch 69 - iter 14/25 - loss 2.35287110 - samples/sec: 113.17 - lr: 0.000977 2021-03-26 08:52:42,298 epoch 69 - iter 16/25 - loss 2.34908857 - samples/sec: 126.83 - lr: 0.000977 2021-03-26 08:52:43,292 epoch 69 - iter 18/25 - loss 2.43797569 - samples/sec: 128.98 - lr: 0.000977 2021-03-26 08:52:44,154 epoch 69 - iter 20/25 - loss 2.44098268 - samples/sec: 148.57 - lr: 0.000977 2021-03-26 08:52:45,230 epoch 69 - iter 22/25 - loss 2.44536105 - samples/sec: 119.21 - lr: 0.000977 2021-03-26 08:52:46,184 epoch 69 - iter 24/25 - loss 2.45193523 - samples/sec: 134.34 - lr: 0.000977 2021-03-26 08:52:46,553 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:52:46,554 EPOCH 69 done: loss 2.4442 - lr 0.0009766 2021-03-26 08:52:47,314 DEV : loss 6.722614288330078 - score 0.9094 2021-03-26 08:52:47,341 BAD EPOCHS (no improvement): 3 2021-03-26 08:52:47,342 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:52:48,216 epoch 70 - iter 2/25 - loss 2.31417513 - samples/sec: 146.87 - lr: 0.000977 2021-03-26 08:52:49,169 epoch 70 - iter 4/25 - loss 2.40965217 - samples/sec: 134.51 - lr: 0.000977 2021-03-26 08:52:50,170 epoch 70 - iter 6/25 - loss 2.42630080 - samples/sec: 128.04 - lr: 0.000977 2021-03-26 08:52:51,162 epoch 70 - iter 8/25 - loss 2.35043907 - samples/sec: 129.32 - lr: 0.000977 2021-03-26 08:52:52,096 epoch 70 - iter 10/25 - loss 2.41967034 - samples/sec: 137.19 - lr: 0.000977 2021-03-26 08:52:53,180 epoch 70 - iter 12/25 - loss 2.37920668 - samples/sec: 118.21 - lr: 0.000977 2021-03-26 08:52:54,122 epoch 70 - iter 14/25 - loss 2.41693105 - samples/sec: 136.07 - lr: 0.000977 2021-03-26 08:52:55,081 epoch 70 - iter 16/25 - loss 2.46385759 - samples/sec: 133.75 - lr: 0.000977 2021-03-26 08:52:56,056 epoch 70 - iter 18/25 - loss 2.49344937 - samples/sec: 131.49 - lr: 0.000977 2021-03-26 08:52:56,987 epoch 70 - iter 20/25 - loss 2.46795544 - samples/sec: 137.77 - lr: 0.000977 2021-03-26 08:52:57,961 epoch 70 - iter 22/25 - loss 2.43951023 - samples/sec: 131.56 - lr: 0.000977 2021-03-26 08:52:59,009 epoch 70 - iter 24/25 - loss 2.45959963 - samples/sec: 122.26 - lr: 0.000977 2021-03-26 08:52:59,439 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:52:59,440 EPOCH 70 done: loss 2.4916 - lr 0.0009766 2021-03-26 08:53:00,188 DEV : loss 6.721915245056152 - score 0.9094 2021-03-26 08:53:00,204 BAD EPOCHS (no improvement): 4 2021-03-26 08:53:00,205 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:53:01,251 epoch 71 - iter 2/25 - loss 2.93990111 - samples/sec: 122.48 - lr: 0.000488 2021-03-26 08:53:02,230 epoch 71 - iter 4/25 - loss 2.70402431 - samples/sec: 131.10 - lr: 0.000488 2021-03-26 08:53:03,145 epoch 71 - iter 6/25 - loss 2.52367194 - samples/sec: 140.12 - lr: 0.000488 2021-03-26 08:53:04,058 epoch 71 - iter 8/25 - loss 2.46390080 - samples/sec: 140.35 - lr: 0.000488 2021-03-26 08:53:05,079 epoch 71 - iter 10/25 - loss 2.46541066 - samples/sec: 125.51 - lr: 0.000488 2021-03-26 08:53:06,070 epoch 71 - iter 12/25 - loss 2.41675409 - samples/sec: 129.43 - lr: 0.000488 2021-03-26 08:53:07,068 epoch 71 - iter 14/25 - loss 2.40485142 - samples/sec: 128.34 - lr: 0.000488 2021-03-26 08:53:08,182 epoch 71 - iter 16/25 - loss 2.39172129 - samples/sec: 115.13 - lr: 0.000488 2021-03-26 08:53:09,358 epoch 71 - iter 18/25 - loss 2.42713354 - samples/sec: 108.98 - lr: 0.000488 2021-03-26 08:53:10,436 epoch 71 - iter 20/25 - loss 2.43891806 - samples/sec: 118.88 - lr: 0.000488 2021-03-26 08:53:11,471 epoch 71 - iter 22/25 - loss 2.41635902 - samples/sec: 123.89 - lr: 0.000488 2021-03-26 08:53:12,392 epoch 71 - iter 24/25 - loss 2.41604218 - samples/sec: 139.07 - lr: 0.000488 2021-03-26 08:53:12,808 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:53:12,808 EPOCH 71 done: loss 2.4220 - lr 0.0004883 2021-03-26 08:53:13,571 DEV : loss 6.722039699554443 - score 0.9098 2021-03-26 08:53:13,594 BAD EPOCHS (no improvement): 1 2021-03-26 08:53:13,594 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:53:14,545 epoch 72 - iter 2/25 - loss 2.27463222 - samples/sec: 134.91 - lr: 0.000488 2021-03-26 08:53:15,464 epoch 72 - iter 4/25 - loss 2.16546875 - samples/sec: 139.50 - lr: 0.000488 2021-03-26 08:53:16,499 epoch 72 - iter 6/25 - loss 2.27853771 - samples/sec: 123.95 - lr: 0.000488 2021-03-26 08:53:17,515 epoch 72 - iter 8/25 - loss 2.27974728 - samples/sec: 126.14 - lr: 0.000488 2021-03-26 08:53:18,556 epoch 72 - iter 10/25 - loss 2.32490218 - samples/sec: 123.11 - lr: 0.000488 2021-03-26 08:53:19,579 epoch 72 - iter 12/25 - loss 2.34221594 - samples/sec: 125.28 - lr: 0.000488 2021-03-26 08:53:20,592 epoch 72 - iter 14/25 - loss 2.33135453 - samples/sec: 126.62 - lr: 0.000488 2021-03-26 08:53:21,578 epoch 72 - iter 16/25 - loss 2.36180919 - samples/sec: 129.96 - lr: 0.000488 2021-03-26 08:53:22,646 epoch 72 - iter 18/25 - loss 2.36696060 - samples/sec: 119.98 - lr: 0.000488 2021-03-26 08:53:23,788 epoch 72 - iter 20/25 - loss 2.34693076 - samples/sec: 112.30 - lr: 0.000488 2021-03-26 08:53:24,843 epoch 72 - iter 22/25 - loss 2.38968690 - samples/sec: 121.54 - lr: 0.000488 2021-03-26 08:53:25,993 epoch 72 - iter 24/25 - loss 2.42801057 - samples/sec: 111.46 - lr: 0.000488 2021-03-26 08:53:26,411 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:53:26,412 EPOCH 72 done: loss 2.4201 - lr 0.0004883 2021-03-26 08:53:27,138 DEV : loss 6.722582817077637 - score 0.9094 2021-03-26 08:53:27,161 BAD EPOCHS (no improvement): 2 2021-03-26 08:53:27,162 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:53:28,166 epoch 73 - iter 2/25 - loss 2.58827686 - samples/sec: 127.78 - lr: 0.000488 2021-03-26 08:53:29,233 epoch 73 - iter 4/25 - loss 2.46295500 - samples/sec: 120.18 - lr: 0.000488 2021-03-26 08:53:30,238 epoch 73 - iter 6/25 - loss 2.42940970 - samples/sec: 127.65 - lr: 0.000488 2021-03-26 08:53:31,226 epoch 73 - iter 8/25 - loss 2.37589297 - samples/sec: 129.86 - lr: 0.000488 2021-03-26 08:53:32,190 epoch 73 - iter 10/25 - loss 2.33973050 - samples/sec: 133.05 - lr: 0.000488 2021-03-26 08:53:33,204 epoch 73 - iter 12/25 - loss 2.38512383 - samples/sec: 126.40 - lr: 0.000488 2021-03-26 08:53:34,377 epoch 73 - iter 14/25 - loss 2.35500358 - samples/sec: 109.17 - lr: 0.000488 2021-03-26 08:53:35,339 epoch 73 - iter 16/25 - loss 2.44799797 - samples/sec: 133.37 - lr: 0.000488 2021-03-26 08:53:36,293 epoch 73 - iter 18/25 - loss 2.42572118 - samples/sec: 134.30 - lr: 0.000488 2021-03-26 08:53:37,246 epoch 73 - iter 20/25 - loss 2.40007474 - samples/sec: 134.59 - lr: 0.000488 2021-03-26 08:53:38,217 epoch 73 - iter 22/25 - loss 2.39740131 - samples/sec: 132.04 - lr: 0.000488 2021-03-26 08:53:39,237 epoch 73 - iter 24/25 - loss 2.40759586 - samples/sec: 125.56 - lr: 0.000488 2021-03-26 08:53:39,595 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:53:39,596 EPOCH 73 done: loss 2.4429 - lr 0.0004883 2021-03-26 08:53:40,362 DEV : loss 6.72260856628418 - score 0.9094 2021-03-26 08:53:40,387 BAD EPOCHS (no improvement): 3 2021-03-26 08:53:40,388 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:53:41,387 epoch 74 - iter 2/25 - loss 2.16657722 - samples/sec: 128.27 - lr: 0.000488 2021-03-26 08:53:42,381 epoch 74 - iter 4/25 - loss 2.09330899 - samples/sec: 129.02 - lr: 0.000488 2021-03-26 08:53:43,306 epoch 74 - iter 6/25 - loss 2.12494469 - samples/sec: 138.70 - lr: 0.000488 2021-03-26 08:53:44,271 epoch 74 - iter 8/25 - loss 2.17297184 - samples/sec: 132.92 - lr: 0.000488 2021-03-26 08:53:45,324 epoch 74 - iter 10/25 - loss 2.23438246 - samples/sec: 121.75 - lr: 0.000488 2021-03-26 08:53:46,306 epoch 74 - iter 12/25 - loss 2.32831556 - samples/sec: 130.51 - lr: 0.000488 2021-03-26 08:53:47,318 epoch 74 - iter 14/25 - loss 2.38299196 - samples/sec: 126.63 - lr: 0.000488 2021-03-26 08:53:48,329 epoch 74 - iter 16/25 - loss 2.35490888 - samples/sec: 127.34 - lr: 0.000488 2021-03-26 08:53:49,330 epoch 74 - iter 18/25 - loss 2.33497572 - samples/sec: 128.13 - lr: 0.000488 2021-03-26 08:53:50,309 epoch 74 - iter 20/25 - loss 2.32606256 - samples/sec: 130.97 - lr: 0.000488 2021-03-26 08:53:51,387 epoch 74 - iter 22/25 - loss 2.38837590 - samples/sec: 118.87 - lr: 0.000488 2021-03-26 08:53:52,300 epoch 74 - iter 24/25 - loss 2.42492236 - samples/sec: 140.46 - lr: 0.000488 2021-03-26 08:53:52,709 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:53:52,710 EPOCH 74 done: loss 2.4280 - lr 0.0004883 2021-03-26 08:53:53,450 DEV : loss 6.722847938537598 - score 0.9094 2021-03-26 08:53:53,474 BAD EPOCHS (no improvement): 4 2021-03-26 08:53:53,475 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:53:54,521 epoch 75 - iter 2/25 - loss 2.46175408 - samples/sec: 122.60 - lr: 0.000244 2021-03-26 08:53:55,421 epoch 75 - iter 4/25 - loss 2.41136295 - samples/sec: 142.55 - lr: 0.000244 2021-03-26 08:53:56,511 epoch 75 - iter 6/25 - loss 2.28692702 - samples/sec: 117.55 - lr: 0.000244 2021-03-26 08:53:57,471 epoch 75 - iter 8/25 - loss 2.30924901 - samples/sec: 133.50 - lr: 0.000244 2021-03-26 08:53:58,455 epoch 75 - iter 10/25 - loss 2.30625987 - samples/sec: 130.32 - lr: 0.000244 2021-03-26 08:53:59,578 epoch 75 - iter 12/25 - loss 2.36866377 - samples/sec: 114.07 - lr: 0.000244 2021-03-26 08:54:00,636 epoch 75 - iter 14/25 - loss 2.39279367 - samples/sec: 121.22 - lr: 0.000244 2021-03-26 08:54:01,651 epoch 75 - iter 16/25 - loss 2.32599847 - samples/sec: 126.27 - lr: 0.000244 2021-03-26 08:54:02,626 epoch 75 - iter 18/25 - loss 2.35100826 - samples/sec: 131.52 - lr: 0.000244 2021-03-26 08:54:03,615 epoch 75 - iter 20/25 - loss 2.42343712 - samples/sec: 129.99 - lr: 0.000244 2021-03-26 08:54:04,589 epoch 75 - iter 22/25 - loss 2.39739620 - samples/sec: 131.60 - lr: 0.000244 2021-03-26 08:54:05,609 epoch 75 - iter 24/25 - loss 2.42359165 - samples/sec: 125.70 - lr: 0.000244 2021-03-26 08:54:06,013 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:54:06,014 EPOCH 75 done: loss 2.4093 - lr 0.0002441 2021-03-26 08:54:06,779 DEV : loss 6.722868919372559 - score 0.9094 2021-03-26 08:54:06,797 BAD EPOCHS (no improvement): 1 2021-03-26 08:54:06,797 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:54:07,740 epoch 76 - iter 2/25 - loss 1.97292817 - samples/sec: 136.00 - lr: 0.000244 2021-03-26 08:54:08,727 epoch 76 - iter 4/25 - loss 2.30610174 - samples/sec: 129.98 - lr: 0.000244 2021-03-26 08:54:09,685 epoch 76 - iter 6/25 - loss 2.43610533 - samples/sec: 133.72 - lr: 0.000244 2021-03-26 08:54:10,579 epoch 76 - iter 8/25 - loss 2.45422328 - samples/sec: 143.49 - lr: 0.000244 2021-03-26 08:54:11,519 epoch 76 - iter 10/25 - loss 2.53655019 - samples/sec: 136.48 - lr: 0.000244 2021-03-26 08:54:12,621 epoch 76 - iter 12/25 - loss 2.52940021 - samples/sec: 116.63 - lr: 0.000244 2021-03-26 08:54:13,782 epoch 76 - iter 14/25 - loss 2.54521218 - samples/sec: 110.44 - lr: 0.000244 2021-03-26 08:54:14,836 epoch 76 - iter 16/25 - loss 2.51659589 - samples/sec: 121.67 - lr: 0.000244 2021-03-26 08:54:15,836 epoch 76 - iter 18/25 - loss 2.49805910 - samples/sec: 128.13 - lr: 0.000244 2021-03-26 08:54:16,857 epoch 76 - iter 20/25 - loss 2.47484744 - samples/sec: 125.62 - lr: 0.000244 2021-03-26 08:54:18,039 epoch 76 - iter 22/25 - loss 2.50383779 - samples/sec: 108.43 - lr: 0.000244 2021-03-26 08:54:19,048 epoch 76 - iter 24/25 - loss 2.48512718 - samples/sec: 126.98 - lr: 0.000244 2021-03-26 08:54:19,599 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:54:19,599 EPOCH 76 done: loss 2.4889 - lr 0.0002441 2021-03-26 08:54:20,401 DEV : loss 6.722756862640381 - score 0.9094 2021-03-26 08:54:20,428 BAD EPOCHS (no improvement): 2 2021-03-26 08:54:20,429 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:54:21,570 epoch 77 - iter 2/25 - loss 2.60748661 - samples/sec: 112.30 - lr: 0.000244 2021-03-26 08:54:22,597 epoch 77 - iter 4/25 - loss 2.72534341 - samples/sec: 124.91 - lr: 0.000244 2021-03-26 08:54:23,694 epoch 77 - iter 6/25 - loss 2.57711697 - samples/sec: 116.75 - lr: 0.000244 2021-03-26 08:54:24,640 epoch 77 - iter 8/25 - loss 2.57871893 - samples/sec: 135.53 - lr: 0.000244 2021-03-26 08:54:25,687 epoch 77 - iter 10/25 - loss 2.60255415 - samples/sec: 122.44 - lr: 0.000244 2021-03-26 08:54:26,692 epoch 77 - iter 12/25 - loss 2.64264709 - samples/sec: 127.58 - lr: 0.000244 2021-03-26 08:54:27,695 epoch 77 - iter 14/25 - loss 2.64700404 - samples/sec: 127.73 - lr: 0.000244 2021-03-26 08:54:28,648 epoch 77 - iter 16/25 - loss 2.62158404 - samples/sec: 134.46 - lr: 0.000244 2021-03-26 08:54:29,513 epoch 77 - iter 18/25 - loss 2.54049926 - samples/sec: 148.20 - lr: 0.000244 2021-03-26 08:54:30,385 epoch 77 - iter 20/25 - loss 2.55079408 - samples/sec: 146.97 - lr: 0.000244 2021-03-26 08:54:31,391 epoch 77 - iter 22/25 - loss 2.53056429 - samples/sec: 127.47 - lr: 0.000244 2021-03-26 08:54:32,377 epoch 77 - iter 24/25 - loss 2.48710426 - samples/sec: 129.97 - lr: 0.000244 2021-03-26 08:54:32,814 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:54:32,814 EPOCH 77 done: loss 2.4795 - lr 0.0002441 2021-03-26 08:54:33,578 DEV : loss 6.7229108810424805 - score 0.9094 2021-03-26 08:54:33,596 BAD EPOCHS (no improvement): 3 2021-03-26 08:54:33,597 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:54:34,543 epoch 78 - iter 2/25 - loss 2.53820717 - samples/sec: 135.43 - lr: 0.000244 2021-03-26 08:54:35,576 epoch 78 - iter 4/25 - loss 2.44104677 - samples/sec: 124.18 - lr: 0.000244 2021-03-26 08:54:36,596 epoch 78 - iter 6/25 - loss 2.45857209 - samples/sec: 125.67 - lr: 0.000244 2021-03-26 08:54:37,609 epoch 78 - iter 8/25 - loss 2.51672958 - samples/sec: 126.60 - lr: 0.000244 2021-03-26 08:54:38,658 epoch 78 - iter 10/25 - loss 2.58631538 - samples/sec: 122.16 - lr: 0.000244 2021-03-26 08:54:39,592 epoch 78 - iter 12/25 - loss 2.54622651 - samples/sec: 137.35 - lr: 0.000244 2021-03-26 08:54:40,594 epoch 78 - iter 14/25 - loss 2.48726221 - samples/sec: 127.97 - lr: 0.000244 2021-03-26 08:54:41,568 epoch 78 - iter 16/25 - loss 2.41305912 - samples/sec: 131.63 - lr: 0.000244 2021-03-26 08:54:42,555 epoch 78 - iter 18/25 - loss 2.45293010 - samples/sec: 129.87 - lr: 0.000244 2021-03-26 08:54:43,512 epoch 78 - iter 20/25 - loss 2.43574464 - samples/sec: 133.87 - lr: 0.000244 2021-03-26 08:54:44,494 epoch 78 - iter 22/25 - loss 2.46249009 - samples/sec: 130.58 - lr: 0.000244 2021-03-26 08:54:45,578 epoch 78 - iter 24/25 - loss 2.45119586 - samples/sec: 118.29 - lr: 0.000244 2021-03-26 08:54:46,007 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:54:46,007 EPOCH 78 done: loss 2.4746 - lr 0.0002441 2021-03-26 08:54:46,815 DEV : loss 6.723141670227051 - score 0.9094 2021-03-26 08:54:46,840 BAD EPOCHS (no improvement): 4 2021-03-26 08:54:46,841 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:54:47,790 epoch 79 - iter 2/25 - loss 2.38345897 - samples/sec: 135.18 - lr: 0.000122 2021-03-26 08:54:48,784 epoch 79 - iter 4/25 - loss 2.43685853 - samples/sec: 128.89 - lr: 0.000122 2021-03-26 08:54:49,760 epoch 79 - iter 6/25 - loss 2.47693690 - samples/sec: 131.40 - lr: 0.000122 2021-03-26 08:54:50,768 epoch 79 - iter 8/25 - loss 2.40356702 - samples/sec: 127.29 - lr: 0.000122 2021-03-26 08:54:51,693 epoch 79 - iter 10/25 - loss 2.47259574 - samples/sec: 138.51 - lr: 0.000122 2021-03-26 08:54:52,575 epoch 79 - iter 12/25 - loss 2.50643518 - samples/sec: 145.33 - lr: 0.000122 2021-03-26 08:54:53,494 epoch 79 - iter 14/25 - loss 2.56891007 - samples/sec: 139.52 - lr: 0.000122 2021-03-26 08:54:54,511 epoch 79 - iter 16/25 - loss 2.45421075 - samples/sec: 126.04 - lr: 0.000122 2021-03-26 08:54:55,512 epoch 79 - iter 18/25 - loss 2.46415308 - samples/sec: 127.97 - lr: 0.000122 2021-03-26 08:54:56,580 epoch 79 - iter 20/25 - loss 2.48957347 - samples/sec: 120.02 - lr: 0.000122 2021-03-26 08:54:57,569 epoch 79 - iter 22/25 - loss 2.53055824 - samples/sec: 129.73 - lr: 0.000122 2021-03-26 08:54:58,509 epoch 79 - iter 24/25 - loss 2.53782905 - samples/sec: 136.41 - lr: 0.000122 2021-03-26 08:54:58,882 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:54:58,882 EPOCH 79 done: loss 2.5240 - lr 0.0001221 2021-03-26 08:54:59,619 DEV : loss 6.723060131072998 - score 0.9094 2021-03-26 08:54:59,643 BAD EPOCHS (no improvement): 1 2021-03-26 08:54:59,644 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:00,590 epoch 80 - iter 2/25 - loss 2.24184740 - samples/sec: 135.56 - lr: 0.000122 2021-03-26 08:55:01,578 epoch 80 - iter 4/25 - loss 2.30228198 - samples/sec: 129.73 - lr: 0.000122 2021-03-26 08:55:02,567 epoch 80 - iter 6/25 - loss 2.19478816 - samples/sec: 129.71 - lr: 0.000122 2021-03-26 08:55:03,494 epoch 80 - iter 8/25 - loss 2.23243473 - samples/sec: 138.34 - lr: 0.000122 2021-03-26 08:55:04,476 epoch 80 - iter 10/25 - loss 2.29499611 - samples/sec: 130.42 - lr: 0.000122 2021-03-26 08:55:05,464 epoch 80 - iter 12/25 - loss 2.33867406 - samples/sec: 129.75 - lr: 0.000122 2021-03-26 08:55:06,444 epoch 80 - iter 14/25 - loss 2.35571365 - samples/sec: 130.84 - lr: 0.000122 2021-03-26 08:55:07,476 epoch 80 - iter 16/25 - loss 2.43696856 - samples/sec: 124.27 - lr: 0.000122 2021-03-26 08:55:08,400 epoch 80 - iter 18/25 - loss 2.44001334 - samples/sec: 138.73 - lr: 0.000122 2021-03-26 08:55:09,364 epoch 80 - iter 20/25 - loss 2.48000899 - samples/sec: 133.03 - lr: 0.000122 2021-03-26 08:55:10,349 epoch 80 - iter 22/25 - loss 2.50344019 - samples/sec: 130.23 - lr: 0.000122 2021-03-26 08:55:11,333 epoch 80 - iter 24/25 - loss 2.47424016 - samples/sec: 130.27 - lr: 0.000122 2021-03-26 08:55:11,724 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:11,725 EPOCH 80 done: loss 2.4586 - lr 0.0001221 2021-03-26 08:55:12,493 DEV : loss 6.722960472106934 - score 0.9094 2021-03-26 08:55:12,511 BAD EPOCHS (no improvement): 2 2021-03-26 08:55:12,512 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:13,442 epoch 81 - iter 2/25 - loss 2.23566240 - samples/sec: 137.85 - lr: 0.000122 2021-03-26 08:55:14,385 epoch 81 - iter 4/25 - loss 2.49205181 - samples/sec: 136.14 - lr: 0.000122 2021-03-26 08:55:15,306 epoch 81 - iter 6/25 - loss 2.55151534 - samples/sec: 139.09 - lr: 0.000122 2021-03-26 08:55:16,308 epoch 81 - iter 8/25 - loss 2.50896537 - samples/sec: 128.08 - lr: 0.000122 2021-03-26 08:55:17,513 epoch 81 - iter 10/25 - loss 2.48709247 - samples/sec: 106.37 - lr: 0.000122 2021-03-26 08:55:18,463 epoch 81 - iter 12/25 - loss 2.54523623 - samples/sec: 134.97 - lr: 0.000122 2021-03-26 08:55:19,496 epoch 81 - iter 14/25 - loss 2.48191902 - samples/sec: 124.10 - lr: 0.000122 2021-03-26 08:55:20,493 epoch 81 - iter 16/25 - loss 2.55448468 - samples/sec: 128.46 - lr: 0.000122 2021-03-26 08:55:21,492 epoch 81 - iter 18/25 - loss 2.46937637 - samples/sec: 128.27 - lr: 0.000122 2021-03-26 08:55:22,468 epoch 81 - iter 20/25 - loss 2.45314644 - samples/sec: 131.29 - lr: 0.000122 2021-03-26 08:55:23,382 epoch 81 - iter 22/25 - loss 2.46111201 - samples/sec: 140.31 - lr: 0.000122 2021-03-26 08:55:24,310 epoch 81 - iter 24/25 - loss 2.49231903 - samples/sec: 138.22 - lr: 0.000122 2021-03-26 08:55:24,770 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:24,771 EPOCH 81 done: loss 2.4980 - lr 0.0001221 2021-03-26 08:55:25,498 DEV : loss 6.722949981689453 - score 0.9094 2021-03-26 08:55:25,520 BAD EPOCHS (no improvement): 3 2021-03-26 08:55:25,521 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:26,455 epoch 82 - iter 2/25 - loss 2.47193301 - samples/sec: 137.23 - lr: 0.000122 2021-03-26 08:55:27,478 epoch 82 - iter 4/25 - loss 2.63737261 - samples/sec: 125.33 - lr: 0.000122 2021-03-26 08:55:28,439 epoch 82 - iter 6/25 - loss 2.50588151 - samples/sec: 133.44 - lr: 0.000122 2021-03-26 08:55:29,414 epoch 82 - iter 8/25 - loss 2.43969005 - samples/sec: 131.43 - lr: 0.000122 2021-03-26 08:55:30,411 epoch 82 - iter 10/25 - loss 2.53163679 - samples/sec: 128.63 - lr: 0.000122 2021-03-26 08:55:31,489 epoch 82 - iter 12/25 - loss 2.43317682 - samples/sec: 118.98 - lr: 0.000122 2021-03-26 08:55:32,472 epoch 82 - iter 14/25 - loss 2.46037332 - samples/sec: 130.34 - lr: 0.000122 2021-03-26 08:55:33,534 epoch 82 - iter 16/25 - loss 2.42180952 - samples/sec: 120.75 - lr: 0.000122 2021-03-26 08:55:34,533 epoch 82 - iter 18/25 - loss 2.50896418 - samples/sec: 128.27 - lr: 0.000122 2021-03-26 08:55:35,466 epoch 82 - iter 20/25 - loss 2.50509016 - samples/sec: 137.43 - lr: 0.000122 2021-03-26 08:55:36,436 epoch 82 - iter 22/25 - loss 2.48789684 - samples/sec: 132.13 - lr: 0.000122 2021-03-26 08:55:37,362 epoch 82 - iter 24/25 - loss 2.45878165 - samples/sec: 138.38 - lr: 0.000122 2021-03-26 08:55:37,842 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:37,843 EPOCH 82 done: loss 2.4828 - lr 0.0001221 2021-03-26 08:55:38,598 DEV : loss 6.72271728515625 - score 0.9094 2021-03-26 08:55:38,619 BAD EPOCHS (no improvement): 4 2021-03-26 08:55:38,620 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:38,621 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:38,621 learning rate too small - quitting training! 2021-03-26 08:55:38,621 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:47,863 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:47,864 Testing using best model ... 2021-03-26 08:55:47,864 loading file /home/tmp/megahedm/models/multipos/multipos_UDMADAR_4Diale-LEV_EGY_GLF_MGR__fasttext_flairbwfw__64__0.5_202103260834/best-model.pt 2021-03-26 08:55:55,013 0.901 2021-03-26 08:55:55,014 Results: - F-score (micro): 0.8974 - F-score (macro): 0.5188 - Accuracy (incl. no class): 0.901 By class: precision recall f1-score support INTJ 0.8182 0.9000 0.8571 10 NOUN 0.9009 0.9402 0.9201 435 NUM 0.9524 0.8333 0.8889 24 ADJ 0.8762 0.7603 0.8142 121 ADP 0.9903 0.9623 0.9761 106 CCONJ 0.9600 0.9730 0.9664 74 PROPN 0.9333 0.9333 0.9333 15 ADV 0.9135 0.8051 0.8559 118 VERB 0.8852 0.9231 0.9038 117 PRON 0.9620 0.9465 0.9542 187 SCONJ 0.8571 0.9474 0.9000 19 PART 0.9350 0.9791 0.9565 191 DET 0.9348 0.9149 0.9247 47 PUNCT 1.0000 1.0000 1.0000 35 AUX 0.9286 0.9811 0.9541 53 MENTION 0.9231 1.0000 0.9600 12 V 0.8571 0.8780 0.8675 82 FUT_PART+V+PREP+PRON 1.0000 0.0000 0.0000 1 PROG_PART+V+PRON+PREP+PRON 0.0000 1.0000 0.0000 0 ADJ+NSUFF 0.6111 0.8462 0.7097 26 NOUN+NSUFF 0.8182 0.8438 0.8308 64 PREP+PRON 0.9565 0.9565 0.9565 23 PUNC 0.9941 1.0000 0.9971 169 EOS 1.0000 1.0000 1.0000 70 NOUN+PRON 0.6986 0.8500 0.7669 60 V+PRON 0.7258 0.8036 0.7627 56 PART+PRON 1.0000 0.9474 0.9730 19 PROG_PART+V 0.8333 0.9302 0.8791 43 DET+NOUN 0.9625 1.0000 0.9809 77 NOUN+NSUFF+PRON 0.9091 0.7143 0.8000 14 PROG_PART+V+PRON 0.7083 0.9444 0.8095 18 PREP+NOUN+NSUFF 0.6667 0.4000 0.5000 5 NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 3 CONJ 0.9722 1.0000 0.9859 35 V+PRON+PRON 0.6364 0.5833 0.6087 12 FOREIGN 0.6667 0.6667 0.6667 3 PREP+NOUN 0.6316 0.7500 0.6857 16 DET+NOUN+NSUFF 0.9000 0.9310 0.9153 29 DET+ADJ+NSUFF 1.0000 0.5714 0.7273 7 CONJ+PRON 1.0000 0.8750 0.9333 8 NOUN+CASE 0.0000 0.0000 0.0000 2 DET+ADJ 1.0000 0.6667 0.8000 6 PREP 1.0000 0.9718 0.9857 71 CONJ+FUT_PART+V 0.0000 0.0000 0.0000 1 CONJ+V 0.6667 0.7500 0.7059 8 FUT_PART 1.0000 1.0000 1.0000 2 ADJ+PRON 1.0000 0.0000 0.0000 8 CONJ+PREP+NOUN+PRON 1.0000 0.0000 0.0000 1 CONJ+NOUN+PRON 0.3750 1.0000 0.5455 3 PART+ADJ 1.0000 0.0000 0.0000 1 PART+NOUN 0.5000 1.0000 0.6667 1 CONJ+PREP+NOUN 1.0000 0.0000 0.0000 1 CONJ+NOUN 0.7000 0.7778 0.7368 9 URL 1.0000 1.0000 1.0000 3 CONJ+FUT_PART 1.0000 0.0000 0.0000 1 FUT_PART+V 0.8571 0.6000 0.7059 10 PREP+NOUN+NSUFF+NSUFF 1.0000 0.0000 0.0000 1 HASH 1.0000 0.9412 0.9697 17 ADJ+PREP+PRON 1.0000 0.0000 0.0000 3 PREP+NOUN+PRON 0.0000 0.0000 0.0000 1 EMOT 1.0000 0.8889 0.9412 18 CONJ+PREP 1.0000 0.7500 0.8571 4 PREP+DET+NOUN+NSUFF 1.0000 0.7500 0.8571 4 PRON+DET+NOUN+NSUFF 0.0000 1.0000 0.0000 0 V+PREP+PRON 1.0000 0.0000 0.0000 5 V+PRON+PREP+PRON 0.0000 1.0000 0.0000 0 CONJ+NOUN+NSUFF 0.5000 0.5000 0.5000 2 V+NEG_PART 1.0000 0.0000 0.0000 2 PREP+DET+NOUN 0.9091 1.0000 0.9524 10 PREP+V 1.0000 0.0000 0.0000 2 CONJ+PART 1.0000 0.7778 0.8750 9 CONJ+V+PRON 1.0000 1.0000 1.0000 5 PROG_PART+V+PREP+PRON 1.0000 0.5000 0.6667 2 PREP+NOUN+NSUFF+PRON 1.0000 1.0000 1.0000 1 ADJ+CASE 1.0000 0.0000 0.0000 1 PART+NOUN+PRON 1.0000 1.0000 1.0000 1 PART+V 1.0000 0.0000 0.0000 3 PART+V+PRON 0.0000 1.0000 0.0000 0 FUT_PART+V+PRON 0.0000 1.0000 0.0000 0 FUT_PART+V+PRON+PRON 1.0000 0.0000 0.0000 1 CONJ+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+V+PRON+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+V+PREP+PRON 0.0000 1.0000 0.0000 0 CONJ+DET+NOUN+NSUFF 1.0000 0.0000 0.0000 1 CONJ+DET+NOUN 0.6667 1.0000 0.8000 2 CONJ+PREP+DET+NOUN 1.0000 1.0000 1.0000 1 PREP+PART 1.0000 0.0000 0.0000 2 PART+V+PRON+NEG_PART 0.3333 0.3333 0.3333 3 PART+V+NEG_PART 0.3333 0.5000 0.4000 2 PART+PREP+NEG_PART 1.0000 1.0000 1.0000 3 PART+PROG_PART+V+NEG_PART 1.0000 0.3333 0.5000 3 PREP+DET+NOUN+NSUFF+PREP+PRON 1.0000 0.0000 0.0000 1 PREP+PRON+DET+NOUN 0.0000 1.0000 0.0000 0 PART+NSUFF 1.0000 0.0000 0.0000 1 CONJ+PROG_PART+V+PRON 1.0000 1.0000 1.0000 1 PART+PREP+PRON 1.0000 0.0000 0.0000 1 CONJ+PART+PREP 1.0000 0.0000 0.0000 1 NUM+NSUFF 0.6667 0.6667 0.6667 3 CONJ+PART+V+PRON+NEG_PART 1.0000 1.0000 1.0000 1 PART+NOUN+NEG_PART 1.0000 1.0000 1.0000 1 CONJ+ADJ+NSUFF 1.0000 0.0000 0.0000 1 PREP+ADJ 1.0000 0.0000 0.0000 1 ADJ+NSUFF+PRON 1.0000 0.0000 0.0000 2 CONJ+PROG_PART+V 1.0000 0.0000 0.0000 1 CONJ+PART+PROG_PART+V+PREP+PRON+NEG_PART 1.0000 0.0000 0.0000 1 CONJ+PART+PREP+PRON+NEG_PART 0.0000 1.0000 0.0000 0 PREP+PART+PRON 1.0000 0.0000 0.0000 1 CONJ+ADV+NSUFF 1.0000 0.0000 0.0000 1 CONJ+ADV 0.0000 1.0000 0.0000 0 PART+NOUN+PRON+NEG_PART 0.0000 1.0000 0.0000 0 CONJ+ADJ 1.0000 1.0000 1.0000 1 micro avg 0.8977 0.8971 0.8974 2662 macro avg 0.8002 0.6126 0.5188 2662 weighted avg 0.9091 0.8971 0.8912 2662 2021-03-26 08:55:55,015 ---------------------------------------------------------------------------------------------------- 2021-03-26 08:55:55,015 ----------------------------------------------------------------------------------------------------