2023-04-06 05:00:57,477 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:00:57,477 Model: "SequenceTagger( (embeddings): StackedEmbeddings( (list_embedding_0): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.25, inplace=False) (encoder): Embedding(275, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=275, bias=True) ) ) (list_embedding_1): FlairEmbeddings( (lm): LanguageModel( (drop): Dropout(p=0.25, inplace=False) (encoder): Embedding(275, 100) (rnn): LSTM(100, 2048) (decoder): Linear(in_features=2048, out_features=275, bias=True) ) ) ) (word_dropout): WordDropout(p=0.05) (locked_dropout): LockedDropout(p=0.5) (embedding2nn): Linear(in_features=4096, out_features=4096, bias=True) (rnn): LSTM(4096, 256, batch_first=True, bidirectional=True) (linear): Linear(in_features=512, out_features=23, bias=True) (loss_function): ViterbiLoss() (crf): CRF() )" 2023-04-06 05:00:57,479 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:00:57,480 Corpus: "Corpus: 3200 train + 401 dev + 401 test sentences" 2023-04-06 05:00:57,480 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:00:57,480 Parameters: 2023-04-06 05:00:57,480 - learning_rate: "0.100000" 2023-04-06 05:00:57,481 - mini_batch_size: "32" 2023-04-06 05:00:57,481 - patience: "3" 2023-04-06 05:00:57,481 - anneal_factor: "0.5" 2023-04-06 05:00:57,481 - max_epochs: "10" 2023-04-06 05:00:57,482 - shuffle: "True" 2023-04-06 05:00:57,482 - train_with_dev: "False" 2023-04-06 05:00:57,482 - batch_growth_annealing: "False" 2023-04-06 05:00:57,483 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:00:57,483 Model training base path: "db/kgc_models/flair-spec-cons/trained_model" 2023-04-06 05:00:57,483 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:00:57,483 Device: cuda:0 2023-04-06 05:00:57,484 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:00:57,484 Embeddings storage mode: gpu 2023-04-06 05:00:57,484 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:00:58,191 epoch 1 - iter 10/100 - loss 2.54841670 - samples/sec: 453.70 - lr: 0.100000 2023-04-06 05:00:58,981 epoch 1 - iter 20/100 - loss 2.33677462 - samples/sec: 406.16 - lr: 0.100000 2023-04-06 05:00:59,670 epoch 1 - iter 30/100 - loss 2.15419054 - samples/sec: 464.94 - lr: 0.100000 2023-04-06 05:01:00,342 epoch 1 - iter 40/100 - loss 1.98991539 - samples/sec: 477.32 - lr: 0.100000 2023-04-06 05:01:01,018 epoch 1 - iter 50/100 - loss 1.89150122 - samples/sec: 474.61 - lr: 0.100000 2023-04-06 05:01:01,710 epoch 1 - iter 60/100 - loss 1.80637456 - samples/sec: 463.46 - lr: 0.100000 2023-04-06 05:01:02,411 epoch 1 - iter 70/100 - loss 1.74580158 - samples/sec: 457.25 - lr: 0.100000 2023-04-06 05:01:03,155 epoch 1 - iter 80/100 - loss 1.67068108 - samples/sec: 430.69 - lr: 0.100000 2023-04-06 05:01:03,906 epoch 1 - iter 90/100 - loss 1.61186574 - samples/sec: 427.18 - lr: 0.100000 2023-04-06 05:01:04,621 epoch 1 - iter 100/100 - loss 1.57518982 - samples/sec: 448.53 - lr: 0.100000 2023-04-06 05:01:04,621 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:04,622 EPOCH 1 done: loss 1.5752 - lr 0.100000 2023-04-06 05:01:05,978 Evaluating as a multi-label problem: False 2023-04-06 05:01:05,992 DEV : loss 0.9883462190628052 - f1-score (micro avg) 0.5583 2023-04-06 05:01:05,998 BAD EPOCHS (no improvement): 0 2023-04-06 05:01:06,000 saving best model 2023-04-06 05:01:07,645 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:07,958 epoch 2 - iter 10/100 - loss 1.07820137 - samples/sec: 1030.85 - lr: 0.100000 2023-04-06 05:01:08,263 epoch 2 - iter 20/100 - loss 1.04921650 - samples/sec: 1053.15 - lr: 0.100000 2023-04-06 05:01:08,548 epoch 2 - iter 30/100 - loss 1.01492418 - samples/sec: 1130.08 - lr: 0.100000 2023-04-06 05:01:08,880 epoch 2 - iter 40/100 - loss 1.01476922 - samples/sec: 969.31 - lr: 0.100000 2023-04-06 05:01:09,162 epoch 2 - iter 50/100 - loss 0.98812696 - samples/sec: 1137.46 - lr: 0.100000 2023-04-06 05:01:09,492 epoch 2 - iter 60/100 - loss 0.96852050 - samples/sec: 974.46 - lr: 0.100000 2023-04-06 05:01:09,771 epoch 2 - iter 70/100 - loss 0.96147093 - samples/sec: 1155.02 - lr: 0.100000 2023-04-06 05:01:10,078 epoch 2 - iter 80/100 - loss 0.94764571 - samples/sec: 1046.79 - lr: 0.100000 2023-04-06 05:01:10,380 epoch 2 - iter 90/100 - loss 0.93687541 - samples/sec: 1063.94 - lr: 0.100000 2023-04-06 05:01:10,680 epoch 2 - iter 100/100 - loss 0.92128929 - samples/sec: 1074.48 - lr: 0.100000 2023-04-06 05:01:10,681 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:10,682 EPOCH 2 done: loss 0.9213 - lr 0.100000 2023-04-06 05:01:11,480 Evaluating as a multi-label problem: False 2023-04-06 05:01:11,493 DEV : loss 0.6865214705467224 - f1-score (micro avg) 0.6667 2023-04-06 05:01:11,501 BAD EPOCHS (no improvement): 0 2023-04-06 05:01:11,502 saving best model 2023-04-06 05:01:13,122 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:13,415 epoch 3 - iter 10/100 - loss 0.80292226 - samples/sec: 1098.71 - lr: 0.100000 2023-04-06 05:01:13,703 epoch 3 - iter 20/100 - loss 0.80121644 - samples/sec: 1116.92 - lr: 0.100000 2023-04-06 05:01:14,000 epoch 3 - iter 30/100 - loss 0.78842391 - samples/sec: 1082.53 - lr: 0.100000 2023-04-06 05:01:14,300 epoch 3 - iter 40/100 - loss 0.78760832 - samples/sec: 1073.66 - lr: 0.100000 2023-04-06 05:01:14,624 epoch 3 - iter 50/100 - loss 0.78730520 - samples/sec: 991.17 - lr: 0.100000 2023-04-06 05:01:14,931 epoch 3 - iter 60/100 - loss 0.77427673 - samples/sec: 1049.45 - lr: 0.100000 2023-04-06 05:01:15,236 epoch 3 - iter 70/100 - loss 0.76768125 - samples/sec: 1054.20 - lr: 0.100000 2023-04-06 05:01:15,519 epoch 3 - iter 80/100 - loss 0.75606934 - samples/sec: 1138.62 - lr: 0.100000 2023-04-06 05:01:15,821 epoch 3 - iter 90/100 - loss 0.75874242 - samples/sec: 1064.79 - lr: 0.100000 2023-04-06 05:01:16,109 epoch 3 - iter 100/100 - loss 0.74525913 - samples/sec: 1116.10 - lr: 0.100000 2023-04-06 05:01:16,110 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:16,110 EPOCH 3 done: loss 0.7453 - lr 0.100000 2023-04-06 05:01:16,887 Evaluating as a multi-label problem: False 2023-04-06 05:01:16,899 DEV : loss 0.614136278629303 - f1-score (micro avg) 0.7101 2023-04-06 05:01:16,906 BAD EPOCHS (no improvement): 0 2023-04-06 05:01:16,908 saving best model 2023-04-06 05:01:18,556 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:18,835 epoch 4 - iter 10/100 - loss 0.64661913 - samples/sec: 1153.86 - lr: 0.100000 2023-04-06 05:01:19,137 epoch 4 - iter 20/100 - loss 0.62497328 - samples/sec: 1065.76 - lr: 0.100000 2023-04-06 05:01:19,419 epoch 4 - iter 30/100 - loss 0.63771202 - samples/sec: 1143.26 - lr: 0.100000 2023-04-06 05:01:19,759 epoch 4 - iter 40/100 - loss 0.64536114 - samples/sec: 944.68 - lr: 0.100000 2023-04-06 05:01:20,029 epoch 4 - iter 50/100 - loss 0.65670237 - samples/sec: 1189.03 - lr: 0.100000 2023-04-06 05:01:20,327 epoch 4 - iter 60/100 - loss 0.65096773 - samples/sec: 1079.77 - lr: 0.100000 2023-04-06 05:01:20,611 epoch 4 - iter 70/100 - loss 0.64386307 - samples/sec: 1132.05 - lr: 0.100000 2023-04-06 05:01:20,903 epoch 4 - iter 80/100 - loss 0.64342225 - samples/sec: 1102.42 - lr: 0.100000 2023-04-06 05:01:21,199 epoch 4 - iter 90/100 - loss 0.64639085 - samples/sec: 1088.24 - lr: 0.100000 2023-04-06 05:01:21,486 epoch 4 - iter 100/100 - loss 0.64023060 - samples/sec: 1119.16 - lr: 0.100000 2023-04-06 05:01:21,487 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:21,487 EPOCH 4 done: loss 0.6402 - lr 0.100000 2023-04-06 05:01:22,479 Evaluating as a multi-label problem: False 2023-04-06 05:01:22,490 DEV : loss 0.5573540925979614 - f1-score (micro avg) 0.7454 2023-04-06 05:01:22,497 BAD EPOCHS (no improvement): 0 2023-04-06 05:01:22,498 saving best model 2023-04-06 05:01:24,161 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:24,456 epoch 5 - iter 10/100 - loss 0.61543932 - samples/sec: 1093.46 - lr: 0.100000 2023-04-06 05:01:24,754 epoch 5 - iter 20/100 - loss 0.58400124 - samples/sec: 1079.67 - lr: 0.100000 2023-04-06 05:01:25,064 epoch 5 - iter 30/100 - loss 0.56852493 - samples/sec: 1038.59 - lr: 0.100000 2023-04-06 05:01:25,358 epoch 5 - iter 40/100 - loss 0.57995167 - samples/sec: 1094.36 - lr: 0.100000 2023-04-06 05:01:25,683 epoch 5 - iter 50/100 - loss 0.57421334 - samples/sec: 987.87 - lr: 0.100000 2023-04-06 05:01:25,987 epoch 5 - iter 60/100 - loss 0.56522019 - samples/sec: 1059.99 - lr: 0.100000 2023-04-06 05:01:26,284 epoch 5 - iter 70/100 - loss 0.56915244 - samples/sec: 1082.58 - lr: 0.100000 2023-04-06 05:01:26,587 epoch 5 - iter 80/100 - loss 0.56741243 - samples/sec: 1060.07 - lr: 0.100000 2023-04-06 05:01:26,873 epoch 5 - iter 90/100 - loss 0.56403810 - samples/sec: 1122.34 - lr: 0.100000 2023-04-06 05:01:27,162 epoch 5 - iter 100/100 - loss 0.56619930 - samples/sec: 1116.38 - lr: 0.100000 2023-04-06 05:01:27,163 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:27,163 EPOCH 5 done: loss 0.5662 - lr 0.100000 2023-04-06 05:01:27,972 Evaluating as a multi-label problem: False 2023-04-06 05:01:27,984 DEV : loss 0.4954551160335541 - f1-score (micro avg) 0.7711 2023-04-06 05:01:27,992 BAD EPOCHS (no improvement): 0 2023-04-06 05:01:27,994 saving best model 2023-04-06 05:01:29,618 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:29,920 epoch 6 - iter 10/100 - loss 0.51276362 - samples/sec: 1067.10 - lr: 0.100000 2023-04-06 05:01:30,220 epoch 6 - iter 20/100 - loss 0.51676351 - samples/sec: 1074.30 - lr: 0.100000 2023-04-06 05:01:30,528 epoch 6 - iter 30/100 - loss 0.51920724 - samples/sec: 1043.78 - lr: 0.100000 2023-04-06 05:01:30,844 epoch 6 - iter 40/100 - loss 0.53617170 - samples/sec: 1016.64 - lr: 0.100000 2023-04-06 05:01:31,157 epoch 6 - iter 50/100 - loss 0.52303169 - samples/sec: 1029.79 - lr: 0.100000 2023-04-06 05:01:31,472 epoch 6 - iter 60/100 - loss 0.52088512 - samples/sec: 1019.45 - lr: 0.100000 2023-04-06 05:01:31,875 epoch 6 - iter 70/100 - loss 0.51524863 - samples/sec: 797.62 - lr: 0.100000 2023-04-06 05:01:32,197 epoch 6 - iter 80/100 - loss 0.52052504 - samples/sec: 999.15 - lr: 0.100000 2023-04-06 05:01:32,524 epoch 6 - iter 90/100 - loss 0.51383911 - samples/sec: 984.59 - lr: 0.100000 2023-04-06 05:01:32,835 epoch 6 - iter 100/100 - loss 0.50979660 - samples/sec: 1039.46 - lr: 0.100000 2023-04-06 05:01:32,836 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:32,836 EPOCH 6 done: loss 0.5098 - lr 0.100000 2023-04-06 05:01:33,785 Evaluating as a multi-label problem: False 2023-04-06 05:01:33,799 DEV : loss 0.45735692977905273 - f1-score (micro avg) 0.7692 2023-04-06 05:01:33,807 BAD EPOCHS (no improvement): 1 2023-04-06 05:01:33,812 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:34,146 epoch 7 - iter 10/100 - loss 0.48405916 - samples/sec: 964.88 - lr: 0.100000 2023-04-06 05:01:34,473 epoch 7 - iter 20/100 - loss 0.46699604 - samples/sec: 986.48 - lr: 0.100000 2023-04-06 05:01:34,807 epoch 7 - iter 30/100 - loss 0.47639370 - samples/sec: 961.54 - lr: 0.100000 2023-04-06 05:01:35,134 epoch 7 - iter 40/100 - loss 0.48267184 - samples/sec: 983.20 - lr: 0.100000 2023-04-06 05:01:35,468 epoch 7 - iter 50/100 - loss 0.47247635 - samples/sec: 962.07 - lr: 0.100000 2023-04-06 05:01:35,772 epoch 7 - iter 60/100 - loss 0.47543941 - samples/sec: 1061.04 - lr: 0.100000 2023-04-06 05:01:36,101 epoch 7 - iter 70/100 - loss 0.47814133 - samples/sec: 977.64 - lr: 0.100000 2023-04-06 05:01:36,440 epoch 7 - iter 80/100 - loss 0.47698574 - samples/sec: 948.03 - lr: 0.100000 2023-04-06 05:01:36,747 epoch 7 - iter 90/100 - loss 0.47987035 - samples/sec: 1047.95 - lr: 0.100000 2023-04-06 05:01:37,066 epoch 7 - iter 100/100 - loss 0.47570336 - samples/sec: 1008.92 - lr: 0.100000 2023-04-06 05:01:37,067 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:37,070 EPOCH 7 done: loss 0.4757 - lr 0.100000 2023-04-06 05:01:37,941 Evaluating as a multi-label problem: False 2023-04-06 05:01:37,952 DEV : loss 0.44222691655158997 - f1-score (micro avg) 0.7888 2023-04-06 05:01:37,960 BAD EPOCHS (no improvement): 0 2023-04-06 05:01:37,961 saving best model 2023-04-06 05:01:40,394 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:40,854 epoch 8 - iter 10/100 - loss 0.45383867 - samples/sec: 700.65 - lr: 0.100000 2023-04-06 05:01:41,173 epoch 8 - iter 20/100 - loss 0.43853387 - samples/sec: 1008.63 - lr: 0.100000 2023-04-06 05:01:41,480 epoch 8 - iter 30/100 - loss 0.44842076 - samples/sec: 1045.92 - lr: 0.100000 2023-04-06 05:01:41,815 epoch 8 - iter 40/100 - loss 0.44778312 - samples/sec: 962.63 - lr: 0.100000 2023-04-06 05:01:42,123 epoch 8 - iter 50/100 - loss 0.45261274 - samples/sec: 1044.30 - lr: 0.100000 2023-04-06 05:01:42,467 epoch 8 - iter 60/100 - loss 0.45202269 - samples/sec: 932.97 - lr: 0.100000 2023-04-06 05:01:42,766 epoch 8 - iter 70/100 - loss 0.44615702 - samples/sec: 1078.30 - lr: 0.100000 2023-04-06 05:01:43,090 epoch 8 - iter 80/100 - loss 0.44471005 - samples/sec: 990.40 - lr: 0.100000 2023-04-06 05:01:43,390 epoch 8 - iter 90/100 - loss 0.44290559 - samples/sec: 1074.92 - lr: 0.100000 2023-04-06 05:01:43,716 epoch 8 - iter 100/100 - loss 0.44319155 - samples/sec: 984.69 - lr: 0.100000 2023-04-06 05:01:43,717 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:43,720 EPOCH 8 done: loss 0.4432 - lr 0.100000 2023-04-06 05:01:44,604 Evaluating as a multi-label problem: False 2023-04-06 05:01:44,615 DEV : loss 0.4376998245716095 - f1-score (micro avg) 0.7692 2023-04-06 05:01:44,623 BAD EPOCHS (no improvement): 1 2023-04-06 05:01:44,625 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:44,962 epoch 9 - iter 10/100 - loss 0.38774550 - samples/sec: 954.84 - lr: 0.100000 2023-04-06 05:01:45,286 epoch 9 - iter 20/100 - loss 0.41776223 - samples/sec: 991.11 - lr: 0.100000 2023-04-06 05:01:45,599 epoch 9 - iter 30/100 - loss 0.42511250 - samples/sec: 1028.88 - lr: 0.100000 2023-04-06 05:01:45,945 epoch 9 - iter 40/100 - loss 0.42112254 - samples/sec: 930.71 - lr: 0.100000 2023-04-06 05:01:46,274 epoch 9 - iter 50/100 - loss 0.42444511 - samples/sec: 976.78 - lr: 0.100000 2023-04-06 05:01:46,603 epoch 9 - iter 60/100 - loss 0.42389968 - samples/sec: 977.72 - lr: 0.100000 2023-04-06 05:01:46,926 epoch 9 - iter 70/100 - loss 0.41802363 - samples/sec: 996.06 - lr: 0.100000 2023-04-06 05:01:47,281 epoch 9 - iter 80/100 - loss 0.41442777 - samples/sec: 906.83 - lr: 0.100000 2023-04-06 05:01:47,611 epoch 9 - iter 90/100 - loss 0.41460733 - samples/sec: 975.57 - lr: 0.100000 2023-04-06 05:01:48,003 epoch 9 - iter 100/100 - loss 0.41394752 - samples/sec: 820.27 - lr: 0.100000 2023-04-06 05:01:48,005 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:48,011 EPOCH 9 done: loss 0.4139 - lr 0.100000 2023-04-06 05:01:48,916 Evaluating as a multi-label problem: False 2023-04-06 05:01:48,928 DEV : loss 0.4502558410167694 - f1-score (micro avg) 0.7756 2023-04-06 05:01:48,936 BAD EPOCHS (no improvement): 2 2023-04-06 05:01:48,940 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:49,272 epoch 10 - iter 10/100 - loss 0.37656011 - samples/sec: 980.38 - lr: 0.100000 2023-04-06 05:01:49,616 epoch 10 - iter 20/100 - loss 0.39638115 - samples/sec: 936.73 - lr: 0.100000 2023-04-06 05:01:49,952 epoch 10 - iter 30/100 - loss 0.39364339 - samples/sec: 956.19 - lr: 0.100000 2023-04-06 05:01:50,279 epoch 10 - iter 40/100 - loss 0.39287102 - samples/sec: 984.94 - lr: 0.100000 2023-04-06 05:01:50,603 epoch 10 - iter 50/100 - loss 0.39715304 - samples/sec: 992.09 - lr: 0.100000 2023-04-06 05:01:50,938 epoch 10 - iter 60/100 - loss 0.38995911 - samples/sec: 961.94 - lr: 0.100000 2023-04-06 05:01:51,244 epoch 10 - iter 70/100 - loss 0.39104831 - samples/sec: 1050.41 - lr: 0.100000 2023-04-06 05:01:51,569 epoch 10 - iter 80/100 - loss 0.39384103 - samples/sec: 988.09 - lr: 0.100000 2023-04-06 05:01:51,891 epoch 10 - iter 90/100 - loss 0.39865212 - samples/sec: 1000.92 - lr: 0.100000 2023-04-06 05:01:52,221 epoch 10 - iter 100/100 - loss 0.40034652 - samples/sec: 975.07 - lr: 0.100000 2023-04-06 05:01:52,222 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:52,222 EPOCH 10 done: loss 0.4003 - lr 0.100000 2023-04-06 05:01:53,082 Evaluating as a multi-label problem: False 2023-04-06 05:01:53,094 DEV : loss 0.4299003481864929 - f1-score (micro avg) 0.7812 2023-04-06 05:01:53,102 BAD EPOCHS (no improvement): 3 2023-04-06 05:01:58,853 ---------------------------------------------------------------------------------------------------- 2023-04-06 05:01:58,855 loading file db/kgc_models/flair-spec-cons/trained_model/best-model.pt 2023-04-06 05:02:08,102 SequenceTagger predicts: Dictionary with 23 tags: O, S-Item, B-Item, E-Item, I-Item, S-Activity, B-Activity, E-Activity, I-Activity, S-Observation, B-Observation, E-Observation, I-Observation, S-Consumable, B-Consumable, E-Consumable, I-Consumable, S-Specifier, B-Specifier, E-Specifier, I-Specifier, , 2023-04-06 05:02:09,735 Evaluating as a multi-label problem: False 2023-04-06 05:02:09,746 0.7132 0.7412 0.7269 0.5798 2023-04-06 05:02:09,747 Results: - F-score (micro) 0.7269 - F-score (macro) 0.75 - Accuracy 0.5798 By class: precision recall f1-score support Item 0.6489 0.7064 0.6764 361 Activity 0.8324 0.8556 0.8438 180 Observation 0.7108 0.6860 0.6982 172 Consumable 0.7907 0.7727 0.7816 44 Specifier 0.7500 0.7500 0.7500 8 micro avg 0.7132 0.7412 0.7269 765 macro avg 0.7466 0.7541 0.7500 765 weighted avg 0.7152 0.7412 0.7275 765 2023-04-06 05:02:09,747 ----------------------------------------------------------------------------------------------------