Hugging Face's logo

nlp-tlp
/
mwo-ner-test

Token Classification
Flair
PyTorch
English
sequence-tagger-model
Model card Files Community
mwo-ner-test
File size: 13,571 Bytes 
2a843a2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
 
2022-08-25 06:06:10,564 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:10,565 Model: "SequenceTagger(
  (embeddings): StackedEmbeddings(
    (list_embedding_0): FlairEmbeddings(
      (lm): LanguageModel(
        (drop): Dropout(p=0.25, inplace=False)
        (encoder): Embedding(275, 100)
        (rnn): LSTM(100, 2048)
        (decoder): Linear(in_features=2048, out_features=275, bias=True)
      )
    )
    (list_embedding_1): FlairEmbeddings(
      (lm): LanguageModel(
        (drop): Dropout(p=0.25, inplace=False)
        (encoder): Embedding(275, 100)
        (rnn): LSTM(100, 2048)
        (decoder): Linear(in_features=2048, out_features=275, bias=True)
      )
    )
  )
  (word_dropout): WordDropout(p=0.05)
  (locked_dropout): LockedDropout(p=0.5)
  (embedding2nn): Linear(in_features=4096, out_features=4096, bias=True)
  (rnn): LSTM(4096, 256, batch_first=True, bidirectional=True)
  (linear): Linear(in_features=512, out_features=47, bias=True)
  (loss_function): ViterbiLoss()
  (crf): CRF()
)"
2022-08-25 06:06:10,567 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:10,567 Corpus: "Corpus: 100 train + 20 dev + 20 test sentences"
2022-08-25 06:06:10,567 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:10,568 Parameters:
2022-08-25 06:06:10,568  - learning_rate: "0.100000"
2022-08-25 06:06:10,568  - mini_batch_size: "32"
2022-08-25 06:06:10,568  - patience: "3"
2022-08-25 06:06:10,569  - anneal_factor: "0.5"
2022-08-25 06:06:10,569  - max_epochs: "10"
2022-08-25 06:06:10,569  - shuffle: "True"
2022-08-25 06:06:10,569  - train_with_dev: "False"
2022-08-25 06:06:10,570  - batch_growth_annealing: "False"
2022-08-25 06:06:10,570 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:10,571 Model training base path: "db/kgc_models/test-kgc-sm-flair/trained_model"
2022-08-25 06:06:10,571 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:10,571 Device: cuda:0
2022-08-25 06:06:10,571 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:10,572 Embeddings storage mode: gpu
2022-08-25 06:06:10,572 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:10,659 epoch 1 - iter 1/4 - loss 4.03916875 - samples/sec: 371.51 - lr: 0.100000
2022-08-25 06:06:10,731 epoch 1 - iter 2/4 - loss 3.89573674 - samples/sec: 443.37 - lr: 0.100000
2022-08-25 06:06:10,805 epoch 1 - iter 3/4 - loss 3.73763981 - samples/sec: 441.41 - lr: 0.100000
2022-08-25 06:06:10,853 epoch 1 - iter 4/4 - loss 3.73055211 - samples/sec: 669.71 - lr: 0.100000
2022-08-25 06:06:10,854 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:10,854 EPOCH 1 done: loss 3.7306 - lr 0.100000
2022-08-25 06:06:10,934 Evaluating as a multi-label problem: False
2022-08-25 06:06:10,941 DEV : loss 3.0921895503997803 - f1-score (micro avg)  0.3087
2022-08-25 06:06:10,942 BAD EPOCHS (no improvement): 0
2022-08-25 06:06:10,944 saving best model
2022-08-25 06:06:12,789 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:12,816 epoch 2 - iter 1/4 - loss 2.91073551 - samples/sec: 1237.62 - lr: 0.100000
2022-08-25 06:06:12,842 epoch 2 - iter 2/4 - loss 2.94079960 - samples/sec: 1251.27 - lr: 0.100000
2022-08-25 06:06:12,874 epoch 2 - iter 3/4 - loss 2.78518824 - samples/sec: 1014.72 - lr: 0.100000
2022-08-25 06:06:12,891 epoch 2 - iter 4/4 - loss 2.80299381 - samples/sec: 1978.97 - lr: 0.100000
2022-08-25 06:06:12,892 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:12,892 EPOCH 2 done: loss 2.8030 - lr 0.100000
2022-08-25 06:06:12,932 Evaluating as a multi-label problem: False
2022-08-25 06:06:12,939 DEV : loss 2.827807903289795 - f1-score (micro avg)  0.2556
2022-08-25 06:06:12,940 BAD EPOCHS (no improvement): 1
2022-08-25 06:06:12,941 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:13,054 epoch 3 - iter 1/4 - loss 2.59037260 - samples/sec: 284.59 - lr: 0.100000
2022-08-25 06:06:13,081 epoch 3 - iter 2/4 - loss 2.60334442 - samples/sec: 1216.48 - lr: 0.100000
2022-08-25 06:06:13,108 epoch 3 - iter 3/4 - loss 2.57638893 - samples/sec: 1264.18 - lr: 0.100000
2022-08-25 06:06:13,126 epoch 3 - iter 4/4 - loss 2.53864971 - samples/sec: 1827.04 - lr: 0.100000
2022-08-25 06:06:13,126 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:13,127 EPOCH 3 done: loss 2.5386 - lr 0.100000
2022-08-25 06:06:13,167 Evaluating as a multi-label problem: False
2022-08-25 06:06:13,173 DEV : loss 2.7169787883758545 - f1-score (micro avg)  0.3168
2022-08-25 06:06:13,174 BAD EPOCHS (no improvement): 0
2022-08-25 06:06:13,175 saving best model
2022-08-25 06:06:14,714 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:14,743 epoch 4 - iter 1/4 - loss 2.71329416 - samples/sec: 1140.13 - lr: 0.100000
2022-08-25 06:06:14,769 epoch 4 - iter 2/4 - loss 2.38254593 - samples/sec: 1254.42 - lr: 0.100000
2022-08-25 06:06:14,803 epoch 4 - iter 3/4 - loss 2.37460295 - samples/sec: 958.15 - lr: 0.100000
2022-08-25 06:06:14,825 epoch 4 - iter 4/4 - loss 2.37499146 - samples/sec: 1527.35 - lr: 0.100000
2022-08-25 06:06:14,826 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:14,826 EPOCH 4 done: loss 2.3750 - lr 0.100000
2022-08-25 06:06:14,864 Evaluating as a multi-label problem: False
2022-08-25 06:06:14,870 DEV : loss 2.749277114868164 - f1-score (micro avg)  0.0
2022-08-25 06:06:14,871 BAD EPOCHS (no improvement): 1
2022-08-25 06:06:14,872 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:14,901 epoch 5 - iter 1/4 - loss 2.32981579 - samples/sec: 1152.14 - lr: 0.100000
2022-08-25 06:06:14,928 epoch 5 - iter 2/4 - loss 2.20069920 - samples/sec: 1219.29 - lr: 0.100000
2022-08-25 06:06:14,965 epoch 5 - iter 3/4 - loss 2.27754946 - samples/sec: 869.53 - lr: 0.100000
2022-08-25 06:06:14,986 epoch 5 - iter 4/4 - loss 2.25410557 - samples/sec: 1547.91 - lr: 0.100000
2022-08-25 06:06:14,987 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:14,987 EPOCH 5 done: loss 2.2541 - lr 0.100000
2022-08-25 06:06:15,029 Evaluating as a multi-label problem: False
2022-08-25 06:06:15,047 DEV : loss 2.7171242237091064 - f1-score (micro avg)  0.2903
2022-08-25 06:06:15,049 BAD EPOCHS (no improvement): 2
2022-08-25 06:06:15,050 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:15,089 epoch 6 - iter 1/4 - loss 2.18811890 - samples/sec: 858.42 - lr: 0.100000
2022-08-25 06:06:15,124 epoch 6 - iter 2/4 - loss 2.14233002 - samples/sec: 929.04 - lr: 0.100000
2022-08-25 06:06:15,156 epoch 6 - iter 3/4 - loss 2.20570734 - samples/sec: 1025.55 - lr: 0.100000
2022-08-25 06:06:15,173 epoch 6 - iter 4/4 - loss 2.18303889 - samples/sec: 1924.88 - lr: 0.100000
2022-08-25 06:06:15,174 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:15,174 EPOCH 6 done: loss 2.1830 - lr 0.100000
2022-08-25 06:06:15,210 Evaluating as a multi-label problem: False
2022-08-25 06:06:15,217 DEV : loss 2.6913464069366455 - f1-score (micro avg)  0.2314
2022-08-25 06:06:15,218 BAD EPOCHS (no improvement): 3
2022-08-25 06:06:15,220 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:15,253 epoch 7 - iter 1/4 - loss 2.14348509 - samples/sec: 988.78 - lr: 0.100000
2022-08-25 06:06:15,281 epoch 7 - iter 2/4 - loss 2.00850553 - samples/sec: 1178.96 - lr: 0.100000
2022-08-25 06:06:15,308 epoch 7 - iter 3/4 - loss 2.06308898 - samples/sec: 1208.59 - lr: 0.100000
2022-08-25 06:06:15,327 epoch 7 - iter 4/4 - loss 2.06392736 - samples/sec: 1723.48 - lr: 0.100000
2022-08-25 06:06:15,327 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:15,328 EPOCH 7 done: loss 2.0639 - lr 0.100000
2022-08-25 06:06:15,368 Evaluating as a multi-label problem: False
2022-08-25 06:06:15,376 DEV : loss 2.2334768772125244 - f1-score (micro avg)  0.5
2022-08-25 06:06:15,377 BAD EPOCHS (no improvement): 0
2022-08-25 06:06:15,381 saving best model
2022-08-25 06:06:16,903 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:16,938 epoch 8 - iter 1/4 - loss 1.97582934 - samples/sec: 924.66 - lr: 0.100000
2022-08-25 06:06:16,971 epoch 8 - iter 2/4 - loss 1.89160690 - samples/sec: 1003.74 - lr: 0.100000
2022-08-25 06:06:17,003 epoch 8 - iter 3/4 - loss 1.92096321 - samples/sec: 1015.15 - lr: 0.100000
2022-08-25 06:06:17,024 epoch 8 - iter 4/4 - loss 1.90466607 - samples/sec: 1613.37 - lr: 0.100000
2022-08-25 06:06:17,024 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:17,025 EPOCH 8 done: loss 1.9047 - lr 0.100000
2022-08-25 06:06:17,076 Evaluating as a multi-label problem: False
2022-08-25 06:06:17,086 DEV : loss 2.3985493183135986 - f1-score (micro avg)  0.3448
2022-08-25 06:06:17,087 BAD EPOCHS (no improvement): 1
2022-08-25 06:06:17,090 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:17,130 epoch 9 - iter 1/4 - loss 1.87079010 - samples/sec: 826.37 - lr: 0.100000
2022-08-25 06:06:17,167 epoch 9 - iter 2/4 - loss 1.85216901 - samples/sec: 864.81 - lr: 0.100000
2022-08-25 06:06:17,203 epoch 9 - iter 3/4 - loss 1.90496054 - samples/sec: 924.19 - lr: 0.100000
2022-08-25 06:06:17,225 epoch 9 - iter 4/4 - loss 1.89038416 - samples/sec: 1452.27 - lr: 0.100000
2022-08-25 06:06:17,226 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:17,226 EPOCH 9 done: loss 1.8904 - lr 0.100000
2022-08-25 06:06:17,276 Evaluating as a multi-label problem: False
2022-08-25 06:06:17,284 DEV : loss 2.121741533279419 - f1-score (micro avg)  0.4627
2022-08-25 06:06:17,285 BAD EPOCHS (no improvement): 2
2022-08-25 06:06:17,288 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:17,323 epoch 10 - iter 1/4 - loss 1.80540347 - samples/sec: 930.59 - lr: 0.100000
2022-08-25 06:06:17,354 epoch 10 - iter 2/4 - loss 1.72573052 - samples/sec: 1054.27 - lr: 0.100000
2022-08-25 06:06:17,386 epoch 10 - iter 3/4 - loss 1.77370743 - samples/sec: 1004.27 - lr: 0.100000
2022-08-25 06:06:17,407 epoch 10 - iter 4/4 - loss 1.75549973 - samples/sec: 1615.25 - lr: 0.100000
2022-08-25 06:06:17,407 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:17,408 EPOCH 10 done: loss 1.7555 - lr 0.100000
2022-08-25 06:06:17,453 Evaluating as a multi-label problem: False
2022-08-25 06:06:17,461 DEV : loss 2.2982170581817627 - f1-score (micro avg)  0.3333
2022-08-25 06:06:17,462 BAD EPOCHS (no improvement): 3
2022-08-25 06:06:19,099 ----------------------------------------------------------------------------------------------------
2022-08-25 06:06:19,100 loading file db/kgc_models/test-kgc-sm-flair/trained_model/best-model.pt
2022-08-25 06:06:27,749 SequenceTagger predicts: Dictionary with 47 tags: O, S-Item, B-Item, E-Item, I-Item, S-Activity, B-Activity, E-Activity, I-Activity, S-Observation, B-Observation, E-Observation, I-Observation, S-Location, B-Location, E-Location, I-Location, S-Consumable, B-Consumable, E-Consumable, I-Consumable, S-Agent, B-Agent, E-Agent, I-Agent, S-Attribute, B-Attribute, E-Attribute, I-Attribute, S-Specifier, B-Specifier, E-Specifier, I-Specifier, S-Cardinality, B-Cardinality, E-Cardinality, I-Cardinality, S-Event, B-Event, E-Event, I-Event, S-Time, B-Time, E-Time, I-Time, <START>, <STOP>
2022-08-25 06:06:28,150 Evaluating as a multi-label problem: False
2022-08-25 06:06:28,159 0.3281	0.3333	0.3307	0.2386
2022-08-25 06:06:28,159 
Results:
- F-score (micro) 0.3307
- F-score (macro) 0.125
- Accuracy 0.2386

By class:
              precision    recall  f1-score   support

        Item     0.2679    0.6250    0.3750        24
    Activity     0.8571    0.6667    0.7500         9
 Observation     0.0000    0.0000    0.0000        13
    Location     0.0000    0.0000    0.0000         5
  Consumable     0.0000    0.0000    0.0000         4
       Agent     0.0000    0.0000    0.0000         3
 Cardinality     0.0000    0.0000    0.0000         2
        Time     0.0000    0.0000    0.0000         2
   Attribute     0.0000    0.0000    0.0000         1

   micro avg     0.3281    0.3333    0.3307        63
   macro avg     0.1250    0.1435    0.1250        63
weighted avg     0.2245    0.3333    0.2500        63

2022-08-25 06:06:28,160 ----------------------------------------------------------------------------------------------------