EC2 Default User commited on
Commit
102bb34
1 Parent(s): 6f3e94b

Update spaCy pipeline

Browse files
README.md CHANGED
@@ -14,47 +14,62 @@ model-index:
14
  metrics:
15
  - name: NER Precision
16
  type: precision
17
- value: 0.737704918
18
  - name: NER Recall
19
  type: recall
20
- value: 0.679245283
21
  - name: NER F Score
22
  type: f_score
23
- value: 0.7072691552
 
 
 
 
 
 
 
24
  - task:
25
  name: POS
26
  type: token-classification
27
  metrics:
28
- - name: POS Accuracy
29
  type: accuracy
30
- value: 0.9715755942
31
  - task:
32
- name: SENTER
33
  type: token-classification
34
  metrics:
35
- - name: SENTER Precision
36
- type: precision
37
- value: 0.9862475442
38
- - name: SENTER Recall
39
- type: recall
40
- value: 0.9901380671
41
- - name: SENTER F Score
42
- type: f_score
43
- value: 0.9881889764
44
  - task:
45
- name: UNLABELED_DEPENDENCIES
46
  type: token-classification
47
  metrics:
48
- - name: Unlabeled Dependencies Accuracy
49
  type: accuracy
50
- value: 0.9224188392
 
 
 
 
 
 
 
51
  - task:
52
  name: LABELED_DEPENDENCIES
53
  type: token-classification
54
  metrics:
55
- - name: Labeled Dependencies Accuracy
56
- type: accuracy
57
- value: 0.9224188392
 
 
 
 
 
 
 
58
  ---
59
  ### Details: https://spacy.io/models/ja#ja_core_news_md
60
 
@@ -63,8 +78,8 @@ Japanese pipeline optimized for CPU. Components: tok2vec, morphologizer, parser,
63
  | Feature | Description |
64
  | --- | --- |
65
  | **Name** | `ja_core_news_md` |
66
- | **Version** | `3.2.0` |
67
- | **spaCy** | `>=3.2.0,<3.3.0` |
68
  | **Default Pipeline** | `tok2vec`, `morphologizer`, `parser`, `attribute_ruler`, `ner` |
69
  | **Components** | `tok2vec`, `morphologizer`, `parser`, `senter`, `attribute_ruler`, `ner` |
70
  | **Vectors** | 480443 keys, 20000 unique vectors (300 dimensions) |
@@ -76,13 +91,12 @@ Japanese pipeline optimized for CPU. Components: tok2vec, morphologizer, parser,
76
 
77
  <details>
78
 
79
- <summary>View label scheme (66 labels for 4 components)</summary>
80
 
81
  | Component | Labels |
82
  | --- | --- |
83
  | **`morphologizer`** | `POS=NOUN`, `POS=ADP`, `POS=VERB`, `POS=SCONJ`, `POS=AUX`, `POS=PUNCT`, `POS=PART`, `POS=DET`, `POS=NUM`, `POS=ADV`, `POS=PRON`, `POS=ADJ`, `POS=PROPN`, `POS=CCONJ`, `POS=SYM`, `POS=NOUN\|Polarity=Neg`, `POS=AUX\|Polarity=Neg`, `POS=INTJ`, `POS=SCONJ\|Polarity=Neg` |
84
  | **`parser`** | `ROOT`, `acl`, `advcl`, `advmod`, `amod`, `aux`, `case`, `cc`, `ccomp`, `compound`, `cop`, `csubj`, `dep`, `det`, `dislocated`, `fixed`, `mark`, `nmod`, `nsubj`, `nummod`, `obj`, `obl`, `punct` |
85
- | **`senter`** | `I`, `S` |
86
  | **`ner`** | `CARDINAL`, `DATE`, `EVENT`, `FAC`, `GPE`, `LANGUAGE`, `LAW`, `LOC`, `MONEY`, `MOVEMENT`, `NORP`, `ORDINAL`, `ORG`, `PERCENT`, `PERSON`, `PET_NAME`, `PHONE`, `PRODUCT`, `QUANTITY`, `TIME`, `TITLE_AFFIX`, `WORK_OF_ART` |
87
 
88
  </details>
@@ -95,18 +109,18 @@ Japanese pipeline optimized for CPU. Components: tok2vec, morphologizer, parser,
95
  | `TOKEN_P` | 97.65 |
96
  | `TOKEN_R` | 97.90 |
97
  | `TOKEN_F` | 97.77 |
98
- | `POS_ACC` | 97.13 |
99
- | `MORPH_ACC` | 0.40 |
100
  | `MORPH_MICRO_P` | 34.01 |
101
  | `MORPH_MICRO_R` | 98.04 |
102
  | `MORPH_MICRO_F` | 50.51 |
103
- | `SENTS_P` | 98.62 |
104
- | `SENTS_R` | 99.01 |
105
- | `SENTS_F` | 98.82 |
106
- | `DEP_UAS` | 92.24 |
107
  | `DEP_LAS` | 90.75 |
108
- | `TAG_ACC` | 97.16 |
109
- | `LEMMA_ACC` | 96.59 |
110
- | `ENTS_P` | 73.77 |
111
- | `ENTS_R` | 67.92 |
112
- | `ENTS_F` | 70.73 |
 
14
  metrics:
15
  - name: NER Precision
16
  type: precision
17
+ value: 0.7384196185
18
  - name: NER Recall
19
  type: recall
20
+ value: 0.6817610063
21
  - name: NER F Score
22
  type: f_score
23
+ value: 0.7089601046
24
+ - task:
25
+ name: TAG
26
+ type: token-classification
27
+ metrics:
28
+ - name: TAG (XPOS) Accuracy
29
+ type: accuracy
30
+ value: 0.9712488769
31
  - task:
32
  name: POS
33
  type: token-classification
34
  metrics:
35
+ - name: POS (UPOS) Accuracy
36
  type: accuracy
37
+ value: 0.9708372531
38
  - task:
39
+ name: MORPH
40
  type: token-classification
41
  metrics:
42
+ - name: Morph (UFeats) Accuracy
43
+ type: accuracy
44
+ value: 0.0
 
 
 
 
 
 
45
  - task:
46
+ name: LEMMA
47
  type: token-classification
48
  metrics:
49
+ - name: Lemma Accuracy
50
  type: accuracy
51
+ value: 0.965013864
52
+ - task:
53
+ name: UNLABELED_DEPENDENCIES
54
+ type: token-classification
55
+ metrics:
56
+ - name: Unlabeled Attachment Score (UAS)
57
+ type: f_score
58
+ value: 0.9211237169
59
  - task:
60
  name: LABELED_DEPENDENCIES
61
  type: token-classification
62
  metrics:
63
+ - name: Labeled Attachment Score (LAS)
64
+ type: f_score
65
+ value: 0.9075282365
66
+ - task:
67
+ name: SENTS
68
+ type: token-classification
69
+ metrics:
70
+ - name: Sentences F-Score
71
+ type: f_score
72
+ value: 0.9940828402
73
  ---
74
  ### Details: https://spacy.io/models/ja#ja_core_news_md
75
 
 
78
  | Feature | Description |
79
  | --- | --- |
80
  | **Name** | `ja_core_news_md` |
81
+ | **Version** | `3.3.0` |
82
+ | **spaCy** | `>=3.3.0.dev0,<3.4.0` |
83
  | **Default Pipeline** | `tok2vec`, `morphologizer`, `parser`, `attribute_ruler`, `ner` |
84
  | **Components** | `tok2vec`, `morphologizer`, `parser`, `senter`, `attribute_ruler`, `ner` |
85
  | **Vectors** | 480443 keys, 20000 unique vectors (300 dimensions) |
 
91
 
92
  <details>
93
 
94
+ <summary>View label scheme (64 labels for 3 components)</summary>
95
 
96
  | Component | Labels |
97
  | --- | --- |
98
  | **`morphologizer`** | `POS=NOUN`, `POS=ADP`, `POS=VERB`, `POS=SCONJ`, `POS=AUX`, `POS=PUNCT`, `POS=PART`, `POS=DET`, `POS=NUM`, `POS=ADV`, `POS=PRON`, `POS=ADJ`, `POS=PROPN`, `POS=CCONJ`, `POS=SYM`, `POS=NOUN\|Polarity=Neg`, `POS=AUX\|Polarity=Neg`, `POS=INTJ`, `POS=SCONJ\|Polarity=Neg` |
99
  | **`parser`** | `ROOT`, `acl`, `advcl`, `advmod`, `amod`, `aux`, `case`, `cc`, `ccomp`, `compound`, `cop`, `csubj`, `dep`, `det`, `dislocated`, `fixed`, `mark`, `nmod`, `nsubj`, `nummod`, `obj`, `obl`, `punct` |
 
100
  | **`ner`** | `CARDINAL`, `DATE`, `EVENT`, `FAC`, `GPE`, `LANGUAGE`, `LAW`, `LOC`, `MONEY`, `MOVEMENT`, `NORP`, `ORDINAL`, `ORG`, `PERCENT`, `PERSON`, `PET_NAME`, `PHONE`, `PRODUCT`, `QUANTITY`, `TIME`, `TITLE_AFFIX`, `WORK_OF_ART` |
101
 
102
  </details>
 
109
  | `TOKEN_P` | 97.65 |
110
  | `TOKEN_R` | 97.90 |
111
  | `TOKEN_F` | 97.77 |
112
+ | `POS_ACC` | 97.08 |
113
+ | `MORPH_ACC` | 0.00 |
114
  | `MORPH_MICRO_P` | 34.01 |
115
  | `MORPH_MICRO_R` | 98.04 |
116
  | `MORPH_MICRO_F` | 50.51 |
117
+ | `SENTS_P` | 99.41 |
118
+ | `SENTS_R` | 99.41 |
119
+ | `SENTS_F` | 99.41 |
120
+ | `DEP_UAS` | 92.11 |
121
  | `DEP_LAS` | 90.75 |
122
+ | `TAG_ACC` | 97.12 |
123
+ | `LEMMA_ACC` | 96.50 |
124
+ | `ENTS_P` | 73.84 |
125
+ | `ENTS_R` | 68.18 |
126
+ | `ENTS_F` | 70.90 |
accuracy.json CHANGED
@@ -3,8 +3,8 @@
3
  "token_p": 0.9764591282,
4
  "token_r": 0.9790021974,
5
  "token_f": 0.9777290092,
6
- "pos_acc": 0.971323379,
7
- "morph_acc": 0.0040005162,
8
  "morph_micro_p": 0.3401360544,
9
  "morph_micro_r": 0.9803921569,
10
  "morph_micro_f": 0.5050505051,
@@ -25,116 +25,116 @@
25
  "f": 0.0
26
  }
27
  },
28
- "sents_p": 0.9862475442,
29
- "sents_r": 0.9901380671,
30
- "sents_f": 0.9881889764,
31
- "dep_uas": 0.9224188392,
32
- "dep_las": 0.9075365579,
33
  "dep_las_per_type": {
34
  "cc": {
35
- "p": 0.7708333333,
36
- "r": 0.7708333333,
37
- "f": 0.7708333333
38
  },
39
  "compound": {
40
- "p": 0.9421390999,
41
- "r": 0.908680947,
42
- "f": 0.925107604
43
  },
44
  "obl": {
45
- "p": 0.8222778473,
46
- "r": 0.8202247191,
47
- "f": 0.82125
48
  },
49
  "case": {
50
- "p": 0.9907763259,
51
- "r": 0.9794832827,
52
- "f": 0.9850974398
53
  },
54
  "dislocated": {
55
- "p": 0.5833333333,
56
- "r": 0.5384615385,
57
- "f": 0.56
58
  },
59
  "nsubj": {
60
- "p": 0.8061420345,
61
- "r": 0.8061420345,
62
- "f": 0.8061420345
63
  },
64
  "nmod": {
65
- "p": 0.88639201,
66
- "r": 0.8304093567,
67
- "f": 0.8574879227
68
  },
69
  "root": {
70
- "p": 0.9741035857,
71
- "r": 0.9644970414,
72
- "f": 0.9692765114
73
  },
74
  "aux": {
75
- "p": 0.977818854,
76
- "r": 0.982358403,
77
- "f": 0.9800833719
78
  },
79
  "advcl": {
80
- "p": 0.6726057906,
81
- "r": 0.6786516854,
82
- "f": 0.6756152125
83
  },
84
  "mark": {
85
- "p": 0.9599198397,
86
- "r": 0.958,
87
- "f": 0.958958959
88
  },
89
  "fixed": {
90
- "p": 0.9533213645,
91
- "r": 0.9654545455,
92
- "f": 0.9593495935
93
  },
94
  "acl": {
95
- "p": 0.8561946903,
96
- "r": 0.8505494505,
97
- "f": 0.8533627343
98
  },
99
  "obj": {
100
- "p": 0.9695121951,
101
- "r": 0.9607250755,
102
- "f": 0.9650986343
103
  },
104
  "nummod": {
105
- "p": 0.9868421053,
106
- "r": 0.8875739645,
107
- "f": 0.9345794393
108
  },
109
  "advmod": {
110
- "p": 0.6323529412,
111
- "r": 0.6142857143,
112
- "f": 0.6231884058
113
  },
114
  "amod": {
115
- "p": 0.9,
116
  "r": 0.7297297297,
117
- "f": 0.8059701493
118
  },
119
  "cop": {
120
- "p": 0.9585798817,
121
  "r": 0.9418604651,
122
- "f": 0.9501466276
123
  },
124
  "ccomp": {
125
- "p": 1.0,
126
- "r": 0.7272727273,
127
- "f": 0.8421052632
128
  },
129
  "det": {
130
- "p": 0.9615384615,
131
- "r": 0.9433962264,
132
- "f": 0.9523809524
133
  },
134
  "csubj": {
135
- "p": 0.6923076923,
136
- "r": 0.75,
137
- "f": 0.72
138
  },
139
  "dep": {
140
  "p": 0.0,
@@ -142,11 +142,11 @@
142
  "f": 0.0
143
  }
144
  },
145
- "tag_acc": 0.9715755942,
146
- "lemma_acc": 0.9659109444,
147
- "ents_p": 0.737704918,
148
- "ents_r": 0.679245283,
149
- "ents_f": 0.7072691552,
150
  "ents_per_type": {
151
  "DATE": {
152
  "p": 0.9722222222,
@@ -154,29 +154,29 @@
154
  "f": 0.9677419355
155
  },
156
  "ORG": {
157
- "p": 0.6218487395,
158
- "r": 0.5401459854,
159
- "f": 0.578125
160
  },
161
  "TITLE_AFFIX": {
162
- "p": 0.7727272727,
163
- "r": 0.5666666667,
164
- "f": 0.6538461538
165
  },
166
  "PERSON": {
167
- "p": 0.7407407407,
168
- "r": 0.7194244604,
169
- "f": 0.7299270073
170
  },
171
  "GPE": {
172
- "p": 0.6774193548,
173
- "r": 0.670212766,
174
- "f": 0.6737967914
175
  },
176
  "PRODUCT": {
177
- "p": 0.5806451613,
178
- "r": 0.4285714286,
179
- "f": 0.4931506849
180
  },
181
  "TIME": {
182
  "p": 0.6666666667,
@@ -184,49 +184,49 @@
184
  "f": 0.8
185
  },
186
  "QUANTITY": {
187
- "p": 0.8382352941,
188
  "r": 0.8636363636,
189
- "f": 0.8507462687
190
  },
191
  "NORP": {
192
- "p": 0.7407407407,
193
- "r": 0.625,
194
- "f": 0.6779661017
195
  },
196
  "ORDINAL": {
197
- "p": 0.5357142857,
198
  "r": 0.6818181818,
199
- "f": 0.6
200
  },
201
  "WORK_OF_ART": {
202
- "p": 0.8,
203
- "r": 0.7058823529,
204
- "f": 0.75
205
  },
206
  "FAC": {
207
- "p": 0.619047619,
208
- "r": 0.3513513514,
209
- "f": 0.4482758621
210
  },
211
  "PERCENT": {
212
  "p": 1.0,
213
  "r": 0.2857142857,
214
  "f": 0.4444444444
215
  },
 
 
 
 
 
216
  "CARDINAL": {
217
  "p": 0.0,
218
  "r": 0.0,
219
  "f": 0.0
220
  },
221
- "EVENT": {
222
- "p": 0.75,
223
- "r": 0.6923076923,
224
- "f": 0.72
225
- },
226
  "LOC": {
227
- "p": 0.5333333333,
228
- "r": 0.8,
229
- "f": 0.64
230
  },
231
  "MOVEMENT": {
232
  "p": 0.0,
@@ -238,16 +238,21 @@
238
  "r": 0.3333333333,
239
  "f": 0.5
240
  },
 
 
 
 
 
241
  "MONEY": {
242
- "p": 0.875,
243
  "r": 1.0,
244
- "f": 0.9333333333
245
  },
246
  "LANGUAGE": {
247
- "p": 0.8571428571,
248
  "r": 1.0,
249
- "f": 0.9230769231
250
  }
251
  },
252
- "speed": 4621.4641630782
253
  }
 
3
  "token_p": 0.9764591282,
4
  "token_r": 0.9790021974,
5
  "token_f": 0.9777290092,
6
+ "pos_acc": 0.9708372531,
7
+ "morph_acc": 0.0,
8
  "morph_micro_p": 0.3401360544,
9
  "morph_micro_r": 0.9803921569,
10
  "morph_micro_f": 0.5050505051,
 
25
  "f": 0.0
26
  }
27
  },
28
+ "sents_p": 0.9940828402,
29
+ "sents_r": 0.9940828402,
30
+ "sents_f": 0.9940828402,
31
+ "dep_uas": 0.9211237169,
32
+ "dep_las": 0.9075282365,
33
  "dep_las_per_type": {
34
  "cc": {
35
+ "p": 0.8125,
36
+ "r": 0.8125,
37
+ "f": 0.8125
38
  },
39
  "compound": {
40
+ "p": 0.9490930369,
41
+ "r": 0.9143179256,
42
+ "f": 0.9313809934
43
  },
44
  "obl": {
45
+ "p": 0.80875,
46
+ "r": 0.8077403246,
47
+ "f": 0.808244847
48
  },
49
  "case": {
50
+ "p": 0.9884925201,
51
+ "r": 0.9791033435,
52
+ "f": 0.9837755297
53
  },
54
  "dislocated": {
55
+ "p": 0.6666666667,
56
+ "r": 0.4615384615,
57
+ "f": 0.5454545455
58
  },
59
  "nsubj": {
60
+ "p": 0.8146718147,
61
+ "r": 0.8099808061,
62
+ "f": 0.812319538
63
  },
64
  "nmod": {
65
+ "p": 0.8869565217,
66
+ "r": 0.8350877193,
67
+ "f": 0.8602409639
68
  },
69
  "root": {
70
+ "p": 0.966,
71
+ "r": 0.9526627219,
72
+ "f": 0.959285005
73
  },
74
  "aux": {
75
+ "p": 0.9768732655,
76
+ "r": 0.9805013928,
77
+ "f": 0.9786839666
78
  },
79
  "advcl": {
80
+ "p": 0.6876404494,
81
+ "r": 0.6876404494,
82
+ "f": 0.6876404494
83
  },
84
  "mark": {
85
+ "p": 0.9696969697,
86
+ "r": 0.96,
87
+ "f": 0.9648241206
88
  },
89
  "fixed": {
90
+ "p": 0.9566003617,
91
+ "r": 0.9618181818,
92
+ "f": 0.9592021759
93
  },
94
  "acl": {
95
+ "p": 0.8278867102,
96
+ "r": 0.8351648352,
97
+ "f": 0.8315098468
98
  },
99
  "obj": {
100
+ "p": 0.9456193353,
101
+ "r": 0.9456193353,
102
+ "f": 0.9456193353
103
  },
104
  "nummod": {
105
+ "p": 0.9325153374,
106
+ "r": 0.899408284,
107
+ "f": 0.9156626506
108
  },
109
  "advmod": {
110
+ "p": 0.7054263566,
111
+ "r": 0.65,
112
+ "f": 0.6765799257
113
  },
114
  "amod": {
115
+ "p": 0.9642857143,
116
  "r": 0.7297297297,
117
+ "f": 0.8307692308
118
  },
119
  "cop": {
120
+ "p": 0.9700598802,
121
  "r": 0.9418604651,
122
+ "f": 0.9557522124
123
  },
124
  "ccomp": {
125
+ "p": 0.95,
126
+ "r": 0.8636363636,
127
+ "f": 0.9047619048
128
  },
129
  "det": {
130
+ "p": 0.9807692308,
131
+ "r": 0.9622641509,
132
+ "f": 0.9714285714
133
  },
134
  "csubj": {
135
+ "p": 0.6153846154,
136
+ "r": 0.6666666667,
137
+ "f": 0.64
138
  },
139
  "dep": {
140
  "p": 0.0,
 
142
  "f": 0.0
143
  }
144
  },
145
+ "tag_acc": 0.9712488769,
146
+ "lemma_acc": 0.965013864,
147
+ "ents_p": 0.7384196185,
148
+ "ents_r": 0.6817610063,
149
+ "ents_f": 0.7089601046,
150
  "ents_per_type": {
151
  "DATE": {
152
  "p": 0.9722222222,
 
154
  "f": 0.9677419355
155
  },
156
  "ORG": {
157
+ "p": 0.6097560976,
158
+ "r": 0.5474452555,
159
+ "f": 0.5769230769
160
  },
161
  "TITLE_AFFIX": {
162
+ "p": 0.8636363636,
163
+ "r": 0.6333333333,
164
+ "f": 0.7307692308
165
  },
166
  "PERSON": {
167
+ "p": 0.762962963,
168
+ "r": 0.7410071942,
169
+ "f": 0.7518248175
170
  },
171
  "GPE": {
172
+ "p": 0.7282608696,
173
+ "r": 0.7127659574,
174
+ "f": 0.7204301075
175
  },
176
  "PRODUCT": {
177
+ "p": 0.4375,
178
+ "r": 0.3333333333,
179
+ "f": 0.3783783784
180
  },
181
  "TIME": {
182
  "p": 0.6666666667,
 
184
  "f": 0.8
185
  },
186
  "QUANTITY": {
187
+ "p": 0.8769230769,
188
  "r": 0.8636363636,
189
+ "f": 0.8702290076
190
  },
191
  "NORP": {
192
+ "p": 0.72,
193
+ "r": 0.5625,
194
+ "f": 0.6315789474
195
  },
196
  "ORDINAL": {
197
+ "p": 0.5172413793,
198
  "r": 0.6818181818,
199
+ "f": 0.5882352941
200
  },
201
  "WORK_OF_ART": {
202
+ "p": 0.5789473684,
203
+ "r": 0.6470588235,
204
+ "f": 0.6111111111
205
  },
206
  "FAC": {
207
+ "p": 0.625,
208
+ "r": 0.4054054054,
209
+ "f": 0.4918032787
210
  },
211
  "PERCENT": {
212
  "p": 1.0,
213
  "r": 0.2857142857,
214
  "f": 0.4444444444
215
  },
216
+ "EVENT": {
217
+ "p": 0.7619047619,
218
+ "r": 0.6153846154,
219
+ "f": 0.6808510638
220
+ },
221
  "CARDINAL": {
222
  "p": 0.0,
223
  "r": 0.0,
224
  "f": 0.0
225
  },
 
 
 
 
 
226
  "LOC": {
227
+ "p": 0.5384615385,
228
+ "r": 0.7,
229
+ "f": 0.6086956522
230
  },
231
  "MOVEMENT": {
232
  "p": 0.0,
 
238
  "r": 0.3333333333,
239
  "f": 0.5
240
  },
241
+ "PET_NAME": {
242
+ "p": 0.0,
243
+ "r": 0.0,
244
+ "f": 0.0
245
+ },
246
  "MONEY": {
247
+ "p": 1.0,
248
  "r": 1.0,
249
+ "f": 1.0
250
  },
251
  "LANGUAGE": {
252
+ "p": 1.0,
253
  "r": 1.0,
254
+ "f": 1.0
255
  }
256
  },
257
+ "speed": 8044.6200194392
258
  }
attribute_ruler/patterns CHANGED
Binary files a/attribute_ruler/patterns and b/attribute_ruler/patterns differ
 
config.cfg CHANGED
@@ -35,8 +35,9 @@ overwrite = true
35
  scorer = {"@scorers":"spacy.morphologizer_scorer.v1"}
36
 
37
  [components.morphologizer.model]
38
- @architectures = "spacy.Tagger.v1"
39
  nO = null
 
40
 
41
  [components.morphologizer.model.tok2vec]
42
  @architectures = "spacy.Tok2VecListener.v1"
@@ -66,7 +67,7 @@ nO = null
66
  @architectures = "spacy.MultiHashEmbed.v2"
67
  width = 96
68
  attrs = ["NORM","PREFIX","SUFFIX","SHAPE"]
69
- rows = [5000,2500,2500,2500]
70
  include_static_vectors = true
71
 
72
  [components.ner.model.tok2vec.encode]
@@ -104,8 +105,9 @@ overwrite = false
104
  scorer = {"@scorers":"spacy.senter_scorer.v1"}
105
 
106
  [components.senter.model]
107
- @architectures = "spacy.Tagger.v1"
108
  nO = null
 
109
 
110
  [components.senter.model.tok2vec]
111
  @architectures = "spacy.Tok2Vec.v2"
@@ -134,7 +136,7 @@ factory = "tok2vec"
134
  @architectures = "spacy.MultiHashEmbed.v2"
135
  width = ${components.tok2vec.model.encode:width}
136
  attrs = ["NORM","PREFIX","SUFFIX","SHAPE"]
137
- rows = [5000,2500,2500,2500]
138
  include_static_vectors = true
139
 
140
  [components.tok2vec.model.encode]
@@ -171,7 +173,7 @@ dropout = 0.1
171
  accumulate_gradient = 1
172
  patience = 5000
173
  max_epochs = 0
174
- max_steps = 0
175
  eval_frequency = 1000
176
  frozen_components = []
177
  before_to_disk = null
 
35
  scorer = {"@scorers":"spacy.morphologizer_scorer.v1"}
36
 
37
  [components.morphologizer.model]
38
+ @architectures = "spacy.Tagger.v2"
39
  nO = null
40
+ normalize = false
41
 
42
  [components.morphologizer.model.tok2vec]
43
  @architectures = "spacy.Tok2VecListener.v1"
 
67
  @architectures = "spacy.MultiHashEmbed.v2"
68
  width = 96
69
  attrs = ["NORM","PREFIX","SUFFIX","SHAPE"]
70
+ rows = [5000,1000,2500,2500]
71
  include_static_vectors = true
72
 
73
  [components.ner.model.tok2vec.encode]
 
105
  scorer = {"@scorers":"spacy.senter_scorer.v1"}
106
 
107
  [components.senter.model]
108
+ @architectures = "spacy.Tagger.v2"
109
  nO = null
110
+ normalize = false
111
 
112
  [components.senter.model.tok2vec]
113
  @architectures = "spacy.Tok2Vec.v2"
 
136
  @architectures = "spacy.MultiHashEmbed.v2"
137
  width = ${components.tok2vec.model.encode:width}
138
  attrs = ["NORM","PREFIX","SUFFIX","SHAPE"]
139
+ rows = [5000,1000,2500,2500]
140
  include_static_vectors = true
141
 
142
  [components.tok2vec.model.encode]
 
173
  accumulate_gradient = 1
174
  patience = 5000
175
  max_epochs = 0
176
+ max_steps = 100000
177
  eval_frequency = 1000
178
  frozen_components = []
179
  before_to_disk = null
ja_core_news_md-any-py3-none-any.whl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c851e760a3b18f48359d9c52828ced1d1bf2560063d8fcdfc161203e3eaf35eb
3
- size 43045518
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80bc4423a0a9d8ce761c25750c28885c055caa83bb4ae45b14937975f9b63bcb
3
+ size 41986714
meta.json CHANGED
@@ -1,14 +1,14 @@
1
  {
2
  "lang":"ja",
3
  "name":"core_news_md",
4
- "version":"3.2.0",
5
  "description":"Japanese pipeline optimized for CPU. Components: tok2vec, morphologizer, parser, senter, ner, attribute_ruler.",
6
  "author":"Explosion",
7
  "email":"contact@explosion.ai",
8
  "url":"https://explosion.ai",
9
  "license":"CC BY-SA 4.0",
10
- "spacy_version":">=3.2.0,<3.3.0",
11
- "spacy_git_version":"bb26550e2",
12
  "vectors":{
13
  "width":300,
14
  "vectors":20000,
@@ -65,10 +65,6 @@
65
  "obl",
66
  "punct"
67
  ],
68
- "senter":[
69
- "I",
70
- "S"
71
- ],
72
  "attribute_ruler":[
73
 
74
  ],
@@ -120,8 +116,8 @@
120
  "token_p":0.9764591282,
121
  "token_r":0.9790021974,
122
  "token_f":0.9777290092,
123
- "pos_acc":0.971323379,
124
- "morph_acc":0.0040005162,
125
  "morph_micro_p":0.3401360544,
126
  "morph_micro_r":0.9803921569,
127
  "morph_micro_f":0.5050505051,
@@ -142,116 +138,116 @@
142
  "f":0.0
143
  }
144
  },
145
- "sents_p":0.9862475442,
146
- "sents_r":0.9901380671,
147
- "sents_f":0.9881889764,
148
- "dep_uas":0.9224188392,
149
- "dep_las":0.9075365579,
150
  "dep_las_per_type":{
151
  "cc":{
152
- "p":0.7708333333,
153
- "r":0.7708333333,
154
- "f":0.7708333333
155
  },
156
  "compound":{
157
- "p":0.9421390999,
158
- "r":0.908680947,
159
- "f":0.925107604
160
  },
161
  "obl":{
162
- "p":0.8222778473,
163
- "r":0.8202247191,
164
- "f":0.82125
165
  },
166
  "case":{
167
- "p":0.9907763259,
168
- "r":0.9794832827,
169
- "f":0.9850974398
170
  },
171
  "dislocated":{
172
- "p":0.5833333333,
173
- "r":0.5384615385,
174
- "f":0.56
175
  },
176
  "nsubj":{
177
- "p":0.8061420345,
178
- "r":0.8061420345,
179
- "f":0.8061420345
180
  },
181
  "nmod":{
182
- "p":0.88639201,
183
- "r":0.8304093567,
184
- "f":0.8574879227
185
  },
186
  "root":{
187
- "p":0.9741035857,
188
- "r":0.9644970414,
189
- "f":0.9692765114
190
  },
191
  "aux":{
192
- "p":0.977818854,
193
- "r":0.982358403,
194
- "f":0.9800833719
195
  },
196
  "advcl":{
197
- "p":0.6726057906,
198
- "r":0.6786516854,
199
- "f":0.6756152125
200
  },
201
  "mark":{
202
- "p":0.9599198397,
203
- "r":0.958,
204
- "f":0.958958959
205
  },
206
  "fixed":{
207
- "p":0.9533213645,
208
- "r":0.9654545455,
209
- "f":0.9593495935
210
  },
211
  "acl":{
212
- "p":0.8561946903,
213
- "r":0.8505494505,
214
- "f":0.8533627343
215
  },
216
  "obj":{
217
- "p":0.9695121951,
218
- "r":0.9607250755,
219
- "f":0.9650986343
220
  },
221
  "nummod":{
222
- "p":0.9868421053,
223
- "r":0.8875739645,
224
- "f":0.9345794393
225
  },
226
  "advmod":{
227
- "p":0.6323529412,
228
- "r":0.6142857143,
229
- "f":0.6231884058
230
  },
231
  "amod":{
232
- "p":0.9,
233
  "r":0.7297297297,
234
- "f":0.8059701493
235
  },
236
  "cop":{
237
- "p":0.9585798817,
238
  "r":0.9418604651,
239
- "f":0.9501466276
240
  },
241
  "ccomp":{
242
- "p":1.0,
243
- "r":0.7272727273,
244
- "f":0.8421052632
245
  },
246
  "det":{
247
- "p":0.9615384615,
248
- "r":0.9433962264,
249
- "f":0.9523809524
250
  },
251
  "csubj":{
252
- "p":0.6923076923,
253
- "r":0.75,
254
- "f":0.72
255
  },
256
  "dep":{
257
  "p":0.0,
@@ -259,11 +255,11 @@
259
  "f":0.0
260
  }
261
  },
262
- "tag_acc":0.9715755942,
263
- "lemma_acc":0.9659109444,
264
- "ents_p":0.737704918,
265
- "ents_r":0.679245283,
266
- "ents_f":0.7072691552,
267
  "ents_per_type":{
268
  "DATE":{
269
  "p":0.9722222222,
@@ -271,29 +267,29 @@
271
  "f":0.9677419355
272
  },
273
  "ORG":{
274
- "p":0.6218487395,
275
- "r":0.5401459854,
276
- "f":0.578125
277
  },
278
  "TITLE_AFFIX":{
279
- "p":0.7727272727,
280
- "r":0.5666666667,
281
- "f":0.6538461538
282
  },
283
  "PERSON":{
284
- "p":0.7407407407,
285
- "r":0.7194244604,
286
- "f":0.7299270073
287
  },
288
  "GPE":{
289
- "p":0.6774193548,
290
- "r":0.670212766,
291
- "f":0.6737967914
292
  },
293
  "PRODUCT":{
294
- "p":0.5806451613,
295
- "r":0.4285714286,
296
- "f":0.4931506849
297
  },
298
  "TIME":{
299
  "p":0.6666666667,
@@ -301,49 +297,49 @@
301
  "f":0.8
302
  },
303
  "QUANTITY":{
304
- "p":0.8382352941,
305
  "r":0.8636363636,
306
- "f":0.8507462687
307
  },
308
  "NORP":{
309
- "p":0.7407407407,
310
- "r":0.625,
311
- "f":0.6779661017
312
  },
313
  "ORDINAL":{
314
- "p":0.5357142857,
315
  "r":0.6818181818,
316
- "f":0.6
317
  },
318
  "WORK_OF_ART":{
319
- "p":0.8,
320
- "r":0.7058823529,
321
- "f":0.75
322
  },
323
  "FAC":{
324
- "p":0.619047619,
325
- "r":0.3513513514,
326
- "f":0.4482758621
327
  },
328
  "PERCENT":{
329
  "p":1.0,
330
  "r":0.2857142857,
331
  "f":0.4444444444
332
  },
 
 
 
 
 
333
  "CARDINAL":{
334
  "p":0.0,
335
  "r":0.0,
336
  "f":0.0
337
  },
338
- "EVENT":{
339
- "p":0.75,
340
- "r":0.6923076923,
341
- "f":0.72
342
- },
343
  "LOC":{
344
- "p":0.5333333333,
345
- "r":0.8,
346
- "f":0.64
347
  },
348
  "MOVEMENT":{
349
  "p":0.0,
@@ -355,18 +351,23 @@
355
  "r":0.3333333333,
356
  "f":0.5
357
  },
 
 
 
 
 
358
  "MONEY":{
359
- "p":0.875,
360
  "r":1.0,
361
- "f":0.9333333333
362
  },
363
  "LANGUAGE":{
364
- "p":0.8571428571,
365
  "r":1.0,
366
- "f":0.9230769231
367
  }
368
  },
369
- "speed":4621.4641630782
370
  },
371
  "sources":[
372
  {
@@ -389,7 +390,7 @@
389
  }
390
  ],
391
  "requirements":[
392
- "sudachipy>=0.4.9",
393
- "sudachidict-core>=20200330"
394
  ]
395
  }
 
1
  {
2
  "lang":"ja",
3
  "name":"core_news_md",
4
+ "version":"3.3.0",
5
  "description":"Japanese pipeline optimized for CPU. Components: tok2vec, morphologizer, parser, senter, ner, attribute_ruler.",
6
  "author":"Explosion",
7
  "email":"contact@explosion.ai",
8
  "url":"https://explosion.ai",
9
  "license":"CC BY-SA 4.0",
10
+ "spacy_version":">=3.3.0.dev0,<3.4.0",
11
+ "spacy_git_version":"849bef2de",
12
  "vectors":{
13
  "width":300,
14
  "vectors":20000,
 
65
  "obl",
66
  "punct"
67
  ],
 
 
 
 
68
  "attribute_ruler":[
69
 
70
  ],
 
116
  "token_p":0.9764591282,
117
  "token_r":0.9790021974,
118
  "token_f":0.9777290092,
119
+ "pos_acc":0.9708372531,
120
+ "morph_acc":0.0,
121
  "morph_micro_p":0.3401360544,
122
  "morph_micro_r":0.9803921569,
123
  "morph_micro_f":0.5050505051,
 
138
  "f":0.0
139
  }
140
  },
141
+ "sents_p":0.9940828402,
142
+ "sents_r":0.9940828402,
143
+ "sents_f":0.9940828402,
144
+ "dep_uas":0.9211237169,
145
+ "dep_las":0.9075282365,
146
  "dep_las_per_type":{
147
  "cc":{
148
+ "p":0.8125,
149
+ "r":0.8125,
150
+ "f":0.8125
151
  },
152
  "compound":{
153
+ "p":0.9490930369,
154
+ "r":0.9143179256,
155
+ "f":0.9313809934
156
  },
157
  "obl":{
158
+ "p":0.80875,
159
+ "r":0.8077403246,
160
+ "f":0.808244847
161
  },
162
  "case":{
163
+ "p":0.9884925201,
164
+ "r":0.9791033435,
165
+ "f":0.9837755297
166
  },
167
  "dislocated":{
168
+ "p":0.6666666667,
169
+ "r":0.4615384615,
170
+ "f":0.5454545455
171
  },
172
  "nsubj":{
173
+ "p":0.8146718147,
174
+ "r":0.8099808061,
175
+ "f":0.812319538
176
  },
177
  "nmod":{
178
+ "p":0.8869565217,
179
+ "r":0.8350877193,
180
+ "f":0.8602409639
181
  },
182
  "root":{
183
+ "p":0.966,
184
+ "r":0.9526627219,
185
+ "f":0.959285005
186
  },
187
  "aux":{
188
+ "p":0.9768732655,
189
+ "r":0.9805013928,
190
+ "f":0.9786839666
191
  },
192
  "advcl":{
193
+ "p":0.6876404494,
194
+ "r":0.6876404494,
195
+ "f":0.6876404494
196
  },
197
  "mark":{
198
+ "p":0.9696969697,
199
+ "r":0.96,
200
+ "f":0.9648241206
201
  },
202
  "fixed":{
203
+ "p":0.9566003617,
204
+ "r":0.9618181818,
205
+ "f":0.9592021759
206
  },
207
  "acl":{
208
+ "p":0.8278867102,
209
+ "r":0.8351648352,
210
+ "f":0.8315098468
211
  },
212
  "obj":{
213
+ "p":0.9456193353,
214
+ "r":0.9456193353,
215
+ "f":0.9456193353
216
  },
217
  "nummod":{
218
+ "p":0.9325153374,
219
+ "r":0.899408284,
220
+ "f":0.9156626506
221
  },
222
  "advmod":{
223
+ "p":0.7054263566,
224
+ "r":0.65,
225
+ "f":0.6765799257
226
  },
227
  "amod":{
228
+ "p":0.9642857143,
229
  "r":0.7297297297,
230
+ "f":0.8307692308
231
  },
232
  "cop":{
233
+ "p":0.9700598802,
234
  "r":0.9418604651,
235
+ "f":0.9557522124
236
  },
237
  "ccomp":{
238
+ "p":0.95,
239
+ "r":0.8636363636,
240
+ "f":0.9047619048
241
  },
242
  "det":{
243
+ "p":0.9807692308,
244
+ "r":0.9622641509,
245
+ "f":0.9714285714
246
  },
247
  "csubj":{
248
+ "p":0.6153846154,
249
+ "r":0.6666666667,
250
+ "f":0.64
251
  },
252
  "dep":{
253
  "p":0.0,
 
255
  "f":0.0
256
  }
257
  },
258
+ "tag_acc":0.9712488769,
259
+ "lemma_acc":0.965013864,
260
+ "ents_p":0.7384196185,
261
+ "ents_r":0.6817610063,
262
+ "ents_f":0.7089601046,
263
  "ents_per_type":{
264
  "DATE":{
265
  "p":0.9722222222,
 
267
  "f":0.9677419355
268
  },
269
  "ORG":{
270
+ "p":0.6097560976,
271
+ "r":0.5474452555,
272
+ "f":0.5769230769
273
  },
274
  "TITLE_AFFIX":{
275
+ "p":0.8636363636,
276
+ "r":0.6333333333,
277
+ "f":0.7307692308
278
  },
279
  "PERSON":{
280
+ "p":0.762962963,
281
+ "r":0.7410071942,
282
+ "f":0.7518248175
283
  },
284
  "GPE":{
285
+ "p":0.7282608696,
286
+ "r":0.7127659574,
287
+ "f":0.7204301075
288
  },
289
  "PRODUCT":{
290
+ "p":0.4375,
291
+ "r":0.3333333333,
292
+ "f":0.3783783784
293
  },
294
  "TIME":{
295
  "p":0.6666666667,
 
297
  "f":0.8
298
  },
299
  "QUANTITY":{
300
+ "p":0.8769230769,
301
  "r":0.8636363636,
302
+ "f":0.8702290076
303
  },
304
  "NORP":{
305
+ "p":0.72,
306
+ "r":0.5625,
307
+ "f":0.6315789474
308
  },
309
  "ORDINAL":{
310
+ "p":0.5172413793,
311
  "r":0.6818181818,
312
+ "f":0.5882352941
313
  },
314
  "WORK_OF_ART":{
315
+ "p":0.5789473684,
316
+ "r":0.6470588235,
317
+ "f":0.6111111111
318
  },
319
  "FAC":{
320
+ "p":0.625,
321
+ "r":0.4054054054,
322
+ "f":0.4918032787
323
  },
324
  "PERCENT":{
325
  "p":1.0,
326
  "r":0.2857142857,
327
  "f":0.4444444444
328
  },
329
+ "EVENT":{
330
+ "p":0.7619047619,
331
+ "r":0.6153846154,
332
+ "f":0.6808510638
333
+ },
334
  "CARDINAL":{
335
  "p":0.0,
336
  "r":0.0,
337
  "f":0.0
338
  },
 
 
 
 
 
339
  "LOC":{
340
+ "p":0.5384615385,
341
+ "r":0.7,
342
+ "f":0.6086956522
343
  },
344
  "MOVEMENT":{
345
  "p":0.0,
 
351
  "r":0.3333333333,
352
  "f":0.5
353
  },
354
+ "PET_NAME":{
355
+ "p":0.0,
356
+ "r":0.0,
357
+ "f":0.0
358
+ },
359
  "MONEY":{
360
+ "p":1.0,
361
  "r":1.0,
362
+ "f":1.0
363
  },
364
  "LANGUAGE":{
365
+ "p":1.0,
366
  "r":1.0,
367
+ "f":1.0
368
  }
369
  },
370
+ "speed":8044.6200194392
371
  },
372
  "sources":[
373
  {
 
390
  }
391
  ],
392
  "requirements":[
393
+ "sudachipy>=0.5.2,!=0.6.1",
394
+ "sudachidict-core>=20211220"
395
  ]
396
  }
morphologizer/model CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f3910cf5c3ecff8a4fbb14f474cc856df94c7e5546ad252c9e48158c9a3705a7
3
- size 7749
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5489ddf9014098b25974a1783b6b2bcb71cbd118776e7a1fbbf5e8e34756652f
3
+ size 7801
ner/model CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d360074597ca1a41a1280edf2e51492a3c37a9f32d261c5aca94146bb424fa8f
3
- size 6961103
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c1e7e625dc2fa41ce822a430d8745261ddaf0c2a0084c84305847db5e73f993
3
+ size 6385103
parser/model CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cb62ea02bfeb31fe1ad0251e679371bed90741e4bb5e0ee80c04d2cbf3ab5037
3
  size 299888
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0789362978a8d034979025d4b3a086def9e3f44845be928da0d143d457613208
3
  size 299888
parser/moves CHANGED
@@ -1 +1 @@
1
- ��moves�~{"0":{"":75051},"1":{"":81581},"2":{"compound":22178,"nmod":11296,"obl":10522,"nsubj":6649,"acl":6185,"advcl":5956,"obj":4364,"nummod":2247,"advmod":1841,"punct":1169,"det":822,"cc":699,"amod":357,"ccomp":335,"dislocated":233,"csubj":139,"dep":0},"3":{"case":35390,"punct":15051,"aux":14506,"fixed":7377,"mark":6390,"cop":2079,"compound":542,"advcl":148,"dep":56},"4":{"ROOT":6810}}�cfg��neg_key�
 
1
+ ��moves�~{"0":{"":77992},"1":{"":83293},"2":{"compound":23506,"nmod":11446,"obl":11030,"nsubj":6884,"advcl":6063,"acl":6020,"obj":4629,"nummod":2487,"advmod":1922,"punct":1321,"det":830,"cc":726,"amod":372,"ccomp":325,"dislocated":235,"csubj":133,"dep":0},"3":{"case":35913,"punct":15455,"aux":14940,"fixed":7391,"mark":6644,"cop":2100,"compound":598,"advcl":152,"dep":58},"4":{"ROOT":7050}}�cfg��neg_key�
senter/model CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8d17cd0004e8619051f1230c1c7edf9a7714c18fa39596d611ae64aa00f166bc
3
- size 213211
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:19c712710d460306df4e72866bd5221d72a27366f3dca83c5d66ec0c55409481
3
+ size 213263
tok2vec/model CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9cffa83574efe695c59f59eb47aa26a625fb16b358f34bb167cb8cc4a45233f7
3
- size 6811418
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e69d2a9239d6d8c26d8033ff7e2a6d193369dfd5ffa319ef4f8c18a11537e7c9
3
+ size 6235418
vocab/strings.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8144aa155796b950df8a0ecdcb2a3d094ef4cdd57a2c4afb1df0be95d1aa7f7d
3
- size 15570329
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44e5c002847a11120089fb9a481003dfd02492a2c8669ca9625cd0a09c8ab22d
3
+ size 15613806