Hugging Face's logo

spacy
/
ja_core_news_sm

Token Classification
spaCy
Japanese
Eval Results
Model card Files Community
osanseviero commited on
Commit
ffe2dc9
·
1 Parent(s): 856faeb

Update spaCy pipeline

Browse files
Files changed (18) hide show
  1. LICENSES_SOURCES +2 -2
  2. README.md +36 -28
  3. accuracy.json +156 -139
  4. attribute_ruler/patterns +0 -0
  5. config.cfg +46 -17
  6. ja_core_news_sm-any-py3-none-any.whl +2 -2
  7. meta.json +185 -145
  8. morphologizer/cfg +46 -0
  9. morphologizer/model +3 -0
  10. ner/model +0 -0
  11. ner/moves +1 -1
  12. parser/model +0 -0
  13. parser/moves +1 -1
  14. senter/cfg +1 -1
  15. senter/model +0 -0
  16. tok2vec/model +0 -0
  17. vocab/strings.json +2 -2
  18. vocab/vectors.cfg +3 -0
LICENSES_SOURCES CHANGED
@@ -1,4 +1,4 @@
1
- # UD Japanese GSD v2.6
2
 
3
  * Author: Omura, Mai; Miyao, Yusuke; Kanayama, Hiroshi; Matsuda, Hiroshi; Wakasa, Aya; Yamashita, Kayo; Asahara, Masayuki; Tanaka, Takaaki; Murawaki, Yugo; Matsumoto, Yuji; Mori, Shinsuke; Uematsu, Sumire; McDonald, Ryan; Nivre, Joakim; Zeman, Daniel
4
  * URL: https://github.com/UniversalDependencies/UD_Japanese-GSD
@@ -438,7 +438,7 @@ Creative Commons may be contacted at creativecommons.org.
438
 
439
 
440
 
441
- # UD Japanese GSD v2.6 NER
442
 
443
  * Author: Megagon Labs Tokyo
444
  * URL: https://github.com/megagonlabs/UD_Japanese-GSD
 
1
+ # UD Japanese GSD v2.8
2
 
3
  * Author: Omura, Mai; Miyao, Yusuke; Kanayama, Hiroshi; Matsuda, Hiroshi; Wakasa, Aya; Yamashita, Kayo; Asahara, Masayuki; Tanaka, Takaaki; Murawaki, Yugo; Matsumoto, Yuji; Mori, Shinsuke; Uematsu, Sumire; McDonald, Ryan; Nivre, Joakim; Zeman, Daniel
4
  * URL: https://github.com/UniversalDependencies/UD_Japanese-GSD
 
438
 
439
 
440
 
441
+ # UD Japanese GSD v2.8 NER
442
 
443
  * Author: Megagon Labs Tokyo
444
  * URL: https://github.com/megagonlabs/UD_Japanese-GSD
README.md CHANGED
@@ -4,7 +4,7 @@ tags:
4
  - token-classification
5
  language:
6
  - ja
7
- license: CC-BY-SA-4.0
8
  model-index:
9
  - name: ja_core_news_sm
10
  results:
@@ -14,61 +14,61 @@ model-index:
14
  metrics:
15
  - name: NER Precision
16
  type: precision
17
- value: 0.7163232964
18
  - name: NER Recall
19
  type: recall
20
- value: 0.5772669221
21
  - name: NER F Score
22
  type: f_score
23
- value: 0.639321075
24
  - task:
25
  name: POS
26
  type: token-classification
27
  metrics:
28
  - name: POS Accuracy
29
  type: accuracy
30
- value: 0.9721899386
31
  - task:
32
  name: SENTER
33
  type: token-classification
34
  metrics:
35
  - name: SENTER Precision
36
  type: precision
37
- value: 0.9860557769
38
  - name: SENTER Recall
39
  type: recall
40
- value: 0.9880239521
41
  - name: SENTER F Score
42
  type: f_score
43
- value: 0.9870388833
44
  - task:
45
  name: UNLABELED_DEPENDENCIES
46
  type: token-classification
47
  metrics:
48
  - name: Unlabeled Dependencies Accuracy
49
  type: accuracy
50
- value: 0.916212877
51
  - task:
52
  name: LABELED_DEPENDENCIES
53
  type: token-classification
54
  metrics:
55
  - name: Labeled Dependencies Accuracy
56
  type: accuracy
57
- value: 0.916212877
58
  ---
59
  ### Details: https://spacy.io/models/ja#ja_core_news_sm
60
 
61
- Japanese pipeline optimized for CPU. Components: tok2vec, parser, senter, ner, attribute_ruler.
62
 
63
  | Feature | Description |
64
  | --- | --- |
65
  | **Name** | `ja_core_news_sm` |
66
- | **Version** | `3.1.0` |
67
- | **spaCy** | `>=3.1.0,<3.2.0` |
68
- | **Default Pipeline** | `tok2vec`, `parser`, `attribute_ruler`, `ner` |
69
- | **Components** | `tok2vec`, `parser`, `senter`, `attribute_ruler`, `ner` |
70
  | **Vectors** | 0 keys, 0 unique vectors (0 dimensions) |
71
- | **Sources** | [UD Japanese GSD v2.6](https://github.com/UniversalDependencies/UD_Japanese-GSD) (Omura, Mai; Miyao, Yusuke; Kanayama, Hiroshi; Matsuda, Hiroshi; Wakasa, Aya; Yamashita, Kayo; Asahara, Masayuki; Tanaka, Takaaki; Murawaki, Yugo; Matsumoto, Yuji; Mori, Shinsuke; Uematsu, Sumire; McDonald, Ryan; Nivre, Joakim; Zeman, Daniel)<br />[UD Japanese GSD v2.6 NER](https://github.com/megagonlabs/UD_Japanese-GSD) (Megagon Labs Tokyo) |
72
  | **License** | `CC BY-SA 4.0` |
73
  | **Author** | [Explosion](https://explosion.ai) |
74
 
@@ -76,10 +76,11 @@ Japanese pipeline optimized for CPU. Components: tok2vec, parser, senter, ner, a
76
 
77
  <details>
78
 
79
- <summary>View label scheme (47 labels for 3 components)</summary>
80
 
81
  | Component | Labels |
82
  | --- | --- |
 
83
  | **`parser`** | `ROOT`, `acl`, `advcl`, `advmod`, `amod`, `aux`, `case`, `cc`, `ccomp`, `compound`, `cop`, `csubj`, `dep`, `det`, `dislocated`, `fixed`, `mark`, `nmod`, `nsubj`, `nummod`, `obj`, `obl`, `punct` |
84
  | **`senter`** | `I`, `S` |
85
  | **`ner`** | `CARDINAL`, `DATE`, `EVENT`, `FAC`, `GPE`, `LANGUAGE`, `LAW`, `LOC`, `MONEY`, `MOVEMENT`, `NORP`, `ORDINAL`, `ORG`, `PERCENT`, `PERSON`, `PET_NAME`, `PHONE`, `PRODUCT`, `QUANTITY`, `TIME`, `TITLE_AFFIX`, `WORK_OF_ART` |
@@ -91,14 +92,21 @@ Japanese pipeline optimized for CPU. Components: tok2vec, parser, senter, ner, a
91
  | Type | Score |
92
  | --- | --- |
93
  | `TOKEN_ACC` | 99.69 |
94
- | `TAG_ACC` | 97.22 |
95
- | `POS_ACC` | 96.40 |
96
- | `MORPH_ACC` | 0.00 |
97
- | `DEP_UAS` | 91.62 |
98
- | `DEP_LAS` | 89.41 |
99
- | `ENTS_P` | 71.63 |
100
- | `ENTS_R` | 57.73 |
101
- | `ENTS_F` | 63.93 |
102
- | `SENTS_P` | 98.61 |
103
- | `SENTS_R` | 98.80 |
104
- | `SENTS_F` | 98.70 |
 
 
 
 
 
 
 
 
4
  - token-classification
5
  language:
6
  - ja
7
+ license: cc-by-sa-4.0
8
  model-index:
9
  - name: ja_core_news_sm
10
  results:
 
14
  metrics:
15
  - name: NER Precision
16
  type: precision
17
+ value: 0.6767371601
18
  - name: NER Recall
19
  type: recall
20
+ value: 0.5635220126
21
  - name: NER F Score
22
  type: f_score
23
+ value: 0.6149622512
24
  - task:
25
  name: POS
26
  type: token-classification
27
  metrics:
28
  - name: POS Accuracy
29
  type: accuracy
30
+ value: 0.9715755942
31
  - task:
32
  name: SENTER
33
  type: token-classification
34
  metrics:
35
  - name: SENTER Precision
36
  type: precision
37
+ value: 0.9823529412
38
  - name: SENTER Recall
39
  type: recall
40
+ value: 0.9881656805
41
  - name: SENTER F Score
42
  type: f_score
43
+ value: 0.9852507375
44
  - task:
45
  name: UNLABELED_DEPENDENCIES
46
  type: token-classification
47
  metrics:
48
  - name: Unlabeled Dependencies Accuracy
49
  type: accuracy
50
+ value: 0.921981982
51
  - task:
52
  name: LABELED_DEPENDENCIES
53
  type: token-classification
54
  metrics:
55
  - name: Labeled Dependencies Accuracy
56
  type: accuracy
57
+ value: 0.921981982
58
  ---
59
  ### Details: https://spacy.io/models/ja#ja_core_news_sm
60
 
61
+ Japanese pipeline optimized for CPU. Components: tok2vec, morphologizer, parser, senter, ner, attribute_ruler.
62
 
63
  | Feature | Description |
64
  | --- | --- |
65
  | **Name** | `ja_core_news_sm` |
66
+ | **Version** | `3.2.0` |
67
+ | **spaCy** | `>=3.2.0,<3.3.0` |
68
+ | **Default Pipeline** | `tok2vec`, `morphologizer`, `parser`, `attribute_ruler`, `ner` |
69
+ | **Components** | `tok2vec`, `morphologizer`, `parser`, `senter`, `attribute_ruler`, `ner` |
70
  | **Vectors** | 0 keys, 0 unique vectors (0 dimensions) |
71
+ | **Sources** | [UD Japanese GSD v2.8](https://github.com/UniversalDependencies/UD_Japanese-GSD) (Omura, Mai; Miyao, Yusuke; Kanayama, Hiroshi; Matsuda, Hiroshi; Wakasa, Aya; Yamashita, Kayo; Asahara, Masayuki; Tanaka, Takaaki; Murawaki, Yugo; Matsumoto, Yuji; Mori, Shinsuke; Uematsu, Sumire; McDonald, Ryan; Nivre, Joakim; Zeman, Daniel)<br />[UD Japanese GSD v2.8 NER](https://github.com/megagonlabs/UD_Japanese-GSD) (Megagon Labs Tokyo) |
72
  | **License** | `CC BY-SA 4.0` |
73
  | **Author** | [Explosion](https://explosion.ai) |
74
 
 
76
 
77
  <details>
78
 
79
+ <summary>View label scheme (66 labels for 4 components)</summary>
80
 
81
  | Component | Labels |
82
  | --- | --- |
83
+ | **`morphologizer`** | `POS=NOUN`, `POS=ADP`, `POS=VERB`, `POS=SCONJ`, `POS=AUX`, `POS=PUNCT`, `POS=PART`, `POS=DET`, `POS=NUM`, `POS=ADV`, `POS=PRON`, `POS=ADJ`, `POS=PROPN`, `POS=CCONJ`, `POS=SYM`, `POS=NOUN\|Polarity=Neg`, `POS=AUX\|Polarity=Neg`, `POS=INTJ`, `POS=SCONJ\|Polarity=Neg` |
84
  | **`parser`** | `ROOT`, `acl`, `advcl`, `advmod`, `amod`, `aux`, `case`, `cc`, `ccomp`, `compound`, `cop`, `csubj`, `dep`, `det`, `dislocated`, `fixed`, `mark`, `nmod`, `nsubj`, `nummod`, `obj`, `obl`, `punct` |
85
  | **`senter`** | `I`, `S` |
86
  | **`ner`** | `CARDINAL`, `DATE`, `EVENT`, `FAC`, `GPE`, `LANGUAGE`, `LAW`, `LOC`, `MONEY`, `MOVEMENT`, `NORP`, `ORDINAL`, `ORG`, `PERCENT`, `PERSON`, `PET_NAME`, `PHONE`, `PRODUCT`, `QUANTITY`, `TIME`, `TITLE_AFFIX`, `WORK_OF_ART` |
 
92
  | Type | Score |
93
  | --- | --- |
94
  | `TOKEN_ACC` | 99.69 |
95
+ | `TOKEN_P` | 97.65 |
96
+ | `TOKEN_R` | 97.90 |
97
+ | `TOKEN_F` | 97.77 |
98
+ | `POS_ACC` | 96.25 |
99
+ | `MORPH_ACC` | 0.40 |
100
+ | `MORPH_MICRO_P` | 34.01 |
101
+ | `MORPH_MICRO_R` | 98.04 |
102
+ | `MORPH_MICRO_F` | 50.51 |
103
+ | `SENTS_P` | 98.24 |
104
+ | `SENTS_R` | 98.82 |
105
+ | `SENTS_F` | 98.53 |
106
+ | `DEP_UAS` | 92.20 |
107
+ | `DEP_LAS` | 90.69 |
108
+ | `TAG_ACC` | 97.16 |
109
+ | `LEMMA_ACC` | 96.59 |
110
+ | `ENTS_P` | 67.67 |
111
+ | `ENTS_R` | 56.35 |
112
+ | `ENTS_F` | 61.50 |
accuracy.json CHANGED
@@ -1,211 +1,232 @@
1
  {
2
- "token_acc": 0.9968965945,
3
- "tag_acc": 0.9721899386,
4
- "pos_acc": 0.9639755682,
5
- "morph_acc": 0.0,
6
- "dep_uas": 0.916212877,
7
- "dep_las": 0.894130554,
8
- "ents_p": 0.7163232964,
9
- "ents_r": 0.5772669221,
10
- "ents_f": 0.639321075,
11
- "sents_p": 0.9860557769,
12
- "sents_r": 0.9880239521,
13
- "sents_f": 0.9870388833,
14
- "speed": 14745.6510174231,
15
  "morph_per_feat": {
16
  "Polarity": {
 
 
 
 
 
 
 
 
 
 
17
  "p": 0.0,
18
  "r": 0.0,
19
  "f": 0.0
20
  }
21
  },
 
 
 
 
 
22
  "dep_las_per_type": {
23
  "cc": {
24
- "p": 0.7659574468,
25
- "r": 0.7826086957,
26
- "f": 0.7741935484
27
- },
28
- "nummod": {
29
- "p": 0.9730769231,
30
- "r": 0.8694158076,
31
- "f": 0.9183303085
32
  },
33
  "compound": {
34
- "p": 0.9299242424,
35
- "r": 0.9143389199,
36
- "f": 0.9220657277
37
  },
38
  "obl": {
39
- "p": 0.7782857143,
40
- "r": 0.8185096154,
41
- "f": 0.7978910369
42
  },
43
  "case": {
44
- "p": 0.9808429119,
45
- "r": 0.9774723177,
46
- "f": 0.9791547141
47
  },
48
  "dislocated": {
49
- "p": 0.4285714286,
50
- "r": 0.3157894737,
51
- "f": 0.3636363636
52
- },
53
- "nmod": {
54
- "p": 0.8792650919,
55
- "r": 0.8072289157,
56
- "f": 0.8417085427
57
  },
58
  "nsubj": {
59
- "p": 0.8020833333,
60
- "r": 0.8020833333,
61
- "f": 0.8020833333
 
 
 
 
 
62
  },
63
  "root": {
64
- "p": 0.967611336,
65
- "r": 0.9540918164,
66
- "f": 0.9608040201
67
  },
68
  "aux": {
69
- "p": 0.9545454545,
70
- "r": 0.9586956522,
71
- "f": 0.9566160521
72
  },
73
  "advcl": {
74
- "p": 0.6402877698,
75
- "r": 0.6223776224,
76
- "f": 0.6312056738
77
  },
78
  "mark": {
79
- "p": 0.953815261,
80
- "r": 0.9350393701,
81
- "f": 0.944333996
 
 
 
 
 
82
  },
83
  "acl": {
84
- "p": 0.8161434978,
85
- "r": 0.8017621145,
86
- "f": 0.8088888889
87
  },
88
  "obj": {
89
- "p": 0.9254658385,
90
- "r": 0.9085365854,
91
- "f": 0.9169230769
92
  },
93
- "fixed": {
94
- "p": 0.9572192513,
95
- "r": 0.9835164835,
96
- "f": 0.9701897019
97
  },
98
  "advmod": {
99
- "p": 0.724137931,
100
- "r": 0.5,
101
- "f": 0.5915492958
102
  },
103
  "amod": {
104
- "p": 0.7941176471,
105
- "r": 0.675,
106
- "f": 0.7297297297
107
  },
108
  "cop": {
109
- "p": 0.9482758621,
110
- "r": 0.9065934066,
111
- "f": 0.9269662921
112
  },
113
  "ccomp": {
114
- "p": 0.9473684211,
115
  "r": 0.8181818182,
116
- "f": 0.8780487805
117
  },
118
  "det": {
119
- "p": 0.9607843137,
120
- "r": 0.9607843137,
121
- "f": 0.9607843137
122
- },
123
- "csubj": {
124
- "p": 0.8333333333,
125
- "r": 0.7692307692,
126
- "f": 0.8
127
  },
128
  "dep": {
129
- "p": 0.0,
130
- "r": 0.0,
131
- "f": 0.0
 
 
 
 
 
132
  }
133
  },
 
 
 
 
 
134
  "ents_per_type": {
135
  "DATE": {
136
- "p": 0.9272727273,
137
- "r": 0.9444444444,
138
- "f": 0.9357798165
139
  },
140
- "PERSON": {
141
- "p": 0.6464646465,
142
- "r": 0.4604316547,
143
- "f": 0.5378151261
144
  },
145
  "GPE": {
146
- "p": 0.6455696203,
147
  "r": 0.5425531915,
148
- "f": 0.5895953757
149
  },
150
  "PRODUCT": {
151
- "p": 0.55,
152
- "r": 0.2682926829,
153
- "f": 0.3606557377
154
  },
155
  "TIME": {
156
- "p": 0.6666666667,
157
  "r": 1.0,
158
- "f": 0.8
159
  },
160
  "QUANTITY": {
161
- "p": 0.9090909091,
162
- "r": 0.9090909091,
163
- "f": 0.9090909091
 
 
 
 
 
164
  },
165
  "NORP": {
166
- "p": 0.7826086957,
167
  "r": 0.5625,
168
- "f": 0.6545454545
169
- },
170
- "ORDINAL": {
171
- "p": 0.5909090909,
172
- "r": 0.6842105263,
173
- "f": 0.6341463415
174
  },
175
  "TITLE_AFFIX": {
176
- "p": 0.7647058824,
177
- "r": 0.4333333333,
178
- "f": 0.5531914894
179
  },
180
- "ORG": {
181
- "p": 0.504950495,
182
- "r": 0.3893129771,
183
- "f": 0.4396551724
184
  },
185
  "WORK_OF_ART": {
186
- "p": 0.7222222222,
187
- "r": 0.7647058824,
188
- "f": 0.7428571429
 
 
 
 
 
189
  },
190
  "PERCENT": {
191
  "p": 1.0,
192
- "r": 0.4285714286,
193
- "f": 0.6
194
  },
195
- "EVENT": {
196
- "p": 0.7619047619,
197
- "r": 0.6153846154,
198
- "f": 0.6808510638
199
  },
200
  "LOC": {
201
- "p": 0.6363636364,
202
  "r": 0.7,
203
- "f": 0.6666666667
204
  },
205
  "FAC": {
206
- "p": 0.6315789474,
207
- "r": 0.3243243243,
208
- "f": 0.4285714286
209
  },
210
  "MOVEMENT": {
211
  "p": 0.0,
@@ -213,9 +234,9 @@
213
  "f": 0.0
214
  },
215
  "LAW": {
216
- "p": 1.0,
217
- "r": 0.3333333333,
218
- "f": 0.5
219
  },
220
  "MONEY": {
221
  "p": 1.0,
@@ -226,11 +247,7 @@
226
  "p": 1.0,
227
  "r": 1.0,
228
  "f": 1.0
229
- },
230
- "CARDINAL": {
231
- "p": 0.0,
232
- "r": 0.0,
233
- "f": 0.0
234
  }
235
- }
 
236
  }
 
1
  {
2
+ "token_acc": 0.9968649485,
3
+ "token_p": 0.9764591282,
4
+ "token_r": 0.9790021974,
5
+ "token_f": 0.9777290092,
6
+ "pos_acc": 0.9624933535,
7
+ "morph_acc": 0.0040005162,
8
+ "morph_micro_p": 0.3401360544,
9
+ "morph_micro_r": 0.9803921569,
10
+ "morph_micro_f": 0.5050505051,
 
 
 
 
11
  "morph_per_feat": {
12
  "Polarity": {
13
+ "p": 1.0,
14
+ "r": 0.9803921569,
15
+ "f": 0.9900990099
16
+ },
17
+ "Inflection": {
18
+ "p": 0.0,
19
+ "r": 0.0,
20
+ "f": 0.0
21
+ },
22
+ "Reading": {
23
  "p": 0.0,
24
  "r": 0.0,
25
  "f": 0.0
26
  }
27
  },
28
+ "sents_p": 0.9823529412,
29
+ "sents_r": 0.9881656805,
30
+ "sents_f": 0.9852507375,
31
+ "dep_uas": 0.921981982,
32
+ "dep_las": 0.9069390694,
33
  "dep_las_per_type": {
34
  "cc": {
35
+ "p": 0.8163265306,
36
+ "r": 0.8333333333,
37
+ "f": 0.824742268
 
 
 
 
 
38
  },
39
  "compound": {
40
+ "p": 0.9411421911,
41
+ "r": 0.9103720406,
42
+ "f": 0.9255014327
43
  },
44
  "obl": {
45
+ "p": 0.8113207547,
46
+ "r": 0.8052434457,
47
+ "f": 0.8082706767
48
  },
49
  "case": {
50
+ "p": 0.9892638037,
51
+ "r": 0.9802431611,
52
+ "f": 0.9847328244
53
  },
54
  "dislocated": {
55
+ "p": 0.7,
56
+ "r": 0.5384615385,
57
+ "f": 0.6086956522
 
 
 
 
 
58
  },
59
  "nsubj": {
60
+ "p": 0.8202676864,
61
+ "r": 0.8234165067,
62
+ "f": 0.8218390805
63
+ },
64
+ "nmod": {
65
+ "p": 0.8793532338,
66
+ "r": 0.8269005848,
67
+ "f": 0.8523206751
68
  },
69
  "root": {
70
+ "p": 0.9741035857,
71
+ "r": 0.9644970414,
72
+ "f": 0.9692765114
73
  },
74
  "aux": {
75
+ "p": 0.9796296296,
76
+ "r": 0.982358403,
77
+ "f": 0.9809921187
78
  },
79
  "advcl": {
80
+ "p": 0.6893424036,
81
+ "r": 0.6831460674,
82
+ "f": 0.6862302483
83
  },
84
  "mark": {
85
+ "p": 0.9755600815,
86
+ "r": 0.958,
87
+ "f": 0.9667003027
88
+ },
89
+ "fixed": {
90
+ "p": 0.9569892473,
91
+ "r": 0.9709090909,
92
+ "f": 0.963898917
93
  },
94
  "acl": {
95
+ "p": 0.8239130435,
96
+ "r": 0.832967033,
97
+ "f": 0.8284153005
98
  },
99
  "obj": {
100
+ "p": 0.9507692308,
101
+ "r": 0.9335347432,
102
+ "f": 0.9420731707
103
  },
104
+ "nummod": {
105
+ "p": 0.9934640523,
106
+ "r": 0.899408284,
107
+ "f": 0.9440993789
108
  },
109
  "advmod": {
110
+ "p": 0.6771653543,
111
+ "r": 0.6142857143,
112
+ "f": 0.6441947566
113
  },
114
  "amod": {
115
+ "p": 0.8709677419,
116
+ "r": 0.7297297297,
117
+ "f": 0.7941176471
118
  },
119
  "cop": {
120
+ "p": 0.9464285714,
121
+ "r": 0.9244186047,
122
+ "f": 0.9352941176
123
  },
124
  "ccomp": {
125
+ "p": 0.8571428571,
126
  "r": 0.8181818182,
127
+ "f": 0.8372093023
128
  },
129
  "det": {
130
+ "p": 1.0,
131
+ "r": 0.9811320755,
132
+ "f": 0.9904761905
 
 
 
 
 
133
  },
134
  "dep": {
135
+ "p": 0.0714285714,
136
+ "r": 0.1428571429,
137
+ "f": 0.0952380952
138
+ },
139
+ "csubj": {
140
+ "p": 0.6363636364,
141
+ "r": 0.5833333333,
142
+ "f": 0.6086956522
143
  }
144
  },
145
+ "tag_acc": 0.9715755942,
146
+ "lemma_acc": 0.9659109444,
147
+ "ents_p": 0.6767371601,
148
+ "ents_r": 0.5635220126,
149
+ "ents_f": 0.6149622512,
150
  "ents_per_type": {
151
  "DATE": {
152
+ "p": 0.9285714286,
153
+ "r": 0.9541284404,
154
+ "f": 0.9411764706
155
  },
156
+ "ORG": {
157
+ "p": 0.5046728972,
158
+ "r": 0.3941605839,
159
+ "f": 0.4426229508
160
  },
161
  "GPE": {
162
+ "p": 0.6296296296,
163
  "r": 0.5425531915,
164
+ "f": 0.5828571429
165
  },
166
  "PRODUCT": {
167
+ "p": 0.4285714286,
168
+ "r": 0.2142857143,
169
+ "f": 0.2857142857
170
  },
171
  "TIME": {
172
+ "p": 0.5714285714,
173
  "r": 1.0,
174
+ "f": 0.7272727273
175
  },
176
  "QUANTITY": {
177
+ "p": 0.7972972973,
178
+ "r": 0.8939393939,
179
+ "f": 0.8428571429
180
+ },
181
+ "PERSON": {
182
+ "p": 0.6346153846,
183
+ "r": 0.4748201439,
184
+ "f": 0.5432098765
185
  },
186
  "NORP": {
187
+ "p": 0.6666666667,
188
  "r": 0.5625,
189
+ "f": 0.6101694915
 
 
 
 
 
190
  },
191
  "TITLE_AFFIX": {
192
+ "p": 0.7142857143,
193
+ "r": 0.5,
194
+ "f": 0.5882352941
195
  },
196
+ "ORDINAL": {
197
+ "p": 0.5909090909,
198
+ "r": 0.5909090909,
199
+ "f": 0.5909090909
200
  },
201
  "WORK_OF_ART": {
202
+ "p": 0.7692307692,
203
+ "r": 0.5882352941,
204
+ "f": 0.6666666667
205
+ },
206
+ "EVENT": {
207
+ "p": 0.6,
208
+ "r": 0.4615384615,
209
+ "f": 0.5217391304
210
  },
211
  "PERCENT": {
212
  "p": 1.0,
213
+ "r": 0.2857142857,
214
+ "f": 0.4444444444
215
  },
216
+ "CARDINAL": {
217
+ "p": 0.0,
218
+ "r": 0.0,
219
+ "f": 0.0
220
  },
221
  "LOC": {
222
+ "p": 0.7,
223
  "r": 0.7,
224
+ "f": 0.7
225
  },
226
  "FAC": {
227
+ "p": 0.44,
228
+ "r": 0.2972972973,
229
+ "f": 0.3548387097
230
  },
231
  "MOVEMENT": {
232
  "p": 0.0,
 
234
  "f": 0.0
235
  },
236
  "LAW": {
237
+ "p": 0.0,
238
+ "r": 0.0,
239
+ "f": 0.0
240
  },
241
  "MONEY": {
242
  "p": 1.0,
 
247
  "p": 1.0,
248
  "r": 1.0,
249
  "f": 1.0
 
 
 
 
 
250
  }
251
+ },
252
+ "speed": 4844.6752174676
253
  }
attribute_ruler/patterns CHANGED
Binary files a/attribute_ruler/patterns and b/attribute_ruler/patterns differ
 
config.cfg CHANGED
@@ -1,10 +1,8 @@
1
  [paths]
2
- train = "corpus/ja-core-news/train.spacy"
3
- dev = "corpus/ja-core-news/dev.spacy"
4
  vectors = null
5
- raw = null
6
  init_tok2vec = null
7
- vocab_data = null
8
 
9
  [system]
10
  gpu_allocator = null
@@ -12,7 +10,7 @@ seed = 0
12
 
13
  [nlp]
14
  lang = "ja"
15
- pipeline = ["tok2vec","parser","senter","attribute_ruler","ner"]
16
  disabled = ["senter"]
17
  before_creation = null
18
  after_creation = null
@@ -27,12 +25,29 @@ split_mode = null
27
 
28
  [components.attribute_ruler]
29
  factory = "attribute_ruler"
 
30
  validate = false
31
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
  [components.ner]
33
  factory = "ner"
34
  incorrect_spans_key = null
35
  moves = null
 
36
  update_with_oracle_cut_size = 100
37
 
38
  [components.ner.model]
@@ -66,6 +81,7 @@ factory = "parser"
66
  learn_tokens = false
67
  min_action_freq = 30
68
  moves = null
 
69
  update_with_oracle_cut_size = 100
70
 
71
  [components.parser.model]
@@ -84,6 +100,8 @@ upstream = "tok2vec"
84
 
85
  [components.senter]
86
  factory = "senter"
 
 
87
 
88
  [components.senter.model]
89
  @architectures = "spacy.Tagger.v1"
@@ -130,17 +148,17 @@ maxout_pieces = 3
130
 
131
  [corpora.dev]
132
  @readers = "spacy.Corpus.v1"
133
- limit = 0
134
- max_length = 0
135
- path = ${paths:dev}
136
  gold_preproc = false
 
 
137
  augmenter = null
138
 
139
  [corpora.train]
140
  @readers = "spacy.Corpus.v1"
141
- path = ${paths:train}
142
- max_length = 5000
143
  gold_preproc = false
 
144
  limit = 0
145
  augmenter = null
146
 
@@ -173,9 +191,8 @@ compound = 1.001
173
  t = 0.0
174
 
175
  [training.logger]
176
- @loggers = "spacy.WandbLogger.v1"
177
- project_name = "spacy-v3.0.0a2"
178
- remove_config_values = []
179
 
180
  [training.optimizer]
181
  @optimizers = "Adam.v1"
@@ -189,21 +206,26 @@ eps = 0.00000001
189
  learn_rate = 0.001
190
 
191
  [training.score_weights]
 
 
 
192
  dep_uas = 0.0
193
- dep_las = 0.45
194
  dep_las_per_type = null
195
  sents_p = null
196
  sents_r = null
197
- sents_f = 0.06
198
- ents_f = 0.5
199
  ents_p = 0.0
200
  ents_r = 0.0
201
  ents_per_type = null
 
 
202
 
203
  [pretraining]
204
 
205
  [initialize]
206
- vocab_data = ${paths.vocab_data}
207
  vectors = ${paths.vectors}
208
  init_tok2vec = ${paths.init_tok2vec}
209
  before_init = null
@@ -211,6 +233,13 @@ after_init = null
211
 
212
  [initialize.components]
213
 
 
 
 
 
 
 
 
214
  [initialize.components.ner]
215
 
216
  [initialize.components.ner.labels]
 
1
  [paths]
2
+ train = null
3
+ dev = null
4
  vectors = null
 
5
  init_tok2vec = null
 
6
 
7
  [system]
8
  gpu_allocator = null
 
10
 
11
  [nlp]
12
  lang = "ja"
13
+ pipeline = ["tok2vec","morphologizer","parser","senter","attribute_ruler","ner"]
14
  disabled = ["senter"]
15
  before_creation = null
16
  after_creation = null
 
25
 
26
  [components.attribute_ruler]
27
  factory = "attribute_ruler"
28
+ scorer = {"@scorers":"spacy.attribute_ruler_scorer.v1"}
29
  validate = false
30
 
31
+ [components.morphologizer]
32
+ factory = "morphologizer"
33
+ extend = true
34
+ overwrite = true
35
+ scorer = {"@scorers":"spacy.morphologizer_scorer.v1"}
36
+
37
+ [components.morphologizer.model]
38
+ @architectures = "spacy.Tagger.v1"
39
+ nO = null
40
+
41
+ [components.morphologizer.model.tok2vec]
42
+ @architectures = "spacy.Tok2VecListener.v1"
43
+ width = ${components.tok2vec.model.encode:width}
44
+ upstream = "tok2vec"
45
+
46
  [components.ner]
47
  factory = "ner"
48
  incorrect_spans_key = null
49
  moves = null
50
+ scorer = {"@scorers":"spacy.ner_scorer.v1"}
51
  update_with_oracle_cut_size = 100
52
 
53
  [components.ner.model]
 
81
  learn_tokens = false
82
  min_action_freq = 30
83
  moves = null
84
+ scorer = {"@scorers":"spacy.parser_scorer.v1"}
85
  update_with_oracle_cut_size = 100
86
 
87
  [components.parser.model]
 
100
 
101
  [components.senter]
102
  factory = "senter"
103
+ overwrite = false
104
+ scorer = {"@scorers":"spacy.senter_scorer.v1"}
105
 
106
  [components.senter.model]
107
  @architectures = "spacy.Tagger.v1"
 
148
 
149
  [corpora.dev]
150
  @readers = "spacy.Corpus.v1"
151
+ path = ${paths.dev}
 
 
152
  gold_preproc = false
153
+ max_length = 0
154
+ limit = 0
155
  augmenter = null
156
 
157
  [corpora.train]
158
  @readers = "spacy.Corpus.v1"
159
+ path = ${paths.train}
 
160
  gold_preproc = false
161
+ max_length = 0
162
  limit = 0
163
  augmenter = null
164
 
 
191
  t = 0.0
192
 
193
  [training.logger]
194
+ @loggers = "spacy.ConsoleLogger.v1"
195
+ progress_bar = false
 
196
 
197
  [training.optimizer]
198
  @optimizers = "Adam.v1"
 
206
  learn_rate = 0.001
207
 
208
  [training.score_weights]
209
+ pos_acc = 0.11
210
+ morph_micro_f = 0.33
211
+ morph_per_feat = null
212
  dep_uas = 0.0
213
+ dep_las = 0.21
214
  dep_las_per_type = null
215
  sents_p = null
216
  sents_r = null
217
+ sents_f = 0.03
218
+ ents_f = 0.21
219
  ents_p = 0.0
220
  ents_r = 0.0
221
  ents_per_type = null
222
+ morph_acc = 0.11
223
+ speed = 0.0
224
 
225
  [pretraining]
226
 
227
  [initialize]
228
+ vocab_data = null
229
  vectors = ${paths.vectors}
230
  init_tok2vec = ${paths.init_tok2vec}
231
  before_init = null
 
233
 
234
  [initialize.components]
235
 
236
+ [initialize.components.morphologizer]
237
+
238
+ [initialize.components.morphologizer.labels]
239
+ @readers = "spacy.read_labels.v1"
240
+ path = "corpus/labels/morphologizer.json"
241
+ require = false
242
+
243
  [initialize.components.ner]
244
 
245
  [initialize.components.ner.labels]
ja_core_news_sm-any-py3-none-any.whl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b45400d19cd9682fb40946402532000ad81164abbaff6828abb012929ea918c8
3
- size 12941362
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be790d131bfe78f83dfdf9205bdefd603a8438a8b3f125e85dcd3a309f605f53
3
+ size 13032813
meta.json CHANGED
@@ -1,14 +1,14 @@
1
  {
2
  "lang":"ja",
3
  "name":"core_news_sm",
4
- "version":"3.1.0",
5
- "description":"Japanese pipeline optimized for CPU. Components: tok2vec, parser, senter, ner, attribute_ruler.",
6
  "author":"Explosion",
7
  "email":"contact@explosion.ai",
8
  "url":"https://explosion.ai",
9
  "license":"CC BY-SA 4.0",
10
- "spacy_version":">=3.1.0,<3.2.0",
11
- "spacy_git_version":"caba63b74",
12
  "vectors":{
13
  "width":0,
14
  "vectors":0,
@@ -18,6 +18,27 @@
18
  "labels":{
19
  "tok2vec":[
20
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
  ],
22
  "parser":[
23
  "ROOT",
@@ -78,12 +99,14 @@
78
  },
79
  "pipeline":[
80
  "tok2vec",
 
81
  "parser",
82
  "attribute_ruler",
83
  "ner"
84
  ],
85
  "components":[
86
  "tok2vec",
 
87
  "parser",
88
  "senter",
89
  "attribute_ruler",
@@ -93,213 +116,234 @@
93
  "senter"
94
  ],
95
  "performance":{
96
- "token_acc":0.9968965945,
97
- "tag_acc":0.9721899386,
98
- "pos_acc":0.9639755682,
99
- "morph_acc":0.0,
100
- "dep_uas":0.916212877,
101
- "dep_las":0.894130554,
102
- "ents_p":0.7163232964,
103
- "ents_r":0.5772669221,
104
- "ents_f":0.639321075,
105
- "sents_p":0.9860557769,
106
- "sents_r":0.9880239521,
107
- "sents_f":0.9870388833,
108
- "speed":14745.6510174231,
109
  "morph_per_feat":{
110
  "Polarity":{
 
 
 
 
 
 
 
 
 
 
111
  "p":0.0,
112
  "r":0.0,
113
  "f":0.0
114
  }
115
  },
 
 
 
 
 
116
  "dep_las_per_type":{
117
  "cc":{
118
- "p":0.7659574468,
119
- "r":0.7826086957,
120
- "f":0.7741935484
121
- },
122
- "nummod":{
123
- "p":0.9730769231,
124
- "r":0.8694158076,
125
- "f":0.9183303085
126
  },
127
  "compound":{
128
- "p":0.9299242424,
129
- "r":0.9143389199,
130
- "f":0.9220657277
131
  },
132
  "obl":{
133
- "p":0.7782857143,
134
- "r":0.8185096154,
135
- "f":0.7978910369
136
  },
137
  "case":{
138
- "p":0.9808429119,
139
- "r":0.9774723177,
140
- "f":0.9791547141
141
  },
142
  "dislocated":{
143
- "p":0.4285714286,
144
- "r":0.3157894737,
145
- "f":0.3636363636
146
- },
147
- "nmod":{
148
- "p":0.8792650919,
149
- "r":0.8072289157,
150
- "f":0.8417085427
151
  },
152
  "nsubj":{
153
- "p":0.8020833333,
154
- "r":0.8020833333,
155
- "f":0.8020833333
 
 
 
 
 
156
  },
157
  "root":{
158
- "p":0.967611336,
159
- "r":0.9540918164,
160
- "f":0.9608040201
161
  },
162
  "aux":{
163
- "p":0.9545454545,
164
- "r":0.9586956522,
165
- "f":0.9566160521
166
  },
167
  "advcl":{
168
- "p":0.6402877698,
169
- "r":0.6223776224,
170
- "f":0.6312056738
171
  },
172
  "mark":{
173
- "p":0.953815261,
174
- "r":0.9350393701,
175
- "f":0.944333996
 
 
 
 
 
176
  },
177
  "acl":{
178
- "p":0.8161434978,
179
- "r":0.8017621145,
180
- "f":0.8088888889
181
  },
182
  "obj":{
183
- "p":0.9254658385,
184
- "r":0.9085365854,
185
- "f":0.9169230769
186
  },
187
- "fixed":{
188
- "p":0.9572192513,
189
- "r":0.9835164835,
190
- "f":0.9701897019
191
  },
192
  "advmod":{
193
- "p":0.724137931,
194
- "r":0.5,
195
- "f":0.5915492958
196
  },
197
  "amod":{
198
- "p":0.7941176471,
199
- "r":0.675,
200
- "f":0.7297297297
201
  },
202
  "cop":{
203
- "p":0.9482758621,
204
- "r":0.9065934066,
205
- "f":0.9269662921
206
  },
207
  "ccomp":{
208
- "p":0.9473684211,
209
  "r":0.8181818182,
210
- "f":0.8780487805
211
  },
212
  "det":{
213
- "p":0.9607843137,
214
- "r":0.9607843137,
215
- "f":0.9607843137
216
- },
217
- "csubj":{
218
- "p":0.8333333333,
219
- "r":0.7692307692,
220
- "f":0.8
221
  },
222
  "dep":{
223
- "p":0.0,
224
- "r":0.0,
225
- "f":0.0
 
 
 
 
 
226
  }
227
  },
 
 
 
 
 
228
  "ents_per_type":{
229
  "DATE":{
230
- "p":0.9272727273,
231
- "r":0.9444444444,
232
- "f":0.9357798165
233
  },
234
- "PERSON":{
235
- "p":0.6464646465,
236
- "r":0.4604316547,
237
- "f":0.5378151261
238
  },
239
  "GPE":{
240
- "p":0.6455696203,
241
  "r":0.5425531915,
242
- "f":0.5895953757
243
  },
244
  "PRODUCT":{
245
- "p":0.55,
246
- "r":0.2682926829,
247
- "f":0.3606557377
248
  },
249
  "TIME":{
250
- "p":0.6666666667,
251
  "r":1.0,
252
- "f":0.8
253
  },
254
  "QUANTITY":{
255
- "p":0.9090909091,
256
- "r":0.9090909091,
257
- "f":0.9090909091
 
 
 
 
 
258
  },
259
  "NORP":{
260
- "p":0.7826086957,
261
  "r":0.5625,
262
- "f":0.6545454545
263
- },
264
- "ORDINAL":{
265
- "p":0.5909090909,
266
- "r":0.6842105263,
267
- "f":0.6341463415
268
  },
269
  "TITLE_AFFIX":{
270
- "p":0.7647058824,
271
- "r":0.4333333333,
272
- "f":0.5531914894
273
  },
274
- "ORG":{
275
- "p":0.504950495,
276
- "r":0.3893129771,
277
- "f":0.4396551724
278
  },
279
  "WORK_OF_ART":{
280
- "p":0.7222222222,
281
- "r":0.7647058824,
282
- "f":0.7428571429
 
 
 
 
 
283
  },
284
  "PERCENT":{
285
  "p":1.0,
286
- "r":0.4285714286,
287
- "f":0.6
288
  },
289
- "EVENT":{
290
- "p":0.7619047619,
291
- "r":0.6153846154,
292
- "f":0.6808510638
293
  },
294
  "LOC":{
295
- "p":0.6363636364,
296
  "r":0.7,
297
- "f":0.6666666667
298
  },
299
  "FAC":{
300
- "p":0.6315789474,
301
- "r":0.3243243243,
302
- "f":0.4285714286
303
  },
304
  "MOVEMENT":{
305
  "p":0.0,
@@ -307,9 +351,9 @@
307
  "f":0.0
308
  },
309
  "LAW":{
310
- "p":1.0,
311
- "r":0.3333333333,
312
- "f":0.5
313
  },
314
  "MONEY":{
315
  "p":1.0,
@@ -320,23 +364,19 @@
320
  "p":1.0,
321
  "r":1.0,
322
  "f":1.0
323
- },
324
- "CARDINAL":{
325
- "p":0.0,
326
- "r":0.0,
327
- "f":0.0
328
  }
329
- }
 
330
  },
331
  "sources":[
332
  {
333
- "name":"UD Japanese GSD v2.6",
334
  "url":"https://github.com/UniversalDependencies/UD_Japanese-GSD",
335
  "license":"CC BY-SA 4.0",
336
  "author":"Omura, Mai; Miyao, Yusuke; Kanayama, Hiroshi; Matsuda, Hiroshi; Wakasa, Aya; Yamashita, Kayo; Asahara, Masayuki; Tanaka, Takaaki; Murawaki, Yugo; Matsumoto, Yuji; Mori, Shinsuke; Uematsu, Sumire; McDonald, Ryan; Nivre, Joakim; Zeman, Daniel"
337
  },
338
  {
339
- "name":"UD Japanese GSD v2.6 NER",
340
  "url":"https://github.com/megagonlabs/UD_Japanese-GSD",
341
  "license":"CC BY-SA 4.0",
342
  "author":"Megagon Labs Tokyo"
 
1
  {
2
  "lang":"ja",
3
  "name":"core_news_sm",
4
+ "version":"3.2.0",
5
+ "description":"Japanese pipeline optimized for CPU. Components: tok2vec, morphologizer, parser, senter, ner, attribute_ruler.",
6
  "author":"Explosion",
7
  "email":"contact@explosion.ai",
8
  "url":"https://explosion.ai",
9
  "license":"CC BY-SA 4.0",
10
+ "spacy_version":">=3.2.0,<3.3.0",
11
+ "spacy_git_version":"bb26550e2",
12
  "vectors":{
13
  "width":0,
14
  "vectors":0,
 
18
  "labels":{
19
  "tok2vec":[
20
 
21
+ ],
22
+ "morphologizer":[
23
+ "POS=NOUN",
24
+ "POS=ADP",
25
+ "POS=VERB",
26
+ "POS=SCONJ",
27
+ "POS=AUX",
28
+ "POS=PUNCT",
29
+ "POS=PART",
30
+ "POS=DET",
31
+ "POS=NUM",
32
+ "POS=ADV",
33
+ "POS=PRON",
34
+ "POS=ADJ",
35
+ "POS=PROPN",
36
+ "POS=CCONJ",
37
+ "POS=SYM",
38
+ "POS=NOUN|Polarity=Neg",
39
+ "POS=AUX|Polarity=Neg",
40
+ "POS=INTJ",
41
+ "POS=SCONJ|Polarity=Neg"
42
  ],
43
  "parser":[
44
  "ROOT",
 
99
  },
100
  "pipeline":[
101
  "tok2vec",
102
+ "morphologizer",
103
  "parser",
104
  "attribute_ruler",
105
  "ner"
106
  ],
107
  "components":[
108
  "tok2vec",
109
+ "morphologizer",
110
  "parser",
111
  "senter",
112
  "attribute_ruler",
 
116
  "senter"
117
  ],
118
  "performance":{
119
+ "token_acc":0.9968649485,
120
+ "token_p":0.9764591282,
121
+ "token_r":0.9790021974,
122
+ "token_f":0.9777290092,
123
+ "pos_acc":0.9624933535,
124
+ "morph_acc":0.0040005162,
125
+ "morph_micro_p":0.3401360544,
126
+ "morph_micro_r":0.9803921569,
127
+ "morph_micro_f":0.5050505051,
 
 
 
 
128
  "morph_per_feat":{
129
  "Polarity":{
130
+ "p":1.0,
131
+ "r":0.9803921569,
132
+ "f":0.9900990099
133
+ },
134
+ "Inflection":{
135
+ "p":0.0,
136
+ "r":0.0,
137
+ "f":0.0
138
+ },
139
+ "Reading":{
140
  "p":0.0,
141
  "r":0.0,
142
  "f":0.0
143
  }
144
  },
145
+ "sents_p":0.9823529412,
146
+ "sents_r":0.9881656805,
147
+ "sents_f":0.9852507375,
148
+ "dep_uas":0.921981982,
149
+ "dep_las":0.9069390694,
150
  "dep_las_per_type":{
151
  "cc":{
152
+ "p":0.8163265306,
153
+ "r":0.8333333333,
154
+ "f":0.824742268
 
 
 
 
 
155
  },
156
  "compound":{
157
+ "p":0.9411421911,
158
+ "r":0.9103720406,
159
+ "f":0.9255014327
160
  },
161
  "obl":{
162
+ "p":0.8113207547,
163
+ "r":0.8052434457,
164
+ "f":0.8082706767
165
  },
166
  "case":{
167
+ "p":0.9892638037,
168
+ "r":0.9802431611,
169
+ "f":0.9847328244
170
  },
171
  "dislocated":{
172
+ "p":0.7,
173
+ "r":0.5384615385,
174
+ "f":0.6086956522
 
 
 
 
 
175
  },
176
  "nsubj":{
177
+ "p":0.8202676864,
178
+ "r":0.8234165067,
179
+ "f":0.8218390805
180
+ },
181
+ "nmod":{
182
+ "p":0.8793532338,
183
+ "r":0.8269005848,
184
+ "f":0.8523206751
185
  },
186
  "root":{
187
+ "p":0.9741035857,
188
+ "r":0.9644970414,
189
+ "f":0.9692765114
190
  },
191
  "aux":{
192
+ "p":0.9796296296,
193
+ "r":0.982358403,
194
+ "f":0.9809921187
195
  },
196
  "advcl":{
197
+ "p":0.6893424036,
198
+ "r":0.6831460674,
199
+ "f":0.6862302483
200
  },
201
  "mark":{
202
+ "p":0.9755600815,
203
+ "r":0.958,
204
+ "f":0.9667003027
205
+ },
206
+ "fixed":{
207
+ "p":0.9569892473,
208
+ "r":0.9709090909,
209
+ "f":0.963898917
210
  },
211
  "acl":{
212
+ "p":0.8239130435,
213
+ "r":0.832967033,
214
+ "f":0.8284153005
215
  },
216
  "obj":{
217
+ "p":0.9507692308,
218
+ "r":0.9335347432,
219
+ "f":0.9420731707
220
  },
221
+ "nummod":{
222
+ "p":0.9934640523,
223
+ "r":0.899408284,
224
+ "f":0.9440993789
225
  },
226
  "advmod":{
227
+ "p":0.6771653543,
228
+ "r":0.6142857143,
229
+ "f":0.6441947566
230
  },
231
  "amod":{
232
+ "p":0.8709677419,
233
+ "r":0.7297297297,
234
+ "f":0.7941176471
235
  },
236
  "cop":{
237
+ "p":0.9464285714,
238
+ "r":0.9244186047,
239
+ "f":0.9352941176
240
  },
241
  "ccomp":{
242
+ "p":0.8571428571,
243
  "r":0.8181818182,
244
+ "f":0.8372093023
245
  },
246
  "det":{
247
+ "p":1.0,
248
+ "r":0.9811320755,
249
+ "f":0.9904761905
 
 
 
 
 
250
  },
251
  "dep":{
252
+ "p":0.0714285714,
253
+ "r":0.1428571429,
254
+ "f":0.0952380952
255
+ },
256
+ "csubj":{
257
+ "p":0.6363636364,
258
+ "r":0.5833333333,
259
+ "f":0.6086956522
260
  }
261
  },
262
+ "tag_acc":0.9715755942,
263
+ "lemma_acc":0.9659109444,
264
+ "ents_p":0.6767371601,
265
+ "ents_r":0.5635220126,
266
+ "ents_f":0.6149622512,
267
  "ents_per_type":{
268
  "DATE":{
269
+ "p":0.9285714286,
270
+ "r":0.9541284404,
271
+ "f":0.9411764706
272
  },
273
+ "ORG":{
274
+ "p":0.5046728972,
275
+ "r":0.3941605839,
276
+ "f":0.4426229508
277
  },
278
  "GPE":{
279
+ "p":0.6296296296,
280
  "r":0.5425531915,
281
+ "f":0.5828571429
282
  },
283
  "PRODUCT":{
284
+ "p":0.4285714286,
285
+ "r":0.2142857143,
286
+ "f":0.2857142857
287
  },
288
  "TIME":{
289
+ "p":0.5714285714,
290
  "r":1.0,
291
+ "f":0.7272727273
292
  },
293
  "QUANTITY":{
294
+ "p":0.7972972973,
295
+ "r":0.8939393939,
296
+ "f":0.8428571429
297
+ },
298
+ "PERSON":{
299
+ "p":0.6346153846,
300
+ "r":0.4748201439,
301
+ "f":0.5432098765
302
  },
303
  "NORP":{
304
+ "p":0.6666666667,
305
  "r":0.5625,
306
+ "f":0.6101694915
 
 
 
 
 
307
  },
308
  "TITLE_AFFIX":{
309
+ "p":0.7142857143,
310
+ "r":0.5,
311
+ "f":0.5882352941
312
  },
313
+ "ORDINAL":{
314
+ "p":0.5909090909,
315
+ "r":0.5909090909,
316
+ "f":0.5909090909
317
  },
318
  "WORK_OF_ART":{
319
+ "p":0.7692307692,
320
+ "r":0.5882352941,
321
+ "f":0.6666666667
322
+ },
323
+ "EVENT":{
324
+ "p":0.6,
325
+ "r":0.4615384615,
326
+ "f":0.5217391304
327
  },
328
  "PERCENT":{
329
  "p":1.0,
330
+ "r":0.2857142857,
331
+ "f":0.4444444444
332
  },
333
+ "CARDINAL":{
334
+ "p":0.0,
335
+ "r":0.0,
336
+ "f":0.0
337
  },
338
  "LOC":{
339
+ "p":0.7,
340
  "r":0.7,
341
+ "f":0.7
342
  },
343
  "FAC":{
344
+ "p":0.44,
345
+ "r":0.2972972973,
346
+ "f":0.3548387097
347
  },
348
  "MOVEMENT":{
349
  "p":0.0,
 
351
  "f":0.0
352
  },
353
  "LAW":{
354
+ "p":0.0,
355
+ "r":0.0,
356
+ "f":0.0
357
  },
358
  "MONEY":{
359
  "p":1.0,
 
364
  "p":1.0,
365
  "r":1.0,
366
  "f":1.0
 
 
 
 
 
367
  }
368
+ },
369
+ "speed":4844.6752174676
370
  },
371
  "sources":[
372
  {
373
+ "name":"UD Japanese GSD v2.8",
374
  "url":"https://github.com/UniversalDependencies/UD_Japanese-GSD",
375
  "license":"CC BY-SA 4.0",
376
  "author":"Omura, Mai; Miyao, Yusuke; Kanayama, Hiroshi; Matsuda, Hiroshi; Wakasa, Aya; Yamashita, Kayo; Asahara, Masayuki; Tanaka, Takaaki; Murawaki, Yugo; Matsumoto, Yuji; Mori, Shinsuke; Uematsu, Sumire; McDonald, Ryan; Nivre, Joakim; Zeman, Daniel"
377
  },
378
  {
379
+ "name":"UD Japanese GSD v2.8 NER",
380
  "url":"https://github.com/megagonlabs/UD_Japanese-GSD",
381
  "license":"CC BY-SA 4.0",
382
  "author":"Megagon Labs Tokyo"
morphologizer/cfg ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "extend":true,
3
+ "labels_morph":{
4
+ "POS=NOUN":"",
5
+ "POS=ADP":"",
6
+ "POS=VERB":"",
7
+ "POS=SCONJ":"",
8
+ "POS=AUX":"",
9
+ "POS=PUNCT":"",
10
+ "POS=PART":"",
11
+ "POS=DET":"",
12
+ "POS=NUM":"",
13
+ "POS=ADV":"",
14
+ "POS=PRON":"",
15
+ "POS=ADJ":"",
16
+ "POS=PROPN":"",
17
+ "POS=CCONJ":"",
18
+ "POS=SYM":"",
19
+ "POS=NOUN|Polarity=Neg":"Polarity=Neg",
20
+ "POS=AUX|Polarity=Neg":"Polarity=Neg",
21
+ "POS=INTJ":"",
22
+ "POS=SCONJ|Polarity=Neg":"Polarity=Neg"
23
+ },
24
+ "labels_pos":{
25
+ "POS=NOUN":92,
26
+ "POS=ADP":85,
27
+ "POS=VERB":100,
28
+ "POS=SCONJ":98,
29
+ "POS=AUX":87,
30
+ "POS=PUNCT":97,
31
+ "POS=PART":94,
32
+ "POS=DET":90,
33
+ "POS=NUM":93,
34
+ "POS=ADV":86,
35
+ "POS=PRON":95,
36
+ "POS=ADJ":84,
37
+ "POS=PROPN":96,
38
+ "POS=CCONJ":89,
39
+ "POS=SYM":99,
40
+ "POS=NOUN|Polarity=Neg":92,
41
+ "POS=AUX|Polarity=Neg":87,
42
+ "POS=INTJ":91,
43
+ "POS=SCONJ|Polarity=Neg":98
44
+ },
45
+ "overwrite":true
46
+ }
morphologizer/model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4342197134b78e4331bceae7c61472c399e737fbd929670c5e28d29c4c9850f
3
+ size 7749
ner/model CHANGED
Binary files a/ner/model and b/ner/model differ
 
ner/moves CHANGED
@@ -1 +1 @@
1
- ��moves��{"0":{},"1":{"DATE":4112,"ORG":3465,"PERSON":2992,"QUANTITY":2502,"GPE":1927,"PRODUCT":1317,"FAC":1230,"ORDINAL":1095,"WORK_OF_ART":1022,"EVENT":865,"NORP":732,"LOC":557,"MONEY":400,"TITLE_AFFIX":343,"TIME":294,"PERCENT":272,"MOVEMENT":148,"LAW":94,"LANGUAGE":78,"CARDINAL":27,"PET_NAME":19,"PHONE":4},"2":{"DATE":4112,"ORG":3465,"PERSON":2992,"QUANTITY":2502,"GPE":1927,"PRODUCT":1317,"FAC":1230,"ORDINAL":1095,"WORK_OF_ART":1022,"EVENT":865,"NORP":732,"LOC":557,"MONEY":400,"TITLE_AFFIX":343,"TIME":294,"PERCENT":272,"MOVEMENT":148,"LAW":94,"LANGUAGE":78,"CARDINAL":27,"PET_NAME":19,"PHONE":4},"3":{"DATE":4112,"ORG":3465,"PERSON":2992,"QUANTITY":2502,"GPE":1927,"PRODUCT":1317,"FAC":1230,"ORDINAL":1095,"WORK_OF_ART":1022,"EVENT":865,"NORP":732,"LOC":557,"MONEY":400,"TITLE_AFFIX":343,"TIME":294,"PERCENT":272,"MOVEMENT":148,"LAW":94,"LANGUAGE":78,"CARDINAL":27,"PET_NAME":19,"PHONE":4},"4":{"DATE":4112,"ORG":3465,"PERSON":2992,"QUANTITY":2502,"GPE":1927,"PRODUCT":1317,"FAC":1230,"ORDINAL":1095,"WORK_OF_ART":1022,"EVENT":865,"NORP":732,"LOC":557,"MONEY":400,"TITLE_AFFIX":343,"TIME":294,"PERCENT":272,"MOVEMENT":148,"LAW":94,"LANGUAGE":78,"CARDINAL":27,"PET_NAME":19,"PHONE":4,"":1},"5":{"":1}}�cfg��neg_key�
 
1
+ ��moves��{"0":{},"1":{"DATE":4200,"ORG":3487,"PERSON":3042,"QUANTITY":2519,"GPE":1953,"PRODUCT":1328,"FAC":1243,"ORDINAL":1114,"WORK_OF_ART":1053,"EVENT":869,"NORP":734,"LOC":563,"MONEY":400,"TITLE_AFFIX":344,"TIME":300,"PERCENT":274,"MOVEMENT":148,"LAW":94,"LANGUAGE":82,"CARDINAL":27,"PET_NAME":20,"PHONE":4},"2":{"DATE":4200,"ORG":3487,"PERSON":3042,"QUANTITY":2519,"GPE":1953,"PRODUCT":1328,"FAC":1243,"ORDINAL":1114,"WORK_OF_ART":1053,"EVENT":869,"NORP":734,"LOC":563,"MONEY":400,"TITLE_AFFIX":344,"TIME":300,"PERCENT":274,"MOVEMENT":148,"LAW":94,"LANGUAGE":82,"CARDINAL":27,"PET_NAME":20,"PHONE":4},"3":{"DATE":4200,"ORG":3487,"PERSON":3042,"QUANTITY":2519,"GPE":1953,"PRODUCT":1328,"FAC":1243,"ORDINAL":1114,"WORK_OF_ART":1053,"EVENT":869,"NORP":734,"LOC":563,"MONEY":400,"TITLE_AFFIX":344,"TIME":300,"PERCENT":274,"MOVEMENT":148,"LAW":94,"LANGUAGE":82,"CARDINAL":27,"PET_NAME":20,"PHONE":4},"4":{"DATE":4200,"ORG":3487,"PERSON":3042,"QUANTITY":2519,"GPE":1953,"PRODUCT":1328,"FAC":1243,"ORDINAL":1114,"WORK_OF_ART":1053,"EVENT":869,"NORP":734,"LOC":563,"MONEY":400,"TITLE_AFFIX":344,"TIME":300,"PERCENT":274,"MOVEMENT":148,"LAW":94,"LANGUAGE":82,"CARDINAL":27,"PET_NAME":20,"PHONE":4,"":1},"5":{"":1}}�cfg��neg_key�
parser/model CHANGED
Binary files a/parser/model and b/parser/model differ
 
parser/moves CHANGED
@@ -1 +1 @@
1
- ��moves�q{"0":{"":75008},"1":{"":80671},"2":{"compound":20642,"obl":11201,"nmod":11139,"nsubj":6348,"acl":6215,"advcl":6023,"obj":4334,"nummod":3800,"advmod":1393,"punct":1249,"det":813,"cc":695,"amod":366,"ccomp":327,"dislocated":266,"csubj":159,"dep":0},"3":{"case":35563,"aux":18454,"punct":14888,"mark":6577,"fixed":2698,"cop":2198,"compound":248,"dep":0},"4":{"ROOT":6787}}�cfg��neg_key�
 
1
+ ��moves�~{"0":{"":75051},"1":{"":81581},"2":{"compound":22178,"nmod":11296,"obl":10522,"nsubj":6649,"acl":6185,"advcl":5956,"obj":4364,"nummod":2247,"advmod":1841,"punct":1169,"det":822,"cc":699,"amod":357,"ccomp":335,"dislocated":233,"csubj":139,"dep":0},"3":{"case":35390,"punct":15051,"aux":14506,"fixed":7377,"mark":6390,"cop":2079,"compound":542,"advcl":148,"dep":56},"4":{"ROOT":6810}}�cfg��neg_key�
senter/cfg CHANGED
@@ -1,3 +1,3 @@
1
  {
2
-
3
  }
 
1
  {
2
+ "overwrite":false
3
  }
senter/model CHANGED
Binary files a/senter/model and b/senter/model differ
 
tok2vec/model CHANGED
Binary files a/tok2vec/model and b/tok2vec/model differ
 
vocab/strings.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cf36b06704e078e9c477ce1b341ece677cf7cf1a1c059de6d37cc9e50bb987c5
3
- size 610901
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ddb6fa8049cb20badee921092f615ca507a8b27a731a85d02502ebd8a765608
3
+ size 1559195
vocab/vectors.cfg ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "mode":"default"
3
+ }