Token Classification
Transformers
Safetensors
French
deberta-v2
Inference Endpoints
bourdoiscatie commited on
Commit
410dc65
·
verified ·
1 Parent(s): 36a7175

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1305 -63
README.md CHANGED
@@ -1,92 +1,1334 @@
1
  ---
2
- library_name: transformers
3
  license: mit
4
  base_model: almanach/camembertav2-base
5
- tags:
6
- - generated_from_trainer
7
  metrics:
8
  - precision
9
  - recall
10
  - f1
11
  - accuracy
12
  model-index:
13
- - name: camembertav2-base-frenchNER_3entities
14
  results: []
 
 
 
 
 
 
 
 
 
15
  ---
16
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
  ```
18
- {'LOC': {'precision': 0.9341895320551385,
19
- 'recall': 0.9516260108445131,
20
- 'f1': 0.9428271615530316,
21
- 'number': 75061},
22
- 'O': {'precision': 0.9953844747581743,
23
- 'recall': 0.9930766705362066,
24
- 'f1': 0.9942292334305959,
25
- 'number': 932066},
26
- 'ORG': {'precision': 0.8804077936494026,
27
- 'recall': 0.8825734282116606,
28
- 'f1': 0.8814892808048901,
29
- 'number': 34149},
30
- 'PER': {'precision': 0.9657491578607356,
31
- 'recall': 0.973339689331225,
32
- 'f1': 0.9695295670905427,
33
- 'number': 86008},
34
- 'overall_precision': 0.985463290528385,
35
- 'overall_recall': 0.985463290528385,
36
- 'overall_f1': 0.985463290528385,
37
- 'overall_accuracy': 0.985463290528385}
38
- ```
39
-
40
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
41
- should probably proofread and complete it, then remove this comment. -->
42
 
43
- # camembertav2-base-frenchNER_3entities
44
 
45
- This model is a fine-tuned version of [almanach/camembertav2-base](https://huggingface.co/almanach/camembertav2-base) on an unknown dataset.
46
- It achieves the following results on the evaluation set:
47
- - Loss: 0.0880
48
- - Precision: 0.9859
49
- - Recall: 0.9859
50
- - F1: 0.9859
51
- - Accuracy: 0.9859
52
 
53
- ## Model description
54
 
55
- More information needed
56
 
57
- ## Intended uses & limitations
58
 
59
- More information needed
 
 
 
 
60
 
61
- ## Training and evaluation data
62
 
63
- More information needed
64
 
65
- ## Training procedure
66
 
67
- ### Training hyperparameters
 
 
 
 
 
 
 
 
 
 
 
68
 
69
- The following hyperparameters were used during training:
70
- - learning_rate: 2e-05
71
- - train_batch_size: 8
72
- - eval_batch_size: 8
73
- - seed: 42
74
- - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
75
- - lr_scheduler_type: linear
76
- - num_epochs: 3
 
 
 
 
77
 
78
- ### Training results
 
 
 
 
 
 
 
79
 
80
- | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
81
- |:-------------:|:-----:|:------:|:---------------:|:---------:|:------:|:------:|:--------:|
82
- | 0.0335 | 1.0 | 43650 | 0.0854 | 0.9833 | 0.9833 | 0.9833 | 0.9833 |
83
- | 0.0169 | 2.0 | 87300 | 0.0821 | 0.9854 | 0.9854 | 0.9854 | 0.9854 |
84
- | 0.0103 | 3.0 | 130950 | 0.0880 | 0.9859 | 0.9859 | 0.9859 | 0.9859 |
 
 
 
 
 
 
 
85
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
86
 
87
- ### Framework versions
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
88
 
89
- - Transformers 4.46.1
90
- - Pytorch 2.4.0+cu121
91
- - Datasets 2.21.0
92
- - Tokenizers 0.20.1
 
1
  ---
 
2
  license: mit
3
  base_model: almanach/camembertav2-base
 
 
4
  metrics:
5
  - precision
6
  - recall
7
  - f1
8
  - accuracy
9
  model-index:
10
+ - name: NERmemberta-3entities
11
  results: []
12
+ datasets:
13
+ - CATIE-AQ/frenchNER_3entities
14
+ language:
15
+ - fr
16
+ widget:
17
+ - text: "Le dévoilement du logo officiel des JO s'est déroulé le 21 octobre 2019 au Grand Rex. Ce nouvel emblème et cette nouvelle typographie ont été conçus par le designer Sylvain Boyer avec les agences Royalties & Ecobranding. Rond, il rassemble trois symboles : une médaille d'or, la flamme olympique et Marianne, symbolisée par un visage de femme mais privée de son bonnet phrygien caractéristique. La typographie dessinée fait référence à l'Art déco, mouvement artistique des années 1920, décennie pendant laquelle ont eu lieu pour la dernière fois les Jeux olympiques à Paris en 1924. Pour la première fois, ce logo sera unique pour les Jeux olympiques et les Jeux paralympiques."
18
+ library_name: transformers
19
+ pipeline_tag: token-classification
20
+ co2_eq_emissions: 14
21
  ---
22
 
23
+
24
+ # NERmemBERT2-3entities
25
+
26
+ ## Model Description
27
+
28
+ We present **NERmemBERTa-3entities**, which is a [CamemBERTa v2 base](https://huggingface.co/almanach/camembertav2-base) fine-tuned for the Name Entity Recognition task for the French language on five French NER datasets for 3 entities (LOC, PER, ORG).
29
+ All these datasets were concatenated and cleaned into a single dataset that we called [frenchNER_3entities](https://huggingface.co/datasets/CATIE-AQ/frenchNER_3entities).
30
+ This represents a total of over **420,264 rows, of which 346,071 are for training, 32,951 for validation and 41,242 for testing.**
31
+ Our methodology is described in a blog post available in [English](https://blog.vaniila.ai/en/NER_en/) or [French](https://blog.vaniila.ai/NER/).
32
+
33
+
34
+ ## Evaluation results
35
+
36
+ ### frenchNER_3entities
37
+
38
+ For space reasons, we show only the F1 of the different models. You can see the full results below the table.
39
+
40
+ <table>
41
+ <thead>
42
+ <tr>
43
+ <th><br>Model</th>
44
+ <th><br>PER</th>
45
+ <th><br>LOC</th>
46
+ <th><br>ORG</th>
47
+ </tr>
48
+ </thead>
49
+ <tbody>
50
+ <tr>
51
+ <td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td>
52
+ <td><br>0.941</td>
53
+ <td><br>0.883</td>
54
+ <td><br>0.658</td>
55
+ </tr>
56
+ <tr>
57
+ <td rowspan="1"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td>
58
+ <td><br>0.942</td>
59
+ <td><br>0.882</td>
60
+ <td><br>0.647</td>
61
+ </tr>
62
+ <tr>
63
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td>
64
+ <td><br>0.966</td>
65
+ <td><br>0.940</td>
66
+ <td><br>0.876</td>
67
+ </tr>
68
+ <tr>
69
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td>
70
+ <td><br>0.967</td>
71
+ <td><br>0.942</td>
72
+ <td><br>0.875</td>
73
+ </tr>
74
+ <tr>
75
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td>
76
+ <td><br><b>0.970</b></td>
77
+ <td><br>0.943</td>
78
+ <td><br>0.881</td>
79
+ </tr>
80
+ <tr>
81
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td>
82
+ <td><br>0.969</td>
83
+ <td><br><b>0.947</b></td>
84
+ <td><br><b>0.890</b></td>
85
+ </tr>
86
+ </tr>
87
+ </tbody>
88
+ </table>
89
+
90
+
91
+ <details>
92
+ <summary>Full results</summary>
93
+ <table>
94
+ <thead>
95
+ <tr>
96
+ <th><br>Model</th>
97
+ <th><br>Metrics</th>
98
+ <th><br>PER</th>
99
+ <th><br>LOC</th>
100
+ <th><br>ORG</th>
101
+ <th><br>O</th>
102
+ <th><br>Overall</th>
103
+ </tr>
104
+ </thead>
105
+ <tbody>
106
+ <tr>
107
+ <td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td>
108
+ <td><br>Precision</td>
109
+ <td><br>0.918</td>
110
+ <td><br>0.860</td>
111
+ <td><br>0.831</td>
112
+ <td><br>0.992</td>
113
+ <td><br>0.974</td>
114
+ </tr>
115
+ <tr>
116
+ <td><br>Recall</td>
117
+ <td><br>0.964</td>
118
+ <td><br>0.908</td>
119
+ <td><br>0.544</td>
120
+ <td><br>0.964</td>
121
+ <td><br>0.948</td>
122
+ </tr>
123
+ <tr>
124
+ <td>F1</td>
125
+ <td><br>0.941</td>
126
+ <td><br>0.883</td>
127
+ <td><br>0.658</td>
128
+ <td><br>0.978</td>
129
+ <td><br>0.961</td>
130
+ </tr>
131
+ <tr>
132
+ <td rowspan="3"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td>
133
+ <td><br>Precision</td>
134
+ <td><br>0.929</td>
135
+ <td><br>0.861</td>
136
+ <td><br>0.813</td>
137
+ <td><br>0.991</td>
138
+ <td><br>0.974</td>
139
+ </tr>
140
+ <tr>
141
+ <td><br>Recall</td>
142
+ <td><br>0.956</td>
143
+ <td><br>0.905</td>
144
+ <td><br>0.956</td>
145
+ <td><br>0.965</td>
146
+ <td><br>0.948</td>
147
+ </tr>
148
+ <tr>
149
+ <td>F1</td>
150
+ <td><br>0.942</td>
151
+ <td><br>0.882</td>
152
+ <td><br>0.647</td>
153
+ <td><br>0.978</td>
154
+ <td><br>0.961</td>
155
+ </tr>
156
+ <tr>
157
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td>
158
+ <td><br>Precision</td>
159
+ <td><br>0.961</td>
160
+ <td><br>0.935</td>
161
+ <td><br>0.877</td>
162
+ <td><br>0.995</td>
163
+ <td><br>0.986</td>
164
+ </tr>
165
+ <tr>
166
+ <td><br>Recall</td>
167
+ <td><br>0.972</td>
168
+ <td><br>0.946</td>
169
+ <td><br>0.876</td>
170
+ <td><br>0.994</td>
171
+ <td><br>0.986</td>
172
+ </tr>
173
+ <tr>
174
+ <td>F1</td>
175
+ <td><br>0.966</td>
176
+ <td><br>0.940</td>
177
+ <td><br>0.876</td>
178
+ <td><br>0.994</td>
179
+ <td><br>0.986</td>
180
+ </tr>
181
+ <tr>
182
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td>
183
+ <td><br>Precision</td>
184
+ <td><br>0.964</td>
185
+ <td><br>0.935</td>
186
+ <td><br>0.872</td>
187
+ <td><br>0.995</td>
188
+ <td><br>0.985</td>
189
+ </tr>
190
+ <tr>
191
+ <td><br>Recall</td>
192
+ <td><br>0.967</td>
193
+ <td><br>0.949</td>
194
+ <td><br>0.878</td>
195
+ <td><br>0.993</td>
196
+ <td><br>0.984</td>
197
+ </tr>
198
+ <tr>
199
+ <td>F1</td>
200
+ <td><br>0.967</td>
201
+ <td><br>0.942</td>
202
+ <td><br>0.875</td>
203
+ <td><br>0.994</td>
204
+ <td><br>0.985</td>
205
+ </tr>
206
+ <tr>
207
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M)</a></td>
208
+ <td><br>Precision</td>
209
+ <td><br>0.966</td>
210
+ <td><br>0.934</td>
211
+ <td><br>0.880</td>
212
+ <td><br>0.995</td>
213
+ <td><br>0.985</td>
214
+ </tr>
215
+ <tr>
216
+ <td><br>Recall</td>
217
+ <td><br>0.973</td>
218
+ <td><br>0.952</td>
219
+ <td><br>0.883</td>
220
+ <td><br>0.993</td>
221
+ <td><br>0.985</td>
222
+ </tr>
223
+ <tr>
224
+ <td>F1</td>
225
+ <td><br>0.970</td>
226
+ <td><br>0.943</td>
227
+ <td><br>0.881</td>
228
+ <td><br>0.994</td>
229
+ <td><br>0.985</td>
230
+ </tr>
231
+ <tr>
232
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td>
233
+ <td><br>Precision</td>
234
+ <td><br>0.946</td>
235
+ <td><br>0.884</td>
236
+ <td><br>0.859</td>
237
+ <td><br>0.993</td>
238
+ <td><br>0.971</td>
239
+ </tr>
240
+ <tr>
241
+ <td><br>Recall</td>
242
+ <td><br>0.955</td>
243
+ <td><br>0.904</td>
244
+ <td><br>0.550</td>
245
+ <td><br>0.993</td>
246
+ <td><br>0.971</td>
247
+ </tr>
248
+ <tr>
249
+ <td>F1</td>
250
+ <td><br>0.951</td>
251
+ <td><br>0.894</td>
252
+ <td><br>0.671</td>
253
+ <td><br>0.988</td>
254
+ <td><br>0.971</td>
255
+ </tr>
256
+ </tbody>
257
+ </table>
258
+ </details>
259
+
260
+
261
+
262
+
263
+ In detail:
264
+
265
+ ### multiconer
266
+
267
+ For space reasons, we show only the F1 of the different models. You can see the full results below the table.
268
+
269
+ <table>
270
+ <thead>
271
+ <tr>
272
+ <th><br>Model</th>
273
+ <th><br>PER</th>
274
+ <th><br>LOC</th>
275
+ <th><br>ORG</th>
276
+ </tr>
277
+ </thead>
278
+ <tbody>
279
+ <tr>
280
+ <td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td>
281
+ <td><br>0.940</td>
282
+ <td><br>0.761</td>
283
+ <td><br>0.723</td>
284
+ </tr>
285
+ <tr>
286
+ <td rowspan="1"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td>
287
+ <td><br>0.921</td>
288
+ <td><br>0.748</td>
289
+ <td><br>0.694</td>
290
+ </tr>
291
+ <tr>
292
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td>
293
+ <td><br>0.960</td>
294
+ <td><br>0.887</td>
295
+ <td><br>0.876</td>
296
+ </tr>
297
+ <tr>
298
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmembert2-3entities (111M)</a></td>
299
+ <td><br>0.958</td>
300
+ <td><br>0.876</td>
301
+ <td><br>0.863</td>
302
+ </tr>
303
+ <tr>
304
+ <td rowspan="1"><br>NERmemberta-3entities (111M) (this model)</td>
305
+ <td><br>0.964</td>
306
+ <td><br>0.865</td>
307
+ <td><br>0.859</td>
308
+ </tr>
309
+ <tr>
310
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td>
311
+ <td><br><b>0.965</b></td>
312
+ <td><br><b>0.902</b></td>
313
+ <td><br><b>0.896</b></td>
314
+ </tr>
315
+ </tr>
316
+ </tbody>
317
+ </table>
318
+
319
+
320
+ <details>
321
+ <summary>Full results</summary>
322
+
323
+ <table>
324
+ <thead>
325
+ <tr>
326
+ <th><br>Model</th>
327
+ <th><br>Metrics</th>
328
+ <th><br>PER</th>
329
+ <th><br>LOC</th>
330
+ <th><br>ORG</th>
331
+ <th><br>O</th>
332
+ <th><br>Overall</th>
333
+ </tr>
334
+ </thead>
335
+ <tbody>
336
+ <tr>
337
+ <td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td>
338
+ <td><br>Precision</td>
339
+ <td><br>0.908</td>
340
+ <td><br>0.717</td>
341
+ <td><br>0.753</td>
342
+ <td><br>0.987</td>
343
+ <td><br>0.947</td>
344
+ </tr>
345
+ <tr>
346
+ <td><br>Recall</td>
347
+ <td><br>0.975</td>
348
+ <td><br>0.811</td>
349
+ <td><br>0.696</td>
350
+ <td><br>0.878</td>
351
+ <td><br>0.880</td>
352
+ </tr>
353
+ <tr>
354
+ <td>F1</td>
355
+ <td><br>0.940</td>
356
+ <td><br>0.761</td>
357
+ <td><br>0.723</td>
358
+ <td><br>0.929</td>
359
+ <td><br>0.912</td>
360
+ </tr>
361
+ <tr>
362
+ <td rowspan="3"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td>
363
+ <td><br>Precision</td>
364
+ <td><br>0.885</td>
365
+ <td><br>0.738</td>
366
+ <td><br>0.737</td>
367
+ <td><br>0.983</td>
368
+ <td><br>0.943</td>
369
+ </tr>
370
+ <tr>
371
+ <td><br>Recall</td>
372
+ <td><br>0.960</td>
373
+ <td><br>0.759</td>
374
+ <td><br>0.655</td>
375
+ <td><br>0.882</td>
376
+ <td><br>0.877</td>
377
+ </tr>
378
+ <tr>
379
+ <td>F1</td>
380
+ <td><br>0.921</td>
381
+ <td><br>0.748</td>
382
+ <td><br>0.694</td>
383
+ <td><br>0.930</td>
384
+ <td><br>0.909</td>
385
+ </tr>
386
+ <tr>
387
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td>
388
+ <td><br>Precision</td>
389
+ <td><br>0.957</td>
390
+ <td><br>0.894</td>
391
+ <td><br>0.876</td>
392
+ <td><br>0.986</td>
393
+ <td><br>0.972</td>
394
+ </tr>
395
+ <tr>
396
+ <td><br>Recall</td>
397
+ <td><br>0.962</td>
398
+ <td><br>0.880</td>
399
+ <td><br>0.878</td>
400
+ <td><br>0.985</td>
401
+ <td><br>0.972</td>
402
+ </tr>
403
+ <tr>
404
+ <td>F1</td>
405
+ <td><br>0.960</td>
406
+ <td><br>0.887</td>
407
+ <td><br>0.876</td>
408
+ <td><br>0.985</td>
409
+ <td><br>0.972</td>
410
+ </tr>
411
+ <tr>
412
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td>
413
+ <td><br>Precision</td>
414
+ <td><br>0.951</td>
415
+ <td><br>0.906</td>
416
+ <td><br>0.853</td>
417
+ <td><br>0.984</td>
418
+ <td><br>0.967</td>
419
+ </tr>
420
+ <tr>
421
+ <td><br>Recall</td>
422
+ <td><br>0.966</td>
423
+ <td><br>0.848</td>
424
+ <td><br>0.874</td>
425
+ <td><br>0.984</td>
426
+ <td><br>0.967</td>
427
+ </tr>
428
+ <tr>
429
+ <td>F1</td>
430
+ <td><br>0.958</td>
431
+ <td><br>0.876</td>
432
+ <td><br>0.863</td>
433
+ <td><br>0.984</td>
434
+ <td><br>0.967</td>
435
+ </tr>
436
+ <tr>
437
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td>
438
+ <td><br>Precision</td>
439
+ <td><br>0.962</td>
440
+ <td><br>0.859</td>
441
+ <td><br>0.862</td>
442
+ <td><br>0.985</td>
443
+ <td><br>0.970</td>
444
+ </tr>
445
+ <tr>
446
+ <td><br>Recall</td>
447
+ <td><br>0.967</td>
448
+ <td><br>0.871</td>
449
+ <td><br>0.857</td>
450
+ <td><br>0.984</td>
451
+ <td><br>0.970</td>
452
+ </tr>
453
+ <tr>
454
+ <td>F1</td>
455
+ <td><br>0.964</td>
456
+ <td><br>0.865</td>
457
+ <td><br>0.859</td>
458
+ <td><br>0.985</td>
459
+ <td><br>0.970</td>
460
+ </tr>
461
+ <tr>
462
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td>
463
+ <td><br>Precision</td>
464
+ <td><br>0.960</td>
465
+ <td><br>0.903</td>
466
+ <td><br>0.916</td>
467
+ <td><br>0.987</td>
468
+ <td><br>0.976</td>
469
+ </tr>
470
+ <tr>
471
+ <td><br>Recall</td>
472
+ <td><br>0.969</td>
473
+ <td><br>0.900</td>
474
+ <td><br>0.877</td>
475
+ <td><br>0.987</td>
476
+ <td><br>0.976</td>
477
+ </tr>
478
+ <tr>
479
+ <td>F1</td>
480
+ <td><br>0.965</td>
481
+ <td><br>0.902</td>
482
+ <td><br>0.896</td>
483
+ <td><br>0.987</td>
484
+ <td><br>0.976</td>
485
+ </tr>
486
+ </tbody>
487
+ </table>
488
+ </details>
489
+
490
+
491
+ ### multinerd
492
+
493
+ For space reasons, we show only the F1 of the different models. You can see the full results below the table.
494
+
495
+ <table>
496
+ <thead>
497
+ <tr>
498
+ <th><br>Model</th>
499
+ <th><br>PER</th>
500
+ <th><br>LOC</th>
501
+ <th><br>ORG</th>
502
+ </tr>
503
+ </thead>
504
+ <tbody>
505
+ <tr>
506
+ <td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td>
507
+ <td><br>0.962</td>
508
+ <td><br>0.934</td>
509
+ <td><br>0.888</td>
510
+ </tr>
511
+ <tr>
512
+ <td rowspan="1"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td>
513
+ <td><br>0.972</td>
514
+ <td><br>0.938</td>
515
+ <td><br>0.884</td>
516
+ </tr>
517
+ <tr>
518
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td>
519
+ <td><br>0.985</td>
520
+ <td><br>0.973</td>
521
+ <td><br>0.938</td>
522
+ </tr>
523
+ <tr>
524
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmembert2-3entities (111M)</a></td>
525
+ <td><br>0.985</td>
526
+ <td><br>0.972</td>
527
+ <td><br>0.933</td>
528
+ </tr>
529
+ <tr>
530
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td>
531
+ <td><br>0.986</td>
532
+ <td><br>0.974</td>
533
+ <td><br>0.945</td>
534
+ </tr>
535
+ <tr>
536
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td>
537
+ <td><br><b>0.987</b></td>
538
+ <td><br><b>0.979</b></td>
539
+ <td><br><b>0.953</b></td>
540
+ </tr>
541
+ </tr>
542
+ </tbody>
543
+ </table>
544
+
545
+
546
+ <details>
547
+ <summary>Full results</summary>
548
+ <table>
549
+ <thead>
550
+ <tr>
551
+ <th><br>Model</th>
552
+ <th><br>Metrics</th>
553
+ <th><br>PER</th>
554
+ <th><br>LOC</th>
555
+ <th><br>ORG</th>
556
+ <th><br>O</th>
557
+ <th><br>Overall</th>
558
+ </tr>
559
+ </thead>
560
+ <tbody>
561
+ <tr>
562
+ <td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td>
563
+ <td><br>Precision</td>
564
+ <td><br>0.931</td>
565
+ <td><br>0.893</td>
566
+ <td><br>0.827</td>
567
+ <td><br>0.999</td>
568
+ <td><br>0.988</td>
569
+ </tr>
570
+ <tr>
571
+ <td><br>Recall</td>
572
+ <td><br>0.994</td>
573
+ <td><br>0.980</td>
574
+ <td><br>0.959</td>
575
+ <td><br>0.973</td>
576
+ <td><br>0.974</td>
577
+ </tr>
578
+ <tr>
579
+ <td>F1</td>
580
+ <td><br>0.962</td>
581
+ <td><br>0.934</td>
582
+ <td><br>0.888</td>
583
+ <td><br>0.986</td>
584
+ <td><br>0.981</td>
585
+ </tr>
586
+ <tr>
587
+ <td rowspan="3"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td>
588
+ <td><br>Precision</td>
589
+ <td><br>0.954</td>
590
+ <td><br>0.908</td>
591
+ <td><br>0.817</td>
592
+ <td><br>0.999</td>
593
+ <td><br>0.990</td>
594
+ </tr>
595
+ <tr>
596
+ <td><br>Recall</td>
597
+ <td><br>0.991</td>
598
+ <td><br>0.969</td>
599
+ <td><br>0.963</td>
600
+ <td><br>0.975</td>
601
+ <td><br>0.975</td>
602
+ </tr>
603
+ <tr>
604
+ <td>F1</td>
605
+ <td><br>0.972</td>
606
+ <td><br>0.938</td>
607
+ <td><br>0.884</td>
608
+ <td><br>0.987</td>
609
+ <td><br>0.983</td>
610
+ </tr>
611
+ <tr>
612
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td>
613
+ <td><br>Precision</td>
614
+ <td><br>0.974</td>
615
+ <td><br>0.965</td>
616
+ <td><br>0.910</td>
617
+ <td><br>0.999</td>
618
+ <td><br>0.995</td>
619
+ </tr>
620
+ <tr>
621
+ <td><br>Recall</td>
622
+ <td><br>0.995</td>
623
+ <td><br>0.981</td>
624
+ <td><br>0.968</td>
625
+ <td><br>0.996</td>
626
+ <td><br>0.995</td>
627
+ </tr>
628
+ <tr>
629
+ <td>F1</td>
630
+ <td><br>0.985</td>
631
+ <td><br>0.973</td>
632
+ <td><br>0.938</td>
633
+ <td><br>0.998</td>
634
+ <td><br>0.995</td>
635
+ </tr>
636
+ <tr>
637
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td>
638
+ <td><br>Precision</td>
639
+ <td><br>0.975</td>
640
+ <td><br>0.960</td>
641
+ <td><br>0.902</td>
642
+ <td><br>0.999</td>
643
+ <td><br>0.995</td>
644
+ </tr>
645
+ <tr>
646
+ <td><br>Recall</td>
647
+ <td><br>0.995</td>
648
+ <td><br>0.985</td>
649
+ <td><br>0.967</td>
650
+ <td><br>0.995</td>
651
+ <td><br>0.995</td>
652
+ </tr>
653
+ <tr>
654
+ <td>F1</td>
655
+ <td><br>0.985</td>
656
+ <td><br>0.972</td>
657
+ <td><br>0.933</td>
658
+ <td><br>0.997</td>
659
+ <td><br>0.995</td>
660
+ </tr>
661
+ <tr>
662
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td>
663
+ <td><br>Precision</td>
664
+ <td><br>0.976</td>
665
+ <td><br>0.961</td>
666
+ <td><br>0.915</td>
667
+ <td><br>0.999</td>
668
+ <td><br>0.995</td>
669
+ </tr>
670
+ <tr>
671
+ <td><br>Recall</td>
672
+ <td><br>0.997</td>
673
+ <td><br>0.987</td>
674
+ <td><br>0.976</td>
675
+ <td><br>0.996</td>
676
+ <td><br>0.995</td>
677
+ </tr>
678
+ <tr>
679
+ <td>F1</td>
680
+ <td><br>0.986</td>
681
+ <td><br>0.974</td>
682
+ <td><br>0.945</td>
683
+ <td><br>0.997</td>
684
+ <td><br>0.995</td>
685
+ </tr>
686
+ <tr>
687
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td>
688
+ <td><br>Precision</td>
689
+ <td><br>0.979</td>
690
+ <td><br>0.970</td>
691
+ <td><br>0.927</td>
692
+ <td><br>0.999</td>
693
+ <td><br>0.996</td>
694
+ </tr>
695
+ <tr>
696
+ <td><br>Recall</td>
697
+ <td><br>0.996</td>
698
+ <td><br>0.987</td>
699
+ <td><br>0.980</td>
700
+ <td><br>0.997</td>
701
+ <td><br>0.996</td>
702
+ </tr>
703
+ <tr>
704
+ <td>F1</td>
705
+ <td><br><b>0.987</b></td>
706
+ <td><br><b>0.979</b></td>
707
+ <td><br><b>0.953</b></td>
708
+ <td><br><b>0.998</b></td>
709
+ <td><br><b>0.996</b></td>
710
+ </tr>
711
+ </tr>
712
+ </tbody>
713
+ </table>
714
+ </details>
715
+
716
+
717
+
718
+ ### wikiner
719
+
720
+ For space reasons, we show only the F1 of the different models. You can see the full results below the table.
721
+
722
+ <table>
723
+ <thead>
724
+ <tr>
725
+ <th><br>Model</th>
726
+ <th><br>PER</th>
727
+ <th><br>LOC</th>
728
+ <th><br>ORG</th>
729
+ </tr>
730
+ </thead>
731
+ <tbody>
732
+ <tr>
733
+ <td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td>
734
+ <td><br><b>0.986</b></td>
735
+ <td><br><b>0.966</b></td>
736
+ <td><br><b>0.938</b></td>
737
+ </tr>
738
+ <tr>
739
+ <td rowspan="1"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td>
740
+ <td><br>0.983</td>
741
+ <td><br>0.964</td>
742
+ <td><br>0.925</td>
743
+ </tr>
744
+ <tr>
745
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td>
746
+ <td><br>0.969</td>
747
+ <td><br>0.945</td>
748
+ <td><br>0.878</td>
749
+ </tr>
750
+ <tr>
751
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmembert2-3entities (111M)</a></td>
752
+ <td><br>0.969</td>
753
+ <td><br>0.946</td>
754
+ <td><br>0.866</td>
755
+ </tr>
756
+ <tr>
757
+ <td rowspan="1"><br>NERmemberta-3entities (111M) (this model)</td>
758
+ <td><br>0.971</td>
759
+ <td><br>0.948</td>
760
+ <td><br>0.885</td>
761
+ </tr>
762
+ <tr>
763
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td>
764
+ <td><br>0.972</td>
765
+ <td><br>0.950</td>
766
+ <td><br>0.893</td>
767
+ </tr>
768
+ </tr>
769
+ </tbody>
770
+ </table>
771
+
772
+
773
+ <details>
774
+ <summary>Full results</summary>
775
+ <table>
776
+ <thead>
777
+ <tr>
778
+ <th><br>Model</th>
779
+ <th><br>Metrics</th>
780
+ <th><br>PER</th>
781
+ <th><br>LOC</th>
782
+ <th><br>ORG</th>
783
+ <th><br>O</th>
784
+ <th><br>Overall</th>
785
+ </tr>
786
+ </thead>
787
+ <tbody>
788
+ <tr>
789
+ <td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td>
790
+ <td><br>Precision</td>
791
+ <td><br>0.986</td>
792
+ <td><br>0.962</td>
793
+ <td><br>0.925</td>
794
+ <td><br>0.999</td>
795
+ <td><br>0.994</td>
796
+ </tr>
797
+ <tr>
798
+ <td><br>Recall</td>
799
+ <td><br>0.987</td>
800
+ <td><br>0.969</td>
801
+ <td><br>0.951</td>
802
+ <td><br>0.965</td>
803
+ <td><br>0.967</td>
804
+ </tr>
805
+ <tr>
806
+ <td>F1</td>
807
+ <td><br><b>0.986</b></td>
808
+ <td><br><b>0.966</b></td>
809
+ <td><br><b>0.938</b></td>
810
+ <td><br><b>0.982</b></td>
811
+ <td><br><b>0.980</b></td>
812
+ </tr>
813
+ <tr>
814
+ <td rowspan="3"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td>
815
+ <td><br>Precision</td>
816
+ <td><br>0.982</td>
817
+ <td><br>0.951</td>
818
+ <td><br>0.910</td>
819
+ <td><br>0.998</td>
820
+ <td><br>0.994</td>
821
+ </tr>
822
+ <tr>
823
+ <td><br>Recall</td>
824
+ <td><br>0.985</td>
825
+ <td><br>0.963</td>
826
+ <td><br>0.940</td>
827
+ <td><br>0.966</td>
828
+ <td><br>0.967</td>
829
+ </tr>
830
+ <tr>
831
+ <td>F1</td>
832
+ <td><br>0.983</td>
833
+ <td><br>0.964</td>
834
+ <td><br>0.925</td>
835
+ <td><br>0.982</td>
836
+ <td><br>0.80</td>
837
+ </tr>
838
+ <tr>
839
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td>
840
+ <td><br>Precision</td>
841
+ <td><br>0.971</td>
842
+ <td><br>0.947</td>
843
+ <td><br>0.866</td>
844
+ <td><br>0.994</td>
845
+ <td><br>0.989</td>
846
+ </tr>
847
+ <tr>
848
+ <td><br>Recall</td>
849
+ <td><br>0.969</td>
850
+ <td><br>0.942</td>
851
+ <td><br>0.891</td>
852
+ <td><br>0.995</td>
853
+ <td><br>0.989</td>
854
+ </tr>
855
+ <tr>
856
+ <td>F1</td>
857
+ <td><br>0.969</td>
858
+ <td><br>0.945</td>
859
+ <td><br>0.878</td>
860
+ <td><br>0.995</td>
861
+ <td><br>0.989</td>
862
+ </tr>
863
+ <tr>
864
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td>
865
+ <td><br>Precision</td>
866
+ <td><br>0.971</td>
867
+ <td><br>0.946</td>
868
+ <td><br>0.863 </td>
869
+ <td><br>0.994</td>
870
+ <td><br>0.988</td>
871
+ </tr>
872
+ <tr>
873
+ <td><br>Recall</td>
874
+ <td><br>0.967</td>
875
+ <td><br>0.946</td>
876
+ <td><br>0.870</td>
877
+ <td><br>0.995</td>
878
+ <td><br>0.988</td>
879
+ </tr>
880
+ <tr>
881
+ <td>F1</td>
882
+ <td><br>0.969</td>
883
+ <td><br>0.946</td>
884
+ <td><br>0.866</td>
885
+ <td><br>0.994</td>
886
+ <td><br>0.988</td>
887
+ </tr>
888
+ <tr>
889
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td>
890
+ <td><br>Precision</td>
891
+ <td><br>0.972</td>
892
+ <td><br>0.946</td>
893
+ <td><br>0.865</td>
894
+ <td><br>0.995</td>
895
+ <td><br>0.987</td>
896
+ </tr>
897
+ <tr>
898
+ <td><br>Recall</td>
899
+ <td><br>0.970</td>
900
+ <td><br>0.950</td>
901
+ <td><br>0.905</td>
902
+ <td><br>0.995</td>
903
+ <td><br>0.987</td>
904
+ </tr>
905
+ <tr>
906
+ <td>F1</td>
907
+ <td><br>0.971</td>
908
+ <td><br>0.948</td>
909
+ <td><br>0.885</td>
910
+ <td><br>0.995</td>
911
+ <td><br>0.987</td>
912
+ </tr>
913
+ <tr>
914
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td>
915
+ <td><br>Precision</td>
916
+ <td><br>0.973</td>
917
+ <td><br>0.953</td>
918
+ <td><br>0.873</td>
919
+ <td><br>0.996</td>
920
+ <td><br>0.990</td>
921
+ </tr>
922
+ <tr>
923
+ <td><br>Recall</td>
924
+ <td><br>0.990</td>
925
+ <td><br>0.948</td>
926
+ <td><br>0.913</td>
927
+ <td><br>0.995</td>
928
+ <td><br>0.990</td>
929
+ </tr>
930
+ <tr>
931
+ <td>F1</td>
932
+ <td><br>0.972</td>
933
+ <td><br>0.950</td>
934
+ <td><br>0.893</td>
935
+ <td><br>0.996</td>
936
+ <td><br>0.990</td>
937
+ </tr>
938
+ </tr>
939
+ </tbody>
940
+ </table>
941
+ </details>
942
+
943
+ ### wikiann
944
+
945
+ For space reasons, we show only the F1 of the different models. You can see the full results below the table.
946
+
947
+ <table>
948
+ <thead>
949
+ <tr>
950
+ <th><br>Model</th>
951
+ <th><br>PER</th>
952
+ <th><br>LOC</th>
953
+ <th><br>ORG</th>
954
+ </tr>
955
+ </thead>
956
+ <tbody>
957
+ <tr>
958
+ <td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td>
959
+ <td><br>0.867</td>
960
+ <td><br>0.722</td>
961
+ <td><br>0.451</td>
962
+ </tr>
963
+ <tr>
964
+ <td rowspan="1"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td>
965
+ <td><br>0.862</td>
966
+ <td><br>0.722</td>
967
+ <td><br>0.451</td>
968
+ </tr>
969
+ <tr>
970
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td>
971
+ <td><br>0.947</td>
972
+ <td><br>0.906</td>
973
+ <td><br>0.886</td>
974
+ </tr>
975
+ <tr>
976
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmembert2-3entities (111M)</a></td>
977
+ <td><br>0.950</td>
978
+ <td><br>0.911</td>
979
+ <td><br><b>0.910</b></td>
980
+ </tr>
981
+ <tr>
982
+ <td rowspan="1"><br>NERmemberta-3entities (111M) (this model)</td>
983
+ <td><br><b>0.953</b></td>
984
+ <td><br>0.902</td>
985
+ <td><br>0.890</td>
986
+ </tr>
987
+ <tr>
988
+ <td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td>
989
+ <td><br>0.949</td>
990
+ <td><br><b>0.912</b></td>
991
+ <td><br>0.899</td>
992
+ </tr>
993
+ </tr>
994
+ </tbody>
995
+ </table>
996
+
997
+
998
+ <details>
999
+ <summary>Full results</summary>
1000
+ <table>
1001
+ <thead>
1002
+ <tr>
1003
+ <th><br>Model</th>
1004
+ <th><br>Metrics</th>
1005
+ <th><br>PER</th>
1006
+ <th><br>LOC</th>
1007
+ <th><br>ORG</th>
1008
+ <th><br>O</th>
1009
+ <th><br>Overall</th>
1010
+ </tr>
1011
+ </thead>
1012
+ <tbody>
1013
+ <tr>
1014
+ <td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td>
1015
+ <td><br>Precision</td>
1016
+ <td><br>0.862</td>
1017
+ <td><br>0.700</td>
1018
+ <td><br>0.864</td>
1019
+ <td><br>0.867</td>
1020
+ <td><br>0.832</td>
1021
+ </tr>
1022
+ <tr>
1023
+ <td><br>Recall</td>
1024
+ <td><br>0.871</td>
1025
+ <td><br>0.746</td>
1026
+ <td><br>0.305</td>
1027
+ <td><br>0.950</td>
1028
+ <td><br>0.772</td>
1029
+ </tr>
1030
+ <tr>
1031
+ <td>F1</td>
1032
+ <td><br>0.867</td>
1033
+ <td><br>0.722</td>
1034
+ <td><br>0.451</td>
1035
+ <td><br>0.867</td>
1036
+ <td><br>0.801</td>
1037
+ </tr>
1038
+ <tr>
1039
+ <td rowspan="3"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td>
1040
+ <td><br>Precision</td>
1041
+ <td><br>0.862</td>
1042
+ <td><br>0.700</td>
1043
+ <td><br>0.864</td>
1044
+ <td><br>0.867</td>
1045
+ <td><br>0.832</td>
1046
+ </tr>
1047
+ <tr>
1048
+ <td><br>Recall</td>
1049
+ <td><br>0.871</td>
1050
+ <td><br>0.746</td>
1051
+ <td><br>0.305</td>
1052
+ <td><br>0.950</td>
1053
+ <td><br>0.772</td>
1054
+ </tr>
1055
+ <tr>
1056
+ <td>F1</td>
1057
+ <td><br>0.867</td>
1058
+ <td><br>0.722</td>
1059
+ <td><br>0.451</td>
1060
+ <td><br>0.907</td>
1061
+ <td><br>0.800</td>
1062
+ </tr>
1063
+ <tr>
1064
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td>
1065
+ <td><br>Precision</td>
1066
+ <td><br>0.948</td>
1067
+ <td><br>0.900</td>
1068
+ <td><br>0.893</td>
1069
+ <td><br>0.979</td>
1070
+ <td><br>0.942</td>
1071
+ </tr>
1072
+ <tr>
1073
+ <td><br>Recall</td>
1074
+ <td><br>0.946</td>
1075
+ <td><br>0.911</td>
1076
+ <td><br>0.878</td>
1077
+ <td><br>0.982</td>
1078
+ <td><br>0.942</td>
1079
+ </tr>
1080
+ <tr>
1081
+ <td>F1</td>
1082
+ <td><br>0.947</td>
1083
+ <td><br>0.906</td>
1084
+ <td><br>0.886</td>
1085
+ <td><br>0.980</td>
1086
+ <td><br>0.942</td>
1087
+ </tr>
1088
+ <tr>
1089
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td>
1090
+ <td><br>Precision</td>
1091
+ <td><br>0.962</td>
1092
+ <td><br>0.906</td>
1093
+ <td><br>0.890</td>
1094
+ <td><br>0.971</td>
1095
+ <td><br>0.941</td>
1096
+ </tr>
1097
+ <tr>
1098
+ <td><br>Recall</td>
1099
+ <td><br>0.938</td>
1100
+ <td><br>0.917</td>
1101
+ <td><br>0.884</td>
1102
+ <td><br>0.982</td>
1103
+ <td><br>0.941</td>
1104
+ </tr>
1105
+ <tr>
1106
+ <td>F1</td>
1107
+ <td><br>0.950</td>
1108
+ <td><br>0.911</td>
1109
+ <td><br>0.887</td>
1110
+ <td><br>0.976</td>
1111
+ <td><br>0.941</td>
1112
+ </tr>
1113
+ <tr>
1114
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td>
1115
+ <td><br>Precision</td>
1116
+ <td><br>0.961</td>
1117
+ <td><br>0.902</td>
1118
+ <td><br>0.899</td>
1119
+ <td><br>0.972</td>
1120
+ <td><br>0.942</td>
1121
+ </tr>
1122
+ <tr>
1123
+ <td><br>Recall</td>
1124
+ <td><br>0.946</td>
1125
+ <td><br>0.918</td>
1126
+ <td><br>0.881</td>
1127
+ <td><br>0.982</td>
1128
+ <td><br>0.942</td>
1129
+ </tr>
1130
+ <tr>
1131
+ <td>F1</td>
1132
+ <td><br>0.953</td>
1133
+ <td><br>0.902</td>
1134
+ <td><br>0.890</td>
1135
+ <td><br>0.977</td>
1136
+ <td><br>0.942</td>
1137
+ </tr>
1138
+ <tr>
1139
+ <td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td>
1140
+ <td><br>Precision</td>
1141
+ <td><br>0.958</td>
1142
+ <td><br>0.917</td>
1143
+ <td><br>0.897</td>
1144
+ <td><br>0.980</td>
1145
+ <td><br><b>0.948</b></td>
1146
+ </tr>
1147
+ <tr>
1148
+ <td><br>Recall</td>
1149
+ <td><br>0.940</td>
1150
+ <td><br>0.915</td>
1151
+ <td><br>0.901</td>
1152
+ <td><br>0.983</td>
1153
+ <td><br><b>0.948</b></td>
1154
+ </tr>
1155
+ <tr>
1156
+ <td>F1</td>
1157
+ <td><br><b>0.949</b></td>
1158
+ <td><br><b>0.912</b></td>
1159
+ <td><br><b>0.899</b></td>
1160
+ <td><br><b>0.983</b></td>
1161
+ <td><br><b>0.948</b></td>
1162
+ </tr>
1163
+ </tbody>
1164
+ </table>
1165
+ </details>
1166
+
1167
+
1168
+ ## Usage
1169
+ ### Code
1170
+
1171
+ ```python
1172
+ from transformers import pipeline
1173
+
1174
+ ner = pipeline('token-classification', model='CATIE-AQ/NERmembert2-base-3entities', tokenizer='CATIE-AQ/NERmembert2-base-3entities', aggregation_strategy="simple")
1175
+
1176
+ result = ner(
1177
+ "Le dévoilement du logo officiel des JO s'est déroulé le 21 octobre 2019 au Grand Rex. Ce nouvel emblème et cette nouvelle typographie ont été conçus par le designer Sylvain Boyer avec les agences Royalties & Ecobranding. Rond, il rassemble trois symboles : une médaille d'or, la flamme olympique et Marianne, symbolisée par un visage de femme mais privée de son bonnet phrygien caractéristique. La typographie dessinée fait référence à l'Art déco, mouvement artistique des années 1920, décennie pendant laquelle ont eu lieu pour la dernière fois les Jeux olympiques à Paris en 1924. Pour la première fois, ce logo sera unique pour les Jeux olympiques et les Jeux paralympiques."
1178
+ )
1179
+
1180
+ print(result)
1181
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1182
 
 
1183
 
1184
+ ### Try it through Space
1185
+ A Space has been created to test the model. It is available [here](https://huggingface.co/spaces/CATIE-AQ/NERmembert).
 
 
 
 
 
1186
 
 
1187
 
1188
+ ## Environmental Impact
1189
 
1190
+ *Carbon emissions were estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). The hardware, runtime, cloud provider, and compute region were utilized to estimate the carbon impact.*
1191
 
1192
+ - **Hardware Type:** A100 PCIe 40/80GB
1193
+ - **Hours used:** 1h51min
1194
+ - **Cloud Provider:** Private Infrastructure
1195
+ - **Carbon Efficiency (kg/kWh):** 0.055 (estimated from [electricitymaps](https://app.electricitymaps.com/zone/FR) for the day of November 21, 2024.)
1196
+ - **Carbon Emitted** *(Power consumption x Time x Carbon produced based on location of power grid)*: 0.014 kg eq. CO2
1197
 
 
1198
 
 
1199
 
1200
+ ## Citations
1201
 
1202
+ ### NERmemBERT2-base-3entities
1203
+ ```
1204
+ @misc {NERmemberta2024,
1205
+ author = { {BOURDOIS, Loïck} },
1206
+ organization = { {Centre Aquitain des Technologies de l'Information et Electroniques} },
1207
+ title = { NERmemberta-base-3entities },
1208
+ year = 2024,
1209
+ url = { https://huggingface.co/CATIE-AQ/NERmemberta-base-3entities },
1210
+ doi = { 10.57967/hf/1752 },
1211
+ publisher = { Hugging Face }
1212
+ }
1213
+ ```
1214
 
1215
+ ### NERmemBERT
1216
+ ```
1217
+ @misc {NERmembert2024,
1218
+ author = { {BOURDOIS, Loïck} },
1219
+ organization = { {Centre Aquitain des Technologies de l'Information et Electroniques} },
1220
+ title = { NERmembert-base-3entities },
1221
+ year = 2024,
1222
+ url = { https://huggingface.co/CATIE-AQ/NERmembert-base-3entities },
1223
+ doi = { 10.57967/hf/1752 },
1224
+ publisher = { Hugging Face }
1225
+ }
1226
+ ```
1227
 
1228
+ ### CamemBERT
1229
+ ```
1230
+ @inproceedings{martin2020camembert,
1231
+ title={CamemBERT: a Tasty French Language Model},
1232
+ author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t},
1233
+ booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
1234
+ year={2020}}
1235
+ ```
1236
 
1237
+ ### CamemBERT 2.0
1238
+ ```
1239
+ @misc{antoun2024camembert20smarterfrench,
1240
+ title={CamemBERT 2.0: A Smarter French Language Model Aged to Perfection},
1241
+ author={Wissam Antoun and Francis Kulumba and Rian Touchent and Éric de la Clergerie and Benoît Sagot and Djamé Seddah},
1242
+ year={2024},
1243
+ eprint={2411.08868},
1244
+ archivePrefix={arXiv},
1245
+ primaryClass={cs.CL},
1246
+ url={https://arxiv.org/abs/2411.08868},
1247
+ }
1248
+ ```
1249
 
1250
+ ### multiconer
1251
+ ```
1252
+ @inproceedings{multiconer2-report,
1253
+ title={{SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)}},
1254
+ author={Fetahu, Besnik and Kar, Sudipta and Chen, Zhiyu and Rokhlenko, Oleg and Malmasi, Shervin},
1255
+ booktitle={Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)},
1256
+ year={2023},
1257
+ publisher={Association for Computational Linguistics}}
1258
+
1259
+ @article{multiconer2-data,
1260
+ title={{MultiCoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognition}},
1261
+ author={Fetahu, Besnik and Chen, Zhiyu and Kar, Sudipta and Rokhlenko, Oleg and Malmasi, Shervin},
1262
+ year={2023}}
1263
+ ```
1264
+
1265
+ ### multinerd
1266
+ ```
1267
+ @inproceedings{tedeschi-navigli-2022-multinerd,
1268
+ title = "{M}ulti{NERD}: A Multilingual, Multi-Genre and Fine-Grained Dataset for Named Entity Recognition (and Disambiguation)",
1269
+ author = "Tedeschi, Simone and Navigli, Roberto",
1270
+ booktitle = "Findings of the Association for Computational Linguistics: NAACL 2022",
1271
+ month = jul,
1272
+ year = "2022",
1273
+ address = "Seattle, United States",
1274
+ publisher = "Association for Computational Linguistics",
1275
+ url = "https://aclanthology.org/2022.findings-naacl.60",
1276
+ doi = "10.18653/v1/2022.findings-naacl.60",
1277
+ pages = "801--812"}
1278
+ ```
1279
+
1280
+ ### pii-masking-200k
1281
+ ```
1282
+ @misc {ai4privacy_2023,
1283
+ author = { {ai4Privacy} },
1284
+ title = { pii-masking-200k (Revision 1d4c0a1) },
1285
+ year = 2023,
1286
+ url = { https://huggingface.co/datasets/ai4privacy/pii-masking-200k },
1287
+ doi = { 10.57967/hf/1532 },
1288
+ publisher = { Hugging Face }}
1289
+ ```
1290
 
1291
+ ### wikiann
1292
+ ```
1293
+ @inproceedings{rahimi-etal-2019-massively,
1294
+ title = "Massively Multilingual Transfer for {NER}",
1295
+ author = "Rahimi, Afshin and Li, Yuan and Cohn, Trevor",
1296
+ booktitle = "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics",
1297
+ month = jul,
1298
+ year = "2019",
1299
+ address = "Florence, Italy",
1300
+ publisher = "Association for Computational Linguistics",
1301
+ url = "https://www.aclweb.org/anthology/P19-1015",
1302
+ pages = "151--164"}
1303
+ ```
1304
+
1305
+ ### wikiner
1306
+ ```
1307
+ @article{NOTHMAN2013151,
1308
+ title = {Learning multilingual named entity recognition from Wikipedia},
1309
+ journal = {Artificial Intelligence},
1310
+ volume = {194},
1311
+ pages = {151-175},
1312
+ year = {2013},
1313
+ note = {Artificial Intelligence, Wikipedia and Semi-Structured Resources},
1314
+ issn = {0004-3702},
1315
+ doi = {https://doi.org/10.1016/j.artint.2012.03.006},
1316
+ url = {https://www.sciencedirect.com/science/article/pii/S0004370212000276},
1317
+ author = {Joel Nothman and Nicky Ringland and Will Radford and Tara Murphy and James R. Curran}}
1318
+ ```
1319
+
1320
+ ### frenchNER_3entities
1321
+ ```
1322
+ @misc {frenchNER2024,
1323
+ author = { {BOURDOIS, Loïck} },
1324
+ organization = { {Centre Aquitain des Technologies de l'Information et Electroniques} },
1325
+ title = { frenchNER_3entities },
1326
+ year = 2024,
1327
+ url = { https://huggingface.co/CATIE-AQ/frenchNER_3entities },
1328
+ doi = { 10.57967/hf/1751 },
1329
+ publisher = { Hugging Face }
1330
+ }
1331
+ ```
1332
 
1333
+ ## License
1334
+ MIT