Hugging Face's logo

CATIE-AQ
/
NERmembert-large-4entities

Token Classification
Transformers
Safetensors
French
camembert
Inference Endpoints
Model card Files Community
NERmembert-large-4entities / README.md
bourdoiscatie's picture
bourdoiscatie
Update README.md
07dbafe verified
preview code
|
raw
history blame
30.2 kB
---
license: mit
base_model: camembert/camembert-large
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: NERmembert-large-4entities
results: []
datasets:
- CATIE-AQ/frenchNER_4entities
language:
- fr
widget:
- text: "Assurés de disputer l'Euro 2024 en Allemagne l'été prochain (du 14 juin au 14 juillet) depuis leur victoire aux Pays-Bas, les Bleus ont fait le nécessaire pour avoir des certitudes. Avec six victoires en six matchs officiels et un seul but encaissé, Didier Deschamps a consolidé les acquis de la dernière Coupe du monde. Les joueurs clés sont connus : Kylian Mbappé, Aurélien Tchouameni, Antoine Griezmann, Ibrahima Konaté ou encore Mike Maignan."
library_name: transformers
pipeline_tag: token-classification
co2_eq_emissions: 80
---
# NERmembert-large-4entities
## Model Description
We present **NERmembert-large-4entities**, which is a [CamemBERT large](https://huggingface.co/camembert/camembert-large) fine-tuned for the Name Entity Recognition task for the French language on four French NER datasets for 4 entities (LOC, PER, ORG, MISC).
All these datasets were concatenated and cleaned into a single dataset that we called [frenchNER_4entities](https://huggingface.co/datasets/CATIE-AQ/frenchNER_4entities).
There are a total of **384,773** rows, of which **328,757** are for training, **24,131** for validation and **31,885** for testing.
Our methodology is described in a blog post available in [English](https://blog.vaniila.ai/en/NER_en/) or [French](https://blog.vaniila.ai/NER/).
## Dataset
The dataset used is [frenchNER_4entities](https://huggingface.co/datasets/CATIE-AQ/frenchNER_4entities), which represents ~385k sentences labeled in 4 categories:
| Label | Examples |
|:------|:-----------------------------------------------------------|
| PER | "La Bruyère", "Gaspard de Coligny", "Wittgenstein" |
| ORG | "UTBM", "American Airlines", "id Software" |
| LOC | "République du Cap-Vert", "Créteil", "Bordeaux" |
| MISC | "Wolfenstein 3D", "Révolution française", "Coupe du monde" |
The distribution of the entities is as follows:
<table>
<thead>
<tr>
<th><br>Splits</th>
<th><br>O</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
</tr>
</thead>
<tbody>
<td><br>train</td>
<td><br>7,539,692</td>
<td><br>307,144</td>
<td><br>286,746</td>
<td><br>127,089</td>
<td><br>799,494</td>
</tr>
<tr>
<td><br>validation</td>
<td><br>544,580</td>
<td><br>24,034</td>
<td><br>21,585</td>
<td><br>5,927</td>
<td><br>18,221</td>
</tr>
<tr>
<td><br>test</td>
<td><br>720,623</td>
<td><br>32,870</td>
<td><br>29,683</td>
<td><br>7,911</td>
<td><br>21,760</td>
</tr>
</tbody>
</table>
## Evaluation results
The evaluation was carried out using the [**evaluate**](https://pypi.org/project/evaluate/) python package.
### frenchNER_4entities
For space reasons, we show only the F1 of the different models. You can see the full results below the table.
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>0.971</td>
<td><br>0.947</td>
<td><br>0.902</td>
<td><br>0.663</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>0.974</td>
<td><br>0.948</td>
<td><br>0.892</td>
<td><br>0.658</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>0.978</td>
<td><br>0.957</td>
<td><br>0.904</td>
<td><br>0</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-4entities">NERmembert-base-4entities</a></td>
<td><br>0.978</td>
<td><br>0.958</td>
<td><br>0.903</td>
<td><br>0.814</td>
</tr>
<tr>
<td rowspan="1"><br>NERmembert-large-4entities (this model)</td>
<td><br>0.982</td>
<td><br>0.964</td>
<td><br>0.919</td>
<td><br>0.834</td>
</tr>
</tbody>
</table>
<details>
<summary>Full results</summary>
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>Metrics</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
<th><br>O</th>
<th><br>Overall</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>Precision</td>
<td><br>0.952</td>
<td><br>0.924</td>
<td><br>0.870</td>
<td><br>0.845</td>
<td><br>0.986</td>
<td><br>0.976</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.990</td>
<td><br>0.972</td>
<td><br>0.938</td>
<td><br>0.546</td>
<td><br>0.992</td>
<td><br>0.976</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.971</td>
<td><br>0.947</td>
<td><br>0.902</td>
<td><br>0.663</td>
<td><br>0.989</td>
<td><br>0.976</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>Precision</td>
<td><br>0.962</td>
<td><br>0.933</td>
<td><br>0.857</td>
<td><br>0.830</td>
<td><br>0.985</td>
<td><br>0.976</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.987</td>
<td><br>0.963</td>
<td><br>0.930</td>
<td><br>0.545</td>
<td><br>0.993</td>
<td><br>0.976</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.974</td>
<td><br>0.948</td>
<td><br>0.892</td>
<td><br>0.658</td>
<td><br>0.989</td>
<td><br>0.976</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>Precision</td>
<td><br>0.973</td>
<td><br>0.955</td>
<td><br>0.886</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.983</td>
<td><br>0.960</td>
<td><br>0.923</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.978</td>
<td><br>0.957</td>
<td><br>0.904</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-4entities">NERmembert-base-4entities</a></td>
<td><br>Precision</td>
<td><br>0.973</td>
<td><br>0.951</td>
<td><br>0.888</td>
<td><br>0.850</td>
<td><br>0.993</td>
<td><br>0.984</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.983</td>
<td><br>0.964</td>
<td><br>0.918</td>
<td><br>0.781</td>
<td><br>0.993</td>
<td><br>0.984</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.978</td>
<td><br>0.958</td>
<td><br>0.903</td>
<td><br>0.814</td>
<td><br>0.993</td>
<td><br>0.984</td>
</tr>
<tr>
<td rowspan="3"><br>NERmembert-large-4entities (this model)</td>
<td><br>Precision</td>
<td><br>0.977</td>
<td><br>0.961</td>
<td><br>0.896</td>
<td><br>0.872</td>
<td><br>0.993</td>
<td><br>0.986</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.987</td>
<td><br>0.966</td>
<td><br>0.943</td>
<td><br>0.798</td>
<td><br>0.995</td>
<td><br>0.986</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.982</td>
<td><br>0.964</td>
<td><br>0.919</td>
<td><br>0.834</td>
<td><br>0.994</td>
<td><br>0.986</td>
</tr>
</tbody>
</table>
</details>
In detail:
### multiconer
For space reasons, we show only the F1 of the different models. You can see the full results below the table.
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>0.940</td>
<td><br>0.761</td>
<td><br>0.723</td>
<td><br>0.560</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>0.921</td>
<td><br>0.748</td>
<td><br>0.694</td>
<td><br>0.530</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>0.960</td>
<td><br>0.887</td>
<td><br>0.877</td>
<td><br>0</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-4entities">NERmembert-base-4entities</a></td>
<td><br>0.960</td>
<td><br>0.890</td>
<td><br>0.867</td>
<td><br>0.852</td>
</tr>
<tr>
<td rowspan="1"><br>NERmembert-large-4entities (this model)</td>
<td><br>0.969</td>
<td><br>0.919</td>
<td><br>0.904</td>
<td><br>0.864</td>
</tr>
</tbody>
</table>
<details>
<summary>Full results</summary>
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>Metrics</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
<th><br>O</th>
<th><br>Overall</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>Precision</td>
<td><br>0.908</td>
<td><br>0.717</td>
<td><br>0.753</td>
<td><br>0.620</td>
<td><br>0.936</td>
<td><br>0.889</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.975</td>
<td><br>0.811</td>
<td><br>0.696</td>
<td><br>0.511</td>
<td><br>0.938</td>
<td><br>0.889</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.940</td>
<td><br>0.761</td>
<td><br>0.723</td>
<td><br>0.560</td>
<td><br>0.937</td>
<td><br>0.889</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>Precision</td>
<td><br>0.885</td>
<td><br>0.738</td>
<td><br>0.737</td>
<td><br>0.589</td>
<td><br>0.928</td>
<td><br>0.881</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.960</td>
<td><br>0.759</td>
<td><br>0.655</td>
<td><br>0.482</td>
<td><br>0.939</td>
<td><br>0.881</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.921</td>
<td><br>0.748</td>
<td><br>0.694</td>
<td><br>0.530</td>
<td><br>0.934</td>
<td><br>0.881</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>Precision</td>
<td><br>0.957</td>
<td><br>0.894</td>
<td><br>0.876</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.962</td>
<td><br>0.880</td>
<td><br>0.878</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.960</td>
<td><br>0.887</td>
<td><br>0.877</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-4entities">NERmembert-base-4entities</a></td>
<td><br>Precision</td>
<td><br>0.954</td>
<td><br>0.893</td>
<td><br>0.851</td>
<td><br>0.849</td>
<td><br>0.979</td>
<td><br>0.954</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.967</td>
<td><br>0.887</td>
<td><br>0.883</td>
<td><br>0.855</td>
<td><br>0.974</td>
<td><br>0.954</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.960</td>
<td><br>0.890</td>
<td><br>0.867</td>
<td><br>0.852</td>
<td><br>0.977</td>
<td><br>0.954</td>
</tr>
<tr>
<td rowspan="3"><br>NERmembert-large-4entities (this model)</td>
<td><br>Precision</td>
<td><br>0.964</td>
<td><br>0.922</td>
<td><br>0.904</td>
<td><br>0.856</td>
<td><br>0.981</td>
<td><br>0.961</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.975</td>
<td><br>0.917</td>
<td><br>0.904</td>
<td><br>0.872</td>
<td><br>0.976</td>
<td><br>0.961</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.969</td>
<td><br>0.919</td>
<td><br>0.904</td>
<td><br>0.864</td>
<td><br>0.978</td>
<td><br>0.961</td>
</tr>
</tbody>
</table>
</details>
### multinerd
For space reasons, we show only the F1 of the different models. You can see the full results below the table.
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>0.962</td>
<td><br>0.934</td>
<td><br>0.888</td>
<td><br>0.419</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>0.972</td>
<td><br>0.938</td>
<td><br>0.884</td>
<td><br>0.430</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>0.985</td>
<td><br>0.973</td>
<td><br>0.938</td>
<td><br>0</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-4entities">NERmembert-base-4entities</a></td>
<td><br>0.985</td>
<td><br>0.973</td>
<td><br>0.938</td>
<td><br>0.770</td>
</tr>
<tr>
<td rowspan="1"><br>NERmembert-large-4entities (this model)</td>
<td><br>0.987</td>
<td><br>0.976</td>
<td><br>0.948</td>
<td><br>0.790</td>
</tr>
</tbody>
</table>
<details>
<summary>Full results</summary>
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>Metrics</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
<th><br>O</th>
<th><br>Overall</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>Precision</td>
<td><br>0.931</td>
<td><br>0.893</td>
<td><br>0.827</td>
<td><br>0.725</td>
<td><br>0.979</td>
<td><br>0.966</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.994</td>
<td><br>0.980</td>
<td><br>0.959</td>
<td><br>0.295</td>
<td><br>0.990</td>
<td><br>0.966</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.962</td>
<td><br>0.934</td>
<td><br>0.888</td>
<td><br>0.419</td>
<td><br>0.984</td>
<td><br>0.966</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>Precision</td>
<td><br>0.954</td>
<td><br>0.908</td>
<td><br>0.817</td>
<td><br>0.705</td>
<td><br>0.977</td>
<td><br>0.967</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.991</td>
<td><br>0.969</td>
<td><br>0.963</td>
<td><br>0.310</td>
<td><br>0.990</td>
<td><br>0.967</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.972</td>
<td><br>0.938</td>
<td><br>0.884</td>
<td><br>0.430</td>
<td><br>0.984</td>
<td><br>0.967</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>Precision</td>
<td><br>0.974</td>
<td><br>0.965</td>
<td><br>0.910</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.995</td>
<td><br>0.981</td>
<td><br>0.968</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.985</td>
<td><br>0.973</td>
<td><br>0.938</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-4entities">NERmembert-base-4entities</a></td>
<td><br>Precision</td>
<td><br>0.976</td>
<td><br>0.961</td>
<td><br>0.91</td>
<td><br>0.829</td>
<td><br>0.991</td>
<td><br>0.983</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.994</td>
<td><br>0.985</td>
<td><br>0.967</td>
<td><br>0.719</td>
<td><br>0.993</td>
<td><br>0.983</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.985</td>
<td><br>0.973</td>
<td><br>0.938</td>
<td><br>0.770</td>
<td><br>0.992</td>
<td><br>0.983</td>
</tr>
<tr>
<td rowspan="3"><br>NERmembert-large-4entities (this model)</td>
<td><br>Precision</td>
<td><br>0.979</td>
<td><br>0.967</td>
<td><br>0.922</td>
<td><br>0.852</td>
<td><br>0.991</td>
<td><br>0.985</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.996</td>
<td><br>0.986</td>
<td><br>0.974</td>
<td><br>0.736</td>
<td><br>0.994</td>
<td><br>0.985</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.987</td>
<td><br>0.976</td>
<td><br>0.948</td>
<td><br>0.790</td>
<td><br>0.993</td>
<td><br>0.985</td>
</tr>
</tbody>
</table>
</details>
### wikiner
For space reasons, we show only the F1 of the different models. You can see the full results below the table.
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>0.986</td>
<td><br>0.966</td>
<td><br>0.938</td>
<td><br>0.938</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>0.983</td>
<td><br>0.964</td>
<td><br>0.925</td>
<td><br>0.926</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>0.970</td>
<td><br>0.945</td>
<td><br>0.878</td>
<td><br>0</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-4entities">NERmembert-base-4entities</a></td>
<td><br>0.970</td>
<td><br>0.945</td>
<td><br>0.876</td>
<td><br>0.872</td>
</tr>
<tr>
<td rowspan="1"><br>NERmembert-large-4entities (this model)</td>
<td><br>0.975</td>
<td><br>0.953</td>
<td><br>0.896</td>
<td><br>0.893</td>
</tr>
</tbody>
</table>
<details>
<summary>Full results</summary>
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>Metrics</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
<th><br>O</th>
<th><br>Overall</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>Precision</td>
<td><br>0.986</td>
<td><br>0.962</td>
<td><br>0.925</td>
<td><br>0.943</td>
<td><br>0.998</td>
<td><br>0.992</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.987</td>
<td><br>0.969</td>
<td><br>0.951</td>
<td><br>0.933</td>
<td><br>0.997</td>
<td><br>0.992</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.986</td>
<td><br>0.966</td>
<td><br>0.938</td>
<td><br>0.938</td>
<td><br>0.998</td>
<td><br>0.992</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>Precision</td>
<td><br>0.982</td>
<td><br>0.964</td>
<td><br>0.910</td>
<td><br>0.942</td>
<td><br>0.997</td>
<td><br>0.991</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.985</td>
<td><br>0.963</td>
<td><br>0.940</td>
<td><br>0.910</td>
<td><br>0.998</td>
<td><br>0.991</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.983</td>
<td><br>0.964</td>
<td><br>0.925</td>
<td><br>0.926</td>
<td><br>0.997</td>
<td><br>0.991</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>Precision</td>
<td><br>0.971</td>
<td><br>0.947</td>
<td><br>0.866</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.969</td>
<td><br>0.943</td>
<td><br>0.891</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.970</td>
<td><br>0.945</td>
<td><br>0.878</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf.co/CATIE-AQ/NERmembert-base-4entities">NERmembert-base-4entities</a></td>
<td><br>Precision</td>
<td><br>0.970</td>
<td><br>0.944</td>
<td><br>0.872</td>
<td><br>0.878</td>
<td><br>0.996</td>
<td><br>0.986</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.969</td>
<td><br>0.947</td>
<td><br>0.880</td>
<td><br>0.866</td>
<td><br>0.996</td>
<td><br>0.986</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.970</td>
<td><br>0.945</td>
<td><br>0.876</td>
<td><br>0.872</td>
<td><br>0.996</td>
<td><br>0.986</td>
</tr>
<tr>
<td rowspan="3"><br>NERmembert-large-4entities (this model)</td>
<td><br>Precision</td>
<td><br>0.975</td>
<td><br>0.957</td>
<td><br>0.872</td>
<td><br>0.901</td>
<td><br>0.997</td>
<td><br>0.989</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.975</td>
<td><br>0.949</td>
<td><br>0.922</td>
<td><br>0.884</td>
<td><br>0.997</td>
<td><br>0.989</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.975</td>
<td><br>0.953</td>
<td><br>0.896</td>
<td><br>0.893</td>
<td><br>0.997</td>
<td><br>0.989</td>
</tr>
</tbody>
</table>
</details>
## Usage
### Code
```python
from transformers import pipeline
ner = pipeline('token-classification', model='CATIE-AQ/NERmembert-large-4entities', tokenizer='CATIE-AQ/NERmembert-large-4entities', aggregation_strategy="simple")
results = ner(
"Assurés de disputer l'Euro 2024 en Allemagne l'été prochain (du 14 juin au 14 juillet) depuis leur victoire aux Pays-Bas, les Bleus ont fait le nécessaire pour avoir des certitudes. Avec six victoires en six matchs officiels et un seul but encaissé, Didier Deschamps a consolidé les acquis de la dernière Coupe du monde. Les joueurs clés sont connus : Kylian Mbappé, Aurélien Tchouameni, Antoine Griezmann, Ibrahima Konaté ou encore Mike Maignan."
)
print(result)
```
```python
```
### Try it through Space
A Space has been created to test the model. It is available [here](https://huggingface.co/spaces/CATIE-AQ/NERmembert).
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
### Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
|:-------------:|:-----:|:------:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.0347 | 1.0 | 41095 | 0.0537 | 0.9832 | 0.9832 | 0.9832 | 0.9832 |
| 0.0237 | 2.0 | 82190 | 0.0448 | 0.9858 | 0.9858 | 0.9858 | 0.9858 |
| 0.0119 | 3.0 | 123285 | 0.0532 | 0.9860 | 0.9860 | 0.9860 | 0.9860 |
### Framework versions
- Transformers 4.36.2
- Pytorch 2.1.2
- Datasets 2.16.1
- Tokenizers 0.15.0
## Environmental Impact
*Carbon emissions were estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). The hardware, runtime, cloud provider, and compute region were utilized to estimate the carbon impact.*
- **Hardware Type:** A100 PCIe 40/80GB
- **Hours used:** 4h17min
- **Cloud Provider:** Private Infrastructure
- **Carbon Efficiency (kg/kWh):** 0.078 (estimated from [electricitymaps](https://app.electricitymaps.com/zone/FR) for the day of January 10, 2024.)
- **Carbon Emitted** *(Power consumption x Time x Carbon produced based on location of power grid)*: 0.08 kg eq. CO2
## Citations
### NERmembert-large-4entities
```
TODO
```
### multiconer
> @inproceedings{multiconer2-report,
title={{SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)}},
author={Fetahu, Besnik and Kar, Sudipta and Chen, Zhiyu and Rokhlenko, Oleg and Malmasi, Shervin},
booktitle={Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)},
year={2023},
publisher={Association for Computational Linguistics}}
> @article{multiconer2-data,
title={{MultiCoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognition}},
author={Fetahu, Besnik and Chen, Zhiyu and Kar, Sudipta and Rokhlenko, Oleg and Malmasi, Shervin},
year={2023}}
### multinerd
> @inproceedings{tedeschi-navigli-2022-multinerd,
title = "{M}ulti{NERD}: A Multilingual, Multi-Genre and Fine-Grained Dataset for Named Entity Recognition (and Disambiguation)",
author = "Tedeschi, Simone and Navigli, Roberto",
booktitle = "Findings of the Association for Computational Linguistics: NAACL 2022",
month = jul,
year = "2022",
address = "Seattle, United States",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.findings-naacl.60",
doi = "10.18653/v1/2022.findings-naacl.60",
pages = "801--812"}
### pii-masking-200k
> @misc {ai4privacy_2023,
author = { {ai4Privacy} },
title = { pii-masking-200k (Revision 1d4c0a1) },
year = 2023,
url = { https://huggingface.co/datasets/ai4privacy/pii-masking-200k },
doi = { 10.57967/hf/1532 },
publisher = { Hugging Face }}
### wikiner
> @article{NOTHMAN2013151,
title = {Learning multilingual named entity recognition from Wikipedia},
journal = {Artificial Intelligence},
volume = {194},
pages = {151-175},
year = {2013},
note = {Artificial Intelligence, Wikipedia and Semi-Structured Resources},
issn = {0004-3702},
doi = {https://doi.org/10.1016/j.artint.2012.03.006},
url = {https://www.sciencedirect.com/science/article/pii/S0004370212000276},
author = {Joel Nothman and Nicky Ringland and Will Radford and Tara Murphy and James R. Curran}}
### frenchNER_4entities
```
TODO
```
### CamemBERT
> @inproceedings{martin2020camembert,
title={CamemBERT: a Tasty French Language Model},
author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t},
booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
year={2020}}
## License
[cc-by-4.0](https://creativecommons.org/licenses/by/4.0/deed.en)