Cyrile commited on
Commit
f9ace7f
1 Parent(s): efd60c3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -27,24 +27,24 @@ Evaluation results
27
 
28
  | **class** | **precision (%)** | **recall (%)** | **f1 (%)** | **support (#sub-word)** |
29
  | :------------: | :---------------: | :------------: | :--------: | :---------------------: |
30
- | **global** | 98.35 | 98.36 | 98.35 | 492'243 |
31
- | **PER** | 96.22 | 97.41 | 96.81 | 27'842 |
32
- | **LOC** | 93.93 | 93.50 | 93.72 | 31'431 |
33
- | **ORG** | 85.13 | 87.08 | 86.10 | 7'662 |
34
- | **MISC** | 88.55 | 81.84 | 85.06 | 13'553 |
35
- | **O** | 99.40 | 99.55 | 99.47 | 411'755 |
36
 
37
  Benchmark
38
  ---------
39
 
40
- This model performance is compared to 2 reference models (see below) with the metric [MCC (Matthews Correlation Coefficient)](https://en.wikipedia.org/wiki/Phi_coefficient). The score is given with a factor x100 and the delta gain with DistilCamemBERT-NER used in reference is in parantheses. For the mean inference time measure, an AMD Ryzen 5 4500U @ 2.3GHz with 6 cores was used:
41
 
42
- | **model** | **time (ms)** | **PER** | **LOC** | **ORG** | **MISC** | **O** |
43
  | :---------------------------------------------------------------------------------------------------------------: | :----------------: | :--------------: | :--------------: | :--------------: | :--------------: | :------------- : |
44
- | [cmarkea/distilcamembert-base-ner](https://huggingface.co/cmarkea/distilcamembert-base-ner) | **43.44** | **93.91** | **88.26** | **84.03** | **82.74** | **91.45** |
45
- | [Davlan/bert-base-multilingual-cased-ner-hrl](https://huggingface.co/Davlan/bert-base-multilingual-cased-ner-hrl) | 87.56<br/>(+102%) | 79.93<br/>(-15%) | 70.39<br/>(-22%) | 60.26<br/>(-28%) | n/a<br/>(n/a%) | 69.95<br/>(-24%) |
46
- | [flair/ner-french](https://huggingface.co/flair/ner-french) | 314.96<br/>(+625%) | 80.18<br/>(-15%) | 72.11<br/>(-18%) | 67.29<br/>(-20%) | 72.39<br/>(-17%) | 74.34<br/>(-19%) |
47
- <!--- | [Jean-Baptiste/camembert-ner](https://huggingface.co/Jean-Baptiste/camembert-ner) | 83.70<br/>(+93%) | 95.20<br/>(+1%) | 90.85<br/>(+3%) | 89.50<br/>(+6%) | 89.02<br/>(+8%) | 92.86<br/>(+2%) | problème de sur-apprentissage car pas moyen de savoir quelles sont les observations d'éval --->
48
 
49
  How to use DistilCamemBERT-NER
50
  ------------------------------
 
27
 
28
  | **class** | **precision (%)** | **recall (%)** | **f1 (%)** | **support (#sub-word)** |
29
  | :------------: | :---------------: | :------------: | :--------: | :---------------------: |
30
+ | **global** | 98.17 | 98.19 | 98.18 | 378'776 |
31
+ | **PER** | 96.78 | 96.87 | 96.82 | 23'754 |
32
+ | **LOC** | 94.05 | 93.59 | 93.82 | 27'196 |
33
+ | **ORG** | 86.05 | 85.92 | 85.98 | 6'526 |
34
+ | **MISC** | 88.78 | 84.69 | 86.69 | 11'891 |
35
+ | **O** | 99.26 | 99.47 | 99.37 | 309'409 |
36
 
37
  Benchmark
38
  ---------
39
 
40
+ This model performance is compared to 3 reference models (see below) with the metric f1 score. For the mean inference time measure, an AMD Ryzen 5 4500U @ 2.3GHz with 6 cores was used:
41
 
42
+ | **model** | **time (ms)** | **PER (%)** | **LOC (%)** | **ORG (%)** | **MISC (%)** | **O (%)** |
43
  | :---------------------------------------------------------------------------------------------------------------: | :----------------: | :--------------: | :--------------: | :--------------: | :--------------: | :------------- : |
44
+ | [cmarkea/distilcamembert-base-ner](https://huggingface.co/cmarkea/distilcamembert-base-ner) | **43.44** | 96.82 | 93.82 | 85.98 | 86.69 | 99.37 |
45
+ | [Jean-Baptiste/camembert-ner](https://huggingface.co/Jean-Baptiste/camembert-ner) | 88.49 | **98.82** | **97.35** | **95.03** | **95.34** | **99.67** |
46
+ | [Davlan/bert-base-multilingual-cased-ner-hrl](https://huggingface.co/Davlan/bert-base-multilingual-cased-ner-hrl) | 87.56 | 79.93 | 72.89 | 61.34 | n/a | 96.04 |
47
+ | [flair/ner-french](https://huggingface.co/flair/ner-french) | 314.96 | 83.55 | 78.13 | 74.19 | 77.13 | 97.88 |
48
 
49
  How to use DistilCamemBERT-NER
50
  ------------------------------