---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- f1
- accuracy
model-index:
- name: distilBERT-finetuned-resumes-sections
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilBERT-finetuned-resumes-sections

This model is a fine-tuned version of [Geotrend/distilbert-base-en-fr-cased](https://huggingface.co/Geotrend/distilbert-base-en-fr-cased) on a private resume sections dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0487
- F1: 0.9512
- Roc Auc: 0.9729
- Accuracy: 0.9482

## Model description
This model classifies a resume section into 12 classes.

### Possible classes for a resume section
```awards```
 ```certificates```
 ```contact/name/title```
 ```education```
 ```interests```
 ```languages```
 ```para```
 ```professional_experiences```
 ```projects```
 ```skills```
 ```soft_skills```
 ```summary```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step  | Validation Loss | F1     | Roc Auc | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:------:|:-------:|:--------:|
| 0.058         | 1.0   | 1083  | 0.0457          | 0.9186 | 0.9494  | 0.9020   |
| 0.0277        | 2.0   | 2166  | 0.0393          | 0.9327 | 0.9614  | 0.9251   |
| 0.0154        | 3.0   | 3249  | 0.0333          | 0.9425 | 0.9671  | 0.9367   |
| 0.0104        | 4.0   | 4332  | 0.0408          | 0.9357 | 0.9645  | 0.9293   |
| 0.0084        | 5.0   | 5415  | 0.0405          | 0.9376 | 0.9643  | 0.9298   |
| 0.0065        | 6.0   | 6498  | 0.0419          | 0.9439 | 0.9699  | 0.9385   |
| 0.0051        | 7.0   | 7581  | 0.0450          | 0.9412 | 0.9674  | 0.9376   |
| 0.0034        | 8.0   | 8664  | 0.0406          | 0.9433 | 0.9684  | 0.9372   |
| 0.0035        | 9.0   | 9747  | 0.0441          | 0.9403 | 0.9664  | 0.9358   |
| 0.0024        | 10.0  | 10830 | 0.0492          | 0.9419 | 0.9678  | 0.9367   |
| 0.0026        | 11.0  | 11913 | 0.0470          | 0.9468 | 0.9708  | 0.9436   |
| 0.0022        | 12.0  | 12996 | 0.0514          | 0.9424 | 0.9679  | 0.9395   |
| 0.0013        | 13.0  | 14079 | 0.0458          | 0.9478 | 0.9715  | 0.9441   |
| 0.0019        | 14.0  | 15162 | 0.0494          | 0.9477 | 0.9711  | 0.9450   |
| 0.0007        | 15.0  | 16245 | 0.0492          | 0.9496 | 0.9719  | 0.9464   |
| 0.0009        | 16.0  | 17328 | 0.0487          | 0.9512 | 0.9729  | 0.9482   |
| 0.001         | 17.0  | 18411 | 0.0510          | 0.9480 | 0.9711  | 0.9441   |
| 0.0006        | 18.0  | 19494 | 0.0532          | 0.9477 | 0.9709  | 0.9441   |
| 0.0007        | 19.0  | 20577 | 0.0511          | 0.9487 | 0.9720  | 0.9445   |
| 0.0005        | 20.0  | 21660 | 0.0522          | 0.9471 | 0.9710  | 0.9436   |


### Framework versions

- Transformers 4.20.1
- Pytorch 1.12.0+cu113
- Datasets 2.3.2
- Tokenizers 0.12.1