---
license: apache-2.0
tags:
- summarization
- generated_from_trainer
metrics:
- rouge
model-index:
- name: BARTkrame-abstract
  results: []
widget:
- text: "Nous voulons que tous les êtres humains, ensemble ou pris isolément, jeunes ou vieux, riches ou pauvres, nobles ou roturiers, hommes ou femmes, puissent pleinement s'instruire et devenir des êtres achevés. Nous voulons qu'ils soient instruits parfaitement et formés non seulement sur tel ou tel point, mais également sur tout ce qui permet à l'homme de réaliser intégralement son essence"
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# BARTkrame-abstract

This model is a fine-tuned version of [krm/BARTkrame-abstract](https://huggingface.co/krm/BARTkrame-abstract) on the [krm/for-ULPGL-Dissertation](https://huggingface.co/datasets/krm/for-ULPGL-Dissertation) dataset.
It achieves (15/10/2022) the following results on the evaluation set:
- Loss: 2.4196
- Rouge1: 0.2703
- Rouge2: 0.1334
- Rougel: 0.2392
- Rougelsum: 0.2419

## Model description

This model is primarly a finetuned version of [moussaKam/mbarthez](https://huggingface.co/moussaKam/mbarthez).

## Intended uses & limitations

More information needed

## Training and evaluation data
We have used the [krm/for-ULPGL-Dissertation](https://huggingface.co/datasets/krm/for-ULPGL-Dissertation) dataset reduced to :

> **Training data :** **5000** samples taken at random with *seed=42*.

> **Validation data :** **100** samples taken at random with *seed=42*.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5.6e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 12

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|
| 0.1316        | 9.0   | 1250 | 2.3251          | 0.2505 | 0.1158 | 0.2150 | 0.2184    |
| 0.0894        | 10.0   | 2500 | 2.3467          | 0.2526 | 0.1073 | 0.2067 | 0.2124    |
| 0.045         | 11.0   | 3750 | 2.3742          | 0.2593 | 0.1211 | 0.2281 | 0.2308    |
| 0.0242        | 12.0   | 5000 | 2.4196          | 0.2703 | 0.1334 | 0.2392 | 0.2419    |


### Framework versions

- Transformers 4.23.1
- Pytorch 1.12.1+cu113
- Datasets 2.6.1
- Tokenizers 0.13.1