---
tags:
- generated_from_trainer
datasets:
- id_liputan6
model-index:
- name: bert2bert-extabs-canonicalcleandata-lr-5e-05-batchsize-4-encmaxlen-512-decmaxlen-256
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bert2bert-extabs-canonicalcleandata-lr-5e-05-batchsize-4-encmaxlen-512-decmaxlen-256
10 Epoch Extractive Training + 10 Epoch Abtractive Training
- Dev Set: Canonical Clean Data & Extreme Clean Data
- Encoder max length (input): 512
- Decoder max length (output): 256

This model was trained from scratch on the id_liputan6 dataset.
It achieves the following results on the evaluation set:
- Loss: 3.0021
- R1 Precision: 0.3553
- R1 Recall: 0.2599
- R1 Fmeasure: 0.2974
- R2 Precision: 0.1458
- R2 Recall: 0.1039
- R2 Fmeasure: 0.12
- Rl Precision: 0.2925
- Rl Recall: 0.2139
- Rl Fmeasure: 0.2448

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 18
- eval_batch_size: 18
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step   | Validation Loss | R1 Precision | R1 Recall | R1 Fmeasure | R2 Precision | R2 Recall | R2 Fmeasure | Rl Precision | Rl Recall | Rl Fmeasure |
|:-------------:|:-----:|:------:|:---------------:|:------------:|:---------:|:-----------:|:------------:|:---------:|:-----------:|:------------:|:---------:|:-----------:|
| 1.7393        | 1.0   | 10772  | 2.6782          | 0.3432       | 0.2497    | 0.2864      | 0.1375       | 0.0975    | 0.113       | 0.2828       | 0.206     | 0.2361      |
| 1.4091        | 2.0   | 21544  | 2.6063          | 0.3486       | 0.2534    | 0.2907      | 0.142        | 0.1004    | 0.1164      | 0.2878       | 0.2094    | 0.2401      |
| 1.246         | 3.0   | 32316  | 2.6079          | 0.3535       | 0.2578    | 0.2955      | 0.1457       | 0.1036    | 0.1199      | 0.2917       | 0.2131    | 0.244       |
| 1.1175        | 4.0   | 43088  | 2.6382          | 0.3579       | 0.2618    | 0.2996      | 0.1488       | 0.106     | 0.1225      | 0.2956       | 0.2163    | 0.2475      |
| 1.0102        | 5.0   | 53860  | 2.6818          | 0.3574       | 0.2609    | 0.2987      | 0.1478       | 0.1052    | 0.1217      | 0.2949       | 0.2154    | 0.2466      |
| 0.9141        | 6.0   | 64632  | 2.7428          | 0.3571       | 0.2616    | 0.2992      | 0.148        | 0.1056    | 0.122       | 0.2938       | 0.2152    | 0.2461      |
| 0.8261        | 7.0   | 75404  | 2.8255          | 0.3534       | 0.2582    | 0.2956      | 0.1457       | 0.1039    | 0.12        | 0.2906       | 0.2126    | 0.2432      |
| 0.7509        | 8.0   | 86176  | 2.8975          | 0.3517       | 0.2572    | 0.2943      | 0.1428       | 0.1016    | 0.1175      | 0.289        | 0.2113    | 0.2418      |
| 0.6822        | 9.0   | 96948  | 2.9586          | 0.3557       | 0.2599    | 0.2975      | 0.1466       | 0.1043    | 0.1206      | 0.2936       | 0.2145    | 0.2455      |
| 0.6289        | 10.0  | 107720 | 3.0021          | 0.3553       | 0.2599    | 0.2974      | 0.1458       | 0.1039    | 0.12        | 0.2925       | 0.2139    | 0.2448      |


### Framework versions

- Transformers 4.39.3
- Pytorch 2.2.1
- Datasets 2.18.0
- Tokenizers 0.15.2