longformer-simple
This model is a fine-tuned version of allenai/longformer-base-4096 on the stab-gurevych-essays dataset. It achieves the following results on the evaluation set:
- Loss: 0.3326
- Claim: {'precision': 0.6375421311900441, 'recall': 0.6000488042947779, 'f1-score': 0.6182275298554368, 'support': 4098.0}
- Majorclaim: {'precision': 0.8534005037783375, 'recall': 0.7853500231803431, 'f1-score': 0.8179623370352487, 'support': 2157.0}
- O: {'precision': 0.9584632404706829, 'recall': 0.9674144756877474, 'f1-score': 0.9629180559765586, 'support': 9851.0}
- Premise: {'precision': 0.884906500445236, 'recall': 0.9064994298745724, 'f1-score': 0.8955728286583305, 'support': 13155.0}
- Accuracy: 0.8752
- Macro avg: {'precision': 0.8335780939710751, 'recall': 0.8148281832593602, 'f1-score': 0.8236701878813937, 'support': 29261.0}
- Weighted avg: {'precision': 0.8727042457708365, 'recall': 0.8751580602166706, 'f1-score': 0.8736819489681839, 'support': 29261.0}
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5
Training results
Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
---|---|---|---|---|---|---|---|---|---|---|
No log | 1.0 | 41 | 0.5843 | {'precision': 0.48984771573604063, 'recall': 0.14128843338213762, 'f1-score': 0.21931818181818183, 'support': 4098.0} | {'precision': 0.5396243701328447, 'recall': 0.5461288827074641, 'f1-score': 0.5428571428571428, 'support': 2157.0} | {'precision': 0.8916069169126951, 'recall': 0.858390011166379, 'f1-score': 0.8746832169640548, 'support': 9851.0} | {'precision': 0.7743724104313917, 'recall': 0.9660965412390726, 'f1-score': 0.8596746372645179, 'support': 13155.0} | 0.7834 | {'precision': 0.673862853303243, 'recall': 0.6279759671237634, 'f1-score': 0.6241332947259743, 'support': 29261.0} | {'precision': 0.7566882370115429, 'recall': 0.7833635214107515, 'f1-score': 0.7516910901801511, 'support': 29261.0} |
No log | 2.0 | 82 | 0.4171 | {'precision': 0.5352343493936415, 'recall': 0.39848706686188384, 'f1-score': 0.45684711148412366, 'support': 4098.0} | {'precision': 0.8575712143928036, 'recall': 0.5303662494204914, 'f1-score': 0.6553995989687769, 'support': 2157.0} | {'precision': 0.9516030844155844, 'recall': 0.952086082631205, 'f1-score': 0.9518445222509768, 'support': 9851.0} | {'precision': 0.8249001331557922, 'recall': 0.9418472063854048, 'f1-score': 0.8795031055900621, 'support': 13155.0} | 0.8389 | {'precision': 0.7923271953394554, 'recall': 0.7056966513247462, 'f1-score': 0.7358985845734849, 'support': 29261.0} | {'precision': 0.8293966272342979, 'recall': 0.8388640169508903, 'f1-score': 0.8281446341741303, 'support': 29261.0} |
No log | 3.0 | 123 | 0.3525 | {'precision': 0.6357409713574097, 'recall': 0.49829184968277207, 'f1-score': 0.5586867305061559, 'support': 4098.0} | {'precision': 0.7525641025641026, 'recall': 0.8164116828929068, 'f1-score': 0.7831887925283523, 'support': 2157.0} | {'precision': 0.9471273523847455, 'recall': 0.9655872500253782, 'f1-score': 0.956268221574344, 'support': 9851.0} | {'precision': 0.8749451192741109, 'recall': 0.9089319650323071, 'f1-score': 0.8916147794638529, 'support': 13155.0} | 0.8637 | {'precision': 0.8025943863950922, 'recall': 0.797305686908341, 'f1-score': 0.7974396310181762, 'support': 29261.0} | {'precision': 0.8567240306977373, 'recall': 0.863675199070435, 'f1-score': 0.8587617347894375, 'support': 29261.0} |
No log | 4.0 | 164 | 0.3385 | {'precision': 0.6185015290519877, 'recall': 0.5922401171303074, 'f1-score': 0.6050860134629769, 'support': 4098.0} | {'precision': 0.7913082842915347, 'recall': 0.8103847936949466, 'f1-score': 0.8007329363261567, 'support': 2157.0} | {'precision': 0.9529177057356608, 'recall': 0.9697492640341082, 'f1-score': 0.9612598108271282, 'support': 9851.0} | {'precision': 0.8938411050904373, 'recall': 0.8903078677309008, 'f1-score': 0.8920709878894051, 'support': 13155.0} | 0.8694 | {'precision': 0.8141421560424051, 'recall': 0.8156705106475658, 'f1-score': 0.8147874371264168, 'support': 29261.0} | {'precision': 0.8676102420265399, 'recall': 0.8694166296435528, 'f1-score': 0.8684387980236479, 'support': 29261.0} |
No log | 5.0 | 205 | 0.3326 | {'precision': 0.6375421311900441, 'recall': 0.6000488042947779, 'f1-score': 0.6182275298554368, 'support': 4098.0} | {'precision': 0.8534005037783375, 'recall': 0.7853500231803431, 'f1-score': 0.8179623370352487, 'support': 2157.0} | {'precision': 0.9584632404706829, 'recall': 0.9674144756877474, 'f1-score': 0.9629180559765586, 'support': 9851.0} | {'precision': 0.884906500445236, 'recall': 0.9064994298745724, 'f1-score': 0.8955728286583305, 'support': 13155.0} | 0.8752 | {'precision': 0.8335780939710751, 'recall': 0.8148281832593602, 'f1-score': 0.8236701878813937, 'support': 29261.0} | {'precision': 0.8727042457708365, 'recall': 0.8751580602166706, 'f1-score': 0.8736819489681839, 'support': 29261.0} |
Framework versions
- Transformers 4.45.2
- Pytorch 2.5.0+cu124
- Datasets 2.19.1
- Tokenizers 0.20.1
- Downloads last month
- 25
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
the model is not deployed on the HF Inference API.
Model tree for Theoreticallyhugo/longformer-simple
Base model
allenai/longformer-base-4096