metadata
library_name: transformers
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- stab-gurevych-essays
metrics:
- accuracy
model-index:
- name: longformer-sep_tok_full_labels
results:
- task:
name: Token Classification
type: token-classification
dataset:
name: stab-gurevych-essays
type: stab-gurevych-essays
config: sep_tok_full_labels
split: train[0%:20%]
args: sep_tok_full_labels
metrics:
- name: Accuracy
type: accuracy
value: 0.8874031749771744
longformer-sep_tok_full_labels
This model is a fine-tuned version of allenai/longformer-base-4096 on the stab-gurevych-essays dataset. It achieves the following results on the evaluation set:
- Loss: 0.2775
- B-claim: {'precision': 0.6083333333333333, 'recall': 0.5140845070422535, 'f1-score': 0.5572519083969466, 'support': 284.0}
- B-majorclaim: {'precision': 0.88, 'recall': 0.624113475177305, 'f1-score': 0.7302904564315352, 'support': 141.0}
- B-premise: {'precision': 0.8373266078184111, 'recall': 0.9378531073446328, 'f1-score': 0.8847435043304464, 'support': 708.0}
- I-claim: {'precision': 0.6361367606688295, 'recall': 0.5500647388864911, 'f1-score': 0.5899780118041893, 'support': 4634.0}
- I-majorclaim: {'precision': 0.8413284132841329, 'recall': 0.793733681462141, 'f1-score': 0.8168383340797134, 'support': 2298.0}
- I-premise: {'precision': 0.8758342602892102, 'recall': 0.9255749026522665, 'f1-score': 0.9000178603322022, 'support': 13611.0}
- O: {'precision': 1.0, 'recall': 0.9986967500203633, 'f1-score': 0.999347950118184, 'support': 12277.0}
- Accuracy: 0.8874
- Macro avg: {'precision': 0.8112799107705595, 'recall': 0.7634458803693505, 'f1-score': 0.782638289356174, 'support': 33953.0}
- Weighted avg: {'precision': 0.8826579231427218, 'recall': 0.8874031749771744, 'f1-score': 0.8840991775809467, 'support': 33953.0}
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5
Training results
| Training Loss | Epoch | Step | Validation Loss | B-claim | B-majorclaim | B-premise | I-claim | I-majorclaim | I-premise | O | Accuracy | Macro avg | Weighted avg |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 41 | 0.4487 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 284.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.7102510460251046, 'recall': 0.9590395480225988, 'f1-score': 0.8161057692307693, 'support': 708.0} | {'precision': 0.5242566510172144, 'recall': 0.0722917565817868, 'f1-score': 0.1270623933244832, 'support': 4634.0} | {'precision': 0.635728952772074, 'recall': 0.6736292428198434, 'f1-score': 0.6541305725755335, 'support': 2298.0} | {'precision': 0.7685153090699018, 'recall': 0.9773712438468886, 'f1-score': 0.8604508262992788, 'support': 13611.0} | {'precision': 0.9707444699912788, 'recall': 0.9973120469169993, 'f1-score': 0.9838489353153878, 'support': 12277.0} | 0.8279 | {'precision': 0.5156423469822248, 'recall': 0.5256634054554452, 'f1-score': 0.49165692810649325, 'support': 33953.0} | {'precision': 0.7884799553707518, 'recall': 0.827879716078108, 'f1-score': 0.7793158674251499, 'support': 33953.0} |
| No log | 2.0 | 82 | 0.3651 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 284.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.6355475763016158, 'recall': 1.0, 'f1-score': 0.7771679473106476, 'support': 708.0} | {'precision': 0.5581831831831832, 'recall': 0.3208890807078118, 'f1-score': 0.40750890654973965, 'support': 4634.0} | {'precision': 0.888728323699422, 'recall': 0.5352480417754569, 'f1-score': 0.66811515480717, 'support': 2298.0} | {'precision': 0.8056312443233424, 'recall': 0.9775181838219088, 'f1-score': 0.8832901812387971, 'support': 13611.0} | {'precision': 0.9999184139675288, 'recall': 0.9982894844017268, 'f1-score': 0.9991032852368142, 'support': 12277.0} | 0.8537 | {'precision': 0.5554298202107274, 'recall': 0.5474206843867007, 'f1-score': 0.5335979250204527, 'support': 33953.0} | {'precision': 0.8341039518604557, 'recall': 0.8537095396577622, 'f1-score': 0.832398123732452, 'support': 33953.0} |
| No log | 3.0 | 123 | 0.2896 | {'precision': 0.47393364928909953, 'recall': 0.352112676056338, 'f1-score': 0.40404040404040403, 'support': 284.0} | {'precision': 0.9333333333333333, 'recall': 0.2978723404255319, 'f1-score': 0.45161290322580644, 'support': 141.0} | {'precision': 0.7856328392246295, 'recall': 0.9731638418079096, 'f1-score': 0.8694006309148264, 'support': 708.0} | {'precision': 0.6642079381805409, 'recall': 0.4080707811825637, 'f1-score': 0.5055473867130063, 'support': 4634.0} | {'precision': 0.7170077628793226, 'recall': 0.8842471714534378, 'f1-score': 0.7918939984411536, 'support': 2298.0} | {'precision': 0.8606260075228371, 'recall': 0.9413709499669385, 'f1-score': 0.8991894452436927, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9978822187830904, 'f1-score': 0.9989399869536856, 'support': 12277.0} | 0.8782 | {'precision': 0.7763916472042519, 'recall': 0.6935314256679729, 'f1-score': 0.7029463936475108, 'support': 33953.0} | {'precision': 0.8699976208166521, 'recall': 0.8782140017082437, 'f1-score': 0.867649200314498, 'support': 33953.0} |
| No log | 4.0 | 164 | 0.2798 | {'precision': 0.5757575757575758, 'recall': 0.5352112676056338, 'f1-score': 0.5547445255474452, 'support': 284.0} | {'precision': 0.9054054054054054, 'recall': 0.475177304964539, 'f1-score': 0.6232558139534884, 'support': 141.0} | {'precision': 0.8377358490566038, 'recall': 0.940677966101695, 'f1-score': 0.8862275449101796, 'support': 708.0} | {'precision': 0.6079838528818121, 'recall': 0.5850237375917134, 'f1-score': 0.596282854943363, 'support': 4634.0} | {'precision': 0.8411037107516651, 'recall': 0.7693646649260226, 'f1-score': 0.8036363636363636, 'support': 2298.0} | {'precision': 0.8822353864820498, 'recall': 0.9081625156123724, 'f1-score': 0.8950112229382376, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9976378594119084, 'f1-score': 0.9988175331294598, 'support': 12277.0} | 0.8828 | {'precision': 0.8071745400478731, 'recall': 0.7444650451734122, 'f1-score': 0.7654251227226482, 'support': 33953.0} | {'precision': 0.881207953399647, 'recall': 0.882779135864283, 'f1-score': 0.8814327847290655, 'support': 33953.0} |
| No log | 5.0 | 205 | 0.2775 | {'precision': 0.6083333333333333, 'recall': 0.5140845070422535, 'f1-score': 0.5572519083969466, 'support': 284.0} | {'precision': 0.88, 'recall': 0.624113475177305, 'f1-score': 0.7302904564315352, 'support': 141.0} | {'precision': 0.8373266078184111, 'recall': 0.9378531073446328, 'f1-score': 0.8847435043304464, 'support': 708.0} | {'precision': 0.6361367606688295, 'recall': 0.5500647388864911, 'f1-score': 0.5899780118041893, 'support': 4634.0} | {'precision': 0.8413284132841329, 'recall': 0.793733681462141, 'f1-score': 0.8168383340797134, 'support': 2298.0} | {'precision': 0.8758342602892102, 'recall': 0.9255749026522665, 'f1-score': 0.9000178603322022, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9986967500203633, 'f1-score': 0.999347950118184, 'support': 12277.0} | 0.8874 | {'precision': 0.8112799107705595, 'recall': 0.7634458803693505, 'f1-score': 0.782638289356174, 'support': 33953.0} | {'precision': 0.8826579231427218, 'recall': 0.8874031749771744, 'f1-score': 0.8840991775809467, 'support': 33953.0} |
Framework versions
- Transformers 4.45.2
- Pytorch 2.5.0+cu124
- Datasets 2.19.1
- Tokenizers 0.20.1