--- license: apache-2.0 base_model: facebook/bart-base tags: - generated_from_trainer model-index: - name: pubmed-abs-sub-03 results: [] --- # pubmed-abs-sub-03 This model is a fine-tuned version of [facebook/bart-base](https://huggingface.co/facebook/bart-base) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.1916 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 10 - num_epochs: 3 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:-----:|:---------------:| | 0.5622 | 0.11 | 500 | 0.4754 | | 0.4051 | 0.21 | 1000 | 0.3731 | | 0.3978 | 0.32 | 1500 | 0.3332 | | 0.3669 | 0.43 | 2000 | 0.3047 | | 0.3059 | 0.54 | 2500 | 0.2824 | | 0.294 | 0.64 | 3000 | 0.2632 | | 0.2774 | 0.75 | 3500 | 0.2579 | | 0.3134 | 0.86 | 4000 | 0.2408 | | 0.2865 | 0.96 | 4500 | 0.2345 | | 0.2103 | 1.07 | 5000 | 0.2320 | | 0.1974 | 1.18 | 5500 | 0.2289 | | 0.2026 | 1.28 | 6000 | 0.2303 | | 0.1931 | 1.39 | 6500 | 0.2155 | | 0.2027 | 1.5 | 7000 | 0.2167 | | 0.1977 | 1.61 | 7500 | 0.2120 | | 0.1836 | 1.71 | 8000 | 0.2100 | | 0.2204 | 1.82 | 8500 | 0.2043 | | 0.1731 | 1.93 | 9000 | 0.2009 | | 0.1479 | 2.03 | 9500 | 0.2004 | | 0.1277 | 2.14 | 10000 | 0.2040 | | 0.136 | 2.25 | 10500 | 0.2022 | | 0.1577 | 2.35 | 11000 | 0.1986 | | 0.1305 | 2.46 | 11500 | 0.1963 | | 0.159 | 2.57 | 12000 | 0.1936 | | 0.1603 | 2.68 | 12500 | 0.1939 | | 0.1195 | 2.78 | 13000 | 0.1926 | | 0.1203 | 2.89 | 13500 | 0.1922 | | 0.1437 | 3.0 | 14000 | 0.1916 | ### Framework versions - Transformers 4.33.3 - Pytorch 2.0.1 - Datasets 2.14.5 - Tokenizers 0.13.3