zwellington
/

bart-pubhealth-expanded-hi-grad

+---
+license: apache-2.0
+base_model: facebook/bart-large
+tags:
+- generated_from_trainer
+datasets:
+- clupubhealth
+metrics:
+- rouge
+model-index:
+- name: bart-pubhealth-expanded-hi-grad
+  results:
+  - task:
+      name: Sequence-to-sequence Language Modeling
+      type: text2text-generation
+    dataset:
+      name: clupubhealth
+      type: clupubhealth
+      config: expanded
+      split: test
+      args: expanded
+    metrics:
+    - name: Rouge1
+      type: rouge
+      value: 30.2592
+---
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+# bart-pubhealth-expanded-hi-grad
+This model is a fine-tuned version of [facebook/bart-large](https://huggingface.co/facebook/bart-large) on the clupubhealth dataset.
+It achieves the following results on the evaluation set:
+- Loss: 2.0581
+- Rouge1: 30.2592
+- Rouge2: 11.7027
+- Rougel: 24.1706
+- Rougelsum: 24.3596
+- Gen Len: 19.95
+## Model description
+More information needed
+## Intended uses & limitations
+More information needed
+## Training and evaluation data
+More information needed
+## Training procedure
+### Training hyperparameters
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- gradient_accumulation_steps: 950
+- total_train_batch_size: 15200
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 10
+### Training results
+| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
+|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
+| 3.7893        | 0.49  | 2    | 2.3943          | 20.5187 | 5.4764  | 15.9378 | 16.2797   | 20.0    |
+| 3.4045        | 0.98  | 4    | 2.1599          | 24.0858 | 7.8207  | 19.0412 | 19.1609   | 19.88   |
+| 3.2488        | 1.47  | 6    | 2.1026          | 27.3466 | 9.369   | 21.1419 | 21.3136   | 19.865  |
+| 3.1823        | 1.96  | 8    | 2.1324          | 28.825  | 9.6007  | 22.0963 | 22.3776   | 19.82   |
+| 3.1263        | 2.44  | 10   | 2.1105          | 29.2694 | 10.5001 | 23.2842 | 23.5473   | 19.85   |
+| 3.0834        | 2.93  | 12   | 2.0837          | 28.5975 | 10.2016 | 22.048  | 22.1341   | 19.915  |
+| 3.0283        | 3.42  | 14   | 2.0773          | 28.5813 | 10.447  | 22.7456 | 22.8496   | 19.91   |
+| 3.0301        | 3.91  | 16   | 2.0730          | 30.1049 | 11.4375 | 24.083  | 24.3045   | 19.945  |
+| 2.9851        | 4.4   | 18   | 2.0775          | 29.2224 | 10.2722 | 22.7019 | 23.0038   | 19.95   |
+| 2.9769        | 4.89  | 20   | 2.0777          | 29.6981 | 10.7044 | 23.2487 | 23.5232   | 19.96   |
+| 2.9623        | 5.38  | 22   | 2.0711          | 29.0438 | 10.5105 | 23.1751 | 23.415    | 19.92   |
+| 2.9421        | 5.87  | 24   | 2.0676          | 29.096  | 10.6599 | 23.1381 | 23.3765   | 19.985  |
+| 2.9234        | 6.36  | 26   | 2.0646          | 29.6561 | 10.9096 | 23.2384 | 23.4265   | 19.985  |
+| 2.9107        | 6.85  | 28   | 2.0616          | 29.7134 | 11.1686 | 23.272  | 23.4475   | 19.985  |
+| 2.9077        | 7.33  | 30   | 2.0593          | 29.5055 | 11.0256 | 23.4406 | 23.6653   | 19.955  |
+| 2.9072        | 7.82  | 32   | 2.0585          | 30.0504 | 11.433  | 23.9176 | 24.1728   | 19.95   |
+| 2.8951        | 8.31  | 34   | 2.0583          | 29.9401 | 11.602  | 23.948  | 24.1323   | 19.95   |
+| 2.8955        | 8.8   | 36   | 2.0584          | 30.1158 | 11.4745 | 24.0509 | 24.2465   | 19.94   |
+| 2.8774        | 9.29  | 38   | 2.0582          | 30.0476 | 11.4465 | 23.8956 | 24.0527   | 19.945  |
+| 2.8851        | 9.78  | 40   | 2.0581          | 30.2592 | 11.7027 | 24.1706 | 24.3596   | 19.95   |
+### Framework versions
+- Transformers 4.31.0
+- Pytorch 2.0.1+cu117
+- Datasets 2.7.1
+- Tokenizers 0.13.2