File size: 4,372 Bytes
05c3d30 da8c5b2 05c3d30 da8c5b2 c2100d7 678a8a5 c2100d7 da8c5b2 c2100d7 da8c5b2 9a7a861 da8c5b2 9a7a861 da8c5b2 9a7a861 da8c5b2 9a7a861 da8c5b2 9a7a861 da8c5b2 9a7a861 da8c5b2 9a7a861 da8c5b2 9a7a861 da8c5b2 05c3d30 c2100d7 b63d8fa 71feb83 b63d8fa c2100d7 b63d8fa c2100d7 c33f650 c2100d7 b63d8fa c2100d7 b63d8fa c2100d7 678a8a5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 |
---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- generated_from_trainer
- stacked summaries
- xsum
datasets:
- stacked-summaries/stacked-xsum-1024
pipeline_tag: summarization
model-index:
- name: flan-t5-large-stacked-XSUM-1024-WIP-2p8-850-stacked-xsum-1024-evaluated
results:
- task:
type: summarization
name: Summarization
dataset:
name: xsum
type: xsum
config: default
split: test
metrics:
- type: rouge
value: 39.3614
name: ROUGE-1
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWZmZDNhNWM5YjcyMzVjNjUwMWE1NDg4YmRiNGMwY2EyZDYzMGZkY2NlNWE0MzQwNDYzN2JkNzYyOGUxNmI3ZiIsInZlcnNpb24iOjF9.1ucBm8VOqZgLXmUyDkPisiFfHJ8VYvOdvUsk6R_F0QGLIBXOCf2s_pbqHauTyEQM2mAn762DpR5L4AZg7hF_BA
- type: rouge
value: 17.5887
name: ROUGE-2
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDU3MDQwNjYzMTE2MjU5NTE0ODU1ZmI2ZjhlY2QxODA3YTYyOWExZDdiM2Y4YzZhMTU3N2IwMGQ4M2MxMTNmZiIsInZlcnNpb24iOjF9.lb6R_xg5R1TABUCSRgvEGmdkxhSRavrfllxhsk_NxKA53EC4MXeE6o7nRWPoo2nrBOb5Lcajy_5y4oPOkv84Ag
- type: rouge
value: 32.6489
name: ROUGE-L
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmFkOTc2MTIxMmYyNTY2MWE3Y2E4ZWYwODQ5MmU3NTIxZWM2Yzg2ZDNkYjE3NDgzM2VjYTMwOTkxNjQ1YmIyYiIsInZlcnNpb24iOjF9.AAAh5SnRDnTMCEXMfEp9N7pwHITv-crNloZTnbW7TMPXtMUe7vzATOxGVMZpMe-Nsf3Wkc3JbUdaZZ9bOb17Ag
- type: rouge
value: 32.6435
name: ROUGE-LSUM
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjg1ZmNkODZlMzdkODA4MDUxMGQyNjFiMTkyYjIzMTE2NGMyOWQ1NmQ2YjY0OTRmZjVjZWNhODBiOWI1YzVlOCIsInZlcnNpb24iOjF9.GUVl2J3DCRQUqueSuCsFM8v7IDXH7EATFlQbFl730Bo8Y2aolA-V9uN7pkaU9IM1wWBz7hvILElBCE0sln6SAQ
- type: loss
value: 1.4964560270309448
name: loss
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTViZTkwMzQ3MGNlZDJhNTk3NDE5NzBkMDZjMGEyNzNkZTI4ZmJhMWRlYTMwNmRmN2JhNzdkNTQ3N2FlODBmNyIsInZlcnNpb24iOjF9.lNWUw12R20SwZMZEuUnxYsWrkFBNoU9_5ZOiuFF5aT9QsHJC-FSmZ8DXTdVudv6J-BoeA-l5KYowr7GJfbzlDQ
- type: gen_len
value: 18.7302
name: gen_len
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWM2MWQzN2YyY2U0NWZhZGFkMjk0MzFlMTA1YTgxYzAzYjhhZmVmZDI5Mzk4ODgzOGU1NjVhNTk3NmYyNzhkMSIsInZlcnNpb24iOjF9.bL56u1G46OIwdIqZJ-6og_T2yCKFTXrlPQeguZps3ixXokfKqlfCDxz3641yKA3AdMlLe5lDcN3UQReHtiWwBg
---
# flan-t5-large-stacked-XSUM-1024
<a href="https://colab.research.google.com/gist/pszemraj/561263b04b33d5aec04a18f572d68011/brief-demo-flan-t5-stacked-xsum.ipynb">
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
</a>
This model is a fine-tuned version of [google/flan-t5-large](https://huggingface.co/google/flan-t5-large) on the stacked-summaries/stacked-xsum-1024 dataset.
It achieves the following results on the evaluation set:
- eval_loss: 1.3314
- eval_rouge1: 46.5061
- eval_rouge2: 22.0588
- eval_rougeL: 37.5235
- eval_rougeLsum: 39.0234
- eval_gen_len: 46.1807
- eval_runtime: 9456.3608
- eval_samples_per_second: 1.896
- eval_steps_per_second: 0.119
> Note that the evaluation set is `stacked-summaries/stacked-xsum-1024` and not `xsum` itself
## Model description
This model card presents a model trained on a stacked dataset that aims to improve summarization by testing the benefits of "task-oriented pretraining". The model is designed to learn how to effectively condense and distill information from text by stacking summaries and separating them into independent concepts. In this way, the model can learn to identify essential information without simply mimicking the style of the dataset summaries.
The token used to identify a new concept in the summary is `[NEXT_CONCEPT]`. You can split an output summary based on this token to see how it split the input text information: `summary_text.split("[NEXT_CONCEPT]")` etc.
## Intended uses & limitations
- max input length (in tokens): 1024
## Training and evaluation data
Refer to `stacked-summaries/stacked-xsum-1024`
Trained for approx 3 epochs before ROUGE scores stabilized on most recent run:
### scores
### gradients
 |