Introduction
A led-base-16384 model to summarize ArXiv papers. Inputs are the abstracts of papers and full documents, and outputs are the summaries of the papers.
Allenai's Longformer Encoder-Decoder (LED).
As described in Longformer: The Long-Document Transformer by Iz Beltagy, Matthew E. Peters, Arman Cohan, led-base-16384 was initialized from bart-base since both models share the exact same architecture. To be able to process 16K tokens, bart-base's position embedding matrix was simply copied 16 times.
Rouge 2
| Type | Score |
|---|---|
precision |
0.1839148953011932 |
recall |
0.14904707945189774 |
fmeasure |
0.1580026685776864 |
- Downloads last month
- 10
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Evaluation results
- ROUGE-1 on ccdv/arxiv-summarizationtest set verified37.325
- ROUGE-2 on ccdv/arxiv-summarizationtest set verified10.895
- ROUGE-L on ccdv/arxiv-summarizationtest set verified20.387
- ROUGE-LSUM on ccdv/arxiv-summarizationtest set verified33.301
- loss on ccdv/arxiv-summarizationtest set verified3.182
- gen_len on ccdv/arxiv-summarizationtest set verified145.590