DEIM Forum 2023 1b-5-2

Efficient Summarization of Long Documents Using Hybrid
Extractive-Abstractive Method
Weichao CHEN†

Mizuho IWAIHARA‡

Graduate School of Information, Production and Systems, Waseda University
2-7 Hibikino, Wakamatsu-ku, Kitakyushu, Fukuoka, 808-0135 Japan
E-mail: †vico_chen@akane.waseda.jp, ‡iwaihara@waseda.jp
Abstract Automatic summarizers based on pre-trained language models (PLMs) have achieved great success in the fieldof short text summarization. However, the complexity of the self-attention mechanism on which PLMs rely grows
quadratically with the input length, thus limiting its application to long document summarization. As a solution, we investigate
an efficient hybrid long document summarization method that enables compression and summarization of original long
documents by combining content selection and abstractive summarization. Long documents will first be extracted for salient
sentences and reorganized into compressed documents, the length of which is within the limits of the PLMs, and finallygenerate the final summary by abstractive summarization. Experiments show that the proposed method can achieve goodsummarization performance for general long documents in single-GPU or low-resource scenarios.
Keyword Natural language processing， Long document summarization，Pretrained language models
these models. In most cases we simply truncate long

1. Introduction
A summary is a concise and comprehensive version of

documents to a limited length for input into these models.

the original document. A summary can help readersThis approach is usually effective, but obviously will lose

quickly understand the main content of the whole article .

semantic information of the truncated part, so the results

The task of generating a summary from the originalobtained by this approach are not comprehensive [9].

document is called text summarization , which can be

Existing research has focused on adapting PLMs to long

abstractive

documents. The main long document adaptation methods

summarizations [1]. Extractive summarization methods,

can be broadly divided into two categories: Efficientsuch as text ranking and clustering, work by selecting andself-attention mechanisms and hybrid summarization.

further

dividedinto

extractive

andcompiling parts of the original text, while abstractive

Efficient self-attention. In terms of reason analysis,summarization generates a new summary that captures the

pre-trained language models cannot be applied to longmain ideas and concepts in the original tex t, where

documents mainly because of the computational c ost of

passages not appearing in the original document may befull self-attention in the transformer structure. The mostgenerated. Abstractive summarization has the advantage

straightforward solution is to improve the self -attentionof smoothness and fluency, while keeping the content

mechanism to make it more efficient. Currently, the mostinformation, so our discussion will mainly focus on

popular approach is to replace the full attention with

abstractive summarization [6].

sparse attention mechanisms such as local attention anduse

sliding windows, which allows to focus on specific parts

Theseof the input while also reducing computational complexity.

algorithms have had great success in the field of short text

Models that use this approach include Longformer [3] andsummarization. However, the self-attention mechanism of

BigBird [14]. For example, Longformer incorporates

transformer-based models has a quadratic level of space

sliding window attention, dilated sliding windows, andcomplexity,

theglobal + sliding windows, which enables the processing ofcomputational consumption of these models as the number

long documents with more than 4096 token s in length.

of input words increases. So, usually these models have a

These approaches have shown reasonable performance on

limit on the number of input tokens , such as 1024 tokens

long document summarization tasks.

Typicalabstractive

seq2seq-basedmodelsleadingtosummarization
such

a

as

algorithms

transformers.

dramatic

increaseinlimit in BART [10]. For relatively long documents, such as

Hybridsummarization(content

selection+press releases, scientific literature, specialized reports,

abstractive summarization). Hybrid summarization isthey often have lengths that far exceed these limits,

anothermaking it impossible to use the entir e content as input ofsummarization

importantthatapproach

for

long

combines

content

document

selectionandabstractive summarization. This approach does not change

we can conclude that this model has good summarization

the structure of the original pre-trained language model

results for both general long documents and extreme long

(PLM); instead, it solves the long document problem by

documents. Compared to BigBird and Longformer, whichchanging the input of the PLMs. In this approach, lo ng

rely on GPU clusters, our model effect has a 1.5 ROUGE

documents are first processed through a content selection

scores improvement and outperforms the other models inprocess,

BERTScore.

where

the

most

beneficial

sentencesorparagraphs for generating a summary are selected andImpressively,

it

has

low

resource

consumption for processing documents of any length.

reorganized into a new, compressed document that is longenough for downstream summarization w ithin the limits of

2. Related Workthe PLMs. For example, Bajaj et al. [2] investigated aLong document summarization using hybrid models iscombined BERT [4] and BART model for long document

an emerging research topic, which is highly related tosummarization, using BERT as a salient sentence classifier

abstractive summarization and extractive content selection

to filter the source document sentences and BART as amethods such as sentence similarity calculation.

summarizer

for

the

summarization

task.

The

main

advantage of this approach is that it p reserves thesemantic information while being efficient.
Despitetherecenteffortson

2.1 Bi-Encoder & Cross-Encoder & SimCSE
In content selection using extractive models, sentence

efficientattention

pair similarity calculation is a popular method. Traditional

mechanisms, there are still potential limitations to their

methods include Tf-Idf and TextRank, but more recent

use in summarizing long documents. For example, thedeep learning-based methods such as Bi-Encoder andreduced complexity of these mechanisms may come at the

Cross-Encoder tend to be more accurate for similarity

cost of summary quality to the original document.

calculation.

Additionally, models using efficient attention mechanisms,
such as Longformer and Big Bird, may still struggle withhigh resource consumption and difficulty in capturing theoverall context of the document. In contrast, the hybrid
model is much lighter and more reliable. An example isthe two-stage model, CogLTX [5] proposed by Ding et al.
whichachieves

SOTA

resultsonlong

document

classification and Q&A tasks by using key sentence
selection and downstream models. This approach can beapplied not only to the above domains but also provide aparadigm for long document summarization.
Based on the successful precedent of CogLTX for long
document classification and Q&A tasks, in this paper, we
further

exploreits

application

summarization.

Our

extension

to
of

long

document

CogLTX

tolong

document summarization can be summarized in 3 parts —(1)

weutilize

BART

toprocessthedownstreamsummarization task in place of BERT;(2) a compression mechanism based on the combination ofsimilarity scoring and latent intervention is introduced;
(3)

more

initialization

methods

specifictothesummarization task are utilized in the model.
In summary, the main contribution of this paper is to
propose a hybrid summarization model, which combines
extractive and abstractive summarization s by conducting
co-training of the content selection model BERT and theabstractive summarization model BART. From the results,

Figure 1. Structures of Bi-Encoder and Cross-Encoder
Bi-Encoder is a simple encoder architecture using
BERT with separate encoders but shared parameters for
input

and

combines

outputtherepresentation.

sequences,while

sequences

into

The

difference

keyaCross -Encoder

singlesequence

betweenthese

architectures is how the pretrained language model isfinetuned, which can affect model performance andproperties [13].
SimCSE [7] is particularly effective for learningsentence embeddings that capture semantic similarity, as
demonstrated by its SOTA performance on a variety ofbenchmark tasks, including semantic textual similarity,
natural language inference, and text classification. One of
the advantages of SimCSE is that it does not require anylabeled data and can be trained on large amounts ofunlabeled text. It also has a simple architecture that can betrained efficiently on a single GPU.

Figure 2. Overall structure of our proposed model
2.2 BART

multi-step reasoning. The second one is the joint training

BART (Bidirectional Encoder Representations fromof two BERTs, that is why this model has two -stage

Transformers) is a sequence -to-sequence language model

processing. In summary, this model has shown outstandingthat has been used for tasks such as text generation,

performance on classification and Q&A tasks over longtranslation, and summarization. It is a transformer -based

documents, that inspire us to think about the possibilities

model that uses attention mechanisms to process longon long document summarization task.

input sequences and generate output sequences. It istrained on large amounts of data using an auto -regressive
language modeling objective.

3. Methodology
Figure 2 shows the overall structure of our proposed

The architecture of BART is a combination of BERT asmodel. It is important to emphasize that our mode l

the encoder and GPT [12] as the decoder which allows

discards the notion of "natural guidance" in CogLTX

capturing

contextual

because there is no natural prompt for summarization task.

information effectively. This has been shown by its

We use a combination of similarity scoring and latentstate-of-the-art results on a variety of NLP tasks such as

intervention to implement an unsupervised process. Morequestion answering, and language understanding. BART is

importantly, we have introduced B ART and utilized a newhighly flexible and can be fine-tuned for different tasks

compression mechanism to expand the compress ion size toand languages.

1024. The main process of the model is as follows.

2.3 CogLTX

dynamically split into blocks that represent the minimum

long-range

dependencies

and

For the training process, long documents are first
CogLTX [5] is a two-stage model that partitions long

granularity

ofcompression.

These

blocksarethendocuments into blocks, scores each block with a judge

initialized to determine the initial relevan ce. The positivemodel, and then combines the high -scoring blocks forand negative sample blocks based on the relevance aretraining downstream tasks. The model utilizes two criticalused as training data to train the BERT scorer, so that wemechanisms, one is MemRecall, which works by splitting

can obtain the score of each block of the original long

documents into blocks, connecting each block with

document. Based on these scores, it is then possible toavailable key information, and obtaining the averagedetermine which blocks in the source document are thesimilarity score of the blocks by a scoring machine, thenmost promising, and the highest scoring blocks are"forgetting" the blocks that did not score well, andselected and reorganized in their order in the original

repeatingdocument to produce compressed do cuments that aretheprocess

multiple

times

to

completeshorter than 1024 -- suitable to be processed directly by

Bi-Encoder and SimCSE methods have the advantage of

BART. The compressed documents are then used as input

generalizing

tonormalmaking them better suited for sentence pair similarity

abstractive summarization training. Finally, through a

calculation. In our proposed model, we use each of them to"latent intervention" method, the blocks of the compressedcompute the similarity between each block of a long

document are again checked to confirm that they are

document and the corresponding summary. The blocks

useful for generating summa ries and reducing the loss ofwith the highest similarity are considered as "relevant"

BART summarizer. The results of the latent intervention

while the rest are "irrelevant" by default.

thedownstreamsummarization

task

for

semantic

information

more

accurately,are used as feedback for the next epoch training.
For the inference process, the long document is

3.3 BERT and BART training

similarly dynamically split into blocks. These blocks are

After obtaining the initialized positive and negativescored by BERT through a dynamic scoring method. The

sample blocks, we can construct the dataset used to train

top-K blocks with the highest scores are selected and

BERT. Here BERT serves as a classifier to score thereorganized into compressed documents in their order inblocks. To ensure the integrity of the learning content, we

the document. The compressed document is passed to theuse a dataset that consists of short spans randomly

BART summarizer to produce the final summary.

extracted from long documents, along with other spansmade up of positive and negative samples.

3.1 Dynamic splitting

After collecting BERT scores for each block of the long

Dynamic splitting is a method for dividing long

document, the highest scoring blocks are collected,

documents with smaller granularity. The length limit 𝐿𝑏

reorganized in their original order within the lon gof the block determines the minimum granularity of the

document, which is referred to as document compression

model input; too small a block may cause semantic

in this paper and represents the end of the first stage. The

information to be fragmented and scattered, while too

length of the compressed document is less than thelarge a block may cause detailed semantic information to

maximum length of 1024 limited by BART, but it contains

be lost. This method is based on the cost of text splitting.

the best information of the original document, so it can be

For a long document, the document is first segmented

used to perform the summarization task on behalf of thebased on punctuation, and if a piece of text exceeds 𝐿𝑏 , itoriginal document. The second stage is fine -tuning BART

is truncated and segmented, resulting in a significant cost.

using the compressed document set.

Initial segmentation has variable block lengths, a better
approach is to dynamically aggregate the smaller blocks

3.4 Latent intervention

again, ensuring that there is the smallest possible sum ofBlocks that are initially marked as "relevant" do notcosts without the aggregated blocks exceeding 𝐿𝑏 inalways maintain their status, in fact, they can be pulled

length. In this way we can obtain the most comprehensive

down after each training epoch. This "rechecking" process

information about the text block.

is called "latent intervention". Latent intervention works
according to the contribution of the block to the generated

3.2 Initialization

summary. Specifically, if a block in a compressed

Initialization represents the pre-processing of blocks
split

from

a

long

document

with

the

purposedocument is discarded causing an increase in the loss ofofthe BART summarizer, this means that we better not lose it

determining the relevance of the blocks before the training

and it is relevant. Conversely, if a block in a compressed

process begins. The relevance reflects the importance of

document is discarded causing a decrease in the loss of thethe block in the long document and represents the degree

BART summarizer, this means that the block played a badof influence of this block on the summary's generation.

role in the generation of the summary, and it is probably

The simplest and most efficient way to determine theirrelevant. This allows us to reposition the relevance of

relevance of a block is by computing the semantic

these blocks based on the results of the latent intervention

similarity between the target summary and each block.

in order to correct our initial judgments and better help the

Unlike traditional word frequency -based methods suchasTf-Idf,thedeeplearning-based

Cross-Encoder,training of the next epoch. This idea is known as
unsupervised training in CogLTX.

Figure 3. The inference process of our proposed model.
3.5 InferenceWith a trained BERT compressor and BART summarizer,
long documents can be applied the intended compression
process and summarization process after being split into
blocks. As shown in Figure 3, the compression process isTable 1. A comparison of Pubmed, arXiv and GovReport
datasets. #(𝐷, 𝑆) means the number of document-summary
pairs, Avg.|𝐷| and Avg.|𝑇| represents the average length of
documents and summaries in each dataset.

based on a multi-layer dynamic compression mechanism.

Dataset#(𝑫, 𝑺)

Avg.|𝑫|

Avg.|𝑺|Specifically, the number of blocks contained in the

Pubmed1332153000

215

compressed document is limited to a maximum of 𝑁𝑏

arXiv

2159136000

300

due to the limitation of the BART input, where 𝑁𝑏 also

GovReport

19463

9000

500

represents the size of the working memory. It is wellknown that in the human brain, the data in working
memory can be cleared, updated, or added at any time.

PubMed is a database of medical literature, including

Multi-layer dynamic compression is based on such aabstracts and full-text articles from a wide range ofprinciple. The compressed document is filled step by step

medical journals. The articles in this dataset may bethrough multiple memory iteration cycles, each of wh ich

suitable for long document summarization in the field of

dynamically updates and retains a certain number of the

medicine. arXiv contains electronic preprints of scientificmost valuable memories based on the prediction scores.

papers in a variety of fields, including mathematics,

Only three cycles are demonstrated in the figure, with thecomputer science, physics, biology, finance, and statistics.

number of memory blocks retained in each cycle being 3,

These papers can be quite long and suitable for long

3, and 10, corresponding to the blue, red, and green blocks

document summarization in the field of scientific research.

in the figure, respectively.

GovReport [8] is a database of government reports,

The abstractive summarization process is relatively

including reports from federal agencies, congressional

simple, and the length of the compressed document

committees, and independent organizations. These reports

obtained by multi-layer dynamic compression is kept

can vary in length and suitable for long document

within the maximum limit of BART, which can directly

summarization in the field of government research andgenerate a target summary corresponding to the original

policy. In summary, these three datasets have their own

long document. This is shown in the right part of Figure 3.

specific domains. They are also different in terms of thelength that we care most about.

4. Datasets and Experiments
A good long document summarization model should be
adaptable to documents of various lengths. In order tofully evaluate the model's summarization performance, we
selected datasets of differen t length classes for training.

4.1 Evaluation Metrics
ROUGE(Recall-Oriented

Understudy

forGisting

Evaluation) is a widely used evaluation me tric forsummarization

tasks.

It

comparesanautomaticallygenerated summary to a reference summary and calculates
the overlap between the two. There are several variations

4.3 Experiment Settingsof ROUGE, including ROUGE-N, which measures the

The experiments were implemented on a standalone

overlap between the summary and the reference in terms

environment with a GeForce RTX 3090. Block size 𝐿𝑏 isof n-grams (sequences of n words), and ROUGE-L, which

set to 64 and the corresponding number 𝑁𝑏 of blocks inmeasures the longest common subsequence between theworking memory is 16. For SimCSE initialization, we didsummary and the reference. ROUGE scores are typicallyunsupervised

used to evaluate the performance of summarization

domain-specific datasets to extract senten ces as training

systems and to compare the quality of different summaries.

data. We chose RoBERTa, a variant of BERT, as the model

However, ROUGE scores are calculated based on word

for the compressor. The batch size of the compressor was

frequencies, and it has its limitations in reflecting the true

set to 32 and the learning rate was set to 2e -5. Since thesemantic content and information coherence. Therefore,

training dataset of RoBERTa needed to be sufficiently

we also utilize BERTScore [16] which can better compare

large to contain every block in the original long document,

and evaluate summaries at the semantic level.

we used data enhancement by sampling the same document

trainingof

SimCSEbyselecting

BERTScore is a metric for evaluating the quality of text

multiple times for the training data. The number ofgeneration tasks, including summarization. It is based on

samples was set to 5. The batch size of the summarizer

the BERT language model and measures the similarity

BART was set to 2 and the learning rate was set at a

between the generated text and the reference text using a

relatively small 1e-5. In the latent intervention stage, thecombination of precision and recall. BERTScore is

relevance of text blocks was rechecked based on adesigned to be less sensitive to the length of the generated

threshold value that increased the relevance of blocks only

text than other evaluation metrics, such as ROUGE, which

when the BART loss difference exceeded 𝑡𝑢𝑝 = 0.15 andcan be useful when evaluating summaries of different

decreased the relevance of blocks when it was less thanlengths. BERTScore has been shown to be effective at𝑡𝑑𝑜𝑤𝑛 = −0.07 . The training of the compressor andevaluating the quality of summaries and other types ofsummarizer was set to 6 epochs.

generated text.

In the inference stage, we set three layers of dynamic
compression with the number of reserved blocks in each

4.2 Baseline Models

layer being 3, 3, and 10, resp ectively.

We chose three types of widely used methods in thefield of long document summarization as baseline models
for comparison.

5. Experimental Results
For different initialization strategies, we conducted

The first type of methods is unsupervised extractive

comparative experiments, and the results are shown insummarization methods, which mainly includes the most

Table 2. Here we also introduce Rouge Similarity, which

representative

statistics

uses the ROUGE scores of the sentence pairs as their

method Tf-Idf, as well as classical BERT-based extractive

similarity scores. The experiment is based on the partial

summarization methods. The summaries generated byGovReport dataset with 5 epochs training. It can be seen

thesethat the unsupervised trained SimCSE shows a slightmethods

traditional

are

word

derivedfrequency

from

the

sentence

reorganization extracted from the source documents.
The second type is supervised abstractive end -to-end
methods based on BART or PEGASUS [15]. This type ofapproach differs from the two-stage model proposed in ouradvantage in the final summar ization results, so in thefollowing experiments, we use the unsupervised trained
SimCSE as the default initialization strategy.
Table 2. different initialization strategies comparison

paper, which emphasizes end -to-end models. Models using
efficient self-attention such as Longformer and Bigbird
are also included in this category.
The third type is hybrid summarization based oncontent selection and abstractive summarization, where
LoBART [11] is a recognized representative work in thisarea, and our work also belongs to this type. We will focus
on this type of comparison.

Model

R-1

R-2

R-L

Rouge similarity

51.44

18.12

25.53

Bi-Encoder

50.58

18.01

24.69

Cross-Encoder

50.75

18.16

25.06

SimCSE (trained)

51.82

18.22

27.33

Table 3. Experimental results and comparison with other baseline models
Datasets

R-1

R-2

PubmedR-L
BERTScore

R-1

arXiv
R-L

R-2

BERTScore

R-1

R-2

GovReport
R-L
BERTScore

Unsupervised Extractive
Tf-Idf

/

/

/

/

34.4

9.5

28.5

0.822

/

/

/

/

BERT

/

/

/

/

35.1

9.8

29.0

0.819

/

/

/

/

Supervised Abstractive – End-to-End
BART-only

42.1

17.5

37.1

/

41.3

15.3

36.8

0.846

52.8

19.1

49.9

/

Longformer

46.9

20.2

42.8

/

46.7

19.6

41.8

0.865

/

/

/

/

BigBird

46.3

20.6

42.3

/

46.2

19.0

41.5

0.855

56.8

22.6

53.8

/

0.845

/

/

/

/

0.868

55.3

21.9

41.6

0.865

Supervised Abstractive – Hybrid summarization
LoBART

48.0

20.9

43.5

/

48.6

20.1

42.2

Our Proposed Model
Ours

46.8

20.3

35.7

0.863

47.9

Table 3 shows the experimental results of our proposed

19.5

35.6

document summarization method. Compared with generalmodel on three different long document datasets. As seen

extractivein Table 3, we observed a 0.5 R-1 improvement whenself-attention based long document adaptation mechanisms,

compared to BigBird on the Pubmed dataset, and thisour proposed model can achieve a balance in resource

result is also close to the result of Longformer. For the

consumption and model performance, which is especially

arXiv dataset, our model has a relatively 1.7 / 1.2 R -1

important for implementing long document summarization

improvement when compared to B igBird and Longformer.

in single-GPU or low resource scenario s. Also, as an

For the GovReport dataset which has the longest averageadvantage of the hybrid summarization model, our model

length, our model also achieves a good result in R-1 andhas an impressive performance in handling documents of

R-2 scores, it is slightly lower than the result of BigBird.

any length, and even outperforms SOTA on specific

For the Pubmed and arXiv datasets, LoBART has adomains. As an additional result, the summaries werelatively high ROUGE score, but for the arXiv dataset,generated are more closely aligned with human judgment

our model has the highest BERTScore.

(See Appendix Table 5), as demonstrated by the fact that

However, our model has a relatively poor performancein terms of R-L scores compared to other models, assummarization

modelsandefficient

our model obtained a higher BERTScore than otherbaseline models.

evidenced by the fact that our model generates summaries

For future work, we tend to further improve thethat have a small common subsequence with the target

performance of the model, especially when dealing withsummary. This can be explained by the characteristics of

extremely long documents. As a further addition to thethe content selection mechanism. Content selection can

CogLTX series of studies, we are hopeful to implement

pick out the most salient sentences in the originalmodel

document, however, the selected sentence s are often

adaptation

discrete, so it will cause the reduction of R -L. The

extraction and generation, multi -label text classification,

decrease of R-L corresponds to the increase of BERTScore,etc., to address the long -document problem faced by theseand how to ensure the balance between the two is also adomains as well.

key point to be considered in our future work.

6. Conclusion and Future Work
For

the

long

document

problem

that

is

often

encountered when using traditional pre -trained language
models for text summarization, we propose a hybrid long

migration
in

and

other

corresponding
domainslong -document

includingkeyphraseReferences[1] Abualigah, Laith, et al. "Text summarization: a brief
review." Recent Advances in NLP: the case of Arabic
language (2020): 1-15.
[2] Bajaj, Ahsaas, et al. "Long Document Summarization
in a Low Resource Setting using Pretrained Language
Models." arXiv preprint arXiv:2103.00751 (2021).
[3] Beltagy, Iz, Matthew E. Peters, and Arman Cohan.
"Longformer: The long-document transformer."arXiv preprint arXiv:2004.05150 (2020).
[4] Devlin, Jacob, et al. "Bert: Pre-training of deep
bidirectionaltransformersforlanguage
understanding." arXiv preprint arXiv:1810.04805(2018).
[5] Ding, Ming, et al. "Cogltx: Applying bert to longtexts." Advances in Neural Information Processing
Systems 33 (2020): 12792-12804.
[6] El-Kassas, Wafaa S., et al. "Automatic text
summarization: A comprehensive survey." Expert
systems with applications 165 (2021): 113679.
[7] Gao, Tianyu, Xingcheng Yao, and Danqi Chen.
"Simcse: Simple contrastive learning of sentence
embeddings." arXiv preprint arXiv:2104.08821(2021).
[8] Huang, Luyang, et al. "Efficient attentions for long
document
summarization."arXiv
preprint
arXiv:2104.02112 (2021).

[9] Koh, Huan Yee, et al. "An Empirical Survey on Long
Document Summarization: Datasets, Models, andMetrics." ACM computing surveys 55.8 (2022): 1 -35.
[10] Lewis,
Mike,
et
al.
"Bart:
Denoisingsequence-to-sequence
pre-trainingfornaturallanguagegeneration,translation,
andcomprehension." arXiv preprint arXiv:1910.13461(2019).
[11] Manakul, Potsawee, and Mark JF Gales. "Long -spansummarization via local attention and content
selection." arXiv preprint arXiv:2105.03801 (2021).
[12] Radford, Alec, et al. "Improving language
understanding by generative pre-training." (2018).
[13] Reimers, Nils, and Iryna Gurevych. "Sentence -bert:
Sentence embeddings using siamese bert -networks."
arXiv preprint arXiv:1908.10084 (2019).
[14] Zaheer, Manzil, et al. "Big bird: Transformers forlonger sequences." Advances in Neural Informati on
Processing Systems 33 (2020): 17283 -17297.
[15] Zhang, Jingqing, et al. "Pegasus: Pre -training withextracted
gap-sentencesforabstractive
summarization."International
ConferenceonMachine Learning. PMLR, 2020.
[16] Zhang, Tianyi, et al. "Bertscore: Evaluating text
generation
withbert."
arXiv
preprint
arXiv:1904.09675 (2019).

Appendix
Table 4. Sample summaries for arXiv dataset
Target Summaries
additive models play an important role in semipara metricstatistics . this paper gives learning rates for regularizedkernel based methods for additive models . these learning
rates compare favourably in particular in high dimensions
to recent results on optimal learning rates for purely
nonparametric regularized kernel based quantile regression
using the gaussian radial basis function kernel , provided
the assumption of an additive model is valid . additionally ,a concrete example is presented to show that a gaussianfunction depending only on one variab le lies in a
reproducing kernel hilbert space generated by an additive
gaussian kernel , but does not belong to the reproducing
kernel hilbert space generated by the multivariate gaussian
kernel of the same variance . * key words and phrases . *additive model , kernel , quantile regression ,semiparametric , rate of convergence , support vector
machine .
the effect of a random phase diffuser on fluctuations oflaser light ( scintillations ) is studied . not only spatial but
also temporal phase variations introduced by the phasediffuser are analyzed . the explicit dependence of thescintillation index on finite - time phase variations is
obtained for long propagation paths . it is shown that forlarge amplitudes of phase fluctuations , a finite - time effectdecreases the ability of phase diffuser to suppre ss the
scintillations .

Generated Summaries
This paper discusses the importance of additive models insemiparametric statistics and presents learning rates forregularized kernel-based methods for these models. The
learning rates are compared favorably, particularly in highdimensions, to recent results on optimal learning rates forpurely nonparametric regularized kernel -based quantile
regression using the Gaussian radial basis function kernel,
provided the assumption of an additive model is valid. The
paper also provides a concrete example to demonstrate thata Gaussian function depending only on one variable lies ina reproducing kernel Hilbert space generated by an additive
Gaussian kernel but does not belong to the reproducing
kernel Hilbert space generated by the multivariate Gaussian
kernel of the same variance. The key terms and phrases
discussed in the paper include additive model, kernel,
quantile regression, semiparametric, rate of convergence,and support vector machine.
The paper examines the impact of a random phase diffuser
on fluctuations of laser light, specifically scintillations.
Both spatial and temporal phase variations caused by the
diffuser are analyzed. The paper derives the explicit
relationship between the scintillation index and finite -timephase variations for long propagation paths. The study
shows that for large amplitudes of phase fluctuations, afinite-time effect reduces the ability of the phase diffuser tosuppress the scintillations.