# Attentional Extractive Summarization +
## 2 Related Work  
In this section, different approaches to extractive and mixed summarization are described, in particular those state-of-the-art systems used in the experimental comparison of this work. Recently, the construction of large corpora [22; 23; 25; 26] has allowed the training of Deep Learning systems for the automatic summarization problem. A very robust baseline used commonly in newspaper summarization is the Lead system. It is based on extracting the first \(k\) sentences of the documents to compose a summary. Although it seems naive, it is especially robust when it is applied to articles in newspapers, since in this domain, generally, the first sentences are dedicated to condensing the information of the whole document and they are used to attract the reader's attention. Extractive approaches can be divided into two different categories: those which use an oracle algorithm to label the sentences of the documents before training the models, and those which directly optimize the ROUGE evaluation metric by means of Reinforcement Learning strategies. Regarding the extractive systems based on oracles, the first approaches were proposed by [PERSON] et al. [6] and [PERSON] et al. [7]. In [6], an encoder-decoder approach for extractive single-document summarization was proposed. In [7] (SummaRunner), [PERSON] et al. presented two versions of Hierarchical Attention Networks to select sentences from the documents as a binary sequence classification problem. One of these versions is trained using the samples provided by the corpus without a previous sentence labeling. The other version requires a greedy algorithm as an oracle for labeling the corpus at sentence level, selecting as a reference summary the set of sentences from the document that maximize the similarity with respect to the reference summary. Recently, the great impact of the Transformer architecture [21] in Natural Language Processing tasks, and particularly in language modeling [27], has boosted the results in automatic summarization by fine-tuning powerful pre-trained language models. The most relevant example is the BertSumEXT system [8], which is based on the fine-tuning of pre-trained Bidirectional Encoder Representations from Transformers (BERT) models [27]. [PERSON] et al. [8] also proposed abstractive and mixed strategies for generating summaries starting from the pre-trained BERT. Reinforcement Learning strategies for automatic summarization have received great interest from the research community. Despite the first works on Reinforcement Learning being intended to perform abstractive summarization by [PERSON] et al. [28], recently these strategies have been widely used for extractive text summarization, directly optimizing the ROUGE evaluation measure [10; 11; 12; 13; 14]. [PERSON] et al. [10] argued about the application of cross-entropy with ground-truth sentence labels to optimize neural summarization models, and they proposed the application of the REINFORCE algorithm [29] for extractive summarization in order to train a hierarchical encoder-decoder. [PERSON] et al. [11] also discussed the suboptimal nature of the labels obtained using oracles. They presented a latent variable extractive model, which can also be viewed as a Reinforcement Learning approach, where the reward is defined as a weighted sum of two measures related to the precision and the recall. These measures were computed from the likelihood of a summary sentence and a document sentence, estimated using an attention-based sequence-to-sequence sentence compression model. This system can be trained in an extractive (Latent) or in a compressive way (Latent-Comp). A theoretically grounded method (BanditSum) was proposed by [PERSON] et al. [12] to model the extractive summarization problem by means of a bandit formalism. They proposed a novel structure for computing the conditional probability of a subset of document sentences given the document, which avoids privileging early sentences over later ones. An approach based on Deep Q Learning (DQN) was proposed by [PERSON] et al.