# Abstractive vs. Extractive Summarization: An Experimental Review
## 3 Evaluation
### Experimental Setup and Evaluation Results
#### 3.2.1 Abstractive Models  
In this section, we describe the _Transformer_-based models and setups that appear in Table 2 and are based on the abstractive approaches presented in Section 2. All models set the dimension for the input layer to 1024, the only exception being _mT5-multilingual-XLSum_ for which it is set to 768. Specifically:  
* _BART-large-CNN_ is a finetuned version of the large sized model of _BART_ (_BART-large_) on _CNN/Daily Mail_. The _BART-large_ model uses the same number (i.e., 12) for each of the encoder, decoder, and hidden layers. The maximum sequence length (_max_position_embeddings_) of the model is 1024 tokens, while the output lengths (_min_length_, _max_length_) are set to (56, 142) tokens.
* _BART-large-XSum_ is a finetuned version of the large sized model of _BART_ on _XSum_. It uses the same number of layers as the previous model. The max sequence length (_max_position_embeddings_) of the model is 1024 tokens, while the output lengths (_min_length_, _max_length_) are set to (11, 62) tokens.
* _BART-large-CNN-SAMSum_ uses the _BART-large-CNN_ model; it is further finetuned on examples from the _SAMSum_ dataset. It has the same number of layers, maximum sequence, and output lengths as the _BART-large-CNN_ model.
* _PEGASUS-large_ is actually the \"_PEGASUS\({}_{\textit{LARCE}}\) (mixed, stochastic)_\" model reported in [9]. The key difference of this model is that it was pretrained on two large text corpora, namely _C4_ and _HugeNews_, with a different training configuration compared to the other PEGASUS models reported in [9]. This model features an updated tokenizer and an enhanced training process. The model has an encoder, a decoder, and a hidden state, each consisting of 16 layers. The max sequence length (_max_position_embeddings_) of the model is 1024 tokens, while the maximum output length (_max_length_) is set to 256 tokens.
* _PEGASUS-multi_news_ uses the _PEGASUS-large_ model finetuned on the _Multi-News_[63] dataset, which was originally made for the task of multi-document summarization. It has the same number of maximum sequence and output lengths.
* _PEGASUS-XSum_ uses the _PEGASUS-large_ model finetuned on _XSum_. It has a maximum sequence length (_max_position_embeddings_) of 512 tokens and an output length limit (_max_length_) of 64 tokens.  
* _PEGASUS-CNN_dailymail_ uses the _PEGASUS-large_ model finetuned on _CNN/Daily Mail_. It has a maximum sequence length (_max_position_embeddings_) of 1024 tokens and an output length limit (_max_length_) of 128 tokens.
* _DistilBART-CNN-12-6_ is a student model of _BART_ that is trained for _CNN/Daily Mail_ using the distillation technique of KD. It features an encoder and a hidden state, each consisting of 12 layers, and a decoder consisting of 6 (half of the teacher model's decoder layers). The student model has a maximum sequence length (_max_position_embeddings_) of 1024 tokens, while output length limits (_min_length_, _max_length_) are set to (56, 142) tokens.
* _DistilBART-XSum-12-6_ is a student model of _BART_ that is trained for the _XSum_ dataset using the same distillation technique as the previous model. It features an encoder and a hidden state with 12 layers, and a decoder with 6 of the teacher model's layers. The student model has a maximum sequence length (_max_position_embeddings_) of 1024 tokens, while output length limits (_min_length_, _max_length_) are set to (11, 62) tokens.
* _mT5-multilingual-XLSum_ is a model that uses Google's _mT5_ (multilingual-T5) base model finetuned on the _XLSum_ dataset for the task of multilingual summarization [55]. The model has an encoder, a decoder, and a hidden state, each consisting of 12 layers. The original authors truncated the input sequence to 512 tokens, while the maximum output length (_max_length_) is set to 84 tokens.