# NoSummary: Control Tokens for Romanian News Summarization
## 2 Method
### Control Tokens  
Starting from previous studies presented in the Introduction and related to the specifics of the summarization task, we chose to specify a set of 4 control tokens representative of various characteristics of the text, namely:  
* **NoSentences** indicates the number of sentences that the summary should have;
* **NoWords** indicates the number of words to be generated within the summary;
* **RatioTokens** reflects how many times the sequence of tokens of the summary must be longer than the input;
* **LexOverlap** is the ratio of the number of 4-grams from the summary that also appears in the reference text; stop words and punctuation marks were omitted.  
The first 3 control tokens are purely quantitative and reflect different use-case scenarios: a summary containing at most a specific number of sentences, a summary having an imposed number of words, or a compression ratio to be used globally. The last control token ensures a lower or higher degree of lexical overlap between the two texts.  
The prompt for the summarization task was the following:  
\[Text:\{article\}\ Summary:\{summary\}
 \tag{3}\]  
The model learns that, after the control token **\"Summary:\"**, it must generate the summary of the text preceding that token. Control tokens are specified before the token that indicates the input (i.e., marked by the **Text** token), while the token specific to the task is placed after the end. The prompt used for an item from the dataset used for training is the following:  
\[FeatureToken:\{value\}\ Text:\{article\}\ Summary:\{summary\}
 \tag{4}\]  
where FeatureToken is <_NoSentences_>, <_NoWords_>, <_RatioTokens_>, or <_LexOverlap_>.  
Following the initial experimentation, we noticed that the model learns best when subsequent entries have the same input text, but with different values for the control tokens and a different text to be generated; this refers to the extraction of fragments from the original summary and their use as the output. This variation is reflected in the text to be generated and was used for the <NoSentences>, <NoWords>, and <RatioToken> control tokens. The generation of multiple variations was applied if the summary text had more than 3 sentences; thus, incremental examples were generated by adding sentences and calculating the value for the control token each time. An example for a summary comprising 4 sentences \(s_{1}\), \(s_{2}\), \(s_{3}\), \(s_{4}\) and <NoWords> would consider two entries in the training dataset: the first item would consist of the first 3 sentences and the corresponding <NoWords> for this first shorter summary and a second item where the \(s_{4}\) sentence would be added and <NoWords> is set at the global count of words from the summary.  
Besides training the summarization model with each control token individually, we also considered combinations of 2 control tokens, namely: <NoWords>-<NoSentences>, <RatioTokens>-<NoSentences>, and <LexOverlap>-<NoWords>. The combination <NoWords>-<NoSentences> was chosen because it reflects the most straightforward manner to manually enforce the length of the summary by an end user (i.e., specify an approximate number of words and the number of sentences that the generated summary should have). <RatioTokens> presents the same idea as <NoWords>, only that it is much more difficult to learn by the model as it represents the ratio between the length of the news and that of the summary. The combination of <LexOverlap>-<NoWords> is interesting because it forces the model to generate a text with an approximate number of words. Still, the generated text must not match the one received by the model. <NoWords> indicates how many words the summary should have, while <LexOverlap> restricts the percentage of combinations of words that are present in the news and generated text by the model; a small value for <LexOverlap> indicates that the model must reformulate an idea from the news, whereas a large value makes the model extract the most important phrases within a word limit.