## 3 Proposed Methodology
### Training with the T5 Model  
Our training procedure consists of two parts: a supervised part and an unsupervised part. In the supervised part, we use small labeled data for warm-up while following the same procedure as that in the original T5 model. In this part, we fine-tune the T5 model with pairs of articles and summaries using different prefixes to indicate the generation direction. The loss function for the supervised part is cross-entropy, which is the same loss as that used in the original T5 model. In the unsupervised part, we adopt a training procedure inspired by the CycleGAN architecture, thus incorporating identity mapping loss and cycle consistency loss. The identity mapping loss deters the model from re-summarizing a summary or expanding a full article by minimizing the difference between the input and output texts. Meanwhile, the cycle consistency loss ensures that the model preserves the source text after a cyclical transfer by minimizing the difference between the input and reconstructed texts. Figure 4 illustrates these two processes. We propose a novel training procedure that uses a single T5 model for both generation tasks with different prefixes. Given an article \(a\) and its summary \(s\), we use the T5 model to generate a fake summary \(s^{fake}\) from \(a\) and a fake article \(a^{fake}\) from \(s\). To indicate the desired  
Figure 4: CycleGAN losses of the proposed model. Figure 3: T5 model with different prefixes. task, we prepend a prefix string to the input text. The generation process can be formulated as follows:  
\[ \begin{split} s^{fake}&=T_{s}(a)=T(P_{s}\oplus a)\\ a^{fake}&=T_{e}(s)=T(P_{e}\oplus s)\end{split}  \tag{1}\]  
where \(T_{s}()\) and \(T_{e}()\) denote the T5 model with the summary prefix and the expansion prefix, respectively. The training process follows a typical supervised paradigm, a cross-entropy loss [32] is calculated to measure the difference between two texts, and the model is trained via back-propagation. \[L(x,x^{fake})=-\sum_{i=1}^{C}p_{i}(x)\log p_{i}(x^{fake}) \tag{2}\]  
where C is the vocabulary size, and \(p_{i}()\) is the probability of i-th word in the vocabulary. For the rest of the dataset, where an article \(a\) and a summary \(s\) are not paired, we calculate the two consistency losses. The identity mapping loss is calculated by re-summarizing a summary or re-expanding an article as follows:  
\[ \begin{split} a^{idt}&=T_{e}(a)\qquad\quad s^{idt} =T_{s}(s)\\ L_{a}^{idt}&=L(a,a^{idt})\qquad\quad L_{s}^{idt}= L(s,s^{idt})\end{split}  \tag{3}\]  
As for the cycle consistency loss, the model first generates \(s^{fake}\) and \(a^{fake}\) as stated before; then, it regenerates \(a^{cycle}\) and \(s^{cycle}\) based on \(s^{fake}\) and \(a^{fake}\). After such a cycle, the losses are calculated as follows:  
\[ \begin{split} a^{fake}&=T_{s}(a)\qquad\quad s^{ fake}=T_{e}(s)\\ a^{cyc}&=T_{e}(s^{fake})\qquad\quad s^{cyc}=T_{s}( a^{fake})\\ L_{a}^{cyc}&=L(a,a^{cyc})\qquad\quad L_{s}^{cyc}= L(s,s^{cyc})\end{split}  \tag{4}\]  
The training algorithm is, thus, adapted as in Algorithm 2 (T for T5 model, \(\oplus\) for concatenation of texts). We use \(P_{s}\) and \(P_{e}\) to denote \(prefix\_summarize\) and \(prefix\_expand\), respectively.