## 3 Proposed Methodology  
### Overall  
This section presents the foundation of our semi-supervised method for automatic text summarization. Unlike existing models, which rely heavily on paired text for supervised training, our approach leverages a small paired dataset followed by a semi-supervised training process with unpaired corpora. The algorithm used in our method is illustrated in Algorithm 1, where \(L\) denotes the loss incurred by comparing two texts. Our approach is inspired by the CycleGAN architecture, which uses two generators to facilitate style transfer in two respective directions. The first part of our method comprises a warm-up step that employs real text pairs to clarify the tasks of the style transferers \(T_{a2s}\) and \(T_{52a}\) and generate basic outputs. The subscripts \(a2s\) and \(s2a\), which represent \"article-to-summary\" and vice versa, are employed to clarify the transfer direction. The second part adopts a similar training procedure to that of CycleGAN with consistency loss functions to further train the models without supervision. Specifically, the identity mapping loss ensures that a text should not be summarized if it is already a summary and vice versa. The corresponding training procedure is based on calling the model to re-generate an _identity_ of the input text. The loss is then calculated by measuring the difference between the original text and the generated identity. This part is designed to train the model to be capable of identifying the characteristics of two distinct text domains. In the following sections of the paper, a superscript _idt_ is used to indicate re-generated identity texts. Figure 2: Architecture of the T5 model. In contrast, the cycle consistency loss trains the model to reconstruct a summary after expanding it or vice versa. The corresponding training procedure follows a cyclical process: For a real summary \(s\), the model \(T_{s2a}\) first expands it and generates a fake article. The term \"fake\" indicates that it is generated by our model, rather than a real example from datasets. Next, the fake article is sent to \(T_{a2s}\) to re-generate its summary. For real articles, the same cycle steps are utilized. This part is designed to train the model to be capable of transferring texts between two domains. In the following, a superscript _fake_ is used to indicate the fake texts generated by the models, and a superscript _cyc_ is used to indicate the final outputs after such a cycle procedure. ```
1:for each batch \(\in\) gold_batchesdo
2: fine-tune \(T_{a2s}\) and \(T_{s2a}\) with batch \(\triangleright\) Finetune with real text pairs
3:endfor
4:for\(epoch\in[1,nb\_epochs]\)do
5:for all\((a_{i},s_{i})\) such that \(a_{i}\in Articles\) and \(s_{i}\in Summaries\)do
6:\((a_{i}^{idt},\ s_{i}^{idt})\leftarrow(T_{s2a}(a_{i}),\ T_{a2s}(s_{i}))\)\(\triangleright\) Re-expand and re-summary
7:\((L_{a}^{idt},\ L_{a}^{idt})\leftarrow(L(a_{i},a_{i}^{idt}),\ L(s_{i},s_{i}^{idt}))\)\(\triangleright\) identity mapping loss
8:\((s_{i}^{fake},\ a_{i}^{fake})\leftarrow(T_{a2s}(a_{i}),\ T_{s2a}(s_{i}))\)\(\triangleright\) Generate fake summary and article
9:\((a_{i}^{cyc},\ s_{i}^{cyc})\leftarrow(T_{s2a}(s_{i}^{fake}),\ T_{a2s}(a_{i}^{ fake}))\)\(\triangleright\) Restore article and summary
10:\((L_{a}^{cyc},\ L_{s}^{cyc})\leftarrow(L(a_{i},a_{i}^{cyc}),\ L(s_{i},s_{i}^{ cyc}))\)\(\triangleright\) cycle consistency loss
11:\(Loss\gets L_{a}^{idt}+L_{s}^{idt}+L_{a}^{cyc}+L_{s}^{cyc}\)\(\triangleright\) Total loss
12: Back-propagation of Loss
13:endfor
14:endfor
```  
**Algorithm 1** Semi-supervised automatic text summarization. As observed, despite the integration of the CycleGAN loss functions, we refrain from constructing a GAN architecture for our task. This decision arises from two factors: firstly, the challenge involved in the back-propagation phase of discrete sampling during text generation; secondly, the lack of discernible improvement vis-a-vis our method during development and the inherent instability in the training process.