LLM aided semi-supervision for efficient Extractive Dialog Summarization
Authors: Nishant Mishra
Gaurav Sahu
Iacer Calixto
Ameen Abu-Hanna
Issam H. Laradji
Venue: EMNLP 2023 Findings (2023)
Keywords: Dialog Summarization
Large Language Models
Semi-supervised learning
Transformers
BART
GPT
Rouge

Abstract: Generating high-quality summaries for chat dialogs often requires large labeled datasets. We propose a method to efficiently use unlabeled data for extractive summarization of customer-agent dialogs. In our method, we frame summarization as a question-answering problem and use state-of-the-art large language models (LLMs) to generate pseudo-labels for a dialog. We then use these pseudo-labels to fine-tune a chat summarization model, effectively transferring knowledge from the large LLM into a smaller specialized model. 
We demonstrate our method on the TWEETSUMM dataset, and show that using 10\% of the original labelled data set we can achieve 65.9/57.0/61.0 ROUGE-1/-2/-L, whereas the current state-of-the-art trained on the entire training data set obtains 65.16/55.81/64.37 ROUGE-1/-2/-L. In other words, in the worst case (i.e., ROUGE-L) we still effectively retain 94.7\% of the performance while using only 10\% of the data.

LLM aided semi-supervision for Extractive Dialog Summarization
Nishant Mishral:3 Gaurav Sahu 3.4 lacer Calixto 1.2
Ameen Abu-Hanna Issam H. Laradji
Amsterdam UMC, Department of Medical Informatics, University of Amsterdam Amsterdam Public Health, Methodology, Amsterdam, The Netherlands 3 University of Waterloo "ServiceNow Research 5 University of British Columbia
Abstract
the summary consists of parts of the original in- put (Feigenblat et al., 2021; Liu, 2019). In this work, we propose a novel extractive dialog summa- rization method based on a semi- cemi-supervised learn- ing paradigm, and demonstrate that we either im- prove perform comparably to the current state- of-the-art on the TWEETSUMM dataset (Feigenblat et al., 2021), while using only 10% of the training data.
Generating high-quality summaries for chat di- alogs often requires large labeled datasets. We propose method to efficiently use unlabeled data for extractive summarization of customer- agent dialogs. In method, we frame problem
marization as a use state-of-the-arr large language models (LLMs) to generate pscudo-labels for a dia- log. We then use these pseudo-labels to fine- tune a chat summarization model, effectively transferring knowledge from the large L.LM into a smaller specialized model. We demon- strate our method on the TWEETSUMM dataset, and show that using 10% of the original la- belled data set we can achieve 65.9/57.0/61 ROUGE- 1/-2/-L, whereas the current state-of- the-art trained on the entire training data set obtains 65. 16/55.81/64... ROUGE- 1/-2/-I. In other words, in the worst case (i.., ROUGE-L) we still effectively retain 94.7% of the perfor- mance while using only 10% of the data.
Pseudolabeling(Le, 2013; Pham al., 2021) is a popular and efficient method for semi-supervised learning. Pre-trained LLMs encode a vast breadth of knowledge various tasks and can generate hu- man like response. They are good annotators( (Wang et al., 2021; Ding et al., 2023) and also ideal for transferring knowledge through distillation( Kim et al., 2022: Kim and Rush, 2016: Liu al., 2021). Hence, decided use LLMs 1) as weak la- bellers to generate automatic summaries for a large set of unlabelled examples (i.c., pseudo-labels); and 2) as evaluators to choose only a small num- ber of high-quality (pseudo-labelled) examples to include in the next training cycle. We report strong results on the TWEETSUMM dataset in zero- and few-shot supi-supe settings. Our main tributions are follows.
1 Introduction
Customer support chats are gaining popularity as a primary medium for servicing clients in indus- try. Summarization models that robustly capture critical information are therefore extremely benefi- cial for companies to support various downstream activities such record- record-keeping, training. Unfortu- nately, standard supervised training of these models rely on large labeled datasets, which can be pro- hibitively expensive to build. While unsupervised summarization techniques exist (Zou et al., 2021; Zhang et al., 2021; Shang et al., 2018), enforcing the summary quality and style is still an ongoing challenge.
We introduce a semi-supervised method for extractive summarization to distill knowledge from general-purpose LLMs into smaller spe- cialized summarization models.
We show that our methods have better or comparable performance to the current state- of-the-art trained on the entire training data (according to ROUGE), while requiring only 10% of the labelled data.
In general terms, chat summarization can be abstractive, where the generated summary is constrained(Goo and Chen, 2018; Chen and Yang, 2021; Gupta and Gupta, 2019), or extractive, where
We show that framing extractive summa- rization as gnesttoon-answeiing problem allows us to efficiently leverage large lan- guage models for weak supervision (c.g., GPT- 3.5; Ouyang et al. 2022).
Corresponding author n.mi wwwsshaamttetedmmmmmmm nl. Work (partially) done during a research visit at ServiceNow.

---

iterate C/ times
pseudo-labelled examples (Dp) for the next step without replacement, and 3) retrain summarization model using labelled and pseudo-labelled examples DE
New labelled set D2
Labelled data Dz
Step 2: Evoluste
Filterad pseudo- labelled data P
(laken probaiilly)
3: Train DistlBART
In practice, we pseudo-label all unlabelled sam- ples DD only once using GPT-3.5 and re-use the predicted labels at each eycle E C. We can do this efficiently for two reasons: the number of un- labelled samples in TWEETSUMM is small (i.c.. 850 data points); and we do not fine-tune GPT-3.5, which is an efficient and performant alternative (as we will show in our experimental results).
Uniabelled data Do
Step1: Weakly label GPT-CA promppi
Pseudo- labelled Gala 2
New unlabelled Po's D2p
Figure 1: Overview of our approach.
Related work Chen and Yang (2021) introduced Conversational Data Augmentation (CODA), sim- plc conversational data augmentation framework for abstractive dialog summarization with an op- tional extension for a semi-supervised learning set- ting (He et al., 2020; Xie et al., 2020). Sznajder et al. (2022) propose a heuristic- based weakly- supervised abstractive summarization model for the TWEETSUMM dataset. They first train separate models for customer and agent on a weakly-labeled set obtained by using the LONG and LEAD heuris- ties and then finetune such models labelled con- versation data. Liu and Lapata (2019) use pre- trained encoders like BERT and propose PreSumm, a model for abstractive and extractive summariza- tion based on sentence classification.
1) Weak labelling Before the first cycle (c 0), we use GPT-3.5 (OpenAI, 2023) in a few-shot ser- ting to generate pseudo-labels for unlabelled ex- amples Dr. We refer to this set of pscudo-labelled examples as Dp = {uz, 8 We framed the extractive summarization task a question- answering problem for GPT-3.5. The dialogues were converted into sequence of numbered sen- tences. The prompt consisted of an instruction that asked the model to return sentence numbers that adequately summarised the dialogue from both per- spectives. We also provided several examples as context. We then extracted the sentences the dia- log corresponding to the numbers that the model re- turned to construct the pseudo-labele summaries. This helped ensure that the pseudo-labels thus ob- tained were always extractive. Please refer to Ap- pendix A for more details about the overall process of peuud-labelling and samples of the prompts used.
2 Methodology
2.1 Notation
We assume training dataset D Dr U Di with labelled (D,) and unlabelled examples (Du). La- belled examples Dr = (w, s consist of a dialog instance u; = [ui,1, U4,,, , com- posed of N input tokens and dialog un's summary = (s s1,,, si.Mr } with M tokens. Unla- belled examples Du- { consist of a dialog instance uj = {u U3,2 ... =5,K] with K input tokens for which there is no summary. In most practical cases, Du >> DL-
2) Evaluating and selecting pseudo-labelled ex- amples We first evaluate the quality of each gen- erated pseudo-labelled example in Dp. We use the big-robbaylityy assigned by GPT-3.5 to tokens cor- responding to the numbers of all sentences included as part of the summary in step 1, and sum these log- probabilities to compute a score for cach pscudo- labelled example in Dp. We then select subset D'p C Dp of the D'p highest-scoring examples. Fi- nally, we merge the original labelled data D2 with the selected iseudo-labelled data D'p, generating a new set of labelled examples D D,, U D'p to use to train the summarization model.
2.2 Iterative procedure for knowledge distillation
In Figure 1, we show an overview of our approach. We use an idea similar to teacher-student trans- fer lcarning(Sanh et al., 2020; Shleifer and Rush, 2020a; Tang and Huang, 2022) and propose an iterative training procedure with C cycles. Each cycle of training consists of three steps: 1) Use a weak-labeller to generate pscudo-labels for di- alogs uy E Du, 2) evaluate and select high-quality
3) Train summarization model using DDew We train a sequence-to-sequenee model on the updated set of labelled examples Drew, which includes the original labelled examples Dí and the pscudo- labelled examples D'p selected in step 2. We train

---

this model to generate an extractive summary si for a dialog u, by minimising the negative log- likelihood ((S) - ((Dz) + ((D)), given that
with three random seeds and average results ac- count for variance. All the evaluations of extractive summarization for the models described in the ex- periments below are performed in a limited length of 80 tokens setting.
Dr.
((Dc) = Σ
log p( 81,1 St, uz 0z
Our Method We used the method described use Distil- BART (Shleifer and Rush, 2020b>-a distilled ver- sion of BART (Lewis al., 2019) with 6 encoder and 6 decoder layers-as as the sequence-to-sequene model. We set the number of examples D'p se- lected by the evaluator at each cycle as 16, the total number of cycles C = 10, and in each cycle we train the extractive summarization model for 10 epochs.
D2p
in section 2.2. Concretely,
((D()) = Σ
- -logp(83 18,00, ug;
where st (3),z) are the first t - 1 tokens of the gold-standard summary 8; (the generated summary 8)).
4) Convert generative summaries to Extractive The summarization model is given a dataset where the ground truth summaries always contain sen- tences from within the dialog, thus we hypothesize that the model should learn representations in a way output exclusively extractive summaries. Even with qualitative and quantitative results validating this, there is no guarantce of an exclusively ex- tractive output from a generative seq-to-seq model. Thus we add another step to ensure extractive sum- marization.
Vanilla method We also explore 'vanilla' ver- sion of our method. Crucially, with this method we do not transfer knowledge from a massive LLM (c.g., GPT-3.5) into the summarization model but use the summarization model in two-step noisy self- telf-training paradigm(Chen et al., 2021).
3.1 Iterative training and GPT-3.5
We devise number of baselines for this experi- ment. We use LEAD-1 and LEAD-2 (Kryseinski et al., 2019), two competitive extractive summa- rization baselines which take the first one (two) leading sentences each from the customer and from the agent as the final summary. We also use LONG- 1 where the agent's and the customer's longest sentences are taken as the final summary. These baselines are not iterative. We directly measure these heuristically generated summaries on the test set. We propose two itcrative semi-supervised baselines where use the agent's and the cus- tomer's first sentence (LEAD-1-i) and longest sen- tence (LONG-I-i), respectively, as the initial sum- mary ($ 2.2, step 1), but instead of selecting the highest- examples ($ 2.2, step 2), we ran- domly select X examples to use as D'p ($ 2.2, step 3).
The generated summaries s; are converted to ex- tractive summaries by sentence matching proce- dure. Each sentence in the generated summary and the dialogue was first embedded using a pre-trained BERT-based sentence transformer model (Reimers (Reimess and Gurevych, 2019). For each sentence embed- ding in the generated summary, we calculated its co- sine distance with each sentence embedding from the original dialogue to obtain semantic textual sim- ilarity. We then replaced the summary sentences with the corresponding sentence in the dialog that had the highest similarity i.e. lowest cosine dis- tance from them, constructing the final extractive summary.
3 Experimental Setup
Data In our experiments, we use the TWEET- SUMM dataset (Feigenblat et al., 2021), which con- tains 1, 100 labelled examples Dr of dialog be- tween customers and customer support agent. We use the original splits and have 880, 110, and 110 examples for training, model selection, and testing, respectively. In semi-supervised experiments, we subsample a fraction of data as labelled and rest used as unlabelled. We repeat these experiments
3.2 Few-shot learning
For this set of experiments, we retrain state-of-the- art task-tuned language models for extractive dia- log summarization. We use DistilBART fine-tuned on the XSum (Narayan et al., 2018. DistilBART- xsum) and CNN-dailymail (Nallapati et al., 2016. DistilBART--a RT-enn) datasets, and further fine-tune these two models once using various subsamples of the TWEETSUMM dataset.
We used the TranencceeTannsfmmerss library

---

Models
Iter.
R-1
R-2
R-L
Models
Iter.
# labelled unlabelled data points
0/850
8/842
770
LEAD-1
x
40.89
34.22
38.74
D-BART-xsum
3.93
33.51
44.56
47.16
LEAD-2
x
53.51
43.27
47.76
D-BART-cnn
30.95
42.87
46.65
50.75
LONG-1
X
54.47
46.56
50.58
Ours (vanilla)"
50.72
50.16
51.42
51.83
LEAD-1-i
62.64
53.13
57.46
Ours (GPT-3.5)
53.75
54.15
55.82
56.94
LONG-1-i
59.80
50.12
54.36
Table 2: ROUGE-2 F-measures on TWEETSUMM's test
Ours (GPT-3.5)
65.93
56.94
60.96
set. D-BART DistilBART: t; Our methods either only use DistilBART-cmi as weak labeller and sum- marizer (vanilla) or use GPT-3.5 as weak labeller and DistilBART-enn as summarizer (for details, see 2.2).
Table I: Results on TWEETSUMM's test set. Compari- son between five simple baselines and our method with GPT 3.5 as weak labeller and evaluator and DistilBART- cnn as summarizer. Baselines can be iterative (v) or not (X). R-1, R-2, and R-L correspond to ROUGE-1, ROUGE-2, and ROUGE-L F-measure, respectively.
10% of the training data. We use the ROUGE-1, ROUGE-2 and ROUGE-L metrics (Lin, 2004) since these are widely used metrics to automati- cally evaluate text summarization models, and also for comparing results with existing work.
3.3 Full data scenario
Here, we use state-of-the-art models for ex- tractive dialog sunmarization-inclueee models used in the original TWFETSUMM dataset pa- per (Feigenblat al., 2021) well as instruction- tuned LLMs to directly summarize dialogs. Pre- Summ (Liu and Lapata, 2019) frames extractive summarization
4.1 Iterative training
We report results in Table 1. We observe that Our approach significantly outperforms baselines di- rectly using first and longest sentences from the customer and agent as the summary in both itera- tive and non-iterative scenario across all metrics. Moreover, cyclic baselines clearly outperform their non-cyclic versions, highlighting the importance of distilling knowledge into the summarization model in small steps.
sentence classification problem by predicting whether sentences are noteworthy, and is the current SotA on TWEETSUMM for ex- GPT-3.5 we directly
tractive summarization. prompt the model to auto-regressively generate the entire summary for a dialog.
4.2 Few-shot learning
In GPT-3.5-QA we treat extractive summariza- tion as wwtpesttionnnnsweriin problem as we do in our method ($ 2.2),
In Table 2, we first observe consistent gains from using semi-supervised learning and iteratively train- ing the summarization model (DistilBART) us- ing more and more pseudo-labelled dialogs, over the same models fine-tuned on only labelled data. We also consistently improve DistilBART's per- formance by using GPT-3.5 as the weak labeller and evaluator, i.e., between 3.3%-5.19 % increase in ROUGE-2 compared our vanilla method.
We provide more details on how we prompt GPT-3.5 autoregressively and in QA setting in Appendix A. For both GPT-3.5 and GPT-3.5-QA, we experiment in different few-shot in-context (Brown al., 2020) settings- (0,1,2,4.8)-withh) the 4096 token limit. We measure the performance using Prompted Direct Inference(PDI), directly measuring the performance on test data.
4.3 Full data scenario
In Table 3, we show that our best model using GPT- 3.5-trained on only 10% of the data outperforss PreSumm-whicl is trained on the entire training dataset-accordiigg according to ROUGE-1 and ROUGE-2, and is only 3.5% behind according to ROUGE-L. We also outperform DistillBART models finetuned on the entire dataset by almost points across all metrics using only 10% of the labelled data. Among the in-context prompt based models, GPT- 3.5 A--QA outperforms GPT-3.5 which is notable as shows why we used for geeudo-labelligg The
4 Results
In $ 4.1, we first discuss how our method compares to simple baselines (iterative and non-iterative), clearly showing it surpasses both. In 4.2, we ex- amine how our approach fares in few-shot Iearn- ing setting where there no-to-little training data available. In 4.3, we compare our best few-shot models with the current SotA using all training data available, and show that our method performs comparably to the current SotA while using only

---

Models
Iter.
Semi.
R-1
R-2
R-L
tasks will help consolidate the performance of our method.
GPT 3.5
X
in-etx
58.11
49.48
54.86
GPT 3.5-QA
X
in-etx
62.05
52.98
57.63
Another significant limitation is the closed source nature of the model we are using for pseudo- labelling i. GPT-3.5(text-daaiinc--000 makes it prohibitively expensive to fine-tune reproduce in cases where the number of samples to be pseudo-labelled is substantial. Therefore we ended up using fixed teacher model and generat- ing the pseudo-labels only once. Having a more tunable, open-source instruction LLM as the weak labeller would have enabled prompt-learning to further improve pseudo-labels. This might have helped us unlock other aspects of teacher-student learning by allowing feedback from the smaller student model to improve/align the larger model updating the pseudo-labels every cycle i.e meta pseudo-labeling (Pham et al., 2021). It would also have provided more insight into the method while being less expensive for our task. We tried to over- come this limitation by exploring souncsource in- struction tuned models like alpaca-7B, vicuna-133 13B as our weak labeller. But the compute requirements to host and infer with these models, inferior and slow inference using the low parameter and quan- tized versions proved that they are not yet tractable and robust.
Full data
This
D-BART-ssum
X
X
61.24
54.05
58,10
D-BART-enn
X
63.61
54.84
58.91
PreSumm
x
65.16
55.81
64.37
10% of training data
Ours (vanilla)²
61.30
51.83
56.89
Ours (GPT-3.5)
65.93
56.94
60.96
Table 3: Results on TWEETSUMM'S test set. D-BART DistilBART. f PreSumm is the model proposed in Liu and Lapata (2019). f; These are our best models in each case, which always the corresponding model trained on 10% of the training data. R-1, R-2, and R-L correspond to ROUGE-I, ROUGE-2, and ROUGE-L F-measure, respectively.
scores for GPT-3.5 and GPT-3.5-QA are from two- shot in-context learning which outperformed zero, one, four and eight shot performances.
5 Conclusions and Future Work
In this work, we introduce a semi-supervised approach for extractive summarization dis- till knowledge from general-purpose LLMs into smaller specialized summarization models. We demonstrate our method on the TWEETSUMM dataset, and show that while training on only 10% of the data our method is competitive to PreSumm, the current state-of-the-art, which uses 100% of the training data. We outperform PreSumm by 0.3% according ROUGE-1 and ROUGE-2, but are still 4% behind according ROUGE-L. As future work, we will extend our method to other text summarization datasets. We will also explore using various open-source, tunable LLMs for both summarization and pscudolabelling.
Ethics Statement
The dataset we used, TWEETSUMM is constructed from an open-source customer support dataset available here The data, although from real pub- lic interaction, has been anonymized. It's annota- tion was crowdsourced. Further we use GPT-3.5 for weeudo-labelling.Sine Since this is from real-world customer-agent interactions we can, in rare cases and based on context get spurious, undesirable or biased outputs. Although chances that are mini- mized by framing it into a QA problem and limiting the generated tokens to a very small window.
Limitations
We show promising results achieved using instruction-tuned LLMs to help with semi- supervised dialog summarization, But there are a few limitations to the work that are laid out in this section. One of the major issues is in terms of breadth of the experiments. The current work deals with only one dataset i.c TweetSumm and one kind of summarization, Extractive dialogue summariza- tion. Further experiments involving other popular datasets comprising different domains, sizes and
References
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss Voss, Gretchen Krueger, Tom Henighan. Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu. Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.

---

Language models are fow-shot learners. In Ad- vances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Ves Stoyanov, and Luke 7etlemoyer. 2019. Bart: De- noising sequence-t--eqqemeee pre-training for natural language generation, translation, and comprehension. arXiv preprint (oXiv:1910) 1910.13461.
Jiaao Chen and Diyi Yang. 2021. Simple conversational data augmentation for semi-supervised abstractive dialogue summarization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, pages 6605-6616.
Chin- Yew Lin. 2004, ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, 1,, Barcelona, Spain. Association for Computational Linguisties.
Yiming Chen, Yan Zhang, Chen Zhang, Grandee Lee, Ran Cheng, and Haizhou Li. 2021. Revisiting self- training for few-shot learning of language model.
Yang Liu. 2019. Fine-tune bert for extractive summa- rization. arXiv preprint arXiv: 1903.70318 10318.
Yang Liu and Mirella Lapata. 2019. Text summariza- tion with pretrained encoders. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3730-3740, Hong Kong, China. Association for Computational Linguistics.
Bosheng Ding, Chengwei Qin, Linfin Liu, Yew Ken Chia, Shafiq Joty, Boyang Li, and Lidong Bing. 2023. Is gpt-3 a good data annotator?
Guy Feigenblat, Chulaka Gunasekara, Benjamin Szna- jder, Sachindra Joshi, David Konopnicki, and Ranit Aharonov. 2021. Tweetsumm-4 dialog summariza- tion dataset for customer service. arXiv preprint 1xXiv:2111.1180
Yang Liu, Sheng Shen, and Mirella Lapata. 2021. Noisy self lell-knowledge distillation for text summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 692-703, Online. Association for Computa- tional Linguistics.
Chin-Wen Goo and Yun-Nung (Vivian) Chen. 2018. Abstractive dialogue summarization with sentence- gated modeling optimized by dialogue acts. 2018 IEEE Spoken Language Technology Workshop (SLT).
pages 735-742.
Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çaglar Gulçehre, and Bing Xiang. 2016. Abstrac- tive text summarization using sequence-to-sequee RNNs and beyond. In Proceedings of the 20th SIGNLL Conference on Computational Natural Lan- guage Learning, pages 280-290, Berlin, Germany. Association for Computational Linguisties.
Som Gupta and Sanjai Kumar Gupta. 2019. Abstractive summarization: An overview of the state of the art. Expert Systems with Applications, 121:49-65.
Junxian He, Jiatao Gu, Jiajun Shen, and Mare' Aurelio Ranzato. 2020. Revisiting self-training for neural sequence generation. In Proceedings of ICLR.
Shashi Narayan. Shay B. Cohen, and Mirella Lapata. 2018. Don't give me the details, just the summary! topic-aware convolutional neural networks for ex- treme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1797-1807, Brussels, Bel- gium. Association for Computational Linguistics.
Su Young Kim, Hyeonjin Park, Kyuyong Shin, and Kyung-Min Kim. 2022. Ask me what you need: Product retrieval using knowledge from gpt-3, arXiv preprint arXiv:2207.0255 12.
Yoon Kim and Alexander M. Rush. 2016. Sequence- level knowledge distillation. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 1317-1327, Austin, Texas, Association for Computational Linguistics.
OpenAL. 2023. GPT-3.5. "https://platform. openai. www. hoon/docs/model/// http://oolllld///e///s
Long Ouyang, Jeff Wu, Xu Jiang. Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback.
Wojeiech Kryscinski, Nitish Shirish Keskar, Bryan Mc- Cann, Caiming Xiong, and Richard Socher. 2019. Neural text summarization: A critical evaluation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (FMNL.P-IJCNLP) pages 540-551, Hong Kong, China. Association for Computational Linguis- tics.
Hieu Pham, Zihang Dai, Qizhe Xie, and Quoe V. Le. 2021. Meta pseudo labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 11557-11568.
Dong-Hyun Lee. 2013. Pseudo-label: The simple and efficient sumi-super learning method for deep neural networks.
Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks.

---

Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2020. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.
A Appendix
A.1 Pseudo-labelling using GPT
We used instruction tuned GPT-3.5(text-dayv 003) for Prompt Guided Unlabeled data annotation for extractive summarization. We explore two dif- ferent approaches to the problem that are described below. We test and report results for the two meth- ods through Prompted Direct Inference(PDI,, di- rectly measuring the performance by annotating test data.
Guokan Shang, Wensi Ding, Zekun Zhang, Antoine Tix- ier, Polykarpos Meladianos, Michalis Vazirgiannis, and Jean-Pierre Lorré. 2018, Unsupervised abstrac- tive meeting summarization with multi-sentence com- pression and budgeted submodular maximization. In Proceedings of the 56th Annual Meeting of the As- sociation for Computational Linguistics ( (Volume I: Long Papers), pages 664-674, Melbourne, Australia. Association for Computational Linguisties.
A.1.1 GPT-3.5 QA
Sam Shleifer and Alexander M. Rush. 2020a. Pre- trained summarization distillation.
Here we frame the task of weak labelling a qneestion-answering problem: we first split a given dialog uj into sentences using a sentence splitter; we then number these sentences from 1 until N, where N is the total number of sentences found in "j (e.g.. if the original sentence reads 'This is a sentence. and it is the 2nd sentence in the dialog, it becomes '2) This is sentence."; "); In order to vert the summary to the suitable format, compare the sentences in summary to the sentences in dialogue to find their positions in the dialog and then map the summary into a fixed structured sen- tence that highlights the numbers of the sentence containing the summary from both perspectives, We build prompt containing a few labelled examples (i.e., gold-standard dialog and summary) as context followed by the dialog ny for which we wish to generate a summary; we prompt GPT-3.5 asking it for the nunbers of all the sentences that must be included as part of the summary. To be more precise, we ask it to answer with sentence numbers that describe the issue being faced by the customer in the dialog and sentence numbers that best highlight the answers provided by the agent. Once we obtain the numbers of the sentences GPT-3.5 deems suitable to be part of the summary, we build the summary 8j by simply extracting these sentences from the original input uz.
Sam Shleifer and Alexander M Rush. 2020b. Pre-
trained summarization distillation. arXiv preprint arXiv:2010. 13002.
Benjamin Sznajder, Chulaka Gunasekara, Guy Lev, Sachin Joshi, Eyal Shnarch, and Noam Slonim. 2022. Heuristic-based inter-training to improve few- shot multi- pantiipperpeeetivee dialog summarization, arXiv preprint 22Xi::2003.55900
first
Ying-Jhe Tang and Hen-Hsen Huang. 2022. A teacher- student approach to cross-domail transfer learning with multi-level attention. In IEEE/WIC/ACM Inter- national Conference on Web Intelligence and Intelli- gent Agent Technology, WI-IAT '21, page 494-499, New York, NY, USA. Association for Computing Machinery.
Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. 2021. Want to reduce la- beling cost? GPT-3 can help. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4195 4205, Punta Cana, Dominican Re- public. Association for Computational Linguisties.
Qizhe Xie, Eduard H. Hovy, Minh-Thang Luong, and Quoc V. Le. 2020. Self-training with noisy student improves imagenet classification. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pages 10684-10695.
Figure 2 shows an example of a prompt we built for pscudolabelling along with the response from GPT-3.5.
Xinyuan Zhang, Ruiyi Zhang, Manzil Zaheer, and Amr Ahmed. 2021. Unsupervised abstractive dialogue summarization for tete-a-tetes. In Proceedings of the AAA/ Conference on Artificial Intelligence, vol- ume 35, pages 14489-14497.
A.1.2 GPT-3.5
In GPT-3.5 we directly prompt the model auto- regressively generate the entire summary for a dia- log. We provide labeled pairs of dialogue and sum- mary as context and ask it to generate summary for the next dialogue. Since this is an extractive
Yicheng Zou, Jun Lin, Lujun Zhao, Yangyang Kang, Zhuoren Jiang, Changlong Sun, Qi Zhang, Xuanjing Huang, and Xiaozhong Liu. 2021. Unsupervised summarization for chat logs with topic-oriented rank- ing and context-aware auto-encoders. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 35, pages 14682.
We used the spaCy sentencizer for this

---

bust glughtia the ou drlng atick cest te cas canal the scldoon
pier eurooie dhoguet seleer calthne suspon aed edtathe Learty meay cook
the do iove and reseucin the bed and dagogee You shou crocis yi lust seneecoe and noi
th mudices and walk dens
Ouroner
plidl reat siice
Grosurrtt ahoeing
oor mir drwwe
car snderg
with came
intared rectrnt
moker
Promppt
DCT0000 Copprny Dove
aerfence
Durage
Health
Catamar:
g110000 ineetgaaa agenae:
dhe decppu Misanii lndting
Forwarp
Asswer
The prompt used for GPT-3.5 QA in- context extractive dialog summarization and pseudo- labelling. The first part contains the prompt with instruc- tions and examples, and the second part is the response generated. Blue represents the dialogs, red represents the sample summaries/answen part of the prompt and green represents the final summary/answer generated. We show only one shot in-context here for brevity
Figure 2:
Outs
Figure 3: The prompt used for GPT-3.5 in-context extractive dialog summarization. The first part con- tains the prompt with instructions and examples, and the second part is the response generated. Blue represents the dialogs, red represents the sample summaries part of the prompt and green represents the final summary generated. We show only one shot in-context here for brevity
summarization problem, we make sure to clarify in the prompt instruction to use sentences that are part of the dialog and not to synthesize new sen- tences or paraphrase. Figure shows the prompt that we used for getting summaries with GPT-3.5 by completion.
B Test learning curves
58
Figure 4 shows the test learning curves for our summarization model(distilBBBRTTenn)) across 10 cycles of semi-supervised training. Each curve represents different settings in terms of numbers of labelled samples at the start. Every cycle, we add new pseulo-labelled samples generated with GPT-3.5 QA.
54
50
Test
#Labeled Samples
44
85
4
Cycles
Figure 4: Test curves for our method (using GPT-3.5 weak labeller and evaluator) when trained on 1, 8, and 85 samples (i.e., approximately 0. 11,% 1%, and 10% of the TWEETSUMM training data).


Reviews:
[{"review_id": "eh04VLzFsr", "reviewer": "Reviewer_Bx3Q", "summary": "", "questions": "1. Is there an oracle result for this TWEETSUMM dataset?\n2. Have the authors conducted the experiment that: 1) uses GPT-3.5 as the labeler and evaluator, and the PreSumm as the summarizer; 2) Use PreSumm as the labeler and summarizer (vanilla)? This is concerned as a missing part of the ablation study.\n3. Case Study is needed.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful to the reviewer for his insightful comments and questions. We have tried to address specific concerns and questions in a concise manner below\n\n**The method is not efficient enough**\nOur method is focused on data efficiency, as we try to replicate full data performance using only a fraction of it while maintaining the training efficiency.\nHaving said that since we are incrementally adding examples for training every cycle by selecting the most confident pseudo-samples and finetuning the summarizer in every cycle on the new composite dataset, it is important to train it for several epochs for it to learn from the samples and converge. It is a standard process while training any generative models **[2]**, as well as semi-supervised learning **[1]**.\n \n**‘Using GPT-3.5 as the labeler and evaluator does not largely improve, requires further analysis’**, It’s not clear what the improvement is compared against.  In Table 3 we show that our method using GPT as a weak labeler and evaluator improves upon the state of the art(PreSumm) on the tweetsumm Dataset using only 10% of the annotated dataset, which is a significant improvement as you rightly noted in the main contributions. It is also ~4 points higher than the BART semi-supervised model. \n\n**Authors wrongly used GPT-3 API (text-davinci-003 is claimed in the paper) rather than GPT-3.5 (gpt-3.5-turbo)**\n* Text-davinci-003 belongs to the instructGPT family of models which is an instruction-tuned iteration of GPT-3. It is thus a part of the GPT-3.5 family of APIs as can be verified from the openAI page **[3]**, while davinci is a part of GPT3. \n* Gpt-3.5-turbo is a chat-optimized version of it. We used text-davinci-003 because it consistently outperformed gpt-3.5-turbo in a few shot settings using prompted direct inference. \n* Also, it provides us with token probabilities that we use to get a confidence score as explained in section 2.2.2, which is not available for gpt-3.5-turbo \n\n**GPT-3.5 is a generative model, which does not guarantee extractive summarization.** As described in section 2.2(lines 134-136)  and demonstrated in appendix A.1.1, a major ingenuity of the work is in how we guarantee that the GPT pseudo labels are extractive by mapping the summarization as a QA task making it generate sentence numbers that it feels describe the issues and solutions. We then use these numbers to extract the specific sentences thus forming our summary. Also, we observed zero cases during testing where the model generated hallucinations/sentence numbers outside the document size, but we had fallbacks in place to handle those, such as discarding unrealistic sentence numbers and falling back on the LEAD heuristic in case of malfunction.\n\n**On the Questions**\n* To the best of our knowledge, and based on an exhaustive literature review around this dataset, there doesn’t exist an oracle score for TweetSumm.\n* Tuning parameters for GPT-3.5(temperature 0.2, top_p=1, max_tokens=48 with a logit bias for numeric tokens)  and distillbart will be added.\n* Adapting the available implementation of PreSumm for tweetsumm dataset is non-trivial, and given the limited scope of a short paper, we thus only used it as a baseline. But the suggested experiments by the reviewer are indeed interesting ablations that we will add to the final version of the paper. \n* It is unclear what is meant here by case study.\n\n**References**\n\n[1] Jiaao Chen and Diyi Yang. 2021. Simple conversational 376 data augmentation for semi-supervised abstractive 377 dialogue summarization. In Proceedings of the 2021 378 Conference on Empirical Methods in Natural Lan- 379 guage Processing, pages 6605–6616.\n\n[2] Sam Shleifer and Alexander M Rush. 2020b. Pre- 492 trained summarization distillation. arXiv preprint 493 arXiv:2010.13002\n\n[3] https://platform.openai.com/docs/models/gpt-3-5", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper proposes a novel approach to extractive summarization using LLMs.\n\n2. The paper is easy to follow.\n\n2. The proposed method demonstrates its effectiveness on the TWEETSUMM dataset while using 10% of the labeled data.", "reasons_to_reject": "1. The method is not efficient enough. It needs to train the model for ten iterations, and in each iteration, there are ten epochs of training. In Table 3, using GPT-3.5 as the labeler and evaluator does not largely improve the performance; this requires further analysis.\n2. Writing could be improved, having some typos and format issues. I am doubtful that the authors wrongly used GPT-3 API (text-davinci-003 is claimed in the paper) rather than GPT-3.5 (gpt-3.5-turbo)\n3. GPT-3.5 is a generative model, which does not guarantee extractive summarization.", "questions_to_authors": "1. Is there an oracle result for this TWEETSUMM dataset?\n2. Have the authors conducted the experiment that: 1) uses GPT-3.5 as the labeler and evaluator, and the PreSumm as the summarizer; 2) Use PreSumm as the labeler and summarizer (vanilla)? This is concerned as a missing part of the ablation study.\n3. Case Study is needed.", "paper_topic_and_main_contributions": "This paper proposes a method for extractive summarization of customer-agent dialogs, using LLMs to generate pseudo-labels for a dialog. The main contribution is a novel use of pseudo-labels for semi-supervised learning, which allows for the transfer of knowledge from the large LLM into a smaller specialized model. The paper demonstrates the effectiveness of this method on the TWEETSUMM dataset and shows that it achieves a good performance while using 10% of the labeled data.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. Lines 326 & 532, text-davinci-003 is GPT-3, not GPT-3.5\n2. Lines 570 & 310, \"pseudolabelling\" -> \"pseudo-labelling\"\n3. Both \"labeled\" and \"labelled\" appear multiple times, the writing is inconsistent.\n4. Line 057, \"we decided\" --> \"we decide\", the tense should be consistent with the context.\n5. Line 070, \"our methods have\" --> \"our method has\""}}, {"review_id": "G6zJ5JNKJI", "reviewer": "Reviewer_2gVZ", "summary": "", "questions": "- Are the notations in Sec 2.1 correct? I assume that gold summaries are given to a sequence of utterances, but the notations in the paper assume that a summary is given to each utterance (u_{i}).\n\n- In the equation written in line160-161, this modelling is for abstractive summarization not for extractive, thus, this model cannot guarantee extractive output. If my understanding is wrong, please point it out.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are thankful to the reviewer for their insightful review, we also acknowledge the reviewer's detailed explanation and close reading, highlighting notational and style issues. We elaborate on some of the points raised below.\n\n**The synthesized dataset is not guaranteed for a purely extractive setting**: As described in section 2.2 (lines 134-136)  and demonstrated in appendix A.1.1, a major ingenuity of the work is in how we guarantee that the GPT pseudo labels are extractive by mapping the summarization as a QA task making it generate sentence numbers that it \"thinks\" should describe the issues and solutions. We then use these numbers to extract the specific sentences thus forming our summary. Also, we observed zero cases during testing where the model generated hallucinations/sentence numbers outside the document size, but we had fallbacks in place to handle those, such as discarding unrealistic sentence numbers and falling back on the LEAD heuristic in case of malfunction. We include a discussion in how we handle these exceptions, were they to happen, in the camera ready if our paper is accepted for publication. \n\n**The authors trained a generative model although the authors focus on an extractive setting. This modeling does not guarantee the output summary is extractive.**\n* Generative models such as BART-large performed comparably to encoder-based classifier models for extractive summarization in our experiments,\n* Thus, our hypothesis was that given a dataset where the ground truth summaries always contain sentences from within the document, the model will learn representations in a way to output extractive summaries. \n* We obtained both qualitative and quantitative results to highlight it. \n* _However, the point raised by the reviewer regarding the lack of such a guarantee is still valid_ and we added an additional step to ensure am extractive output. We took the output from the BART model and performed a per-sentence match with the dialog using sentence transformers dot score, to extract the closest matching sentence from the dialog to make sure it's always extractive. Adding this additional step guarantees extractive summarization.\n* As an ablation, we shall be adding experiments with PreSumm, which is an extractive model as the summarizer instead of BART in the final version of the paper. \n\n**On the Questions**\n* Yes the notations are correct, but I think our wording is confusing, by utterance (u_{i}) here, we mean a dialogue i.e. set of utterances. We shall fix it, thanks for the catch.\n* Based on the above point, we will add an additional step to represent matching on top of abstractive summarization in the equation, along with a section on preSumm.\n\n**Regarding the point about novelty**, we might not have been clear but would like to stress that this is not merely a data augmentation technique. We show here that knowledge from generic high-parameter models trained on billions of tokens can serve as a useful starting point for smaller specialized models distilling their knowledge before finetuning in a very specific use-case thus helping in low-data and no-data scenarios. We also show the effectiveness of these models as a good substitute for human annotators, lowering costs while improving performance. It also lays the foundation for an incremental semi-supervised method for summarization, which can be extended to using open-source models, enabling even better results through fine-tuning, and improving the quality of pseudo-labels every cycle.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This paper is well-written and easy to follow.\n- The experiments show that the performance of the model trained on the proposed pseudo dataset is comparable to the model trained on the human-authored dataset.\n- Straightforward and reasonable usage of GPT3.5.", "reasons_to_reject": "- The synthesized dataset is not guranteed for a purely extractive setting.\n (i.e., the authors used ``You should choose at least two sentences and at most four sentences in total'' for the prompt, but GPT3.5 is a language generator, thus, the produced dataset is not guaranteed to contain only extracted sentences.)\n\n- The authors trained a generative model although the authors focus on an extractive setting. This modeling does not guarantee  the output summary is extractive.", "questions_to_authors": "- Are the notations in Sec 2.1 correct? I assume that gold summaries are given to a sequence of utterances, but the notations in the paper assume that a summary is given to each utterance (u_{i}).\n\n- In the equation written in line160-161, this modelling is for abstractive summarization not for extractive, thus, this model cannot guarantee extractive output. If my understanding is wrong, please point it out.", "paper_topic_and_main_contributions": "This paper investigates the use of an instruct-tuned model, GPT3.5 for data augmentation in an extractive dialogue summarization setting. The authors propose to automatically obtain summary sentences from dialogue by GPT3.5. Then a log likelihood-based filtering method is applied to filter instances with low confidence. Finally, the synthesized summary sentences are used for training a generative summarizer.\n\nAs a strength, The experiments show that the performance of the model trained on the proposed pseudo dataset is comparable to the model trained on the human-authored dataset.\n\nThis paper is easy to follow, however, some notations in Sec. 2.1 does seem not clear (see the questions for authors.).\nAlthough the authors claim that they focus on the extractive setting, the trained model is not guaranteed to be an extractive model (see the weakness below.).\nOverall, this paper proposes a straightforward usage of GPT3.5 and comprehensive evaluations provide insights for the community, However, the proposed idea is a simple replacement of existing data augmentation techniques by a model with few-shot ability, which may provides less novelty.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- \\bf{Related Work} -> \\section{Related Work}."}, "extra_text": {}}, {"review_id": "TV984a3SYF", "reviewer": "Reviewer_P49m", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would first like to thank the reviewer for their insightful comments, and for finding our work interesting.\n\nWe agree with the reviewer's point regarding the use of a closed-source model. Not being able to use an open-source model largely has to do with logistic bottlenecks such as lack of computing resources on our end, and stable and accurate open-source models.\n\nWith open-source models increasingly inching closer to the performance of closed-source models on a range of tasks and benchmarks, the claims and results of our paper should be reproducible with open-source models (e.g., Llama-2). They will also provide an additional advantage of low cost and the ability to fine-tune the models or prompts to generate higher quality pseudo labels every cycle as mentioned by the reviewer, thus in fact leading to better results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper proposes a new method of efficiently using unlabeled data on chat dialogs summarization task and it is hopefully popularized to other tasks which have a need for a large number of labeled data.", "reasons_to_reject": "The method proposed by this paper uses a closed source model to form pseudo labeled data, which means it may be difficult to fine-tuning this model for a higher quality of pseudo labeled data. And experiments using this method on open-source models have not been successful, making it difficult to prove that this method can be extended to other language processing tasks.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a new method that can effectively use unlabeled data to generate high-quality summaries for chat dialogs. This method frames summarization as a question-answering problems and use a large language model to generate summaries for unlabeled data to form pseudo labeled data and fine-tunes the summarization model to generate summaries using pseudo labeled data. Overall, the model achieved results similar to state-of-the-art models using only 10% of data.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The three papers propose methods for extractive summarization of customer-agent dialogs using large language models (LLMs) to generate pseudo-labels for dialog data. The paper's main contribution is the novel use of pseudo-labels for semi-supervised learning, which enables knowledge transfer from the LLMs to specialized models. They demonstrate the effectiveness of their methods on different datasets, achieving good performance with minimal labeled data. However, several concerns about the efficiency of the methods, and uncertainty about guaranteeing extractive summarization in some cases need to be addressed. Overall, these papers propose innovative techniques for dialogue summarization, but certain limitations and clarity issues require further revision.
Decision: Accept-Findings