## 2 Related Work
Many studies use deep learning technology to obtain important information from text data, so that the machine can understand the text and generate a text-related summary. Therefore, many recent studies have applied pre-training models to text summarization. [PERSON] [36] proposed a framework built using BERT. The framework can provide extractive and abstractive types of text summarization. The experimental results on three data sets (CNN/DailyMail, NYT and XSum) show that their model achieves the state-of-the-art result under the automatic and human-based evaluation protocol. [PERSON] [37] used the BERT model and GPT-2 model to conduct text summarization on the open research data set of COVID-19 and extracted the keywords and semantic meaning of the sentences from the original text data through the above-mentioned pre-trained model. Their system provides a concise and meaningful summary for the medical literature. [PERSON] [38] used the pre-trained Transformer language model to generate summaries from the original text data. Compared with the pre-trained Transformer encoder-decoder network, this model has significantly improved results. [PERSON] [39] proposed a pre-trained model HIBERT (Hierarchical Bidirectional Encoder Representations from Transformers) for pre-training on text data and unlabeled data. This model can achieve higher results on the CNN/Dailymail data set and the New York Times data set. Therefore, the experimental results show that the use of pre-trained models can make large open data sets acquire better results. [PERSON] [40] mentioned two approaches to deal with the task of text summarization using mT5 model and ParsBERT model and got good results on a data set named pn-summary for Persian abstractive text summarization. [PERSON] [41] proposed a pre-trained model T-BERTSum for text summarization, which captures the key words of the topic information of social media, understands the meaning of the sentence and judges the topic of the message discussion, and then generates a high-quality section Summary. [PERSON] [42] uses BERT, Seq2 seq and reinforcement learning to form a text summary model. [PERSON] [43] uses T5, one of the most advanced pre-trained models, to perform a summary task on a data set with 80,000 news articles, and the results indicate that the summary generated by T5 has better quality than those generated by other models. [PERSON] [44] has developed a pre-trained language model ENEMAbst that can be used in extractive and abstractive summarization techniques. Both abstractive summaries and extractive summaries on the CNN/DailyMail dataset obtain the most excellent results. [PERSON] [45] proposed a pre-trained language model that can be used for abstractive text summarization, and combined with the use of BERT to experiment on the German summarization data set. After performing the contextual word embedding task through BERT, the model can improve the model's ability to generate high-quality German summaries. In Table 1, some important related work is shown for comparison. In summary, in our work, social media (such as Twitter), pre-trained models, and text summaries (including extractive and abstractive methods) are the three key elements of event summarization. From the above literature review, the text summaries of specific event tweets on social media have gradually become one of the most popular summarization research topics in recent years [1; 2; 3; 41; 42]. However, extractive summarization cannot obtain many key sentences in the tweet-form data, resulting in unsatisfactory quality of the generated summaries. Furthermore, due to our task requires extensive domain knowledge beyond the meaning of individual words and sentences, in this work we limit our focus to abstractive summarization algorithms. On the other hand, in the recent references of text summarization, there are more and more work using pre-training language models [35; 36; 37; 38; 39; 40; 41; 42; 43; 44]. Among them, BERT is the pre-trained model mainly used by most research teams.