## 1 Introduction  
Users encounter news of various fields through a variety of platforms due to the development of social media. Unlike how the press and TV networks used to dominate news broadcasting, users of social media can directly participate in the news while the users quickly propagate the news due to the high accessibility and speed. Furthermore, fake news is propagated more quickly. The propagation of fake news inflicts significant damage to news consumers [1]. Individuals who produce fake news have been producing fake news for various purposes, such as political purposes, economic benefits, and so on. It is difficult to detect such fake news because they are generally written in exactly the same format as true news [2]. Accordingly, countries such as the United States and the United Kingdom have been paying attention to research on fake news detection [3], and fact-checking websites have emerged as a part of such research efforts [4]. The authenticity of the news is judged through fact-checking websites by verifying through the press. There are several fact-checking websites being operated overseas, but it is very insufficient compared to the amount of fake news being created. In spite of the insufficiency, research is ongoing on fake news detection using the data provided by fact-checking websites [5, 6]. On the other hand, in Korea, about 10 fact-checking websites, including SNU FactCheck ([[https://factcheck.snu.ac.kr/](https://factcheck.snu.ac.kr/)]([https://factcheck.snu.ac.kr/](https://factcheck.snu.ac.kr/)) (accessed on 15 February 2022)) and FactChecknet ([[https://factcheck.snu.ac.kr/](https://factcheck.snu.ac.kr/)]([https://factcheck.snu.ac.kr/](https://factcheck.snu.ac.kr/)) (accessed on 15 February 2022)), are operated, and research on fake news detection has a serious problem of a lack of data because the number of verifiedfact-checking websites is fewer and the amount of usable data is also inadequate. Hence, the frequently used fake news detection model is based on the term frequency-inverse document frequency (TF-IDF) [7]. The similarity of documents is checked using the frequency of words appearing in a document, which has the limitation of being incapable of reflecting the contextual information of sentences. Afterwards, various fake news detection models reflecting the contextual information of a document based on embedding techniques have been proposed [8; 9]. The performance of fake news detection has drastically improved recently due to the development of language models such as BERT, which undergoes pre-training based on large-scale data [10]. Due to advancements in BERT, most research on fake news detection was conducted by embedding entire documents [11]. When embedding the entire document of fake news, however, discriminating fake news from real news becomes challenging because fake news maliciously includes truthful sentences [12]. In addition, embedding the entire document leads to performance degradation of a model since fake news is created so as to appear as real as possible. In particular, fake news detection in a specific category may not demonstrate a performance improvement in the BERT model due to insufficient data. With the recent emergence of COVID-19, fake news on COVID-19 is spreading faster than real news on social media and in a few press outlets. Fake news starting with shocking headlines can lead to serious results since infections, in particular, are directly associated with human lives [13]. Research on detecting fake news on COVID-19 is actively conducted overseas since there is an ample amount of usable data amid the COVID-19 outbreak [14; 15]. However, research is less actively conducted in Korea due to a lack of useful data. This paper, therefore, builds a dataset on COVID-19 fake news collected from multiple fact-checking websites and proposes a model for detecting fake sentences related to COVID-19.