## 1 Introduction  
Fake information appears in a variety of forms, including videos, audio, images, and text. Furthermore, fake information in text form can be classified as news, social network services, speeches, documents, and so on. This study proposes a model for fake news detection by focusing on text-based fake news. Fake news has recently become a widespread problem worldwide. Fraudulent or falsified information can spread rapidly and become a problem if readers fail to detect, at a glance, whether or not the given information is fake news. In 2015, the International Fact-Checking Network (IFCN) was established by Poynter, an American media education agency. IFCN observes fact check trends and provides training programs for fact checkers. In addition, various efforts have been undertaken to prevent the spread of fake news by providing a code of principles that fact check organizations around the world can use. Politifact ([[https://www.politifact.com](https://www.politifact.com)]([https://www.politifact.com](https://www.politifact.com))) and snopes ([[https://www.snopes.com](https://www.snopes.com)]([https://www.snopes.com](https://www.snopes.com))) developed a Fake news detection tool to classify the level of fake news in stages based on the presented criteria. However, these tools are time-consuming and expensive as they require manual work and judgment. Therefore, a model that automatically detects fake news is required. There are several tasks involved in detecting fake news. Fake news challenge stage \(1\) (FNC-1) (www.fakenewschallenge.org) involves classifying the stance of a body text from a news article relative to a headline. The body text may agree, disagree, discuss or be unrelated to the headline. The Web Search and Data Mining (WSDM) 2019 fake news challenge (www.kaggle.com/c/fake-news-pair-classification-challenge) detects fake news by classifying the title of a news article. Given the title of a fake news article A and the title of a coming news article B, participants were asked to classify B into one of the three categories [1; 2; 3]. In addition, there are other tasks involved in fake news detection using Social Network Services (SNS) data. This task of the challenge (www.clickbait-challenge.org) is to develop a classifier that rates how clickbaiting a social media post is [4; 5; 6]. Finally, one of the tasks in the artificial intelligence research and development challenge (www.ai-challenge.kr) in Korea involves the detection of unrelated contextual news content in the news bodytext. This is done to prevent a situation in which the reader is exposed to unintended information. However, the results of these challenges and tasks do not exhibit good performance and can be improved. In addition, word embedding is an important factor for improving the performance of the model. word2 vec [7] and fastText [8] were previously used for word embedding. These do not exhibit good performance because they employ fixed vector values rather than fluid vector values for words. To complement this, we use contextual word embedding. Typical models include Embeddings from Language Models (ELMO) [9], BERT [10], and Generative Pre-Training (GPT) [11]. ELMO uses a bi-LSTM structure [12] and a feature-based approach that requires the application of many hyperparameters. The feature-based approach includes a pre-trained language representation of an additional feature of networks that perform specific tasks. On the other hand, BERT and GPT used a transformer structure and a fine-tuning approach to minimize the hyperparameters. The fine-tuning approach involves reducing the parameter of a specific task as much as possible and slightly changing the pre-trained parameters by training downstream tasks. We considered the above factors and applied fake news data to the BERT model, allowing us to better analyze the meaning of the news article. In this paper, we propose BAKE, an automatic fake news detection model that improves upon BERT by mitigating the data imbalance problem.