# SSCL-TransMD: Semi-Supervised Continual Learning Transformer for Malicious Software Detection
## 1 Introduction  
In recent years, with the rapid development of digitization and the increasing convenience of internet technology, mobile devices such as smartphones have rapidly evolved. The Android system, due to its open source nature, has gradually become the mobile operating system with the highest market share. However, this also makes it the primary target for malicious software developers, which brings great harm to individual and enterprise data users. According to the \"2020 Android Platform Security Situation Analysis Report\" by Qi An Xin Threat Intelligence Center, a total of 2.3 million malicious Android program samples were intercepted in 2020, with an average of 6301 malicious program samples intercepted per day. According to the \"China Mobile Security Situation Report for the First Half of 2022\" released by 360 Anti-Phishing Center, approximately 10.797 million mobile malicious program samples were intercepted in the first half of 2022, a year-on-year increase of 180.5% compared to the first half of 2021. The rapid spread of malicious software has brought about a large number of harms, such as fee consumption, privacy leakage, and remote control, all of which mobile smartphone users have to bear. According to the \"2020 Android Platform Security Situation Analysis Report\", security governance of mobile internet is relatively weak globally and presents various means, such as online banking theft and Trojan problems, which posea great threat to users' property. Unlike other operating systems, the Android operating system allows users to obtain applications from third-party app stores, which brings convenience to users but also risks and hazards. The presence of unverified applications in third-party app stores increases the likelihood of users downloading malicious software. Although there have been continuous developments in technologies for malware detection, and significant progress has been made, malware developers are also utilizing the latest technologies to update their malicious software. Different categories of malware, such as Trojans, adware, and riskware, are constantly being developed. Therefore, efficient malware detection methods are essential. Deep learning, as a powerful tool for data pattern mining, has been widely applied in the field of malware detection. In the training of malware detection models, static datasets collected in advance are usually used for learning. This results in the inability of the models to detect unknown malicious behaviors. Therefore, predicting unknown malicious behaviors based on historical samples of malicious behaviors is a challenging task, and the ability to continuously learn new unlabeled malicious software behavior information is crucial for the lifespan of a malware detection system. In recent years, the Transformer model, known for its high degree of parallelism and scalability in parallel computing systems, has become a star in the field of deep learning. Many improved Transformer models have achieved excellent results in malware detection. Although these improved Transformer models have clear advantages, there are two pressing issues that need to be addressed for application in the field of malware detection: (1) Due to the continuous updates of malware data, the obtained samples of malicious software lack labels, which poses great difficulties in training malware detection models; (2) The fixed structure of deep learning neural networks determines the limited capacity of the models [1]. When neural networks learn new malicious software behaviors, they may forget historical data, leading to catastrophic forgetting [2]. To address catastrophic forgetting, one approach is to incorporate new samples into the historical training dataset and retrain the network using a dataset that contains both new and old training data. However, starting from scratch to train the model to adapt to newly generated malicious software data every day is time-consuming and highly inefficient. This paper makes the following contributions:  
1.