## 1 Introduction
Then, to further improve the interference mitigation performance, dilated convolution and designated contrast learning strategy were utilized for network construction and training, respectively [22]. By integrating the data-driven properties of DL and the advantages of compressed sensing (CS)-based interpretable optimization, an efficient feature-oriented unsupervised adaptive network called FUAS-Net was presented to suppress mutual interference [23]. The transformer is a powerful and promising technology in signal processing [24]. Transformer-based networks are especially adept at capturing repetitive patterns  
Figure 1: The scenario in which mutual interference occurs. with extended dependencies, which are well suited for reconstructing the periodic target signals from the interfered signal. In preliminary work [25], the authors developed a novel end-to-end transformer-based interference mitigation structure named RIMformer, which performs well in interference cancellation and target signal reconstruction. However, most DL-based approaches have a common drawback. The architectures of deep neural networks tend to be deeper and wider in order to obtain better results, which places high demands on the computational and memory costs for inference. The high memory and computational costs hinder the deployment of deep neural networks on resource-constrained devices. The deployment of DL models on edge computing systems poses a significant burden. Real interference mitigation tasks require a balance between performance and processing speed. The lightweight design of models is necessary to achieve decent resource requirements and retain a high interference mitigation performance. Model compression and acceleration of deep neural networks are needed to satisfy the real-time constraints of real-world tasks. Lightweight networks such as MobileNet [26] and ShuffleNet [27] have been developed for convolutional neural networks (CNNs). For transformer networks, reducing the complexity of attention computations is a straightforward approach to minimizing computing cost while maintaining high recognition accuracy [28]. To tackle the extensive computation demands in self-attention layers, long-short-range attention (LSRA) [29] was introduced to extract global and local features separately. The Reformer [30] adopts locality-sensitive hashing to substitute dot product operations in attention, which reduces the computation requirements. The Linformer [31] decomposes self-attention into multiple low-rank matrices via linear projection, and the complexity is decreased from \(O(n^{2})\) to \(O(n)\). Knowledge distillation (KD) is also an effective model compression technique aimed at transferring knowledge from a more extensive network to a smaller one [32]. KD algorithms are further improved through alternative loss functions like contrastive-based loss [33] and minimizing the maximum mean discrepancy between models [34]. A direct distillation approach [35] was proposed to leverage the teacher's classifier and align intermediate features with an L2 loss. In [36], a teacher-free KD employed self-training and manually designed regularization terms to improve the student model. On the basis of our preliminary work [25], we introduce a lightweight transformer based on linear attention and depthwise separable convolution to mitigate interference and restore target signals. In order to address the performance loss caused by network size reduction, a KD strategy is applied by combining both feature-based and response-based knowledge, which can significantly improve the performance of the student network. It is observed that the effectiveness of KD decreases dramatically when there is a substantial disparity in complexity between the teacher and student models. To overcome this drawback, a medium-sized assistant model is utilized to bridge the gap between the teacher and student models, which facilitates more effective knowledge transfer. The main contributions of this paper are as follows:  
1.