## 2 Method
### Local Linear Embedding Method  
In manifold learning, LLE methods aim to maintain linear relationships between samples within a neighborhood. High-dimensional data often represent complex real-world objects, such as gene expressions or image pixels. After these data are reduced by LLE, the distribution in the low-dimensional space reflects the relative positions and similarity relationships of the original high-dimensional data, thereby preserving both the local and global structure. Figure 3 illustrates the process of LLE mapping 3D data (B) to 2D data (C). The data points in the low-dimensional space maintain the structure of the high-dimensional data, demonstrating the effectiveness of LLE. Figure 4 demonstrates that data points in the low-dimensional space should maintain the reconstruction relationship of the original high-dimensional data [17]. The \"reconstruction relationship\" means that, through the process of dimensionality reduction, the data points in the low-dimensional space can \"reconstruct\" or \"reflect\" the data structure of the original high-dimensional space to some extent. This involves retaining enough information in the low-dimensional space to recover important features of the high-dimensional data. If the dimensionality reduction is successful, similar sample points in the high-dimensional space should also remain similar in the low-dimensional space, effectively extracting and representing the key information of the data. Figure 4: The samples in the high-dimensional space maintain reconstruction relations in the low-dimensional space. Figure 3: Mapping of high-dimensional data to low-dimensional data. (**a**) Raw data structure; (**b**) Three-dimensional data; (**c**) Two-dimensional data. Based on the above theory, a hypothesis is made: \"using the LLE algorithm as the loss function trained before model pruning plays a positive role in reducing the loss of model performance in the pruning process\". According to this assumption, the LLE method is used as the loss function during model pruning training, and the specific calculation steps are as follows:  
1. The weight parameter of the filter in each layer of the network before the model is pruned is taken as the sample point, and the k nearest neighbor points of each sample point are calculated by the KNN strategy; meanwhile, the \(k\) sample points with the closest Euclidean distance relative to the sample point are specified as the \(k\) nearest neighbors of the sample point, and \(k\) is a pre-given value. 2. Calculate the local reconstruction weight matrix \(\alpha\) of the sample points and use the mean squared error as the loss function of the regression problem. The calculation formula of mean squared error is shown in Equation (12). \[J(\omega)=\sum_{i=1}^{N}\Big{|}\Big{|}w_{i}-\sum_{j\in Q(i)}\alpha_{ij}w_{j} \Big{|}\Big{|}_{2}^{2} \tag{12}\]  
where the sample points \(w_{i}\) and \(w_{j}\) represent the weight parameters before the model is pruned, \(w_{j}\) is the jth neighbor of \(w_{i}\), \(a_{ij}\) is the weight coefficient of the linear relationship between the sample point \(w_{i}\) and its neighbor \(w_{j}\), and \(Q(i)\) represents the set of \(k\) neighbors of \(w_{i}\). The weight coefficient \(a_{ij}\) is normalized, and the weight coefficient needs to satisfy the condition in Formula (13). \[\sum_{j\in Q(i)}\alpha_{ij}=1 \tag{13}\]  
For sample \(w_{i}\) that is not in the neighborhood of sample \(w_{j}\), let the corresponding \(\alpha_{ij}=0\). The weight coefficients of the linear relationship between \(w_{i}\) and its neighbors can be determined by Formulas (12) and (13). Then, the above optimization problems are solved by the Lagrangian multiplier method, and Formula (12) is obtained by matriization.