# Algorithm for Filling High Rank Matrix of Network Big Data Based on Density Peak Clustering

Authors: Deqiang Liu
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

The traditional filling method for network big data matrix has poor filling effect and suffers from noise. Therefore, a filling algorithm for network big data high rank matrix based on density peak clustering is proposed. The missing data are replaced by small-interval data, the information entropy of the high rank matrix of network big data is calculated, the density peak clustering algorithm is optimized through the cluster center selection strategy, the block data set is obtained through the unknown block method, and the block filling is realized by the host filling algorithm. Experimental results show that the filling accuracy of the proposed algorithm is as high as 0.895, and the loss rate is between 2% and 12%.


---

## Body

## 1. Introduction

Matrix filling is one of research hotspots in the fields of matrix analysis, optimization, image processing, means filling the missing elements accurately through known elements in the case of missing elements in the sampling matrix, and finally completing the sampling matrix [1]. In practice, the sampling matrix sometimes has special structures, such as symmetric matrix and Toeplitz matrix, which play an important role in communication engineering and power system, especially in the field of signal and image processing [2–4].There are two kinds of methods to deal with the problem of matrix rank minimization: one is to relax the rank function convex into the matrix kernel norm and establish the optimization model of the kernel norm; the second is to give the rank of the matrix in advance and establish a low-rank decomposition model [5]. Many domestic experts and scholars have also carried out a lot of researches on matrix filling and applied the research results to the fields of image processing, text analysis, and recommendation system [6].In the aspect of image restoration, the large singular values in the data matrix retain more characteristics of the original data, while the small singular values contain more noise. Matrix filling technology has been widely used in data analysis, recommendation system, image filling, video denoising, and machine learning [7, 8].The researches on big data require the involvement of a huge amount of information, which is usually collected and stored in daily life, but the process is carried out without supervision. Once external interference occurs, it will inevitably produce some missing data [9]. The collected data usually contain important information, and if it cannot be processed in time or improperly, there will be a serious impact on the real-time and effectiveness of the data, and even some wrong data information may appear, leading to users’ wrong decisions. In view of missing data, we need to fill in the data in time [10–15].Relevant scholars have studied this problem and achieved some results. Sun et al. [16] proposed the missing data filling algorithm based on the improved neural process [16], expressed the observed time series in a single way, obtained their respective characterization vectors by the neural network, obtained the distribution function of the data through the neural process model, introduced the correction coefficient in the training stage, determined the sampling rate of the training data more accurately according to the data missing rate, and estimated the missing value of the data through the trained model. The results show that this algorithm has good filling effect in the context of small data sets, but the filling accuracy of missing data is poor. Lin et al. [17] optimized K based on cuckoo algorithm—the missing data filling algorithm of means clustering [17]. By taking the training error of neural network as the fitness function, the weight and threshold of neural network are optimized, the weight of neural network through cuckoo search algorithm is calculated, and K-means clustering algorithm is adopted to realize the optimization of missing data threshold and the filling of missing data. This method has high missing data filling efficiency, but the data loss rate is unacceptable.This paper presents a high rank matrix filling algorithm for network big data based on density peak clustering. The specific steps are as follows.In the first step, density peak clustering is introduced, the local density is calculated based on cutoff kernel function, and the clustering center is obtained by comprehensively considering the local density value and minimum distance value of data points [18–22]; the second step is to classify the nonclustering center points, identify the abnormal points, replace the lost data with small-interval data, and calculate the information entropy of the filling data; in the third step, the distance measurement is obtained by the density peak clustering algorithm to realize data segmentation, and the complete sub-data set is obtained by the host filling method to complete the high rank matrix filling algorithm of network big data [23–28].The fourth step is to verify the effectiveness of the proposed method and draw a conclusion.


## 2. High Rank Matrix Filling Algorithm for Network Big Data

### 2.1. Big Data Information Entropy Calculation

Set the data with missing correlation toW=U,A, U represents the object set, and A represents the attribute set. If ui∈U, i=1,2,…U, attribute α∈A, that is, the interval data will be missing data fui,α, which can be represented by “∗.”Compared with a missing data set, if the missing data are replaced by the inter-cell, the information entropy of the large data set will increase significantly [29–36].On the big data setW, the attribute condition B∈A is met, and b1,b2,…,bN∈B, Uk∈U. Assuming that the m data bm is missing data, the data set Uk that has not been filled is called the original data, and the information entropy is calculated according to the formula as follows:(1)HλNA|0=−∑i=1U1Ulg1U∑j=1UPuiujbl∑l=1Nλlk,where Puiujbl represents the minimum multi-interval similarity between ui and uj, and λlk represents the correlation coefficient between the l index and the k index, both of which can be expressed as follows:(2)Puiujbl=fui,bl∩fuj,blfui,bl∪fuj,bl,λlk=λbk,bl∑l−1n−1λbk,bl+λbk,bk,where f⋅ represents the interval of missing data.Letbkm∗ be a small-interval data, replace the lost data bkm in the big data set with bkm∗, and at the same time, fill the Uk data set in the original big data.(3)fuU∗,bkm∗=fuU∗,bkm∗,fuU∗,bkm∗,fuU∗,bkm∗=fSuU∗,bkm∗−fXuU∗,bkm∗=δ,where S represents the lower limit of the interval and X represents the upper limit of the interval.The newly obtained large data set can be expressed according to(4)W∗=U∗,A∗.The information entropy of the newly obtained data set can be calculated by(5)HλNA∗|U=∑i=1U∗1U∗×lg1U∗∑j=1U∗∑i=1NλlkPuiujbl.bkm indicates that the missing data in the large data set are small-interval data, so ∑i=1U∗Puiujbl is almost equal to zero, and Puiujbl=1. That is, HλNA∗ is(6)HλNA∗=lgU∗−1U∗∑l=1Nλlk.Them data dhm in the original data Uk is missing data, and the length can be regarded as zero, while the original big data are λmkPujukbkm=0. It can be confirmed that the big data information entropy and the original data information entropy after filling Uk meet the following relationship:(7)HλNA∗|0>HλNA|0.By using relatively small-interval data to replace missing data and continuously expanding the interval range, the information entropy in certain ranges will continue to decrease [37–40]. The similarity relationship of the above-mentioned use interval expands the range of the interval. The greater the position relative to the newly filled data interval, the greater the possibility that the object will be classified into other object data sets [41, 42]. When the filling range of the data interval is too large, the data classification is abnormal, resulting in data confusion. When the newly filled interval range increases from the minimum to the maximum, there will be at least one entropy representing the minimum [43–47].


### 2.2. Determination of Density Peak Clustering Center

According to the calculated information entropy of filling big dataδi and ρi, the relationship between them is analyzed and the basis is put forward. The density peak clustering algorithm is optimized by using the clustering center selection strategy [48, 49]. According to the principle of cluster center selection, the difference degree of cluster points is measured by using normalized product of adjacent distance δi and density ρi. According to the statistical characteristics and change trend of the difference degree, the largest group of points is selected as the cluster center. After obtaining the cluster center, the network big data are divided into different clusters according to the adjacent distance label so as to realize clustering [50–52].In order to quantify the degree to which a data point of network big data is offset from the originδ and ρ after normalization, the cluster center weight is introduced according to the positive proportional relationship:(8)ωi=δiρi.In order to obtain the data point set with the largest deviation, the cluster center weights are arranged in the order from large to small. The firstN points are taken, and N is usually set to 30. Take the point with the greatest deviation from the origin as the inflection point of the overall downward trend of the cluster center weight from acute to slow.The downward trend of the weight of the survey center is described by the slope of the two-point line segment:(9)kiN=δi+N−δiN.In equation (9), kiN denotes the average change rate of cluster center weight within i,i+N range, which reflects overall change trend of a certain range ψ. Then, the inflection point can be described as(10)v=argmaxki1k1i−1.In equation (10), k1i−1 represents the slope from the first point to the i th point, which is the average change rate of point set 1,2,…,i; kil is used to describe the slope from the i th point to the i+1 st point.Based on the above analysis, the cluster center selection process is given:(1)
Calculate the difference degree of each network data pointψ.(2)
The cluster center weights are arranged in order from large to small.(3)
Calculatekil and k1i−1 as well as the maximum value of ki1/k1i−1, and determine the inflection point i=v.(4)
Take the network data point1,2,…,v before the inflection point as the cluster center point.


### 2.3. Unknown Block Calculation Method

Based on the understanding of the data set, the correlation missing data can be divided into two types: block known and block unknown. For known missing data, it can be directly divided into blocks by known information. In this case, there will be fewer variables, and the meaning of the variables is clear. In this paper, several data sets are used for experimental verification.In actual operation, most of the missing data sets are unknown blocks, especially in cloud computing with missing big data, which makes it difficult to distinguish block information. In the case that the block is unknown, this paper adopts the density peak clustering algorithm and uses the method of improving the distance measurement to achieve the block of missing data.The density peak clustering algorithm does not need to specify the number of categories in advance. The calculation method in the data set is as follows:Input: the number of clusters isA, and the data set is K.Output:αi clusters of all objects.Step 1: randomly selectK cluster centers, denoted by u1,u2,…,uk∈RP.Step 2: iterate until convergence.Calculate the cluster belonging to all objectsαi in A:(11)ci=argminjαi−uj2.Update the center of the class based on all the classesj:(12)uj=∑i=1n1ci=jai∑i=1n1ci=j.Before describing the block calculation, the following problems should be solved:(1)
The clustering method can be divided intoQ-type and R-type clustering. The K-means calculation method is actually the Q-type clustering method. R-type refers to relative variable clustering. To cluster the variables using the density peak clustering algorithm, the data set should be transferred to AT∈Rp×n first and then to cluster A′.(2)
Since the block processing effect is poor, when the sample size ofn is relatively small, AT after transposition will be an uncertain data set and density peak clustering algorithm has no limitations in clustering. However, when n is relatively large, AT is still a missing data after replacement, and the density peak clustering algorithm has certain limitations in clustering. In order to solve this problem, the sparse expression is used to select variables. The central idea is to constrain the weights of variables in the objective function, forcing variables with relatively small weights to not cluster, thereby retaining variables with large weights. In this way, the selection of variables is realized, and the objective function of the result is defined as follows:(13)ci=argminj∑u=1pwuαui−uju2,s.t.w2≤1,w1≤s,wu≥0∀.uwhere w represents the variable weight vector, wu represents the coefficient of the u variable weight, and s represents the adjusted parameter.For the method mentioned above, selection of variables is adopted to solve the limitation of the density peak clustering algorithm when the number is large.(3)
Compared with the classical clustering algorithm, the Euclidean distance is usually used to calculate the distance. When a large amount of data is missing, it is difficult to calculate the Euclidean distance. Therefore, it is necessary to define the distance between the missing objectsαi and αj:(14)dαi,αj=n∑u=1nIu∑u=1nαui,αuj2Iu.were αuj represents the value of the object αi on the u variable.(4)
For classical clustering methods, arithmetic averaging is usually used to update the cluster centers, but this method is not applicable when the data are missing. Therefore, the paper proposes the clustering centers in the case of missing data. Thes object α1,α2,…,αs in the j cluster includes the missing part; that is, the cluster center uj is taken in the i variable. The formula is(15)uji=s∑u=1sIiu∑u=1sαiuIiu,where Iiu=0,αiu=∗1,αiu≠∗, which is uj=uj1,uj2,…,ujp.After solving the above problems, the calculation method of KMB is given below:Input: data setA, number of clusters K.Output: block data set.Step 1: transposeA to obtain AT.Step 2: arbitrarily selectK cluster center points u1,u,2u,…,uK∈Rn.Step 3: iterate until convergence.Calculate the cluster belonging to all objectsαi in AT:(16)ci=argminjn2∑u=1nαui−uju2Iu∑u=1nIu22.Relative to all classesj, update the center of the cluster according to the definition in the text:(17)uj=uj1,uj2,…,ujp.Step 4: convert the cluster to complete the data set and obtainA1.Step 5: segment the data setA1 according to the clustering result to obtain a block data set.Block filling is a missing data filling method obtained by dividing the data set into blocks according to the characteristics of the data set. It is suitable for a wide range of data filling and relying on other variables, which is a big advantage. It is also called the host method that is widely used. Most of the traditional filling can be used in the host algorithm proposed in the article, except for mode filling and mean filling. However, some filling algorithms do not rely on variables, which meet the conditions as follows:Input: Missing data setS=U,A,V,f.Output: Complete data setS′=U′,A,V,f′.Step 1: Determine the data block. If the block is known, then it can be directly divided; if it is unknown, it needs to be divided by the KMB algorithm.Step 2: Through the block information. Segmentation of the missing data setS yields K sub-data sets RSi=U,Ai,V,f and i=1,2,…,K.Step 3: Compared with all missing data sets, the complete sub-data setSi=U′,Ai,V,f′, i=1,2,…,K can be obtained by using host filling.Step 4: Combine all the complete sub-data setsSi to obtain a complete data set S′=U′,A,V,f′.To block the initial data set and fill all blocks in parallel, the calculation time for filling should be reduced tomaxt1,t2,…,tk, where K represents the number of blocks and tk represents the filling time of K blocks. When the data set dimensions and data volume are large, the block filling effect is obvious.


## 2.1. Big Data Information Entropy Calculation

Set the data with missing correlation toW=U,A, U represents the object set, and A represents the attribute set. If ui∈U, i=1,2,…U, attribute α∈A, that is, the interval data will be missing data fui,α, which can be represented by “∗.”Compared with a missing data set, if the missing data are replaced by the inter-cell, the information entropy of the large data set will increase significantly [29–36].On the big data setW, the attribute condition B∈A is met, and b1,b2,…,bN∈B, Uk∈U. Assuming that the m data bm is missing data, the data set Uk that has not been filled is called the original data, and the information entropy is calculated according to the formula as follows:(1)HλNA|0=−∑i=1U1Ulg1U∑j=1UPuiujbl∑l=1Nλlk,where Puiujbl represents the minimum multi-interval similarity between ui and uj, and λlk represents the correlation coefficient between the l index and the k index, both of which can be expressed as follows:(2)Puiujbl=fui,bl∩fuj,blfui,bl∪fuj,bl,λlk=λbk,bl∑l−1n−1λbk,bl+λbk,bk,where f⋅ represents the interval of missing data.Letbkm∗ be a small-interval data, replace the lost data bkm in the big data set with bkm∗, and at the same time, fill the Uk data set in the original big data.(3)fuU∗,bkm∗=fuU∗,bkm∗,fuU∗,bkm∗,fuU∗,bkm∗=fSuU∗,bkm∗−fXuU∗,bkm∗=δ,where S represents the lower limit of the interval and X represents the upper limit of the interval.The newly obtained large data set can be expressed according to(4)W∗=U∗,A∗.The information entropy of the newly obtained data set can be calculated by(5)HλNA∗|U=∑i=1U∗1U∗×lg1U∗∑j=1U∗∑i=1NλlkPuiujbl.bkm indicates that the missing data in the large data set are small-interval data, so ∑i=1U∗Puiujbl is almost equal to zero, and Puiujbl=1. That is, HλNA∗ is(6)HλNA∗=lgU∗−1U∗∑l=1Nλlk.Them data dhm in the original data Uk is missing data, and the length can be regarded as zero, while the original big data are λmkPujukbkm=0. It can be confirmed that the big data information entropy and the original data information entropy after filling Uk meet the following relationship:(7)HλNA∗|0>HλNA|0.By using relatively small-interval data to replace missing data and continuously expanding the interval range, the information entropy in certain ranges will continue to decrease [37–40]. The similarity relationship of the above-mentioned use interval expands the range of the interval. The greater the position relative to the newly filled data interval, the greater the possibility that the object will be classified into other object data sets [41, 42]. When the filling range of the data interval is too large, the data classification is abnormal, resulting in data confusion. When the newly filled interval range increases from the minimum to the maximum, there will be at least one entropy representing the minimum [43–47].


## 2.2. Determination of Density Peak Clustering Center

According to the calculated information entropy of filling big dataδi and ρi, the relationship between them is analyzed and the basis is put forward. The density peak clustering algorithm is optimized by using the clustering center selection strategy [48, 49]. According to the principle of cluster center selection, the difference degree of cluster points is measured by using normalized product of adjacent distance δi and density ρi. According to the statistical characteristics and change trend of the difference degree, the largest group of points is selected as the cluster center. After obtaining the cluster center, the network big data are divided into different clusters according to the adjacent distance label so as to realize clustering [50–52].In order to quantify the degree to which a data point of network big data is offset from the originδ and ρ after normalization, the cluster center weight is introduced according to the positive proportional relationship:(8)ωi=δiρi.In order to obtain the data point set with the largest deviation, the cluster center weights are arranged in the order from large to small. The firstN points are taken, and N is usually set to 30. Take the point with the greatest deviation from the origin as the inflection point of the overall downward trend of the cluster center weight from acute to slow.The downward trend of the weight of the survey center is described by the slope of the two-point line segment:(9)kiN=δi+N−δiN.In equation (9), kiN denotes the average change rate of cluster center weight within i,i+N range, which reflects overall change trend of a certain range ψ. Then, the inflection point can be described as(10)v=argmaxki1k1i−1.In equation (10), k1i−1 represents the slope from the first point to the i th point, which is the average change rate of point set 1,2,…,i; kil is used to describe the slope from the i th point to the i+1 st point.Based on the above analysis, the cluster center selection process is given:(1)
Calculate the difference degree of each network data pointψ.(2)
The cluster center weights are arranged in order from large to small.(3)
Calculatekil and k1i−1 as well as the maximum value of ki1/k1i−1, and determine the inflection point i=v.(4)
Take the network data point1,2,…,v before the inflection point as the cluster center point.


## 2.3. Unknown Block Calculation Method

Based on the understanding of the data set, the correlation missing data can be divided into two types: block known and block unknown. For known missing data, it can be directly divided into blocks by known information. In this case, there will be fewer variables, and the meaning of the variables is clear. In this paper, several data sets are used for experimental verification.In actual operation, most of the missing data sets are unknown blocks, especially in cloud computing with missing big data, which makes it difficult to distinguish block information. In the case that the block is unknown, this paper adopts the density peak clustering algorithm and uses the method of improving the distance measurement to achieve the block of missing data.The density peak clustering algorithm does not need to specify the number of categories in advance. The calculation method in the data set is as follows:Input: the number of clusters isA, and the data set is K.Output:αi clusters of all objects.Step 1: randomly selectK cluster centers, denoted by u1,u2,…,uk∈RP.Step 2: iterate until convergence.Calculate the cluster belonging to all objectsαi in A:(11)ci=argminjαi−uj2.Update the center of the class based on all the classesj:(12)uj=∑i=1n1ci=jai∑i=1n1ci=j.Before describing the block calculation, the following problems should be solved:(1)
The clustering method can be divided intoQ-type and R-type clustering. The K-means calculation method is actually the Q-type clustering method. R-type refers to relative variable clustering. To cluster the variables using the density peak clustering algorithm, the data set should be transferred to AT∈Rp×n first and then to cluster A′.(2)
Since the block processing effect is poor, when the sample size ofn is relatively small, AT after transposition will be an uncertain data set and density peak clustering algorithm has no limitations in clustering. However, when n is relatively large, AT is still a missing data after replacement, and the density peak clustering algorithm has certain limitations in clustering. In order to solve this problem, the sparse expression is used to select variables. The central idea is to constrain the weights of variables in the objective function, forcing variables with relatively small weights to not cluster, thereby retaining variables with large weights. In this way, the selection of variables is realized, and the objective function of the result is defined as follows:(13)ci=argminj∑u=1pwuαui−uju2,s.t.w2≤1,w1≤s,wu≥0∀.uwhere w represents the variable weight vector, wu represents the coefficient of the u variable weight, and s represents the adjusted parameter.For the method mentioned above, selection of variables is adopted to solve the limitation of the density peak clustering algorithm when the number is large.(3)
Compared with the classical clustering algorithm, the Euclidean distance is usually used to calculate the distance. When a large amount of data is missing, it is difficult to calculate the Euclidean distance. Therefore, it is necessary to define the distance between the missing objectsαi and αj:(14)dαi,αj=n∑u=1nIu∑u=1nαui,αuj2Iu.were αuj represents the value of the object αi on the u variable.(4)
For classical clustering methods, arithmetic averaging is usually used to update the cluster centers, but this method is not applicable when the data are missing. Therefore, the paper proposes the clustering centers in the case of missing data. Thes object α1,α2,…,αs in the j cluster includes the missing part; that is, the cluster center uj is taken in the i variable. The formula is(15)uji=s∑u=1sIiu∑u=1sαiuIiu,where Iiu=0,αiu=∗1,αiu≠∗, which is uj=uj1,uj2,…,ujp.After solving the above problems, the calculation method of KMB is given below:Input: data setA, number of clusters K.Output: block data set.Step 1: transposeA to obtain AT.Step 2: arbitrarily selectK cluster center points u1,u,2u,…,uK∈Rn.Step 3: iterate until convergence.Calculate the cluster belonging to all objectsαi in AT:(16)ci=argminjn2∑u=1nαui−uju2Iu∑u=1nIu22.Relative to all classesj, update the center of the cluster according to the definition in the text:(17)uj=uj1,uj2,…,ujp.Step 4: convert the cluster to complete the data set and obtainA1.Step 5: segment the data setA1 according to the clustering result to obtain a block data set.Block filling is a missing data filling method obtained by dividing the data set into blocks according to the characteristics of the data set. It is suitable for a wide range of data filling and relying on other variables, which is a big advantage. It is also called the host method that is widely used. Most of the traditional filling can be used in the host algorithm proposed in the article, except for mode filling and mean filling. However, some filling algorithms do not rely on variables, which meet the conditions as follows:Input: Missing data setS=U,A,V,f.Output: Complete data setS′=U′,A,V,f′.Step 1: Determine the data block. If the block is known, then it can be directly divided; if it is unknown, it needs to be divided by the KMB algorithm.Step 2: Through the block information. Segmentation of the missing data setS yields K sub-data sets RSi=U,Ai,V,f and i=1,2,…,K.Step 3: Compared with all missing data sets, the complete sub-data setSi=U′,Ai,V,f′, i=1,2,…,K can be obtained by using host filling.Step 4: Combine all the complete sub-data setsSi to obtain a complete data set S′=U′,A,V,f′.To block the initial data set and fill all blocks in parallel, the calculation time for filling should be reduced tomaxt1,t2,…,tk, where K represents the number of blocks and tk represents the filling time of K blocks. When the data set dimensions and data volume are large, the block filling effect is obvious.


## 3. Experimental Study

This experiment uses the movie rating data information obtained from the Movie Lens data set, randomly selects the rating data of 1500 users for 3000 movies, and converts it into a 3000 × 1500 partial observable user-movie rating data matrixX. Given three cases of rank r=10,500,1000, the proportion of observable items is set to rate=0.1,0.3,0.5,0.8 in the three cases. Comparative analysis is conducted in terms of the iteration convergence time time, the number of iterations n, and the relative error ε=X−MF2/MF2 between the repair matrix and the original matrix. The results are shown in Figure 1.Figure 1
Line graph of the relationship between the ratio of observable items and the convergence error. (a)r = 10. (b) r = 500. (c) r = 1000.
(a)(b)(c)It can be seen from Figure1 that when the rank of the matrix to be processed is 10, with the increase of observable items, the relative convergence errors of the four filling algorithms all have a downward trend in a certain period of time. That is to say, when there are more observable items, the accuracy of the algorithm convergence is higher, and the singular value threshold truncation algorithm and the algorithm proposed in this paper have higher accuracy of convergence than the other two algorithms. It can be seen from Figure 1(b) that when the rank is 500, with the increase of observable items, the relative error accuracy of the proposed matrix filling algorithm is better than other algorithms in a certain period of time. As an improved method of accelerating the nearest neighbor gradient algorithm, the augmented Lagrangian algorithm has better convergence rate and accuracy. It can be seen from Figure 1(c) that when the rank is 1000, under the condition of less observable items, the accuracy of augmented Lagrangian algorithm is better than other algorithms. In the case of many observable items, the error convergence accuracy of the proposed matrix filling algorithm is significantly better than other algorithms.


### 3.1.d2 Comparison of Filling Accuracy

The filling accuracies of different methods are measured by taking two standards as indicators. One standard isd2, which is used to measure the degree of matching between the real value and the filling value.(18)d2=1−∑i=1nei−ri2∑i=1nei−E+ri−R2.According to Table1, it can be seen that for any combination of missing, the proposed algorithm is obviously higher than the other two algorithms. In addition, the more the missing data of correlation, the lower the d2 obtained by the other two methods. The filling accuracy will decrease with the missing rate of the data. However, the filling accuracy of the proposed algorithm has always been maintained at a very high level. In terms of d2, the proposed filling method is obviously higher than the other two algorithms as well.Table 1
d2 filling accuracy index.
CombinationAlgorithmMissing rate/%Missing patternProposed algorithmFIMUSDMI1Single0.8370.7480.734Multiple0.8190.7290.7233Single0.8950.7290.714Multiple0.8460.7070.6985Single0.8530.6940.683Multiple0.8430.6840.67410Single0.8670.6590.646Multiple0.8460.6380.618


### 3.2. RMSE Mean Analysis

The average error between the filled value and the true value is measured according to(19)RMSE=1n∑i=1nri−ei21/2.where n is the number of missing values, ri is the true value of the i missing value, ei is the filled value of the i missing value, R is the average value of ri, and E is the average value of ei, i=1,2,…,n, the meaning of the two standards. The larger the value of d2, the higher the filling accuracy. Conversely, the smaller the RMSE value, the higher the filling accuracy.As can be seen from Figure2, the filling accuracy of the proposed method is relatively stable, the data missing will be between 2% and 12%, the value will be above 0.8, and the RMSE value will be between 0.15 and 0.2. Compared with the single correlation missing, the filling accuracy of single missing mode will be significantly higher than that of multi-filling mode. Since the missing data of multi-fill pattern are relatively large, the interference of feature extraction and restoration is higher than that of single fill pattern. This proves that the proposed method has stronger stability and higher filling accuracy than the other two methods.Figure 2
RMSE average.


### 3.3. Analysis of Filling Effect of High Rank Matrix under Multiple Data Sets

In order to further verify the filling effect of high rank matrix of network big data, filling accuracy of density peak clustering method, means clustering method and improved neural process method are comparatively analyzed based on Google dataset search dataset (https://toolbox.google.com/datasetsearch), Google Trends dataset (https://trends.google.com/trends/explore), and EU open data portal dataset (https://data.europa.eu/euodp/en/data/), as shown in Tables 2–4.Table 2
High rank matrix filling accuracy of density peak clustering method.
Data volume (GB)High rank matrix filling accuracy (%)Google Dataset SearchGoogle TrendsEU Open Data Portal5092.6596.2898.3210097.2699.3299.0115098.9398.7199.8520097.6199.6396.16Table 3
Cuckoo algorithm optimizationK high rank matrix filling accuracy of means clustering method.
Data volume (GB)High rank matrix filling accuracy (%)Google Dataset SearchGoogle TrendsEU Open Data Portal5053.1558.8853.6310057.2360.1262.1015060.0256.2656.0920055.9653.1752.61Table 4
High rank matrix filling accuracy of improved neural process method.
Data volume (GB)High rank matrix filling accuracy (%)Google Dataset SearchGoogle TrendsEU Open Data Portal5053.2649.8249.3210055.1250.3359.1615060.9856.2948.2220059.9058.3666.10According to Tables2–4, the highest filling accuracy of means clustering method under Google dataset search, Google Trends, and EU open data portal datasets is 60.02%, 60.12%, and 62.10%, respectively. In contrast, the highest filling accuracy of the improved neural process method under Google dataset search, Google Trends, and EU open data portal datasets is 60.98%, 56.29%, and 66.10%, respectively. The highest filling accuracy of the improved neural process method under Google Dataset Search, Google Trends, and EU Open Data Portal data sets is 60.98%, 56.29%, and 66.10%, respectively. The highest filling accuracy of density peak clustering method under Google Dataset Search, Google Trends, and EU Open Data Portal data sets is 98.93%, 99.63, and 99.85%, respectively. The above data show that the density peak clustering method has higher filling accuracy as it uses small-interval data to replace the lost data. By optimizing the density peak clustering algorithm through the cluster center selection strategy, obtaining the block data set through the unknown block method, and realizing the block filling by using the host filling algorithm, the filling noise of the network big data matrix can be effectively avoided and the filling effect can be improved.


## 3.1.d2 Comparison of Filling Accuracy

The filling accuracies of different methods are measured by taking two standards as indicators. One standard isd2, which is used to measure the degree of matching between the real value and the filling value.(18)d2=1−∑i=1nei−ri2∑i=1nei−E+ri−R2.According to Table1, it can be seen that for any combination of missing, the proposed algorithm is obviously higher than the other two algorithms. In addition, the more the missing data of correlation, the lower the d2 obtained by the other two methods. The filling accuracy will decrease with the missing rate of the data. However, the filling accuracy of the proposed algorithm has always been maintained at a very high level. In terms of d2, the proposed filling method is obviously higher than the other two algorithms as well.Table 1
d2 filling accuracy index.
CombinationAlgorithmMissing rate/%Missing patternProposed algorithmFIMUSDMI1Single0.8370.7480.734Multiple0.8190.7290.7233Single0.8950.7290.714Multiple0.8460.7070.6985Single0.8530.6940.683Multiple0.8430.6840.67410Single0.8670.6590.646Multiple0.8460.6380.618


## 3.2. RMSE Mean Analysis

The average error between the filled value and the true value is measured according to(19)RMSE=1n∑i=1nri−ei21/2.where n is the number of missing values, ri is the true value of the i missing value, ei is the filled value of the i missing value, R is the average value of ri, and E is the average value of ei, i=1,2,…,n, the meaning of the two standards. The larger the value of d2, the higher the filling accuracy. Conversely, the smaller the RMSE value, the higher the filling accuracy.As can be seen from Figure2, the filling accuracy of the proposed method is relatively stable, the data missing will be between 2% and 12%, the value will be above 0.8, and the RMSE value will be between 0.15 and 0.2. Compared with the single correlation missing, the filling accuracy of single missing mode will be significantly higher than that of multi-filling mode. Since the missing data of multi-fill pattern are relatively large, the interference of feature extraction and restoration is higher than that of single fill pattern. This proves that the proposed method has stronger stability and higher filling accuracy than the other two methods.Figure 2
RMSE average.


## 3.3. Analysis of Filling Effect of High Rank Matrix under Multiple Data Sets

In order to further verify the filling effect of high rank matrix of network big data, filling accuracy of density peak clustering method, means clustering method and improved neural process method are comparatively analyzed based on Google dataset search dataset (https://toolbox.google.com/datasetsearch), Google Trends dataset (https://trends.google.com/trends/explore), and EU open data portal dataset (https://data.europa.eu/euodp/en/data/), as shown in Tables 2–4.Table 2
High rank matrix filling accuracy of density peak clustering method.
Data volume (GB)High rank matrix filling accuracy (%)Google Dataset SearchGoogle TrendsEU Open Data Portal5092.6596.2898.3210097.2699.3299.0115098.9398.7199.8520097.6199.6396.16Table 3
Cuckoo algorithm optimizationK high rank matrix filling accuracy of means clustering method.
Data volume (GB)High rank matrix filling accuracy (%)Google Dataset SearchGoogle TrendsEU Open Data Portal5053.1558.8853.6310057.2360.1262.1015060.0256.2656.0920055.9653.1752.61Table 4
High rank matrix filling accuracy of improved neural process method.
Data volume (GB)High rank matrix filling accuracy (%)Google Dataset SearchGoogle TrendsEU Open Data Portal5053.2649.8249.3210055.1250.3359.1615060.9856.2948.2220059.9058.3666.10According to Tables2–4, the highest filling accuracy of means clustering method under Google dataset search, Google Trends, and EU open data portal datasets is 60.02%, 60.12%, and 62.10%, respectively. In contrast, the highest filling accuracy of the improved neural process method under Google dataset search, Google Trends, and EU open data portal datasets is 60.98%, 56.29%, and 66.10%, respectively. The highest filling accuracy of the improved neural process method under Google Dataset Search, Google Trends, and EU Open Data Portal data sets is 60.98%, 56.29%, and 66.10%, respectively. The highest filling accuracy of density peak clustering method under Google Dataset Search, Google Trends, and EU Open Data Portal data sets is 98.93%, 99.63, and 99.85%, respectively. The above data show that the density peak clustering method has higher filling accuracy as it uses small-interval data to replace the lost data. By optimizing the density peak clustering algorithm through the cluster center selection strategy, obtaining the block data set through the unknown block method, and realizing the block filling by using the host filling algorithm, the filling noise of the network big data matrix can be effectively avoided and the filling effect can be improved.


## 4. Conclusion

This paper presents a high rank matrix filling algorithm for network big data based on density peak clustering. The density peak clustering is introduced, the missing data are replaced by small-interval data, and the information entropy of filled data is calculated. Combined with the density peak clustering algorithm and the improved distance measurement in the case of missing data, the missing data are partitioned, and the host filling can be used to obtain a complete sub-data set. The following conclusions are drawn through experiments:(1)
When the rank is 500, with the increase of observable terms, the relative error accuracy of the matrix filling algorithm proposed in this paper is better than that of other algorithms. When there are many observable items, the error convergence accuracy of the proposed matrix filling algorithm is obviously better than that of other algorithms as well.(2)
For any missing combination, the proposed algorithm is obviously higher than the other two algorithms. In addition, in the case of more correlation missing data, the filling accuracy of the proposed method is always stable.(3)
The filling accuracy of the proposed method is relatively stable, with missing data ranging between 2% and 12%,D2 value more than 0.8, and the RMSE value between 0.15 and 0.2. The filling accuracy of single missing mode will be significantly higher than that of multi-filling mode, because the missing data of multi-filling mode are relatively large, and the interference to feature extraction and restoration is higher than that of single missing mode.



---

*Source: 8662238-2022-06-25.xml*