# Local Gravitation Clustering-Based Semisupervised Online Sequential Extreme Learning Machine

Authors: Xingbiao Wang; Bin Gu; Quanyi Zou; Rui Lei
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Due to the limited number of labeled samples, semisupervised learning often leads to a considerable empirical distribution mismatch between labeled samples and unlabeled samples. To this end, this paper proposes a novel semisupervised algorithm named Local Gravitation-based Semisupervised Online Sequential Extreme Learning Machine (LGS-OSELM), learning to unlabeled samples follows from easy to difficult. Each sample is formulated as an object with mass and associated with local gravitation generated from its neighbors. The similarity between samples is measurable by the local gravitation measures (centrality CE and coordination CO). First, the LGS-OSELM uses the labeled samples to learn the initialization model by implementing ELM. Second, the unlabeled samples with a high confidence level that is easy to learn are labeled with the pseudo label. Then, these samples are utilized to iterate the neural network by implementing OS-ELM. The proposed approach ultimately realizes effective learning of all samples through successive learning unlabeled samples and iterating neural networks. We implement experiments on several standard benchmark data sets to verify the performance of the proposed LGS-OSELM, which demonstrates that our proposed approach outperforms state-of-the-art methods in terms of accuracy.


---

## Body

## 1. Introduction

In past decades, neural networks have been intensively studied and applied as computational tools to solve various engineering problems [1, 2]. Although overwhelming traditional neural networks (such as BP algorithm) and their derivatives can approximate any nonlinear continuous function [3, 4], their performance is insufficient in learning speed when exposed to large-scale data sets [5]. To address this issue, Huang et al. [6, 7] have proposed an alternative to neural networks called extreme learning machine (ELM). Unlike traditional neural networks, which rely on an iterative method to adjust connection weights, ELM’s output weights are determined by using Moore Penrose generalized inverse, and the random feature mapping process determines its input weights [8]. Because the entire ELM learning process does not need any iteration, it has the ability to learn rapidly [5, 9, 10].The original ELM performs offline learning [11, 12]. For online prediction problems, the prediction model must be updated as newly arriving data. If the ELM is directly employed for the online prediction, the ELM must repeat training with past data as well as new data. With the continuous arrival of new data and the repeated training on the past data, the ELM’s learning speed gradually declines and disappears, and even worse its learning ability collapses altogether. To solve the above problem, the Online Sequential Extreme Learning Machine (OS-ELM) has been proposed [13–15], an online ELM variant. The OS-ELM only trains on newly arriving data and then combines the existing prediction model and the training results to update a new prediction model. However, the OS-ELM is overwhelmingly dependent on abundant labeled samples. Many data sets usually contain a few labeled samples and large unlabeled samples [16, 17]. If only applying labeled samples train the prediction model, the valuable information from unlabeled samples is wasted and would lead to inferior generalization performance of the model. Fortunately, Semisupervised Learning (SSL) provides a feasible strategy to effectively utilize the information of unlabeled samples for boosting the model performance [18, 19].Some Semisupervised ELM (SS-ELM) variants [20–26] have been presented. These semisupervised ELM are implemented by labeling the unlabeled samples once or calculating graph Laplacians between labeled and unlabeled samples once that generally used distance to measure similarity between samples. Semisupervised OS-ELM [27] based on the OS-ELM and SS-ELM [20] have been designed to adapt learning for unlabeled samples chunk by chunk. The effectiveness of distance-based semisupervised learning algorithms relies on the hypothesis that the same class is near feature space. However, for many complex data sets, distance-based methods cannot effectively achieve clustering [28]. To this end, Xia et al. [29] have been proposed a Density-based Semisupervised Online Sequential Extreme Learning Machine (D-SOS-ELM) based on local density and distance to measure similarity between labeled samples and unlabeled samples. The most existing SSL methods are based on the assumption that the empirical distribution of labeled samples follows from the actual samples’ distribution. It can hardly obtain the underlying distribution with only limited labeled samples called the empirical distribution mismatch issue. An example of Jain’s toy data [30] is illustrated in Figure 1, in which we plot tiny amounts of labeled samples and affluent unlabeled samples. It can be observed that unlabeled samples well describe Jain’s toy data distribution compared to labeled samples. Most SLL methods deem that similar samples have similar labels [31]. That is, employing the similarity between labeled and unlabeled samples determines the possibility of unlabeled sample labels. However, due to the sampling bias issue, it is infeasible to determine the relationship between all unlabeled and labeled samples at once, resulting in the inferior performance of model.Figure 1
An example of the empirical distribution mismatch between labeled and unlabeled samples with the Jain’s toy data. The labeled and unlabeled samples are displayed in the (A) of (b) and (c), and the kernel density estimations of theirx-axis projection are plotted in the (B) of (b) and (c), respectively. (a) Original samples. (b) Labeled samples Sl. (c) Unlabeled samples (Su).
(a)(b)(c)To tackle the empirical distribution mismatch issue, a novel semisupervised OS-ELM named Local Gravitation Semisupervised OS-ELM (LGS-OSELM) is proposed in the paper, which deems that the larger the scale of labeled samples is, the closer the empirical distribution of labeled samples is to the real sample distribution. Therefore, a label propagation strategy from labeled samples to unlabeled samples and from easy to difficult is used to gradually raise the scale of labeled samples.Recently, Wang et al. [32] proposed local gravitation clustering, responding to the relevance between each sample and its neighbors, demonstrating that clustering data with complex features can be better clustered by local gravitation than density and distance clustering. Inspiring by local gravitation clustering, in our proposed method, the gravitation measures, including centrality (CE) and coordination (CO), between each unlabeled sample and labeled samples from different categories are calculated to measure similarity. These gravitation measures determine its confidence level. The unlabeled samples with the higher confidence level are easier to learn. Therefore, these unlabeled samples are labeled pseudo label, to label, and then they are employed to iterate the neural network by implementing OS-ELM for embedding information of newly labeled samples. Through continuous incremental learning, the effective learning of unlabeled samples is finally realized to facilitate effectively the learning accuracy. This method inherits the advantages of the OS-ELM and local gravitation clustering; uses local gravitation to express the similarity measure; and uses OS-ELM for incremental learning to reduce the empirical distribution mismatch gradually. Experimental results on several benchmark data sets, including two synthetic data sets and eleven real-world data sets, demonstrate that the classification accuracy of our approach outperforms state-of-the-art methods.Contributions of this work are summarized as follows:(1)
The proposed LGS-OSELM offers a new incremental learning perspective to reduce gradually distribution mismatch between labeled and unlabeled samples by a label propagation strategy from labeled samples to unlabeled samples.(2)
This paper proposes a step-by-step learning strategy, that is, each iteration selects samples with high similarity from the unlabeled sample set for learning.(3)
This paper proposed the local gravitation measures, extending the similarity measurement between labeled samples and unlabeled samples method.The remainder of this paper is organized as follows. Section2 presents representative works related to OS-ELM and gravitational clustering. Section 3 describes our newly proposed approach. Section 4 evaluates the performance of the proposed approach experimentally. Section 5 presents the conclusion.


## 2. Related Works

### 2.1. Extreme Learning Machine

The original ELM is a variant of the feed-forward neural network, in which parameters of hidden layers (input weights bias) are randomly chosen and the output weight vector is determined analytically via a Moore-Penrose generalized inverse. With L hidden nodes, an original ELM model can be expressed as(1)fLx=∑i=1Lβihix=hxβ,where x=xi,x2,…,xm denotes the input feature and hx=hix,h2x,…,hLx is the output row vector of the hidden layer. ELM learning aims to determine the output weight vector β=β1,β2,…,βL by minimizing the estimation error. The loss is expressed as follows:(2)minβJ=12Hβ−T2,where T=t1,t2,…,tN⊤, and H is the hidden layer output matrix. It can be expressed as(3)H=hx1⋮hxN=gw1,x1,b1⋯gwL,x1,bL⋮⋱⋮gw1,xN,b1⋯gwL,xN,bL,where w donates the input weight of hidden layer, b donates the bias of hidden node, and g(.) is the activation function. If regularized optimization is utilized, the optimal weight vector is given by the solution of the following optimization problem:(4)minβJ=12Hβ−T2+C2β2,where C denotes the regularization factor and is a positive constant, and the solution for p is given:(5)β=1C+HTH−1HTT.


### 2.2. Online Sequential Extreme Learning Machine

The OS-ELM is an online version of the ELM. It can gradually learn newly arriving samples. Suppose that there is an initial sample setS0=xi0,ti0i=1N0 contained N0 samples. In initial phase, the neural network is trained using the original ELM, the initial output weight vector is given by(6)β0=P0H0⊤T0,where P0=1/C+H0TH0−1. Suppose that the sequence sample set Sk=xik,tiki=kNk is available. For each newly arriving sample set Sk, the neural network should be updated as(7)Pk=Pk−1−Pk−1Hk⊤I+HkPk−1Hk⊤−1HkPk−1,(8)βk=βk−1+PkHk⊤Tk−Hkβk−1,where Hk is the output matrix of the k-th iteration. It be calculated as(9)Hk=gw1,x1k,b1…gwL,x1k,bL⋮⋱⋮gw1,xNkk,b1…gwL,xNkk,bL.


### 2.3. Local Gravitation Clustering

The local gravitation clustering approach inspired by Newton’s law of gravitation responds to association between two objects. Each sample is viewed as an object with mass. The attractive force between two objects can be determined using the following formula:(10)F⟶q1,q2=Gm1m2Dq1,q2D^q1,q2,where F⟶q1,q2 denotes the attractive force between two objects q1 and q2, m1 and m2 respectively denote the mass of q1 and q2, Dq1,q2 is the distance between these two objects, and D^q1,q2 denotes the unit vector of the direction of the line that connects the two objects.In a local region, there is existing an assumption that the distances between the current object and its neighbors have not varied significantly. Therefore, equation (10) can be simplified as(11)F⟶q1,q2=Gm1m2D^q1,q2.The Local Resultant Force (LRF) of an object generated by itsk-nearest neighbors can be computed as(12)F⟶i=mi∑j=1kmjD^ij,where the unit vector D^ij generalizes the directional information, and k is the number of neighbors. The object with larger mass generates more influence to its neighbors, while the object with smaller mass is more sensitive to the influence from its neighbors. Motivated by this formulation, the LRF is newly defined as(13)F⟶i=1mi∑j=1kD^ij,where the mass m of the object is defined as follows:(14)mi=1∑j=1kDij.In the higher density regions, the mass of object will become larger according to equation (14), vice versa. Equation (13) is not sum of scalars, but vectors. The denser the region the larger the mass, the smaller the magnitude for ∑j=1kD^ij. Figure 2(a) shows that there exists a significant difference between the LRFs of the samples close to the center of cluster and those at border of the cluster. To encapsulate the information of the LRF, two local gravitation measures based on the local gravitation intensity have been proposed: (1) the centrality (CE) and (2) the coordination (CO). The CEi and COi of sample Xi are defined as(15)CEi=1k∑j=1kcosF⟶j,D⟶ij,(16)COi=1k∑j=1kF⟶j⋅F⟶i,where D⟶ij refers to the displacement vector from data point xi to its j−th neighbor. Figure 2(b) shows that points in the cluster interior usually have CE values greater than 0. A sample, whose CO value is larger, indicates that its LRF has approximately same direction as its neighbors, and it may be located at the border. According to equations (15) and (16), CE∈−1,1, and CO has no clear value range. Figure 3 shows CE and CO values of samples in different localities. From Figure 3, the closer the sample is to the edge, the larger its CO is, the smaller its CE is. Table 1 shows the differences between interior and boundary samples in term of the mass, the magnitude of the LRF, the CE, and the CO. The local gravitation clustering computes the LRF, the CE, and the CO of each data sample and uses this information to distinguish samples in the central region from those at the border region.Figure 2
The rules of LRFs and CE are shown in 2-D. (a) LRFs of the synthetic data set. The LRFs usually have relatively small magnitude in the central regions of the clusters and with relatively large magnitude at the border regions. (b) Points with CE > 0 are represented with red dotted-markers, most of which are located in center areas.
(a)(b)Figure 3
CE and CO values of samples in different localities are shown. (a) CE; (b) CO.
(a)(b)Table 1
Differences between interior samples and boundary samples on several local gravitation measures.
MassLRFCECOInteriorLargeSmallLargeSmallEdgeSmallLargeSmallLarge


## 2.1. Extreme Learning Machine

The original ELM is a variant of the feed-forward neural network, in which parameters of hidden layers (input weights bias) are randomly chosen and the output weight vector is determined analytically via a Moore-Penrose generalized inverse. With L hidden nodes, an original ELM model can be expressed as(1)fLx=∑i=1Lβihix=hxβ,where x=xi,x2,…,xm denotes the input feature and hx=hix,h2x,…,hLx is the output row vector of the hidden layer. ELM learning aims to determine the output weight vector β=β1,β2,…,βL by minimizing the estimation error. The loss is expressed as follows:(2)minβJ=12Hβ−T2,where T=t1,t2,…,tN⊤, and H is the hidden layer output matrix. It can be expressed as(3)H=hx1⋮hxN=gw1,x1,b1⋯gwL,x1,bL⋮⋱⋮gw1,xN,b1⋯gwL,xN,bL,where w donates the input weight of hidden layer, b donates the bias of hidden node, and g(.) is the activation function. If regularized optimization is utilized, the optimal weight vector is given by the solution of the following optimization problem:(4)minβJ=12Hβ−T2+C2β2,where C denotes the regularization factor and is a positive constant, and the solution for p is given:(5)β=1C+HTH−1HTT.


## 2.2. Online Sequential Extreme Learning Machine

The OS-ELM is an online version of the ELM. It can gradually learn newly arriving samples. Suppose that there is an initial sample setS0=xi0,ti0i=1N0 contained N0 samples. In initial phase, the neural network is trained using the original ELM, the initial output weight vector is given by(6)β0=P0H0⊤T0,where P0=1/C+H0TH0−1. Suppose that the sequence sample set Sk=xik,tiki=kNk is available. For each newly arriving sample set Sk, the neural network should be updated as(7)Pk=Pk−1−Pk−1Hk⊤I+HkPk−1Hk⊤−1HkPk−1,(8)βk=βk−1+PkHk⊤Tk−Hkβk−1,where Hk is the output matrix of the k-th iteration. It be calculated as(9)Hk=gw1,x1k,b1…gwL,x1k,bL⋮⋱⋮gw1,xNkk,b1…gwL,xNkk,bL.


## 2.3. Local Gravitation Clustering

The local gravitation clustering approach inspired by Newton’s law of gravitation responds to association between two objects. Each sample is viewed as an object with mass. The attractive force between two objects can be determined using the following formula:(10)F⟶q1,q2=Gm1m2Dq1,q2D^q1,q2,where F⟶q1,q2 denotes the attractive force between two objects q1 and q2, m1 and m2 respectively denote the mass of q1 and q2, Dq1,q2 is the distance between these two objects, and D^q1,q2 denotes the unit vector of the direction of the line that connects the two objects.In a local region, there is existing an assumption that the distances between the current object and its neighbors have not varied significantly. Therefore, equation (10) can be simplified as(11)F⟶q1,q2=Gm1m2D^q1,q2.The Local Resultant Force (LRF) of an object generated by itsk-nearest neighbors can be computed as(12)F⟶i=mi∑j=1kmjD^ij,where the unit vector D^ij generalizes the directional information, and k is the number of neighbors. The object with larger mass generates more influence to its neighbors, while the object with smaller mass is more sensitive to the influence from its neighbors. Motivated by this formulation, the LRF is newly defined as(13)F⟶i=1mi∑j=1kD^ij,where the mass m of the object is defined as follows:(14)mi=1∑j=1kDij.In the higher density regions, the mass of object will become larger according to equation (14), vice versa. Equation (13) is not sum of scalars, but vectors. The denser the region the larger the mass, the smaller the magnitude for ∑j=1kD^ij. Figure 2(a) shows that there exists a significant difference between the LRFs of the samples close to the center of cluster and those at border of the cluster. To encapsulate the information of the LRF, two local gravitation measures based on the local gravitation intensity have been proposed: (1) the centrality (CE) and (2) the coordination (CO). The CEi and COi of sample Xi are defined as(15)CEi=1k∑j=1kcosF⟶j,D⟶ij,(16)COi=1k∑j=1kF⟶j⋅F⟶i,where D⟶ij refers to the displacement vector from data point xi to its j−th neighbor. Figure 2(b) shows that points in the cluster interior usually have CE values greater than 0. A sample, whose CO value is larger, indicates that its LRF has approximately same direction as its neighbors, and it may be located at the border. According to equations (15) and (16), CE∈−1,1, and CO has no clear value range. Figure 3 shows CE and CO values of samples in different localities. From Figure 3, the closer the sample is to the edge, the larger its CO is, the smaller its CE is. Table 1 shows the differences between interior and boundary samples in term of the mass, the magnitude of the LRF, the CE, and the CO. The local gravitation clustering computes the LRF, the CE, and the CO of each data sample and uses this information to distinguish samples in the central region from those at the border region.Figure 2
The rules of LRFs and CE are shown in 2-D. (a) LRFs of the synthetic data set. The LRFs usually have relatively small magnitude in the central regions of the clusters and with relatively large magnitude at the border regions. (b) Points with CE > 0 are represented with red dotted-markers, most of which are located in center areas.
(a)(b)Figure 3
CE and CO values of samples in different localities are shown. (a) CE; (b) CO.
(a)(b)Table 1
Differences between interior samples and boundary samples on several local gravitation measures.
MassLRFCECOInteriorLargeSmallLargeSmallEdgeSmallLargeSmallLarge


## 3. Proposed LGS-OSELM

In SLL, available samples include few labeled samplesSl=xil,tili=1Nl and plenty of unlabeled samples Su=xiui=1Nu. The previous semisupervised learning is based on two assumptions (1) that labeled and unlabeled samples follow the same true samples distribution and (2) that if two samples x1 and x2 are similar to each other, the conditional probabilities Pt|x1 and Pt|x2 should be the same or similar. In the previous most of the SLL based on ELM [20, 21, 33, 34], the Laplacian graph built from both labeled and unlabeled samples is built based on these assumptions. However, the empirical distribution of labeled samples usually deviates from the true sample distribution, due to the randomness in sampling of labeled samples and the small sample scale. Sampling bias issue is commonly mentioned in the works under the supervised learning and domain adaptation scenario [35, 36]. In these works, sampling bias is that training data and testing data are differently distributed. In SLL, the number of labeled samples is usually small (Nl≪Nu), which would lead to a notable empirical distribution difference with the unlabeled samples, and sampling bias between labeled samples and unlabeled samples has rarely been discussed in the previous SLL.In addition, measuring similarity between two samples is based on distance, but the distance cannot reflect the similarity in many cases [37]. Two examples that semisupervised learning-based distance cannot handle are shown in Figure 4. Figure 4(a) shows the point distribution of flamed data [38], the unlabeled samples located in rectangular areas with black border are easily misclassified when using the distance-based semisupervised learning, because the average distance from these rectangular areas to another category is smaller than that of their real category. Another example of Jain’s toy data [30] in Figure 4(b) also shows that the distance-based method may cause the misclassification in the rectangular areas with black border, making learning worse.Figure 4
Two examples that the distance-based method cannot handle. (a) Flame data and (b) Jain’s toy data. Blue circles and red squares denote two different classes. The unlabeled samples located in rectangular areas with black border are easily misclassified.
(a)(b)To effectively end the aforementioned problems, a new semisupervised learning strategy based on local gravitation and OS-ELM is proposed in this paper, which deems that the larger the scale of labeled samples is, the closer the empirical distribution of labeled samples is to the real sample distribution. Therefore, a label propagation strategy from label samples to unlabeled samples and from easy to difficult is used to gradually raise the scale of labeled samples. Figure5 shows the illustration of the label propagation with the Jain’s toy data. From left to right is the label propagation in Figure 5(a), and the kernel density estimations of their x-axis projection are plotted in Figure 5(b), respectively. From the figure, we can note that the empirical distribution of labeled data deviates from the true sample distribution due to the limited sampling size of labeled data in the initial stage. As the size of the labeled data continues to increase by a label propagation strategy, the empirical distribution of the labeled data gradually approaches the true sample distribution.Figure 5
An illustration of the label propagation with the Jain’s toy data. From left to right is the label propagation in (a), and the kernel density estimations of theirx-axis projection are plotted in (b), respectively.
(a)(b)First, the LGS-OSELM uses the labeled samples to learn the initialization model by implementing ELM. Second, at each incremental learning, the unlabeled samples with high confidence level that is easy to learn are labeled with pseudo label, and then these samples are utilized to iterate the neural network by implementing OS-ELM. The confidence level of unlabeled samples belonging to a certain category is determined by the gravitational similarity. Through successive learning unlabeled samples and iterating neural network, the proposed approach ultimately realizes effective learning of all samples. We employ relative local gravitation measure between unlabeled and labeled samples to determine confidence level of unlabeled samples. The labeled sample setSl=xil,tili=1Nl is divided into different groups Sql=xi,ql,cqi=1Nq with the same label, where ti∈c1,c2…,cn,Sl, Sl=∪q=1lSql, and Nl=∑q=1nNl. In the work, we define a relative mass between xiu and Sql as(17)mi,cqu=1∑j=1kDijcq,where Dijcq denotes the distance between xiu and its j−th neighbor from Sql. Here, k-nearest neighbors are from Sql. An unlabeled sample has different relative LRFs with different groups. The relative LRFs are defined as(18)F⟶i,cq=1mi,cqu∑j=1kD^ijcq.Wang et al.’s work [32] indicated that CE gradually decreases from the inside to the edge, and CE∈−1,1. The relative CE is defined as(19)CEi,cq=1k∑j=1kcosF⟶j,D⟶ijcq,where F⟶j denotes the LRF of the j−th neighbor of xiu, which are calculated by the nearest neighbors from the same group. The relative CO between xiu is defined as(20)COi,cq=1k∑j=1kF⟶j⋅F⟶i,cq.The closer the unlabeled samplexiu is to the edge of Sql, the larger the COi,cq is, vice versa. COi,cq has no clear value range. Here, COi,cq is modified as(21)COi,cq˜=1−e−COi,cq1+e−COi,cq,where COi,cq˜∈−1,1. The points of most central areas and the edge areas are significantly different in terms of CE and CO. If CEi,cq≥0 0, xiu has a higher probability of locating at interior of Sql. However, a sample with CEi,cq<0, xiu with a smaller probability is in the center of Sql. Since the points in the cluster center are subjected to forces in all directions, their CE are not significantly larger than zero [32]. Fortunately, COi,cq˜ provides auxiliary information to avoid that xiu in the central region be misclassified as edge sample. The center sample usually has a smaller CO value, COi,cq˜<0. When all internal samples are labeled, the edged samples have smaller CE and larger CO. It is difficult to determine which cluster they belong to. To end the problem, we compare magnitude of relative LRFs. As Figure 6(b) shows, the Q and R are edged data points. For data point Q,F⟶Q,c1>F⟶Q,c2 so that data point with higher confidence level Q belongs to c1.Figure 6
Data LRF of edge data points are shown. (a) Only the edge data points are not labeled. (b) TheQ and R are edged data points. For data point Q, F⟶Q,c1>F⟶Q,c2, so that data point with higher confidence level Q is belonged to c1. The Q and R are edged data points. For data point R, F⟶R,c1<F⟶R,c2, so that data point with higher confidence level, R, is belonged to c2.
(a)(b)To sum up, according to the local gravitation theory, the unlabeled samples with high confidence are selected in the following three cases.(1)
For each unlabeled samplexiu, we calculate its relative CE to each category from labeled samples, and then find CEi,max=maxCEi,c1,CEi,c2…CEi,cm. If CEi,max≥0 and CEi,max=CEi,cq with higher confidence level located at the interior of Sql.(2)
IfCEi,max<0 and CEi,max=CEi,cq, we calculate COi,cq˜ of xiu. If COi,cq˜<0, xiu with higher confidence level locates at the center of Sql.(3)
If it does not meet (1) and (2),xiu is the edge data point. We calculate magnitude of relative LRFs and find F⟶i,max=maxF⟶i,c1,F⟶i,c2,…,F⟶i,cn, 仔 F⟶i,max=F⟶i,cq with higher confidence level belonging to Sql.The algorithm is described in Algorithm 1. Through the calculation of sample’s local gravitation to determine its confidence level, the unlabeled samples with higher confidence level are inserted into neural network to generate pseudo labels [29] and then continue to be iterated until all the unlabeled samples are learned. The proposed method is more conducive to improving the reliability and accuracy of network learning.Algorithm 1: The proposed LGS-OSELM algorithm.
input. Labeled samples set Sl=xil,tili=1Nl, unlabeled samples set Su=xiui=1Nu, and activation function, g⋅.Output: The decision functions: f (x) = hxβ.UseSl to build the initial ELM model by implementing ELM. That is, according to equation (6), calculate β0A temporary sample setZ=∅ is established.whileSu is not empty doFor each unlabeled samplesxiu, calculate its relative CEs andthen findCEi,cq=maxCEi,c1,CEi,c2…CEi,cm.IfCEi,cq≥0thenxu is removed into Z from set Su.elseCalculateCOi,cq˜ifCOi,cq˜ < 0 thenxu is removed into Z from set SuelseFor each unlabeled samplexiu, calculate all its relative LRFs, and then find F⟶i,max=maxF⟶i,c1,F⟶i,c2,…,F⟶i,cn,xu is removed into Z from set Su.end ifend ifUnlabeled samples inZ are inserted into the ELM model to generate pseudo labels. Remove the samples into Sl from Z.Update the network model with the result of the labeled samples. That is, updateβ and P based on equations (7) and (8).end while


## 4. Experiments and Results

In order to verify the effectiveness of the proposed LGS-OSELM, several comparative experiments have been empirically conducted on synthetic and real data sets.


### 4.1. Baseline Methods

To demonstrate the capabilities of LGS-OSELM algorithm, comparisons are made with different algorithms such as the SS-ELM, Laplacian support vector machine (Lap-SVM), LapTELM D-SOS-ELM, and STAR-SVM. These comparison algorithms are briefly introduced as follows:(1)
ELM [6]: ELM is efficient single layer feed-forward neural network. Its details have been introduced in 2.1.(2)
SS-ELM [20]: the semisupervised learning algorithm is based on Laplacian graph between labeled data and unlabeled data. It initializes the ELM model, then iterative selects the best until convergence.(3)
Lap-SVM [39]: the semisupervised learning algorithm is based on Laplacian graph and SVM, which introduces an additional regularization term on the geometry of both labeled and unlabeled samples.(4)
STAR-SVM [40]: STAR-SVM is designed to adaptively modify the optimization by adjusting the weights at each iteration. At each iteration, the regularization parameters are adapted to better reflect label confidence and class proportion and to gradually include more unlabeled samples.(5)
LapTELM [22]: LapTELM uses manifold regularization term to explore the geometry structure information of unlabeled samples, which simultaneously trains two related and paired semisupervised ELMs with two nonparallel separating planes for the final classification.(6)
D-SOS-ELM [29]: D-SOS-ELM uses local density and distance which are used to the similarity of sample.


### 4.2. Data Sets

The experimental data sets include synthetic data sets and real data sets. Two synthetic data sets are Jain’s toy data set and the Flame data, which are shown in Figure4. For real data sets, we use benchmark data sets, which were widely used to evaluate machine learning algorithms. The details of these data sets are shown in Table 2. Among them, USPS and MNIST are the two well-known handwritten data sets with samples ranging from “0” to “9.” The other 10 real-world data sets are selected from UCI machine learning repository.Table 2
Specifications of benchmark data sets.
Data setInstancesFeaturesClassesAustralian690142EEG-Eye14980142Glass21496Heart270132MNIST200078410Mushroom8124222Ionosphere351342Pima78682Seeds21073Sonar208602USPS_20185625610Wine178133


### 4.3. Experimental Setup

To evaluate the semisupervised learning algorithms, each data set is split into a training data set and a testing data set (denoted byT). Then, the training data set is further divided into a labeled set (ℒ), a validation set (V), and an unlabeled set (U). When a semisupervised learning algorithm is trained, the labeled data from L and the unlabeled data from U are utilized. In this paper, we randomly select 10% as L from the original data, 50% as U, 10% as V, and 30% as T. The performance of model is sensitive to involved parameters of algorithm. In the experiments, we use the training set for a 7-fold cross-validation to find the optimal parameters. In the simulations of ELM, SS-ELM, and LGS-OSELM algorithms on the EEG-Eye, MNIST, and USPS20, the number of hidden layer nodes is set to be 2000, while on the remaining data sets, it is set to be 200, since ELM-based models can realize good generalization performance as long as the number of hidden layer nodes is large enough. The details of the involved parameters in each algorithm are listed as follows:(i)
ELM: the trade-off parameterc(ii)
SS-ELM: the trade-off parametersc and λ and the number of the nearest neighbors k(iii)
Lap-SVM: the number of nearest neighborsk and the bandwidth parameter σ(iv)
STAR-SVM: trade-off parameterc and confidence weight γ(v)
LGS-OSELM: trade-off parameterc and the number of nearest neighbors k(vi)
LapTELM: trade-off parametersc1 and c2(vii)
D-SOS-ELM: trade-off parameterscThese parameters are selected from the following sequences:(i)
c and c1: 10−6,10−4,10−2,1,102,104,106(ii)
c2: {0.001, 0.01, 0.05, 0.1, 0.5, 1}(iii)
σ: 10−4,10−2,100,102,104(iv)
k: {5, 10, 15, 20}(v)
λ: 10−6,10−4,10−2,100,102,104,106(vi)
γ: {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1}


### 4.4. Experimental Results

#### 4.4.1. The Results on the Synthetic Data Sets

Two synthetic data sets randomly divided the data set into 4 parts, the first part is 10% of the data set, the second part is 50%, the third part is 10%, and the fourth part is 30%. The traditional ELM method is divided into two types of training: one selects the first part of labeled samples to train the model (ELMA), and another simulation employs the first part and second part labeled samples to train the model (ELMB). The SS-ELM, LapTELM, D-SOS-ELM, and LGS-OSELM select the first part as ℒ and second part as U. The third and fourth parts of data respectively are set as V and T for all algorithms.This experiment is mainly used to find the optimal. Table3 shows the experimental results of LGS-OSELM on these synthetic data sets. In Table 3, it can be seen that when the parameter k increases from 5 to 20, classification accuracy is different. On the Flame data set, when the number of nearest neighbors k is 15, the experimental result is the best; for Jain data sets, the number of nearest neighbors k is 10.Table 3
Classification accuracy of LGS-OSELM with differentk on V for two synthetic data sets.
kFlameJainAverage (%)Best (%)Average (%)Best (%)595.4098.3492.5692.311096.5499.0197.361001598.8310095.311002097.5010094.1699.65Table4 shows the classification accuracy of ELM, SS-ELM, D-SOS-ELM, LapTELM, and LGS-OSELM on the synthetic data sets, where “Acc%” denotes the average classification accuracy on the testing set T according to the optimal parameters “(c),” “(c1,c2),” “(c, λ, k),” or “(c, k).” It can be seen from Table 4 that the ELMB has the best experimental result; ELMA is the worst. In terms of the number of samples used in training, the ELMA model is the least, so the result is the worst. The number of training samples for SS-ELM and LGS-OSELM is the same as that of ELMB. The difference is that the ELMB model is trained by a large number of labeled samples, while SS-ELM, D-SOS-ELM, LapTELM, and GLS-OSELM have very few labeled samples, and most of the samples are unlabeled, and it is obvious that when the labeled data are relatively small, LGS-OSELM outperforms the ELMa, D-SOS-ELM, LapTELM, and SS-ELM. On the Flame data set, the classification accuracy achieved by LGS-OSELM is 8% and 19.1% higher than these of the ELMa and SS-ELM, respectively. On the Jain data set, the classification accuracy achieved by LGS- OSELM is 13.13% and 21.61% higher than those of the ELMa and SS-ELM, respectively. The proposed LGS-OSELM is slightly excellent than D-SOS-ELM and LapTELM on two synthetic data sets.Table 4
Average classification accuracy of the ELM, SS-ELM, D-SOS-ELM, LapTELM, and LGS-OSELM onT for two synthetic data sets.
DatasetELMaELMBSS-ELMD-SOS-ELMLapTELMLGS-OSELMAcc% (c)Acc% (c)Acc% (c, A, k)Acc% (c)Acc% (C1, c2, k)Acc% (c,k)Flame79.7399.2390.8397.6797.8698.83(100)(100)(100, 102, 5)(100)(100, 0.01, 15)(100, 15)Jain76.5598.3985.0394.8795.6498.16(10−2)(10−2)(10−2, 102, 15)(102)(10−2, 0.05, 10)(10−2, 10)


#### 4.4.2. The Results on Real-World Data Sets

The performance comparison on real-world data sets is displayed in Table5. Note that the training sets of the ELM consist of only the 10% labeled data. As can be seen from Table 5, LGS-OSELM, LapTELM, and D-SOS-ELM achieve better results than ELM, Lap-SVM, STAR-SVM, and SS-ELM on almost all the employed data sets. LGS-OSELM achieves better performance than LapTELM and D-SOS-ELM in most cases. From the last line of Table 5, the average classification accuracy achieved by LGS-OSELM is 82.15% on the T, which is 11.29%, 4.25%, 8.94%, 6.51%, 1.87%, and 1.12% higher than those of the ELM, STA-AVM, Lap-SVM, SS-ELM, D-SOS-ELM, and LapTEM across twelve data sets, respectively. The average classification accuracy achieved by LGS-OSELM is 83.63% on the U, which is 10.89%, 4.59%, 7.58%, 6.34%, 2.23%, and 2.21% higher than these of the ELM, STA-SVM, Lap-SVM, SS-ELM, D-SOS-ELM, and LapTELM across twelve data sets, respectively.Table 5
Classification accuracy of seven algorithms onT and U for real-world data sets.
ELMSTAR-SVMLap-SVMSS-ELMD-SOS-ELMLapTEMLGS-OSELMDatasetAcc%Acc%Acc%Acc%Acc%Acc%Acc%(c)(c, γ)(c, σ, k)(c, λ, k)(c)(c1, c2, k)(c, k)AustralianT66.7869.7369.9670.8176.3475.4377.34(10−2)(100, 0.7)(100, 10−2, 5)(102, 100, 10)(102)(102, 0.01, 10)(10−2, 10)U67.8270.5670.3870.8577.5677.4379.28(10−2)(100, 0.7)(100, 10−2, 5)(102, 100, 10)(102)(102, 0.01, 10)(10−2, 10)EEG-EyeT73.3594.2373.4276.3093.2594.6297.65(100)(102, 0.5)(102, 10−4, 15)(102, 100, 15)(102)(102, 0.01, 10)(102, 20)U75.4595.3674.3577.4292.1494.2597.58(100)(102, 0.5)(102, 10−4, 15)(102, 100, 15)(102)(102, 0.01, 10)(102, 20)GlassT64.5268.2364.5266.3570.4672.1373.34(10−2)(10−2, 0.7)(10−2, 102, 5)(10−2, 102, 5)(10−2)(10−2, 0.05, 15)(10−2, 10)U67.5271.6668.1466.6372.4572.3477.42(10−2)(10−2, 0.7)(10−2, 102, 5)(10−2, 102, 5)(10−2)(102, 0.05, 15)(10−2, 10)HeartT66.3472.4572.5264.5275.1976.2678.39(100)(102, 0.4)(102, 102, 5)(10−4, 102, 5)(10−4)(102, 0.1, 15)(100, 15)U67.1273.4375.7267.6475.2977.0977.65(100)(102, 0.4)(102, 102, 5)(10−4, 102, 5)(10−4)(102, 0.1, 15)(100, 15)MNISTT81.5788.2388.5689.4591.1892.5792.32(100)(102, 0.3)(102, 102, 5)(10−4, 10−2, 15)(10−4)(10−4, 0.05, 15)(102, 15)U83.0689.5689.8391.2691.6293.5292.35(100)(102, 0.3)(102, 102, 5)(10−4, 10−2, 15)(10−4)(10−4, 0.05, 15)(102, 15)MushroomT88.4590.3487.8490.3292.2489.1591.65(100)(102, 0.5)(100, 102, 5)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 5)U89.2192.6688.9390.4793.5489.2593.25(100)(102, 0.5)(100, 102, 5)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 5)IonosphereT82.1786.2584.7284.1490.9490.9492.46(10−2)(102, 0.7)(10−2, 102, 15)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 15)U82.4386.3787.8284.4391.4691.4292.86(10−2)(102, 0.7)(10−2, 102, 15)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 15)PimaT62.5267.4465.4567.4468.1568.2469.23(10−4)(10−4, 0.3)(10−4, 100, 10)(10−4, 100, 15)(10−4)(10−4, 0.1, 5)(10−4, 5)U63.6869.0565.8669.1768.5269.4571.80(10−4)(10−4, 0.3)(10−4, 100, 10)(10−4, 100, 15)(10−4)(10−4, 0.1, 5)(10−4, 5)SeedsT58.4365.6160.4564.5668.6268.1667.52(102)(102, 0.7)(102, 10−2, 10)(10−4, 100, 10)(10−4)(10−4, 0.01, 20)(102, 20)U58.4467.2662.1669.3069.1268.6271.35(102)(102, 0.7)(102, 10−2, 10)(10−4, 100, 10)(10−4)(10−4, 0.01, 20)(102, 20)SonarT73.4577.3267.2777.3480.2486.7285.68(10−2)(10−2, 0.5)(10−4, 102, 5)(10−4, 10−2, 5)(10−2)(104, 0.05, 15)(104, 15)U73.5579.1769.1179.0082.1485.6887.19(10−2)(10−2, 0.5)(10−4, 102, 5)(10−4, 10−2, 5)(10−2)(104, 0.05, 15)(104, 15)USPS_20T84.4496.5293.0495.2696.7298.1597.75(10−2)(10−2, 0.6)(10−4, 102, 5)(10−2, 10−2, 5)(10−2)(10−2, 0.1, 5)(100, 5)U86.1492.3697.7695.5497.4298.2597.12(10−2)(10−2, 0.6)(10−4, 102, 5)(10−2, 10−2, 5)(10−2)(10−2, 0.1, 5)(100, 5)WineT48.3458.5251.3661.4158.6358.3262.52(102)(102, 0.7)(102, 102, 15)(102, 10−2, 15)(10−2)(102, 0.01, 15)(102, 15)U58.4760.5862.4565.7861.6360.5364.27(102)(102, 0.7)(102, 102, 15)(102, 10−2, 15)(10−2)(102, 0.01, 15)(102, 15)AverageT70.8677.9173.2275.6580.2981.0482.15U72.7479.0476.1377.2981.3981.4283.63


### 4.5. Experimental Analysis

For further analyzing the significant difference between the five models on the employed real-world data sets, we resort to the well-known Friedman test with the corresponding post hoc tests. The Friedman statistics is written as follows:(22)τχ2=12nmm+1∑i=1mR¯i2−mm+124,where m and n are the numbers of the involved models and data sets and R denotes the average ranks. The statistics τχ2 follows distribution with m-1 degree of freedom. The average ranks of the five models testing sets of all the real-world data sets are displayed in Table 6. Firstly, we make a comparison of the five models on T. It is clear that m = 7 and n = 12. Based on the average ranks displayed in Table 7, we can calculate the statistics τχ2  = 56.267, and(23)τF=n−1τχ2mn−1−τχ2,which is distributed following the F-distribution with (m − 1) and (m − 1) (n − 1) degrees of freedom. We can calculate τF=29.865. For the confidence level 90% α=01, Fα6,66=1.865 < τF=29.865, which means the null hypothesis that the there is no significant difference among the all models. Next, the Nemenyi post hoc test is performed to further compare the five models in pairs. The critical difference (CD) is obtained(24)CDα,m=qα,mmm+16n,where qα,m is a definite value that related to m and the significance level and the confidence level α. It is available by looking up specific table; qα,m=2.693 can be obtained querying specific table and then we obtain the critical difference CD0.1,6  = 2.375. If the average ranks of two models differ by at least CD, their performance is significantly different. Statistical variables and the difference of average ranks between and other models are displayed in Table 7. The statistical variable τF is greater than Fα6,66, which indicates the null hypothesis. We can summarize that the proposed LGS-OSELM performs significantly better than ELM, STAR-SVM, Lap-SVM, and SS-ELM. There is no significant difference between D-SOS-ELM, LapTELM, and our proposed LGS-OSELM on T and the U.Table 6
Relevant values in the Friedman test with Nemenyi’s post hoc test on the 12 real-world data sets.
τFd (ELM- LGS-OSELM)d (STAR-SVM- LGS-OSELM)d (LapSVM- LGS-OSELM)d (SS-ELM- LGS-OSELM)d (D-SOS-ELM- LGS-OSELM)d (LapTELM- LGS-OSELM)T56.26729.8655.21 > CD2.83 > CD4.29 > CD3.17 > CD1.13 < CD0.88 < CDU45.15215.5945.17 > CD2.92 > CD3.7 > CD2.83 > CD1.38 < CD1.46 < CDTable 7
Average ranks of the seven algorithms on the real-world data sets.
ELMSTAR-SVMLap-SVMSS-ELMD-SOS-ELMLapTEMLGS-OSELMT6.714.335.794.672.632.381.50U6.674.425.254.332.882.961.50We further analyze the performance of the proposed LGS-OSELM with different scales of labeled data. More specifically, for all the employed data sets, 70% data are randomly selected from each class to form the training set, and the others are left to compose the testing set. Then,Q% data in the training set are randomly selected as ℒ and others as U, where the values of Q are 10, 20, 30, 40, 50, 60, and 70. The learning results are displayed in Figure 7. It is easy to know that the classification accuracy of the five algorithms is generally increased with the increasing of the values of Q. That is to say, increasing the scale of labeled data can generally boost the performance of algorithm.Figure 7
The classification accuracy of the five algorithms with different scales of labeled data onT.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)Further, it can be found that the LGS-OSELM performs better than other baseline methods in most cases, and the advantage is more significant when the value ofQ is small. In specific, no matter what the scale of labeled data is, the proposed LGS-OSELM performs better than SS-ELM, Lap-SVM and STAR-SVM, D-SOS-ELM, and LapTELM.In order to evaluate the computational efficiency of the proposed algorithm, we present the training times of six semisupervised algorithms on different data sets in Table8. From this table, training time is essentially the same order of magnitude. The proposed algorithm is the most efficient among the six algorithms, in which LGS-OSELM spends the least amount of training time on four of the twelve data sets. Because the proposed method is based on distance and density, the calculation of the local gravitation is very time consuming.Table 8
Training time of classifiers on the real-world data sets (in seconds).
STAR-SVMLap-SVMSS-ELMD-SOS-ELMLapTEMLGS-OSELMAustralian0.03140.02210.02440.02550.02900.0276EEG-Eye1.86151.45331.35321.54231.07321.0234Glass0.45120.35120.15260.13430.15120.1543Heart0.02520.00850.00520.00420.00840.0055MNIST0.25230.08350.05410.05430.08420.0643Mushroom2.34321.31412.84833.42530.93321.0023Ionosphere0.05560.03560.00420.00560.00530.0024Pima0.02420.03080.02010.05230.04710.0435Seeds0.0920.05740.07420.04710.04070.0427Sonar0.13450.09450.08520.05950.04730.0425USPS_200.41520.19330.2120.19230.23530.2391Wine0.31520.09330.12120.09230.21350.0912To summarize, two synthetic data sets and 12 groups of real-world data sets are used as the experimental data. For the two synthetic data sets, two types of ELM (ELM trained by few labeled samples and ELMB trained by abundant labeled samples), SS-ELM D-SOS-ELM and LapTELM, are employed as comparisons. The experimental results showed that the LGS-OSELM, trained by few labeled samples and abundant unlabeled samples, has the higher accuracy than SS-ELM, D-SOS-ELM, LapTELM, and ELM is and slightly less than ELMb. For real-world data sets, the experimental results showed that the LGS-OSELM has higher prediction accuracy than state-of-the-art semisupervised methods including STAR-SVM, Lap-SVM and SS-ELM, D-SOS-ELM, and LapELM. For further fair comparisons of five methods, we resort to the well-known Friedman test with the corresponding post hoc tests, which indicates that the proposed LGS-OSELM performs significantly better than ELM, STAR-SVM, Lap-SVM, and SS-ELM, and there is no significant difference between D-SOS-ELM, LapTELM, and our proposed LGS-OSELM on the T and the U across 12 real-world data sets. Experiments with the different scale of the labeled sample are carried out to further explore discrepancy among the five methods. Results indicate that the classification accuracy of these five algorithms generally increases as the scale of the labeled sample increases, and no matter what the scale of labeled data is, the proposed LGS-OSELM performs better than other baseline methods.


## 4.1. Baseline Methods

To demonstrate the capabilities of LGS-OSELM algorithm, comparisons are made with different algorithms such as the SS-ELM, Laplacian support vector machine (Lap-SVM), LapTELM D-SOS-ELM, and STAR-SVM. These comparison algorithms are briefly introduced as follows:(1)
ELM [6]: ELM is efficient single layer feed-forward neural network. Its details have been introduced in 2.1.(2)
SS-ELM [20]: the semisupervised learning algorithm is based on Laplacian graph between labeled data and unlabeled data. It initializes the ELM model, then iterative selects the best until convergence.(3)
Lap-SVM [39]: the semisupervised learning algorithm is based on Laplacian graph and SVM, which introduces an additional regularization term on the geometry of both labeled and unlabeled samples.(4)
STAR-SVM [40]: STAR-SVM is designed to adaptively modify the optimization by adjusting the weights at each iteration. At each iteration, the regularization parameters are adapted to better reflect label confidence and class proportion and to gradually include more unlabeled samples.(5)
LapTELM [22]: LapTELM uses manifold regularization term to explore the geometry structure information of unlabeled samples, which simultaneously trains two related and paired semisupervised ELMs with two nonparallel separating planes for the final classification.(6)
D-SOS-ELM [29]: D-SOS-ELM uses local density and distance which are used to the similarity of sample.


## 4.2. Data Sets

The experimental data sets include synthetic data sets and real data sets. Two synthetic data sets are Jain’s toy data set and the Flame data, which are shown in Figure4. For real data sets, we use benchmark data sets, which were widely used to evaluate machine learning algorithms. The details of these data sets are shown in Table 2. Among them, USPS and MNIST are the two well-known handwritten data sets with samples ranging from “0” to “9.” The other 10 real-world data sets are selected from UCI machine learning repository.Table 2
Specifications of benchmark data sets.
Data setInstancesFeaturesClassesAustralian690142EEG-Eye14980142Glass21496Heart270132MNIST200078410Mushroom8124222Ionosphere351342Pima78682Seeds21073Sonar208602USPS_20185625610Wine178133


## 4.3. Experimental Setup

To evaluate the semisupervised learning algorithms, each data set is split into a training data set and a testing data set (denoted byT). Then, the training data set is further divided into a labeled set (ℒ), a validation set (V), and an unlabeled set (U). When a semisupervised learning algorithm is trained, the labeled data from L and the unlabeled data from U are utilized. In this paper, we randomly select 10% as L from the original data, 50% as U, 10% as V, and 30% as T. The performance of model is sensitive to involved parameters of algorithm. In the experiments, we use the training set for a 7-fold cross-validation to find the optimal parameters. In the simulations of ELM, SS-ELM, and LGS-OSELM algorithms on the EEG-Eye, MNIST, and USPS20, the number of hidden layer nodes is set to be 2000, while on the remaining data sets, it is set to be 200, since ELM-based models can realize good generalization performance as long as the number of hidden layer nodes is large enough. The details of the involved parameters in each algorithm are listed as follows:(i)
ELM: the trade-off parameterc(ii)
SS-ELM: the trade-off parametersc and λ and the number of the nearest neighbors k(iii)
Lap-SVM: the number of nearest neighborsk and the bandwidth parameter σ(iv)
STAR-SVM: trade-off parameterc and confidence weight γ(v)
LGS-OSELM: trade-off parameterc and the number of nearest neighbors k(vi)
LapTELM: trade-off parametersc1 and c2(vii)
D-SOS-ELM: trade-off parameterscThese parameters are selected from the following sequences:(i)
c and c1: 10−6,10−4,10−2,1,102,104,106(ii)
c2: {0.001, 0.01, 0.05, 0.1, 0.5, 1}(iii)
σ: 10−4,10−2,100,102,104(iv)
k: {5, 10, 15, 20}(v)
λ: 10−6,10−4,10−2,100,102,104,106(vi)
γ: {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1}


## 4.4. Experimental Results

### 4.4.1. The Results on the Synthetic Data Sets

Two synthetic data sets randomly divided the data set into 4 parts, the first part is 10% of the data set, the second part is 50%, the third part is 10%, and the fourth part is 30%. The traditional ELM method is divided into two types of training: one selects the first part of labeled samples to train the model (ELMA), and another simulation employs the first part and second part labeled samples to train the model (ELMB). The SS-ELM, LapTELM, D-SOS-ELM, and LGS-OSELM select the first part as ℒ and second part as U. The third and fourth parts of data respectively are set as V and T for all algorithms.This experiment is mainly used to find the optimal. Table3 shows the experimental results of LGS-OSELM on these synthetic data sets. In Table 3, it can be seen that when the parameter k increases from 5 to 20, classification accuracy is different. On the Flame data set, when the number of nearest neighbors k is 15, the experimental result is the best; for Jain data sets, the number of nearest neighbors k is 10.Table 3
Classification accuracy of LGS-OSELM with differentk on V for two synthetic data sets.
kFlameJainAverage (%)Best (%)Average (%)Best (%)595.4098.3492.5692.311096.5499.0197.361001598.8310095.311002097.5010094.1699.65Table4 shows the classification accuracy of ELM, SS-ELM, D-SOS-ELM, LapTELM, and LGS-OSELM on the synthetic data sets, where “Acc%” denotes the average classification accuracy on the testing set T according to the optimal parameters “(c),” “(c1,c2),” “(c, λ, k),” or “(c, k).” It can be seen from Table 4 that the ELMB has the best experimental result; ELMA is the worst. In terms of the number of samples used in training, the ELMA model is the least, so the result is the worst. The number of training samples for SS-ELM and LGS-OSELM is the same as that of ELMB. The difference is that the ELMB model is trained by a large number of labeled samples, while SS-ELM, D-SOS-ELM, LapTELM, and GLS-OSELM have very few labeled samples, and most of the samples are unlabeled, and it is obvious that when the labeled data are relatively small, LGS-OSELM outperforms the ELMa, D-SOS-ELM, LapTELM, and SS-ELM. On the Flame data set, the classification accuracy achieved by LGS-OSELM is 8% and 19.1% higher than these of the ELMa and SS-ELM, respectively. On the Jain data set, the classification accuracy achieved by LGS- OSELM is 13.13% and 21.61% higher than those of the ELMa and SS-ELM, respectively. The proposed LGS-OSELM is slightly excellent than D-SOS-ELM and LapTELM on two synthetic data sets.Table 4
Average classification accuracy of the ELM, SS-ELM, D-SOS-ELM, LapTELM, and LGS-OSELM onT for two synthetic data sets.
DatasetELMaELMBSS-ELMD-SOS-ELMLapTELMLGS-OSELMAcc% (c)Acc% (c)Acc% (c, A, k)Acc% (c)Acc% (C1, c2, k)Acc% (c,k)Flame79.7399.2390.8397.6797.8698.83(100)(100)(100, 102, 5)(100)(100, 0.01, 15)(100, 15)Jain76.5598.3985.0394.8795.6498.16(10−2)(10−2)(10−2, 102, 15)(102)(10−2, 0.05, 10)(10−2, 10)


### 4.4.2. The Results on Real-World Data Sets

The performance comparison on real-world data sets is displayed in Table5. Note that the training sets of the ELM consist of only the 10% labeled data. As can be seen from Table 5, LGS-OSELM, LapTELM, and D-SOS-ELM achieve better results than ELM, Lap-SVM, STAR-SVM, and SS-ELM on almost all the employed data sets. LGS-OSELM achieves better performance than LapTELM and D-SOS-ELM in most cases. From the last line of Table 5, the average classification accuracy achieved by LGS-OSELM is 82.15% on the T, which is 11.29%, 4.25%, 8.94%, 6.51%, 1.87%, and 1.12% higher than those of the ELM, STA-AVM, Lap-SVM, SS-ELM, D-SOS-ELM, and LapTEM across twelve data sets, respectively. The average classification accuracy achieved by LGS-OSELM is 83.63% on the U, which is 10.89%, 4.59%, 7.58%, 6.34%, 2.23%, and 2.21% higher than these of the ELM, STA-SVM, Lap-SVM, SS-ELM, D-SOS-ELM, and LapTELM across twelve data sets, respectively.Table 5
Classification accuracy of seven algorithms onT and U for real-world data sets.
ELMSTAR-SVMLap-SVMSS-ELMD-SOS-ELMLapTEMLGS-OSELMDatasetAcc%Acc%Acc%Acc%Acc%Acc%Acc%(c)(c, γ)(c, σ, k)(c, λ, k)(c)(c1, c2, k)(c, k)AustralianT66.7869.7369.9670.8176.3475.4377.34(10−2)(100, 0.7)(100, 10−2, 5)(102, 100, 10)(102)(102, 0.01, 10)(10−2, 10)U67.8270.5670.3870.8577.5677.4379.28(10−2)(100, 0.7)(100, 10−2, 5)(102, 100, 10)(102)(102, 0.01, 10)(10−2, 10)EEG-EyeT73.3594.2373.4276.3093.2594.6297.65(100)(102, 0.5)(102, 10−4, 15)(102, 100, 15)(102)(102, 0.01, 10)(102, 20)U75.4595.3674.3577.4292.1494.2597.58(100)(102, 0.5)(102, 10−4, 15)(102, 100, 15)(102)(102, 0.01, 10)(102, 20)GlassT64.5268.2364.5266.3570.4672.1373.34(10−2)(10−2, 0.7)(10−2, 102, 5)(10−2, 102, 5)(10−2)(10−2, 0.05, 15)(10−2, 10)U67.5271.6668.1466.6372.4572.3477.42(10−2)(10−2, 0.7)(10−2, 102, 5)(10−2, 102, 5)(10−2)(102, 0.05, 15)(10−2, 10)HeartT66.3472.4572.5264.5275.1976.2678.39(100)(102, 0.4)(102, 102, 5)(10−4, 102, 5)(10−4)(102, 0.1, 15)(100, 15)U67.1273.4375.7267.6475.2977.0977.65(100)(102, 0.4)(102, 102, 5)(10−4, 102, 5)(10−4)(102, 0.1, 15)(100, 15)MNISTT81.5788.2388.5689.4591.1892.5792.32(100)(102, 0.3)(102, 102, 5)(10−4, 10−2, 15)(10−4)(10−4, 0.05, 15)(102, 15)U83.0689.5689.8391.2691.6293.5292.35(100)(102, 0.3)(102, 102, 5)(10−4, 10−2, 15)(10−4)(10−4, 0.05, 15)(102, 15)MushroomT88.4590.3487.8490.3292.2489.1591.65(100)(102, 0.5)(100, 102, 5)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 5)U89.2192.6688.9390.4793.5489.2593.25(100)(102, 0.5)(100, 102, 5)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 5)IonosphereT82.1786.2584.7284.1490.9490.9492.46(10−2)(102, 0.7)(10−2, 102, 15)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 15)U82.4386.3787.8284.4391.4691.4292.86(10−2)(102, 0.7)(10−2, 102, 15)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 15)PimaT62.5267.4465.4567.4468.1568.2469.23(10−4)(10−4, 0.3)(10−4, 100, 10)(10−4, 100, 15)(10−4)(10−4, 0.1, 5)(10−4, 5)U63.6869.0565.8669.1768.5269.4571.80(10−4)(10−4, 0.3)(10−4, 100, 10)(10−4, 100, 15)(10−4)(10−4, 0.1, 5)(10−4, 5)SeedsT58.4365.6160.4564.5668.6268.1667.52(102)(102, 0.7)(102, 10−2, 10)(10−4, 100, 10)(10−4)(10−4, 0.01, 20)(102, 20)U58.4467.2662.1669.3069.1268.6271.35(102)(102, 0.7)(102, 10−2, 10)(10−4, 100, 10)(10−4)(10−4, 0.01, 20)(102, 20)SonarT73.4577.3267.2777.3480.2486.7285.68(10−2)(10−2, 0.5)(10−4, 102, 5)(10−4, 10−2, 5)(10−2)(104, 0.05, 15)(104, 15)U73.5579.1769.1179.0082.1485.6887.19(10−2)(10−2, 0.5)(10−4, 102, 5)(10−4, 10−2, 5)(10−2)(104, 0.05, 15)(104, 15)USPS_20T84.4496.5293.0495.2696.7298.1597.75(10−2)(10−2, 0.6)(10−4, 102, 5)(10−2, 10−2, 5)(10−2)(10−2, 0.1, 5)(100, 5)U86.1492.3697.7695.5497.4298.2597.12(10−2)(10−2, 0.6)(10−4, 102, 5)(10−2, 10−2, 5)(10−2)(10−2, 0.1, 5)(100, 5)WineT48.3458.5251.3661.4158.6358.3262.52(102)(102, 0.7)(102, 102, 15)(102, 10−2, 15)(10−2)(102, 0.01, 15)(102, 15)U58.4760.5862.4565.7861.6360.5364.27(102)(102, 0.7)(102, 102, 15)(102, 10−2, 15)(10−2)(102, 0.01, 15)(102, 15)AverageT70.8677.9173.2275.6580.2981.0482.15U72.7479.0476.1377.2981.3981.4283.63


## 4.4.1. The Results on the Synthetic Data Sets

Two synthetic data sets randomly divided the data set into 4 parts, the first part is 10% of the data set, the second part is 50%, the third part is 10%, and the fourth part is 30%. The traditional ELM method is divided into two types of training: one selects the first part of labeled samples to train the model (ELMA), and another simulation employs the first part and second part labeled samples to train the model (ELMB). The SS-ELM, LapTELM, D-SOS-ELM, and LGS-OSELM select the first part as ℒ and second part as U. The third and fourth parts of data respectively are set as V and T for all algorithms.This experiment is mainly used to find the optimal. Table3 shows the experimental results of LGS-OSELM on these synthetic data sets. In Table 3, it can be seen that when the parameter k increases from 5 to 20, classification accuracy is different. On the Flame data set, when the number of nearest neighbors k is 15, the experimental result is the best; for Jain data sets, the number of nearest neighbors k is 10.Table 3
Classification accuracy of LGS-OSELM with differentk on V for two synthetic data sets.
kFlameJainAverage (%)Best (%)Average (%)Best (%)595.4098.3492.5692.311096.5499.0197.361001598.8310095.311002097.5010094.1699.65Table4 shows the classification accuracy of ELM, SS-ELM, D-SOS-ELM, LapTELM, and LGS-OSELM on the synthetic data sets, where “Acc%” denotes the average classification accuracy on the testing set T according to the optimal parameters “(c),” “(c1,c2),” “(c, λ, k),” or “(c, k).” It can be seen from Table 4 that the ELMB has the best experimental result; ELMA is the worst. In terms of the number of samples used in training, the ELMA model is the least, so the result is the worst. The number of training samples for SS-ELM and LGS-OSELM is the same as that of ELMB. The difference is that the ELMB model is trained by a large number of labeled samples, while SS-ELM, D-SOS-ELM, LapTELM, and GLS-OSELM have very few labeled samples, and most of the samples are unlabeled, and it is obvious that when the labeled data are relatively small, LGS-OSELM outperforms the ELMa, D-SOS-ELM, LapTELM, and SS-ELM. On the Flame data set, the classification accuracy achieved by LGS-OSELM is 8% and 19.1% higher than these of the ELMa and SS-ELM, respectively. On the Jain data set, the classification accuracy achieved by LGS- OSELM is 13.13% and 21.61% higher than those of the ELMa and SS-ELM, respectively. The proposed LGS-OSELM is slightly excellent than D-SOS-ELM and LapTELM on two synthetic data sets.Table 4
Average classification accuracy of the ELM, SS-ELM, D-SOS-ELM, LapTELM, and LGS-OSELM onT for two synthetic data sets.
DatasetELMaELMBSS-ELMD-SOS-ELMLapTELMLGS-OSELMAcc% (c)Acc% (c)Acc% (c, A, k)Acc% (c)Acc% (C1, c2, k)Acc% (c,k)Flame79.7399.2390.8397.6797.8698.83(100)(100)(100, 102, 5)(100)(100, 0.01, 15)(100, 15)Jain76.5598.3985.0394.8795.6498.16(10−2)(10−2)(10−2, 102, 15)(102)(10−2, 0.05, 10)(10−2, 10)


## 4.4.2. The Results on Real-World Data Sets

The performance comparison on real-world data sets is displayed in Table5. Note that the training sets of the ELM consist of only the 10% labeled data. As can be seen from Table 5, LGS-OSELM, LapTELM, and D-SOS-ELM achieve better results than ELM, Lap-SVM, STAR-SVM, and SS-ELM on almost all the employed data sets. LGS-OSELM achieves better performance than LapTELM and D-SOS-ELM in most cases. From the last line of Table 5, the average classification accuracy achieved by LGS-OSELM is 82.15% on the T, which is 11.29%, 4.25%, 8.94%, 6.51%, 1.87%, and 1.12% higher than those of the ELM, STA-AVM, Lap-SVM, SS-ELM, D-SOS-ELM, and LapTEM across twelve data sets, respectively. The average classification accuracy achieved by LGS-OSELM is 83.63% on the U, which is 10.89%, 4.59%, 7.58%, 6.34%, 2.23%, and 2.21% higher than these of the ELM, STA-SVM, Lap-SVM, SS-ELM, D-SOS-ELM, and LapTELM across twelve data sets, respectively.Table 5
Classification accuracy of seven algorithms onT and U for real-world data sets.
ELMSTAR-SVMLap-SVMSS-ELMD-SOS-ELMLapTEMLGS-OSELMDatasetAcc%Acc%Acc%Acc%Acc%Acc%Acc%(c)(c, γ)(c, σ, k)(c, λ, k)(c)(c1, c2, k)(c, k)AustralianT66.7869.7369.9670.8176.3475.4377.34(10−2)(100, 0.7)(100, 10−2, 5)(102, 100, 10)(102)(102, 0.01, 10)(10−2, 10)U67.8270.5670.3870.8577.5677.4379.28(10−2)(100, 0.7)(100, 10−2, 5)(102, 100, 10)(102)(102, 0.01, 10)(10−2, 10)EEG-EyeT73.3594.2373.4276.3093.2594.6297.65(100)(102, 0.5)(102, 10−4, 15)(102, 100, 15)(102)(102, 0.01, 10)(102, 20)U75.4595.3674.3577.4292.1494.2597.58(100)(102, 0.5)(102, 10−4, 15)(102, 100, 15)(102)(102, 0.01, 10)(102, 20)GlassT64.5268.2364.5266.3570.4672.1373.34(10−2)(10−2, 0.7)(10−2, 102, 5)(10−2, 102, 5)(10−2)(10−2, 0.05, 15)(10−2, 10)U67.5271.6668.1466.6372.4572.3477.42(10−2)(10−2, 0.7)(10−2, 102, 5)(10−2, 102, 5)(10−2)(102, 0.05, 15)(10−2, 10)HeartT66.3472.4572.5264.5275.1976.2678.39(100)(102, 0.4)(102, 102, 5)(10−4, 102, 5)(10−4)(102, 0.1, 15)(100, 15)U67.1273.4375.7267.6475.2977.0977.65(100)(102, 0.4)(102, 102, 5)(10−4, 102, 5)(10−4)(102, 0.1, 15)(100, 15)MNISTT81.5788.2388.5689.4591.1892.5792.32(100)(102, 0.3)(102, 102, 5)(10−4, 10−2, 15)(10−4)(10−4, 0.05, 15)(102, 15)U83.0689.5689.8391.2691.6293.5292.35(100)(102, 0.3)(102, 102, 5)(10−4, 10−2, 15)(10−4)(10−4, 0.05, 15)(102, 15)MushroomT88.4590.3487.8490.3292.2489.1591.65(100)(102, 0.5)(100, 102, 5)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 5)U89.2192.6688.9390.4793.5489.2593.25(100)(102, 0.5)(100, 102, 5)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 5)IonosphereT82.1786.2584.7284.1490.9490.9492.46(10−2)(102, 0.7)(10−2, 102, 15)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 15)U82.4386.3787.8284.4391.4691.4292.86(10−2)(102, 0.7)(10−2, 102, 15)(10−2, 102, 10)(10−2)(10−2, 0.1, 15)(100, 15)PimaT62.5267.4465.4567.4468.1568.2469.23(10−4)(10−4, 0.3)(10−4, 100, 10)(10−4, 100, 15)(10−4)(10−4, 0.1, 5)(10−4, 5)U63.6869.0565.8669.1768.5269.4571.80(10−4)(10−4, 0.3)(10−4, 100, 10)(10−4, 100, 15)(10−4)(10−4, 0.1, 5)(10−4, 5)SeedsT58.4365.6160.4564.5668.6268.1667.52(102)(102, 0.7)(102, 10−2, 10)(10−4, 100, 10)(10−4)(10−4, 0.01, 20)(102, 20)U58.4467.2662.1669.3069.1268.6271.35(102)(102, 0.7)(102, 10−2, 10)(10−4, 100, 10)(10−4)(10−4, 0.01, 20)(102, 20)SonarT73.4577.3267.2777.3480.2486.7285.68(10−2)(10−2, 0.5)(10−4, 102, 5)(10−4, 10−2, 5)(10−2)(104, 0.05, 15)(104, 15)U73.5579.1769.1179.0082.1485.6887.19(10−2)(10−2, 0.5)(10−4, 102, 5)(10−4, 10−2, 5)(10−2)(104, 0.05, 15)(104, 15)USPS_20T84.4496.5293.0495.2696.7298.1597.75(10−2)(10−2, 0.6)(10−4, 102, 5)(10−2, 10−2, 5)(10−2)(10−2, 0.1, 5)(100, 5)U86.1492.3697.7695.5497.4298.2597.12(10−2)(10−2, 0.6)(10−4, 102, 5)(10−2, 10−2, 5)(10−2)(10−2, 0.1, 5)(100, 5)WineT48.3458.5251.3661.4158.6358.3262.52(102)(102, 0.7)(102, 102, 15)(102, 10−2, 15)(10−2)(102, 0.01, 15)(102, 15)U58.4760.5862.4565.7861.6360.5364.27(102)(102, 0.7)(102, 102, 15)(102, 10−2, 15)(10−2)(102, 0.01, 15)(102, 15)AverageT70.8677.9173.2275.6580.2981.0482.15U72.7479.0476.1377.2981.3981.4283.63


## 4.5. Experimental Analysis

For further analyzing the significant difference between the five models on the employed real-world data sets, we resort to the well-known Friedman test with the corresponding post hoc tests. The Friedman statistics is written as follows:(22)τχ2=12nmm+1∑i=1mR¯i2−mm+124,where m and n are the numbers of the involved models and data sets and R denotes the average ranks. The statistics τχ2 follows distribution with m-1 degree of freedom. The average ranks of the five models testing sets of all the real-world data sets are displayed in Table 6. Firstly, we make a comparison of the five models on T. It is clear that m = 7 and n = 12. Based on the average ranks displayed in Table 7, we can calculate the statistics τχ2  = 56.267, and(23)τF=n−1τχ2mn−1−τχ2,which is distributed following the F-distribution with (m − 1) and (m − 1) (n − 1) degrees of freedom. We can calculate τF=29.865. For the confidence level 90% α=01, Fα6,66=1.865 < τF=29.865, which means the null hypothesis that the there is no significant difference among the all models. Next, the Nemenyi post hoc test is performed to further compare the five models in pairs. The critical difference (CD) is obtained(24)CDα,m=qα,mmm+16n,where qα,m is a definite value that related to m and the significance level and the confidence level α. It is available by looking up specific table; qα,m=2.693 can be obtained querying specific table and then we obtain the critical difference CD0.1,6  = 2.375. If the average ranks of two models differ by at least CD, their performance is significantly different. Statistical variables and the difference of average ranks between and other models are displayed in Table 7. The statistical variable τF is greater than Fα6,66, which indicates the null hypothesis. We can summarize that the proposed LGS-OSELM performs significantly better than ELM, STAR-SVM, Lap-SVM, and SS-ELM. There is no significant difference between D-SOS-ELM, LapTELM, and our proposed LGS-OSELM on T and the U.Table 6
Relevant values in the Friedman test with Nemenyi’s post hoc test on the 12 real-world data sets.
τFd (ELM- LGS-OSELM)d (STAR-SVM- LGS-OSELM)d (LapSVM- LGS-OSELM)d (SS-ELM- LGS-OSELM)d (D-SOS-ELM- LGS-OSELM)d (LapTELM- LGS-OSELM)T56.26729.8655.21 > CD2.83 > CD4.29 > CD3.17 > CD1.13 < CD0.88 < CDU45.15215.5945.17 > CD2.92 > CD3.7 > CD2.83 > CD1.38 < CD1.46 < CDTable 7
Average ranks of the seven algorithms on the real-world data sets.
ELMSTAR-SVMLap-SVMSS-ELMD-SOS-ELMLapTEMLGS-OSELMT6.714.335.794.672.632.381.50U6.674.425.254.332.882.961.50We further analyze the performance of the proposed LGS-OSELM with different scales of labeled data. More specifically, for all the employed data sets, 70% data are randomly selected from each class to form the training set, and the others are left to compose the testing set. Then,Q% data in the training set are randomly selected as ℒ and others as U, where the values of Q are 10, 20, 30, 40, 50, 60, and 70. The learning results are displayed in Figure 7. It is easy to know that the classification accuracy of the five algorithms is generally increased with the increasing of the values of Q. That is to say, increasing the scale of labeled data can generally boost the performance of algorithm.Figure 7
The classification accuracy of the five algorithms with different scales of labeled data onT.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)Further, it can be found that the LGS-OSELM performs better than other baseline methods in most cases, and the advantage is more significant when the value ofQ is small. In specific, no matter what the scale of labeled data is, the proposed LGS-OSELM performs better than SS-ELM, Lap-SVM and STAR-SVM, D-SOS-ELM, and LapTELM.In order to evaluate the computational efficiency of the proposed algorithm, we present the training times of six semisupervised algorithms on different data sets in Table8. From this table, training time is essentially the same order of magnitude. The proposed algorithm is the most efficient among the six algorithms, in which LGS-OSELM spends the least amount of training time on four of the twelve data sets. Because the proposed method is based on distance and density, the calculation of the local gravitation is very time consuming.Table 8
Training time of classifiers on the real-world data sets (in seconds).
STAR-SVMLap-SVMSS-ELMD-SOS-ELMLapTEMLGS-OSELMAustralian0.03140.02210.02440.02550.02900.0276EEG-Eye1.86151.45331.35321.54231.07321.0234Glass0.45120.35120.15260.13430.15120.1543Heart0.02520.00850.00520.00420.00840.0055MNIST0.25230.08350.05410.05430.08420.0643Mushroom2.34321.31412.84833.42530.93321.0023Ionosphere0.05560.03560.00420.00560.00530.0024Pima0.02420.03080.02010.05230.04710.0435Seeds0.0920.05740.07420.04710.04070.0427Sonar0.13450.09450.08520.05950.04730.0425USPS_200.41520.19330.2120.19230.23530.2391Wine0.31520.09330.12120.09230.21350.0912To summarize, two synthetic data sets and 12 groups of real-world data sets are used as the experimental data. For the two synthetic data sets, two types of ELM (ELM trained by few labeled samples and ELMB trained by abundant labeled samples), SS-ELM D-SOS-ELM and LapTELM, are employed as comparisons. The experimental results showed that the LGS-OSELM, trained by few labeled samples and abundant unlabeled samples, has the higher accuracy than SS-ELM, D-SOS-ELM, LapTELM, and ELM is and slightly less than ELMb. For real-world data sets, the experimental results showed that the LGS-OSELM has higher prediction accuracy than state-of-the-art semisupervised methods including STAR-SVM, Lap-SVM and SS-ELM, D-SOS-ELM, and LapELM. For further fair comparisons of five methods, we resort to the well-known Friedman test with the corresponding post hoc tests, which indicates that the proposed LGS-OSELM performs significantly better than ELM, STAR-SVM, Lap-SVM, and SS-ELM, and there is no significant difference between D-SOS-ELM, LapTELM, and our proposed LGS-OSELM on the T and the U across 12 real-world data sets. Experiments with the different scale of the labeled sample are carried out to further explore discrepancy among the five methods. Results indicate that the classification accuracy of these five algorithms generally increases as the scale of the labeled sample increases, and no matter what the scale of labeled data is, the proposed LGS-OSELM performs better than other baseline methods.


## 5. Conclusion

In this work, a local gravitation-based incremental semisupervised classification algorithm is proposed. The proposed algorithm uses the local gravitation measures (CE and CO) between unlabeled data and label database on analysis of the relationship between sample and their neighborhoods to screen the unlabeled samples with high confidence joining OS-ELM to improve the accuracy of the semisupervised learning. For many complex data sets, the traditional method of using distance as a similarity index is difficult to realize the correct calculation of the similarity between samples, which leads to the low accuracy of semisupervised learning. By calculating the local gravitation measures (CE and CO) of unlabeled samples, this method can effectively determine the similarity of the samples and can evaluate the reliability of the unlabeled samples. Because each iteration selects the samples with high confidence for incremental semisupervised learning, the accuracy of learning is greatly improved compared with the existing methods. The average test accuracy of the proposed model on two synthetic data sets and eleven real-world data sets are higher than that of SS-ELM, Lap-SVM, and STAR-SVM. This method can make use of a small number of labeled data and a large number of unlabeled data to train a practicable model, which can improve the utilization of data, reduce the manufacturing cost of data labeling, and improve the efficiency of learning.



---

*Source: 1735573-2022-05-11.xml*