# Correntropy-Based Constructive One Hidden Layer Neural Network
## 4 Proposed Method
### C2n2: Objective Function for Training the New Node  
In this subsection, we combine the idea of constructive SLFN with the idea of cornertropy and propose a new strong constructive network that is robust to impulsive noise. The proposed method employs correntropy as the objective function to adjust the input \((w_{L},b_{L},L=1,\ldots,\infty)\) and output \((\beta_{L},L=1,\ldots,\infty)\) parameters of the network. To the best of our knowledge, it is the first time that correntropy with the Gaussian kernel has been employed for training all the parameters of a constructive SLFN. C2N2 starts with zero hidden nodes. The first hidden node is added to the network. First, the input parameters \((w_{1},b_{1})\) of the hidden node are calculated by employing a correntropy objective function with a Gaussian kernel. Then, they are fixed and the output parameter \((\beta_{1})\) of the node is adjusted by the correntropy objective function with Gaussian kernel. After the parameters of the first node are obtained, they are then fixed and the next hidden node is added to the network and trained. This process is iterated until the stopping condition is satisfied. The proposed method can be viewed as an extension of CCOEN [28] with the following differences:  
1. In contrast to CCOEN, which uses correntropy with a sigmoid kernel to adjust the input parameters of a cascade network, the proposed method uses correntropy with a Gaussian kernel to adjust the whole parameters of an SLFN. 2. CCOEN uses correntropy to adjust the input parameters of the new node in a cascade network to provide a more robust method. However, the output parameter of the new node in a cascade network is still adjusted based on the least mean square error. In contrast, the proposed method uses correntropy with Gaussian kernel to obtain both the input and output parameters of the new node in a constructive SLFN. Therefore, the proposed method is more robust than CCOEN and other networks introduced in [3] when the dataset is contaminated by impulsive noise. 3. Employing Gaussian kernel for correntropy as the objective function to adjust the network's parameters provides a closed-form formula introduced in the next section. In other words, both the input and output parameters are adjusted by two closed-form formulas. For the proposed network, each newly added node (\(L\)-th added node where \(L=1,\ldots,\infty\)) is trained using the two phases. In the first phase, the new node is selected from \(\mathcal{G}\), using the following optimization problem: where  
\[g_{L}^{\text{sim}(e)}=\text{argmax}_{g_{L}\in\mathcal{G}}\{V(g_{L})\},\]  
\[V(g_{L})=E\Bigg{(}\frac{1}{\sqrt{2\pi\sigma}}\exp\Bigg{(}-\frac{\|e_{L-1}-k_{ L}g_{L}\|^{2}}{2\sigma^{2}}\Bigg{)}\Bigg{)}.\]  
From the definition of the kernel, the most similar \(\Big{(}g_{L}^{\text{sim}(e)}\Big{)}\) activation function to the residual error of \(L-1\) nodes network is selected from \(\mathcal{G}\) as this node is selected to maximize:  
\[V(g_{L})= E(\langle\Phi(e_{L-1}),\Phi(k_{L}g_{L})\rangle)\] \[,k_{L}\in\mathbb{R}-\{0\}\]  
where \(\Phi\) is feature mapping. Consequently, the biggest reduction in error is obtained and the network has a more compact architecture. In the second phase, the output parameter \((\beta_{L})\) of the new node is adjusted whereby  
\[\beta_{L}^{\text{sim}(e)}=\text{argmax}_{\beta_{L}}\{V(\beta_{L}),\beta_{L} \in\mathbb{R}\},\]  
\[V(\beta_{L})=E\Bigg{(}\frac{1}{\sqrt{2\pi\sigma}}\exp\Bigg{(}-\frac{\left\|e _{L-1}-\beta_{L}g_{L}^{\text{sim}(e)}\right\|^{2}}{2\sigma^{2}}\Bigg{)}\Bigg{)}.\]  
These two phases are iterated and a new node is added in each iteration until the certain stopping condition is satisfied. This is discussed in Section 5.