# Correntropy-Based Constructive One Hidden Layer Neural Network
## 2 Mathematical Notations, Definitions and Preliminaries
### Measure Space, Probability Space and Function Space  
As mentioned in [3], let \(\mathcal{X}\) be the input space that is a bounded measurable subset in \(\mathbb{R}^{d}\) and \(L^{2}(\mathcal{X})\) be the space of all function \(f\) that is \(\int_{\mathcal{X}}(f(x))^{2}d\mu(x)<\infty\). For \(u,v\in L^{2}(\mathcal{X})\), the inner product is defined as follows:  
\[\langle u,v\rangle_{\mu}:=\int_{\mathcal{X}}u(x)v(x)d\mu(x).\]  
where \(\mu\) is a positive measure on input space. Under the measure \(\mu\), the \(l2\) norm in \(L^{2}(\mathcal{X})\) space is denoted as \(|\cdot|_{2}\). The closeness between \(u\) and \(v\) is measured by  
\[|u-v|_{2}=\left(\int_{\mathcal{X}}(u(x)-v(x))^{2}d\mu(x)\right)^{\frac{1}{2}}\]  
The angle between \(u\) and \(v\) is defined by  
\[\theta_{u,v}:=\arccos\biggl{(}\frac{\langle u,v\rangle_{\mu}}{|u|_{2}|v|_{2}} \biggr{)}\]  
**Definition 1** ([28; 29]). : _Let \(W\) be a probability space that is a measure space with a total measure one. This space is represented as follows:_  
\[W=(\Omega,\mathcal{F},\mathcal{P})\]  
_where \(\Omega\) is its sample space. In this paper, \(\Omega\) is considered a compact subset of \(\mathbb{R}^{d}\), \(\mathcal{F}\) is a sigma-algebra of events and \(\mathcal{P}\) is a probability measure that is a measure on \(\mathcal{F}\) with \(\mathcal{P}(\Omega)=1\)._  
**Definition 2** ([28; 29]). : _Let \(\mathcal{L}^{p}(\Omega,\mathcal{F},\mathcal{P}),1\leq p<\infty\) be a set of all \(p\) - integrable random variables \(X:\Omega\rightarrow\mathbb{R}\), i.e.,_  
\[\|X\|_{p}=\left(\int_{\Omega}X^{p}d\mathcal{P}\right)^{\frac{1}{p}}=(E(X^{p}) )^{\frac{1}{p}}<\infty\]  
_This is a vector space and the inner product in this space is defined as follows:_  
\[\langle X,Y\rangle:=\int_{\Omega}X(\omega)Y(\omega)d\mathcal{P}=E(XY)\]  
_where \(X,Y\in\mathcal{L}^{p}(\Omega,\mathcal{F},\mathcal{P})\) and \(E(\cdot)\) is expectation in probability theory. The closeness between two random variables \(X\) and \(Y\) is measured by \(\mathcal{L}^{p}(\Omega,\mathcal{F},\mathcal{P})\) norm:_  
\[\|X-Y\|_{p}=\left(\int_{\Omega}|X(\omega)-Y(\omega)|^{p}d\mathcal{P}\right)^{ \frac{1}{p}}=\]  
In ITL, the correlation between random variables is generalized to correntropy, which is a measure of similarity [15]. Let \(X\) and \(Y\) be two given random variables; the correntropy in the sense of [15] is defined as  
\[V_{\mathcal{M}}(X,Y):=E(k_{\mathcal{M}}(X,Y))\]  
where \(k_{\mathcal{M}}(\cdot,\cdot)\) is a Mercer kernel function. In general, a Mercer kernel function is a type of positive semi-definite kernel function that satisfies Mercer's condition. Formally, a symmetric function \(k_{M}\) is a Mercer kernel function if, for any positive integer \(m\) and any set of random variables \(X_{1},\ldots,X_{m}\in\mathcal{L}^{p}(\Omega,\mathcal{F},\mathcal{P})\), the corresponding Gram matrix \(K_{ij}=k_{M}(X_{i},X_{j})\) is positive semi-definite. In the definition of correntropy, \(E(\cdot)\) implies the expected value of the random variable and \(\mathcal{M}\) is replaced by \(\alpha\) or \(\sigma\) if the sigmoid or Gaussian kernel (radial basis) are used, respectively. In our recent work [28], we use a sigmoid kernel, which is defined as  
\[k_{\alpha}(X,Y)=\tanh(\alpha\langle X,Y\rangle+c),\]  
where \(\alpha,\ c\in\mathcal{R}\) are scale and offset hyperparameters of the sigmoid kernel. The offset parameter \(c\) in the sigmoid kernel influences the shape of the kernel function. A higher value of \(c\) leads to a steeper sigmoid curve, making the kernel function more sensitive to variations in the input space. It is important to note that the choice of hyperparameters, including \(c\) and \(\alpha\), can significantly impact the performance of a machine learning model using the sigmoid kernel. These parameters are often tuned during the training process to optimize the model for a specific task or dataset.