# Robust Hessian Locally Linear Embedding Techniques for High-Dimensional Data
## 2 Reformulating HLLE through Truncation Function from Differential Manifold
### Reformulating Hessian LLE
Together with \(u_{i}^{i}=\langle V^{i},x_{i}-x_{i}\rangle=0\), we have:  
\[f(x_{j})=g(u_{j}^{i})=g(0)+(\
abla g)^{T}u_{j}^{i}+\frac{1}{2}u_{j}^{i}{}^{T}H_ {f}^{i}u_{j}^{i}+O(\|u_{j}^{i}\|^{3}) \tag{5}\]  
Over the neighborhood of \(u_{j}^{i}\), we develop the operator \(b^{i}\) that approximates the function \(g(u_{j}^{i})\) by its projection on the basis \(U_{j}^{i}=\{1,u_{j_{1}}^{i},\ldots,u_{j_{d}}^{i},(u_{j_{1}}^{i})^{2},\ldots,(u_ {j_{d}}^{i})^{2},\ldots,u_{j_{1}}^{i}\times u_{j_{2}}^{i},\ldots,u_{j_{d-1}}^{i }\times u_{j_{d}}^{i}\}\), we have:  
\[f(x_{j})=g(u_{j}^{i})=(b^{i})^{T}\cdot U_{j}^{i} \tag{6}\]Let \(b^{i}=[g(0),\
abla g,h^{i}]\in\mathbb{R}^{1+d+d(d+1)/2}\); then, \(h^{i}\in\mathbb{R}^{d(d+1)/2}\) is the vector form of local Hessian matrix \(H^{i}_{f}\) over neighborhood \(N(x_{i})\). The least-squares estimation of the operator \(b^{i}\) can be obtained by:  
\[\operatorname*{argmin}_{b^{i}}\sum_{j=1}^{k}(f(x_{j})-\langle b^{i},U^{i}_{j} \rangle)^{2} \tag{7}\]  
The least-squares solution is \(b^{i}=(U^{i})^{\intercal}f^{i}\), where \(f^{i}=[f(x_{1}),\dots,f(x_{k})]\in\mathbb{R}^{k}\), \(U^{i}=[U^{i}_{1};U^{i}_{2};\dots;U^{i}_{k}]\in\mathbb{R}^{k\times(1+d+d(d+1)/2)}\) and \((U^{i})^{\intercal}\) denotes the pseudo-inverse of \(U^{i}\). Note that the the last \(d(d+1)/2\) components of \(b^{i}\) correspond to \(h^{i}\), which is the vector form of local Hessian matrix \(H^{i}_{f}\). The local Hessian operator \(H^{i}\in\mathbb{R}^{(d(d+1)/2)\times k}\) is constructed by the last \(d(d+1)/2\) rows of \((U^{i})^{\intercal}\), and we have: \(h^{i}=H^{i}\cdot f^{i}\). Therefore, the local quadratic form \(\|H^{tan}_{f}(x_{i})\|_{F}^{2}\) can be estimated with:  
\[\|H^{tan}_{f}(x_{i})\|_{F}^{2}=(h^{i})^{T}(h^{i})=(f^{i})^{T}(H^{i})^{T}(H^{i} )\big{(}f^{i}\big{)} \tag{8}\]  
\(\|H^{tan}_{f}(x_{i})\|_{F}^{2}\) is a local object defined on the manifold, while \(\mathcal{H}(f)\) is a global object defined on the manifold. We use the truncation function from the differential manifold to connect them together. The truncation function is an important tool in the differential manifold to establish a relationship between the local and global properties of the manifold. Suppose \(U\) and \(V\) are two nonempty subsets of a smooth manifold \(\mathcal{M}\), where \(\bar{V}\) is compact and \(\bar{V}\subset U\) (\(\bar{V}\) is the closure of \(V\)). Then the truncation function [24] is defined as a smooth function \(s:\mathcal{M}\rightarrow\mathbb{R}\) such that:  
\[s(p)= \begin{cases}1,&p\in V\\ 0,&p\
otin U\end{cases}  \tag{9}\]  
The truncation function \(s\) can be discretely approximated by the 0-1 selection matrix \(S^{i}\in\mathbb{R}^{N\times k}\) and an entry of \(S^{i}\) is defined as:  
\[(S^{i})_{pq}= \begin{cases}1,&p=N_{i}\{q\}\\ 0,&p\
eq N_{i}\{q\}\end{cases}  \tag{10}\]  
where \(N_{i}=\{i_{1},\dots,i_{k}\}\) denotes the set of indices for the \(k\)-nearest neighborhood of data point \(x_{i}\). Thus, the local object \(f^{i}=[f(x_{1}^{i}),\dots,f(x_{k}^{i})]\in\mathbb{R}^{k}\) can be expressed by \(f^{i}=(S^{i})^{\intercal}f\), where \(f=[f(x_{1}),\dots,f(x_{N})]\in\mathbb{R}^{N}\) is a global object defined on the whole manifold. By using the truncation function and its discrete approximation, together with Equation (8), the global functional \(\mathcal{H}\) can be discretely approximated as follows:  
\[ \begin{split}\mathcal{H}(f)&=\int_{\mathcal{M}}\|H^{ tan}_{f}(x)\|_{F}^{2}dx=\frac{1}{N}\sum_{i=1}^{N}\|H^{tan}_{f}(x_{i})\|_{F}^{2}\\ &=\frac{1}{N}\sum_{i=1}^{N}(f^{i})^{T}(H^{i})^{T}(H^{i})(f^{i})= \frac{1}{N}\sum_{i=1}^{N}f^{T}S^{i}(H^{i})^{T}(H^{i})(S^{i})^{T}f\\ &=f^{T}\mathcal{H}f\end{split}  \tag{11}\]  
where \(\mathcal{\bar{H}}=\frac{1}{N}\sum_{i=1}^{N}S^{i}(H^{i})^{T}(H^{i})(S^{i})^{T}  
The global embedding coordinates \(Y=[y_{1},y_{2},\ldots,y_{N}]\in\mathbb{R}^{d\times N}\) can be obtained by minimizing the functional \(\mathcal{H}(f)\). Let \(y=f=[f(x_{1}),\ldots,f(x_{N})]\) be a row vector of \(Y\).