# Beyond Human Vision: Unlocking the Potential of Augmented Reality for Spectral Imaging
## 2 Method
### Step 2: Asset Management  
Building on the datasets acquired in the first step, step 2 comprises the necessary tasks to analyze the spectral dataset, construct the relevant images, and merge them with the 3D digital model of the sample. This step is performed in the form of Python scripts and streamlined user-centric actions. First, having the spectral dataset acquired, it is necessary to analyze it and construct the relevant maps to be merged with the 3D digital model. While this process could be performed in the headset itself and incorporated into the software solution, we opted to perform it previously for efficiency purposes. Indeed, the analysis of spectral data typically requires a significant amount of pre-processing such as baseline removal [28] or filtering [29], which, in addition to the fact that spectral imaging datasets are typically large in memory size (in the order of GB of data), require a lot of processing power for efficient operation. For this work, and having utilized LIBS, we applied a baseline removal procedure using Asymmetrical Least Squares Smoothing (ALSS) [28], and the main spectral lines corresponding to each characteristic spectral signature were extracted. Next, we select a set  
Figure 1: Schematic of the described methodology to deploy an interactive visualization tool for a HoloLens 2 device, broken into the three steps referred to in the manuscript. **(Left, bottom)** Spectral map of the cobalt emission line at 345.3 nm and 3D model of the analyzed sample. **(Middle, bottom)** Illustrative results of the superimposition of the 3D model with the spectral data correspondent to a cobalt emission line at 345.3 nm. **(Right, bottom)** Illustration of the final user-friendly interface in unity. of \(N_{i}\) relevant spectral lines and construct the spectral maps by integrating them in the \(\lambda\) dimension around the selected line and applying a spatial Gaussian filter. All of these steps are performed in a single Python script, outputting a series of \(N_{i}\) images for the relevant features selected (see Figure 2). An additional image with a mean spectrum is also saved to a file to be displayed in the final solution as further described in step 3. Subsequently, and in order to merge with the 3D digital model, we need to superimpose the spectroscopic data in the corresponding surface of the sample. This is performed using a mesh matching algorithm previously presented in the literature [6], which consists of determining the following three transformations: translation, rotation, and scaling between two point clouds, one in the 3D model and the other one in the 2D spectral image, minimizing the mean square error between the clouds. Mathematically speaking, these selected points, \(\vec{p}_{i}\) and \(\vec{q}_{i}\), corresponding to the same spatial region in both the 3D model and the 2D image, can be represented by two matrices (Equation (1)) that contain the set of points intended to be aligned as follows:  
\[\mathbf{P}= \begin{pmatrix}x_{1}&y_{1}&z_{1}\\ x_{2}&y_{2}&z_{2}\\ \vdots&\vdots&\vdots\\ x_{n}&y_{n}&z_{n}\end{pmatrix} \quad\text{and}\quad\mathbf{Q}= \begin{pmatrix}i _{1}&j_{1}&k_{1}\\ i_{2}&j_{2}&k_{2}\\ \vdots&\vdots&\vdots\\ i_{n}&j_{n}&k_{n}\end{pmatrix}  \tag{1}\]  
According to ref. [30], this minimization problem can be solved by computing the cross-covariance matrix, \(\boldsymbol{\Sigma}_{PQ}\), for the point patterns \(\mathbf{P}\) and \(\mathbf{Q}\) as  
\[\boldsymbol{\Sigma}_{PQ}=\frac{1}{n}\sum_{i=1}^{n}(\vec{q}_{i}-\vec{q}_{i})^{ T}(\vec{p}_{i}-\vec{u}_{p}) \tag{2}\]  
By decomposing the matrix \(\boldsymbol{\Sigma}_{PQ}\) through Singular Value Decomposition (SVD), i.e., \(\boldsymbol{\Sigma}_{PQ}=\mathbf{U}\mathbf{D}\mathbf{V}^{T}\), it is possible to achieve the optimal transformation parameters that minimize the mean squared error given by the following:  
\[