# Beyond Human Vision: Unlocking the Potential of Augmented Reality for Spectral Imaging
## 1 Introduction
In this case, AR can help visualize cross-sectional maps with subsurface 3D reconstructions in real time, holding interesting potential for applications in subjects such as artwork restoration, where understanding complex stratigraphy or volumetric characteristics may be critical. From the perspective of AR, the key idea is to superimpose digital information into the real world, contrasting with virtual reality which immerses the user in a fully synthetic 3D space, often disconnecting them from their physical surroundings. Thus, AR maintains a connection with the physical space, allowing for a more contextual and interactive experience with digital models and data [15], supporting contextual awareness, in situ annotation, and collaborative analysis within shared physical environments. These characteristics are particularly relevant in domains such as cultural heritage diagnostics and the mining industry, where users may need to compare physical samples, access physical documentation, or interact collaboratively in shared spaces. In this context, spectral imaging holds the potential to add novel multiple layers of information, superimposing them both onto these digital models or even in real-world objects, allowing humans to see beyond human vision and enable deeper analysis and better insights [16; 17]. Indeed, some pioneering results in the literature already hint towards these directions, either by utilizing projection methods [18] or interactive headsets, with the latter significantly enhancing the experience via additional embedded sensors. For example, ref. [17] explores the combination of a fiber optic Raman probe and an augmented reality localization strategy and interface to create a visualization of a chemical image of samples, overlaid yet not anchored to the physical object. Regarding hyperspectral imaging, interesting approaches include medical applications such as the classification of tissues during in vivo surgical procedures [19] and a toolset for guided brain tumor phantom resection which utilizes an HoloLens AR headset [16], to mention a few illustrative examples. In the geological sciences domain, ref. [20] presents a toolset to analyze rock surfaces and mineral types, which may be promising to assist in situ operations, but again missing the contextual interpretation of 3D models. In general, although these approaches demonstrate the potential of combining AR and spectral imaging, the analytical potential of the solution is a characteristic often overlooked. To the best of our knowledge, no toolset fuses interactive 3D digital models with spectral imaging, adding the capabilities to arbitrarily scale the samples to enhance details, access other perspectives, or interactively explore spectral information. In this manuscript, we describe a full-stack methodology and pipeline to integrate spectral imaging datasets onto 3D object digital models, deploying it as a user-friendly interface that provides an interactive environment to explore the object and dataset characteristics. For that purpose, a Unity software solution for spectral analysis will be developed for the Microsoft Hololens 2 device, taking advantage of the Mixed Reality Feature Toolkit needed to add interaction such as zoom adjustment and rotation, in addition to more in-depth spectrum analysis tools. The results presented demonstrate the potential of the fusion of AR and spectral imaging to open new avenues for research and innovation, enabling us to perceive and interact with our world in ways previously unattainable and enhancing human vision capabilities.