Development of real-time multimodal optical imaging systems with augmented reality
Creators: name: Karchegani, Maziyar Askari
Publisher: name: University of Iowa
Year: 2026
Subjects: Distance Aware Registration
Fluorescence Imaging
Image Registration
Stereoscopic Imaging
Stereoscopic Magnification
Thermal Imaging
Optics

description: Advancements in the development of imaging software and hardware, has led to emerging utilization of optical imaging in medicine and industry. There is growing research interest in integration of multiple optical modalities, offering structural and functional information simultaneously. Fluorescence and thermal imaging are examples of functional imaging, while color imaging provides structural information. Current multimodal optical imaging systems rely either on a hardware or a software approach for image registration. Hardware implementation solutions are often heavy and bulky structures with relatively high costs in mechanical design and optical componentry. Feature-based registration methods on the other hand, represent a pure software approach that rely on mutual features between source and target. Once applied across different optical wavelength bands, these methods become inaccurate and inconsistent due to low number of matching features among different imaging modalities.
description: Leveraging an integrated design, advantages of hardware and software can be combined to overcome the shortcomings. Aimed at the improving multimodal optical imaging, my doctoral thesis embodies hardware and software development, system characterization, and experimental investigation of novel optical imaging systems in the medical field, with a focus on registration of multiple optical imaging modalities, and augmented reality in visualization.
description: I designed a broadly applicable cyber-physical framework to perform real-time multimodal distance-aware image registration. Based on this framework, a prototype multimodal imaging system was designed and built for real-time distance-aware image registration of thermal and color imageries. The hardware and algorithm development and accuracy characterization were performed using the prototype. In another work, I developed a multimodal intraoperative imaging system for computer-aided surgery to register fluorescence and color imaging modalities based on the designed framework. A prototype was built and successfully applied to image-guided surgery in biological models, including live swine. In exploring the applications of our framework, a novel method for high-throughput temperature screening was developed. The method is applicable to fever monitoring in facilities with large crowds. In addition, the industrial applications of colorthermal and near infrared-thermal dual-modal imaging systems were explored.
description: To improve the user experiment with wearable imaging systems in clinical applications, I developed augmented reality systems with dynamic 3-dimensional image magnification capability, presented on partially offset display panels to increase situational awareness during surgical procedures. These systems pave the way for future clinical translation. The framework and systems developed in this work have shown great potential to facilitate various medical and industrial real-time imaging applications.