Diagnosis of Osteosarcoma Based on Multimodal Microscopic Imaging and Deep Learning

Authors:
raw_author_name: Zihan Wang
raw_affiliation_string: Department of Biomedical Engineering, College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu 211106, P. R. China
raw_author_name: Bing Wang
raw_affiliation_string: Department of Biomedical Engineering, College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu 211106, P. R. China
raw_author_name: Qingxia Wu
raw_affiliation_string: Department of Biomedical Engineering, College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu 211106, P. R. China
raw_author_name: Lan Li
raw_affiliation_string: Department of Pathology, The Second Xiangya Hospital of Central South University, Changsha, Hunan 410011, P. R. China
raw_author_name: Huijie Wang
raw_affiliation_string: Department of Biomedical Engineering, College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu 211106, P. R. China
raw_author_name: Chao Tu
raw_affiliation_string: Department of Orthopaedics, The Second Xiangya Hospital of Central South University, Changsha, Hunan 410011, P. R. China; Shenzhen Research Institute of Central South University Shenzhen, Guangdong 518063, P. R. China
raw_author_name: Jianhua Yin
raw_affiliation_string: Department of Biomedical Engineering, College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu 211106, P. R. China

Abstract: Osteosarcoma is the most common primary bone tumor with high malignancy. It is particularly necessary to achieve rapid and accurate diagnosis in its intraoperative examination and early diagnosis. Accordingly, the multimodal microscopic imaging diagnosis system constructed by bright field, spontaneous fluorescence and polarized light microscopic imaging was used to study the pathological mechanism of osteosarcoma from the tissue microenvironment level and achieve rapid and accurate diagnosis. First, the multimodal microscopic images of normal and osteosarcoma tissue slices were collected to characterize the overall morphology of the tissue microenvironment of the samples, the arrangement structure of collagen fibers and the content and distribution of endogenous fluorescent substances. Second, based on the correlation and complementarity of the feature information contained in the three single-mode images, combined with convolutional neural network (CNN) and image fusion methods, a multimodal intelligent diagnosis model was constructed to effectively improve the information utilization and diagnosis accuracy. The accuracy and true positivity of the multimodal diagnostic model were significantly improved to 0.8495 and 0.9412, respectively, compared to those of the single-modal models. Besides, the difference of tissue microenvironments before and after cancerization can be used as a basis for cancer diagnosis, and the information extraction and intelligent diagnosis of osteosarcoma tissue can be achieved by using multimodal microscopic imaging technology combined with deep learning, which significantly promoted the application of tissue microenvironment in pathological examination. This diagnostic system relies on its advantages of simple operation, high efficiency and accuracy and high cost-effectiveness, and has enormous clinical application potential and research significance.