Abstract PR005: Deep learning-based multimodal integration of histology and genomics improves cancer origin prediction
Authors: Shaban Muhammad , Lu Ming Y. , Williamson Drew F.K. , Chen Richard J. , Lipkova Jana , Chen Tiffany Y. , Mahmood Faisal 
Published: January 2023

Accurate identification of a primary origin of metastatic tumors is essential for optimizing treatment and involves the integration of multiple forms of data during the examination of tissue by a pathologist. However, despite the use of highly sensitive and specific immunohistochemical stains for some cell lineages, pathologists cannot reliably determine the origin of every metastatic tumor, with 1-2% classified as cancers of unknown primary (CUP) even with the integration of other clinical data [1]. Previous work has shown the possibility of using artificial intelligence algorithms to predict primary origin using histology [2] or different forms of molecular data, including genomics [3], transcriptomics [4], or methylation profiles [5]. We present a multimodal deep learning algorithm that leverages routinely acquired histology slides, associated clinically-available genomics data, and patient sex to classify tumors into 18 different primary origins. Our approach shows substantial improvement over unimodal deep learning using histology or genomic data alone, achieving an accuracy of 88.1% and 92.0% on a held-out test (n=4,881) and external test set (n=660), respectively. Furthermore, on CUP cases (n=283), we observed an agreement of 85.5% between the model’s three most likely predicted origins and the differential diagnoses assigned in the associated pathology reports. At test time, our flexible model design enables origin prediction to be made from only histology or genomics alone, if necessary due to missing data. Additionally, our model allows us to perform interpretability studies to observe which parts of the histology and which genes contribute most to the prediction of a particular origin, a potentially useful tool for quality control and knowledge discovery. References: [1] Rassy, Elie, and Nicholas Pavlidis. "Progress in refining the clinical management of cancer of unknown primary in the molecular era." Nature reviews Clinical oncology 17.9 (2020): 541-554. [2] Lu, Ming Y., et al. "AI-based pathology predicts origins for cancers of unknown primary." Nature 594.7861 (2021): 106-110. [3] Jiao, Wei, et al. "A deep learning system accurately classifies primary and metastatic cancers using passenger mutation patterns." Nature communications 11.1 (2020): 1-12. [4] Grewal, Jasleen K., et al. "Application of a neural network whole transcriptome–based pan-cancer method for diagnosis of primary and metastatic cancers." JAMA network open 2.4 (2019): e192597-e192597. [5] Zheng, Chunlei, and Rong Xu. "Predicting cancer origins with a DNA methylation-based deep neural network model." PloS one 15.5 (2020): e0226461.

Citation Format: Muhammad Shaban, Ming Y. Lu, Drew F.K. Williamson, Richard J. Chen, Jana Lipkova, Tiffany Y. Chen, Faisal Mahmood. Deep learning-based multimodal integration of histology and genomics improves cancer origin prediction [abstract]. In: Proceedings of the AACR Special Conference: Cancer Metastasis; 2022 Nov 14-17; Portland, OR. Philadelphia (PA): AACR; Cancer Res 2022;83(2 Suppl_2):Abstract nr PR005.