Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer’s Disease using structural MR and FDG-PET images
Authors: Lu Donghuan , Popuri Karteek , Ding Gavin Weiguang , Balachandar Rakesh , Beg Mirza Faisal ,   Alzheimer’s Disease Neuroimaging Initiative, Weiner Michael , Aisen Paul , Petersen Ronald , Jack Cliford , Jagust William , Trojanowki John , Toga Arthur , Beckett Laurel , Green Robert , Saykin Andrew , Morris John , Shaw Leslie , Kaye Jefrey , Quinn Joseph , Silbert Lisa , Lind Betty , Carter Raina , Dolen Sara , Schneider Lon , Pawluczyk Sonia , Beccera Mauricio , Teodoro Liberty , Spann Bryan , Brewer James , Vanderswag Helen , Fleisher Adam , Heidebrink Judith , Lord Joanne , Mason Sara , Albers Colleen , Knopman David , Johnson Kris , Doody Rachelle , Villanueva-Meyer Javier , Chowdhury Munir , Rountree Susan , Dang Mimi , Stern Yaakov , Honig Lawrence , Bell Karen , Ances Beau , Carroll Maria , Creech Mary , Franklin Erin , Mintun Mark , Schneider Stacy , Oliver Angela , Marson Daniel , Grifth Randall , Clark David , Geldmacher David , Brockington John , Roberson Erik , Love Marissa Natelson , Grossman Hillel , Mitsis Efe , Shah Raj , deToledo-Morrell Leyla , Duara Ranjan , Varon Daniel , Greig Maria , Roberts Peggy , Albert Marilyn , Onyike Chiadi , D’Agostino Daniel , Kielb Stephanie , Galvin James , Cerbone Brittany , Michel Christina , Pogorelec Dana , Rusinek Henry , de Leon Mony , Glodzik Lidia , Santi Susan De , Doraiswamy P. , Petrella Jefrey , Borges-Neto Salvador , Wong Terence , Coleman Edward , Smith Charles , Jicha Greg , Hardy Peter , Sinha Partha , Oates Elizabeth , Conrad Gary , Porsteinsson Anton , Goldstein Bonnie , Martin Kim , Makino Kelly , Ismail M. , Brand Connie , Mulnard Ruth , Thai Gaby , Mc-Adams-Ortiz Catherine , Womack Kyle , Mathews Dana , Quiceno Mary , Levey Allan , Lah James , Cellar Janet , Burns Jefrey , Swerdlow Russell , Brooks William , Apostolova Liana , Tingus Kathleen , Woo Ellen , Silverman Daniel , Lu Po , Bartzokis George , Graf-Radford Neill , Parftt Francine , Kendall Tracy , Johnson Heather , Farlow Martin , Hake Ann Marie , Matthews Brandy , Brosch Jared , Herring Scott , Hunt Cynthia , Dyck Christopher , Carson Richard , MacAvoy Martha , Varma Pradeep , Chertkow Howard , Bergman Howard , Hosein Chris , Black Sandra , Stefanovic Bojana , Caldwell Curtis , Hsiung Ging-Yuek Robin , Feldman Howard , Mudge Benita , Assaly Michele , Finger Elizabeth , Pasternack Stephen , Rachisky Irina , Trost Dick , Kertesz Andrew , Bernick Charles , Munic Donna , Mesulam Marek-Marsel , Lipowski Kristine , Weintraub Sandra , Bonakdarpour Borna , Kerwin Diana , Wu Chuang-Kuo , Johnson Nancy , Sadowsky Carl , Villena Teresa , Turner Raymond Scott , Johnson Kathleen , Reynolds Brigid , Sperling Reisa , Johnson Keith , Marshall Gad , Yesavage Jerome , Taylor Joy , Lane Barton , Rosen Allyson , Tinklenberg Jared , Sabbagh Marwan , Belden Christine , Jacobson Sandra , Sirrel Sherye , Kowall Neil , Killiany Ronald , Budson Andrew , Norbash Alexander , Johnson Patricia Lynn , Obisesan Thomas , Wolday Saba , Allard Joanne , Lerner Alan , Ogrocki Paula , Tatsuoka Curtis , Fatica Parianne , Fletcher Evan , Maillard Pauline , Olichney John , DeCarli Charles , Carmichael Owen , Kittur Smita , Borrie Michael , Lee T.-Y. , Bartha Rob , Johnson Sterling , Asthana Sanjay , Carlsson Cynthia , Potkin Steven , Preda Adrian , Nguyen Dana , Tariot Pierre , Burke Anna , Trncic Nadira , Reeder Stephanie , Bates Vernice , Capote Horacio , Rainka Michelle , Scharre Douglas , Kataki Maria , Adeli Anahita , Zimmerman Earl , Celmins Dzintra , Brown Alice , Pearlson Godfrey , Blank Karen , Anderson Karen , Flashman Laura , Seltzer Marc , Hynes Mary , Santulli Robert , Sink Kaycee , Gordineer Leslie , Williamson Jef , Garg Pradeep , Watkins Franklin , Ott Brian , Querfurth Henry , Tremont Geofrey , Salloway Stephen , Malloy Paul , Correia Stephen , Rosen Howard , Miller Bruce , Perry David , Mintzer Jacobo , Spicer Kenneth , Bachman David , Pomara Nunzio , Hernando Raymundo , Sarrael Antero , Relkin Norman , Chaing Gloria , Lin Michael , Ravdin Lisa , Smith Amanda , Raj Balebail Ashok , Fargher Kristin 
Published: April 2018

Alzheimer’s Disease (AD) is a progressive neurodegenerative disease where biomarkers for disease based on pathophysiology may be able to provide objective measures for disease diagnosis and staging. Neuroimaging scans acquired from MRI and metabolism images obtained by FDG-PET provide in-vivo measurements of structure and function (glucose metabolism) in a living brain. It is hypothesized that combining multiple different image modalities providing complementary information could help improve early diagnosis of AD. In this paper, we propose a novel deep-learning-based framework to discriminate individuals with AD utilizing a multimodal and multiscale deep neural network. Our method delivers 82.4% accuracy in identifying the individuals with mild cognitive impairment (MCI) who will convert to AD at 3 years prior to conversion (86.4% combined accuracy for conversion within 1–3 years), a 94.23% sensitivity in classifying individuals with clinical diagnosis of probable AD, and a 86.3% specificity in classifying non-demented controls improving upon results in published literature.