Using structural MRI to identify bipolar disorders – 13 site machine learning study in 3020 individuals from the ENIGMA Bipolar Disorders Working Group
Authors: Nunes Abraham , Schnack Hugo G. , Ching Christopher R. K. , Agartz Ingrid , Akudjedu Theophilus N. , Alda Martin , Alnæs Dag , Alonso-Lana Silvia , Bauer Jochen , Baune Bernhard T. , Bøen Erlend , Bonnin Caterina del Mar , Busatto Geraldo F. , Canales-Rodríguez Erick J. , Cannon Dara M. , Caseras Xavier , Chaim-Avancini Tiffany M. , Dannlowski Udo , Díaz-Zuluaga Ana M. , Dietsche Bruno , Doan Nhat Trung , Duchesnay Edouard , Elvsåshagen Torbjørn , Emden Daniel , Eyler Lisa T. , Fatjó-Vilas Mar , Favre Pauline , Foley Sonya F. , Fullerton Janice M. , Glahn David C. , Goikolea Jose M. , Grotegerd Dominik , Hahn Tim , Henry Chantal , Hibar Derrek P. , Houenou Josselin , Howells Fleur M. , Jahanshad Neda , Kaufmann Tobias , Kenney Joanne , Kircher Tilo T. J. , Krug Axel , Lagerberg Trine V. , Lenroot Rhoshel K. , López-Jaramillo Carlos , Machado-Vieira Rodrigo , Malt Ulrik F. , McDonald Colm , Mitchell Philip B. , Mwangi Benson , Nabulsi Leila , Opel Nils , Overs Bronwyn J. , Pineda-Zapata Julian A. , Pomarol-Clotet Edith , Redlich Ronny , Roberts Gloria , Rosa Pedro G. , Salvador Raymond , Satterthwaite Theodore D. , Soares Jair C. , Stein Dan J. , Temmingh Henk S. , Trappenberg Thomas , Uhlmann Anne , van Haren Neeltje E. M. , Vieta Eduard , Westlye Lars T. , Wolf Daniel H. , Yüksel Dilara , Zanetti Marcus V. , Andreassen Ole A. , Thompson Paul M. , Hajek Tomas ,   for the ENIGMA Bipolar Disorders Working Group
Published: August 2018

Bipolar disorders (BDs) are among the leading causes of morbidity and disability. Objective biological markers, such as those based on brain imaging, could aid in clinical management of BD. Machine learning (ML) brings neuroimaging analyses to individual subject level and may potentially allow for their diagnostic use. However, fair and optimal application of ML requires large, multi-site datasets. We applied ML (support vector machines) to MRI data (regional cortical thickness, surface area, subcortical volumes) from 853 BD and 2167 control participants from 13 cohorts in the ENIGMA consortium. We attempted to differentiate BD from control participants, investigated different data handling strategies and studied the neuroimaging/clinical features most important for classification. Individual site accuracies ranged from 45.23% to 81.07%. Aggregate subject-level analyses yielded the highest accuracy (65.23%, 95% CI = 63.47–67.00, ROC-AUC = 71.49%, 95% CI = 69.39–73.59), followed by leave-one-site-out cross-validation (accuracy = 58.67%, 95% CI = 56.70–60.63). Meta-analysis of individual site accuracies did not provide above chance results. There was substantial agreement between the regions that contributed to identification of BD participants in the best performing site and in the aggregate dataset (Cohen’s Kappa = 0.83, 95% CI = 0.829–0.831). Treatment with anticonvulsants and age were associated with greater odds of correct classification. Although short of the 80% clinically relevant accuracy threshold, the results are promising and provide a fair and realistic estimate of classification performance, which can be achieved in a large, ecologically valid, multi-site sample of BD participants based on regional neurostructural measures. Furthermore, the significant classification in different samples was based on plausible and similar neuroanatomical features. Future multi-site studies should move towards sharing of raw/voxelwise neuroimaging data.