Reliability and Validity of Smartphone Cognitive Testing for Frontotemporal Lobar Degeneration
Authors: Adam M. Staffaroni, Annie L. Clark, Jack C. Taylor, Hilary W. Heuer, Mark Sanderson-Cimino, Amy B. Wise, Sreya Dhanam, Yann Cobigo, Amy Wolf, Masood Manoochehri, Leah Forsberg, Carly Mester, Katherine P. Rankin, Brian S. Appleby, Ece Bayram, Andrea Bozoki, David Clark, R. Ryan Darby, Kimiko Domoto-Reilly, Julie A. Fields, Douglas Galasko, Daniel Geschwind, Nupur Ghoshal, Neill Graff-Radford, Murray Grossman, Ging-Yuek Hsiung, Edward D. Huey, David T. Jones, Maria I. Lapid, Irene Litvan, Joseph C. Masdeu, Lauren Massimo, Mario F. Mendez, Toji Miyagawa, Belen Pascual, Peter Pressman, Vijay K. Ramanan, Eliana Marisa Ramos, Katya Rascovsky, Erik D. Roberson, M. Carmela Tartaglia, Bonnie Wong, Bruce L. Miller, John Kornak, Walter Kremers, Jason Hassenstab, Joel H. Kramer, Bradley F. Boeve, Howard J. Rosen, Adam L. Boxer, Liana Apostolova, Brian Appleby, Sami Barmada, Ece Bayram, Bradley Boeve, Hugo Botha, Adam L. Boxer, Andrea Bozoki, Danielle Brushaber, Annie L. Clark, Yann Cobigo, R. Ryan Darby, Gregg S Day, Sreya Dhanam, Bradford Dickerson, Dennis Dickson, Kimiko Domoto-Reilly, Fanny Elahi, Kelley Faber, Anne Fagan, Julie A. Fields, Jamie Fong, Tatiana Foroud, Leah K. Forsberg, Douglas R. Galasko, Ralitza Gavrilova, Tania Gendron, Daniel Geschwind, Nupur Ghoshal, Jill Goldman, Neill Graff-Radford, Jonathan Graff-Radford, Ian M. Grant, Murray Grossman, Matthew Hall, Chadwick M. Hales, Hilary W. Heuer, Lawrence S. Honig, Ging-Yuek (Robin) Hsiung, Eric Huang, Edward D. Huey, David Irwin, Noah Johnson, David T. Jones, Kejal Kantarci, David Knopman, Tyler Kolander, John Kornak, Walter Kremers, Justin Kwan, Argentina Lario Lago, Maria Lapid, Shannon B. Lavigne, Suzee Lee, Gabriel C. Léger, Irene Litvan, Peter Ljubenkov, Diane Lucente, Ian R. Mackenzie, Masood Manoochehri, Joseph C. Masdeu, Lauren Massimo, Scott McGinnis, Corey T. McMillan, Mario F. Mendez, Carly Mester, Joie Molden, Toji Miyagawa, Georges Naasan, Chiadi Onyike, Alexander Pantelyat, Emily Paolillo, Belen Pascual, Henry Paulson, Leonard Petrucelli, Peter Pressman, Rosa Rademakers, Vijay Ramanan, Eliana Marisa Ramos, Katherine P. Rankin, Meghana Rao, Katya Rascovsky, Kristoffer W. Rhoads, Jessica Rexach, Aaron Ritter, Erik D. Roberson, Emily Rogalski, Julio C. Rojas, Howard J. Rosen, Rodolfo Savica, William Seeley, Allison Snyder, Adam M. Staffaroni, Anne C. Sullivan, Jeremy M. Syrjanen, M. Carmela Tartaglia, Jack Taylor, Philip W. Tipton, Marijne Vandebergh, Arthur Toga, Lawren VandeVrede, Sandra Weintraub, Dylan Wint, Amy B. Wise, Amy Wolf, Bonnie Wong, Zbigniew K. Wszolek, Jennifer Yokoyoma
Journal: JAMA Network Open (2024)

Importance Frontotemporal lobar degeneration (FTLD) is relatively rare, behavioral and motor symptoms increase travel burden, and standard neuropsychological tests are not sensitive to early-stage disease. Remote smartphone-based cognitive assessments could mitigate these barriers to trial recruitment and success, but no such tools are validated for FTLD. Objective To evaluate the reliability and validity of smartphone-based cognitive measures for remote FTLD evaluations. Design, Setting, and Participants In this cohort study conducted from January 10, 2019, to July 31, 2023, controls and participants with FTLD performed smartphone application (app)–based executive functioning tasks and an associative memory task 3 times over 2 weeks. Observational research participants were enrolled through 18 centers of a North American FTLD research consortium (ALLFTD) and were asked to complete the tests remotely using their own smartphones. Of 1163 eligible individuals (enrolled in parent studies), 360 were enrolled in the present study; 364 refused and 439 were excluded. Participants were divided into discovery (n = 258) and validation (n = 102) cohorts. Among 329 participants with data available on disease stage, 195 were asymptomatic or had preclinical FTLD (59.3%), 66 had prodromal FTLD (20.1%), and 68 had symptomatic FTLD (20.7%) with a range of clinical syndromes. Exposure Participants completed standard in-clinic measures and remotely administered ALLFTD mobile app (app) smartphone tests. Main Outcomes and Measures Internal consistency, test-retest reliability, association of smartphone tests with criterion standard clinical measures, and diagnostic accuracy. Results In the 360 participants (mean [SD] age, 54.0 [15.4] years; 209 [58.1%] women), smartphone tests showed moderate-to-excellent reliability (intraclass correlation coefficients, 0.77-0.95). Validity was supported by association of smartphones tests with disease severity ( r range, 0.38-0.59), criterion-standard neuropsychological tests ( r range, 0.40-0.66), and brain volume (standardized β range, 0.34-0.50). Smartphone tests accurately differentiated individuals with dementia from controls (area under the curve [AUC], 0.93 [95% CI, 0.90-0.96]) and were more sensitive to early symptoms (AUC, 0.82 [95% CI, 0.76-0.88]) than the Montreal Cognitive Assessment (AUC, 0.68 [95% CI, 0.59-0.78]) ( z of comparison, −2.49 [95% CI, −0.19 to −0.02]; P = .01). Reliability and validity findings were highly similar in the discovery and validation cohorts. Preclinical participants who carried pathogenic variants performed significantly worse than noncarrier family controls on 3 app tasks (eg, 2-back β = −0.49 [95% CI, −0.72 to −0.25]; P &amp;amp;lt; .001) but not a composite of traditional neuropsychological measures (β = −0.14 [95% CI, −0.42 to 0.14]; P = .32). Conclusions and Relevance The findings of this cohort study suggest that smartphones could offer a feasible, reliable, valid, and scalable solution for remote evaluations of FTLD and may improve early detection. Smartphone assessments should be considered as a complementary approach to traditional in-person trial designs. Future research should validate these results in diverse populations and evaluate the utility of these tests for longitudinal monitoring.