Reliability and Validity of Smartphone Cognitive Testing for Frontotemporal Lobar Degeneration
Authors: Staffaroni Adam M. , Clark Annie L. , Taylor Jack C. , Heuer Hilary W. , Sanderson-Cimino Mark , Wise Amy B. , Dhanam Sreya , Cobigo Yann , Wolf Amy , Manoochehri Masood , Forsberg Leah , Mester Carly , Rankin Katherine P. , Appleby Brian S. , Bayram Ece , Bozoki Andrea , Clark David , Darby R. Ryan , Domoto-Reilly Kimiko , Fields Julie A. , Galasko Douglas , Geschwind Daniel , Ghoshal Nupur , Graff-Radford Neill , Grossman Murray , Hsiung Ging-Yuek , Huey Edward D. , Jones David T. , Lapid Maria I. , Litvan Irene , Masdeu Joseph C. , Massimo Lauren , Mendez Mario F. , Miyagawa Toji , Pascual Belen , Pressman Peter , Ramanan Vijay K. , Ramos Eliana Marisa , Rascovsky Katya , Roberson Erik D. , Tartaglia M. Carmela , Wong Bonnie , Miller Bruce L. , Kornak John , Kremers Walter , Hassenstab Jason , Kramer Joel H. , Boeve Bradley F. , Rosen Howard J. , Boxer Adam L. ,   ALLFTD Consortium, Apostolova Liana , Appleby Brian , Barmada Sami , Bayram Ece , Boeve Bradley , Botha Hugo , Boxer Adam L. , Bozoki Andrea , Brushaber Danielle , Clark Annie L. , Cobigo Yann , Darby R. Ryan , Day Gregg S , Dhanam Sreya , Dickerson Bradford , Dickson Dennis , Domoto-Reilly Kimiko , Elahi Fanny , Faber Kelley , Fagan Anne , Fields Julie A. , Fong Jamie , Foroud Tatiana , Forsberg Leah K. , Galasko Douglas R. , Gavrilova Ralitza , Gendron Tania , Geschwind Daniel , Ghoshal Nupur , Goldman Jill , Graff-Radford Neill , Graff-Radford Jonathan , Grant Ian M. , Grossman Murray , Hall Matthew , Hales Chadwick M. , Heuer Hilary W. , Honig Lawrence S. , Hsiung Ging-Yuek (Robin) , Huang Eric , Huey Edward D. , Irwin David , Johnson Noah , Jones David T. , Kantarci Kejal , Knopman David , Kolander Tyler , Kornak John , Kremers Walter , Kwan Justin , Lario Lago Argentina , Lapid Maria , Lavigne Shannon B. , Lee Suzee , Léger Gabriel C. , Litvan Irene , Ljubenkov Peter , Lucente Diane , Mackenzie Ian R. , Manoochehri Masood , Masdeu Joseph C. , Massimo Lauren , McGinnis Scott , McMillan Corey T. , Mendez Mario F. , Mester Carly , Molden Joie , Miyagawa Toji , Naasan Georges , Onyike Chiadi , Pantelyat Alexander , Paolillo Emily , Pascual Belen , Paulson Henry , Petrucelli Leonard , Pressman Peter , Rademakers Rosa , Ramanan Vijay , Ramos Eliana Marisa , Rankin Katherine P. , Rao Meghana , Rascovsky Katya , Rhoads Kristoffer W. , Rexach Jessica , Ritter Aaron , Roberson Erik D. , Rogalski Emily , Rojas Julio C. , Rosen Howard J. , Savica Rodolfo , Seeley William , Snyder Allison , Staffaroni Adam M. , Sullivan Anne C. , Syrjanen Jeremy M. , Tartaglia M. Carmela , Taylor Jack , Tipton Philip W. , Vandebergh Marijne , Toga Arthur , VandeVrede Lawren , Weintraub Sandra , Wint Dylan , Wise Amy B. , Wolf Amy , Wong Bonnie , Wszolek Zbigniew K. , Yokoyoma Jennifer 
Published: April 2024

Importance

Frontotemporal lobar degeneration (FTLD) is relatively rare, behavioral and motor symptoms increase travel burden, and standard neuropsychological tests are not sensitive to early-stage disease. Remote smartphone-based cognitive assessments could mitigate these barriers to trial recruitment and success, but no such tools are validated for FTLD.

Objective

To evaluate the reliability and validity of smartphone-based cognitive measures for remote FTLD evaluations.

Design, Setting, and Participants

In this cohort study conducted from January 10, 2019, to July 31, 2023, controls and participants with FTLD performed smartphone application (app)–based executive functioning tasks and an associative memory task 3 times over 2 weeks. Observational research participants were enrolled through 18 centers of a North American FTLD research consortium (ALLFTD) and were asked to complete the tests remotely using their own smartphones. Of 1163 eligible individuals (enrolled in parent studies), 360 were enrolled in the present study; 364 refused and 439 were excluded. Participants were divided into discovery (n = 258) and validation (n = 102) cohorts. Among 329 participants with data available on disease stage, 195 were asymptomatic or had preclinical FTLD (59.3%), 66 had prodromal FTLD (20.1%), and 68 had symptomatic FTLD (20.7%) with a range of clinical syndromes.

Exposure

Participants completed standard in-clinic measures and remotely administered ALLFTD mobile app (app) smartphone tests.

Main Outcomes and Measures

Internal consistency, test-retest reliability, association of smartphone tests with criterion standard clinical measures, and diagnostic accuracy.

Results

In the 360 participants (mean [SD] age, 54.0 [15.4] years; 209 [58.1%] women), smartphone tests showed moderate-to-excellent reliability (intraclass correlation coefficients, 0.77-0.95). Validity was supported by association of smartphones tests with disease severity (r range, 0.38-0.59), criterion-standard neuropsychological tests (r range, 0.40-0.66), and brain volume (standardized β range, 0.34-0.50). Smartphone tests accurately differentiated individuals with dementia from controls (area under the curve [AUC], 0.93 [95% CI, 0.90-0.96]) and were more sensitive to early symptoms (AUC, 0.82 [95% CI, 0.76-0.88]) than the Montreal Cognitive Assessment (AUC, 0.68 [95% CI, 0.59-0.78]) (z of comparison, −2.49 [95% CI, −0.19 to −0.02]; P = .01). Reliability and validity findings were highly similar in the discovery and validation cohorts. Preclinical participants who carried pathogenic variants performed significantly worse than noncarrier family controls on 3 app tasks (eg, 2-back β = −0.49 [95% CI, −0.72 to −0.25]; P &amp;lt; .001) but not a composite of traditional neuropsychological measures (β = −0.14 [95% CI, −0.42 to 0.14]; P = .32).

Conclusions and Relevance

The findings of this cohort study suggest that smartphones could offer a feasible, reliable, valid, and scalable solution for remote evaluations of FTLD and may improve early detection. Smartphone assessments should be considered as a complementary approach to traditional in-person trial designs. Future research should validate these results in diverse populations and evaluate the utility of these tests for longitudinal monitoring.