Integrating Home-Based Video Teleneuropsychology Into Clinical Practice: Typical Versus Atypical Alzheimer's Disease
Phase: NA
Conditions: Suspected Typical Alzheimer's Disease (AD), Suspected Atypical Alzheimer's Disease (AD)
Sponsor: University of Colorado, Denver

Official title: VIdeo Telehealth in ALzheimer's: NeuroPsychology (VITAL-NP) 
NCT number: NCT05379023  
Document date (IRB approved protocol): January 11, 2022 
Document type: Study Protocol with Statistical Analysis Plan 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
I. Hypotheses and Specific Aims:   
Specific Aims 
The outbreak of COVID-19 has raised critical questions regarding the implementation of 
teleneuropsychology (TeleNP) using video administration of cognitive assessments in patients’ 
homes. Remote administration of brief cognitive assessments has been shown to be a valid 
alternative to in-person testing with typical, memory-predominant Mild Cognitive Impairment 
(MCI) and Alzheimer’s Disease (AD) dementia. However, these studies have focused primarily 
on virtual testing services to a patient in a remote clinic rather than appraising feasibility and 
validity of virtual testing services to a patient in their home.  Whether due to the COVID-19 
pandemic, geographical barriers to healthcare access, or functional limitations, many older 
patients have limited access to remote clinic offices to participate in an evaluation.  
Moreover, prior studies have selectively focused on research participants with typical 
presentations of AD, raising concerns as to the generalizability of TeleNP to patients who are 
diagnostically challenging and more representative of clinic settings. For example, patients with 
complex presentations of AD and related disorders (ADRD) are more likely to be referred to 
University clinics than patients with memory-predominant AD. Given that University clinics are 
typically located in urban areas, patients with complex ADRD often require more extensive 
travel and resources to obtain a dementia neuropsychological evaluation.  In order to increase 
access to neuropsychological assessments, improve the precision of neurodegenerative 
diagnoses, and ensure that TeleNP is generalizable to complex presentations of ADRD, 
appraisal of the feasibility and preliminary validity of home-based TeleNP in both typical 
and atypical AD is critically needed. 
The overarching goal of this proposal is to a) assess the feasibility and preliminary validity of 
home-based delivery of TeleNP to patients with suspected AD, referred for cognitive 
assessments in a Neurology Clinic; and b) elucidate whether TeleNP is equivalent to face-to-
face evaluation (FF) for diagnostic adjudication of atypical versus typical AD. To accomplish 
these goals, we will conduct a pilot clinical trial of two testing modalities, TeleNP versus FF, for 
delivery of neuropsychological evaluations. Neurologist-referred patients with typical (i.e., 
memory-predominant; n=40) and atypical (i.e., complex and/or non-memory predominant; n=40) 
presentations of possible AD will be enrolled in the study. Care partners of participants will also 
be consented to the study and included in stakeholder surveys/interviews. Using a cross-over 
design, all participants will undergo evaluations (clinical interview + Core Battery of cognitive 
tests) in both modalities, approximately 4-6 weeks apart to account for potential practice effects. 
Although all participants will be referred due to suspected AD, the neuropsychologist will be 
blinded to the specific diagnosis (i.e. typical versus atypical) from the neurologist. The Core 
Battery will include measures of memory, executive, visuospatial, and language functions. To 
further minimize practice effects, alternate assessment forms will be implemented and order of 
test forms will be counterbalanced. Some tests are not readily adaptable to video-based delivery 
due to tactile/physical stimuli; as such, we will assess whether a Flexible Assessment (i.e., 
option to include additional tests, after completion of Core Battery) in the FF modality alters 
diagnostic conclusions. Finally, in an exploratory aim, we will identify stakeholder perspectives 
of TeleNP that inform pragmatic implementation of remote neuropsychological evaluations. Our 
Aims are: 
 

Aim 1: Determine the validity of home-based video administration of neuropsychological 
tests (TeleNP) for assessing AD. 1a: Are cognitive test results equivalent across TeleNP and 
FF conditions in patients with suspected AD? 1b. Is this equivalence consistent across referral 
diagnoses (i.e., Typical vs. Atypical AD)? This Aim will test the validity of cognitive measures 
assessed with TeleNP and determine whether the validity of TeleNP differs based on referral 
diagnosis. 
Aim 2: Evaluate the impact of home-based TeleNP on the differential diagnosis of typical 
and atypical AD.  2a. Is the neuropsychologist’s diagnosis comparable across TeleNP and FF 
conditions? 2b. Does flexibility to incorporate additional tests (Flexible Assessment) that are not 
readily adaptable to video administration alter diagnostic conclusions? Given that 
neuropsychologists’ conceptualizations of patient presentations may differ from the referring 
neurologist, this Aim will appraise whether TeleNP results in the same diagnoses from 
neuropsychologists compared to FF (gold standard) conditions. 
Aim 3: Appraise the feasibility and acceptability of TeleNP in the evaluation of AD 
patients. 3a. What is the user experience of TeleNP by patients, care partners, 
neuropsychologists, and psychometrists?  3b. Are there barriers or unique benefits of TeleNP 
that may influence future use and scalability in clinical care? This Aim will utilize mixed methods, 
including interviews about the user experience and surveys of intervention feasibility and 
acceptability, to address quantitative and qualitative aspects of TeleNP that may influence the 
pragmatic implementation of remote assessments in a future, definitive trial. 
 
II. Background and Significance:  
In the context of the COVID-19 pandemic and other limitations to accessing face-to-face care, 
critical questions have been raised regarding the feasibility and validity of implementing 
teleneuropsychology (TeleNP) using video administration of cognitive assessments to patients in 
their homes. The knowledge gap regarding TeleNP resulted in a considerable decrease in 
cognitive assessments during this pandemic. Cognitive assessments, ranging from brief screens 
to comprehensive testing batteries, are an important clinical service to better elucidate the 
diagnostic presentation and severity of older adult patients, including Mild Cognitive Impairment 
(MCI) and Alzheimer’s disease (AD) dementia1-3. Despite their integral use in clinical diagnosis, 
symptom tracking, and treatment planning, there are several barriers to the widespread 
translation of these services to a remote-delivery modality. Whereas remote administration of brief 
cognitive screens4-8 and assessments9-11 has been shown to be a valid alternative to in-person 
testing with typical, memory-predominant MCI and AD, these studies have focused primarily on 
virtual testing services to a patient in a remote clinic rather than appraising feasibility and validity 
of virtual administration of cognitive tests to a patient in their home.  
Moreover, prior studies have selectively focused on older adult participants with typical 
presentations of AD, often with previously confirmed memory-predominant syndromes 
adjudicated by interdisciplinary research conferences.12-14 This approach is important and 
standard for initial validations; however, it raises questions regarding the generalizability of 
TeleNP to patients who are diagnostically challenging and more representative of 
subspecialty Neurology clinic referrals. For example, patients with atypical and/or complex 
presentations of AD and related disorders (ADRD) are more likely to be referred to subspecialty 
University clinics than patients with classic AD presentations15. Given that University clinics are 
often located in urban areas, patients may require more extensive travel and financial resources 
to obtain a comprehensive dementia evaluation. To improve patient access to specialized 
neurodegenerative disorder evaluations, pragmatic appraisal of the feasibility and 

diagnostic validity of home-based TeleNP in patients with typical and atypical ADRD 
presentations is critically needed. 
To overcome these significant gaps in prior research, we propose to demonstrate the feasibility 
and validity of home-based, comprehensive TeleNP. We will conduct a cross-over pilot trial to 
evaluate the feasibility and preliminary validity of standard, face-to-face (FF) neuropsychological 
assessment versus home-based video TeleNP in adults with suspected typical or atypical AD. 
Germane to the generalizability of TeleNP to clinical settings, the trial will be embedded in the 
University of Colorado Hospital (UCH) Memory Disorders Clinic, which will further facilitate an 
appraisal of whether TeleNP assessments are appropriate for diagnostic adjudication of typical 
vs. atypical ADRD phenotypes. In an exploratory aim, we will utilize a mixed methods approach 
to assess the feasibility and acceptability of TeleNP among key stakeholders: patients, their care 
partners, neuropsychologists, and a psychometrist. This approach will be guided by the Unified 
Theory of Acceptance and Use of Technology (UTAUT) model; this theoretical framework 
identifies “social influence” and “facilitating conditions”, which can assess the potential influence 
of the home environment on user acceptance and intentions to use.16 This pilot clinical trial will 
delineate the feasibility of implementing TeleNP in a Neurology clinic setting and thereby prepare 
us for a future definitive pragmatic trial of TeleNP effectiveness across multiple clinics. 
 
III. Preliminary Studies/Progress Report:   
This is an NIH-funded R21 application with no current pilot data. There are published studies of 
telehealth and neuropsychology; however, the vast majority of studies using remote 
neuropsychological testing have been delivery of services to another health care facility, with the 
presence of on-site support to the patient8, 9, 17-20. Although those studies emulated a “controlled” 
setting of a clinic, they did not address the feasibility of implementing TeleNP in a more real-world 
context, when the patient is at home with no staff support. Whether due to rural-based access 
to care barriers, functional limitations, or COVID-19, many older patients have limited 
access to remote clinic offices for evaluation, hence raising questions regarding scalability of 
clinic-to-remote clinic neuropsychological assessments. As such, we will appraise the feasibility 
and preliminary validity of TeleNP as a comprehensive neuropsychological assessment via home-
based video telehealth administration. 
 
IV. Research Methods 
 
A. Description of Population to be Enrolled:   
 
Clinical Trial Participants (Patients): 
We will recruit 80 patients from the UCH Memory Disorders Clinic (considering possible screen 
fails, up to 150 may ultimately be recruited). To understand the feasibility and preliminary validity 
of TeleNP, we will enroll 1) 40 adults with suspected typical AD (i.e., memory-predominant) and 
2) 40 adults wherein AD is on the neurologist’s diagnostic differential, but the presentation is 
atypical (i.e., complex/non-memory predominant presentations). The typical and atypical AD 
groups will range in severity from MCI to mild dementia (based on referring neurologist’s 
determination); this is consistent with current conceptualizations of AD, in which clinical syndrome 
(AD) is distinct from severity of presentation (MCI; dementia).1, 2  
All participants with suspected AD will be referred by neurologists. Our specific inclusion/exclusion 
criteria will be: 

Inclusion criteria:  
Eligible participants will be adults ≥age 60 years and <90 years who are undergoing evaluation 
for possible AD and whose severity ranges from MCI to mild dementia. Of note, we are excluding 
individuals aged 90 and over, as recent studies suggest that individuals in the oldest-old category 
differ from other individuals with MCI or dementia in terms of risk factors for AD and prevalence 
of non-AD pathologies.21-23 In addition, limited age-referenced normative data for 
neuropsychological tests are available for individuals 90 and over, which impacts our ability to 
address Aims 1 and 2.  
Eligible participants must carry a diagnosis of either suspected typical AD or atypical AD, based 
on the referring neurologist’s diagnosis.  
• 
Typical AD: Defined as a memory-predominant presentation of AD.  
• 
Atypical/Complex AD: Defined as a complex (i.e., AD is under consideration, but other 
diagnoses may be included in neurologist’s differential diagnosis) and/or non-memory 
predominant phenotype of AD.  
 
Both patient groups will be based solely on the neurologist’s clinical diagnosis after the initial 
Neurology visit, and in the absence of formal neuropsychological test results, to minimize 
circularity.  
 
Exclusion criteria:  
We will exclude individuals with moderate to severe levels of dementia and individuals with active 
delirium (per neurologist’s determination), as these patients are less likely to be referred for 
neuropsychological evaluation than individuals with mild presentations. We will exclude patients 
who are legally blind or deaf, due to the auditory and visual components of the study. We will 
exclude individuals with a brain tumor and individuals who have a confirmed non-AD neurological 
diagnosis (e.g, Multiple Sclerosis; FTD); moreover, we will exclude individuals who are being 
evaluated outside of the UCH Memory Disorders Clinic. Finally, we will exclude participants who 
report active (i.e., within the last 6 months) substance use of illicit drugs such as cocaine or 
methamphetamines, or who carry a diagnosis of a major psychiatric disorder (e.g., schizophrenia; 
bipolar disorder). 
 
Additional Consenting Participants:  
Participants will be encouraged to bring a care partner, if available, with them to visits (i.e., FF 
and TeleNP), and the care partner will also provide consent to complete surveys and interviews. 
Consistent with the target sample for the primary participants, we will recruit up to 80 care 
partners; however, consistent with standard clinical practice, the availability of a care partner will 
not be a requirement.  There are no age requirements or other inclusion/exclusion criteria for the 
care partner. We anticipate that most care partners will be co-residing caregivers, although 
participants may select a close friend as their care partner.  
 
Total potential enrollment number including coverage for screen failures/withdrawals = 300. (Up 
to 150 patients, plus their care partners if applicable). 
 
Recruitment and Screening:   
We will work directly with the Memory Disorders Clinic to recruit participants. In the last two years, 
the UCH Memory Disorders Clinic served over 5,000 patients.  

The referring neurologists (sub-investigators on this study) may directly approach their patients 
to discuss the proposed study and determine interest in study enrollment. They (i.e., referring 
neurologist) will provide baseline demographic and diagnostic information (typical vs. atypical 
AD), patient severity level (MCI vs. Mild Dementia), and their (neurologist) judgment of capacity 
to consent through a brief REDCap survey to the study psychometrist.  
The psychometrist will contact the prospective participant to review the study procedures and 
obtain informed consent from the patient and/or legally authorized representative [LAR]. 
Although most patients with MCI retain capacity to consent to a research study, diminished 
capacity is present across the MCI and early dementia spectrum. We will consider the capacity 
judgment provided by the neurologist; in addition, the following questions will be used to assess 
whether or not the patient can consent for themselves. If they cannot answer these questions 
after consent discussion, consent will be obtained from an LAR: 
• Describe in your own words the purpose of this study? 
• 
What are you being asked to do? 
• 
What are the side effects/risks of being in the study? 
• 
Is this voluntary?  
• 
What do you do if you have questions or possible side effects? 
 
The psychometrist will present these questions; if any questions arise regarding capacity, then a 
neuropsychologist or physician will be available to complete a capacity to consent to research 
assessment. 
 
The basis and conclusions of the capacity to consent assessment will be recorded.  If 
cognitively capable of providing consent in the clinical opinion of the evaluator, the study 
participant will review and sign the consent form. If a study participant has an LAR (including 
durable/medical power of attorney or guardian) and are judged to be unable to provide consent, 
then the consent of the LAR will also be required. Verbal assent will be obtained from the 
decisionally challenged patient. The participant’s LAR may also serve in the role of consenting 
care partner. 
 
Informed consent, interviews, and surveys may be conducted via HIPAA-compliant platforms 
using remote assessments. Specifically, e-consent may be obtained remotely using HIPAA-
compliant video programs (e.g., Zoom, Skype, Vidyo) and e-signature using REDCap and/or 
phone by the site investigator or study staff. 
 
HIPAA privacy notification language will be incorporated into the informed consent document 
and will be signed by all participants prior to any procedures being carried out.  Informed 
consent language will include key facts about the study, the study events, timeline, the study 
participant’s role in the study, potential risks and benefits and highlight the fact that participation 
is completely voluntary and will signed by all study participants. The informed consent/HIPAA 
form will specify that the database will receive and store all research data. Information regarding 
the purpose and procedures for this project will be given in both oral and written form.  All study 
participants will be consented in privacy and will be given as much time as they need to read, 
discuss and ask questions about study events and their respective roles and participation in the 
study. They will be reassured that the same ethical and legal codes that govern medical practice 

also apply to clinical research and they will be given the opportunity to ask questions and 
receive information regarding the goals and details of this study. If the Principal Investigator or 
Co-Investigators do not initially participate in the discussion, they will be available to answer 
questions prior to obtaining written informed consent/authorization. 
 
Subjects may withdraw participation in the study at any time by calling, writing, or emailing the 
Principal Investigator or study coordinator. Subjects will be unable to withdraw data that has 
already been analyzed, presented, or published. 
 
 
 
B. Study Design and Research Methods   
 
Randomization, Intervention and Neuropsychological Assessment Methods 
Study participants will undergo both modalities (FF; TeleNP), conducted 4-6 weeks apart 
(allowing for maximum of 90 days). Order of modality will be randomized at baseline using a 
computerized process through REDCap, developed by Dr. Carlson. Similarly, assignment of 
supervising neuropsychologist will be randomized and counterbalanced across testing conditions. 
Our goal of using two neuropsychologists is to minimize bias; specifically, use of the same 
neuropsychologist for the same patient across both modalities would likely influence their 
interview approach and diagnostic adjudication due to recollection of the prior assessment. 
 
 
 
Intervention Components: 
 
Pre-Visit Call: The psychometrist will place a pre-visit call to the participant and care partner up 
to 5 days prior to the scheduled FF visit or TeleNP visit. For the FF condition, the participant will 
be reminded to bring their care partner to provide additional details regarding symptom history 
during the interview. For the TeleNP condition, the psychometrist will have sent/delivered a study 
tablet, to the patient’s home between the recruitment/screening call and the pre-visit call. During 
this pre-visit TeleNP call, the psychometrist will remind the participant to include their care partner 
on the assessment day, as the care partner will be encouraged to assist with setting up their home 
testing environment and to provide additional details regarding history during the TeleNP 

interview. A test video call will also be conducted during this session, and the psychometrist will 
assist with troubleshooting.  
 
Protocol and Core Battery (2 hours): In both conditions (FF; TeleNP), the neuropsychologist 
will first conduct a structured, 30-minute interview with the participant and care partner, if 
available, which is a standard component of neuropsychological assessments. The 
psychometrist will subsequently conduct a Core Battery of standardized neuropsychological 
assessments (see Data Collection below). The majority (60%) of clinical neuropsychologists 
employ psychometrists to administer their tests, thus our approach simulates clinical practice.24 
Tests in the Core Battery were selected based on: a) breadth: coverage of all primary domains 
(i.e., memory25, 26, attention/executive functions27, 28, visuospatial29, and language functions30); b) 
use in clinical practice: frequent use in neuropsychological evaluations with established, validated 
norms in older adult populations; c) availability of alternate forms: presence of alternate forms, 
where possible, for tests most sensitive to practice effects (e.g., memory, verbally-mediated 
executive measures); and d) adaptability to TeleNP: inclusion of tests that minimize use of 
physical/tactile stimuli; i.e., inclusion of tests that may be screen shared via remote video 
technology and/or permit the patient to utilize their own piece of paper. The psychometrist will 
score the Core Battery based on published normative data for an older adult population using an 
automated 
excel 
calculator 
and 
provide 
the 
demographic-adjusted 
scores 
to 
the 
neuropsychologist. The neuropsychologist will document their diagnosis and severity of the 
participant’s presentation. 
 
Assessment Modality Details:  
Face-to-face (FF) Assessment (control condition): The participant will undergo an in-person 
evaluation with the neuropsychologist and psychometrist at the Clinical & Translational Research 
Center (CTRC) or in Building 400 at CU-AMC.  The standard protocol will be administered, 
including the interview and Core Battery. After completion of the Core Battery and submission of 
initial ratings, the supervising neuropsychologists may elect to have additional tests administered 
by the psychometrist as part of a Flexible Assessment. The Flexible Assessment will include a 
menu of tests that have physical/tactile stimuli (e.g., Design Fluency; Trail Making Test), all of 
which are excluded from the Core Battery due to lack of adaptability to video-based 
administration. Upon receipt of the scores from the Flexible Assessment, the neuropsychologist 
will complete a second form indicating any alterations to their diagnosis based on new information.  
 
TeleNP Assessment (intervention condition): The participant will undergo a video-based TeleNP 
assessment using a HIPAA-compliant platform (e.g., Zoom). The participant will be at home using 
a study-provided tablet, and the neuropsychologist and psychometrist will administer the 
evaluation from either the CTRC or their CU-AMC office. During the scheduled TeleNP 
assessment, the standard protocol will be administered (i.e., interview and Core Battery). The 
psychometrist will assist with tablet collection at no direct cost to the participant (i.e., will provide 
a paid return box; alternatively, participant may drop off tablet if convenient to them) 
 
Data Collection for Clinical Trial Participants: 
Medical records:  

--Participants will be asked to allow their past, current and future medical record information, to 
be placed in our RedCap database. The goal is to collect health information pertinent to 
cognitive functioning in older adults. Because concurrent medical conditions and treatments 
may impact the participants’ health, it is likely that all pertinent past, current and future medical 
record information will be considered for inclusion in the RedCap database. This will include: 
symptom history and diagnostic considerations from the participants’ neurology notes; 
laboratory results; health history (i.e., major medical problems); brain imaging results. 
 
Demographics:  
--Participants’ age, birthdate, sex, gender, education level (Years), race/ethnicity, prior 
occupation, etc. will be obtained. Participant’s contact information and address will also be 
obtained to facilitate scheduling baseline and follow-up appointments.  
 
Interview: 
--Participant’s symptom history will be documented on a standard data collection sheet during 
the interview portion. 
 
Neuropsychological Testing, Core Battery (May not be given in its entirety): 
• 
Cognitive/Mood Surveys: 
o Geriatric Depression Scale 
 
• 
Cognitive Assessments: 
o Global: 
§ 
MoCA, with alternate forms 
o Memory:  
§ 
CVLT-II, with alternate forms 
§ 
Benson Figure Recall / RBANS Figure Recall 
o Speed/Executive Functions:  
§ 
Oral Trail Making Test 
§ 
Oral Symbol Digit Modality 
§ 
Digit Span, with alternate forms 
§ 
Verbal Fluency, with alternate forms (FAS; BHR) 
§ 
Color Word Inhibition 
o Language:  
§ 
Sentence Repetition 
§ 
Boston Naming Test-15 item, with alternate forms  
§ 
Verbal Agility 
§ 
WRAT Reading 
§ 
Abbreviated Peabody Picture Vocabulary Test 
§ 
Syntactic Comprehension 
o Visuospatial and Posterior Cortical Function 
§ 
Judgment of Line Orientation (Short Form) 
§ 
Benson Figure Copy / RBANS Figure Copy 
§ 
Calculation Screen 

 
Neuropsychological Testing, Flexible Assessment: 
• 
Cognitive Assessments in addition to Core Battery (May not be administered in its 
entirety): 
o Grooved Pegboard 
o Clock Drawing Test 
o Rey-Osterrieth Complex Figure 
o BVMT 
o Navon Figures 
o Face Perception/Affect Naming 
o VOSP Number Localization 
o Design Fluency 
o Written Trail Making Test A & B 
 
Surveys: 
• 
4-item measures of acceptability and feasibility 
• 
19-item modified UTAUT after TeleNP portion 
 
Cognitive Composites, Neuropsychologist Diagnosis, and Interviews: See Outcome Measures 
Below 
 
Data Collection for Care Partner Participants: 
 
Demographics:  
--Care partner’s age, birthdate, sex, gender, education level (Years), race/ethnicity, prior 
occupation, etc. will be obtained. Participant’s contact information and address will also be 
obtained to facilitate scheduling baseline and follow-up appointments.  
 
Surveys: 
• 
4-item measures of acceptability and feasibility 
• 
19-item modified UTAUT after TeleNP portion 
 
Interviews: See Outcome Measures Below 
 
Incorporation of Results into Clinical Care: Because the set battery used in the FF assessment 
uses validated tests that are normally used for these patients in clinical care, it is likely that 
patients in this study will not undergo a separate standard of care neuropsychological 
assessment.  Therefore, a brief research overview of the results of the FF assessment will be 
provided to the referring neurologist and the participant after both conditions are complete (and 
posted to Epic Medical Record as a research note), for use in their clinical care. Based on a 

national survey, the median time between neuropsychological assessment and completion of 
reports is 3 weeks; our proposed time is slightly longer than the median, but remains within an 
expected range for clinical reports.31 Of note, when considering the amount of time between 
referral and scheduled date of assessment in SOC (often on the order of months), our window 
will likely be shorter/faster than SOC given dedicate research time. If any safety, psychological or 
other signs of distress related to potential diagnosis of AD are noted during either assessment, 
appropriate referrals will be coordinated by Drs. Bettcher and Greher. Drs. Bettcher and Greher 
will coordinate with the neurologist to determine if prompt discussion of the first assessment is 
clinically prudent. The research note posted to Epic EMR will include a brief background/history, 
table of scores for FF condition, and summary/recommendations, consistent with standard of care 
approaches. 
 
C.  Outcome Measure(s):   
 
Aim 1: Equivalence of Cognitive Test Results: The goal of this Aim is to test the concurrent 
validity of cognitive measures across testing modalities and determine whether the test results 
differ based on referral diagnosis. Z-scores will be calculated based on age-referenced normative 
data for each cognitive test (Table 1) in the Core Battery. For data reduction purposes, a mean z-
score for each cognitive domain will be used (i.e., memory, executive functions, spatial abilities, 
language). This will allow us to address whether neuropsychological test results are equivalent 
across modality (i.e., FF vs. TeleNP) and whether complexity of referral impacts comparability of 
test results. 
 
Aim 2: Equivalence of Neuropsychologist’s Diagnosis: Considering that neuropsychologists’ 
conceptualizations of patient presentations may differ from the referring neurologist, the goal 
of this Aim is to appraise whether TeleNP results in the same diagnoses from 
neuropsychologists compared to FF (gold standard) conditions, and determine whether 
inclusion of flexibility to incorporate additional tests (Flexible Assessment) that are not readily 
adaptable to video administration alters diagnostic considerations. Neuropsychologist-
determined diagnosis (i.e., typical AD, atypical/complex AD, not AD) and severity (i.e., 
subjective concerns; MCI; dementia) will be assessed immediately after the standard protocol 
via a REDCap survey. To clarify whether the Flexible Assessment alters the 
neuropsychologists’ conceptualizations, neuropsychologist diagnosis and severity will also be 
documented after the Flexible Assessment is completed.  
 
Aim 3: Feasibility and Acceptability of TeleNP:  This Aim will utilize mixed methods, including 
patient, care partner, neuropsychologist, and psychometrist interviews about the user 
experience and surveys of intervention feasibility and acceptability (i.e., modified UTAUT), to 
address quantitative and qualitative aspects of TeleNP that may influence the pragmatic 
implementation of remote assessments in a future, definitive trial. 
Qualitative:  
• 
Patients and Care Partners: A subset of diverse patients (n=20) and any associated 
care partners will be invited to participate in semi-structured interviews with the 
psychometrist within one week of completing both FF and TeleNP conditions. We will 
sample for diverse perspectives, including participants from both genders and typical vs. 
atypical AD. These interviews will focus on the participant’s user experience, guided by 

UTAUT domains and specific attention to home setting and potential care partner 
involvement.  
• 
Psychometrist and Neuropsychologists: The psychometrist and neuropsychologists will 
also participate in brief, separate interviews to garner information regarding feasibility, 
acceptability, and strengths/weaknesses of TeleNP; these interviews will be conducted 
by Dr. Lum on four occasions, with each interview occurring after completion of 20 study 
visits.  
 
Quantitative: To appraise feasibility of TeleNP, we will include measures of enrollment tracking 
(i.e., enrolled vs. decline), protocol assessment time, tracking of protocol adherence (i.e., 
completion of Core Battery), and presence of technological difficulties (e.g., termination of 
assessment due to technological difficulties). The psychometrist will ask all patients (n=80) 
and associated care partners (up to n=80) to complete validated 4-item Acceptability and 
Feasibility of Intervention Measures32, 33 after both conditions, and a 19-item UTAUT questionnaire 
(adapted to TeleNP) after the TeleNP condition to measure determinants of intention to use. 
 
We will document the fidelity of assessment administration and scoring using a fidelity checklist 
in a random 20% sample of participants. 
 
Costs and Payments 
All costs associated with the implementation and maintenance of the study shall be supported 
by Dr. Bettcher’s grants. There is no charge to participants or participants’ insurance for 
participation.  
 
Clinical Trial participants will receive $25 to compensate for their time after both visits are 
completed. Care partners will not be compensated. 
 
C. Description, Risks and Justification of Procedures and Data Collection Tools: 
 
Description of Risks: 
Potential risks to participants, by intervention, including physical, psychological, social, cultural, 
financial, and legal risks: For both conditions, there are no expected physical or legal risks. 
Participants will be advised that the clinical trial is voluntary, and that they may obtain a clinical 
neuropsychological evaluation in lieu of the research study as part of standard of care.  
Risks associated with neuropsychological assessments:  
• 
Both conditions (FF [control] and TeleNP [intervention]): The potential risks associated with 
participation in the study are minimal and include the risk of anxiety and/or fatigue, boredom 
or frustration during the completion of the neuropsychological testing sessions. An additional 
potential risk is loss of confidentiality, which could lead to potential social or cultural risks, if 
personal sensitive information is shared outside of the intervention. 
• 
TeleNP: For the TeleNP condition, there are no expected physical, social, cultural, financial 
or legal risks. One potential risk is loss of privacy and/or confidentiality, as described below. 
We will employ video-based assessments for the TeleNP section, using HIPAA-compliant 
platforms. 
 

Mild practice effects may be observed regardless of order of administration, which is commonly 
noted in standard of care with serial testing; however, given that results will be provided from the 
FF condition (i.e., the condition most comparable to standard of care), there is a small possibility 
that this may result in a slight overestimation of participant’s cognitive abilities for those who 
undergo the FF condition second. See mitigation of risks below, as we will use validated mitigation 
strategies used in standard of care for serial testing. 
The neuropsychological tests selected for this research study are identical to measures used in 
standard of care, with the FF condition being comprehensive and comparable to SOC. Standard 
of care for neuropsychology is not set in stone, meaning that the duration of the assessment is 
often provider-dependent. For reference, the tests and the length of the assessment are 
comparable to the standard neuropsychology battery – both in terms of content and duration – 
that the PI (Bettcher) uses in clinic for the same patient population. All tests selected were chosen 
based on their standard use in clinic, their ability to carefully characterize cognitive strengths and 
weaknesses, and their utility in assisting with differential diagnosis. Thus, we do not think that that 
the participant would receive a less comprehensive assessment in the research study than they 
would receive as part of standard of care. 
Risks to privacy and/or confidentiality: Participants face a potential risk to privacy or loss of 
confidentiality related to private identifiable information and the research materials (test results) 
that they may provide as a study participant. 
 
Risk of a delay in diagnosis: Based on a national survey, the median time between assessment 
date and report is approximately 3 weeks; however, this does not include available windows to 
schedule neuropsychology visits in clinic, which often is on the order of months (i.e., once a 
patient is referred to neuropsychology at UCH, it frequently takes more than a month to 
schedule them with a provider). Our study will involve a 4-6 week process (maximum of 3 month 
window), during which time both the FF and TeleNP conditions will be conducted. This is still 
within the range of expected result times when considering the SOC provider scheduling 
process. In addition, the clinical neuropsychological assessment is one portion of a much larger 
assessment conducted by the neurologists; the neurologist is ultimately the provider that will 
provide a definitive diagnosis to the patient, and their diagnostic regimen typically occurs over a 
3- to 6-month period, during which reversible dementia labs, brain MRI, and cognitive testing 
are ordered. Our proposed timeline fits within the expected parameters of a neurology 
assessment; thus, we do not expect a risk for a delay in diagnosis. 
 
Alternative treatments and procedures: Because patients will all be seen by the UCH Memory 
Disorders Clinic, patients have access to the Neuropsychology Service as part of usual care. 
Based on neurologist determination, patients are frequently referred for a neuropsychological 
assessment as part of the interprofessional care team approach. 
 
Adequacy of Protection Against Risks 
Informed Consent and Assent: Because the targeted patient groups suffer from degenerative 
diseases, safeguards, and guidelines are in place for assessing the capacity of potential 
subjects and the use of surrogates/legally authorized representatives in the setting when an 
individual is determined unable to consent for themselves. Participants will be recruited through 
the UCH Memory Disorders Clinic, which is a subspecialty clinic in the department of Neurology. 
Dr. Holden and other neurologists at the UCH Memory Disorders clinic may directly approach 

their patients to participate in the study, and they will also document their initial determination of 
capacity to consent based on clinical assessment. If the patient is interested in participating, the 
neurologist will provide brief (e.g., contact information; diagnosis; capacity) information to the 
psychometrist and the psychometrist will set up the initial screening and consent call. All 
potential participants will be encouraged to include a care partner in the process (i.e., 
screening/consent, study visits).  
 
All participants will subsequently participate in an informed consent process; if the referring 
neurologist notes a lack of capacity, the participant will be asked to include their legally authorized 
representative (LAR) during the screening/consent call. All aspects of the project will be reviewed 
by the University of Colorado Multiple Institutional Review Board (COMIRB). Before enrollment, 
study personnel will ensure the subject’s full understanding of the research project, including the 
study objective, risks and benefits, the right to non-participation, review processes, emergency 
medical treatment, and financial responsibility. Informed consent will provide explicit details 
regarding confidentiality and sharing of de-identified data with qualified researchers; more 
specifically, consent forms will indicate that: de-identified data may be shared with qualified 
investigators outside the University; identifiable information from their in-person assessment will 
be provided to them and their referring neurologist in a research note. Dr. Bettcher, Dr. Greher, 
and Dr. Holden will also be available to answer questions regarding consent and procedures. 
 
Strategies to minimize risk: Risks to subjects are minimal and may include loss of confidentiality 
and psychological discomfort about discussing issues related to their neuropsychological 
assessment. Prior to enrollment, the study will be explained in a clear and detailed manner by the 
study coordinator (serving in role as coordinator and psychometrist) and investigators. All 
participants will be reminded at the beginning of all assessments of their right to terminate the 
assessment at any time. In the study consent procedure and study activities, we will clearly state 
that all participants can refuse to answer any questions or to review any of the study materials. 
Research staff will be trained to address psychological distress and will follow standard 
procedures for referral for mental health evaluation. We will also reassure patients that if they 
choose not to be in the study, it will not affect the medical care they normally receive.  
 
We will provide neuropsychological test results from the FF condition (i.e., condition that is most 
congruent with standard of care) to the referring neurologist and the patient, and this information 
will also be placed in a research note in their UCH EMR. This will be explicitly noted in consent 
form and reviewed in detail with the participant. In terms of study results, there is a possibility of 
practice effects, particularly on memory tests and fluency tests, with serial testing. Studies suggest 
minimal practice effects in individuals with symptomatic Alzheimer’s disease; however, given that 
we will provide results from the FF assessment to be used in clinical care and potentially for 
diagnosis, we will mitigate risk in the following way: a) use of alternate forms, where possible and 
b) require of at least one month between conditions. Based on these strategies, which are also 
used in standard of care with neuropsychological serial testing, we do not think enrollment in the 
study poses a significant risk of a clinically meaningful practice effect.  
 
Strategies to manage and protect the privacy of participants and confidentiality of research data: 
A copy of the consent for will be given to each participant. Maintenance of subject confidentiality 
is a high priority for the proposed research project. Video-based TeleNP will be conducted using 

HIPAA-compliant platform supported by UC Health and CU Anschutz Medical Campus. Video 
assessments will not be recorded. All written and audio-recorded data (i.e., stakeholder semi-
structured interviews) and consent materials will be given anonymous identification numbers and 
will be stored on a password protected private database on computers in locked offices. The 
consent forms and the key for identification of subject names corresponding to identification 
numbers will be stored in a separate locked file cabinet. For audio-recorded data (i.e., semi-
structured interviews), research assistants will redact any spoken identifiers (names, etc.) from 
transcripts and will label audio-files with a sequential de-identified number. Study data will be 
collected and managed using REDCap (Research Electronic Data Capture). Only key study 
personnel will have access to these data. REDCap is a secure web application designed to 
support data capture for research studies, providing user-friendly web-based case report forms, 
real-time data entry validation (e.g. for data types and range checks), audit trails, and a de-
identified data export mechanism to common statistical packages (SPSS, SAS, Stata, R/S-Plus). 
 
Plans for ensuring professional intervention if indicated: Participants will be given the PI’s contact 
information for any questions or concerns about the study. Staff will be trained to identify and 
address psychological distress, reportable events, or threats to society including elder abuse, 
suicide or homicide ideation or intent, violent behavior or threats, excessive emotional outbursts, 
a witnessed psychotic break with visual or auditory hallucinations, verbalization of anxiety or 
flashbacks to past traumatic experiences, and reports of major depression (assessed in the 
study). For acute issues, hospital security will be called (720-848-7777). For nonacute issues, the 
research team will be available in clinic to help avert crises and to schedule potential needed 
follow-up. If elder abuse is suspected, we will contact the patient’s Colorado county law 
enforcement agency and county-specific adult protective services intake phone number through 
standard reporting mechanisms. The informed consent forms state that if elder abuse is 
suspected it will be reported, in accordance with State of Colorado law. A toll free crisis line will 
be made available to participants who may be in abusive situations (1-888-885-1222), and the 
referring neurologist will be alerted to suspected elder abuse. 
 
Plans for handling incidental findings: Participants will receive results from their FF research 
assessment. Through the process of completing the study procedures, the most likely potential 
incidental findings will be new or worsened physical or psychological distress, or delirium. If the 
intervention is stopped due to discomfort and/or the patient desires for the research staff to share 
information about their current needs with the patient’s UCH Memory Disorders Clinic neurologist, 
the psychometrist will communicate this information on the patient’s behalf. Of note, if information 
emerges during either the TeleNP or FF visit that requires immediate clinical action, or clearly 
would impact treatment, the neuropsychologist will immediately notify the participant and 
neurologist to streamline care and minimize any delays in treatment. Moreover, if any safety, 
psychological or other signs of distress related to potential diagnosis of AD are noted during either 
assessment, appropriate referrals will be coordinated by Drs. Bettcher and Greher.  
 
To improve reach to participants who have limited access to technology and conduct 
TeleNP using a standardized technology platform, we will provide a 4G tablet to all participants 
via a research lending library. Participants will be encouraged to bring a care partner, if available, 
with them to visits (i.e., FF and TeleNP); however, consistent with standard clinical practice, the 
availability of a care partner will not be a requirement.  
 

E.   Potential Scientific Problems:   
Practice Effects: Repeated administration of cognitive tests is sometimes associated with 
practice effects, which may impact interpretation of results. Practice effects have been shown to 
be lower in symptomatic (MCI, dementia) patients relative to asymptomatic adults34; with respect 
to individual domains, verbal memory tests and verbally-mediated executive function measures 
appear to be most vulnerable to practice effects, with visuospatial measures less impacted. In 
estimating practice effects over a 3- to 6-month period, maximum effect sizes have ranged from 
0.10 to 0.15 based on drug trials, which are quite small.34 Our proposed period (4-6 weeks) occurs 
over a shorter duration. Based on recommendations for clinical trials34, we will use alternate forms, 
where available (i.e., memory tests; fluency measures; attention/working memory). Of note, we 
will provide feedback to the patient and referring neurologist based on the FF assessment. In 
cases where the participant undergoes the FF assessment after the TeleNP condition, it is 
possible that aforementioned practice effects, albeit very small, may influence the mean scores 
obtained for each cognitive domain. Serial testing is common in standard neuropsychological 
practice, thus we will use validated approaches (i.e., alternate forms) for tests that are most 
vulnerable to this effect. 
Diagnosis: While neuropsychologists will be apprised of the study design, they will be blinded to 
the specific referral diagnosis (i.e., typical vs atypical AD); moreover, the neuropsychologist will 
be asked to provide their own determination and given the option of including “Not AD” as their 
provisional diagnosis. The goal of this approach is to minimize bias and circularity; however, it 
remains possible that neuropsychologist-specific effects will be present (i.e., tendency to rate 
participants as “complex”). We will assess in exploratory analyses whether there were provider-
specific effects on outcomes. We acknowledge that the division of diagnoses into three categories 
(i.e., subjective concerns; MCI; dementia) does not fully map onto NIA-AA diagnostic criteria for 
MCI2 and AD dementia1; however, most neuropsychologists do not have access to fluid/PET 
biomarkers in clinical practice, thus our approach more pragmatically represents standard of care. 
Nonetheless, neuropsychologists often do provide specific diagnoses for complex phenotypes, 
including focal presentations (e.g., posterior cortical atrophy). As an exploratory strategy, we will 
consider providing an additional diagnostic field for the neuropsychologist to indicate specific 
considerations in their differential.  
Statistical Analysis Framework: We are evaluating equivalence by examining mean Z-scores 
per cognitive domain, aggregated across multiple tests; however, individual tests have different 
test-retest reliabilities, which are often evaluated on a patient-specific level through the use of 
reliable change indices (RCIs). Given the pilot nature of this clinical trial, it is possible that a .5SD 
difference in mean domain score will not accurately reflect a clinically meaningful difference in 
individual test scores. As such, we will consider examining test-specific differences across 
modalities using published test-rest reliability. 
Technological Difficulties: We have incorporated several approaches to minimize and 
troubleshoot technological difficulties, including a pre-visit test call, inclusion of care partner in the 
process, and provision of a study tablet to all participants. Consistent with Aim 3, we will track 
technological issues throughout the study procedures to inform changes that might be 
beneficial to a larger clinical trial. In addition, we will include a discussion during monthly team 
meetings regarding any technological issues that arise.   
 
F.   Statistical Analysis Plan:   
For Aim 1, we will adopt an equivalence testing framework. The null hypothesis is that there is an 
average difference of at least .5 SD units in the difference in the z-scores, where .5 SD defines a 
potentially clinically meaningful difference. The alternative hypothesis is that the mean of the 

difference in the z-scores is zero. This is the reverse of standard statistical testing. We elected a 
.5 SD margin as an initial threshold for a clinically meaningful difference; the reason for this 
decision is that once an individual reaches a level deemed to be clinically significant impairment 
(e.g., mild impairment = -1.5 SD; moderate impairment =-2.0 SD), a change in .5 SD equates to 
a categorical difference in their overall severity rating.  To conduct our equivalence test, we will 
compute the difference in the z-score between the FF and TeleNP conditions and compute the 
mean difference and a 90% confidence interval based on a t-test. This results in an equivalence 
test with an alpha=0.05. If the entirety of bounds of the 90% confidence interval are between (-
.5SD, .5SD), then the two assessments will be considered equivalent. We will repeat this 
computation stratified by referring diagnosis (typical vs. atypical), and also by sex or race/ethnicity 
(white vs. non-White).  
For Aim 2, we will compute a 3x3 table with the rows as TeleNP diagnosis and the columns as 
FF diagnosis to summarize the results. To assess the equivalence of TeleNP and FF for each 
condition we will create 3, 2x2 tables. For example, when assessing for differences in typical AD 
we will collapse over atypical AD and Not AD.  Following prior work35, we will compute the 
difference proportion of typical AD for the TeleNP and FF setting and construct a 90% confidence 
interval. A difference of proportions greater than 5% will be considered a clinically meaningful 
difference. If the 90% confidence interval is between (-5%,5%), then the two modalities will be 
considered equivalent. We will repeat this process for the two other neuropsychologist-
determined categories (atypical AD, Not AD). We will conduct the same analyses, stratified by 
severity. We will assess whether the neuropsychologists’ diagnosis is equivalent to the Flexible 
Assessment by repeating the above analysis using the Flexible vs FF diagnosis (i.e., diagnosis 
reported prior to initiating Flexible Assessment).  
For Aim 3, Feasibility and Acceptability of Intervention Measures and the UTAUT will be analyzed 
descriptively with means and interquartile ranges. Analysis of qualitative data will be a continuous 
process beginning with initial interviews and continuing throughout the data generation period. All 
interviews will be audio-recorded and transcribed verbatim. Directed by Dr. Lum, rapid qualitative 
analysis36 will be used to understand multiple user perspectives on feasibility, acceptability, and 
other potential factors affecting use, including home context, of TeleNP compared to FF. At least 
2 team members will analyze transcripts.37 All data will be entered into ATLAS.ti v8.0 (GmbH, 
Berlin) for data management. Transcripts will be analyzed to develop the major themes or 
concepts reported by interviewees. Using the quantitative and qualitative data, we will conduct 
sequential mixed methods analyses using data from Aim 3.38 The synthesis stage of data analysis 
involves triangulating the findings from interviews and quantitative data related to feasibility (i.e., 
assessment time, protocol adherence, technical issues, etc.), acceptability ratings, availability of 
a care partner, etc. We will interpret and summarize feasibility and acceptability for each modality 
from the different user perspective 
G.  Summarize Knowledge to be Gained:   
Appraisal of the feasibility and validity of TeleNP in patients with complex AD presentations will 
provide a critical platform for the future development of a pragmatic, multi-site trial. Validation of 
TeleNP in a real-world setting will further increase access to neuropsychological evaluations and 
reduce patient burden. Specific knowledge to be gained includes: 
• 
Comparison of two testing modalities for neuropsychology: telehealth and standard, 
face-to-face assessment 
• 
Comparison of diagnostic adjudication for these two neuropsychology modalities 
• 
Test- and interview-specific differences in feasibility and validity in these two modalities 

• 
Qualitative information regarding feasibility, participant experience and acceptability, care 
partner experience and acceptability, and psychometrist & neuropsychologist 
experience. 
• 
Greater understanding of technological concerns that may arise in telehealth 
• 
Preliminary understanding of potential benefits of telehealth, both qualitatively and 
quantitatively 
 
 
 H. References: 
 
 
 
1. 
McKhann GM, Knopman DS, Chertkow H, et al. The diagnosis of dementia due to Alzheimer's 
disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on 
diagnostic guidelines for Alzheimer's disease. Alzheimer's & dementia : the journal of the Alzheimer's 
Association 2011;7:263-269. 
2. 
Albert MS, DeKosky ST, Dickson D, et al. The diagnosis of mild cognitive impairment due to 
Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association 
workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement 2011;7:270-279. 
3. 
Ewers M, Walsh C, Trojanowski JQ, et al. Prediction of conversion from mild cognitive impairment 
to Alzheimer's disease dementia based upon biomarkers and neuropsychological test performance. 
Neurobiol Aging 2010;2010/12/17. 
4. 
Ciemins EL, Holloway B, Coon PJ, McClosky-Armstrong T, Min SJ. Telemedicine and the mini-
mental state examination: assessment from a distance. Telemed J E Health 2009;15:476-478. 
5. 
DeYoung N, Shenal BV. The reliability of the Montreal Cognitive Assessment using telehealth in a 
rural setting with veterans. J Telemed Telecare 2019;25:197-203. 
6. 
Grosch MC, Weiner MF, Hynan LS, Shore J, Cullum CM. Video teleconference-based 
neurocognitive screening in geropsychiatry. Psychiatry Res 2015;225:734-735. 
7. 
Loh PK, Donaldson M, Flicker L, Maher S, Goldswain P. Development of a telemedicine protocol 
for the diagnosis of Alzheimer's disease. J Telemed Telecare 2007;13:90-94. 
8. 
McEachern W, Kirk A, Morgan DG, Crossley M, Henry C. Reliability of the MMSE administered 
in-person and by telehealth. Can J Neurol Sci 2008;35:643-646. 
9. 
Cullum CM, Weiner MF, Gehrmann HR, Hynan LS. Feasibility of telecognitive assessment in 
dementia. Assessment 2006;13:385-390. 
10. 
Galusha-Glasscock JM, Horton DK, Weiner MF, Cullum CM. Video Teleconference 
Administration of the Repeatable Battery for the Assessment of Neuropsychological Status. Arch Clin 
Neuropsychol 2016;31:8-11. 
11. 
Wadsworth HE, Galusha-Glasscock JM, Womack KB, et al. Remote Neuropsychological 
Assessment in Rural American Indians with and without Cognitive Impairment. Arch Clin Neuropsychol 
2016;31:420-425. 
12. 
Brearly TW, Shura RD, Martindale SL, et al. Neuropsychological Test Administration by 
Videoconference: A Systematic Review and Meta-Analysis. Neuropsychol Rev 2017;27:174-186. 
13. 
Grosch MC, Gottlieb MC, Cullum CM. Initial practice recommendations for teleneuropsychology. 
Clin Neuropsychol 2011;25:1119-1133. 
14. 
Munro Cullum C, Hynan LS, Grosch M, Parikh M, Weiner MF. Teleneuropsychology: evidence for 
video teleconference-based neuropsychological assessment. J Int Neuropsychol Soc 2014;20:1028-
1033. 
15. 
Tonelli M, Wiebe N, Manns BJ, et al. Comparison of the Complexity of Patients Seen by Different 
Medical Subspecialists in a Universal Health Care System. JAMA Netw Open 2018;1:e184852. 

16. 
Venkatesh V, Morris MG, David GB, Davis FD. User Acceptance of Information Technology: 
Toward a Unified View. MIS Quarterly 2003;27:425-478. 
17. 
Vahia IV, Ng B, Camacho A, et al. Telepsychiatry for Neurocognitive Testing in Older Rural Latino 
Adults. Am J Geriatr Psychiatry 2015;23:666-670. 
18. 
Harrell KM, Wilkins SS, Connor MK, Chodosh J. Telemedicine and the evaluation of cognitive 
impairment: the additive value of neuropsychological assessment. J Am Med Dir Assoc 2014;15:600-606. 
19. 
Hildebrand R, Chow H, Williams C, Nelson M, Wass P. Feasibility of neuropsychological testing 
of older adults via videoconference: implications for assessing the capacity for independent living. J 
Telemed Telecare 2004;10:130-134. 
20. 
Wadsworth HE, Dhima K, Womack KB, et al. Validity of Teleneuropsychological Assessment in 
Older Patients with Cognitive Disorders. Arch Clin Neuropsychol 2018;33:1040-1045. 
21. 
Corrada MM, Hayden KM, Paganini-Hill A, et al. Age of onset of hypertension and risk of 
dementia in the oldest-old: The 90+ Study. Alzheimers Dement 2017;13:103-110. 
22. 
Robinson JL, Corrada MM, Kovacs GG, et al. Non-Alzheimer's contributions to dementia and 
cognitive resilience in The 90+ Study. Acta Neuropathol 2018;136:377-388. 
23. 
Trieu T, Sajjadi SA, Kawas CH, Nelson PT, Corrada MM. Risk factors of hippocampal sclerosis in 
the oldest old: The 90+ Study. Neurology 2018;91:e1788-e1798. 
24. 
Sweet JJ, Klipfel KM, Nelson NW, Moberg PJ. Professional practices, beliefs, and incomes of 
U.S. neuropsychologists: The AACN, NAN, SCN 2020 practice and "salary survey". Clin Neuropsychol 
2021;35:7-80. 
25. 
Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning Test, 2nd Edition. San 
Antonio, TX: The Psychological Corporation, 2000. 
26. 
Delis DC, Massman PJ, Butters N, Salmon DP, Cermak LS, Kramer JH. Profiles of demented and 
amnesic patients on the California Verbal Learning Test: Implications for the Assessment of Memory 
Disorders. Psychological Assessment: A Journal of Consulting and Clinical Psychology 1991;3:19-26. 
27. 
Delis DC, Kramer JH, Kaplan E, Holdnack J. Reliability and validity of the Delis-Kaplan Executive 
Function System: an update. J Int Neuropsychol Soc 2004;10:301-303. 
28. 
Mrazik M, Millis S, Drane DL. The oral trail making test: effects of age and concurrent validity. 
Arch Clin Neuropsychol 2010;25:236-243. 
29. 
Kramer JH, Jurik J, Sha SJ, et al. Distinctive neuropsychological patterns in frontotemporal 
dementia, semantic dementia, and Alzheimer disease. Cognitive and behavioral neurology : official 
journal of the Society for Behavioral and Cognitive Neurology 2003;16:211-218. 
30. 
Weintraub S, Mesulam MM, Wieneke C, Rademaker A, Rogalski EJ, Thompson CK. The 
northwestern anagram test: measuring sentence production in primary progressive aphasia. Am J 
Alzheimers Dis Other Demen 2009;24:408-416. 
31. 
Postal K, Chow C, Jung S, Erickson-Moreo K, Geier F, Lanca M. The stakeholders' project in 
neuropsychological report writing: a survey of neuropsychologists' and referral sources' views of 
neuropsychological reports. Clin Neuropsychol 2018;32:326-344. 
32. 
Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed 
implementation outcome measures. Implement Sci 2017;12:108. 
33. 
Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual 
distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011;38:65-76. 
34. 
Goldberg TE, Harvey PD, Wesnes KA, Snyder PJ, Schneider LS. Practice effects due to serial 
cognitive assessment: Implications for preclinical Alzheimer's disease randomized controlled trials. 
Alzheimers Dement (Amst) 2015;1:103-111.