Expanding Access to Comprehensive Geriatrics Care Via Telehealth (QUE 20-010)
Phase: NA
Conditions: Geriatric Syndrome
Sponsor: VA Office of Research and Development

Expanding Access to Comprehensive Geriatrics Care via Telehealth (QUE 20-010) 
 
NCT05166603 
 
March 15, 2021 

Page 1 of 9 
 
  
 
TELE-GRACE IMPLEMENTATION PROJECT 
STATISTICAL ANALYSIS PLAN: Aim 1 Effectiveness  
Table of Contents 
OVERALL TELEGRACE PROJECT RATIONALE ..................................................................... 1 
THIS ANALYSIS PLAN FOCUSES ONLY ON EFFECTIVENESS ............................................. 3 
METHODS ................................................................................................................................. 3 
Intervention: In-Person versus Telehealth .............................................................................. 3 
Eligible Veteran Patients ........................................................................................................ 4 
Exclusion Criteria ................................................................................................................... 4 
Outcome Measures ................................................................................................................ 4 
Randomization ....................................................................................................................... 4 
Data Sources .......................................................................................................................... 5 
Corporate Data Warehouse (CDW) data ............................................................................. 5 
Chart Review data .............................................................................................................. 5 
Follow-up Period .................................................................................................................... 5 
ANALYSIS PLAN ....................................................................................................................... 5 
Intention-to-Treat Analysis ...................................................................................................... 5 
Per-Protocol Analyses ............................................................................................................ 5 
Sensitivity Analyses ................................................................................................................ 6 
POWER ..................................................................................................................................... 7 
DATA MONITORING ................................................................................................................. 7 
Enrollment .............................................................................................................................. 7 
Interim Analysis ...................................................................................................................... 8 
Safety ..................................................................................................................................... 8 
REFERENCES .......................................................................................................................... 8 
 
OVERALL TELEGRACE PROJECT RATIONALE 
The Geriatric Resources for Assessment and Care of Elders (GRACE) program is a 
collaborative, multidisciplinary care model that includes geriatricians, social workers, nurse 
practitioners, and primary care providers and which provides home-based geriatric care 
management. GRACE has been demonstrated in a randomized controlled trial to improve 
quality of care and reduce healthcare utilization compared to usual care.4 The VA-GRACE 
model of care was evaluated in a controlled study in a large VAMC among elderly Veterans who 
were recently admitted to the hospital, finding that GRACE was associated with 7.1% fewer ED 

Page 2 of 9 
 
visits, 14.8% fewer 30-day readmissions, 37.9% fewer hospital admissions, and 28.5% fewer 
bed-days of care, saving the VAMC an estimated $200,000 per year after program costs.3 
 
The Indianapolis VAMC GRACE program has been demonstrated to improve care and 
outcomes for older Veterans. In a propensity weighted data analysis with 634 GRACE patients 
and 5614 matched control patients, GRACE was significantly associated with lower all-cause 
mortality after discharge from an index hospitalization: all-cause mortality 90-day post discharge 
was 1.3% for GRACE patients and 8.5% for control patients. However, the reach of the program 
is limited due to logistical constraint of the 20-mile driving distance that the GRACE teams use 
for their home visits. The overall objective of the TeleGRACE project is to improve the care 
and outcomes of older Veterans with a recent inpatient stay by expanding access to the 
GRACE program, by evaluating a telehealth implementation. Specifically, the TeleGRACE 
project seeks to overcome the drive-time distance barrier that currently prevents the widespread 
application of GRACE to eligible older Veterans by implementing virtual home visits instead of 
in-person home visits. 
 
The TeleGRACE evaluation will focus on three primary aims and a secondary aim: 
Primary Aim 1: To examine the effectiveness of the TeleGRACE program, we designed a 
randomized controlled implementation trial (RCT) powered for the primary outcome of 90-day 
all-cause mortality. We will also examine its effectiveness for the secondary outcomes including 
90-day readmissions, 1-year ED utilization (VA and non-VA), 1-year all-cause readmissions, 1-
year mortality, as well as patient, caregiver, and staff satisfaction. We hypothesize that patients 
who receive TeleGRACE will have lower 90-day mortality than patients in usual care.  
 
Primary Aim 2: to examine the implementation of the TeleGRACE program. The 
implementation strategy is reflecting & evaluating.  Implementation outcomes are based on the 
REAIM framework and include reach, efficacy (Aim 1), and implementation (total number of 
Veterans served, fidelity).  
 
Primary Aim 3: to conduct a business-case analysis (BCA).  The business case analysis will 
calculate the net financial savings or loss for TeleGRACE as the difference in the overall 
intervention costs and savings due to downstream benefits for patients receiving TeleGRACE 
versus usual care controls. 
 

Page 3 of 9 
 
 
 
THIS ANALYSIS PLAN FOCUSES ONLY ON EFFECTIVENESS  
To examine the effectiveness of the TeleGRACE program, we designed a randomized 
controlled implementation trial (RCT) powered for the primary outcome of 90-day all-cause 
mortality. We will also examine its effectiveness for the secondary outcomes including 90-day 
readmissions, 1-year ED utilization (VA and non-VA), 1-year all-cause readmissions, 1-year 
mortality, as well as patient, caregiver, and staff satisfaction. We hypothesize that patients who 
receive TeleGRACE will have lower 90-day mortality than patients in usual care.  
 
METHODS 
The TeleGRACE project will be a prospective, randomized controlled trial. The unit of 
randomization is the patient. This project is an evaluation of a clinical program and is 
classified as quality improvement and not research. 
 
Intervention: In-Person versus Telehealth  
The GRACE program involves geriatric-focused home visits by a nurse practitioner and a social 
worker followed by multidisciplinary meetings to discuss each Veteran’s plan of care along with 
ongoing collaboration with VA primary care. This “in-person” model has been in effect at the 
Indianapolis VA medical center since 2010. In the telehealth approach, a staff member will be 
trained to conduct home visits using the GRACE protocol and will use video telehealth (i.e., VA 
video connect [VVC]) from the patient’s home to connect with the GRACE multidisciplinary team 
members located at the coordinating center in Indianapolis. The use of the virtual home visit 
instead of the in-person home visit is the only change that is made for the TeleGRACE 
program. The multidisciplinary team meetings as well as the follow-up and coordination of care 
with the primary care PACT teams will be unchanged. The number of visits and the content of 
the assessments will be the same for TeleGRACE patients as for in-person GRACE patients.  
 
 
 
 

Page 4 of 9 
 
Eligible Veteran Patients  
Patients eligible to receive TeleGRACE services include: 
• 
Veterans discharged from the Indianapolis VA medical center (VAMC) for an 
medical/surgical diagnosis (excludes substance use disorder-related admissions; 
excludes planned admissions)  
• 
Age ≥70 years 
• 
Not enrolled in home-based primary care (HBPC) 
• 
Not enrolled in hospice 
• 
Not in dialysis 
• 
Primary care visit within VA in the prior 2 years 
• 
Not residing in nursing home, skilled nursing facility, or CLC 
• 
CAN score ≥95th percentile for mortality or missing CAN score 
• 
Discharged from the hospital alive 
 
Exclusion Criteria  
• 
Patients who have been randomized to the control arm after their index hospitalization 
who become readmitted to the hospital may not be re-randomized. 
• 
Enrolled in in-person GRACE  
• 
Enrolled in home-based primary care (HBPC) 
• 
Enrolled in hospice 
• 
Enrolled in complex care program 
• 
Dialysis (hemodialysis or peritoneal dialysis) 
• 
Residing in nursing home, skilled nursing facility, or CLC 
• 
Living >60 miles from the Indianapolis VAMC facility  
 
Outcome Measures 
The primary outcome for the TeleGRACE evaluation will be: 
• 
90-day all-cause mortality (measured as 90-days from discharge from the index hospital 
stay) 
 
The secondary outcomes are:  
• 
90-day readmissions  
• 
1-year all-cause readmissions 
• 
1-year mortality 
• 
1-year ED utilization (VA and non-VA) 
• 
1-year falls 
 
We will explore using time to nursing home placement as a secondary outcome. We have 
requested the data and will evaluate the data source for completeness; comparing it against 
chart review. If the team considers the data source to be accurate, then we will examine time to 
nursing home placement as a secondary analysis. 
 
Randomization 
Patients will be randomized in ratio of 1 (control):2 (intervention) stratified by site of primary care 
(Indianapolis VAMC versus non-Indianapolis [e.g., CBOCs]).  We created a computer-generated 
randomization list (with random block sizes of 3 or 6) for each strata. With the current capacity 

Page 5 of 9 
 
of VA GRACE program staff, our goal is to enroll 5 patients to TeleGRACE each month. The 
project will seek to enroll a total of 180 patients to TeleGRACE and 360 to usual care in a three-
year recruitment period.   
 
Data Sources 
Corporate Data Warehouse (CDW) data 
CDW data will be used to assess patient baseline characteristics (e.g., age, demographics, past 
medical history, prior healthcare utilization) as well as outcomes (i.e., ED utilization, 
readmission, mortality, and falls). In this way, the same data source is used to identify outcomes 
for both intervention and control patients. 
 
Chart Review data 
Chart review will be used to validate the TeleGRACE dose classification that is based on CDW 
data (see Per-Protocol analysis described below). 
 
Follow-up Period 
Patients will be followed for one year from the date of discharge from their index hospitalization.  
Unlike clinical trials that require in-person outcome assessments and hence implement 
“windows” for data collection, this evaluation will use electronic health record data for outcome 
assessment and hence no windows will be specified. 
 
 
ANALYSIS PLAN 
 
Intention-to-Treat Analysis 
We will compare baseline characteristics between TeleGRACE patients and control patients 
using chi-square tests for categorical data and t-tests for continuous characteristics. 
 
In the primary intention-to-treat (ITT) analysis, we will compare outcomes between patients 
randomized to usual care and those to TeleGRACE adjusting for the baseline characteristics 
that are significantly different by logistic regression for the binary outcome and Poisson 
regression for count data. If the sample size is sufficient, we will examine the effect of the 
TeleGRACE program by site of receipt of primary care services.  
 
Per-Protocol Analyses 
Not all patients who are randomized to receive TeleGRACE will receive services from the 
TeleGRACE staff. This situation is anticipated for patients who decline GRACE and others who 
become ineligible after discharge from the index hospitalization but before their first virtual home 
visit post-discharge (e.g., patients who die, get readmitted, or move to a skilled nursing facility 
after discharge). In the exceptionally rare case, patients may be discharged from TeleGRACE 
due to safety concerns for our staff (e.g., sexually or physically aggressive behavior, 
unwillingness to cage animals, unwillingness to store firearms). The per-protocol analyses will 
be similar to the planned intention-to-treat analysis but will compare the outcomes between 
patients who received GRACE service versus patients in usual care.    
 

Page 6 of 9 
 
Given the expected heterogeneity in the number of TeleGRACE visits, patients will be classified 
into four categories as follows: 
1. Full TeleGRACE dose: the patient received a virtual post-discharge transitional visit, 
followed by a virtual baseline visit with multidisciplinary team discussion, and then follow-
up virtual discussion to ensure that the patient and caregiver agree with and understand 
the plan of care.  These patients may receive additional follow-up contacts with the 
TeleGRACE/GRACE team members over the course of the one year follow-up period.  
Note: in the atypical situation where the patient did not receive a transitional visit but did 
receive a baseline assessment, the patient will still be classified as “full TeleGRACE 
dose). 
2. Transitional visit dose: the patient received a virtual post-discharge transitional visit but 
did not receive a baseline visit. These patients may receive additional follow-up contacts 
(typically by telephone) with the TeleGRACE/GRACE team members over the course of 
the one-year follow-up period.  In some cases, the patient may receive multiple 
transitional visits (e.g., after readmissions). 
3. Partial dose: the patient received neither a post-discharge transitional visit nor a 
baseline visit but had other contact with the TeleGRACE/GRACE team (typically 
telephone visits). 
4. No TeleGRACE: patients may have been contacted by the TeleGRACE/GRACE team in 
an attempt to schedule a visit, but these patients received no clinical visits or 
encounters.  Patients who decline TeleGRACE services are included in this category.  
Please note: in a research randomized controlled trial, only patients who are potentially 
interested in participation are consented and then randomized. TeleGRACE is a quality 
improvement project, therefore enrollment follows procedures used in clinical care. 
 
We will further examine the outcomes among patients across these 4 categories as well as the 
control patients. 
 
Sensitivity Analyses 
Considering the heterogeneity in the TeleGRACE patients, we plan to conduct the following 
sensitivity analyses 
1. Enrolled versus Randomized to TeleGRACE: We will conduct the ITT analysis 
described above which compares patients randomized to TeleGRACE versus usual care 
controls. In addition, we will compare patients who enrolled in TeleGRACE versus 
control patients. 
2. Dose: In addition to the description of TeleGRACE services as provided in the per-
protocol analysis, the dose of the TeleGRACE program will be assessed in terms of:  
a. The total number of GRACE contacts (telephone, virtual or in-person) in the one-
year post-discharge from the index event. 
b. Typically, home visits are conducted by a dyad of a nurse practitioner and social 
worker. Given staffing constraints, some patients may only receive a visit by a 
social worker or a nurse practitioner but not both. Therefore, each transitional, 
baseline, and follow-up visit will be assessed to describe whether it was 
conducted by social work, nursing, or other staff.  
c. A key element of the GRACE program is the development of a multidisciplinary 
plan via discussion at the weekly GRACE rounds. The number of times an 
individual patient is discussed by the entire team in the year post-discharge will 

Page 7 of 9 
 
be recorded (the geriatrician’s summary of the multidisciplinary discussion will be 
used to identify whether a patient was discussed during team rounds). 
 
3. Discontinuous Eligibility Periods:  Patients who become permanently ineligible for 
TeleGRACE (e.g., are readmitted and then subsequently discharged to a long-term care 
facility) will be censored on the date they became ineligible for the outcome measures 
because they became ineligible for services.  However, this censoring plan will be 
implemented retrospectively because it is expected that the older persons who will be 
enrolled in this study may have intermittent periods of ineligibility (e.g., during a hospital 
stay and then followed by short-term rehabilitation stay). Such patients may still benefit 
from the intervention and hence will not be censored either from the intervention or 
control groups.  
a. For all intervention patients, we will calculate the proportion of eligible days a 
patient is cared for by the GRACE team as: the number of days that the patient 
was under the care of the GRACE team divided by the total number of days the 
patient was potentially eligible for GRACE.  
b. We will note whether the period of eligibility was either continuous or 
discontinuous. 
c. For patients with a discontinuous eligibility period, we will describe the patterns of 
discontinuity. For example, some patients may have a short hospital stay early 
post-discharge but then receive a relatively long period of GRACE services in the 
year post-discharge.  In contrast, other patients may have recurrent 
hospitalizations some of which may be followed by short-term rehab stays 
resulting in fragmented eligibility periods of relatively short durations. 
d. If there are more than 10% patients censored for the outcomes, we will conduct 
Cox-regression analysis for the hazard of death and risk-exposure adjusted 
Poisson regression for the number of readmissions and number of falls.   
 
POWER  
For Aim 1 (effectiveness of TeleGRACE versus usual care), we expect to observe 2% and 8% 
90-day all-cause mortality for the TeleGRACE and usual care patients, respectively. This 
projected effect size is conservative based on our propensity-score weighting analysis 
comparing GRACE patients with usual care patients. With 180 TeleGRACE and 360 usual care 
patients, the study has 0.96 power at detect the efficacy of TeleGRACE program with a one-
sided normal test at a significance level 0.05. Considering a 10% drop-out, the analysis has 
0.94 power to establish the TeleGRACE efficacy with the same effect size.  
 
DATA MONITORING  
 
Enrollment 
We will closely monitor the randomization of TeleGRACE with the expected rate of 5 per month. 
We will plot the number of actual randomized patients and its expected number against the 
study time and update the figure on a monthly basis. Prior to the DSMB meeting, we will 
estimate the current randomization rate and project the final number of randomized patients by 
end of the fourth year with the current rate along with its 95% confidence interval based on the 
Poisson distribution. If the upper bound of the 95% interval is below the target number of 180. 
We will modify the enrollment strategy to ensure that we achieve the enrollment goal.   

Page 8 of 9 
 
Interim Analysis 
Since the observed effect size of in-person GRACE versus usual care is quite substantial with 
respect to 90-day all-cause mortality, we plan an interim analysis when a half of the enrolled 
patients can be evaluated for the primary outcome. One-sided normal testing will be applied for 
the interim analysis using O’Brien-Fleming group sequential test with the overall type error 
controlled at 0.05. The hypotheses are:  
𝐻0: 𝑝1 = 𝑝2 𝑣𝑠. 𝐻1: 𝑝1 < 𝑝2, 
where 𝑝1and 𝑝2 stand for the 90-day all-cause mortality in TeleGRACE and usual care arms, 
respectively.  
If the 𝑍−test statistic crosses the boundary of -2.54 at the interim analysis, the trial will be 
recommended for stopping for efficacy; otherwise, the trial will continue to the end with the 
crossing boundary for the test statistic at -1.66. The overall power with the interim analysis for 
this study is 0.82.   
 
Safety 
This is a quality improvement project; few safety issues are anticipated.  The primary safety 
concern would be related to loss of data confidentiality for patients or staff interviewees.  Any 
breaches of confidentiality will be reported. 
 
 
REFERENCES 
 
1. Statistics NCfVAa. Veteran Population: U.S. Department of Veterans Affairs, Fiscal 
Year 2016. 
2. Statistics NCfVAa. Comparison of VA Users and Non-Users: FY2016: U.S. 
Department of Veterans Affairs, 2017. 
3. Schubert CC, Myers LJ, Allen K, et al. Implementing Geriatric Resources for 
Assessment and Care of Elders Team Care in a Veterans Affairs Medical Center: 
Lessons Learned and Effects Observed. Journal of the American Geriatrics 
Society 2016;64(7):1503-9. doi: 10.1111/jgs.14179 [published Online First: 
2016/06/16] 
4. Counsell SR, Callahan CM, Clark DO, et al. Geriatric care management for low-
income seniors: a randomized controlled trial. Jama 2007;298(22):2623-33. doi: 
10.1001/jama.298.22.2623 [published Online First: 2007/12/13] 
5. Luck J, Parkerton P, Hagigi F. What is the business case for improving care for 
patients with complex conditions? J Gen Intern Med 2007;22 Suppl 3:396-402. 
doi: 10.1007/s11606-007-0293-2 [published Online First: 2007/12/06] 
6. Leatherman S, Berwick D, Iles D, et al. The business case for quality: case studies 
and an analysis. Health Aff (Millwood) 2003;22(2):17-30. doi: 
10.1377/hlthaff.22.2.17 [published Online First: 2003/04/04] 
7. Smith M, Barnett P, Phibbs C, et al. Microcost methods of determining VA healthcare 
costs. In: VA, ed. Menlo Park, CA: VA Health Economics Resource Center, 2010. 

Page 9 of 9 
 
8. Rollins A, Kukla M, Salyers M, et al. Comparing the Costs and Acceptability of Three 
Fidelity Assessment Methods for Assertive Community Treatment. Adm Policy 
Ment Health 2017;44(5):810-16. doi: 10.1007/s10488-016-0785-7 
9. Xu H, Daggy J, Yu D, et al. Joint modeling of medical cost and survival in complex 
sample surveys. Stat Med 2013;32(9):1509-23. doi: 10.1002/sim.5617 [published 
Online First: 2012/10/02] 
10. VA Health Economic Resource Center (HERC). Budget Impact Analysis. 
11. Mauskopf JA, Sullivan SD, Annemans L, et al. Principles of good practice for budget 
impact analysis: report of the ISPOR Task Force on good research practices--
budget impact analysis. Value in health : the journal of the International Society 
for Pharmacoeconomics and Outcomes Research 2007;10(5):336-47. doi: 
10.1111/j.1524-4733.2007.00187.x [published Online First: 2007/09/25] 
12. Druss BG, von Esenwein SA, Compton MT, et al. Budget impact and sustainability 
of medical care management for persons with serious mental illnesses. The 
American journal of psychiatry 2011;168(11):1171-8. doi: 
10.1176/appi.ajp.2011.11010071 [published Online First: 2011/06/17] 
13. Sullivan SD, Mauskopf JA, Augustovski F, et al. Budget impact analysis-principles of 
good practice: report of the ISPOR 2012 Budget Impact Analysis Good Practice 
II Task Force. Value in health : the journal of the International Society for 
Pharmacoeconomics and Outcomes Research 2014;17(1):5-14. doi: 
10.1016/j.jval.2013.08.2291 [published Online First: 2014/01/21]