FINAL EVALUATION 

REPORT 

St. Elizabeth Medical Center 

January 2015 
2 

FINAL EVALUATION REPORT 

St. Elizabeth Medical Center

January 2015 

Acknowledgements 

This report was developed under grant #10SIHKY001 

 
3 

Table of Contents

I. INTRODUCTION 4

II. METHODS 6

A. EVALUATION DESIGN 6

B. DATA COLLECTION 7

III. FINDINGS 7

A. PROGRAM IMPLEMENTATION 8

B. PROGRAM IMPACT 12

C. DEMOGRAPHICS 14

D. SHORT-TERM (0-12 MONTHS) 15

E. INTERMEDIA TE OUTCOMES (12-24 MONTHS) 18

F. LONG-TERM IMPACT OUTCOMES (24-36 MONTHS) 21

G. LONGITUDINAL ANALYSIS 22

IV. SUMMARY 24

V. LESSONS LEARNED 24

VI. NEXT STEPS 25
4 

St. Elizabeth Healthcare 

Access to Mental Health Services Through Emergency Department Telepsychiatry 

I. Introduction 

Rising costs and lack of health insurance places the emergency department (ED) as a safety net 

for those who cannot find healthcare elsewhere. This is especially true for patients with mental health 

and substance abuse (MHSA) conditions. According to the American College of Emergency Physicians 

(ACEP), the “inability to access psychiatric care has caused patients needing psychiatric care to seek care 

in emergency departments or go without”.1 Additionally, patients with a mental health/substance abuse 

(MHSA) diagnosis are nearly 2.5 times more likely to be admitted through the emergency department 

(ED) when compared to patients with no MHSA condition.2 

Despite significant utilization of the ED by those with mental health conditions, few specialized 

resources are available to the ED. According to Treatment Advocacy Center, Kentucky has a “severe” 

shortage of public psychiatric hospitals with only 15.6 beds per 100,000 people, compared to a national 

average of 17 beds per 100,000 people in 2005 (down from 340 beds/100,000 in 1955).3 Data from 2005 

indicate that 2.8 % (or 2.7 million) of more than 99 million total emergency department visits in the 

United States were associated with a primary mental health diagnosis. 4 In 2011, the St. Elizabeth 

Healthcare System reported 208,660 visits to its five emergency departments.5 Of these, 6,954 (3.3% of 

the total) visits resulted in a primary mental health diagnosis, above the national average (~3%). Of 

those with mental health as a primary diagnosis, 39% (2,692) were admitted to the hospital for inpatient 

care. This exceeds the national average of 20.7% reported by the Agency for Healthcare Research and 

Quality.6 

In addition to a shortage of specialized mental health services in emergency departments, 

patients presenting in the ED with a MHSA condition are costing patients and hospitals billions. In 2007, 

total MHSA expenses for adults ages 18–64 were approximately $36.5 billion, $13 billion higher than in 

1997 (adjusted to 2007 dollars). In addition, a higher proportion of adults ages 18–64 reported a MHSA 

condition related expense in 2007 than in 1997 (12.6 versus 7.9, respectively). Among all Americans, 

36.2 million people paid for mental health services totaling $57.5 billion in 2006 (this includes children 

and adolescents).7 At St. Elizabeth, about half (48%) of the overall mental health population who 

received services from St. Elizabeth Healthcare were Medicaid or Self Pay patients. For their emergency 

department care, St. Elizabeth incurred financial losses for their treatment exceeding $2.5 million from 

April 2010 through June 2011. 

 

1

 American College of Emergency Physicians (ACEP) Psychiatric and Substance Abuse Survey 2008. Fact Sheet. 

Available at: http://www.acep.org/uploadedFiles/ACEP/Advocacy/federal_issues/PsychiatricBoardingSummary.pdf 

2

 Healthcare Cost and Utilization Project, Statistical Brief #92. Available at: http://www.hcupus.ahrq.gov/reports/statbriefs/sb92.pdf 

3

 March, 2008:The Shortage of Public Hospital Beds for Mentally Ill Persons: A Report of the Treatment Advocacy 

Center http://www.treatmentadvocacycenter.org/storage/documents/the_shortage_of_publichospital_beds.pdf 

4

 2005 Nationwide Emergency Department Sample (NEDS), Healthcare Cost and Utilization Project (HCUP), Agency 

for Healthcare Research and Quality. 

5

 Mental health related diagnoses, for the purpose of this project, include the following the diagnoses associated 

with Mental Health ICD-9-CM diagnosis codes. Examples of mental health related diagnoses include: anxiety 

disorders, attention-deficit/hyperactivity disorders, eating disorders, mood disorders (bipolar, disthymic, and 

major depressive disorders), personality disorders, substance-abuse disorders and schizophrenia. 

6

 Source: Center for Financing, Access and Cost Trends, AHRQ, Household Component of the Medical Expenditure 

Panel Survey, 2007 

7

 Brown, E. Health Care Expenditures for Adults Ages 18–64 with a Mental Health or Substance Abuse 

Related Expense: 2007 versus 1997. Statistical Brief #319. March 2011. AHRQ, Rockville, MD. 
5 

With the rapid advances in technology, telemedicine has now become a viable alternative for 

assessing/treating patients when services are not readily available such as specialty services in remote 

rural areas. As psychiatric access can be limited in outlying areas, a Telepsychiatry program is especially 

crucial for access for these patients. Timely, expert assessments may also lead to a reduction in 

inappropriate inpatient admissions, thereby reducing charges. Patients who are assessed by a mental 

health professional will also receive more timely and appropriate treatment and better linkages to 

outpatient care. This will likely reduce the number of unnecessary re-visits to the emergency 

department and better long-term patient health. Mental health assessments conducted by licensed 

mental health professionals will help to provide a standardized mental health assessment to patients 

presenting in emergency departments, followed by providing them with a behavioral plan of care and 

referrals via a Telepsychiatry program offered 24 hours daily. In addition to ensuring a quality, 

consistent service, a goal is also to shorten wait times in the ED and improve disposition such that ED 

visits, readmission rates, and costs are ultimately reduced in patients receiving these services over time 

compared to those who do not receive this service. It is also expected that satisfaction with the ED is 

maintained throughout the integration of the Telepsychiatry Service. An intermediate goal is to track a 

cohort of patients to assess how well they complete follow ups with referrals to community mental 

health resources. 

The Access to Mental Health Services through Emergency Department Telepsychiatry program is 

employing an innovative strategy of linking five geographically separate emergency departments to a 

crisis team of mental health professionals at a community-based healthcare organization to provide 

more timely, accurate and effective diagnosis and disposition (care or referral) to these individuals. The 

program addresses access to care and reduces health risks/disparities by offering psychiatric 

assessments via telemedicine by a mental health professional at any time, 24/7, and at any of our 

hospital emergency departments. The new program relieves the ED physicians from this service by 

providing a timely mental health assessment of the mental health patient by a mental health 

professional. Consequently, the mental health professional works with the on-site physician during this 

single ED visit to develop a detailed plan of care so that appropriate and timely mental health care 

assessments are conducted with the goal of improving mental health care in high risk patients. This will 

increase access to a mental health professional assessment, increase timeliness of a mental health 

assessment, decrease overall cost per patient, and decrease ED re-visits and inpatient admissions. 

St. Elizabeth Healthcare operates six facilities throughout Northern Kentucky—St. Elizabeth(s) 

Covington, Edgewood, Falmouth, Florence, Fort Thomas, and Grant. Before implementation of the 

Telepsychiatry Program, three (Edgewood, Covington and Florence) of the five St. Elizabeth EDs had onsite access to mental health assessment for people who seek care at them; two (Ft. Thomas and Grant) 

offered no on-site access. At the three with access, service hours are limited to only 4-14 hours/day 

depending on the ED. St. Elizabeth’s, in collaboration with NorthKey Community Care, a regional 

community mental health center (CMHC), established a telepsychiatry program allowing patients 

presenting to any of the system’s five EDs with mental health needs to be assessed by a licensed mental 

health professional. Mental health professionals (MHP) at two fixed-site offices at the main Edgewood 

Campus are available via audio/visual technology to complete real time patient assessments of mental 

health patients at the other four EDs during the day and evening hours. Another site through NorthKey 

covers the night hours, thus enabling northern Kentucky residents to have access to mental health 

professionals 24/7. 

St. Elizabeth developed the logic model below (Figure 1) from the Kellogg Foundation’s Logic 

Model Development Guide (2004). The logic model details the process for implementing the St. 

Elizabeth Telepsychiatry program through inputs and activities as well as the outputs and expected 

short-term, intermediate, and long-term outcomes. 
6 

Figure 1: St. Elizabeth Healthcare Telepsychiatry Logic Model 

II. Methods 

The Telepsychiatry Program is offered to every individual presenting with mental health concerns 

that meets criteria at any one of the five St. Elizabeth Emergency Department. Patients are offered a 

description of the Program and must give their consent to participate. Data is collected through 

electronic medical records (EMR) as well as through self-report surveys given to patients. 

A. Evaluation Design 

The Telepsychiatry Program was evaluated based on two phases: program implementation and 

program impact. 
7 

1. Program Implementation 

Elements of program implementation were evaluated using monitoring logs and survey tools. The 

quantity and quality of the Telepsychiatry program was measured by the number of staff trained and 

the number of patients receiving a screening. 

2. Impact Evaluation 

This report details a preliminary data analysis of the impact of the Telepsychiatry program. Data 

were analyzed between the Target Groups (Years 1 and 2) and an Archival Comparison Group. 

Preliminary aggregated findings were also included for Target Group Year 3 as a reference. The Groups 

are detailed below: 

 The ArchivalComparison Group is comprised of archival data of patients seen in any of 

the five St. Elizabeth ED facilities for mental health concerns from April 1, 2011 to March 

31, 2013 (Year 0) before the Telepsychiatry Program was implemented. 

 Target Group Year 1 includes participants in the St. Elizabeth’s Telepsychiatry Program 

who consented to the Telepsychiatry intervention during ED intake from April 1, 2012 to 

March 31, 2013. 

 Target Group Year 2 includes participants in the St. Elizabeth’s Telepsychiatry Program 

who consented to the Telepsychiatry intervention during ED intake from April 1, 2013 to 

March 31, 2014. 

 Target Group Year 3 includes participants from the first two quarters (6 months) of the 

third year of operation (April to September 2014). These data were received in 

aggregate form and were unable to be included in the analyses. 

The proposed evaluation was to include a matched comparison group including individuals 

presenting at the ED with mental health concerns who were either unable or elected not to receive 

treatment through the St. Elizabeth Telepsychiatry Program. However, not enough patients have 

refused the Telepsychiatry Program in Year 1 or Year 2 to develop a matched comparison group. 

B. Data Collection 

Program implementation data were collected by survey and weekly inventories and compiled from all 

trainings by program administrators. Additional implementation data were pooled from relevant EPIC 

(St. Elizabeth’s EMR) data and shared with evaluators. Program impact data were collected from 

relevant information pooled by an EPIC specialist on a monthly basis and shared with the evaluators. 

During Year 1, it was noted that some staff members were unsure of what screens on EPIC certain 

variables were reported and contributed to a low amount of data for that variable. An IT professional 

addressed this challenge in Year 2 by developing flow-sheets and adding specific data points that could 

be captured through our electronic health record. Specifically, a data point was added to display the 

time the mental health professional started his/her assessment so that the length of the assessment 

could be captured. 

 

III. Findings 

Data provided below details the implementation and impact findings for the St. Elizabeth Healthcare 

Telepsychiatry Program. 
8 

A. Program Implementation 

Table 1: Summary of Implementation Indicators and Corresponding Findings: Detailed findings are provided for each question below. 

Research Question Indicator Finding

1.

To 

Program Implementation: (Year 

what extent did training activities 

1 Only)

take place as planned? (Logic Model A1)

a. How many staff were trained? Number of staff trained. N=150 expected. 146 staff were trained

b. Did staff attend all required training dates? Number of trainings attended per staff member. 100% of staff attended all required trainings

c.

Did the staff think that the training was adequate? 

And/or did the training increase staff’s comfort level 

with the program and using the equipment?

Ratings of 3 

Percentage 

or 

of 

greater 

positive 

on individual survey responses. 

written responses.

93% of staff gave a rating of 

conduct 

3 

a 

or higher for “I am able to set 

Telepsychiatric Assessment”

up equipment and 

2.

To 

Program Implementation: (Year 1 Only)

what extent did St. Elizabeth Healthcare establish the needed infrastructure for the ED Telepsychiatry Program? (Logic Mod el A2)

a. Was the 

needs?

equipment adequate for the program’s Integrity and fidelity of Telepsychiatry equipment The equipment was adequate for the Program.

b. Was the infrastructure available 

anticipated timeframe?

within the Availability of equipment for patients as needed Equipment was 

modified 

ready for the start of the Telepsychiatry Program and 

as needed to reflect ED volume and patient needs.

was 

3.

To 

Program Implementation:

what extent did the screening and intake process facilitate patient access to Telepsychiatry Program services? (Logic Model A3 and A4)

a. How many mental health screenings were provided? Number of 

expected.

mental health screenings at intake. N=2000 

Screenings offered to determine eligibility

Target Group Year 1 = 1920 screenings were offered to 

Target Group Year 2 = 1973 screenings were offered to 

Target Group, Year 3 (6 mos.) = 921 screenings offered

1674

1723

patients

patients

b. How many patients 

program at intake?

consent to the Telepsychiatry 

Number of patients who consent and receive 

Telepsychiatry services. N=1800 meet program criteria 

with 95% level of participation in Telepsychiatry 

services. 

Target Group Year 1

1314 ED visits involving 1122 patients consented to Telepsychiatry Services 

patients did not consent, with a 98.1% patient participation rate)

Target Group Year 2

1359 ED visits involving 1160 patients consented to Telepsychiatry Services 

patients did not consent, with a 99.1% patient participation rate )

(22

(11 

c. How many Telepsychiatry needs/risk assessments 

were provided? Number of Telepsychiatry assessments.

Telepsychiatry Assessments Completed

Target Group Year 1 = 1314

Target Group Year 2 = 1359

Target Group, Year 3 (6 mos.) = 595

d. How many patients 

referral?

received a plan of care and Number of patients with Plan of Care and/or referral.

100% of patients that had a telepsychiatry assessment received a plan of care 

and a referral to primary physicians, outpatient services at St. Elizabeth’s, 

NorthKey, or other community mental health resources.
9 

4.

Are 

Program Implementation:

study measures being monitored across sites to ensure continuous quality improvement and impact? (Logic Model A5)

a. Are recruitment goals being met consistently? Number of patients who 

Telepsychiatry services.

consent and receive 

Target Group Year 1 = 1314 ED visits

received Telepsychiatry Services

Target Group Year 2 = 1359 ED visits

received Telepsychiatry Services

involving 

involving 

1122 patients 

1160 patients 

consented 

consented 

and 

and 

b. What 

lower 

factors are affecting 

staff ratios, etc.)?

recruitment (e.g. busy ED, ED volume and staffing records 

Telepsychiatry participation.

compared to Year 

Year 

1: 

2: 

Staffing and technology issues

Minimal complications
10

1. Staff Trainings (Logic Model OUT1) 

In February 2012, 146 staff members received a mandatory educational training on the 

Telepsychiatry processes, as well as procedures for triage, screening, and eligibility. 100% of staff 

attended all required trainings. Trainings were 4 hours and took place on site in the computer rooms to 

display documentation. The equipment and assessments were available for staff to use and learn. 99.3 

percent of trained staff members agreed or strongly agreed that they were “able to state the 

Telepsychiatric Program process and procedures for arrival types, safety screenings, and behavioral 

precautions.” Additionally, 98.7 percent agreed or strongly agreed “I am able to locate the appropriate 

flow sheet in EMR and demonstrate accurate and complete documentation.” 93.0 percent agreed or 

strongly agreed to “I am able to set up equipment and conduct a Telepsychiatric Assessment.” All 

evaluation questions received an average rating of 3.4 or higher out of a possible 4. Table 2 below 

details the full survey results. 

Table 2: Telepsychiatry Program Staff Education Evaluation

Strongly 

Agree Agree Total Average Rating(out of 4)

I am able to state the Telepsychiatric Program process 

and procedures for arrival types, safety screening and 

behavioral precautions.

44.2% 55.1% 3.43

I am able to locate the appropriate flow sheet in EMR and 

demonstrate accurate and complete documentation. 55.5% 43.2% 3.53

I am able to describe the process and procedures for 

assessing proper placement for the patient. 47.0% 49.7% 3.42

I am able to communicate with the ED team the 

procedure forpatients meeting criteria for Telepsychiatric 

assessment.

49.7% 47.6% 3.45

I am able to set up equipment and conduct a 

Telepsychiatric Assessment. 34.3% 58.7% 3.26

I anticipate St. Elizabeth Healthcare will see positive 

patient outcomes as a result of the application of my 

learning.

45.2% 51.4% 3.40

I found the room comfortable and experienced minimal 

distractions during the class. 57.7% 38.9% 3.53

I am clear about what is expected of me as a result of this 

educational class. 50.0% 45.9% 3.45

2. Infrastructure and Operation (Logic Model OUT2) 

Telepsychiatry equipment was purchased and installed. Equipment is portable and can be easily 

moved to the patient’s room. The camera allows the Telepsychiatry social worker to pan the room and 

focus on different objects or people as needed. 

Equipment was modified to ensure successful implementation of the Program. After starting the 

program, it was determined that the picture and sound quality would be greater if the system was hard 

wired instead of run over a wireless connection. Additionally, an extra Telepsychiatry unit was added at 

the Florence Emergency Department to deal with an increase in volume. 
11

3. Screening, Intake, MHP Assessment and Education (Logic Model OUT3 and OUT4)

All patients presenting with a mental health concern in the 5 emergency departments were 

screened to determine eligibility for the Telepsychiatry program. Eligibility is determined based on 

considerations such as; can the equipment be left in the room with the patient, can the patient 

cooperate with the mental health assessment, and can the patient communicate with the mental health 

professional. Additionally, if a patient is acutely intoxicated, requires restraint or seclusion, presentsin 

the ED with a court ordered hold, is unstable or violent, or speaks a foreign language without a medical 

certified interpreter present they were also excluded.

A total of 1920 screenings were offered to 1674 patients in year 1 and 1973 screenings were offered 

to 1723 patients in year 2. At intake, a nurse does a psychosocial assessment on everyone who presents 

with mental health concern. Then it is determined if the patient is eligible for Telepsychiatry Program or 

if the mental health assessment will be conducted face to face, or by a doctor.

Although data was sparse in determining consent, we feel confident that most if not all of the 

persons who we do not have consent for would have consented because of the high rate of usage and 

the high patient satisfaction scores. Inyear 1, 1144 patients, over 1347 ED visits, were determined to 

be eligible for Telepsychiatry, and 1122 patients with 1314 ED visits consented. The patient 

participation rate was 98.1 percent. In year 2, 1171 patients, over 1379 ED visits, were determined to 

be eligible for Telepsychiatry, and 1160 patients with 1359 ED visits consented. The patient 

participation rate was 99.1 percent. Some consenting patients were seen in person by mental health 

professionals and others, in some cases, were assessed by the ED doctor. Patients were assessed by ED 

doctors either because of long wait times for the mental health professional, or because the ED doctor 

was familiar with the patient from previous visits. These factors were not captured in the data exported 

from EPIC and are recognized as possible confounds in the current evaluation.

Our initial assessment found that all year 1 patients (100 percent) who received a Telepsychiatry 

assessment received a plan of care and a referral following their treatment in the St. Elizabeth’s 

Emergency Department. Patients are referred to primary physicians, outpatient services at St. 

Elizabeth’s, NorthKey, or other community mental health resources, such as Cincinnati Children’s 

Hospital, Mercy Mt. Airy, Lindner Center of Hope, or The Ridge Behavioral Health System in Lexington.

4. Mental Health ProfessionalContinuous Quality Improvement Monitoring (Logic Model OUT5)

Telepsychiatry recruitment goals were met having 1122 patients in year 1 and 1160 patients in year 

2 consenting to use the Telepsychiatry Program. In year 1, issues with staffing, such as physicians not 

wanting to wait for a Telepsychiatry consult and instead conducting the mental health assessment on 

their own, may have had an effect on recruitment. 

Any time there was an issue with equipment, it was reported to the IT department Help Desk. 

Before the system was fully implemented, two patients’ Telepsychiatry visits were terminated due to 

lack of connectivity. From April 2013 to September 2014, there were 14 reports: 

 4 hardware issues (e.g., blown fuses)

 4 user errors (e.g., leaving a phone off the hook)

 2 configuration errors necessitating changing a network setting or rebooting

 2 server errors

 2 broken screens (one from a stretcher banging into the monitor and the other was damaged by 

a patient)- Two new monitors are on backorder and should arrive soon. One monitor was sent 

from Edgewood to Ft. Thomas to accommodate for their broken monitor. The other broken 

screen was in Florence. Since they had two screens, they are down to one until the damaged 

one is replaced. 
12

B. Program Impact 

Table 3: Summary of Impact Indicators and Corresponding Findings: Detailed findings are provided for each question below. 

Research Question Indicator Finding

Program 

1.

Impact:

Does the St. Elizabeth Telepsychiatry Program improve ED mental healthcare?

a. To what extent did the program increase access to 

assessment by a mental health professional? (Logic 

ST1)

Model 

Percentage 

by a mental

of mental health patients who 

 health professional in the ED.

consent to Telepsychiatry care Patients consented to 

of Target Group Year 1 

Group Year 2 visits.

Telepsychiatry care 

visits and 99.1% of 

for 98.1%

Target 

b. To what extent did the program reduce time from 

mental health assessment? (Logic Model ST2)

arrival to Minutes from arrival to assessment. (Wait Time) Average wait 

Target Group 

Target Group 

time 

Year 

Year 

from arrival to assessment

1 = 120.9 minutes

2 = 109.6 minutes

c. To what extent did 

discharge, transfer 

Model ST3)

the program reduce 

or admission (length 

time from arrival 

of stay)? (Logic 

to Length of Stay (LOS) in minutes, which is recorded in minutes from the 

point of intake. LOS is inclusive of wait time and describes the period of 

time between arrival to final disposition whether this is by discharge, 

transfer, or admission from the ED

Average Length of Stay:

Archival Comparison Group = 205.3 minutes

Target Group Year 1 = 256.9 minutes

Target Group Year 2 = 262.3 minutes

d. To what extent did the program increase the number of 

mental health patients who receive follow-up care outside 

the ED? (Logic Model ST4)

Follow up care received from referral service or resource. 100 patient records randomly selected 

27% scheduled an outpatient appointment

20 of 27 attended their appointment

8 non-compliant patientsreached

Barriers: Finances, availability of appointment 

and difficulty navigating resources

times, 

Program 

2.

Impact:

Compared to patients receiving usual care (i.e.: Assessment by an ED professional) do those individuals who elect to receive the Telepsychiatry program experience improved outcomes?

a. To what extent did the program reduce subsequent ED 

visits among patients receiving a care plan? (Logic Model 

INT1)

Number of return visits within 2 years of receiving a Plan of Care. Average Revisits in 30 days per Patient:

Archival Comparison Group = .17 visits

Target Group Year 1 = .07 visits

Target Group Year 2 = .07 visits

Average Revisits in 1 year per Patient:

Archival Comparison Group = .35 visits

Target Group Year 1 = .17 visits

Target Group Year 2 = .17 visits

b. To what extent did the program reduce 

within 30 days (Logic Model INT2)

readmission rates Number of readmissions within 30 days. Average Readmissions in 30 Days Per Patient:

Archival Comparison Group = .06 readmissions

Target Group Year 1 = .05 readmissions

Target Group Year 2 = .04 readmissions

c. What percentage 

care (Logic Model 

of patients 

INT3)

report satisfaction with ED Proportion of mental 

patient care survey.

health ED patients with positive responses on Preliminary data shows a 95.3%

satisfaction

reported overall 

3. Program Impact:

To what extent did the program decrease overall inpatient 

admissions for patients in the ED (Logic Model LT1)?

Number of inpatient admissions Overall Admission Rates:

Archival Comparison Group = 

Target Group Year 1 = 61.5%

Target Group Year 2 = 67.5%

36.4% 
13

a. To what extent, does the Telepsychiatry program reduce 

the costs of mental healthcare in the ED? (Logic Model 

LT2)

Calculated average cost per patient in ED visits and eventual disposition.

Average ED Cost per Patient (US$):

Archival Comparison Group = 1624.11

Target Group Year 1 = 1947.78

Target Group Year 2 = 2087.25
14

C. Demographics 

The Archival Comparison Group included 5361 patients with a total of 7264 emergency 

department visits. The Year 1 Target Group included 1122 patients with a total of 1314 emergency 

department visits where patients received Telepsychiatry Services. The Year 2 Target Group included 

1160 patients with a total of 1359 emergency department visits where patients received Telepsychiatry 

Services. Table 4 displays the demographics of all three groups. 

Adequate resources could not be acquired to conduct a third year of evaluation for the 

Telepsychiatry program. However, we were able to gather aggregated participation data for the first two 

quarters of the third year of operation. Demographics for these individuals are illustrated below in 

Table 4. Across both quarters (April to September 2014), 921 patient visits to St. Elizabeth emergency 

departments for mental health complaints were recorded. Of these cases, 595 received Telepsychiatry 

assessments, 270 received in-person psychiatric assessments, and 56 received no assessment. Note that 

the 56 cases with no assessment were part of the aggregate report and could not be excluded from the 

demographic statistics below. 

Table 4: Demographics* 

Demographics

Archival 

Comparison

N=5361

Target Year 1

N=1122

Target Year 2

N=1160

Target Year 3

N=921

Gender

Female 2590 (48.3%) 523 (46.6%) 499 (43.0%) 207 (22.5%)

Male 2771 (51.7%) 539 (48.0%) 612 (52.8%) 234 (25.4%)

Unknown 0 (0.0%) 60 (5.3%) 49 (4.2%) 0 (0.0%)

Age

18 - 39 years 3217 (60.0%) 622 (55.4%) 690 (59.5%) 259 (28.1%)

40 - 64 years 2144 (40.0%) 414 (36.9%) 419 (36.1%) 182 (19.8%)

Unknown 0 (0.0%) 86 (7.7%) 51 (4.4%) 0 (0.0%)

Race

White/Caucasian 4944 (92.2%) 972 (86.6%) 1017 (87.7%) 399 (43.4%)

Black/African-American 285 (5.3%) 71 (6.3%) 67 (5.8%) 26 (2.9%)

Native American/Hawaiian or API 12 (0.2%) 1 (0.1%) 1 (0.1%) 1 (0.1%)

Asian 12 (0.2%) 2 (0.2%) 5 (0.4%) 1 (0.2%)

Other 4 (0.1%) 0 (0.0%) 0 (0.0%) 7 (0.8%)

Unknown 104 (1.9%) 76 (6.8%) 70 (6.0%) 6 (0.6%)

Ethnicity

Non-Hispanic 5254 (98.0%) 1051 (93.7%) 1093 (94.2%) 431 (46.8%)

Hispanic 86 (1.6%) 11 (1.0%) 18 (1.6%) 9 (0.9%)

Unknown 21 (0.4%) 60 (5.3%) 49 (4.2%) 1 (0.1%)

Language

English 5279 (98.5%) 1058 (94.3%) 1109 (95.6%) 439 (47.7%)

Spanish 23 (0.4%) 1 (0.1%) 2 (0.2%) 2 (0.2%)

Other 5 (0.1%) 0 (0.0%) 0 (0.0%) 0 (0.0%)

Unknown 54 (1.0%) 63 (5.6%) 49 (4.2%) 0 (0.0%)
15

Demographics

Archival 

Comparison

N=5361

Target Year 1

N=1122

Target Year 2

N=1160

Target Year 3

N=921

Insurance/ Payment

Type

Private Insurance 1517 (28.3%) 252 (22.5%) 263 (22.7%) 94 (10.2%)

Medicaid 946 (17.6%) 207 (18.4%) 261 (22.5%) 227 (24.7%)

Medicare 618 (11.5%) 128 (11.4%) 141 (12.2%) 82 (8.9%)

Self-pay 2214 (41.3%) 410 (36.5%) 417 (35.9%) 32 (3.4%)

Other government (VA, Tricare) 64 (1.2%) 8 (0.7%) 18 (1.6%) 6 (0.6%)

Unknown 2 (0.0%) 117 (10.4%) 60 (5.2%) 0 (0.1%)

*Chi squared analyses were performed to analyze demographic differences between groups (“unknown” and 3rd year 

demographics were excluded). Insurance Payment Type significantly differed by group, χ2(8, N=7464) = 35.58, p<.001. There 

were no significant differences between groups for gender, age, race, ethnicity, and language.

D. Short-Term (0-12 Months)

Once the Telepsychiatry program was successfully implemented and MHP hours were expanded 

at all 5 emergency departments, short-term outcomes were monitored to assess increased access to a 

MHP for patients presenting in the ED with a mental health concern, a reduction in ED wait times and ED 

length of stay, and an increase in the number of mental health patients receiving follow-up care in the 

ED.

1. Access: (Logic Model ST1)

Prior to the Telepsychiatry Program, only three of the five Emergency Departments had on-site 

access to a mental health professional and service hours were limited. Implementing the Telepsychiatry 

Program increased access to a MHP for patients presenting in all five of the Emergency Departments

with a mental health concern at any time. Telepsychiatry provides an opportunity to standardize the 

assessment and referral process across sites, ensuring that patients receive a plan of care.

During intake, a nurse does a psychosocial assessment to determine any mental health 

concerns. Following this assessment, the patients presenting with mental health concerns is screened 

for Telepsychiatry and asked to consent. Patients eligible for Telepsychiatry consented to using the 

program 98.7 percent of visits in Year 1 and 99.3 percent of visits in Year 2.

2. ED Wait Times (Logic Model ST2)

In year 1, data collection for Telepsychiatry patient’s ED wait times began in January 2013 and is 

only available for a limited number of patients because it took some time to improve the reporting 

process in EPIC to obtain accurate data. Wait times were collected for the entirety of year 2. More 

reliable median wait times are reported along with means due to a large amount of variability. The 

median wait time from arrival to mental health assessment is 93 minutes in Year 1 and 88 in Year 2 

(Table 5).
16

Table 5: Emergency Department Wait Times (Minutes) – January 2013 to March 2014 

Target Group Year 1 Target Group Year 2

Median Mean Standard 

Deviation Median Mean Standard Deviation

Door to Mental Health Assessment 

Order 38 58.6 87.30 35 53.0 64.20

Mental Health Assessment Order to 

Mental Health Professional 76 98.9 87.79 80 99.5 85.03

Door to Mental Health Professional 97 128.9 134.17 102 129.4 109.29

Door to Mental Health Assessment 

Start

93 120.9 102.72 88 108.7 81.48

Mental Health Assessment Start to 

Complete 51 59.9 36.64 - - -

Note. Target Group Year 3 wait times (in minutes) for “Door to Mental Health Professional” and “Mental Health Assessment Order to Mental 

Health Professional” were 20.3 and 48.0, respectively. No further Year 3 wait time data are available at this time. 

3. ED Stay Length (Logic Model ST3) 

 Data was analyzed to compare the difference in length of stay for patients in Year 0 before 

Telepsychiatry (Archival Comparison Group) to Years 1 and 2 during Telepsychiatry (Target Groups). 

More reliable median wait times are reported along with means due to a large amount of variability. In 

addition, two outliers were removed prior to the analysis due to invalid length of stay times (values were 

negative). The Year 1 Target Groups’ median length of stay was 54 minutes per visit longer than the 

comparison group and the Year 2 Target Groups’ median length of stay was 64 minutes per visit longer 

than the comparison group (Table 6). In addition, the first 6 months of Year 3 continue this trend. 

Finally, Table 7 shows length of stay by common primary diagnoses. Similar to the overall patter, the 

higher lengths of stay for Target group patients were present within patients of similar diagnoses. It 

should be noted that this could be due to a more thorough assessment occurring with the Telepsychiatry 

Program. 

Table 6: Length of Stay in Minutes Per ED Visit 

Median 25th

Percentile

75th

Percentile Mean (SD) 95% Confidence Interval

Lower Bound Upper Bound

Archival Comparison 

Group

(N=7262)

179.0 111 265 205.3 (137.4) 202.2 208.5

Target Group Year 1

(N=1235) 233.0 174 311 256.9 (132.8) 249.5 264.3

Target Group Year 2

(N=1359) 243.0 181 321 262.3 (114.6) 256.2 268.4

Target Group Year 3*

(N=865) - - - 269.0 - -

*Target Group Year 3 data were received in an aggregated report. It was not possible to conduct more sophisticated 

analyses to assess its differences from the prior years.
17

Table 7: Length of Stay in Minutes Per ED Visit by Common Primary Diagnoses 

Primary 

Diagnosis

Archival Comparison Group

(N=7262)

Target Group Year 1

(N=1235)

Target Group Year 2

(N=1359)

% with 

Diagnosis

Mean 

(SD)

95% CI % with 

Diagnosis

Mean 

(SD)

95% CI % with 

Diagnosis

Mean 

(SD)

95% CI

Lower 

Bound

Upper 

Bound

Lower 

Bound

Upper 

Bound

Lower 

Bound

Upper 

Bound

Schizophrenia 2.1% 227.8 

(136.2) 207.8 247.8 1.4% 195.6 (75.3) 136.9 254.2 1.8% 236.3 (68.8) 185.5 287.0

Acute 

Depression 3.4% 239.6 (89.7) 223.8 255.5 4.9% 253.1 (95.1) 221.2 285.0 4.3% 272.5 (95.5) 240.1 304.9

Psychotic 

Disorder 3.4% 249.3 (108.7) 233.5 265.1 3.6% 276.7 (122.6) 239.2 314.2 4.0% 263.5 (96.6) 230.0 297.1

Anxiety 

Disorder 14.6% 122.3 (67.8) 114.6 129.9 2.7% 160.6 (88.5) 118.5 202.6 3.7% 201.6 (94.8) 166.4 236.8

Opioid Abuse 3.2% 162.1 

(118.3) 145.7 178.6 1.9% 228.5 (73.8) 178.8 278.3 2.0% 229.3 (111.2) 181.4 277.2

Alcohol 

Abuse 15.5% 269.0 (196.7) 261.6 276.4 2.7% 382.0 (222.6) 340.5 423.5 4.0% 328.2 (194.1) 294.3 362.0

Chronic 

Depression 14.8% 222.8 (125.5) 215.2 230.4 20.7% 243.4 (104.5) 228.0 258.8 17.8% 254.2 (115.9) 238.2 270.2

Suicidal 

Ideation 8.5% 252.1 (104.1) 242.1 262.1 25.7% 267.2 (107.6) 253.6 280.9 31.1% 277.8 (100.2) 265.7 289.9

4. Follow-up Care (Logic Model ST4)8

 To determine if patients that participated in Telepsychiatry services followed through with 

outpatient referrals, 100 patient records with visits between January 2013 and June 2013 were 

randomly selected and evaluated to assess compliance. In addition, non-compliant patients (i.e., 

patients that did not schedule and attend an outpatient appointment) were contacted via telephone to 

discuss barriers. 

 Of the 100 selected patients, 27 scheduled an outpatient appointment and 20 of those 27 

attended their appointment. Of the 80 non-compliant patients, 59 were called to discuss barriers. Eight 

(14%) patients were successfully reached where they indicated finances, availability of appointment 

times, and difficulty navigating resources as their primary barriers. 

Note that 6 patients within this sample already had an outpatient appointment scheduled prior 

to their Telepsychiatry assessment. Taking this into consideration, the remaining 94 patients were used 

to create 95% confidence intervals. Based on these data, it estimated that 15-32% of patients with 

outpatient referrals will schedule an appointment and approximately 54-89% of those patients will 

attend their appointment. The average time for patients to schedule an appointment from the time of 

their Telepsychiatry assessment is estimated to be 3.8-11.4 days. 

 

8

 Outpatient follow-up data was acquired and computed by C. Faith Denigan of Northern Kentucky University as 

part of a clinical cap-stone project entitled “Emergency Department Telepsychiatry Referrals and Perceived 

Barriers” completed in April of 2014.
18

These findings are preliminary and were gathered using a qualitative survey. A new follow-up 

survey will be developed that will make use of more quantitative measures. The implementation and 

analysis of the new follow-up data will be carried out beyond the scope of this grant. 

 

E. Intermediate Outcomes (12-24 Months)

 As the Telepsychiatry program accrues more patients, intermediate outcomes should include 

reduced rates of return ED visits and readmission rates as well as higher rates of patient satisfaction with 

ED care. 

1. ED Visits (Logic Model INT1) 

 Analyses were conducted to determine whether or not Telepsychiatry Program reduce a 

patient’s revisit rate to St. Elizabeth Emergency Departments. Compared to the Archival Comparison 

Group receiving usual care (i.e., Assessment by a ED physician), Year 1 and 2 Target patients receiving a 

Telepsychiatry plan of care showed a reduction in subsequent ED visits over both 30 day and 1 year 

periods. 

 A one-way analysis of variance (ANOVA) showed that revisits within 30 days differed between 

groups, F(2, 7640) = 16.16, p<.001 (Table 8). The sample achieves greater than 99% power to detect a 

small effect size (f = 0.07). A Bonferroni post hoc analysis indicated that Target Years 1 and 2 had 

significantly fewer revisits within 30 days than the Archival Comparison group (p<.001). 

Table 8: Number of Revisits within 30 Days per Patient 

Mean SD

95% Confidence Interval

Lower Bound Upper Bound

Archival Comparison Group

(N=5361) .17 .85 .15 .20

Target Group Year 1 

(N=1122) .07 .32 .05 .09

Target Group Year 2 

(N=1160) .07 .31 .05 .09

 In addition to the 30 days analysis, a one-way ANOVA showed that subsequent ED visits were 

more common among patients in the baseline group across one year, F(2, 7640) = 27.26, p<.001 (Table 

9). The sample achieves greater than 99% power to detect a small effect size (f = 0.08). A Bonferroni post 

hoc analysis indicated that Target Years 1 and 2 had significantly fewer revisits within one year than the 

Archival Comparison group (p<.001). 

Table 9: Number of Return Visits within 1 Year per Patient 

Mean SD

95% Confidence Interval

Lower Bound Upper Bound

Archival Comparison Group

(N=5361) .35 1.13 .32 .39

Target Group Year 1 

(N=1122) .17 .55 .14 .20

Target Group Year 2 

(N=1160) .17 .54 .14 .20
19

2. Psychiatry Inpatient Admission Rates (Logic Model INT2)

 Overall, patients participating in the Telepsychiatry program (Target Groups Years 1-3) had 

higher rates of admission into both St. Elizabeth’s facilities and higher rates of transfers to non-St. 

Elizabeth’s facilities than the Archival Comparison Group (Table 10). It should be noted that the Target 

Group data includes patients admitted to St. Elizabeth Hospital and St. Elizabeth Mental Health units. 

Data detailing patient’s location after being admitted to St. Elizabeth was not available for the Archival 

Comparison Group. This difference in data collection could contribute to an increase in admissions due 

to specificity for the Target Group. 

Table 10: Overall Admission Rates per Visit

Admitted to a St. Elizabeth

Facility

Transferred to a Non-St. 

Elizabeth Facility

Archival Comparison Group

(N=7264)

36.4% 2.8%

Target Group Year 1 

(N=1312) 61.5% 3.4%

Target Group Year 2 

(N=1358) 67.5% 0.8%

Target Group Year 3 

(N=865) 70.0% 0.6%

In addition to the admission rates, one-way ANOVAs were conducted to compare the difference 

in readmission and transfer rates between groups. There were no significant differences between 

groups on readmission rates within a year or within 30 days (Tables 11-12). A significant difference was 

present between groups for transfers to non-St. Elizabeth facilities, F(2, 7640)=10.92, p<.001 (Table 13).

A Bonferroni post hoc analysis indicated that Target Year 2 had significantly fewer transfers to non-St. 

Elizabeth facilities than both the Archival Comparison group (p<.001) and Target Group Year 1 (p=.001). 

Table 11: Readmission Rates over 30 Days per Patient 

Mean SD 95% Confidence Interval

Lower Bound Upper Bound

Archival Comparison Group

(N=5361) .06 .34 .05 .07

Target Group Year 1 

(N=1122) .051

.24 .04 .06

Target Group Year 2 

(N=1160) .042

.22 .03 .06

1Did not significantly differ from Archival Comparison group (p=.859) 

2Did not significantly differ from Archival Comparison group (p=.272) 

Table 12: Readmission Rates within 1 Year per Patient 

Mean SD 95% Confidence Interval

Lower Bound Upper Bound

Archival Comparison Group

(N=5361) .09 .42 .08 .10

Target Group Year 1 

(N=1122) .081

.34 .06 .10
20

Target Group Year 2 

(N=1160) .102

.36 .08 .12

1Did not significantly differ from Archival Comparison group (p=1.00) 

2Did not significantly differ from Archival Comparison group (p=1.00) 

Table 13: Patients Transferred to a Non-St. Elizabeth Facility 

Mean SD 95% Confidence Interval

Lower Bound Upper Bound

Archival Comparison Group

(N=5361) .04 .20 .03 .04

Target Group Year 1 

(N=1122) .041

.20 .03 .05

Target Group Year 2 

(N=1160) .012

.11 .00 .02

1Did not significantly differ from Archival Comparison group (p=.859) 

2Significantly fewer transfers than the Archival Comparison group (p<.001) 

3. Patient Satisfaction (Logic Model INT3) 

Patients were given a survey related to patients’ satisfaction with various aspects of emergency 

department care, such as patient care, time spent, etc. Nurses hand out satisfaction surveys after 

patients see the mental health professional. Data were collected for 55 patients in Year 1 and 9 patients 

in Year 2. Due to the limited Year 2 sample size, all patient satisfaction data were merged and analyzed 

together (n = 64). 

Of the patients surveyed (Table 14), 79.7 percent strongly agreed that “staff prepared me for 

the video process” and 70.3 percent strongly agreed that it was “easy to talk to a mental health 

professional over the video equipment.” Overall, 71.9 percent strongly agreed that overall they were 

satisfied. In addition, data was collected from all of St. Elizabeth’s emergency departments to assess 

overall patient satisfaction (Table 15). Telepsychiatry patients were more satisfied with their emergency 

department experience (71.9 percent strongly agree) than other patients seen in the emergency 

department in 2013 (51.8 percent) and 2014 (55.2 percent). 

Table 14: Telepsychiatry Patient Satisfaction Survey Results 
21

Strongly 

Disagree Disagree Agree

Strongly 

Agree

Staff Prepared me for the video 

process

1.6% - 18.8% 79.7%

Easy to talk with MHP over the video 

equipment - 3.1% 26.6% 70.3%

Sound/Image quality were good - 1.6% 23.4% 75.0%

Time with MHP was private 1.6% 4.7% 23.4% 70.3%

Felt MHP cared about my problem - 1.6% 25.0% 73.4%

Was comfortable receiving assessment 

by video 3.1% 4.7% 23.4% 68.8%

Satisfied with services 3.2% - 28.6% 68.3%

Would be comfortable using video 

again 1.6% 4.7% 26.6% 67.2%

Assessment by video are of value 1.6% 4.7% 35.9% 57.8%

Overall was satisfied - 4.7% 23.4% 71.9%

Table 15: Overall Emergency Department Patient Satisfaction Data 

2013 Florence

N=1500

Ft. Thomas

N=1200

Covington

N=1200

Edgewood

N=2400

Grant

N=900

Average Rating

N=7200

Overall ED rating 

(% rating 9 or 10 out of 10) 54.4% 58.6% 58.9% 62.7% 57.0% 58.9%

Would Recommend 

(% saying yes) 69.7% 70.9% 72.9% 77.5% 66.9% 72.7%

Instructions or Explanations of 

Treatment/Tests from Nurses 

(% rating excellent)

49.6% 50.5% 50.7% 50.1% 49.4% 50.1%

Instructions or Explanations of 

Treatment/Tests from Doctors 

(% rating excellent)

49.5% 50.9% 49.7% 50.8% 48.52% 50.1%

Overall Quality of Care

(% rating Excellent) 49.1% 50.8% 51.8% 55.0% 48.7% 51.8%

2014 (YTD) Florence

N=937

Ft. Thomas

N=750

Covington

N=750

Edgewood

N=1500

Grant

N=563

Average Rating

N=4500

Overall ED rating 

(% rating 9 or 10 out of 10) 57.7% 66.7% 65.6% 66.7% 64.5% 64.4%

Would Recommend 

(% saying yes) 68.5% 74.7% 73.3% 78.0% 71.1% 73.8%

Instructions or Explanations of 

Treatment/Tests from Nurses 

(% rating excellent)

49.0% 53.6% 52.1% 51.5% 52.8% 51.6%
22

Instructions or Explanations of 

Treatment/Tests from Doctors 

(% rating excellent)

52.5% 55.7% 51.0% 53.7% 54.8% 53.5%

Overall Quality of Care

(% rating Excellent) 50.6% 58.8% 54.4% 55.7% 57.4% 55.2%

F. Long-Term Impact Outcomes (24-36 months) 

1. Readmission Rates Within 2 Years 

As noted in Table 12 above, the mean rate of readmissions within 2 years of receiving a plan of 

care was .10 (N=1160) while Year 1 was .08 (N=1122). The Bonferroni post-hoc analysis indicated that 

Target Group Year 2 did not differ from Year 1 (p=983) in contrast to our predictions. 

2. ED Cost Analysis (Logic Model LT2) 

Average ED costs per patient were calculated (adjusted for inflation) to determine if the 

Telepsychiatry program successfully reduced ED costs (see Table 16). A one-way ANOVA was conducted 

to examine ED costs by year of operation, yielding a significant difference between groups, F(2, 7632) = 

245.57, p<.001. A Bonferroni post-hoc test showed that each year significantly differed from one 

another at a significance level of p<.001. The data demonstrates a significant increase in costs for each 

successive year, in contrast to the logic model (LT2). 

Table 16: Average ED Costs Per Patient in US Dollars 

Mean SD 95% Confidence Interval

Lower Bound Upper Bound

Archival Comparison Group

(N=5357) 1624.11 780.66 1603.20 1645.02

Target Group Year 1 

(N=1118) 1947.78 575.75 1913.99 1981.56

Target Group Year 2 

(N=1160) 2087.25 592.49 2053.12 2121.38

F(2, 7632) = 245.57, p<.001 

G. Longitudinal Analysis 

It was recognized, during Year 1, that some patients visited St. Elizabeth emergency departments 

across multiple years of operation. In order to better understand the impact of the Telepsychiatry 

program, longitudinal analyses were conducted to determine patient outcomes from the Archive 

Comparison Group (Year 0) to Target Group Year 2. A total of 49 eligible, consenting patients visited St. 

Elizabeth emergency departments during all 3 observed years of ED operation. A summary of the 

program’s longitudinal impact can be seen below (Table 17).

Table 17: Summary of Longitudinal Impact Measures 

Archival 

Comparison 

Group

(Year 0)

Target Group

(Year 1)

Target Group

(Year 2)

Average Length of Stay Per Visit Mean (SD) 236.4 (138.3) 266.1 (125.8) 291.5 (119.5)
23

(in minutes) 95% CI 196.7, 276.1 229.9, 302.2 257.1, 325.8

Average Number of Revisits 

Within 1 Year

Mean (SD) 2.1 (2.4) 0.6 (1.1) 0.6 (1.0)

95% CI 1.4, 2.8 0.3, 0.9 0.3, 0.9

Average Number of Revisits

Within 30 Days

Mean (SD) 1.1 (1.9) 0.2 (0.7) 0.2 (0.6)

95% CI 0.5, 1.6 0.0, 0.4 0.0, 0.4

Average Number of Readmissions

Within 1 Year

Mean (SD) 0.7 (1.1) 0.3 (0.6) 0.4 (0.8)

95% CI 0.4, 1.0 0.1, 0.5 0.2, 0.6

Average Number of Readmissions

Within 30 Days

Mean (SD) 0.5 (0.7) 0.1 (0.4) 0.1 (0.3)

95% CI 0.3, 0.7 0.0, 0.2 0.0, 0.2

Average Number of Transfers

to a Non-St. Elizabeth Facility

Mean (SD) 0.2 (0.6) 0.0 (0.3) 0.0 (0.0)

95% CI 0.0, 0.4 0.0, 0.1 0.0, 0.0

1. ED Stay Length

Since patients often had multiple visits within a year, each patient’s average length of stay per visit 

was calculated for each year and compared across the three observed years of ED operation. A 

repeated measures ANOVA showed that length of stay had a marginally significant difference across the 

three years, F(2, 96) = 2.861, p=.062. Though not significant, LOS increased in a linear pattern from Year 

0 to Year 2 (see Table 17). 

2. ED Visits

Repeated measures ANOVAs were conducted to determine whether or not a patient’s revisit rate 

within a year and within 30 days changed from Year 0 to Year 2. Revisits within a year showed a 

significant change across ED years of operation, F(2, 96) = 16.401, p<.001. A Bonferroni post-hoc test 

indicated that revisits within a year were significantly more common in Year 0 than both Year 1 (p<.001) 

and Year 2 (p<.001). Revisits within 30 days also showed a significant difference across ED years of 

operation, F(2, 96) = 8.190, p=.001. A Bonferroni post-hoc test indicated that revisits within 30 days 

were significantly more common in Year 0 than both Year 1 (p=.008) and Year 2 (p=.016). See Table 17

for descriptive statistics.

3. Psychiatry Inpatient Admission Rates

Repeated measures ANOVAs were conducted to determine whether readmission rateswithin a 

year, readmissions within 30 days, and transfers to Non-St. Elizabeth facilities changed over time. 

Readmissions within a year showed a significant change across ED years of operation, F(2, 96) = 3.902, 

p=.023. A Bonferroni post-hoc test indicated that readmissions within a year were significantly more 

common in Year 0 than Year 1 (p=.013). Readmissions within 30 days also showed a significant 

difference across ED years of operation, F(2, 96) = 7.764, p=.001. A Bonferroni post-hoc test indicated 

that revisits within 30 days were significantly more common in Year 0 than both Year 1 (p=.009) and 

Year 2 (p=.013). Finally, the number of transfers to Non-St. Elizabeth facilities were marginally significant 

across ED years of operation, F(2, 96) = 2.777, p=.067. Descriptive statistics in Table 17 indicate a drop 

in transfers from Year 0 to Year 1, but no transfers in Year 2.
24

IV. Summary 

 Staff members at all five emergency departments attended all required trainings and 93 percent of 

those staff felt they were able to use the Telepsychiatry process and equipment. 

 The Telepsychiatry Program increased access to a mental health professional by implementing a 

process to ensure patients are screened for mental health concerns and referred to a mental health 

professional as needed. For patients eligible to receive Telepsychiatry, patients consented to 

receiving services in 98.1 percent of visits in year 1 and 99.1 percent of visits in year 2. 

 Infrastructure was built to sustain the Telepsychiatry Program and increased to meet the needs of 

the emergency department volume. 

 The Target Groups (Years 1 to 3) had longer lengths of stay and higher rates of admission, in 

contrast to our logic model predictions. Table 7 indicated that this trend was present across patients 

with similar primary diagnoses. It is possible that the thoroughness of the mental health assessment 

increased patients’ length of stay; however, due to missing data for the length of assessments for 

the Archival Comparison group and Target Year 2, this possibility could not be explored in depth. As 

such, data collection will require improvement to sufficiently capture determinants of longer stays 

at EDs. 

 The Target Groups (Years 1 and 2) showed a significantly lower rate of return visits to the 

emergency department within 30 days and within 1 year. 

 There were no significant differences between groups for readmissions within 30 days and 1 year. 

Target Group Year 2 had fewer transfers to non-St. Elizabeth’s facility than Target Group Year 1 

and the Archival Comparison Group. 

 Of 100 randomly selected patients, 100 received referrals to providers or outside community mental 

health agency for follow up. Of these, 27 (27%) scheduled an outpatient appointment and 20 of 

those 27 (74%) who made an appointment reported attending their appointment. According to 8 

non-compliant patients, finances, availability of appointment times, and difficulty navigating 

resources were their primary barriers. 

 ED costs significantly increased each successive year, in contrast to the logic model (LT2). 

 Longitudinal analyses indicated that return visits and readmission rates within 30 days and within a 

year significantly decreased over time. Differences in length of stay and transfers were marginally 

significant. 

 Patients are reporting 95.3 percent satisfaction rate with the Telepsychiatry Program. However, 

the sample size was very low (n=64). 

V. Lessons Learned 

 Keep Telepsychiatry program small at start-up. St. Elizabeth’s Telepsychiatry program began with 5 

participating emergency departments. This allowed for opportunities to specialize optimal processes 

and expand the program from that point. 

 At minimum, utilize practitioners full-time to maintain efficiency. 

 Start-up costs included staff training and payments to NorthKey for night coverage. 

 Consider costs and minimum technology requirements when selecting equipment. The 

telecommunication equipment used was larger than necessary. In addition, the individual 

components came as part of a package that was not readily compatible with smaller models making 

replacement costly and complicated. 

 Ensure that all participating sites have compatible software (e.g., EPIC) to ensure information is 

transferred with ease. 
25

VI. Next Steps 

St. Elizabeth is further promoting the model by: 

 Improving upon the follow-up survey by adding more quantitative elements. 

 Expanding to New Horizons Medical Center in Owenton, Kentucky. Owen County is a secondary 

service area for St. Elizabeth currently served by NorthKey. 

 Providing service in additional communities (without St. Elizabeth facilities). This way the program 

can assure more equitable access to services beyond the primary service region. 

 Expanding telemedicine services to cardiology and endocrinology.