25

Risk Prediction Models for Hospital Readmission

A Systematic Review Evidence-based Synthesis Program

APPENDIX A. Search Strategy

Databases: Ovid Medline(R) 1996 to June 3 2010/Ovid Medline(R) and Ovid 

OLDMEDLINE(r) 1948 to June Week 3 2010/Ovid MEDLINE(R) In-Process & Other 

Non-Indexed Citations

Initial search: June 24, 2010

Update search: March 31, 2011

# Searches Results

1 exp Patient Readmission/ 5473

2 readmi$.mp. 12202

3 rehosp$.mp. 2507

4 1 or 2 or 3 13761

5 exp Risk/ 620133

6 model$.mp. 1748945

7 predict$.mp. 693749

8 risk$.mp. 1163956

9 util$.mp. 380641

10 5 or 6 or 7 or 8 or 9 3398624

11 4 and 10 5897

12 “smith index”.mp. 3

13 “Probability of Repeated Admissions”.mp. 1

14 11 or 12 or 13 5901

15 limit 14 to “all adult (19 plus years)” 4095

16 remove duplicates from 15 4001

Update search additional yield 558 

Databases: Cochrane Central Trial Registry (CCTR)/Cochrane Database of Systematic 

Reviews (CDSR)/Database of Abstracts of Reviews of Effectiveness (DARE)

Searched: June 24, 2010

Update search: March 31, 2011

# Searches Results

1 exp Patient Readmission/ 401

2 readmi$.mp. 1478

3 rehosp$.mp. 523

4 1 or 2 or 3 1857

5 exp Risk/ 18547

6 model$.mp. 37010

7 predict$.mp. 28433

8 risk$.mp. 63301

9 util$.mp. 13015

10 5 or 6 or 7 or 8 or 9 112650

11 4 and 10 975

12 “smith index”.mp. 0

13 “Probability of Repeated Admissions”.mp. 0

14 11 or 12 or 13 975

After deduplication with previous search 960 unique citations

Update search additional yield: 137
26

Risk Prediction Models for Hospital Readmission

A Systematic Review Evidence-based Synthesis Program

Database: EMBASE 

Date: August 29, 2011 

# Searches Results

1 ‘hospital readmission’/exp 7,965

2 readmi* AND [embase]/lim 13,680

3 rehosp* AND [embase]/lim 2,913

4 #1 OR #2 OR #3 17,205

5 ‘risk’/exp 1,050,898

6 model* AND [embase]/lim 1,786,265

7 predict* AND [embase]/lim 920,849

8 risk* AND [embase]/lim 1,496,969

9 util* AND [embase]/lim 502,028

10 #5 OR #6 OR #7 OR #8 OR #9 4,175,899

11 #4 AND #10 7,846

12 ‘smith index’AND [embase]/lim 3

13 ‘probability of repeated admissions’ 1

14 #11 OR #12 OR #13 7,850

15 #11 OR #12 OR #13 AND ([adult]/lim OR [aged]/lim) 4,185

after deduplication with previous searches 1358 unique citations

Database: CINAHL Plus with Full Text 1986 - 2010 

Initial search: June 24, 2010 

Update search: March 31, 2011

Search 

ID# Search Terms Search Options Results

S21 #S16 NOT #S20 Search modes - Boolean/Phrase 122

S20 S13 or S14 or S18 Narrow by SubjectAge3: - Aged, 80 and over 

Narrow by SubjectAge2: - Adult: 19-44 years 

Narrow by SubjectAge1: - Middle Aged: 45-64 years 

Narrow by SubjectAge0: - Aged: 65+ years 

Search modes - Boolean/Phrase 

1024

S19 S13 or S14 or S18 Search modes - Boolean/Phrase 1789

S18 S11 and S17 Search modes - Boolean/Phrase 1789

S17 S1 or S2 Search modes - Boolean/Phrase 3606

S16 S12 or S13 or S14 Narrow by SubjectAge3: - Aged, 80 and over 

Narrow by SubjectAge2: - Adult: 19-44 years 

Narrow by SubjectAge1: - Middle Aged: 45-64 years 

Narrow by SubjectAge0: - Aged: 65+ years 

Search modes - Boolean/Phrase 

1146

S15 S12 or S13 or S14 Search modes - Boolean/Phrase 2013

S14 “probability 

of repeated 

admissions”

Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean/Phrase 

1
27

Risk Prediction Models for Hospital Readmission

A Systematic Review Evidence-based Synthesis Program

Search 

ID#

Search Terms Search Options Results

S13 “smith index” Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean/Phrase 

0

S12 S4 and S11 Search modes - Boolean/Phrase 2013

S11 S5 or S6 or S7 or 

S8 or S9 or S10

Search modes - Boolean/Phrase 501973

S10 util* Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean/Phrase 

108769

S9 risk* Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean/Phrase 

259597

S8 predict* Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean/Phrase 

87796

S7 “model*” Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean/Phrase 

149297

S6 (MH “Risk 

Factors”)

Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean/Phrase 

55640

S5 (MH “Risk 

Assessment”)

Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean/Phrase 

27742

S4 S1 or S2 or S3 Search modes - Boolean/Phrase 3959

S3 rehosp* Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean/Phrase 

684

S2 readmi* Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean/Phrase 

3606

S1 (MH 

“Readmission”)

Limiters - Published Date from: 19860101-20100631 

Search modes - Boolean /Phrase 

2514

Update search additional yield 122
28

Risk Prediction Models for Hospital Readmission

A Systematic Review Evidence-based Synthesis Program

APPENDIX B. Inclusion/exclusion criteria for review of full-text articles

1. Is the full text of the article in English? 

Yes .......................... ..........................................................................................Proceed to #2

No ............................................................................................................... Code X1. STOP 

2. Does the study population include adult patients admitted to a medical service? 

Yes ............................ ........................................................................................Proceed to #3

No ............................................................................................................... Code X2. STOP 

3. Is the article a primary study that develops or tests prediction models for risk of hospital 

readmission?

Yes ........................................................................................................................ Proceed to #4

No .........................................................................................................Code X3. Proceed to #5

4. Is the model tested in both a derivation and validation cohort, or is it a validation of a 

previously developed model?

Yes ......................................................................................................... Code I4. Proceed to #6 

No .........................................................................................................Code X4. Proceed to #6

5. Is the article a systematic review or meta-analysis of prediction models for risk of hospital 

readmission?

Yes ........................................................................................................Code X5. Proceed to #6

	 (Eligible primary studies identified in systematic reviews will be coded I4).

No ......................................................................................................................... Proceed to #6

6. If article meets none of the above criteria but may be useful for background/discussion, add 

code “B.”

Population: Adult patients admitted to a medical service. Post-surgical patients and psychiatric 

re-admissions are excluded. 

Intervention: Risk prediction models derived and validated in a cohort of medical inpatients.

Comparator: Studies comparing the performance of two or more risk prediction models in a 

population will be included.

Outcomes: Hospital readmission – including all-cause readmissions, condition-specific readmissions, 

and potentially preventable readmissions. Readmission of inpatients to ICU is excluded. 

Timing: No restrictions. 

Setting: Exclude studies conducted in health systems of developing nations. 
29

tal Readmission: A Systematic Review Evidence-based Synthesis Program

tudy quality assessment criteria

f 

Non-biased selection† Low loss to followup‡ Adequate prognostic factor measurement§ Adequate outcome measurement|| Method of validation

Unsure 

(appears to be 

consecutive series but 

does not explicitly state 

this)

Unsure 

(did not report follow-up)

Yes Yes Derived and validated 

in same population 

using cross-validation 

metholodogy (75% 

derivation, 25% 

validation repeated 

1000 times)

ption 

o 

erall 

)

Yes Unsure

(did not report follow-up)

No

(several prognostic factors 

not clearly described)

No, included interhospital 

transfers and did not exclude 

deaths

Different large cohort

Yes Unsure Yes Partly - unclear if transfers 

and elective readmissions 

were excluded.

Split sample in a 

large cohort

Yes Yes Partly - unsure how accurate 

geographic deprivation 

scores are

Partly - unclear if transfers 

and elective readmissions 

were excluded.

Split sample in a 

large cohort

Yes Yes Yes Partly - unlikely that 

readmissions to other 

hospitals were captured

Split sample in a 

small cohort

Yes Unsure Yes Yes Split sample in a 

large cohort

Yes Partly 

(they excluded the 11% 

of patients for whom 

incomplete information 

was available)

Yes Yes Split sample in a 

large cohort

Yes Unsure Yes Yes Split sample in a 

large cohort
30

tal Readmission: A Systematic Review Evidence-based Synthesis Program

f 

Non-biased selection† Low loss to followup‡ Adequate prognostic factor measurement§ Adequate outcome measurement|| Method of validation

Yes Partly - use of Medicare 

data ensures good degree 

of f/u, but data on transfers 

to skilled nursing facilities 

from home not readily 

available

Yes Partly - outcome was 

complicated care transitions 

which included admission 

to skilled nursing facility 

from home. Not clear that 

such transfers were reliably 

identified using available 

datasets

Different large 

cohorts

on 

ll 

Unsure (probably 

consecutive series)

Unsure Yes Partly - outcome combined 

hospital readmissions, 

skilled nursing transfer from 

hospital, and stay longer 

than mean expected for 

DRG. It is unclear how valid 

the use of this combined 

measure is. Not clear that 

elective readmissions and 

transfers were excluded.

Split sample in a 

small cohort

Yes Unsure Yes Yes Split sample in a 

large cohort

Yes Unsure Yes Partly – 20% of registry 

patient files could not be 

linked to Medicare claims 

data and, therefore, is 

not included in outcome 

determination

Bootstrapping in 

large cohort

Partly - large # 

excluded because they 

did not complete intake 

interview, mostly 

because they were 

discharged before they 

could be interviewed. 

Sample therefore will 

be skewed towards 

longer LOS patients

Unsure - # completing 30 

D f/u to gather self-report 

utilization data unclear

Yes Partly - readmissions to 

non-index hospitalization 

based on self-report and 

therefore subject to recall 

bias. Elective readmissions 

were included

Split sample in a 

large cohort
31

tal Readmission: A Systematic Review Evidence-based Synthesis Program

f 

Non-biased selection† Low loss to followup‡ Adequate prognostic factor measurement§ Adequate outcome measurement|| Method of validation

on 

rly 

Yes Yes Yes Yes Split sample in a 

large cohort

Yes Yes Partly - well-described 

methods, but administrative 

data only and comorbidity 

variables could have 

captured complications 

rather than comorbidities. 

No mention of effort to 

validate administrative data 

against chart review data.

Yes Split sample in a 

large cohort

Yes Yes Yes Partly - unclear how 

transfers were handled

Split sample in a 

large cohort

Yes Unsure Yes Partly - readmissions 

from state-specific HCFA 

database - cross-state 

readmissions wouldn’t be 

captured

Different large 

cohorts

Yes Unsure Partly - data based on 

medical chart review, but 

unclear how well certain 

factors such as smoking 

history were documented in 

the medical record

Partly - readmissions to 

other facilities were not 

captured, but it was a rural 

area and there were few 

other options for care

Different small 

cohorts

Yes Unsure Yes Yes Jackknife approach 

using portions of 

large cohort

on 

 

6 

is 

 was 

Partly - consecutive 

patients but large # 

not reached before 

discharge and large # 

refused participation, 

many because they did 

not feel well and this 

could have skewed 

sample.

Yes Yes Partly - not clear that 

elective readmissions were 

excluded

Moderate size 

validation cohort 

only (validation of a 

previously derived 

instrument)
32

tal Readmission: A Systematic Review Evidence-based Synthesis Program

f 

Non-biased selection† Low loss to followup‡ Adequate prognostic factor measurement§ Adequate outcome measurement|| Method of validation

Yes Unsure Unsure Partly - unclear if transfers 

and elective readmissions 

were excluded.

Split sample in a 

large cohort

Yes Unsure

(did not report % of 

patients with outcome 

available, mean “followup” 6.9 months)

Partly - calendar year 

readmissions used as 

outcome and f/u interval 

was not included as a 

covariate

Partly

(used calendar year 

readmissions rather than 12 

months follow-up period, 

making admissions later in 

year less applicable. And, 

cross-state readmissions not 

captured)

Split sample in a 

large cohort

Yes Partly

(21.9% of patient records 

were not available through 

the end of 1988 and thus 

were excluded, but a 

selectively corrected two 

stage probit model no 

common influence between 

likelihood of missing data 

and readmission)

Yes Yes Split sample in a 

large cohort

Partly - enrolled 

patients and provider 

participants over only a 

5 week period

Partly - unclear how many 

patients were contacted 

successfully for self-report 

utilization measure

Yes Yes Validation of a 

previously derived 

instrument (PRA)

Yes Unsure (did not report) Partly

(comorbidities only assessed 

via discharge ICD-9 coding, 

not via actual measurement 

methods/scales)

Partly

(only included readmission 

to Baylor MC, not to outside 

facilities)

Split sample in a 

large cohort

Yes Yes Yes Yes Split sample in a 

small cohort

Yes Unsure Yes Partly - a single center 

study and unclear how well 

readmissions to outside 

hospitals were captured

Validation of a 

previously derived 

instrument
33

tal Readmission: A Systematic Review Evidence-based Synthesis Program

f 

Non-biased selection† Low loss to followup‡ Adequate prognostic factor measurement§ Adequate outcome measurement|| Method of validation

f 

as 

at 

> 

t 

is 

and 

were 

his 

Partly - consecutive 

patients, but 20% 

could not be reached 

before discharge 

and another 20% 

declined to participate. 

Characteristics of these 

pts “were similar”.

Unsure Yes Yes Bootstrapping in a 

small cohort

Yes Unsure Partly - unclear how severity 

and complexity variables 

were calculated

Yes different cohort 

(unsure of size)

Yes Yes, follow-up on 95.6% of 

population

Yes Partly - outcome was selfreport. Readmissions were 

considered unplanned if not 

arranged when patient had 

been discharged from index 

hospitalization.

Different large cohort

for selecting patients, and for enrolled patients describes duration and severity of symptoms, demographics (at least age), and setting (primary 

attempting to enroll) a consecutive series of patients meeting inclusion criteria, or a random sample.

lable for at least 80% of patients at 6 months or later of follow-up.

appropriate methods for measuring prognostic factors.

 appropriate methods to define and identify readmission; transfers and deaths during index hospitalization were excluded.