Health Systems Evidenc

Evidence to Support 

Policymaking and Managemen

<< healthsy g stemsevidence.org >>

Veterans Health Administration 

September 14th September 14 , 2010

Lori Greco MHSc, Knowledge Broker

Kaelan Moat Doctoral Student Kaelan Moat, Doctoral Student
Agenda

 Overview of Health Systems Evidence

 What it is  How is was created Why use it What is in it How to search it

 Tour of Health Systems Evidence Demonstration searches
Health Systems Evidence Partne

3
Interactive Question 1

 What is your role?

ClinicianR h esearc

her

Manager or Administrator

Other
Background

 J pg ohn Lavis’ research program

 To promote the use of research evidence 

policymaking and management

 Michael Wilsons’ work to develop the data
What is Health Systems Evidence

 Online re

p y ositor

y

 Continuously updated Easy s c to 

sear

c

h
What is Health Systems Evidence

Contains policy-relevant synthesized researc

evidence about:

 governance financial and delivery arrangem

governance, financial and delivery arrangem

within health systems 

 implementation strategies that can support 

change within health systems.
Interactive Question 2

 Given what you know so far, are you likely to use Health Sy

Evidence to find synthesized research evidence about governa

to find synthesized research evidence about governa

financial and delivery arrangements within health systems and implementation strategies within health systems?

YesNo

Maybe 
Methods: How we build and mainta

 On

-going process of: going process of:

 Identifying and screening abstracts of systematic reviews/o

syntheses for inclusion; 

 Coding each included synthesis according to type and how 

health systems arrangements and implementation strategies

 Searching for and linking to free online pdfs, full text files, 

Searching for and linking to free online pdfs, full text files, 

summaries, abstracts;

 Applying a validated quality rating tool to produce a metho

q alit score for each s nthesis and q

uality score for each

s

ynthesis and;

 Identifying countries in which the studies included in each 

conducted
How we build and maintain HSE 

 Identifying abstracts of research syntheses

 Centre for Reviews and Dissemination (UK) Data

Abstracts of Reviews of Effects Abstracts of Reviews of Effects

—

“DARE

” (wee

(wee

updates)

 Cochrane librar

y y ( y

 of s

ystematic reviews 

(monthl

y u

 Cochrane Effective Practice and Organization of 

Group (EPOC)

 Other syntheses and sources of syntheses that are 

identified (personal knowledge, forwards from colis

t-servs, ) etc.

)
How we build and maintain HSE 

 Screening abstracts of systematic reviews/other research s

Screening abstracts of systematic reviews/other research s

for inclusion

 Reviews/overviews of reviews: at least two databases 

h b hd l l / l have 

been searc

h

e

d, exp

licit inc

lusion

/exc

lusion criter

be included

 Polic

y briefs: must address two or more of the 

p y

olic

y

options for addressing the problem, and implementatistrategies; and; they draw (at least in part) on systematreviews to describe what is known about two or more problem, options for addressing the problem, and implementation strategies

 Performed independently by at least two reviewers Performed independently by at least two reviewers
How we build and maintain HSE 

 Coding each included synthesis according to type:

• Policy brief

• Oi f i i Overview o

f systematic revi

e

w

• Systematic review• S

y () stematic review 

(Cochrane

)

• Systematic review protocol (often Cochrane)

 Coding each included synthesis according to type of quest

• Effectiveness• Not effectiveness• Many
How we build and maintain HSE 

 Coding based on which health system arrangement(s) and 

implementation strategies they address:

 Go

v g( / ernance Arran

gements 

(i.e. centralization

/decentra

of policy authority)

 Financial Arrangements (i.e. the use of fee for service a

form of remuneration) form of remuneration)

 Delivery Arrangements (i.e. substitution of nurse pract

for general physicians)

 Implementation strategies (i.e. providing consumers wi

information or education about a program)

 Independently coded by two reviewers Independently coded by two reviewers
Governance Arrangement Financial Arrangement Delive

Health System Arrangements

g

(79)

g

(104)

Arrangeme

• Policy Authority (14) • Financing (35) • To whom care is pr

what efforts to reac

• Organizational authority 

(32)

• Professional authority (20)

• Funding (21)• Remuneration (44)

• By whom care is pr

Professional authority (20) • Where care is prov

• Commercial authority (11)

C d k h ld

• Financial incentives for 

patients (27)

R ll i (15)

Where care is prov

• With what informa

communication tec

• Consumer and stakeholder p id d? (215)

involvement (23)

• Resource allocation (15) provided? (215)

• With what level of 

is care provided (37
Consumer-targeted (135) Provider-targeted (181) O

Implementation strategies

• Information or education 

provision (104)

• Behaviour chan

ge support 

• Educational material (80)• Educational meeting (92)• Educational outreach visit (37) g pp (64)

• Skills and competencies 

development (32)

Educational outreach visit (37)

• Local opinion leader (7)• Local consensus process (4)

P i (0) • (Personal) Support (32)

• Communication and 

decision-making facilitation 

• Peer review (0)

• Audit and feedback (49)• Reminders and prompt (0)

(15)

• System participation (1)

• Tailored intervention (2)• Patient-mediated intervention (18)• Multi-faceted intervention (89)

( )
How we build and maintain HSE 

 Searching for and linking to free online pdfs, full text 

user friendly summaries, abstracts

 URLs for publicly available copies of included syn

and user-friendly summaries of these syntheses are identified and included in the database for each syidentified and included in the database for each sy

 i.e. links to PubMed abstracts, Australasian Cochra

Centre Policy Liaison Initiative user-friendly summPDF files if the review is published in an open accjournal
How we build and maintain HSE 

 Applying a validated quality rating tool to produce a 

methodological quality score for each synthesis

 T i id d l h 

Two reviewers 

i

n

depen

dent

ly assess eac

h systema

review in the database using the AMSTAR quality tool

 Reconciled scores are reported with each review

 Country coding

 Two reviewers independently scan reviews to iden

countries in which the studies included were cond

 Reconciled codes appear in the database with eac

Reconciled codes appear in the database with eac
Why use Health Systems Eviden

 Saves time by helping to rapidly identify: 

 synthesis of the best available research evidence o

synthesis of the best available research evidence o

given topic 



pp p re

pared in a systematic and trans

parent wa

y

 how recently the search for studies was conducted quality of the synthesis countries in which the studies included in the syn

were conducted

 key findings from the synthesis key findings from the synthesis
What is in Health Systems Evidenc

 Details about systematic reviews relevant t

he lth s stems he

alth

s

ystems

 policy briefs overviews of systematic reviews overviews of systematic reviews systematic reviews review 

protocols 

 links to any user-friendly summaries, scientific ab

and full-text reports currently available
What is in Health Systems Evidence

 Over 1300 citations (at 2010-8-12)

 17 policy 

briefs

 15 overviews of systematic reviews 842 systematic reviews

• 287 Cochrane reviews

 146 protocols for Cochrane reviews 
Evidence Service 

 General 

• All new reviews (coming soon)

 Topic-specific services 

• Primary healthcare• Ch i di t Chron

ic disease managemen

t

• Diabetes management• Chronic pain management• Rural healthcare strategy

 Subscribe at 

www.mcmasterhealthforum.org/healthsystemsevidence/evidence servwww.mcmasterhealthforum.org/healthsystemsevidence/evidence

_serv
Finding Research Evidence

 Search by: 

 health system topic Canadian priority area (which may overlap with priorit

in other jurisdictions)

 type of synthesis type of synthesis type of research question author open search recent update any combination of these options any combination of these options
Finding Research Evidence (

 Each search will yield a listing of syntheses an

 type of synthesis (e.g., policy brief) type of question (e.g., effectiveness) health system topic (e.g., skill mix – role expansion or  title of the synthesis of the synthesis last year literature searched quality rating (using AMSTAR criteria) countries in which studies included in the synthesis we

conducted

 links to any freely available user to any freely available user

-friendly summaries s

friendly summaries, s

abstracts, and full-text reports.
Navigating Health Systems Eviden

www.mcmasterhealthforum.org
Health Systems Evidence Home P

www healthsystemsevidence org www.healthsystemsevidence.org
Interactive Question 3

 What topp q 

ic areas or 

practice/ research 

questions woul

use Health Systems Evidence to find research 

about?
HSE Search Interface (1)
HSE Search Interface (2)
HSE Search Interface (3)
HSE Search Interface (4)
A Searchable Question

What do 

you want to find out?

 About the problem Ab p s dd ss p b out o

ption

s to address the 

pro

blem

 About option implementation considerations
Sample Search

Scenario:

 We want to provide a patient We want to provide a patient-centered medical ho

centered medical ho

model, e.g., more shared decision-making with paand better coordination of care provided by speci

i t f ti t ith h i di improve outcomes for patients with chronic disea

healthsystemsevidence.org
Sample Search - Topic
Sample Search - Type
Search Results
Review Links and Details

links to any freely available user-friendly summascientific abstracts, and full-text reports.

Review summaries


Full text p y olic brief available at 

www.mcmasterhealthforum.org
More Sample Scenarios 1

 Sample Scenario (1)

We want to 

p p rovide a 

patientcentered medical home model, 

e.g., improving access (particularly with the use of technology)
Results
More Sample Scenarios 2

 Sample Scenario (2) Al lh lh h i A local healt

h aut

hority 

seeks information about alternative healthcare 

delivery models for rural delivery models for rural

communities
Results
More Sample Scenarios 3

 Sample Scenario (3) A research-funding 

agency needs to identify agency needs to identify

what’s known about implementation strate

g pp ies to support 

national policy priorities
Results
Next Steps for HSE 

 Ongoing monthly updates New topic-specific bulletins  General evidence service Tutorials on content and use  Webinar series with the Canadian Cochrane C Podcasts of webinars 
Interactive Question 4

 Do 

y qy ou have other 

questions that you would use Hea

Systems Evidence to search for research evidence 
Interactive Question 5

 Given what you now know, are you likely to use Health Sy

E id v ence to fd h d h d b fin

d synthesize

d research evidence a

bout governa

financial and delivery arrangements within health systems and impgy lementation strategies within health systems ?

Yes

No 

Maybe 
Questions?

Contacts: Lori Greco Kno ledge Broker Lori Greco, Knowledge Broker

lgreco@mcmaster.ca

 Kaelan Moat, Doctoral Student

moatk @a mcmaster.ca