# Presentation: 026006

## Health Services Research Doctoral Training Competencies

**Christopher B. Forrest, MD, PhD**
**Johns Hopkins Bloomberg School of Public Health**
**Erin Holve, MPH, MPP**
**Johns Hopkins Bloomberg School of Public Health**
**Diane Martin, PhD**
**University of Washington School of Public Health**

- Funding: Health Services Research Doctoral Training Competencies Conference sponsored by AHRQ R13 HS016070-01, Baltimore, MD, Sept 8-9, 2005

## Content

- Context & Statement of Problem
- Methods
- Core Competencies
- Discussion and Next Steps

## Context

- HSR is a scientific field with an emerging professional identify.  But,
- No common approach to educating HSR professionals.
- No common set of skills and knowledge bases.
- We need to elucidate the knowledge base and skills needed to be a successful health services researcher.
- Why?
- To aid in defining the field.
- To communicate to employers and students the unique expertise of health services researchers.
- To develop similar competence among graduates from various programs.
- Other reasons (curricula development, continuing education, etc.).

## Goal of the Competency Project

- To develop HSR core competencies:
- A minimum set
- Common to all HSR professionals
- That most programs could implement

## HSR Core Competency

*A knowledge-based or skill-based asset that all doctoral trainees in Health Services Research should acquire during their training.  Core competencies are the common denominator across all training programs. They are the skills and knowledge sets that all graduates of HSR training programs should have when they finish their doctoral training.*

## Example of Competency Levels: Health Disparities

**Awareness**
**Describe health disparities and their origins.**
**(2)** **Critique and Explain**
- 	**Critique alternative explanations for the  origins of disparities.**
**(3)** **Integrate and Innovate**
- 	**Propose health system solutions to health disparities.**

## Methods Used to Develop Core Competencies

## Initial Workplan and Timeline

**Council on Education for Public Health (CEPH) Self-Studies (n=27 schools)**

**Job Postings**
** ****(n=320)**

**T32 Training Grants (n=14)**

**Degree and Title Review from AH Membership File        (n=3,623)**

**Continuing Ed. Options for HSR Professionals**

**Sample of Universities with HSR PhD Programs. Review for major categories, themes**

**Professional identities according to AH members. Distribution of degrees held within major categories:  HSR, policy analyst, physician, etc.**

**Summary of course offerings and objectives with emphasis on highly requested skills and topics**

**Review of member survey conducted by Academy Health with attention to professional development needs**

**Summary of Academy Health Member Survey (n= 1,793)**

**Literature Review**

**August 2004**

**October/November/December  2004**

**January 2005**

**July 2004**

**Plan for Structured Analysis of Skills/**
**Behaviors/ Knowledge Bases / Competencies **

**	**

**Review and revise occupational categories and classification of skills.**

**Summarize**
**documents**

## Literature Review 

- Followed Cochrane Collaborative protocol, reviewing peer and non-peer reviewed literature
- Major keywords “competencies” “health services”, “health policy” and “health systems administration”
- Most material focused on objectives for management training rather than research
**No specific literature on health services or health policy competencies**

## T32 Training Programs

- Reviewed 14 AHRQ-Sponsored Institutional Training Grant Programs in HSR (T32)
- Learning objectives for doctoral training are ***not a requirement*** of NIH or AHRQ-funded training programs.
*** ******No grant applications contained learning objectives for doctoral training or sufficient information to infer learning objectives or training competencies***

## Competency Review: Taxonomy of Learning Objectives

- Interobserver Agreement = 93%
- Disagreement arbitrated by PI

**CEPH-Accreditation Self-studies** section V.C. “Learning Objectives
-  (n=21 HSR PhD Programs)

- Observer 1:
- Learning Objectives entered into **MS Excel**

- Learning Objectives transferred to **Atlas ti 5.0** (text analysis software)
- (n=650)

- Observer 2:
- Learning Objectives entered into **MS Excel**

- PI Reviews full set of competencies and summarizes into domains and core competencies: **Develop**
**Initial Taxonomy**

- Revised Taxonomy Based on Work Group Comments (Current v.3)

- JHU and U Wash Faculty Review and Comment on Taxonomy
- (v2)

- Final Taxonomy Based on Meeting

## Health Services Research Doctoral Training Competencies Conference 

**Objectives**:
- To provide a forum for exploring the possibility of developing core competencies for the field;
- To produce a consensus-derived set of knowledge-based and skills-based HSR doctoral training competencies and their content areas.

## Organizations Attending the HSR Competency Conference

- AcademyHealth
- CEPH
- AHRQ
- BC/BS National Association
- Boston University
- CMS
- Harvard University
- Johns Hopkins University
- Urban Health Institute

- Mathematica
- Texas A&M
- University of Alabama
- University of CA-Berkley
- University of Michigan
- University of Minnesota
- University of NC-Chapel Hill
- University of Washington
- Students from 4 programs

## Core Competencies

## HSR Core Competencies: Summary

***Breadth of**** HSR theoretical and ****conceptual knowledge***
***In-depth disciplinary knowledge**** and skills*
***Application of HSR foundational knowledge**** to health policy problems*
***Pose innovative HSR questions***
*Interventional and Observational ****Study Designs***
***Primary data collection methods***
***Secondary data**** acquisition methods*

*Conceptual ****models and ****operational ****measures***
*Implementation of ****research protocols***
* ****Responsible conduct of research***
*** ******Multi-disciplinary teamwork***
* ****Data analysis***
* ****Scientific communication***
*** ******Stakeholder collaboration and knowledge translation***

## (1) Breadth of HSR theoretical and conceptual knowledge 

- Know how to apply alternative theoretical and conceptual models from a range of relevant disciplines to HSR.

_**Comments**_
**1.  All health services researchers should be versed in a wide range of **
**theoretical and conceptual frameworks.**
**The work group felt that programs could define the range of disciplines**
**and fields.  As a field, we have not yet defined these.**

## (2) In-depth disciplinary knowledge and skills 

- Apply in-depth disciplinary knowledge and skills relevant to health services research.

_**Comments**_
**Each HSR professional should have expertise in at least one discipline**
**or interdisciplinary area, and should be able to apply this expertise in an **
**independent research project.**
**The work group left unresolved the question of whether HSR itself is a**
**field or discipline.**

## (3) Application of HSR foundational knowledge to health policy problems 

- Use knowledge of the structures, performance, quality, policy, and environmental context of health and health care to formulate solutions for health policy problems.

_**Comments**_
**HSR has a common foundational knowledge.**
**2.  HSR professionals should be able to synthesize this knowledge to**
**create novel solutions to vexing health policy problems.**

## (4) Pose Innovative HSR Questions

- Pose innovative and important research questions, informed by systematic reviews of the literature, stakeholder needs, and relevant theoretical and conceptual models.

_**Comments**_
**Literature review techniques are a competency.**
**2.  Knowing how to include stakeholders into research question development**
**is a critically important skill.**

## (5) Interventional and Observational Study Designs 

- Select appropriate interventional (experimental and quasi-experimental) or observational (qualitative, quantitative, or mixed methods) study designs to address specific health services research questions.

_**Comments**_
**Knowledge of qualitative and mixed methods research is an important skill.**
**2.  Quasi-experimental designs distinguish HSR from other quantitative fields**
**in public health, such as epidemiology.**

## (6) Primary data collection methods 

- Know how to collect primary health and health care data obtained by survey, qualitative, or mixed methods.

_**Comments**_
**The level of competence in primary data collection was a large point of**
**discussion during the competency conference.  We agreed on an intermediate**
**level for both primary and secondary data analysis; however, there was a **
**minority opinion that simply “knowing about” primary data collection may be**
**adequate for some HSRers (e.g., health economists who conduct secondary**
**data analysis exclusively).**

## (7) Secondary data acquisition methods 

- Know how to assemble secondary data from existing public and private sources.

_**Comments**_
**Manipulation of existing data is a strength of HSR.  The types of data-sets**
**that HSRers have expertise in is a distinguishing characteristic of the field.**
**There was less contention on the importance of secondary data**
**acquistion than there was for primary data collection.**

## (8) Conceptual models and operational measures 

- Use a conceptual model to specify study constructs for a health services research question and develop variables that reliably and validly measure these constructs.

_**Comments**_
**This competency was not contentious and is relevant for many fields of **
**social science research.**

## (9) Implementation of research protocols 

- Implement research protocols with standardized procedures that ensure reproducibility of the science.

_**Comments**_
**Learning how to create “electronic research records” was seen as an**
**important new competency for future HSR professionals.**

## (10) Responsible conduct of research 

- Ensure the ethical and responsible conduct of research in the design, implementation, and dissemination of health services research.

_**Comments**_
**The competency should address both existing and newly created**
**data sources.**

## (11) Multi-disciplinary teamwork 

- Work collaboratively in multi-disciplinary teams.

_**Comments**_
**Although HSR is characterized by multi-disciplinary teams and most**
**thought that this competency was important, there was a sense that the**
**roadmap for how to teach it was unclear.**

## (12) Data analysis 

- Use appropriate analytical methods to clarify associations between variables and to delineate causal inferences.

_**Comments**_
**The specific techniques could be statistical, economic, or other types of**
**analysis.  The competency intentionally does not specify which set.**

## (13) Scientific communication 

- Effectively communicate the findings and implications of health services research through multiple modalities to technical and lay audiences.

_**Comments**_
**Communication refers to both written and oral formats.**

## (14) Stakeholder collaboration and knowledge translation 

- Understand the importance of collaborating with policymakers, organizations, and communities to plan, conduct, and translate health services research into policy and practice.

_**Comments**_
**The importance of including stakeholders is emphasized in this**
**TRIP competency.**

## Essential HSR Educational Domains for the Core Competencies

- Health
- Financing of health care
- Organization of health care
- Health policy
- Health care seeking, access and use
- Quality of care
- Data acquisition and quality control
- Research ethics
- Teamwork
- Proposal development
- Dissemination

- Health informatics
- Scientific method and theory
- Literature review
- Study design
- Survey research
- Qualitative research
- HSR data sources
- Measurement and variables
- Advanced HSR analytic techniques
- Economic evaluation and decision sciences

## Optional HSR Educational Domains for the Core Competencies

- Health policy analysis
- Health care law
- Ethics and health care
- Quality improvement applications

- Health care decision-making
- Project management
- Professional development
- Pedagogy

##   HSR Doctoral Education Next Steps

- Diane Martin, PhD
- University of Washington

## Continuing the Dialog

- Presentations
- NRSA Trainees Conference, June 24
- Academy Health, June 26
- APHA, November 6
- Journal article submission to HSR

## Program Discussions

- Discuss among faculty
- Discuss with students
- Draft competencies or modify existing competencies
- Map competencies to courses or experiential learning

## Program Considerations

- Program faculty interests and expertise
- Admissions: bachelor vs master
- Work independently & in multidisciplinary teams
- Traditional dissertation vs journal articles
- Average time to completion; # yrs coursework

## HSR Doctoral Education

**Past Experience & Training**

**Doctoral Courses & Seminars**
**General requirements**
**Specific Concentration**
**Electives**

**Required Exams & Other Benchmarks**

**Research & Policy Experience**
- RA, independent study – multi-disciplinary team
- Dissertation – independent leadership

**Successful HSR Graduate**

**Mentorship**
- Faculty
- Research & Policy Partners
- Peers

**Professional Development & Socialization into HSR**

- Teaching Experience

## Delivery of Competencies

- Deliver competencies via didactic courses, seminars, indep. study, field work, dissertation project
- Competencies delivered in short courses by experts
- Doctoral e-learning?

## Assessment

- Course grades, field work assessments
- Program exams: qualifying exams; comprehensive exams; defense of dissertation proposal; dissertation defense
- Faculty and university review
- Alumni jobs, grants, publications
- Employer feedback about alumni

## Promising Practices 

- Innovative financial support
- Shared curriculum across universities
- Student experiences: mentoring, prof. development, teamwork, field exp.
- Partnerships: industry, gov’t., HSR orgs., other academic institutions, community groups