# Presentation: 150908

## Raising the Quality Bar on Public Health Course Content

*Stacy Baker, MSEd, Public Health Foundation*
*Leanne Jeffers, MPH, Regional Institute for Health and Environmental Leadership*
*Karen Ngowe, MA, Michigan Department of Community Health*

## Raising the Quality Bar

*At issue:*
- Over 1,200 courses
- High stakes for poor choices and training
- How can people identify what is quality?
- How can we raise the quality of offerings?

**Notes:**
Start with...a survey (surprise!)
What can we do that might help? 

## How other professions address C.E. quality

**Usually in context of licensure, certification, and C.E. requirements** *Examples:*
- Physicians, nurses
- Accountants
- Architects
- Challenges for public health

**Notes:**
Opportunities to Rate or Raise the Quality of Courses –
Creates opportunities to influence or review the provider quality, review course in context of awarding CE credits
Physicians:
CME required to maintain licensure
Opp - Review course for awarding CME to ensure it meets minimal standards
Opp - Review and accredit education provider for meeting standards. Ex: ACCME (Accreditation Council for Continuing Medical Education, ACCME) Essential Areas for organizational assessment – Written CME mission; Educational Planning & Evaluation - Uses needs assessment data, Evaluates effectiveness of programs/improve; Effective Administration - business/management practices....   [Note limited evidence that the accreditation at organization level is a good way to improve or rate the effectiveness of CME. – Leist et al 2004]

Nurses:
American Nurses Credentialing Center – Subsidiary of ANA - Accredits educational providers for nursing CE, approvers, and programs. Standards for Professional Development – voluntary (fee - $3,600 – 6,600)
Architects – 
Membership in Prof. Association – (American Institute of Architects) requires Learning Units
30 states require CE units for re-licensure.
Opp: AIA/Continuing Education Systems group offers “Guidelines for Educational Excellence” addressing org’s commitment to quality programming. Maintains a network of >2500 registered CE providers.  AIA offers provider training workshops, program development assistance, and annual Award for Excellence to providers.
Course level – Approves courses for CE, typically accepted by multiple states. Association also monitors quality through reports filed by members, SMEs, providers, AIA will audit programs at request of any licensing board to determine how well it meets learning objectives.
Accountants
CPE Standards for CPAs.  Courses approved for CPE must meet standards related to piloting, learning objectives/methods, instructor qualifications, examinations to show satisfactory completion (for self-study), mechanism for participant assessment of whether objectives met.

Many professional entities put out model rules to aid states with continuing education requirements and standards related to providers or the award of CE

CHALLENGES FOR PH WORKFORCE NOT LICENCED OR CREDENTIALED 
No model found of blue ribbon panel that reviewed everything

How to figure out what’s effective and feasible for us...A survey (surprise?)
Colorado, PHF, and TRAIN – a marriage of interests

## Survey methods

- Online survey open to the 22 affiliatesFifteen respondents (68%) – 10 complete; 5 partial
- Questions addressed:
- Criteria and processes affiliates use to approve course providers and courses
- Effectiveness and feasibility of potential strategies for improving course quality

**Notes:**
Primary TRAIN site administrators were the state respondents invited

Second bullet is focus for today’s presentation

## Survey findings

**Strategy Effectiveness and Feasibility**(See report, page 5)
- Only four strategies considered most effective AND most feasible by over 35% of respondents
- Only _one_ strategy highly recommended by the majority of respondents

**Notes:**
Pages 5-6
Limited agreement about the most promising quality assurance approaches

## Survey findings

**Top Strategies and Surprises**
- (#1) Encourage providers to use existing accreditation options (CME, CNE, etc.) to demonstrate quality of material
- Raise standards for course providers to be approved to list courses on TRAIN

**Notes:**
ASTD=American society for Training and Development

## Survey findings

**Top Strategies and Surprises** (continued)
- Encourage providers to use existing instructional design standards (e.g., ASTD)
- Develop a mechanism for affiliates to request inspection copies
- 3rd Party Review not viewed as a particularly feasible option

## Survey findings

**Top Concerns about QA Processes**
- Staff & resource limitations
- Discouraging course providers
- Delays in posting courses
- Duplicating review efforts

**Notes:**
Staff and resource limitations to support a more rigorous review
Potential to discourage course providers from participating
Potential time delays in posting courses to the system
Duplication of effort by multiple affiliates reviewing the same courses

## Survey recommendations

- Informed by these survey findings, TRAIN affiliates endorsed the following seven recommendations from the Learning Content Workgroup:

**Notes:**
See p8 for list of recommendations. Please refer to the "Recommendations—Discussion" section (p. 9) for a discussion of each recommendation and proposed action steps.

## Survey recommendations

**Actively encourage more course providers to use existing accreditation options (such as CME, CNE, CEU, CLE, and CHES)** for their continuing education programs targeting professionals who protect the public’s health.

## Survey recommendations

**Encourage more providers to use existing instructional design standards** set by organizations such as American Society for Training and Development (ASTD).

## Survey recommendations

- Explore potential mechanisms for affiliates or managers to request inspection copies of learning materials, guest user access, or the right to “try before you buy” through TRAIN.

## Survey recommendations

- Facilitate the sharing of lessons learned among TRAIN affiliates that are piloting higher standards for course provider or course approval, and reconsider in 2006 whether system-wide standards changes are warranted.

## Survey recommendations

- Affiliates should use and promote the existing "Accreditation" field on course listings to indicate approval by any review bodies.

## Survey recommendations

**Identify potential options to increase the feasibility of using third party review panels** to rate and raise the quality of courses, and then reassess interests among affiliates, course providers, and learners.

## Survey recommendations

**Federal and national organizations should explore the creation of a "Public Health CEU" accreditation type** that would be voluntary for course providers and tied to certification, licensure, or other incentives for public health workers.

## Work group poll 

- TRAIN Learning Content Work Group incorporated recommendations into its action plans
- Combined with other work group interests (policy & funding strategies)
- Polled affiliates again to identify volunteers and priorities for the _work group_ this year

## Work group action plans 

**Top 3 Priority Areas for Action:**
- ID and Promote Existing Quality Standards
- Advocate for “National Public Health CE/CEU”
- Encourage Funding & Policy Strategies
**Opportunities for Involvement**
- We Want YOU!

## Questions?  Comments?

## Slide 20

**Notes:**
I love www.despair.com  Good slide for question break? Goes with the “we want your help” theme 

## Small group discussion

- These are Public Health training issues, NOT limited to TRAIN Affiliates
- We need input from all stakeholders
- Let’s start with the top 3 priority items

## Small groups:

- Instructional Design Standards
- “National Public Health CE/CEU”
- Funding & Policy Strategies

## Small group directions

- Move to one of three groups
- Questions, handouts at each table
- Process (Facilitation; flip charts, need to record and report highlights)
- 20 min. for discussion + 5 min. each report
- All recorded ideas will be posted online

## Discussion Questions

- 1.	What opportunities already exist that could forward activities and efforts in this area?
- 2.	What barriers will need to be considered and addressed in this area?
- 3. 	What actions can individuals (you) take now to begin to address course quality at the local, state and/or national levels?

## Breakout reports

- Instructional design standards group
- “Public health CEU” group
- Funding and policy strategies group

## Wrap up

- Opportunities for involvement in TRAIN Learning Content Work Group
- Other action steps