# Presentation: 486729

## SPF SIG Overview and Expectations

**New Grantee Workshop**

## Slide 2

## SPF SIG Goals

- Prevent the onset and reduce the progression of substance abuse, including childhood and underage drinking
- Reduce substance abuse-related problems in the communities
- Build prevention capacity and infrastructure at the State and community levels
- “SAMHSA envisions the SPF SIGs being implemented through partnerships between States and Communities”

## Key Principles of the SPF SIG

- Public Health Approach
- Outcomes Based Prevention
- Follows a Strategic Planning Process – SPF SIG
- Use Data throughout the process to inform decisions

## Public Health Approach

- The SPF SIG takes a ***public health approach*** to prevent substance related problems.
- A public health approach focuses on change for entire populations (collections of individuals who have one or more personal or environmental characteristic in common).
- Population-based public health considers an entire range of factors that determine health.

## Outcomes-based prevention

- Effective prevention is grounded in a solid understanding of*** ***alcohol tobacco and other drug*** consumption and consequence patterns ***that need to be addressed.
- Understanding the nature and extent of consumption (e.g., underage drinking) and consequences (e.g., motor-vehicle crashes) from the beginning is critical for determining prevention priorities and aligning strategies to address them.

## Outcomes-Based Prevention

**Intervening Variables**

**Strategies/**
**Programs**

**Planning, Monitoring, Evaluation and Replanning**

## SAMHSA’s Strategic Prevention Framework Steps

## Infusing Data throughout SPF SIG Decision Making

- SPF SIG strives to ***use data across all steps*** of the Strategic Prevention Framework (SPF).
- Deliberate process to collect, analyze, interpret, and apply lessons from substance use and consequence data to drive state efforts across the entire SPF.
- Ongoing and integrated data analyses are critical to:
- identify problems and set priorities,
- assess and mobilize capacity for using data,
- inform prevention planning and funding decisions,
- guide selection of strategies to address problems and goals,
- monitor key milestones and outcomes and adjust plans as needed.

## Characteristics of the SPF SIG

- Community Level Change: Community is Unit of Analysis
- Community Level efforts funded by the SPF SIG include all five steps of the SPF
- Capacity and infrastructure that can be sustained over time
- The SPF focuses on a “systematic process”, not just a funding and program implementation decisions
- Dynamic Process: requires grantees to ask questions and use information for decision making

## SPF SIG and SIG

- Similarities
- Differences:
- Addresses the entire lifespan and the population
- Encourages programs, policies, and practices
- Requires data-driven decision-making and facilitates development and utilization of monitoring and surveillance systems

## DHHS commitment thru SAMHSA to bring down underage drinking rates; target of close to $30 million in FY2004;
States are required to include the prevention of underage alcohol consumption and provide a comprehensive strategy that addresses Underage drinking priorities. . .Underage drinking must be included in all five steps of the Strategic Prevention Framework Expectations
States are required to report on underage drinking-related SPF SIG activities as a condition of the grant.
Lead agency for underage drinking is required to be part of SPF SIG Advisory Council.

- States are required to include the prevention of underage alcohol consumption and provide a comprehensive strategy that addresses Underage drinking priorities. . .Underage drinking must be included in all five steps of the Strategic Prevention Framework Expectations
- States are required to report on underage drinking-related SPF SIG activities as a condition of the grant.
- Lead agency for underage drinking is required to be part of SPF SIG Advisory Council.

- SPF SIG Underage Drinking Component

## State Epidemiological Workgroups

- States are required to establish and manage (or work with an existing) State Epidemiological Workgroup (SEW)
- RFA states SAMHSA’s expectation that an average of $200,000/yr be spent for SEW/needs assessment activities
- Term & Condition requires State grantee to confirm that the State will expend a minimum of $150,000 each year from grant or other funds for SEW activities
- RFA requires that the SEW function for duration of 5-year funding

## State Epidemiological WorkgroupsPurpose

- Bring systematic, analytical thinking to the causes and consequences of the use of substances in order to effectively and efficiently utilize prevention resources
- Promote data-driven decision making at all stages in the Strategic Prevention Framework
- Promote cross systems planning, implementation, and monitoring efforts
- Provide core support to the SPF Advisory Council

**Notes:**
Emphasis is on promoting data driven decision making

## SPF Advisory Council

**Required Membership**
- Office of the Governor
- Core Group—drug and alcohol related agencies
- Demand reduction coordinator from DEA
- Underage drinking lead agency
- SAMHSA/CSAP
**Encouraged Membership**
- Other State, community and nonprofit organizations
**Chair Appointed by Governor**

## SPF SIG Advisory Council (cont’d)

- SPF SIG Advisory Council Expectations:
- SPF SIG Meetings
- Connection to the Governor’s Office
- Community as well State perspective
- Diversity
- Focus on getting the work done
- Relationship to SPF SIG Staff
- Relationship to SPF SIG Epi Workgroup

## Expectations of SPF SIG Grantees

- CSAP expects that State grantees will:
- Maintain timely communication with their CSAP Project Officers
- Document their activities
- Seek State Project Officer feedback and guidance as needed on:
- Each step of the Five-Step SPF Process
- The State Strategic Plan
- Selection of effective strategies
- Evidence-Based Programs, Policies, and Practices

## Quarterly Reports

- Report format is under development (draft being distributed)
- Please review and provide comments
- Final report format will be provided by January 1st
- First Quarterly Report will be submitted by January 31st

## Expectations of CSAP Staff

- CSAP expects that the State Project Officers will:
- Maintain ongoing communication with State grant staff.
- Serve as a member of the Advisory Committee.
- Ensure the grantee’s collaboration with its State Epidemiological Workgroup.
- Participate in policy, steering, and other SPF SIG workgroups.
- Facilitate the provision of technical assistance as requested by State grantees or identified by the SPO.
- Ensure that quarterly reports are submitted by deadline.
- Review quarterly reports and provide appropriate feedback.

## Expectations of CSAP Staff (cont’d)

- CSAP expects that the State Project Officers will:
- Work closely with grantees on development of their Strategic Plans.
- Complete review of strategic plan within 3 weeks of receipt.
- Provide any feedback and comments necessary for approval of Strategic Plan. Review revisions upon receipt.
- Provide written approval of Strategic Plan to State.
- Review and approve the State sub-recipient funding mechanisms(s) within 3 weeks of receipt.
- Monitor the development and collection of outcome data.

## SPF SIG Cross-Site Evaluation

- SPF SIG New Grantee Meeting

## Cross-site Evaluation Team

- Westat
- Pacific Institute for Research and Evaluation (PIRE)
- MayaTech Corporation
- HMS Technologies

## Collaborative Model

- Recognition of multiple federal stakeholders whose priorities may differ
- Due consideration of program aspects critical to CSAP:
- strategic and data-driven planning,
- state-level system change,
- environmental change at all levels, and
- underage drinking in addition to illicit drug use
- Vision of grantees as full partners in design and implementation of the national evaluation, with continuing collaboration over life of the contract
- We (the cross-site evaluation) exist but for them.
- A critic silenced is not a critic persuaded.
- Relationships with state evaluators essential to:
- Allow for (and learn from) processes and evaluation activities that may be unique to each state.
- Maximize contributions from the states and their evaluators to the cross-site evaluation

## Collaborative Model (cont.)

- Potential areas for collaboration include:
- developing measures;
- identifying data sources;
- facilitating data collection;
- reviewing drafts of data collection instruments;
- interpreting the evaluation findings;
- disseminating information from the evaluation; and
- using the findings for project revisions and strategic planning.
- At the same time, we will need to:
- meet certain obligations (e.g., delivery of the OMB package within xx weeks EDOC.) and
- assert more directive leadership as needed to protect the integrity of the evaluation.

## General Design Framework-Logic Model

## General Design Framework - Features

- A rigorous, yet practical approach to evaluating processes and outcomes at state, community, and individual levels
- Due consideration of program aspects critical to CSAP
- Tailored program logic models and within-state analyses
- A vision of grantees as full partners in the design and implementation of the national evaluation
- Explicit consideration of program selection and implementation fidelity in interpreting community- and individual-level outcomes.

**Notes:**
grounded in lessons learned from our experience with the current SIG cross-site evaluation.
Discussed before

## General Design Framework - Features

- Standardization of data collection at all levels, with built-in reliability checks
- Tailored use of secondary data sources in combination with state logic models, site visits and other sources, to detect anticipated effects of each state’s SPF SIG processes on individual behavioral outcomes
- Leveraging relationships with SEWs to yield data that benefits the national evaluation.
- Use of states’ own SPF SIG evaluations to augment/aid interpretation of national evaluation data.

## General Design Framework - Features

- Use of natural variation and replications within and across states -- concurrently and over time – in tandem with SEW comparison states to triangulate on effect estimates at the community and individual levels.
- Multilevel modeling and meta-analytic methods to explain cross-site variation in state-, community-, and individual-level outcomes.

## Measurement Domains

- Systemic change at state and community levels
- SPF implementation and program fidelity at community level
- Changes in substance use and related outcomes at state and community levels

## Systemic Change at the State Level

- To determine the States’ baseline characteristics:
- Implementation of the strategic planning framework
- Changes in the state ATOD prevention system
- Changes in the state environment regarding ATOD use
- Use of evidence-based programs (EBP)

## Systemic Change at the State Level

- Site-visits to SFP/SIG states
- Initial Site-Visits (1st round)
- Instrumentation development –
- Initially an inductive process
- Benchmarks for an effective state-wide system
- Clarification of state-specific logic models (planned processes and goals)
- Clarification of evaluation expectations with state SPF/SIG staff and the SEWs
- Collaboration with state evaluators

## Systemic Change at the State Level

- Annual Site-Visits (2nd through 6th rounds)
- Expand to SEW-only states
- Qualitative data via interviews
- Coding and reliability
- Documenting environmental changes
- Tracking state system changes
- Quantitative data

## Systemic Change at the Community Level

- Collect base-line data for local systems
- Changes in systems at the regional, county or community level.
- Compare community-level changes across communities and across states
- Changes in environmental factors relevant to ATOD use
- Implementation of the strategic planning framework
- Use of evidence-based programs (EBP)

## Systemic Change at the Community Level

- Site-visits to randomly selected communities in SPF/SIG states (rounds 2-6) in conjunction with state site-visits.
- Modification of state-level site-visits
- Qualitative data
- Quantitative data via MayaTech and web-based data collection.

## Systemic Change at the Community Level

- Structured, community level forms
- Designed to collect information with which to evaluate community capacity building activities
- Will capture community level data related to improving organizational and community resources, infrastructure development activities, relationship building activities, contextual factors, & systemic and environmental factors.
- Will measure progress across the life of the grant, not just baseline
- Will include both quantitative and qualitative response options

## SPF Implementation and Program Fidelity at Community Level

- State Level Data Collection (site visits)
- Assess the extent to which EBPs are being implemented across all SPF SIG communities
- Assess mechanisms for promoting EBPs across the State
- Identification and delivery of EBPs
- Training
- Monitoring fidelity
- Community Level Data Collection (site visits and web-based)
- Assess the extent to which communities are delivering EBPs
- Assess the type of training and TA they receive
- Assess the extent to which they are monitoring program fidelity and how they are using such data
- Are adaptations culturally competent?

## Changes in Substance Use and Related Outcomes at State and Community Levels 

- Compare SPF and non-SPF states on state-level outcomes
- Using national data sources
- Using state-specific data sources (via SEWs)
- Compare SPF and non-SPF communities on community-level outcomes
- Using national data sources
- Using state-specific data sources (via SEWs)
- Selected subgroup analyses (state and community levels)

## Cultural Competence Foci

- State Level:
- Establish & Monitor Cultural Competence Policy Statewide
- Community:
- Implement Policy & Monitor Prevention Program Service Delivery & Evaluation
- Program Level:
- Deliver culturally Appropriate Prevention Services & Implement Culturally Tailored/Designed and Appropriate Evaluation

## Integration of Process and Outcome Analyses

- Building a State and Program-level Database to be merged with outcome data
- Standardized coding, ratings, composite scale development, reliability assessment
- Extending prior analysis strategy
- Currently being implemented in SIG
- Prior cross-site evaluations
- Relationships between site characteristics and outcomes
- Multi-level modeling
- Explanatory meta-analysis

## Preliminary Year One Timeline

- Instrument development
- Data collection
- Quarterly reports
- Site visits
- Late January to early April
- Two days
- Confer with:
- Projector directors
- SEW chair persons
- Evaluators.

## Preliminary Year One Timeline

- Planning
- Internal workgroups
- Grantee subcommittees
- Prior cross-site evaluations

## Lunch

## SPF SIG Step #1

- Conducting a State-wide Needs Assessment

## What is Needs Assessment?

- Assessment of substance use and related problems leading to recommendations regarding state and community priorities
- Assessment of capacities, resources, gaps, and readiness
- 5 year requirement for SPF SIG which includes ongoing monitoring/surveillance (cyclical part of SPF 5 Steps)

## SPF Step #1 Requirements Profile population needs, resources, and readiness to address the problems and gaps in service delivery

**State Requirements**
- Develop a State Epidemiological Workgroup.
- Conduct a statewide needs assessment using epidemiological data.
- The Assessment should include:
- A profile of consumption and related problems in the State
- Current capacity to implement the SPF
- Gaps in services and capacity;
- Readiness to act;
- Support the SPF implementation in communities.

## Components of a Needs Assessment

- Set the Stage: Define purpose, goals, procedures and timeframe
- Data Collection: Define data/data sources. Collect data.
- Data Analysis: Explore trends, patterns and baselines
- Integrate and Communicate Results: Develop Needs Assessment report and/or other documents, presentations, maps, etc. to communicate findings

## Setting the Stage

- Develop SPF SIG Advisory Group and  State Epidemiological Workgroup (SEW)
- Establish structural and procedural connections between Advisory Group and SEW
- Advisory Group and SEW collaborate to define purpose, goals and timeline for Needs Assessment

## Data Collection

- Identify set of state specific indicators of
- Substance use-related consequences
- Substance use patterns related to these consequences

## Substance Use Related Problems

|  | Tobacco | Alcohol | Illicit Drugs |
| --- | --- | --- | --- |
| Illness | Lung Cancer
Heart Disease
Lung Disease
Reproductive health         
Fetal effects | Cirrhosis
Cancer
Heart Disease
Fetal effects | Overdose
HIVFetal effects |
| Injury | Burns | MVA
Suicide
Homicide | MVA
Suicide |
| Other |  | Crime
Work/School | Crime
Work/School |

## Consumption

- Overall Consumption
- Acute, heavy consumption
- Consumption in risky situations
- Drinking and driving
- Consumption by high risk groups
- Youth, College Students
- Pregnant women

## Criteria for Selecting Indicators

- Reflect critical substance use related problem or consumption pattern
- Valid
- Periodic collection
- Consistent measurement
- Available/Limited burden
- Dissagregated geographically, by age, gender and/or race/ethnicity

## Tips

- Don’t go overboard!
- Limit the data to what you believe is essential.
- Use existing data.
- In 1st year, rely primarily on existing data.  Data needs may and should be identified but focus on completing initial assessment with existing data to see what you learn before developing new data collection mechanisms

## Data Analysis

- Levels
- Establish baseline
- Trends
- Over time
- Patterns
- By age, gender, race/ethnicity

## Integration and Communication

- Creating a State Profile:
- Description of substance use related problems
- across State and across time
- by age, gender, race/ethnicity
- Description of consumption patterns that contribute to these problems
- across State and across time
- by age, gender, race/ethnicity

## Needs Assessment Process and Outcomes

| Steps | Outcomes |
| --- | --- |
| Set Stage
 | Description of purpose, goals, general procedures and timelines |
| Data Collection | Identified set of State indicators and data sources |
| Data Analysis | Descriptions of baseline, trends, patterns in data |
| Integration and Communication | State Profile integrating findings and presenting cohesive report
 |

## Next Steps

- Develop and utilize a prioritization process for consumption patterns and problems
- Recommendations for Resource Allocation based on Needs Assessment Data
- Feed recommendations into Capacity Development (Step 2) and State Strategic Plan (Step 3)

## Key milestones in Step 1 include:

- Formation of an Epidemiological Workgroup
- Collaboration with advisory groups (SPF SIG Advisory Council)
- Collection of epidemiological data
- Analysis of epidemiological data
- Development of problem statements
- Identification of potential geographic target areas and populations
- Assessment of readiness, external factors, and potential barriers to success
- Assessment of organizational, fiscal, and leadership capacity
- Assessment of cultural competence
- Analysis of service gaps

## Key products for Step 1:

- Epidemiological Workgroup Report
- Clear, concise, and data-driven problem statement(s)
- Data sources for ongoing assessment
- Gap analysis and community program/resource/service baseline inventory

## SPF Step #1 RequirementsProfile population needs, resources, and readiness to address the problems and gaps in service delivery

**Community Requirements**
- Accurately assess substance-abuse related problems using epidemiological data
- Identify the magnitude of the (substance abuse) problem and where it is greatest
- Identify risk and protective factors
- Assess community assets and resources, gaps in services and capacity, and readiness to act.

## SPF SIG Step #2

- Building State and Community Prevention System Capacity

## Step 2.  Capacity

- Capacity involves:
- Mobilizing resources within a geographic area (state/community).
- Engaging key stakeholders, coalitions, and service providers to plan and implement sustainable prevention efforts in Steps 3-4 of the SPF.
- Developing cultural competence and building on the existing prevention infrastructure.
- Mobilizing both financial and organizational resources

## Step 2.  Capacity (cont’d)

- Capacity involves:
- The ongoing development of State and Community level capacity as part of each of the five steps of the SPF
- Creating partnerships.
- Developing readiness and leadership.
- Strengthening capacity through education and training.
- Focusing on sustainability and evaluation capacity.

## SPF Step #2 RequirementsMobilize and/or build capacity to address needs

- State Requirements
- Engage key stakeholders across the State
- Engage key stakeholders within the target communities that are selected for capacity building planning and implementation activities

## Key milestones of Step 2 include:

- Creation of sustainable partnerships
- Introduction of training and education to promote readiness, cultural competence, leadership, and data collection, monitoring, and evaluation capacity
- Meetings and workshops with key stakeholders, coalitions, and service providers

## Key outcomes for Step 2 :

- A State prevention system with the capacity to plan and address the needs identified in Step 1.
- The infrastructure to implement and sustain the SPF.

## SPF Step #2 RequirementsMobilize and/or build capacity to address needs

- Community Requirements
- Engage key stakeholders at the State and community level to plan and implement successful prevention activities that will be sustained over time
- Key tasks may include:
- Convening community leaders and stakeholders
- Building coalitions
- Training stakeholders, coalitions, and service providers
- Organizing agency networks
- Leveraging resources

## SPF SIG Step #3

- Developing a Comprehensive Statewide
- Strategic Plan

## Step #3 Developing a Comprehensive Strategic Plan

- Planning involves:
- Developing a comprehensive, logical, and data driven plan to address the problems identified in Step 1 with the current and future capacity developed and identified in Step 2 of the SPF.
- The planning process produces Strategic Goals, Objectives, and Performance Targets as well as Logic Models and in some cases Action Plans.  In addition to the Strategic Goals, Objectives, and Performance Targets, as well as Logic Models.

## Step #3 Developing a Comprehensive Strategic Plan (cont’d)

- In addition to the Strategic Goals, Objectives, and Performance Targets, the Strategic Plan lays the groundwork for:
- The State’s Implementation activities, including:
- Capacity Expansion
- Training
- Support for Community SPF processes
- Development of monitoring and evaluation systems
- The identification of strategies
- The selection of evidence based programs, policies, and practices

## SPF Step #3 Requirements Develop a Comprehensive Strategic Plan

- State Requirements:
- Develop a State Strategic Plan that uses statewide needs assessment data to:
- Articulate a vision
- Identify priorities
- Identify key milestones and outcomes
- Involve private and public service systems
- Identify required training
- Identify appropriate funding mechanisms to allocate resources to target communities
- Sustain infrastructure

## SPF Step #3 Requirements (continued)

- Describe how your State will address:
- Necessary infrastructure development
- Underage Drinking Initiative
- The State Strategic Plan must be data-driven and focused on addressing the most critical needs in the State.
**The State Strategic Plan must be approved by the SAMHSA/CSAP Project Officer before implementation activities can begin.**

## Strategic Plan Expectations

- The State Strategic Plan should:
- Summarize needs, identify contextual conditions, and discuss Steps 1-3.
- Include the identification of target populations/communities, but should not go as far as to specify individual programs, policies, and/or practices that an individual community may implement based on its SPF process.
- Address cultural competency

## Strategic Plan Expectations (cont’d)

- Grantee Strategic Plans should also discuss:
- The role of stakeholders in the development of the plan
- Data used for decision making
- Potential barriers to Implementation
- How the proposed SPF SIG activities fit with other State activities

## Getting Started

- Gather background information including
- Previous strategic and operational plans
- RFP’s, work plans, annual reports
- Budget requests, legislation
- Current metrics/performance measures
- Past evaluations, evaluations of similar programs
- Interview managers and staff involved with the program

## Key milestones of Step 3 include:

- Planning meetings and strategy development sessions
- Strategic Goals, Objectives, and Performance Targets
- Logic Models
- Draft Strategic Plan
- Creation of State data collection, monitoring, and evaluation plan and identification of measures

## Key interim products may include:

- Draft Strategic Plan
- Logic Models
- Action Plans
- Performance Outcomes
- Data collection, monitoring/surveillance, and evaluation plan

## SPF Step #3 RequirementsDevelop a Comprehensive Strategic Plan

- Community Requirements:
- Develop a strategic plan that articulates a vision for prevention activities and organizing and implementing prevention efforts based on:
- Documented needs
- Identified resources and strengths
- Measurable objectives and performance measures
- Baseline data
- Adjust plans as the result of ongoing needs assessment and monitoring
- Create a long-term strategy to sustain policies, programs, and practices

## SPF SIG Step #4

- Implementing evidence-based prevention programs and infrastructure development activities

## Step 4.  Implementation

- Implementation involves:
- Taking action guided by the Strategic Plan created in Step 3 of the SPF.

## SPF Step #4 Requirements Implement infrastructure development activities

- State Requirements:
- Build capacity to provide the infrastructure and other necessary support for State and local level SPF process
- Needs assessment
- Capacity building
- Strategic Planning
- Implementation
- Monitoring and Evaluation

## SPF Step #4 RequirementsImplement evidence-based prevention programs and infrastructure development activities

- Community Requirements:
- Use the findings of needs assessments to guide selection and implementation of evidence-based policies, programs, and practices
- Strategies must be evidence based
- Federal Model Program List
- Peer reviewed journal with proven effectiveness
- Documented effectiveness

## Outcomes-Based Prevention

**Intervening Variables**

**Strategies/**
**Programs**

**Planning, Monitoring, Evaluation and Replanning**

**Notes:**
The project is organized around outcomes-based prevention principles

This perspective identifies 
-what you are trying to affect- outcomes
What you do to affect it -strategies
What the intervening mechanisms are that you might change-intervening variables 

## Substance-Related Problems (Examples)

|   | TOBACCO | ALCOHOL |  ILLICIT DRUG |
| --- | --- | --- | --- |
| Illness | Lung Cancer
Heart Disease
Lung Disease
Reproductive            health
Fetal effects | Cirrhosis
Cancer
Heart Disease
Fetal effects | Overdose
HIV
Fetal Effects |
| Injury | Burns | MVA
Suicide
Homicide | MVA
Suicide |
| Other |   | Crime
Work/School | Crime
Work/School |

## Substance Use 

- Overall Consumption
- Acute, heavy consumption
- Consumption in risky situations
- Drinking and driving
- Bars
- Consumption by high risk groups
- youth

## Causal Factors (Examples)

- Community Level Factors
- Availability of substances
- Promotion of substances
- Social Norms regarding use
- Enforcement of Policies and Social Norms
- Individual Level Factors
- Perceptions of risk
- Perceptions of harm

## Choosing Strategies 

- Must address the problem identified
- Must address the causal factor(s) believed to be involved

## Strategy matches Problem and Cause

**Alcohol easily available in bars**
**Little enforcement of drinking driving laws **

**No service to intoxicated patrons**
**Sobriety checkpoints**

## Strategy does not match Cause or Problem

**Alcohol easily available in bars**
**Little enforcement of drinking driving laws **

**Middle school  curriculum **
**Mentoring Program**

- Strategy does not match Cause or Problem

## Comprehensive Analysis for Planning

- Substance Use

- Causal Factors

- Strategies

**High Rate of Alcohol-Related Crashes **

**High Rates of Binge Drinking**

**High Rates of Drinking and Driving**

**Low Perceived Risk of Alcohol Use**

**Social Norms Encouraging  Binge Drinking**

**Little Enforcement of Drinking and Driving**

**Curriculum to Increase Knowledge about Risk**

**Media campaign to correct perceptions of ‘normal’ consumption**

**Checkpoints**

- Comprehensive Analysis for Planning

## Comprehensive Analysis for Planning

- Substance Use

- Causal Factors

- Strategies

**High Rate of Alcohol-Related Crashes **

**High Rates of Drinking and Driving**

**Little Enforcement of Drinking and Driving**

**Checkpoints**

- Comprehensive Analysis for Planning

## Who does what (Possible Steps)

- Needs Assessment
- State identifies problem
- Community further defines problem (State TA)
- Problem Analysis
- State develops framework for identifying and analyzing causal factors
- Community and State analyze causal factors
- State provides TA on analyzing causal factors
- Strategy Identification
- State provides TA on choosing strategies (possibly providing sets of options or criteria for choosing strategies)
- Community chooses strategies

## Timelines

- Strategy identification starts after State level needs assessment and resource allocation and
- After State and Community level causal factor analysis

## SPF SIG Step #5

- Monitor process, evaluate effectiveness, sustain effective programs/activities, and improve or replace those that fail

## General Themes in re SPF-SIG Data

- Data processes are at the Beginning (NA) and End (Eval), and throughout the 5 Steps
- Data serve Monitoring and Management purposes
- For Administration of Grant Progress
- For Surveillance of the Substance Abuse Problem
- There are required National Outcomes, but there is flexibility in State level data gathering and reporting

## Cross-Over of Evaluation and the Other SPF Steps: Needs Assessment

- What is the link between the Epi process and the Evaluation data to be collected and reported?
- What data were collected and how were the data analyzed?
- What trends do the data suggest and how are these trends considered in planning?
- Are data collected and reviewed on an ongoing basis, and how is the epi workgroup convened for this purpose?

## Cross-Over of Evaluation and the Other SPF Steps:  Capacity Building

- What are the identified resources and resource needs in the state?
- What data were collected and how were the data analyzed to determine this?
- What efforts are undertaken to mobilize and build capacity? (training?  staffing?  coordination?)

## Cross-Over of Evaluation and the Other SPF Steps:  Strategic Planning

- How well does the planning process reflect the identified needs and resources?
- What is the role of the State Advisory Council in the Planning process?
- How are issues discussed/decisions made?
- Are the right parties involved and do they “ante up” as appropriate?
- How is the implementation plan expressed and going to be managed?

## Cross-Over of Evaluation and the Other SPF Steps:  Implementation

- Does the  implementation match the plan?
- What is implemented – what programs, strategies, activities and by whom?
- What can we say about “ implementation fidelity” – how is the implementation to be tracked?
- What changes are made along the way?

## SPF Step #5 RequirementsMonitor process, evaluate effectiveness, sustain effective programs/activities, and improve or replace those that fail

- State Requirements:
- Provide ongoing monitoring and evaluation of all SPF SIG activities
- Provide training and technical assistance to communities regarding evaluation and performance measurement
- Assess program effectiveness

## SPF Step #5 Requirements(Cont’d)

- State Requirements:
- Identify successes
- Encourage needed improvement
- Promote sustainability of effective policies, programs, and practices
- Adjust implementation plans based on monitoring/evaluation activities

## State Evaluation Expectations

- Coordinate data collection as much as possible with Epi Workgroup
- Collect and report data on SAMHSA’s National Outcome Measures at all relevant levels – lots more to come on this
- Do a good state-level evaluation
- Provide quarterly reports, including evaluation information
- Participate in cross-site evaluation, including site visits and providing data to CSAP

## Key milestones of Step 5 include:

- Consultation and collaboration with evaluation team
- Process evaluation
- Collection of required outcome data
- Outcome evaluation
- Review of policy, program, and practice effectiveness
- Development of recommendations for quality improvement

## Key products for Step 5 include:

- Evaluation Report and updates
- Recommendations for quality improvement

## SPF Step #5 RequirementsMonitor process, evaluate effectiveness, sustain effective programs/activities, and improve or replace those that fail

- Community Requirements:
- Monitor and evaluate activities
- Provide performance data to the State

## Now, to the heart of the matter!

- SAMHSA’s National Outcomes:
- What are the measures?
- What are the implications of the National Outcomes for States and Communities?
- SEE HANDOUTS

## Questions we hope to answer in the context of the National Outcome Measures

- What’s up with GPRA?
- What’s up with the CSAP Core Measures?
- What are the requirements in re:
- Community Strategies?
- Program Participant level data gathering?
- Is the list final?
- When will I have to report on the NOMs for the Block Grant?

## Cultural Competence, Sustainability, and the SPF SIG Process

- SPF SIG New Grantees Meeting

## The Need for Cultural Competence in the SPF SIG Program

- Why is there a compelling need for cultural competence?
- To respond to current and projected demographic changes in the US.
- To eliminate long standing disparities in the quality  of prevention services to people of diverse racial, ethnic, and linguistic populations; and consideration of gender, disabilities, and sexual orientation in delivery of prevention services.
- To improve the effectiveness, quality of services, and positive prevention outcomes to targeted populations.

## The Need for Cultural Competence in the SPF SIG Program (continued)

- Why is there a compelling need for cultural competence?
- To meet DHHS, SAMHSA and CSAP requirements.
- To add to the body of knowledge, awareness and skill in the design, implementation, and delivery of  prevention programs, policies, and practices.
- To promulgate the philosophy that an effective program is a culturally competent program across the board, i.e., agency, policies, staff, and programs/practices.

## Cultural Competence: Two Definitions

- A set of academic & interpersonal skills that allow individuals to increase their understanding & appreciation of cultural differences & similarities within, among & between groups.  This requires a willingness & ability to draw on community-based values, traditions, & customs & to work with knowledgeable persons of & from the community in developing focused interventions, communications, & other supports.  Orlandi et.al.,(1992)
- ...the attainment of knowledge, skills & attitudes to enable administrators & practitioners within systems of care to provide for diverse populations.  This includes an understanding of that group’s or members language, beliefs, norms and values, as well as socioeconomic & political factors that may have a significant impact on their well-being, & incorporating those variables into assessment & treatment. CSAP, (1993)

## Cultural Competence and the SPG SIG  Program

- Cultural competence is a critical component of the SPF SIG program. CSAP has methodically and systemically addressed the issue of cultural competence in its publications, programs, training and technical assistance and policy statements for over a decade.   Today CSAP and SAMHSA routinely inform applicants in its documents such as the SPF SIG SP 04-002 Initial Announcement of its expectations regarding cultural competence

## Cultural Competence Foci

- State Level:
- Establish and Monitor Cultural Competence as it relates to the steps of the SPF
- Community:
- Implement Policy and Monitor Prevention Program Service Delivery and Evaluation
- Program Level:
- Deliver Culturally Appropriate Prevention Programs, Policies, and Practices and Implement Culturally Tailored/Designed and Appropriate Evaluation

## SPF-SIG Cultural Competence Requirements 

- Build a state & program level data base consisting of the following descriptive information:
- Types of plans and strategies designed to implement culturally appropriate policies, programs & practices;
- Organizational capability & experience of awardees in implementing culturally appropriate/competent prevention interventions;
- Utilization, reliability & validity of culturally appropriate psychometric methods and measures;
- Types and utilization of culturally appropriate adaptations with evidence based programs, policies, and practices.

## Potential Sources/Domains Where Cultural Competence May Be Found

- Organizational Values
- Governance
- Planning and Monitoring/Evaluation
- Communication
- Staff Development
- Organizational Infrastructure
- Services/Interventions
- (Lewin Group, et. al., 2002)