Table 1. Summary statistics of key baseline measures for youth completing [Survey Name] as of [Time Stamp] Intervention Comparison Intervention versus comparison Baseline measure Mean or % (standard deviation)a Mean or % (standard deviation) a Mean difference p-value of difference Age or grade level Gender (female) Race/ethnicity American Indian or Alaska Native Asian Native Hawaiian or Pacific Islander Black or African American White More than one race Unknown or Not Reported Behavioral measure 1 Behavioral measure 2 Sample size notes: [Describe the analytic procedure used to test the intervention-comparison group difference in baseline means – e.g., a t-test or a HLM analysis.] a Include standard deviation results in parentheses if a non-dichotomous measures (e.g., for age or grade level, or a count of sexual partners in the past 3 months).
EXHIBIT F: Implementation Analysis Plan Template (Tier 2B Grantees Only) Example data and methods that may be used to assess implementation elements Implementation element Types of data used to assess whether the element of the intervention was implemented as intended Methods used to operationalize and describe each implementation element (variable or data to be reported is shown in italics) Adherence How much of the program (and each of its components) was offered—for example, the number of sessions delivered, average frequency, and number of visits held Length (number of minutes) of program sessions that were captured in the data system Number of text messages sent that were recorded from the messenger service Average session duration, calculated as the average of the observed session lengths (measured in minutes) Average number of text messages sent across all individuals in the treatment What content was delivered to the youths—for example, the total number of topics covered and the proportion of material that was received Number of topics covered, as measured through a fidelity tracking log or protocol List of sessions recorded in attendance logs Total number of topics covered, calculated as the average of the total number of topics checked by each health educator in the daily fidelity tracking log or protocol Who delivered the material to the youths—for example, the number and type of staff delivering the program to participants, their position requirements or qualifications, and the percentage of staff trained and receiving ongoing support List of staff members hired Training logs Records on facilitators for each session Percentage of staff trained, calculated as the number of staff who were trained divided by the total number of staff who delivered the program Dosage How much programming was received—for example, the average number (percentage) of sessions attended, the percentage of the sample that did not attend at all, and the total average time spent on the application attendance records Views on a website, captured in web analytics Number of text message confirmations received Percentage of students attending the required or recommended proportion of lessons Average time spent on the web application, calculated as the average length of time a youth actively clicks on the content (measured in minutes) Percentage of text messages received, calculated as the number of text message confirmations received divided
by the number of text messages sent Quality What was the quality of interactions—for example, were facilitators knowledgeable and was pacing adequate Observations of interaction quality, using protocol developed by evaluators Percentage of sessions with high quality interactions, calculated as the percentage of observed interactions that the independent evaluator scored as “high quality” What was the quality of participant engagement with program—for example, did the youths actively engage with the material, and were the participants satisfied with the programming Observations of engagement, using the Youth Program Quality Assessment (YPQA) Ratings from facilitator fidelity logs Engagement ratings from participant satisfaction surveys Percentage of sessions with moderate youth engagement, calculated as the percentage of sessions in which the independent evaluator scored youth engagement as “moderately engaged” or higher Average parent engagement rating, calculated as the average of the engagement scale scores (ranging from 1–5) across parent satisfaction surveys Context and counterfactual What other TPP programming was available or offered to study participants and those in the comparison condition—for example, what are the sexual education requirements in the target area and is programming available through community settings District website listing all available TPP programming Interviews with school district curriculum director Survey items on baseline and follow-up assessments List of TPP programming, all TPP programming available to both treatment and comparison groups described on district websites will be listed Percentage receiving in-class TPP programming, the data on the survey question on experiences of the counterfactual at follow-up should be presented as frequency counts and percentages What external events affected implementation—for example, was there staffing turnover, were there budget cuts to the program, and were there any technology issues in the area List of school closures Interviews with community representatives List of organizational changes Number of schools that were closed as a result of district turnaround initiatives (unrelated to the TPP programming), if any Percentage and total number of planned cohorts missed due to community issues, if any What were the unplanned adaptations to key program Documentation of request to or approval by OAH to make List of unplanned adaptations, to be provided if any occur
components—for example, did the setting change, were additional sessions added, and were components cut an unplanned adaptation to the program Note: The of data and methods can be tailored to your program – feel free to add, edit, or delete components of this table, as appropriate for your implementation study.