Federal Register / Vol. 88, No. 204 / Tuesday, October 24, 2023 / Notices 73021 

Name of Committee: Disease, 

Disability, and Injury Prevention and 

Control Special Emphasis Panel (SEP)— 

CE24–011, Grants to Support New 

Investigators in Conducting Research 

Related to Understanding Drug Use and 

Overdose Risk and Protective Factors 

(K01). 

Date: March 5, 2024. 

Time: 8:30 a.m.–5 p.m., EST. 

Place: Web Conference. 

Agenda: To review and evaluate grant 

applications. 

For Further Information Contact: 

Aisha L. Wilkes, M.P.H., Scientific 

Review Officer, National Center for 

Injury Prevention and Control, Centers 

for Disease Control and Prevention, 

4770 Buford Highway NE, Mailstop 

S106–9, Atlanta, Georgia 30341, 

Telephone: (404) 639–6473; Email: 

AWilkes@cdc.gov. 

The Director, Office of Strategic 

Business Initiatives, Office of the Chief 

Operating Officer, Centers for Disease 

Control and Prevention, has been 

delegated the authority to sign Federal 

Register notices pertaining to 

announcements of meetings and other 

committee management activities, for 

both the Centers for Disease Control and 

Prevention and the Agency for Toxic 

Substances and Disease Registry. 

Kalwant Smagh, 

Director, Office of Strategic Business 

Initiatives, Office of the Chief Operating 

Officer, Centers for Disease Control and 

Prevention. 

[FR Doc. 2023–23457 Filed 10–23–23; 8:45 am] 

BILLING CODE 4163–18–P 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Administration for Children and 

Families 

Title IV–E Prevention Services 

Clearinghouse Handbook of Standards 

and Procedures, Draft Version 2.0 

AGENCY: Administration for Children 

and Families, Department of Health and 

Human Services. 

ACTION: Request for public comment. 

SUMMARY: The Administration for 

Children and Families (ACF), within the 

U.S. Department of Health and Human 

Services (HHS), oversees the Title IV–E 

Prevention Services Clearinghouse. ACF 

seeks comments on proposed changes 

and clarifications to existing standards 

and procedures in the Handbook of 

Standards and Procedures, Version 2.0. 

DATES: The deadline for comments on 

this notice is November 24, 2023. 

ADDRESSES: Interested parties may 

submit written questions, comments, 

and supplementary documents by email 

to preventionservices@abtassoc.com 

with ‘‘Title IV–E Prevention Services 

Clearinghouse FRN comment’’ in the 

subject line. To ensure that your 

comments have maximum effect, please 

identify clearly the section of the draft 

Handbook of Standards and Procedures, 

Version 2.0 that your comments 

address. 

Readers are referred to the full version 

of the draft Handbook of Standards and 

Procedures, Version 2.0 on the 

Clearinghouse website (https://

preventionservices.acf.hhs.gov/ 

resources/comment-draft-handbook). 

SUPPLEMENTARY INFORMATION: 

1.0 Background and Legislative 

Context 

The Family First Prevention Services 

Act (FFPSA) was signed into law as part 

of the Bipartisan Budget Act (H.R. 1892) 

on February 9, 2018. FFPSA amended 

the Social Security Act (the Act) to 

enable use of Federal funds available 

under parts B and E of title IV of the 

Social Security Act to provide enhanced 

support to children and families and 

prevent foster care placements through 

the provision of evidence-based ‘‘mental 

health and substance abuse prevention 

and treatment services, in-home parent 

skill-based programs, and kinship 

navigator services.’’ As described in the 

statutory language, these services and 

programs are intended ‘‘for children 

who are candidates for foster care or 

who are pregnant or parenting foster 

youth and the parents or kin caregivers 

of the children.’’ The Act requires an 

independent systematic review of 

evidence to designate programs and 

services as ‘‘promising,’’ ‘‘supported,’’ 

and ‘‘well-supported’’ practices. 

In order to meet these requirements, 

ACF established the Title IV–E 

Prevention Services Clearinghouse (the 

Clearinghouse). The Clearinghouse 

carries out a systematic review process 

implemented by trained reviewers using 

consistent, transparent standards and 

procedures. The Handbook of Standards 

and Procedures, Version 1.0 (https://

preventionservices.acf.hhs.gov/reviewprocess) provides a detailed description 

of the standards used to identify and 

review programs and services for the 

Clearinghouse and the procedures 

followed by the Clearinghouse staff. The 

Handbook of Standards and Procedures, 

Version 1.0 was informed by public 

comments submitted in response to 

Federal Register Notice 83 FR 29122 

(https://www.federalregister.gov/ 

documents/2018/06/22/2018-13420/ 

decisions-related-to-the-development-ofa-clearinghouse-of-evidence-basedpractices-in-accordance), consultations 

with research and practice experts, and 

the review processes developed and 

used by other prominent evidence 

clearinghouses. 

2.0 Overview of 2021 Request for 

Public Comment on Title IV–E 

Prevention Services Clearinghouse 

Handbook of Standards and 

Procedures, Version 1.0 

ACF solicited feedback on the 

Prevention Services Clearinghouse 

Handbook of Standards and Procedures, 

Version 1.0 (subsequently referred to as 

Handbook Version 1.0) through a 

Federal Register Notice 86 FR 37332 

(https://www.federalregister.gov/ 

documents/2021/07/15/2021-15065/ 

title-iv-e-prevention-servicesclearinghouse-handbook-of-standardsand-procedures) published on July 15, 

2021. This comment period was open 

for 30 days and closed on August 16, 

2021. One hundred four unique 

commenters submitted feedback, 

including 10 commenters from state and 

local child welfare agencies. 

Commenters included state and local 

government administrators, program 

and service developers, Federal staff, 

researchers and evaluators, foundation 

and non-profit organization staff, and 

other interested parties. ACF ensured 

the careful review and consideration of 

all of the comments in developing the 

draft Handbook of Standards and 

Procedures, Version 2.0 (subsequently 

referred to as Handbook Version 2.0). 

Comments were considered within the 

context of the statutory requirements of 

FFSPA, the necessity to conduct a 

systematic, objective, and transparent 

evidence review, and resource 

considerations. The public comments 

informed discussions with a large 

number of experts whose comments 

were also considered in developing the 

proposed revisions. 

Summary of Comments. Comments 

highlighted how the standards and 

procedures specified in Handbook 

Version 1.0 might be revised to better 

reflect the goals and requirements of the 

Executive Order on Advancing Racial 

Equity and Support for Underserved 

Communities Through the Federal 

Government. For example, commenters 

recommended prioritizing the review of 

programs and services that have been 

implemented and/or studied with 

diverse populations (Section 2.2). 

Commenters also recommended 

engaging diverse individuals and those 

with lived experience to inform the 

systematic review process and allowing 

greater flexibility for culturally adapted 

programs and services. Commenters 


ddrumheller on DSK120RN23PROD with NOTICES1
73022 Federal Register / Vol. 88, No. 204 / Tuesday, October 24, 2023 / Notices 

recommended providing additional 

detail to clarify the existing standards 

and procedures. For example, comments 

requested technical clarification 

regarding the definition of an available 

written protocol, manual, or other 

documentation (Section 2.1.2), 

determination of the length of time after 

the end of treatment (Section 6.2.3), 

determination of whether program or 

service or study adaptations are 

substantial (Section 4.1.6), and 

calculations of effect size and statistical 

significance (Section 5.1.0). 

Commenters recommended broadening 

the definitions of the program or service 

areas (Section 2.1.2) to be more 

inclusive regarding the types of 

programs and services that may be 

eligible for review. Commenters 

recommended broadening the definition 

of eligible comparison conditions 

(Section 4.1.4) and making the design 

and execution standards (Chapter 5), 

particularly those related to baseline 

equivalence (Section 5.7), more flexible. 

Finally, commenters provided 

recommendations to ACF that did not 

pertain to the Clearinghouse. For 

example, comments recommended ACF 

provide further support and investment 

in building evidence, particularly of 

programs and services designed to serve 

communities of color and others 

disproportionally represented in the 

child welfare system as well as for 

kinship navigator programs. 

Summary of Proposed Revisions. The 

draft Handbook Version 2.0 aims to be 

responsive to the diversity of comments 

received, to enhance the transparency of 

the systematic review process, and to 

support efforts to advance equity in 

accordance with the Executive Order on 

Advancing Racial Equity and Support 

for Underserved Communities Through 

the Federal Government. For example, 

revised program or service area 

definitions (Section 2.1) are inclusive of 

a broader range of programs and 

services, new program or service 

prioritization criteria have been added 

to consider the child welfare relevance 

and diversity of populations served 

(Section 2.2) with similar criteria also 

added for study prioritization (Section 

2.3), and the range of eligible 

comparison conditions for studies has 

been expanded to include studies that 

compare one intervention to another 

intervention (Section 4.1.7). Additional 

clarification and guidance are now 

provided on program or service and 

study adaptations, including new 

examples of how standards are applied 

to culturally adapted programs and 

services (Sections 2.3.2 and 4.1.9). 

Clarification is also provided that 

eligible outcomes and outcome 

measures may be defined differently 

across studies to reflect the different 

ages, backgrounds, cultures, locations, 

and contexts of the study participants, 

with examples provided (Section 4.1.8). 

Formulae used in effect size and 

statistical significance calculations are 

now provided directly in the Handbook 

(Chapter 6) and additional guidance and 

clarification is provided on design 

confounds, including clarification that 

studies with a single provider unit 

shared across the intervention and 

comparison conditions are not 

considered a confound (Section 5.9.3). 

A broader range of options is provided 

for establishing baseline equivalence 

and low attrition randomized group 

design contrasts are no longer assessed 

for baseline equivalence (Section 5.7). 

The Handbook now provides additional 

information on how the risk of harm 

assessment is conducted, with 

additional considerations for cases 

where the comparison group receives 

another intervention (Section 7.2.1). 

Further, additional clarification on how 

time since the end of treatment is 

calculated is provided (Section 7.2.3). 

The Handbook now clearly specifies 

how any member of the public can 

submit recommendations of programs or 

services for review or information about 

studies of those recommended programs 

and services to the Clearinghouse at any 

time (Chapters 1 and 3). 

Additional Relevant Activities. The 

Clearinghouse also intends to conduct 

additional activities to be responsive to 

public comments and to support efforts 

to advance equity in accordance with 

the Executive Order on Advancing 

Racial Equity and Support for 

Underserved Communities Through the 

Federal Government. First, the 

Clearinghouse is planning to display 

study participant characteristics on the 

program or service page of the 

Clearinghouse website. Display of 

participant characteristics is intended to 

promote transparency on the extent to 

which diverse populations are 

represented in research reviewed by the 

Clearinghouse. Second, the 

Clearinghouse plans to develop two new 

reports focused on equity. These two 

reports are intended to provide 

additional information about diverse 

populations included in studies of the 

programs and services that have been 

reviewed by the Clearinghouse and 

identify gaps in evidence. Third, 

enhanced activities are planned for 

future public calls for program and 

service recommendations in order to 

comprehensively identify culturally 

adapted and culturally grounded 

programs and services that may be 

eligible for review. The Clearinghouse 

plans to conduct targeted outreach to 

providers of culturally adapted and 

culturally grounded programs and 

services and community-based 

organizations serving diverse 

populations to improve engagement. 

The Clearinghouse also plans to clearly 

communicate in future public calls how 

the public, including community-based 

organizations and providers of 

culturally adapted and culturally 

grounded programs and services, can 

recommend programs and services and 

submit relevant studies of programs and 

services to the Clearinghouse. Further, 

the Clearinghouse plans to make future 

public call materials available in both 

English and Spanish. Fourth, the 

Clearinghouse intends to revise its 

author Reporting Guide to clarify 

recommended reporting related to 

culturally adapted and culturally 

grounded programs and services and the 

characteristics of their participants. 

Fifth, the Clearinghouse intends to 

revise existing resources for 

Clearinghouse users, such as its 

Frequently Asked Questions (FAQ) 

website section and fact sheet resources, 

with person-centered design principles 

to ensure information about the 

Clearinghouse and its standards and 

procedures are accessible. Sixth, the 

Clearinghouse plans to publicly post all 

programs and services that have been 

recommended for review and will 

continue to explore additional ways to 

improve transparency such as through 

data sharing. 

A comprehensive list of specific 

revisions and clarifications to the 

Clearinghouse’s Standards and 

Procedures is provided in the following 

section. Subsequent chapter and section 

numbers all refer to the chapter and 

section numbering for the draft 

Handbook Version 2.0 unless the text 

explicitly indicates a reference to 

Handbook Version 1.0 chapter and 

section numbering. 

3.0 Revisions and Clarifications to the 

Clearinghouse’s Standards and 

Procedures in the Draft Handbook 

Version 2.0 

3.1 Introduction 

The revised introduction includes a 

description of the Clearinghouse 

website and resources available on the 

website. This includes reference to the 

FAQ section that includes information 

on how members of the public can 

submit a program or service 

recommendation and how to provide 

information about studies to the 

Clearinghouse. 


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 204 / Tuesday, October 24, 2023 / Notices 73023 

3.2 Chapter 1. Identify Programs and 

Services 

Revisions clarify that all program and 

service recommendations are retained 

for consideration, including those 

submitted during public calls and ad 

hoc recommendations submitted to the 

Prevention Services Clearinghouse 

inbox. Revisions also clarify that any 

member of the public may submit a 

program or service recommendation at 

any time to the Clearinghouse via email 

and that suggested information to 

include as part of a program or service 

recommendation can be found on the 

FAQ section of the Clearinghouse 

website. Additionally, this section now 

indicates that all programs and services 

identified as potential candidates for 

review will be posted on the 

Clearinghouse website. 

3.3 Chapter 2. Prioritize and Select 

Programs and Services 

3.3.1 Revisions and Clarifications to 

Program or Service Area Definitions 

(Section 2.1.1) 

Based on FRN feedback and 

consultation with experts in the fields of 

mental health, substance use, parenting 

and parent skill-based programs and 

services, kinship navigator programs, 

and child welfare, the draft Handbook 

Version 2.0 revised and clarified the inhome parent-skill based and substance 

use prevention and treatment program 

or service area definitions, as noted 

below. 

• In-home parent skill-based 

programs and services. The revised 

definition is more flexible and now 

indicates that eligible programs and 

services involve direct intervention with 

a parent or caregiver and target 

parenting skills or other skills that can 

be applied to where the child resides, 

including in the home. The revised 

definition also clarifies that delivery of 

programs and services can occur in the 

home or other settings and defines 

necessary content for a program or 

services to be considered ‘‘skill-based.’’ 

Revised examples of eligible and 

ineligible in-home skill-based programs 

and services are provided in Exhibit 2.3. 

• Substance use prevention and 

treatment programs and services. The 

revised definition clarifies that 

programs or services: 

• targeting recovery from substance 

use (as well as those targeting 

prevention, treatment, remediation, 

elimination and/or reduction of 

substance use or misuse) are eligible; 

and 

• without client-oriented substance 

use prevention or treatment 

components, such as mass 

communications/media campaigns or 

interventions that solely target broader 

community-level or policy systems, 

remain not eligible. 

Revised examples of eligible and not 

eligible programs and services are now 

provided in Exhibit 2.2. Specifically, 

one new example clarifies that programs 

or services targeting parents or 

caregivers aiming to prevent substance 

use among children and youth are 

eligible. 

Minor wording changes were made to 

the kinship navigator program or service 

area definition for clarification 

purposes. Experts did not suggest any 

changes to eligible outcomes for kinship 

navigator programs and services. 

No changes were made to the mental 

health prevention and treatment 

programs and services definition. New 

examples of eligible and ineligible 

programs and services are provided in 

Exhibit 2.1. 

3.3.2 Clarifications to Available 

Protocols, Manuals, or Other 

Documentation (Section 2.1.2) 

To be eligible for review by the 

Prevention Services Clearinghouse, 

programs and services must be clearly 

defined and replicable. To meet this 

criterion, programs and services must 

have available written or recorded 

protocols, manuals, or other 

documentation that describes how to 

implement or administer the practice 

(referred to subsequently in this notice 

as a ‘‘manual’’ for brevity). Revisions to 

this section clarify that materials to 

satisfy this requirement may be 

presented in a web-based format and 

that ‘‘manual’’ can include recorded 

videos or online learning systems if 

these materials describe how to 

implement or administer the practice. 

The Clearinghouse notes that, consistent 

with Handbook Version 1.0, there are no 

language requirements for manual 

eligibility. 

3.3.3 Revisions and Clarifications to 

Program or Service Prioritization 

(Section 2.2) 

As of July 2023, the Prevention 

Services Clearinghouse has reviewed 

148 programs and services. Yet there 

remains a high volume of potentially 

eligible programs and services identified 

for review. As a result, the Prevention 

Services Clearinghouse must continue 

to prioritize programs and services for 

review. The draft Handbook Version 2.0 

continues to highlight the prioritization 

of programs and services with available 

evidence of eligibility and programs and 

services in active use (Section 2.2). New 

to this section is further clarification 

about additional prioritization 

considerations. These additional 

prioritization criteria were informed by 

recommendations from public 

comments and consultation with 

experts. Listed below are the additional 

prioritization criteria included in the 

draft Handbook Version 2.0. 

• Number and source of program or 

service recommendations received; 

• Child welfare relevance; 

• Population(s) served; 

• Previous evaluations and studies; 

and 

• Implementation supports. 

The Clearinghouse continues to 

prioritize programs and services in a 

way that ensures representation across 

the four program and service areas. 

Additional clarification is provided in 

draft Handbook Version 2.0 noting that 

the Clearinghouse assesses prioritization 

criteria by examining publicly available 

information, other clearinghouses’ 

websites, and materials submitted with 

program or service recommendations. 

3.3.4 Clarifications on Program or 

Service Selection (Section 2.3.1) 

Given the large volume of programs 

and services identified, resource 

considerations mean that not all 

programs and services can be selected 

for review at once. To help clarify the 

distinction between the prioritization 

and reviewing process, the draft 

Handbook Version 2.0 adds a new 

section on selection of a program or 

service for review (Section 2.3.1). Based 

on the prioritization process, specific 

programs and services are selected for 

review at a given time, as indicated by 

publication on the working list of 

programs and services planned for 

review available on the Prevention 

Services Clearinghouse website. The 

final eligibility of a program or service 

for review by the Clearinghouse is 

determined after a program or service is 

selected for the working list. 

3.3.5 Revisions to Program or Service 

Adaptations Criteria (Section 2.3.2) 

Multiple public comments requested 

clarification regarding the program or 

service adaptation standards specified 

in Handbook Version 1.0 (found in 

Section 4.1.6 of this version) and 

recommended increased inclusivity, 

particularly with respect to cultural 

adaptations. The Prevention Services 

Clearinghouse sought input from a range 

of experts specifically focusing on 

program or service adaptations, 

including those with expertise in 

cultural adaptations designed to serve 

historically underserved communities. 

Underserved communities, as 

articulated in the Executive Order on 

Advancing Racial Equity and Support 


ddrumheller on DSK120RN23PROD with NOTICES1
73024 Federal Register / Vol. 88, No. 204 / Tuesday, October 24, 2023 / Notices 

for Underserved Communities Through 

the Federal Government, include Black, 

Latino, and Indigenous and Native 

American persons, Asian Americans 

and Pacific Islanders and other persons 

of color; members of religious 

minorities; lesbian, gay, bisexual, 

transgender, and queer (LGBTQ+) 

persons; persons with disabilities; 

persons who live in rural areas; and 

persons otherwise adversely affected by 

persistent poverty or inequality. 

To meet the eligibility criteria of being 

clearly defined and replicable, a 

program or service must have publicly 

available written or recorded protocols, 

manuals, or the documentation 

(hereafter referred to as ‘‘manuals’’) that 

describe how to implement the practice 

(Section 2.1.2). A new section (2.3.2) 

clarifies the procedures used to identify 

and review relevant manuals for a 

program or service. This includes 

procedures for identifying a primary 

manual for review and addressing cases 

with multiple potential manuals. 

Many programs and services have 

multiple manuals, including manual 

editions (e.g., editions of a manual as a 

program or service evolves over time or 

expands) and manual variants (e.g., 

adaptations of a program or service or a 

manual to address new issues, different 

populations, or alternative approaches 

to delivering the program or service). 

This section clarifies the standard 

process by which the Prevention 

Services Clearinghouse assesses 

whether alternative manual editions or 

variants have any substantial 

adaptations, compared to the primary 

manual identified. This process consists 

of the following steps, followed as 

needed based on the nature of the 

program or service: 

• Step 1: Determining whether the 

adaptation is explicitly prohibited in the 

primary program or service manual 

under review or is the result of adding 

another separate program or service to 

the existing program or service (i.e., 

‘‘bundling’’); 

• Step 2: Determining whether the 

adaptation is explicitly allowed by the 

primary program or service manual 

under review; 

• Step 3: Determining whether the 

adaptation substantially changes a 

program element in the primary 

program or service manual under 

review; 

• Step 4: Gathering additional 

information and consulting with senior 

content experts on the Clearinghouse. 

A revised table (Exhibit 2.4) classifies 

program elements and gives examples of 

acceptable and substantial 

adaptations—including expanded 

examples of adaptations that may be 

made in the process of culturally 

adapting a program or service. (These 

criteria and procedures are aligned with 

those used to assess any program or 

service adaptations identified in studies 

during the study eligibility process, 

described in Section 4.1.9). Manuals 

that are substantially adapted from a 

primary manual may be considered as a 

separate program or service when 

reviewing studies. Studies with these 

substantial adaptations would be 

ineligible in a review based on the 

primary manual identified for a 

particular program or service. 

Alternatively, manuals without 

substantial adaptations may be 

considered the same program or service 

when reviewing studies. Studies 

without substantial adaptations would 

be included in a review based on the 

primary manual. 

3.4 Chapter 3. Literature Search 

To help ensure identification of 

studies conducted with American 

Indian and Alaska Native populations, 

the draft Handbook Version 2.0 adds 

Healthy Native Youth to its list of 

clearinghouses used to identify relevant 

research. The list of bibliographic 

databases has been trimmed for 

efficiency and resource considerations. 

Some databases in Handbook Version 

1.0 were largely providing duplicative 

results. This section clarifies that any 

publicly available research from 

program or service websites is 

incorporated into the search. 

Clarification is also provided on 

procedures for incorporation of research 

that is submitted to the Prevention 

Services Clearinghouse inbox ad hoc or 

during public calls. 

3.5 Chapter 4. Study Eligibility 

Screening and Prioritization 

3.5.1 Revision to Study Definition 

(Section 4.1) 

In alignment with other Federal 

evidence clearinghouses, the Prevention 

Services Clearinghouse intends to focus 

on degree of sample overlap in applying 

its definition of a study as ‘‘one research 

investigation of a defined subject 

sample, and the interventions, 

measures, and statistical analyses 

applied to that sample.’’ Additional 

study definition criteria (based on the 

What Works Clearinghouse v4.0 study 

definition) in Handbook Version 1.0 

have been dropped in the draft 

Handbook Version 2.0. 

3.5.2 Clarifications on Source of 

Publication Criteria (Section 4.1.2), 

Language of Publication (Section 4.1.3) 

and Location of Study (Section 4.1.4) 

The draft Handbook Version 2.0 

clarifies the definition of ‘‘publicly 

available’’ and ‘‘published’’ for the 

source of publication standard (Section 

4.1.2), in response to public comments. 

Dissertations, theses, and conference 

papers remain ineligible. Given the 

priority of reviewing a large number of 

programs and services, the Prevention 

Services Clearinghouse intends to 

continue to exclude such sources in the 

interests of efficiency. 

Some public comments indicated 

confusion about whether studies 

conducted outside of the United States 

or those conducted in non-Englishspeaking countries are eligible. The 

draft Handbook Version 2.0 clarifies 

that the standard from Handbook 

Version 1.0 that studies must be 

available in English (Section 4.1.3) is 

inclusive of studies originally published 

in another language that have published 

English language translations available. 

The draft Handbook Version 2.0 

explicitly clarifies that studies 

conducted in any country are eligible 

(Section 4.1.4), as they were under 

Handbook Version 1.0. 

3.5.3 Revisions to Study Design and 

Intervention Condition Criteria (Sections 

4.1.5, 4.1.6) 

The draft Handbook Version 2.0 

provides clarification on definitions for 

randomized group designs and quasiexperimental group designs with respect 

to eligible study designs (Section 4.1.5). 

It clarifies that single-group pretestposttest designs and interrupted time 

series designs without comparison 

groups are not eligible. It also clarifies 

that group assignment must be exclusive 

for an outcome measured at a given 

point in time—that is, participants 

cannot be counted in both the 

intervention and comparison condition. 

The criterion for eligible intervention 

conditions—that the intervention group 

is offered an eligible program or service 

that is essentially the same for all 

participants in the group—remains the 

same as in Handbook Version 1.0, with 

minor clarifications, but is presented as 

a distinct subsection in the draft 

Handbook Version 2.0 (Section 4.1.6) 

for clarity. 

3.5.4 Revisions to Eligible Comparison 

Conditions (Section 4.1.7) 

Many public comments requested 

expansion of eligible study comparison 

conditions beyond no or minimal 

treatment and treatment as usual to 


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 204 / Tuesday, October 24, 2023 / Notices 73025 

include more active comparison 

conditions. Many experts also 

recommended that the Prevention 

Services Clearinghouse consider 

including active comparison conditions. 

One consideration voiced by multiple 

experts consulted is that active 

comparison conditions are increasingly 

recommended, particularly if there are 

other available interventions considered 

to be efficacious. Revision to this 

standard was considered in the context 

of the FFSPA legislative criterion that a 

program or service must be 

demonstrated as being superior to an 

appropriate comparison practice. 

The draft Handbook Version 2.0 

allows for five types of eligible 

comparison conditions: 

• No intervention or wait list—offered 

no services or services at a later date 

(clarifying that outcomes measured after 

a wait list group is offered the 

intervention are not eligible). 

• Minimal intervention—including 

informational materials or 

psychoeducation, referrals to available 

services, or similar nominal services. 

• Placebo or attention control— 

conditions designed to account for 

nonactive effects of treatment, such as 

participants’ expectations, contact time 

with an interventionist, or the 

relationship between interventionist 

and participants; includes psychological 

or pharmacological placebos, attention 

placebos, and nonspecific therapy in 

which participants receive the same or 

similar amount of attention or contact as 

the participants in the intervention 

condition. 

• Treatment as usual—The draft 

Handbook Version 2.0 clarifies that both 

‘‘usual or typical services’’ (i.e., 

individuals do not receive anything they 

would not have been able to receive 

outside the context of the study) or 

‘‘services consistent with usual or 

typical services’’ (i.e., services as part of 

the study that are not offered in the 

community but are clearly described as 

consistent with the usual or typical 

services that would be received by 

individuals or families similar to those 

in the study) are considered eligible 

under treatment as usual. Therapeutic 

or pharmacological interventions that 

meet the definition of treatment as usual 

are eligible. 

• Head-to-head comparisons— 

assigned to another intervention that is 

not a variant of the program or service 

under review (may also be referred to as 

alternative interventions, active 

interventions, or comparator 

interventions); excluded are 

comparisons to pharmacological 

interventions that do not meet the 

definition of treatment as usual above. 

The draft Handbook Version 2.0 

indicates three types of comparison 

conditions that are explicitly not 

eligible for review and provides a 

rationale for each: 

• Intervention variants—assigned to 

an intervention that is a variation of the 

intervention under review. Examples 

include dismantling studies (e.g., full 

version of intervention compared to one 

lacking one or more components); 

bundled intervention studies (e.g., full 

version of intervention compared to a 

version with a second intervention 

added); studies comparing different 

delivery modes, providers, dosage, or 

fidelity levels for the same intervention; 

sequencing studies (e.g., both conditions 

receive the same interventions, but in a 

different order). 

• Population-level data or 

benchmarks—constructed from 

population norms or statistics derived 

from other studies, surveys, censuses, or 

similar sources. 

• Comprised only of intervention 

refusers or dropouts—composed 

entirely of individuals who were offered 

the intervention condition but refused 

the offer or dropped out of the 

intervention after being offered the 

intervention. 

3.5.5 Revisions to Outcomes (Section 

4.1.8) 

Definitions of outcome domain, 

outcome, and outcome measurement 

have been provided for clarity. 

Clarifications have been included 

regarding eligible outcomes within the 

child safety and child permanency 

outcome domains and family 

functioning outcomes within the adult 

well-being outcome domain. The 

clarifications to the child safety and 

child permanency outcomes were 

previously described in the FAQ section 

of the Prevention Services 

Clearinghouse website. Additionally, 

eligible educational achievement and 

attainment outcomes in the child wellbeing outcome domain have been 

expanded to include school attendance 

and absenteeism as eligible outcomes. 

These outcomes, though not direct 

measures of educational achievement 

and attainment, are viewed as closely 

related and relevant outcomes. 

Clarification is provided that outcomes 

that are composites of one or more 

eligible outcomes within the eligible 

outcome domains are eligible; those that 

are composites of eligible and ineligible 

outcomes are not eligible. Clarification 

is also provided that eligible outcomes 

and outcome measures may be defined 

differently across studies to reflect the 

different ages, backgrounds, cultures, 

locations, and contexts of the study 

participants, with examples provided. 

The Prevention Services 

Clearinghouse currently does not have 

measurement standards for assessing the 

validity or reliability of biomarker 

measures (i.e., a physiological measure 

used as an indicator of a physical, 

psychological or emotional state), such 

as the use of cortisol as a measure of 

psychological stress. Expert 

consultations on biomarkers did not 

indicate a clear set of standards that 

could be broadly applied for review of 

such measures. As a result, the draft 

Handbook Version 2.0 indicates that 

biomarker measures are not currently 

eligible for review as child well-being or 

adult well-being outcomes. 

3.5.6 Revisions to Study Program or 

Service Adaptations Criteria (Section 

4.1.9) 

Consistent with Handbook Version 

1.0, the draft Handbook Version 2.0 

indicates that, to be eligible for review, 

studies of a program or service must all 

represent similar implementations of 

the program or service selected for 

review. Revisions in the draft Handbook 

Version 2.0 clarify that the process of 

assessing program or service adaptations 

for study eligibility is based on having 

identified a particular manual (or set of 

manuals) of the program or service 

under review (see Sections 2.3.1, 2.3.2). 

The standard process used to identify 

whether program or service adaptations 

are present in the studies being screened 

for eligibility is clarified. The 

procedures and criteria for assessing 

whether adaptations identified in 

studies are acceptable or substantial 

mirror those specified in Section 2.3.2 

for adaptations found in manual 

editions or variants. The end result of 

these procedures is the determination of 

study eligibility for the particular 

program or service under review (in 

Section 2.3.2, the end determination is 

whether two manuals are substantively 

similar or represent different programs 

or services). Studies with any 

substantial adaptations are ineligible for 

review as a study of the program or 

service under review (such studies may 

be eligible for review as a study of 

different program or service and its 

associated manual). Studies with only 

minor adaptations may potentially be 

eligible if all other study eligibility 

criteria are met. 

3.5.7 Revisions to Study Review 

Prioritization Criteria (Section 4.2) 

The Prevention Services 

Clearinghouse notes that study 

prioritization criteria are distinct from 

study eligibility criteria. When a 


ddrumheller on DSK120RN23PROD with NOTICES1
73026 Federal Register / Vol. 88, No. 204 / Tuesday, October 24, 2023 / Notices 

program or service has more than 15 

studies eligible for review, study 

prioritization criteria are applied to 

order the review of eligible studies. The 

study prioritization process ensures 

efficiencies in the reviewing process to 

review a large number of programs and 

services. 

The Prevention Services 

Clearinghouse notes that only 12 of the 

148 programs and services reviewed as 

of July 2023 had more than 15 eligible 

studies identified, requiring the use of 

study prioritization criteria in these 

reviews to prioritize the first 15 eligible 

studies for review using the design and 

execution standards. Of these 12 

programs and services, nine had 16 to 

25 eligible studies, with a few having a 

much larger number of eligible studies 

(e.g., 75 or 90). All other programs and 

services reviewed had 15 or fewer 

eligible studies, with all eligible studies 

reviewed using the design and 

execution standards. Therefore, as in 

Handbook Version 1.0, the study 

prioritization criteria continue to apply 

only when there are 15 or more eligible 

studies of a program or service in the 

draft Handbook Version 2.0. 

Three modifications have been made 

to the process of assigning prioritization 

points for identifying the order in which 

studies are reviewed in the draft 

Handbook Version 2.0. First, given that 

programs or services must demonstrate 

sustained favorable effects 6 or 12 

months beyond the end of treatment 

(Section 7.2.3) to receive a rating of 

supported or well-supported, the 

Prevention Services Clearinghouse 

intends to increase the prioritization 

points given to studies that include 

outcomes measured 6 or 12 months 

beyond the end of treatment to ensure 

that these studies are reviewed earlier 

when present, increasing the 

prioritization points for such studies to 

3 and 6 points, respectively (compared 

to 1 and 2 points, respectively, in 

Handbook Version 1.0). Second, some 

public commenters and experts 

consulted noted the importance of 

statistical power for being able to detect 

intervention effects. The draft 

Handbook Version 2.0 adds one 

prioritization score point for studies that 

report an analysis of statistical power. 

Third, many public comments 

recommended that points be awarded to 

studies based on populations served. 

The draft Handbook Version 2.0 intends 

to add one prioritization score point for 

the child welfare relevance of 

populations served and two 

prioritization points for studies with 

samples from underserved 

communities. Prioritization points for 

studies with outcomes in multiple 

outcome domains have been decreased 

from a maximum of three to a maximum 

of one. The draft Handbook Version 2.0 

provides procedural details clarifying 

how ties in prioritization scores are 

resolved in cases where more than 15 

eligible studies are identified. 

The draft Handbook Version 2.0 

includes efficiency enhancements based 

on the study prioritization process for 

programs and services where more than 

15 eligible studies are identified. If, after 

review of the first 15 eligible studies 

prioritized for review, a program or 

service has not achieved a rating of 

well-supported, additional studies are 

reviewed using the design and 

execution standards in their prioritized 

order until either no eligible studies 

remain that could result in further 

improvement to the program or service 

rating or all eligible studies have been 

reviewed. Determination of potential for 

program or service ratings to improve 

upon review of additional eligible 

studies is based on (1) the program 

rating from studies already reviewed 

using the design and execution 

standards and (2) the duration of effects 

examined in the remaining studies (as 

assessed according to study review 

prioritization criteria). Detailed 

examples of the application of this 

policy are described in Section 4.2. The 

draft Handbook Version 2.0 retains the 

policy from Handbook Version 1.0 of 

reviewing all studies against design and 

execution standards when 15 or fewer 

eligible studies are identified. All 

eligible studies are reviewed for risk of 

harm. 

3.6 Chapter 5. Evidence Review Using 

the Design and Execution Standards 

3.6.1 Revisions and Clarifications to 

Contrasts Rated, Design and Execution 

Rating Categories, Method of 

Assignment, and Integrity of Random 

Assignment (Sections 5.1 to 5.5) 

The draft Handbook Version 2.0 

indicates that contrasts from all eligible 

comparison conditions (Section 5.1) 

will be rated, whereas under Handbook 

Version 1.0, only contrasts from the 

least-intensive eligible comparison 

condition for a particular contrast were 

rated if multiple comparison conditions 

were eligible for review (Handbook 

Version 1.0, Section 4.1.4). Given the 

priority of reviewing a large number of 

programs and services, the draft 

Handbook Version 2.0 retains the policy 

from Handbook Version 1.0 of only 

reviewing full-sample analyses and not 

reviewing subgroup or sensitivity 

analyses due to resource considerations. 

For any studies that receive a moderate 

or high design and execution rating and 

report subgroup analyses, the 

Clearinghouse intends to indicate 

whether subgroup analyses were 

conducted for informational purposes 

only. New and revised examples are 

provided to clarify integrity of 

randomization standards for individual 

and cluster-assignment designs. 

3.6.2 Revisions and Clarifications to 

Attrition, Baseline Equivalence, and 

Pretest Standards (Sections 5.6 to 5.8) 

Based on expert feedback, and in 

alignment with other Federal 

clearinghouses (in particular, the What 

Works Clearinghouse and Home Visiting 

Evidence of Effectiveness [HomVEE]), 

the draft Handbook Version 2.0 no 

longer requires baseline equivalence to 

be established for a contrast from a low 

attrition randomized group design to 

receive a ‘‘High’’ support of causal 

evidence rating. 

Public comments expressed a desire 

for greater flexibility regarding options 

for demonstrating baseline equivalence 

and reconsideration of participant 

sociodemographic characteristics that 

could be used to establish baseline 

equivalence when a pretest alternative 

is not available. Informed by expert 

consultations, the draft Handbook 

Version 2.0 maintains a general 

preference for using the same (or nearly 

the same) measure as the outcome (i.e., 

a ‘‘direct pretest’’) for baseline 

equivalence but now allows any eligible 

outcome measure demonstrated to be 

correlated with the outcome at a 

threshold of 0.60 or higher to be used 

to establish baseline equivalence (here 

referred to as a ‘‘correlated pretest 

measure’’). Also informed by expert 

feedback, when a correlated pretest 

measure or pretest alternative is not 

available, the draft Handbook Version 

2.0 provides greater flexibility in the 

form of two options for establishing 

equivalence on sociodemographic 

characteristics, allowing an expanded 

set of individual characteristics and the 

use of a set of neighborhood 

characteristics if only one individual 

characteristic is available. Option 1 

requires demonstration of baseline 

equivalence on at least two of the 

following individual characteristics: 

race or ethnicity, socioeconomic status, 

household composition, or age. If only 

one of the four individual 

characteristics from Option 1 is 

available, baseline equivalence can still 

be established under Option 2 if 

equivalence is demonstrated on a 

measure of each of the following 

neighborhood characteristics: race or 

ethnicity, socioeconomic status, and 

household composition. When 

sociodemographics are used to establish 


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 204 / Tuesday, October 24, 2023 / Notices 73027 

baseline equivalence, a new 

requirement indicates that study authors 

must clearly describe all criteria used to 

create the intervention and comparison 

groups and affirmatively indicate that 

the same or similar criteria were used to 

create each group. 

Binary measures have different 

statistical properties than continuous 

measures that can potentially reduce 

their reliability as indicators of baseline 

equivalence—particularly when events 

are rare or in smaller samples. To 

address this, the draft Handbook 

Version 2.0 indicates a preference for 

continuous correlated pretests over 

direct pretests when establishing 

baseline equivalence for a binary 

outcome. It also permits use of 

continuous pretest alternative measures 

when outcomes are binary, even if it 

was feasible to measure a direct pretest. 

Specifically, continuous measures that 

meet the correlated pretest measure or 

pretest alternative criteria are preferred 

over a direct pretest of the binary 

measure, when available. 

3.6.3 Revisions and Clarifications to 

Statistical Model Standards (Section 

5.9) 

The statistical model standards 

(Section 5.9.1) have been revised in the 

draft Handbook Version 2.0 to clarify 

procedures used when statistical models 

do not meet standards and alternative 

statistical models are not available or do 

not meet standards. In such cases, the 

Prevention Services Clearinghouse will 

seek to review the contrast based on 

unadjusted means and standard 

deviations and the statistical 

significance test procedures specified in 

Chapter 6. 

The measurement reliability standard 

for inter-rater reliability in Handbook 

Version 1.0 was revised in the draft 

Handbook Version 2.0 (Section 5.9.2), 

with specific thresholds for inter-rater 

reliability (correlation), inter-rater 

agreement on the basis of percentage 

agreement (0.80 or higher), and interrater agreement based on kappa (0.60 or 

higher). These revised standards are 

aligned with current What Works 

Clearinghouse standards. 

Some public comments expressed 

concern that confound standards 

prevent inclusion of studies conducted 

in rural or underserved areas where 

only a single service provider is 

available may not be able to meet 

standards. The draft Handbook Version 

2.0 clarifies that studies with a single 

person or administrative unit are not 

automatically confounded, with 

detailed clarifying examples added to 

this section. Specifically, if a single 

provider (or a single administrative 

unit) provides treatment or services to at 

least some participants in both the 

intervention and comparison condition, 

a design confound is not considered to 

be present. Expert feedback indicated 

that the confound standards in 

Handbook Version 1.0 were appropriate 

causal evidence standards, informing 

the retention of these confound 

standards in the draft Handbook 

Version 2.0. 

3.7 Chapter 6. Record and 

Characterize Impact Estimates 

Public comments requested additional 

information about the formulae used for 

computing effect sizes and procedures 

used for determining statistical 

significance. The draft Handbook 

Version 2.0 provides all standard 

formulae used in computing effect sizes 

reported and for computing statistical 

significance. For models that meet 

statistical model standards in the design 

and execution requirements (Section 

5.9), the draft Handbook Version 2.0 

indicates that author-reported statistical 

significance is preferred in covariateadjusted models and certain models for 

which the Prevention Services 

Clearinghouse does not currently have 

standards for computing statistical 

significance (e.g., time-to-event models). 

When such models are not available or 

do not meet statistical model standards, 

the formulae provided are used to 

conduct a post-hoc statistical 

significance test based on the natural 

metric of the outcome reported (e.g., 

continuous, binary, count, or time-toevent). 

Clarification is provided on 

information needed and procedures 

used to compute effect sizes and 

statistical significance for repeated 

measures models (e.g., growth curve 

models). In alignment with other 

Federal clearinghouses (in particular, 

What Works Clearinghouse, HomVEE), 

point-in-time estimates for each 

measurement time period are required. 

If such information is not reported, 

unadjusted means and standard 

deviations for each point in time are 

used (or requested if not reported), with 

appropriate post-hoc significance tests 

performed based on the natural metric 

of the outcome. 

3.8 Chapter 7. Program or Service 

Ratings 

3.8.1 Revisions and Clarifications to 

Program or Service Ratings (Section 7.1) 

and Risk of Harm (Section 7.2.1) 

No changes were made to the criteria 

for promising, supported, or wellsupported program or service ratings in 

the draft Handbook Version 2.0 (Section 

7.1). This section clarifies that intention 

of the Prevention Services 

Clearinghouse is for program or service 

ratings from reviews conducted under 

Handbook Version 1.0 to be retained 

until such time that a program or service 

is re-reviewed under Handbook Version 

2.0 (see Section 8.5.1 below regarding 

re-review procedures). 

A new standard specified in the risk 

of harm section (Section 7.2.1) of the 

draft Handbook Version 2.0 is that 

contrasts in head-to-head comparison 

conditions or placebo or attention 

control comparison conditions where 

the comparison condition has any 

evidence for risk of harm cannot 

contribute to a promising, supported, or 

well-supported rating. If risk of harm is 

present in these kinds of comparison 

conditions, impact estimates are not 

clearly interpretable as evidence of 

intervention effectiveness—as it is 

possible that both the intervention and 

comparison condition could be made 

worse off than if they had not 

participated in the study at all. When 

risk of harm is not present in the 

comparison condition, favorable 

impacts can be interpreted as the 

intervention group being at least better 

off than they would have been if no 

treatment had been offered at all and 

can potentially contribute as evidence of 

effectiveness. Standard procedures for 

identifying potential risk of harm in 

comparison conditions are detailed in 

this section. 

3.8.2 Revisions and Clarifications to 

Usual Care or Practice Settings 

Definition (Section 7.2.2) 

The definition of usual care or 

practice settings (Section 7.2.2) in the 

draft Handbook Version 2.0 has been 

clarified to indicate that community 

settings, such as schools, with 

embedded service providers that may 

provide eligible programs or services as 

part of their typical operations (e.g., 

school counselors), are also considered 

usual care or practice settings. It 

clarifies that clinics that provide 

services solely for participants in 

research studies or clinical trials (i.e., 

that do not provide any services to 

persons not participating in research 

studies as part of their typical 

operations) do not constitute usual care 

or practice settings. 

3.8.3 Revisions and Clarifications to 

Beyond the End of Treatment (Section 

7.2.3) 

Some public comments requested 

clarification on how the Prevention 

Services Clearinghouse assesses the 

duration of sustained effects, 

particularly in cases where the end of 


ddrumheller on DSK120RN23PROD with NOTICES1
73028 Federal Register / Vol. 88, No. 204 / Tuesday, October 24, 2023 / Notices 

treatment is flexible across participants. 

Section 7.2.3 of the draft Handbook 

Version 2.0 includes revisions to clarify 

the order of preference for information 

that may be provided in studies about 

the end of treatment and procedures for 

computing the duration of sustained 

effects when the duration of treatment is 

fixed, when the duration of treatment is 

defined and varies across participants, 

and when the duration of treatment is 

undefined. Treatment of boosters in 

computing the duration of sustained 

effects is now explicitly addressed. 

Detailed procedures and examples can 

be found in Section 7.2.3 of the draft 

Handbook Version 2.0. 

3.9 Chapter 8. Prevention Services 

Clearinghouse Procedures 

The draft Handbook Version 2.0 

represents the first update to the 

Handbook of Standards and Procedures 

since the beginning of the Title IV–E 

Prevention Services Clearinghouse in 

2018. The basic procedures for 

identifying eligible studies (Section 8.3) 

and reviewing studies against the design 

and execution standards (Section 8.4) 

remain essentially the same, with minor 

clarifications to operational procedures. 

Author query policies (Section 8.4.2) 

have been clarified; new content has 

been added clarifying the reasons that 

the Prevention Services Clearinghouse 

may query program and service 

developers for information about 

programs or services (Section 8.4.3). 

New content and more substantive 

revisions are described below. 

3.9.1 Selection of Handbook of 

Standards and Procedures Version To 

Use in Reviews (Section 8.2) 

The intention of the Prevention 

Services Clearinghouse is to conduct 

reviews of any program or service not 

previously reviewed under Handbook 

Version 1.0 solely under the standards 

and procedures specified in Handbook 

Version 2.0 once it is finalized. 

Programs or services that are included 

on the working list prior to when 

Handbook Version 2.0 is finalized may 

be reviewed under Handbook Version 

1.0 or Handbook Version 2.0. The 

version of the handbook used to 

conduct a review (or re-review) of a 

program or service will be clearly stated 

on the working list and on the program 

or service’s review page on the 

Prevention Services Clearinghouse 

website. 

3.9.2 Program and Service Re-Reviews 

and Study Re-Reviews (Sections 8.5.1, 

8.5.2) 

The Prevention Services 

Clearinghouse intends to conduct 

program and service re-reviews solely 

under Handbook Version 2.0 after it is 

finalized (Section 8.5.1). The intention 

of the Prevention Services 

Clearinghouse is that all existing 

program and service ratings determined 

under Handbook Version 1.0 will 

remain in effect until such time that a 

program or service re-review is 

conducted of a program or service. 

Programs and services reviewed by 

the Prevention Services Clearinghouse 

under Handbook Version 1.0 may be 

considered for re-review under 

Handbook Version 2.0 if a re-review has 

the potential to change the program or 

service rating (Section 8.5.1). Program or 

service ratings could potentially change 

due to application of Handbook Version 

2.0 standards to studies already 

identified in a prior review (e.g., studies 

previously ineligible now being eligible; 

studies being able to demonstrate 

baseline equivalence under revised 

standards) or the emergence of new 

evidence since the original review. The 

intention of the Prevention Services 

Clearinghouse is that the rating of a rereviewed program or service would be 

based solely on the standards and 

procedures in Handbook Version 2.0 

(i.e., the previously assigned rating 

would no longer be in effect). 

The intention of the Prevention 

Service Clearinghouse is to conduct 

study re-reviews (i.e., due to missing 

information or errors in the currently 

published review of an individual 

study) under the version of the 

handbook used to review the program or 

service (Section 8.5.2). That is, for a 

program or service reviewed under 

Handbook Version 1.0 where the 

program or service has not been rereviewed under Handbook Version 2.0, 

a study re-review would be conducted 

under Handbook Version 1.0. For a 

program or service where a program or 

service rating has been assigned using 

Handbook Version 2.0, study re-reviews 

would be conducted using Handbook 

Version 2.0. This policy is consistent 

with other Federal evidence 

clearinghouses with multiple handbook 

versions (e.g., HomVEE). The Prevention 

Services Clearinghouse’s intention is 

that the emergence of substantial new 

evidence that has the potential to 

change program or service ratings (e.g., 

a newly published study) should be 

addressed through a program or service 

re-review. Similarly, cases where 

application of Handbook Version 2.0 

standards to a study reviewed under 

Handbook Version 1.0 could affect the 

program or service rating are intended 

to be addressed through a program or 

service re-review. Study re-reviews are 

intended to be limited solely to 

addressing missing information or errors 

in studies already reviewed. 

3.9.3 Manual Citation Updates 

(Section 8.5.3) 

The Prevention Services 

Clearinghouse recognizes that program 

or service manuals may be updated in 

the course of time after a review of a 

program or service has been published. 

Should a new manual edition (as 

defined in Section 2.3.2) be published, 

the public may request consideration of 

an update to the manual citation used 

for the program or service as outlined in 

Section 8.5.3 of the draft Handbook 

Version 2.0. If updated manual editions 

do not have substantive modifications 

or adaptations from the manual 

reviewed (per the criteria specified in 

Section 2.3), a manual citation may be 

updated to reflect that a newer manual 

edition is in active use that is 

substantively similar to the original 

primary manual selected for the review 

of the program or service. In considering 

whether an update to a manual citation 

is warranted, the Prevention Services 

Clearinghouse must have sufficient 

information available to be able to apply 

the procedures specified in Section 2.3 

for determining whether any substantive 

adaptations are present in the newer 

manual edition compared to the original 

edition reviewed. If the manual citation 

is updated, the original manual citation 

used to conduct the review of evidence 

for the program or service will also be 

noted for clarity. 

4.0 Timeline for the Clearinghouse To 

Apply New Standards and Procedures 

The Prevention Services 

Clearinghouse proposes to apply the 

standards and procedures upon 

publication of a final Handbook Version 

2.0. The public will be clearly notified 

on the Prevention Services 

Clearinghouse website and via other 

avenues (e.g., email to subscribers to the 

Prevention Services Clearinghouse 

email list) when the final published 

Handbook Version 2.0 will go into effect 

for reviewing programs and services. 

Per the procedures in Chapters 7 and 

8 of the draft Handbook Version 2.0, all 

existing program and service ratings 

established under Handbook Version 

1.0 will remain in effect until such time 

that a program or service re-review is 

conducted of a program or service under 

Handbook Version 2.0. 

5.0 Request for Information (RFI) 

To facilitate the review of 

submissions, please identify the 

chapter, section, and/or page number of 

the draft Handbook of Standards and 

Procedures, Version 2.0 (https:// 


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 204 / Tuesday, October 24, 2023 / Notices 73029 

preventionservices.acf.hhs.gov/ 

resources/comment-draft-handbook) 

that your comments address. This RFI is 

for information and planning purposes 

only and should not be construed as a 

solicitation or as an obligation on the 

part of ACF or HHS. For more 

information about the Prevention 

Services Clearinghouse, visit: https://

preventionservices.acf.hhs.gov. 

Lauren Supplee, 

Deputy Assistant Secretary for Planning, 

Research, and Evaluation. 

[FR Doc. 2023–23391 Filed 10–23–23; 8:45 am] 

BILLING CODE 4184–01–P 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Administration for Community Living 

Agency Information Collection 

Activities; Proposed Collection; 

Comment Request; Alzheimer’s and 

Dementia Program Data Reporting 

Tool (ADP–DRT) OMB Control Number 

0985–0022 

AGENCY: Administration for Community 

Living, HHS. 

ACTION: Notice. 

SUMMARY: The Administration for 

Community Living (ACL) is announcing 

an opportunity for the public to 

comment on the proposed collection of 

information listed above. Under the 

Paperwork Reduction Act of 1995 (the 

PRA), Federal agencies are required to 

publish a notice in the Federal Register 

concerning each proposed collection of 

information, including each proposed 

extension of an existing collection of 

information, and to allow 60 days for 

public comment in response to the 

notice. This notice solicits comments on 

the proposed Revision for the 

information collection requirements 

related to Alzheimer’s and Dementia 

Program Data Reporting Tool (ADP– 

DRT). 

DATES: Comments on the collection of 

information must be submitted 

electronically by 11:59 p.m. (EST) or 

postmarked by December 26, 2023. 

ADDRESSES: Submit electronic 

comments on the collection of 

information to: Erin Long (erin.long@

acl.hhs.gov). Address written comments 

on the collection of information to 

Administration for Community Living, 

Washington, DC 20201, Attention: Erin 

Long PRA comments Alzheimer’s and 

Dementia Program Data Reporting Tool 

(ADP–DRT). 

FOR FURTHER INFORMATION CONTACT: Erin 

Long, erin.long@acl.hhs.gov, 202–795– 

7389. 

SUPPLEMENTARY INFORMATION: Under the 

PRA (44 U.S.C. 3501–3520), Federal 

agencies must obtain approval from the 

Office of Management and Budget 

(OMB) for each collection of 

information they conduct or sponsor. 

‘‘Collection of information’’ is defined 

as and includes agency requests or 

requirements that members of the public 

submit reports, keep records, or provide 

information to a third party. The PRA 

requires Federal agencies to provide a 

60-day notice in the Federal Register 

concerning each proposed collection of 

information, including each proposed 

extension of an existing collection of 

information, before submitting the 

collection to OMB for approval. To 

comply with this requirement, ACL is 

publishing a notice of the proposed 

collection of information set forth in 

this document. With respect to the 

following collection of information, 

ACL invites comments on our burden 

estimates or any other aspect of this 

collection of information, including: 

(1) whether the proposed collection of 

information is necessary for the proper 

performance of ACL’s functions, 

including whether the information will 

have practical utility; 

(2) ways to enhance the quality, 

utility, and clarity of the information to 

be collected; 

(3) accuracy of ACL’s estimate of the 

burden of the proposed collection of 

information, including the validity of 

the methodology and assumptions used 

to determine burden estimates; 

And (4) ways to minimize the burden 

of the collection of information on 

respondents, including through the use 

of automated collection techniques 

when appropriate, and other forms of 

information technology. 

The Older American’s Act requires 

ACL to evaluate ‘‘demonstration 

projects that support the objectives of 

this Act, including activities to bring 

effective demonstration projects to scale 

with a prioritization of projects that 

address the needs of underserved 

populations, and promote partnerships 

among aging services, community-based 

organizations, and Medicare and 

Medicaid providers, plans, and health 

(including public health) systems. 

(Section 201 (42 U.S.C. 3011) Sec. 127. 

Research and Evaluation). 

To fulfill the evaluation requirements 

and allow for optimal federal and statelevel management of ACL’s Alzheimer’s 

Disease Program, specific information 

must be collected from grantees. 

The current reporting tool is set to 

expire 12/31/2023. The Alzheimer’s and 

Dementia Program (ADP) Project Officer 

has reviewed the current data collection 

procedures to ensure the acceptability of 

these items as appropriate and thorough 

evaluation of the program, while 

minimizing burden for grantees. 

The result of this process is the 

proposed modifications to the existing 

data collection tool. ACL is aware that 

different grantees have different data 

collection capabilities. It is understood 

that, following the approval of the 

modified data collection tool, ACL will 

work with its grantees to offer regular 

training to ensure minimal burden. 

To support alignment with Executive 

Order 13985 on Advancing Racial 

Equity and Support for Underserved 

Communities Through the Federal 

Government, Executive Order 14075 on 

Advancing Equality for Lesbian, Gay, 

Bisexual, Transgender, Queer, and 

Intersex Individuals, and Executive 

Order 13988 on Preventing and 

Combating Discrimination on the Basis 

of Gender Identity and Sexual 

Orientation, ACL is adding three sexual 

orientation and gender identity (SOGI) 

items to the ADP–DRT. Understanding 

these disparities can and should lead to 

improved service delivery for ACL’s 

programs and populations served. 

The proposed data collection tools 

may be found on the ACL website for 

review at https://www.acl.gov/aboutacl/public-input. 

Estimated Program Burden: 

ACL estimates the burden associated 

with this collection of information as 

follows: 

Type of respondent Form name Number of 

respondents 

Frequency of 

response 

Average time 

per response 

(in hours) 

Total burden 

hours 

(annual) 

Grantee ............................................. ADSSP–DRT .................................... 69 2 6.64 916.32 

Total ........................................... ........................................................... ........................ ........................ ........................ 916.32 


ddrumheller on DSK120RN23PROD with NOTICES1