44728 Federal Register / Vol. 86, No. 154 / Friday, August 13, 2021 / Notices 

most message testing. This package 

includes generic questions and formats 

that can be used to develop health 

message testing data collection 

instruments. These include a list of 

screening questions, comprised of 

demographic and introductory 

questions, along with other questions 

that can be used to create a mix of 

relevant questions for each proposed 

message testing data collection method. 

Programs may request to use additional 

questions if needed. 

Message testing questions will focus 

on issues such as comprehension, 

impressions, personal relevance, 

content and wording, efficacy of 

response, channels, and spokesperson/ 

sponsor. Such information will enable 

message developers to enhance the 

effectiveness of messages for intended 

audiences. 

Data collection methods proposed for 

HMTS include intercept interviews, 

telephone interviews, focus groups, 

online surveys, and cognitive 

interviews. In almost all instances, data 

will be collected by outside 

organizations under contract with CDC. 

For many years CDC programs have 

used HMTS to test and refine message 

concepts and test draft materials for 

clarity, salience, appeal, and 

persuasiveness to target audiences. 

Having this generic clearance available 

has enabled them to test their 

information and get critical health 

information from the public quickly. 

Over the last three years, more than 32 

messages have been tested using this 

clearance. Examples include: 

CDC Older Adult Injury Prevention 

Creative Campaign—Survey. This health 

communication campaign aimed to 

support and expand upon CDC’s older 

adult injury prevention efforts and to 

raise awareness among older adults and 

their caregivers about preventable 

injuries that disproportionately impact 

them, steps to reduce their risk of 

injuries, and increase education about 

risk factors. Information collected can 

assist in the most effective use of CDC 

communication resources and 

opportunities by assessing clarity, 

appeal, persuasiveness and effectiveness 

of campaign material and 

advertisements (i.e., poster or video 

advertisement). 

The Division of Tuberculosis 

Elimination (DTBE) obtained OMB 

approval through HMTS for Health 

Communications Testing for Latent 

Tuberculosis Infections (LTBI) 

Campaign for CDC’s National Center for 

HIV/AIDS, Viral Hepatitis, STD, and TB 

Prevention. This collection was used to 

inform NCHHSTP/DTBE’s future public 

service campaign efforts targeted to 

consumers at high-risk for LTBI, and the 

providers who serve them. This 

information collection activity is 

essential because it will provide CDC 

with effective messages for 

communicating about this disease and 

infection to motivate at-risk consumers 

to get preventive screening and, if 

infected, treatment, and to motivate 

healthcare providers to encourage 

testing and early detection. 

The Division of Diabetes Translation 

obtained OMB approval through HMTS 

for Message Testing for Diabetes SelfManagement Education and Support 

(DSMES) Marketing Support: Card Sort 

Activity. Findings from this message 

testing effort were used by the Centers 

for Disease Control and Prevention’s 

(CDC) Division of Diabetes Translation 

(DDT) to inform how best to 

communicate with key audiences about 

DSMES services. Specifically, 

information about which attributes of 

DSMES services are most important to 

each audience will be identified and 

will serve as the basis for messages 

developed to promote DSMES services. 

This work will help increase the 

likelihood that messages will resonate 

and be understood as intended. 

Over 27,696 respondents were 

queried and over 6,100 burden hours 

used during this time period. Because 

the availability of this ICR has been so 

critical to programs in disseminating 

their materials and information to the 

public in a timely manner, OADC is 

requesting a three-year extension of this 

information collection. Based on 

anticipated use, CDC requests approval 

for an estimated 2,470 annual burden 

hours (7,410 over the course of the 

three-year approval). There is no cost to 

the respondents other than their time. 

ESTIMATED ANNUALIZED BURDEN HOURS

Type of respondents Form name Number of 

respondents 

Number of 

responses per 

respondent 

Average 

burden per 

response 

(in hours) 

Public Health Professionals, Health Care Providers, State and Local Public Health Officials, Emergency Responders, General 

Public.

Moderator’s Guides, Eligibility Screeners, 

Interview Guides, Opinion Surveys, Consent Forms.

18,525 1 8/60 

Jeffrey M. Zirger, 

Lead, Information Collection Review Office, 

Office of Scientific Integrity, Office of Science, 

Centers for Disease Control and Prevention. 

[FR Doc. 2021–17350 Filed 8–12–21; 8:45 am] 

BILLING CODE 4163–18–P 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Centers for Disease Control and 

Prevention 

[30Day–21–0222] 

Agency Forms Undergoing Paperwork 

Reduction Act Review 

In accordance with the Paperwork 

Reduction Act of 1995, the Centers for 

Disease Control and Prevention (CDC) 

has submitted the information 

collection request titled The 

Collaborating Center for Questionnaire 

Design and Evaluation Research 

(CCQDER) to the Office of Management 

and Budget (OMB) for review and 

approval. CDC previously published a 

‘‘Proposed Data Collection Submitted 

for Public Comment and 

Recommendations’’ notice on May 26, 

2021 to obtain comments from the 

public and affected agencies. CDC 

received no comments related to the 

previous notice. This notice serves to 

allow an additional 30 days for public 

and affected agency comments. 

CDC will accept all comments for this 

proposed information collection project. 

The Office of Management and Budget 

is particularly interested in comments 

that: 


jbell on DSKJLSW7X2PROD with NOTICES
Federal Register / Vol. 86, No. 154 / Friday, August 13, 2021 / Notices 44729 

(a) Evaluate whether the proposed 

collection of information is necessary 

for the proper performance of the 

functions of the agency, including 

whether the information will have 

practical utility; 

(b) Evaluate the accuracy of the 

agencies estimate of the burden of the 

proposed collection of information, 

including the validity of the 

methodology and assumptions used; 

(c) Enhance the quality, utility, and 

clarity of the information to be 

collected; 

(d) Minimize the burden of the 

collection of information on those who 

are to respond, including, through the 

use of appropriate automated, 

electronic, mechanical, or other 

technological collection techniques or 

other forms of information technology, 

e.g., permitting electronic submission of 

responses; and 

(e) Assess information collection 

costs. 

To request additional information on 

the proposed project or to obtain a copy 

of the information collection plan and 

instruments, call (404) 639–7570. 

Comments and recommendations for the 

proposed information collection should 

be sent within 30 days of publication of 

this notice to www.reginfo.gov/public/ 

do/PRAMain. Find this particular 

information collection by selecting 

‘‘Currently under 30-day Review—Open 

for Public Comments’’ or by using the 

search function. Direct written 

comments and/or suggestions regarding 

the items contained in this notice to the 

Attention: CDC Desk Officer, Office of 

Management and Budget, 725 17th 

Street NW, Washington, DC 20503 or by 

fax to (202) 395–5806. Provide written 

comments within 30 days of notice 

publication. 

Proposed Project 

The Collaborating Center for 

Questionnaire Design and Evaluation 

Research (CCQDER) (OMB Control No. 

0920–0222, Exp. 08/31/2021)— 

Revision—National Center for Health 

Statistics (NCHS), Centers for Disease 

Control and Prevention (CDC). 

Background and Brief Description 

Section 306 of the Public Health 

Service (PHS) Act (42 U.S.C. 242k), as 

amended, authorizes that the Secretary 

of Health and Human Services (DHHS), 

acting through NCHS, shall undertake 

and support (by grant or contract) 

research, demonstrations, and 

evaluations respecting new or improved 

methods for obtaining current data to 

support statistical and epidemiological 

activities for the purpose of improving 

the effectiveness, efficiency, and quality 

of health services in the United States. 

The Collaborating Center for 

Questionnaire Design and Evaluation 

Research (CCQDER) is the focal point 

within NCHS for questionnaire and 

survey development, pre-testing, and 

evaluation activities for CDC surveys 

such as: The National Survey of Family 

Growth (NSFG), the Research and 

Development Survey (RANDS) 

(including RANDS COVID), and other 

federally sponsored surveys. 

The CCQDER and other NCHS 

programs conduct cognitive interviews, 

focus groups, in-depth or ethnographic 

interviews, usability tests, field tests/ 

pilot interviews, and experimental 

research in laboratory and field settings, 

both for applied questionnaire 

development and evaluation, as well as 

more basic research on measurement 

errors and survey response. 

Various techniques to evaluate 

interviewer-administered, selfadministered, telephone, ComputerAssisted Personal Interviewing (CAPI), 

Computer Assisted Self-Interviewing 

(CASI), Audio Computer-Assisted SelfInterviewing (ACASI), and web-based 

questionnaires are used. 

The most common questionnaire 

evaluation method is the cognitive 

interview. These evaluations are 

conducted by the CCQDER. The 

interview structure consists of 

respondents first answering a draft 

survey question and then providing 

textual information to reveal the 

processes involved in answering the test 

question. Specifically, cognitive 

interview respondents are asked to 

describe how and why they answered 

the question as they did. Through the 

interviewing process, various types of 

question-response problems that would 

not normally be identified in a 

traditional survey interview, such as 

interpretive errors and recall accuracy, 

are uncovered. By conducting a 

comparative analysis of cognitive 

interviews, it is also possible to 

determine whether particular 

interpretive patterns occur within 

particular sub-groups of the population. 

Interviews are generally conducted in 

small rounds totaling 40–100 

interviews; ideally, the questionnaire is 

re-worked between rounds, and 

revisions are tested iteratively until 

interviews yield relatively few new 

insights. 

Cognitive interviewing is inexpensive 

and provides useful data on 

questionnaire performance while 

minimizing respondent burden. 

Cognitive interviewing offers a detailed 

depiction of meanings and processes 

used by respondents to answer 

questions, a process that ultimately 

produces the survey data. As such, the 

method offers an insight that can 

transform understanding of question 

validity and response error. 

Documented findings from these studies 

represent tangible evidence of how the 

question performs. Such documentation 

also serves CDC data users, allowing 

them to be critical users in their 

approach and application of the data. 

In addition to cognitive interviewing, 

a number of other qualitative and 

quantitative methods are used to 

investigate and research measurement 

errors and the survey response process. 

These methods include conducting 

focus groups, usability tests, in-depth or 

ethnographic interviews, and the 

administration and analysis of questions 

in both representative and nonrepresentative field tests. Focus groups 

are conducted by the CCQDER. Focus 

groups are group interviews whose 

primary purpose is to elicit the basic 

sociocultural understandings and 

terminology that form the basis of 

questionnaire design. Each group 

typically consists of one moderator and 

4–10 participants, depending on the 

research question. In-depth or 

ethnographic interviews are one-on-one 

interviews designed to elicit the 

understandings or terminology that are 

necessary for question design, as well as 

to gather detailed information that can 

contribute to the analysis of both 

qualitative and quantitative data. 

Usability tests are typically one-on-one 

interviews that are used to determine 

how a given survey or information 

collection tool functions in the field, 

and how the mode and layout of the 

instrument itself may contribute to 

survey response error and the survey 

response process. 

In addition to these qualitative 

methods, NCHS also uses various tools 

to obtain quantitative data, which can 

be analyzed alone or analyzed alongside 

qualitative data to give a much fuller 

accounting of the survey response 

process. For instance, phone, internet, 

mail, and in-person follow-up 

interviews of previous NCHS survey 

respondents may be used to test the 

validity of survey questions and 

questionnaires and to obtain more 

detailed information that cannot be 

gathered on the original survey. 

Additionally, field or pilot tests may be 

conducted on both representative and 

non-representative samples, including 

those obtained from commercial survey 

and web panel vendors. Beyond looking 

at traditional measures of survey errors 

(such as item missing rates, nonresponse, and don’t know rates), these 

pilot tests can be used to run 


jbell on DSKJLSW7X2PROD with NOTICES
44730 Federal Register / Vol. 86, No. 154 / Friday, August 13, 2021 / Notices 

experimental designs in order to capture 

how different questions function in a 

field setting. Similar methodology has 

been adopted by other federal agencies, 

as well as by academic and commercial 

survey organizations. 

In 2021–2024, NCHS/CCQDER staff 

plan to continue research on methods 

evaluation and general questionnaire 

design research. We envision that over 

the next three years, NCHS/CCQDER 

will work collaboratively with survey 

researchers from universities and other 

Federal agencies to define and examine 

several research areas, including, but 

not limited to: (1) Differences between 

face-to-face, telephone, and virtual/ 

video over internet cognitive 

interviewing, (2) effectiveness of 

different approaches to cognitive 

interviewing, such as concurrent and 

retrospective probing, (3) reactions of 

both survey respondents and survey 

interviewers to the use of ComputerAssisted Personal Interviewing (CAPI), 

Audio Computer-Assisted SelfInterview (ACASI), video over internet/ 

virtual, (4) social, cultural and linguistic 

factors in the question response process, 

and (5) recruitment and respondent 

participation at varying levels of 

incentive in an effort to establish 

empirical evidence regarding 

remuneration and coercion. Procedures 

for each of these studies will be similar 

to those applied in the usual testing of 

survey questions. For example, 

questionnaires that are of current 

interest (such as RANDS and NIOSH) 

may be evaluated using several of the 

techniques described above. In addition, 

different versions of a survey question 

will be developed, and the variants then 

administered to separate groups of 

respondents in order to study the 

cognitive processes that account for the 

differences in responses obtained across 

different versions. 

These studies will be conducted 

either by CCQDER staff, DHHS staff, or 

NCHS contractors who are trained in 

cognitive interviewing techniques. The 

results of these studies will be applied 

to our specific questionnaire 

development activities in order to 

improve the methods that we use to 

conduct questionnaire testing, and to 

guide questionnaire design in general. 

OMB approval is requested for three 

years. Participation is voluntary. We are 

requesting 9,455 annualized hours, 

totaling 28,365 over three years. This is 

an increase of 1,672 hours per year or 

5,016 hours over three years. The 

requested increases are due to an 

anticipated increase in the number and 

size of projects being undertaken. 

ESTIMATED ANNUALIZED BURDEN TABLE

Type of respondent Form name Number of 

respondents 

Number of 

responses per 

respondent 

Average 

burden per 

response 

(in hours) 

Individuals or households ............................... Eligibility Screeners ........................................ 4,400 1 5/60 

Individuals or households ............................... Developmental Questionnaires ...................... 8,750 1 55/60 

Individuals or households ............................... Respondent Data Collection Sheet ................ 8,750 1 5/60 

Individuals or households ............................... Focus Group Documents ............................... 225 1 90/60 

Jeffrey M. Zirger, 

Lead, Information Collection Review Office, 

Office of Scientific Integrity, Office of Science, 

Centers for Disease Control and Prevention. 

[FR Doc. 2021–17349 Filed 8–12–21; 8:45 am] 

BILLING CODE 4163–18–P 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Centers for Medicare & Medicaid 

Services 

[Document Identifier CMS–1880 and CMS– 

1856] 

Agency Information Collection 

Activities: Submission for OMB 

Review; Comment Request 

AGENCY: Centers for Medicare & 

Medicaid Services, Health and Human 

Services (HHS). 

ACTION: Notice. 

SUMMARY: The Centers for Medicare & 

Medicaid Services (CMS) is announcing 

an opportunity for the public to 

comment on CMS’ intention to collect 

information from the public. Under the 

Paperwork Reduction Act of 1995 

(PRA), federal agencies are required to 

publish notice in the Federal Register 

concerning each proposed collection of 

information, including each proposed 

extension or reinstatement of an existing 

collection of information, and to allow 

a second opportunity for public 

comment on the notice. Interested 

persons are invited to send comments 

regarding the burden estimate or any 

other aspect of this collection of 

information, including the necessity and 

utility of the proposed information 

collection for the proper performance of 

the agency’s functions, the accuracy of 

the estimated burden, ways to enhance 

the quality, utility, and clarity of the 

information to be collected, and the use 

of automated collection techniques or 

other forms of information technology to 

minimize the information collection 

burden. 

DATES: Comments on the collection(s) of 

information must be received by the 

OMB desk officer by September 13, 

2021. 

ADDRESSES: Written comments and 

recommendations for the proposed 

information collection should be sent 

within 30 days of publication of this 

notice to www.reginfo.gov/public/do/ 

PRAMain. Find this particular 

information collection by selecting 

‘‘Currently under 30-day Review—Open 

for Public Comments’’ or by using the 

search function. 

To obtain copies of a supporting 

statement and any related forms for the 

proposed collection(s) summarized in 

this notice, you may make your request 

using one of following: 

1. Access CMS’ website address at 

website address at: https://

www.cms.gov/Regulations-andGuidance/Legislation/ 

PaperworkReductionActof1995/PRAListing.html. 

FOR FURTHER INFORMATION CONTACT: 

William Parham at (410) 786–4669. 

SUPPLEMENTARY INFORMATION: Under the 

Paperwork Reduction Act of 1995 (PRA) 

(44 U.S.C. 3501–3520), federal agencies 

must obtain approval from the Office of 

Management and Budget (OMB) for each 

collection of information they conduct 

or sponsor. The term ‘‘collection of 

information’’ is defined in 44 U.S.C. 

3502(3) and 5 CFR 1320.3(c) and 

includes agency requests or 

requirements that members of the public 

submit reports, keep records, or provide 

information to a third party. Section 

3506(c)(2)(A) of the PRA (44 U.S.C. 

3506(c)(2)(A)) requires federal agencies 

to publish a 30-day notice in the 

Federal Register concerning each 


jbell on DSKJLSW7X2PROD with NOTICES