Document ID: FDA-2023-N-0894-0001
Agency: fda
Document Type: Notice
Title: Agency Information Collection Activities; Proposed Collection; Comment Request; The Real Cost Monthly Implementation Assessment
Posted Date: 2023-04-27T04:00Z

[Federal Register Volume 88, Number 81 (Thursday, April 27, 2023)]
[Notices]
[Pages 25660-25663]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2023-08945]

-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Food and Drug Administration

[Docket No. FDA-2023-N-0894]

Agency Information Collection Activities; Proposed Collection; 
Comment Request; The Real Cost Monthly Implementation Assessment

AGENCY: Food and Drug Administration, HHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Food and Drug Administration (FDA or Agency) is announcing 
an opportunity for public comment on the proposed collection of certain 
information by the Agency.

[[Page 25661]]

Under the Paperwork Reduction Act of 1995 (PRA), Federal Agencies are 
required to publish a notice in the Federal Register concerning each 
proposed collection of information and to allow 60 days for public 
comment in response to the notice. This notice solicits comments on a 
proposed information collection entitled ``The Real Cost Monthly 
Implementation Assessment.''

DATES: Either electronic or written comments on the collection of 
information must be submitted by June 26, 2023.

ADDRESSES: You may submit comments as follows. Please note that late, 
untimely filed comments will not be considered. The https://www.regulations.gov electronic filing system will accept comments until 
11:59 p.m. Eastern Time at the end of June 26, 2023. Comments received 
by mail/hand delivery/courier (for written/paper submissions) will be 
considered timely if they are postmarked or the delivery service 
acceptance receipt is on or before that date.

Electronic Submissions

    Submit electronic comments in the following way:
     Federal eRulemaking Portal: https://www.regulations.gov. 
Follow the instructions for submitting comments. Comments submitted 
electronically, including attachments, to https://www.regulations.gov 
will be posted to the docket unchanged. Because your comment will be 
made public, you are solely responsible for ensuring that your comment 
does not include any confidential information that you or a third party 
may not wish to be posted, such as medical information, your or anyone 
else's Social Security number, or confidential business information, 
such as a manufacturing process. Please note that if you include your 
name, contact information, or other information that identifies you in 
the body of your comments, that information will be posted on https://www.regulations.gov.
     If you want to submit a comment with confidential 
information that you do not wish to be made available to the public, 
submit the comment as a written/paper submission and in the manner 
detailed (see ``Written/Paper Submissions'' and ``Instructions'').

Written/Paper Submissions

    Submit written/paper submissions as follows:
     Mail/Hand Delivery/Courier (for written/paper 
submissions): Dockets Management Staff (HFA-305), Food and Drug 
Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852.
     For written/paper comments submitted to the Dockets 
Management Staff, FDA will post your comment, as well as any 
attachments, except for information submitted, marked and identified, 
as confidential, if submitted as detailed in ``Instructions.''
    Instructions: All submissions received must include the Docket No. 
FDA-2023-N-0894 for ``The Real Cost Monthly Implementation 
Assessment.'' Received comments, those filed in a timely manner (see 
ADDRESSES), will be placed in the docket and, except for those 
submitted as ``Confidential Submissions,'' publicly viewable at https://www.regulations.gov or at the Dockets Management Staff between 9 a.m. 
and 4 p.m., Monday through Friday, 240-402-7500.
     Confidential Submissions--To submit a comment with 
confidential information that you do not wish to be made publicly 
available, submit your comments only as a written/paper submission. You 
should submit two copies total. One copy will include the information 
you claim to be confidential with a heading or cover note that states 
``THIS DOCUMENT CONTAINS CONFIDENTIAL INFORMATION.'' The Agency will 
review this copy, including the claimed confidential information, in 
its consideration of comments. The second copy, which will have the 
claimed confidential information redacted/blacked out, will be 
available for public viewing and posted on https://www.regulations.gov. 
Submit both copies to the Dockets Management Staff. If you do not wish 
your name and contact information to be made publicly available, you 
can provide this information on the cover sheet and not in the body of 
your comments and you must identify this information as 
``confidential.'' Any information marked as ``confidential'' will not 
be disclosed except in accordance with 21 CFR 10.20 and other 
applicable disclosure law. For more information about FDA's posting of 
comments to public dockets, see 80 FR 56469, September 18, 2015, or 
access the information at: https://www.govinfo.gov/content/pkg/FR-2015-09-18/pdf/2015-23389.pdf.
    Docket: For access to the docket to read background documents or 
the electronic and written/paper comments received, go to https://www.regulations.gov and insert the docket number, found in brackets in 
the heading of this document, into the ``Search'' box and follow the 
prompts and/or go to the Dockets Management Staff, 5630 Fishers Lane, 
Rm. 1061, Rockville, MD 20852, 240-402-7500.

FOR FURTHER INFORMATION CONTACT: JonnaLynn Capezzuto, Office of 
Operations, Food and Drug Administration, Three White Flint North, 10A-
12M, 11601 Landsdown St., North Bethesda, MD 20852, 301-796-3794, 
[email protected].

SUPPLEMENTARY INFORMATION: Under the PRA (44 U.S.C. 3501-3521), Federal 
Agencies must obtain approval from the Office of Management and Budget 
(OMB) for each collection of information they conduct or sponsor. 
``Collection of information'' is defined in 44 U.S.C. 3502(3) and 5 CFR 
1320.3(c) and includes Agency requests or requirements that members of 
the public submit reports, keep records, or provide information to a 
third party. Section 3506(c)(2)(A) of the PRA (44 U.S.C. 3506(c)(2)(A)) 
requires Federal Agencies to provide a 60-day notice in the Federal 
Register concerning each proposed collection of information before 
submitting the collection to OMB for approval. To comply with this 
requirement, FDA is publishing notice of the proposed collection of 
information set forth in this document.
    With respect to the following collection of information, FDA 
invites comments on these topics: (1) whether the proposed collection 
of information is necessary for the proper performance of FDA's 
functions, including whether the information will have practical 
utility; (2) the accuracy of FDA's estimate of the burden of the 
proposed collection of information, including the validity of the 
methodology and assumptions used; (3) ways to enhance the quality, 
utility, and clarity of the information to be collected; and (4) ways 
to minimize the burden of the collection of information on respondents, 
including through the use of automated collection techniques, when 
appropriate, and other forms of information technology.

The Real Cost Monthly Implementation Assessment

OMB Control Number 0910-NEW

    This information collection supports the development and 
implementation of FDA public education campaigns related to tobacco 
use. To reduce the public health burden of tobacco use in the United 
States and educate the public--especially young people--about the 
dangers of tobacco use, the FDA Center for Tobacco Products (CTP) is 
developing and implementing multiple public education campaigns.
    FDA launched ``The Real Cost'' in February 2014, seeking to reduce 
tobacco use among at-risk youth ages 12 to 17 in the United States who 
are open

[[Page 25662]]

to smoking cigarettes and/or using electronic nicotine delivery systems 
(ENDS) products, or who have already experimented with cigarettes and/
or ENDS products. Complementary evaluation studies, including the 
``Evaluation of FDA's Public Education Campaign on Teen Tobacco 
(ExPECTT),'' were implemented to measure awareness of ``The Real Cost'' 
paid media campaign among youth ages 12 to 17 in the United States, and 
to understand how awareness is related to change in key outcomes.
    Although outcome evaluation studies of ``The Real Cost'' have and 
continue to assess the impact of awareness on outcomes, no studies have 
sought to assess the implementation of ``The Real Cost.'' As FDA 
continues to increase the presence of ``The Real Cost'' on digital 
channels (e.g., Hulu, YouTube, Instagram), the need for an 
implementation evaluation has become clear as these messages are 
received by the target audience on digital channels differently 
compared to how the messages are received on broadcast channels. Before 
the migration of campaign ads to digital channels, ads from ``The Real 
Cost'' were primarily aired on broadcast TV. In the broadcast space, 
for people to avoid receiving the message, they needed to be proactive 
(e.g., finding the remote to change the channel or leaving the room). 
In the digital space, however, people need to be proactive to watch the 
full message, like stopping scrolling on social media or skipping the 
ad on YouTube. Assessment of this information is integral to 
understanding self-reported ad awareness levels, as well as how our 
audience experiences and processes the ads as they are airing in a 
digital, real-world setting.
    Therefore, we propose a study to help us understand, in a digital 
setting, how youth experience the messages, how they engage with 
messages, the extent to which youth report being exposed to messages, 
and how youth process the messages. Studying exposure to ad messages as 
it naturally occurs in the real world can help us understand the points 
of connection--or disconnection--between the results of copy testing 
studies (which assess responses to the ads with forced exposure to 
them) and outcome evaluation findings (which are based on natural 
exposure to ads in the real world). Data gathered from this assessment 
will also provide the necessary and timely information to optimize 
campaign messages, the digital media buy (i.e., where, how, and when 
ads are shown), and creative rotations (i.e., which ads are shown).
    ``The Real Cost'' Monthly Implementation Assessment is a repeated 
cross-sectional survey that will be conducted using web-based surveys 
that are self-administered on personal computers or web enabled mobile 
devices to collect rapid data on ``The Real Cost'' ads. Data from up to 
2,000 youth in the United States will be collected each month for up to 
24 months. To be eligible, youth and young adults must be between the 
ages of 12 to 20 and have not taken ``The Real Cost'' Monthly 
Implementation Assessment survey within the past 3 months. We will use 
an Ipsos Knowledge Networks Panel to collect data on ``The Real Cost'' 
ads. This design offers flexibility to assess new ad messages, as they 
air across various digital platforms, examine their performance over 
time, as well as the ability to pivot and add new survey measures as 
necessary. Monthly data will also allow us to obtain timely information 
on ad awareness, perceived effectiveness, as well as on youth attention 
and processing of the ads.
    The purpose of FDA's ``The Real Cost'' Monthly Implementation 
Assessment is to evaluate the following key components about ``The Real 
Cost'' ads:
     Awareness of ``The Real Cost'' ads.
     Attention behaviors when seeing ``The Real Cost'' ads.
     Processing of ``The Real Cost'' ads, including:
    [cir] Engagement with the ads.
    [cir] Main message comprehension.
    [cir] Acceptance and/or rejection of the ads.
     Perceived effectiveness of ``The Real Cost'' ads.
     Belief and knowledge tracking of ``The Real Cost'' ads.
    In addition to the above components, the survey will ask 
participants to report on tobacco use and other psychographic and 
demographic items. The time frame that the survey items will ask about 
for ad awareness (i.e., past 30 days or past week) will depend on 
several factors, including how long the ad was on air. The survey will 
take an average of approximately 25 minutes to complete per 
participant. As the survey items are tested, any irrelevant items will 
be cut as necessary. Ad creative for both vaping and cigarette products 
will be assessed; therefore, two similar surveys (one on ENDS-focused 
ads and one on cigarette-focused ads) will be fielded as appropriate, 
but not within the same month. In support of the provisions of the 
Tobacco Control Act that require FDA to protect the public health and 
to reduce tobacco use by minors, FDA requests OMB approval to collect 
information to evaluate CTP's public education campaign ``The Real 
Cost'' through the Monthly Implementation Assessment.
    FDA estimates the burden of this collection of information as 
follows:

                                                     Table 1--Estimated Annual Reporting Burden \1\
--------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                   Number of
          Type of respondent/activity              Number of     responses per       Total             Average burden per response          Total hours
                                                  respondents     respondent       responses
--------------------------------------------------------------------------------------------------------------------------------------------------------
Parent/Young Adult Screener...................       2,688,000               1       2,688,000  0.05 (3 minutes)........................         134,400
Parent Permission.............................       2,016,000               1       2,016,000  0.05 (3 minutes)........................         100,800
Youth Screener................................       2,016,000               1       2,016,000  0.05 (3 minutes)........................         100,800
Youth Assent..................................          36,000               1          36,000  0.05 (3 minutes)........................           1,800
Young Adult Consent...........................          12,000               1          12,000  0.05 (3 minutes)........................             600
Online Survey.................................          48,000               1          48,000  0.42 (25 minutes).......................          20,160
Invitation Email/Reminder Emails/Thank you              48,000               1          48,000  0.42 (25 minutes).......................          20,160
 Email.
                                               ---------------------------------------------------------------------------------------------------------
    Total.....................................  ..............  ..............  ..............  ........................................         378,720
--------------------------------------------------------------------------------------------------------------------------------------------------------
\1\ There are no capital costs or operating and maintenance costs associated with this collection of information.

    Data collection for the Monthly Implementation Assessment will 
consist of administering a monthly survey to participants aged 12 to 20 
over the course of 2 years (24 months). We expect the screening process 
(3 minutes

[[Page 25663]]

per response) to yield an approximate 2.3 to one ratio of eligible 
participants. We will need to screen approximately 112,000 potential 
parents and young adults each month (resulting in 2,688,000 screeners) 
over the study period. Since the eligible age for data collection is 12 
to 20 years old, we intend to screen parents of eligible youth, as well 
as young adults. Parents of the youth participants determined to be 
eligible through the screener will provide parent permission (3 minutes 
per response). We estimate that 2,016,000 of the parents who complete 
the screener will provide their permission for their youth to complete 
the online survey (approximately 75 percent of the 2,688,000 screened). 
Eligible youth (2,016,000) will provide their assent (3 minutes per 
response) to participate in the online survey (25 minutes per 
response). Participants that are 18 to 20 (19 to 20 in Alabama and 
Nebraska in accordance with state law) will complete the screener for 
themselves and provide their consent (3 minutes per response) to 
participate in the online survey. We estimate that approximately 25 
percent of the 48,000 completed surveys will come from young adults 
aged 18 to 20 (19 to 20 in Alabama and Nebraska).
    Over the course of the study period, we intend to survey 
approximately 2,000 youth and young adults ages 12 to 20 per month for 
24 months. From these completed screeners, we estimate that we will 
obtain data from 36,000 youth and 12,000 young adults. This will give 
us a total of 48,000 participants for the study. The survey will be 
repeated with a new cross-sectional sample approximately every month 
over a period of 24 months; however, some participants will complete 
more than one wave. These 48,000 respondents will receive an invitation 
email with a link to take the survey (4 minutes), six reminder emails 
(3 minutes each), and a thank you email (3 minutes) upon completion of 
the study for a total of 25 minutes for respondents to read and respond 
to the emails.

    Dated: April 20, 2023.
Lauren K. Roth,
Associate Commissioner for Policy.
[FR Doc. 2023-08945 Filed 4-26-23; 8:45 am]
BILLING CODE 4164-01-P