Document ID: EPA-HQ-OA-2007-0607-0004
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2020-08-12T04:00Z

Supporting Statement for

Information Collection Request No. 2255.01

 “Performance Measurement Reporting for Training 

and Education/Outreach” 

PART B OF THE SUPPORTING STATEMENT

Each expedited clearance request submitted to OMB will include a Part B
Document that provides a full description of the proposed approach for
collecting information, including data analysis methods, if any.  To
complete the document, assistance agreement recipients would refer to
the guidance in the reporting tools, including Attachment A (guidance on
collection strategies, including sample design) and Attachment B
(guidance on preparing the Part B Document).  These guidances suggest
best practices and recommendations for the design and implementation of
a successful survey.  The guidances offer user-friendly reference
materials and approaches on, for example, how to determine a minimum
acceptable sample size, maximize response rates, and obtain
representative responses, so that statistically valid results are
obtained.  Examples of acceptable approaches are also provided, to
minimize the level of burden imposed on assistance agreement recipients
and ensure that recipients understand the level of detail needed to
obtain OMB approval.  Any questions that arise can be directed to the
EPA Project Officer and/or the Gatekeeper for assistance.

The length and complexity of the Part B Document will depend on the
proposed survey approach.  If a sampling approach will be used instead
of a census, for example, more information is generally required in
order to explain how the representativeness of the sample will be
ensured.  If a particular section of the Part B Document is not
applicable to a particular collection, this will be indicated briefly.

1.	Survey Objectives, Key Variables, And Other Preliminaries

(1)	Survey Objectives

The Part B Document will describe the survey objectives.  The objectives
will be stated in the form of issues or questions for resolution. In
general, survey objectives will be limited to a reasonable number, based
on the assistance agreement recipient’s and EPA’s information needs.
 

(2)	Key Variables

The Part B Document will briefly describe the characteristics of the
individuals being surveyed that could affect the way in which they
respond to the survey questions.  The description will try to give the
reviewer a sense of how these variables might influence the survey
results.  However, if the key variables are not known, this will be
explained briefly (e.g., “insufficient information exists on
population characteristics”).  

(3)	Sampling Approach

This section will state whether a census or sampling approach will be
used. If a sampling approach is proposed, details will be presented in
accordance with the procedures laid out below.

(4)	Feasibility

The feasibility of the proposed survey effort will be described briefly.
 This entails a brief discussion of likely problems and how they will be
addressed.

2. 	Survey Design

This section will describe the mechanics of the survey design – from
how the respondent pool is selected to what type of questions will be
asked in the survey.  If a sampling approach is being proposed, all four
information parameters below will be addressed.  If a census is
proposed, only the last parameter will be addressed.

(1)	Target Population and Coverage

This section will briefly describe the overall group of individuals or
organizations that are the focus of the survey (i.e., the “target
population”).  This section also will briefly describe if any key
subgroups within the target population will not be surveyed.  

(2)	Sample Design

The following issues will be addressed with regard to the sampling
design if a census approach will not be used:

Indication of the sampling frame.  Guidance is provided in the tools on
sample frame selection.

Indication of the sample size, including how it was derived.  Guidance
is provided in the tools on determining an appropriate sample size for a
simple analysis of the number of respondents scoring at or above a
pre-determined benchmark level.  A user-friendly reference table is
included that identifies the appropriate sample size range for this type
of analysis based on the total population size, eliminating the need for
calculations. Assistance agreement recipients that wish to conduct more
sophisticated analyses would need to establish appropriate sample sizes
accordingly.  

Brief description of the sampling method.  The generally recommended
approach is a random, or probability, sampling.  However, in some
situations a more complex method (e.g. stratified random sampling) may
be appropriate. This section will discuss why the selected method is
being proposed, including an explanation why the proposed approach will
yield results that are representative of the population sampled and will
meet the objectives of the survey.

Brief description of multi-stage sampling, if applicable.  It is
doubtful that most recipients will use multi-stage sampling for the
questionnaires.  If multi-state sampling will not be used, this will be
indicated briefly.  

Brief description of stratification variables, if applicable, including
how they were selected and how they will improve the survey estimates. 
It is doubtful that most recipients will use stratification for the
questionnaires in this guidance.  If stratification will not be used,
this will be indicated briefly.  

(3)	Precision Requirements

This section will describe the precision targets and sampling error.  It
also will discuss the potential sources of nonsampling error, the steps
to be taken to minimize its impact on sample estimates, and what effect
it may have on the value of the survey as a decision-making tool.  

(4)	Description of Questionnaire

The questions included in the questionnaire will be described briefly,
including a general indication that they are expected to achieve the
survey objectives.  In addition, the steps taken to ensure the
reliability and usefulness of the design will be described.  

3.	Pretests And Pilot Tests

This section will include the following boilerplate text:

“EPA conducted a pilot test of the generic questions and questionnaire
formats to identify and resolve potential problems in regard to
effectiveness, clarity, feasibility, and burden/usability.  The pilot
test lasted about two months and all of the questionnaire formats were
tested, as well as the Assistance Agreement Recipient Reporting Form. 
EPA solicited feedback from assistance agreement recipients as well as
those individuals who completed a questionnaire (e.g., training
participants), and addressed their feedback in finalizing the forms. 
See Section 7(g) in the supporting statement of ICR 2255.01,
“Performance Measurement Reporting for Training and
Education/Outreach,” for additional information on the pilot tests. 
In no case were identical questions asked of more than nine persons.”

4.	Collection Methods And Follow-up

This section will describe methods to be used for collecting data,
including any follow-up activities.  

(1)	Collection Methods

This section will identify which collection method(s) have been selected
and why.  Guidance is provided in the tools on data collection methods.

(2)	Survey Response And Follow-up

This section will state the target response rate and describe how the
actual response rate will be measured.   Guidance is provided in the
tools for assistance agreement recipients to maximize response rates,
including use of the Dillman method for following up on surveys that
have not been submitted, offering incentives, and offering anonymity to
respondents if possible.

  

5.	Analyzing And Reporting Survey Results

Proposed data entry procedures and quality control measures will be
described. This will include a description of the procedures to be used
for accurately counting the data and entering it on the data reporting
forms provided by EPA.  If additional analysis will be conducted, these
will be described.

  PAGE   1