Administration for Community Living

Office of Performance and Evaluation

ACL’s Data Collection Process

November 2021
1

Acknowledgment 

This document was prepared by the Office of Performance and Evaluation (OPE), Center for Policy and 

Evaluation (CPE), Administration for Community Living (ACL), U.S. Department of Health and Human 

Services (HHS), in partnership with New Editions Consulting, Inc., under contract number 

HHSP233201500113I/75P00119F37003.
2 

Purpose

The collection, analysis, and dissemination of timely, reliable, and quality data are essential to the 

success of the ACL mission of maximizing the independence, well-being, and health of older adults, 

people with disabilities across the lifespan, and their families and caregivers.

As ACL builds on its existing programming and implements new programming, new data collection 

processes need to be created to evaluate the performance of these programs and collect data about 

individuals served. 

The purpose of this flowchart presentation is to illustrate the processes needed to roll out a new ACL 

data collection, including key activity areas, action steps, constraints and requirements, and time 

estimates. 

Key Activities for New ACL Data Collections

Generally, the following six key activities must be completed before new ACL data collection can be 

implemented: 

1. Identifying the data need

2. Assessing existing data and data collection processes

4. Obtaining required data collection approvals

3. Developing new data elements

5. Defining a data collection and submission process

6. Rolling out the new data collection
3 

1. Identifying the Data Need

Data collections by ACL are driven by authorizing legislation, federal reporting requirements, 

program needs, and needs for information that would support ACL in achieving its mission. These 

sources stipulate the types of data ACL is required to collect, as well as the limitations on what 

data and how much data ACL can collect. 

Steps for Identifying Data Needs 

Review 

authorizing 

legislation 

• Authorizing statutes establish or continue federal agencies or programs,

establish the terms and conditions under which they operate, authorize

the enactment of appropriations, and specify how appropriated funds

are to be used.

• ACL administers programs authorized through a variety of statutes,

including the Developmental Disabilities Act and the Older Americans

Act.

Review federal 

reporting 

requirements 

• Under the Government Performance and Results Act of

1993 (GPRA) and the GPRA Modernization Act of 2010,

every federal agency is required to develop performance

measures, which include inputs, outcomes, outputs, and

impacts.

• ACL’s outcomes measures are described, by program, in its

annual Congressional Budget Justification, including

measures of program efficiency.

Review programmatic 

guidance, and 

information needs, and 

obtain input from 

program managers 

• Each ACL program has performance goals

reflecting its purpose and target

population(s).

• ACL program managers oversee programs

that support the development and

maintenance of reporting standards,

provide guidance on program performance,

and communicate data trends.
4 

2. Assessing Existing Data and Data Collection Processes

Once the data needs have been determined, existing data and data collection processes must be 

reviewed to avoid duplication in data collection and to minimize the burden of data collection on ACL 

stakeholders (e.g., grantees, subgrantees, service recipients). 

Steps for Assessing Existing Data and Data Collection Processes

Review current 

program 

performance 

reports

• In compliance with GPRA and federal program reporting requirements, 

ACL grantees (i.e., states, the District of Columbia, and U.S. territories) 

submit program performance reports on the activities carried out 

under their programs. In the case of Councils on Developmental 

Disabilities, the five-year state plan drives the subsequent annual work 

plan and annual performance progress report. Existing performance 

reporting tools have been approved by the Office of Management and 

Budget (OMB). 

Assess current 

data collection 

systems

• Many programs have approved and tested reporting 

systems to collect program performance data and data 

about the program’s client population. These include, but 

are not limited to, the Burn Model System Database, the 

National Adult Maltreatment Reporting System, the SHIP 

Tracking and Reporting Systems, the Older Americans Act 

Performance System, and the Traumatic Brain Injury 

Database. 

Develop/revise 

program logic 

model

• A logic model is a visual and systematic way 

to describe the relationships between 

available resources, program activities, and 

anticipated changes or results. It can be 

helpful to compare the outputs and 

outcomes listed in a logic model to the 

program performance data to see if there are 

any gaps or if there are data being collected 

that are not directly related to the program 

goals.
5 

3. Developing New Data Elements

New data elements are developed after the data needs have been determined, existing sources have 

been reviewed, and ACL program staff have been consulted. When developing new data elements, it is 

important to have a clear understanding what the goal of the new data collection is – i.e., what are we 

trying to achieve with the data. This will help ensure that data needs can be prioritized and data 

elements are relevant for addressing the identified needs. It is also important to work with program 

stakeholders, such as grantees, to develop the data elements and to ensure buy-in for them. 

Steps for Developing New Data Elements

Prioritize data 

needs

• Resources to collect, store, and analyze new data elements are likely 

limited. Since grantees’ focus is primarily on the implementation of 

program activities and services, the burden on grantees of data collection 

should be kept to a minimum.

• Data elements need to be responsive to legislation or reporting 

requirements, new and existing authorizing legislation or reporting 

requirements, or new and existing program needs.

Select data 

collection 

method

• Data collection methods need to balance data collection 

burden, the level of specificity needed to meet data usage 

needs, and data quality standards. The selection of methods 

also depends on who the respondents are, how they can be 

reached, and what resources are available. 

• Data collection methods may include surveys, interviews, 

observations, or routine administrative data collections.

Draft data 

collection tool(s)

• Data collection tools refer to the devices/instruments 

used to collect data, such as questionnaires, phone 

interview questions, or computer-assisted interviewing. 

Tools and their content need to be designed with the end 

user in mind and to help ensure the quality of the data.

• Consider existing data collection tools that have been 

developed for similar purposes or that measure similar 

concepts of interest. In many cases, there may be existing 

valid and appropriate tools that can be adapted for the 

specific purposes of your data collection activity. 

Leveraging existing tools can add efficiencies to the data 

collection tool development process and is generally a 

best practice for tool development/selection.

• When developing new data elements, work with program 

stakeholders, including grantees, to help determine 

appropriate data collection strategies and questions, and 

to obtain their buy-in. 
6 

4. Obtaining Required Data Collection Approvals

Federal agencies, including ACL, must follow stringent procedures and obtain approval before collecting 

any new data or information from the public. In addition, the Paperwork Reduction Act of 1995 (PRA), 

44 U.S.C. §§ 3501–3521, stipulates that federal agencies must minimize the burden resulting from the 

collection of information by or for the federal government on individuals; small businesses; educational 

and nonprofit institutions; state, local, and tribal governments; and other persons. To ensure that all of 

these needs are met, all new data collections must undergo several reviews and approval processes. 

Obtain ACL 

internal 

approvals

• All new data collection activities must be first reviewed within ACL by the 

appropriate Office and Center Directors and the Office of Information 

Resources Management (OIRM). In addition, new planned activities need to be 

accounted for in the Office or Center’s funding plan. 

• It may also be helpful to consult with OPE as data elements and data collection 

methods are crafted to ensure their soundness and facilitate timely internal 

approvals.

Obtain 

OMB 

PRA/IRB 

clearance

• As per the PRA, every U.S. government agency must obtain approval 

from the OIRA within OMB before collecting any information by means 

of identical questions posed to – or identical reporting, recordkeeping, or 

disclosure requirements imposed on – 10 or more persons from the 

public. This is a Congressionally mandated requirement. 

• The application for OMB PRA clearance includes a description of the 

purpose and procedures, a supporting statement, the data collection, 

instruments, results from pilot testing of data collection instruments, an 

estimate of the total burden, opportunities for public comment, and 

documentation of approval or review exemption from an Institutional 

Review Board (IRB). See the ACL intranet for guidance.

Conduct 

privacy 

impact 

assessment

• Privacy Impact Assessments (PIAs) provide members of the public with 

essential information about the information collected and maintained 

about themselves by federal agencies. The E-Government Act, OMB M03-22 requires federal agencies to conduct PIAs before developing or 

procuring information technology systems that collect, maintain, or 

disseminate personally identifiable information (PII) or creating 

electronic information collections involving 10 or more individuals. 

Consult with OIRM regarding PIA requirements.

• If the data collection mechanism is a system that will collect and store 

PII, a System of Records Notice (SORN) may be required. A system of 

records is a group of any records under the control of any agency from 

which information is retrieved by the name of the individual or by 

some identifying number, symbol, or other identifier assigned to the 

individual. The Privacy Act requires each agency to publish notice of its 

systems of records (a SORN) in the Federal Register. Consult with OIRM 

regarding PIA and SORN requirements.
7 

5. Defining a Data Collection and Submission Process

After all required data collection approvals have been obtained, the process for collecting and 

submitting the data must be defined, developed, and tested. ACL’s OIRM is a key partner during this 

stage, to guide and review the development of any new technology services for ACL. The actual 

development of the system is performed by a qualified entity based on the system requirements 

established by ACL. 

Steps for Defining a Data Collection and Submission Process

Consult with 

OIRM

• OIRM oversees and coordinates the provision of information technology

services for ACL. OIRM serves as the primary liaison to the HHS Office of 

Chief Information Officer (CIO), who provides management for the 

maintenance and operation of ACL’s information technology systems 

infrastructure, including the local area network, personal computers, 

software, hosting, and support services. The director of OIRM serves as 

ACL's CIO and Privacy Officer.

Develop 

requirements/ 

award contract

• As part of developing the system requirements/awarding

contracts, ACL determines the needed features and the

requirements for developing and implementing the data

collection system. Requirements may include specifications

for developing a database backend and functions that link

existing data systems, export data, etc.

Test and 

approve the 

system

• Alpha testing is a form of internal acceptance

testing performed to identify potential issues

and bugs in the individual system

components. It is performed by the system

development team.

• Beta testing is conducted in a "real-world”

environment with a small group of users who

are not members of the system development

team. It is considered a form of external user

acceptance testing.
8 

6. Rolling Out the New Data Collection

After the new data elements have been developed, data submission processes have been defined, and 

approvals have been obtained, the new data collection is rolled out to for testing with end-users. During 

this step, the data collection approach and content is tested. This may include testing the approach for 

selecting respondents (e.g., sample vs. census), piloting recruitment materials, and collecting responses. 

Final changes to the new data collection are made based on the results of the testing.

Steps for Rolling Out the New Data Collection

Conduct pilot 

testing

• Before the new data collection is rolled out, the data collection 

approach and tools are pilot-tested in the “real world” for usability. This 

may include cognitive testing (identifies potential problems in questions 

by evaluating the cognitive processes respondents use to answer them), 

testing the effectiveness of recruitment materials for recruiting 

respondents, assessing the usefulness of data element definitions, etc. 

ACL’s OPE can help with this.

Complete 

debriefings and 

make changes

• As an outcome of the testing of the data collection 

approach and tools, debriefings are conducted to identify 

and incorporate final improvements, and begin the final 

rollout to all end users.

Provide training 

and technical 

assistance

• As part of the rollout, training and technical 

assistance (T/TA) is provided to all users. T/TA 

is provided to ACL staff who are responsible 

for overseeing, managing, and using the new 

data. In addition, T/TA is provided to system 

users who are responsible for entering and 

submitting data. 

• T/TA for users may have two foci – the data 

collection system and the data elements. 

T/TA focused on the system is provided by 

the developers and supports system users on 

how to use and navigate the system/portal. 

T/TA focused on the data elements is 

generally provided by subject matter experts 

who offer guidance on the content of the 

data collection, including definition of data 

elements. 
9 

Estimated Timeline

The timeline presents a general framework to show that the process of implementing a new data collection is a complex and long process. It 

shows how long each activity may take to complete on average. Note: Timeframes will wary based on the needs and scope of the data collection 

requirements. For example, timeframes may be short for data collections that do not require the development of a new web-based data 

collection system.
10

Summary

As highlighted by the Evidence-Based Policymaking Act, “government decisions should be based on 

rigorous evidence and data about what works.”

As ACL makes changes to its programs and implements new programs, new data elements and data 

collection systems need to be developed, tested, and implemented to evaluate the performance of 

these programs and determine their reach. However, as this flowchart highlights, developing and 

implementing new data collection activities is a stringent and lengthy process that requires dedicated 

staff and resources. Thus, data needs have to be balanced with available resources. In addition, every 

effort should be made to keep the burden of data collection to grantees and other nonfederal 

participants to a minimum.

For additional information contact:

Office of Performance and Evaluation

Center for Policy and Evaluation

Administration for Community Living

330 C St SW

Washington, DC 20201

www.acl.gov/contact