Paperwork Reduction Act: Agencies Could Better Leverage Review Processes and Public Outreach to Improve Burden Estimates
Released: 2018-08-10T00:00:00

Letter

Background

According to OMB’s regulation implementing the PRA, “information” is broadly defined as any statement or estimate of fact or opinion, regardless of form or format, whether in numerical, graphic, or narrative form, and whether oral or maintained on paper, electronic, or other media. Federal agencies collect this information in various formats, such as forms and applications, recordkeeping information maintained by entities, and third-party disclosures (see figure 1). Agencies collect this information to ensure that the public is kept safe from harm, that qualified recipients receive benefits to which they are entitled, and that agencies otherwise fulfill their respective missions.

For example, DOT requires commercial motor vehicle drivers to record information about the hours drivers spend operating their vehicles with the goal of improving operational safety and reducing crashes, injuries, and fatalities involving trucks or buses. USDA oversees SNAP, which provides food assistance to low-income individuals and families, and state agencies that administer the program must collect information from applicants in order to determine eligibility. HHS collects compliance information from entities and businesses to ensure that they are appropriately safeguarding individuals’ health information. While such information collection activities are important for the fulfillment of agency missions, they have the potential to impose significant burdens on individuals, businesses, and other entities.

The PRA created the Office of Information and Regulatory Affairs (OIRA) within OMB to review and approve individual ICRs and oversee how agencies implement the PRA. OIRA provides agencies with instructions for preparing the supporting statements required for each ICR submitted for review. OIRA also provides agencies with guidance documents on specific information collection requirements, including how to conduct pre- testing on new or complex information collections, and how PRA applies to the use of social media.

Estimating Information Collection Burden

An integral part of an agency’s ICR submission is the estimated burden on the public associated with the collection—in terms of both time (i.e., burden hours) and costs (i.e., dollars spent). Under the PRA and OMB regulations, agencies are required to develop a specific, objectively supported estimate of the burden associated with each collection. OMB directs agencies to estimate burden hours and costs to respondents for each information collection as part of the ICR justification in the supporting statement. According to OMB staff, estimated costs to respondent should include the wage rate and any applicable employee fringe benefits, such as paid leave, insurance, and retirement contributions. The formulas shown in figure 2 illustrate the calculations generally used by agencies to determine burden hours and costs associated with the collections.

ICRs are subject to multiple levels of review to ensure that they comply with the requirements of the PRA. Programs or components within an agency (e.g., bureaus) often perform an initial review. In addition, PRA requires that agencies have an independent review process, whereby agency staff who are independent of program responsibilities review the ICRs. During this review, staff evaluate the need for the information collection and the burden estimate, including whether the information collection minimizes burden on the public, among other things. Before an information collection is submitted by an agency to OMB for final review and approval, the independent reviewer must certify that the collection meets the standards that are set forth in the PRA. These standards include ensuring that the collection contains sufficient information to allow respondents to evaluate the estimated burden.

Once an ICR has been submitted to OMB, OMB will then review it for compliance with procedural requirements of the PRA and OMB’s PRA regulations. OMB can approve an ICR without changes or request changes or additional information from the agency. OMB can approve a collection for up to 3 years at one time. If the agency wants to continue to collect the information after the approval period, it must submit another ICR to OMB for approval and provide the public with an opportunity to comment on the continuation of the collection.

Public Comment and Consultation

The PRA requires agencies to solicit public input on their ICRs as a means of validating their burden estimates. Agencies can engage the public in a variety of ways such as through a notice of proposed rulemaking (NPRM), a PRA 60-day notice published in the Federal Register, or other agency-specific mechanisms. Figure 3 shows the process that agencies generally use to engage the public, which involves a PRA 60-day notice published in the Federal Register. In some circumstances, an NPRM in the Federal Register can be used to solicit input on an information collection in lieu of the 60-day notice where an information collection is part of larger rulemaking.

Selected Agencies Used Data and Professional Judgment to Estimate Burden, but Limitations Include Inconsistent Cost Estimates and Insufficient Review

IRS Used Original Data to Estimate Burden, While Other Agencies Used Existing Data and Professional Judgment

When estimating burden hours, USDA, HHS, DOT, and IRS used data and professional judgment to develop their estimates. The PRA requires that agencies develop an objectively supported burden hour estimate, but neither the act, nor OMB regulations, prescribe how agencies should develop these estimates. Among our four agencies, IRS was the only one to report gathering original data on public burden through surveys of individual taxpayers and businesses to help inform the estimates for its two largest ICRs. Each year, IRS surveys a representative sample of taxpayers who submitted completed tax returns, according to IRS officials. The surveys collect information on the actual time and cost that taxpayers invest in paperwork-related activities.

For the U.S. Individual and Business Tax Return ICRs—the federal government’s two largest information collections—IRS used its Taxpayer Burden Model to combine original survey results with existing taxpayer data to estimate taxpayer burden in terms of both time and out-of-pocket costs. The survey results also help IRS forecast its burden hour estimates each year, taking into account changes in law, regulations, and technology.

For the remaining six case study ICRs that we reviewed, USDA, HHS, and DOT used already existing data and information to estimate at least one burden hour element (i.e., number of respondents, frequency of responses, or average burden time per response), such as in the following examples:

Historical data. To estimate the number of respondents for the SNAP ICR, USDA’s Food and Nutrition Service used historical program data on the number of applicants in previous years.

Other internal agency data. To estimate the number of respondents, such as drivers and motor carriers, for the Hours of Service of Drivers Regulations ICR, DOT’s Federal Motor Carrier Safety Administration (FMCSA) used data from the 2014 Pocket Guide to Large Truck and Bus Statistics, according to agency officials. This publication compiles data from the Federal Highway Administration, the National Highway Traffic Safety Administration, and FMCSA’s Motor Carrier Management Information System, including data on the number of commercial motor vehicle drivers operating in the United States.

Third-party data. To estimate the frequency of response for some of the third-party disclosures included in its Prescription Drug Labeling ICR, HHS’s Food and Drug Administration used data from a survey conducted by the National Association of Boards of Pharmacy, according to agency officials. The data included the number of drugs on the market and the percentage of drugs requiring medication guides. These data helped Food and Drug Administration staff estimate how often the pharmaceutical industry might need to comply with the information collection.

Research studies. To help estimate the burden of applying for SNAP benefits, USDA’s Food and Nutrition Service relied, in part, on a program research study. Specifically, a 2004 Food Stamp Program Access Study estimated that applicants spend, on average, 2.2 hours travelling for face-to-face interviews during the application process. Food and Nutrition Service staff incorporated this information in its burden hour estimates to determine the average burden time per response for the SNAP ICR.

In cases where data did not exist to inform burden hour estimates, such as for average burden time per response, the selected agencies relied on their professional judgment to develop estimates, informed in some instances by internal consultation or public input. For example, HHS’s Office for Civil Rights (OCR) did not have data for the average burden time per response for some Health Insurance Portability and Accountability Act (HIPAA) Privacy, Security, and Breach Notification Rules information collection activities. The information collection addresses HIPAA requirements related to the use, disclosure, and safeguarding of individually identifiable health information. According to HHS officials, some of the reporting or recordkeeping activities required by HIPAA may be conducted by security experts. To help estimate the average burden time for these particular activities, OCR officials stated that they consulted with internal HHS security experts to determine the time it might take a security expert to complete the applicable information collection activities.

When available, public input helped some agencies refine their burden hour estimates. For example, as part of its Prescription Drug Labeling information collection, HHS’s Food and Drug Administration had originally estimated that it would take approximately 5 seconds for a pharmacist to provide a patient with a medication guide. During the 2001 renewal of the ICR, however, the agency received a comment from a distributor stating that such disclosure could take additional time, especially if the pharmacist did not already receive the medication guides and had to print them on-site, according to agency officials. After receiving this comment, agency officials considered that some distributions of medication guides to patients may take longer than others, and revised its previous estimate from 5 seconds to 3 minutes.

Agencies Did Not Always Estimate Time Costs nor Estimate Time Costs Consistently

In some cases, the four agencies did not estimate respondent time costs as a monetized dollar amount in their supporting statement, as required by OMB. Of the 200 ICRs reviewed (including the 8 largest ICRs), 76 ICRs did not include respondent time cost estimates.

Specifically, of the 50 ICRs with the largest burden hours at each agency, we found agencies did not include total annual respondent time costs for IRS—all 50, including its 2 largest ICRs;

DOT—19, including its 2 largest ICRs;

HHS—5, including its second-largest ICR; and

USDA—2 ICRs.

OMB requires agencies to include estimated respondent time costs in the ICR supporting statements. Supporting statements provide the public with detail information about the burden estimates and underlying methodology used to calculate them, among other things. However, OMB reviewed and approved all 76 ICRs we identified that did not include these estimates.

Agencies provided a variety of reasons for not including these estimates in the supporting documents. For example:

According to IRS officials the model they used to generate burden estimates for all but their two largest information collection requests is unable to calculate respondent time costs. They told us that the model they used to generate burden estimates for their two largest collections—called the Taxpayer Burden Model— does calculate respondent time cost, but the IRS did not include this information in the supporting statement. According to IRS officials, OMB is aware of the old model’s limitation and told IRS its resources should be devoted to transitioning these collections to its Taxpayer Burden Model. IRS officials said that the agency plans, but has not developed a timeline, to use the Taxpayer Burden Model on future ICRs and to phase in use of the new model over a number of years, giving higher priority to tax forms that affect the most taxpayers.

DOT officials stated that some respondent time costs were not included in ICR supporting statements because, based on their professional judgment, the information collection activities are incidental to routine business operations and therefore should not be included in respondent time cost estimates. While OMB’s guidance states that agencies should not include burden hour estimates for customary and usual business practices, the guidance also instructs agencies to estimate respondent time costs for any estimated burden hours included in the supporting statement. That is, any estimated burden hours should have a corresponding time cost for carrying out the information collection activities. DOT’s inclusion of burden hours in the supporting statement in these cases indicates that the information collection activities are not incidental to routine business operations, and that respondent time costs should have been provided based on OMB’s guidance.

An HHS official told us that, in general, while costs are not ignored, they are also not considered high-impact information.

USDA officials said that it did not include respondent time costs in the supporting statement in part because an ICR had been merged with another information collection, but stated the agency would include these costs in the ICR renewal’s supporting statement.

Unless OMB takes action to ensure that agencies consistently follow its guidelines to include respondent time costs, agencies will likely continue to not meet the requirement and omit this information. The PRA requires that OMB establish and oversee standards and guidelines by which agencies are to estimate the burden to comply with a proposed collection of information. As part of its guidelines, OMB directs agencies to provide certain standard information in its supporting statements, including estimated respondent time costs. According to OMB staff, OMB reviews these supporting statements as part of its ICR review process and has the option of requesting changes from the agency prior to approving the ICR. However, OMB staff said that the process is decentralized with individual OMB desk officers responsible for managing their own review of ICRs. OMB reviewed and approved all 76 ICRs we identified that did not include these estimates. OMB officials told us that they will review the findings in this report to determine what response is needed. Monetized respondent time cost estimates will be particularly important if agencies can use reductions in paperwork to offset new regulations under Executive Order 13771.

For ICRs with monetized respondent time costs, agencies were inconsistent in whether they included fringe benefits, such as paid leave, insurance, and retirement contributions. OMB’s instructions for submitting ICRs direct agencies to provide respondent time costs, but the instructions do not specify how to calculate such costs. Of the 119 ICRs we identified where employees might complete an information collection activity on behalf an employer, 35 applied fringe benefits to their respondent time cost estimates and 84 did not, as shown in table 2.

Table 3 shows that including fringe benefits in respondent time cost estimates can have a significant effect on the total estimated respondent time costs for an information collection. Of the two ICRs with the largest burden hour estimates at USDA, the Mandatory Country of Origin Labeling of Covered Commodities ICR includes fringe benefits, while the SNAP ICR does not. The SNAP ICR’s respondent time cost for state employees would have been $118 million higher if it had applied the same fringe benefit estimate (33 percent of the wage rate) as the Mandatory Country of Origin Labeling ICR. While different types of respondents (e.g., state employees, farmers, or doctors) may not receive the same percentage of wages as fringe benefits, the exclusion of such benefits leads to an underestimate of respondent time costs.

OMB has not provided agencies with any formal, final guidance for calculating respondent time costs or applying fringe benefits. The PRA requires the Director of OMB to develop standards and guidelines for information collections. Additionally, Standards for Internal Control in the Federal Government states that management should externally communicate the necessary quality information to external parties to achieve the entity’s objectives. OMB provided non-binding draft guidance on reviewing agency information collections in 1999 that states that any wage rates used to estimate respondent time costs should be “fully-loaded” to reflect the full cost of labor, including employee fringe benefits, such as paid leave, insurance, and retirement contributions. OMB staff told us that OMB continues to believe that using “fully-loaded” wage rates is important. Without formal, final guidance clearly communicating how agencies should apply fringe benefits to respondent time cost estimates, agencies may continue to calculate costs inconsistently.

Agencies’ inconsistent application of fringe benefits could contribute to agencies underestimating the burden costs. Such underestimation could contribute to inconsistent implementation of Executive Order 13771. As previously stated, OMB guidance implementing the Executive Order states that agencies may offset the incremental costs of new regulations through the repeal or streamlining of mandatory information collection burdens. HHS officials said that they have considered potential information collection burden reductions as part of their efforts to comply with the order. While USDA and IRS officials said that the agencies were aware of the order, they did not yet have specific plans to reduce information collection burden for the purposes of the order. Without clear guidance about how to consistently estimate respondent time costs, Congress and the administration cannot effectively compare information collection cost savings for the public.

Agencies’ Independent Review Processes Did Not Detect Errors in Burden and Cost Estimates or Other Discrepancies

While our selected agencies reported having multiple levels of independent review processes in place as part of the overall process for preparing ICR burden estimates, we found instances where USDA, HHS, and DOT did not detect math errors or inconsistencies. We found multiple calculation errors in the supporting statements at three of the four selected agencies—USDA, HHS, and DOT—that over- or underestimated burden hours and costs to varying degrees, sometimes by millions of hours or hundreds of millions of dollars. We also found inconsistencies among estimates in Reginfo.gov and supporting statements. Reginfo.gov provides summary information to the public on information collections, including information on the estimated time and cost burdens. Supporting statements provide the public with more detailed information on the underlying methodology used to estimate burden, among other things.

The PRA requires that agencies establish a process independent of program responsibility to review each ICR before submission to OMB for approval, including a specific, objectively supported estimate of the burden. Agency officials reported that reviewers assessed the reasonableness of burden estimates by reviewing calculations, comparing current estimates to previously approved estimates, or reading the accompanying narrative in the supporting statement, which contains the assumptions used in calculating burden hours and costs. However, the agencies we reviewed did not adequately follow their own review processes, resulting in estimates that misrepresented the burden hours and costs of information collection activities, as described in the following examples.

Department of Agriculture: We found math errors in the supporting statement of USDA’s second largest ICR based on estimated burden hours. Specifically, we found that USDA’s Agricultural Marketing Service did not follow its stated assumptions in calculating burden hours and respondent time costs for the Mandatory Country of Origin Labeling ICR, resulting in an overestimation of hours and an underestimation of costs. By using the incorrect number of respondents when calculating burden hours, the Agricultural Marketing Service overestimated the Mandatory Country of Origin Labeling ICR’s total burden by 171,444 hours.

In addition, the agency did not consistently apply its stated assumptions (e.g., about the average burden time per response) in the development of respondent time cost estimates, resulting in underestimated costs presented to the public, as shown in table 4. For instance, the ICR contained two different maintenance recordkeeping costs: one described in the narrative and another in the summary tables in the supporting statement. Both underestimated recordkeeping costs. Our review found that, had the agency’s stated assumptions been consistently applied, the actual cost estimate would have been approximately $463.2 million, or $104.5 million higher than the largest maintenance recordkeeping cost estimate in the supporting statement. According to Agricultural Marketing Service officials, external pressure and accelerated timelines resulted in a less effective review of the ICR. Agency officials acknowledged that they did not follow review processes or adequately review the supporting statement. An official said that the agency will ensure that estimates for this ICR are corrected in the future.

Department of Health and Human Services: We found both math errors in the supporting statements and inconsistencies between the supporting statements and Reginfo.gov for some of HHS’s 50 largest IRCs based on estimated burden hours. Specifically, HHS did not detect calculation errors in the supporting statements in 6 of 50 ICRs (none of which were the top two case study ICRs for HHS) that we reviewed, resulting in incorrect burden hour or cost estimates. For example, in a Centers for Medicare & Medicaid Services (CMS) ICR, we found that the agency correctly stated its assumptions in the supporting statement but, due to a math error that was not detected during the review process, incorrectly calculated the respondent time cost in the second year of the collection based on these assumptions. Because it did not detect this error, HHS underestimated respondent time costs by approximately $14.4 million or about 40 percent of the published total respondent time costs in the second year for that ICR, as shown in table 5.

In another ICR, published in July 2013, CMS overestimated the public’s burden by approximately 12.8 million hours. Agency officials attributed the discrepancy to two significant math errors. Officials said that these math errors were resolved and the burden hours recalculated in a subsequent renewal of the information collection in 2017. According to HHS officials, ICRs go through multiple levels of review before HHS approves the ICR. Program offices conduct an initial review of ICRs before passing them on to the Office of the Chief Information Officer (OCIO), which then conducts a final review of ICRs before final submission to OMB, including a basic check of the math used in calculating burden hours.

Additionally, in 19 of the 50 HHS ICRs we reviewed, including HHS’s largest ICR and the two CMS examples above, we identified discrepancies in reported burden hours between the supporting statements and Reginfo.gov (see table 6). For the public to evaluate the methodology used to develop the final burden estimate posted on Reginfo.gov, the two sources need to be consistent. Reasons for such discrepancies, according to HHS officials, included data entry errors, estimate changes made in supporting statements that were not reflected on Reginfo.gov, and calculation errors. We also found 14 instances where HHS did not include an ICR supporting statement on Reginfo.gov. Based on our findings, HHS examined the discrepancies in table 6, and in June 2018 reported that all of the issues that we identified had been corrected. CMS said that in general most of the issues identified can be attributed to human error due in part to staff shortages and tight ICR submission timelines. CMS said that it takes the errors very seriously and will continue to work to refine its internal review processes to improve the quality of its ICR submissions.

HHS officials attributed some the discrepancies between the burden estimates found in the supporting statements and the estimates found on Reginfo.gov to instances when OMB works directly with program offices within the department on revisions to a burden estimate without involving OCIO. For example, according to HHS officials, in one instance shown in table 6, the supporting statement reported an initial estimate of 30,708 burden hours. Later, the estimate was revised based on input that OMB provided directly to the program office and was reported on Reginfo.gov as 12,845,827 burden hours. OCIO was not aware that the change had been made. Officials from HHS OCIO said the office is working to improve coordination and serve more as an intermediary between OMB and HHS components.

Department of Transportation: We found, and DOT officials acknowledged, a calculation error in the supporting statement for the Inspection, Repair, and Maintenance ICR, DOT’s second largest ICR based on estimated burden hours. In the ICR’s supporting statement, DOT calculated its total annual burden hours by using an average burden time per response of 170 seconds for one information collection activity. However, DOT did not include 30 seconds for one of the inspection tasks that was stated in the calculation’s assumptions found in the supporting statement. Officials said that this error may have been an inadvertent miscalculation and identified 200 seconds as the accurate average burden time per response. As a result, DOT underestimated the ICR’s total annual burden by approximately 450,000 hours, nearly 4 percent of the reported total annual burden hours for the ICR (see table 7).

As part of its review process, DOT uses a checklist for reviewing ICRs, which includes checking the math for burden hours and costs in supporting statements. An official acknowledged that, while DOT follows its review process, the agency missed this calculation error at multiple steps. The official said that the information collection had been active for a long period of time and that not detecting the error was not material in terms of PRA compliance. However, this example illustrates that a small error of 30 seconds per response can have a large impact on the overall burden hour and cost estimate. The DOT official said that it will correct the error in a revised ICR.

In the examples above, both OMB and agencies reviewed and approved the ICRs—including some of the largest ICRs at each of the agencies— but did not detect or address the math errors in the supporting statements, inconsistencies between published estimates in supporting statements and Reginfo.gov, or missing supporting statements during the review process, allowing incorrect burden hour and cost estimates to be publicly released. The PRA requires OMB to provide directions and oversee the review and approval of collections of information and the reduction of the information collection burden. OMB reviews the ICRs to ensure they are consistent with applicable laws and policies related to information quality. According to OMB officials, OMB desk officers check the burden calculations for consistency and reasonableness. The desk officers also check that the estimates are properly and consistently calculated. This includes reviewing the burden calculations to ensure that they are mathematically accurate. When we asked OMB officials about the ICRs cited in this report where we detected mathematical errors, they told us that they will review these ICRs to determine what response is needed.

Until agencies ensure that their review processes adequately detect errors and inconsistencies, the agencies cannot ensure that their burden estimates are reliable, may result in less confidence in agencies’ ability to accurately compute and report burden and as such, less confidence in agencies’ ability to effectively manage and minimize the burden they impose on the public. According to the Standards for Internal Control in the Federal Government, management should periodically review policies, procedures, and related control activities for continued relevance and effectiveness in achieving the entity’s objectives. Therefore, if agencies’ review processes do not detect errors or inconsistencies in supporting documents, then the public may have inaccurate or incomplete information on the burden imposed by an information collection. Additionally, without reliable burden estimates and complete information, Congress and the administration cannot ensure that agencies’ efforts to reduce burden to comply with Executive Order 13771 are effective.

Agencies We Reviewed Conducted Varying Levels of Public Consultation and Received Little Input on Burden Estimates

Agencies Conducted Public Outreach, but Public Notices Received Few Comments and Did Not Always Contain Needed Information to Evaluate Burden Estimates

Agencies met the PRA requirement to post Federal Register notices and solicit public comments for all 200 information collections that we reviewed. They generally received few, if any, comments from the public in response. We found that 161 ICRs solicited comments through 60- day notices. Table 8 shows that while 35 of the 161 ICRs received comments (including 3 of the top 8 case study ICRs), only 10 received comments that were related to the burden estimates (including only one case study ICR), according to the ICR supporting statement and related documents.

Based on our review of these 10 comments, 2 comments resulted in increases to agencies’ burden estimates while another 5 resulted in no burden estimate change, but allowed the agency to further explain the basis for its estimates, and provided increased transparency for the public. For example, as a result of feedback from a trade association, USDA revised its burden hour estimate for a particular component of the National Organic Program ICR from 1 hour to 10 hours. The trade association said it would take 10 to 60 hours to develop a label and get it approved. Because this estimate was not based on a formal survey, USDA did not use the upper range provided by the commenter but did acknowledge that it may have underestimated the burden, according to the ICR supporting statement. In the other example, DOT officials revised the overall burden for an ICR on drivers’ medical certificates from 9.8 million hours to 10.2 million hours based on a comment that called to their attention an incorrect assumption about the collection’s frequency. The agencies also made more transparent the specific sources used to determine the burden estimates for five of the ICRs in response to public comments. For example, one comment prompted HHS to provide additional details on the components of the burden hour estimate for the ICR and identify the relevant source data.

Officials at DOT and HHS said that, in some cases, they rely on public input in response to the 60-day Federal Register notices to validate their burden estimates, and if they do not receive any comments, they do not make any changes to the ICRs. For example, according to DOT officials, FMCSA relies on public comments to suggest revisions to ICRs that are up for renewal. In cases where no one has submitted any comments on the burden estimate, DOT officials reported that they assume the burden hour estimate per respondent is accurate and do not change the estimate.

Despite the value of public input, agencies’ Federal Register notices did not always contain a complete description of the elements that make up the burden estimates. As a result, the public may not have had enough information to comment on the reasonableness of the estimates. PRA and OMB regulations require that agencies solicit comments from the public both in the Federal Register and through other means, in part to evaluate the accuracy of the agency’s burden estimate, including the validity of the methodology and assumptions used to calculate the burden. Generally, three basic elements of the burden estimate formula in the Federal Register notices provide the public with sufficient information to review the burden estimates. As previously stated, these elements are: (1) the number of respondents, (2) the frequency of response, and (3) the average burden time per response. However, agencies do not consistently include all of these elements in the Federal Register notices.

Figures 4 and 5 show two examples of 60-day Federal Register notices in which agencies provided varying levels of detail on the burden hour estimates. In figure 5, USDA has provided burden information in its SNAP information collection by using tables and a summary that provides the estimated number of respondents, the frequency of response (i.e., number of responses per respondent), and the average burden time per response. In addition to providing data on these three elements, USDA grouped burden estimates by activity and type of respondent. This made it possible for the public to be able to review and comment on the specific assumptions used to develop the estimated hours per response. In particular, the notice shows burden data for time spent on the application, recertification application, reports, and notices for both state agencies and households.

Conversely, figure 5 shows an example where IRS has only provided the estimated total annual burden hours for the ICR without providing any of the elements used to calculate the estimated burden hours—the frequency of the information collection, the number of respondents, and the average burden time per response.

A member of the public is more likely to be able to meaningfully comment on the average burden time per response (e.g., 19 minutes for a household to complete the initial SNAP application, as shown in figure 4) than an aggregate estimate (e.g., 284,599 total burden hours to complete a form used to report and summarize income from rents, royalties, partnerships, and other sources, as shown in figure 5).

In 9 of the 10 instances in which USDA, HHS, and DOT received comments related to the burden estimates, the 60-day notices contained either all of these required elements or sufficient information to be able to calculate all of these elements. Agencies that do not consistently include these basic elements of the burden estimate reduce the likelihood that the public will be able to provide meaningful input to improve the accuracy of their burden estimates. For the two ICRs that resulted in a change in the burden estimate, USDA and DOT included detailed information for the burden time per response in their respective 60-day notices, which allowed the public to comment on these estimates.

As shown in table 9, of the 200 ICRs that we reviewed, 25 did not contain enough information to allow the public to reasonably determine the frequency of response, number of respondents, or average burden time per response: 13 at IRS, 11 at DOT, and 1 at HHS. In general, if agencies do not provide sufficient data for the elements needed to evaluate burden estimates, they may not benefit from receiving well- informed comments to ensure more reliable estimates, or to provide an opportunity for greater transparency concerning their rationale for existing burden estimates.

Our analysis found that 13 IRS ICRs (none of which were case study ICRs) did not have enough information on the frequency of the collection to allow the public to reasonably review the burden estimate and thereby provide meaningful input. IRS officials said that they did not always include data on the frequency of the collection because it might cause confusion for those instances where only a portion of the respondent population will respond to the collection more than once in a given year. Instead, IRS officials noted that their current policy is to include the estimated number of respondents, the estimated time per response, and the estimated total burden hours in each Federal Register notice for ICRs.

In early 2017, IRS established a new position to review ICRs and ensure that the agency’s PRA policies are properly implemented. According to IRS, this has helped to ensure that IRS includes estimated number of respondents, the estimated time per response, and the estimated total burden hours in each Federal Register notice. However, if IRS does not also report on the frequency of the collection and the frequency cannot be calculated using the other elements, the public may not be able to fully evaluate the burden estimate.

According to a DOT official, in some instances, program officials did not follow DOT’s prescribed templates for Federal Register notices, which direct officials to include the number of respondents, the frequency of response, and burden time per respondent to be able to calculate the total annual burden hours in each notice. According to the DOT official, in response to our findings, DOT’s Office of General Counsel is conducting an education campaign to reinforce the necessity of providing fully transparent information regarding ICR burden during all stages of the notice process.

In addition, DOT did not always include the average burden time per response in its Federal Register notices, in part because DOT’s templates do not direct officials to provide this information. A DOT official said that the templates presume that the individual reading the notice will have sufficient information to calculate this element. However, we found that average burden time per response could not reasonably be calculated using the other information provided in 10 of the notices that we reviewed in part because one or more other elements of the burden estimate were missing. Average burden time per response is a key element for the public to be able to reasonably evaluate the burden. A DOT official said that DOT plans to update the template based on our findings.

Agencies Did Not Always Consult with the Public beyond Public Notices and Did Not Solicit Input about Burden Estimates

The agencies we reviewed did not always consult with the public on information collections beyond Federal Register notices, as required by the PRA and regulations. While the PRA requires consultation on every ICR, OMB guidance only recommends public consultation in general but does not direct agencies to consult beyond the publication of the notices. When they did consult with the public, agencies did not always use these consultations as an opportunity to explicitly ask about burden hour estimates. Of the 200 information collections we reviewed, 113 contained information in their supporting statements indicating that the agencies performed public consultations beyond the Federal Register notices (see table 10). Only 3 of the 8 case study ICRs that we reviewed indicated that the agencies performed public consultations beyond the Federal Register notices.

In the 50 ICRs we reviewed, DOT provided information about public consultation in about one-quarter of its ICR supporting statements. A DOT official stated that DOT generally conducts outreach through the rulemaking process through discussions with stakeholders about the activities and fundamentals of the rule. Through this outreach process, rather than speaking explicitly about estimated burden hours, DOT and stakeholders discuss what the regulations require and whether those requirements are burdensome. According to the official, stakeholders will tell OMB if the burden estimate is incorrect. But DOT generally does not conduct additional outreach about burden estimates during ICR renewals, which occur at least every 3 years. HHS also provided information about public consultation in about one-third of its ICRs.

Agencies’ public consultation beyond the publication of Federal Register notices include federal advisory committee meetings, board meetings, webinars, and periodic stakeholder meetings. In addition, the outreach targeted a wide range of stakeholders, including associations, individuals subject to the information collection, and industry representatives.

OMB’s guidance directs agencies to include descriptions in ICR supporting statements of efforts to consult with the public about information collection burden. However, only 6 of these 113 ICR supporting statements—4 at USDA and 2 at DOT—indicated that public outreach was related to the burden hour estimates, despite OMB’s guidance. Agencies generally did not use public consultation beyond the publication of Federal Register notices to seek input on burden estimates. For example, USDA officials said that the Agricultural Marketing Service engages industry on a regular basis through meetings and seminars, but that it does not explicitly ask for feedback on the ICR burden hour or cost estimates and assume that industry representatives will raise any existing issues with the ICRs at these meetings.

At HHS, Office for Civil Rights (OCR) officials stated that there are instances where they receive feedback during conferences or through communications initiated by the public or members of Congress. However, OCR officials let people bring up the subject of the accuracy of OCR’s burden hour estimates on their own. At DOT, for five of the six Federal Railroad Administration ICRs we reviewed that involved consultation with the Railroad Safety Advisory Committee, the supporting statements do not show evidence of discussions of the ICRs’ burden estimates during committee meetings, and the agency did not reference any comments on these estimates or summarize them in the supporting statements. At IRS, 40 of the 43 IRS ICR supporting statements that identified public consultation specifically highlighted periodic meetings to discuss tax laws and tax forms with representatives of professional associations in the fields of law and accounting. IRS officials said that they do not specifically raise the issue of burden hour estimates during these meetings, but only ask stakeholders for general comments.

The lack of public consultation beyond Federal Register notices is due, in part, to a lack of guidance from OMB. In 2005, we recommended that OMB alter its current guidance to all federal agencies to direct agencies to consult with potential respondents beyond the publication of Federal Register notices. OMB disagreed with this recommendation, stating that it interprets publication in the Federal Register as the principal means of agency consultation with the public, with PRA notices on forms providing an opportunity for further public input. OMB staff told us in January 2018 that they still hold this view. Specifically, OMB staff said that additional consulting should occur for those ICRs where important information may be missed by the notice and comment period. In a June 2018 conversation, OMB staff acknowledged that public consultation could be particularly beneficial the first time that an ICR is renewed after the initial approval. At that point, the public will have had its first experiences responding to the information collection, which can inform its feedback to agencies. However, given the different types of changes that can occur over time that could affect burden estimates—such as changes in technology, the economy, and the original source data used to generate burden estimates—we continue to believe that it is important to actively consult with the public on each renewal, particularly given the low level of response that agencies receive in response to Federal Register notices for ICR renewals.

In our 2005 report, agencies also expressed concerns that consultation for every ICR would not be a good use of agency resources. Officials stated that the greatest opportunity is at the rulemaking stage. However, as previously discussed, agencies have existing public outreach efforts whose broader use would not require significant additional time and resources. Without leveraging opportunities to engage in direct public consultation with the public for every ICR, agencies may miss opportunities to obtain additional comments on ICRs, which some agencies stated they rely on to check the accuracy of their estimates and in two cases have resulted in significant revisions. Further, emphasizing those elements of the burden estimate where quality data are limited and stakeholder experiences are most relevant (e.g., the time per response) could help agencies focus outreach on the most pertinent information.

We maintain that the PRA requirement regarding public consultation in addition to the 60-day Federal Register notice is clear: both requirements are introduced together, with no distinction between them: agencies shall “provide 60-day notice in the Federal Register, and otherwise consult with members of the public and affected agencies concerning each proposed collection.” Based on our review of the four agencies, we believe that such consultation can be completed in an efficient and effective consultation manner using many of the outreach mechanisms currently in place. However, given OMB’s continued disagreement with our 2005 recommendation, congressional action may be needed to clarify the language in the PRA to more explicitly require federal agencies to consult with potential respondents on each information collection beyond the publication of Federal Register notices.

Conclusions

One of the PRA’s key requirements is for agencies to produce estimates of the burden that information collections will impose on the public. This information is essential for agencies to appropriately balance the burden of these information collections with their public benefit and for properly measuring progress toward applicable burden reduction goals. The PRA provides two mechanisms to help ensure the quality of these estimates: a multi-layered independent review process and mandatory public consultation requirements. However, the errors, omissions, and other discrepancies that we found in agencies’ ICRs indicate these mechanisms are not operating as effectively as they could be.

Independent review processes are only able to ensure an accurate and reliable estimate when agencies and OMB use them consistently to detect errors and correct them. However, we found that USDA, HHS, and DOT failed to adequately apply their own review processes, resulting errors and discrepancies between the supporting statement and Reginfo.gov. Similarly, OMB approved ICRs containing mathematical errors in the supporting statements and inconsistencies between the supporting statements and Reginfo.gov. If the agencies’ and OMB’s review processes do not detect mathematical errors and inconsistencies, then Congress and the public may have inaccurate or incomplete information on the estimated burden imposed by an information collection may result in less confidence in agencies’ abilities to accurately compute and report the burden and as such, less confidence in agencies’ ability to effectively manage and minimize the burden they impose on the public.

As part of its review process, OMB also reviews ICRs for compliance with PRA and applicable regulations, and policies. However, OMB approved numerous ICRs without the required respondent time cost information. If OMB does not take action to ensure that agencies consistently follow its guidance to include respondent time costs, agencies will likely continue to and omit this information. In addition, OMB’s current formal guidance does not offer specific instructions on when and how to include fringe benefits like paid leave and retirement contributions in respondent time costs. Without clear guidance, agencies may continue to inconsistently estimate respondent time costs, which could potentially result in underestimated time costs at some agencies as well as inconsistent implementation of efforts to reduce regulatory burden.

Public input, when available, often resulted in improvements in the quality of agencies’ burden estimates. However, the four agencies are missing opportunities to improve the quality of their estimates by not better leveraging existing public outreach efforts. While Federal Register notices provide the public with an opportunity to comment on the burden estimates, we found that DOT and IRS did not always provide sufficient information in their notices on the methodologies used to calculate the burden to allow the public to meaningfully comment on agencies’ burden estimates.

At the same time, given the few comments that agencies receive in response to these notices, it is clear that Federal Register notices alone are not sufficient. We found that agencies are already actively engaging with stakeholders through a number of means, including federal advisory committee meetings, periodic stakeholder meetings, and webinars, but are not fully using these opportunities to explicitly seek input on their burden estimates. Emphasizing those elements of the burden estimate (e.g., average time per response) during these events could help the agencies target their outreach on the most pertinent information.

IRS uses a methodically rigorous process to develop the initial burden estimates for the federal government’s two largest information collections—U.S. Individual and Business Tax Return ICRs. This process includes gathering detailed information from the public on the time and money spent on tax preparation through its taxpayer surveys. IRS plans to transition additional information collections to this more rigorous approach in the coming years. This could improve the quality of burden hour estimates and provide the cost estimates that IRS is currently lacking for other collections. IRS could also do more do consult with the public after the initial burden estimate has been developed. IRS reported that it already periodically meets with representatives from professional associations to discuss tax laws and tax forms. If IRS used these opportunities to explicitly seek input on the initial burden estimate, the agency could both obtain valuable feedback on burden estimates and comply with the consultation requirements in PRA.

OMB could help ensure that agencies more consistently obtain valuable public input on each of their ICRs by providing guidance directing agencies to consult with the public beyond the Federal Register notices on every ICR, as required in the PRA and as we previously recommended. However, while we consider the PRA requirement regarding public consultation in addition to the 60-day Federal Register notice for each ICR to be clear, OMB continues to believe that additional consulting should occur for those ICRs where important information may be missed by the public notice and comment period. We maintain that agencies should comply with the additional consultation requirement in the PRA. We acknowledge OMB’s concerns that public consultation not overly be burdensome to agencies. However, we found that the agencies we reviewed have already identified efficient and effective mechanisms for gathering input from the public, such as through periodic stakeholder meetings and webinars. Given that OMB continues to disagree with our 2005 recommendation, congressional action to clarify the legal requirement may be required.

Matter for Congressional Consideration

We are making the following matter for congressional consideration: Congress should consider amending the Paperwork Reduction Act to more explicitly require federal agencies to consult with potential respondents on each information collection beyond the publication of Federal Register notices using efficient and effective consultation methods. (Matter for Consideration 1)

Recommendations for Executive Action

We are making a total of 11 recommendations, including 2 to OMB; 2 each to the Departments of Agriculture and Health and Human Services; 3 to the Department of Transportation; and 2 to the Internal Revenue Service.

The Director of OMB should ensure the consistent application of the requirement for respondent time costs, including clarifying instructions for when and how to include fringe benefits. (Recommendation 1)

The Director of OMB should review the policies, procedures, and related control activities to ensure that the agency’s Paperwork Reduction Act review process is operating effectively. (Recommendation 2)

The Secretary of Agriculture should review the policies, procedures, and related control activities to ensure that the agency’s Paperwork Reduction Act review process is operating effectively. (Recommendation 3)

The Secretary of Agriculture should leverage existing consultation with stakeholders and the public to explicitly seek input on the burden imposed by information collections. (Recommendation 4)

The Secretary of Health and Human Services should review the policies, procedures, and related control activities to ensure that the agency’s Paperwork Reduction Act review process is operating effectively. (Recommendation 5)

The Secretary of Health and Human Services should leverage existing consultation with stakeholders and the public to explicitly seek input on the estimated burden imposed by information collections. (Recommendation 6)

The Secretary of Transportation should review the policies, procedures, and related control activities to ensure that the agency’s Paperwork Reduction Act review process is operating effectively. (Recommendation 7)

The Secretary of Transportation should leverage existing consultation with stakeholders and the public to explicitly seek input on the estimated burden imposed by information collections. (Recommendation 8)

The Secretary of Transportation should include enough information in Federal Register notices to allow the public to reasonably calculate or determine the number of respondents, the frequency of response, and the average burden time per response for each information collection activity. (Recommendation 9)

The Commissioner of Internal Revenue should leverage existing consultation with stakeholders and the public to explicitly seek input on the estimated burden imposed by information collections. (Recommendation 10)

The Commissioner of Internal Revenue should include enough information in Federal Register notices to allow the public to reasonably calculate or determine the number of respondents, the frequency of response, and the average burden time per response for each information collection activity. (Recommendation 11)

Agency Comments and Our Evaluation

We provided a draft of this report to the Director of OMB; the Secretaries of USDA, HHS, DOT; and the Commissioner of the IRS for comment, respectively. OMB did not provide written comments, and OMB staff informed us that OMB neither agreed nor disagreed with our recommendations to the agency. The Audit Liaison from the USDA’s Office of the Chief Information Officer informed us in an email that USDA concurs with our recommendations to the agency. HHS, DOT, and IRS provided written comments, which we have reprinted in appendixes II, III and IV.

In its written comments, HHS and DOT concurred with our recommendations to the agencies. HHS said it intends to continually review PRA processes and procedures as well as closely monitor their implementation to further reduce human error. DOT stated that the agency has taken action to improve its PRA program, and reported that it issued a new Federal Register notice for its Inspection, Repair, and Maintenance ICR to address mathematical errors identified in this report.

In its written comments, IRS also concurred with our recommendations to the agency. However, IRS stated that its existing PRA policies and procedures sufficiently address the PRA requirements. In response to our recommendation on leveraging existing consultation mechanisms to obtain public comments on ICR burdens, IRS noted its public participation process includes consultation with stakeholders. Specifically, IRS said that, as resources allow, it partners with industry and stakeholder groups to consult with taxpayers on tax product development and assess the burden experience in understanding the tax forms and complying with requirements to complete them.

However, we believe that IRS could better leverage this stakeholder consultation. Our analysis of supporting statements and interviews with IRS officials indicates that IRS did not explicitly seek input on burden estimates for its largest collections during these consultations. Soliciting input through the published forms themselves provides additional opportunities to obtain valuable stakeholder input, but it is not a substitute for actively reaching out to stakeholders for input on its burden estimates prior to approval, which could be readily accomplished through the mechanisms IRS already has in place for stakeholder consultation.

In response to our recommendation on providing the public with sufficient information in its public notices to allow the public to evaluate an ICR’s burden, IRS acknowledged that public notices issued before February 2017 may not have included all the elements needed by the public to be able to evaluate the burden estimates (number of respondents, frequency of response, and average burden time per response). However, according to IRS, the agency has since implemented a procedure to ensure that these elements are in the ICRs and that recent ICR public notices contain all three elements. If effectively implemented, these new procedures could help ensure that the public has the information it needs to review and provide input to on the specific assumptions used to develop the burden estimates.

The public notices we reviewed for this report were all published prior to February 2017. When we spoke with IRS officials in February 2018, they said that their current policy is to include the estimated number of respondents, the estimated time per response, and the estimated total burden hours in each Federal Register notice for ICRs. IRS officials added that they did not always include data on the frequency of the collection because it might cause confusion for those instances where only a portion of the respondent population will respond to the collection more than once in a given year. Although IRS stated in its written comments that it had implemented new procedures to include all the necessary elements, we found some ICRs issued after February 2017 that do not contain the necessary elements, including frequency, to allow the public to evaluate the specific assumptions used to develop the burden estimates. We will follow-up with IRS to ensure that new ICR procedures fully address the issues we identified.

OMB, USDA, HHS, and DOT also provided technical comments, which we incorporated as appropriate throughout our report.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Director of the Office of Management and Budget; the Secretaries of the Departments of Agriculture, Health and Human Services, and Transportation; and the Commissioner of the Internal Revenue Service, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff has any questions about this report, please contact Tranchau (Kris) T. Nguyen at (202) 512-2660 or Nguyentt@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report. Key contributors to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

This report examines (1) how agencies estimate both the burden hours and costs of their information collections, and any limitations of their approaches, and (2) the extent to which agencies consult with and receive comments from the public on the collections’ estimated burden.

To address both of these objectives, we reviewed Office of Management and Budget (OMB) data on federal information collection requests (ICR) available on Reginfo.gov as of April 7, 2017. To obtain more information about the ICR process at the agency level, we selected four agencies to serve as case studies. We identified the four agencies with the largest number of total annual estimated burden hours across the federal government based on the Reginfo.gov data. The selected agencies are the Internal Revenue Service (IRS)—which alone accounts for approximately 70 percent of the federal government’s total information collection burden hours—and the Departments of Health and Human Services (HHS), Agriculture (USDA), and Transportation (DOT). HHS represents 12 percent of the total burden hours in the federal government, while USDA and DOT each represent 2 percent of the federal information collections burden. The four selected agencies represent more than 85 percent of the total estimated burden hours across the federal government.

For each of the four agencies, we selected the 50 largest ICRs based on total annual burden hours, for a total of 200 ICRs, to provide us with information about the agencies’ efforts to consult with the public and their approach for estimating burden hours, respondent time costs, and resource costs related to each ICR. As part of this analysis, we reviewed information about (1) burden hour and (2) cost estimates and public consultation from Reginfo.gov and the ICR supporting statement.

To obtain a more detailed understanding of the methodologies, policies, and public outreach efforts related to estimating and reviewing the burden associated with each ICR, we selected the two ICRs with the largest burden hour estimates from each selected agency as case studies. The eight case study ICRs, shown in table 11, represent the majority of information collection burden at each agency and roughly 59 percent of the federal government’s total burden hours. We reviewed the supporting statements for each case study ICR to determine how the agencies calculated burden hours, respondent time costs, and respondent resource costs.

We reviewed the Federal Register notices issued by the agencies to solicit comments from the general public on these ICRs, as well as the comments received in response to the 60-day notices. We reviewed the Paperwork Reduction Act and OMB guidance issued to assist agencies in developing and reviewing their information collections. We interviewed knowledgeable officials at the four selected agencies to obtain information on the methodologies used to estimate burden time and costs, the processes and policies for ICR review and submission, and public participation in providing comments about the burden estimates. In addition, we interviewed OMB staff to obtain information about its role in reviewing ICRs submitted by agencies, as well as its relationship with the selected agencies.

To assess the reliability of Reginfo.gov data on burden hours and annualized costs for each ICR, we interviewed OMB staff and reviewed documentation of the Reginfo.gov website and the Regulatory Information Service Center/Office of Information and Regulatory Affairs Consolidated Information System (ROCIS), which is the system that agencies use to track information collection requests and that underlies information provided on Reginfo.gov. We compared the data from Reginfo.gov with the data found in the supporting statements. We interviewed agency officials and OMB staff about the discrepancies between these two information sources. We found that the Reginfo.gov data were sufficiently reliable for the purpose of selecting the case study agencies and ICRs subject to our review.

We conducted this performance audit from January 2017 to July 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Health and Human Services

Appendix III: Comments from the Department of Transportation

Appendix IV: Comments from the Department of the Treasury

Appendix V: GAO Contacts and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the above contact, Thomas J. McCabe (Assistant Director) and Joseph L. Santiago (Analyst-in-Charge) supervised the development of this report. Michael Bechetti, Tim Bober, Alyssia Borsella, Jaqueline Chapin, Steven Flint, Tim Guinane, Heather Krause, Ying Long, Sharon Miller, Ed Nannenhorn, Kayla Robinson, Robert Robinson, Cindy Saunders, Wesley Sholtes, and Chris Zakroff made major contributions to this report. Dawn Bidne, Jeffrey DeMarco, Jessica Nierenberg, and Laurel Plume verified the contents of this report.