Recovery Act: Recipient Reported Jobs Data Provide Some Insight into Use of Recovery Act Funding, but Data Quality and Reporting Issues Need Attention
Released: 2009-11-19T00:00:00

Letter

Background

In December 2007, the United States entered what has turned out to be the deepest recession since the end of World War II. In responding to this downturn, the Recovery Act employs a combination of tax relief and government spending. About one-third of the funds provided by the act are for tax relief to individuals and businesses; one-third is in the form of temporary increases in entitlement programs to aid people directly affected by the recession and provide some fiscal relief to states; and one- third falls into the category of grants, loans, and contracts.

As of September 30, 2009, approximately $173 billion, or about 22 percent, of the $787 billion provided by the Recovery Act had been paid out by the federal government. Nonfederal recipients of Recovery Act-funded grants, contracts, and loans are required to submit reports with information on each project or activity, including the amount and use of funds and an estimate of jobs created or retained. Of the $173 billion paid out, about $47 billion—a little more than 25 percent—is covered by this recipient report requirement. Neither individuals nor recipients receiving funds through entitlement programs, such as Medicaid, or through tax programs are required to report. In addition, the required reports cover direct jobs created or retained as a result of Recovery Act funding; they do not include the employment impact on materials suppliers (indirect jobs) or on the local community (induced jobs), as shown in figure 1.

To implement the recipient reporting data requirements, OMB has worked with the Recovery Accountability and Transparency Board (Recovery Board) to deploy a nationwide data collection system at www.federalreporting.gov, while the data reported by recipients are available to the public for viewing and downloading on www.recovery.gov (Recovery.gov). OMB’s June 22, 2009, guidance on recipient reporting also includes a requirement for data quality review. Prime recipients have been assigned the ultimate responsibility for data quality checks and the final submission of the data. Because this is a cumulative reporting process, additional corrections can take place on a quarterly basis.

The first of the required recipient reports cover cumulative activity since the Recovery Act’s passage in February 2009 through the quarter ending September 30, 2009. As shown in figure 2, OMB specified time frames for different stages in the reporting process: for this current report, prime recipients and delegated subrecipients were to prepare and enter their information from October 1 to October 10; prime recipients were able to review the data for completeness and accuracy from October 11 to October 21, and a federal agency review period began October 22. The final recipient reporting data for the first round of reports were first made available on October 30.

To assess the reporting process and data quality efforts, GAO performed an initial set of edit checks and basic analyses on the final recipient report data that first became available at Recovery.gov on October 30, 2009. We built on information collected at the state, local, and program level as part of our bimonthly reviews of selected states’ and localities’ uses of Recovery Act funds. These bimonthly reviews focus on Recovery Act implementation in 16 states and the District of Columbia, which contain about 65 percent of the U.S. population and are estimated to receive collectively about two-thirds of the intergovernmental federal assistance funds available through the Recovery Act.

To understand state quality review and reporting procedures, we visited the 16 selected states and the District of Columbia during late September and October 2009 and discussed with prime recipients projects associated with 50 percent of the total funds reimbursed as of September 4, 2009, for that state in the Federal-Aid Highway Program administered by the Department of Transportation (DOT). Prior to the start of the reporting period on October 1, we obtained information on prime recipients’ plans for the jobs data collection process. After the October 10 data reporting period, we went back to see if prime recipients had followed their own plans and subsequently talked with at least two subrecipients to gauge their reactions to the reporting process and assess the documentation they were required to submit. We gathered and examined issues raised by recipients in these jurisdictions regarding reporting and data quality and interviewed recipients on their experiences using the Web site reporting mechanism. During the interviews, we looked at state plans for managing, tracking, and reporting on Recovery Act funds and activities. In a similar way, we examined a nonjudgmental sample of Department of Education (Education) Recovery Act projects at the prime and subrecipient level. We also collected information from selected transit agencies and housing authorities as part of our bimonthly Recovery Act reviews.

To gain insight into and understanding of quality review at the federal level, we interviewed federal agency officials who have responsibility for ensuring a reasonable degree of quality across their program’s recipient reports. We assessed the reports from the Inspectors General (IG) on Recovery Act data quality reviews from 15 agencies. We are also continuing to monitor and follow up on some of the major reporting issues identified in the media and by other observers. For example, a number of press articles have discussed concerns with the jobs reporting done by Head Start grantees. According to a Health and Human Services (HHS) Recovery Act official, HHS is working with OMB to clarify the reporting policy as it applies to Head Start grantees. We will be reviewing these efforts as they move forward.

For our discussion of how macroeconomic data and methods and recipient reporting together can be used to assess the employment effects of the Recovery Act, we analyzed economic and fiscal data using standard economic principles and reviewed the economic literature on the effect of monetary and fiscal policies for stimulating the economy. We also reviewed the guidance that OMB developed for Recovery Act recipients to follow in estimating the effect of funding activities on employment, reviewed reports that the Council of Economic Advisers (CEA) issued on the macroeconomic effects of the Recovery Act, and interviewed officials from CEA, OMB, and the Congressional Budget Office (CBO).

Our work was conducted in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audits to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence we obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Initial Observations on Recipient Reporting Data Identify Areas for Further Review

As detailed in our report, our analysis and fieldwork indicate there are significant issues to be addressed in reporting, data quality, and consistent application of OMB guidance in several areas.

Erroneous or questionable data entries. Many entries merit further attention due to an unexpected or atypical data value or relationship between data.

Quality review by federal agencies and prime recipients. o Coverage: While OMB estimates that more than 90 percent of recipients reported, questions remain about the other 10 percent. o Review: Over three quarters of the prime reports were marked as having undergone review by a federal agency, while less than 1 percent were marked as having undergone review by the prime recipient Issues in the calculation of full-time equivalents (FTE). Different interpretations of OMB guidance compromise the ability to aggregate the data.

Erroneous or Questionable Data Entries

We performed an initial set of edit checks and basic analyses on the recipient report data available for download from Recovery.gov on October 30. As part of our review, we examined the relationship between recipient reports showing the presence or absence of any full-time equivalent (FTE) counts with the presence or absence of funding amounts shown in either or both data fields for “amount of Recovery Act funds received” and “amount of Recovery Act funds expended.” Forty-four percent of the prime recipient reports showed an FTE value. However, as shown in table 1, we identified 3,978 prime recipient reports where FTEs were reported but no dollar amount was reported in the data fields for amount of Recovery Act funds received and amount of Recovery Act funds expended. These records account for 58,386 of the total 640,329 FTEs reported. There were also 9,247 reports that showed no FTEs but did show some funding amount in either or both of the funds received or expended data fields. The total value of funds reported in the expenditure field on these reports was $965 million. Those recipient reports showing FTEs but no funds and funds but no FTEs constitute a set of records that merits closer examination to understand the basis for these patterns of reporting.

Our review also identified a number of cases in which other anomalies suggest a need for review: discrepancies between award amounts and the amounts reported as received, implausible amounts, or misidentification of awarding agencies. While these occurred in a relatively small number of cases, they indicate the need for further data quality efforts.

Quality Review by Federal Agencies or Prime Recipients

OMB guidance assigns responsibility for data quality to the prime recipient and provides for federal agency review. A correction could be initiated by either the prime recipient or the reviewing agency. OMB requires that federal agencies perform limited data quality reviews of recipient data to identify material omissions and significant reporting errors and notify the recipients of the need to make appropriate and timely changes to erroneous reports. The prime recipient report records we analyzed included data on whether the prime recipient and the agency reviewed the record in the data quality review time frames. Over three quarters of the prime recipient reports were marked as having undergone federal agency review.

Less than 1 percent of the records were marked as having undergone review by the prime recipient. The small percentage reviewed by the prime recipients themselves during the OMB review time frame warrants further examination. While it may be the case that the recipients’ data quality review efforts prior to initial submission of their reports were seen as not needing further revision during the review timeframe, it may also be indicative of problems with the process of noting and recording when and how the prime recipient reviews occur and the setting of the review flag. In addition, the report record data included a flag as to whether a correction was initiated. Overall, slightly more than a quarter of the reports were marked as having undergone a correction during the period of review.

Issues in Calculation of Full-Time Equivalents

In its guidance to recipients for estimating employment effects, OMB instructed recipients to report solely the direct employment effects as “jobs created or retained” as a single number. Recipients are not expected to report on the employment impact on materials suppliers (“indirect” jobs) or on the local community (“induced” jobs). OMB guidance stated that “the number of jobs should be expressed as ‘full-time equivalents (FTEs),’ which is calculated as total hours worked in jobs created or retained divided by the number of hours in a full-time schedule, as defined by the recipient.” Consequently, the recipients are expected to report the amount of labor hired or not fired as result of having received Recovery Act funds. It should be noted that one FTE does not necessarily equate to the job of one person. Organizations may choose to increase the hours of existing employees, for example, which can certainly be said to increase employment but not necessarily be an additional job in the sense of adding a person to the payroll.

Problems with the interpretation of this guidance or the calculation of FTEs were one of the most significant problems we found. Jobs created or retained expressed in FTEs raised questions and concerns for some recipients. While reporting employment effects as FTEs should allow for the aggregation of different types of jobs—part-time, full-time, or temporary—and different employment periods, if the calculations are not consistent, the ability to aggregate the data is compromised.

One source of inconsistency was variation in the period of performance used to calculate FTEs, which occurred in both the highway and education programs we examined. For example, in the case of federal highways projects, some have been ongoing for six months, while others started in September 2009. In attempting to address the unique nature of each project, DOT’s Federal Highway Administration (FHWA) faced the issue of whether to report FTE data based on the length of time to complete the entire project (project period of performance) versus a standard period of performance, such as a calendar quarter, across all projects. According to FHWA guidance, which was permitted by OMB, FTEs reported for each highway project are expressed as an average monthly FTE. Because FTEs are calculated by dividing hours worked by hours that represent a full-time schedule, a standard period of performance is important if numbers are to be added across programs.

As an illustration, take a situation in which one project employed 10 people full time for 1 month, another project employed 10 people full time for 2 months, and a third project employed 10 people full time for 3 months. FHWA’s use of average monthly FTE would result in FTEs being overstated compared either with using OMB’s June 22 guidance or to standardizing the reports for one quarter. Under FHWA’s approach, 30 FTEs would be reported (10 for each of the three projects); on the other hand, using a standardized measure, 20 FTEs would be reported (3-1/3 for the first project, 6-2/3 for the second project, and 10 for the third). Conversely, if a project starts later than the beginning of the reporting period, applying OMB’s June 22 guidance, which requires reporting of FTEs on a cumulative basis, could result in reporting fewer FTEs than would be the case under a standardized reporting period approach. In either case, failure to standardize on a consistent basis prevents meaningful comparison or aggregation of FTE data.

This was also an issue for education programs. For example, in California, two higher education systems calculated FTE differently. In the case of one, they chose to use a 2-month period as the basis for the FTE performance period. The other chose to use a year as the basis for the FTE. The result is almost a three-to-one difference in the number of FTEs reported for each university system in the first reporting period. Although Education provides alternative methods for calculating an FTE, in neither case does the guidance explicitly state the period of performance of the FTE.

OMB’s decision to convert jobs into FTEs provides a consistent lens to view the amount of labor being funded by the Recovery Act, provided each recipient uses a standard time frame in calculating the FTE. The current OMB guidance, however, creates a situation where, because there is no standard starting or ending point, an FTE provides an estimate for the life of the project. Without normalizing the FTE, aggregate numbers should not be considered, and the issue of a standard period of performance is magnified when looking across programs and across states.

Recipients were also confused about counting a job created or retained even though they knew the number of hours worked that were paid for with Recovery Act funds. While OMB’s guidance explains that in applying the FTE calculation for measuring the number of jobs created or retained recipients will need the total number of hours worked that are funded by the Recovery Act, it could emphasize this relationship more thoroughly throughout its guidance.

While there were problems of inconsistent interpretation of the guidance, the reporting process went relatively well for highway projects. DOT had an established procedure for reporting prior to enactment of the Recovery Act. As our report shows, in the cases of Education and the Department of Housing and Urban Development, which do not have this prior reporting experience, we found more problems. State and federal officials are examining identified issues and have stated their intention to deal with them.

Recommendations for Executive Action

In our report, we make a number of recommendations to OMB to improve the consistency of FTE data collected and reported. OMB should continue to work with federal agencies to increase recipient understanding of the reporting requirements and application of the guidance. Specifically, OMB should clarify the definition and standardize the period of measurement for FTEs and work with federal agencies to align this guidance with OMB’s guidance and across agencies; given its reporting approach, consider being more explicit that “jobs created or retained” are to be reported as hours worked and paid for with Recovery Act funds; and continue working with federal agencies and encourage them to provide or improve program-specific guidance to assist recipients, especially as it applies to the full-time equivalent calculation for individual programs.

Given some of the issues that arose in our review of the reporting process and data, we also recommend that OMB should work with the Recovery Board and federal agencies to re-examine review and quality assurance processes, procedures, and requirements in light of experiences and identified issues with this round of recipient reporting and consider whether additional modifications need to be made and if additional guidance is warranted.

Agency Comments and Our Evaluation

In commenting on a draft of our report, OMB staff told us that OMB generally accepts the report’s recommendations. It has undertaken a lessons-learned process for the first round of recipient reporting and will generally address the report’s recommendations through that process.

As recipient reporting moves forward, we will continue to review the processes that federal agencies and recipients have in place to ensure the completeness and accuracy of data, including reviewing a sample of recipient reports across various Recovery Act programs to assure the quality of the reported information. As existing recipients become more familiar with the reporting system and requirements, these issues may become less significant; however, communication and training efforts will need to be maintained and in some cases expanded as new recipients of Recovery Act funding enter the system. In addition to our oversight responsibilities specified in the Recovery Act, we are also reviewing how several federal agencies collect information and provide it to the public for selected Recovery Act programs, including any issues with the information’s usefulness. Our subsequent reports will also discuss actions taken on the recommendations in this report and will provide additional recommendations, as appropriate.

Recipient Reports and Economic Methods Together Can Offer Insights into Employment Impact

While the recipient reports provide a real-time window on the use and results of Recovery Act spending, the data will represent only a portion of the employment effect, even after data quality issues are addressed. A fuller picture of the employment effect would include not only the direct jobs reported but also the indirect and induced employment gains resulting from government spending. In addition, the entitlement spending and tax benefits included in the Recovery Act also create employment. Therefore, both the data reported by recipients and other macroeconomic data and methods are helpful in gauging the overall employment effects of the stimulus.

Economists will use statistical models to estimate a range of potential effects of the stimulus program on the economy. In general, the estimates are based on assumptions about the behavior of consumers, business owners, workers, and state and local governments.

Neither the recipients nor analysts can identify with certainty the impact of the Recovery Act because of the inability to compare the observed outcome with the unobserved, counterfactual scenario (in which the stimulus does not take place). At the level of the national economy, models can be used to simulate the counterfactual, as CEA and others have done. At smaller scales, comparable models of economic behavior either do not exist or cover only a very small portion of all the activity in the macroeconomy.

Our report discusses a number of the issues that are likely to affect the impact of the Recovery Act, including the potential effect of different types of stimulus. We also discuss state and sectoral employment trends and that the impact of the Recovery Act will vary across states. The employment effects of Recovery Act funds are likely to vary with the condition of a state’s labor market, as measured by its unemployment rate. Labor markets in every state weakened over the course of the recession, but the degree to which this has occurred varies widely across states.

Figure 3 illustrates this—it shows the geographic distribution of the magnitude of the recession’s impact on unemployment as measured by the percentage change in unemployment between December 2007 and September 2009.

The impact of funds allocated to state and local governments will also likely vary with states’ fiscal conditions.

GAO’s Review of Allegations of Fraud, Waste, and Abuse Submitted to FraudNet

Finally, let me provide the committee with an update on allegations of fraud, waste, and abuse made to our FraudNet site. As of November 13, 2009, FraudNet has received 106 Recovery Act–related allegations that were considered credible enough to warrant further review. We referred 33 allegations to the appropriate agency Inspectors General for further review and investigation. Our Forensic Audits and Special Investigations unit is actively pursuing 8 allegations, which include wasteful and improper spending; conflicts of interest; and grant, contract, and identity fraud. Another 9 are pending further review by our criminal investigators, and 15 were referred to other GAO teams for consideration in their ongoing work. We will continue to monitor these referrals and will inform the committee when outstanding allegations are resolved. The remaining 41 allegations were found not to address waste, fraud, or abuse; lacked specificity; were not Recovery Act-related; or reflected only a disagreement with how Recovery Act funds are being disbursed. We consider these allegations to be resolved and no further investigation is necessary.

Mr. Chairman and Members of the Committee, this concludes my statement. I would be pleased to respond to any questions you may have.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.