Recovery Act: Recipient Reported Jobs Data Provide Some Insight into Use of Recovery Act Funding, but Data Quality and Reporting Issues Need Attention
Published: 2009-11-19T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: This testimony discusses the report being issued today on the first set of recipient reports made available in October 2009 in response to the American Recovery and Reinvestment Act's section 1512 requirement. On October 30, Recovery.gov (the federal Web site on Recovery Act spending) reported that more than 100,000 recipients had reported hundreds of thousands of jobs created or retained. GAO is required to comment quarterly on the estimates of jobs created or retained as reported by direct recipients of Recovery Act funding from federal agencies. In the first quarterly GAO report, being released today, we address the following issues: (1) the extent to which recipients were able to fulfill their reporting requirements and the processes in place to help ensure recipient reporting data quality and (2) how macroeconomic data and methods, and the recipient reports, can be used to help gauge the employment effects of the Recovery Act. Because the recipient reporting effort will be an ongoing process of cumulative reporting, our review represents a snapshot in time. At this juncture, given the national scale of the recipient reporting exercise and the limited time frames in which it was implemented, the ability of the reporting mechanism to handle the volume of data from a wide variety of recipients represents a solid first step in moving toward more transparency and accountability for federal funds; however, there is a range of significant reporting and quality issues that need to be addressed. Consequently, our report contains several recommendations to improve data quality that Office of Management and Budget (OMB) staff generally agreed to implement. We will continue to review the processes that federal agencies and recipients have in place to ensure the future completeness and accuracy of data reported. Finally, our report notes that because the recipient reports cover about one-third of Recovery Act funds, both the data in those reports and other macroeconomic data and methods together can offer a more complete view of the overall employment impact of the Recovery Act.
section_title: What GAO Found
paragraphs: As detailed in our report, our analysis and fieldwork indicate there are significant issues to be addressed in reporting, data quality, and consistent application of OMB guidance in several areas. Many entries merit further attention due to an unexpected or atypical data value or relationship between data. As part of our review, we examined the relationship between recipient reports showing the presence or absence of any full-time equivalent (FTE) counts with the presence or absence of funding amounts shown in either or both data fields for "amount of Recovery Act funds received" and "amount of Recovery Act funds expended." Forty-four percent of the prime recipient reports showed an FTE value. However,we identified 3,978 prime recipient reports where FTEs were reported but no dollar amount was reported in the data fields for amount of Recovery Act funds received and amount of Recovery Act funds expended. These records account for 58,386 of the total 640,329 FTEs reported. While OMB estimates that more than 90 percent of recipients reported, questions remain about the other 10 percent. Less than 1 percent of the records were marked as having undergone review by the prime recipient. The small percentage reviewed by the prime recipients themselves during the OMB review time frame warrants further examination. While it may be the case that the recipients' data quality review efforts prior to initial submission of their reports were seen as not needing further revision during the review timeframe, it may also be indicative of problems with the process of noting and recording when and how the prime recipient reviews occur and the setting of the review flag. In addition, the report record data included a flag as to whether a correction was initiated. Overall, slightly more than a quarter of the reports were marked as having undergone a correction during the period of review. In its guidance to recipients for estimating employment effects, OMB instructed recipients to report solely the direct employment effects as "jobs created or retained" as a single number. Problems with the interpretation of this guidance or the calculation of FTEs were one of the most significant problems we found. Jobs created or retained expressed in FTEs raised questions and concerns for some recipients. One source of inconsistency was variation in the period of performance used to calculate FTEs, which occurred in both the highway and education programs we examined. While there were problems of inconsistent interpretation of the guidance, the reporting process went relatively well for highway projects. DOT had an established procedure for reporting prior to enactment of the Recovery Act. As our report shows, in the cases of Education and the Department of Housing and Urban Development, which do not have this prior reporting experience, we found more problems. State and federal officials are examining identified issues and have stated their intention to deal with them.

section_title: Letter
paragraphs: I appreciate the opportunity to be here to discuss the report we are issuing  today on the first set of recipient reports made available in October 2009 in  response to the American Recovery and Reinvestment Act’s section 1512  requirement. On October 30, Recovery.gov (the federal Web site on  Recovery Act spending) reported that more than 100,000 recipients had  reported hundreds of thousands of jobs created or retained. GAO is  required to comment quarterly on the estimates of jobs created or retained  as reported by direct recipients of Recovery Act funding from federal  agencies.
In the first quarterly GAO report, being released today, we address the  following issues: (1) the extent to which recipients were able to fulfill their  reporting requirements and the processes in place to help ensure recipient  reporting data quality and (2) how macroeconomic data and methods, and  the recipient reports, can be used to help gauge the employment effects of  the Recovery Act. Because the recipient reporting effort will be an  ongoing process of cumulative reporting, our review represents a snapshot  in time.
At this juncture, given the national scale of the recipient reporting exercise  and the limited time frames in which it was implemented, the ability of the  reporting mechanism to handle the volume of data from a wide variety of  recipients represents a solid first step in moving toward more  transparency and accountability for federal funds; however, there is a  range of significant reporting and quality issues that need to be addressed.  Consequently, our report contains several recommendations to improve  data quality that Office of Management and Budget (OMB) staff generally  agreed to implement. We will continue to review the processes that federal  agencies and recipients have in place to ensure the future completeness  and accuracy of data reported. Finally, our report notes that because the  recipient reports cover about one-third of Recovery Act funds, both the  data in those reports and other macroeconomic data and methods  together can offer a more complete view of the overall employment impact  of the Recovery Act.
subsections: section_title: Background
paragraphs: In December 2007, the United States entered what has turned out to be the  deepest recession since the end of World War II. In responding to this  downturn, the Recovery Act employs a combination of tax relief and  government spending. About one-third of the funds provided by the act are  for tax relief to individuals and businesses; one-third is in the form of  temporary increases in entitlement programs to aid people directly  affected by the recession and provide some fiscal relief to states; and one- third falls into the category of grants, loans, and contracts.
As of September 30, 2009, approximately $173 billion, or about 22 percent,  of the $787 billion provided by the Recovery Act had been paid out by the  federal government. Nonfederal recipients of Recovery Act-funded grants,  contracts, and loans are required to submit reports with information on  each project or activity, including the amount and use of funds and an  estimate of jobs created or retained. Of the $173 billion paid out, about $47  billion—a little more than 25 percent—is covered by this recipient report  requirement. Neither individuals nor recipients receiving funds through  entitlement programs, such as Medicaid, or through tax programs are  required to report. In addition, the required reports cover direct jobs  created or retained as a result of Recovery Act funding; they do not  include the employment impact on materials suppliers (indirect jobs) or  on the local community (induced jobs), as shown in figure 1.
To implement the recipient reporting data requirements, OMB has worked  with the Recovery Accountability and Transparency Board (Recovery  Board) to deploy a nationwide data collection system at  www.federalreporting.gov, while the data reported by recipients are  available to the public for viewing and downloading on www.recovery.gov  (Recovery.gov). OMB’s June 22, 2009, guidance on recipient reporting  also includes a requirement for data quality review. Prime recipients have  been assigned the ultimate responsibility for data quality checks and the  final submission of the data. Because this is a cumulative reporting  process, additional corrections can take place on a quarterly basis.
The first of the required recipient reports cover cumulative activity since  the Recovery Act’s passage in February 2009 through the quarter ending  September 30, 2009. As shown in figure 2, OMB specified time frames for  different stages in the reporting process: for this current report, prime  recipients and delegated subrecipients were to prepare and enter their  information from October 1 to October 10; prime recipients were able to  review the data for completeness and accuracy from October 11 to  October 21, and a federal agency review period began October 22. The  final recipient reporting data for the first round of reports were first made  available on October 30.
To assess the reporting process and data quality efforts, GAO performed  an initial set of edit checks and basic analyses on the final recipient report  data that first became available at Recovery.gov on October 30, 2009. We  built on information collected at the state, local, and program level as part  of our bimonthly reviews of selected states’ and localities’ uses of  Recovery Act funds. These bimonthly reviews focus on Recovery Act  implementation in 16 states and the District of Columbia, which contain  about 65 percent of the U.S. population and are estimated to receive  collectively about two-thirds of the intergovernmental federal assistance  funds available through the Recovery Act.
To understand state quality review and reporting procedures, we visited  the 16 selected states and the District of Columbia during late September  and October 2009 and discussed with prime recipients projects associated  with 50 percent of the total funds reimbursed as of September 4, 2009, for  that state in the Federal-Aid Highway Program administered by the  Department of Transportation (DOT). Prior to the start of the reporting  period on October 1, we obtained information on prime recipients’ plans  for the jobs data collection process. After the October 10 data reporting  period, we went back to see if prime recipients had followed their own  plans and subsequently talked with at least two subrecipients to gauge  their reactions to the reporting process and assess the documentation they  were required to submit. We gathered and examined issues raised by  recipients in these jurisdictions regarding reporting and data quality and  interviewed recipients on their experiences using the Web site reporting  mechanism. During the interviews, we looked at state plans for managing,  tracking, and reporting on Recovery Act funds and activities. In a similar  way, we examined a nonjudgmental sample of Department of Education  (Education) Recovery Act projects at the prime and subrecipient level. We  also collected information from selected transit agencies and housing  authorities as part of our bimonthly Recovery Act reviews.
To gain insight into and understanding of quality review at the federal  level, we interviewed federal agency officials who have responsibility for  ensuring a reasonable degree of quality across their program’s recipient  reports. We assessed the reports from the Inspectors General (IG) on  Recovery Act data quality reviews from 15 agencies. We are also  continuing to monitor and follow up on some of the major reporting issues  identified in the media and by other observers. For example, a number of  press articles have discussed concerns with the jobs reporting done by  Head Start grantees. According to a Health and Human Services (HHS)  Recovery Act official, HHS is working with OMB to clarify the reporting  policy as it applies to Head Start grantees. We will be reviewing these  efforts as they move forward.
For our discussion of how macroeconomic data and methods and  recipient reporting together can be used to assess the employment effects  of the Recovery Act, we analyzed economic and fiscal data using standard  economic principles and reviewed the economic literature on the effect of  monetary and fiscal policies for stimulating the economy. We also  reviewed the guidance that OMB developed for Recovery Act recipients to  follow in estimating the effect of funding activities on employment,  reviewed reports that the Council of Economic Advisers (CEA) issued on  the macroeconomic effects of the Recovery Act, and interviewed officials  from CEA, OMB, and the Congressional Budget Office (CBO).
Our work was conducted in accordance with generally accepted  government auditing standards. Those standards require that we plan and  perform the audits to obtain sufficient, appropriate evidence to provide a  reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence we obtained provides a  reasonable basis for our findings and conclusions based on our audit  objectives.
section_title: Initial Observations on Recipient Reporting Data Identify Areas for Further Review
paragraphs: As detailed in our report, our analysis and fieldwork indicate there are  significant issues to be addressed in reporting, data quality, and consistent  application of OMB guidance in several areas.
Erroneous or questionable data entries. Many entries merit  further attention due to an unexpected or atypical data value or  relationship between data.
Quality review by federal agencies and prime recipients.  o  Coverage: While OMB estimates that more than 90 percent  of recipients reported, questions remain about the other 10  percent.  o  Review: Over three quarters of the prime reports were  marked as having undergone review by a federal agency,  while less than 1 percent were marked as having undergone  review by the prime recipient  Issues in the calculation of full-time equivalents (FTE).  Different interpretations of OMB guidance compromise the ability to  aggregate the data.
subsections: section_title: Erroneous or Questionable Data Entries
paragraphs: We performed an initial set of edit checks and basic analyses on the  recipient report data available for download from Recovery.gov on  October 30. As part of our review, we examined the relationship between  recipient reports showing the presence or absence of any full-time  equivalent (FTE) counts with the presence or absence of funding amounts  shown in either or both data fields for “amount of Recovery Act funds  received” and “amount of Recovery Act funds expended.” Forty-four  percent of the prime recipient reports showed an FTE value. However, as  shown in table 1, we identified 3,978 prime recipient reports where FTEs  were reported but no dollar amount was reported in the data fields for  amount of Recovery Act funds received and amount of Recovery Act funds  expended. These records account for 58,386 of the total 640,329 FTEs  reported. There were also 9,247 reports that showed no FTEs but did show  some funding amount in either or both of the funds received or expended  data fields. The total value of funds reported in the expenditure field on  these reports was $965 million. Those recipient reports showing FTEs but  no funds and funds but no FTEs constitute a set of records that merits  closer examination to understand the basis for these patterns of reporting.
Our review also identified a number of cases in which other anomalies  suggest a need for review: discrepancies between award amounts and the  amounts reported as received, implausible amounts, or misidentification  of awarding agencies. While these occurred in a relatively small number of  cases, they indicate the need for further data quality efforts.
section_title: Quality Review by Federal Agencies or Prime Recipients
paragraphs: OMB guidance assigns responsibility for data quality to the prime recipient  and provides for federal agency review. A correction could be initiated by  either the prime recipient or the reviewing agency. OMB requires that  federal agencies perform limited data quality reviews of recipient data to  identify material omissions and significant reporting errors and notify the  recipients of the need to make appropriate and timely changes to  erroneous reports. The prime recipient report records we analyzed  included data on whether the prime recipient and the agency reviewed the  record in the data quality review time frames. Over three quarters of the  prime recipient reports were marked as having undergone federal agency  review.
Less than 1 percent of the records were marked as having undergone  review by the prime recipient. The small percentage reviewed by the prime  recipients themselves during the OMB review time frame warrants further  examination. While it may be the case that the recipients’ data quality  review efforts prior to initial submission of their reports were seen as not  needing further revision during the review timeframe, it may also be  indicative of problems with the process of noting and recording when and  how the prime recipient reviews occur and the setting of the review flag.  In addition, the report record data included a flag as to whether a  correction was initiated. Overall, slightly more than a quarter of the  reports were marked as having undergone a correction during the period  of review.
section_title: Issues in Calculation of Full-Time Equivalents
paragraphs: In its guidance to recipients for estimating employment effects, OMB  instructed recipients to report solely the direct employment effects as  “jobs created or retained” as a single number. Recipients are not expected  to report on the employment impact on materials suppliers (“indirect”  jobs) or on the local community (“induced” jobs). OMB guidance stated  that “the number of jobs should be expressed as ‘full-time equivalents  (FTEs),’ which is calculated as total hours worked in jobs created or  retained divided by the number of hours in a full-time schedule, as defined  by the recipient.” Consequently, the recipients are expected to report the  amount of labor hired or not fired as result of having received Recovery  Act funds. It should be noted that one FTE does not necessarily equate to  the job of one person. Organizations may choose to increase the hours of  existing employees, for example, which can certainly be said to increase  employment but not necessarily be an additional job in the sense of adding  a person to the payroll.
Problems with the interpretation of this guidance or the calculation of  FTEs were one of the most significant problems we found. Jobs created or  retained expressed in FTEs raised questions and concerns for some  recipients. While reporting employment effects as FTEs should allow for  the aggregation of different types of jobs—part-time, full-time, or  temporary—and different employment periods, if the calculations are not  consistent, the ability to aggregate the data is compromised.
One source of inconsistency was variation in the period of performance  used to calculate FTEs, which occurred in both the highway and education  programs we examined. For example, in the case of federal highways  projects, some have been ongoing for six months, while others started in  September 2009. In attempting to address the unique nature of each  project, DOT’s Federal Highway Administration (FHWA) faced the issue of  whether to report FTE data based on the length of time to complete the  entire project (project period of performance) versus a standard period of  performance, such as a calendar quarter, across all projects. According to  FHWA guidance, which was permitted by OMB, FTEs reported for each  highway project are expressed as an average monthly FTE. Because FTEs  are calculated by dividing hours worked by hours that represent a full-time  schedule, a standard period of performance is important if numbers are to  be added across programs.
As an illustration, take a situation in which one project employed 10  people full time for 1 month, another project employed 10 people full time  for 2 months, and a third project employed 10 people full time for 3  months. FHWA’s use of average monthly FTE would result in FTEs being  overstated compared either with using OMB’s June 22 guidance or to  standardizing the reports for one quarter. Under FHWA’s approach, 30  FTEs would be reported (10 for each of the three projects); on the other  hand, using a standardized measure, 20 FTEs would be reported (3-1/3 for  the first project, 6-2/3 for the second project, and 10 for the third).  Conversely, if a project starts later than the beginning of the reporting  period, applying OMB’s June 22 guidance, which requires reporting of  FTEs on a cumulative basis, could result in reporting fewer FTEs than  would be the case under a standardized reporting period approach. In  either case, failure to standardize on a consistent basis prevents  meaningful comparison or aggregation of FTE data.
This was also an issue for education programs. For example, in California,  two higher education systems calculated FTE differently. In the case of  one, they chose to use a 2-month period as the basis for the FTE  performance period. The other chose to use a year as the basis for the  FTE. The result is almost a three-to-one difference in the number of FTEs  reported for each university system in the first reporting period. Although  Education provides alternative methods for calculating an FTE, in neither  case does the guidance explicitly state the period of performance of the  FTE.
OMB’s decision to convert jobs into FTEs provides a consistent lens to  view the amount of labor being funded by the Recovery Act, provided each  recipient uses a standard time frame in calculating the FTE. The current  OMB guidance, however, creates a situation where, because there is no  standard starting or ending point, an FTE provides an estimate for the life  of the project. Without normalizing the FTE, aggregate numbers should  not be considered, and the issue of a standard period of performance is  magnified when looking across programs and across states.
Recipients were also confused about counting a job created or retained  even though they knew the number of hours worked that were paid for  with Recovery Act funds. While OMB’s guidance explains that in applying  the FTE calculation for measuring the number of jobs created or retained  recipients will need the total number of hours worked that are funded by  the Recovery Act, it could emphasize this relationship more thoroughly  throughout its guidance.
While there were problems of inconsistent interpretation of the guidance,  the reporting process went relatively well for highway projects. DOT had  an established procedure for reporting prior to enactment of the Recovery  Act. As our report shows, in the cases of Education and the Department of  Housing and Urban Development, which do not have this prior reporting  experience, we found more problems. State and federal officials are  examining identified issues and have stated their intention to deal with  them.
section_title: Recommendations for Executive Action
paragraphs: In our report, we make a number of recommendations to OMB to improve  the consistency of FTE data collected and reported. OMB should continue  to work with federal agencies to increase recipient understanding of the  reporting requirements and application of the guidance. Specifically, OMB  should  clarify the definition and standardize the period of measurement for  FTEs and work with federal agencies to align this guidance with  OMB’s guidance and across agencies;  given its reporting approach, consider being more explicit that “jobs  created or retained” are to be reported as hours worked and paid for  with Recovery Act funds; and  continue working with federal agencies and encourage them to  provide or improve program-specific guidance to assist recipients,  especially as it applies to the full-time equivalent calculation for  individual programs.
Given some of the issues that arose in our review of the reporting process  and data, we also recommend that OMB should work with the Recovery  Board and federal agencies to re-examine review and quality assurance  processes, procedures, and requirements in light of experiences and  identified issues with this round of recipient reporting and consider  whether additional modifications need to be made and if additional  guidance is warranted.
section_title: Agency Comments and Our Evaluation
paragraphs: In commenting on a draft of our report, OMB staff told us that OMB  generally accepts the report’s recommendations. It has undertaken a  lessons-learned process for the first round of recipient reporting and will  generally address the report’s recommendations through that process.
As recipient reporting moves forward, we will continue to review the  processes that federal agencies and recipients have in place to ensure the  completeness and accuracy of data, including reviewing a sample of  recipient reports across various Recovery Act programs to assure the  quality of the reported information. As existing recipients become more  familiar with the reporting system and requirements, these issues may  become less significant; however, communication and training efforts will  need to be maintained and in some cases expanded as new recipients of  Recovery Act funding enter the system. In addition to our oversight  responsibilities specified in the Recovery Act, we are also reviewing how  several federal agencies collect information and provide it to the public for  selected Recovery Act programs, including any issues with the  information’s usefulness. Our subsequent reports will also discuss actions  taken on the recommendations in this report and will provide additional  recommendations, as appropriate.
section_title: Recipient Reports and Economic Methods Together Can Offer Insights into Employment Impact
paragraphs: While the recipient reports provide a real-time window on the use and  results of Recovery Act spending, the data will represent only a portion of  the employment effect, even after data quality issues are addressed. A  fuller picture of the employment effect would include not only the direct  jobs reported but also the indirect and induced employment gains  resulting from government spending. In addition, the entitlement spending  and tax benefits included in the Recovery Act also create employment.  Therefore, both the data reported by recipients and other macroeconomic  data and methods are helpful in gauging the overall employment effects of  the stimulus.
Economists will use statistical models to estimate a range of potential  effects of the stimulus program on the economy. In general, the estimates  are based on assumptions about the behavior of consumers, business  owners, workers, and state and local governments.
Neither the recipients nor analysts can identify with certainty the impact  of the Recovery Act because of the inability to compare the observed  outcome with the unobserved, counterfactual scenario (in which the  stimulus does not take place). At the level of the national economy,  models can be used to simulate the counterfactual, as CEA and others  have done. At smaller scales, comparable models of economic behavior  either do not exist or cover only a very small portion of all the activity in  the macroeconomy.
Our report discusses a number of the issues that are likely to affect the  impact of the Recovery Act, including the potential effect of different types  of stimulus. We also discuss state and sectoral employment trends and  that the impact of the Recovery Act will vary across states. The  employment effects of Recovery Act funds are likely to vary with the  condition of a state’s labor market, as measured by its unemployment rate.  Labor markets in every state weakened over the course of the recession,  but the degree to which this has occurred varies widely across states.
Figure 3 illustrates this—it shows the geographic distribution of the  magnitude of the recession’s impact on unemployment as measured by the  percentage change in unemployment between December 2007 and  September 2009.
The impact of funds allocated to state and local governments will also  likely vary with states’ fiscal conditions.
section_title: GAO’s Review of Allegations of Fraud, Waste, and Abuse Submitted to FraudNet
paragraphs: Finally, let me provide the committee with an update on allegations of  fraud, waste, and abuse made to our FraudNet site. As of November 13,  2009, FraudNet has received 106 Recovery Act–related allegations that  were considered credible enough to warrant further review. We referred  33 allegations to the appropriate agency Inspectors General for further  review and investigation. Our Forensic Audits and Special Investigations  unit is actively pursuing 8 allegations, which include wasteful and  improper spending; conflicts of interest; and grant, contract, and identity  fraud. Another 9 are pending further review by our criminal investigators,  and 15 were referred to other GAO teams for consideration in their  ongoing work. We will continue to monitor these referrals and will inform  the committee when outstanding allegations are resolved. The remaining  41 allegations were found not to address waste, fraud, or abuse; lacked  specificity; were not Recovery Act-related; or reflected only a  disagreement with how Recovery Act funds are being disbursed. We  consider these allegations to be resolved and no further investigation is  necessary.
Mr. Chairman and Members of the Committee, this concludes my  statement. I would be pleased to respond to any questions you may have.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.