National Institutes of Health: Employment and Other Impacts Reported by NIH Recovery Act Grantees
Published: 2011-11-10T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The American Recovery and Reinvestment Act of 2009 (Recovery Act) included $8.2 billion in funding for the National Institutes of Health (NIH) to be used to support additional scientific research-including extramural grants at universities and other research institutions. In 2009, the Acting Director of NIH testified that each extramural grant awarded with Recovery Act funding had the potential of supporting employment--full- or part-time scientific jobs--in addition to other impacts, such as contributing to advances in improving public health. GAO was asked to examine the use of Recovery Act funds by NIH grantees. Specifically, GAO addresses the information available from NIH and its grantees about the extent to which NIH Recovery Act funding (1) supported jobs, and (2) had other impacts. To obtain information on job impacts, GAO reviewed a database containing information NIH Recovery Act grantees reported to the national data collection system and interviewed NIH officials. To obtain more specific jobs information about individual grants, GAO administered a Web-based data collection instrument to 50 selected principal investigators who direct research at grantee institutions--10 principal investigators at each of five selected grantee institutions. The selected principal investigators had generally received awards of $500,000 or more. To obtain information on other Recovery Act impacts, GAO used information from the data collection instrument and interviewed NIH officials.
section_title: What GAO Found
paragraphs: Data reported by all of NIH's Recovery Act grantee institutions to the national data collection system at www.federalreporting.gov and available to NIH indicate that the number of full-time equivalent (FTEs) jobs supported by NIH Recovery Act funds increased from December 2009 through September 2010, and then remained steady from December 2010 through June 2011--the most recent quarter for which data are available. The number of FTEs supported by NIH Recovery Act funds increased from about 12,000 in the reporting quarter ending December 2009 to about 21,000 in the quarter ending in June 2011. The 50 selected principal investigators who direct research at the grantee institutions in GAO's review provided additional information explaining how the Recovery Act funding supported FTEs. Nearly one-third of the selected principal investigators reported that the NIH Recovery Act funding they received supported new positions, and about half of the principal investigators reported that the funding they received allowed them to avoid reductions in jobs or avoid a reduction in the number of hours worked by current employees. The selected principal investigators also reported that the Recovery Act funding they received primarily supported scientists and other faculty. NIH officials we interviewed reported that they receive some information from principal investigators about the other impacts of NIH-funded research, such as preliminary research results included in annual progress reports. NIH is also participating in the Star Metrics program--a multiagency venture to monitor the scientific, social, and economic impacts of federally funded science--which NIH officials expect could provide more information about these impacts. While Star Metrics is currently developing an approach to capture information about the other impacts of NIH grant funding, there is no expected completion date for reporting this information. In response to GAO's data collection instrument, selected principal investigators who direct research at the grantee institutions in GAO's review reported that the use of Recovery Act funds resulted in purchases of research supplies, equipment, laboratory testing services, and scientific training of health care professionals. The majority of the 50 selected principal investigators in GAO's review also reported preliminary results from their Recovery Act-funded research that could contribute to future scientific developments in prevention and early detection of disease, improvements in medical therapies, and improved research capabilities. The principal investigators in GAO's review and NIH officials GAO interviewed reported that they track the scientific impact of NIH research--including the impact of research funded through the Recovery Act--primarily through peer-reviewed publications, but also through other metrics such as the filing and approval of patent applications. According to NIH officials, when a sufficiently large body of research results has accumulated, NIH plans to prepare reports--similar to its existing publicly available Investment Reports--that will highlight the impact of its Recovery Act-funded research. The Department of Health and Human Services provided technical comments on a draft of this report, which GAO incorporated as appropriate.

section_title: Letter
paragraphs: The American Recovery and Reinvestment Act of 2009 (Recovery Act)  was enacted to, among other things, support job creation and  preservation, infrastructure investment, energy efficiency, and scientific  research. The act included $8.2 billion in funding for the National  Institutes of Health (NIH) to be used to support additional scientific  research, including extramural grants at universities, medical schools,  and other research institutions. The act required that Recovery Act funds  be obligated by NIH in fiscal years 2009 and 2010.
In March 2009, the Acting Director of NIH testified on the potential  employment, economic, as well as scientific benefits of the Recovery Act  funding NIH received. Specifically the Acting Director testified that each  research grant awarded with Recovery Act funding has the potential of  supporting several full- or part-time scientific jobs. During testimony, the  Acting Director also pointed out that the extramural grants made with  Recovery Act funding could have other impacts, such as contributing to  advances in improving public health.
Universities or other research institutions that receive grant funding from  NIH could directly support employment in a number of ways, including  increasing the number of hours worked by existing part-time employees  or hiring new full- or part-time employees who may work on research  projects supported by one or more NIH grant awards. Recovery Act  funding—including NIH Recovery Act funding—could also indirectly  support jobs, such as if vendors that supply research equipment and  services increased their employees due to increased business from  research institutions. Any direct or indirect support of jobs could also lead  to induced effects on other jobs, such as when employees make  purchases at local businesses. However, past GAO reports have found  that it is difficult to measure the indirect and induced impacts of Recovery  Act funding, in part because the information needed to measure these  impacts is often not available. Recipients of Recovery Act funding are  only required to report on jobs directly supported by Recovery Act funds.  These reports do not include the employment impact on materials  suppliers (indirect jobs) or on the local community (induced jobs).
To measure direct support of jobs, the Office of Management and Budget  (OMB) requires recipients of Recovery Act funds—including NIH  Recovery Act grantees—to report on the number of jobs supported using  these funds to a nationwide data collection system at  www.federalreporting.gov. Guidance provided by OMB provides more  detail on this requirement, including that recipients report on the number  of full-time-equivalent (FTE) jobs that were directly funded by the  Recovery Act during each quarter. OMB’s guidance also directs  recipients of Recovery Act funding to report on the results of funded  projects as well as provide narrative descriptions of the types of jobs  funded by the Recovery Act, which provide information that the number of  FTEs does not capture. Federal agencies, such as NIH, and recipients,  such as NIH Recovery Act grantees, may also have additional information  on the other impacts of Recovery Act funding—that may include scientific  impacts, impacts in the local community, and impacts on NIH Recovery  Act grantees. Grant awards for scientific research supported by NIH  Recovery Act funds were generally made to grantee institutions, such as  universities, for research activities that are directed by a principal  investigator employed by the grantee institution. When we use the term  “grantee” in this report, we are referring to the grantee institution,  including the principal investigator who is designated by the grantee  institution to direct the NIH Recovery Act-funded research.
You requested that we examine the use of Recovery Act funds by NIH  and its grantees. In August 2010, we reported on the process and criteria  NIH used to award grants with funding made available by the Recovery  Act, the characteristics of the grants, and information NIH has made  publicly available about the grants. This report addresses the information  available from NIH and its grantees about the extent to which (1) NIH  Recovery Act funding supported jobs, and (2) NIH Recovery Act funding  had other impacts.
To obtain the information NIH and selected NIH Recovery Act grantees  have on the jobs supported with NIH Recovery Act funding, we  interviewed NIH officials about the information they have on the FTEs  supported by the Recovery Act, and reviewed (1) NIH data containing  information reported to the nationwide data collection system on the FTEs  supported by NIH Recovery Act funding, (2) annual progress reports for  fiscal year 2010 that NIH Recovery Act grantees are required to submit to  NIH, and (3) other jobs information that NIH gathers from other sources.  In addition, we selected five grantee institutions, which are universities  that employ principal investigators who received NIH Recovery Act  funding. We obtained from NIH, information on the FTEs supported by  NIH Recovery Act funding at each of these five grantee institutions as  reported to the nationwide data collection system at  www.federalreporting.gov.
To gather more specific information about individual grants, we  disseminated a Web-based data collection instrument (DCI) to 50  selected principal investigators—10 principal investigators at each of the  same five grantee institutions. (See app. I for the selection criteria for  grantee institutions and principal investigators). The Web-based DCI  contained questions about the types and number of jobs supported by the  Recovery Act funding received from NIH. The selected principal  investigators and their grantee institutions are not representative of all  grantee institutions and principal investigators who received NIH  Recovery Act funding.
The information on the number of FTEs supported by NIH Recovery Act  funding reported to the nationwide data collection system by recipients of  Recovery Act funding has certain limitations. First, OMB guidance  requires FTE numbers to be reported quarterly, and FTEs should not be  added across quarters to obtain a cumulative number of FTEs. In  addition, the calculation of FTEs may reflect full-time and/or multiple part- time jobs, therefore FTEs cannot be used to determine the total number  of individual jobs. Moreover, because of a change in OMB’s reporting  requirements FTE data for the first reporting quarter may not be  comparable to the data reported for subsequent reporting quarters.  Finally, the number of FTEs represents only the jobs directly supported by  the Recovery Act but does not capture other jobs, such as those indirectly  supported by the act, or other impacts of the spending.
To obtain information NIH and selected NIH grantees have on the other  impacts—such as impacts in the local community, scientific impacts, and  impacts on grantee institutions and principal investigators of the NIH  Recovery Act funding, we disseminated the Web-based DCI to the 50  principal investigators at the five selected grantee institutions and  interviewed NIH officials. We also asked NIH and NIH grantees to identify  the metrics they use to measure and track other impacts such as impacts  on science, the local community, and on the grantee institutions and  principal investigators. Finally, we reviewed relevant NIH Recovery Act  grant guidance as well as OMB’s Recovery Act guidance to identify  Recovery Act grantee requirements for reporting information on FTEs and  on the impacts of the Recovery Act grants to the nationwide data  collection system. See appendix I for more details about our scope and  methodology.
To assess the reliability of the data provided by NIH, we obtained  information from agency officials knowledgeable about (1) NIH grant  award data, (2) NIH Recovery Act grantee recipient reports, and (3) job  information that NIH gathers from other sources. We obtained information  from administrators at the selected grantee institutions about the quality  of their recipient reports and performed data quality checks to assess  the reliability of the Recovery Act grants data file received from NIH.  These data quality checks involved an assessment to identify incorrect  and erroneous entries or outliers. Based on the information we obtained  and analyses we conducted, we determined that the data were sufficiently  reliable for the purposes of this report.
We conducted this performance audit from October 2010 to November  2011, in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: As of September 30, 2010—the end of the 2 fiscal years during which  Recovery Act awards were made—NIH made more than 21,500 grant  awards using Recovery Act funds. In August 2010, we reported that NIH  used standard review processes—peer review or administrative review— and standard criteria to award extramural scientific research grants with  Recovery Act funding. These NIH Recovery Act grant awards were  made to three grant categories. The grants varied in award size,  geographic distribution, award duration, and research methods,  consistent with scientific research grants funded with annual  appropriations. The act required that these funds be obligated by NIH  within a 2-year window—specifically, in fiscal years 2009 and 2010,  though the activities funded by the grant may occur after fiscal year 2010.
subsections: section_title: Recovery Act Reporting Requirements
paragraphs: OMB guidance requires recipients of Recovery Act funding—including  NIH Recovery Act grantees—to report on the number of jobs supported  by the Recovery Act on a quarterly basis to the nationwide data collection  system. OMB developed recipient reporting guidance and deployed a  nationwide data collection system at www.federalreporting.gov. According  to OMB guidance, a grantee’s estimate of the number of jobs supported  by the Recovery Act each quarter must be expressed in terms of FTEs,  which are calculated as the total number of hours worked and funded by  the Recovery Act within a reporting quarter divided by the quarterly hours  in a full-time schedule, as defined by the recipient.
According to the OMB guidance, federal agencies that award Recovery  Act funds should establish internal controls to ensure data quality,  completeness, accuracy, and timely reports to the  www.federalreporting.gov Web site. In reviewing a selection of the reports  submitted to www.federalreporting.gov by grantees of agencies across  the Department of Health and Human Services (HHS), the HHS Office of  Inspector General found that HHS had processes in place for reporting  the use of Recovery Act funds. NIH officials also reported that HHS  assesses the quality of reports filed by NIH Recovery Act grantees. For  example, using data assessments performed by NIH, HHS assesses the  quality of the data reported by Recovery Act grantees.
section_title: Sources of Information on Jobs and Other Impacts of Recovery Act Grants
paragraphs: NIH and NIH Recovery Act grantees collect information about the FTEs  supported by NIH Recovery Act funding as well as information on the  other impacts of this funding from a variety of sources. Specifically, NIH  collects information about FTEs supported by the Recovery Act from the  www.federalreporting.gov Web site. NIH grantees, including NIH  Recovery Act grantees, also submit annual progress reports to NIH that  include information such as the goals and progress of their research. NIH  is also participating in the development of a multiagency collaboration  (called Star Metrics) to track the employment, scientific, and economic  impacts of its funded research projects—including Recovery Act grants.
In addition, NIH gathers information from principal investigators working  on priority research areas and prepares publicly available reports (known  as Investment Reports) about the potential scientific impacts of NIH- funded research. NIH Institutes and Centers (IC) select the topics  featured in these reports based on (1) the importance of the topic area  within the body of research funded by the IC, (2) the level of funding  provided by the IC to the topic area, and (3) the level of public interest in  the topic area. NIH grantees also collect information about the jobs as  well as other impacts of NIH grants, including those funded by the  Recovery Act, using payroll records, and effort reporting systems—such  as time cards, other internal accounting records, and publications.
section_title: NIH and Its Grantees Reported That Recovery Act Funding Generally Increased FTEs at Grantee Institutions and Primarily Supported Scientists and Other Faculty
paragraphs: Data reported by all NIH Recovery Act grantee institutions to the  nationwide data collection system and available to NIH indicate that the  number of FTEs supported by NIH Recovery Act funds generally  increased from December 2009 through September 2010, then generally  remained steady from December 2010 through June 2011—the most  recent quarters for which data are available. As shown in figure 1, the  number of FTEs supported by NIH Recovery Act funding ranged from  about 12,000 in the reporting quarter ending December 2009 to about  21,000 in the quarter ending in June 2011.
According to NIH officials, Recovery Act funds could eventually support a  total of approximately 54,000 FTEs.  This figure represents NIH’s  estimated total of FTEs that could be supported throughout the Recovery  Act. According to NIH officials, this estimate is projected based on the  quarterly expenditure of funds reported by grantee institutions and the  projected number of FTEs that NIH expects that these funds could  support over the life of the Recovery Act. NIH expects that the Star  Metrics program will provide additional information about the number and  types of jobs funded by the Recovery Act. NIH officials reported that the  Star Metrics program is an ongoing initiative and that the program is  expected to release preliminary results regarding jobs in 2012.
Like other NIH Recovery Act grantee institutions, data reported by our five  grantee institutions also showed a general increase in FTEs. Specifically,  the five institutions combined reported almost 1,000 FTEs in the quarter  ending in December 2009, increasing to almost 2,000 supported FTEs in  the most recent quarter for which data are available that ended in June  2011. (See fig. 2).
Through responses to our data collection instrument 50 selected principal  investigators at five grantee institutions provided additional information  explaining how the Recovery Act funding supported FTEs. Nearly   30 percent of the 50 selected principal investigators reported that the NIH  funding they received supported new positions, and about half of the  principal investigators reported that the funding they received allowed  them to avoid reductions in the number of employees at their institution or  avoid a reduction in the number of hours worked by current employees.  For example, according to the selected principal investigators, 29 percent  of the jobs supported by NIH Recovery Act funding at the five grantee  institutions were new employees hired by the institution using Recovery  Act funding, and 54 percent were current employees. One principal  investigator reported using NIH Recovery Act funding to hire more than  10 employees, many of whom had recently been laid off or had been out  of work for several months. According to selected principal investigators,  a majority (54 percent) of the job positions supported by NIH Recovery  Act funds were parttime and the mean number of hours worked per  week for all supported positions was about 20, including for example, a  mean of 9 hours per week for professors and 35 hours per week for  students pursuing postgraduate degrees. (See app. II for more descriptive  information about the FTEs supported by NIH Recovery Act funding.)
section_title: NIH and Its Grantees Reported Other Impacts of NIH Recovery Act Funding
paragraphs: NIH officials currently receive some information reported by NIH grantees  about other impacts of NIH’s Recovery Act funding, and NIH is  participating in a program that NIH officials expect could help track these  other impacts. In response to our data collection instrument, two-thirds of  our 50 selected principal investigators—who direct research at the  grantee institutions—reported that the Recovery Act funding received in  fiscal years 2009 and 2010 was used to purchase research supplies and  equipment and lab testing services. In addition, the majority of our 50  selected principal investigators and NIH also reported preliminary results  from research projects funded by the Recovery Act.
subsections: section_title: NIH Is Participating in a Program to Track Other Impacts, and Selected NIH Grantees Reported Other Impacts such as Purchases of Supplies and Equipment
paragraphs: NIH officials we interviewed said that principal investigators—who direct  research at the grantee institutions—including those which received  Recovery Act funding—currently report some information to NIH about  the other impacts of NIH-funded research. This information generally  includes purchases made by the principal investigators, as well as  preliminary research results submitted to NIH in their annual progress  reports. NIH is participating in the Star Metrics program—a multiagency  collaboration currently involving about 77 grantee institutions—to track,  among other things, the scientific and nonscientific impacts of its funded  research grants, including social and workforce outcomes and economic  growth. NIH officials expect that the Star Metrics program could provide  more information about these other impacts. Officials told us that Star  Metrics is currently developing an approach to capture this information,  and that they expect to pilot the approach in 2012. However, at this time  there is no expected completion date for reporting this information.
In their responses to our data collection instrument, many of our 50  selected principal investigators reported that they used the Recovery Act  funding they received from fiscal years 2009 through 2010 to purchase  supplies, equipment, and testing services used in research. Some of the  principal investigators also reported that in the course of conducting some  of their Recovery Act-funded research, they were able to provide scientific  training to health care professionals. The selected principal investigators  provided anecdotal information about the other impacts of the selected  grants. Recipients of Recovery Act funding, such as grantee institutions,  do not systematically track these other impacts; however, they are not  required by the Recovery Act to do so. In previous work on the Recovery  Act, GAO identified difficulties in assessing other impacts, particularly in  instances when data on the other impacts are not readily available. (See  app. III for more details of the other impacts of NIH Recovery Act funding  as reported by selected principal investigators.)
Purchasing Supplies and Equipment. In their responses to our data  collection instrument, two-thirds of our 50 selected principal investigators  reported that they used the Recovery Act funding they received from NIH  to purchase or lease laboratory equipment and supplies needed to  conduct research. These transactions, which we corroborated by  conducting a selected review of NIH Annual Progress Reports and  Recovery Act recipient reports, could translate into additional sales and  revenues for the vendors. According to the principal investigators, their  transactions included biomedical equipment and supplies, office supplies,  computer equipment, and software licenses. For example, one principal  investigator reported purchasing highly specialized imaging equipment  for $27,000, as well as other medical, laboratory, and office supplies.
Purchasing Specialized Services. Over a quarter of our 50 selected  principal investigators reported that they used NIH’s Recovery Act funding  to purchase certain laboratory testing services—such as genetic  sequencing—from other research facilities that were better equipped to  perform the testing and analyses. For example, one principal investigator  reported contracting with a small local research company to perform  specialized DNA analysis needed to determine the causes of immune  deficiency disorders. In addition, a couple of principal investigators  reported that they used NIH’s Recovery Act funding to contract for  consultations services, such as statistical analyses and the design of  models needed for their research. Some principal investigators also  purchased ancillary services that they said were needed to support  clinical trials, such as services providing patient transportation,  recruitment, and care.
Scientific Training for Health Care Professionals. Nine of our 50 selected  principal investigators also reported in our data collection instrument that  in the course of conducting their Recovery Act-funded research they were  able to provide scientific training to health care professionals. Some of  these principal investigators cited the importance of exposing current and  future physicians to research-based approaches for diagnosing and  treating patients. For example, one principal investigator reported that  while researching how to select treatments for cancer patients, new  oncology researchers—fellows and junior faculty—were trained about the  effects of human genetics on care delivery for cancer patients. According  to this principal investigator, understanding the effects of genetics on  cancer allows physicians to personalize the treatment options they offer  to patients. The principal investigator also noted that the next generation  of physicians needs to be knowledgeable about genomic approaches to  cancer care, while developing the foundation for their research careers.  According to another principal investigator, as part of research to  determine why certain genes contribute to Alzheimer’s disease, health  care professionals were trained to analyze complex genetic datasets and  to develop software packages needed to efficiently perform the analysis.
section_title: Most NIH Grantees and NIH Reported Preliminary Research Illustrating Potential Scientific Impacts from Projects Funded by the Recovery Act
paragraphs: In responses to our data collection instrument, a majority of our 50  selected principal investigators who direct research at the grantee  institutions reported on the preliminary results from their research projects  supported with Recovery Act funds. According to the majority of our  selected principal investigators these preliminary results could contribute  to future scientific developments in preventive medicine, the early  detection of diseases, and medical therapies. Additionally, one principal  investigator reported that some of the results of their research could lead  to the development of research capabilities to be used by other  researchers. A few principal investigators, however, stated that it was  premature to report any preliminary results from their NIH Recovery Act- funded research, because they were still conducting clinical trials and  analyzing data. In general, scientific research—including NIH-funded  projects—can be lengthy and complex, and take years to obtain results.
Grantee institutions and principal investigators in our review and NIH  officials we interviewed reported that they track the scientific impact of  NIH research—including preliminary results from research funded through  the Recovery Act—primarily through peer-reviewed publications. NIH  officials also reported that they track certain priority research areas and  communicate potential scientific impacts through its Investment  Reports. According to NIH, when a sufficiently large body of research  results have accumulated the agency plans to prepare reports (similar to  its Investment Reports) that highlight the impact of its Recovery Act- funded research. Other metrics used to track scientific impacts—including  for Recovery Act-funded research—as reported by principal investigators  in our review include the filing and approval of patent applications, the  ability to secure future grant funding, presentations at professional  meetings, utilization of products produced from their research, and  changes to health care policies and clinical practices implemented as a  result of their research.
As noted earlier, the majority of our selected principal investigators  provided preliminary results from their research projects supported with  Recovery Act funds. The following are examples of these preliminary  results:    Prevention of Diseases. One principal investigator reported that their  Recovery Act-supported research on coronary heart disease indicated  that high levels of calcified and noncalcified plaque, which can clog  arteries and contribute to heart disease, is present in young healthy  people who have a family history of premature coronary disease.  According to this principal investigator, the results of this research  could be used to identify persons who would benefit from heart  imaging tests and preventative therapy for coronary heart disease.    Early Detection of Diseases. One principal investigator reported that  their Recovery Act-supported research resulted in the identification of  several hundred proteins that are associated with chronic pancreatitis.  According to this principal investigator, this research could contribute  towards creating new blood tests for detecting chronic pancreatitis.  Another principal investigator reported identifying the symptoms that  are the most important and efficient for making a diagnosis of autism  in young children.
Improvements in Medical Therapies. One principal investigator  reported that data collected for their Recovery Act grant has yielded  results in developing personalized therapeutic approaches for patients  with idiopathic pulmonary fibrosis, a fatal disorder. This principal  investigator noted that these results could help to simplify decision  making regarding therapeutic interventions, such as for patients  undergoing an organ transplant. Another principal investigator cited  progress toward overcoming the resistance of colon cancer to existing  treatment therapies, and another assessed two alternative therapies  for coronary heart disease. A principal investigator also reported that  their Recovery Act-supported research contributed to the  development of a kidney dialysis monitoring device that could be less  invasive and more cost-effective than the current surgically implanted  monitoring systems.
Improved Research Capabilities. One principal investigator reported  that their Recovery Act-supported research contributed to the  development of a new approach that is being utilized by other  researchers studying the connections between different genes and  traits, such as those that may lead to heart disease.
section_title: Agency Comments
paragraphs: A draft of this report was provided to HHS for review and comment. HHS  provided technical comments that were incorporated as appropriate.
As arranged with your offices, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days after its  issue date. At that time, we will send copies of this report to other  interested congressional committees, the Secretary of Health and Human  Services, and the Director of the National Institutes of Health. This report  will also be available on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions regarding this report, please  contact Linda T. Kohn at (202) 512-7114 or kohnl@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. Key contributors to this report are  listed in appendix IV.
section_title: Appendix I: Scope and Methods
paragraphs: To obtain the information National Institutes of Health (NIH) and selected  NIH Recovery Act grantees have on the jobs supported with NIH  Recovery Act funding, we interviewed NIH officials about the information  they have on the full-time-equivalents (FTE) supported by the Recovery  Act, and reviewed (1) NIH data containing information reported by  grantee institutions to a nationwide data collection system at  www.federalreporting.gov on the FTEs supported by NIH Recovery Act  funding, (2) annual progress reports for fiscal year 2010 that NIH  Recovery Act grantees are required to submit to NIH, and (3) other jobs  information that NIH gathers from other sources.
To assess the reliability of the data provided by NIH, we obtained  information from agency officials knowledgeable about (1) NIH grant  award data, (2) NIH Recovery Act grantee recipient reports, and (3) the  jobs information that NIH gathers from other sources. We also performed  data quality checks to assess the reliability of the Recovery Act grants  data file received from NIH. These data quality checks involved an  assessment to identify incorrect and erroneous entries or outliers. Based  on the information we obtained and analyses we conducted, we  determined that the data were sufficiently reliable for the purposes of this  report.
In addition, we selected five grantee institutions, which were universities  that employ principal investigators who received NIH Recovery Act  funding. The five selected grantee institutions met the following criteria:  (1) received the largest portion of Recovery Act funds from NIH,   (2) received the largest number of grants, and (3) reported the highest  number of FTEs supported by NIH Recovery Act funds. The selected  institutions were Johns Hopkins University, University of Michigan,  University of Washington, University of Pennsylvania, and Duke  University. The selected grantee institutions are not representative of all  institutions that received Recovery Act funding. (See table 1 for more  information about the five selected grantee institutions.)
To gather more specific information about individual grants, we created a  Web-based data collection instrument (DCI) and disseminated it to 50  selected principal investigators—10 principal investigators at each of the  same five grantee institutions. The Web-based DCI contained questions  about the types and number of jobs supported by the Recovery Act  funding received from NIH. The selected principal investigators received  grant awards that met the following criteria: (1) the grant was a new grant  award and not a supplement to an existing grant, (2) the grant award was  for $500,000 or greater (see table 2 for more details), and (3) the grant  award was made on or before December 1, 2009. We reviewed the  abstracts for all the grants that met the above criteria and made a  judgmental selection of the final 50 grants—making sure to include a  variety of grant types such as Challenge grants and Grand Opportunity  (GO) grants that were developed for the Recovery Act. The 50 selected  grant awards ranged in size from $500,000 to about $11,000,000. The  principal investigators for these selected grants are not representative of  all principal investigators who received NIH Recovery Act funding. To  gather information about the grants from an institutional perspective, we  also created a second Web-based DCI and disseminated it to an official  involved in coordinating Recovery Act reporting at each of the five  selected grantee institutions.
We performed follow-up information gathering from selected principal  investigators and administrators at grantee institutions that completed the  DCI to supplement the information provided in the DCI. We also obtained  and reviewed information reported by grantee institutions to the  nationwide data collection system at www.federalreporting.gov about the  number of jobs supported by the Recovery Act.
The information on the number of FTEs supported by NIH Recovery Act  funding reported to the nationwide data collection system by recipients of  Recovery Act funding has certain limitations. First, the Office of  Management and Budget (OMB) guidance requires FTE numbers to be  reported quarterly and FTEs should not be added across quarters to  obtain a cumulative number of FTEs. In addition, the calculation of FTEs  may reflect full-time and/or multiple part-time jobs, therefore FTEs cannot  be used to determine the total number of individual jobs. Moreover,  because of a change in OMB’s reporting guidelines, FTE data for the first  reporting quarter may not be comparable to the data reported for  subsequent reporting quarters. The number of FTEs represents only the  jobs directly supported by the Recovery Act but does not capture the jobs  indirectly supported by the act or other impacts of the spending.
To identify the information NIH and selected grantee institutions and  principal investigators have on the other impacts of the NIH Recovery Act  funding they received, we utilized the Web-based DCI disseminated to  the same 50 principal investigators—10 principal investigators at each of  the five selected grantee institutions—noted earlier, and interviewed NIH  officials. We asked the grantee institution and principal investigators to  identify other impacts such as scientific impacts, impacts in the local  community, and impacts on the grantee institution and principal  investigators. We also asked NIH and principal investigators to identify  the metrics they use to measure and track these other impacts. We  contacted the State Recovery Act representative in two of the states in  which our selected universities are located (North Carolina and  Pennsylvania) to identify information on the other impacts of NIH  Recovery Act funding in their jurisdictions. Finally, we reviewed relevant  NIH Recovery Act grant guidance as well as OMB’s Recovery Act  guidance to identify Recovery Act grantee requirements for reporting  information on FTEs and on the impacts of the Recovery Act grants to  NIH and the nationwide data collection system at  www.federalreporting.gov.
section_title: Appendix II: Analysis of Information Reported by Selected Principal Investigators on Jobs Supported by Recovery Act Funding
paragraphs: We disseminated a Web-based data collection instrument (DCI) to a total  of 50 selected principal investigators (10 principal investigators at each of  five selected grantee institutions). The data collection instrument  included questions about the jobs supported by NIH Recovery Act  funding. Detailed results from selected questions in our data collection  instrument related to the jobs supported by Recovery Act funding cited in  this report are listed below in tables 3-6. For example, information about  (1) the number of supported positions that existed before the Recovery  Act and (2) the average number of hours worked by each supported job  category. Not all totals add to 100 percent because respondents were  given multiple answers and asked to check all that apply.
section_title: Appendix III: Analysis of Information Reported by Selected Principal Investigators on Other Impacts of Recovery Act Funding
paragraphs: We disseminated a Web-based data collection instrument to a total of 50  selected principal investigators (10 principal investigators at each of five  selected grantee institutions). The data collection instrument included  questions about the other impacts of NIH Recovery Act funding. Detailed  results from selected questions in our data collection instrument related to  the other impacts of Recovery Act funding cited in this report are listed in  tables 7-10. For example, information about (1) the types of nonscientific  impacts reported by selected principal investigators, and (2) the metrics  used to track and measure scientific impacts.
section_title: Appendix IV: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Will Simerl, Assistant Director;   N. Rotimi Adebonojo; Leonard Brown; Carolyn Garvey; Krister Friday;  Daniel S. Ries; and Monica Perez-Nelson made key contributions to this  report.