Small Business Research Programs: Actions Needed to Improve Compliance with Spending and Reporting Requirements
Published: 2013-09-09T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Small Business Act established the SBIR and STTR programs to use small businesses to meet federal R&D needs. The law mandates that agencies, with extramural R&D budgets that meet the thresholds for participation, must spend a percentage of these annual budgets on the SBIR and STTR programs. The agencies are to report on their activities to SBA and, in turn, SBA is to report to Congress. Eleven agencies participate in SBIR, and five of them also participate in STTR. The act's 2011 reauthorization mandates that GAO review SBA's and the agencies' compliance with spending and reporting requirements, and other program aspects, for fiscal years 2006 to 2011. GAO determined (1) the extent to which participating agencies complied with spending requirements and how the agencies calculated these requirements, (2) the extent to which participating agencies and SBA complied with certain reporting requirements, (3) the potential effects of basing the spending requirements on an agency's total R&D budget, and (4) the cost to participating agencies of SBIR and STTR program administration. GAO reviewed agency calculations of spending requirements and the required reports and interviewed SBA and participating agency program and financial officials.
section_title: What GAO Found
paragraphs: Using data agencies had reported to the Small Business Administration (SBA), GAO found that 8 of the 11 agencies participating in the Small Business Innovation Research (SBIR) program and 4 of the 5 agencies participating in the Small Business Technology Transfer (STTR) program did not consistently comply with spending requirements for fiscal years 2006 to 2011. In calculating their annual spending requirements for these programs, some agencies made improper exclusions from their extramural research and development (R&D) budgets and used differing methodologies. SBA, which oversees the programs, provided guidance in policy directives for agencies on calculating these requirements, but the directives do not provide guidance on calculating the requirements when appropriations are late and spending is delayed, resulting in agencies using differing methodologies. This made it difficult to determine whether agencies' calculations were correct. Without further SBA guidance, agencies will likely continue calculating spending requirements in differing ways.
The participating agencies and SBA have not consistently complied with certain program reporting requirements. For example, in their methodology reports to SBA, the agencies submitted different levels of detail on their methodologies, such as the programs excluded from the extramural budget and the reasons for the exclusions. SBA's guidance states that the methodology reports are to itemize each R&D program excluded from the calculation of the agency's extramural budget and explain why a program is excluded but does not specify the format of the methodology reports to ensure consistency. Also, SBA's annual reports to Congress contained limited analysis of the agencies' methodologies, often not including information on particular agencies. Without more guidance to agencies on the formats of their methodology reports and more analysis of the contents of those reports, SBA cannot provide Congress with information on the extent to which agencies are reporting what is required. Further, SBA has not submitted an annual report on these programs for fiscal years 2009 to 2011 but plans to submit the reports to Congress later in 2013--making the data available to Congress on the programs 2 to 4 years late.
Potential effects of basing each participating agency's spending requirement on its total R&D budget instead of its extramural R&D budget include an increase in the amount of the spending requirement--for some agencies more than others--depending on how much of the agency's R&D budget is composed of extramural spending. Also, if the thresholds of the spending requirements for participation in the programs did not change, changing the base to an agency's total R&D budget would increase the number of agencies required to participate.
The agencies' cost of administering the programs could not be determined because the agencies have not consistently tracked that cost as they are not required to by the authorizing legislation of the programs. Nine of the 11 agencies in SBIR provided GAO with estimates of some of these costs for fiscal year 2011--most of which were for salaries and expenses. With the start of a pilot program allowing agencies to use up to 3 percent of SBIR program funds for administrative costs in 2013, SBA plans to require agencies to track and report administrative costs paid from program funds.
section_title: What GAO Recommends
paragraphs: GAO recommends, among other things, that SBA provide additional guidance to agencies for spending and reporting requirements and provide Congress with a more timely annual report with more analysis of the agencies' methodologies. SBA stated that it agrees with the recom-mendations and will implement them.

section_title: Letter
paragraphs: Small businesses are a major driver of high-technology innovation and  economic growth in the United States, generating significant employment,  new markets, and high-growth industries, according to the National  Academy of Sciences. The Small Business Innovation Research (SBIR)  and Small Business Technology Transfer (STTR) programs target small  businesses for federal research or research and development (R&D)  funding to develop and commercialize innovative technologies. The  SBIR and STTR programs, managed by the Small Business  Administration (SBA), were established in 1982 and 1992, respectively, in  amendments to the Small Business Act. Most recently, these programs  were reauthorized through September 30, 2017, by the National Defense  Authorization Act for Fiscal Year 2012.
In this report we refer to federal departments and agencies as agencies. through fiscal year 2011, federal agencies made about 126,000 awards  under the SBIR program totaling about $30.4 billion and about 9,000  awards under the STTR program totaling about $2.3 billion.
Currently, 11 agencies participate in SBIR programs: the U.S.  Department of Agriculture (USDA); the Departments of Commerce  (Commerce), Defense (DOD), Education (Education), Energy (DOE),  Health and Human Services (HHS), Homeland Security (DHS), and  Transportation (DOT); the Environmental Protection Agency (EPA); the  National Aeronautics and Space Administration (NASA); and the National  Science Foundation (NSF). Of these agencies, 5 also participate in the  STTR program: DOD, DOE, HHS, NASA, and NSF.
The Small Business Act sets certain reporting requirements for  participating agencies and SBA. Among other things, such agencies are  to report to SBA on their methodology for calculating extramural R&D  spending requirements within 4 months of enactment of their annual  appropriations. Agencies are also directed to submit annual reports to  SBA on their SBIR and STTR spending requirements and the amount  spent after the end of each fiscal year. Furthermore, SBA is to annually  report to Congress on all the participating agencies’ SBIR and STTR  programs, including an analysis of the agencies’ budget calculation  methodologies. The act also created a pilot program beginning in fiscal  year 2013 that would allow a portion of SBIR program funds to be used  for administrative costs, outreach and technical assistance, contract  processing, and other specified purposes. Agencies are otherwise  generally not permitted to spend program funds on administrative costs.
In addition, the Small Business Act mandates that GAO review the  participating agencies’ compliance with spending and reporting  requirements for the programs for fiscal years 2006 to 2011, as well as  other aspects of the programs.extent to which the participating agencies complied with the programs’  spending requirements and how they calculated these requirements, (2)
Our objectives were to determine (1) the  the extent to which participating agencies and SBA have complied with  certain reporting requirements, (3) the potential effects of basing the  spending requirements for the SBIR and STTR programs on an agency’s  total R&D budget instead of its extramural R&D budget, and (4) the cost  to participating agencies of SBIR and STTR program administration and  how agency funds were used to cover these costs.
To gather information on both programs, we reviewed the laws governing  the programs and the guidance that the SBA has issued regarding the  programs. We reviewed prior GAO, SBA Inspector General, and other  reports related to the programs. We also met with the SBA’s SBIR and  STTR program staff and interviewed program and financial management  officials at each of the 11 agencies participating in SBIR. To determine  the extent to which participating agencies have complied with the  programs’ spending requirements, we compared spending requirements  for fiscal years 2006 to 2011 with the amounts agencies reported  spending in each annual report to SBA. We determined that an agency  met its spending requirement if the agency’s reported spending for these  programs was equal to or greater than the reported spending  requirement. We used the agencies’ obligations data to represent  spending for the programs in part because obligations data were readily  available from each of the agencies for program purposes, and  obligations provided a reasonable measure of the spending for the  programs in each year. In addition, we collected data and documents  from agencies and interviewed agency officials about how they calculated  their extramural R&D budgets, including any amounts agencies are  authorized to exclude from those budgets. To assess the reliability of the  spending data, we questioned agency officials about the source and data  quality control procedures and reviewed relevant documentation. We  determined that these spending data were sufficiently reliable for the  purposes of this report.
To determine the extent to which participating agencies and SBA have  complied with certain reporting requirements for calculating their  extramural research budgets, we compared the agencies’ methodology  and annual reports to SBA and SBA’s annual report to Congress for fiscal  years 2006 to 2011, to the extent available, with requirements in the  Small Business Act and SBA’s related policy directives.documentation of SBA’s review and analysis of the methodology reports  and copies of SBA’s annual reports to Congress on the SBIR and STTR  programs to determine the extent to which SBA carried out the mandated  analysis of methodologies and publication of results in its annual reports.
To determine the potential effects of basing spending requirements for  the SBIR and STTR programs on an agency’s total R&D budget, we used  actual data on total R&D budget authority from the President’s budget to  calculate potential spending requirements for each federal agency for  fiscal years 2006-2011.requirements, we assumed that the same funding percentages currently  required by the Small Business Act would apply to total R&D budgets. We  also assumed that the current spending thresholds that require agency  participation in the SBIR and STTR programs— $100 million and $1  billion in extramural funding respectively—would instead apply to total  R&D funding. In other words, an agency with total annual R&D funding of  $100 million would be required to participate in the SBIR program, while  an agency with total annual R&D funding of $1 billion would be required  to participate in both the SBIR and STTR programs. We compared the  spending requirements under this first scenario with the spending  requirements under the current law to determine the potential effects of  changing the methodology to calculate the spending requirements for  fiscal years 2006 to 2011. We also developed two additional funding  scenarios. We compared the potential spending requirements of these  scenarios with the spending requirements under the current law for fiscal  years 2006 to 2011.
In calculating these potential spending  To determine the cost to participating agencies for SBIR and STTR  program administration and how agency funds were used, we collected  existing administrative cost data directly from agencies and interviewed  program and financial officials at each agency to (1) learn the scope of  existing data available on administrative costs, (2) identify amounts and  categories of administrative costs, and (3) request existing annual cost  data from them. We determined that these data were too incomplete and  from such varied sources that an assessment of the available data’s  reliability was not possible. Thus, we could not use the data in our report.  In addition, we gathered information on the agencies’ implementation of  the pilot program that allows a portion of SBIR program funds to be used  for specific administrative costs.
We conducted this performance audit from April 2012 to August 2013 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: SBIR has four overarching purposes: to (1) use small businesses to meet  federal R&D needs, (2) stimulate technological innovation, (3) increase  commercialization of innovations derived from federal R&D efforts, and  (4) encourage participation in technological innovation by small  businesses owned by women and disadvantaged individuals. The SBIR  program has a three-phase structure as follows:  In Phase I, agencies award up to $150,000 for a period of about 6 to 9  months to small businesses to determine the scientific and technical  merit and feasibility of ideas that appear to have commercial potential.
In Phase II, small businesses whose Phase I projects demonstrate  scientific and technical merit, in addition to commercial potential, may  compete for awards of up to $1 million to continue the R&D for an  additional period, normally not to exceed 2 years.
Phase III is for small businesses to pursue commercialization  objectives resulting from the Phase I and II R&D activities, where  appropriate. Phase III is the period in which Phase II innovation  moves from the laboratory to the marketplace. SBIR Phase III work  completes an effort made under prior SBIR phases, but it is funded by  sources other than the SBIR program. To commercialize their  products, small businesses are expected to raise additional funds  from private investors, the capital markets, or from non-SBIR sources  within the agency that made the initial award.
According to SBA documents, STTR’s purpose is to stimulate a  partnership of ideas and technologies between innovative small  businesses and research institutions through federally funded R&D.program provides funding for research proposals that are developed and  executed cooperatively between small businesses and research  institutions. Like the SBIR program, the STTR program is structured in  three phases as follows:
Phase I aims to establish the technical merit, feasibility, and  commercial potential of the proposed R&D efforts and to determine  the quality of performance of the small businesses. STTR Phase I  awards generally do not exceed $150,000 for 1 year.
Phase II funding is based on the results achieved in Phase I and the  scientific and technical merit and commercial potential of the Phase II  project proposed. STTR Phase II awards generally do not exceed $1  million total costs for 2 years.
Phase III is for small businesses to pursue commercialization of  research or technology resulting from the Phase I and II R&D  activities, and it completes an effort made under the prior STTR  phases but is funded by sources other than the STTR program.  According to SBA, Phase III work can involve commercial application  of R&D financed by nonfederal capital, including STTR products or  services intended for use by the federal government and continuation  of R&D that has been competitively selected in Phases I and II,  according to the 2012 SBA policy directive.
As noted, federal agencies with a budget of more than $100 million for  extramural R&D are required to have an SBIR program, while federal  agencies with extramural R&D budgets that exceed $1 billion annually are  required to have an STTR program. Generally, the extramural R&D  budget is defined as the sum of the total obligations for R&D minus  amounts to be obligated for intramural R&D, that is, R&D conducted by  employees of a federal agency in or through government-owned,  government-operated facilities. In determining their extramural R&D  budget, agencies have authority to exclude certain R&D programs from  the extramural R&D base used for calculating SBIR and STTR spending  requirements, such as facilities and equipment used for R&D and certain  intelligence activities. For example, under the Small Business Act, DOE  must exclude amounts obligated for its naval reactor program from its  extramural R&D budget. Likewise, under the act, DOD excludes  programs carried out by certain elements of the intelligence community.  In addition, DOT must exclude funds obligated for the Federal Highway  Administration State Planning and Research program from its extramural  R&D budget.
In fiscal year 2011, the 11 participating agencies reported spending a  total of $2.2 billion for SBIR, and the 5 participating agencies reported  spending a total of $251 million for STTR, with DOD spending the most  on these programs––$1.1 billion and $121 million, respectively. DOD’s  reported spending constituted 48.8 percent of total SBIR spending and  48.2 percent of total STTR spending in fiscal year 2011.
According to SBA documents, the agency’s role is to serve as the  oversight and coordinating agency for the SBIR and STTR programs and  is to direct and assist the agencies’ implementation of the programs,  review their progress, collect agency reports, analyze the information in  the agencies’ methodology reports, and report annually to Congress on  the programs. In this role, SBA issued SBIR and STTR policy directives in  September 2002 and December 2005, respectively, and updated them in  August 2012; these directives provide agencies with detailed guidance on  implementation of the SBIR and STTR programs.
section_title: Data Indicate Most Agencies Have Not Consistently Complied with Mandated Requirements, Because of Improper Exclusions and Differing Methodologies
paragraphs: Data reported by the participating agencies to SBA for fiscal years 2006  to 2011 indicate that most of the agencies have not consistently complied  with the mandated spending requirements for SBIR and STTR. In  calculating their spending requirements, some participating agencies  made improper exclusions and used differing methodologies.
subsections: section_title: Agency Data Indicate That Agencies Did Not Consistently Comply with Mandated Program Spending Requirements
paragraphs: Our analysis of data the participating agencies reported to SBA indicates  that, from fiscal years 2006 to 2011, most agencies did not consistently  comply with mandated spending requirements. Specifically, 8 of the 11  agencies did not consistently meet annual spending requirements for  SBIR. Data from 3 of the agencies—DHS, Education, and HHS—indicate  that they met their spending requirements for all 6 years. For STTR, 4 of  5 agencies did not consistently meet annual spending requirements. Data  from 1 agency—HHS—indicate that it met its STTR spending  requirements for all 6 years. Figure 1 shows the number of years that  each agency met its annual SBIR and STTR spending requirements,  based on the information submitted to SBA in each agency’s annual  reports.spending on these programs.
See appendix I for further details on each agency’s reported  Evaluation budget of the United States Special Operations Command  subunit from its extramural R&D budget because the budget is less  than the $1 billion required for participating in the STTR program.
HHS reported excluding the extramural R&D budgets of the Centers  for Disease Control and Prevention and the Food and Drug  Administration because these subunit budgets are less than the $1  billion required for participating in the STTR program.
In another example, a third agency, DOT reported excluding the Federal  Aviation Administration’s extramural R&D budget, which is well in excess  of $100 million annually, from the SBIR program budget calculation. We  asked DOT and FAA to provide the legal authority for this exclusion, but  they did not supply this information.
Improperly excluding subunits reduced calculations of three agencies’  respective extramural R&D and, in turn, the agencies’ spending  requirements for SBIR and STTR. Over fiscal years 2006 to 2011, these  improper exclusions resulted in a $7.7 million reduction to DOD’s STTR  spending requirement, a $34.7 million reduction to DOT’s SBIR  requirement, and HHS’ exclusions resulted in a $6.1 million reduction to  its STTR spending requirement. Officials at DOD said they changed the  agency’s policy on exclusions as of fiscal year 2013 and that the new  policy, which will not allow these improper exclusions, is currently being  implemented. DOT provided no further information on its exclusion. HHS  met its overall agency spending requirement even with the improper  exclusions, according to the data in HHS’ annual reports to SBA.
In addition to identifying improper exclusions, we found that, when  appropriations were received late in the year, agencies used differing  methodologies to calculate their spending requirements, making it difficult  to determine whether agencies’ calculations were correct. For example,  some agency program managers told us that, when appropriations were  received late in the year, they used their prior year actual spending to  calculate their current year spending requirement, while others calculated  their current year spending requirement using some other methodology.  Specifically, program managers at the National Institute of Standards and  Technology (NIST)—a subunit of Commerce—stated that program  managers at NIST used the past year’s actual SBIR spending to calculate  the current year’s requirement. In contrast, NASA calculated its SBIR  spending requirement by determining what percentage its extramural  R&D spending comprised of its total R&D spending in the prior year.  NASA then applied this percentage to the current year’s total R&D budget  to calculate the current year’s extramural budget, which it then used as  the basis for calculating the SBIR and STTR spending requirements.  Although SBA provided guidance in policy directives for participating  agencies on calculating their spending requirements, neither SBA’s prior  nor its current policy directives provide guidance to agencies on how to  calculate such spending requirements when agency appropriations are  delayed. Without such guidance, agencies will likely continue to calculate  spending requirements in differing ways.
section_title: Agencies and SBA Have Not Consistently Complied with Certain Reporting Requirements
paragraphs: Agencies participating in the SBIR and STTR programs have not  consistently complied with Small Business Act requirements for annually  reporting a description of their methodologies for calculating their  extramural R&D budgets to SBA. In addition, SBA has not consistently  complied with the act’s requirements for annually reporting to Congress,  including reporting on SBA’s analysis of the agencies’ methodologies for  calculating their extramural R&D budgets.
subsections: section_title: Agencies’ Reporting of Methodologies Did Not Consistently Comply with Requirements
paragraphs: With the exception of NASA in certain years, agencies did not submit their  methodology reports to SBA within the time frame required by the Small  Business Act for fiscal years 2006 to 2011 for the SBIR and STTR  programs. The act requires that agencies report to SBA their  methodologies for calculating their extramural budgets within 4 months  after the date of enactment of their respective appropriations acts.  However, most participating agencies documented their methodologies  for calculating their extramural R&D budgets for these fiscal years and  submitted them to SBA after the close of the fiscal year with their annual  reports, but three agencies—USDA, Education, and DOT—did not  provide a methodology report for 1 fiscal year during this period. USDA  did not submit a report on its fiscal year 2007 methodology because  agency officials said it was identical to prior years. Officials from  Education and DOT said they typically submitted their methodology  reports with their annual reports. However, they told us that for fiscal year  2011 they did not submit their methodology reports to SBA on time  because that was the first year that agencies were required to submit  their annual reports to SBA’s automated system and there was not a  place in SBA’s system to submit methodology reports. SBA officials said  that they nonetheless expected agencies to submit their methodology  reports and that there are several methods to transmit this information,  such as by memorandum or e-mail. Education officials later told us they  submitted their 2011 methodology report to SBA in January 2013.
SBA officials said that they have not held the agencies to the act’s  deadline for submitting methodology reports, in part because continuing  resolutions enacting final appropriations have sometimes not been  passed until the middle of the fiscal year. This timing for appropriations  has pushed the required reporting date of the methodology report—due 4  months after appropriations are set—until late in the fiscal year. SBA  officials said this has made it more convenient for participating agencies  to submit the methodology report with the annual report. Further, SBA  officials said the agency uses the methodology reports for their annual  reports to Congress. By not having the methodology reports earlier in the  year as specified by law, however, SBA does not have an opportunity to  promptly analyze these methodologies and provide the agencies with  timely feedback to assist agencies in accurately calculating their spending  requirements. SBA officials said they have provided feedback orally and  through e-mails to the participating agencies about their methodology  reports, but many agency program managers said that SBA has provided  little feedback. By not providing such feedback, SBA is forgoing the  opportunity to assist agencies in correctly calculating their program  spending requirements and helping to ensure that they meet mandated  requirements.
More significantly, the majority of the agencies did not include information  consistent with a provision in SBA’s SBIR and STTR policy directives that  specifies the act’s requirement for a methodology report from each  agency. Specifically, the SBA policy directives state that the  methodology report must include an itemization of each R&D program  excluded from the calculation of the agency’s extramural budget and a  brief explanation of why it is excluded. In our review, we found that  two of the participating agencies—EPA and HHS—complied fully with  the requirements because they included in their methodology reports  an itemization of the programs excluded from the calculation of their  extramural R&D budget and an explanation of why the programs were  excluded for all 6 fiscal years in our review; and six agencies—DHS, DOD, DOE, DOT, NASA and NSF—did not fully  meet these requirements for the 6 fiscal years in our review because  their methodology reports either identified some excluded programs  but not others that we identified or the reports omitted explanations for  exclusions.
As a result, agencies submitted different information, including different  levels of detail on their methodologies. For example, some agencies  provided itemization of each R&D program excluded, including dollar  amounts and statutory authority, as part of the calculation of the agency’s  extramural budget and a brief explanation of why it is excluded, while  other agencies only provided a brief explanation. SBA officials told us that  most participating agencies’ methodology reports have changed little from  year to year, so SBA does not raise questions about details of their  methodologies. In the absence of guidance from SBA on the format in  which the methodology reports are to be presented, DOD developed a  methodology template that guides the calculation of DOD’s extramural  R&D budget and in turn the programs’ spending requirements, including  the identification of any R&D programs excluded from the basis for  calculating their spending requirements and a brief explanation of why  they are excluded. Without guidance on the format of methodology  reports, participating agencies are likely to continue to provide SBA with  broad, incomplete, or inconsistent information about their methodologies  and spending requirements. Furthermore, without having more consistent  information from agencies, it is difficult for SBA to comprehensively  analyze the methodologies and determine whether agencies are  accurately calculating their spending requirements.
In addition, for the agency annual report requirement, SBA has provided a  template that asks agencies for the extramural R&D budget base used to  calculate the SBIR or STTR spending requirements, but it does not ask  for the specific calculations used to derive that budget base. Unlike the  requirement set by law and SBA policy directives for methodology reports  to include a description of their methodologies for calculating extramural  R&D budgets, information on actual calculations, such as identifying  exclusions, is not required for agency annual reports to SBA. However,  because annual reports show the results of the previously described  methodology including such information in the annual reports is important.  By not requesting that agencies include calculations used to derive the  budget base in its template, SBA has been receiving incomplete  information from the participating agencies, which limits the usefulness of  data the agency reports to Congress. SBA officials told us that  participating agencies’ calculations of spending requirements have  changed little from year to year, and so SBA does not raise questions  about the calculations. SBA likewise does not request that agencies  include information in their annual reports that would enable SBA to  conduct better oversight, including information on (1) whether agencies  met the mandated spending requirements, (2) the reasons for any  noncompliance with these requirements, and (3) the agencies’ plans for  meeting any noncompliance in future years. By including this information,  SBA could more fully oversee the programs and provide more complete  information to Congress.
section_title: SBA Annual Reports to Congressional Committees Are Often Late and Incomplete
paragraphs: SBA has not consistently complied with the requirement for reporting its  analysis of the agencies’ methodologies in its annual report to  Congress, as required by the Small Business Act. Over the 6 years  covered in our review, SBA reported to Congress for 3 of those, fiscal  years 2006, 2007, and 2008. Furthermore, these reports contained limited  analyses of the agencies’ methodologies, and some of the analyses were  inaccurate. For example, SBA’s analysis was limited to a table attached  to the annual report to Congress that often did not include information on  particular agencies; SBA provided no other documentation showing the  results of its analysis of the agency methodology reports. For example, in  its fiscal year 2006 annual report, SBA concluded that all of the  participating agencies complied with program requirements in calculating  their extramural R&D budgets but did not present the basis for its  conclusion. As noted earlier, our review showed that some participating  agencies improperly excluded some extramural programs from their  funding base calculation and did not consistently comply with SBA’s  instructions in its policy directive to itemize the exclusions in their  calculation of the R&D extramural budget for either the SBIR or STTR  program. Without more comprehensive analysis and accurate information  on participating agencies in SBA’s annual report, Congress does not have  information on the extent to which agencies are reporting what is required  by law or if they are under spending by, for example, taking improper  exclusions.
Moreover, SBA officials said they delayed submitting their annual reports  to Congress for fiscal years 2009, 2010, and 2011 to reconcile significant  inconsistencies the agency found between spending data submitted by  participating agencies in their annual reports to SBA and data routinely  collected in SBA’s automated system from agencies and awardees on  SBIR and STTR awards made during the fiscal year. In commenting on a  draft of this report in August 2013, SBA program officials said that the  three reports had been consolidated into one report that was being  reviewed by the Office of Management and Budget. The agency plans to  submit the reports to Congress in 2013, making the data available to  Congress on the programs 2 to 4 years after the end of the fiscal year.
section_title: Basing Spending Requirements on Agencies’ Total R&D Budgets Could Increase Spending and Program Participation
paragraphs: Changing the methodology to calculate the SBIR and STTR spending  requirements based on each agency’s total R&D budget instead of each  agency’s extramural R&D budget would increase the amount of each  agency’s spending requirement for the programs, some much more than  others, depending on the assumptions about how the funding base  change is implemented. Also, it would increase the number of agencies  that would be required to participate in the programs. Some agencies  reported that such a change could have effects on their R&D programs  and may create challenges.
subsections: section_title: Changing the Base for SBIR and STTR Budgets to Total R&D Would Increase Agency Spending Requirements
paragraphs: If the SBIR and STTR spending requirements under current law were  applied to an agency’s total R&D budget rather than to the extramural  R&D budget, the spending requirements for each agency would increase  because their extramural R&D budget is a part of, and therefore smaller,  than their total R&D budget. Table 3 shows a comparison of the  agencies’ spending requirements for the SBIR and STTR programs in  fiscal year 2011 under the current law, based on an agency’s extramural  R&D budget, and this alternative methodology.
As shown in table 3, some agencies’ spending requirements would  increase more than others. This is due to differences in the relative  proportions of the extramural and intramural R&D budgets among  agencies. Examples are as follows:
NSF used almost its entire R&D budget for extramural R&D in fiscal  year 2011 and was required to spend about $124 million on its SBIR  program in that year. Its spending requirement would have increased  slightly to $127 million—about a 3 percent increase—if the spending  requirement were based on the total R&D budget instead of the  extramural R&D budget. For NSF’s STTR program, the spending  requirement in fiscal year 2011 would have increased about 3  percent—the same percentage increase as for SBIR.
Commerce used a relatively small percentage of its total R&D  budget—about 25 percent—for extramural R&D in fiscal year 2011,  and its spending requirement for SBIR would have more than  quadrupled from about $6 million to $26 million—333 percent—in  fiscal year 2011 if the calculation methodology changed. While  Commerce does not participate in the STTR program currently, it  would have to participate in the STTR program if the calculation  methodology changed. Its spending requirement would have been $3  million rather than zero.
To put these figures in perspective, if the funding percentage in law were  applied to the total R&D budget instead of the extramural budget, NSF’s  spending on SBIR in 2011 would have increased to about 2.6 percent of  its extramural R&D budget, while Commerce’s spending would have  increased to about 10.7 percent of its extramural R&D budget. For the  STTR program, if the funding percentage in the law were applied to the  total R&D budget instead of the extramural budget, NSF’s spending on  STTR would have increased to about 2.7 percent of the extramural R&D  budget, and Commerce’s spending would have increased from zero.
In addition to the changes in the dollar amount of funds available for  STTR and SBIR spending requirements, agencies said that changing the  base for calculation of budgets for these programs would affect agency  operations, depending on assumptions about how the funding base  change is implemented. For example, changing the base would increase  SBIR and STTR budgets and could result in reductions in certain types of  intramural R&D with corresponding reductions in full-time equivalent  staffing of these programs. In addition, some agency officials said there  were potential changes in the content of the agency’s extramural R&D  effort because of changes in the types of businesses that receive grants  and contracts.
In addition to applying the same percentages as used under current law  to the total R&D budget, we analyzed the potential changes to spending  requirements using two other alternative scenarios that apply different  percentages to the total R&D budget. In these scenarios, some agencies  would have experienced an increase in spending requirements, while  others would have experienced a decrease. Appendix II contains a  discussion of the alternate scenarios and the results of our analysis.
section_title: Changing the Calculation Methodology Would Increase the Number of Agencies Required to Participate
paragraphs: Changing the calculation methodology to the total R&D budget would also  increase the number of agencies that would be required to participate in  the SBIR and STTR programs, assuming the same dollar thresholds for  participating in the programs were applied to the total R&D budget rather  than only the extramural R&D budget. For example, our analysis of the  total R&D budget for all federal agencies for fiscal year 2011 indicates  that
For SBIR, two additional agencies—the Departments of Veterans  Affairs (VA) and the Interior—would have been required to participate  in fiscal year 2011 if total R&D budgets had been the criteria. These  agencies reported total R&D budgets to SBA in excess of $100  million, which is the threshold for participation in the SBIR program.  Adding these two agencies to participating SBIR agencies in fiscal  year 2011 with the total R&D budget as the base, would have  increased total federal SBIR spending by $48 million.
For STTR, three additional agencies—Commerce, USDA, and VA— would also have been required to participate in the STTR program for  fiscal year 2011 if total R&D budgets had been the criteria for meeting  the threshold. These agencies reported total R&D budgets in excess  of $1 billion, which is the threshold for participation in the STTR  program. Adding these three agencies to STTR in fiscal year 2011  with the total R&D budget as the base would have increased total  STTR spending by $13 million.
Table 4 shows these agencies’ R&D budgets and what their SBIR and  STTR spending requirements for fiscal year 2011 would have been if the  spending methodology was changed to the total R&D budget.
section_title: Agencies’ Cost for the Administration of the Programs Cannot Currently Be Determined Because the Agencies Do Not Identify or Track All Costs
paragraphs: The participating agencies’ cost of administering the SBIR and STTR  programs cannot be determined because the agencies neither collect that  information nor have the systems to do so. Neither the authorizing  legislation for the programs nor SBA guidance directs agencies to track  and estimate all administrative costs, and neither the law nor SBA  guidance defines these administrative costs. Estimates agencies provided  indicated that the greatest amounts of administrative costs in fiscal year  2011 were for salaries and expenses, contract processing, outreach  programs, technical assistance programs, support contracts, and other  purposes. With the implementation in 2013 of a pilot program allowing  agencies under certain conditions to use up to 3 percent of SBIR program  funds for certain administrative costs, SBA expects to require agencies in  the pilot program to track and report the spending of that 3 percent but  not all of their administrative costs.
subsections: section_title: Agencies Have Not Comprehensively Identified or Tracked Administrative Costs for Several Reasons
paragraphs: The participating agencies have not comprehensively identified or tracked  the cost of administering the SBIR and STTR programs for several  reasons. Agency officials said that the costs cannot be determined  because the agencies do not have the systems for collecting the data.  Neither the authorizing legislation for the programs nor SBA guidance  directs agencies to track and estimate administrative costs, and neither  the law nor SBA guidance defines these administrative costs. We found  that the amount of funds that participating agencies spent administering  the SBIR and STTR programs—and the way the funds were used— cannot currently be estimated because the agencies have not identified or  tracked many categories of program administrative costs. Agency officials  said an important reason that administrative costs for the SBIR and STTR  programs are not comprehensively identified or tracked is that using SBIR  or STTR budgets to fund administrative costs has been generally  prohibited. The Small Business Act generally prohibits agencies, except  for DOD, from using any of their SBIR or STTR budgets to fund  administrative costs of the programs, including the cost of salaries.  Agencies reported that administrative costs of the programs were paid out  of budget accounts other than the SBIR and STTR accounts.
In addition, agency officials told us that the SBIR and STTR programs cut  across many agency programs and disciplines and that the staff  supporting the programs may work on a full-time or part-time basis,  making identification and estimation of costs more difficult. For example,  DOE reported the administrative costs of the SBIR and STTR program  office only, but pointed out that the programs involved the part-time efforts  of 70 to 100 additional people throughout DOE, including technical  program managers, grants specialists and contracting officers, whose  costs were not estimated.
Similarly, HHS officials said it would be an exceptionally complex  calculation to determine how much is currently spent on the  administrative costs of the SBIR or STTR program because a large  number of HHS staff work a fraction of their time on these programs.  Officials in HHS’ National Institutes of Health (NIH), which accounts for  the majority of HHS’ SBIR and STTR R&D programs, said there were a  small number of full-time staff on these programs; rather, NIH officials  said that most staff managing the programs do so as a collateral part of  their duties and are not required to track the portion of their time spent on  the programs. NASA reported that its budget estimate included a  separate line for SBIR and STTR program management that covers  personnel costs, travel, and procurement costs. NASA officials noted,  however, that other costs to operate the programs are not included in this  budget estimate, including the cost of NASA technical experts to review  proposals and the cost of technical and contracting representatives  interacting with small businesses. NASA officials did provide a rough  estimate for the number of hours and full time equivalent staff spent by  NASA technical reviewers and contracting personnel in a typical year as  25 to 38 full-time equivalent staff. The officials noted that the estimate  does not include hours spent by others involved: mission directorate  representatives, center chief technologists, contracting officers, support  contractors, procurement support, and legal support. They also said that  they did not have estimates for such categories as support contracts,  outreach, and technical assistance.
section_title: Most Estimated SBIR and STTR Administrative Costs Were in Salaries and Expenses, Contract Processing, Outreach and Technical Assistance, Support Contracts, and Other Categories
paragraphs: In response to our request for information on administrative costs for  fiscal year 2011, 9 of the 11 participating agencies provided us with  estimates of a portion of their costs to administer the SBIR and STTR  programs in fiscal year 2011. Of the administrative costs estimated by  these 9 agencies, the greatest amounts were for salaries and expenses,  contract processing costs, outreach programs, technical assistance  programs, and support contracts, and the “other” category. In some  cases, officials for some agencies identified having costs in these  categories or several others but provided no estimates of the amounts.  The agency with the most administrative costs estimated in the most  categories for 2011 was DOD, which provided estimates in 10 cost  categories.
Of the 11 participating agencies, Commerce and HHS did  not provide estimated administrative costs or identify having  administrative costs in any category.
In response to our data requests and questions regarding fiscal year  2011, the 9 agencies provided some estimates, identified unestimated  costs, or had no response in many of the costs categories for which we  requested data. An overview of the information we obtained is contained  in appendix III.
section_title: Participation in the Administrative Costs Pilot Program Will Require Agencies to Track Certain, but Not All Costs
paragraphs: As noted earlier, the National Defense Authorization Act for Fiscal Year  2012 created a pilot program beginning in fiscal year 2013 that would  allow up to 3 percent of SBIR program funds to be used for administrative  costs, the provision of outreach and technical assistance and contract  processing, and other specified purposes. Agencies are otherwise  generally not permitted to spend SBIR or STTR program funds on  administrative costs.
According to SBA’s policy directive, funding under this pilot is not  intended to and must not replace current agency administrative funding in  support of SBIR activities. Rather, funding under this pilot program is  intended to support additional initiatives. SBA issued its guidance for the  pilot program as part of its revised policy directive of August 2012 and  requires agencies to submit annual work plans to SBA for approval on  spending priorities, amounts, milestones, expected results, and  performance measures before agencies can begin the pilot.guidance also directs agencies to report to them on the use of the funds  allowed to be spent on administrative costs under the pilot program  authority in their annual reports. However, agencies will not identify or  track all of their administrative costs so SBA will not be able to report to  Congress on total administrative costs. Of the 11 agencies participating in   The SBA  the SBIR program, 10 have submitted plans for the pilot program to SBA.  SBA officials told us that, as of August 2013, all 10 of the agencies’ pilot  plans had been approved for implementation in the current fiscal year.
section_title: Conclusions
paragraphs: To help small businesses develop and commercialize innovative  technologies, federal agencies have awarded billions of dollars to such  businesses under the SBIR and STTR programs, which SBA oversees. In  its role overseeing the programs, SBA has issued policy directives that  provide agencies with guidance on the implementation of the programs.  Agencies participating in the programs are required by law to spend a  specific minimum portion of their extramural R&D budgets on these  awards and to report certain information related to their spending to SBA.  In turn, SBA is to review this information and report on it annually to  Congress. However, participating agencies’ compliance with the  programs’ spending requirements is unclear because some agencies  improperly calculated their spending requirements and—in the absence of  specific guidance from SBA when their appropriations were delayed— agencies used differing methodologies for calculating these requirements.  Without guidance from SBA, agencies will likely continue to calculate  spending requirements in differing ways, which will continue to raise  questions about their compliance. In addition, most agencies’ reports to  SBA about their methodologies for calculating their spending  requirements did not contain key details, such as the identification of any  R&D programs excluded from the basis for calculating their spending  requirements and a brief explanation of why they are excluded, which is  required both by law and SBA policy directives. Agencies also submitted  differing information in these reports because SBA’s policy directives do  not specify the format for the reports. Without guidance on the format of  methodology reports, participating agencies are likely to continue to  provide SBA with broad, incomplete, or inconsistent information about  their methodologies and spending requirements. Furthermore, without  more complete and consistent information from agencies, it is difficult for  SBA to comprehensively analyze the methodologies and whether  agencies are accurately calculating their spending requirements.  Moreover, according to agency officials, SBA provided little timely  feedback about the agencies’ methodology reports. By not providing such  feedback, SBA is forgoing the opportunity to assist agencies in correctly  calculating their program spending requirements and helping to ensure  that they meet mandated requirements.
In addition, for the participating agencies’ annual report requirement, SBA  has provided a template requesting the extramural R&D budget base that  agencies used to calculate the programs’ spending requirements, but the  template does not request the specific calculations agencies used to  derive those requirements. By not requesting such calculations, SBA has  been receiving inconsistent and incomplete information from the  participating agencies, which limits the usefulness of data it reports to  Congress. SBA likewise does not request that agencies include  information in their annual reports that would enable better oversight,  including information on (1) whether agencies met the mandated  spending requirements, (2) the reasons for any noncompliance with these  requirements, and (3) the agencies’ plans for meeting any noncompliance  in future years. Finally, SBA’s annual reports to Congress have been  years late or contained little analysis of the methodology reports agencies  submitted to describe how they calculated their spending requirements.  Without more rigorous oversight by SBA and more timely and detailed  reporting on the part of both SBA and participating agencies, it will be  difficult for SBA to ensure that intended benefits of these programs are  being attained and that Congress receives critical information to oversee  these programs.
section_title: Recommendations for Executive Action
paragraphs: To ensure that participating agencies and SBA comply with spending and  reporting requirements for the SBIR and STTR programs, we recommend  the SBA Administrator take the following four actions:
Provide additional guidance on how agencies should calculate  spending requirements when agency appropriations are received late  in the fiscal year and the format agencies are to include in their  methodology reports.
Provide timely annual feedback to each agency following submission  of its methodology report on whether its method for calculating the  extramural R&D budget used as the basis for the SBIR and STTR  spending requirements complies with program requirements including  an itemization of and an explanation for all exclusions from the basis  for the calculations.
Direct participating agencies to include in their annual reports the  calculation of the final extramural R&D budget used as the basis for  their SBIR and STTR spending requirements and, if they did not meet  the spending requirements, the reasons why not and how they plan to  meet the spending requirements in the future.
Provide Congress with a timely annual report that includes a  comprehensive analysis of the methodology each agency used for  calculating the SBIR and STTR spending requirements, providing a  clear basis for SBA’s conclusions about whether these calculations  meet program requirements.
section_title: Agency Comments
paragraphs: We provided copies of our draft report to the Secretaries of USDA,  Commerce, DOD, Education, DOE, HHS, DHS, and DOT; the  Administrators of SBA, EPA, and NASA; and the Director of NSF for  review and comment.  In response, six of the agencies—USDA,  Education, DOE, EPA, NASA, and NSF—stated by e-mail that they had  no technical or written comments.  Five other agencies—Commerce,  DHS, DOD, DOT, and HHS—provided technical comments by e-mail,  which we incorporated into the draft report as appropriate.
SBA provided technical comments on the draft report and officials of  SBA’s Office of Technology said by e-mail through the Office of  Congressional and Legislative Affairs that they agree with the findings of  the report and will work to implement the recommendations.  Specifically,  in response to our recommendation to provide additional guidance on  how agencies should calculate spending requirements, SBA said it plans  a training session for all SBIR and STTR agencies to provide guidance  and uniformity in the calculation of extramural budgets. In response to our  finding that SBA is not receiving timely methodology reports from the  agencies in order to provide feedback, SBA said it has strongly  encouraged the agencies to submit their methodologies to SBA in a  timely manner. We incorporated SBA’s technical comments into the  report as appropriate.
We are sending copies of this report to the Secretaries of Agriculture,  Commerce, Defense, Education, Energy, Health and Human Services,  Homeland Security, and Transportation; the Administrators of the Small  Business Administration, the Environmental Protection Agency, and the  National Aeronautics and Space Administration; the Director of the  National Science Foundation; the appropriate congressional committees;  and other interested parties. In addition, this report is available at no  charge on the GAO website at http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at (202) 512-3841 or ruscof@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. Key contributors to this report are listed in  appendix IV.
section_title: Appendix I: Comparison of Participating Agencies’ Reported Requirements and Spending for the SBIR and STTR Programs, Fiscal Years 2006 to 2011
paragraphs: Figures 2 through 12 compare reported spending requirements of the 11  agencies participating in the Small Business Innovation Research (SBIR)  program with their reported spending over fiscal years 2006 to 2011.
Figures 13 through 17 compare reported spending requirements of the 5  agencies participating in the Small Business Technology Transfer (STTR)   The  program with their reported spending over fiscal years 2006 to 2011.Department of Homeland Security also had an STTR program in FY 2006  and 2007. However, the agency was not required to participate, because  its extramural R&D budget for each of those years was actually below the  $1 billion threshold required for participation in the STTR program. The  agency stated it had inadvertently used an incorrect extramural budget  amount to determine its participation requirement. Since the agency was  not required to participate in STTR and therefore had no spending  requirement, no figure is included here for its STTR expenditures.
section_title: Appendix II: Analysis of Alternative Scenarios for Basing SBIR and STTR Spending Requirements on Total R&D Budget
paragraphs: To calculate the expenditure requirements for the SBIR and STTR  programs, we used two key variables: (1) the “base,” which is the  research and development (R&D) funding from which the requirement is  calculated, and (2) the percentage that is applied to that base. For  example, for fiscal years 2006 to 2011, the base for SBIR and STTR  funding is the extramural R&D budget, and the mandated percentages  applied to that base were 2.5 and 0.3 percent, respectively. We tested  three alternative scenarios that vary the percentage applied to the total  R&D budget to illustrate the potential effects of changing the methodology  to calculate agencies’ SBIR and STTR expenditure requirements. The  scenarios analyzed were as follows:
For scenario 1, we applied the same percentages for the expenditure  requirements under the current law to the total R&D budget instead of  the extramural R&D budget.
For scenario 2, using fiscal year 2006 numbers as our base, we  determined the percentage to apply to the total R&D budget of all  participating agencies for fiscal years 2006 through 2011 that would  hold the total expenditure requirement constant for the programs.
For scenario 3, using fiscal year 2006 numbers as our base, we  determined the percentage to apply to the total R&D budget of each  individual agency for fiscal years 2006 through 2011 to hold each  individual agency’s expenditure requirement constant for the  programs.
See the details of the three scenarios and current law in table 5, and their  effects on spending requirements are in table 6.
section_title: Appendix III: Description of SBIR and STTR Administrative Cost Data Obtained from the Participating Agencies
paragraphs: The following describes the administrative cost data obtained, identified,  or not available from the participating agencies for fiscal year 2011.
Department of Commerce (Commerce): Officials with the two  participating subunits, the National Institute of Standards and  Technology (NIST) and the National Oceanic and Atmospheric  Administration (NOAA), said that the administrative costs for the  program included salaries and expenses but that they did not have an  estimate of them. The officials said the agencies did not specifically  track administrative costs. Such costs were not allowed to be charged  against the SBIR funds.
Department of Homeland Security (DHS): Officials in DHS’ Science  and Technology Directorate, which is one of two subunits managing  the SBIR program at DHS, provided a partial list of administrative  costs for the fiscal year. These included salary, travel, and other costs  (e.g. contracting fees, support contracts and audit costs) that were  estimated at $962,000. Categories of costs that the agency identified  but did not estimate included salaries and expenses of other DHS  supporting staff and contractors. DHS officials in the Science and  Technology Directorate said that the directorate’s management and  administrative budget began fully funding the administrative costs for  SBIR in 2011; previously, these costs were funded from the  extramural R&D budget of the directorate. The other DHS unit with an  SBIR program, the Domestic Nuclear Detection Office did not identify  or estimate administrative costs.
DOD: Agency officials said that while DOD had not tracked  administrative costs of the SBIR or STTR programs through 2011  agency-wide, such costs had been reported to various extents by the  13 DOD subunits that participate in one or both of the programs.  Based on reports from some subunits, DOD’s partial administrative  costs totaled $30.2 million. The 13 DOD subunits varied in their  identification of administrative costs. Some identified none; some  identified a few; and others identified many categories but did not  provide estimates for each cost category. DOD Office of Small  Business Programs officials stated that the department did not track  “non-legislated administrative expenses,” which were described as the  program administrative costs before the start of the administrative  costs pilot program.
Department of Energy (DOE): Agency officials in the DOE Office of  Science, which manages most of the SBIR and STTR programs in  DOE, said administrative costs for these programs were in three  categories: salaries and benefits, support contracts, and travel, and  totaled $1.2 million in fiscal year 2011. According to these officials,  these costs did not include personnel expenses for over 70 specialists  who spend a fraction of their time on the programs.
Department of Transportation (DOT): Agency officials said  administrative costs in fiscal year 2011 were estimated at $363,000,  primarily for salaries and expenses but also including travel and other  smaller categories. These program managers said this represents  part of its administrative costs that directly support the SBIR  program’s management but not other support activities like  procurement and legal services that are provided by other DOT  subunits.
Department of Education (Education): Agency officials estimated  administrative costs for fiscal year 2011 totaled $479,000, of which  $4,000 was for travel, $174,000 for salaries and benefits of  department employees who administer the program, including  preparing solicitations, running competitions, performing oversight,  congressional reporting, and monitoring awards. Officials said the  2011 total included $38,000 for salary and benefits of the department  contracts and acquisitions management staff, and $263,000 for salary  and benefits of personnel assisting with application reviews.
Environmental Protection Agency (EPA): Agency officials said that  known administrative costs for fiscal year 2011 were $953,000, which  included $533,000 for the salaries and expenses of four FTE staff that  run the program, $350,000 for external peer review of SBIR proposals  at various funding phases, and $70,000 for a contract to provide  program support. Officials said there were other administrative costs  that were associated with staff that also manage other grant programs  and that these costs were not easily separated by program and they  were not tracked.
Department of Health and Human Services (HHS): Agency officials  said they do not track or report the amount of administrative costs for  the SBIR and STTR programs. They said it would be very difficult to  determine how much is currently spent on the administrative costs of  the program. HHS officials reported that since authorizing legislation  did not allow SBIR/STTR funds to be spent on administration, funding  for administrative costs, such as salary and expenses, training and  travel, comes from other accounts.
National Aeronautics and Space Administration (NASA): Agency  officials estimated administrative costs for certain categories as  roughly $11.9 million in fiscal year 2011, which included, among other  things, $8.6 million for procurement costs, about $3 million for salaries  and expenses, and $151,000 for travel. According to these officials,  other identified unestimated and untracked costs to administer the  SBIR and STTR programs include the costs of technical experts  within NASA reviewing proposals, the cost of holding review panels,  and the cost that technical and contracting representatives spend  interacting with companies seeking and receiving funding.
National Science Foundation (NSF): Agency officials said they  identified administrative costs of $4 million for the SBIR and STTR  programs. These include 10 FTE within the agency costing  approximately $2 million in salaries and benefits and $2 million NSF  designates from its extramural research and spends for SBIR and  STTR administrative costs, primarily for contracted technical and  administrative support. NSF has contracted for this support for many  years because of the high volume of actions in the program and the  time frames that need to be met in the process. NSF officials said  there were other administrative costs, including the efforts of federal  staff that devote substantial time to the programs, but these have not  been tracked or estimated.
U.S. Department of Agriculture (USDA): Agency officials said that in  fiscal year 2011 administrative costs for SBIR included $184,000 for  experts who provided peer review of project proposals to cover such  costs as honoraria and travel. Officials said USDA does not break out  administrative costs for the SBIR program beyond honoraria and  travel.
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact:
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Tim Minelli, Assistant Director;  Hilary Benedict; Antoinette Capaccio; Cindy Gilbert; Richard Johnson;  Cynthia Norris; John Scott; Ilga Semeiks; and Vasiliki Theodoropoulos  made key contributions to this report.