Nanotechnology: Improved Performance Information Needed for Environmental, Health, and Safety Research
Published: 2012-05-21T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Nanotechnology involves the ability to control matter at approximately 1 to 100 nanometers. Worldwide trends suggest that products that rely on nanotechnology will be a $3 trillion market by 2020. However, some of the EHS impacts of nanotechnology are unknown. The NSTC coordinates and oversees the NNI, an interagency program that, among other things, develops national strategy documents for federal efforts in nanotechnology.
In this context, GAO examined: (1) changes in federal funding for nanotechnology EHS research from fiscal years 2006 to 2010; (2) the nanomaterials that NNI member agencies EHS research focused on in fiscal year 2010; (3) the extent to which NNI member agencies collaborate with stakeholders on this research and related strategies; and (4) the extent to which NNI strategy documents address desirable characteristics of national strategies. GAOs review included seven NNI agencies that funded 93 percent of the EHS research dollars in fiscal year 2010. This report is based on analysis of NNI and agency documents and responses to a questionnaire of nonfederal stakeholders.
section_title: What GAO Found
paragraphs: From fiscal years 2006 to 2010, the National Science and Technology Council (NSTC) reported more than a doubling of National Nanotechnology Initiative (NNI) member agencies funding for nanotechnology environmental, health, and safety (EHS) researchfrom approximately $38 million to $90 million. Reported EHS research funding also rose as a percentage of total nanotechnology funding over the same period, ending at about 5 percent in 2010. However, GAO identified several reporting problems that raise concerns about the quality of EHS funding data reported. For example, for 18 percent of the 2010 projects GAO reviewed that were reported as EHS research, it was not clear that the projects were primarily directed at EHS risks. In addition, NNI member agencies did not always report funding using comparable data. The absence of detailed guidance on how agencies should report funding for their nanotechnology research has contributed to these problems, as GAO also reported in 2008 and made a related recommendation.
In 2010, EHS research at the NNI member agencies GAO reviewed most frequently focused on carbon nanotubes, nanosilver, and nanoscale titanium dioxide. NNI has not prioritized nanomaterials for EHS research, but NNIs 2011 EHS research strategy outlines criteria for NNI member agencies to use in doing so. It is too soon to tell how these criteria will influence NNI member agencies decisions about which nanomaterials to prioritize, and it is unclear if information needed to use the NNI criteria is available.
The NNI member agencies have collaborated extensively on EHS research and strategies. They have collaborated through the NSTC to develop joint EHS research strategies and have initiated numerous formal collaborative EHS research projects. Nonfederal stakeholders who responded to GAOs web-based questionnaire on nanotechnology EHS research told GAO that they benefited from collaboration with the NNI member agencies but identified some challenges, including a lack of funding and limited awareness of collaboration opportunities, among others. Most respondents rated the 2011 NNI EHS research strategy as somewhat or very effective at addressing nanotechnology EHS research needs.
NNI strategy documents for EHS research issued by the NSTC address two and partially address the other four of the six desirable characteristics of national strategies identified by GAO that offer a management tool to help ensure accountability and more effective results. For example, the NNI strategy documents provide a clear statement of purpose, define key terms, and discuss the quality of currently available data, among others. However, they do not include performance informationsuch as performance measures, targets, and time frames for meeting those measuresthat would allow stakeholders to evaluate progress towards the goals and research needs of the NNI. In addition, the documents do not include, or sufficiently describe, estimates of the costs and resources needed for the strategy. Without this information, it may be difficult for agencies and stakeholders to implement the strategy and report on progress toward achieving the research needs and assess if investments are commensurate with costs of the identified needs.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Director of the Office of Science and Technology Policy (OSTP), which administers the NSTC, (1) coordinate development of performance information for NNI EHS research needs and publicly report this information; and (2) estimate the costs and resources necessary to meet the research needs. OSTP and the seven included agencies neither agreed nor disagreed with the recommendations.

section_title: Letter
paragraphs: Nanotechnology is the understanding and control of matter at  approximately 1 and 100 nanometers, known as the nanoscale. For  illustration, a sheet of paper is about 100,000 nanometers thick, a human  hair is about 80,000 nanometers in diameter, and three gold atoms lying  side by side are about 1 nanometer long. Unusual properties can emerge  in materials manufactured at the nanoscale—including electrical,  magnetic, mechanical, optical, and thermal properties—that differ in  important ways from the properties of conventionally scaled materials.  Materials at this scale that have been manufactured are sometimes  referred to as engineered nanomaterials. Nanotechnology has enabled  or facilitated novel research in areas such as computing, medicine,  energy conversion and storage, water purification, agriculture and food  systems, synthetic biology, aerospace, and geoengineering. Consumer  products that incorporate nanotechnology are as diverse as clothing,  cosmetics, household appliances, and sporting goods. For example,  nanoscale particles of titanium dioxide used in sunscreens act as physical  filters that absorb UV light without affecting the opacity of the product. A  2010 study estimated that values for products enabled by nanotechnology  were worth about $91 billion in the United States and $254 billion  worldwide in 2009. Trends suggest that the number of nanotechnology  products and workers employed in related fields will double every 3 years  worldwide, achieving a $3 trillion market and 6 million workers by 2020.
Even with its increasing commercialization, much remains unknown about  nanotechnology, including some of the environmental, health, and safety  (EHS) impacts of nanomaterials. For example, there are few tools and  methods—such as models to predict the behavior of nanomaterials in the  environment—for conducting research. It is difficult to assess the risk of  nanomaterials because these materials are too varied to generalize how  they will behave; nonetheless, risks associated with particular uses of  specific nanomaterials can be assessed. Further, there is little information  on the number of workers exposed to nanomaterials in the workplace or  the effects on human health of such exposure. However, as we reported  in 2010, some research indicates that the toxicity of certain  nanomaterials, such as some forms of carbon nanotubes and nanoscale  titanium dioxide, may pose a risk to human health.
As of 2010, the NNI member agencies include the Departments of Defense, Education,  Energy, Homeland Security, Justice, Labor, State, Treasury, and Transportation; the  Department of Agriculture’s National Institute of Food and Agriculture and Forest Service;  the Department of Commerce’s Bureau of Industry and Security, National Institute of  Standards and Technology, and the U.S. Patent and Trademark Office; the Department of  Health and Human Services’ Food and Drug Administration, National Institute for  Occupational Safety and Health, and the National Institutes of Health; the Department of  the Interior’s U.S. Geological Survey; the Consumer Product Safety Commission;  Environmental Protection Agency; intelligence agencies; the International Trade  Commission; National Aeronautics and Space Administration; Nuclear Regulatory  Commission; National Science Foundation; Office of Management and Budget; and Office  of Science and Technology Policy. support to the NSET. The NNI, through the NNCO and appropriate  agencies, among others, is directed to (1) establish goals, priorities, and  metrics for evaluation for federal nanotechnology research, development,  and other activities; (2) invest in federal research and development  programs in nanotechnology and related sciences to achieve these goals;  and (3) provide for interagency coordination of federal nanotechnology  research, development, and other activities.
The federal nanotechnology commitment is significant. Cumulative  reported NNI funding since fiscal year 2001 amounts to more than $14  billion. Nanotechnology research projects and activities undertaken by the  NNI member agencies are categorized into eight program component  areas. Agencies report their research funding for each program  component area to the Office of Management and Budget (OMB) as part  of the annual federal budget process. The seventh program component  area comprises research “primarily directed at understanding the EHS  impacts of nanotechnology development and corresponding risk  assessment, risk management, and methods for risk mitigation,” which  we refer to in this report as EHS research.
These three strategy documents are: (1) NSTC, Committee on Technology,  Subcommittee on Nanoscale Science, Engineering, and Technology, National  Nanotechnology Initiative Strategic Plan (February 2011); (2) NSTC, Committee on  Technology, Subcommittee on Nanoscale Science, Engineering, and Technology,  National Nanotechnology Initiative Environmental, Health, and Safety Research Strategy  (October 2011); and (3) NSTC, Committee on Technology, Subcommittee on Nanoscale  Science, Engineering, and Technology, The National Nanotechnology Initiative  Supplement to the President’s 2012 Budget (February 2011). economy and levels of government and involve a range of public and  private organizations and entities. The 2011 NNI strategic plan provides  a vision and overarching goals for the program, including responsible  development of nanotechnology, which includes EHS research. The 2011  NNI EHS research strategy expands on the goal of the responsible  development of nanotechnology by describing the state of science and  research needed to ensure that nanotechnology provides maximum  benefits to the environment and human well-being. The NNI Supplement  to the President’s 2012 Budget serves as the annual report for the NNI  and describes recent activities by the NNI member agencies and funding  of the NNI, including the funding level for the program component area  related to EHS research. These NNI strategy documents are the principal  public record of the NSTC’s coordinating activities and reflect the NSTC’s  guidance to agencies as they separately determine their budgets and  activities for nanotechnology EHS research.
In prior work, we have identified six desirable characteristics for national  strategies and reviewed strategies that the federal government has used  to plan for and report on crosscutting issues including emergency  preparedness, financial literacy, influenza, and terrorism for these  characteristics. programs, priorities, resource allocations, and standards that would  enable federal agencies and other stakeholders to implement the  strategies and achieve the identified results. National strategies that  address these characteristics offer policymakers and implementing  agencies a management tool that can help ensure accountability and  more effective results.
GAO, National Capital Region: 2010 Strategic Plan is Generally Consistent with  Characteristics of Effective Strategies, GAO-12-276T (Washington, D.C.: Dec. 7, 2011);  Influenza Pandemic: Further Efforts Are Needed to Ensure Clearer Federal Leadership  Roles and an Effective National Strategy, GAO-07-781 (Washington, D.C.: Aug. 14,  2007); Financial Literacy and Education Commission: Further Progress Needed to Ensure  an Effective National Strategy, GAO-07-100 (Washington, D.C.: Dec. 4. 2006); Combating  Terrorism: Evaluation of Selected Characteristics in National Strategies Related to  Terrorism, GAO-04-408T (Washington, D.C.: Feb. 3, 2004).
In this context, you asked us to review federal nanotechnology EHS  research. This report examines (1) changes in federal funding for  nanotechnology EHS research from fiscal years 2006 to 2010; (2) the  nanomaterials that NNI member agencies focused on in their EHS  research in fiscal year 2010; (3) the extent to which NNI member  agencies collaborate with stakeholders on nanotechnology EHS research  and related strategies; and (4) the extent to which NNI strategy  documents address desirable characteristics of national strategies.
To conduct this work, we reviewed EHS research efforts funded by seven  NNI agencies, which collectively funded 93 percent of EHS research  dollars in fiscal year 2010: the National Science Foundation (NSF),  National Institutes of Health (NIH), Environmental Protection Agency  (EPA), National Institute for Occupational Safety and Health (NIOSH),  Food and Drug Administration (FDA), National Institute of Standards and  Technology (NIST), and the Consumer Product Safety Commission  (CPSC). The first six of these agencies represent the top six providers of  EHS research funding from fiscal year 2006 to fiscal year 2010, and  CPSC has an important role in ensuring the safe use of nanotechnology  in consumer products.
To examine recent changes in federal funding for nanotechnology EHS  research, we reviewed information published by NSTC in the NNI  Supplements to the President’s Budget. Specifically, we reviewed funding  reported in each program component area for fiscal years 2006 through  2010 for all NNI member agencies funding nanotechnology research. For  the dollar amounts that we adjusted for inflation, we used the Biomedical  Research and Development Price Index to report funding in constant  2010 dollars. We consulted with agency and OMB officials to determine  the type of budget information reported in these documents for the  agencies’ investments. For fiscal year 2010, the most recent year for  which agencies’ nanotechnology investment data were available, we also  collected quantitative and qualitative project-level data on all research  projects that the seven selected agencies had categorized as EHS  research (program component area 7). The agencies do not report  project-level data to OMB annually but report their total EHS research  funding to OMB annually for inclusion in the NNI budget supplements  published by NSTC. Therefore, we reviewed data on the individual  projects included in the agencies’ total EHS research funding in 2010. To  assess the reliability of the agencies’ data, we obtained information  regarding the data and the information systems used to produce and  store them. We also reviewed related supporting documentation and  consulted with agency officials. From our assessment, we determined  that the data were sufficiently reliable for our purposes. We analyzed  these data and consulted with agency officials to assess whether the  agencies had appropriately categorized as EHS research the projects  they reported as program component area 7—research “primarily directed  at understanding the EHS impacts of nanotechnology development and  corresponding risk assessment, risk management, and methods for risk  mitigation.” We also reviewed our 2008 report in this area, which found  that 22 of 119 fiscal year 2006 projects reported as EHS research were  miscategorized, and recommended that the Director of OSTP, in  consultation with the Directors of NNCO and OMB, provide better  guidance to agencies on how to report nanotechnology EHS research. To  identify the nanomaterials studied by the fiscal year 2010 EHS research  projects, we reviewed qualitative project data such as abstracts for the  projects we determined were primarily directed at EHS and consulted with  agency officials. We then grouped the nanomaterials into five  categories.
To determine the extent to which the NNI agencies collaborate with  stakeholders on nanotechnology EHS research and related strategies, we  (1) discussed with agency officials how their agencies collaborate on  nanotechnology EHS research and NNI’s role in facilitating that  collaboration and (2) obtained documentation on these collaborative  efforts. We conducted a review of formal collaborative efforts that focused  on nanotechnology EHS research initiated from February 2008 to October  2011. We administered a web-based questionnaire to a nonprobability  sample of 223 nonfederal stakeholders, including those affiliated with  academia, companies, nongovernmental organizations (NGO), and state  and local governments, to obtain their views on collaboration with the NNI  member agencies on EHS research and the NNI EHS research  strategies. Of the 223, 138 completed the questionnaire, for an overall  response rate of 62 percent. This sample included individuals who had  expertise in the field of nanotechnology, had interacted with the NNI in the  past few years, or who were representatives of organizations and  companies suggested to us through our scoping interviews.
To determine the extent to which the NNI strategy documents address  desirable characteristics of national strategies, we analyzed three key  NNI strategy documents related to nanotechnology EHS for the presence  of six desirable characteristics we previously identified for national  strategies. We also reviewed NNI member agencies’ management  documents for the use of performance information related to  nanotechnology EHS research and the extent to which they were linked  with the NNI national priorities. Appendix I presents a more detailed  description of our scope and methodology.
We conducted this performance audit from February 2011 through May  2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Nanomaterials come in a variety of forms based both on their chemical  composition and physical structure. For example, carbon-based  nanomaterials can be produced in a number of physical structures such  as sheets (graphene), tubes (carbon nanotubes), and particles (carbon  black). Nanomaterials can enter the marketplace as materials  themselves, as intermediates that either have nanoscale features or  incorporate nanomaterials, and as final nano-enabled products.
The use of nanomaterials in commercial applications raises questions  about the potential risks that might arise from exposures to nanomaterials  and the differences in exposure during their manufacture, use, and  disposal. For example, some, but not all, research studies have shown  adverse respiratory or cellular effects in animals exposed to some types  of carbon nanotubes. Observed effects include early onset and  persistence of pulmonary fibrosis and interference with cell division. The  risk posed by a material is a combination of the hazard or negative effect  that material may have on an organism and the extent of the organism’s  exposure to that material. Therefore, a highly poisonous material—that is,  one with high hazard—may nonetheless pose little risk if susceptible  groups have little or no contact with the material. For instance, many  household chemicals are hazardous to human health but pose little risk  when exposure is limited by safe handling. Conversely, a material with  relatively mild health effects may pose a large risk if people or the  environment are exposed to large amounts or over prolonged periods.
NRC, A Research Strategy for Environmental, Health, and Safety Aspects of  Engineered Nanomaterials (2012).
Development has developed a list of 13 representative manufactured  nanomaterials now or soon to be in commerce as priority testing  targets.
Regulatory agencies, like FDA, CPSC, and EPA, may not have complete  information on the uses or risks of some nanomaterials. In 2010, we  reported that uncertainties persist about how to evaluate the safety of  engineered nanomaterials in food and that nanomaterials may enter the  food supply in certain products generally recognized as safe without   We also reported in 2010 that EPA has taken a  FDA’s knowledge. variety of actions to better understand and regulate the risks of  nanomaterials but that the agency faces challenges that might impede its  ability to regulate nanomaterials effectively.that the Toxic Substances Control Act gives EPA authority to issue rules  requiring companies to submit certain information about chemicals. EPA  recently amended a regulation to require companies to report certain  information regarding production of chemicals above certain thresholds  but those thresholds may not capture nanomaterials if they are produced  in amounts below the thresholds. EPA currently has no plans to further  reduce the thresholds. In addition, according to officials at CPSC, which is  charged with protecting the public from unreasonable risks of injury or  death from thousands of types of consumer products, CPSC does not  have the statutory authority to require pre-market approval for products,  including those incorporating nanomaterials.
For example, we reported  The NNI was codified in law by the 21st Century Nanotechnology  Research and Development Act in 2003. The act requires the President  to implement a national nanotechnology program and charges the NSTC  itself—or through a subgroup—with overseeing the planning,  management, and coordination of the program. The NSTC carries out  these tasks through the NSET Subcommittee, which includes a co-chair  from OSTP as well as representatives from the member agencies of the  NNI. The NSET oversees working groups, including the Nanotechnology  Environmental and Health Implications (NEHI) working group which  supports federal activities to protect public health and the environment.  Figure 1 provides an overview of the organizational structure of the NNI.
The NNI does not fund research directly; rather, each of its member  agencies determines its nanotechnology activities based on its individual  mission and priorities. The NNI provides a framework for a  comprehensive nanotechnology research and development program by  establishing shared goals, priorities, and strategies among member  agencies; and providing avenues for member agencies to leverage the  resources of all participating agencies. The four goals of the NNI are to  (1) advance a world-class nanotechnology research and development  program; (2) foster the transfer of new technologies into products for  commercial and public benefit; (3) develop and sustain educational  resources, a skilled workforce, and the supporting infrastructure and tools  to advance nanotechnology; and (4) support responsible development of  nanotechnology.
Efforts of the NNI member agencies are reported through triennial  strategic plans and annual budget supplements. The act directs the  NSTC, itself or through an appropriate subgroup it designates or  establishes, to develop and update every 3 years a strategic plan to guide  the activities of the program. The NSTC published its most recent  strategic plan in February 2011. In addition to the statutorily required  strategic plan, in 2008, the NSTC published a Strategy for  Nanotechnology-Related Environmental, Health, and Safety Research  and updated this document in 2011. The EHS strategies published in  2008 and 2011 expand on the goal of the responsible development of  nanotechnology by describing the state of science and research needed  to ensure that nanotechnology provides maximum benefits to the  environment and human well-being. The act also requires the NSTC to  prepare an annual report to be submitted to congressional committees on  the national nanotechnology program’s budget and an analysis of the  progress made toward achieving the goals and priorities established for  the program, among other things. In the NNI Supplements to the  President’s Budget, the NSTC reports overall NNI spending and  describes research efforts and investments within eight program  component areas, as seen in table 1. These program component areas  provide an organizational framework for categorizing the activities of the  NNI. Data presented in the NNI annual supplements is collected from the  NNI member agencies by OMB as part of the annual budget formulation  process.
The act requires triennial external reviews of the national nanotechnology  program. Specifically, the act requires the NNCO to contract with the  NRC to conduct the triennial evaluations of the national nanotechnology  program. The NRC draws on expertise from outside government,  including from academia, companies, and NGOs. The NRC completed  reviews in 2002, 2006, and 2009 based on the work of 15 to 23 panelists  chosen by the NRC and identified in the review. In addition, the NRC also  published an independent research strategy to address EHS aspects of  nanomaterials in 2012.
The act also requires the President to establish or designate a National  Nanotechnology Advisory Panel. The advisory panel, by statute, must  consist primarily of members from academic institutions and industry, and  panel members must be qualified to provide advice and information on  nanotechnology research, development, demonstrations, education,  technology transfer, commercial application, or societal and ethical  concerns. Since 2004, the President has designated the President’s  Council of Advisors on Science and Technology (PCAST) to function as  the advisory panel. PCAST members are generally senior managers in  major corporations and academia selected for diverse expertise in  science and technology issues. The advisory panel is required to report  not less frequently than once every 2 fiscal years on its assessment of the  national nanotechnology program and recommendations for ways to  improve the program. The advisory panel has produced three  assessments to date, in 2005, 2008, and 2010. The first and second  assessments were authored by a subset of PCAST membership, and the  third assessment was authored by a working group of three PCAST  members and additional external experts, but PCAST as a whole  approved the assessments. The first and second assessments created  Nanotechnology Technical Advisory Groups, which were comprised of  approximately 40 members who provided written responses to  questionnaires developed by PCAST. The membership of these groups is  not identified in the assessments. The external members of the third  assessment’s working group were selected by PCAST members and are  identified in PCAST’s assessment.
section_title: Funding of EHS Research by NNI Member Agencies Has Increased, but Quality of Funding Data Is Uncertain
paragraphs: NSTC reported more than a doubling of funding for nanotechnology EHS  research by NNI member agencies from fiscal years 2006 to 2010.  Reported EHS funding also rose as a percentage of total NNI funding  during this period, ending up at about 5 percent in 2010. We also  identified several reporting problems related to the continued absence of  detailed guidance on how agencies should report funding for their  nanotechnology research, raising concerns about the quality of EHS  funding data reported.
subsections: section_title: EHS Research Funding by NNI Member Agencies More Than Doubled From 2006 to 2010 and Amounted to About 5 Percent of Total NNI Funding in 2010
paragraphs: From fiscal years 2006 to 2010, NSTC reported more than a doubling of  NNI member agencies’ funding for nanotechnology EHS research in the  NNI Supplements to the President’s Budget––from approximately $38  million to $90 million.reported to be by NSF, NIH, and EPA, and the largest increases in  reported EHS research funding over this period were at NIH and EPA.
section_title: EHS Research Funded in 2010 Focused On Metal- and Carbon-Based Nanomaterials
paragraphs: In fiscal year 2010, EHS research at the seven NNI member agencies we  reviewed focused on two categories of nanomaterials more than others— metal- and carbon-based nanomaterials, which are used in a variety of  applications, including electronics, consumer products such as  sunscreens, medical products, and protective coatings (see fig. 4). NNI  has not prioritized nanomaterials for EHS research, but in October 2011 it  outlined criteria for its member agencies to use in doing so. It is not yet  clear what effect these criteria will have on how agencies prioritize the  nanomaterials they focus on in their EHS research.
As shown in figure 5, we found that in the 193 fiscal year 2010 research  projects we determined were primarily directed at EHS, the nanomaterials  that were most frequently the focus of this research were carbon  nanotubes (70 projects), nanosilver (60 projects), and nanoscale titanium  dioxide (48 projects).
Carbon nanotubes are nanoscale cylinders consisting of seamlessly  “rolled” sheets of graphene, a form of carbon. They are extraordinarily  strong, flexible, lightweight, heat resistant, and have high electrical  conductivity. Carbon nanotubes are currently used in a variety of  applications including conductive coatings for touchscreens and solar  cells, and sporting goods such as bicycle frames and baseball bats.  According to a 2009 NIOSH report, carbon nanotubes have been  shown to produce adverse effects in the respiratory systems of rats.
Nanosilver is a metal-based nanomaterial that, according to a 2010  EPA report, is currently used in an increasing number of consumer  and medical products because of its remarkably strong antimicrobial  properties. For example, EPA reported that nanosilver is being  incorporated into clothing, food contact materials such as packaging  and cutting boards, household appliances, cosmetics, and children’s  toys. It is also used in industrial processes because of its ability to  catalyze many reactions. According to EPA’s 2010 report, there is  clear evidence that nanosilver is toxic to aquatic and terrestrial  organisms and may be detrimental to human health. In this report,  EPA noted that several studies have shown that nanosilver can be  released into the wastewater stream during washing, such as from  socks containing nanosilver. The nanosilver released may disrupt the  helpful bacteria used in wastewater treatment processes or be  released into the environment.
Nanoscale titanium dioxide is a metal-based nanomaterial used in  sunscreens, protective coatings, and other materials to manage heat  and light by blocking UV light from the sun’s rays. It is also being  added to paints, cements, windows, tiles, and other products for its  sterilizing and deodorizing properties and is being used for antifogging  coatings and self-cleaning windows. In addition, it is being  investigated for use in removing contaminants from drinking water.  According to a 2011 NIOSH report, nanoscale titanium dioxide is a  potential occupational carcinogen when inhaled. Regarding  exposure to this nanomaterial in sunscreens, a 2010 FDA publication  found that nanoscale titanium dioxide included in a formulation similar  to currently marketed sunscreens is unlikely to significantly penetrate  human skin.
Particles,” Toxicological Sciences, vol. 115,  no. 1 (2010):156-66.
subsections: section_title: Effect of Recent NNI Criteria for Agencies’ Prioritization of Nanomaterials Is Not Yet Clear
paragraphs: NNI has not prioritized nanomaterials for EHS research, but the 2011 NNI  EHS research strategy outlines criteria to assist its member agencies in  doing so. The main criteria NNI proposed are the extent to which (1)  study of a particular nanomaterial may provide a major contribution to the  research knowledge base, and (2) nanomaterials and nanotechnology- enabled products may pose a safety concern to workers, consumers, and  the environment. NNI further identified five criteria for assessing whether  a material may pose a safety concern: (1) the nanomaterial’s potential for  hazard, (2) likelihood of exposure, (3) high reactivity, (4) biological  novelty, and (5) the involvement of the material in an event that produced  health or environmental impacts. However, because the criteria were  published in October 2011, it is too soon to tell how they will influence  NNI member agencies’ decisions about which nanomaterials to prioritize.
Furthermore, it is unclear if the information needed to use the NNI criteria  is available. As we reported in May 2010, predicting and assessing the  potential hazards, exposures, and resulting risks from nanomaterials is  difficult, and current understanding of nanomaterial toxicity and exposure  is limited. For example, the findings from completed toxicity studies of a  nanomaterial constructed in one manner may not be applicable to  understanding the risks posed by the same nanomaterial constructed in a  different manner and, therefore, studies of similar nanomaterials may not  be comparable. Also, according to the NRC’s 2012 draft strategy for  nanotechnology EHS research, there is incomplete information on the  effects of the array of nanomaterials used in products and a lack of  information on effects of chronic exposures. NRC reported that most  toxicity studies test a single material and usually focus on effects of acute  exposures.
In addition, reliable, comprehensive information is not readily available on  the likelihood of exposure to nanomaterials. Consumers and workers are  more likely to be exposed to nanomaterials that are already produced in  large quantities or incorporated into a larger number of products, but  information on the prevalence or production volumes of nanomaterials or  nanotechnology-enabled products is currently limited. As previously noted  in this report, FDA, CPSC, and EPA may not receive complete  information on what nanomaterials are already in use in the case of  certain products. CPSC officials told us that one way the agency collects  information on the use of nanomaterials in products to better understand  potential exposures is by purchasing information from commercial  vendors. However, we found that obtaining such information from  commercial vendors such as market research companies can be costly,  and it is difficult to assess the reliability of such research because it can  involve data and methods, including modeling based on various  assumptions, that may be proprietary.
Another source of information on nanomaterials in commerce is the  Woodrow Wilson International Center for Scholars’ Project on Emerging  Nanotechnologies, which maintains an inventory of nanotechnology- based products. As of the March 2011 update to the list, the inventory  contained approximately 1,300 nanotechnology-based products.  However, the Wilson Center’s list is not comprehensive, and it consists of  consumer products. As we reported in May 2010, occupational  exposure is a particular concern because the exposure and risk to  workers is potentially greater than the risk to consumers. At present,  though, there is little information on the exposure of workers to  nanomaterials in the workplace.
section_title: NNI Member Agencies Have Collaborated Extensively on Nanotechnology EHS Research
paragraphs: The NNI member agencies have collaborated extensively with each other  and nonfederal stakeholders on EHS research and strategies. The NNI  member agencies participated in interagency efforts to develop joint  strategies related to EHS research. They have also undertaken over 40  collaborative nanotechnology EHS research projects in recent years,  signing agreements with both federal and nonfederal stakeholders. Many  of the nonfederal stakeholders responding to our questionnaire rated their  collaborative activities with NNI agencies as very or generally useful,  although they have identified some challenges.
subsections: section_title: NNI Member Agencies Have Collaborated to Develop Joint Strategies
paragraphs: NNI member agencies have collaborated through the NSTC to develop  joint EHS research strategies. The NSTC primarily coordinates  nanotechnology EHS research through the NSET Subcommittee and the  NEHI, an interagency working group. The NEHI’s purpose includes  providing for an exchange of information among agencies and with  nonfederal stakeholders; facilitating the identification, prioritization, and  implementation of nanotechnology EHS research; and managing the EHS  interagency research strategy and facilitating its implementation. The  2011 NNI EHS research strategy was developed by NNI member  agencies working through the NEHI working group. The strategy sets a  common vision for nanotechnology EHS research, EHS research  categories and needs, and key principles intended to assist the NNI  member agencies make strategic decisions about research programs that  will advance the NNI EHS research agenda while meeting their respective  missions. According to several NNI member agency officials, individual  agencies’ implementation efforts of the 2011 NNI EHS research strategy  are discussed at NEHI meetings. However, since the strategy was  finalized in October 2011, it is too soon to tell to what extent it will be used  to integrate the NNI member agencies’ implementation plans. The 2011  NNI EHS research strategy updated and replaced the 2008 NNI Strategy  for Nanotechnology-Related EHS Research, which had established  common priority EHS research needs. The 2008 NNI Strategy for  Nanotechnology-Related EHS Research was also developed by the NEHI  working group and informed by earlier publications of NNI EHS research  needs.
Officials from five of the NNI member agencies told us that they use the  NEHI working group as a forum to collaborate on nanotechnology EHS  research projects. For example, CPSC officials told us that they identify  agencies with expertise on testing nanomaterials found in consumer  products through discussions in the NSET Subcommittee and NEHI  working group, which has led to collaborative testing of exposure and  release of selected nanomaterials. This collaboration has resulted in data  that support the respective missions of the collaborative agencies.  Officials from the National Toxicology Program at NIH noted that they  selected nanomaterials for study based in part on discussions in the NEHI  working group and NNI sponsored workshops. Officials from NIST told us  that NNI member agencies are also working through the NEHI working  group to develop an inventory of collaborative EHS research activities.  Collaborative activities related to EHS research have been reported in the  2011 NNI EHS research strategy and annual NNI Supplements to the  President’s Budget.
section_title: NNI Member Agencies Have Engaged In Numerous Collaborative EHS Research Projects
paragraphs: The NNI member agencies have recently initiated numerous  nanotechnology EHS research projects in collaboration with other federal  agencies and nonfederal stakeholders. We reviewed formal collaborative  nanotechnology EHS research projects initiated from February 2008 to  October 2011, while the 2008 NNI Strategy for Nanotechnology-Related  EHS Research was in effect. The 2008 NNI Strategy for Nanotechnology- Related EHS Research called for the NEHI and the NNI member  agencies to coordinate agency efforts to address priority research needs  and, among others, identify opportunities for collaboration and joint  development and use of resources where appropriate, facilitate  partnerships with industry, and coordinate and support international  efforts, support development of consensus-based standards, and  facilitate wide dissemination of research results and other nonproprietary  EHS information. During this period, NNI member agencies initiated 43  formal collaborative projects related to EHS research––24 interagency  collaborations and 19 collaborations that included nonfederal  stakeholders. Most of the interagency collaborations were among the NNI  member agencies included in our review, but a few were with other  federal agencies such as the Department of Defense, the Department of  Labor’s Occupational Safety and Health Administration, and the  Department of Agriculture. Nonfederal stakeholders included foreign  governments, such as the United Kingdom (UK) and China; universities,  such as the University of Massachusetts, Lowell and Rice University; and  NGOs, such as the International Alliance for NanoEHS Harmonization  (IANH) and the International Life Sciences Institute Research  Foundation.
NNI member agencies used these collaborative projects to extend their  capability to achieve their individual missions. For example, CPSC  collaborated with four agencies to conduct research on nanomaterials  found in consumer products. NIOSH, an agency within the Centers for  Disease Control and Prevention that conducts research and makes  recommendations for the prevention of work-related injury and illness,  partnered with entities that operate nanomanufacturing facilities to better  understand the potential for occupational exposure to nanomaterials.  NIST, which aims to advance measurement science, standards, and  technology, signed an agreement with the IANH to jointly create protocols  for toxicological tests of selected representative nanoparticles, confirming  inter-laboratory reproducibility and verifying the predictability of certain  procedures.
Collaborative EHS research projects have resulted in transfers of funding  as well as sharing of expertise, facilities, and other resources, as shown  in the following examples:
An interagency agreement between CPSC and NIOSH to study the  pulmonary effects of titanium dioxide nanoparticles released from  aerosol spray products involved CPSC providing the product to be  tested and transferring funding to NIOSH to construct the testing  equipment, with NIOSH providing expertise and staff time to run the  tests and produce a report.
An agreement among the NIH’s National Cancer Institute, the  National Institute of Environmental Health Sciences (NIEHS), and the  National Institute of Biomedical Imaging and Bioengineering to  develop, maintain, and operate a web-based nanomaterial registry  specified that the National Institute of Biomedical Imaging and  Bioengineering will develop the registry to provide consistent and  curated information on the biological and environmental interactions of  well-characterized nanomaterials, as well as links to associated  publications, modeling tools, computational results, and  manufacturing guidance from existing databases. The National  Cancer Institute and the NIEHS provided funding. NIEHS also  committed to provide input to the constitution of an advisory board  and attend its meetings, as well as be represented in the nano registry  project management team.
Agencies have also issued joint solicitations for grant applications,  such as one between EPA and a number of UK agencies to solicit  proposals for research on environmental and health implications of  nanotechnology. EPA and the UK agencies signed an agreement that  notes (1) the funding of grants will be made consistent with the budget  priorities of each party and (2) they are to work jointly to define the  scientific priorities, issue the request, assist with peer review and  selection of grantees, and disseminate the research results.
Details on these and other collaborative EHS research projects are  provided in appendix II.
section_title: Nonfederal Stakeholders We Surveyed Benefited from NNI Collaboration but Identified Some Challenges
paragraphs: The 138 nonfederal stakeholders who responded to our questionnaire  reported that they benefited from collaboration with NNI member  agencies but faced some challenges. Most respondents indicated that the  2011 NNI EHS research strategy was very or somewhat effective at  addressing the EHS research needs. Respondents reported that they  also obtained information on nanotechnology EHS risks from NNI  member agencies and nongovernmental sources.
The following three formal mechanisms for collaboration with NNI  member agencies were most frequently identified by respondents as  generally or very useful to them: (1) joint data gathering and sharing; (2)  joint research solicitations or funding of research consortia; and (3)  competitive grants. See figure 6 for respondents’ ratings of seven formal  collaborative mechanisms we identified for them to rate. Some  respondents who provided comments in response to optional open-ended  questions also identified public-private partnerships, such as joint  academic, government, and industry information exchange and research  programs, as collaboration mechanisms the NNI member agencies  should consider. A few respondents who commented in response to  optional open-ended questions also cited benefits to informal  collaboration with NNI agencies, such as discussions during workshops  and conferences.
Since both the 2011 NNI EHS research strategy and its predecessor, the  2008 NNI Strategy for Nanotechnology-Related EHS Research, called for  partnerships with industry and other nonfederal stakeholders, we included  questions about collaboration with NNI member agencies in our  questionnaire. When asked about the usefulness of their collaboration  with the NNI member agencies, more than half of respondents rated their  collaborative EHS research or related activities for each of the NNI  member agencies as generally or very useful, as seen in figure 7.
Stakeholders responding to our questionnaire indicated that certain  challenges to collaboration with NNI member agencies we identified on  EHS research apply to them. As seen in figure 8, more than two-thirds of  those who rated the challenge of lack of funding and the challenge of  limited awareness of collaboration opportunities indicated that each of  these challenges apply to them. This is consistent with what we heard  from representatives of a nanotechnology trade association and an NGO,  who stated that there is limited funding for corporate EHS research and  development efforts because they do not generate revenues. Some  respondents who provided comments in response to optional open-ended  questions stated that the private sector needs government funding for  EHS research. A number of respondents also commented that  government funding for EHS research was inadequate or an “after- thought” to other purposes. Some respondents also commented that  funding seems to be targeted toward large, multiyear centers rather than  smaller targeted projects that can help address near-term EHS needs.
More than half of those responding noted that each of the challenges of  regulatory uncertainty, lack of standardization—such as different  terminology or protocols—and concerns regarding disclosure of  proprietary or confidential business information, apply to them. Some  respondents cited regulatory uncertainty—that is, lack of a clear  regulatory environment to enable commercialization and protect  consumers and the general public—as a challenge to collaboration with  their work with NNI member agencies on nanotechnology EHS research.  For example, a number of respondents who provided comments in  response to optional open-ended questions pointed to regulatory  uncertainty as limiting the ability of companies to determine their EHS  responsibilities. Some respondents also commented on difficulties in  collaborating with NNI member agencies resulting from differences in  expertise and regulatory approaches across different government  agencies or even within the same agency. With regard to lack of  standardization as a challenge to collaboration on nanotechnology EHS  research, some respondents provided examples related to the lack of  scientific consensus on the definition of nanomaterials, the testing  methods for toxicology research, and common terminology across  research and regulatory agencies.
When asked to rate the effectiveness of mechanisms we identified that  NNI member agencies can use for obtaining input from nonfederal  stakeholders on the development of the NNI EHS research strategies,  respondents most frequently identified the following three mechanisms as  being used somewhat effectively or effectively: (1) workshops,  conferences, and other public speaking engagements; (2) advisory  councils; and (3) the NNI web portal.
When asked to rate the 2011 NNI EHS research strategy, most of those  responding rated this strategy as somewhat or very effective at  addressing nanotechnology EHS research needs. As seen in figure 10,  while just over half of those responding to these questions indicated the  2008 NNI Strategy for Nanotechnology-Related EHS Research  addressed these needs somewhat or very effectively, 79 out of 98  indicated the 2011 NNI EHS research strategy addressed these needs  somewhat or very effectively. Some of the respondents who commented  in response to optional open-ended questions commended the 2011 NNI  EHS research strategy for accurately capturing the input of the  workshops’ participants and summarizing the research needs, but others  pointed to shortcomings, including a lack of prioritization and insufficient  focus on implementation. For example, one respondent commented that  the strategy would only be effective if it was implemented, and that it may  not be sufficient to rely on the agencies’ voluntary implementation efforts.
Additionally, respondents identified various sources, including the NNI  member agencies, from which they obtained information on the potential  EHS risks of nanotechnology. When asked to estimate the frequency with  which they obtained nanotechnology EHS information from NNI member  agencies in recent years, respondents reported obtaining information  from the NNI member agencies in varying frequencies. More than half of  the respondents to these questions reported occasionally or often  obtaining information from EPA and NIOSH, as seen in figure 11.
The 2011 NNI strategic plan calls for providing a “one-stop” Internet- based portal for nanotechnology information, including, among other  things, scientific data such as characterization and toxicity  measurements. About 80 percent of the 131 stakeholders responding to a  question about how useful such a portal would be indicated that an  Internet-based portal for nanotechnology EHS information would be  generally or very useful to them. The NNI does not currently provide a  portal that contains this information, but an NNCO official told us that the  NNCO is implementing a web-based map that identifies the  instrumentation and facilities for nanotechnology research at major  universities and centers in the U.S. and federal labs.
When asked to rate the frequency with which they obtained information  on the potential EHS risks of nanotechnology from nongovernmental  sources, as seen in figure 12, more than two-thirds of respondents  reported that they occasionally or often obtained information from each of  the following four sources: (1) peer-reviewed scientific publications; (2) in- house research; (3) online databases; and (4) news outlets.
section_title: NNI Strategy Documents Address or Partially Address Desirable Characteristics of National Strategies
paragraphs: NNI strategy documents address two and partially address four of the six  desirable characteristics of effective national strategies that we identified  in prior work and that offer policymakers and implementing agencies a  management tool that can help ensure accountability and more effective  results. The three NNI strategy documents––the 2011 NNI strategic plan,  the 2011 NNI EHS research strategy, and the NNI Supplement to the  President’s 2012 Budget––compose a national strategy for  nanotechnology EHS research and create a framework for achieving NNI  program goals and priorities.
We rated the NNI strategy documents to determine how well they jointly  addressed the six characteristics that we have identified for effective  national strategies, as described in table 3.
Ideally, effective national strategies should fully address all of these  characteristics. However, we recognize that by their nature, national  strategies are intended to provide broad direction and guidance—rather  than be prescriptive, detailed mandates—to the relevant implementing  agencies. Thus it is unrealistic to expect all national strategies to provide  details on each and every characteristic we identified. Moreover, the NNI  member agencies have different statutory authority and functions that  may significantly affect their research and research priorities.  Nonetheless, we believe the more detail a strategy provides, the easier it  is for the responsible parties to implement it and achieve its goals.
We reviewed the three NNI strategy documents for elements related to  these characteristics, and based on the inclusion of these elements, rated  how well the strategic documents address the six characteristics.  According to our methodology, the strategy documents “address” a  characteristic when they explicitly include all, or nearly all, elements of the  characteristic and have sufficient specificity and detail. The documents  “partially address” a characteristic when they include some or most of the  elements of the characteristic with sufficient specificity and detail. The  documents “do not address” a characteristic when they do not include any  elements of a characteristic or references are too vague or general to be  useful. Additional details about our ratings and the elements that make up  these characteristics are available in appendix I.
As described in table 4, we found, when reviewed as a whole, that the  strategy documents address or partially address all of the desirable  characteristics of a national strategy.
Taken as a whole, the NNI strategy documents address the first two  characteristics—purpose, scope, and methodology and problem definition  and risk assessment—by including all, or nearly all, of their elements.  These characteristics describe the scope of the strategy, describe how  and why it was produced, define the problems the strategy intends to  address, discuss causes and the operating environment, and provide a  risk assessment or broad description of potential risks. Elements of these  characteristics that are included in the documents are providing a clear  statement of purpose, defining key terms, delineating what major  functions or mission areas are covered, discussing problems with the  current understanding of EHS implications of nanotechnology, and the  quality of currently available data, among others.
Goals, Subordinate Objectives,  Activities, and Performance  Measures (Desirable  Characteristic 3)
The NNI strategy documents partially address the third characteristic  regarding goals, subordinate objectives, activities, and performance  measures. This characteristic describes the overall desired results,  hierarchy of strategic goals and subordinate objectives, priorities,  milestones, outcome-related performance measures, and process for  monitoring and reporting on progress, among others. The NNI strategy  documents include a number of elements such as desired results,  strategic goals, and subordinate objectives, but do not include, or do not  fully develop, other elements such as priorities, milestones, or outcome- related performance measures. Performance information—such as  outcome-related performance measures and milestones comprised of  targets and time frames for meeting those measures—allows managers  to identify performance problems and develop approaches that improve  results. For the purposes of this report, we define performance  information to mean data collected to measure progress toward achieving  an established goal. A wide range of information can be relevant to  program performance. Performance information can focus on various  dimensions of performance such as outcomes, outputs, quality,  timeliness, customer satisfaction, or efficiency. It can inform key  management decisions such as setting program priorities, allocating  resources, identifying program problems, and taking corrective action to  solve those problems; or it can help determine progress in meeting the  goals of programs or operations. The NNI strategy documents contain  detailed research needs for nanotechnology EHS and report the annual  funding of EHS research and the number of projects supported for  selected years, but the documents do not prioritize among these needs or  include outcome-related performance measures, targets, or time frames  that allow for monitoring and reporting on progress toward meeting the  research needs.
Independent reviews of the prior NNI strategy documents also noted an  absence of performance information. In 2010, the National  Nanotechnology Advisory Panel recommended that the NNCO monitor  metrics that assess, among other things, the NNI’s progress on  developing methodologies to assess plausible risks of nanotechnology.  The advisory panel also recommended that the NEHI working group  develop and implement a strategy that links EHS research activities with  knowledge gaps and decision-making needs. The 2009 NRC review  concluded that the 2008 NNI Strategy for Nanotechnology-Related EHS  Research could be an effective tool, but did not include a number of  elements, including measures of research progress. We previously  reported that, ideally, a national strategy would set clear desired results  and priorities, outcome-related performance measures, and specific  milestones while giving implementing parties flexibility to pursue and  achieve those results within a reasonable time frame.
The NNI strategy documents give agencies wide latitude to develop  research programs to meet the goals and research needs of the NNI, but  we found that NNI member agencies vary in their identification and  reporting of agency-specific performance information for nanotechnology  EHS research that could align with the NNI research needs. The 2011  NNI EHS research strategy states that prioritization, timing, and staging of  the research needs identified by the strategy are components of an  implementation plan and should be developed within agency missions  and appropriations. Similarly, the 2011 NNI strategic plan states that the  document serves as a guide for individual agency implementation. Six of  the seven NNI agencies we reviewed have documented performance  measures, targets, or time frames for their EHS research. Three of the  NNI member agencies—CPSC, FDA, and NIH—annually report EHS- related performance information through their publicly available annual  performance reports. Three other NNI member agencies—EPA, NIOSH,  and NIST—identify strategic goals and performance measures and  targets in their program-level nanotechnology EHS strategies, but have  not collected, or have not publically reported results of these performance  measures. NIOSH has previously published research summaries in its  nanotechnology research progress reports. NIOSH officials told us that a  forthcoming progress report will also document the results of performance  measures identified in their nanotechnology EHS strategy. NSF reports  planned EHS research activities as part of its annual budget request, but  does not identify performance targets or measures for its nanotechnology  EHS research beyond funding requested. According to an NNCO official,  the NEHI working group is piloting an effort to gather information from the  NNI member agencies to assess high-level progress toward meeting NNI  research needs, but has not released the results of this work.
Resources, Investments, and  Risk Management and  Organizational Roles,  Responsibilities, and  Coordination (Desirable  Characteristics 4 and 5)
The NNI strategy documents partially address the fourth and fifth  characteristics describing resources, investments, and risk management  and organizational roles, responsibilities, and coordination. We found that  the documents include some, but not all, of the elements of these  characteristics. With respect to the fourth characteristic, we found that the  documents include elements relating to agency resources associated with  the strategy such as descriptions of current activities of NNI member  agencies and current investment levels. However, we found that the  documents do not include, or do not fully develop, other elements such as  the costs and types of resources—for example human capital or research  and development costs—associated with implementing the strategy. An  NNCO official told us that the NNI does not analyze the costs of  resources to meet the EHS research needs because it does not control  the funding for member agencies, and that each agency allocates funding  and prioritizes its activities based on its mission area and available  budget. According to this official, the 2011 NNI EHS research strategy  identifies research goals, but it is up to the agencies to determine how  their funding should be spent. Similarly, a NIST official told us that the  NNI cannot dictate to agencies how or what research to fund. With  respect to the fifth characteristic pertaining to organizational roles,  responsibilities, and coordination, we found that the documents include  elements such as lead, support, and partner roles for agencies and  processes for coordination and collaboration, but do not include, or do not  fully develop, other elements such as an accountability and oversight  framework. The strategy documents include detailed descriptions of the  NNI structure, describe coordination mechanisms such as the NEHI  working group, and provide examples of collaborative activities related to  EHS research. As we reported above, we found that NNI member  agencies have engaged in numerous collaborative projects with federal  and nonfederal partners and nonfederal partners generally rated these  collaborations as generally or very useful. However, the strategy  documents do not provide details on oversight of agency roles or how  agencies will be held accountable to the goals and research needs of the  NNI strategy documents.
Integration and Implementation  (Desirable Characteristic 6)
The NNI strategy documents also partially address the sixth characteristic  describing integration and implementation. This characteristic describes  how the national strategy documents relate to subordinate levels of  government and their plans to implement the strategy. As we have  previously reported, to achieve a common outcome, collaborating  agencies need to establish strategies that work in concert with those of  their partners or are joint in nature. We found that the NNI strategy  documents include limited information regarding the elements of this  characteristic. For example, the 2011 NNI EHS research strategy  describes a framework for implementation, but does not provide detailed  implementation guidance. In addition, we found that the NNI strategy  documents generally do not include information on the NNI member  agencies’ nanotechnology EHS strategies or plans or their integration  with the national strategy.
Five of the seven NNI member agencies we reviewed have developed  such agency-level nanotechnology EHS research strategies. As noted  above, EPA, NIOSH, and NIST have developed strategic plans  specifically for EHS research. In addition, FDA has published a strategic  plan for its regulatory science research efforts, which include  nanotechnology, and developed a research plan specifically targeting   Furthermore, NSF officials told us that the agency  nanotechnology. describes its strategic direction for nanotechnology, including current and  proposed activities and funding by PCA, in its annual budget request.CPSC includes nanotechnology as an external risk factor for its  agencywide strategic plan, but the plan does not address EHS research  directly. NIH officials told us they do not have agency or institute-specific  strategic plans that specifically address nanotechnology EHS research,  but nanotechnology EHS research plans developed by its institutes  conform to the NNI strategy. Of the five agencies with nanotechnology  EHS-related strategies, two—FDA and NIOSH—explicitly align agency  research goals in their strategic plans for nanotechnology to NNI EHS  research needs. The FDA and NIOSH strategic plans describe the  agencies’ EHS research goals, their framework for meeting those goals,  and which NNI EHS research needs will be advanced. NIST’s Nano-EHS  program plan, EPA’s Nanomaterial Research Strategy, and NSF’s budget  request to Congress describe agency research objectives. The  documents do not explicitly identify NNI research needs; however,  agency officials told us that they reflect the NNI strategic documents. The  2011 NNI strategic plan reports that NNI member agencies have initiated  a mapping exercise to evaluate how this strategic plan relates to NNI  member agencies’ strategic plans and to the priorities of the  administration. In addition, according to NNI member agency officials,  individual agencies’ implementation efforts of the 2011 NNI EHS research  strategy are discussed at NEHI meetings. However, the results of this  mapping exercise and proceedings of the NEHI are not publically  available.
section_title: Conclusion
paragraphs: The increasing commercialization of nanotechnology-enabled products and  information gaps related to nanotechnology EHS impacts underscore the  importance of ensuring progress toward meeting EHS strategic goals and  research needs. The NNI strategy documents help shape the policies,  programs, priorities, resource allocations, and standards related to  nanotechnology EHS research and activities. The NNI strategy documents  address or partially address the desirable characteristics of effective  national strategies, including overarching strategic goals and objectives  such as the EHS research needs. However, the documents do not include  some elements of the characteristics, which could make it more difficult for  federal agencies and other stakeholders to implement the strategy and  achieve the identified results. For example, progress toward achieving the  strategy documents’ goals and research needs cannot be tracked because  the NNI does not report specific performance information, such as  performance measures, targets, or time frames. In addition, the NNI  strategy documents do not include estimates of the costs and types of  resources associated with these goals and research needs.
Developing and coordinating performance information and cost estimates  for the NNI’s nanotechnology EHS research may be challenging because  of the varied missions and priorities of the NNI member agencies.  However, not having performance information that is aligned with the  strategic goals and research needs of the NNI makes it difficult for  agencies, policymakers, and stakeholders to determine the collective  progress of the national nanotechnology program. Furthermore,  developing performance information and including it in NNI strategy  documents could help to strengthen two other desirable characteristics  that the documents partially address. Specifically, this information could  also support an accountability and oversight framework—that is currently  not well-developed—by linking agency activities to specific measures of  performance and creating consistent benchmarks to judge the overall  progress of the NNI member agencies. In addition, developing this  information could facilitate better integration between the NNI strategy  documents and the NNI member agencies’ nanotechnology EHS  research strategies and goals. Providing cost estimates related to the NNI  EHS research needs would allow the NNI member agencies,  policymakers, and stakeholders to assess if investments are  commensurate with costs of the identified needs. The NSTC, which is  administered by the OSTP, does not direct funding to meet the NNI  research needs; however, rough estimates of their costs could serve as  guidance to the NNI member agencies as they individually determine their  own budgets and priorities. In addition, the NSTC is well-positioned to  develop these estimates due to its coordinating role across the member  agencies of the NNI and corresponding access to expertise at these  agencies. Developing performance information and cost estimates could  also support the analysis of the progress made toward achieving the  goals and priorities established for the program, as required by the 21st  Century Nanotechnology Research and Development Act.
In addition, we are reiterating our 2008 recommendation that the Director  of OSTP, in consultation with the Directors of the NNCO and OMB,  provide better guidance to agencies on how to report their  nanotechnology EHS research. The continued absence of detailed  guidance on how to report research under PCA 7 has contributed to the  data reporting problems we identified. Therefore, agencies, policymakers,  and stakeholders do not have access to accurate, consistent, and  complete data on the federal government’s investment in nanotechnology  EHS research. Without reliable and up-to-date data, it will be difficult for  agencies to accurately assess and report their progress toward their own  performance measures, as well as the EHS research goals and needs  identified in the 2011 NNI EHS research strategy.
section_title: Recommendations for Executive Action
paragraphs: To better offer policymakers and implementing agencies a management  tool that can help ensure accountability and more effective results, we are  making two recommendations to the Director of OSTP:  We recommend that the Director of OSTP coordinate development by the  NNI member agencies of performance measures, targets, and time  frames for nanotechnology EHS research that align with the research  needs of the NNI, consistent with the agencies’ respective statutory  authorities, and include this information in publicly available reports.
We also recommend that, to the extent possible, the Director of OSTP  coordinate the development by the NNI member agencies of estimates of  the costs and types of resources necessary to meet the EHS research  needs.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Director, OSTP; Secretary,  Commerce; Chairman, CPSC; Administrator, EPA; Secretary, Health and  Human Services; and Director, NSF. OSTP and the agencies neither  agreed nor disagreed with the recommendations. OSTP and Health and  Human Services provided technical and clarifying comments, which we  incorporated as appropriate. The Department of Commerce, CPSC, EPA,  and NSF indicated they had no comments on the report.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Director, OSTP;  Secretary, Commerce; Chairman, CPSC; Administrator, EPA; Secretary,  Health and Human Services; Director, NSF; and other interested parties.  The report also will be available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-3841 or ruscof@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made major contributions to this report  are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our review examines (1) changes in federal funding for nanotechnology  environmental, health, and safety (EHS) research from fiscal years 2006  to 2010; (2) the nanomaterials that National Nanotechnology Initiative  (NNI) member agencies focused on in their EHS research in fiscal year  2010; (3) the extent to which NNI member agencies collaborate with  stakeholders on nanotechnology EHS research and related strategies;  and (4) the extent to which NNI strategy documents address desirable  characteristics of national strategies.
To conduct this work, we reviewed EHS research efforts funded by seven  NNI member agencies, which collectively funded 93 percent of EHS  research dollars in fiscal year 2010: the National Science Foundation  (NSF), National Institutes of Health (NIH), Environmental Protection  Agency (EPA), National Institute for Occupational Safety and Health  (NIOSH), Food and Drug Administration (FDA), National Institute of  Standards and Technology (NIST), and the Consumer Product Safety  Commission (CPSC). The first six of these agencies represent the top six  providers of EHS research funding from fiscal year 2006 to fiscal year  2010, and CPSC has an important role in ensuring the safe use of  nanotechnology in consumer products.
To examine recent changes in federal funding for nanotechnology EHS  research, we reviewed information published by the National Science and  Technology Council (NSTC) in the NNI Supplements to the President’s  Budget. Specifically, we reviewed the actual agency investments reported  in each program component area for fiscal years 2006 through 2010 for  all NNI member agencies funding nanotechnology research. For the  dollar amounts that we adjusted for inflation, we used the Biomedical  Research and Development Price Index to report funding in constant  2010 dollars. We consulted with the Office of Management and Budget  (OMB) and officials at the seven selected agencies to determine the type  of budget information reported as actual agency investments in these  documents. For fiscal year 2010, the most recent year for which actual  agency investment data were available, we also collected quantitative  and qualitative project-level data (such as project funding amounts and  project abstracts or progress reports) on all research projects that the  seven selected agencies had categorized as Program Component Area  (PCA) 7 (EHS research). The agencies do not report project-level data to  OMB annually but report their total EHS research funding to OMB  annually for inclusion in the NNI budget supplements published by NSTC.  Therefore, we reviewed data on the individual projects included in the  agencies’ total EHS research funding in 2010. We provided the agencies  with a spreadsheet template to use in listing their projects, which we  based on the spreadsheet OMB had used in its data calls to collect  project-level data from agencies on their 2006 and 2009 EHS research  projects. Consistent with the approach used in OMB’s data calls, we did  not ask the agencies to use a standardized definition of what constitutes a  “project;” instead, we deferred to each agency to identify their projects  and consulted with agency officials as needed.
For some projects at NIH, NIOSH, and NIST, we grouped together some  of the agencies’ data entries and counted them together for the purposes  of our analyses. Consequently, the numbers of projects we are reporting  may not match the number reported by these agencies elsewhere.  Specifically, for NIH, we grouped the 14 supplement grants together with  their main grants and counted each of those groups as a single project.  We also grouped together the individual subprojects of each of NIH’s  multiproject “parent” grants reported as EHS and counted each of those  groups as a single project, for a total of seven groups of multiproject  grants. In addition, we grouped together two grants made by different NIH  institutes for the same project. For NIOSH, we identified two project  entries with identical project descriptions. After consulting with NIOSH  officials, we grouped those two entries together and counted them as a  single project. For NIST, the agency reported a separate entry for each of  its Project Tracking Numbers and explained that a single technical project  is often associated with more than one Project Tracking Number. NIST  officials instructed us to group together the entries with identical titles and  count each of those groups as a single project.
To assess the reliability of the agencies’ fiscal year 2010 data, we sent  each of the seven selected agencies a set of questions regarding the data  and the information systems used to produce and store them. We also  reviewed related supporting documentation, such as user manuals and  data dictionaries for information systems, and, for some projects, copies  of the associated interagency agreements. We examined the data for  obvious errors, compared each agency’s total fiscal year 2010 PCA 7  funding reported to us to the totals reported in the NNI Supplement to the  President’s 2012 Budget, and consulted with agency officials to  understand the reasons for any differences. We determined that the data  were sufficiently reliable for the purposes of our analyses. However, we  did not attempt to verify the funding amounts reported for each project— for example, by reviewing documentation of agencies’ actual  expenditures for each project.
We reviewed the qualitative project data, including abstracts and project  reports, to assess whether the agencies had appropriately categorized  the research projects they reported as EHS research—that is, whether  the projects met the definition of program component area (PCA) 7. As  described in this report, the definition of PCA 7—research primarily  directed at understanding the EHS impacts of nanotechnology  development and corresponding risk assessment, risk management, and  methods for risk mitigation—is the only written guidance available to  agencies on how to report their nanotechnology research in this program  component area. Determining whether research meets this definition is an  inherently subjective process. Therefore, rather than definitively conclude  that any projects did not meet the definition, we assigned projects a  designation of “not clearly PCA 7” if we were ultimately unable to identify  any EHS research, or it was not clear that EHS was the primary focus of  the research. For example, some projects studied various  nanotechnology-enabled drugs, but it was not clear to what extent the  research was directed at the safety of the drugs versus their efficacy.  Some other projects appeared to be closely related to program  component areas other than PCA 7, such as PCA 4 (instrumentation  research, metrology, and standards for nanotechnology) or PCA 8  (education and societal dimensions), and it was not clear that PCA 7 was  the most appropriate category. To minimize bias and to ensure the  consistency of our evaluation, two analysts independently analyzed the  project data and used their professional judgment to assess whether each  project met the PCA 7 definition. The two analysts discussed those  projects where they did not agree on whether the research was primarily  directed at EHS and reached agreement. For categorization of projects  that appeared questionable to us, we generally asked agencies to explain  why they had reported those projects as EHS research. We reviewed  agencies’ responses and modified our determinations for projects as  appropriate given the additional support provided by the agencies. For  two projects funded by NIST and 18 by NSF, the agencies had reported  only a portion of the total project funding under PCA 7. In those instances,  we assessed whether at least some of the project was primarily directed  at EHS. However, we did not verify that the funding amounts reported as  EHS research were correct because that information is, according to  these agencies’ officials, based on the discretion of the knowledgeable  agency program staff. We also reviewed our 2008 report in this area,  which found that 22 of 119 fiscal year 2006 projects reported as EHS  research were miscategorized, and recommended that the Director of the  Office of Science and Technology Policy (OSTP), in consultation with the  Directors of the National Nanotechnology Coordinating Office and OMB,  provide better guidance to agencies on how to report nanotechnology  EHS research.
GAO, Nanotechnology: Better Guidance Is Needed to Ensure Accurate Reporting of  Federal Research Focused on Environmental, Health, and Safety Risks, GAO-08-402  (Washington, D.C.: Mar. 31, 2008). experience in nanotechnology helped identify the categories we used and  verified that the materials were placed in the appropriate categories for  each project. After reviewing the fiscal year 2010 EHS research by the  selected agencies, we selected the five categories of nanomaterials we  used, based on the primary composition of the materials, to encompass  the range of materials identified in the projects. To inform our selection of  the categories we used for grouping nanomaterials, we reviewed  literature related to nanotechnology EHS issues from federal agencies  (NIH and EPA), nonprofit organizations (the Woodrow Wilson  International Center for Scholars’ Project on Emerging Nanotechnologies  and the World Technology Evaluation Center, Inc.), and a market  research firm that tracks nanotechnology (Lux Research). We found that  various classes or categories were used to present information on  nanomaterials. For materials identified as composites, we assigned them  to categories based on the primary nanomaterials from which they were  formed—for example, we assigned carbon nanotube nanocomposites to  the carbon-based nanomaterials category.
To determine the extent to which the NNI member agencies collaborate  with stakeholders on nanotechnology EHS research and related  strategies, we (1) discussed with agency officials how their agencies  collaborate on nanotechnology EHS research and NNI’s role in facilitating  that collaboration, and (2) we obtained documentation on these  collaborative efforts. We conducted a review of formal collaborative  efforts (i.e., those that are documented in written agreements) that  focused on nanotechnology EHS research initiated from February 2008 to  October 2011 while the 2008 NNI Strategy for Nanotechnology-Related  EHS Research was in effect. We chose this time frame because this  strategy—the first NNI strategy to focus on EHS research—established  collaboration as necessary for its implementation efforts. The strategy  was in effect for a number of years, which allowed us to get a more  complete picture of the number of collaborative projects completed during  its implementation. We only included collaborative efforts that were  formalized by written agreements because those provided a clearer  description of the agencies’ contributions of resources. We obtained an  initial list of such efforts from published NNI sources as well as an OMB  data call used to compile EHS-related nanotechnology projects for the  2011 NNI EHS research strategy. We sent our initial list to the selected  NNI agencies for their review and revision, and analyzed supporting  documents provided by the agencies. We provide relevant information  from the supporting documents for each of the projects within our scope  in appendix II.
We administered a web-based questionnaire to a nonprobability sample  of nonfederal stakeholders, including those affiliated with academia,  companies, nongovernmental organizations (NGO), and state and local  governments, to obtain their views on collaboration with the NNI member  agencies on EHS research and the EHS research strategies. Because  the population of nonfederal stakeholders is diverse, wide-ranging, and  difficult to reliably define, we chose to use a nonprobability sample of  individuals who (1) had expertise in the field of nanotechnology, and (2)  had interacted with the NNI in the past few years, or who were  representatives of organizations and companies suggested to us through  our scoping interviews. We identified relevant potential respondents from  several sources, including the list of participants in NNI workshops that  were used to solicit input for the 2011 NNI EHS research strategy and  participants in a review of the NNI by the President’s Council of Advisors  on Science and Technology. We also included members of  nanotechnology-related trade associations and other organizations  through interviews with nanotechnology experts using an iterative  process, often referred to as “snowball sampling.” We excluded federal  employees and contractors, individuals or organizations that are not  located in the United States, and those for whom we could not obtain  contact information. In addition, individuals who were not considered to  have expertise in the nanotechnology field based on their job titles were  excluded.chosen as potential respondents. Of these 228 individuals, five  subsequently were excluded from our sample because they had invalid e- mail addresses, were duplicates, or were not available for an extended  period, giving us a final sample of 223 potential respondents. We  categorized the population surveyed by affiliation type, usually based on  information provided on each entity’s website, as follows: (1) “academic,”  which includes universities or university-affiliated programs or centers; (2)  “company,” which includes companies, corporations, or other for-profit  entities whose primary purpose is to sell products, services, or  information; (3) “organization,” which includes NGOs or other entities  whose primary purpose is public interest or mutual benefit; and (4) “state  and local government.”
These sources provided us with a list of 228 stakeholders  The questions included in the questionnaire were developed using issues  raised in our scoping interviews and background literature search. Most of  our questions provided respondents with answer options identified  through our interviews and background literature searches. In certain  cases, we also provided respondents with an “other” category which  allowed them to write in additional responses we did not identify.  However, because few respondents chose this option, we presented only  some of these responses as illustrative examples. The questionnaire  went through internal reviews, including independent GAO survey  experts, as well as four pretests with two stakeholders who work for  companies, one academic, and one stakeholder from an NGO. We chose  the pretesters to reflect the diversity of affiliations of the nanotechnology  experts included among our respondents. We revised and clarified the  questions and introductory material to the questionnaire based on  comments obtained in the internal reviews and from the pretesters.
We administered the questionnaire through a secure server. When we  completed the final survey questions and format, we sent an e-mail  announcement of the questionnaire to the nonfederal stakeholders in our  sample on September 23, 2011. Stakeholders were notified that the  questionnaire was available online and were given unique passwords and  user names on October 11, 2011. We sent follow-up e-mail messages on  October 18, 2011, October 25, 2011, and November 2, 2011 to those who  had not yet responded. Then we contacted all nonrespondents by  telephone, starting on October 21, 2011. The questionnaire was available  online until December 6, 2011.
Because this was not a sample survey, it has no sampling errors.  However, the practical difficulties of conducting any survey may introduce  errors, commonly referred to as nonsampling errors. For example,  difficulties in interpreting a particular question, sources of information  available to respondents, or entering data into a database or analyzing  them can introduce unwanted variability into the survey results. We took  steps in developing the questionnaire, collecting the data, and analyzing  them to minimize such nonsampling errors. For example, we worked with  social science survey specialists to design the questionnaire. We  pretested the draft questionnaire with four nonfederal stakeholders to  ensure that the questions were relevant, clearly stated, and easy to  understand. When we analyzed the data, an independent analyst  checked all computer programs. Since this was a web-based  questionnaire, respondents entered their answers directly into the  electronic questionnaire, eliminating the need to key data into a database,  minimizing error.
Of the 223 potential respondents, 138 completed the questionnaire during  our time frame, for an overall response rate of about 62 percent. The  numbers responding by affiliation type as categorized by us were   (1) “academic”—42 out of 62; (2) “company”—65 out of 114;   (3) “organization”—24 out of 39; and (4) “state and local government”—  7 out of 8.
To determine the extent the NNI strategy documents address desirable  characteristics of national strategies, we compared three key NNI  strategic documents against criteria for desirable characteristics of national strategies. Prior GAO reports have identified six desirable  characteristics for national strategy documents that would help shape the  policies, programs, priorities, resource allocations, and standards that  would enable federal agencies and other stakeholders to implement the  strategies and achieve the identified results. National strategies that  address these characteristics offer policymakers and implementing  agencies a management tool that can help ensure accountability and  more effective results. Table 5 provides the desirable characteristics  and the elements we looked for.
To assess whether the documents addressed these desirable  characteristics, two analysts independently reviewed the three NNI  strategy documents for the elements of each characteristic. We examined  the documents for inclusion of each element and for sufficient specificity  and detail. The two analysts compared their assessments, discussed any  discrepancies, and agreed upon a determination for each element. Each  characteristic was then given a rating of “address,” “partially address,” or  “does not address.” According to our methodology, the strategy  documents “address” a characteristic when it explicitly includes all, or  nearly all, elements of the characteristic and has sufficient specificity and  detail. The documents “partially address” a characteristic when it includes  some or most of the elements with sufficient specificity and detail. The  “partially address” category includes a range that varies from explicitly  including most of the elements to including as few as one of the elements  of a characteristic. A strategy “does not address” a characteristic when it  does not include any elements of a characteristic or references are too  vague or general to be useful.
Because the three NNI documents have different purposes and  audiences, an individual document need not address each element of  each desirable national strategy characteristic. Therefore, we reviewed  the three NNI strategy documents as a whole. In the case of the 2011  NNI strategic plan and Supplement to the President’s 2012 Budget, we  focused on sections related to EHS research, where applicable.
We identified agency documents related to nanotechnology EHS  research strategies and performance. Documents were identified based  on interviews with agency officials, review of agency websites, and a data  collection instrument sent to agencies requesting any documentation  related to the planning or performance of their nanotechnology EHS  research programs. In order to assess the extent of linkage between  agencies’ nanotechnology EHS research and national priorities of the  NNI, we evaluated agency management documents for explicit referrals  to NNI activities or guidance, including strategic documents such as the  NNI strategic plan or EHS research strategy, or activities of the NNI’s  planning and coordinating bodies. We also reviewed agencies’  Performance Reports in order to assess agencies' use of performance  information related to nanotechnology EHS research efforts.
We conducted this performance audit from February 2011 through May  2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that the  evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Collaborative Nanotechnology Environmental, Health, and Safety Research Agreements
paragraphs: Table 6 identifies collaborative nanotechnology EHS research project  agreements initiated by the NNI member agencies included in our review  from February 2008 to October 2011. The table identifies only those  projects specifically focused on nanotechnology. All the information in the  table is based on the written agreements, which were provided by NNI  member agencies’ officials.
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Dan Haas (Assistant Director),  Krista Anderson, Nirmal Chaudhary, Elizabeth Curda, Lorraine Ettaro,  Alison O’Neill, Tind Shepper Ryen, Jeanette Soares, Ruth Solomon, Hai  Tran, and Jack Wang made key contributions to this report.