Advanced Manufacturing: Commerce Could Strengthen Collaboration with Other Agencies on Innovation Institutes
Published: 2017-04-06T00:00:00

Fast Facts:
Over the past few decades, the United States has become less competitive in manufacturing. In response, the Manufacturing USA program supports public-private manufacturing innovation institutes. The institutes are intended to develop and promote advanced manufacturing technologies, such as 3D printing.
An interagency team was responsible for outlining agencies' roles and responsibilities in the program, but didn't include all relevant agencies in the process or fully identify how agencies could contribute. We recommended that the Department of Commerce, in its coordinating role, take action to address this concern.

Highlights:
section_title: Why GAO Did This Study
paragraphs: The RAMI Act includes a provision for GAO to assess the program every two years, with a final assessment in 2024. This is GAO's first report in response to the statutory provision. Among other objectives, GAO examined (1) the status of the network and use of the institutes, (2) the extent to which performance measures are in place to assess progress toward achieving the statutory program purposes, and (3) the extent to which Commerce has taken steps to coordinate agencies contributing to the program.
GAO reviewed documentation and interviewed officials from Commerce, DOD, DOE, DOL, the Department of Education, and Manufacturing USA institutes; and held discussion groups with a nongeneralizable sample of institute member representatives.
section_title: What GAO Found
paragraphs: As of December 2016, the Departments of Defense (DOD), Energy (DOE), and Commerce (Commerce) collectively had signed agreements to establish 11 manufacturing innovation institutes. Four of these institutes were established prior to enactment of the Revitalize American Manufacturing and Innovation Act of 2014 (RAMI Act), which requires Commerce to establish a network of institutes for manufacturing innovation. Since 2014, the network—called the Manufacturing USA network—has grown as DOD, DOE, and Commerce have established seven more institutes, and Commerce, DOD, and DOE plan to sponsor up to four more institutes. Each institute is a public-private partnership between the sponsoring federal agency and a nonfederal entity in charge of operations, with the nonfederal entity matching or exceeding the federal financial assistance. GAO's analysis of institute membership from May through September 2016 shows that about 780 members had joined the seven institutes that were operating during GAO's review (i.e., supporting research projects in their technology focus areas). Members receive a variety of benefits, such as access to intellectual property and networking opportunities.
Commerce, DOD, and DOE worked together to develop initial performance measures to track progress toward the Manufacturing USA program's statutory purposes. Additionally, DOD, working with Commerce and DOE, hired a consultant to review the Manufacturing USA program. The consultant's January 2017 report included recommendations on revised measures to track program progress. After considering the results of this review, Commerce plans to work with DOD and DOE to reach agreement on a revised set of measures. While Commerce may face challenges with assessing the program, such as the timeframe over which results may need to be measured, it has taken steps or has identified options to address these challenges.
Commerce has used a variety of mechanisms to coordinate the Manufacturing USA program. These mechanisms incorporate several key practices GAO has identified for enhancing and sustaining interagency collaboration. However, GAO found that the process used to develop a governance system that outlines agencies' responsibilities for contributing to the program did not include all relevant non-sponsoring agencies (agencies that do not sponsor institutes), or ensure that their roles and responsibilities for contributing to the program are fully identified. Specifically, non-sponsoring agencies, such as the Department of Labor (DOL)—which administers discretionary grant programs that can help increase the number of skilled workers in advanced manufacturing—were not actively involved in developing the governance system. Additionally, the governance system does not specify any responsibility for non-sponsoring agencies to provide information or expertise related to their activities to the program. A Commerce official told GAO that the governance system is subject to revision, but participation in the program is up to each agency. However, including all relevant agencies in the process of revising the system and fully identifying non-sponsoring agencies' roles and responsibilities could strengthen Commerce's efforts to leverage and coordinate agencies' contributions to the program, consistent with key practices for interagency collaboration.
section_title: What GAO Recommends
paragraphs: GAO recommends that Commerce work with all relevant federal agencies to fully identify roles and responsibilities for how agencies that do not sponsor institutes could contribute to the Manufacturing USA program. Commerce agreed, but suggested an alternative recommendation. GAO modified the recommendation to clarify its intent.

section_title: Letter
paragraphs: The manufacturing sector encompasses businesses across a variety of  industries and plays a critical role in the U.S. economy and national  defense. However, according to a June 2011 report from the President’s  Council of Advisors on Science and Technology (PCAST), the share of  U.S. economic activity from the manufacturing sector has declined over  several decades, and the U.S. trade balance in advanced technology  products, which was long a relative strength of the United States, shifted  from a surplus to a deficit starting in 2001. The report also stated that the  United States has been steadily losing the research and development  activities linked to manufacturing—along with associated high-skilled  jobs—to other nations, as well as losing the competitiveness to  manufacture products invented in the United States.
Beginning with its 2011 report, PCAST issued a series of reports that  recommended a number of steps to increase U.S. competitiveness in  advanced manufacturing—that is, manufacturing that uses innovative  technology to improve products or processes. PCAST’s  recommendations included that the federal government establish a  national network of manufacturing innovation institutes as public-private  partnerships to create a manufacturing research infrastructure to help  bridge the gap between research and development activities and  domestic production. In 2012, at the request of the President and using  existing statutory authorities, the Department of Defense (DOD)  established a pilot manufacturing innovation institute with co-funding and  participation from the Department of Energy (DOE) and other agencies.  This pilot institute focused on additive manufacturing technology. In  2012, the President set forth a goal of establishing a network of up to 15  institutes. Also, in 2013, the National Science and Technology Council  released a report outlining a preliminary design for the national network,  citing the pilot manufacturing innovation institute as a benchmark for other  institutes. Subsequently, using their existing statutory authorities, DOD  and DOE established additional institutes, each with its own technological  focus.
In December 2014, Congress passed, and the President signed into law,  the Revitalize American Manufacturing and Innovation Act of 2014 (RAMI  Act). Among other things, the RAMI Act requires the Secretary of  Commerce to establish a program within the Department of Commerce’s  (Commerce) National Institute of Standards and Technology (NIST); the  act refers to this program as the Network for Manufacturing Innovation  program. In September 2016, the Secretary of Commerce announced a  new public name for the program, Manufacturing USA; we use this name  for the program in this report. According to the RAMI Act, the purposes  of the program include stimulating U.S. leadership in advanced  manufacturing research, innovation, and technology; accelerating  development of an advanced manufacturing workforce; and creating and  preserving jobs. The RAMI Act requires the Secretary of Commerce to  establish, within NIST, a national program office to oversee and carry out  the program; this office is known as the Advanced Manufacturing National  Program Office (AMNPO). The RAMI Act specifies a number of functions  for the national program office, such as establishing procedures,  processes, and criteria as may be necessary and appropriate to maximize  cooperation and to coordinate the activities of the program with programs  and activities of other federal agencies whose missions contribute to or  are affected by advanced manufacturing.
As part of the program, the RAMI Act requires the Secretary of  Commerce to establish a network of centers for manufacturing innovation  (which we refer to as the Manufacturing USA network). There are two  types of centers (which we refer to as Manufacturing USA institutes):  those that receive financial assistance from Commerce under the RAMI  Act and those that do not. Institutes that do not receive financial  assistance from Commerce pursuant to the RAMI Act either (1) are  considered institutes by the act because they were in existence and  formally recognized pursuant to law or executive action as manufacturing  innovation centers before the law’s enactment or (2) are recognized by  the Secretary of Commerce, at the institute’s request, as an institute for  manufacturing innovation for purposes of participating in the network and  are substantially similar to those established by Commerce under the act.  For example, under the RAMI Act the National Additive Manufacturing  Innovation Institute is considered an institute and the Secretary of  Commerce has recognized institutes established by DOD and DOE  pursuant to their statutory authority as part of the network. Each institute  is a public-private partnership between the sponsoring federal agency  and a nonfederal entity (generally a nonprofit organization or university) in  charge of the institute’s operations. The institutes have members that  include companies, nonprofit organizations, and academic institutions,  among others.
The RAMI Act also contains a provision for GAO to assess not less  frequently than once every 2 years the operation of the Manufacturing  USA program and to provide a final assessment not later than December  31, 2024. The act requires that each assessment include, among other  things, a review of the management, coordination, and industry utility of  the program and an assessment of the extent to which the program has  furthered the purposes described in the RAMI Act. This is our first report  on this program in response to the statutory provision. This report  examines (1) the status of establishing the Manufacturing USA network  and the extent to which manufacturers and other entities have used the  institutes; (2) how federal agencies manage the Manufacturing USA  institutes and what challenges, if any, have been encountered to date; (3)  the extent to which performance measures are in place to help  Commerce assess progress toward achieving the statutory purposes of  the program; and (4) the extent to which Commerce has taken steps to  coordinate the efforts of agencies that contribute to the Manufacturing  USA program.
To obtain information on Manufacturing USA institute status, use,  management, performance measures, and coordination, we reviewed  Manufacturing USA program and institute documentation, including the  cooperative agreements establishing the institutes; these agreements  include amounts of federal and nonfederal financial assistance. We also  interviewed Commerce, DOD, and DOE officials to obtain information  about the institutes their agencies sponsor. In addition, we asked  Manufacturing USA institute officials to respond to a structured set of  questions related to their institute’s (1) structure and funding; (2) industry  use and data collection; (3) management, coordination, and challenges;  and (4) program assessment; among other questions. We analyzed  membership data provided by institute officials to obtain information on  membership participation by manufacturers and other entities. We relied  on institute officials to determine which of their members were from small  or large manufacturers because there is no agreed-upon definition for  manufacturers in each of these categories. We obtained information on  membership levels, benefits, and rights from institute documentation and  websites and verified its accuracy with institute officials.
We also held three discussion groups with 19 representatives of institute  members to obtain their perspectives on various issues, such as how  their organizations benefited from membership in a Manufacturing USA  institute, and challenges related to institute operations. We held one  group for small manufacturers comprised of four institute member  representatives; one group for large manufacturers comprised of eight  representatives; and one group for academic and nonprofit organizations  comprised of seven representatives. We selected institute members for  the discussion groups based on Manufacturing USA institute officials’  recommendations, to provide a nongeneralizable sample of perspectives.  Our selection is not representative of the full population of institute  members and does not include entities that have not yet signed a  membership agreement with an institute.
To obtain additional information on coordination of the efforts of agencies  that contribute to the Manufacturing USA program, we interviewed  Commerce, DOD, and DOE officials as well as officials from the  Departments of Education (Education) and Labor (DOL). We included  Education and DOL in the scope of our work because the RAMI Act  includes as purposes of the program accelerating the development of an  advanced manufacturing workforce and creating and preserving jobs. To  evaluate Commerce’s coordination of the agencies that contribute to the  Manufacturing USA program, we compared Commerce’s efforts against  selected key practices for interagency collaboration that we have  previously identified. We selected the key practices that were most  relevant to Commerce’s efforts to coordinate contributing agencies.
We conducted this performance audit from February 2016 to April 2017 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: This section provides information on the history of the Manufacturing USA  program, the provisions of the RAMI Act, and the awards process and  membership of the institutes.
subsections: section_title: Manufacturing USA Program History
paragraphs: The June 2011 PCAST report was followed in July 2012 by another  report, prepared by the Advanced Manufacturing Partnership Steering  Committee; this report contained 16 recommendations to improve the  manufacturing competitiveness of the United States. One of the  recommendations was to establish a national network of manufacturing  innovation institutes as public-private partnerships. After DOD established  the first such institute as a pilot institute in 2012, the National Science and  Technology Council released its January 2013 report proposing a  preliminary design for the network. This report envisioned a network of  public-private partnerships to be supported with a co-investment of  federal obligations ranging from $70 million to $120 million for each  institute and equal or greater amounts in nonfederal pledges (also known  as cost-share). The amount of investment would depend on such factors  as the magnitude of the opportunity and maturity of the technology and  would be distributed across 5 to 7 years. Total capitalization of an institute  across this time period was envisioned to be $140 million to $240 million.  The report also envisioned that an institute would become sustainable  within 7 years of the public announcement of an award of financial  resources to start up and operate the institute, and that the institute would  provide long-term economic impact in its region and nationwide. Building  on the July 2012 Advanced Manufacturing Partnership report, an October  2014 PCAST report provided 12 recommendations with further guidance  on how the United States could sustain its lead in innovation. For  example, one of the recommendations was to create—through the  National Economic Council, the Office of Science and Technology Policy,  and the implementing agencies and departments—a shared  Manufacturing USA governance structure that includes input from various  agencies as well as from private sector experts, organized labor, and  academia.
section_title: RAMI Act Provisions
paragraphs: Under the RAMI Act, the purposes of the Manufacturing USA program are  to  improve the country’s manufacturing competitiveness and increase  production of goods manufactured predominantly within the United  States;  stimulate U.S. leadership in advanced manufacturing research,  innovation, and technology;  facilitate transition of innovative technologies into scalable, cost- effective, and high-performing manufacturing capabilities;  facilitate access by manufacturing enterprises to capital-intensive  infrastructure; accelerate development of an advanced manufacturing workforce;  facilitate peer exchange of and documentation of best practices in  addressing advanced manufacturing challenges;  leverage nonfederal sources of support to promote a stable and  sustainable business model without the need for long-term federal  funding; and  create and preserve jobs.
The RAMI Act requires the Secretary of Commerce to report annually to  Congress, until December 31, 2024, on the program’s performance,  including an assessment of Commerce’s institutes and an assessment of  the participation in, and contributions to, the network by institutes  established by other agencies. Annual reports are also to include an  assessment of how well the program is meeting its statutory purposes.
The RAMI Act also contains a number of provisions related to  collaboration between Commerce and other agencies. For example, in  awarding financial assistance to establish institutes, the RAMI Act  requires that the Secretary of Commerce, through AMNPO, collaborate  with federal departments and agencies whose missions contribute to or  are affected by advanced manufacturing. In addition, several of the  functions of the AMNPO under the RAMI Act also pertain to collaboration.  The functions that the RAMI Act identifies for the national office include:
Overseeing the planning, management, and coordination of the
Entering into memorandums of understanding with federal  departments and agencies whose missions contribute to or are  affected by advanced manufacturing, to carry out the program’s  statutory purposes;
Developing, not later than 1 year after the date of enactment of the  RAMI Act, and updating not less frequently than once every 3 years  thereafter, a strategic plan to guide the program;
Establishing such procedures, processes, and criteria as may be  necessary and appropriate to maximize cooperation and coordinate  the activities of the program with programs and activities of other  federal departments and agencies whose missions contribute to or  are affected by advanced manufacturing;
Establishing a clearinghouse of public information related to the  activities of the program; and
Acting as a convener of the network.
Additionally, the RAMI Act requires that, when developing and updating  the strategic plan, the Secretary of Commerce solicit recommendations  and advice from a wide range of stakeholders, including industry, small  and medium-sized manufacturing enterprises, research universities,  community colleges, and other relevant organizations and institutions on  an ongoing basis. The RAMI Act also requires the Secretary of  Commerce to ensure that the AMNPO incorporates NIST’s Hollings  Manufacturing Extension Partnership into Manufacturing USA program  planning to ensure that the results of the program reach small and  medium-sized entities.
section_title: Manufacturing USA Institute Awards Process and Membership
paragraphs: DOD, DOE, and Commerce have provided competitive financial  assistance for institutes using similar processes for soliciting applications  and making awards. DOD and DOE have identified prospective  technology focus areas through requests for information from nonfederal  organizations and meetings with industry officials and other stakeholders.  For Commerce institutes, the RAMI Act requires that the Secretary of  Commerce ensure an open process that allows for consideration of all  applications relevant to advanced manufacturing regardless of technology  focus area. Subsequently, each of the sponsoring agencies released  notices of intent announcing a competition for their new institutes and  then released federal funding opportunity announcements. The federal  funding announcements included goals and objectives for the institutes  that were related to the Manufacturing USA program and to the agency  mission. Potential applicants generally have had opportunities to ask  clarifying questions via mechanisms such as webinars and e-mail.  Applicants submitted pre-application concept papers, which were  evaluated by subject matter experts in the government to determine, for  example, the potential to fulfill a recognized national need. Based on  these subject matter experts’ review, the sponsoring agencies invited  selected applicants to submit full proposals. Full proposals included  detailed information—such as a business strategy, leadership and  sustainment plan, and technology investment plan—and were evaluated  by multi-agency source selection team. Ultimately, an official from the  sponsoring federal agency selected the winning applicant, after which the  agency and the applicant negotiated an agreement, such as a  cooperative agreement, to establish and manage the institute. These  awards provided financial resources to start up and operate an institute  focusing on the specific technology area.
The institutes provide members with access to shared facilities and  equipment. Members can take advantage of these resources in a variety  of ways, such as by collaborating on research related to the technology  focus area of the institute. Membership is open to all U.S. industrial  organizations, academic institutions, nonprofit organizations, and  government agencies that want to further technology and education in a  certain focus area. Each institute has its own membership terms— including a range of costs, rights, and benefits—and required time  commitment. Specific terms and conditions are detailed in formal  membership agreements.
section_title: In 2016, the Manufacturing USA Network Included Eleven Established Institutes, and up to Four Additional Institutes Are Planned
paragraphs: As of December 2016, DOD, DOE, and Commerce collectively had  established a total of eleven innovation institutes, which have become  part of the Manufacturing USA network. Seven of these eleven institutes  were operating (i.e., supporting research projects in their technology  focus areas) as of December 2016. In addition, one DOD institute, two  DOE institutes, and potentially one additional Commerce institute were in  the process of being established through a competitive solicitation. Our  analysis of institute membership between May 2016 and September 2016  shows that about 780 manufacturers and other entities, such as academic  institutions and nonprofit organizations, have become members of the  operating institutes. Institute members receive a variety of benefits and  rights, such as networking opportunities.
subsections: section_title: Seven of Eleven Established Institutes in the Network Were Operating in 2016
paragraphs: As of December 2016, DOD, DOE, and Commerce collectively had  established a total of eleven institutes that have become part of the  Manufacturing USA network. Four institutes—three DOD institutes and  one DOE institute—were established prior to the enactment of the RAMI  Act in 2014. The remaining seven institutes were established after the  enactment of the RAMI Act. Four of these institutes—Advanced  Functional Fabrics of America, the Advanced Tissue Biofabrication  Manufacturing Innovation Institute, the Clean Energy Smart  Manufacturing Innovation Institute, and the National Institute for  Innovation in Manufacturing Biopharmaceuticals—were established in  2016 and were not yet operating as of December 2016. However, all  eleven institutes are part of the network. Appendix I provides additional  information on the DOD and DOE institutes that were operating as of  December 2016.
The institutes, which are located across the United States, focus on a  wide variety of areas of advanced manufacturing, including additive  manufacturing (three-dimensional, or 3D, printing), lightweight metals,  integrated photonics circuits, novel fiber and textiles, and  biopharmaceuticals. DOD, DOE, and Commerce plan to provide over  $800 million in financial assistance, generally through cooperative  agreements, for the eleven established institutes they sponsor. According  to DOD, DOE, and NIST officials, the agencies plan to distribute this  assistance over a 5 to 7 year period, at the end of which they expect the  institutes to be sustainable without guaranteed federal financial  assistance. Award recipients have pledged about $1.9 billion in  nonfederal funds over the same 5 to 7 year period. However, the officials  also said they expect the institutes will be eligible to receive additional  federal financial assistance, such as competitively awarded grants for  research.
According to the February 2016 Manufacturing USA program annual  report, the seven institutes established by DOD are designed to  overcome the challenge DOD faces when the existing industrial  infrastructure is insufficient to develop and produce new and emerging  technologies that hold strategic promise. DOD considers its institutes to  represent a key investment strategy for the department and its  Manufacturing Technology program. Table 1 provides an overview of  the seven institutes DOD had established as of December 2016.
DOE’s mission is to ensure America’s security and prosperity by  addressing its energy, environmental, and nuclear challenges through  transformative science and technology solutions. DOE officials told us  that the three institutes established by DOE support this mission by  focusing support on innovative, advanced manufacturing technologies  that will enhance domestic advanced manufacturing competitiveness and  create jobs for American workers by maintaining U.S. global  competitiveness in clean energy manufacturing. Table 2 provides an  overview of the three institutes DOE had established as of December  2016.
In December 2016, the Secretary of Commerce announced the selection  of an award recipient for the first institute sponsored by Commerce, the  National Institute for Innovation in Manufacturing Biopharmaceuticals.  According to Commerce, this institute is expected to help advance U.S.  leadership in the biopharmaceutical industry, foster economic  development, improve medical treatments, and ensure a qualified  workforce. Table 3 provides an overview of the institute Commerce had  established as of December 2016.
section_title: DOD, DOE, and Commerce Were in the Process of Sponsoring up to Four New Institutes as of December 2016
paragraphs: DOD, DOE, and Commerce were in the process of evaluating  applications, selecting award recipients, and negotiating agreements to  establish up to four additional institutes as of December 2016. The DOD- sponsored institute, the Robots in Manufacturing Environments  Manufacturing Innovation Institute, will focus on robotic technologies in  manufacturing, such as human robot interaction, learning, and mobility.  The two DOE institutes, Rapid Advancement in Process Intensification  Deployment and Reducing Embodied-energy and Decreasing Emissions,  will have the following technology focus areas: (1) developing  breakthrough technologies to boost domestic energy productivity and  energy efficiency of manufacturing processes, and (2) reuse, recycling,  and remanufacturing of materials. In addition, Commerce officials were  considering whether to establish a second institute, subject to the  availability of funds. This second Commerce-sponsored institute could  address any area of advanced manufacturing as long as it does not  duplicate the technology focus areas and programs of existing or  announced federally-funded institutes. Table 4 provides an overview of  the institutes that DOD and DOE were in the process of establishing as of  December 2016.
section_title: About 780 Manufacturers and Other Entities Were Members of the Operating Institutes in 2016
paragraphs: Our analysis of institutes’ membership data showed that about 520  manufacturers (mostly small manufacturers according to a Commerce  official) were members of the seven DOD and DOE institutes that were  operating as of December 2016. In addition, about 260 other entities,  including academic institutions, state government agencies, nonprofit  organizations, and NIST’s Hollings Manufacturing Extension Partnership  centers were members of these institutes. Some institute officials told us  that additional members were pending. For example, America Makes  officials told us that as of July 2016, they were processing 14 new  member requests.
Manufacturers and other entities obtain a variety of benefits and rights,  such as networking opportunities and intellectual property access, based  on membership level. Each institute has developed a membership  structure that provides different costs, benefits, and rights at each level.  The number of membership levels ranges from 3 levels at one institute to  10 levels at another institute. For manufacturers, membership structures  generally consist of 3 to 4 levels. For other entities, membership  structures and the number of membership levels vary. At the highest  membership levels, members have the greatest influence on the  governance of the institute and the direction of research and  development. Benefits and rights at those levels could include  participation in technology roadmap development, greater access to  intellectual property, and participation in determining the order in which  projects are executed. Benefits and rights at lower membership levels  could include assistance with navigating government contracts,  collaboration with other members, access to member-only data, and  participation in research and development projects. See appendix I for  additional information on membership benefits and rights for DOD and  DOE institutes that were operating as of December 2016.
Figure 1 illustrates participation by manufacturers and other entities at the  highest 3 levels of membership. At the highest 2 membership levels,  which we denote as Tier 1 and Tier 2, participation by manufacturers and  other entities is fairly evenly divided. At the membership level which we  denote as Tier 3, manufacturers represent a larger proportion of  participating members.
Figure 2 illustrates participation by large and small manufacturers at the  highest 3 levels of membership, which we denote as Tier 1, Tier 2, and  Tier 3. Manufacturers who join Manufacturing USA institutes at a higher  membership level are typically larger manufacturers.
Membership costs vary depending on the institute and type of entity.  Some institutes allow membership costs to be provided by members as  in-kind contributions. In-kind contributions can consist of items such as  equipment, software, or facility space. Our analysis of institutes’  membership data showed that for manufacturers, costs for the highest  membership level range from $200,000 cash or in-kind annually, with at  least a 3-year commitment, to $5 million provided over a period of up to 5  years. Costs for the lowest membership level range from no fee, but with  a 3-year commitment, to $15,000 cash or in-kind annually, with at least a  3-year commitment. For other entities—institutes that delineate between  manufacturers and other types of members—membership costs generally  range from no fee, with a 1-year commitment, to $5 million in  expenditures, with a 5-year commitment. According to an institute official,  in-kind contributions can attract more small manufacturers and other  organizations to become members, and to become members at higher  levels, than if membership costs were restricted to cash membership  fees. See appendix I for additional information on membership costs for  each institute.
During discussion groups with representatives of institute members, 19  representatives of manufacturers and other entities that are members of  Manufacturing USA institutes provided their perspectives on how their  organizations have benefited from institute membership. Representatives  of institute members in each of the discussion groups identified some  examples of benefits, including the following:
Small manufacturers. All four representatives of small manufacturers  identified networking opportunities as a benefit of institute  membership. They told us they have established working relationships  with other large and small manufacturers, and with suppliers, and that  they obtained contracts they otherwise might not have been able to  secure. One small manufacturer representative also said that, through  networking, he had gained a more comprehensive understanding of  his company’s technology. Another small manufacturer representative  said that, through networking, he has kept up with market trends and  what other manufacturers are trying to achieve.
Large manufacturers. Four of seven large manufacturer  representatives cited acceleration of technology development as a  benefit of institute membership. For example, three large  manufacturer representatives stated that networking has helped  accelerate the development of their technology by between 2 and 5  years, thereby shortening the time needed to get their products ready  for commercial release.
Academic and Nonprofit. Five of eight other entities’ representatives  said they have been able to strengthen their educational programs in  ways that help prepare students to enter a certain manufacturing  industry—for example, by adding material relevant to a particular  technology, such as semiconductors. One academic entity  representative also said that some students work on institute projects  that make the students more marketable upon graduation.
section_title: DOD and DOE’s Operating Institutes Are Primarily Managed through Cooperative Agreements, and Institute Officials and their Members Identified Several Operating Challenges
paragraphs: DOD and DOE primarily use cooperative agreements to manage their  Manufacturing USA institutes that were operating as of December 2016.  These agreements include terms and conditions for the institutes, and the  agencies seek to ensure these terms and conditions are met through  meetings, reports, and other mechanisms. Manufacturing USA institute  officials and institute member representatives identified various  challenges related to institute operations.
subsections: section_title: DOD and DOE Primarily Rely on Cooperative Agreements to Manage Institutes and Use a Variety of Mechanisms to Ensure Terms and Conditions Are Met
paragraphs: DOD and DOE manage the Manufacturing USA institutes that were  operating as of December 2016 primarily through cooperative  agreements that define the terms and conditions that govern the  relationships between the agencies and the institutes. For example, the  cooperative agreements identify the amount of federal financial  assistance agencies provide to each institute and the period of  performance for the institute to carry out work under the agreement.  According to AMNPO officials, the sponsoring federal agency determines  the federal financial assistance provided through these cooperative  agreements, and this amount can be up to $120 million, which is matched  or exceeded by funding from private industry and other nonfederal  sources over a 5- to 7-year period. Cooperative agreements may also  address other aspects of the relationships, including how facilities and  infrastructure will be established, how research projects will be  determined, and how projects will be managed. In addition, the  cooperative agreements may require Manufacturing USA institutes to  develop governance plans that define types of membership, establish  sustainability goals, and help manage intellectual property.
According to institute and agency officials, DOD and DOE have  established mechanisms to ensure that their Manufacturing USA  institutes meet the terms and conditions specified in the cooperative  agreements. Institute officials told us that DOD and DOE may conduct  weekly, monthly, and quarterly meetings with each of their institutes. For  example, officials from one DOD institute said that DOD officials perform  program management functions through weekly teleconferences with the  institute’s executive director, executive staff, and finance staff. Similarly,  officials from one DOE institute said that a DOE Technology Manager  conducts weekly teleconferences with the institute’s management staff.  Institute officials said that DOD’s program managers may conduct  monthly visits to their institutes, and DOE officials conduct monthly  meetings with their institutes to assess progress toward achieving key  goals specified in the cooperative agreements. DOD and DOE also host  quarterly meetings for the institute directors and program managers of  their respective institutes, at which they discuss a variety of topics. For  example, DOD officials said the focus of their quarterly meetings is to  ensure their institutes are on track to being sustainable. Finally, DOD and  DOE have reporting requirements for the institutes they have sponsored.  The cooperative agreements all contain requirements for reports to be  submitted to the sponsoring agency, such as expenditure reports,  progress reports, technical roadmaps, and presentations from program  reviews. For example, DOD officials said their Manufacturing USA  institutes are required to report semi-annually on their progress toward  becoming sustainable. DOD uses semi-annual program management  review presentations to assess its Manufacturing USA institutes’  sustainability plans and progress toward implementing these plans. In  addition, DOE officials said that they require their institutes to report on  sustainability along with other program milestones through quarterly  technical reviews.
section_title: Institute Officials and Member Representatives Identified Several Operating Challenges
paragraphs: Manufacturing USA institute officials and the institute member  representatives we spoke with identified several operating challenges  faced by the institutes. These included:
Negotiating agreements and approving projects. Officials from  three DOD institutes and one DOE institute described challenges  related to negotiating and approving membership and project  agreements. For membership agreements, there are challenges in  negotiating rights to any intellectual property created; for example,  potential members may be hesitant to share these rights with the  institute. These negotiation challenges can cause schedule delays for  institute operations. For project agreements, there are challenges in  negotiating the terms and conditions of the agreements and with the  amount of time it takes for sub-recipients and sponsoring agencies to  review and approve the agreements. Some institutes have taken  steps to address these challenges. For example, one DOD institute  used language from another DOD institute’s membership agreement  to avoid schedule delays that could result from developing its own. In  addition, one DOD institute has increased the amount of time  allocated for the development of project agreements.
Meeting cost-share requirements. Representatives of two academic  institutions and two small manufacturers expressed concerns about  cost-share requirements for institute participation. For example, one  representative said universities are not capable of providing the  amount of cash some institutes require under their membership  agreements, and another representative said it is a challenge for  small manufacturers to participate in some institutes because of the  cost-share requirements. However, according to an institute official,  the possibility of providing in-kind contributions could help some small  manufacturers and other organizations address these concerns.
Ensuring institute sustainability. Going forward, as institutes reach  the end of the period of performance for their agreements, officials  from all DOD institutes identified the sustainability of the institutes as  a challenge, although they told us several different reasons for their  concerns:
Amount of federal financial assistance. Officials from all DOD  institutes said the amount of federal financial assistance provided  under their cooperative agreements may not be enough for the  institutes to become sustainable over the 5- to 7-year period of  performance.
Length of federal financial assistance. Officials from three DOD  institutes said the 5- to 7-year period of performance for their  cooperative agreements may not be sufficient to become  sustainable.
Collaborating with foreign-owned organizations. Officials from two  DOD institutes expressed concerns about constraints on  collaborating with foreign-owned organizations because these  constraints affect their sustainability plans. For example, officials  from one DOD institute said that the institute’s cooperative  agreement requires DOD approval for foreign-owned  organizations’ participation and that DOD may not provide  approval, limiting potential sources of institute funding.
Type of activities funded. Some DOD institute officials said that  the cooperative agreements require them to carry out activities  that do not contribute to sustainability. For example, a senior  official from one DOD institute said that education and workforce  development activities are required as a project component under  the institute’s cooperative agreement, but these activities make  only a minor contribution to the institute’s sustainability.
Institute officials identified several options for how they might address  sustainability challenges. For example, officials from one DOD institute  said they will base their decision about which type of activities to fund,  after federal financial assistance ends, on which ones help make the  institute sustainable as well as on the availability of funds from nonfederal  sources. Similarly, another DOD institute official said that the institute  may stop funding education and workforce development activities after  federal financial assistance under the cooperative agreement ends.  Further, officials from one DOD institute said they may not require the  same level of scrutiny on collaborating with foreign-owned organizations  after federal financial assistance under their cooperative agreement ends.
section_title: Commerce, DOD, and DOE Have Developed Performance Measures and Commerce Is Taking Steps to Address Potential Challenges in Assessing Program Progress
paragraphs: AMNPO led a collaboration with DOE and DOD to develop an initial set of  performance measures to assess the Manufacturing USA program’s  progress toward furthering its eight statutory purposes. AMNPO, DOE,  and DOD continue to work together to improve agreed upon performance  measures for the institutes. Commerce also has taken steps or has  identified options to address challenges in measuring program  performance.
subsections: section_title: Commerce, DOD, and DOE Developed an Initial Set of Performance Measures for the Program
paragraphs: AMNPO led a collaboration with DOE and DOD to develop guidance for  Manufacturing USA institutes and sponsoring agencies that included  initial performance measures that could be used to track progress toward  achieving the eight statutory purposes of the Manufacturing USA  program. The performance measures outlined in this guidance relate to  the four program goals for Manufacturing USA that were established in  the program’s first strategic plan, published in February 2016. In  developing the strategic plan, AMNPO officials said they worked with  other agency officials, including DOD and DOE officials, to develop  Manufacturing USA program goals that aligned with the initial measures  as well as with the program’s statutory purposes. Figure 3 describes the  relationship between the program’s statutory purposes, program goals,  and initial institute performance measures.
Institutes are only required to report on measures that have been agreed  upon with their sponsoring agencies. According to AMNPO officials,  reporting on institute performance is the responsibility of the respective  “funding agency” (i.e., the sponsoring agency). Also, the RAMI Act does  not include reporting requirements for institutes sponsored by DOD and  DOE. However, as noted previously, the RAMI Act requires the  Secretary of Commerce to report annually to Congress on the  performance of the program, including an assessment of how well the  program is meeting the purposes identified in the RAMI Act.
AMNPO, DOD, and DOE told us that they are working together to  consider whether any revisions may be warranted to these initial  performance measures. Specifically, in coordination with AMNPO and  DOE, DOD hired an external consultant to conduct an assessment of the  Manufacturing USA program design and progress made toward achieving  program objectives, and recommend areas where Manufacturing USA  and its institutes could enhance programs. In January 2017, the external  consultant reported on the results of its review. Among other  recommendations, the report recommended developing measures to  evaluate Manufacturing USA program performance using a phased  approach. For example, the report recommended developing measures at  the program level for two phases:  1.  start-up support and shared services provision (i.e., the phase in  which the program provides support to individual institutes in their  establishment and delivers services applicable to all the institutes);  and  2.  long-term outcomes (i.e., the phase in which the program tracks the  achievement of program goals on a manufacturing sector-wide and  national basis).
AMNPO officials told us that they plan to incorporate the results of this  review into a set of revised performance measures. They also told us  that, subsequently, they plan to work with DOD and DOE to reach  agreement on these revised measures to allow the Secretary of  Commerce to report the data collected on the revised measures in the  Manufacturing USA annual report to Congress for fiscal year 2017. This  annual report is expected to be issued in May 2017.
section_title: Commerce Has Taken Steps or Has Identified Options to Address Challenges in Measuring Program Performance
paragraphs: Institute officials identified challenges related to measuring progress  toward achieving the Manufacturing USA program’s statutory purposes,  including measuring outcomes within a short timeframe, and Commerce  has taken steps to address these challenges. First, institute officials told  us that some of the program’s statutory purposes are difficult to measure.  In particular, some institute officials told us that measuring the number of  jobs created or preserved may not be feasible because, for example,  there are challenges with associating the number of jobs created and  preserved directly to an institute’s activities. Second, several institute  officials also said that the timeline associated with measuring progress  toward achieving the program’s statutory purposes may be too short. For  example, institutes are relatively new and generally still in the process of  establishing mechanisms to collect meaningful data related to many of  the program purposes. Further, measuring economic impact and job  creation, for example, requires performance measures with lagging  indicators that will not be measurable until years into the future, according  to institute officials. AMNPO officials told us that they tried to address  these challenges in the first Manufacturing USA program annual report by  including information on some leading indicators of success among  institutes that had available data, such as the level of involvement by  industry and academia and amount of cost-share. Also, as mentioned  above, AMNPO officials plan to incorporate the results of the recent  external consultant review, which recommended developing measures  that assess progress during the program’s initial start-up phase in  addition to long-term outcome measures. For example, the consultant’s  report suggested developing long-term outcome measures that relate to  achievement of program goals on a manufacturing sector-wide and  nationwide basis, such as increased economic competitiveness,  macroeconomic results, and workforce results.
Agencies may also face challenges collecting performance information  from institutes after their cooperative agreements end, and Commerce  has identified options to continue collecting this information. Institutes  currently track and report information required by their sponsoring agency  and cooperative agreements. Under the DOE cooperative agreements we  reviewed, financial assistance recipients are to continue to report annually  for five years beyond the project period on the utilization and impact of  the institute and technical progress in implementing and deploying the  technologies on the institute’s roadmap. Conversely, DOD officials told  us the cooperative agreements the agency entered into before December  2016 do not require their institutes to continue to report on performance  after the agreements’ period of performance ends. According to AMNPO  officials, one way to continue collecting this information is for institutes to  voluntarily continue providing performance information to federal  agencies, including Commerce. However, in the absence of ongoing  federal financial support from their funding agencies for the efforts and  expense of collecting and providing data, the officials said that institutes  may be reluctant to continue reporting on performance. AMNPO officials  told us of another option to ensure continued performance reporting but  said that they did not have plans in place to use it. Specifically, the  officials said the Secretary of Commerce could establish a requirement  for institutes funded by other agencies to submit performance reports to  Commerce after their cooperative agreements end as a condition of their  continued network participation because the Secretary has the statutory  authority to recognize those institutes for the purpose of participating in  the network. AMNPO officials said they anticipate that institutes will  choose to remain in the network and continue to voluntarily report to  AMNPO because the institutes value the services and other benefits they  receive from participating.
section_title: Commerce Used Various Mechanisms to Help Coordinate the Manufacturing USA Program, but It Has Not Included All Relevant Agencies in Fully Identifying Roles and Responsibilities
paragraphs: AMNPO uses a variety of collaborative mechanisms to coordinate the  efforts of the agencies that contribute to the Manufacturing USA program,  including a network governance system that defines some roles and  responsibilities for agencies that sponsor institutes as well as for agencies  that do not sponsor institutes (non-sponsoring agencies). However, the  process for developing the governance system did not include all relevant  non-sponsoring agencies or ensure that their roles and responsibilities  are fully identified.
subsections: section_title: Various Mechanisms Support Coordination of the Manufacturing USA Program
paragraphs: AMNPO has used a variety of mechanisms to enhance coordination  among the institutes and agencies that contribute to the Manufacturing  USA program. These mechanisms incorporate several key practices for  enhancing and sustaining interagency collaboration. As our prior work  has found, an interagency mechanism for collaboration is any  arrangement or application that can facilitate collaboration between  agencies. One of these mechanisms is the December 2016  Manufacturing USA network charter. The charter says that AMNPO is  responsible for supporting network functions, which include establishing  the network; facilitating intra-network collaboration; fostering robust  communication between the network and external stakeholders; and  sustaining, strengthening, and growing the network. The charter also  identifies several subfunctions for each of these functions. For example,  the facilitating intra-network collaboration function includes, among other  subfunctions, establishing forums for network collaboration, information  exchange, and knowledge management; facilitating the organization and  sharing of lessons learned and best practices across the network; and  facilitating network-level discussions among institutes regarding  management of technology interfaces and technology gaps.
Another mechanism that AMNPO has developed is the Manufacturing  USA strategic plan. AMNPO collaborated with federal agencies that  contribute to the Manufacturing USA program to develop the strategic  plan; these agencies included the Department of Agriculture, Commerce,  DOD, Education, DOE, the National Aeronautics and Space  Administration, and the National Science Foundation. According to the  strategic plan, agencies and institutes participating in the Manufacturing  USA program collectively work toward achieving program goals. For  example, to work toward the program goal of accelerating the  development of an advanced manufacturing workforce, the strategic plan  states that the network can help institutes navigate opportunities for  federal financial assistance that could support their efforts, including  workforce development programs such as those authorized by the  Workforce Innovation and Opportunity Act, as amended. AMNPO  officials told us that AMNPO has an ongoing effort to help institutes  navigate such opportunities. The strategic plan also identified other  opportunities for the network to enhance coordination across agencies  that contribute to the Manufacturing USA program, such as by serving as  a clearinghouse for information about workforce support.
AMNPO also uses mechanisms such as hosting teleconferences,  convening in-person meetings, and using technologies that allow it to  collaborate with representatives of different agencies and institutes. For  example, according to AMNPO officials, AMNPO hosts monthly  teleconferences with directors from DOD and DOE institutes and senior  deputies from DOD and DOE and convenes in-person, semi-annual  Manufacturing USA network meetings that include institute and agency  staff. For instance, an August 2016 Manufacturing USA network meeting  included discussions on a public name for the program, communications,  and establishing a Directors Council. According to AMNPO officials, the  Manufacturing USA Directors Council will further facilitate cooperation  and collaboration among the institutes. The officials said AMNPO has  hosted other meetings, such as a workforce development workshop that  included workforce leaders from all institutes; subject matter experts;  Hollings Manufacturing Extension Partnership center officials; and agency  officials from Education, the National Science Foundation, Commerce,  DOD, and DOE. AMNPO also recently launched a web-based shared  services platform that allows institutes to share best practices and that  allows agencies that participate in the Manufacturing USA network to  share information with the institutes. AMNPO officials told us they are  working with the institutes to provide content for this system and expect to  continue working with them to ensure information is current and useful.
A further coordination mechanism is a governance system that defines  roles and responsibilities for agencies contributing to the Manufacturing  USA program. According to Commerce, the process for developing the  October 2015 governance system document for the Manufacturing USA  network was initiated by the National Science and Technology Council’s  Subcommittee on Advanced Manufacturing, and represented a joint effort  between Commerce, DOD, and DOE. The governance system identifies  the Manufacturing USA network functions and subfunctions for which  agencies that sponsor Manufacturing USA institutes, as well as those that  do not sponsor institutes (non-sponsoring agencies), are responsible,  accountable, informed, and consulted. For example, under the  governance system, as part of the facilitating intra-network collaboration  function, AMNPO and the agencies that sponsor institutes are  responsible for providing situational awareness to individual institutes  regarding key contextual landscape issues, such as industrial  developments. Alternatively, as part of the function to sustain, strengthen,  and grow the network, AMNPO and the agencies that sponsor institutes  are responsible for identifying and helping to establish long-term  nonfinancial support mechanisms for the program, which the governance  document notes should provide valuable nonfinancial support to help  institutes succeed and thrive. Non-sponsoring agencies are responsible  for one general function: promoting advanced manufacturing to a variety  of external stakeholders such as Congress to raise awareness about the  Manufacturing USA institutes.
section_title: The Process for Developing the Governance System Did Not Include All Relevant Non-sponsoring Agencies or Ensure That Their Roles and Responsibilities Are Fully Identified
paragraphs: Although the governance system was developed by an interagency team,  the process used to develop the governance system did not ensure that  all relevant non-sponsoring agencies were included or that their roles and  responsibilities for contributing to the Manufacturing USA program were  fully identified.
Specifically, the process used to develop the governance system did not  include working with all relevant agencies that could contribute to the  Manufacturing USA program. As described above, agencies that  contributed to the development of the governance system included  Commerce, DOD, and DOE. An AMNPO official told us that non- sponsoring agencies had an opportunity to comment on the governance  system but were not involved in its development. According to  Commerce, only the institute-sponsoring agencies were involved in  developing the governance system because the governance document  was chartered by the National Science and Technology Council as a  reference for how Manufacturing USA institutes are to be involved in the  network. Thus, non-sponsoring agencies, such as DOL the National  Science Foundation, the National Aeronautics and Space Administration,  and the Department of Homeland Security, among others, were not  involved in developing the governance system despite having missions  that contribute to or are affected by advanced manufacturing. For  example, DOL’s mission is to foster, promote, and develop the welfare of  the wage earners, job seekers, and retirees of the United States; improve  working conditions; advance opportunities for profitable employment; and  assure work-related benefits and rights. In support of this mission, DOL is  a member of the National Science and Technology Council’s  Subcommittee on Advanced Manufacturing, according to the  subcommittee’s charter, and DOL has created resources to assist with  the development of programs that support secondary to post-secondary  career pathways related to advanced manufacturing. The governance  system itself does not specify which non-sponsoring agencies are  responsible for contributing to the Manufacturing USA program; rather, it  broadly identifies non-sponsoring agencies as agencies not acting in a  lead funding role and that are providing other types of support to an  institute in the network.
Furthermore, in developing the governance system, the process  Commerce, DOD, and DOE used did not include fully identifying how  non-sponsoring agencies could contribute to the Manufacturing USA  program. For example, the governance system does not specify any  responsibility for non-sponsoring agencies to provide information or  expertise related to their activities to the program. Rather, where the  governance system indicates a role for non-sponsoring agencies, it  generally indicates that such agencies are to be informed and  consulted. However, some non-sponsoring agencies may be  implementing programs or other activities that could contribute to the  Manufacturing USA program. For example, DOL administers workforce  development programs that are carried out by state agencies and local  workforce development boards. According to DOL officials, state  workforce development boards could work with Manufacturing USA  institutes to develop sector partnership strategies, which help employers  in an industry address shared goals and hiring needs. In addition, DOL  administers discretionary grant programs, which can help increase the  number of skilled workers in advanced manufacturing. Financial  assistance provided under these discretionary grant programs is being  used to support some Manufacturing USA institutes’ education and  workforce development activities, including:
The Trade Adjustment Assistance Community College and Career  Training grant program, which provided nearly $2 billion to strengthen  manufacturing programs at community colleges; and
The American Apprenticeship Initiative grant program, which will  provide $175 million to train and hire 34,000 new apprentices in  advanced manufacturing over the next five years.
Information or expertise related to other agencies’ programs, such as  these, if provided to AMNPO, could help it support the institutes with  navigating opportunities for federal financial assistance.
Our work has shown that collaborative mechanisms, such as the  Manufacturing USA program’s governance system, benefit from certain  key features, including the clarity of roles and responsibilities and  ensuring that the relevant participants are included in the collaborative  effort. Specifically, key collaboration practices call for clarifying the roles  and responsibilities of all participating agencies and determining whether  all relevant participants have been included. By agreeing on and clearly  defining the roles and responsibilities of their members as well as  documenting decisions, such as in a memorandum of understanding,  collaborating agencies can clarify which agency will do what, organize  their joint and individual efforts, and facilitate decision making. Moreover,  the RAMI Act identifies several AMNPO functions pertaining to  coordination, such as establishing procedures, processes, and criteria as  may be necessary and appropriate to maximize cooperation and  coordinate the activities of the Manufacturing USA program with the  programs and activities of other federal departments and agencies whose  missions contribute to or are affected by advanced manufacturing.
An AMNPO official acknowledged that the governance system will require  revision as the network evolves. The official said that in revising the  system, AMNPO will work with sponsoring agencies to further define roles  and responsibilities. However, the official said that other agencies  participate in the program as it aligns with their respective missions and  AMNPO cannot prescribe functions and subfunctions for other federal  agencies participating in the Manufacturing USA network. However,  without ensuring that all relevant agencies have been included in the  process of developing the governance system and that non-sponsoring  agencies’ roles and responsibilities have been fully identified, AMNPO  may miss opportunities to leverage and coordinate the efforts of non- sponsoring agencies in contributing to the Manufacturing USA program  consistent with key practices for interagency collaboration and effective  implementation of AMNPO’s functions under the RAMI Act.
section_title: Conclusions
paragraphs: In an effort to revitalize the U.S. manufacturing sector and increase U.S.  competitiveness in advanced manufacturing, Congress passed and the  President signed into law the RAMI Act, which requires the Secretary of  Commerce to establish a network of institutes for manufacturing  innovation, among other things. In establishing this Manufacturing USA  network, AMNPO, in collaboration with DOD and DOE, has taken steps to  establish institutes and measure progress toward the statutory purposes  of the program. AMNPO has also worked with other sponsoring and non- sponsoring agencies to coordinate agencies’ contributions to the network  through efforts that incorporate a number of the key practices we have  identified to enhance and sustain interagency collaboration.
However, while Commerce, DOD, and DOE worked together to develop a  governance system that outlines agencies’ roles and responsibilities, the  process for developing the system did not ensure that all relevant non- sponsoring agencies were included or that their roles and responsibilities  for contributing to the Manufacturing USA program were fully identified.  Working with other agencies to revise the governance system, including  ensuring that all relevant agencies are involved in the process to fully  identify non-sponsoring agencies’ roles and responsibilities, would  strengthen AMNPO’s efforts to leverage and coordinate agencies’  contributions to the Manufacturing USA program, consistent with key  practices for interagency collaboration, and would improve  implementation of its function to coordinate the program.
section_title: Recommendation for Executive Action
paragraphs: To enhance interagency collaboration in the Manufacturing USA program,  the Secretary of Commerce should direct the Director of NIST to work  with all non-sponsoring agencies whose missions contribute to or are  affected by advanced manufacturing to revise the Manufacturing USA  governance system to ensure the roles and responsibilities for how these  agencies could contribute to the Manufacturing USA program are fully  identified.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report for review and comment to Commerce,  DOD, Education, DOE, and DOL. We received the following comments:
Commerce provided written comments, which are reproduced in  appendix II. Commerce stated it agreed with the intent of our  recommendation and that it will convene an extended team for  revising the governance model. Commerce provided other comments  related to the role and responsibility of the Department, interagency  engagement, performance metrics, as well as other comments on the  recommendation which are described below.
DOD provided written comments, which are reproduced in appendix  III. DOD also stated that it agreed with the intent of our  recommendation and expanding the network governance model to  better define roles and responsibilities of non-sponsoring agencies.  DOD provided other comments in areas similar to Commerce’s  comments, as described below.
In an email from an audit analyst in its Office of the Chief Financial  Officer, DOE provided general comments, which we discuss below.
Commerce, DOD, and DOE provided technical comments, which we  incorporated as appropriate.
Officials from Education and DOL stated via email that they had no  comments on the report.
Commerce, DOD, and DOE provided several comments related to our  recommendation.
Commerce agreed with the intent of our recommendation but  requested revisions to the wording of the recommendation because  NIST has no authority to compel participation by DOL and Education  in the Manufacturing USA program or to define roles and  responsibilities for these agencies. Relatedly, Commerce and DOD  said that Commerce has no authority to coordinate other agencies in  their programs or any management responsibilities for institutes  sponsored by other agencies. We recognize that Commerce has no  authority to coordinate other agencies’ programs, to manage the  institutes that they sponsor, or to compel agency participation in the  program. We believe that our report correctly characterizes improved  coordination as a collaborative effort between Commerce and other  agencies. The focus of the recommendation in our draft report was  not for NIST to compel agency participation or unilaterally define their  roles and responsibilities but rather for NIST to work with non- sponsoring agencies in an interagency collaborative effort to identify  their roles and responsibilities. As our report noted, one of the  functions of AMNPO specified in the RAMI Act is to establish  procedures, processes and criteria as may be necessary and  appropriate to maximize cooperation and coordinate the program’s  activities with programs and activities of the other federal agencies  whose missions contribute to or are affected by advanced  manufacturing. In response to Commerce’s comments, we clarified  our recommendation to make clear that NIST should engage non- sponsoring agencies in an interagency collaboration to fully identify  their roles and responsibilities for contributing to the program.
Commerce also proposed an alternative recommendation,  commenting that it would be constructive for GAO to recommend that  the Secretary of Commerce connect with counterparts at DOL and  Education. While our report provides more detailed information on  several DOL programs and activities that could potentially contribute  to the Manufacturing USA program (relative to other non-sponsoring  agencies), we provide this information as illustrative examples.  Because we believe collaboration on Manufacturing USA network  governance roles and responsibilities should include all relevant non- sponsoring agencies, we did not adjust our recommendation to  specifically focus on DOL or Education.
Commerce also noted that the first generation of the Manufacturing  USA governance system was focused on how institutes would work  with the network and program, so the responsible entities that  developed it were limited to Commerce, DOD, and DOE. Commerce  further stated that our recommendation speaks to a next generation of  the Manufacturing USA governance with a broader scope. We  modified our report to more clearly reflect that Commerce, DOD, and  DOE were the agencies initially involved in developing the  governance system as chartered by the National Science and  Technology Council. However, we do not believe the scope of the  governance system as evidenced by documentation provided to us  during our review is substantially different than that envisioned by our  recommendation. For example, as we noted in our report, the  governance system assigns responsibility for certain subfunctions to  non-sponsoring agencies.
Finally, DOD stated it would work with the interagency team to assist  in reaching out to all agencies whose missions contribute to or are  affected by advanced manufacturing, and that it supports expanding  the network governance model. In its general comments, DOE agreed  that additional coordination with other governmental agencies would  be useful, and would further serve to reduce potential inefficiencies  and redundancies in federal investments.
Commerce and DOD also provided general comments in three areas (1)  roles and responsibilities of the Department of Commerce, (2)  interagency engagement, and (3) performance metrics.
Roles and responsibilities. Commerce and DOD provided  comments regarding the roles and responsibilities of Commerce that  were also related to the comments on our recommendation and are  discussed above. Commerce and DOD also emphasized the nature of  (1) the AMNPO as an interagency team hosted at NIST, and (2) the  Manufacturing USA institutes as public-private partnerships, owned  and managed by an industry-led consortium, with the sponsoring  federal agency providing a minority of funding. We made several edits  to reflect these points. Commerce and DOD also commented that  present non-federal funding is over a 2 to 1 match to federal financial  assistance. We believe we accurately present information on these  relative contributions in our report.
Interagency engagement. Commerce and DOD stated that all  agencies were invited to be part of the Manufacturing USA program,  and noted that a number of agencies, including some non-sponsoring  agencies, have been members of the interagency team. Commerce  and DOD also stated that GAO correctly stated that DOL was not a  principal agency involved with the program; although, Commerce  noted attempts had been made to reach out to DOL and both  agencies discussed providing information on DOL initiatives to the  Manufacturing USA institutes. We added information to our report to  acknowledge these efforts. Other comments Commerce provided  regarding interagency engagement through the Manufacturing USA  governance system are discussed above.
Performance metrics. Commerce and DOD stated that there is  agreement on the initial set of performance measures for the  Manufacturing USA program and that information on each of these  measures will be provided in the upcoming Manufacturing USA  annual report (for fiscal year 2016). In the draft report, we included  information on the extent to which all DOD and DOE institutes  planned to provide information on all of the initial performance  measures for the Manufacturing USA program. This information was  based on documentation provided to us during the course of our  review, which showed that not all DOD institutes planned to report  data on all of the initial performance measures. In response to the  comments received from Commerce and DOD, we followed up with  DOD. A DOD official responsible for overseeing DOD’s Manufacturing  USA institutes said via email that DOD recently tasked the institutes to  provide data in support of these measures for aggregation in the fiscal  year 2016 report. Therefore, we revised our report to remove  information related to a difference in the extent to which DOD and  DOE planned to have all their institutes report on all the initial  measures. In its comments on our draft report DOD also provided  information on other information DOD collects from its institutes  consistent with its mission, and emphasized that performance  measures included in our report are only initial metrics which could  evolve over time. We did not make any additional changes based on  these comments as we believe these concepts are already reflected  in our report.
Finally, Commerce and DOD commented on the title of our draft report,  which was Advanced Manufacturing: Commerce Needs to Strengthen  Collaboration with Other Agencies on Innovation Institutes, indicating a  preference for a more descriptive report title. In its general comments,  DOE stated that the draft title overstated the severity of the issue. We  believe that Commerce as well as other institute-sponsoring and non- sponsoring agencies have worked collaboratively in implementing the  Manufacturing USA program; however, our report identified an  opportunity to further strengthen this collaboration. We adjusted our  report title and believe that it appropriately highlights this area where  collaboration could be strengthened.
We are sending copies of this report to the appropriate congressional  committees; the Secretaries of Commerce, Defense, Education, Energy,  and Labor; and other interested parties. In addition, the report is available  at no charge on the GAO website at http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact John Neumann at (202) 512-3841 or neumannj@gao.gov, or  Andrew Sherrill at (202) 512-7215 or sherrilla@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. GAO staff who made key  contributions to this report are listed in appendix IV.
section_title: Appendix I: Information on Operating Manufacturing USA Institutes as of December 2016
paragraphs: This appendix provides additional information on each of the five  Department of Defense (DOD) and two Department of Energy (DOE)  Manufacturing USA institutes that were operating as of December 2016.  This information includes, among other things, background information on  the institute, such as its budget and technology focus, as well as data on  the number of members and membership benefits, rights, and fees. The  institutes are listed in chronological order based on the date of the  cooperative agreement with their sponsoring agency.
subsections: section_title: National Additive Manufacturing Innovation Institute (America Makes)
paragraphs: About: America Makes is the flagship Manufacturing USA institute. Its  mission is to accelerate the adoption of additive manufacturing  technologies, also known as three-dimensional, or 3D, printing, in the  U.S. manufacturing sector.
Location: Youngstown, Ohio, facility opened in October 2012. A satellite  center is located at the University of Texas, El Paso.
First cooperative agreement: Signed August 2012. America Makes is  operated by the National Center for Defense Manufacturing and  Machining, a 501(c)(3) nonprofit organization that has a cooperative  agreement with the Air Force Research Laboratory.
Budget: America Makes was established as a pilot institute in August  2012 for the Manufacturing USA program with $30 million in planned  federal obligations and $40 million in pledged nonfederal cost-share. This  amount was later increased to a combined $110 million in planned federal  obligations ($55 million) and pledged nonfederal cost-share ($55 million).
Second cooperative agreement: Signed February 2016 between the  National Center for Defense Manufacturing and Machining and the Air  Force Research Laboratory for up to approximately a combined additional  $76 million in planned federal obligations ($50 million) and pledged  nonfederal cost-share ($26 million). As of June 2016, the second  cooperative agreement consists of one $9 million project.
Technology portfolio: About 60 projects, such as a project to streamline  the design and improve the materials for the production of customized  ankle-foot orthoses, and another project to accelerate the U.S. metal- casting industry with the adoption of America Makes technologies into  their operations.
Membership: About 175 members from industry, academia, government,  nonprofit organizations, and Hollings Manufacturing Extension  Partnership centers.
Summary of membership fees, benefits, and rights: America Makes  provides three membership levels with different fees, benefits, and rights  associated with each level. The three membership levels and their  associated fees are as follows:
Platinum (lead): $200,000 cash or in-kind annually, with the intention  of at least a 3-year commitment
Gold (full): $50,000 cash or in-kind annually, with the intention of at
Silver (supporting): $15,000 cash or in-kind annually, with the  intention of at least a 3-year commitment  Table 5 provides an overview of America Makes membership benefits  and rights. Specific terms and conditions are further detailed in the formal  membership agreement.
section_title: Digital Manufacturing and Design Innovation Institute (DMDII)
paragraphs: About: DMDII’s mission is to digitize American manufacturing. By  capturing data at every stage of the production process—and by  deploying specially designed software and other digital tools— manufacturers can efficiently share and revise their digital designs.
Location: Chicago, Illinois, facility opened in May 2015. Two chapters  were opened in Quad Cities and Rockford, Illinois.
Cooperative agreement: Signed February 2014. DMDII is managed by  UI LABS, a 501(c)(3) nonprofit organization that has a cooperative  agreement with the Army Contracting Command Redstone.
Budget: $176 million from federal and nonfederal financial assistance $70 million in planned federal obligations $106 million pledged nonfederal cost-share  Technology portfolio: About 50 projects, such as a project to use 3D  models to enhance product quality, reduce communication errors, and  reduce development times throughout the supply chain, and another  project to fill the gap between design and manufacturing with the  information necessary to enable services for machine motion control.
Membership: About 260 members from industry, academia, government,  nonprofit organizations, and Hollings Manufacturing Extension  Partnership centers.
Summary of membership fees, benefits, and rights: DMDII provides  three membership levels for industry, four membership levels for  academic and nonprofit organizations, and two government membership  levels with different fees, benefits, and rights associated with each level.  The industry, academic and nonprofit, and government membership  levels and their associated fees are as follows:
Tier 1: $400,000 cash annually and at least $3 million in  expenditures for additional projects with a 5-year commitment
Tier 2: $200,000 cash annually with a 5-year commitment
Tier 3: $500 cash annually with a 5-year commitment
Academic and nonprofit:
Tier 1: $5 million in expenditures with a 5-year commitment
Tier 2: $2 million in expenditures with a 5-year commitment
Tier 3: $1 million in expenditures with a 5-year commitment
Tier 4: $500 cash annually with a 5-year commitment
U.S. government: Obligations as specified in the cooperative
State and local government: At least $5 million in expenditures  Tables 6, 7, and 8 provide an overview of DMDII membership benefits  and rights for industry, academic and nonprofit, and government  members, respectively. Specific terms and conditions are further detailed  in the formal membership agreement.
section_title: Lightweight Innovations for Tomorrow (LIFT)
paragraphs: About: LIFT’s mission is to accelerate the development of new  manufacturing processes for products using lightweight metals, including  aluminum, magnesium, titanium, and advanced high-strength steel alloys.
Location: Detroit, Michigan, facility opened in January 2015. Satellite  locations are in Columbus, Ohio; Ann Arbor, Michigan; Worchester,  Massachusetts; and Golden, Colorado.
Cooperative agreement: Signed February 2014. LIFT is managed by the  American Lightweight Materials Manufacturing Innovation Institute, a  501(c)(3) nonprofit organization that has a cooperative agreement with  the Office of Naval Research.
Budget: $148 million from federal and nonfederal financial assistance $70 million in planned federal obligations $78 million in pledged nonfederal cost-share  Technology portfolio: About 30 projects, either initiated or under  development, in the areas of melt and powder processing,  thermomechanical processing, joining and assembly, coatings, and agile  tooling, such as developing and deploying thin wall ductile iron castings  for high-volume production. LIFT also sponsored a Purdue-designed  Indy 500 Grand Prix for high school students to engineer, build, test, and  market vehicles using lightweight metals.
Membership: About 90 members from industry, academia, government,  nonprofit organizations, and a Hollings Manufacturing Extension  Partnership center.
Summary of membership fees, benefits, and rights: LIFT provides  nine membership levels with different fees, benefits, and rights associated  with each level. The nine membership levels and their associated fees  are as follows:
Gold: $350,000 annually, of which at least $100,000 is provided in  cash, with the intention of at least a 5-year commitment
Silver: $150,000 annually, of which at least $50,000 is provided in  cash, with the intention of at least a 5-year commitment
Bronze: $25,000 cash annually, with the intention of at least a 5-year  In-Kind members (e.g., equipment, software, or engineering service  providers): $150,000 in-kind annually, with the intention of at least a 5- year commitment
Small manufacturers and start-ups: For small manufacturers with 251  to 500 employees, the fee is $5,000 cash annually, with the intention  of at least a 5-year commitment; for small manufacturers with 1 to 250  employees, the fee is $2,500 cash annually, with the intention of at  least a 5-year commitment; and for start-ups with fewer than 50  employees and less than 5 years in business, the fee is $1,000 cash  annually, with the intention of at least a 5-year commitment
Research partners: $10,000 cash annually, with the intention of at  least a 5-year commitment. The annual in-kind fee varies based on  the level of participation.
Trade association: Annual cash and in-kind fees vary based on the  level of participation, with the intention of at least a 5-year  commitment
Professional society: $50,000 in-kind annually, with the intention of at
Education and workforce: Annual cash and in-kind fees vary based on  the level of participation, with the intention of at least a 5-year  commitment  Table 9 provides an overview of LIFT membership benefits and rights  across the nine membership levels. Specific terms and conditions are  further detailed in the formal membership agreement.
section_title: The Next Generation Power Electronics Manufacturing Innovation Institute (PowerAmerica)
paragraphs: About: PowerAmerica’s mission is to develop advanced manufacturing  processes that will enable large-scale production of wide bandgap  semiconductors, which allow power electronic components to be smaller,  faster, and more efficient than silicon-based devices.
Location: Raleigh, North Carolina, facility opened in January 2015.
Cooperative agreement: Signed December 2014. PowerAmerica is led  by North Carolina State University, which has a cooperative agreement  with DOE’s Office of Energy Efficiency and Renewable Energy.
Budget: $140 million from federal and nonfederal financial assistance $70 million in planned federal obligations $70 million in pledged nonfederal cost-share  Technology portfolio: About 30 projects, such as a project developing a  medium voltage fast charger for electric vehicles. Another project  involves working with an existing silicon wafer fabrication facility to update  its capabilities to allow for the production of wide bandgap semiconductor  devices—specifically silicon carbide wafers. This affords PowerAmerica  members an opportunity to accelerate their research by producing and  testing wide bandgap devices at the facility.
Membership: About 30 members from industry, academia, government,  and a Hollings Manufacturing Extension Partnership center.
Summary of membership fees, benefits, and rights: PowerAmerica  provides seven membership levels with different fees, benefits, and rights  associated with each level. The seven membership levels and their  associated fees are as follows:
Full sustaining: $500,000 annually ($250,000 of which may be met by  in-kind contributions), with a 3-year commitment. In general, the in- kind contribution should be made within the membership year;  however, the member may defer 25 percent of the in-kind contribution  for up to 6 months into the following membership year.
Full member: $100,000 cash annually with a 3-year commitment
Affiliate member: $50,000 cash annually with a 3-year commitment
Startup member: $10,000 cash annually with a 3-year commitment
Academic member: $10,000 cash annually with a 3-year commitment
Federal lab: No fee, but involves a 3-year commitment
Associate member: No fee, but involves a 3-year commitment  Table 10 provides an overview of PowerAmerica membership benefits  and rights across the seven membership levels. Specific terms and  conditions are further detailed in the formal membership agreement.
subsections: section_title: Full sustaining
paragraphs: Third  option for  exclusive  commercial  license  Members vote on projects that have gone through the PowerAmerica call for projects process and  that can be funded by industry membership fees.  Enhancement projects are projects funded by industry that are beyond the core funding of the grant.  They are nonproprietary so that all of the members can benefit from the research (as opposed to  other industry-funded projects performed using standard research agreements that have full  overhead and in which the company would own intellectual property generated).
section_title: Institute for Advanced Composites Manufacturing Innovation (IACMI)
paragraphs: About: IACMI’s mission is to accelerate the development and adoption of  cutting-edge manufacturing technologies for low-cost, energy-efficient  manufacturing of advanced polymer composites for vehicles, wind  turbines, compressed gas storage, and other applications.
Location: Oak Ridge, Tennessee, facility opened in June 2015.  Geographic extensions are located in Dayton, Ohio (compressed gas  storage); Golden, Colorado (wind turbines); West Lafayette, Indiana  (design and simulation); and Detroit, Michigan (vehicles). IACMI has also  signed memorandums of understanding with the Carbon Recycling  Technology Center in Port Angeles, Washington, and the Composite  Prototyping Center in Plainview, New York.
Cooperative agreement: Signed June 2015. IACMI is managed by the  Collaborative Composite Solutions Corporation, a 501(c)(3) nonprofit  organization that has a cooperative agreement with DOE’s Office of  Energy Efficiency and Renewable Energy.
Budget: $250 million from federal and nonfederal financial assistance $70 million in planned federal financial obligations $180 million in pledged nonfederal cost-share  Technology portfolio: About three projects, such as a project optimizing  carbon fiber production to enable high-volume manufacturing of  lightweight automotive components, and another project developing  thermoplastic material to lower production costs and improve recyclability  of wind turbine blades.
Membership: About 140 members from industry, academia, government,  nonprofit organizations, and a Hollings Manufacturing Extension  Partnership center.
Summary of membership fees, benefits, and rights: IACMI provides  four membership levels with different fees, benefits, and rights associated  with each level. The four membership levels and their associated fees are  as follows:
Charter: $5 million provided over a period of up to 5 years (at least 50  percent cash and $100,000 overhead annually)
Premium: $1 million provided over a period of up to 5 years (at least  50 percent cash and $20,000 overhead annually)
Resource: $5,000 cash annually for industry members with 500 or  fewer employees, government agencies, and educational institutions;  $10,000 cash annually for industry members with over 500  employees; as well as a resource contribution and a 1-year  commitment
Consortium: $5,000 cash annually for industry members with 500 or  fewer employees, government agencies, and educational institutions;  and $10,000 cash annually for industry members with over 500  employees; involves a 1-year commitment  Table 11 provides an overview of IACMI membership benefits and rights.  Specific terms and conditions are further detailed in the formal  membership agreement.
subsections: section_title: Additional benefits for Premium, Resource, and Consortium members Enterprise-wide proprietary projects
section_title: American Institute for Manufacturing Integrated Photonics (AIM Photonics)
paragraphs: About: AIM Photonics’ mission is to advance integrated photonic circuit  manufacturing technology development while simultaneously providing  access to state-of-the-art fabrication, packaging, and testing capabilities  for small-to-medium enterprises, academia, and the government.
Location: Albany, New York, facility operations began in July 2015.  Institute officials said the Rochester, New York, facility will be the location  for the test, assembly, and packaging operations. Also, AIM Photonics  has satellite hubs at the University of Rochester, Rochester Institute of  Technology, State University of New York Polytechnic Institute,  Massachusetts Institute of Technology, Columbia University, University of  Arizona, and University of California Santa Barbara.
Cooperative agreement: Signed July 2015. AIM Photonics is managed  by the Research Foundation of the State University of New York, a  501(c)(3) nonprofit organization that has a cooperative agreement with  the Air Force Research Laboratory.
Budget: $612 million federal and nonfederal financial assistance $110 million in planned federal obligations $503 million in pledged nonfederal cost-share  Technology portfolio: Institute officials said that the portfolio contains  about 10 projects, such as a project pertaining to 3D displays, and  another project through AIM Photonics’ AIM Academy to study industry  skills gap and employment needs.
Membership: About 30 members from industry, academia, and  government.
Summary of membership fees, benefits, and rights: AIM Photonics  provides four membership levels for industry and three membership  levels for academic and nonprofit organizations with different fees,  benefits, and rights associated with each level. The industry and  academic and nonprofit membership levels and their associated fees are  as follows:
Tier 1: $1 million cash annually (may include cash and in-kind with  a minimum of $100,000 cash through 2020), increasing to full  cash by 2021
Tier 2: $500,000 cash annually (may include cash and in-kind with  a minimum of $100,000 cash through 2020), increasing to full  cash by 2021
Tier 3: $100,000 or greater amount cash annually (may include in- kind through 2020)
Industry observer: $2,500 cash annually, with a 1-year  commitment
Academic and nonprofit:
Tier 1: In-kind and tangible and intangible contributions as  provided in the membership agreement (such as software  licenses, hardware, services, and expertise)
Tier 2: In-kind as provided in the membership agreement (such as  software licenses, hardware, or services and overhead costs)
Academic observer: No fee, with 1-year commitment  Tables 12 and 13 provide an overview of AIM Photonics’ membership  benefits and rights for industry and academic and nonprofit, respectively.  Specific terms and conditions are further detailed in the formal  membership agreements.
section_title: America’s Flexible Hybrid Electronics Manufacturing Institute (NextFlex)
paragraphs: About: NextFlex’s mission is to pioneer a new era of advanced flexible  hybrid electronics manufacturing in the United States. Flexible hybrid  electronics combine the ability to add electronics to materials that are part  of our everyday lives, to create lightweight, low-cost, flexible,  conformable, and stretchable smart products.
Location: San Jose, California, facility opened in August 2016.
Cooperative agreement: Signed August 2015. NextFlex is managed by  FlexTech Alliance, Inc., a 501(c)(6) nonprofit organization that has a  cooperative agreement with the Air Force Research Laboratory.
Budget: $171 million in federal and nonfederal financial assistance $75 million in planned federal obligations $96 million in pledged nonfederal cost-share  Technology portfolio: About 13 projects, such as a project to develop a  flexible smart wound dressing with oxygen release and sensing capability,  and a project to develop a flexible and stretchable monitoring sensor  network to improve aerospace fuel efficiency and flight safety.
Membership: About 45 members from industry, academia, government,  and nonprofit organizations.
Summary of membership fees, benefits, and rights: NextFlex provides  four membership levels for industry, three membership levels for  academic and nonprofit organizations, and a government membership  level with different fees, benefits, and rights associated with each level.  The industry, academic and nonprofit, and government membership  levels and their associated fees are as follows:
Tier 1: $150,000 cash and $250,000 in-kind annually, with the  intention of at least a 3-year commitment
Tier 2: $50,000 cash and $50,000 in-kind annually, with the  intention of at least a 3-year commitment
Tier 3: $10,000 cash annually and in-kind encouraged, with the  intention of at least a 3-year commitment
Observer: $2,500 cash annually and in-kind encouraged, with the  intention of at least a 3-year commitment
Academic and nonprofit:
Tier 1: $15,000 cash and $600,000 in-kind annually, with the  intention of at least a 3-year commitment
Tier 2: $7,500 cash and $300,000 in-kind annually, with the  intention of at least a 3-year commitment
Tier 3: $2,500 cash annually and in-kind encouraged, with the  intention of at least a 3-year commitment
No annual membership fee  In-kind contributions, which are not required, can be cash in  support of the institute  Tables 14, 15, and 16 provide an overview of NextFlex membership  benefits and rights for industry, academic and nonprofit, and government  members, respectively. Specific terms and conditions are further detailed  in the formal membership agreement.
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the contacts named above, key contributors to this report  were William MacBlane, Assistant Director; Christopher Murray, Assistant  Director; Erin Barry; Candace Carpenter; David Dornisch; Brian Egger;  Ellen Fried; Paige Gilbreath; Ruben Gzirian; Jamila Kennedy; Jonathan  Ludwigson; Rachel Pittenger; Dan C. Royer; Kathryn Smith; and Jeanette  Soares.