NOAA: Next Steps to Strengthen Its Acquisition Function
Published: 2006-06-07T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The National Oceanic and Atmospheric Administration (NOAA) accounts for about half of the Department of Commerce's (Commerce) acquisition spending, over $851 million in fiscal year 2005 alone. In recent years however, NOAA has experienced instances of poor contract management. GAO was asked to determine if NOAA is positioned to effectively carry out its acquisition function. Specifically, GAO assessed the extent to which NOAA has structured an acquisition organization that provides appropriate oversight; established policies and processes that promote, among other things, a knowledge-based acquisition process for development and production of complex systems; and planned and managed its contracting workforce to address future retirement challenges.
section_title: What GAO Found
paragraphs: NOAA has yet to structure all of its field acquisition activities under the direct oversight of its acquisition organization, increasing the risk that taxpayers are not getting the best value for their dollars. To improve oversight of its acquisition function, NOAA reorganized in 2005 by having all acquisition divisions report to the NOAA Director of Acquisition and Grants. However, NOAA's acquisition director still lacks direct line authority to oversee a National Weather Service field office that is responsible for one of the agency's contracts worth up to half a billion dollars. Also, without appropriate oversight from the acquisition organization, collateral duty field staff who are not career contracting officers have bought millions of dollars in goods and services. Our work has shown that a well-functioning acquisition organization has direct lines of oversight between the head of acquisition and various components to help enforce policies that enable efficient and effective contract spending. Although NOAA has established clear and consistent policies for some key areas, such as advance acquisition planning, the agency lacks a knowledge-based process for developing and producing complex systems--a situation that can increase the risk of cost increases and schedule delays. NOAA is missing key elements that promote successful outcomes for complex developmental systems because it must adhere to Commerce's acquisition policies that do not support a knowledge-based approach. Without such an approach, the multibillion-dollar satellite investment that NOAA is in the early stages of acquiring is at a higher risk of not meeting program requirements. NOAA has yet to focus on succession planning and management for its contracting workforce, although the agency is pursuing hiring flexibilities to recruit new contracting employees in anticipation of an impending wave of retirements. NOAA is facing a human capital challenge because of its aging contracting workforce and has reported that about 43 percent of contracting employees are now eligible to retire or will be eligible to retire by 2009. While senior acquisition managers are concerned with the loss of a high percentage of their contracting staff to retirement and other attrition, NOAA has not specifically addressed such contracting workforce challenges in its strategic human capital plan. Also, it is unclear whether human capital planning under way by Commerce will determine the gaps in numbers and skills in the contracting workforce. Unless the future retirement and workforce capacity challenges are strategically addressed, NOAA could soon lose a significant portion of its contracting knowledge base.

section_title: Letter
paragraphs: The National Oceanic and Atmospheric Administration (NOAA) is one of  the largest agencies within the Department of Commerce (Commerce) and  accounts for 45 percent of the department’s dollars spent on acquisition.   In fiscal year 2005 alone, NOAA’s acquisition organization contracted for  over $851 million in goods and services in support of the agency’s mission.   NOAA’s mission is to understand and predict changes in the Earth’s  environment and conserve and manage coastal and marine resources to  meet the nation’s economic, social, and environmental needs. To  accomplish this mission, NOAA acquires goods including ships, satellites,  aircraft, and high-performance computers as well as services such as  hydrographic surveys, fisheries analyses, and buoy maintenance. For the  first time, NOAA is managing an expected $6.2 billion acquisition  investment for developing a complex geostationary environmental  satellite.
NOAA relies more and more on contractor support to conduct its mission- critical work. However, in recent years, NOAA has experienced instances  of poor contract management that prompted the Commerce Inspector  General to designate management of Commerce’s acquisition processes as  one of the top 10 departmental management challenges. The Commerce  Inspector General identified a lack of effective oversight and management  controls over acquisition processes as a contributing factor to the poor  contract management.
In response to your request, we examined whether NOAA is positioned to  effectively carry out its acquisition function. Specifically, we assessed the  extent to which NOAA has (1) structured an acquisition organization that  provides appropriate oversight; (2) established clear and consistent  acquisition policies and processes that promote, among other things, a  knowledge-based acquisition process for development and production of  complex systems; and (3) planned and managed its contracting workforce  to address future retirement challenges. See appendix I for additional  details on scope and methodology.
To conduct this work, we assessed information from NOAA headquarters  and acquisition divisions and compared this information against what our  previous work has shown to be best acquisition practices. We reviewed  agency directives, memorandums, and other documentation; interviewed  agency officials; and analyzed agency systems and processes. We  conducted our review from August 2005 to April 2006 in accordance with  generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: The NOAA acquisition function is carried out by the headquarters  acquisition office, headed by the Director of Acquisition and Grants, as  well as 10 divisions, of which 6 are located at NOAA headquarters in Silver  Spring, Maryland, and 4 in regions throughout the country (see table 1). In  2005, NOAA employed 123 contracting and acquisition support staff in its  acquisition function across headquarters and regional locations.
NOAA’s acquisition function also provides procurement services to several  other Commerce components, including the International Trade  Administration, the Bureau of Economic Analysis, and the Economic  Development Administration. In fiscal year 2005, NOAA administered over  15,000 contract actions with total awards over $851 million.
In recent years, NOAA has experienced problems in acquiring goods and  services. For example, NOAA’s inadequate negotiations and oversight, and  lack of proper review, led the National Weather Service to pay for  defective equipment and execute a contract modification without  adequate negotiation for the transition power source for a high-resolution  Doppler weather radar system jointly designed, acquired, and operated by  Commerce, the Department of Defense, and the Department of  Transportation. As a result, contract costs increased by approximately  $4.5 million. Similarly, the Commerce Inspector General reported that  NOAA had insufficient management controls to ensure adequate oversight  of the procurement of over $65 million in new fisheries research vessels.  For example, NOAA did not have the information required to effectively  monitor the schedule and cost of the vessels and failed to implement an  effective award fee plan on a timely basis to encourage superior  performance by the contractor.
section_title: NOAA Lacks Oversight of Some Field Acquisition Activities
paragraphs: Although NOAA has taken some actions, the agency has yet to structure all  of its field acquisition activities under the appropriate oversight of its  acquisition organization, increasing the risk that taxpayers are not getting  best value for their dollars. After the Commerce Inspector General  criticized the lack of effective oversight and management controls over  acquisition processes, NOAA changed the structure of its acquisition  organization in 2005 to have acquisition divisions report to the Director of  Acquisition and Grants. Yet the acquisition director lacks direct line  authority over a field acquisition office in Mississippi. In addition, some  NOAA field operations rely on staff who are not career contracting officers  but whose collateral responsibilities are to procure goods and services  without appropriate oversight from the acquisition organization. Our work  has shown that a well-functioning acquisition organization has direct lines  of oversight between the head of acquisition and various components of  the organization to help enforce acquisition policies that enable the agency  to get the best value on goods and services.
subsections: section_title: Acquisition Office Reorganization Was Not Comprehensive
paragraphs: In some respects, NOAA’s reorganization has fallen short in providing  comprehensive authority to the NOAA acquisition director to provide  appropriate oversight to field acquisition activities. After the Commerce  Inspector General faulted NOAA’s contracting oversight, in an effort to  better ensure oversight and management of acquisition across the agency,  NOAA established an Acquisition and Grants Office in headquarters in  2005. The reorganization elevated the office, formerly part of the NOAA  Office of Finance and Administration, as a new staff office reporting  directly to the Deputy Under Secretary for Oceans and Atmosphere. Also,  the NOAA regional acquisition divisions—which previously were separate  entities reporting to the directors of the regional service centers in which  they were located—all now report directly to the Director of the  Acquisition and Grants Office as part of the reorganization. By elevating its  acquisition office to a level that reports directly to an Under Secretary and  having the divisions as direct reports to the acquisition director, NOAA has  positioned itself to appropriately oversee agencywide acquisition activities  and enforce acquisition policies.
However, the reorganization was not comprehensive because it did not  include the National Weather Service’s National Data Buoy Center  acquisition office at the Stennis Space Center, Mississippi. Specifically, the  Buoy Center does not report to the NOAA acquisition director (see fig. 1),  even though this acquisition office is responsible for administering  multiple contracts, including one of NOAA’s largest contracts, worth up to  $500 million. While contracting officers in the Buoy Center acquisition  office derive their contracting authority from headquarters, they report  through the Buoy Center chain of command, not to NOAA’s acquisition  director.
This reporting relationship was first established more than 20 years ago in  a Commerce appropriation act, which created the Buoy Center acquisition  office, and was stipulated in each subsequent appropriations act through  1988, but not since. During the recent reorganization, NOAA officials  believed these laws prohibited NOAA from changing the reporting  relationship to one where the Buoy Center acquisition office would report  directly to the NOAA Acquisition and Grants Office. However, an analysis  by our general counsel found no law requiring the Buoy Center acquisition  office to report to the Director of the Buoy Center or prohibiting NOAA  from making this change.  Commerce’s General Counsel agreed with our  assessment. Although we did not identify any inefficiencies resulting from  this current reporting relationship, we believe NOAA runs the risk of poor  outcomes of contracts.
section_title: NOAA Relies on Staff Who Have Collateral Contracting Responsibilities
paragraphs: Certain NOAA field offices rely on “collateral duty” contracting officers  who perform contracting duties in addition to their mission-related  responsibilities but who are not career contracting officers. In these  locations, collateral duty contracting officers have authority to make  individual purchases of up to $25,000 for such day-to-day needs as  provisions for ships. While NOAA relies on collateral duty contracting staff  for a relatively small share of total agency procurement spending, these  staff administered about 6,400 contract actions worth over $36 million  alone in fiscal year 2005.
NOAA’s acquisition director as well as senior field division contracting  officers expressed concern about how well they can monitor the actions  by staff with collateral duty contracting responsibilities to ensure that they  follow NOAA acquisition policies. For example, unlike with their own  contracting staff,  some NOAA regional acquisition divisions provide only  sporadic oversight over collateral duty contracting officers through  occasional audits, either performed in person or through contract file  samples sent by these staff for review. One regional acquisition division is  only able to audit work by staff with collateral contracting responsibilities  once every 2 to 3 years. NOAA acquisition managers attributed their  inability to provide regular oversight to limited time and resources, given  other responsibilities.
Our review of contracting weaknesses at a Department of Homeland  Security agency found challenges similar to NOAA’s with regard to  providing appropriate oversight to collateral duty contracting officers. In  view of such situations, we have found that an agency needs to explore  ways of making collateral duty contracting officers in field locations  accountable through the acquisition function for the contracting work.
section_title: Advances in Day-to- Day Acquisition Policies Not Matched in Policies for Complex Developmental Systems
paragraphs: Although NOAA has established clear and consistent acquisition policies  and processes for day-to-day acquisitions of goods and services, the  agency lacks a knowledge-based acquisition process for development and  production of complex systems. For example, NOAA has implemented  planning processes intended to improve opportunities to achieve savings  and reduce administrative burden in acquiring goods and services.  However, NOAA has experienced difficulties in acquiring more complex  developmental systems such as lacking sufficient management controls  when it acquired its new fisheries research vessels. NOAA is missing key  elements that promote successful outcomes because it must adhere to  Commerce’s acquisition policies that do not promote a knowledge-based  approach to acquiring complex developmental systems. Our work has  shown that a knowledge-based approach would enable decision makers to  be reasonably certain, at critical investment decision junctures in the  acquisition life cycle, that the system is likely to meet cost, schedule, and  performance targets.
subsections: section_title: New NOAA Policies and Processes Intended to Improve Acquisition Outcomes
paragraphs: To facilitate effective planning, award, and administration of contracts  that help to ensure the best value on goods and services, NOAA recently  implemented new policies and processes to guide the agency’s acquisition  activities. For example, to ensure consistency of the NOAA acquisition  process, a new acquisition handbook establishes standard operating  procedures, which the agency previously lacked, such as required reviews  and approvals, and quality assurance of acquisition activities within the  agency. To ensure that contracting employees are adhering to proper  procedures, NOAA instituted periodic acquisition management reviews.  Under these reviews NOAA’s acquisition director (or a senior contracting  officer appointed by the director) personally inspects a sample of  completed contract files from an acquisition division in order to identify  and remedy any systemic problem areas and identify best practices to  share with other divisions. In the last few years, reviews in several  divisions found a lack of proper contract file management, as required in  part 4.8 of the Federal Acquisition Regulation. The acquisition director  consequently instructed the division heads to take steps with their staff to  correct this problem.
Recognizing that NOAA lacked effective acquisition planning and that this  lack was hindering effective workload management, the Acquisition and  Grants Office in fiscal year 2005 implemented new advance acquisition  planning requirements. For example, all staff offices and programs must  develop annual acquisition plans that include acquisitions that are over  $100,000 or expected to result in new contracts. To support these new  planning requirements and provide real-time data, the acquisition office  deployed an online strategic acquisition management tool in fall 2005, in  which NOAA staff offices and programs enter their annual acquisition  plans for the upcoming fiscal year. Although the tool is new and still  requires occasional modifications, NOAA officials generally agreed that it  provides considerable benefits for the acquisition process to better plan  the agency’s use of resources and reduce the administrative burden  previously experienced with managing unscheduled contracting  requirements.
section_title: NOAA’s Acquisition Process for Development of Complex Systems Lacks a Knowledge-Based Approach
paragraphs: Because of the high dollar value and risk associated with acquiring  complex developmental systems, Commerce requires that its subordinate  organizations, including NOAA, adhere to departmentwide policy and  processes regarding such acquisitions. However, Commerce policies and  processes do not promote a knowledge-based approach for acquiring  these systems, increasing the risk that Commerce will make decisions to  commit the government to major investments without knowing that  NOAA’s systems can be developed at estimated costs, according to  planned schedules, and with expected performance. Commerce officials  acknowledged that their acquisition policy for developing major systems is  outdated and said they are revising it. While the current policy identifies  key decision points that a program must complete while acquiring a  complex developmental system, it still lacks key features of a knowledge- based acquisition framework—specific criteria and metrics—that can be  used to measure whether a program is ready to move through each key  decision point.
Our work has shown that leading developers in industry and government  organizations use a knowledge-based approach to deliver high-quality  products on time and within budget. An acquisition policy that requires  such a framework for complex developmental systems enables program  managers to be reasonably certain, at critical junctures, or knowledge  points, in the project’s acquisition life cycle, that their products will meet  established cost, schedule, and performance targets. Such an acquisition  policy also enables program managers to provide senior leaders with the  information necessary to make sound project investment decisions for  their agencies.  Highlights of an acquisition policy that incorporates best  practices for the development of complex systems include the equivalent  of    Knowledge point 1: Resources and needs match. This point occurs  when a match is made between the customer’s requirements and  the product developer’s available resources in terms of knowledge,  time, money, and capacity.
Knowledge point 2: Product design is stable. This point occurs when  a program determines that a product’s design will meet customer  requirements, as well as cost, schedule, and reliability targets.
Knowledge point 3: Production processes are mature and the design  is reliable. This is demonstrated when the product can be  manufactured within cost, schedule, and quality targets.  (See app. II for additional information on the knowledge-based acquisition  framework for complex developmental systems.)
Having a knowledge-based framework to guide acquisitions of complex  developmental systems is especially critical given that NOAA is in the  early acquisition stages of developing a geostationary environmental  satellite at a projected investment cost of about $6.2 billion. According to  a Commerce official, in the past NOAA relied on agencies outside of  Commerce to acquire such systems, but experienced cost and schedule  problems. To maintain management control over contractor performance,  NOAA chose to manage the acquisition of its new satellite—the  Geostationary Operational Environmental Satellite (GOES-R), which the  agency plans to launch in 2012.
Amplifying the need for clear guidance, NOAA officials told us the agency  is “feeling its way” through the GOES-R acquisition by relying on existing  Commerce and NOAA policies, but also learning from lessons drawn from  previous satellite acquisitions at other agencies. According to senior  officials directly involved, NOAA was forced to prepare for the most  recent key decision point in 2005—authorizing program definition and risk  reduction activities in the GOES-R acquisition—with vague guidance from  Commerce over what specific criteria and metrics were required for the  program to pass through the key decision point.
In order to be ready for the next key decision point, NOAA officials said  that clearer guidance from Commerce is needed on the information  required. The next key decision point is scheduled for fiscal year 2007 and  will commit the agency to development and production of the GOES-R  series. Commerce acquisition management officials told us they are  briefing NOAA officials and developing a plan for necessary  documentation on the requirements for the project’s next key decision  point. It remains to be seen whether these steps will provide sufficient  clarity not only for what NOAA is expected to do to prepare for the  remaining key decision points but also sufficient knowledge to make a  well-informed decision on proceeding with the GOES-R acquisition.
section_title: NOAA Has Taken Limited Actions to Address Contracting Workforce Retirement Challenges
paragraphs: NOAA is facing a human capital challenge because of its aging contracting  workforce, and senior acquisition managers are concerned about the loss  of a high percentage of their contracting staff to retirement and other  attrition. However, NOAA has yet to focus on succession planning and  management for its contracting workforce, even though the agency has  approval to use flexible direct hire authority in anticipation of an  impending wave of retirements.
NOAA relies on contract specialists and procurement analysts to carry out  its acquisition function. However, NOAA’s acquisition director is worried  about the near-term retirements of experienced contracting employees.  Our analysis of Office of Personnel Management’s (OPM) personnel data  for NOAA confirms there is cause for this sense of urgency, given that  almost 52 percent of the agency’s contract specialists and procurement  analysts are more than 49 years old and that 66 percent had at least 20  years of service (see fig. 2). Moreover, NOAA reported in January 2006 that  about 43 percent of its contracting employees are already eligible to retire  or will become eligible to retire between now and fiscal year 2009. Given  that NOAA has historically found that 50 percent of employees who are  eligible to retire do so within 3 years, it could lose a significant portion of  its contracting workforce within the next few years.
In addition to being concerned about retirements, senior acquisition  managers are very concerned about the significant acceleration in  contracting staff turnover as well as what they view as “fierce”  competition for federal contract specialists that prompts many to accept  jobs outside of NOAA. For example, NOAA reported in January 2006 that  over 12 percent of headquarters contracting positions are vacant.  Moreover, NOAA’s acquisition organization is concerned that the risk of  continuing losses is high, given that governmentwide, Commerce had the  highest reported agency attrition rate of 19 percent for contracting  employees as recently as fiscal year 2004. Our analysis of OPM’s fiscal year  2005 personnel data affirms their concern, since about half of the  contracting specialists and procurement analysts that separated that year  from NOAA transferred to jobs in other agencies (6 of the 13 employee  separations).
In 2006 NOAA’s workforce management office completed a strategic  human capital management plan intended to respond to recognized  challenges and to help ensure the agency has the scientific, technical, and  administrative expertise necessary to accomplish NOAA’s mission.  However, the plan does not specifically address the potential gaps in  NOAA’s contracting workforce even though NOAA is responsible for  executing up to 25 percent of the Department of Commerce’s budget on  acquiring goods and services. Furthermore, because contracting expertise  was not recognized as mission-critical, the acquisition director said she  was not asked and did not provide input about the challenges to the  workforce management office. Involving senior management, employees,  and other stakeholders in strategic human capital planning is an important  step toward its successful implementation.
In addition, Commerce has taken some initial steps to address this  retirement challenge in its contracting workforce. The Secretary of  Commerce’s Office of Acquisition Management and Financial Assistance  hired a contractor to develop a human capital plan for the department’s  contracting workforce. However, it is too early to determine if this  Commerce-wide plan will determine the gaps in numbers and skills in the  contracting workforce to achieve mission-critical work and develop  strategies to address these gaps. Until this assessment of the contracting  workforce is completed at the departmental level, NOAA will be hard  pressed to make its own assessment. Also recognizing a critical staffing  shortage, in January 2006, Commerce approved NOAA’s use of flexible  direct hire authority to recruit qualified individuals. Civilian agency heads  are authorized to determine if a shortage exists in certain federal  acquisition positions so that persons with high qualifications can be  recruited and directly hired. Direct hire authority allows an agency the  flexibility to appoint individuals to positions without adherence to certain  competitive examination requirements. According to a NOAA official, to  date, NOAA has not implemented this authority.
The prospect of many retirements in experienced staff make NOAA’s need  for contracting workforce planning more significant, and NOAA stands to  benefit greatly by planning strategically for its contracting workforce. Our  work has shown that to recruit, develop, and retain the right number of  people with the right skills to support mission needs, leading organizations  go beyond a succession planning approach that focuses on simply  replacing individuals and engage in broad, integrated succession planning  and management efforts that focus on strengthening both current and  future organizational capacity. A key part of succession planning and  management is to identify the talent required to achieve organizational  goals. To that end, effective human capital planning involves   (1) identifying the gaps in skills and competencies that need to be filled to  achieve current and future mission-critical work, (2) developing strategies  that are tailored to address these gaps, and (3) monitoring and evaluating  the progress toward closing the gaps, and adjusting strategies accordingly.  Unless the future retirement and workforce capacity challenges are  strategically addressed, NOAA could soon lose a significant portion of its  contracting knowledge base.
section_title: Conclusions
paragraphs: Effectively and efficiently acquiring goods and services will continue to  play a major role in enabling NOAA to accomplish its mission. Therefore,  NOAA’s success depends greatly on how well it manages its acquisition  function. While NOAA has taken initial steps intended to put in place an  efficient, effective, and accountable acquisition function, more can be  done. NOAA can improve oversight of its field acquisition operations by  requiring direct reporting of all of its acquisition offices and developing an  oversight process for collateral duty contracting officers; adopting a  knowledge-based acquisition process to help ensure that complex  developmental systems meet cost, schedule, and performance targets; and  addressing the imminent retirements of its contracting workforce to  ensure it has a solid knowledge base to carry out its acquisition function.  These next steps can facilitate achieving successful acquisition outcomes  and enable NOAA to responsibly and prudently manage its investments to  help meet the nation’s economic, social, and environmental needs.
section_title: Recommendations for Executive Action
paragraphs: To close the gaps between NOAA’s current acquisition process and best  practices for the development of complex systems, we recommend that  the Secretary of Commerce take the following actions:  Incorporate a knowledge-based framework in the revision of  Commerce’s policy for acquiring complex developmental systems. This  framework should identify requirements for major system projects to  attain specific product knowledge at significant stages in the  acquisition life cycle. Demonstration of this knowledge should be used  as a basis for decision making by departmental leadership at the  following key points:   Before projects are approved to move from concept and technology  development to start the acquisition program, the policy should  require that customer requirements and product developers’  resources match, as indicated by achieving a high level of  technology maturity.
Before projects are approved to move from integration to  demonstration, the policy should require that the product design is  stable and performs as expected.
Before projects are approved to move from demonstration into  production, the policy should require that its design is reliable and  can be manufactured within cost, schedule, and quality targets.
To provide appropriate oversight of NOAA’s field acquisition operations,  we recommend that the Under Secretary of Commerce for Oceans and  Atmosphere take the following two actions:  take steps to ensure that NOAA’s Director of Acquisition and Grants  has direct authority over all acquisition entities within the agency by  realigning the National Data Buoy Center’s Acquisition Office to report  directly to NOAA’s acquisition director and    provide for regular monitoring of collateral duty contracting officers in  field offices to help ensure they are accountable to senior acquisition  officials for their contracting work.
To address the future retirement challenges of NOAA’s contracting  workforce, we recommend that the Under Secretary of Commerce for  Oceans and Atmosphere take the following action:    conduct a data-driven assessment of NOAA’s contracting workforce to  (1) identify skill gaps, develop strategies to address these gaps,  evaluate progress toward closing these gaps, and adjust strategies  accordingly and (2) involve relevant stakeholders in the development  of this assessment.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report signed by the Deputy  Secretary of Commerce, the department agreed with our findings and  recommendations.  Commerce stated that the report presented an  accurate assessment of the major issues NOAA faces and described  current or planned actions to address the recommendations. The  department’s comments are reprinted in appendix III.
As we agreed with your office, unless you publicly announce the contents  of this report earlier, we plan no further distribution of it until 30 days  from the date of this letter. We will then send copies to other interested  congressional committees, the Secretary of Commerce, and the Under  Secretary for Oceans and Atmosphere. We will make copies available to  others on request. In addition, the report will be available at no charge on  GAO’s Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-4841 or shamesl@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix IV.
section_title: Appendix I: Scope and Methodology
paragraphs: To determine if the National Oceanic and Atmospheric Administration  (NOAA) has structured an acquisition organization that provides adequate  oversight, we interviewed NOAA’s Director of Acquisition and Grants, the  acquisition division heads, and Commerce acquisition management  executives regarding NOAA’s acquisition function. We reviewed pertinent  NOAA and Commerce organizational, management, and policy documents  pertaining to the oversight of its acquisition activities. We also performed a  legislative analysis of the establishment of the National Data Buoy Center  in Mississippi to determine if there was a legal basis for not having its  acquisition office report directly to NOAA’s Director of Acquisition and  Grants. In addition, we reviewed reports from Commerce’s Office of the  Inspector General to obtain an understanding of relevant Commerce and  NOAA acquisition management challenges.
To determine if NOAA has established clear and consistent acquisition  policies and processes that promote, among other things, a knowledge- based acquisition approach for development and production of complex  systems, we reviewed acquisition policies, processes, and guidelines. We  interviewed NOAA’s acquisition and grants director and acquisition  division heads and reviewed information regarding new acquisition  policies, handbook guidance, quality assurance, and planning processes.  To determine if NOAA has established policies and processes that promote  a knowledge-based review process for complex developmental systems  acquisitions, we compared and discussed with NOAA and Commerce  acquisition officials their policies for major acquisitions to our knowledge- based acquisition approach. Specifically, we compared and contrasted  Commerce and NOAA’s major system acquisition review process with  criteria contained in GAO best practices work on systems acquisition and  space system acquisitions.
Finally, to determine the extent to which NOAA has planned and managed  its contracting workforce to address future retirement challenges, we  interviewed NOAA and Commerce officials responsible for contracting  workforce and strategic human capital planning. We also analyzed and  compared NOAA’s contracting workforce and strategic human capital  planning efforts with principles of effective human capital management. In  addition to obtaining Commerce and NOAA officials’ views and concerns  regarding future human capital challenges, we analyzed data from Office  of Personnel Management’s (OPM) Central Personnel Data File (CPDF),  which is its database of federal civilian employees. Specifically, to assess  the potential for retirements in the next few years in NOAA’s contracting  workforce, the CPDF data we used reflected information on NOAA  employees—as of September 30, 2005—working in the general series   (GS) 1102 contracting series of contract specialists and procurement  analysts. We limited our analysis to focus on contract specialists and  procurement analysts because NOAA’s Acquisition and Grants Office  workforce mainly consists of this occupational category. We analyzed  CPDF’s separations and demographic data (on the ages and years of  service) and NOAA information for contract specialists and procurement  analysts in order to identify the extent to which NOAA’s experienced  contracting workforce is eligible for and thus has the potential for retiring  in the near future. We did not independently verify the CPDF data on  NOAA’s contracting series employees for the year we reviewed. However,  we previously reported that data from the CPDF for key variables in  another study—occupation, agency/subagency, birth date, and service  computation date—were 99 percent accurate.
We conducted our review from August 2005 through April 2006 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: Knowledge-Based Acquisition Framework for Complex Developmental Systems
paragraphs: The federal government spends billions annually to research, develop, and  produce large custom projects, such as satellites, space projects, weapon  systems, air traffic control systems, and information technology systems.  However, undesirable acquisition outcomes often occur because agency  officials proceed further into development or production without  obtaining sufficient knowledge that the product will be able to meet  established cost, schedule, performance, and quality targets.
The risk of undesirable acquisition outcomes can be significantly reduced.  All product development acquisition projects, whether for a ship, airplane,  missile, or satellite, go through a complex process of building knowledge.  Ultimately, across the acquisition life cycle, this process brings together  and integrates the technology, components, and subsystems needed for  the product to work and be reliably manufactured. Over the past decade,  we have undertaken a body of best practices work that draws upon  lessons learned from product development practices of leading  commercial firms, which expect their program managers to deliver high- quality products on time and within budget.
Our work has identified three discrete points in the process for complex  developmental systems at which obtaining certain levels of knowledge  promote successful project outcomes. These knowledge points, depicted  in figure 3, are defined in the following manner.
Knowledge point 1: Resources and needs match. This point occurs  when a sound business case is made for the system—that is, a match is  made between customer requirements and the product developer’s  available resources in terms of knowledge, time, money, and capacity.  Achieving a high level of technology maturity at the start of system  development acquisition is an important indicator of whether this  match has been made such that the technologies needed to meet  essential product requirements have been demonstrated to work in  their intended environment.
Knowledge point 2: Product design is stable. This point occurs when  a program determines that a product’s design will meet customer  requirements and cost, schedule, and reliability targets. A best practice  is to achieve design stability at the system-level critical design review  usually held midway through development. Completion of at least   90 percent of engineering drawings at the system design review  provides tangible evidence that the design is stable.
Knowledge point 3: Production processes are mature and the design  is reliable. This is demonstrated when a product can be manufactured  within cost, schedule, and quality targets. A best practice is to ensure  that all key manufacturing processes are in statistical control—they are  repeatable, sustainable, and capable of consistently producing parts  within the product’s quality tolerances and standards—at the start of  production.
The attainment of each successive knowledge point builds on the  preceding one. While the knowledge itself builds continuously without  clear lines of demarcation, the attainment of knowledge points is  sequential. In other words, production maturity cannot be attained if the  critical technologies are not mature.
This knowledge-based approach is not just applicable to improving how  the Department of Defense manages the acquisition of weapon systems.  As we have recently reported, utilizing a knowledge-based approach to  manage complex developmental system acquisitions can likewise help  civilian agencies reduce risk by ensuring high levels of knowledge are  attained on individual acquisitions at key decision points, thereby enabling  senior leaders to make informed investment decisions. A knowledge- based approach can also provide inexperienced program managers the  necessary guidance to implement good management practices and ensure  that the correct data are on hand for decision makers. Further, the  methodology used by successful program managers to acquire complex  developmental systems does not differ based upon each individual system  being acquired. Notwithstanding the type of system being acquired, these  managers follow a regimented process that uses common and consistent  criteria for decision making.
section_title: Appendix III: Comments from the Department of Commerce
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
paragraphs: Lisa Shames, (202) 512-4841 or shamesl@gao.gov.
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Carolyn Kirby, Assistant  Director; Ian Jefferies; Jose Ramos; Sylvia Schatz; Bob Swierczek; and  Greg Wilmoth made key contributions to this report.