Space Acquisitions: Improvements Needed in Space Systems Acquisitions and Keys to Achieving Them
Published: 2006-04-06T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DOD's space system acquisitions have experienced problems over the past several decades that have driven up costs by hundreds of millions, even billions of dollars, stretched schedules by years, and increased performance risks. GAO was asked to testify on its findings on space acquisition problems and steps needed to improve outcomes.
section_title: What GAO Found
paragraphs: DOD's space acquisition programs continue to face substantial cost and schedule overruns. At times, cost growth has come close to or exceeded 100-percent, causing DOD to nearly double its investment in face of technical and other problems without realizing a better return on its investment. Along with the cost increases, many programs are experiencing significant schedule delays--as much as 6 years--postponing delivery of promised capabilities to the warfighter. Outcomes have been so disappointing in some cases that DOD has had to go back to the drawing board to consider new ways to achieve the same capability. These problems are having a dramatic effect on DOD's space investment portfolio. Over the next 5 years, there will be about $12 billion less dollars available for new systems as well as for the discovery of promising new technologies because of cost growth. And while DOD is pushing to start new, highly ambitious programs such as the Transformational Satellite and Space Radar, broader analyses of the nation's fiscal future indicate that spending for weapon systems may need to be reduced, rather than increased, to address growing deficits. GAO has identified a number of causes behind these problems, but several stand out. First, DOD starts more space and weapons programs than it can afford, which pressures programs to under estimate costs and over promise capabilities. Second, DOD starts its space programs too early, that is, before it is sure the capabilities it is pursuing can be achieved within available resources and time constraints. DOD has also allowed new requirements to be added well into the acquisition phase. DOD has appointed a new leadership to oversee space acquisitions who have committed to adopting practices GAO has recommended for improving outcomes. These include delegating the maturation of technologies to the S&T community; adopting an evolutionary development approach in which new systems would be developed in a series of discrete increments, or blocks; fund S&T appropriately so that significant technology breakthroughs can be continually pursued; and improving collaboration on requirements. Adopting best practices for space acquisitions will not be an easy undertaking. DOD, as a whole, still operates in an environment that encourages competition for funding, and thus, behaviors that have been detrimental to meeting cost and schedule goals. Moreover, the changes being proposed will require significant shifts in thinking about how space systems should be developed and changes in incentives. By establishing investment priorities, embedding best practices in policy, and addressing capacity shortfalls, DOD can mitigate these challenges and better position programs for success.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the Department of Defense’s  (DOD) space acquisitions. Each year, DOD spends billions to acquire  space-based capabilities to support current military and other government  operations as well as to enable DOD to transform the way it collects and  disseminates information, gathers data on its adversaries, and attacks  targets. In fiscal year 2007 alone, DOD expects to spend almost $20 billion  dollars to develop and procure satellites and other space systems,  including nearly $7 billion on the major space systems. Despite its  growing investment in space, however, DOD’s space system acquisitions  have experienced problems over the past several decades that have driven  up costs by hundreds of millions, even billions of dollars, stretched  schedules by years, and increased performance risks. In some cases,  capabilities have not been delivered to the warfighter after decades of  development.
As a result of these problems, DOD is now contending with important  trade-off decisions such as whether to continue investing in long  beleaguered efforts or undertake more promising alternatives. At the same  time, leadership now recognizes the need to substantially change DOD’s  current space acquisition approach and the value of adopting practices  that will lay a better foundation for program execution. Within this  context, I will discuss our findings on space acquisition problems, recent  steps DOD has taken in an effort to address these problems, and the  changes that still need to occur if DOD is to break the cycle of acquisition  problems.
subsections: section_title: Space Acquisition Problems Persist
paragraphs: The majority of satellite programs we have reviewed over the past 2  decades experienced problems during their acquisition that drove up costs  and schedules and increased technical risks. Several programs were  restructured by DOD in the face of delays and cost growth. At times, cost  growth has come close to or exceeded 100-percent, causing DOD to nearly  double its investment in face of technical and other problems without  realizing a better return on its investment. Along with the cost increases,  many programs are experiencing significant schedule delays—as much as  6 years—postponing delivery of promised capabilities to the warfighter.
Outcomes have been so disappointing in some cases that DOD has had to  go back to the drawing board to consider new ways to achieve the same  capability. It is in such a position today, with its Space-based Infrared  System (SBIRS) High program and possibly its National Polar-orbiting  Operational Environmental Satellite System (NPOESS) program, both of  which have been mired in expanding cost and schedule setbacks.
More specifically, DOD’s investment in SBIRS High, a critical missile  warning system, has been pushed to over $10.5 billion from the initial   $4.1 billion estimate made over 9 years earlier. This 160-percent increase  in estimated costs triggered a fourth Nunn-McCurdy breach   (see 10 U.S.C. 2433), requiring a review by the Secretary of Defense and a  report to Congress, and resulted in the program being restructured for a  third time, in late 2005. With costs and timelines spiraling out of control,  DOD reduced the number of satellites it plans to procure—pushing the  average per unit procurement cost up to 224-percent above 2002 baseline  costs—and is now pursuing an alternative to SBIRS High while it  continues with the scaled back program.
Initial cost and schedule estimates for NPOESS—a new satellite  constellation intended to replace existing weather and environmental  monitoring satellites—have also proven unreliable. NPOESS is managed  by a tri-agency Integrated Program Office consisting of DOD, the National  Oceanic and Atmospheric Administration, and National Aeronautics and  Space Administration. In January 2006, the program reported a Nunn- McCurdy unit cost breach, at the 25-percent threshold, due to continuing  technical problems, including problems with the development of key  sensors. Specifically, in early 2005, DOD learned that a subcontractor  could not meet cost and schedule targets due to significant technical  issues on an imaging sensor known as the visible/infrared imager  radiometer suite (VIIRS) sensor—including problems with the  cryoradiator, excessive vibration of sensor parts, and errors in the sensor’s  solar calibration. These technical problems were further complicated by  subcontractor management problems. To address these issues, DOD  provided additional funds for VIIRS, capped development funding for  other critical technologies, and revised its schedule to keep the program  moving forward. We also reported that based on our own analysis of  contractor trends, the program will most likely overrun costs by   $1.4 billion. Given the challenges currently facing the program, the  scheduled first launch date slipped 17 months to September 2010.
Another recent example of problems is evident in the Advanced Extremely  High Frequency (AEHF) program. We reported in the past that this  program experienced cost increases due to requirements changes,  inadequate contract strategies, and funding shortfalls. We also reported  that DOD had to cut back its planned purchase of satellites from five to  three as a result. The outcome has been an 84-percent unit cost increase— each AEHF satellite is now estimated to cost about $2.1 billion. More  recently, we reported that scheduling delays and the late delivery of  cryptographic equipment have culminated into nearly a 3-year delay in the  launch of the first satellite and that the program still faces schedule risk  due to the continued concurrent development of two critical path items  managed and developed outside the program.
Acquisition problems have not been limited to the development of home- grown systems. DOD’s purchase of an ostensible commercial satellite for  the use of communications, the Wideband Gapfiller Satellite (WGS), is  experiencing about a 70-percent cost growth, due in part to the problems a  subcontractor was experiencing in assembling the satellites. Improperly  installed fasteners on the satellites’ subcomponents have resulted in  rework on the first satellite and extensive inspections of all three satellites  currently being fabricated. The cost for WGS has increased about   $746.3 million but DOD estimates that about $276.2 million of this amount  is largely due to cost growth associated with a production gap between  satellites three and four. The launch of the first satellite has now been  delayed for over 3 years and is currently scheduled for June 2007. The  delay will increase program costs and add at least 22 months to the time it  takes to obtain an initial operational capability from the system.
Figure 1 shows that, overall for fiscal years 2006 through 2011, estimated  costs for DOD’s major space acquisition programs have increased a total  of about $12.2 billion—or nearly 44-percent in total—above initial  estimates. Figure 2 breaks out this trend among key major space  acquisitions.
As both figures illustrate, cost increases have had a dramatic impact on  DOD’s overall space portfolio. To cover the added costs of poorly  performing programs, DOD has shifted scarce resources away from other  programs, creating a cascade of cost and schedule inefficiencies. For  example, to fund other space programs, DOD has had to push off the start  of a new version of the Global Positioning System (GPS), which has forced  costs to increase for the current version under development. Meanwhile,  DOD is also contending with cost increases within its Evolved Expendable  Launch Vehicle (EELV) program. These are largely due to misjudgments  about the extent to which DOD could rely on commercial demand to  leverage its investment. Nevertheless, the resulting $12.6 billion increase  has added pressures to make tradeoffs.
At the same time that DOD is juggling resources on existing programs, it is  undertaking two new efforts—the Transformational Satellite  Communications System (TSAT) program and Space Radar program— which are expected to be among the most ambitious, expensive, and  complex space systems ever. Moreover, DOD is relying heavily on their  planned capabilities to fundamentally enable DOD to transform how  military operations are conducted. In fact, many other weapon systems  will be interfaced with these satellites and highly dependent on them for  their own success. Together, these systems have been preliminarily  estimated to cost about $40 billion.  While DOD is planning to undertake  the new systems, broader analyses of the nation’s fiscal future indicate  that spending for weapon systems may need to be reduced, rather than  increased, to address growing deficits.
section_title: Underlying Causes of Acquisition Problems
paragraphs: Our reviews have identified a number of causes behind the problems just  described, but several consistently stand out. First, on a broad scale, DOD  starts more weapon programs than it can afford, creating a competition for  funding which encourages low cost estimating, optimistic scheduling, over  promising, suppressing bad news, and for space programs, forsaking the  opportunity to identify and assess potentially better alternatives. Programs  focus on advocacy at the expense of realism and sound management.  Invariably, with too many programs in its portfolio, DOD and even  Congress are forced to continually shift funds to and from programs— often undermining well-performing programs to pay for poorly   performing ones.
Second, DOD starts its space programs too early, that is, before it has  assurance that the capabilities it is pursuing can be achieved within  available resources and time constraints. This tendency is caused largely  by the funding process, since acquisition programs attract more dollars  than efforts concentrating solely on proving out technologies.  Nevertheless, when DOD chooses to extend technology invention into  acquisition, programs experience technical problems that have  reverberating effects and require large amounts of time and money to fix.  When programs have a large number of interdependencies, even minor  “glitches” can cause disruptions.
A companion problem for all weapon systems is that DOD allows new  requirements to be added well into the acquisition phase. Many times,  these significantly stretch the technology challenges (and consequently,  budgets) the program is already facing. This was particularly evident in  SBIRS High up until 2004. While experiences would caution DOD not to  pile on new requirements, customers often demand them fearing there  may not be another chance to get new capabilities since programs can  take a decade or longer to complete.
Third, space programs have historically attempted to satisfy all  requirements in a single step, regardless of the design challenge or the  maturity of the technologies to achieve the full capability. Increasingly,  DOD has preferred to make fewer, but heavier, larger, and complex  “Battlestar Galactica-like” satellites, that perform a multitude of missions  rather than larger constellations of smaller, less complex satellites that  gradually increase in sophistication. This has stretched technology  challenges beyond the capability of many potential contractors and vastly  increased the complexities related to software—a problem that affected  SBIRS High and AEHF, for example.
Our reviews have identified additional factors that contribute to space  acquisition problems, though less directly affecting cost and schedule  problems we have reported on. For example, consolidations within  defense supplier base for space programs have made it more difficult for  DOD to incorporate competition into acquisition strategies. Since 1985,  there were at least ten fully competent prime contractors competing for  the large programs and a number that could compete for subcontracts.  Arguably today, there are only two contractors that could handle DOD’s  most complex space programs. DOD has exacerbated this problem by not  seeking opportunities to restructure its acquisitions to maximize  competition, particularly for the small suppliers who have a high potential  to introduce novel solutions and innovations into space acquisitions. In the  1990s, DOD also structured contracts in a way that reduced oversight and  shifted key decisionmaking responsibility onto contractors. DOD later  found that this approach—known as Total System Performance  Responsibility, or TSPR—magnified problems related to requirements  creep and poor contractor performance.
Another factor contributing to problems is the diverse array of officials  and organizations involved with a space program, which has made it even  more difficult to pare back and control requirements. The Space Radar  system, for example, is expected to play a major role in transforming  military as well as intelligence-collecting operations and other critical  governmental functions, such as homeland security. As a result, its  constituency includes combatant commanders, all of the military services,  intelligence agencies, and the Department of Homeland Security. The  Global Positioning System not only serves the military, it provides critical  services to civilian users, the transportation sector, the information  technology sector, among many other industries.
In addition, short tenures for top leadership and program managers within  the Air Force and the Office of the Secretary of Defense has lessened the  sense of accountability for acquisition problems and further encouraged a  short-term view of success, according to officials we have interviewed.  Though still in a pre-acquisition phase, TSAT and Space Radar have  already had one program director each. The SBIRS High program,  meanwhile, has seen at least three program directors. At the highest levels  of leadership, for many years, DOD did not invest responsibilities for its  space activities in any one individual—leaving no one in charge of  establishing an integrated vision for space or of mediating between  competing demands. In 1994, it established such a position within the  Office of the Secretary of Defense, but dissolved this position in 1998. In  2002, DOD established a space leadership position within the Under  Secretary position in the Air Force, combined it with the directorship of  the National Reconnaissance Office in order to better integrate DOD and  intelligence space activities, and allowed the Under Secretary to have  milestone decision authority for major space systems acquisitions. After  the first Under Secretary of the Air Force in charge of space retired in  2005, DOD split these responsibilities and temporarily reclaimed milestone  decision authority for all major space programs. Changes in leadership and  reorganizations are common across DOD, but again, they make it more  difficult to enforce accountability and maintain the right levels of support  for acquisition programs.
Lastly, there are capacity shortfalls that have constrained DOD’s ability to  optimize and oversee its space programs. These include: shortages in the  pipeline of scientists and engineers, shortages of experts in systems and  software engineering, and uneven levels of experience among program  managers. Contractors are also facing workforce pressures similar to  those experienced by the government, that is, not enough technical  expertise to develop complex space systems. In addition, we have  reported that there is a lack of low-cost launch opportunities, which are  needed to increase the level of experimental testing in space.
section_title: DOD Has Expressed Its Commitment to Improve Its Approach to Space Acquisitions
paragraphs: DOD has recently expressed a commitment to improve its approach to  space acquisitions and embrace many of the recommendations we have  made in the past.
Our previous recommendations have been focused on providing a sound  foundation for program execution. Namely, we have recommended that  DOD separate technology discovery from acquisition, follow an  incremental path toward meeting user needs, match resources and  requirements at program start, and use quantifiable data and  demonstratable knowledge to make decisions to move to next phases. In  addition, we have called on DOD to develop an overall investment strategy  for space in order to help DOD rebalance its investments in space  acquisition programs as it continues to contend with cost increases from  its programs.
These recommendations are based on a body of work that we have  undertaken over the last several years that examines weapon acquisition  issues from a perspective that draws upon lessons learned from best  product development practices. Leading commercial firms expect that  their program managers will deliver high-quality products on time and  within budget. Doing otherwise could result in the customer walking  away. Thus, those firms have created an environment and adopted  practices that put their program managers in a good position to succeed in  meeting these expectations. Collectively, these practices comprise a  process that is anchored in knowledge. It is a process in which technology  development and product development are treated differently and  managed separately. The process of developing technology culminates in  discovery—the gathering of knowledge—and must, by its nature, allow  room for unexpected results and delays. Leading firms do not ask their  program or product managers to develop technology. Rather, they give  responsibility for maturing technologies to science and technology  organizations. The process of developing a product culminates in delivery  and, therefore, gives great weight to design and production. The firms  demand—and receive—specific knowledge about a new product before  production begins. A program does not go forward unless a strong  business case on which the program was originally justified continues to  hold true.
While the practices we have recommended represent commonly accepted  sound business practices, until recently, they have not been accepted by  DOD’s space acquisition community for large space acquisitions.  By  contrast, these practices were implemented for the development of a  small, experimental satellite, intended for direct use by a combatant  command, (known as TacSat 1).  We recently reported that by including  only mature technologies and limiting new requirements, DOD was able to  develop the satellite for less than $10 million (including surplus hardware  valued at $5 million) and within 12 months.
In disagreeing with our recommendations, DOD asserted its desire to push  programs to advance technologies as far as possible. Other reasons that  space officials have given for extending technology development into  acquisition include the greater ability to secure funding for costly  technology development within an acquisition program versus a science  and technology program, a belief among the acquisition community that  labs in charge of developing space technologies do not understand their  needs, as well as communication gaps between the science and technology  (S&T) and acquisition communities.
Moreover, while DOD officials told us they were pursuing evolutionary  development for space systems, we found that they were beginning  programs by challenging programs managers to achieve significant leaps  in capability with the intention of abandoning those efforts later in the  development cycle should too many problems be encountered. This is not  a true evolutionary approach, as it leaves DOD facing increased technical  challenges at the beginning of a program and thus, increased risks, and it  raises the expectations on the part of stakeholders who may be unwilling  to accept less capability later on. Two of the systems we were most  concerned about in this respect were and TSAT and Space Radar —they  were already expected to cost about $40 billion. DOD was planning to start  these acquisitions even when many of their critical technologies were still  immature and it was pursuing a highly ambitious path in terms of the  technology push. Given that these systems were among the most complex  programs ever undertaken for space, they were being counted on to  enable wider DOD transformation efforts, and DOD was already  contending with highly problematic space efforts, we believed DOD could  not afford to pursue such risky approaches for TSAT and Space Radar.
Since we last testified before this subcommittee in July 2005, DOD has  appointed a new Under Secretary of the Air Force to be in charge of space  acquisitions, who, in turn, has embraced adopting best practices, or, as he  terms it, “going back to the basics.” Specifically, the Under Secretary has  expressed a desire to    Delegate the maturation of technologies—to the point of being tested  in a relevant environment or operational environment, if appropriate— to the S&T community.
Adopt an evolutionary development approach in which new systems  would be developed in a series of increments, or blocks. Any desired  technology that is not expected to be matured in time to start a new  block would be assigned to a later block. Each block would have a  discrete beginning and end point.
Fund S&T appropriately so that significant technology breakthroughs  can be continually pursued.
Improve collaboration on requirements—consulting with warfighters  on the content of each new block.
In addition, the Under Secretary is focused on estimating cost and funding  new acquisitions to an 80-percent confidence level; strengthening systems  engineering and strengthening the acquisition workforce.
Aspects of this approach have recently been incorporated in to DOD’s  TSAT program. For the first block, satellites 1 and 2, the Air Force has  reduced its expectations in the level of sophistication of these satellites to  increase the confidence in the schedule for launching the first satellite in  2014. Higher performing levels of the technologies to support laser  communications and an Internet-like processor router will be pushed off  to a subsequent block, along with the multi-access laser  communications—a more robust laser capable of transmitting vast  amounts of data within seconds. Program officials have also stated that  the TSAT program will not enter into product development, that is, formal  acquisition, until its critical technologies are proven.
These are good steps when looking at TSAT as an individual program.  It is  important, however, that the Air Force ensure warfighters accept lower  capability and that it makes sense to pursue the current approach versus  the alternative of buying more AEHF or WGS satellites.
section_title: Keys to Realizing DOD’s New Goals for Space Acquisitions
paragraphs: DOD’s desire to adopt best practices for space acquisition is a positive and  necessary first step toward reform. However, these changes will not be  easy to undertake. They require significant shifts in thinking about how  space systems should be developed; changes in incentives and  perceptions; as well as further policy and process changes. Moreover, they  will need to be made within a larger acquisition environment that still  encourages a competition for funding and consequently pressures  programs to view success as the ability to secure the next installment  rather than the end goal of delivering capabilities when and as promised.  In addition, DOD’s space leaders will be challenged to sustain a  commitment to adopting best practices, given the myriad of missions and  programs that compete for the attention of DOD’s leadership and  resources, frequent turnover in leadership positions, and potential  resistance from the many diverse organizations involved with space  acquisitions.
There are steps, however, that DOD can take to substantially mitigate  these challenges.
First, DOD can guide its decisions to start space acquisition programs  with an overall investment strategy. More specifically, DOD could  identify overall capabilities and how to achieve them, that is, what role  space will play versus other air-, sea-, and land-based assets; identify  priorities for funding space acquisitions; and implement mechanisms  that would enforce the strategy and measure progress. Optimally, DOD  would do this for its entire weapon system investment portfolio so that  space systems that are expected to play a critical role in transformation  could be prioritized along with other legacy and transformational  systems and so that DOD could reduce pressures associated with  competition for funding. But in the absence of a departmentwide  strategy, DOD could reexamine and prioritize its space portfolio with  an eye toward balancing investments between legacy programs and  new programs as well as between S&T programs and acquisition  programs. In addition, DOD could prioritize S&T investments. This is  particularly important since DOD is undertaking a range of initiatives— collectively known as operationally responsive space (ORS)—designed  to facilitate evolutionary development, more testing of technologies  before acquisition, and ultimately enable DOD to deliver space-based  capabilities to the warfighter much faster and quicker. While ORS  investments hold great potential, there are other S&T projects  competing for the same resources, including those focused on  discovering and developing technologies and materials that could  greatly enhance future capabilities, reduce costs, and maintain U.S.  superiority in space.
Second, DOD could revise policies and processes supporting space as  needed to adopt the best practices being embraced. For example,  DOD’s space acquisition policy could be further revised to ensure that  a true evolutionary approach is being pursued and that blocks, or  increments, will include only technologies that have been sufficiently  matured. DOD could also implement processes and policies, as needed,  that stabilize requirements, particularly for acquisitions that are being  shared with other stakeholders, such as the intelligence community,  and that ensure warfighters are bought into capabilities being pursued  for each new system increment. In recent years, it has instituted  processes for some individual systems, such as SBIRS High, that could  serve as a model.
Third, DOD could continue to address other capacity shortfalls. These  include shortages of staff with science and engineering backgrounds;  shortages of experience within the program manager workforce;  limited opportunities and funding for testing for space technologies;  and the lack of low-cost launch vehicles. At the same time, DOD could  continue to work toward strengthening relationships between the S&T  and acquisition communities and coordination within the S&T  community. The Under Secretary is uniquely positioned to do this given  his previous position as DOD’s Director of Defense Research and  Engineering and his participation in previous efforts to develop a  strategy for space S&T.
Fourth, we have recommended that DOD take steps departmentwide to  hold people and programs accountable when best practices are not  pursued. This will require DOD to empower program managers to  make decisions related to funding, staffing, and moving into  subsequent phases and to match program manager tenure with  development or delivery of a product. It may also require DOD to tailor  career paths and performance management systems to incentivize  longer tenures. Until these actions have been taken, space leaders  could take steps now to ensure space program managers have the right  levels of experience to execute large programs and have sufficient  authority so that they can be held accountable. Likewise, DOD’s space  leaders can take steps to hold its contractors accountable by  structuring contracts so that incentives actually motivate contractors  to achieve desired acquisition outcomes and withholding award fees  when those goals are not met.
In closing, we are encouraged with the acquisition approach being  embraced by DOD’s space leadership. It can enable DOD to begin to match  resources to requirements before starting new programs and therefore,  better position programs for success. Successful implementation,  however, will hinge on the ability of DOD’s current space leaders to instill  and sustain commitment to adopting best practices over the short and long  term. In doing so, best practice approaches should be reflected in policy  and manifested in decisions on individual programs or reform will be  blunted. They should also be accompanied by an investment strategy for  space, and ultimately DOD, to separate wants from needs and to alleviate  long-standing pressures associated with competition within DOD to win  funding. By embracing a model that incorporates all these elements, DOD  can achieve better outcomes for its space programs.
section_title: Scope and Methodology
paragraphs: In preparing for this testimony, we relied on previously issued GAO  reports on assessments of individual space programs, incentives and  pressures that drive space system acquisition problems, common  problems affecting space system acquisitions, space science and  technology strategy, and DOD’s space acquisition policy, as well as our  reports on best practices for weapon systems development. We also  analyzed DOD’s Selected Acquisition Reports to assess cost increases and  investment trends. In addition, we met with the Air Force Under Secretary  to discuss his “back to basics” approach. We conducted our review  between March 6 and April 3, 2006 in accordance with generally accepted  government auditing standards.
section_title: Contacts and Acknowledgments
paragraphs: For future information, please contact Cristina Chaplain at 202-512-4841 or  chaplainc@gao.gov.  Individuals making contributions to this testimony  include, Art Gallegos, Robert Ackley, Maricela Cherveny, Sharron Candon,  Jean Harker, Leslie Kaas Pollock, and Karen Sloan.
Table 1 highlights recent findings from our reports on cost and schedule  overruns for DOD’s current and planned space programs. The table also  notes that many programs are still addressing past mistakes in acquisition  approaches and contractor oversight as well as technical, design, and  manufacturing problems.