Comparison of DoD and Commercial
Software Acquisition Practices (Cont.)

e Business Practices
— Commercial practice more flexible with greater incentives

e Integration Testing and Delivery
— Commercial provides integration and functional testing according to need
- DoD uses separate test agency with added time and complexity

~ Absence of beta testing within DOD increases costs

e Rights in Data

— Commercial more flexible, especially regarding resales

e Maintenance
— Commercial: Maintenance considered and integrated with development
- DoD: Maintenance not major factor in development process

__ Appendix C provides a more complete comparison of DoD and commercial software acquisition
practices as developed by this Task Force.


3.6 SOFTWARE ARCHITECTURE

Software System Architecture:
The Missing Link?

What is software architecture?
© Software architecture consists of:
— Software system components
- The relationships among those components

~ Rules for their composition (constraints)

e Defining document would address:
- System functionality
— Software system components
— Interfaces, standards, protocols
Execution model
- Data flows
- Control flow
— Critical timing/throughput aspects
- Error handling

Software architecture was emphasized by the Task Force as a means for achieving important ends.
Software architecture consists of software system components, the relationships among these components
and the rules for their composition (constraints). To use architecture as a management tool, DoD needs to
define: system functionality along with software system components to be employed; interfaces,
standards, and protocols to be employed; and the execution model to include data flows, control flow,
critical timing/throughput aspects, and error handling approach.


3.5 ACQUISITION

Acquisition

e Findings
- Acquisition practices have led to:

» Distance between user and developer
» Limited participation by commercial software companies

Adherence to DoD regulations for reviews and documentation is increasing,
software costs

» DoD software costs are estimated to be increased by at least 20 % over
commercial best practice

» Commercial best practice requires much less documentation than DoD

Funding for maintenance planning/execution starts late
» Maintenance assumed organic; inhibits teaming/partnerships

Acquisition focus is on mandatory “how to” specifications and standards
rather than the product (what)

» Lengthens process and adds costs
» Discourages harmonization with commercial practice
» Creates adversarial relationship
Acquisition process does not reward development of reusable software

The Task Force was concerned that the specific acquisition and contracting approaches used by
DoD inhibit use of commercial practices and software. The Task Force findings in this area are shown
above. Strict adherence to DoD regulations for reviews and documentation is increasing software costs.
DoD software costs are estimated to be increased by at least 20% over commercial best practices. The
DoD focus on detailed technical specifications has lengthened the process, added costs, discouraged
harmonization with commercial practice, and created a highly adversarial relationship between the
Government and industry. There is a strong belief that certain commercial companies or divisions of
companies have opted to stay away from government contracts due to the complexity of the acquisition
tules and regulations.


Software Technology Base (Cont.)

Recommendations

e Provide for the evolution of the DoD Software Technology Strategy to align
with emerging commercial technology and practices
e Emphasize Technology Transfer (External and Internal)
- Fund technology transfer programs in such topics as:
» Architectural principles,
» Architecture description languages,
» Standard interfaces, and
» Integration technologies

- Initiate demonstration programs (e.g., ATDs) to facilitate software technology
insertion into systems. Examples of candidate criteria:

» Open standards
» Use of COTS and GOTS
» Frequent releases to include a number of users
» Multiple platforms
» Satisfies commercial standards and interoperability standards across DoD
organizations
e Initiate formal data collection and analysis

The Task Force makes the above recommendations with regard to DoD software technology base
investments. The Task Force supports a DoD technology base program that is more closely aligned with
the wide range of similar efforts ongoing in commercial organizations.


Why Is This Study Important?

e DoD Military Capability Increasingly Software Dependent
e Software One of Few DoD Budget Areas That is Projected to Grow

- Software Has Become the Overall Defense System Schedule and Cost Driver
» Declining DoD Budget -- Affordability a Major Concern
» Escalating Costs of Software Development and Life Cycle

- Significant Level of DoD Resources To Be Spent on Information Technology:
» FY92- FY94: ~$10 Billion/Year*
» EIA Forecast for FY95 - FY98: ~$10 Billion/Year
» In-House vs. Contracted Out: 30% In; 70% Out
~ Will Require Greater Management Control
Rich Commercial Base to Tap; Many Opportunities
— Custom Software - Acquisition Methodologies
— Off-the-Shelf Products - Approach to Requirements Determination
Functionality and Flexibility More Embedded in Software than Hardware
Study is Timely Because DoD Leadership is Focused on Acquisition
Reform

~- Something May Actually Get Done *Source: EIA

Given its increasing reliance upon software as the mechanism for implementing system
capabilities, coupled with the rising costs associated with software development and maintenance, DoD
must take action now to address the issues associated with software acquisition. The commercial sector
provides numerous opportunities upon which the DoD could readily capitalize. The time is ripe for
assertive DoD action, particularly since the current leadership is so strongly focused on acquisition reform.


Briefings Provided to the Task Force

Joint Staff Views on How DoD Can Adopt Commercial Software
Practices

MG David Kelley, Vice Director J6

Army Views on How DoD Can Adopt Commercial Software Practice

LTG Peter Kind, Director, Information Systems for C+

Naval Views on How DoD Can Adopt Commercial Software Practice

RADM John Hekman, Commander, NISMC

Air Force Views on How DoD Can Adopt Commercial Software
Practices

Mr. Lloyd Mosemann, Deputy Assistant Secretary of Air
Force (Communication, Computers & Support Systems)

DISA Views on How DoD Can Adopt Commercial Software Practices

Dr. Mark Scher, Director of Infrastructures, Defense
Information Systems Agency

DoD 5000 Series Regulations

Mr. Gene Porter, Director, Acquisition Program Integration

DoD 8000 Series Regulations

Mr. Harry Pontius, Director. C31 Policy

DoD-STD-2167A/MIL-STD-498

Dr. Singh, Senior Manager, Space & Naval Warfare Systems
Command

Draft White Paper on Software Acquisition

Dr. Jack Ferguson, Program Manager, Software Engineering
Institute

Data Modeling vs. Data Standards

LTG Peter Kind, Director, Information Systems for C4

Case Study: B2

Mr. Fred Schwartz, Director of Engineering,

Case Study: Ensemble of Real-time Software Systems

MajGen Israel, Director, Defense Airborne Reconnaissance
Office

Case Study: Reserve Component Automation System (RCAS)

MG Gary Stemley, Program Manager for RCAS

Case Study: AEGIS

CAPT. Richard Cassidy, AEGIS Technical Director

Case Study: PRISM.

Mr. Robert Kent, ESC/ENS

Case Study: BMC3

Lt Col Robert Phelps, BMDO

Boeing Views on DoD vs. Commercial Software Practices

Mr. John Hanson, Director, Systems Software & Engineering,
Boeing

McDonnell Douglas Views on DoD vs. Commercial Software Practices

Mr. L. George Hite, Senior Manager, McDonnell Douglas

IBM Views on DoD vs. Commercial Software Practices

Mr. Dave Coyer, Program Manager, IBM Federal Systems
Company

Object Technology and a COTS Product Called "SNAP"

Mr. Joe Fox, Chairman, Template Software Inc.

Procurement Regulations/Issues

Mrs. Eleanor Spector, Director, Defense Procurement
OUSD(A&T)

Integrated Computer Aided Software Engineering (ICASE) Program

Col Gary Case, ICASE System Program Director

Computer Sciences Corporation Case Study on Commercial Software
Development

Mr. Daniel Kemp, Senior Partner, Computer Sciences Corp.

Commercial Software Acquisition Practices

Mr. Alex Morrow, General Manager, Lotus Development
Corp.

Experiences in DoD Software Maintenance

Mr. Bobby McDonald, Deputy Director, Electronic Warfare
Directorate, Warner Robins ALC

MITRE Corporation Experiences in Exploiting Commercial Practices

Mr. Steven Crisp, Dept. Head, MITRE Corp.

FAA Study on the Use of Commercial Software

Mr. John Stenbit, Vice President & General Manager, TRW
Systems Integration Group

NSIA Study on the Use of COTS Software

Gen. William Richardson, USA(Ret), Executive Vice President
Army Programs, Burdeshaw Associates, Ltd. and Ms. Linda
Connor, Program Manager, Rockwell International

DoD Software Reuse Initiative

Ms. Linda Brown, ODASD(IM)/Information Technology

MITRE Study on Feasibility of Using a Software Acquisition Maturity
Model in DoD

Ms. Judith Clapp, Division Assitant, The MITRE Corp.

Software Management Metrics and Reliability

Mr. Douglas Putnam and Mr. Lawrence Putnam, Jr.,
Quantitative Software Management, Inc.

Innovative Techniques in DoD SW Acquisition: F-22 Integrated
Product Team Approach

Colonel Robert Kayuha

Legal Impediments to Acquiring Defense Software Commercial

Mr. Robert Gorman, OSD General Counsel


APPENDIX B

BRIEFINGS
PROVIDED TO THE TASK FORCE


Comparison of Software Acquisition Methods!

Requirements Definition

Best Commercial Practice

Current DoD Practice

Requirements based on strategic plan and market
analysis.

Requirements based on using command Mission
Need Statement, master plans, and top-level
certification.

Requirements based on life-cycle resource
constraints.

Requirements based largely on annual budget
resource constraints.

Detailed requirements generated by interdisciplinary
team including users, domain experts, and system
engineers.

Detailed requirements generated by buyer in
collaboration with user. Team generally includes
domain experts and acquisition personnel.

Buyer, user, and vendor are a team. Attitude of
partnership, trust and cooperation. Presumption of
trustworthiness for reputable commercial
organizations.

"Us vs. them" mentality about contractors.
Government thinks in terms of control,
accountability, detailed auditing, and double
checking. Presumption that contractors cannot be
trusted.

Functional specification is modified by knowledge
of availability of existing products.

Functional and/or performance specification; little to
no regard for existing products.

Vendors involved early in study, analysis and
prototyping with emphasis on reuse and evolution
of existing systems.

May contract for prototypes, but contractor
involvement in pre-award discussions is
discouraged.

Need is based on business case and decisions are
based on return on investment.
("time to make")

Need based on Mission Need Statement; decisions
based on need, economics, and politics. ("time to
field")

Efficient decision processes.

Decision processes formal and time-consuming.

Level of documentation is negotiable based on
individual user needs and complexity of system
being developed.

Extensive (often redundant or unnecessary)
documentation required under 2167A.

Tailoring of documentation requirements is often
minimal or discouraged.

More detailed analysis of cost versus feature.
Dropping lower value/higher cost options or
reducing requirements is practiced.

Little or no requirements reductions on high cost
items.

More requirements trade-off decisions (involving
complexity and schedule) for reduced time to field.

Very little flexibility to trade-off requirements creep
versus complexity and schedule.

Selected vendors may assist in preparing
specifications.

Vendors not involved in preparing specifications.

Tools used to create system models for use in
requirements definition; e.g, GUI Building.

Requirements definition based on Mission Need
Statement.

Flexibility allowed in choice of programming
language.

Specific requirements regarding use of
programming language; e.g.,, CMS-2, etc.

Evolutionary and incremental approach favored.

Requirements defined up front with little flexibility
for modifications.

significant cost savin;

Summary
Commercial is more flexible and open between users and suppliers, and requirements are based ona
strategic plan. In the commercial world, there is more willingness to adjust requirements based on
availability of products and thus to filed a system sooner and evolve it to include more capability at

1 This appendix was initially derived from a White Paper on software acquisition methods prepared by the Software
Engineering Institute. The resulting content represents the consensus of this Task Force.

Cl


Rights in Data

Best Commercial Practice

Current DoD Practice

Tf custom development, buyer gets all rights, but
vendor may retain right to subsequent sales.

Specified by contract.
Government usually demands all rights for
government use due to organic support and
maintenance needs, and competition (via statutory

mandate).
If tailored version of standard system, buyer only | Same as commercial. May have exceptions for
gets rights to tailored parts. proprietary material.
Summary

Similar, but commercial is a little more flexible, especially regarding resales.

Maintenance

Best Commercial Practice

Current DoD Practice

Organic support shifting to outsourcing or vendor.

Organic support, with reluctance to be dependent on
vendor. Use of depot maintenance makes
interoperability issues more manageable. Also,
must be responsive to user for critical systems.

Level of discrepancy reporting required is based on
user needs. Problem resolution usually delayed
until next major release of software. Buyer has
limited power to implement fast resolution of
problems.

Bureaucratic and paper-intensive discrepancy
reporting and change control board often imposed.
Award fees may be tied to problem workoff,
resulting in fast resolution of problems.

COTS solutions take advantage of economics of
scale since maintenance costs are leveraged across
| multiple buyers.

Unique software requires custom maintenance all
borne by the single buyer.

Summary
The DoD and industry currently have different requirements, and trends are moving apart. However, DoD

is currently reevaluating its

practices for hardware systems and

perhaps should also reevaluate for software.

C5


THE UNDER SECRETARY OF DEFENSE

WASHINGTON, DC 20301-3000

Dil 86 1993

ACQUISITION

MEMORANDUM FOR CHAIRMAN, DEFENSE SCIENCE BOARD

SUBJECT: Terms of Reference--Defense Science Board Task Force on
Acquiring Defense Software Commercially

You are requested to form a Defense Science Board Task Force
on Acquiring Defense Software Commercially. Determine the
conditions under which the procurement of defense software (i.e.,
operational software, support software, and software tools) can
appropriately use commercial practices and develop a strategy for
defense software procurement that substantially incorporates such
practices. Specifically address within this strategy DoD use of
_commercial software products.

The scope of this effort should include all DoD systems that
are software intensive. It should address all stages in the life
cycle of a software component from initial procurement to
evolutionary upgrade of software or of software/hardware
combinations. This commercially-based strategy should not be
constrained by existing DoD standards; it should be viewed as a
coexisting alternative to, rather than replacement of, the
current DoD procurement strategy. Accordingly, you should
compare your proposed strategy to the current DoD strategy to
indicate circumstances in which each strategy is most beneficial.

To assess the appropriateness of DoD use of commercial
practices, the Task Force should identify and apply objective
measures such as elapsed development time, software life-cycle
cost, management risk, as well as measures of software product
quality.

The Task Force should consider at least the following
topics:

* Technical: state-of-the-art and best commercial
practices, development tools, reusable software components,
and techniques and tools for tailoring commercial software
components for use in defense systems.

+ Management: DoD management of the commercial development
process, software risk management techniques and supporting
tools, minimum delivery time, affordability, maintenance
after product delivery, post-deployment product enhancement,
software process support tools, quality and assured
availability, and use of development and maintenance tools.


Principal Reasons DoD Software
Programs Get Into Trouble

EE

Poor Requirements Definition
- Lack of User Involvement in Development Process
Inability of Users to Foresee Benefits of Automation Without Incremental Capability

@ Inadequate Software Process Management and Control by Contractor

e Lack of Integrated Product Teams
- Failure to Establish “Team” With Vendors and Users
- Little Participation of Functional Area Experts

e Ineffective Subcontractor Management
e Lack of Consistent Attention to Software Process
e Too Little Attention to Software Architecture

¢ Poorly Defined and Inadequately Controlled Interfaces Between Computer
Hardware, Communications and Software

e Assumption That Software Upgrades Can “Fix” Hardware Deficiencies
(Without Assessment of Cost and Schedule Risks)

Focus on Innovation Rather than Cost and Risk

Limited or No Tailoring of Military Specifications Based on Continuing
Cost-Benefit Evaluations

Over the last two decades, a number of DoD software development efforts have gotten into trouble,
particularly in terms of actual costs and schedule vs. expected or predicted costs and schedule. The Task
Force heard briefings from a number of DoD program managers where this was the case. Based on the
specific programs discussed and on other inputs, the Task Force prepared a listing of the principal reasons
DoD software programs have gotten into trouble as shown above. Certain of these reasons are a reflection
of DoD practice. Others are tied to the software engineering level available at the time such programs were
initiated. The Task Force believes that today’s software technology and practices can directly address
many of the root causes for such past problems.


Software System Architecture (Cont.)

Findings
e Why is it important?
- Essential for effective management over the lifecycle
» Software system lifecycle costs ~ 65% for maintenance
» ~ 65% of maintenance costs are due to changes/modifications/upgrades
- Software architecture is a prime enabler of flexibility and reuse

- Well-formulated architecture might reduce costs of changes/upgrades by 30-
50% ($4-$7B/year assuming software expense ~ $30B/year)

e@ Why doesn’t it play a larger role?
; — Focus is usually on initial cost, schedule, functionality — not lifecycle

~ 2167A reinforces this approach - requires proof that design satisfies
functionality

- Difficult to specify, test, etc.
- Not well understood

Software architecture is a prime enabler of flexibility and reuse, and a well-formulated architecture
might reduce costs of changes/upgrades by 30-50% ($4-$7B/year assuming software expense
~$30B/year). Software architecture has not been emphasized because PM focus has usually been on initial
cost, schedule, and functionality and not on the life cycle. DoD-STD-2167A reinforces this approach by
requiring only proof that a design satisfies the required functionality.


Table of Contents

Executive Summary ..............cccccccccccccccccesssssssseeeeeeeeeeeseeeseeeeseenenieseeeeeeeeceeereeeeeeeneaaeea i

1.0

2.0

3.0

4.0

Task Force Overview..................ccccccssseeceeeccccceeenaeeeeeeeeececececaaeeeeeeeeeeeeeeaaeneneees 1
1.1 Terms of Reference

1.2 Caveats
1.3. Topics Addressed...
1.4 Membership ........... cece c ee cee cece ccc e eee ee eee e nee e eens scene rene eee eee saree ae neeet reer eee eaee 4

Current DoD and Commercial Software Acquisition Practices.

2.1. Background
2.2 Comparison of Defense and Commercial Software... cece seeeeeeee 10

Major Findings and Recommendations.
3.1 Process Credibility

3.2 DoD Program Management
3.3. DoD Personnel
3.4 Use and Integration of Commercial Off-the-Shelf Software.
ie C705 03 08 (0) | BEE EEECEECEOOOOOSSSOSSOSSS
3.6 Software Architecture
3.7 Software Technology Base

OTL) AES SESS ESSE SSSSSECOOOOSOOSS
4.1 “Atta Persons”
4.2 Overarching Recommendation ..

Terms of Reference ............ccccccccccccc cece ee cece ee eee eee eee eee eG EE EE ETRE E EEE EEA EEE EERE A-1

Briefings Provided to the Task Force
Comparison of Software Acquisition Methods..................c::ccsceeeeeeeeeeeeeeeeeeees C-1

Proposed Action Plan from Service Software Executive Officials .................... D-1


Use and Integration of
Commercial Off-the-Shelf Software

Findings (Cont.

e DoD has not fully identified the pros and cons associated with the use of COTS
- Pros
» Saves money in design and development of those components
» Can Significantly Reduce Development Risk
eSupport is Concern
Selection of COTS products early in project cycle will enable requirements to be driven by commercial software
capabilities
Very useful in rapid prototyping
- Cons,
‘Commercial products must still be integrated into system and qualified with system
Difficulty in Configurement Management and Support for Older Releases
‘Additional testing may be required to qualify system with commercial components
‘Subsequent releases determined by vendor, not DoD

Security Aspects of Use of COTS Not Well Understood

© DoD not addressing multiple aspects
= Development environment
= Tactical computer program
= Virus protection
= Commercial Espionage
© Classified sc systems a pr -m for rcial
e Inaddition, DoD has not determined when to use COTS

Commercial Off-the-Shelf Software Products May Not Apply to All DoD Systems
Should be used as is - avoid tailoring or special features

— Most weapon system/real-time application software for DoD will not exclusively be “custom,” but
will involve some COTS

In general, DoD has not identified the pros and cons associated with the use of COTS. DoD must
learn how to balance the cost savings associated with design and development of commercial software
products and the significantly reduced development risk with the concern for longer term support and
system security. Commercial products must still be integrated into and qualified within each defense
system and there is difficulty in configuration management and support for older releases. The latter point
is important since many DoD software systems are not deployed before a commercial software product is
retired or replaced. DoD is not addressing many aspects of the use of COTS software:

— Development environments
— Virus protection
— Commercial espionage

In essence, DoD has not developed a corporate way to decide when to use COTS software.
Commercial off-the-shelf software products do not apply to all DoD systems. Most weapon system/real-
time application software for DoD will not exclusively be "off-the-shelf." Integration and configuration
control for COTS software then become important concerns. Further, DoD must learn to use COTS so
that it avoids tailoring or special features.


Assessment of Current DoD
Software Acquisition Approach

e Strengths

- Highly Structured Process Tied to Individual System
Developments

~ Tightly Defined Requirements

— Produces High Quality Product for Mission Critical Systems
That Demand Extremely Low Failure Rates (e.g., Flight
Controls for Man-Rated Platforms)

The strengths of the current DoD approach to software development, acquisition and operation are
summarized above. The Task Force found that the highly structured DoD process has, in fact, provided a
high quality software product, in most cases.


3.0 MAJOR FINDINGS AND RECOMMENDATIONS
3.1 PROCESS CREDIBILITY

Process Credibility

Findings

e Attempts to achieve absolute fairness in competition in contracting
(fair and reasonable pricing) have led to a lack of trust between
government and individuals/contractors

~- Current DoD practice:
Is not compatible with commercial business practices
Is focused on contractor management/audit activities and costs
Hinders flexibility (managers take no personal risk)
Is not well suited to procuring complex, knowledge-intense, "first of a kind" systems
Is too costly
Does not prevent malfeasance or incompetence
Leads to adversarial relationships
Reduces reuse and contractor incentive to invest because of stringent data rights
interpretation
- Contractors must make profit on single contract; no long term relationship (DoD
business practices do not view profit as a legitimate cost of doing business)

~ No individual fully understands or owns total process

The Task Force spent considerable effort on how the requirement for DoD to ensure public trust in
its acquisition process influences DoD’s ability to employ “commercial best practices.” The Task Force
found that attempts to achieve absolute fairness in competition in contracting have, in fact, led to a lack of
trust between the government and individuals/contractors. DoD acquisition processes are focused on’
contractor audit activities to an extent that hinders the flexibility of Program Managers and contractors.
DoD PMs are not incentivized to assume any personal risk associated with allowing for flexibility
comparable to commercial practice. Criminal sanctions are a significant disincentive for such efforts.

DoD’s acquisition system still does not prevent malfeasance or incompetence and leads to
adversarial relationships rather than partnerships which are the norm within commercial industry software
developments. One problem highlighted by the Task Force was that the current DoD system does not
allow one individual or manager to control the total process, even for a specific project. This lack of
control leads to a diffusion of accountability and hinders DoD’s ability to oversee complex “first of a
kind” software developments.


Assessment of Current Commercial
Software Acquisition Approach

® Strengths
- Open Architecture Compatible with Usage of COTS Software
- Much Less Formal Competitive Procurement Process
» Includes Prior Performance as Major Selection Criterion

Trend Toward Joint Assumption of Risks Between Buyers and Suppliers
Across Development, Operation, Maintenance and Modernization

Process is More Flexible

Shorter Cycle (Product Release) Times

Tailorable Level of Documentation and Oversight

Emphasis on Reuse and Tailoring Requirements to Existing Products
Beta Testing Widely Used

e Weaknesses

- Less Responsive to Continual Changes in Requirements
— Less Assurance that Software Will Function Properly Under All Situations
- Potentially Locked into One Vendor's Proprietary Application

The strengths and weaknesses of the current commercial approach to software development,
acquisition and operation are summarized above.


Use and Integration of
Commercial Off-the-Shelf Software

Recommendations

e Require Trade Studies and Analysis of the Use of COTS in DoD’s Software Acquisition

Process Where Effective
— Performed by Acquiring Organization as Essential Part of Defining Requirements and in Rapid Prototyping
Situations
» Employ Broad Agency Announcements or Similar Contractual Approaches to Facilitate Such Studies
= Use of COTS Appropriate When:
Defining Requirements
Rapid Prototyping Situations
Not Required to Tailor COTS Source Code to Application

» Not Required to be Error-Free
» COTS Software is “Close Enough” to Tailor Requirements

Establish “Customer Friendly” Application-Specific Information Technology
“Component Stores”
= Generic Architectures for Specific Domains
Rapid Requirements Definition Process and Prototyping
Reusable, Prequalified Components
Assemble Systems Rather than Develop Them
Reduce Lead Time
Security is not Paramount

Increase tech base funding for security audit tools for systems employing COTS
Capitalize on Innovative Cost-Effective Techniques for Acquiring and Using COTS
Software Products

- Such as Use of Enterprise Licenses

Given these findings, the Task Force makes the above recommendations with regard to DoD use
and integration of commercial off-the-shelf software. The Task Force sees great benefit to be gained
through exploitation of COTS software; however, DoD must develop corporate approaches to the use and
integration of COTS software, if it is to gain this benefit.


Process Comparison Summary

LS

Process Attribute

Custom
Commercial

Civilian’
Commercial
Result

Problem Definition

Shared (Includes
Markot Analysis)

More Practical
Requirements

Process Definition

InternavEvolving

Reduced Cost!
‘Schedule

Flexibility.

Not Constrained

Process Improvement
Bree, Enabling
"Technologies

Mllestones/Reviews
(Duration, Formallty)

‘Short informal
Ineremental
‘Document

Reduced Cost
‘Schedule

[ Customer ser

Involvement in
Development

Tigh Userat Prob.
Def. and,
User = Cust

Tess Requirements

Process Monitoring by
Customer

Some

‘Customer Acceptance
Process

‘Simple

Inspection/Teating

Rigorous/Formal

‘Subcontracting

Brief Product Spec
180 Possible

Use ot
Development Techniques
{le Rause4GU00)

Extensive

‘Source: IBM Federal Systems

As is evident from the process comparison summary above, there are not only differences between
commercial and defense applications, but also in the process used by each to develop, acquire and support
complex software systems. Defense and other federal acquisition regulations and detailed specifications
require a much more complex set of deliverables within the context of DoD’s contracting process.

a


Development Process

Best Commercial Practice

Current DoD Practice

Vendor often tailors existing systems and uses
COTS. System designed to fit in a defined product
or product line architecture.

Varies with application. Some systems use COTS.
However, usually a new system that doesn't reuse
legacy software. Unique systems are built with little
regard for architecture.

Buyer may have heavy involvement in design and
development as part of the team (Integrated Product
Development team).

Formal, structured spiral, or waterfall model. Buyer
oversees, but team approach is not usually
emphasized.

Reviews typically informal and stress progress
against goals.

Reviews usually very formal and include technical
design details in addition to progress metrics.

Buyer actively involved as co-participant in
management of technical details.

Micro management of technical details.

Heavy user involvement.

Limited user involvement.

involvement.

Heavy buyer

Vendor embraces one or more industry standards
which improves interface and integration with COTS
products.

Government and industry standards called out. Not
all government standards enhanced by COTS
products.

Buyer requirements may be translated to more
"general purpose" requirement for potential
software reuse.

Tailored system; little, if any, focus on designing in
reusable code.

Management reviews and degree of oversight are
commensurate with size and risk of program.

Notably more detailed reviews and oversight
performed.

Prototyping common, with joint applications
development teams (user and developer) working to
clarify requirements and incorporate new
requirements that do not affect cost or schedule.

Prototyping seldom used.

Use of flexible architectures allows insertion and
plug and play of COTS products.

Use of MIL standard computers and legacy
instruction set architectures restricts new

development.

existing systems. Multi-year acquisitions not re-

Summ
Commercial more flexible with likelihood of a team approach and is biases toward reuse and tailoring of
justified each year. Product im,

rovements are anticipated.

C3


1.3 TOPICS ADDRESSED

Topics Addressed

(Topic

|Management:

l@ DoD Software Acquisition Policies

[fe DoD Management of Commercial Development Process
[@ Software Risk Management Techniques and Supporting Tools
[@ Minimum Effective Delivery Time

[© Affordability

[© Maintenance After Product Delivery

le Fost-Deployment Product Enhancement

[@ Software Process Support Tools

fe Quality

[© Assured Availability

[© Use of Development and Maintenance Tools

le Technical Data Rights

[© Intellectual Property Rights

[® Viability

je Alternative Forms of Procurement Agreements

[@ Incentives for Creation/Use of Reusable Software Components

[@ Economic Incentives

Technical: .

J@ Importance of Software Architecture

fe State-of-the-Art and Best Commercial Practices

[@ Development Tools

[© Reusable Software Components

[® Techniques/Tools for Tailoring Commercial Components for
Defense Use

In its efforts to assess the appropriateness of DoD use of commercial practices, the Task Force
addressed software management, contracting, and technical issues. The above viewgraph lists the primary
topics considered within each of these categories.


DoD Personnel (Cont.)

Recommendatio
e Establish DoD-wide software program management education and training
initiative
- Change DSMC and IRMC courses for PMs to reflect best commercial practices and other

recommendations of this Task Force and Provide for changes to reflect the dynamics of the
software industry

— Develop and provide interactive training tools for senior managers to perfect software
management skills

— Rotate government and contractor personnel between PM and developer organization to build
understanding and trust; encourage use of IPA’s from industry

~ Incorporate software management principles in senior management education and seminars
(including senior services colleges)

~ Provide mechanisms for keeping software expertise current in the workplace
e Develop Acquisition Managers with software program management expertise
Integrate software-qualified personnel into senior acquisition staff
Establish Norms for the Number of Software Experts in Program Offices
Upgrade Educational Requirement for Personnel Assigned to Acquisition,
Management, Development and Oversight of Software Intensive Programs
e Develop Expertise in Analysis of Domain Software Design

- Promote Software Reuse in the Design

The Task Force makes the above recommendations with regard to DoD software expertise of its
personnel. The Task Force strongly recommends an emphasis on increasing the capability of its personnel
in modern software practices and techniques.




C. DEVELOP DOD "BETA TEST" PHILOSOPHY (ACTION OFFICE OSD(T&E)
1. Team with Universities/other appropriate activities
2. Allow fielding of software direct from test beds with user consent

IV. PERSONNEL

A. DEVELOP SOFTWARE ACQUISITION MANAGERS (ACTION OFFICE
USD(A&T))
1. Provide a career path for software managers
2. Integrate software personnel into senior acquisition staff

B. PROVIDE SOFTWARE EDUCATION FOR SENIOR MANAGERS
(ACTION OFFICE USD(A&T))
1. Develop DoD Senior Software Managers Course
2. Develop and provide interactive training tools for
senior managers to perfect software management skills

C. PUBLISH AND PROMOTE "BEST PRACTICES" HANDBOOK (ACTION
OFFICE OSD(T&E))

D. ENSURE DOMAIN EXPERTISE (ACTION OFFICE MILITARY
DEPARTMENTS/ AGENCIES)

V. SOFTWARE TECHNOLOGY BASE AND TRANSITION (ACTION OFFICE
DDR&E)

A. PROVIDE FOR SOFTWARE TECHNOLOGY INSERTION INTO SYSTEM
ACQUISITION

B. INVEST IN THE DOD SOFTWARE TECHNOLOGY STRATEGY

C. PROVIDE FOR THE EVOLUTION OF THE DOD SOFTWARE
TECHNOLOGY STRATEGY TO CAPTURE EMERGING COMMERCIAL
PRACTICES


3.2 DOD PROGRAM MANAGEMENT

DoD Program Management

Findings
Program management does not encourage “80% solution for 20% cost”
Users often not significantly involved in process

Little emphasis on life-cycle issues (including maintenance and support)

Existing policies, methodologies, and procedures, and their implementation are
inadequate

— Little evidence that policies are influenced by actual experience and vice versa

~ Little effort to measure effectiveness and costs of policy directives

Some indication that DoD is migrating toward development and use of
standards-based architectures

Program Managers lack incentives to allow tailoring of procedures to fit
individual program needs or to develop “corporate” solution (e.g., employ
common architecture or common software components)

The Task Force found that the DoD program management system itself discourages the use of
commercial best practices. These findings are not unique to software, but are particularly important for
software given the significant cost savings associated with software reuse. There are few incentives for
Program Managers (PMs) to develop or even employ corporate solutions (common architectures/software
components), particularly if they are more expensive to acquire. Rather, each PM will tend to optimize
his/her development for its own purpose. Further, it is difficult for users to be significantly involved until
late in a software development process, unless some sort of prototype can be constructed. DoD PMs place
little emphasis on life-cycle issues (such as software maintenance and support). Existing DoD-wide
software policies, methodologies, and procedures, and their implementation by PMs are inadequate.
There is little evidence that policies are influenced by actual experience and vice versa and there is little
effort to measure the effectiveness and costs of policy directives.


DOD Personnel: There is currently a shortage of sufficiently qualified software personnel at all
levels within the Department. DoD should establish a Department-wide software program management
education and training initiative that includes: changing courses for PMs to reflect best commercial
practices and other recommendations of this task force and providing for changes to reflect the dynamics
of the software industry; rotating government and contractor personnel between PM and developer
organizations to build understanding and trust; encouraging use of IPA's from industry; and integrating
software-qualified personnel into senior DoD acquisition staff.

Use and Integration of Commercial Off-the-Shelf (COTS) Software: DoD has not
fully identified the pros and cons associated with the use of COTS software and, as a result, has not
determined when and how best to use COTS software. To facilitate this process, DoD should require
trade studies and analysis of the use of COTS software in DoD’s software acquisition process where
appropriate. Further, DoD should establish “customer friendly,” application-specific information
technology “component stores” to enable program managers to assemble systems rather than develop them
through use of reusable, prequalified components. DoD should also increase technology base funding for
security audit tools for systems employing COTS software and should capitalize on innovative cost-
effective techniques for acquiring and using COTS software products, such as the use of enterprise
licenses.

Software Architecture: Software architecture was emphasized by the Task Force as a means
for achieving important ends. There is currently little emphasis on architecture in DoD software programs
or regulations. As a result, DoD is not benefiting from architecture as a key tool for evolutionary
development and for early (and frequent) involvement of users with functional capability and facilitating
reuse, requirements changes with minimum cost and schedule, and “product line” management. DoD
should require vendors to propose, manage and control the architecture and should establish an early
architecture deliverable in all developments.

Software Technology Base: The current DoD software technology base investment does not
adequately take advantage of commercial R&D. Further, software technology transfer (both internal and
external) is not receiving adequate emphasis within DoD. DoD should provide for the evolution of the
DoD Software Technology Strategy to align with emerging commercial technology and practices.

Overarching Recommendations: To facilitate implementation of the many recommendations
contained in this report, the Task Force concluded that DoD's investments in software require greater
management control and DoD-wide oversight. To this end, the Task Force recommends that the Secretary
of Defense (SECDEF) assign the Under Secretary of Defense (Acquisition and Technology) (USD (A&T))
the responsibility for DoD-wide software technology, policy, practices and acquisition. In carrying out
this responsibility, the Task Force recommends that the USD (A&T) consider forming an executive
council with the Director, Defense Research and Engineering (DDR&E), the Assistant Secretary of
Defense (Command, Control, Communications and Intelligence) (ASD (C3I)) and appropriate
representatives from the Services and Defense Agencies as members. Further, USD (A&T) should
provide for a supporting “process action team” to assist in implementation of the recommendations of this
Task Force study.


Executive Summary

The Defense Science Board Task Force on Acquiring Defense Software Commercially recognizes
that DoD systems are becoming increasingly dependent on the use of software as the mechanism for
implementing operational capabilities. To adapt to changing military and national security situations, DoD
is more dependent than ever on its ability to modify mission software rapidly, often in near real-time.
However, software remains the schedule and cost driver for the development and maintenance of many
important defense systems.

In its review of current DoD and commercial software acquisition practices, the Task Force found
notable differences, as evidenced in Appendix C. There are, however, many similarities between the
various categories of DoD and commercial software systems. Although there are indications that
commercial development efforts have achieved better predictability and lower costs, the Task Force noted a
significant lack of credible, quantitative data to substantiate this assessment.

In general, the Task Force concluded that DoD's investment in software requires greater DoD-wide

management control and oversight in the coming years if the Department is to exploit the use of

commercial software acquisition practices fully, as well as rapid advances in software technology. The
following is a summary of selected findings and recommendations toward that end.

Process Credibility: Current DoD practice is not compatible with commercial business
practices. DoD should work to make necessary changes to acquisition regulations such as:

¢ Having program managers manage 3 of 3 (price/schedule/functionality) but only constrain 2 of 3

* Defining successful performance on contracts as delivering a solution (with predictable price,
schedule and functionality) not adherence to government processes, procedures and specifications

* Not requiring c-level specifications for software projects developed in Ada

¢ Establishing mechanisms to allow both current ability to perform as well as past performance as
key factors in source selection

¢ Encouraging offerors to demonstrate as much functionality as possible as part of bid without
eliminating domain knowledgeable competition

DoD Program Management: DoD program management approaches discourage the use of
commercial practices. Program managers lack incentives to tailor procedures to fit individual program
needs or to develop “corporate” solutions (e.g., employ common architecture or common software
components). DoD should establish and implement overarching software life cycle guidelines more
conducive to the use of commercial practices and products, such as:

¢ Defining software architectures to enable rapid changes and reuse

* Facilitating early system engineering and iterative development

* Participating in development of commercial and international standards

¢ Allowing the fielding of software directly from test beds with user consent

* Requiring program managers to stay with programs at least through beta testing to maintain
continuity of understanding of original nuances in requirements


Comparison of DoD and Commercial
Software Acquisition Practices

e Requirements Definition
- Commercial
- More flexible and open between users and supplier
~ Based on strategic plan and usually cost/schedule driven
- More willing to adjust requirements based on availability of off-the-shelf
products
~ Evolves capability

e Vendor Selection
- Commercial
- Much more flexible; no requirement for fairness or to maintain the public trust
— Encourages vendors to offer best solution, not meet 100% of requirements
- Accommodate teaming and long-term relationships

e Development Process
~- Commercial
- More flexible; product improvements anticipated
~ Team approach with bias toward reuse and tailoring of existing systems
- Multi-year acquisitions not re-justified each year.

In essence, the Task Force found major differences between DoD and commercial software
acquisition practices, as outlined above and on the next page.


1.2. CAVEATS

Caveats

a
o Relied on Inputs from DoD and Industry Experts

« Provided No Detailed, Quantitative Assessment or
Evaluation of Individual Topics

Did Not Address:

— Success or Failure of Ada

— Recommended Software Development Approach for Specific
Programs

— Specific COTS Products for DoD to Exploit
— Trade-Offs Between Hardware and Software

The Task Force relied heavily on inputs from a variety of DoD and industry experts regarding a
wide range of topics related to defense software technology, policy, practices and acquisition. Appendix
B lists the many briefings that were provided. Although the Task Force chose not to provide detailed,
quantitative assessments or evaluation of individual topics, it used the information associated with these
topics in the formulation of findings and recommendations. The areas not addressed by the Task Force,
while important in and of themselves, were determined not to be directly germane to the development of an

overall defense software strategy.


Resource Allocation:
Where Does the Effort Go?

Military Commercial

Engineering 30% 50%
Evaluation 20% 20%
* Management 15% 10%
* Meeting Support 15% 5%

* Documentation 15% 7%

¢ Customer/User Support 5% 8%

Source: Magnavox and Capers Jones

There are some indications that commercial development efforts have achieved better predictability
and lower costs than DoD counterparts. The Task Force solicited quantitative data on this issue from both
government and commercial software experts. The figure above summarizes one type of indicator of the
difference between commercial and government projects (in terms of the percentage of the effort expended
for different aspects of a typical development).


REPORT OF THE
DEFENSE SCIENCE BOARD

TASK FORCE

ON

ACQUIRING DEFENSE SOFTWARE

COMMERCIALLY

JUNE 1994

OFFICE OF THE UNDER SECRETARY OF DEFENSE
FOR ACQUISITION & TECHNOLOGY
WASHINGTON, D.C. 20301-3140


4.2 OVERARCHING RECOMMENDATION

Overarching Recommendation

SS

e SECDEF Assign USD (A&T) the Responsibility for DoD-Wide Software
Technology, Policy, Practices and Acquisition

e This Responsibility Includes Allocating Resources and Assigning
Responsibility for:

Drafting and Institutionalizing a New Software Acquisition Policy That
Includes the Recommendations of this DSB Task Force

Creating Incentives and Ensuring Compliance to the Policy

Creating Terms and Conditions and Financial Rewards to Maximize the
Ability of DoD to Realize the Benefits of Commercial Best Practices

Requiring Source Selection Evaluation of Development Contractors Through
a Formal Software Process Capability Evaluation

Advising the Acquisition Executive on Matters Concerning Software
Technology, Acquisition and Architecture for Major Programs

Maintaining a Digest of Lessons Learned and Best Practices and
Communicating the Same to Program Managers and Contractors

Throughout its deliberations, the Task Force acknowledged that the issues associated with defense
software were applicable across the spectrum of DoD software intensive systems. The Task Force also
frequently learned of obstacles based, in part, on the current dichotomous DoD structure associated with
software technology, policy, practices and acquisition. In order to ensure that the DoD reap maximum
benefit from its recommendations, the Task Force formulated the above overarching recommendation.


2.0

2.1

1D AND CO) RCIAL SOFTWARE ACOUIST PRACTIC

BACKGROUND

Background: Previous Studies

DSB Summer Study on Technology Base (1981)

Joint Service Task Force on Software Problems (1982)

AF SAB High Cost and Risk of Mission Critical Software (1983)

CODSIA Report on DoD Management of Mission-Critical Computer Resources (1984)
DSB Task Force on Military Software (1987)

Ada Board Response to DSB Task Force (1988)

Summer Report on Defense-Wide Audit of Support for Tactical Software (1988)
Workshop on Executive Software Issues (1988)

Workshop on Military Software (1988)

Army Materiel Command Study (1989)

Software Technology Development and Deployment Plan for DoD Technology Base
(1989)

AF SAB Adapting Software Development Policies to Modern Technology (1989)
Draft DoD Software Master Plan (1990)

Draft DoD Software Technology Strategy (1991)

AF SAB Study on Information Architecture (1993)

Study on Military Standards Impacts on the Acquisition Process (1993)
Draft Software Action Plan Working Group Report (1993)

Evolutionary Acquisition Study, AFCEA (June 1993)

As a point of departure, the Task Force noted a number of previous studies addressing issues

related to the defense software technology, policy, practices and acquisition. Despite the increased
emphasis given to software issues by the DoD (as evidenced by the above list), the majority of the
recommendations resulting from these studies have not been implemented.


1.4 MEMBERSHIP

(— Members >

e Dr. George H. Heilmeier Bellcore
© Dr. Larry Druffel Software Engineering Institute
Members:
© Dr. Jacques Gansler TASC
e@ Mr. Jack Hancock Retired Exec VP, Pacific Bell
e@ Mr. Patrick Hillier EDS
e@ = Mr. Arthur E. Johnson Loral Federal Systems
e@ Dr. Bruce Johnson Andersen Consulting
@ Mr. Alan McLaughlin MIT Lincoln Laboratory
e Dr. Alvin E. Nashman Computer Sciences Corporation
@ Mr. John Stenbit TRW
© Dr. Terry Straeter GDE Systems, Inc.
Independent DSB Reviewers
@ = Mrs. Joan Habermann Logistics Management Institute
e@ Mr. Philip A. Odeen BDM International, Inc.
Executive Secretary
e Ms, Virginia L. Castor ODDR&E/AT
\. pss Secretariat Representative
oe _CDR Robert C. Hardee DSB
(— Government Advisors >

OSD:
e@ Ms. Deborah Castleman OASD(C31)
e = Mr. Frank Kendall OUSD(A&T)
e Mr. Gene Porter OUSD(A&T)

~@ Mr. William Mounts ODUSD(AR)
e Ms. Linda Brown ODASD(IM)
e Mr. Chris Dipetto OUSD(A&T)
Joint Staff:
e@ Mr. Joseph Toma J6oA
Services:
e LTG Peter Kind Army
e@ RADM John G. Hekman Navy
© Mr. Lloyd K. Mosemann II Air Force
Agenci

e Ms, Belkis Leong-Hong DISA
e Dr. Ed Thompson ARPA

Task Force members represented a valued cross section of software expertise within both the DoD
and commercial sectors. The government advisors represented senior executives (including the three
Service Software Executive Officials) from the major software management organizations within the
Department.



Vendor Selection

Best Commercial Practice

Current DoD Practice

Solicit multiple (but not all) qualified vendors -a
selected few. Encourage teaming with a view to
attaining a long term relationship that covers the
entire life cycle and fosters trade-offs in cost and
schedule,

Solicit all possible vendors. Vendor proposals must
meet 100% of requirements. Teaming seldom
encouraged; development and maintenance usually
separate entities.

Compare vendor history and experience. Maintain
long-term relationships.

Can compare previous performance, but normally
can't have long-term relationships.

The organization that will be responsible for a
system over its full life cycle is heavily involved
from the beginning.

Maintenance organization not usually involved in
vendor selection process.

Use site visits and demonstrations to gain
knowledge of vendor capabilities.

Site visit only by capability evaluation team, or
other expert teams. Visits are very structured.

Negotiate for best values based on: (1) trade-offs of
costs and requirements licensing; and (2)
consideration of both vendor and buyer best
interests,

Negotiation based on lowest cost and shortest
schedules, endangering maturity of finished
product.

Overall goals: (1) obtain product at reasonable cost
as soon as possible; and (2) achieve the business case
for the system.

Overall goal: Obtain lowest cost product that
rigorously meets all requirements, but be fair.

Relatively few review and approval steps once
vendor is selected.

Review and approval process more structured and
complex once vendor selected.

Past performance weighted heavily (sometimes
primary factor) in selection process.

Past performance considered, but only as a minor
factor.

More flexibility in vendor selection based on metrics
and overall assessment.

Selection of vendor forced by use of pre-defined
metrics for proposal evaluation.

Modifications made as procurement proceeds in
order to get best results.

Change difficult once process begins.

Summ.
Very different processes with commercial much more flexible, but with no requirement for fairness, or to
maintain the public trust. Commercial encourages vendors to offer best solution, but solution may not
meet 100% of the requirements. Teaming and long-term relationships are more easily accommodated by

indus

C2



Software System Architecture (Cont.)

Recommendations

e@ Emphasize Use of Software Architecture
— Establish model and context for architecture selection
» Standards-based with emphasis on “implementable”
» Require vendors to propose, manage and control the architecture

— Require delivery of software architecture definition as first step in any
software acquisition

— Foster migration strategies at architecture level

e Assign responsibility within Government for domain analysis and product
line developments

e Provide expertise and resources to ensure coordinated DoD participation
in commercial/international standards bodies and users groups

The Task Force makes the above recommendations in order to facilitate greater use of software
architecture as a management tool for DoD software programs and activities. In particular, the Task Force
sees great value in requiring the delivery of software architecture definition as a first step in any software
acquisition. Where possible, such software architecture definition should be operational (i.e., executable).


3.3 DOD PERSONNEL

DoD Personnel

Findings
e Ashortage of sufficiently qualified software personnel
currently exists at all levels within the DoD

- Expertise for software acquisitions, software evaluations, and software
maintenance/support

Expertise to represent DoD (customer) interests with commercial sector
Expertise in domain software design and applications

Expertise in software technology to develop policies, standards, and
guidelines

Expertise in software program management

Based on the inputs provided, the Task Force found that a shortage of sufficiently qualified
software personnel exists at all levels within the DoD. Without personnel who are highly qualified in
modern software practices, DoD will not be as capable of effectively exploiting complex software within
its systems. This personnel shortage has been a major contributor to the problems that have arisen in past
DoD software development programs.


Overarching Recommendation (Cont.)

In Carrying Out This Responsibility, Consider Forming an Executive
Council

- Including the DDR&E, the ASD (C31) and Appropriate Representatives from
the Services and Defense Agencies

— Provide Supporting “Process Action Team” to Assist in Implementation

The Task-Force also identified the mechanism by which its overarching recommendation could be
readily implemented.


DoD Program Management (cont.)

Recommendations
e Establish Overarching Software Life Cycle Guidelines

- Tools/Methods
» Define Software Architectures to Enable Rapid Changes and Reuse
» To Achieve the Benefits of Using Standards-Based Architectures, DoD Must Manage Programs Using:
‘© Early system engineering
© erative development
‘¢ Proactive participation in development of these standards
» Promote Development/Use of Community-Wide Metrics and Models (e.g,, SEI’s Capability Maturity Model)

- Acquis mn
» Revise the Milestones for Software-Intensive Development
© Address the need fora softwate-first philosophy
© Provide fora layered software/hardware standards based architecture

‘© Acquisitioin and lifecycle planning should now separate hardware and software fieldings based on the business sense ofthe specific project
Require Early Interaction Between User, Acquisition Agent, and Developer; Identify and Get Early User Involvement
Apply Evolutionary Development with Rapid Deployment of Initial Functional Capability

Encourage Competition of Technical Approach vs. Cost

Provide Incentives and Guidelines to Encourage Software Reuse (Architecture-Based Reuse)

Reduce Documentation and Review Requirements for “Mature” Companies (i.e., Companies Determined to Be “Mature”
‘Through Evaluation Mechanisms)
Tailor operational testing to develop DoD “Beta Test” philosophy

‘© Allow fielding of software direct from testbeds with user consent

Have Program Manager Stay with Programs at Least Through Beta Testing to Maintain Continuity of Understanding of
Original Nuances in Requirements

The Task Force makes the recommendation that DoD establish overarching software lifecycle
guidelines directed at facilitating program manager employment of commercial practices and software and
that a DoD-wide effort be made to oversee implementation of these guidelines. The tools, methods and
acquisition approaches recommended by the Task Force are listed above.


DEPARTMENT OF THE ARMY
OFFICE OF THE SECRETARY OF THE ARMY
407 ARMY PENTAGON
WASHINGTON DC 20310-0107

Office, Director of Information
‘Systems tor Command, Control,
‘Communications, & Computers

MEMORANDUM FOR THE EXECUTIVE SECRETARY, DEFENSE SCIENCE BOARD
TASK FORCE ON DOD ACQUISITION OF COMMERCIAL SOFTWARE

SUBJECT: Proposed Action Plan with Regard to Common Changes Proposed by DoD
Speakers to Revise the Software Acquisition Process

The DSB Task Force asked us to provide a list of common recommended
changes needed to revise DoD's software acquisition process. Our recommended list
of proposed changes is provided in the enclosure to this memorandum. As we worked
together to develop this list, we found it useful to categorize the recommended changes
into five major categories. Several of our recommendations did not receive adequate
emphasis in our presentations, yet are important to the process we are studying. We
feel that if OSD is serious about addressing Software Acquisition issues, a single
senior office with primary responsibility for effecting the reforms proposed by the DSB
will be essential. Further, such an official or office must receive "from the heart"
sustainment and backing from the highest levels within OSD. The enclosed list
addresses our concerns with regards to the questions before the DSB. We hope that
this will be beneficial to the DSB Task Force and we all look forward to continued active
participation in this effort.

PETER A. KIND YD K. MOSE! Nil

Lieutenant General, GS Deputy Assistant Secretary

Director of Information (Communications, Computer, and Support
Systems for Command, Control, Systems)

Communications and Computers

Date 4 Feb 94 pate ¥4E6 2%

LS

IOHN G. HEKMAN
a Rear Admiral, SC, USN
- Commander, Naval Information

Systems Center

Date 4 Feb 94

Printed on @ Recycled Paper


1.0
1.1

Defense Science Board Task Force
on Acquiring Defense Software Commercially

TASK FORCE OVERVIEW
TERMS OF REFERENCE

Terms of Reference

Objectives:

e Determine:
- Conditions Under which Procurement of Defense Software Can Use Commercial Practices
~ Changes Required to Permit Such Use

e Develop Strategy that Incorporates Such Practices
- Not Constrained by Existing DoD Standards
- Viewed as Coexisting Alternative Strategy
- Includes DoD Use of Commercial Software Products

e Compare Proposed Strategy with Current DoD Strategy, Indicating
Circumstances Where Each is Most Beneficial

Scope:
e All Software Intensive Systems
e All Stages of the Software Life-Cycle

Appendix A provides the Terms of Reference by which the Task Force was established. The

objectives and scope of this Task Force are outlined above. At the first meeting of the Task Force, the
sponsor of the study (the Director, Defense Research and Engineering) requested that the Task Force
provide a strategy that: was sensible, pragmatic, and unconstrained by current DoD acquisition practices;
was based on evaluation of mechanisms for integrating defense software efforts with commercial software
efforts; did not require legislative relief; and addressed the full spectrum of DoD software applications.


Assessment of Current DoD
Software Acquisition Approach

i EEE

e Weaknesses
- High Life Cycle Cost in Time and Dollars
- Software Development Cycle Tied to Weapon System Developments
» Incredibly Long , Typically 13 to 15 Years from Concept to Fielding

- Encourages Excessive Acquisition Agent Involvement in Design
Detail and Process

— Based on Mistrust vs. Mutual Trust
» Excessive Documentation
» Excessive Formal Review
» Excessive Testing of Non-Critical Systems
» Poor Communication Between Vendor, Acquisition Agent and User
— No Requirements Addressing Cost and Schedule
- Traditional Approach Used: Design it All and Then Build it
— Little or No Requirements Relaxation for High Cost Items
— Inadequate Beta Testing in Early Phase

— Little Focus on Designing in Reusability

However, there are a number of weaknesses associated with the current defense approach, as
summarized above. Many of these weaknesses derive from the need for a fair and open procurement
process and the necessity to prove that public dollars are wisely spent.




APPENDIX D

PROPOSED ACTION PLAN FROM
SERVICE SOFTWARE EXECUTIVE
OFFICIALS


Proposed Software Life Cycle

Pre-Prososal Demos & Proposal & Demo
Market Analysis Simulation Capability

Evaluations Evaluations Version 1.0

Basic Functions

Release to Beta

USER Site then Users
Version 2.0

Improved Functions

Version 3.0
Improved Functions

To implement this recommendation, the Task Force proposed that DoD evolve a more appropriate
software life cycle approach as depicted above. This approach provides for early capability (even in the
initial bidding process) and for a gradual enhancement in capability over time.


Acquisition Approach

Developing the RFP : Contract Proposal
‘Qualified Software Organization
Domain Experience
SkilleMatix
Development Environment
Process Control
Peer Reviews _ :
‘Metrics and Milestone Plan

© RewsePlan

The Task Force proposed approach to competition is shown in the above figure. The core

government role in such an approach would be to:

Develop the RFP (no “How To” statement of work; no management CDRLs; no government in-
process approvals)

Provide an integrated “Top Level” specification (architecture, COTS/reuse, software engineering
environment and test/validation approach).

Employ software metrics as a key determinant,

Evaluate proposed technical solutions and the proposed management plan

The contractor would then:

ec ee ee ee

Provide an execution plan, management controls and progress milestones/metrics

Describe an in-place, mature software development organization and relevant domain experience
Provide a skills matrix describing personnel to be employed

Identify a robust development environment and describe applicable prior experience

Describe automated process control software

Describe the extent to which peer inspections will be used

Provide a metrics usage plan and purposes for which they will be used

Provide specific reuse and program management plans

Propose specific architecture(s) in executable code




Process Credibility

HN

Findings
e Requirements and Source Selection Inflexibility
~ Vendors attempt to meet every requirement
- Ease of protest
~ Requirements focus vendor on particular solution
= Requirements have become alternative to professional judgment
- Departures from requirements have caused protests and bad publicity

Price/Schedule/Functionality
~ In commercial sector, managers constrain 2 out of 3 (e.g,, cost and schedule)
= In DoD, managers constrain 3 out of 3

Constrained Communication During Solicitation
- Questions are provided to competitors (can give away proprietary concepts)
~ Questions are often misinterpreted and answered incorrectly
- Guarded way of asking questions limits substantive feedback

Complicated Regulations
~ Restrict variety of proposals
- Restrict competition and limit government options
— Consume time and resources
= Drive government employees to follow conservative procedures as safest path (inhibits “best value”)

Government-Unique Accounting Procedures and Audits
Audit requirements limit available vendors
Add substantial cost
Create need for separate accounting systems
Drive vendor to lowest labor cost solution rather than best value solution
Lead to rejection of good solutions based on value pricing vs cost-based pricing

The viewgraph above lists important hindrances that the Task Force sees with regard to the
adoption of commercial software practices. Many of the Task Force recommendations address these
hindrances.


APPENDIX C

COMPARISON OF SOFTWARE
ACQUISITION METHODS




OFFICE OF THE SECRETARY OF DEFENSE
WASHINGTON, D.C. 20301 -3140

30 JUN 1994
DEFENSE SCIENCE
BOARD
Dr. Paul Kaminski
Chairman, Defense Science Board

Office of the Undersecretary for Acquisition and Technology
The Pentagon

Washington, D.C.

Dear Dr. Kaminski:

Enclosed is the final report of the Defense Science Board Task Force on Acquiring Defense
Software Commercially. We were tasked to determine the conditions under which procurement of
defense software can use commercial practices and to define needed changes to permit such use.
In the attached report, we make specific recommendations with regard to DoD process credibility,
software program management, required expertise of DOD personnel using modern software
practices, use and integration of commercial off-the-shelf software, DoD software acquisition
practices, use of software architectures by DoD as a management tool, and DoD’s investment in the
software technology base.

Based on our review, we concluded that issues associated with defense software are
applicable across the wide spectrum of DoD software intensive systems. However, to reap
maximum benefit from improvements related to the myriad aspects associated with software, the
DoD requires a more coordinated approach to the oversight of its diverse software capabilities and
programs.

To provide such Department-wide oversight and to facilitate implementation of the other
specific recommendations contained within this report, the Task Force recommends that the
Secretary of Defense assign to the Under Secretary of Defense (Acquisition and Technology) the
responsibility for DoD-wide software technology, policy, practices and acquisition. This
centralized approach will best serve the Department for all software intensive systems and
programs. To support the implementation of this recommendation and to ensure that the key
stakeholders are participants in the process, the Task Force further recommends the formation of
an Executive Council consisting of the appropriate principals from OSD, the Services, and the
Defense Agencies.

We thank all of the members and government advisors of this Task Force for their
dedicated efforts and significant contributions to this study.

bok) Me ~be5\f EIT Ve Dasee

Latry Dfuffel VA 7 George -Heilmeier \
Co-Chairman Co-Ch:



4.0

4.1

SUMMARY

“ATTA PERSONS”

“Atta Persons”

Electronic Systems Command Software by Assembly of Modules

— Software “Component Store”
A-109 Acquisition Methodology

Use of Commercial Practices and Products in Reserve
Component Automation System (RCAS )

Software Reuse, Prototypes and User Involvement of Global
* Command and Control System (GCCS) Initiative

Perry-Paige Initiative Toward Migration Systems

The Task Force identified several ongoing efforts worthy of note.

The Air Force Electronic Systems Command (ESC) has defined an approach for software system
development through the assembly of pre-qualified software modules (PRISM). ESC is pursuing
the development of such software modules. The Task Force was very supportive of this program.

Office of Management and Budget (OMB) Circular A-109 outlines an approach for major Federal
system acquisitions that encourages: definition of top level needs vs. detailed specifications,
exploitation of innovative private sector contributions and use of early competitive demonstrations
of competing approaches. These all are acquisition attributes recommended by the Task Force.

The Reserve Component Automation System (RCAS) is an ongoing MIS development to support
the reserves. The program has successfully employed the A-109 acquisition approach and
extensively used commercial acquisition practices and products. The Task Force commends this
approach.

The Global Command and Control System (GCCS) is an initiative of the Joint Staff (J3 and J6) to
provide vertical and horizontal interoperability of combat information systems across Services,
Combatant Commands and Agencies. It was highlighted for its software reuse approach, its use of
prototypes and its emphasis on operational user involvement.

The Perry-Paige migration systems initiative has established a focus on selecting a set of target
computing systems (including MIS, C31 and embedded) towards which DoD will aim. This
migration strategy will enable a more cost-effective DoD investment in software across the life-
cycle.


3.7. SOFTWARE TECHNOLOGY BASE

Software Technology Base

Findings

e@ The current DoD software technology base does not
adequately take advantage of commercial R&D

Software technology transfer ( internal and external) is not
receiving adequate emphasis within DoD

There is a paucity of data to support prediction of cost,
schedule and performance

The software technology base (both defense and commercial) provides DoD with ample
opportunity for significantly improving the defense software acquisition life-cycle.


JOINT RECOMMENDED PROPOSED ACTIONS

I, DESIGNATE SINGLE. COMMON DOD SOFTWARE MANAGEMENT OFFICIAL
(ACTION OFFICE (USD))

A. ESTABLISH DEPUTY ASSISTANT SECRETARY OF DEFENSE
(SOFTWARE)

B. PROVIDE SUFFICIENT RESOURCES FOR MISSION

Il. SOFTWARE ACQUISITION AND LIFE-CYCLE MANAGEMENT (ACTION OFFICE
NEW DASD)

A. PROMOTE REUSE BASED ON DOMAIN MANAGEMENT
1. Define domains of interest or "areas of business"
2. Assign responsibilities to manage the domains
3. Establish and manage common architectures within the domains

B. REDUCE MONITORING OF MATURE COMPANIES
1. Identify criteria for assessing/determine process maturity
2. Consider maturity in source selection
3. Allow sole source extension for high quality vendors

C. PROMOTE GOVERNMENT/INDUSTRY TEAMING
1. Reduce Documentation Requirements to a minimum
2. Provide for electronic delivery/evaluation/exchange

D. MANAGE RISK
1. Mandate the use of market research
2. Use metrics effectively for management
3. Develop Risk Management Disciplines
4. Stick with a winner/punish poor performers

Wl. POLICY AND STANDARDS

. ADOPT COMMERCIAL STANDARDS (ACTION OFFICE(ASD(C3D))
1. DoD invests in commercial standards developments
2. Change FAR/DFAR and SD-1 as appropriate

B. REVISE THE MILESTONES FOR SOFTWARE-INTENSIVE
DEVELOPMENTS (ACTION OFFICE USD(A&T)/ASD(C3))
1. Address the need for a software-first philosophy
2. Provide for a layered software/hardware standards based architecture


The Software Domain
is Very Large

LARGE DOD EMBEDDED REAL
TIME SYSTEMS

CUSTOM
COMMERCIAL ‘SMALL DOD.
REAL TIME SYSTEMS EMBEDDED REAL
‘TIME SYSTEMS

‘COMMERCIAL DOD
ENGINEERING ENGINEERING
SYSTEMS SYSTEMS

mz

COMPLEXITY

CUSTOM DOD
BUSINESS SYSTEMS.

‘CUSTOM COMMERCIAL
BUSINESS SYSTEMS.

‘COMPLEX
(COMMERCIAL

PRODUCTS

wOorm<mu O4

COST TO DEVELOP

The software domain reviewed by the Task Force encompasses a wide variety of DoD systems,
ranging from software tools to large embedded real-time systems. The associated cost, complexity, and
time required to develop these systems vary widely, both for commercial and military applications.




Business Practices

Best Commercial Practice

Current DoD Practice

Informal contracting, joint ventures, partnerships
with mutual economic benefit and vested interest in
success.

Difficult to write contract to motivate contractors to
reduce cost; expensive to terminate contractors.

Oversight built on established relationships.

Burdensome cost accounting procedures required;
extensive oversight, reporting, and documentation
requirements.

Can hire and fire vendors and managers.

Government personnel regulations, policies, and
practices determine qualifications of its managers,
rotations of assignment, and training.

Multi-year effort and funding.

Multi-year effort. Yearly, unpredictable funding.

Summary

Commercial

practice more flexible with greater incentives.

Integration Testing and Delivery

Best Commercial Practice

Current DoD Practice

Unless system is for a new plant, then there are
major "cut-over" issues.

Similar "cut-over" or transition issues.

Sometimes difficult to assemble complete system in
laboratory environment due to cost.

Usually integrate system in laboratory prior to
operational testing.
Development testing vs. operational testing via
statutory mandate.

Beta testing widely used to quickly find errors.

Little beta testing.

Ultimate acceptance authority rests with buyer, not a
separate organization.

Structured, specified operational testing conducted
by separate authority. Acceptance authority rests
with buyer.

Buyer/user/vendor are a team.

Emphasis on DoD as oversight and approval
authority.

Summary
Integration and functional testing seem appropriate to the need. DoD use of separate test agency adds time
and complexity. Absence of beta testing increases cost to DoD.

row



2.2 COMPARISON OF DEFENSE AND COMMERCIAL SOFTWARE

/

oo

Comparison of
Defense and Commercial Software Needs

\

DoD and commercial applications for software are different in some ways but very similar in
others. The above table highlights these similarities and differences for real-time applications, command
and control applications, MIS systems and reusable components and products. This apparent disparity in
the classification of software between DoD and commercial vendors increases the complexity of DoD's
management task. Companies (people, methods, and organizations) are usually specialized to one or more
commercial type systems, not to DoD type systems. Defense and commercial software applications needs
are merging, providing the potential that DoD can exploit commercial capabilities more effectively over
time. Major revisions are needed in DoD software management across all DoD applications in order for

e Defense and Commercial Software Applications Needs Are Merging

Breadth of Typical Size Uperade/ ‘Sensitivity to
Application {SLOC) Change Errors
| DoD Real Time Software Unique 400,000 Complex; High
Inflexible
ams Tar Custom Software Integrated Unique 700,000 Complex, Tigh
Into Real-Time Operations inflexible
c dandc ica
cat Moving Toward 200,000 Complex; Flexible Moderate
Wide
[-~Commercial Custom Software Integrated| Moving Toward 200,000 Complex; Moving, ‘Moderate
Into Business Operations Wide “ Toward Flexible
IMIS Systems
| DoD Automated Information Systems Very Wide 200,000 Moving Toward Low
Commercial
= Commercial Automated Information Very Wide 200,000 Exploits Object Tow
Systems ‘Oriented
Technology
|Reusable Components/ Products
|- DoD Component Stores Very Wide 50,000 Tied to Weapon Low to High
System Cycle__ | (Depending on Use)
Commercial Shrink Wrapped ‘Very Wide 0,000 Tied to
Commercial Cycle

nN

—/

DoD to capitalize on the evolving commercial base.


OFFICE OF THE SECRETARY OF DEFENSE
WASHINGTON, D.C. 20301-3140

DEFENSE SCIENCE 30 JUN 1994
BOARD

MEMORANDUM FOR UNDER SECRETARY OF DEFENSE (ACQUISITION &
TECHNOLOGY)

SUBJECT: Report of the Defense Science Board Task Force on
Acquiring Defense Software Commercially

I am pleased to forward the report of the Defense Science
Board Task Force on Acquiring Defense Software Commercially. The
Task Force, co-chaired by Dr. George H. Heilmeier and Dr. Larry
Druffel, was chartered to determine the conditions under which
defense software could be procured using commercial practices and
to develop a strategy for procurement that incorporates such
practices.

‘In its investigation into applying commercial practice to
DoD software procurement, the Task Force reviewed a broad
spectrum of interrelated elements from software program
management policy and DoD software acquisition practices to DoD's
investment in the software technology base. I wholeheartedly
concur with the group's key finding that DoD must develop a more
coordinated approach to the oversight of its diverse software
capabilities and programs. The Task Force's report provides an
outstanding point from which to begin addressing revamping DoD's
software procurement practices.

Crut A. Hermunoke :

Paul G. Kaminski
Chairman


Recommendations-Acquisition

e Recommendations

- Implement the following acquisition approach:

» Establish acquisition focus on functionality and consistency with
“commercial best practice”

Revise procedures encouraging interaction between user and developer
and achieving early functionality

Minimize DoD regulations for review and documentation that are
different than “commercial best practice”

» Require planning for maintenance at beginning of development process
» Provide government funded vehicle in contracts to incentivize
development of reusable SW

— Review all existing military standards and military specifications
pertaining to software development and documentation, for
continued applicability, such as DoD-STD 2167

To this end, the Task Force makes the above recommendations with regard to DoD software
acquisition practices. In essence, these recommendations can move DoD toward an acquisition approach
that is more consistent with commercial approaches, while not requiring changes in statutes.


Comparison of
Commercial and Government Projects

Schedule/Time
Number of Projects if (Months)

[Commercial
|Government
Percent Difference

|Commercial
|Government

[Percent Difference

|Government

Percent Difference

|Government*
Percent Difference +13.46%

Summary of 15.34 40.76%
Average Percent Difference +15.54% +.
* Small sample may be statistically invalid.
Source: QSM, Inc.

This figures summarizes other quantitative indicators of the difference between commercial and
government projects (in terms of the typical size of the code, time to develop the application and overall
level of effort).

It should be noted that the Task Force was unable to find reliable, quantitative data supporting the
notion that commercial practices are more cost-effective than DoD practices. This lack of reliable
indicators was a major concern of the Task Force.


This report is a product of the Defense Science Board (DSB). The DSB is a
Federal Advisory Committee established to provide independent advice to the
Secretary of Defense. Statements, opinions, conclusions and recommendations
in this report do not necessarily represent the official position of the
Department of Defense.

This document is UNCLASSIFIED.

Security review completed 13 July 1994 by OATSD (Public Affairs)
Directorate for Freedom of Information and Security Review.
(Reference # 94-S-2796)


APPENDIX A
TERMS OF REFERENCE


3.4. USE AND INTEGRATION OF COMMERCIAL OFF-THE-SHELF SOFTWARE

Use and Integration of
Commercial Off-the-Shelf Software

Findings:
DoD Does Not Normally Perform COTS Market Analyses Incident to
Requirements Definition

DoD is Beginning to Exploit Evolutionary Development Approaches

Tools/Methodologies Are Evolving to Facilitate Use of COTS
Computer Aided Software Engineering (CASE)
Object oriented methods
Reuse repositories
Integrated product teams
Integrated risk management

DoD does not routinely perform COTS software market analyses during the requirements
definition phase of an acquisition program. Nor does it employ prototypes or simulations of new
capability in a way that could influence the requirement process. Rather, requirements are evolved in a
manner that is disconnected from the availability of commercial applications and are not typically
influenced by such available capability. This situation is true for both hardware and software. Today’s
technology can facilitate early interaction between users and available capability, particularly in software.
Further, DoD is beginning to exploit evolutionary software development approaches that could provide for
the inclusion of existing commercial software functionality into early prototypes and allow users to test
such capabilities. Modern software tools and methodologies have evolved that facilitate use of COTS,
such as those listed above.


Software System Architecture (Cont.)

Findings (Cont.)
e Little emphasis on architecture in DoD software programs or regulations

e Impact of software architecture issue

- DoD not benefitting from architecture as a key tool for:

Evolutionary development

Early (and often) involvement of users with functional capability
Ability to include changing commercial technology

Reuse

Facilitating requirements change with minimum cost and schedule
Facilitating product line management

e Insufficient Progress in:

- Developing models/standards for domain specific software architectures
- Open system architectures work

There is currently little emphasis on architecture in DoD directives or regulations. As a result, DoD
is not benefiting from architecture as a management tool. Further, the Task Force sees insufficient
progress in developing models and standards for domain specific software architectures.


Process Credibility

Recommendations

e@ Make necessary changes to acquisition regulations

Have Program Managers Manage 3 of 3 (Price/Schedule/Functionality) But Only
Constrain 2 of 3

Define Successful Performance on Contracts as Delivering Solution (with Predictable
Price, Schedule and Functionality) Not Adherence to Government Processes,
Procedures and Specifications

Do Not Require C-Level Specifications for Software Projects Developed in Ada

Prior to RFP, Government Should Perform Independent Market Analyses of Off-the-
Shelf and Contractor Products to Assure “Best Value” Solution

Establish Mechanisms to Allow Both Current Ability to Perform as Well as Past
Performance as Key Factors in Source Selection

» Require Source Selection Evaluation of Development Contractors Through a Formal Software
Process Capability Evaluation
Encourage Offerors to Demonstrate as Much Functionality as Possible as Part of Bid
Without Eliminating Domain Knowledgeable Competition
» Executable Architecture as a Minimum
» Weight Heavily in Selection

The Task Force makes the above recommendations with regard to process credibility.


Background: Impediments to Change

e DoD Software Management
No Single Operative Mechanism Exists to Implement Change
Three Separate Domains: MIS, C31, Embedded
Two Separate Acquisition Management Structures
Bureaucratic “Turf” Inhibits Real Reform
e Acquisition Process
5000 and 8000-Series Developments Typically Employ “Waterfall”
Approach; Not Incremental/Spiral Approach
e Culture
Systems Are Stove Pipe -- My System, My Program (PM is King)
DoD Acquisition Training Reinforces the Wrong Approaches

e Procurement
The Contracting Process Inhibits Creativity and Investment by
Contractors; Limits Options
Interpretation of Competition in Contracting Act

To ensure that its recommendations could be readily implemented by the DoD, the Task Force
identified the primary reasons why recommendations from previous studies had not been acted upon. The
Task Force then formulated its recommendations to appropriately address these impediments.






* Contracting: technical data rights, intellectual
property rights, liability, alternative forms of procurement
agreements, and incentives for creation of reusable software
components as well as their subsequent reuse.

The Director, Defense Research and Engineering will sponsor
this study. Dr. George H. Heilmeier and Dr. Larry E. Druffel
will serve as Co-Chairmen. The Office of the Director, Defense
Research and Engineering will provide the necessary funding and
support contractor arrangements. The Executive Secretary will be
Ms. Virginia L. Castor. Commander Robert C. Hardee will be the
Defense Science Board Secretariat representative. It is not
anticipated that this Task Force will need to go into any
"particular matters" within the meaning of Section 208 of Title
18, U.S. Code, nor will it cause any member to be placed in the
position of acting as a procurement official.

\John M. Deutch
\


Background:
DoD Software Acquisition Management

Two Different Sets of Software Policies/Rules Managed by
Two Different Organizations

— USD (A&T) for Software Embedded in Weapons/Systems

- ASD (C3I) for C3I Software and MIS Applications

Majority of Software Issues are Applicable to All DoD
Systems

Growing Similarity Between DoD and Commercial Software
Developments Across the Various Types of DoD Software

— Modern Software Tending to Blur Distinction Between DoD and
Commercial Applications

Central DoD Leadership Needed More than Ever

— Major Revisions Needed in DoD Software Policies/Rules and
Management Across All DoD Applications in Order to Meet SECDEF’s
Acquisition Reform Goals

— Interpretation and Implementation of DoD-Wide Policies/Rules by
Management is a Central Issue

Today, the management of DoD software acquisitions is quite complex. There are two sets of
software policies/rules managed by two different organizations

* USD (A&T) for software embedded in weapons/systems
« ASD (C3)) for C3I software and MIS applications

There currently exists within the DoD a dichotomous organizational structure for the management
of DoD software intensive systems. The Under Secretary of Defense (Acquisition and Technology) is
responsible for weapons systems; the Assistant Secretary of Defense (Command, Control,
Communications, and Intelligence (C31) is responsible for C3I systems and Management Information
Systems (MIS). However, as technology has evolved over the years, the issues associated with the use of
software in all of these systems have become essentially the same. The issuance of different software-
related policies and regulations that are oriented according to the type of system is no longer appropriate.
If the DoD is to adequately address this fundamental issue, central focused management is required.