A11103 08^523

NAFL INST OF STANDARDS & TECH R.I.C,

A1 1 1 03089223

Fife, Dennis W/Computer software managem

QC100 .U57 N0.500-, 11, 1977 C.I NBS-PUB CE & TECHNOLOGY;

COMPUTER

SOFTWARE MANAGEME^

A PRIMER FOR

PROJECT MANAGEMENT

AND OUALITY CONTROL

500-11

7

NBS Special Publication 500-11

U.S. DEPARTMENT OF COMMERCE

National Bureau of Standards
NATIONAL BUREAU OF STANDARDS

The National Bureau of Standards^ was established by an act of Congress March 3, 1901. The Bureau's overall goal is to strengthen and advance the Nation's science and technology and facilitate their effective application for public benefit. To this end, the Bureau conducts research and provides: (1) a basis for the Nation's physical measurement system, (2) scientific and

technological services for industry and government, (3) a technical basis for equity in trade, and (4) technical services to pro- mote public safety. The Bureau consists of the Institute for Basic Standards, the Institute for Materials Research, the Institute for Applied Technology, the Institute for Computer Sciences and Technology, the Office for Information Programs, and the

Office of Experimental Technology Incentives Program.

THE INSTITUTE FOR BASIC STANDARDS provides the central basis within the United States of a complete and consist- ent system of physical measurement; coordinates that system with measurement systems of other nations; and furnishes essential services leading to accurate and uniform physical measurements throughout the Nation's scientific community, industry, and commerce. The Institute consists of the Office of Measurement Services, and the following center and divisions: Applied Mathematics — Electricity — Mechanics — Heat — Optical Physics — Center for Radiation Research — Laboratory Astrophysics" — Cryogenics- — Electromagnetics - — Time and Frequency^.

THE INSTITUTE FOR MATERIALS RESEARCH conducts materials research leading to improved methods of measurement, standards, and data on the properties of well-characterized materials needed by industry, commerce, educational insti- tutions, and Government; provides advisory and research services to other Government agencies; and develops, produces, and

distributes standard reference materials. The Institute consists of the Office of Standard Reference Materials, the Office of Air and Water Measurement, and the following divisions:

Analytical Chemistry — Polymers — Metallurgy — Inorganic Materials — Reactor Radiation — Physical Chemistry.

THE INSTITUTE FOR APPLIED TECHNOLOGY provides technical services developing and promoting the use of avail- able technology; cooperates with public and private organizations in developing technological standards, codes, and test methods; and provides technical advice services, and information to Government agencies and the public. The Institute consists of

the following divisions and centers: Standards Application and Analysis — Electronic Technology — Center for Consumer Product Technology: Product

Systems Analysis; Product Engineering — Center for Building Technology: Structures, Materials, and Safety; Building

Environment; Technical Evaluation and Application — Center for Fire Research: Fire Science; Fire Safety Engineering.

THE INSTITUTE FOR COMPUTER SCIENCES AND TECHNOLOGY conducts research and provides technical services designed to aid Government agencies in improving cost effectiveness in the conduct of their programs through the selection, acquisition, and effective utilization of automatic data processing equipment: and serves as the principal focus wthin the executive branch for the development of Federal standards for automatic data processing equipment, techniques, and computer

languages. The Institute consist of the following divisions: Computer Services — Systems and Software — Computer Systems Engineering — Information Technology.

THE OFFICE OF EXPERIMENTAL TECHNOLOGY INCENTIVES PROGRAM seeks to affect public policy and process

to facilitate technological change in the private sector by examining and experimenting with Government policies and practices in order to identify and remove Government-related barriers and to correct inherent market imperfections that impede

the innovation process. THE OFFICE FOR INFORMATION PROGRAMS promotes optimum dissemination and accessibility of scientific information generated within NBS; promotes the development of the National Standard Reference Data System and a system of in- formation analysis centers dealing with the broader aspects of the National Measurement System; provides appropriate services

to ensure that the NBS staff has optimum accessibility to the scientific information of the world. The Office consists of the following organizational units:

Office of Standard Reference Data — Office of Information Activities — Office of Technical Publications — Library —

Office of International Standards — Office of International Relations.

1 Headquarters and Laboratories at Gaithersburg, Maryland, utiless otherwise noted; mailing address Washington, D.C. 20234.

' Located at Boulder, Colorado 80302.
ANDAJRDS

1977

COMPUTER SCIENCE & TECHNOLOGY:

Computer Software Management: A Primer for

Project Management and Quality Control

Dennis W. Fife

Systems and Software Division

Institute for Computer Science and Technology

National Bureau of Standards

Washington, D.C. 20234

U.S. DEPARTMENT OF COMMERCE, Juanita M. Kreps, Secretary

Dr. Sidney Harman, Under Secretary

Jordan J. Baruch. Assistant Secretary for Science and Technology

i^liMATIONAL BUREAU OF STANDARDS. Ernest Ambler, Acting Director

Issued July 1977
Reports on Computer Science and Technology

The National Bureau of Standards has a special responsibility within the Federal

Government for computer science and technology activities. The programs of the NBS Institute for Computer Sciences and Technology are designed to provide ADP

standards, guidelines, and technical advisory services to improve the effectiveness of

computer utilization in the Federal sector, and to perform appropriate research and

development efforts as foundation for such activities and programs. This publication

series will report these NBS efforts to the Federal computer community as well as to interested specialists in the academic and private sectors. Those wishing to receive

notices of publications in this series should complete and return the form at the end

of this publication.

National Bureau of Standards Special Publication 500-11

Nat. Bur. Stand. (U.S.), Spec. Publ. 500-11, 58 pages (July 1977) CODEN: XNBSAV

Library of Congress Cataloging in Publication Data

Fife, Dennis W.

Computer software management.

(Computer science & technology) (NBS special publication ; 500-1 1) Bibliography: p. Supt. of Docs. No.: Ci3. 10:500-1

1. Electronic data processing—Management, 2. Programming

(Electronic computers)— Management. I. Title. II. Series. III. Series: United States. National Bureau of Standards. Special publication ; 500-11. QC100.U57 no. 500-11 [QA76.9.M3] 602Ms [658'.05'4042] 77-608127

U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON: 1977

For sale by the Superintendent of Documents, U.S. Government Printing Office, Washington, D.C. 20402 - Price $2 Stock No. 003-<X)3-O1796-7
PKEFACt.

Ttie National Bureau of Standards is tne

primary technical advisor on computers tor

government agencies (Brooks Act, Public Law

&9-3k)6) . NBb assists Federal agencies in specitic

projects, ana so obtains first-hand experience

with agency problems and competencies. This guide

was prepared drawing upon extensive contacts with

government computing personnel, and using the

published recommendations of authorities m

software management. A variety of executive

guides and primers in other fields were reviewed

in choosing the scope and approach to the material

covered here. uther staff members ot trie

Institute for Computer Sciences and Technology

assisted by reviewing the material as it was

developed. The author was helped particularly by

the constructive comments of I. Trotter Hardy ana

Theoaore Linden.

NOTICE TO READERS

This guide is intended to become a standard

reference for managing Federal government software

projects. Comments by the readers on its

completeness, clarity, etc. would be valuable in

making appropriate revisions for this purpose. A

postage-free reader evaluation form is included at

the back of the report, and you are encouraged to

complete and return it.

-iii-

TABLE OF CONTENTS

Page

Why this guide was written 1 Who this guide is for 2 What this guide is for 2 What this guide is not for . 3

How to use this guide 3

Why software management is important 3

Why every agency needs software management .... 5 IBasic tasks of software management 7

#The qualities of superior software 13

Organizing for software management 15

Minimizing early problems 21

Guidelines for design and programming 29

Guidelines for testing 33

Avoiding endless software problems 38

The need for software production tools 40

Standards for quality control 42

Fitting controls to the project size 45

Final homilies 47

References 48

Section with specific guidelines.

-V-
List of Figures

Page

1. The Life Cycle of a Software Product 10

2. Chart of Basic Quality Control Documents ... 11

3. A Representative Team Organization 19

4. A Representative Organization for

a Large Project 20

5. Chart of Recommended Minimum Milestones ... 28

6. Chart of Recommended Quality Controls .... 46

-vi-
COMPUTER SOFTWARE MANAGEMENT:

A primer for project management

and quality control

Dennis w. Fife

Today, providing computer software involves

greater cost and risK than providing computer

equipment, because Hardware is mass produced by

industry using proven technology, wnile sottware

IS still produced mostly Dy tne craft of

individual computer programmers. This brief guide

IS intended tor managers who are responsible for

computer projects, to explain the use of quality

controls and software management methods. The

typical problems of software development are

explained. Over twenty distinct quality controls

are defined, and recommendations are given for

software management actions. Empirical

information is included that would help top

executives to appreciate the potential problems

and importance of software management.

Keywords: computer management; computer

programming; computer project control; computer

software; software engineering; software quality;

software reliability.

Whi Trilb oUlbE wAb WRITTEN

Computer software is the computer instructions or programs, as well as the data files and descriptive manuals,

that nave to be provided to maKe computer equipment

(hardware) perform useful services. Today, providing

software involves greater cost and risk than providing the

equipment, because hardware is mass produced by industry

using proven technology, while software is still produced

mostly by the craft of individual computer programmers and

users.

-1-
Software management is the technical and management

control throughout the life cycle of all software that

supports an organization's mission and objectives. Software

management is effective when high quality software is being

obtained with reasonable and controllable costs. Software

management is inadequate when poor decisions and lax

technical controls result in faulty software performance,

computer service fiascos, or project cost overruns. The

unabated high cost of software as well as many publicized

cases of software catastrophes are convincing evidence that

software management is poorly understood and practiced in

most organizations.

The problem is deep-seated and serious, because many

managers who control software projects do not have

professional computing expertise, and many programmers lack

the technical sKills to properly design and produce the

Highly complex computer software needed today. Equally

important, the expansion of computer applications over the

last twenty years has far outpaced the evolution of basic

principles and engineering discipline in software design and

proQuction. Much more applied research is needed before

software engineering will progress to the effectiveness

found in other engineering and production fields.

WHO THIS GUIDE IS FOR

This guide is intended for managers and staff who are

responsible for computer projects. These will be data

processing professionals and supervisors primarily. But the

intended audience also includes executives and managers who

were not trained as computer professionals and who perhaps

have never used a computer themselves. Every reader will

need to understand the rudiments of computer operation and

programming.

WHAT THIS GUIDE IS FUR

The objective of this guide is to give the reaaer an

appreciation of the software pitfalls in computer projects,

and to explain tne use of quality controls ana software

management methods. The typical problems of software

production are highlighted, and recommendations are given

-2-
for setting up software management procedures.

AriAT This bUIDL IS NUT FOK

The brevity ana special purpose of tnis guide mean tnat

It is unsuitable for otner important purposes. In

particular, tnis guide is NOT a survey of programming

methods, NOK a handbook for the direct supervision of

computer programmers. Its use cannot compensate for a lack

of technical expertise that is essential to direct

management of software design, development, and maintenance.

Finally, this guide is NOT a popular treatment to arouse the

interest and concern of the general public. It is expected

that readers have a professional interest and obligation in

decision-making for software projects.

HOW TO USE THIS GUIDE

The sections marked by (#) in the Table of Contents

contain specific recommendations and guidelines for

effective software management. This material should be

utilized to develop appropriate administrative oroers or management procedures to implement the recommended practices

in all software projects. Selected references for further

stuay of the more important topics are given at the end of

this guide. Aaaitional sections of this guide contain

information that woula benefit a senior executive in

unaer stand ing the problems ana importance of software

management.

WHY SOFTWARE MANAGEMENT IS IMPORTANT

Software is a very expensive resource for government.

The procurement and development of computer software is

"big business" for many organizations, including the

government. Estimates of Federal government spending range

as high as as $5 billion annually on software, and the

accumulated investment in current Federal software probably

-3-
exceeds $25 billion dollars. Software costs greatly exceed

equipment costs over the useful life of computer services,

and the initial development costs for government software

systems are growing to enormous levels. For instance, cost

estimates for the newly proposed Tax Administration System

of the Internal Revenue Service are well over $500 million

dollars, and the Air Force has estimated its software costs

for command and control systems in the 1980 's will be

several billion dollars.

*The government software inventory is large and complex.

Among the more than 8,000 government computer

installations, many have 500 or more programs for specific

government tasks, in addition to others acquired from the

computer manufacturer for general support of computer

operation. Government computer applications, which include

sophisticated military systems as well as public service

systems handling millions of accounts, are typical of the

most challenging uses of computer technology within the

sta te-of -the-ar t

*Software flaws and mismanagement can be ruinous to

government projects and public services, and even to

national goals.

Government and business are completely dependent on

computer services today. Manual methods are no longer

considered because human means cannot be organized to handle

the volume of data and short deadlines that are needed in

widespread public services. Social and economic activity is

geared to the performance of computer systems. Examples are

many, as in airline reservations systems or the Social

Security Administration systems that distribute over $50

billion annually to about 25 million individuals. These

benefits are possible only with effective management of the

computer software. In recent years, recurring cases of

major software failures have demonstrated the serious

consequences of software mismanagement to legislators,

journalists, and the public. One dramatic case occurred in

1975, when a government agency made more than $500 million

in overpayments to individuals as a result of erroneous

computer programs.

*No technology or standard practice now exists that will

surely prevent faulty design, logical errors, cost overrun,

or late delivery for any software project.

The preparation and maintenance of computer software is

a laborious, human task that can require the coordinated

effort of dozens or even hundreds of programmers. The

-4-
possibilities for program errors, poor design, or poor

decision-making may be higher than in any other technical

field, because of extreme technical complexity and the lack

of standards and common practices in this new technology.

The speed ana volume of data handling by the computer

creates disastrous consequences from errors in programs and

accentuates the difficulties caused by only minor design

flaws. Tnere is no practical method Known that will

guarantee software as free from errors or design flaws.

Althougn programming productivity has been improving, the

aavances have not kept pace with the tremendous growth m

computer applications ana the aramatic gains in performance

versus cost for computer hardware over the past twenty

years. There are very few standard computer programs for

common applications. There are no standards covering common

work practices and technical goals, such as the measurement

of software reliability. Thus, every software project is a unique effort. Success or failure is determined more by the

individual technical skills and management capability of the

personnel involved, than by any other factor.

WHY EVERY AGENCY NEEDS SOFTWARE MANAGEMENT

Software management must be the responsibility of the

computer user organization.

Over 7S% of all software is developed by the users of

computers, rather than the manufacturers of computer

equipment or the producers of commercial software. Although

the computer industry is working to improve software

management for its products, industry cannot be expected to

provide a comprenensive solution, nor to cover the unique

circumstances of each computer user,

Software design and programming is a complex technical

activity that must be directed effectively.

Although many people nowadays may understand the

rudiments of programming from college or even high school

courses, the simple homework exercises in such courses do

not represent the "real world" of professional practice.

The design and production of numerous programs that must

work together for a major service, such as the payroll of a large business, is not routine effort that can go

unsupervised or be done by junior personnel. The funds and

personnel resources involved may be a major investment for

the organization, and the technical choices may drastically

-5-
affect non-computer work processes in management, clerical,

and client relations areas. Technically complex programs

may demand advanced professional SKill which is very scarce

and must be carefully airected at the critical technical

problems. Management and technical control by a professional software designer is essential for resolving

complex design issues and for directing programmers to

produce a specified product within cost and schedule

targets

Software management establishes accountability for project

decisions and objectives, and provides upper management with

visible measures of progress toward desired goals.

Software projects usually are initiated by managers in

such areas as payroll, engineering, plant and supply, etc.

Although they may control the project budget and schedule,

and have the best information on potential benefits and

requirements, they are often unprepared to conduct a design

project or to judge the results provided by an outside

technical group. Thus, software designers sometimes worK

independently using their own judgement, with little

pressure to reliably estimate their progress and the

remaining work or costs. Software management mcluaes

cneckpoints to assure that important objectives and issues

are carefully evaluated by the concerned parties, especially

m regard to cost , qual i ty , and scneaule, and tnat users and

designers take their appropriate responsibility for tne

decisions made.

Software management provides the technical controls and

resource management to produce high quality software within

acceptable funding and time limits.

Software management emphasizes technical criteria and

working procedures to resolve the technical issues that are

critical to the success or failure of a project. Major

questions commonly arise concerning the feasibility or

performance of a desired system, the attainable quality of

the system considering the resources available to produce

it, and the methodology for design and testing in order to

achieve the most effective software product. Software

management delineates quality controls, standards, planning

factors, and experience data which help designers and

programmers to organize and direct their efforts efficiently

in solving such problems within available cost and time

1 imits

Software management controls costs after software operation

beg ins
The useful life of software for a significant service

typically is over eight years. Experience has shown that

continual changes are necessary in software, to correct

previously unknown errors, to improve useability or

performance, or to include new functions required because of

legal, administrative, or technical changes. As much as 70%

of software cost occurs in this redesign and maintenance,

after software has been delivered and put into use. It is

important to recognize that maintenance is no less

cnallenginy than development, for change to programs

involves new aesign work ana snould be aone accoramg to

planned cost and time goals. software management

t^roceaures, consistently appliea throughout the sottware

life, will maKe best use of limited funas by stressing close

management of effort toward tne most needea sottware

capabil 1 t les

.

BASIC TASKS OF SOFTWARE MANAGEMENT

Software management consists of all the technical and

management activities, decisions, and controls that are

directly required to purchase, produce, or maintain software

throughout the useful life of a computer system or service.

Software management is primarily concerned with

computer programs, data files, and documentation, rather

than the computer equipment configuration that executes the

programs. Even so, software designers must have a say in

nardware decisions if they are to meet the goals for a computer service. The term "system" will be used here for

the combination of hardware and software that provides a computer service. Since software has little purpose without

haraware, the terms "system" ana "software" may be used

in terchangeaoly without causing confusion.

ine basic activities ana aecisions are best introducea

in relation to tne commonly recognizea pnases of a system's

life cycle, snown in tigure 1. Tne INITIATION pnase

involves the assessment of an existing, inadequate system,

in order to determine the feasioility of replacing it witn

an improved system. The four pnases, DEFINITION through

TESTING, are the development period, where a new or improved

software product is being designed and implemented. The

OPERATION phase starts with tne operation of a new system by

Its intended users, and includes the subsequent maintenance

and minor redesigns that user experience may dictate.

Eventually, maintenance no longer suffices for needed

-7-
improvements, and another development cycle is begun.

Specitic guidelines for each phase will be given after

the following summary of the tasks included. Figure 2 helps

to understand each phase by summarizing the documents

produced that are needed for quality control.

*In the INITIATION phase, software management assures that

the technical designers understand the purpose and scope of

the envisioned system, and adequately perceive its required

functions

The INITIATION phase begins when management recognizes

that a need exists for a new or improved computer service,

and that a study should be made to formulate a software

solution. During this phase, software planners or system

analysts identify the intended users and investigate their

work processes. The planners describe the services needed

from the envisioned software, and then conceive and outline

alternative software solutions. A close working

relationship with the intended users is essential. The

alternative approaches should be described in basic terms

that users can understand. User concurrence with the stated

requirements and the recommended software approach is

mandator y

*The INITIATION phase assures that the recommended software

approach is technically and economically feasible.

An estimate must be made of the costs and benefits for

eacn of the potential software solutions. This will lead to

a recommended solution and provide data regarding

feasibility. Comparable existing systems are investigated

to derive cost data and to confirm that proposed solutions

are technically realizable. Any pitfalls, failures, or

questionable design areas in comparable projects should be

covered thoroughly. Comparisons of the proposed project to

past failures or problem areas would assure that they can be

avoided or resolved. General specifications must be written

to show clearly the scope and character of the recommended

solution. The INITIATION phase concludes with a report

delineating the recommended software and the supporting

analysis. Management may then approve the concept and

budget the recommended funds and personnel for acquisition

and operation of the planned system.

*The DEFINITION phase defines the functional and performance

requirements, in order to confirm that the software, when

built, would meet its objectives.

-8-
The DEFINITION phase begins when management commits

resources to the design or purchase of the proposed

software. In this phase, detailed specifications are

created for those externally apparent functions and design

characteristics that are crucial to the purpose and

operation of the software in the intended environment.

Input ana output data, processing operations, and

performance goals are defined. Tne Functional Requirements

define wnat the software must do ana the general aspects of

how It will oe constructea, rather than the details. The

DLFIWITIUN pnase concluaes with a specification that is

sufficiently precise for outsiae contracting, if uesirea.

Equally important, the spec ii ica tion allows the intended

users to confirm that tne proposed system will meet tne need

ana will nave acceptaole qualities of useability,

generality, etc.

The DEFINITION pnase provides a project plan for managing

the acquisition and installation of the system.

A thorough plan for producing or purchasing the

software is prepared as part of the DEFINITION phase.

Projections are made of the detailed costs. A delivery

schedule is laid out, including intermediate milestones and

technical reviews for evaluating progress and insuring that

the project will meet its goals. Working methods are

defined for quality control of intermediate and final

products. The plan should extend to the installation and

initial operation of the system, and should include user

training, document preparation and review, acceptance

testing, and the continuing obligations of any contractors,

vendors, or support organizations. The Project Plan is

extremely important oecause it specifies how the effort will

be managea, how problems will be resolvea, and how the

quality of the final system will be assurea.

-9-
Figure 1. Ttie Lite Cycle ot a software irroauct

Time

Time

Major technical activity
Figure 2. Chart of Basic Quality Control Documents

Life Cycle Phase Documents General Content

Initiation Feasibility Study Report Requirements analysis, definition and evaluation

of alternative solutions, recommended software

concept.

Definition Functional Requirements

Quality and Performance

Requi rements

Project Plan

External specification of software functions and operation, including design constraints.

Desired design attributes and quantitative performance parameters for key processing operations.

Cost and work breakdown, detailed schedule including

quality assurance milestones, and definition of primary management and quality control methods.

Design Design/Coding Standards

Design Specifications

Test Plan

Detailed design attributes and coding conventions

for qual ity

Software architecture, module definitions, interface

specifications, and programming requirements.

Testing criteria, testing schedule and work breakdown,

standard test case specifications.

Programming Review Reports

Unit Test Reports

Inspection Reports

Software Manuals

Program Listings

Identified discrepancies and recommendations from team reviews.

Results of unit test for each module.

Results of inspection sessions for each subsystem.

Usage, operation and maintenance descriptions.

Source Language statements for each tested module.

Testing Test/Fault Report Results of scheduled tests.

Operati on Faul t Report

Specification

Maintenance Plan

•swmn-t-nmc r 1 rri jm*; ffl nr P"; nf dP'f"Prted or SUSDeCted

failure.

Functional and design specifications for maintenance

and improvement work.

Work breakdown, cost estimate, and schedule for proposed rework.

-11-
*Tne DtiSluN pnase produces a tliorough design spec it ica t ion

tor guiding a timely ana pr ODlem-f r ee implementation ot the

proposed sottware.

The DtSIbW phase begins when the Functional

requirements have been validated by intended users, and

implementation of the system is approved. The DbSIbiM phase

results in detailed specifications of the internal

construction of the software, for use by programmers who

will implement the design. The DESIGN phase involves

definition of the internal architecture of the software,

performance trade-off analyses of algorithms, specification

of individual programs or modules, specification of

interactions between components of software, etc. The

Design Specifications are important as a means for resource

management and quality control during PROGRAMMING.

*In the PROGRAMMING phase, software management assures that

appropriate technical choices are made, and that high

quality workmanship is applied in producing code meeting the

Design Specifications.

The PROGRAMMING phase often is merged with DESIGN and

so may have no distinguishable starting point. PROGRAMMING

involves final technical choices for internal program design

and the creation of programming language statements or

"code" that is executable on the computer and that

implements the design. Software management here is the

aay-to-aay supervision of programmers, and guiaing and

reviewing their technical results. PROGRAMwINb effort

includes the testing by each programmer of his individual

programs and the preparation of pertinent management reports

and software aocuments. Test reports, inspection reports,

ana yzo^ram documentation act as tangible quality controls

over PKUGRAWhINu accomplishments.

*In the TESTING phase, software management assures that no

significant errors or design discrepancies will impede usage

of the system m the intended operational environment.

The TESTING phase is concerned with verifying that the

software product meets the functional specifications and

contains no significant discrepancies or logical errors to

impede its use. TESTING proceeds according to a Test Plan

prepared in the DESIGN phase, and focuses on the integration

of individual programmer results in an overall working

system. Noted discrepancies and errors are documented for

corrective action by the programming team. Gross

shortcomings in the original design or functional

specifications may be exposed; this should lead to a major

project review in order to plan and organize additional

-12-
definition and design work.

*ln OPERATION of a delivered system, software management

assures that maintenance is as well managed and controlled

as original development.

Tne OPERATION phase follows the delivery and

installation of the software in the field, and involves

periodic redesign and improvement. Software management

procedures that are recommended during development can be

streamlined to provide similar close control of the reduced

effort committed to corrective design work. Thorough

specifications, quality workmanship, effective testing, and

management review remain highly important in directing

tecnnical effort to meet cost and schedule targets.

buring the development period, the earlier activities may

oe continuea or re^jeatea in oraer to remedy aesign aefects

ana improve quality or tne final system.

Aitnougn tne successive phases defined nere should be

loiioweo whenever practical, some flexibility is aavisable,

especially wnen a project involves unusual complexity or

innovation. For example, it may be important to carry only

a part of the system through to DESIGN and PRO(jRAi'U*iIN(j , m

oraer to confirm feasibility before beginning DEFINITION for

tne entire system. Or, with limited effort available,

aeveiopment of some supporting capabilities or auxiliary

programs may be deferred until the mainstream processing is

successfully completed. So, all parts of the software need

not be at the same stage of development, and the life cycle

orientation may be viewed with some flexibility.

THE QUALITIES OF SUPERIOR SOFTWARE

These general properties are commonly accepted as

characteristic of high quality software. The techniques of

software management may be usea to control quality at

acceptaoly nign levels or to maximize quality insofar as

practical witfi tne resources ana time availaole for a project

C0KKt,(^TwL6i3--progr ams perform exactly ana correctly all

tne functions expectea from the specifications, if

availaole, or else from the documentation. This means tnat

incorrect aocumen tation is as serious as an incorrect

program. Correctness is an ideal quality, that is rarely

-13-
determinable, so a more practical quality is RELIABILITY.

RELIAblLITY —Programs pertorra without significant

detectaDle errors all the functions expected from the

specifications or the documentation. high reliability

inaicates that programs are relatively trouble free m

performing what they are claimea to do. t)Ut an equally

important question is whether tne functions ana performance

are adequate and suitable to a needed purpose. The latter

quality is callea VALILITY.

VALIDITY —Programs provide the performance, all

functions, and appropriate interfaces to other software

components, that are sufficient for beneficial application

m the intended user environment. This means the software,

without additional programming or manual intervention, has

the capabilities that reasonably would be expected for its

purpose. Validity is a quality of specifications as well as

computer programs. Examples of an invalid program would be

an interactive editor that had no online function for

retrieving stored text for inspection, or a FORTRAN language

compiler that had no DO loop implementation. Validity

involves judgement of user requirements, and may change if

the intended application or purpose is altered. Because

poor reliability may render a needed function useless,

reliability is necessary to validity.

RESILIENCE (or ROBUSTNESS) —Programs continue to

perform m a reasonable way aespite violations of the

assumed input and usage conventions. Input of unacceptable

data, or an inconsistent command, should never cause a result that is astonishing ana detrimental to the user--sucn

as tne aeletion of any valia results obtamea previously,

programs should include routine cnecKs and recovery

possibilities that are "forgiving" of common user and aata

errors. Resilience is related to the broader quality of

Ui>bat>lLlTY . USeABILITY —Programs have functions and usage

techniques that are natural and convenient for people,

showing good consideration of human factors ana limitations.

For example, the programs have few arbitrary codes for data

in input or output, have consistent conventions in different

operating modes, and provide thorough diagnostic messages

for errors or violations of use.

CLARITY —The functions and operation of the programs

are easily understood from the user manual, and the program

design and structure are readily apparent from the listing

of program statements. This means that documentation must

be well written, but also that the program is carefully

-14-
aesignea, witn meaningful cnoices of variable names, use of

Known algoritiims, frequent ana effective comments in the

program to descrioe its operation, and a modular structure

tnat isolates separate functions for examination.

MAINTAINABILITY —Programs are well documented by

manuals and internal comments, and so well structured that

another programmer could easily repair defects or make minor

improvements. Clarity is essential for maintainability,

but also implied are a wide variety of good design

attributes, such as program functions that help to diagnose

potential problems, e. g., periodic reports of status or control totals; or, general techniques that can be readily

adapted for change, e. g., the isolation of constants,

report titles, and otner static data as named variables.

iHOLiIFlAbILITY--Frogr am functions tnat might require

major change are well aocumented and isolatea m distinct

mouules. naintainabil 1 ty is essential to moo it iaoii i ty

out mod If laoil ity means tnat a concertea erfort was maae to

anticipate major cnanges, ana to plan the software aesign so

tnat tney coula oe maae easily.

oLi>*bKALlTi--f rogr ams perform their functions over a wiae ranye of input values ana usage moaes. Programs are

not limitea to special cases or ranges of values, when the

functions are commonly or reasonaoly extenaable to a more

general case.

POKTAbILITY--Progr ams are easily installed on another

computer or under another operating system program,

btanaard programming language is used, and hardware or other

software dependent features are isolated for easy change.

TESTABILITY —Programs are simply structured and use

general algorithms, to facilitate step-by-step testing of

all capabilities.

LFFICILNCY/LCONOMY —Programs have high performance

algorithms and conservatively use computer resources, such

as main storage, so that the cost of program operation is

low

Ui\oAwlZlWU tuK ijUFTWAKL nANAvjLi'ibNT

*A project team is tne basic organization for sottware

man ag erne n t

-lb-
The team approach is effective because a mix of needed

skills can be quickly assembled from an organization's

available talent, and the most talented designers can be

consistently utilized on the toughest problems. Team

practices provide better oversight and counseling tnrough

peer reviews, better dissemination of tecnnical and status

information about software modules, and better back-up of

personnel. The project team approacn will be an unusual

practice where computer personnel have been organized

according to specialties such as "operations", "system

analysis" and "systems programming". Nevertheless, the

advantages gained by forming an appropriately skilled team,

dedicated to one product objective, warrant a departure from

the traditional organization. A project team may be formed

temporarily to carry out software development, with further

effort after installation handled in the normal

organizational framework. However, the advantages of team

work apply to maintenance effort in the OPERATION phase as

well as to the development phases. So a project team may be

advisable when major maintenance or improvement activities

are done.

*For projects involving up to about 10 personnel, one

individual should be responsible for the entire system

design and the project management.

This concept has been called the "Chief Programmer"

approach, for the individual must be an exceptional

programmer as well as a very competent manager. Control

tnrough one individual, especially from ulFInITIOn through

Tt-STiNb, offers a better possibility of a consistent, nigh

quality product, than would cooperative work oy several

individuals or smaller teams. The team chief personally

defines and designs the entire system, and produces many of

the critical modules. The cnief exercises complete

authority over the technical results and team production,

he or she represents the project with users or customers and

with higher management, and decides the project commitments

made to them. The chief controls personnel assignments and

schedules, and has supervisory responsibility for all team

members. The team chief usually should have a minimum of

five years of diversified, intensive experience in system

analysis and software design, and should have demonstrated

the personal qualities, such as judgement and maturity, that

are needed for effectively supervising other specialists and

for dealing with clients and superior officials.

*The project team should include an assistant chief,

specialists for key problem areas, and supporting personnel.

-lb-
ine assistant team cniet substitutes tor tne cniet

wnenever necessary, and serves as his primary technical

aavisor. ine assistant may personally oetme, aesign, and

program some parts ot tne system. Ahen needed, particular

specialists become team members to handle difficult design

problems requiring tneir unusual knowledge and experience.

In PRuGRAwHING especially, a programming language specialist

should be available to answer questions about the effect and

performance of various instruction sequences, and to suggest

better ways of programming certain algorithms. Specialists

on program production tools, testing, and data management

techniques are advisable also. The team secretary assists

in clerical tasks aimed at improving the team productivity,

such as the preparation of computer runs, purchase of

computer supplies, handling of project records and software

documents. Additional support is advisable also in

documentation and technical writing, and in administrative

support of tne team on personnel matters, training,

management information support, etc. Figure 3 depicts a representative team structure.

*ream techniques are appropriate also tor very large

projects

When faore tnan aoout ten people are needed to uo a project, one individual cannot effectively supervise all of

them, so tne design and programming must be distriDuted

among several teams. L>4ever tneless , one manager should oe

given authority over tne entire effort, anu snould maKe the

fundamental decisions on worK oreakdown, resource

allocations, and software design. inis project manager and

his subordinate programming managers could be viewed as

comprising a team to carry out system design and project

management. Their working procedures should be similar to

those within a subordinate programming team; for instance,

conducting periodic design conferences where each team chief

presents the designs and other results of his group. The

project manager should exercise considerable technical and

design leadership, and major aspects of overall system

design would be decided by his team. Support arrangements

similar to a programming team are advisable, as suggested in

Figure 4.

*Team techniques are applicable in managing contracted

developments

A team approach is advisable also when the software

development will be accomplished principally by private

contractors. The project manager or team chief will serve

then as the contracting tecnnical monitor. The contracting

team, rather tnan designing and producing programs, will

-17-
prepare the technical statements of work, and other

contractual requirements such as the quality and performance

specifications, and the pro3ect plan in regard to mandatory

schedules, deliverable products, and quality controls.

Technical specialists in particular areas such as testing

and performance evaluation are also necessary on the

contracting team, and special administrative support is also

advisable to improve the quality and timely completion of

contract requirements, technical reviews, acceptance

testing, etc., as the contractors proceeds witn tne

aeveiopmen t

*agency-widfc direction ana evaluation of software policy and

quality control is needed m addition to individual team

luanay emen t

h.n upper level of software management authority is

advisable to provide consistent direction and support for

individual teams with regard to organizational policies,

standards, and objectives in software. Such an individual

or office can contribute significantly in assessing software

technology for project needs, and in reviewing project

accomplishments with a view to standardizing team

procedures, technical aids and software production tools,

and even specific software components that have wide utility

for the organization.

-18-
irigure j. A representative Team urganization

Chief Programmer

Deputy Chief

Programmer

Administrator

Technical | Writer 1

Speci al i sts

0 0 0 0

Up to about 10

0 Language expert

0 Testing specialist

0 Tools specialist
Figure 4. A Representative Organization for a Large Project

Chief Designer

and

Project Manager

Deputy

Desi gner/Manager

Project

Control

Office

Documentati on Group

Tool s Group

I Chief

Programmer

Teams

1

1

» Testing f

Group

1 1
*;30ftware problems originate most frequently in tne analysis

ot requirements and tne design of tne computer software.

Nearly 50% of software flaws and errors exposed m

final testing or initial use of a new system are traceable

to shortcomings and mistakes in the early stages of a project, i. e. from INITIATION through Dt,SIGN. Most

authorities suggest that added effort in the early stages is

the best way to improve the quality of software. The effort

up to the start of PROGRAMMING should comprise about 40% of

the total development effort. A greater allocation is

advisable, provided working methods and review techniques

are geared to strong improvements in the validity and detail

of the Design Specifications.

*The primary quality controls for the INITIATION and

DEFINITION phases are system requirements documents and the

technical reviews made of them by the parties concerned.

quality control in these early stages is necessarily

juogementai, out will be aiaed considerably by stanaardized

documentation practice. Tne National oureau or Standards

nas prepared guidelines, FIPS PUb 38, tnat indicate the

content ror functional requirements, project plans, and

otner software documents. Additional guidelines ana

stanaaras wili be issued m the near future. The use oi

computer-aided text editing and document formatting will oe

especially helpful in rapidly accommodating numerous small

cnanges in specifications during repeated and lengthy

rev lews

*otart a Project Record with the INITIATION pnase and

continue it over the system lite.

The Project Record should be an open record of the

major decisions and of the pertinent documents and

specifications. It should be official, to clearly indicate

firm commitments, yet openly available, so that inadequacies

can be detected as early as possible and so tnat current

goals and constraints are apparent to all. This record is

distinct from a technical notebook, which also is often

effective as an informal medium for exchanging tentative

proposals among designers and programmers.

*In the INITIATION phase, identify all user tasks to be

supported by the software and the important attributes of

those tasKS.

-21-
The documentation of the INITIATION phase should

characterize the working environment in which the proposed

software will operate. This would include a description of

user tasks to be supported, the present methods and computer

aids for carrying out these tasks, and the present

limitations that the proposed system would overcome. The

description should be oriented for the users' review, so

that they may identify omissions or shortcomings of the

proposed software solution.

Maintain a continuing interaction with the users and their

managers during the development period.

The potential users or their managers are authorities

on the working environment m which tne proposed system must

tunction. They can provide vital information on the

acceptaole performance and design goals for the system, such

as input data characteristics, toleraole response time, etc.

Their strong support and involvement increases tne

likelihood of successful planning and design, and helps in

oDtaming needed requirements data and evaluations of the

t>roposed system. A highly recommended approach is to have

selected users serve as members of the project team, not

necessarily to perform design worK, but to help to decide

and to document requirements. Project milestones should

include scheduled review sessions as the requirements and

alternative solutions are evolved, to obtain user comment

and concurrence.

*S tr aightf oward automation of existing manual systems may

not yield the greatest benefits from computers.

The computer's capability to do complex processing of

large volumes of data at high speed generally means that

computers can provide better analytical assistance to human

decision-makers than could any number of clerical people.

The way in which data records are organized for manual

handling also may not be the best to use in computer storage

and retrieval. The INITIATION, DEFINITION and DESIGN phases

should include research toward new algorithms and data

handling procedures that allow the computer to serve as more

than a high-speed clerk and file cabinet. For example,

there are mathematical procedures that can greatly improve

inventory controls, scneduling for scarce resources, and

plans tor performance of complex tasks. Also, there are

well proven computer techniques that provide rapid retrieval

of the very data needed for a human task, rather than

requiring people to manually hunt through long computer

print-outs of all related data. Improved algorithms,

carefully chosen for each application, can increase human

and computer productivity by many times.

-22-
*Kadicai aavances in automation carry exceptional

r isK--exper imen t with new approaches through prototype

software, ana oe preparea to throw away one or more systems

oetore confirming the design.

Too often, the cause for software management failure is

overambition or executive zeal to bring the benefits of

automation to a pedestrian manual operation in one great

leap forwara. Never undertake to produce in a single

project step a system for which there is no technological

precedent. Keep the scope of systems within the bounds of

your experience with successful pr o j ec ts--ad vance your

system capability in modest increments.

*use consultants ana visit other organizations to identify

potential t'roblems in building a system and to verify that

system oojectives are attainable.

^jystems tnat nave any possibility or challenging tne

bta te-ot-the-ar t must be planned cautiously. uesearcners,

consultants, ana organizations wno nave pioneered an

innovative type or system can help aetine tne major pitfalls

ana now tney may oe avoioea.

stematically evaluate available software of a comi->arable

t^pe, wnetner commercial or other-user developed, oerore

maKing a ririii commitment to develop a new system.

ine feature analysis technique is very useful for

evaluating availaole software, and comparing proposed system

requirements against the current state-of- the-art. Feature

analysis proceeds by decomposing the requirements into

inaividual functions ana attributes, and then developing

snort aescriptions of what the available software provides

tor each feature. The descriptions should be organized in a table for easy comparison. Available software, if adequate

to the need, can usually be purchased for much less than the

cost of producing similar programs. Even if new development

is required, existing software may serve for some components

and reduce overall system cost.

*i:ormulate ana evaluate alternative solutions tnat differ m

tne risK anu cost of development.

ir'ruaent management requires a looK at other choices

Detore aecidmg upon one system to be oought or built,

bmce software cost ana risK are now so important, tney

naturally are Key cons laer at ions m conceiving of competing

at^proacnes. ine possiDility of meeting tne need tnrougn

commercial camputmg services or other sources of availaole

sortware certainly should oe presentea as one alternative.
It feasible. No definite guidelines can be offerea on how

to formulate alternatives. Clearly, thorough knowledge of

the state-of-the-art is essential and will indicate

candidate solutions that are more or less innovative.

Estimate the future system costs by several techniques and

adopt a conservative projection.

There is no well formulated cost estimation method that

is highly favored and widely used. Various published data

exist on programmer production rates, but different

variables are involved and some important factors are

omitted in one case or another. BROOKS (see References)

gives a concise summary of published data. YOURDON and

WALSTGN also have data showing the gains possible with

recent innovations such as structured programming.

Experience and judgement, however, are the most accepted

means to reach an overall recommendation on the time and

funding needed. Published data should certainly be used as

a guide, noting carefully the pertinent factors included.

Examine costs of coraparaole systems and advertised costs of

commercial packages for added eviaence. Do a detailed cost

oreakdown of the system by major components and by major

technical activities or work processes. Include operation

and Development costs, to have a full life cycle investment

comparison of alternative systems. Above all, recognize the

uncertainty of any early estimate of cost, and choose a conservatively high, yet reasonable figure. Reevaluate the

cost estimate periodically as the development proceeds and

better definition of the software evolves.

*aave the intended users review and approve the requirements

and recommended solution defined in the INITIATION phase.

User concurrence, along with any evaluations and

recommendations from the user review, should be in the

record concluding the INITIATION phase. This could be as

simple as initials in a project notebook for a very small

project, or may require an official report and approval

letter for a large project. To reach their assessment,

users should be given a cost and benefit analysis of

alternative systems, in addition to the general

specification of each.

Resist generalizing the system capability to accommoaate

requirements that cannot oe resolved before DESIGN begins.

If an imposea schedule will require starting DESIGN

before ail requirements are defined, a judgement must be

made of wnether the uncertainty allows useful effort to

proceed. If it aoes, then the unknown requirements must be

-24-
anticipated and the most likely possibilities defined. The

impact of these alternatives on the system design must be

determined, and if great enough, it may be necessary to

proceed with definition and design of several systems until

the unknowns are eliminated. Be cautious about generalizing

the required system capability to handle arbitrary, unknown

oeraands. (generalized systems often involve great risk in

aevelopment, and may have poorer performance and efficiency

tnan more constrained and specialized designs, be forceful

in investigating potential needs, m order to resolve

requirements questions and uncertainties as early as

possible

*Use engineering analysis, modeling or simulation, and

experience data to establish the performance needed in

critical software operations.

Software designers often assume that necessary

performance can be attained by "tuning" programs once they

are running. On the other hand, early decisions, for

instance on the hardware, the programming language, or the

principal algorithms to be used, could be so poor as to

eliminate any hope of reaching a needed level of

performance. Engineering analysis, coupled with

measurements from comparable existing software, should be

used to investigate the expected system performance as a function of the attainable performance for basic processing

steps. Performance goals should be specified for the

critical operations, to support feasibility judgements and

to serve as validation criteria for the DESIGN and

PKOGRAMMINCj phases. Early decisions that constrain the

design or implementation approach for the software should

snow clear evidence that the necessary performance would be

achievaole

Include witn tne Functional Requirements the performance

and quality objectives that the delivered software must

meet.

ihe specification of quality criteria should begin in

tne ubFlNlTION phase in order to have a basis for judging

tne subsequent design. Use the definitions given above as a starting point, to be expanded and specialized for a given

project. For example, identify the system functions that are

most subject to future modifications and the nature of the

required raodif labil ity . As another example, describe such

useability characteristics as the processing points where

recovery procedures are required and the acceptable form of

recovery. Include the quantitative performance goals for

principal algorithms and processing steps. In the Project

Plan, define the technical reviews and the evaluation

-25-
procedures tnat wiii oe used to assure tnat tne quality and

performance oDjectives are met througnout tne project.

Develop tfie Functional Requirements in a structured format,

to expedite review and validation.

A free-form narrative approach to documenting

requirements will make it difficult, except for very simple

programs, to isolate individual functions and performance

figures for later confirmation of design. A better approach

is first to decompose overall requirements into major

functional groups, such as data entry or report generation,

or into processing modes, such as online query or

transaction auditing. Then describe individual requirements

under each group with a separate statement of a function

that must be provided or a performance or design attribute

that must be achieved. Each requirement should include

criteria for confirming its achievement, the testing or evaluation methods involved m confirmation, and any

comments or discussion that amplify the requirement

statement or snow its origin.

*Do not conclude the DhFIwITIUN pnase without having an

approved specitication tnat incorporates cnanges and

reiinements emerging from user reviews.

wever proceed into detailed design when ambiguity or

Known errors exist in the requirements specifications,

basic disagreements are more liKely to lead to tundamentdl

redesign than to minor and easily rendered change.

Formulate concrete, measurable milestones of progress for

the design, programming, testing, installation, and Initial

operation of the system.

It is impossible to exercise control and take

corrective action without concrete evidence of where

problems lie. This means that progressive milestones must

be conceived covering the life of a project. They must be

events and accomplishments that are directly observable by

the software manager, such as the receipt of a specified

document, the successful operation of a given program, or

the conclusion of a scheduled review meeting. They must be

sufficiently frequent over time to serve as progress

indicators or trouble alerts if not completed. The

dependence of later milestones on earlier events shoula be

revealed by drawing up a graph or network relating all

milestones. Progress during DESIoiNi and PKOGRAWWING should

De measured by requirea events, such as reviews or

Specifications, and not Dy programmer estimates ot percent

of code completea. The accompanying cnart describes a -2b-
minimum recommended set of milestones over a system's life

cycle

Establish controls and milestones tnat will identify and

correct errors or omissions as early as possible.

The cost of correcting an error or adding capability to

a system increases significantly as a project proceeds. by

the time system testing begins, errors are over ten times

more costly to fix tnan if tney naa been recognized in

aesign. Adaitional time given to improving specifications

ana to tnorougnly critizing tnem is wortnwnile in reducing

later t^roject costs.

*Pian at least of total project effort for management

requirements ana tecnnical reviews.

when tne neeaea effort nas oeen underest imatea , wnicn

too often IS the case, careful reviews ana management

analysis may be thrown asiae to put all effort into design

and programming. Obviously this will worK to the detriment

of quality objectives, ana probably will result in even more

time required for testing and rework later. Make sure that

project estimates include the time and effort for detailed

review of work as it progresses, and for continuing review

of project schedules and resource expenditures. Insist on

using this time as intended, so that project status and

problem sources can always be accurately defined.

Plan contracts for design and production only after tne

detailed Functional Requirements and Project Plan have been

prepared ana approved.

Contractors cannot be selected or effectively monitored

without the quality assurance and validation criteria that

are emooaiea in these aocuments. They also proviae the

first practical oasis for aecomposing the development effort

into worK packages that different contractors mignt

unaertaKe. contract requirements particularly snouia call

tor stanaaraizea aocumen tation ana coorainatea tormat among

tne tunctional Kequirements and the succeeding aesign

specifications, to aia in tracing and correlating results as

tne eifort proceeds.

-27-
figure b. Chart ot Kecommended i'tinimum hilestones

Time

INITIATION DEFINITION DESIGN PROGRAMMING TESTING OPERATION

N

AAAAAi\ A AAAA AA AA V

Description of Milestones

1. Initiate Project Record with plan for feasibility study.

2. Complete analysis of user tasks to be supported.

3. Complete definition of alternative software solutions.

4. Complete feasibility study report with recommended solution.

5. Obtain user concurrence with recommended solution; budget approval

for project. ,

6. Complete Functional Requirements.

7. Complete Project Plan.

8. Complete user validation of Functional Requirements.

9. Complete software architecture and definition of major modules.

10. Complete Test Plan; and Design/Coding Standards.

11. Complete Design Specification of all component programs; tools definition,

12. Complete validation of Design Specification.

13. Complete unit testing; accept all modules for source code control.

14. Initiate fault reporting procedures.

15. Complete integration testing.

16. Complete delivery and shakedown.
GUIDELINES FOR DESIGN AND PROGRAMMING

*The second major source of problems lies in the quality of

programs as they are produced.

About 40% of software errors are traceable to

misinterpretations of specifications, errors in designing to

them, or mistakes m writing program statements. Recent

research to improve software management has concentrated on

these problems. A variety of new techniques, such as

structured programming ana automated quaiit;/ controi, are

Deginning to come into widespread practice, but much more

must be aone. misinterpretation of specifications would be

reduced by more formal or matnematical specification methods

rather tnan exclusive use of narrative English. Analytical

metnods of design, replacing tne subjective criteria used

neretoiore, would reduce tne frequency of programming errors

and costly maintenance. Software managers must Keep current

with such developing areas in software engineering research.

Continuing training programs, special in-house worksnops,

and experimental projects are good avenues for stimulating

the computer staff toward improving their working practices

and technical knowledge.

*0n beginning the DESIGN phase, prepare an explicit

specification of the quality design and programming

standards that must be observed in the DESIGN and

PROGRAMMING phases.

The Functional Requirements should be accompanied by

definitions of mandatory quality objectives, that serve as a starting point. Using these, a list of design attrioutes

can be specified, extending in detail to conventions on the

use of programming language features and constraints for

individual program design. This expansion of quality

specifications therefore will encompass the Coding Standards

to be used in the project, and will provide a framework for

starting design of individual modules. For example, the

quality standards may define the extent of diagnostic

messages to oe provided, the types of data errors to oe

detected uy validation algoritnms, the manner of isolating

particular processes subject to future modification, etc.

Develop the Design Specifications as a hierarchy of

processes, specifying the input and output of each process,

and the data flow interconnections.

-29-
A fiierarchy is an ettective means to decoiTipose

tunctionai requirements into successively more aetaiiea

operations. ine approacii is well matched to wnat is called

"top-down" desiyn tor software, wnicn proceeds Dy successive

retinements ot general functions, until arriving at the

component modules or programs that must De implemented to

Duiia tne system. Documentation tor this tunctionai design

approacn is rather simple, and it portrays the overall

system operation in an easily followed graphical way. The

References include books ana reports on the technique.

Complete the Design Specifications before beginning to

coae

.

It is vital that the design be completely worked out

before programming begins, so that programming effort is not

wasted on erroneous design notions and so that programming,

when started, has a complete guide to assure quality work.

Decompose the system into program modules that are simple

and easily understood, usually meaning about 100 program

statements in size.

The advantages of small modules are many. Their

functions are more clearly and simply describable. This

helps direct the programmer toward a clear objective. Their

brevity improves clarity and speeds review for validity and

quality. ir'rogress may be more accurately evaluated when the

units of programming work are small ana readily understooa.

Define general service moaules to reauce duplication m

program modules.

In designing the system into modules, keep track of

replicated functions ana aefine modules that provide such

functions as services that all programs may use. These

service modules usually are those that manipulate standard

data structures in a system of programs, for example,

entering items m tables, retrieving items, managing storage

allocation, etc.

*Use standard high level programming languages.

The use of vendor -un ique programming language,

especially machine or assembly language, should be

prevented, except for very small portions of code where

assembly language may be essential to achieving the

performance goals of the software. These exceptional cases

should be strictly controlled by explicit approval. Keep a standard high level language version of assembly programs

for documentation and maintenance assistance.

-30-
*Use structured programming tecnniques.

Structurea programming is a concept that encompasses

programming team management, design methods, and programming

technology. Numerous books and articles are available,

describing the design and programming techniques especially.

Published results from evaluation projects confirm that

increases in programmer productivity and reductions in

program errors are achieved, often as factors of two or greater. Investment in the training and management

orientation needed to practice these methods will be

wor thwhile

*laentity or develop programming aids to increase team

proauctivity , ana issue guidelines to the team tor using

them

ipeciiic r ecommenaations on programming tools are given

later. selection of tools ana specifications tor unique

tools neeaea lor tne project snoula oe aone as early as

possible in tne ut-hlNITlQw ana DtiiGN phases.

*^^onauct weeKly reviews by tne programming team of

individuals' worK.

inis practice helps all team members to Keep informed

of the design and production as it proceeds, and may act

also as a training opportunity. But primarily it focuses

the combined talent of the team on the quality of the team

output as a whole. Errors or inferior design can be more

quickly exposed than with reviews by a single manager. The

results of each review session should be documented as a guide for the individual programmer to improve or correct

his work. In structured programming, these review sessions

are called "structured wal k-throughs" , to suggest that each

programmer leads the team through the design and operation

of his module in the same pattern as it was developed.

*ijesign anJ program tor quality before performance.

oefore programmers make any effort to tune their

moaules for improved performance, all quality

cnar acter istics required shoula be realized. wecessary

comments ana program arrangements tor clarity,

maintainability, moo it laoil i ty , generality, ana all requirea

lunctions snould be completea ana contirmea in reviews.

*tor large projects, conauct inspections at successive

stages of aesign ana production, and make all necessary

changes before work moves to tne next stage.

-31-
In addition to team reviews of each programmer's

progress, more formal inspections are recommended tor large

projects, to examine subsystems or closely related modules

produced Dy different programming teams. Tnese inspections

siiould serve as progress milestones in tne Project Flan,

rne inspection team may include experts outside of tne

teams, in addition to tne team cniefs and tne responsiole

programmers. Tne inspection team leader (or moderator)

snoulu not nave oeen a participant in producing tne

subsystem under review ano must nave the authority to inform

nigher management of negative findings. A team size of

about ten persons maximum is aovisaole, and separate reviews

snould be made when design is complete, when code and unit

tests are complete, and when first integration tests are

comple te

.

*Use Iterative review techniques for validation.

Validation, also sometimes called verification, means

confirmation that a system is reliable, and meets its

specifications as well as the requirements of the user

environment. Team reviews or inspections should always

trace design specifications back to functional requirements,

taking note of findings from past reviews. In this way, the

logical thread that stems from the original specification is

followed to the working software, ano successively examined

for continuity and possible shortcomings. Reviews should

evaluate the suitability of the software for the intended

use, and whether the necessary quality is being achieved.

*in the FKUGRAMWING phase, the individual programmer

performs "unit testing".

unit testing determines that the individual programs

work m isolation from otners and meet the specifications

given with the assignment. unit testing must oe performed

according to standard criteria, ratner tnan ad noc methods

contrived by each programmer. The programmer provides test

reports for use in review and inspection tasks. Testing

criteria are described in the guidelines tor TbSTINb, which

follow

.

*Put program modules under source code control when unit

testing is completed.

Completion of unit testing signifies internal project

delivery of a module for integration with others in the

overall software system. Source code control restricts

further program changes to formally approved changes that

have been decided from an overall system viewpoint as

necessary to meet project goals. Source code control is

-32-
neeaea tor best use of programming effort in the terminal

period of tne development schedule.

*Before the DESIGN phase is complete, a formal test plan for

tne system should be approved.

The Test Plan should be started in the DEFINITION

phase, for the Project Plan must account for the effort

required for the testing strategy to be used. Some details

on the number of tests, test inputs, etc. cannot be

specified until the design is nearly complete. The Test

Plan describes the TESTING phase effort and the testing

schedule, and defines the standard test set, i. e. the

complete range of test cases that will be repeatedly used to

assure reliaoility of the delivered software. The

development of this standard test set is a significant

effort that must be completed by the start of TESTING. The

test cases must be provided by manual calculations or other

independent means, so that the correct output expected is

Known beforenana. other means may include program

analyzers, simulations, taoles oi ranaom numbers or function

values, results of comparable systems, etc. The aeptn of

tne lest i'lan is inoicative of the reliability tne user may

expect 11 testing is successfully completed. laeally, the

Test i'ian is aeveloped by a tecnnical group tnat is

inoepenaent of tne designers and programmers, ano that can

require tests that focus on perceived weaK points m the

design. users should be represented in test planning, and

tne final plan should oe jointly agreed oetween users and

the chief designer.

GUIDELINES FOR TESTING

Delivered software for large systems typically has errors

at a rate of one m every 3\diu program statements.

wnen iEoiii>4«o oegins, tne errors present may be three or more times nigner than tnis rigure. Testing is the process

of executing programs with representative input data or

conditions, tor wnich tne correct results are known, to

determine wnether incorrect results occur. Testing provides

information that helps to isolate errors. Debugging is tne

- j J-
subsequent work of a programmer to explicitly identify

errors and to correct them by changes to program

instructions or data. Many large program systems never

become error-free, i. e. "correct". Instead, they are

maintained and managed m terms of the number of known

errors that are allowed to exist--i.e. successive versions

or releases are planned to reduce the known errors to some

acceptaole, but perhaps still nign rigure.

*iesting is the only practical way to detect program errors

so that they may oe eliminateo.

Testing only determines if errors are present. It

cannot verify tnat no errors exist in a program. bucn

confirmation is conceivaole m only two ways. One is to

exnaustively test tne program on every possible input. Lven

for simple programs, this could take millions of years even

If done automatically by a computer. The second conceivable

way IS to prove correctness as a logical and mathematical

theorem about the program instructions. This approach is a current research topic in computer science, and proofs have

been done manually for simple programs up to a few hundred

instructions. Research work is now aimed at simplifying

proof requirements through programming practices suited to

this objective, and through computer theorem-proving

software that would reduce the complex mathematical effort

involved. No break-through is predicted in the near future

that would make proof -of -cor r ec tness practical ana

economical for the average programmer to use on normally

complex programs.

*Adopt specific criteria to govern unit testing of all

programs.

wany aifterent testing criteria are conceivable, out

most nave no direct relationship to possible errors ana

aesign flaws, ana so proviOe no information to improve

reliaoility. For example, one could require a speciric

number of test runs per program. but unless eacn test was

appropriately distinct, no more benefit would be obtained

tnan from running only one test. A ranaom sample among most

or a program's input values may be an effective criterion,

but It IS useful only if the valid input values are small m

number, so tnat a significantly large number of random tests

can be generated and run within the time and resources

available. Other possible criteria are:

a) test every instruction at least once;

b) test every branch of the program;

-34-
c) test every ioyicai path within tne program

witn at most one iteration or every loop.

(A logical patn is a sequence ot instructions, some possioly

repeatea, trom input to termination ol a program.) ine above

tnree criteria are successively more stringent; they are

preferable to many others because they are well airectea at

possible faults, and tend to be efficient m using test

ef f or t

Recommended testing guidelines that are broadly

applicable to software cannot be simply stated. The subject

is technically complex and beyond the scope of this guide.

The National Bureau of Standards is currently preparing a guideline that will include results from current research to

develop testing theory and to lay down broadly effective

principles. The most important consideration at this time

is to avoid arbitrarily chosen tests by each programmer,

that do not clearly exercise previously untested functions

ana logic.

*bcneaule progressive tests to build up to a representative

full system test.

Testing snoula begin with discrete tests of oasic

system runctions tnat are neeoea in oraer to go more complex

ana realistic tests. Simple input, output, ana control

functions are examples. Keeping each test restricted to a given function allows the results to clearly identify

problems encounterea. ultimately, tne software should be

exercised oy composite test cases that represent full system

operation and expected usage conditions. Composite tests

Should be conceived to stress the software performance. The

limits on such factors as accuracy, acceptable volume of

data, or number of concurrent users, should be verified if

included in the specifications. The diagnostic features of

tne software should be exercised as well with test cases

having invalid input. The Test Plan should encompass most,

if not all, of the capabilities defined in the Functional

Requirements

*Use program analyzers to assure that all program functions

have been exercised.

A program analyzer is a couiputer prograai that collects

data about another program, including data about it as it

operates. in particular, an analyzer can determine whether

test inputs nave caused all the program instructions to oe

used. This verifies that all capabilities have been

exercised, and identifies extraneous cooe that may have no

purpose and should oe removed.

-j5-
Experience indicates that testing usually requires about

40% of available development time.

Careful, disciplined design should reduce the need for

extensive testing, but more theoretical development and

experience with improved techniques will De needed to assure

this IS always so. Certainly, better specifications would

eliminate rework that sometimes is initiated during TESTIMG

when it becomes clear that the design was incomplete or

haphazaraly contrived. but, it seems best to be

conservative in project scnedules as well as cost estimates,

so allow a major portion of project time ror testing and

uesign rework.

*Use a fault reporting process to manage deougging and

testing

.

It IS good practice to have a formal report system by

which Detected errors ana discrepancies are recorded and

fully described. These reports will help to confirm that

all known errors are fixed before delivery. They also help

to trace multiple instances of the same anomalous behavior,

so that debugging assignments can be made for related

problems and the debugging effort reduced.

Conduct regression tests after program rework has been

done

.

The TESTING phase is primarily concerned with

integration testing, i.e. tests to determine whether

different program modules, produced rather independently,

will work together correctly. /(hen errors are found and

eliminated by debugging, new undetected errors may be

introduced. Regression testing means that previously

successful tests are rerun to detect any introduced errors.

A complete, successful run of the entire standard test set

should be accomplished on tne software oefore concluding

iE^Dl'lNG and delivery to the customer.

Supplement integration testing with system validation

reviews.

Although specifications are expected to be of nigh

caliber when PKUbRAmniNG begins, practical use of the

resulting system during TESTIwG may reveal that original

notions of desired operation were fallacious. Reviews of

test sessions should consider the general suitability of the

system as well as strict adherence to discrete functional

specifications. Any discrepancies should be considered for

rework, rather than delivering a system that in the

judgement of users and designers will not be adequate for

-36-
ttie user's t^urpose.

*lesting snouid assure a niyn measure ot reiiaoiiity tnat

wouia oe expectea in operation ot computer programs.

Correctness is tne aesirea quality tor software, but

also IS an ideal condition that otten is not economically

attainaole or contirmaole for complex software. So, tne

term " r el laDil i ty " is useful to denote the lacK of detectea

errors with respect to some basis of observation.

Observations could be made over a long time perioo of

repeated use of the software, and reliability then defined

as the ratio F/N, with F the number of detected faults, and

N the number of runs or trials of the program.

Software reliability measurements will reflect how the

software is being used, as well as the faults or errors it

contains. If routine usage is limited to previously

debugged functions, reliability will apparently be high. If

usage changes, reliability may suddenly degrade seriously,

as previously unexercised code is used and its errors

revealed

Reliability measurement is a recent subject of software

engineering research. i*iore results are needed before a minimum acceptaole figure can be recommenaea. The most

effective use of reliability oata m project management also

IS not certain until more project experience is at nana.

*k'ian a snaKeaown period atter delivered software is

installea

A shakeaown period may be dictated by hardware alone if a new computer system is involved, but this kind of trial

use may also effectively supplement the previously performed

integration testing of software. The shakedown scenario

should be as close to the actual anticipated usage as costs

and procedures permit, but do not depend upon the shakedown

operation for any operational service. Use the fault

reporting process and records of shakedown run time to

measure and evaluate reliability. The shakedown period

needed to attain stable and reliable operation may be

several months, particularly if hardware installation is

extended over this time and considerable user training is

being done.

-37-
AVOIDING ENDLESS SOFTWARE PROBLEMS

*Use the management techniques for development during the

OPERATION phase as well.

The programming and design ettort available during

OPERATION IS generally reduced compared with tne development

phases. however, the goals of software inanagement ao not

change; indeed, maintenance costs may dominate the total

life cycle cost of a software product. Maintenance worK on

software is still a design cnallenge, but one constrained by

tne program structure ana instructions alreaoy in use.

unless technically controllea, maintenance may aegraae

reliaoility ana otner qualities tnat were initially present,

maintenance tenas to aisorganize and complicate tne prograiii

structure, increasing the potential for errors ana tne cost

of future cnanges. It should not oe assuaiea that the

existing software is a sufficient constraint to prevent

unsupervised maintenance from Decerning costly or haphazara.

Maintenance should be conducted under a tight management

regime that includes oefinition ana scheauling of manageable

work assignments, preparation and review of specifications,

team review of design and code, standard tests, and

documen tat ion

.

*Keep a failure and discrepancy reporting process to

continually manage software reliability.

Failure reports are normal in the TESTING phase and the

initial period of OPERATION, but they continue to be

important as maintenance continues over the years. These

reports are needed for sottware managers to accurately

aetermine if maintenance work is being accomplished as

assigned and if greater testing ana rework effort is

warranted. be cautious about proposed improvements tnat are

not strongly supportea by users or inbicated by problem

r epor ts

.

*Evaluate software periodically ana plan improvements to

avoia oDsolescence ana to minimize future conversion costs.

ine nigh investment in current software ana tne high

cost of replacement often leaa to i^rolongea use or sortware

designs that nave become obsolete. Obsolescence may oe aue

to cnanging technology or economic factors, or to changes in

organizations ana worK loads. Major software subsystems

should be evaluatea periodically, to identify mod ir icat ions

that would eliminate operational limitations, or woula use

-38-
improvea naraware or sottware tecnnolo^y. wajor improvement

m an existiny system may oe economically aone tnrou^jn

progressive stayes ot reworK. equally important, rework to

stanaaroize or restructure component routines may increase

tne leasioility ot translerin9 tne sottware directly to a luture computer com i>j ur a t ion . bucn prospects empnasize tne

importance ot standard documentation, oecause tne

replacement ot modules or interlaces would oe impractical

witnout clear oerinition ot tne existing sottware design.

*use configuration management metliods to control sottware

status

Configuration management involves close control ot

operational software modifications, to assure tnat

continuing service is not adversely affected by cnanges and

tnat the changes made are sufficiently important to justify

their cost within the total maintenance effort available,

proposed changes should be agreed to jointly by tne

programming group and the users, and scneduled with the

appropriate priority as part of the total worKload of the

programming team. Changes should first be made to a test

copy of the software, not tne same copy used tor everyday

service. Thus testing can proceed without delaying

operational service, and only a fully cnecKed out system is

delivered tor use.

i'er lod ically review the t^roject kecord and current

documentation, to insure tnat maintenance results are oemg

recorded tor later use oy other programmers.

rtaintenance responsibility is sometimes misconstrued as

a personal prerogative that may be done with eacn

programmer's style, unless management exerts its autnority

tor quality control to meet tne organization's neeos.

Liocumen tat ion is a tedious but crucial chore, and it is

inexcusable for operational sottware to oe altered witn no

evidence except in the code itself. As in all phases ot

software worK, management action should be as immediate as

possible. The team review process is recommended for it may

remove any appearance of management inquisition. weeKly

meetings that include reviews of documentation updates may

De sufficient to keep project information current and

compl e te

-39-
THE NEED FOR SOFTWARE PRODUCTION TOOLS

Software tools are computer programs that automate tasks in

the management, production, and testing of software.

A wide variety of tools have been developed and used in

the software field, but few standard tools are recognized as

applicable to every project. The most widely used tools are

the compilers for high level programming languages, that

transform program statements into executable machine codes,

inany other tools can appreciably reduce ttie tedium of

programming tasks ana can expedite tne application of

standaras to a programmer's products. Surveys of computer

installations, as well as recent project experiences, show

tnat tne value of errective tools is not wioely appreciated,

ana an aaecjuate repertoire of effective tools is selaom

t>rovided. Tne importance of good software proauction tools

is confirmea over wnelmmgly by all recent experience ana

autnor ita t ive research in software engineering.

*t^roviae interactive computer support for programming on

every project.

Developing programs and documentation on an interactive

computer facility significantly improves the efficiency and

working conditions for the programming team. Programmers

can concentrate on technical problem solving without being

limited and burdened by scheduling of computer runs, as in a batch computer support facility. Tentative design ideas can

be experimentally programmed and evaluated in a short time,

allowing the development of a quality design to move along

rapidly. Various debugging and testing tools can be used to

snorten the time for identifying sources of errors. A

separate interactive support facility is advisable when the

operational computer is unsuitable for interactive usage

because it is intended for a batch or real-time application.

The technology and economics of interactive support software

have been amply proven, and the availability of oasic

systems is adequate for tnis general recommendation.

*use preprocessors to supplement a standard nign level

programming language.

A preprocessor accepts programming statements written

in an improved aialect ol a stanaara progr amiiiing language,

ana translates tnem into stanaara language acceptaole to a compiler program. Tne merit of a preprocessor is tnat

restrictive conventions that still exist in stanaard

languages, sucn as card column orientation, can be

-40-
circumvented tor improving programmer productivity and

reaucmg errors, without losing tne transportability

advantages ot tne standard language. For example,

structurea programming coula De applied even witn t'UKXKAi^,

wnicn is innerently unsuitea tor it. t<qually important,

many or tne coaing stanoaras ot a pro3ect coula de cnecKed

tor eacn program oy a preprocessor, as a means ot automated

quality control. However, a preprocessor may oe aitiicult

to use together witn a symoolic aeougger , oecause the

ueougger output would rerer to tne preprocessor output

statements, ana so aeougging results woula oe aitticult to

correlate witn the source program statements.

*t,stablish a standara liorary ot specialized sottware tools.

Software tools generally are inexpensive, otten

available trom other users for the cost of reproduction, ana

sometimes concise and easily modified for unique project

needs. Tools are usually specific to one programming

language, and they perform a limited set of functions on

programs written in that language. For each language m use

then, a basic set of tools can be assembled, with each one

assisting in an important, distinct design and programming

task

*Use an editor, program librarian, and program analyzer as

standard tools.

An editor assists the programmer in making numerous

Changes ana aaditions to a program, without the burden of

tilling out coding forms or manipulating format aetails, ana

with less risK ot errors in repetitive, trivial changes. A

gooa editor will automatically perform routine tormattmg

ana stanaara ization actions, ana witn prompting ana cnecKing

facilities incluaea, it can prevent or laentity trivial

syntactic errors that a programmer may maKe.

a program librarian is a necessary complement to an

eaitor, ana proviaes for storing on the computer ana

retrieving all program modules and distinct versions of them

that may aevelop auring a project. The usual tile system of

an interactive computer facility may serve as an adequate

program librarian. Features that are useful ana often

provided in special tools for this purpose include

capabilities to time and date stamp different modules, to

employ special naming conventions for programs, to

efficiently store related program versions by storing only

the relative changes, etc.

A program analyzer is software that scans the source

statements in another program to collect information about

-41-
tnem or to provide tor data collection about tne program

behavior wfien it is executed. A variety ot analyzers may be

useful, tor special formatting of programs, for auditing

adherence to coding standards, for extracting cross

reference and data usage reports, for performance

measurement, for test monitoring, and similar purposes.

Many tools of these types are available, and special

tools are often economical to develop for a project.

*(Jse a simple project information system to maintain data on

schedules, milestones, and resources used on distinct tasks

and activities.

Simplicity is the key to keeping the management and

administrative effort within bounds tor small projects. For

large projects, fiscal and contractual requirements may

dictate extensive record Keeping, but it is still advisable

tor technical management to nave simplified project data

that may expose basic problems, ratner than to oe buried

under all possiole details. It a computer mtormation

system is used, it should maintain a gross networK ot major

milestones and produce a tew briet reports on manpower and

other expenditures tor tnese milestones and otner support

act IV 1 ties

.

STANDARDS FOR QUALITY CONTROL

A number of distinct quality control techniques are

known to be effective, and some have been briefly described

in the preceding. In all cases, there is no documented

standard of practice for the technique, but there are

published articles and sources ot experience to help m

developing appropriate standards for an interested

organization. A summary definition of known techniques is

given oelow.

PROJECT RLCORL/--An official journal ot project plans,

specifications, scnedules, worK assignments, oudgets,

technical standards, etc. that serves to inform all

concerned parties ot the goals, worK methoos, nistory, and

-4z-
status of the effort.

ir'ROJECT PLAi\--a detailed plan of milestones, resource

allocations, ana quality control procedures to oe used in

tne aevelopment phases tor a software product. incorporated

in tne ir'roject Kecora ana updatea as neeaea during

aevelopiuen t

L/ULUi-iLi.'^'iaiiow i)TAwijAKDo--unif orm requirements on

uocuiiients for tne project ana the software, tnat aetine

scope ana identification of each aocument, as well as lormat

anu content, ana that cover tne principal specifications ana

tecnnical aescr ipt ions , ootn internal to the project ana

aeliveraole witn tne sortware.

AUTuwATLb DUCUi*ibwTATIOi.M AlDb--Computer software for

text editing ana document formatting, and for automatea

preparation of program documentation such as ilow charts or

cross reference listings of aata elements ana program

s ta temen ts

STRUCTURED SPECIFICATIONS —The approach of coordinating

all specification documents in format and content, and using

formal or mathematical specification techniques to reduce

aiTibiguity. This facilitates successive validation

milestones as increasingly detailed specifications are

developed. The series of specifications involved begins

with the feasibility study description of system concept,

continues with the functional requirements, and extends to

the individual program module specifications.

wuALIT:^ and i^ERFORMANCb RE(^UlKEMENTS--Augmen ta t ion of

tne functional requirements, to state the specitic quality

attriDutes expected of the software and the quantitative

performance necessary m Key operations or test cases.

:3PELlFICATI0w CuwTkOL--A formal process of reviewing

and at^proving proposed changes to specifications, with

consideration of tneir tecnnical validity, impact on project

resources ano scneauie, and impact on worK completed or m

progress. Tne goal is to inhibit continuing, capricious

Changes, and instead stabilize the specifications on

distinct deliverable versions of the software that can be

managed for production.

STRUCTURED pRUGRAMMIwG —Technical practices and

criteria governing particularly the programming techniques

and language usage for increasing programming productivity

and reducing program errors.

-43-
Tup-L»uwiM PKUOKAw utVhLUPiyibw'i--'i'ne approach ot

implemen tincj and testing program systems oy Duiiding program

modules starting witn those at the most general,

application-oriented level (the "top"), and proceeding

successively down to the most specialized, computerdependent level (the "bottom").

DiiSIblM and CODING STAI\IDARDS--De tailed specifications of

generally applicaole design requirements based on quality

oDjectives, including such programming conventions as

indenting and spacing of program statements, use of

comments, naming of variables, use of language features,

etc

.

SOURCE CODE CONTROL —The process of labeling and

archiving program modules accepted for formal technical

control, thereby limiting future programmer modifications to

formally reviewed and approved changes.

TEAM kEVIEAS--Techn ical team conferences to audit

design proposals, specifications, and completed programs for

qual 1 ty

.

UNIT FuLD£-Rb--The programmer's technical records of

design and testing worK on individual program modules,

sometimes callea "unit development folders". As standard

uocuments in a project, these augment the project record and

specifications oy providing more technical oocumen tation as

necessary for reviews and inspections. Especially important

m large projects subject to frequent personnel changes or

r eassignmen ts

.

INbPECTIONS--Formal review and auditing of subsystems

or groups of program modules at design, code, and test

stages, by an independent technical team. Applicable to

larger projects.

STANDARD PRODUCTION TOOLS —Computer software to aid the

programming team in creation, documentation, testing, and

analysis of the deliverable programs. These would include,

for example, the standard programming language compiler,

program librarian, test data generator, symbolic debugger,

etc

.

PROGRAM ANALyZERS--Sof tware tools for automated

analysis of program modules, to confirm complete adherence

to coding standaras ana quality criteria, ana to assure

thorough testing according to standards.

PkuJElt INtORMATION--Kecora mg ana analysis ot the

allocation and expenaiture of team effort on different

-44-
modules ana milestones for the pro-gram system, and on

various production activities, in order to monitor progress

and to support best practical resource management.

TESTINb S1ANDARDS--Spec if ic criteria governing tne

program testing to De performed, to assure that programs are

unirormly tested oy all programmers and tnat all program

instructions and capaoilities are tested to the extent

pr ac t ical

iNuLi^LwubiNT itibT irLAiNS--use Of independent tecnnical

st^ecialists to prepare compr enensive tests of tne integrated

soitware sj-stem.

t\rtuLi Kbir'UKi 1 rormal process for documenting

oDservea errors or discrepancies m program operation ano

capaoility, to aid oeougging and reliaoility assessment, and

to validate sottware design.

CunFIvjUkAi lUN hAhAGLwENT--A formal process of

controlling the operational software status and reviewing

proposed improvements, to Dest direct maintenance effort and

to minimize impact of maintenance and testing on operational

ser V ice

FITTING CONTROLS TO THE PROJECT SIZE

Software projects differ tremendously m complexity,

acceptable scnedule, and available resources. More than

anytning, tne available personnel determine wnat technical

goals are acnievaole and wnat quality controls may

reasonaoly De imposed. Figure 6 illustrates a Dasic

judgement of where the above controls are best utilized m

regard to project size. Tne controls tnat are perceived to

oe more demanding of tecnnical or supporting effort are not

recommended for smaller projects.

-45-
Figure 6. Chart of Recommended quality Controls

Quality Control Development or Improvement

Effort in Man-Years

Less than 1/2 1/2 to 2 2 to 5 Over 5

Project Record X X X X Proiect Plan X X X X Qual i ty/Performance

Requirements Ax YA

YA X i

i Standard Documentation X X X X

1 Program Analyzers X X X X 1

i Standard Production Tools X X X X

i Structured Programming X X X X 1 Ton — Hnu/n PIpwp 1 nnmpn f X X X X !

1

i

i

1

X X

i X

X X X I

1 Automated Documentation

Aids

YA X X i

1 Design/Coding Standards X X

V 1 X Team Reviews X X

VX I Testing Standards X X ^ \ Project Information X X

wX

Source Code Control X X Unit Records X X Fault Reporting X X Configuration Management X X

Inspections X

Independent Test Plans X

-46-
FINAL HOMILIES

A few simple cautions may help to avoid unqualified

failure in undertaking software development, given present

Knowleoge of now best to do it.

Cnoose reasonable goal

. Although computer technology

has amazing capaoil i ties , we remain primarily limited by

human creativity m regara to software acnievemen ts

nave a de tailea plan Im^jortant tasKs may be forgotten

or expectations may expana unless all is recoraeu as a reminuer

tut one per son m control , Tnere snoula be strong

leaaersniiJ towara one well unoerstooa concept.

use gooa tools . i^roauctive tools are tne only certain

way to accelerate nuiuan accompl isnmen ts

Cater to the sof twar e user . The user judgement is tne

ultimate answer on success or failure of tne product.

build up in steps . Confidence from successful

achievement of modest goals will lead more quickly to

grander oojectives.

admit mistakes early . Pursuit of the hopeless leads

only to further despair--poor choices and misconceptions

require drastic remedies as early as possible.

Conside r star ting over . Patcning a poor result has

limiteo return; a wiser action may be to start fresn at the

oeg mn ing .

-47-
REFi:;R£wCh;S

baKer , t, 'I,, "structured ir'rogr amming in a i'roauction

ir^roy ramming environment", ir^roceea ing s ot tne International

Center ence on Kel lable So£ twar e , 21-23 April lb>75, Los

Angeles, ppl7z-lb5, ibt-h Computer bociety. Long beacn,

cal itornia

.

BrooKs, F. P., Jr., Tne ivly tnical Man-£lont_h: t.ssays on

bo£ twar e Eng ineer ing , Add ison-vVesley Publishing Co.,

Reading, Mass., 1975, 195p.

Comptroller General of the United States, Lessons Learned

APou t Acquir ing Financial Management and Other Information

Systems , August 1976, 61p, available from U. S. Government

Printing Office, Washington, D. C. 2ld402, Stock No.

02W-000-00138-1.

Endres, A., "An Analysis of Errors and Their Causes in

System Programs", IEEE Transactions on Software Eng ineer ing , V. SE-1, n. 2, June 1975, 140-149, IEEE Computer Society,

Long Beach, Calif.

Fagan, £., "Design and Code Inspections to Reduce Errors

in program Development", Ibm Systems Journal , v. 15, n. 3,

1975, 182-211.

tiecht, iierbert, measurement , pst imation , and prediction of

^ot tware t^ei labil i ty , NASA Cr-145135, aerospace Corporation,

sjI begunoo, Calif., January ly77, 41p.

IbW Corp., Data Processing uivision. Improved Programiuing

Technolog ies --An over v lew , Installation management Keport

GC2U-l&5^-k) , October 1974 , I9p.

IBh Corp., Data processing Division, _hIPO--A Design Aid and

Documentation Technique , Installation Management Report

GC20-1851-0, October 1974, 130p.

Kernighan, B. w. and Plauger , P. J., The Elements of

Programming Style , McGraw-Hill Book Company, New York, 1974,

147p.

Kernighan, B. w. and Plauger, P. J., Sof twar

e

Tools,

Add ison-Vvesley Publishing Co., Reading, Mass., 1976 , 338p.

Mills, Harlan, "Software Development", IEEE Transactions on

Sof twar e Eng meer ing , December 1976 , 265-273.

-48-
rtyers, Glenford J., Sot twar e Rel iabil i ty : Pr inc iples and

Pr act ices , John wiley and Sons, New York, 1976, 360p.

National Bureau of Standards, Guidel ines tor Documentation

of Computer Programs and Automated Data Systems , Feder al

Information Processing Standards Publication 38, 1976

February 15, 55p.

Rochkind, M. J., "The Source Code Control System",

Proceed mg s of the First National Conference on Sof twar

t.ng ineer mg , 11-12 September, 1975 , Washington, D. C,

pp37-43, Illl Computer Society, Long Beach, Calif.

KUDey, K. J., et al . , "(quantitative Aspects of Software

validation", 1 llL 1 r ansae t ions on Sof twar e Lng ineer mg , v. bt-1, n.2, June 1975, 15to-155, IlLL Computer i»ociety. Long

beacn, Calit.

van Tassel, u . , prog r am S tyle , ues ign , Lf i ic lency , ueougg ing , ana Testing , Pr en t ice-Hall , Inc., Lnglewooa

Clitfs, New Jersey, 1^/4, 256p.

waiston, C. l. and C. P. telix, "A Method of Program

estimation and measurement", It>t»i Systems Journal , v. 1_6, n.

1 , 19 7 7 , 54-73 . Yourdon, l., Techniques of Program Structure and Design ,

ir'r en t ice-riall , Inc., Englewood Cliffs, New Jersey, 1975,

364p.

Yourdon, E., "A Case Study in Structured Programming:

Kedesign of a Payroll System", Digest of Paper s of I£EE

COMPCOn , September 9-11 , 1975 , Washington, D. C, 119-122,

IEEE Computer Society, Long Beach, Calif.

-49-
1

NBS-114A (REV. 7-73)

U.S. DEPT. OF COMM. BIBLIOGRAPHIC DATA

SHEET

1. PUBLICATION OR REPORT NO. NBS SP-5Q0-11

2. Gov't Accession

No.

3. Recipient's Accession No.

4. TITLE AND SUBTITLE

COMPUTER SCIENCE & TECHNOLOGY:

Computer Software Management: A Primer for Project

Management and Quality Control

5. Publication Date July 1977

6. Performing Organization Code

7. AUTHOR(S)

Dennis W. Fife

8. Performing Organ. Report No.

9. PERFORMING ORGANIZATION NAME AND ADDRESS

NATIONAL BUREAU OF STANDARDS

DEPARTMENT OF COMMERCE

WASHINGTON, D.C. 20234

10. Project/Task/Work Unit No.

11. Contract/Grant No.

12. Sponsoring Organization Name and Complete Address (Street, City, State, ZIP)

Same as Item 9

13. Type of Report & Period Covered

14. Sponsoring Agency Code

15. SUPPLEMENTARY NOTES

Library of Congress Catalog Card Number: 77-608127

16. ABSTRACT (A 200-word or less [actual summary of most significant information. If document includes a significant

bibliography or literature survey, mention it here.) Today, providing computer software involves greater cost and risk than

providing computer equipment, because hardware is mass produced by industry

using proven technology, while software is still produced mostly by the craft

of individual computer programmers. This brief guide is intended for managers

who are responsible for computer projects, to explain the use of quality controls

and software management methods. The typical problems of software development

are explained. Over twenty distinct quality controls are defined, and

recommendations are given for software management actions. Empirical information

is included that would help top executives to appreciate the potential problems

and importance of software management.

17. KEY WORDS (six to twelve entries; alphabetical order; capitalize only the first letter of the first key word unless a proper name; separated by semicolons)

Computer management; computer programming; computer project control;

computer software; software engineering; software quality; software reliability.

18. AVAILABILITY ^Unlimited

1 1 For Official Distribution. Do Not Release to NTIS

\yCi Order From Sup. of Doc, U.S. Government Printing Office Washington. D.C. 20402. SD Cat. No. C13.10:50&-11

1 1 Order From National Technical Information Service (NTIS)

Springfield, Virginia 22151

19. SECURITY CLASS

(THIS REPORT)

UNCL ASSIFIED

21. NO. OF PAGES

58

20. SECURITY CLASS

(THIS PAGE)

UNCLASSIFIED

22. Price

USCOMM-DC 29042-P74
READER EVALUATION FORM

NBS Special Publication 500-U , "Computer Software Management"

Your voluntary, anonymous comments after reading this report will help NBS to develop this guide

as a standard reference for software projects of the Federal government.

1. Where do you place yourself among the audience for this report? (Check one best answer).

Practicing computer professional^ 1 Executive or policy officialQ2

Technical professional in another field Layman[ ]4

2. How would you rate the clarity of the presentation? (Check one best answer).

Well written, easy to follow Difficult in spots [ ]& Generally difficult [^7 Mostly vague or unclear[ \8

3. How would you rate the completeness of the guidance given? (Check one best answer)

Complete, with adequate discussion^ 9 Generally complete, but lacking depth [~] 10 Some omissions or inadequate analysis[^ll Serious omissions [

) 12 (Please comment below)

4. How would you rate the validity of the specific guidelines and procedures given?

(Check one best answer)^

Highly practical and consistent with most experience \

] 1

Reasonable and generally recommended 14 Plausible for some cases | | 15 Mostly unjustified Ql6 (Please comment below)

Check your belief regarding each of the following statements on the need for this guide:

Strongly

Disagree

Tend to

Di saqree

Tend to AqreeStrongly

Aqree

a. Better guides already are used by most designers. 17 18 19 20

b. NBS is a well-known source for such information. 21 22 23 24

c. The software problem is not so important. 25 26 27 28

d. The subject is too complex for such a brief guide. 29 30 31 32

e. The problem cannot be reduced by simple rules. 33 34 35 36

f. This guide will improve government operations. 37 38 39 40

g- Standard practices are needed in this field soon. 41 42 43 44

h. Good textbooks are more beneficial than such a guidebook. 45 46 47 48

6. Your narrative comments are welcome.

Fold and Staple here. No postage required.
U.S. DEPARTMENT OF COMMERCE

NATIONAL BUREAU OF STANDARDS

WASHINGTON. D.C. 20234

OFFICIAL BUSINESS

PENALTY FOR PRIVATE USE. S300. RETURN POSTAGE GUARANTEED

National Bureau of Standards

Computer Science Section

TECH A36

7

Washington, DC 20234
ANNOUNCEMENT OF NEW PUBLICATIONS ON

COMPUTER SCIENCE & TECHNOLOGY

Superintendent of Documents,

Government Printing Office,

Washington, D. C. 20402

Dear Sir:

Please add my name to the announcement list of new publications to be issued in the series: National Bureau of Standards Special Publication 500-.

Name

Company

Address

City State Zip Code

(Notification key N-503)

NBS TECHNICAL PUBLICATIONS

PERIODICALS JOI KNAL OF RESEARCH reports National Bureau

of Standards research and development in physics,

mathematics, and chemistry. It is published in two

sections, available separately:

• Physics and Chemistry (Section A) <j\^

Papers of interest primarily to scier* orking in these fields. This section covers a bf .ige of physical and chemical research, wit*- ^^-^ ,,r emphasis on

standards of physical measu*- ^

, fundamental constants, and properties of nv ^'.isued six times a year.

.Annual subscription: D- ^^.f'^.-, $17.00; Foreign, $21.25.

• Mathematical Sci^ ^^C*'**^ Section B)

Studies and com"" ^* .is designed mainly for the mathematician anH .Q<v*<;tical physicist. Topics in mathematical stati#'\^,^c^^.ieory of experiment design, numerical

analysi v^..retical physics and chemistry, logical desigp ^ programming of computers and computer syst^ nort numerical tables. Issued quarterly. Annual

s."^ ..cription: Domestic, $9.00; Foreign, $11.25.

DIMENSIONS/NBS (formerly Technical News Bulletin) —This monthly magazine is published to inform

scientists, engineers, businessmen, industry, teachers,

students, and consumers of the latest advances in science and technology, with primary emphasis on the work at NBS. The magazine highlights and reviews

such issues as energy research, fire protection, building

technology, metric conversion, pollution abatement,

health and safety, and consumer product performance.

In addition, it reports the results of Bureau programs

in measurement standards and techniques, properties of matter and materials, engineering standards and services, instrumentation, and automatic data processing.

Annual subscription: Domestic, .Sl2. 50: Foreign, .$15.65.

NONPERIODICALS

.Monographs—Major contributions to the technical liter- ature on various subjects related to the Bureau's scientific and technical activities. Handbooks—Recommended codes of engineering and

industrial practice (including safety codes) developed

in cooperation with interested industries, professional

organizations, and regulatory bodies.

Special Publications—Include proceedings of conferences

.sponsored by NBS, NBS annual reports, and other

special publications appropriate to this grouping such

as wall charts, pocket cards, and bibliographies.

Applied .Mathematics Series—Mathematical tables, manuals, and studies of special interest to physicists, engineers, chemists, biologists, mathematicians, computer programmers, and others engaged in scientific and technical work.

National Standard Reference Data Series—Provides

quantitative data on the physical and chemical properties of materials, compiled from the world's literature and critically evaluated. Developed under a world-wide

program coordinated by NBS. Program under authority

of National Standard Data Act (Public Law 90-396).

NOTE: At present the principal publication outlet for these data is the Journal of Physical and Chemical

Reference Data (JPCRD) published quarterly for NBS

by the American Chemical Society (ACS) and the American Institute of Physics (AIP). Subscriptions, reprints,

and supplements available from ACS, 1155 Sixteenth

St. N.W., Wash. D. C. 20056. Building Science Series—Disseminates technical infor- mation developed at the Bureau on building materials,

components, systems, and whole structures. The series presents research results, test methods, and performance criteria related to the structural and environmental

functions and the durability and safety characteristics

of building elements and systems.

Technical Notes—Studies or reports which are complete

in themselves but restrictive in their treatment of a subject. Analogous to monographs but not so comprehensive in scope or definitive in treatment of the subject area. Often serve as a vehicle for final reports of work performed at NBS under the sponsorship of other government agencies.

Voluntary Product Standards—Developed under proce- dures published by the Department of Commerce in Part

10, Title 15, of the Code of Federal Regulations. The

purpose of the standards is to establish nationally rec- ognized requirements for products, and to provide all concerned interests with a basis for common under- standing of the characteristics of the products. NBS

administers this program as a supplement to the activities of the private sector standardizing organizations.

Consumer Information Series—Practical information, based on NBS research and experience, covering areas

of interest to the consumer. Easily understandable lang- uage and illustrations provide useful background knowledge for shopping in today's technological marketplace.

Order above NBS publications from: Superintendent

of Documents, Goi>crnmcnt Printing Office, Washington,

D.C. >(>!>() 2. Order following NBS publications—NBSIR's and FIPS

from the National Technical hiformation Services,

Sprinyrield, Va. 221GI. Federal Information Processing Standards Publications (FIPS PUBS) —Publications in this series collectively constitute the Federal Information Processing Standards Register. Register serves as the official source of information in the Federal Government regarding stand- ards issued by NBS pursuant to the Federal Property

and Administrative Services Act of 1949 as amended,

Public Law 89-306 (79 Stat. 1127), and as implemented

by Executive Order 11717 (38 FR 12315, dated May 11, 1973) and Part 6 of Title 15 CFR (Code of Federal

Regulations).

NBS Interagency Reports (NBSIR)—A special series of interim or final reports on work performed by NBS for outside sponsors (both government and non-government). In general, initial distribution is handled by the sponsor; public distribution is by the National Technical Information Services (Springfield, Va. 22161) in paper copy or microfiche form. BIBLIOGRAPHIC SUBSCRIiniON SERVICES

The following current-awareness and literature-survey

bibliographies are issued periodically by the Bureau:

Cryogenic Data Center Current Awareness Service. A

literature survey issued biweekly. Annual subscription: Domestic, 82.5.00 ; Foreign, 8.}0.0(J

Liquified Natural Gas. A literature survey issued quarterly. Annual subscription: $20.00.

Superconducting Devices and Materials. A literature survey issued quarterly. Annual subscription: 1.30.00 . Send subscription orders and remittances for the pre- ceding bibliographic services to National Bureau of Standards, Cryogenic Data Center (275.02) Boulder,

Colorado 80302.
U.S. OEPARTMENT OF COMMERCE

National Bureau of Standards

Washington.D C. 20234

OFFICIAL BUSINESS II Penalty for Private Use, $300

POSTAGE AND FEES PAID

U.S. DEPARTMENT OF COMMERCE

COM-21S

SPECIAL FOURTH-CLASS RATE BOOK