FTWARE

•" VOLUME _

1989

SOFTWARE ENGINEERING LABORATORY SERIES SEL-89-006

COLLECTED SOFTWARE

ENGINEERING PAPERS: VOLUME VII

NOVEMBER 1989

National Aeronautics and

Space Administration

Goddard Space Flight Center

Greenbelt, Maryland 20771

FOREWORD

The Software Engineering Laboratory (SEL) is an organization

sponsored by the National Aeronautics and Space Administration/Goddard Space Flight Center (NASA/GSFC) and created for

the purpose of investigating the effectiveness of software

engineering technologies when applied to the development of

applications software. The SEL was created in 1977 and has

three primary organizational members:

NASA/GSFC (Systems Development Branch)

The University of Maryland (Computer Sciences Department)

Computer Sciences Corporation (Systems Development

Operation)

The goals of the SEL are (I) to understand the software

development process in the GSFC environment; (2) to measure

the effect of various methodologies, tools, and models on

this process; and (3) to identify and then to apply successful development practices. The activities, findings, and

recommendations of the SEL are recorded in the Software

Engineering Laboratory Series, a continuing series of

reports that includes this document. The papers'contained

in this document appeared previously as indicated in each

section.

Single copies of this document can be obtained by writing to

Systems Development Branch

Code 552

NASA/GSFC

Greenbelt, Maryland 20771

5642

PRECEDING PAGE BLANK NOT FJI.M,ED

iii

TABLE QF CONTENTS

Section 1 - Introduction ................ i-i

Section 2 - Software Measurement and Technoloqy

Studies ................... 2-1

Establishing a Measurement Based Maintenance

Improvement Program: ....Lessons Learned in the

SEL, H. Rombach and B. Ulery ............ 2-2

Maintenance = Reuse-Oriented Software Development, V. Basili ................. 2-30

Software Development; A Paradigm for the

Future, V. Basili .................. 2-43

Section 3 - Measurement EnvironMent Studies ....... 3-1

Integrating Automated Support for a Software

Management Cycle Into the TAME System,

V. Basili and T. Sunazuka ............. 3-2

Towards A Comprehenzive Framework for Reuse: A

Reuse-Enabling Software Evolution Environment,

V. Basili and H. Rombach ............. 3-22

Section 4 - Ad_ Technology Studies ........... 4-1

"Evolution of Ada Technology in a Production

Software Environment," F. McGarry, L.Esker,

and K. Quimby .................... 4-2

"Using Ada to Maximize Verbatim Software Reuse,"

M. Stark and E. Booth ................ 4-11

Standard Bibli0graphy.0f SEL Literature

5642

V

SECTION I--INTRODUCTION

SECTION 1 - INTRODUCTION

This document is a collection of selected technical papers

produced by participants in the Software Engineering Laboratory (SEL) during the period December 1988, through October

1989. The purpose of the document is to make available, in

one reference, some results of SEL research that originally

appeared in a number of different forums. This is the seventh such volume of technical papers produced by the SEL.

Although these papers cover several topics related to software engineering, they do not encompass the entire scope of

SEL activities and interests. Additional information about

the SEL and its research efforts may be obtained from the

sources listed in the bibliography at the end of this document.

For the convenience of this presentation, the seven papers

contained here are grouped into three major categories:

• Software Measurement and Technology Studies

• Measurement Environment Studies

• Ada Technology Studies

The first category presents experimental research and evaluation of software measurement and technology; the second

presents studies on software environments pertaining to

measurement. The last category represents Ada technology

and includes research, development, and measurement studies.

The SEL is actively working to increase its understanding

and to improve the software development process at Goddard

Space Flight Center (GSFC). Future efforts will be documented in additional volumes of the Collected Software Engineerinq Papers and other SEL publications.

5642

I-i

SECTION 2--SOFTWARE MEASUREMENT AND

TECHNOLOGY STUDIES

SECTION 2 - SOFTWARW. MEASUREMENT AND TECHNOLOGY STUDIES

The technical papers included in this section were originally

prepared as indicated below.

• Establishinq a Measurement Based Maintenance Impr0vem_nt Pr0gr%m; L@ssons Learned in the SEL,

H. Rombach and B. Ulery, University of Maryland,

Technical Report TR-2252, May 1989

• Maintenance = R_use-Oriented Software Development,

V. Basili, University of Maryland, Technical Report

TR-2244, May 1989

• Software Developm_n_; A Paradiqm for the Future,

V. Basili, University of Maryland, Technical Report

TR-2263, June 1989

5642

2-1
UMIACS-TR-89-54

CS-TR-2252

Establishing a .Measurement Based

Maintenance [mprovement Program:

Lessons Learned in the SEL

H. Dieter Rombache

Institute for Advanced Computer Studies

Department of Computer .Science

Bradford T. Ulery

Department o_ Computer Science

University of Maryland

College Park. MD 20742

May, I989

ABSTRACT

The Software Engineering Laboratory (SEL) is a joint venture between NASAs

Goddard Space Flight Center, the University of MarTland, and Computer Sciences

Corporation. We discuss the use of a goal oriented approach to measurement to establish a maintenance improvement program within the SEL. Differences are tound to

exist between the initial phase of the program and its routine application. \Ve demonstrate our approach through concrete examples, and summarize lesso,s ,,_e i_avc

learned in the establishment of a measurement based, maintenance improvement pro-

_am.

"['hts work was _upp,_r_cd by the NASA grant NSC;-5123

This paper writ _l_o _gpcar m the i_rucccdin.gs ,_t Ihc Cvr_tcrcncc on 5,altware _',l..unttmance 193 _)

5642

2-2
5642

i. INTRODUCTION

The Software Engineering Laboratory (SEL) is an organization sponsored by the National

Aeronautics and Space Administration/Goddard Space Flight Center (NASA/GSFC) and created

for the purpose of investigating the effectiveness of software engineering technologies. The SEL

was created in 1977 and has three primary organizational members: NASA/GSFC, the University

of Maryland, and Computer Sciences Corporation.

NASA/GSFC develops ground control software systems and other support software for

satellites. A large number of case studies and controlled experiments have been conducted in the

past that have resulted in evolutionary changes to NASA's development practices

[Basili85,McGarry85]. Some of the changes include the stricter use of code reading techniques

iBasili87], the use of measurement baselines for management purposes iRamsey88!, measurement

based recommendations for Ada projects [Brophy87!, and most recently the experimental

adoption of the CLEA.NROOM development method !Selby87].

In order to formalize the procedures for investigating software technologies and to organize

the resulting experience, three paradigms have been defined for goal oriented measurement:

(1) Goal/question/mettle !Basili84,Basili85b i. This paradigm is based on the principle tha_

effective measurement procedures should be derived (top-down) from goals. The GQN1

paradigm suggests that measurement needs to start with a precise specification of the goals.

continue with the refinement of each goal into a set of quantifiable questions, and end with

the derivation of a set of metrics. This approach yields a rationale for any chosen set of

metrics all the way back to the original goals. Therefore, it also provides a basis for the

goal oriented interpretation of collected data.

(2) Evaluation [Basili_4]. This paradigm is based on the additional principle that the

measurement process must be designed to fit the production environment,. The evaluation

paradigm simply extends the GQM paradigm by including the actual measuremenL

1

2-3
procedures. These procedures must be tailored to the product or process being studied in

order to obtain valid results. Previous publications also refer to this as the GQM paradigm.

(3) Improvement iBasili85b,Basili88]. This paradigm is based on the principle that

improvement is based on continuous learning. The improvement paradigm provides the

context for evaluating multiple project. It emphasizes recording what has been learned

through measurement so that this knowledge will be available when it is needed.

Knowledge is managed explicitly by modeling the environment and providing feedback from

analysis to production.

Experience from several studies has reaffirmed these principles and our belief in the effectiveness

of goal oriented measurement whether in .x production environment or an expemment. Th_s

experience has led to the formulation of specific measurement guidelines and goal templates

'_Basili88}.

A number of improvement programs have been established based on the improvement

paradigm [Grady87,Rombach87,Basili85,McGarry851. Most of the published results, however.

address specific studies performed under such programs rather than the establishment of the

programs themselves. Examples include industrtal case _tudies

iRombach87, Basili87b, Weiss85, Basili871 as well as academic experiments

'Nehmer87,Katz86,Rombach86}. Two noteworthy exceptions address the managema] and

technology transfer problems associated with the establishment of such a study

[Bretsford88, Grady87b].

An organization's long-term commitment to invest in such a program depends on whether

the potential for future payoff can be demonstrated convincingly. We distinguish between the

"initial program phase" aimed at establishing an effective program and demonstrating Its payoff

potential, and the "routine program phase" aimed at applying an accepted program to routine

projects.

5642

2

2-4
The purpose of the initial phase is to understand the environment sufficiently to identify

high leverage improvement goals and to establish a proper measurement infrastructure

(procedures, managerial commitment, tools, personnel, etc.). The environment should be modeled

explicitly and limited measurement may be required in order to demonstrate the potential

leverage of the stated goals and the feasibility of the procedures. It is important to recognize that

this initial phase represents an investment. What is learned in this phase provides the foundation

for improvements during the routine phase.

This paper reports on our experience from establishing an improvement program for

maintenance in the SEL. Section 2 summarizes the approach used to create the initial

understanding of the environment, the improvement, goals and measurement procedures. Section

3 characterizes the current status of our program. Sections 4 and 5 highlight some important

lessons, and outline future SEL maintenance improvement activities.

2. MAINTENANCE IMPROVEMENT APPROACH

Within the SEL, development and maintenance are performed by separate organizational

units. In late 1987, measurement of maintenance was included in the scope of _he SEL in order to

better understand and eventually improve the software life-cycle. At that time there wa_ little

documentation of the actual maintenance procedures (beyond some general guidelines} on which

to base our initial analysis. Nor was there any explicit feedback to developers about the product

maintainability, or the types and amounts of maintenance required.

Based on past experience, we were confident that the guidelines supporting the improvement

paradigm would be helpful during the initial program phase. We had, however, only vague ideas

as to how these methods should be applied given the lack of explicitly documented experience in

the SEL maintenance environment that was available when we began. We expected to learn

about the strengths and weaknesses of the improvement approach in a situation where we cou[d

5642

3

2-5
not select improvement goals or design measurement procedures based on a mature understanding

of the environment, but rather would have to initially bootstrap that understanding.

The improvement program established for maintenance in the SEL is based on a version of

the improvement paradigm applied to maintenance (Rombach881. This paradigm (see Fig. 1)

suggests that maintenance can be improved by iteratingthe following steps for each project:(I)

characterize the corporate maintenance environment; (2)state improvement goals in quanutative

terms; (3) plan the appropriate maintenance and measurement procedures for the project at hand:

(4) perform maintenance, measure, analyze and provide feedback; and (5) perform post mortem

analysis and provide recommendations for future projects.

II. Characterize the corporate maintenance environment

I2. State improvement goals

a. State improvement goals informally

b. Specify related measurement goals

I3. Plan maintenance

a. Plan appropriate maintenance process

b. Plan appropriate measurement process

I4. Perform maintenance

a. Perform maintenance process

b. Perform rnea.surement process

c..Analyze collecteddata and provide immediate

feedback

15. Perform post-mortem analysisand provide recommendations

16. for Return futureto projects step [I __

Figure 1: The improvement paradigm applied to maintenance.

We applied the principles of the paradigms strictly. However, during the initial phase, our

understanding of she environment, goals, and meazurement procedures did not develop according

5642

2-6
to a straightforward sequential application of the first three steps of the improvement paradigm.

Nor were all supporting metrics identified by a strictly top-down application of the GQM

paradigm. There are two good reasons for not following these steps: (i) we sometimes discover

that our knowledge of prior steps [s inadequate, so we retrace our steps, or (ii) practical

constraints (such as existing data collection forms) preclude a strictly top-down derivation of

procedures.

The initial uncertainty in our understanding of the maintenance environment made it

necessary to allow for planned and ad hoc feedback loops at any time. Such feedback loops

resulted in revisions of the goals and measurement procedures. Mea-*urement procedures were

validated by actually applying them to real projects on a trial basis. The experience from such

trial data collection, validation, and analysis helped us to further improve our understanding of

the environment, and provided objective data to demonstrate the existence or" suspected

maintenance problems. Demonstrating the feasibility of planned measurement procedures on a

trial basis has won confidence in their potential to support the improvement goals.

In summary, a number of quick (and sometimes partial) iterations through the improvement

paradigm eventually resulted in our current status. Based on this status the SEL ha_ reached

consensus that routinely applying this improvement progr£m to all future maintenance projects is

worthwhile.

3. PROGRAM STATUS

This section presents the current status of our maintenance improvement program

according to the outline of the improvement paradigm (Fig. 1). Section 3.1 summarizes our

current understanding of the SEL maintenance environment (corresponding to improvement step

I1). Selected improvement goals and their supporting data collection, and validation procedures

are summarized in sections 3.2. and 3.3 (corresponding to steps I2 and [3.b). Initial mea.surement

5642

5

2-7
dat_ are presented and interpreted in secUoQs 3.4 and 3.G {¢orre_poadln_ _o steps I4.b _n<i [4.c).

This form of presentation LS intended to demonstrate _he use of _he improvement paradigm

durinl; this initial pha_e.

3.1. EN'FIRONMENT

We characterize the SEL maintenance environment in terms of the application, maintained

products, and maintenance process.

Apptlcation

Two mL_iom in _his study are the Coamic Backcround Explorer (COBE) and _he G,_mma

R_y O_ervatory (GRO). They are tentatively scheduled to be launched _n July, 1989, and April,

1990, respectively. COBE's scientific minion is _o investigate _he orilins of the univeme. GRO

will make ot_ervation5 over the enerfy range from .1 to 30,000 NfeV

The Earth R_liation I_udzet Satellite (ERBS) wa_ launched from Space Shuttle Challenger

in October, ],984. ERBS carried t_e Earth IL_distion Budget Experiment (ERBE) and _he

Stratospheric Aerosol and G_ Experiment (SAGE)-II. Mea_urement_ from the_e experimenm are

_ed to understand the earth's chmate and how environmental t'actor_ affect it.

Largely because of the Challenser disaster in January, 198G, COBE will be the flint m_ion

to which _he Flight Dynamics Division of NASA/GSFC h_ conthbu_ed si_nific_[y _ince ERBS

Malntaiaed Produet_

To date, we have monitored five projects representing each of the followin8 _hree major

_yl_e_ o_"systems developed in _he $EL environment.

5642

2-8
(1) Attitude Ground Support Systems (AGSS) provide operational support for a mission. Their

functions include determining spacecraft attitude from telemetry data, verifying the onboard computer's attitude determination and control, supporting star tracking (for

guidance), and more.

(2) Attitude Telemetry Data Simulator Systems produce realistic attitude telemetry and

engineering data files to exercise the algorithms and processing capabilities of .-kGSS's.

Telemetry data includes essentially everything the spacecraft knows and could report back.

(3) Attitude Dynamics Simulator Systems are analytic tools for testing and evaluating (two

subsystems of) the spacecraft simulators. They simulate the environment of the spacecraft,

sensor data, the on-board computer's response (actuator commands), and the resulting

control torques in order to model the spacecraft dynamics.

In addition to the many numerical algorithms, each of these systems manages a user

interface including the control of parameters, reading large data files, and printing tables and

plots.

Maintenance Process

In order to understand the role of maintenance in this environment, why changes occur and

who could benefit from our observations, and in order to design effective measurement

procedures, we hb.ve modeled the software life-cycle (Fig. 2). The Flight Dynamics Division of

NASA/GSFC is divided into four branches, three of which are included in our model. Note that

communications crossing organizational boundaries tend to be more formal and occur less

frequently than internal communications.

Project Requirements are received by the Specification Developers. The Specification

Developers produce Functional Specifications and Mathematical Derivations for use by the

5642

7

2-9
Requirements Analysts. The Requirements ,4.nalysts write the Preliminary Design which includes

the high level system architecture. This design is used by the Software Developers who produce

Code, a User's Guide, and a System Description. The System Description summarizes the

Preliminary and Detailed Designs. The system is then submitted to Acceptance Testing. Upon

acceptance, the system is eventually passed to Maintenance. Maintainers are responsible for the

system until about three months after launch. [f the system is an AGSS, its routine Operation is

handled by Operations Support. Dynamics simulators and telemetry simulators are not passed on

to Operations Support. Although changes frequently occur immediately after a hunch, they are

reportedly quite infrequent during Operational Support.

During maintenance, each change is formally defined by an Operauonal _of_war,

Modification Report (OSMR), a form that specifies the change, and then follows it. g_therlng

dates and signatures as the change is approved, implemented, tested, installed, etc. Typic_Jl}'

there are more outstanding OSMRs than resources. A Project Ta_k Leader is responsible (or

allocating these resources.

OSMRs may be filed for several rea._ons. Acceptance Testing may reveal the need for

enhancements (corrections are still the responsibility of the Software Developers) Lau, r The

Users (same organization unit as maintainers) may request enhancements or identify :he riced f,_r

corrections or adaptations. The Specification Developers may also initiate changes, resulting from

idea_ about similar forthcoming systems. Or, the Project Office may modify the Project

Requirements.

There are three software libraries: Working, Testing, and Operational. Entries in ¢.he

Working library have not yet been accepted and may not be final. SeverM changes are made to

the Testing library at once. These changes are te_ted together. The Operational library is very

stable. Three months prior to launch, the Operational library is frozen. Maintenance

nevertheless continues (the freeze is lifted after launch).

5642

8

2-10
Project

Requirements _ Specifi!a_ion

Analysis Development ]

Software _ I Acceptance

: _-t Maintenance

/

Operational

Support

Figure 2. The software life-cycle in the NASA/GSFC Flight Dynamics

Division. Organizational units are separated by the dotted lines.

5642
Not MI projects conform to the genera/ models, but the models provide a common reference

for tailoring procedure_ to specific projects and making comparisons or generalizations across

projects.

3.2. MAINTENANCE IMPROVEMENT GOALS

We have identified a set of important improvement goals and refined them into quantifiable

questions and metrics according to the GQM paradigm. This subsection presents some

representative goals and questions. We have not included the complete set of goals, questions

and metrics according to the GQM templates suggested in iBasili88!, but only highlight selected

ones. In the questions that refer to specific metrics, the metrics are italicized.

Characterizing maintenance

We study the maintenance process itself to see how maintainers spend their time and wha_

_hey do. We stud[ the entire software life-cycle to understand how Specification Developers.

Software Developers, Users, and Maintainers communicate: why changes are made; and whether

the organizational divisions result in the best use of personnel's skills and knowledge.

Currently we are interested in the following questions:

(1) What are the major maintenance activities? Wha_ are the major software life-cycle

activities? What are the major communication links between activities?

(2} How is productivity related to type3 of change_ (corrections, enhancements, adaptations},

and characteristics of the product (type of product, LOC_? How is effort (hours) distributed

across various activities?

5642

Io

2-12
Charaeteri_.ing the delivered product

The quality of the delivered product influences both what changes will be performed and the

amount of effort that will be required. We therefore characterize the products with the following

objectives in mind: to understand how and why .the product changes: to understand how the

product influences productivity during a change; and to provide historical, baseline data for

future projects.

We are currently interested in characterizing each of the three types of software products in

terms of the following:

(3) What are the static characteristics of each product (#LOC. #components, ._ystem

architecture, programming lanquaqe, types of documents)? What are the functional

characteristics of each?

(4) What types of changes are made (in terms of how both static and functional characteristics

of the system are affected).

Improving maintenance

The maintenance process can be improved by focusing on the maintenance activities

themselves, or by improving the entire software life-cycle of which maintenance is a part.

We are currently interested in the following specific possibilities:

(5) Establishing communication from Maintainers to Software Developers for feedback about

product maintainability.

(6) Providing management with better mechanisms for monitoring the process.

Improving the developed products

Because of the relatively short maintenance phase, improvements to the products will be

directed primarily toward the development of future products.

5642

II

2-13
(7)

(s)

The following ideas have been proposed for improving the product:

The debug code is not well designed from the Maintainers' perspective. Future designs

should allow the Maintainers more control over which messages are turned on.

We are trying to learn more about the relation between system structure and the locality of

changes. A significant number of changes affect five or more files.

3.3. DATA COLLECTION AND VALIDATION PROCEDURES

The measurement procedures presented in this subsection support the stated improvement

goals within the SEL maintenance environment. These procedures include data collection, theLr

validation, analysis, and feedback.

We routinely monitor the effort associated with various maintenance activities, and other

characteristics of the changes. Similar data is available from development. This data will be

used to characterize the maintenance process, the types of changes made to the product, and the

reasons for making the changes.

Routine data collection Ls implemented primarily through the use of forms (Fig. 3). At _he

end of each week, project personnel each complete a Weekly Maintenance Effort Form (\_,._[EF)

which briefly summarizes how they spent their time according to type of changes i correction,

enhancement, adaptation, or other) and maintenance activity (isolation, implementation, uni_

test, integration test, other). Upon completion of each change, a Maintenance Change Report

Form (MCRF, Fig. 4) is filed. The MCRF summarizes the change from a user's perspective

(reason for change and functionality) and from the programmer's perspective (effort spent, parts

of the system modified, etc.). A history of development (phase dates, effort) and product

characteristics (size, number of subsystems, etc.) is summarized on a Project Completion

Statistics Form (PCSF). This data will be made available at the end of development. It will be

5642

12

2-14
compiled through the use of programs which examine the software library and the SEL database.

The data collection forms and procedures reflect our models of the environment, the

maintenance process and products, and measurement procedures. For example, the W_[EF is

filed only by Maintainers (as defined in Fig. 2); Specification Development and Use do not

contribute to the hours monitored {although these hours axe charged to the project); and

Changed Objects (MCRF, section 13) refers to specific documents used in this environment.

Project Task

Leader

Programmers

MCRF

WMEF

._ SEL Library

(Form Validation)

Figure 3. Routine data collection and validauon procedures. WNIEF is

collected weekly from all project personnel (Project Task Leaders, Maintainers.

Managers). MCRF is collected once for each change. These forms are validated,

analyzed, and stored in the SEL, an independent entity.

5642

13

2-15
ORIGINAL PAGE IS

OF POOR QUALITy

MAINTENANCE CHANGE REPORT FORM

OSMR Number:

Date:

SECTION A: Change Request Information

Furctlm/O_¢muorv o_ Cl'annge:

Whm wm me type _ _?

Corra_lon

Enhancement

k:lal_tlon

Whaul catana¢lthe r.r,an_e ?

m Raq_rmmen_si_wm"ca.or_

Software O_s0gn

Codo

Previous r..hange

01her

SECTION B: Change Implementation Inforrnatlon

Components Changed/Adcled/Oeleled:

E_mate the effort spent Iso_/deteffnJntng me c_smgo:

Estimate the effort to damign, Implement, and test the change:

lhrto ldayw 1_to

,tlhr 1day lweok 1 mor'_

C_m¢_ aa ct_anged ol_ec_:

-- Requltwnenl_r,'Sl:_clflcations I_,cumerrl

IDeaUQnOocurnent

Code

Symern Oeg:rt;_n

Usaw's Gui_e

Other

If code c_ecL, ¢haractetlzm the change (c.ecK most

ap_k:alm)

_lon

LogC:controJ sxmczum

(o.o.changed flow of comro:)

(rnodlJi to module c_rnmunlcatJon )

inter/ace (external)

(modlJkl to extema/¢_'nmunicatlon)

Data (vakm or stru<t_m)

(e.g., varta_le or value (:_angeo)

compuuuonaJ

(04.. cJ'tange of math expre_ion)

Otl"dN"(none Of the al:x_ve ;M:)pIy)

E_m me _ o_ Iinu o_ axle (Including comments):

m ctlangld

Entw tl'Nm_ of _:

added rJtanged deleted

Ento¢ the numbe_ o/me added compomm_ thai am

_Wy rmv _t=_/rmua_

m

doIotod

rm._eo wr_

modl flcatlo.-,s

_n

Figure 4. The MCRF. A change report summarizes the change from two

perspectives: the functional perspective ('b]ack box'), and the structural

perspective ('white box').

14

5642

2-16
3.4. COLLECTED MAINTENANCE DATA

We began monitoring maintenance projects on a trial basis in October, 1987 (Fig. 5). Our

initial understanding of the environment reflected many biases from our knowledge of the

Software Development process. Prior to October, 1988, the MCRF had emphasized corrective

maintenance, did not request separate functional and structural descriptions of changes, and the

OSMRs were not monitored. Other minor revisions were also made to the forms and procedures.

The latest revision was made in January, 1989. Figure 6 summarizes the number of forms filed in

total across the various projects.

1987

PROJI

PRO J2

PRO J3

PRO J4

PRO J5

ONDJFMAMJJASONDJF

ct ov ec an g a.p r r ya un ? ug ep ct ov ec _n g

V0 V1

:11

V2

1989

Figure 5. Five maintenance projects have been monitored on a trial basis since

October, 1987. Two changes to the data collection forms took place in October,

1988, and January, 1989.

l

PRO J1 PRO J2 PRO J3 PRO J4 PRO J5 I

MCRFs WMEFs 26671 8618 1228 150 3,°0_

Figure 6. Forms received. PRO J3 and PRO J4 have not been continuously

active.

5642

15

2-17
01/08/88 " ***"

01/15/88 " "**

01/22/'88 " * * "**"

01/29/88 ****

02/05/88 * * * ****

02/12/88 * * * ****

02/19/88 * * ***

02/26/88 * " **"

03/04/'88 * * **"

03/11/88 * ****

03/18/88 * " * *'***

03/25/88 " ****

04/01/88 * ***

04/08/88 * **'*

04/15/88 * * " ****

04/22/88 " 0 " "''*

04/29/88 "* * ***

05/06/88 ** " " **'*

05/13/88 * " '

05/20/88 * * * *****

05/27/88 ** * ** ****

06/03/88 * * " ****

06/10/88 * * * ****

06/17/88 ..... **

06/24/88 * * " "***"

07/01/88 * * * ***

07/08/88 * * 0 **'"

07/15/88 0 0 0  

07/22/88 o o o ** 

07/I29/88 * 0 0 *

08/105/88 0 0 **

08/12/88 * 0 **

o8/19/88 • o o o

08/26/88 " * "'*

09/02/88 "*" * **

09/09/'88 * 0

09/16/88 * *

09/23/88 * *

09/30/88 *

1o,/o7/88 • *

1o/14/88 " o

10/21/88 * *

1o/28/88 o o

11/04/88 0 ***

DATE NI N2 N3 P4

**Z

*Z

0

i

0

0

0

O

P5

*ssm

***s

**m.

0

0

0

P6

_sJl

*zw_

.11z

_**Iz

**z

0

0

0

0

0

PT

i

=

Z

.i

=

z

=

=

=

=

o

0

0

0

0

0

0

SIS

0 = 0 hours (form submitted)

* ----up to I0 hours

** ---_up _ 20 hours

*** = up to 30 hours

  = up to 40 hours

** *** -----up to 50 hours

For each week of the project, to_alexpended effortisshown distributed over the

various personnel. On this project, Maintenance was contracted out to CSC

(CSC manager: M8; CSC programmers: P4 - P7). .Although NASA personnel

NI - N3) were primarily responsible for SpecificationDevelopment,, some hours

meetings, consulting, etc.)were a_tribu_ed to Maintenance.

Figure 7. Weekly Effort, Hist,ograms (PRO.It)

5642

2-18
Real-time analysis of the data is not yet used by the Project Tasks Leaders or managers,

but figure7 suggests one way effortmight be monitored. It shows for each week how much total

efforthas been invested by the various personnel.

Figures 8 through I0 profilesome overall trends. The FORTRAaN subroutines in these

products are not small, therefore entirecomponents are seldom added or deleted.

MCRF

Total LOC LOG LOG Total Files Files Files i

LOC Added Deleted Changed Files Added Deleted Changed

PROJI ] 81K 2484 430 1353 _ 3 3 335 i

PRO J2 I 46K 2323 325 354 433 10 0 [07

PRO J3 ] 52K i0 0 55 242 0 0 0

PRO J4 I 37K 0 0 0 322 0 0 0

PRO J5 I 176K 0 0 0 _ 0 0 o

Figure 8. This table summarizes data from section B of the MCRF. The totals

refer to size at delivery ("?" refers to unavailable PCSF data). Files (usually

single subroutines) are called "components" on the MCRF.

3.5. INITIAL ANALYSIS RESULTS

In the initialphase of improvement, analysis results frequently reveal limitations in _he

measurement procedures and forms as well as the naivete" of early goals and questions.

Obviously, revealing these [imitations and misconceptions is the firststep toward improvement.

The following examples demonstrate how one's understanding and models develop through the

analysis of data. These analyses are based on the current goals, questions,and data.

(I) How expensive is maintenance compared to development?

So far the costs have been low compared to figuresoften quoted in research literature{Fig.

9). There are two very good reasons for this. First, maintenance, as defined in this environment.

17

2-19

5642
does not include Operational Support (during which software changes are presumed infrequent).

Second, excepting PRO J1, none of the projects has completed. Also note that time spent in

Specification Development (during the Maintenance phase) is not included.

(2) What types of changes are made?

Figure 10 shows the distribution of time spent on various types of maintenance over each of

the projects. There are a few significant limitations to this data which make most generalizauons

premature: I) the "other type* category was not included on the WNIEF until October, 1988;

time spent on meetings and management was therefore forced into one of the available categories;

2) there is little total data from some projects; 3} PRO J1 was "maintained" by the original

Software Developers; 4) there is no data summarizing those change requests which were not

tmplemented.

PRO J1

PRO J2

PRO J3

PRO J4

PRO J5

Development .\[aintenance

Hours Hours

17K 3K

I8K 2K

6K 0.2K

12K 0 IK

47K 0.5K

Figure 9. A comparison of total technical and management hours. (PRO J2 was

in maintenance before Oct, 1987 when measurement of maintenance began.)

5642

18

2-20
Total Correction Enhancement Adaptation Other*

PRO J2 [ 2328 44% 54% 0% 0%

PRO J3 [ 234 58% 35% 3% 3%

PRO J4 [ 132 0% 87% 3% 10%

_PROjS_k 459 10% 78% 2% 10%

Total isolation Change Implement Unit Test IntegraHours Design___ tion Test

PRO J2 | 2328 15% 15% 19% 6% 30%

PRO J3 ] 234 53% 12% 23% 6% 4%

PRO J4 [ 132 45% 15% 24% 15% 0%

_P_R_O_ J._5_1 459 15% lo% t5% 4% ..... i6____ _

Figure I0. These two tables show the distributions of effort (WMEF) by type

of change and activity.

* "Other" was only recently added to section B of the WMEF. Most of the I3%

adaptation on PROJI actually represents management.

4. LESSONS LEARNED

The lessons learned from our efforts to establish the improvement program for SEL's

maintenance environment address (i) why the introduction of measurement is significant, (ii) how

well the improvement approach worked, (iii) how we built the credibility of our program, and (iv)

automated support.

4.1. WHY IS THE INTRODUCTION OF MEASUREMENT SIGNIFICANT:

It was again apparent from this study that the introduction of measurement into an

industrial setting represents not just the introduction of another method. Instead it signals a

5642

19

2-21
dramatic change in the organization toward a more engineering oriented software development

and maintenance style. Such a change affects all levels of an organization. Most organizations

are not ready for this kind of change. As a consequence, the initial phaze of the improvement

program must be sensitive to the need of selling measurement as a credible and promising

mechanism.

Specific lessons learned:

LI: Introducing measurement represents a major shift toward a more engineering oriented

software development and maintenance style.

L2: Most environments are not ready for systematic, measurement basedimprovement.

L3: Special effort must be made to build the credibility of the selected improvement _oals

and measurement procedures before measurement is attempted on a large scale.

The SEL adopted measurement as a means for routine improvement of its development

activities over a decade ago. Still, during the introduction of measurement to the SEL s

maintenance activities we encountered the need for selling measurement to a new audience.

Initially, it was not clear whether we should aim to reduce the need for maintenance through

better Specification Development, making a more maintainable product, or using more _trmgent

acceptance testing. Our current goals were influenced as much by management's receptiveness to

our ideas as by our technical understanding of the maintenance process.

4.2. HOW WELL DID THE IMPROVEMENT APPROACH WORK DURING THE

INITIAL PHASE?

It became apparent again from this study that the basic principles of the improvement

paradigm not only apply to the initial program phase; they are even more important for

organizing learning the higher the level of uncertainty is. On the other hand, varying levels of

5642

2o

2-22
understanding of the environment seem to justify (even require

the paradigm.

Specific lessons learned:

different procedures for applying

L4: It is important to distinguish improvement methodologies (which depend on the

environment and the maturity of the program) from principles of the improvement

paradigm (which emphasize explicit learning through the use of measurement). .An

expected, the improvement paradigm was extremely helpful during the initial phase:

however, the steps taken had to be modified according to the maturity of the program

and the need to demonstrate its value.

L5: Most initial learning came from exploratory investigations (e.g., meetings, interviews) and

was based on subjective and intuitive data.

In the initial phase, we learned more from attempting to implement the measurement

procedures than from the data they provided. For example, many of the environment problems

were first revealed to us in exploratory meetings with maintenance personnel. Those meetings

provided the focus which enabled us to follow up on some of these issues in a much more goal

oriented fashion (including the use of measurement data}. Subjective data are very important

during the initial program phase when our understanding does not allow for the definition of

objective metrics, or when the underlying goal does not require or justify the cost of collecting

objective data.

Our application of the improvement paradigm can be characterized a_ prototyping. It

allowed for feedback loops at any time. Sometimes our understanding of the environment was

improved when refining goals into questions, metrics, and measurement procedures. An example

of this is how we learned about the maintenance libraries. Given the use of formal change

requests, we assumed that changes were well defined and that upon completion of a change, a

corresponding change form (MCRF) would be filed. Eventually we learned that changes were

being made Ithe hours showed on weekly formsj, but we were no_ receiving NICRFs. This

21

2-23

5642
inconsistency was only obvious when a new project started. "Completed" changes were being

entered in the working library, but forms were not filed until after several changes were made,

transferred to the test library, tested and approved. Although, we sometimes identified

interesting metrics based on prior experience or intuition, we always eventually justified such

metrics in the context of some improvement goal.

4.3. HOW DID WE BUILD THE CREDIBILITY OF OUR PROGRAM?

Although the actual collection and interpretation of data w_s not the objective of the initial

program phase, we used it on _ trial basis in order to design effective measurement procedures

and to identify further needs for improvement. It, is very hard to convince anyone that you atfocusing on the right problems without actually providing some objective evidence that those

problems actually exist. It is also difficult to convince someone that a program will be effective

without demonstrating that the planned measurement procedures can be implemented in the

current environment. The techniques used to establish procedures and demonstrate their

credibility are important to the success of the program.

Specific lessons learned:

L6: The environment needs to be modeled at a level of detail that enables us to demonstrate

that (i) the chosen goals are justified, (ii} the derived metrics support those goals, and (iii}

the planned measurement procedures can be implemented in the given environment.

LT: Trial data collection and validation may be needed to establish confidence that planned

measurement procedures are effective and that the identified goals address significant

problems.

L8: Depending on the initial level of understanding, data collection may be less accurate.

objective, and complete during the initial phase than during routine application.

5642

22

2-24
L9: The participation of all people concerned adds to the credibility of the program.

We involved representatives from several organizational levels. They helped build initial

hypotheses based on their insights, and served as reviewers of the results produced. Their

comments were solicited,because without theirconfidence that we were addressing the right issues

in a feasibleway, we could not expect theircooperation during actual data collection.

4.4. HOW MUCH AUTOMATED SUPPORT iS NECESSARY?

This study also illustrated the need for automated support. During the initial phase, simple

tools are needed for storing and analyzing the measurement data. A. database system, statistics

package, and report generator will become more important as the measurement procedures

stabilize and the volume of data increases. Tools for modeling and planning could be very helpful

provided they effectively support change.

Specific lessons learned:

L10: Automated support for measurement is less important during the initial phase, but will

be required during the routine phase.

Lll: Graphical models (e.g., Fig. 2), plans (c.g,, relating goals, questions, and metrics}, and

raw data are unwieldy and numerous. In addition to a database, automated support for

designing and managing graphical structures would be extremely useful.

During the initial phase, data collection forms were not stored in a database but simply in

folders, and could not be analyzed automatically. A.s a result it was not always easy to keep

track of forms and we might have lost some. This does not cause severe problems during the

initial program phase, but would during the routine phase when reliance on this data is greater

We are currently in the process of expanding the SEL development database to include

maintenance data.

5642

23

2-25
5. CONCLUSIONS

In this study we have followed the principles of the improvement paradigm while

introducing an improvement program to the SEL maintenance environment. We have observed

that unlike a well conceived experiment and unlike an environment with a history of using

measurement, this maintenance environment required a period of bootstrapping. Several rapid

improvement (or learning) cycles were required to create the initial understanding of this

environment necessary to identify meaningful goals and to design effective measurement

procedures. Using measurement on a trial basis is also important for building the credibility of

the improvement approach before attempting to apply it routinely on a large scale.

We have completed the initial program phase in which the goals and measurement

procedures presented in this paper have been demonstrated to justify routine application to all

maintenance projects within the SEL. The SEL is now providing routine support for improving

maintenance, including data collection, form validation, and database support.

Establishing the SEL maintenance improvement program has been mainly a technology

transfer problem. We tried to import existing technology, and customize it to the _pecific needs

within the SEL. During the course of this study we have identified several problem areas _haL

cannot be solved with existing technology but require additional research. These problem area_

include more formal means for (i) capturing our understanding of an environment, (ii) packaging

it into project specific, domain specific and general knowledge. (iii) relating measurement and the

objects of measurement (i.e. processes and products), (iv) tailoring existing models to specific

needs, and capturing the modeling process itself. Independent of the SEL maintenance

improvement program, other research at the University of Maryland is addressing some of these

issues: the TAME (Tailoring A Measurement Environment) project [Basili88] focuses on problems

(ii) - (iv). The MW'P (Multi-View Process Specification) project iRombach89! focuses on problems

(i) and (iii).

5642

24

2-26
6. ACKNOWLEDGEMENTS

We thank Frank McGarry and Jon Valett (NASA/GSFC) for supporting this project from

the very beginning, all the NASA and CSC maintenance personnel for putting up with several

evolving versions of data collection forms and for giving us valuable feedback, and the librarians

and database administrator of NASA/GSFC for handling the logistics of so many daLa collection

forms.

5642

25

2-27
7. REFERENCES

Basili84

Basili85

Basiti85b

Basili87b

Basili87

Basili88

Brelsford88

Brophy87

Grady87

Grady87b

Katz86

McGarry85

Nehmer87

V. R. Basili and D. M. Weiss, "A Methodology for Collecting Valid Software

Engineering Data," IEEE Transactionn on Software Engineering SE-10(6),

pp.728-738 (November 1984).

V. R. Basili, "Can We Measure Software Technology: Lessons Learned from

Eight Years of Trying," ProceedingJ Tenth Annual Software Enginecr2n9

Workshop, NASA Goddard Space Flight Center (December 1985).

V. R. Basili, "Quantitative Evaluation of Software Engineering Methodology,"

First Pan Pacific Computer Conference (September 1985).

V. R. Basiti and H. D. Rombach. "Tailoring the Software to Project Goals and

Environments," Proc. Ninth International Conference on Software Enqmeer, nq,

pp.345-357. (March 30-April 2, 1987).

V R. Basili and R. W. Selby, Jr., "Comparing the Effectiveness of Software

Testing Strategies," [EEE Transactions on Software Engineering SE-13It2),

pp.1278-1296 (December 1987).

V. R. Basili and H. D. Rombach, "The TPuME Project: Towards ImprovementOriented Software Environments," [EEE Trannactions on Software Eng_necrm9

SE-14(6), pp.758-773 (June 1988).

J. Brelsford, "Establishing a Software Quality Program," Quality Progr_,s

21(11), pp.34-37 (November 1988).

C. Brophy, W. Agresti, and V. R. Basili, "Lessons Learned in t:_e ,_f .-\da

Oriented Design Methods," Proceedings Joint Ada Conference. pp27,1-235

(March 16-17, 1987).

R. B. Grady, "Measuring and Managing Software Maintenance," [EEE Software

4(5), pp.35--45 (September 1987).

R. B. Grady and D. L. Caswell, Software Metrics: Establishing a Company-Wide

Program, Prentice-Hall, Inc., Englewood Cliffs, New Jersey (1987).

E. E. Katz, H. D. Rombach, and V. R. Basili, "Structure and Maintainability of

Ada Programs: Can We Measure the Differences?," Proceedings 9th

Minnowbrook Workshop on Software Performance Evaluation (August 1986).

F. E. McGarry, "Recent SEL Studies," Proceedings Tenth Annual Software

Engineering Workshop, NASA Goddard Space Flight Center (December 1985).

J. Nehmer, D. Haban, F. Mattern, D. Wybranietz, and H. D. Rombach, "Key

Concepts of the INCAS Multicomputer Project," IEEE Transactions on Software

Engineering SE-13(8), pp.913-923 (August 1987).

5642
Ramsey88

Rombach86

Rombach87

Rombach88

Rombach89

Selby87

Weiss85

C. L. Ramsey and V. R. Basili, "Expert Systems for Software Engineering

Management: A Summarized Evaluation," in Ezpert Systems and Advanced

Data Processing, ed. M. L. Emric h, A. R. Sadlowe, and L. F. A.rrowood, Elsevier

Science Publishing Co., Inc. (1988).

H. D. Rombach, V. R. Basili, and R. W. Selby, Jr., "The Role of Code Reading

in the Software Life Cycle," Proceedings 9th Minnowbrook Workshop on Software

Performance Evaluation (August 1986).

H. D. Rombach and V. R. Basili, "A Quantitative Assessment of Software

Maintenance: An Industrial Case Study," Proceedings Conference on Software

Maintenance-1987, pp.134-144 (September 21-24, 1987).

H. D. Rombach and B. T. Ulery, "Improving Software Maintenance through

Mea-*urement," Technical Report CS-TR-2131, Dept. of Computer Science,

University of Maryland, College Park, Maryland (October 1988). Published as

an invited paper, IEEE Proceedings, April 1989.

H. D. Rombach and L. Mark, "Software Process & Product Specifications: A

Basis for Generating Customized SE Information Bases," Proceedings TwentySecond Annual Hawaii International Conference on System Sciences II (1989).

R. W. Selby, Jr., V. R. Basili, and T. Baker, "CLE,M'_-ROOM Sohware

Development: An Empirical Evaluation," [EEE Transactions on Software

Engineering SE-13(9), pp.1027-1037 (September 1987).

D. M. Weiss and V. R. Basili, "Evaluating Software Development by Analysis of

Changes: Some Data from the Software Engineering Laboratory," [EEE

Transactions on Software Engineering SE-11(2), pp.157-168 (February 1985).

5642

27

2-29
UMIACS-TR--89-48

CS-TR-2244

May, 1989

Maintenance = Reuse-Oriented Software

Developmentt

Victor R. Basili¢

Institute for Advanced Computer Studies

Department of Computer Science

University of Maryland

College Park, MD 20742

ABSTRACT

In this paper, we view maintenance as a reuse process. In this context, we discuss a set of models that can be used to support the maintenance process. We present

a high level reuse framework that characterizes the object of reuse, the process tbr

adapting that object for its target application, and the reused object within its target application. Based upon this framework, we offer a qualitative comparison of the three

maintenance process models with regard to their strengths and weaknesses and the circumstances in which they are appropriate. To provide a more systematic, quantitative

approach for evaluating the appropriateness of the particular maintenance model, we

provide a measurement scheme, based upon the reuse framework, in the form of an organized set of questions that need to be answered. To support the reuse perspective, a

set of reuse enablers are discussed.

% This reseatreh w_ supported in part by NASA grant NSG--512.3 and ONR grant N00014-87-K--0307 to the Uruverslty of Maryland.

_: Keynote address, conference on Softwnr_ Maintenance, Phoenix, AZ, Octoqber 1988.

5642

2-30
Maintenance = Reuse-Oriented Software Development

Victor R. Basili

Institute for Advanced Computer Studies

Department of Computer Science

University of Maryland

[Keynote Address, Conference on Software Maintenance,

Phoenix, AZ, October 1988.]

Abstract

In thispaper,we view maintenanceas a reuseprocess,In thiscontext,we discussa setof models that

can be used to support the maintenance process. We present a high level reuse framework that characterizes the

- object of reuse, the process for adapting that object for its target application, and the reused object within its target application. Based upon this framework, we offer a qualitative comparison of the three maintenance process

models with regard to their strengths and weaknesses and the circumstances in which they are appropriate. To

provide a more systematic, quantitative approach for evaluating the appropriateness of the particular maintenance

model, we provide a measurement scheme, based upon the reuse framework, in the form of an organized set of

questions that need to be answered. To support the reuse perspective, a set of reuse enablers are discussed.

Introduction

If we take the view that software should be developed with the goal of maximizing the reuse of prior

experience in the form of knowledge, processes, products and tools, then the maintenance process is logically

ideally suited to a reuse-oriented software development process. There are a variety of reuse models. The key

issue here is which process model is best suited to the particular maintenance problem at hand.

In this paper, we present a high level organizational paradigm for software development and maintenance

in which an organization can learn from prior and current development and maintenance tasks and then apply

that paradigm to several maintenance process models. The paradigm has associated with it a mechanism for setring goals that can be measured so that the organization can evaluate the process and the product and learn from

its experience for future projects or enhancements of the current project.

We begin by identifying three process models that can be used for maintenance. We then present a high

level reuse framework that characterizes the object of reuse, the process for adapting that object for its target

application, and the reuse object within its target application. Based upon this framework, we offer a qualitative

comparison of the three maintenance process models with regard to their snxnghts and weaknesses and the circumstances in which they are appropriate. To provide a systematic, qnantitative approach for evaluadng the

appropriateness of the particular maintenance model, we provide a measurement scheme, using the

Goal/Question/Metric Paradigm. Since reuse requires a supportive environemnt, a set of environmental reuse

enablers are discussed.

Maintenance

The nature of software is that it can be modified without the use of physical tools such as screw drivers

and soldering irons. This has lead to the false assumption that maintenance is easy and inexpensive. Clearly

nothing could be further from the truth.

Most software systems are complex and modification requires a deep understanding of the functional and

non-functional requirements, the mapping of functions to system components, and the interaction of the

5642

2-31
components. Without good documentation of the requirements, design and code with respect to function, traceability and su=ucmre, maintenance becomes a difficult, expensive, error-prone task. As early as 1976, Belady and

Lehman reported on the problems with the evolution of OS 360 [7]. The literature is filled with similar experiences and lessons learned [10,12,16,182.0].

Main_mnco consists o_"several different types of activRie.s: correction of faults exisdng in the system,

the adaptation of the system m a changing operating envimnmem, e.g., new terminals, operating system

modifications, era., and changes m the original requirements. The new system is like the old system but

different in a specific set of characteristics. One can view the new version of the system as a modification of

the old system or a new system which reuses many of the components of the old system. Although these two

views have many aspectsin common, theyam quit_differentwithr_t totheprocessmodels usedand their

effectson futureenvironments.

In fact,wc can identify at leastthr_ processmodels associatedwith maintenancedepending upon the

characteristics of themodification.We willcallthese(1)the quickfixmodel, (2)the itcrativcenhancement

model,and (3)thefullreusemodel. All threemodels reusetheold systemand so arcreuse-oriented.Which

model shouldbc chosenforany particularmodificationisa combinationof management and technicaldecisions.

QuAck Fix Model. The quick fix model involves taking the existing system, usually just the code, and making

the necessary changes to the source code and the accompanying documentation, e.g. requirements, design, and

recompiling the system as a new version. This may be as straightforward as a change to some internal component, e.g. an error correction involving a single component or a structural change or even some functional

enhancement. Here reuse is implicit.

Old System New System

Requirements Requirements <--

Design Design < ........

I

Code ............. > Code .......... >

I

Test Test < .........

Figure 1. Quick Fix Process Model

lterative Enhancement Model. Iterative Enhancement [5"]is an evolutionary model which was proposed

for software development in environments where the complete set of requirements for a sYstem were not fully

tmdersuxxi _ the author did not know how to build the full system. Although it was proposed as a development model, it is well suite_l to mammnance. The process model involves:

1. Starting with the existing system requircmems, design, code, test and analysis documents

2. Redeveloping smmng with the appropriam document based upon analysis of the existing system, propagating

the changes through the full set of documents

3. At each step of the evolutionary process, continuing to redesign, based upon analysis.

5642

2-32
Old System

Requirements

Design

I

Code

I

Test

I

Analysis

New System

> Requirements .... >

Design I

I

Code i

f I

Test i

I I

Analysis

Figure 2. Iterative Enhancement Model

To view this as a maintenance model, assume the initial implementation is the system in its current state

in the evolutionary maintenance process. The process assumes that the maintenance organization has the ability

to analyze the existing product, characterize the proposed set of modifications, and redesign the current version

where necessary for the new capabilities. Again, reuse is implicit.

Full Reuse Process Model. While iterative enhancement starts with evaluating the existing system for

redesign and modification, a lull re,use process model starts with the requirements analysis and design of the new

system, with the concept of reusing whatever requ_ements, design and code are available fram the old system.

The reuse process model involves:

1. Starting the requirements for the new system, reusing as much of the old system as feasible

2. Building a new system using components fram the old system or other systems available in the repository.

developing new components where appropriate.

Old System Repository New System

Requirements --> {Ri} < ...... > Requirements

I I

Design ........ > {Di} < ...... > Design

I I

Code -> {Ci} < ...... > Code

I I

Test .......... > {Ti} < ...... > Test

Figure 3. Full Reuse Process Model.

Here reuse is explicit, packaging of prior components is necessary and analysis is required for the selection of the appropriate components.

5642

2-33
The differencebetween the lasttwo approaches ismore one of perspective.The fullreuse model freesthe

developer to design the solution relativeto the available set of solutions of similar systems. The iterative

enhancement model takes the lastversion of the current system and enhances it. Both approaches encourage

redesign, but the fullreuse model suggests a wider forum and can lead to the development of more reusable

components for furoresystems where the iterativenhancement model suggests the tailoringof an existingsystem for the given extensions.

A Reuse Framework

The existence of several models for maintenance raises several questions. Which is the most appropriate

model for a particular environment? a particular system? a particular set of changes? the task at hand? How

do I improve each step in the process model I have chosen? How do I minimize overall cost and maximize

overall quality?

In order to answer these questions we need a model of the object of reuse, the process of adapring that

object for its target application, and the mused object within its target application. A simple model for reuse is

given in figure 4. In that model, an object is any software process or product and a transformation is the set of

activities that are performed in musing that object. Given that the scope of the new application are understood,

the stepsare:

I.identifyingthe candirl_te reusable piecesof the old object

2. understanding them

3. modifying them to our needs

4. integratingthem intothe process

* context

* I old 1 I new !

*l object I ...... >I transformation I ..... >i objec_ [ "

I repository [

Figure 4. A Simple Reuse Model

To flesh out the model, we need a framework for categorizing objects, transformations, and context. The

framework should cover various categories e.g. reuse object: process, product, Within each category, there are

various classification schemes for produce e.g. requirements document, code module, test plan, and process: e.g.

cost estimation, risk analysis, design.

There arc a variety of approaches to reuse and schemes that classify the object of reuse

[8,9,11,13,15,17,19].We use here a variationof a muse framework [4]thatcaptures severalaspects of the reuse

process, product and context.

Object dimensions include:

5642

2-34
(I) Reuse Object Type: What is a characterization of the candidam reuse object? Sample categories and

classifications are: process (e.g. design, tesO and product (e.g. application, tool).

(2) Self-Contalnedness: How independent and understandable is the candidate reuse object? Sample categories

and classifications are: syntactic independeiace (e.g. tightly coupled), semantic independence (e.g. similar

functionality), and precision of specification (e.g. formal, informal).

(3) Reuse Object Quality: How good is the candidate reuse object? Sample categories and classifications are:

maturity (e.g. newly developed, used in one application) complexity (e.g. low cyclomatic complexity) reliability (e.g. no failures during prior use).

Context dimensions include:

(1) Requirements Domain: How similar are the requirements domains of the candidate reuse objects and

current or future projects? Some example categories and classifications are: application (e.g. ground support software for satellites), distance (e.g. same application, similar algorithms/different problem focus).

(2) Solution Domain: Howsimilar are the evoloution process which resulted in the candidate reuse objects and

the ones used in the current and future projects? Some example categories and classifications ate: process

model (e.g. waterfall model) design method (e.g. function decomposition) programming language (e.g.

FORTRAN).

(3) Knowledge Transfer Mechanism: How is information about the candidate reuse objects and their context

passed to current and future projects? An example classification is: humans (e.g. subset of the development team doing maintenance, separate team doing maintenance).

Transformation dimensions include:

(I) Type of Trausoformations: How do we characterize the transformation activities to be performed? Some

sample categories and classifications are: Percent of Change (e.g. 0%, 5%), Direction (e.g. general to

domain specific, project specific to domain specific), mechanism of modification (e.g. verbatim, parameterization, template-based, unconstrained) and mechanism for identifying (e.g. by name, by functional requirements).

(2) Activity Integration: How do we integrate the transformation ._,'tivities into the new system development?

Some sample categories and classifications are: phase activity performed in the new development planning

(e.g. cost estimation, risk analysis), construction (e.g. requirements development), analysis (e.g. testing).

(3) Transformed Quality: What is the contribution of the reuse object in the context of the new system with

respect to the objectives set for it? Sample category and classifications axe: reliability (e.g. no failures associated with that component) and performance (e.g. satisfies the timing requirement).

Comparing the Models Using the Reuse Framework

When applying the reuse framework to the maintenance process, we are focusing on a set of reuse objects

that are product documents. Let us compare the various models according to the dimensions given.

Consider the reuse object dimensions:

With regard to reuse object type, the object of the quick fix and iterative enhancement models is the set

of documents representing the old system: The object of the full reuse model is the repository including the

old system.

With regard to self-conminedness, all the models depend upon the unit of change. The quick fix model

depends upon how much evolution has taken place since entropy may have unstructured the system. In iterative

enhancement, the evolved system should be improving for the specific application and f_" the appropriate set of

changes, the unit of change should be more visible. In the full reuse model, the evolved system should be

improving with respect to reuse object independence for the general application, depending upon the quality and

maturity of the repository.

With regard to reuse object quality, the quick fix model offers little knowledge of the quality of the old

object. In iterative enhancement, the analysis phase provides a fair assessment of quality with respect to the

particular application. In full reuse, we have an assessment of quality of the reuse object across several systems.

5642

2-35
Consider the context dimensions:

With regard to the requirements domain, the quick fix and i,,'rative enhancement model assume the same

application, in fa#t the same project. The full reuse model allows for manageable variation in the application

domain, depending upon what is available in the repository.

With regard to the solution domain, the quick fix model assumes the same solution structure exists during

maintenance as during development. There is no change in the basic design- or structure of the new system. In

the iterative enhancement model, becatmc redesign is a part of the model, there is some modification to the solution smacture allowed. The full reuse model allows major differences in the solution structure, i.e. a complete

redesign is possible going from functional decomposition to object oriented design.

With regard to knowledge transfer mechanism, the quick fix model and iterative enhancement work best

with the same people. The full reuse model can compensate for having a different team, assuming we have

application specialists and a well documented reuse object repository.

Consider the wansformation dimensions:

With regard to type of activities, the quick fix model typically uses a source code look-up, reading for

understanding, unconstrained modification and re-compilation approach, lteradve enhancement typically begins

with a search Oh'ough the highest relevant document, changing it and continuing through the subsequent documents using a variety of modification mechanisms. The full reuse is uses a library search, and a variety, of

modification mechanisms depending upon the type of change. Here modification is done off-line.

With regard to activity integration, in the quick fix model, all activities are performed at same time. herauve enhancement associates the activities with all the normal development phases. In the full reuse model,

identification of the candidate reusable pieces is done during project planning and the other activities are done

during development.

With regard to transformed quality, the quick fix model usually works best on small well-contained

modifications since their affect on the syste*_ can be understood and verified in context. Iterative enhancement

is more appropriate for larger changes where the analysis phase can provide better assessment of the full effect

of changes. Full reuse is appropriate for large changes and major redesigns. Here, analysis and prior history of

the performance of the reuse objects support quality.

Given these differences, we can provide some analysis of the various maintenance process models and

recommend where they might be most applicable. But first, let's discuss the relationship between the development ancl maintenance process models. In some sense development can be considered a subset of m.a,n,enance.

,Maintenance environments differ from development environments with regard to the constraints on the soiuuon.

customer demand, timeliness of response, and organization.

Most maintenance organizations are set up for the quick fix model but not for the iterative enhancement or

reuse process models. This is because they are responding to timeliness, e.g. a system failure needs to be fixed

immediately, or a customer demand, e.g. a modification of the functionality of the system. Clearly these are

strengths for the quick fix model. But the weaknesses of the model are that the modification is usually a patch.

not well documented, the structure of the system has been partly destroyed which makes future evoluuon of me

system difficult and error-ridden, and it is not compatible with development processes. This model is best used

when timeliness and customer need ate dominant and there is little chance the system will be modified again.

The iterativc enhancement model allows for redesign so the structure of the system evolves and future

modification is easier. It focuses on the particular system, making it as good as possible. It is compatible with

development process models. The drawbacks ate that it is a more costly and possibly less timely approach (in

the short run) than the quick fix model and it provides little support for generics or future similar systems. It is

a good approach to us(: when the product will have a long life and evolve over time. In this case, if timeliness

is also a constraint, the quick fix model can be used as a patch and the iterative enhancement model can be used

for the long term change, replacing the patch.

The full reuse process model provides the maintainer with a broader perspective, focuses on long range

development for a set of products and has the side effect of creating reusable components of all kinds for future

developments. It is compatible with development process models, and in fact, it is the way we would like such

models to evolve. The drawback is that it is more costly in the short run, is not appropriate for small

modifications but can be used in conjunction with other models. It is best used when we are living in multi5642

2-36
product environments or generic development where the product line has a long life.

The assessment given above is informal and intuitive. This is due to the fact that it is a qualitative

analysis. To do a quantitative analysis we need quantitative models of the reuse objects, transformations, and

context. We need a measurement framework for characterizing via categorization and classification, evaluation,

prediction, and motivation to support management and technical decisions. To do this we apply the

goal/question/metric paradigm to the models.

The Goal Question Metric Paradigm

The goal/question/metric (GQM) paradigm [1,2,6] represents a systematic approach for setting the project

goals (tailored to the specific needs of an organization), defining them in an operational, tractable way by

refining them into a set of quanufiable questions that in mrn imply a specific set of metrics and data for collection. The wactability of this software engineering process allows the analysis of the collected data and computed metrics in the appropriate context of the questions and the original goal. This context supports feedback

(by integrating analytic and constructive aspects) and learning (by defining the appropriate synthesis procedure

for lower-level into higher-level pieces of experience).

The process of setting goals and refining them into quantifiable questions is complex and requires experience. In order to support this process, a set of templates for setting goals, and a set of gmdelines for deriving

questions and metrics has been developed [2].

Goals are defined in terms of purpose, perspective and environment. Different sets of guidelines exist for

defining product-related and prooess-related questions. Product-related questions are formulated for the purpose

of defining the product (e.g., physical attributes, cost, changes and defects, context), defining the quality perspective of interest (e.g., reliability, user friendliness), and providing feedback from the particular quality perspective.

Process-related questions are formulated for the purpose of defining the process (quality of use, domain of use),

defining the quality perspective of interest (e.g., reduction of defects, cost effectiveness of use), and providing

feedback from the particular quality perspective.

Application of the Goal Question Metric Paradigm

In applying the goal/question/metric paradigm, we define the goals of the maintenance process and articulate the issues associated with choosing the appropriate process model, providing management with the questions

that need to be answered to make intelligent decisions, understand the trade-offs, and perform risk analysis.

There are a variety of goals we can generate. For example: to determine which process model should be chosen

for a particular product, to improve our performance or evolve a better definition of any of the models for a particular product line.

In what follows we will generate a sample goal for maintenance and provide a partial list of the quesuons

involved. Some of the answers will be obvious, either in the measures they require be taken or the information

required form the experts; others will not. Thus a goal for maintenance in the context of the reuse framework

might be:

Purpose:

To evaluate the new product requirements in order to reuse as much of the available products as possible.

Perspective:

Examine the cost and future evolution of the development from the point of view of the organization.

Environment:

Along with the standard environmental factors, such as resource factor, problem factors, we would like to

pay special attention to the three context dimensions of the reuse framework.

Requirements Domain:

Clearly we are using product objects from the same application domain, although we have the ability

to choose candidate components from other application domains.

5642

2-37
Solution Domain:

This defines the process models, methods and tools that were used M the development of the existing

pnxiucL If the same processes arc to be used for the evolved project then there is not problem with

reuse.However, the reusemodel allowsus tochange thepr_.,csscs(and thus possiblyto the product

strucun_)at tlm costof reusinglessof the priorprojccL If thereare to be changes then we must

evaluam thecostof modificationof theprocessand resultingproductrelativeto thegainsforprocess

change.

Knowledge TransferMechanism:

[fthemaintenancsgroupisthesame as thedevelopmentgroupthenthereisno transferof knowledge

required.If theyaredifferentthentherearcconcernsthatmust be evaluatedwithrespecttoappLication,proce,ss and productknowledgeof themaintainersand thekindsofdocumentationavailable.

ProductDefinition:

In consideringtheproduct,we actuallyhave severalThe new productto be bu/ILi.e.thenew version

of the sysu_m, and the old versions plus any other sysmms that am relevant.

Product Dimensions

New Product:

How many requirementsarethereintotalfor thenew system?

Old Produce

What is the mapping of requirements to system components?

What is the measure of the complexity of the n-accability?

How independent am the components to be modified?

What is the complexity of the system and the individual system components?

Repository:

What candidam components are available in the repository and what are their context, transtormation and

object classifications?

Difference betweennew and old:

How many requirements ate there that are not in the old system? (Categorize by size, new vs. modification

of old vs. deletion of old, etc.)

How many components must be changed, added, deleted? (categorized by size and type of change)

Changes/Defects

How many errors, faults, failures (categorized by class) are associated with the requirements and components

that need to be changed?

What is the profile of changes to the original system prior to this change?

Coat

What is the cost of understanding the new requirements?

What is the estimated cost of building a new system, musing the experience and parts of the old project?

What was the cost of the old system in total?

What was the cost of each version?

What is the estimated cost of modifying the old system to meet the new requirements?

Customer Context

How will the new system be used?

What are the potential fuRu'e modifications based upon our analysis of customer profiles, past modifications and

the state of technologies?

5642

2-38
Per_ecrive:

cost and future evolution of the development

Model of Perspective: cost of modification of the design of the sysmm vs. the expected furore modifications

Parameters:

the life time of the system

the cost of future evolution of the system

the cost of evolving the old system versus rebuilding from old parts

Feedback:

Is the model appropriate?

How can the model be improved?

How can the estimations be improved?

How can classifications be improved?

How can activities be improved?

The GoalK_uestion/Metric paradigm allows us to develop other goals for reuse. These can be developed

for whether the reuse object is a process or a product. Consider the following examples:

Evaluate the modification activity within the reuse process in order to improve it. Examine the cost and correctness of the resulting object from the point of view of the customer.

Predict the appropriate maintenance process model in order to perform the correct one. Examine its cost with

respect to the customer needs and the future evolutions of the system from the point of view of the corporataon.

Evaluate the standard corporate design method in order to assess how it should have been tailored for the current

project. Examine its effectiveness from the point of view of the designer.

Evaluate the components of the existing product in order to determine whether to reuse them. Examine their

independence and functional appropriateness from the point of view of their use in furore systems.

Predict the ability of a set of code components to be integrated into the current system from the point of view of

the developer.

Motivate the development of a reusable set of components in order to engineer them for reuse. Examine the

reward structure from the point of view of the manager and developer.

Reuse Enablers

There are a variety of support mechanisms necessary for achieving maximum reuse that have not been

sufficiently emphaisized in the literature. In this paper we have discussed several of these: a set of maintenance

models, a mechanism for choosing the appropriate such models based upon the goals and characteristics of the

problem at hand, and a measurement and evaluation mechanism. To support these activities there is a need for

an improvement paradigm that aids the organization in evaluating, learning and enhancing the software process

and product, a reuse-oriented evolution environment that motivates and supports reuse, and automated support

for that model as well as the measurement and evaluation process.

The Improvement Paradigm: The improvement paradigm [1] is a high level organiT_,tional process model in

which the organization learns how to improve their product and process. Within this model the organization

should learn how to make better decisions on which process model to use for the maintenance of their future

software products based upon learning from past performance. The paradigm is defined as follows:

I. Planning. There are three integrated activities to planning that are iteratively applied:

(a) Characterize the current project environment. It provides a quantitative analysis of the environment and

a model of the project in the context of that environment. In the context of maintenance, the characterization should provide product dimension dam, change and defect data, cost data and customer

5642

2-39
10

contextdamforearlier versions of the system to be modified, information about what classes of candidate components are available in the repository for the new system, and any information feedback

from prior projects about experience with the different models for the types of modifications required.

Co) Set up goals and refine them into quantifiable questions and metrics using the goal/question/metric paradigm, for successful project performance and improvement over previous project performances. This

consists of a top-down analysis of goals that iteratively decomposes high-level goals into detailed subgoals. The iteration terminates when it has produced sub-goals that we can measure direcdy. For

maintenance this involves the development of specific G/Q/Ms as specified in the prior section.

(c) Choose and tailor the appropriate construction modei for this project and the supporting methods and

tools to satisfy the project goals relative to the characterized environment.. Understanding the environment quantitatively allows us to choose the appropriate process model and fine tune the methods and

tools needed to be most effective. For example, knowing the effect of prior applications of the various

maintenance models and methods in creating new projects from old systems allows us to choose and

fine ame the appropriate process model and methods that have been historically most effective in

creating new systems of the type required from older versions and component parts in the repository.

2. Analysis. Analyze the data to evaluate the current practices, determine problems, record the findings and

make recommendations for improvement. We must conduct dam analysis during and after the project. The

goal/question/memc paradigm provides traceability from goals to metrics and back. This permits the measurement to be interpreted in context ensuring a focused, simpler analysis. The goal-driven operational

measures provide a framework for the kind of analysis needed.

3. Learning and Feedback. This step involves the organizauon and encoding of the quantitative and qualitative

experience gained from the current project into a corporate information base to help improve planning,

development, and assessment for future projects. The results of the analysis and interpretation phase can be

fed back to the organization to change the way it does business based upon explicitly determined successes

and failures. In this way, we can learn how to improve quality and productivity, and how to improve

definition and assessment of goals. We can start the next project armed with the experience gained from

this and previous projects. For example, understanding the problems associated with each new version of a

system, provides insights into the need for redesign and redevelopment.

A Reuse-Oriented Environment: Reuse can be more effectively achieved within an environment that supports

reuse [3,8,13]. Software engineering environments provide such things as a project data bases, and support

the interaction of people with methods, tools and project data. However. experience i.s not controlled by

the project data base or owned by the organization. Reuse only exists implicitly.

We need to be able to incorporate the reuse process model into the context of development. We need to

combine the development and maintenance models in order to maximize the context dimensions. We need to

integrate characterization, evaluation, prediction and motivation into the process. We need to support learning

and feedback to make reuse viable. We propose that the reuse model can exist within the context of the

improvement paradigm, making it possible to support all of the above requirements.

The TAME Project: The improvement paradigm and the reuse oriented process model require automated support

for the dam base, encoded experience, and the repository of prior projects and reusable components [2,3,1,*].

We need to automata as much of the measurement process as possible, and provide a tool environment for

managers and engineers to develop project specific goals, and generate operational definitions based upon these

goals that specify the appropriate metrics needed for evaluation. The evaluation and feedback cannot be done in

real time without automated support. Automated support will help in the post mortems analysis.

The goalof theTAME system[2]istoinstantiateand integratetheimprovementand goal/questionmemc

paradigmsand help in the tailoringof the softwaredevelopmentprocess.But itcan alsosupportthe reuseorientedprocessmodel The TAME envisonmentmodel containsbasicmechanisms forsupportingsystematic

learningand reuse, To helpwithsystematiclearningitprovidessupportforrecordingexperience,off-linegeneralizingor tailoringof experience,and formalizingexperience.To help with systematicreuseit supports

mechanisms forusingexistingexperienceand on-linegeneralizingor tailoringof candidateexperience.In this

way itattemptstointegratebothlearningand reuseintoan overallevolutionmodel.

5642

2-40
11

The application of the TAME system concept to maintenance will provide a mechanism for choosing the

appropriate maintenance process model for a particular project and provide data to help us learn how to do a

better job of maintenance.

Summary

The approach to maintenance depends on the nature of the problem and the size and complexity of the

modification. This paper recommends that we view maintenance as a reuse process. In this way the malntmner

is provided with a reuse model and a framework for viewing maintenance that permits a measurement framework to be applied. A new model of a reuse-oriented evgludon process can be developed in which the existing

models can be defined. Existing models can then be analyzed within this _'nework, allowing an organization

to evaluate the strengths and weaknesses of the different approaches and provides feedback in refining the various process models and creating an experience base from which to support further management and technical

decisions.

The approach provides support for defining activities, determining opdons, and evaluation. If the approach

is not adapted then it is difficult for an organization to know which process model to use for a parucular project,

whether they are evolving the system appropriately, and whether they are maximizing quality and minimizing

cost over the life of the system.

References

[1] V.R. Basili, "Quantitative Evaluation of Software Methodology," Dept. of Computer Science,

University of ,Maryland, College Park, TR-1519, July 1985 [also in Prec. of the First Pan Pacific

Computer Conference, Australia, September 1986].

[2] V.R. BasilL H. D. Rombach "The TAME Project: Towards Improvement-Oriented Software Environmenus," IEEE Transactions on Software Engineering, vol. SE-14, no. 6, June 1988, pp. 758-773.

[3] V.R. Basili, H. D. Rombach "Towards a Comprehensive Framework for Reuse: A Reuse Enabling

Software Evolution Environment," University of Maryland Computer Science Technical Report,

UMIACS-TR-88-92, December 1988.

[4] V.R. Basili, H. D. Rombach, J. Bailey, and B. G. Joo, "Software Reuse: A Framework," Prec. of the

Tenth Minnowbrook Workshop on Software Reuse, Blue Mountain Lake, New York, July 1987.

[5] V.R. Basili, A. J. Turner, "lterative Enhancement: A Practical Technique for Software Development,"

IEEE Transactions on Software Engineering, voi. SE-I, no. 4, pp. 390-396, December, 1975.

[6] V.R. Basili, D. M. Weiss, "A Methodology for Collecting Valid Software Engineering Data," IEEE Transactions on Software Engineering, vol. SE-10, no. 6, pp. 728-738, November 1984.

[7] L. Belady and M. Lehman, "A Model of Large Program Development, IBM Systems Journal, vol. I5, no.3.

1976.

[8] Ted Biggerstaff, "Reusability Framework, Assessment, and Directions," IEEE Software Magazine,

March 1987, pp.41-49.

[9] T.P. Bowen, G. B. Wigle, J. T. Tsai, "Specification of Software Quality Attributes," Technical Report

RADC-TR-85-37, Rome Air Development Center, Griffiss Air Force Base, N.Y. 13441-5700, February

1985.

[10] Federal Information Prtx:essing Standards, "Guideline on Software Maintenance," U.S. Dept. of

Commerce/National Bureau of Standards, lIPS PUB 106, June 1984.

[11] P. Freeman, "Reusable Software Engineering: Concepts and Research Directions," Prec. of the

Workshop on Reusability, September 1983, pp. 63-76.

[12] R. B. Grady, "Measuring and Managing Software Maintenance," IEEE Software, Voi. "L No. 5, September

1987, pp. 35-45.

5642

2-41
12

[13] IEEE Software, special issue on 'Reusing Software', voi.4, no.l, January 1987.

[14] IEEE Software, special issue on 'Tools: Making Reuse a Reality', vol.4, no.7, July 1987.

[15] G. A. Jones, R. Prieto-Diaz, "Building and Managing Software Libraries," Proc. Compsac'88, Chicago,

October 5-7, 1988, pp. 228-236.

[i6] B. P. Lient_ E.B. Swanson, and G. E. Tompkins, "Characteristics of application software maintenance,"

Communications of the AClVl, Vol 21, No. 6, June 1978, pp. 466-471.

[17] R. Prieto-Diaz, P. Freeman, "Classifying Software for Reusability," IEEE Software, vol.4, no.l, January

1987, pp. 6-16

[18] H. D. Rombach, V. R. Basili, "A Quantitative Assessment of Software Maintenance: An Indusmal Case

Study," in Proc. Conf. Software Maintenance, Austin, TX, Sept. 1987, pp. 134-144.

[19] Mary Shaw, "Purposes and Varieties of Software Reuse," Proceedings of the Tenth Minnowbrook

Workshop on Software Reuse, Blue Mountain Lake, New York, July, 1987.

[20] W. Tmcz, "Tutorial on 'Software Reuse: Emerging Technology'," IEEE Catalog Number EHO278-2,

1988.

[21] S. S. Yau, R. A. NichoU, J. J.-P. Tsai, and S.-S. Liu, "An Integrated Life-Cycle Model for Software

Maintenance," [EEE Transactions on Software Engmeenng, Vol. SE-14, No. 8, August 1988, pp. 1128-

1144.

5642

2-42
UMIACS-TR-89-57 "

CS-TR-2263

June, 1989

Software Development: A Paradigm for the Future +

Victor R. Basili

Institute for Advanced Computer Studies

Department of Computer Science

University of Maryland

College Park, MD 20742

ABSTRACT

This paper offers a new paradigm for software development that treats software

development as an experimental activity. It provides built-in mechanisms for learning

how to develop software better and reusing previous experience in the forms t)t

knowledge, processes and products. It uses models and measures to aid m the task,', (_t

characterization, evaluation and motivation. If proposes an organization scheme for

scparatmg the project-specific focus from the organization's learnin,,a and reuses

focuses of software development. It discusses the implications of thls approach for

corporations, research and education and presents some research activities currently

underway at the University of Maryland that support this approach.

- E'{c,c.arch -,upponcd m t',an h,_ x,,_.S,\ _rant \S(;-5123, \}:OSR gram 87-i)13(I ONR grant NI)llIIa-NT-K-aI_{17 .i,I [[ \l

_51l:k rthrougin the I/idll_[Riii ,_,%_l_.lalk._ i_r_4rill;i _1 the [)_;partmeflt ,)1 C_Jmputer SClCllcc) K_'_ll,>tc .l_ldrcx_. t'[]\ll'?_ \t ",'_ L)r

lalltJ,_. I:k. Sept 1()._9

5642

2-43
-2-

1. INTRODUCTION

We have been struggling with the problems of software development for

many years [31,64]. Organizations have been clamoring for mechanisms to

improve the quality and productivity of software. We have evolved from focusing on the project, e.g. schedule and resource allocation concerns, to focusing on

the product, e.g. reliability and maintenance concerns, to focusing on the protess, e.g. improved methods and process models [97,33,39,86 I. We have begun to

understand that software development is not an easy task. There is no simple

set of rules and methods that work under all circumstances. We need to better

understand the application, the environment in which we are developing products, the processes we are using and the product characteristics required.

For example, the application, environment, process and product 'xssoeiated

with the development of a toaster and a spacecraft are quite different with

respect to hardware engineering. No one would assume that the same educational background and training, the same management and technical onvironment. the same pro(tuct characteristics and constraints, and the same processes.

methods and technologies would be appropriate for both. They are :_lso ,l,_ice

different with respect, to software engineering.

We have not fully accepted the need to understand the differences and learn

from our experiences. We have been stow in building models of pro, hlcts and

processes and people for software engineering even though we have such models

for other engineering disciplines. Measurement and evaluation have only recently

become mechanisms for defining, learning, and improving the software process

and product [3.34].

We have not even delineated the differences between such te.l'Ia_ :t_ _chnique, method, process and engineering. For the purpose of this p:q)er w_ _tefine

a technique as a basic techno[ogy for constructing or assessing sot'tw:u'e. _._..

reading or testing. We define a method as an organized management approncil

based upon applying some technique, e.g., design inspections or te._t pl:m_. \\,.

define a process model as an integrated set of methods that covers the life .'ycl_..

e.g.. an iterative enhancement model using structured designs. ,lesign inspections.

etc. We define software engineering as the application and t.ailoring of _,.,'hniques, methods and processes to the problem, project and organizational char:tcteristics.

There is a basically experimental nature to software development. We can

draw analogies from disciplines like experimental physics and the social sciences.

.Ms such we need to treat software developments as experiments from which we

can learn and improve the way in which we build software.

5642

2-44
-3-

2. THE IMPROVElVIENT PARADIGM

Based upon our experiences in trying to evaluate and improve the quality in

several organizations [5,29,53,58], we have concluded that a measurement and

analysis program that extends through the entire life cycle is a necessity. Such a

program requires an organization to adopt a long term, quality-oriented, organi,

zational life cycle mode, which we call the Improvement Paradigm [4,191 . The

paradigm has evolved over time, based upon experiences in applying it to

improve various software related issues, e.g., quality and methodology. In its

current form, it has four essential aspects:

1 Characterizing the environment. This involves data that characterizes the

resource usage, change and defect histories, product dimensions and environmental aspects for prior projects and predictions for the current project. [t

involves information about what processes, methods and techniques have

been successful in the past on projects with these characteristics. [t provides

a quantitative analysis of the environment and a model of the project in the

context of that environment.

'2 Planning. There are two integrated activities to planning that are iteratively applied:

(a) Defining goals for the software process and product operationally relative to the customer, project, and organization. This consists or" :t

top-down analysis of goals that iteratively decomposes high-level goals

into detailed subgoals. The iteration terminates when it has produced

subgoals that we can measure directly. This approach differs from the

usual in that it defines goals relative to a specific project :rod organization from several perspectives. The customer, the developer, and the

development manager all contribute to goal definition. It is. however.

the explicit linkage between goals and me-_surement that ,tisting_ushes

this approach. This not only defines what_ good is but provides :t focus

for what metrics are needed.

(b) Choosing and tailoring the process model, methods, and tools to satisfy

the project goals relative to the charactorized environment. Understanding the environment quantitatively allows us to choose the

appropriate process model and fine tune the methods and tools needed

to be most effective. For example, knowing prior ,let'ect histories

allows us to choose and fine tune the appropriate constructive methods

for preventing those defects during development (e.g. training in the

application to prevent errors in the problem statement) and assessment

methods that have been historically most effective in detecting those

defects (e.g., reading by stepwise abstraction for interface faults).

3 Analysis. We must conduct data analysis during and after the project. The

information should be disseminated to the responsible organizations. The

operational definitions of process and product goals provide traceability to

metrics and back. This permits the measurement to be interpreted in context ens,lring a focused, simpler analysis. The goal-driven oper:_tional rnen._-

ares provide 'a framework for the kin,t _l" :maR'sis n,,'_te_l. D,Irin_ proj_.ct

5642

2-45
-4-

4

development, analysis can provide feedback to the current project in reai

time for corrective action.

Learning and Feedback. The results of the analysis and interpretation phase

can be fed back to the organization to change the way it does business based

upon explicitly determined successes and failures. For example, understanding that we are allowing faults of omission to pass through the inspection

process and be caught in system test provides explicit information on how

we should modify the inspection process. Quantitative histories can improve

that process. In this way. hard-won experience is propagated througilot_t

the organization. We can learn how to improve quality and productiviv,-,

and how to improve definition and assessment of goals. This step involves

the organization of the encoded knowledge into an information reposi_or- ,r

expereince base to help improve planning, development, and assessment.

• Characterize the current project environment.

• Set up goals and refine them into quantifiable questions and metrics for successful proj-cr

performance and improvement over previous project performances.

• Choose the appropriate software project execution model for this projec_ and 5tlpportll_%_

methods and tools.

• Execute t_e chosen processes and construct the products, collect the prescribed data, vatida[e

it, and and analyze the data to provide feedback in real-time for corrective acuon ,m the

current project.

• Analyze the data to evaluate the current practices, determine problems, record the fin,lin_

and make recommendations for improvemen_ for future projects. This is an otT-lin,, ['r-,'-->

which involves the structuring of experience so that it can be reused in the future.

• Proceed to step 1 to start the next project, armed with the recorded. _tructured ,'x.::,,',"i,'n,:,,

gained from this and prevmus projects.

FIGURE 1: THE IMPROVEMENT PARADIGM

The Improvement Paradigm is based upon the assumption that software

product needs directly affect the processes used to develop and maintain the product. We must first specify our project and organizational goals and their

achievement level. This specification helps determine our processes. In other

words, we can't define the processes and then determine how we are going to

achieve and evaluate certain project characteristics. We must define the project

goals explicitly and quantitatively and use them to drive the process.

._s it stands, the improvement paradigm is a generic process whose :t_,p:

need to be instantiated bv various support mechanisms. It, re,quires a me¢'h:_nism

[Zr defining operat, ional goals :m,t t rausformin_ th_,m into metrics (step 2:t). It

5642

2-46
-5-

requires a mechanism for evaluating the measurement in the context of the goaIs

(step 3). It requires a mechanism for feedback and learning (step 4). It requires

a mechanism for storing experience so that it can be reused on other projects

(steps 1,2b). It requires automated support for all of these mechanisms. [n the

next three sections, we will discuss mechanisms that have been used to support

these activities. In the last half of the paper, we will discuss a proposed organizational structure that allows these activities to be managed and evolve.

2.1. The GoalQuestionMetric Paradigm

The Goal/Question/Metric (GQM) paradigm is a mechanism for defining

and evaluating a set of operational goals, using measurement on a specific project. It represents a systematic approach for setting the project goals tailorpd to

the specific needs of an organization, defining them in an operational, tractabl_

way by refining them into a set of quantifiable questions that in turn implien a

specific set of metrics and data for collection. It involves the development ,_['

data collection mechanisms, e.g., forms, automated tools, the colleeth)a :_n,i v:_iidation of data. it includes the analysis of the collected data and rompllte,i

metrics in the appropriate context of the questions and the original _oals.

The GQM paradigm was originally developed for evaluating defects for a _ot

of projects in the NASA/GSFC environment [28]. The application involved :t _ot

of case study experiments. [t was then expanded to include vario_s t vt)o_ _["

experimental approaches, including controlled experiments [4.22,25 I.

The process of setting goals and refining them into quantifiable ,[_,_sri_ns i_

complex and requires experience. In order to support this process. _ s,_t ,>f r..c_tplates for setting goals, and a set, of guidelines for deriving questi_'_ns all_t m_ttl_s

has been developed t19[. These templates and guidelines reflect o_r ,__xp_l'i¢_!_.,., ,

from having applied the GQM paradigm in a variety of environments.

Goals are defined in terms of purpose, perspective and environment. Dit'-

ferent sets of guidelines exist for defining product-related and proeess-rela_e,t

questions. Product-related questions are formulated for the purpose of _letiniag

the product (e.g., physical attributes, cost, changes and defects, user context!.

defining the quality perspective of interest (e.g., functionality, reliability, _ser

friendliness), and providing feedback from the particular quality perspective.

Process-related questions are formulated for the purpose of defining the process

(process conformance, domain conformance), defining the quality perspective of

interest (e.g., reduction of defects, cost effectiveness of use), and providing fee, tback from the particular quality perspective.

The GQM provides a mechanism for supporting step 2(a) of the Improw.-

ment Paradigm which req,dres a mechanism for defining operational goal._ :m_t

transforming t.hem into metrics that ,:an be used for characterization. ,_,,:_ltz:_r,i_,,.

5642

2-47
-6-

d

Goal 1 ......... Goal

Question 1 ......... Que_,tion uestion m

 uo: onA\

d 3 ......... d r m

FIGURE 2: THE GOAL/QUESTION/METRIC PARA_DIGM

prediction and motivation. [t supports step 3 by helping t_) ,b*fine t,he ,_xt,_rlmental context and providing mechanisms for the data collecth>n, validation an,l

analysis activities. It also supports step 4 t)3" provi,ling ql,antit:Ltivo f'e_,tb:t,q_ ,:)n

_he achievement of foals.

The GQM was originally used to define aml eval_late goals for a i_artic,llar

project in a particular environment. In _he context <)t' the [mpr.ow_m_.nt Pr_E':>

di_,m, t,he use of the GQM is expanded. .Noxv. we can use it, for long range corporate goal setting and evaluat, ion. We can hnprove our evat,lat.ion of :t project

by analyzing it in the cont`ext` of several other projects. We can expand our

level of feedback and learning by defining the appropriate synt, hesis procedure for

lower-level into higher-level pieces of experience. A_ part of the IP we can learn

more about, the definition and application of the GQM in a formal way. just as

we would learn about, any or, her experiences.

S

2.2. The TAME Project

The T._,I_ project [18,191 recognizes the nee_l to _-haracterize. int_.,grato :m,t

a,_omate the various act, ivities involved in instant, iating the [mpr<)v_qn_nt P:t,':>

digm. for ,,se _>n projects. [t ,lelineates the step._ perf_rnl.,I b_ th,. i,,-,>j,._.t :m,t

5642

2-48
creates the idea of an experience base as the repository for what we have learned

during prior developments. It recognizes the need for constructive and analytic

activities and supports the tailoring of the software development process.

persp as_l

constructire

characterizing

characterize

planning

I

what how

[

I

, _ plan --

for

set : construction

executing

const, r uct

--tl ............................................................... i .......................

environment __--_. goals plan

ana- t--- --. i_.__ for

lytic i i " ] analysis

4-4

analyze

I

1

' ' i

T

* _' A v A V A v

1

EXPERIENCE BASE J

FIGURE 3: THE T,_ME SYSTEM

The TAIv[E system offers an architecture tbr a software engineering envir<mment that supports the goal generat, ion, me_urement and evaluation activities.

It is aimed at providing automated support for managers and engineers to

develop project specific goals and specify the appropriate metrics needed for

evaluation. It provides automated support for the evaluation and feedback on a

particular project in real time as well as help prepare for post, mort, ems.

The Tame project was initiated to underst, and how to automate as much of

the paradigm as possible using whatever current technology is available and to

determine where research is needed. It provides a vehicle for defining the concepts in the paradigm more rigorously.

A major goal for the T,%ME project is t_ ('reato :_ corporate experience b:_se

which incorporates historical inform:_tiCm ;t_l'¢_s all pl'r_je¢'ts wit, h ro_nr, I r¢_

2-49

5642
-8-

project,product and processdata, packaged in such a way that itcan be useful

to future projects.This experiencebase would containms a minimum the historicaldata base of collected data and interpreted results, the collection of measured

objects, such as project documents, and collection of measurement plans, such as

GQM models for various projects. It should also contain combinations and synthesis of this information to support future software development and maintenance.

T._,IE is an ambitious project. It is assumed it will evolve over time and

that we will learn a great, deal from formalizing the various aspects of the

Improvement Paradigm as well as integrating the various sub-activities. [t will

result in a series of prototypes, the first of which is to build a simple evaiuatioa

environment. Building the various evolving prototypes and applying them in :._

variety of project environments should help us learn and test out ideas.

Tame provides mechanisms for instantiating the [mpr_vement Parmtigm l:,v

providing an experience base to allow the storing of experience so tha_ it ,':m },e

used on other projects (steps t.2a), further defining the various steps ,_o 1,,: p,_rformed (steps 1.2.3.4). and automating whatever is possible.

3. A REUSE-ORIENTED SOFTWARE ENGINEERING MODEL

The Improvement Paradigm, as instantiated in the T.%ME system. :_sunl_s

that improvement can be achieved by iterating planning, execution of plans, an,l

feedback across projects within an organization. Feedback can be 'd_w_,i :ts

reusing experience from the ongoing or prior projects to improve the plannin_ >r

execution of ongoing or future projects. Learning can be viewed :_s the pro,._ess .>f

accumulating and packaging experience so it can be reused effectively. Fh,_. rhu

paradigm explicitly" recognizes the need to capture and re,me knowledge, t,r<,,i_l,t_

and processes from prior projects.

On the other hand. it should be noted that reuse can be n.n et'i',_'ti':, _

mechanism only if it is paired with learning and viewed as an integr'd p:u't of :_n

improvement-oriented software evolution process model. [f we accept the t':/,'t

that a better understanding of a process allows for more effective reuse. "reuse

orientation" and "imlarovement orientation" of a process model are identical

attributes. Both are supported by experimentation.

In a traditional software process model, learning and reuse only' occur

because of individual efforts or by accident. They are not explicitly supported

and called out as desired characteristics of the development process..-ks a consequence, this experience is not owned by the organization (via the project database) but rather owned by individual human beings and lost after the project, has

been completed. A reuse-oriented process model must view reuse, le-_rning :m,t

feedback as integral components, and place all experience, including :__ol'tx_:H-_,

evolution methods :m,1 tools, under the control of an experieiice b:use [21/].

5642

2-50
-9-

Since improvement requires the feedback of available experience and feedback is based on learning and reuse activities, a requirement for such a process

model is that it support systematic learning and reuse. Systematic learning

requires support for the 0ff-line recording, generalizing or tailoring, and formalizing of experience. Systematic reuse requires support for (re-)using existing

experience. Off-line activities are performed independent of any particular project in order to improve the reuse potential of existing experience in the experience base.

Project goals are typically directed towards the development of a specific

system. Thus off-line activities must have their own organizational structure.

They cannot be part of the normal development organization because they

require a different focus, a different set of processes, and an independent cost

base.

For example, the objective of the recording process is to create a repository

or" well specified and organized experience. It requires effective mechanisms for

collecting, validating, storing and retrieving experience. This should not be part

of the project focus. The project can contribute by making its experience avail-

"_ble r.o this independent organization, but cannot itself oversee the recording. [t

might not even be clear to the project what is worth recording.

The objective of generalizing existing experience prior to its reuse is to make

a candidate reuse object useful in a larger set of potential target applications.

The objective of tailoring existing experience prior to its potential reuse is to

fine-tune a candidate reuse object to tit a specific task or exhibit special attributes, such as size or performance. Clearly a project cannot afford to ._eneralize

or tailor experience for another project within its budget constraints. Ewn

wo,'se_ _t may no_ have the perspective _o do so since ol)je_:tives anct :harac_erisdes are differen_ from project to project, and even more so from env,mnment _o

environment. Generalizing and tailoring require a broader perspective or" the

organization and the products it develops.

The object, lye of formalizing existing experience prior _o its potent, ial reuse is

_o encode it in more precise, better understood ways. Off-line tailored or generalized experience needs to be formalized to increase its reuse potential and

satisfy general reuse needs within an organization. The more we can formalize

experience, the better it, can be reused.

Formalization activities include the movement from informal knowledge

(e.g., concepts), to structured or schematized knowledge (e.g., methods), or even

to completely formal knowledge or automation (e.g., tools). [t requires models

of r,he various reuse objects, notations for making the models more precise, notations for abstracting reuse object characteristics, mechanisms for validating these

models, and mechanisms for interpreting models in the appropriate context.

Clearly the project has neither the budget nor the need r,o t'ormalize its own

experience.

2-51

5642
- 10-

Reuse requires a precise specification of the reuse context including the evolution process that is expected to enable reuse, and the characteristics of the

available candidate reuse objects. The objective of a reuse-oriented software

evolution process model is to support the use of previously accumulated experience during such reuse activities as: (a) specifying reuse needs in a way that

allows matching them with descriptions of available experience, (b) finding and

understanding appropriate reuse candidates, (c) evaluating reuse candidates in

order to pick the most promising candidate, (d) actually tailoring the reuse candidate if necessary, (e) integrating the reuse candidate into the ongoing software

project, and (f) evaluating the software project.

A reuse-oriented soRware evolution environment is an integral part of the

improvement paradigm. The mechanisms supplied by the TAS,[E system to support that paradigm are consistent with the mechanisms needed to support the

reuse environment model with its experience base. It provides a mechanism for

evaluating the recorded experience, helping us to decide what and hew to reuse.

tailor and analyze. It captures experience in the form of data from which mo,tets

can be built to formalize experience. It supports continuous learning.

It is clear that an experience base is a key component of the reuse :m,t

improvement paradigms. A project needs help in accessing the reusable exp_'i -

ence. If the experience is available (recorded), appropriate (tailored or genera[-

ized), and well-packaged (formalized), it can be used by a project. But an

experience base is more than a physical entity. It is an organization that must

support all the off-line activities that support its creation and use.

4. DMDING UP THE RESPONSIBILITIES AND ACTMTIES

Based upon the prior discussion, the implementation of the [mprovem,ut

Paradigm would best be served by two separate and distinct organizational s_r'u,-

tures. One organization is project-oriented. Its goal is to deliver the syst_,ms

required by the customer. We will (:all this the Project Organization. The oth,_r

organization, which we will call the Experience Factory. will have r+he role <>t'

monitoring and analyzing project developments, developing and paeka_iag

experience for reuse in the form of knowledge, processes, tools and products, and

supplying it to the Project Organization upon request. The Experience Factor?"

represents the experience base discussed above and the various activities associated with building and modifying it, controlling its access, and interfacing to the

Project Organization.

Each project in a Project Organization can choose its process model based

upon the characteristics of the project, taking advantage of prior experience with

the various process models from the experience base in the Experience Factory.

[t can access information about prior system requirements and solutions, efl'ec[ive

methods and tools and even available system components. Based _pon :recess to

this prior experience, the project can ebonise :t[l,t t.ailor the best possiblo pI'(_._,ss.

methods and tools. It can reuse pt'i(>r pro<l_,+ts ta+ilorecl to its tree,is.

5642

2-52
-11 -

PROJECT ORGANIZATION

TAME Process Model

r

! I

characterize

I

_.___

environment]

set

goals

construct

select 4--

: i

methods :

& l

i analyze

tools

t

a

1

I

0

r

reuse I record

formaliz

)

informal schematized )roductlzed

PR()JECT SPE( IFIC

D()_LAJN SPE( IFIC

3 E N E R',4 L

g

e

n

e

r

3,

1

1

Z

e

Experience Base

EXPERIENCE FACTORY

FIGURE 4: IMPROVEMENT AND REUSE

ORIENTED-SOFTWARE ENGINEERING MODEL

The Experience Factory analyzes the project development, for all systems

developed by the corporar, ion. Based upon this it, recognizes commonality ,tmon_

projects, generalizes knowledge and packages it for use across all projo_.ts, it

creates a repository of re,tsableinformat, ion. For example, it can clevoh)p rf_s(),,r,'e

models. ,bfect mo,lels. _.mt risk management mo, tels :_ml taih>r lliem t'()r rl_'

5642

2-53
-12-

particular projects. [t can develop processes, methods, techniques and tools and

tailor them based u.pon the characteristicsof the particular project. This can be .

accomplished based upon the Factory's analysis of the success and failure of the

various activities across many projects. It can generate system components, at

various levels of the architectural hierarchy based upon its recognition of commonality.

4.1. Some Specific Activities in the Project Organization

Let us consider the activitiesof the Project Organization with regard to the

development of a system and how it might use the Experience Factory while

applying the improvement paradigm.

At the start of a project, project management functions consist of activities

such as resource and schedule planning, organizing, and staffing. Th_se are

covered by the characterizing and planning functions ]n the Improvement Paradigm.

During the characterizing phase, based upon its needs and characteristicz.

the project can access the experience base for the information about similar previous projects. This provides the project manager with a context for planning

that includes resource estimation and allocation information, personnel experience. software and hardware available for reuse, environmental characteristicsof

concern and sets of baselines for resources, schedules, defects, etc. The projec_

can store information on its own characteristics back into the experience bn.-_ for

analysis.

During the planning phase, the project can analyze prior goals aml Else :l,_:n

:_s defined or tailor them (or have them tailored bv the C'ompone,lt Factory! t'of

its nee_ts. It can access the collection of construtive and analytic metho,ts and

tools, that have been effective and choose the appropriate ones tha_ will h_lp

satisfy its goals. The goals and methods are influenced by the knowle_ige _aine_l

t'rom the characterization phase, specifically with regard to elements of prior systems that can be reused. These elements include data, such _ks baselines, process

models that have been successful, including methods and techniques that have

been tailored and tools that support those methods, and components of prior projects such as requirements, design or code that can be adapted for the current

project. The goals define the kinds of data that need to be collected as well as

the mechanisms needed for collection. This provides the manager with information about what feedback will be provided for the project during development.

The goals and process model, as tailored for the project, are stored in the experience base for monitoring the current project and expanding the experience b-_se

for future projects.

5642

2-54
-13-

PROJECT ORGANIZATION

characterizing

needs and characteristics

,{ of previous projects

needs and characteristics

(tailored to current project)

planning

active reuse of previous plans

for construction and analysis

plans for construction and analysis

(tailored to project characteristics)

construction

(according to some

construction model)

construction plans,

t + reuse methods,

tools and products

new products

analysis

(track construction) t+ analysis plans,

reuse measurement tools

collected data

feedback/learning

analysis plans (interpretation)

data from current project,

data,' interpretation from

previous projects

feedback and

new know edge

EXPERIENCE

FACTORY

>

FIGURE 5: ACCESSING THE EXPERIENCE FACTORY

Project execution covers the directing and controlling activities as well ,a_s

the development activities.

During the execution phase, the project proceeds using the tailored process

model, methods, techniques, and tools as specified in the planning phase. It uses

prior product parts, supplied by the experience base. Feedback is supplied to

project management to support directing and controlling of the project. During

execution, project experiences, components and data are returne_t to tile Experience Factory and feedback is provided to the project.

5642

2-55
- 14-

At project conclusion, the overall project is analyzed and the results are fed

back to the project as well as packaged and incorporated into the experience base

for use on future projects.

4.2. Some Specific Activities in the Experience Factory

The Experience Factory plays several roles. It builds and maintains the

experience base, it interfaces with the project in the Project Organization by providing information from the experience base and developing those elements that

are requested by the project based upon its current level of expertise, e.g.,

tailored methods and tools and software components, and it acts as a quality

assurance organization, providing feedback to the project with respect to its

goals..-ks such it has several process models associated with it.

In building and maintaining the experience base, the Experience Factory

performs the learning and reuse activities of recording, generalizing and tailoring.

and formalizing. The degree to which it can perform these activities ,lepends

upon the breadth and depth of the information available and the level of technology.

It records information gathered from the various project developments. For

example, it saves experiences from the projects it is monitoring, such as code

modules, lessons learned on the project from the application of the constr_ctive

and analytic processes and measurement data. such as resource and ,lefect data.

It generalizes or tailors the information that it has gathered. F',r ,exampl_..

it uses the project-specific measurement data across sex'oral p_'ojoc.ts to cr,:_t_

baselines such as defect profiles; it develops generic packages from project spe,'itic

packages or instantiates a generic package for a specific project: it rotines :_

design technology based on the lessons learned from applying it on a specific projeer: it parameterizes a cost model for a project or uses data from the projert :(,

improve the estimation capability of the model.

It formalizes the information in the experience base to enhance its reuse

potential. For example, it supplies code modules with their functional

specifications and other appropriate documentation such as characterizing attributes, when needed: it makes more precise the steps in applying a method based

upon lessons learned from its application; it builds cost models empirically based

upon the data available; it develops management support systems based upon the

available data and lessons learned; it builds automated support for methods.

In responding to requests from a project, it provides whatever information it

has available from the experience base and the people. The level of support

clearly depends upon the state of the art in the packaging of exp,,rib,.co. "t'h_

interface with the Proj(_ct ()rganiz_tion will ,'hango over time. st:trtina t.%i[tl _m:tll

packets of experience xn,t b::il,ting to higher levol ()n(eb.

5642

2-56
- 15-

PROJECT

ORGANIZATION EXPERIENCE FACTORY

4

products

models

data

lessons learned

direct feedback

products

data

lessons learned

models

baselines

tools

consulting

a

n

> a

l

Y

S

I

S

1

I

k----,

I

I

t

s

Y

n

t

h +---

e

S

1

s

Experience

Base

formalize

tailor

i

I

generalize

FIGURE 6: ACTMTIES IN THE EN:PERIENCE FACTORY

The actual information supplied depends upon the request and what is

currently available in the experience base. For example, during characterization.

it provides baselines and estimation models, and information on packaged products, such as requirements templates or code modules. General defect baselines

can be tailored to the specific project by limiting the projects considered to those

with the same characteristics as the current project, e.g., same application

domain, same process model.

During planning it supplies GQIvI models and process moctets, metho,ts, tools

and techniques. These can be obtained directly from the experience b:_._o or

tail>rod for the needs of the proje_:t. For _xample, :L_s,lming th:tt iil_l )_ 'ti¢>(_> :_I'e

5642

2-57
-16-

chosen for the project and knowing the classes of faults found in similarly

classified projects, the component factory might tailor the reading technology

within inspections to concentrate on locating the kinds of faults that tend to

occur in this type of project. They can also provide training and consulting on

the use of the methods and models.

During project execution, they can act as a contractor supplying various levels of project components. In fact from the Project Organization perspective, any

component that can be well specified can be delivered by the Experience Factory.

In turn, the Experience Factory can respond to the request by delivering an existing component, modifying an existing component, e.g. instantiating a generic

package from the experience base, or developing the component from scratch and

adding it to the experience base.

If we view quality assurance as the act of leading, teaching, and auditing r.he

process, then it implies an organizational structure independent but interacdv_

with the projects. (Note that this is different from quality control, which ,,re

define as the act of directing, influencing, _erifying, and correcting the pro, iuct.

which implies a project controlled organization.) The Experience Factory is :m

ideal location for the quality assurance activities.

In acting as a quality assurance organization, the Experience Factor," :tu,ii_s

activities and collects the prescribed data. provides feedback to the project in

real time, and offers training in the various planning, constructive, and analytic

approaches. The quality assurance activities is consistent with the activities of

building and maintaining the experience base and responding to req_ests fl'om

the Project Organization. It also provides an independent chain of comman_t :m,i

a corporate perspective with regard to goals, data collection, pro,'_ss :tn,] products.

4.3. Viewing the Experience Factory as a Component Factory

As a particular dimension of the Project Organization and the Exp_rien,_ _

Factory, consider the activities of the Project Organization with regard to the

development of a system and how it might use the Factory from the point or'

view of code development, e.g., as a Component Factory. We can view the projeer organization within the Project Organization as having the t'oLlowing activities:

Requirements Definition: The system analysts will interact with the customer to determine project requirements. It is assumed that the analysts will know

the application domain and what is available in the repository for reuse. They

will have access to repository information about what kinds of components :_re

available so they can make tradeoff decisions, negotiating with the custonler t'or

function vs. price.

5642

2-58
- 17-

Initially. this negotiation will be limited since the repository will be sparsely

populated. This should change over time as the repository fills with components. It should be noted that the system analyst can use Factory components

for building and analyzing prototypes of the system.

Specification and Design: The requirements will be turned into a system

design and specification for the required components. Those components that

can be well specified can be turned over to the Experience Factory and orders

will be filled fox" components.

Initially, the specifications will be for low level components since the Factory

will begin bottom up..-ks time goes on and the repository builds up in terms of

components, and the technology for recognizing, specifying and integrating larger

pieces of systems develops, larger components can be ordered.

Tile Experience Factory operates according to several process models. When

an order for a component arrives, it can check its repository for tile appropriate

component or order it externally if it is available from an outside vendor. It :an

develop it from scrar, ch, using verification technology, based upon the t'ac_ r:hac ir

has the specification and the component it is developing is limited in size. H_wever. given that it has been required to deliver such a component, is ,:an ,[eci,ie

whether the component is of general use, from its knowledge of other projects.

and can generalize or tailor the component, package it with the necessary astributes for future reuse and store it in the repository.

.-ks an initializing activity, the Factory can analyze prior systems for r_ls'abIe

components and re-engineer them to seed the repository. It can de,.'ot_:_p ,';m_-

ponents, so _hey are easy to combine, modify with respect to certain ¢:rit,,ri:t :_n,[

label and package appropriately.

Integration and Evaluat, ion: The project will have the task of in_e,_;rarin_ _i_e

components into its own specified design. These integrated components mi,ght t,,,

ret, urned to the Factory for future use. It will then evaluate the system t_:t._o,[

upon the customer requirements and deliver the system.

5. IMPLICATIONS OF THE NEW LIFE CYCLE ORGANIZATION

5.1. Implications for Corporations

One of the major problems with software development in the past has been

that projects have been unable to explicitly reuse experience from prior projects

or contribute to the experience base for future projects. This has been _t_e in

part to the fact: that immediate project delivery goals and the more long-range

goals of reuse and learning are distinct and not easily paired. Projec_ _cll_,,l,ll_

often takes prece(lence over the lux!iry of passing on le-trned experiem'e.

2-59

5642
- 18-

The new life cycle organization divides the focus of software development

into two separate organizations. It separates the immediate project goals from

the long range learning and reuse-oriented goals. In the approach, the Project

Organization can focus on the customer needs and has the advantage of access to

a knowledgeable support organization in the form of the Experience Factory.

The Experience Factory focuses on the organization's goals to learn and reuse. It

has the advantage of accumulating experience from a large number of projects

which provides it with a broader perspective than any particular project.

This organizational structure has many advantages. It should promote

higher quality and productivity because of reuse and learning. It can provide

better and more focused education and training for developers and provide better

methods and tools for them to use.

It provides the corporation with a corporate asset in the guise of the Experience Factory. The Experience Factory contains everything the organization has

learned and developed that is useful for future developments as well as an assessment of the status of corporate quality and productivity..-ks the Experience Factory grows in its role and assets, the corporation can learn more and more from

the various experiences across the corporation.

There will be more emphasis on formalization of all parts of management

and development. Formal verification becomes cost effective since the correct

units will be used in many systems: it becomes more applicable since we will be

applying it to smaller units, at least in the beginning, where the technology is

manageable. Formal models of risk assessment can be used since the experienc'e

b:_se should provide a broad basis for understanding and comparison.

The organizational scheme has the advantage that it can start small anal

expand with the growth in technology and the experience base. However. there

are several issues that must be dealt with in putting this or__anization in p[:_c,?.

e.g. financial and organizational.

This organization requires separate cost centers for the Projert <)rganiz:_tion

and the Experience Factory. There are several models of how the funding or' _he

Experience Factory might work. For example, it could be funded out of corporate overhead which would grow with the success of the factory or projects

could be billed for factory items. The right model will depend upon the company and the organization and politics within that company.

This organization requires a careful definition of management and responsibility structures. [t is clear that we do not want to create new conflicts over

responsibility for problems with packaged experience.

This organization needs to be motivated and supported. [ncentivo an,t

reward structures need to be developed. We will need to learn from experience

_ained from applying different financial and management structures.

5642

2-60
- 19-

5.2. Implications for Research

There are several implications for research based upon this organizational

structure. Many of the technologies already developed for programming in the

small are applicable in the factory domain. For example, verification technology

is already available for factory produced components and it is necessary and cost

effective because those units will be reused many times. Research activities can

focus on the transfer of these technologies. Therefore, user friendly tools to support verification are needed. Based upon this formalization, we should learn

more about the relevant primitives for particular application domains and how to

encapsulate t`hem.

There are research activities '-associated with defining and tailoring models.

These include process models, methods and tools: product models of the various

products and qualities of those products: and models of information, like goal

generation languages, cost, resource allocation, risk, and defect prediction.

Models must be defined for the Project, Organization and the Experience F_tctory

and must take into account their interface. This involves the definition or'

languages for defining these models and tool generators, i.e. tools that ,,'an be

instantiated to support` variations cot"a method.

There are research activities associated with generating larger product units

from the Experience Factory. These include defining models of module interconnection languages that scale up, combining specifications and verifying them. and

combining test plans to validate integrated components.

There are research activities associated with the building and accessing _i'

the experience base, e.g., mechanisms for encoding lessons learned into :_ m_,tel.

tools for generating goals and mapping t,hem onto measures, models that p<'mi_

_he model to learn automa¢ically.

5.3. Implications for Education

The organizational scheme provides a focus for many of t,he technologies

already taught, at the University and so makes much of the current education

more relevant. Topics that require more emphasis are formalisms of all kinds,

e.g., verification technologies, formal requirements and specification notations.

formal models of measurement and management. There is a need to teach students how to develop, use and assess methods and tools and deal with access and

retrieval of libraries. Reuse and learning technologies need to be made available.

There is a clear entry path for new software engineers through the Component Factory where they can develop small component_s under careful guidance

and tool support and learn from the general experience base. ,-ks their experience

grows they can be moved into any of the other higher level activities, e.g.. the

Project Organization. or other parts or" the Experience Factory.

5642

2-61
- 20 -

6. RESEARCH ACTMTIES AT M_ARYLAND THAT SUPPORT

THE NEW LIFE CYCLE

The paradigms and organization described in this paper offer a framework

for research that focuses on the key issues for improving the software process and

product in a context that permit the research to be used and experimented with

in an industrial setting. Over the past dozen years, at the University of Maryland. we have been working on several research projects whose goal is to evolve

to this framework.

The projects are organized into those dealing with the instantiation ol_ the

improvement paradigm in the SEL [5,46], where the concepts of the Project

Organization and the Experience Factory have been evolving, the TA_\IE project

which is automating support for this framework in a formal way. and a variety

of odmr projects which are attempting to understand, formalize and improve

various process and product characteristics.

A major source of activity has been the Software En_ineerin,_ Lab()r:Lu)ry

(SEL), a joint venture of the NASA Goddard Space Flight Center. the University

of .Maryland. and Computer Sciences Corporation. The SEL has informally -_cred

an Experience Factory that supports project development. The application

domain is ground support software for satellites. We have been building mo, iets

and supplying these models and lessons learned back to project_ _o they can

improve their process and product. This work has been performed via experiments of various kinds, dealing with resource, defect, process, and pro,tact

models.

In an attempt, tobetter understand t,he environment we have _se_i ,t:_Ia ,'_iletted _turing development to build vario_ts descriptive models _t' the .'<EL

_nvironment. In this way we have formalized knowledge from raw ,late re) I'o.rreal models or baselines and made the results available to the pr@e,:t ,_E'g:tnization for use in characterizing, planning and evaluating the project.

We have collected data on resource expenditures, applied various -xisting

models [32.47,4t/.61 i and eventually built and tailored mo(tels that e×plicitlv

described resource aitocat, ion in the SEL environment [2.8,1().18.30]. These are

used for estimating, planning and evaluating new projects.

We have developed baselines for defects by accumulating defect data over

many projects [62]. These defects a classified by phase and type. They vary

with different project classifications [16]. They provide insight into the environment. support for project management and evaluation, and point to are:us areas

that need improvement in the process [18].

We have used various product metrics [41,-t5] to provide insight into the

characteristics of the products being developed as welt as evaluating t.he _Lsel'ulhess of these metrics for the SEL environment [6,11.26.-t21. Areas _)f new t_,_'hm)logy that have been introduced, like Ad:t havo generate, l the nee, i fc;r ,t,.v,l(_pin_

5642

2-62
-21 -

new metrics to characterize new-product qualities [12,40}. We have used these

metrics as baselines to provide the project manager with insights as to what the

problems may be-with the current development [38].

With regard to process improvement, we have built descriptive and perscriprive models of processes, methods and techniques and experimented with their

application. The results of our studies are formalized and reused for future projeers within the limits of the technology available.

In some cases, we have performed controlled experiments in which we

analyzed the effects of various methods and techniques before recommending

them on actual projects. "vVe would then perform case study experiments Go

evaluate the effect of the method or technology on an actual project development

to assure that it scales up and is applicable to the SEL environment. For exampie, we ran controlled experiments on a set of structured programming methods

and techniques [17], various testing and reading techniques [23.54]. object

oriented design in Ada [12,40] and the Cleanroom process model [5.9I.

\Ve then apply these approaches to projects within the project orgamzation.

We evaluate their effect there, and make recommendations, write lessons learned

documents, and refine or change the models to incorporate what we have learned.

In this way, the experiences gained from applying a particular model from the

experience base is improved based upon the lessons learned from applying the

model so that it can be used for future projects. Two case studies currently

being run in the SEL, based upon controlled experiments, are the use of object

oriented development in Ada [1,14,35) and the application of the Cleanroom procezs.

In other instances we have developed models and experimented directly _>n

the projects. For example, we have evaluated the test methodology _se_l for

acceptance test [.51] and the methods used /'or maintenance i5.-5].

Parts of the data collection process have been automated for the FORTRAN

environment [37. 43] and are being automated for the transition to Ada :3!)!.

Other tools have been developed that help support the various technologies used.

Parts of the evaluation process have been automated using a knowledge base to

create a decision support system [50,60 I.

The Tame project has focused on the architecture for the measurement and

evaluation processes [19[. Work has been done by using studies performed in the

SEL to define the process improvement mechanisms [18]. We have devised a

resource planning and feedback model that is consistent with the Improvement

Paradigm [43].

The Goal/question/Metric Paradigm has been applied in a variety' of

environments other than the SEL and has evolved b:_sed upon these activities

5642

2-63
- 22 -

We are currently working on supporting the automation of the generation of

operational goals in a reasonably complete and consistent manner. A key aspect

of the approach is that project personnel can generate goals that can be measured and evaluated. We are working on extending the GQM templates into a

goal generation language that will aid the goal writer in articulating questions

and metrics based upon the goal and the model of the object of interest. We are

currently experimenting with hypertext and attribute grammar technology to

develop prototypes of this automated support mechanism.

hVe are in various stages in the development of three measurement, tools for

analyzing programs in Ada and C. A source code analyzer for various syntactic

metrics, such as cyciomatic complexity and software science metrics, has been

developed for Ada (ASA.P) [38] and C (CSA.P). A structura[ coverage analyzer

(SCA) is under development for Ada [63}. Data bindings analyzers are being

developed for Ada and C based on prior versions of the tools for FORTRAN.

SIMPL, and PL/C.

"_Ve have developed a set Of requirements and defined a system arohi_ec_LIr_:

for measurement tool generations using a parser generator that retains the p:_r_

tree for further transformations [-t8!. an enhancement of YACC and al'_ exp,.rimenting with the prototypes of this tool generation system.

With regard to reuse, we have developed a model of a reuse supt>_rt

environment that can exist within the T._ME framework [20]. We have appli,,i

the model to the maintenance process to show the advantages o_" viewing [ll_.]nt,,-

nance as a reuse process [7].

'_Ve are developing a mo(lel of revise consistent with the appro,_ch pre_eI_r,_.i

in this paper that classifies the objects as they exist in the experience b:_,s__,,rh_,

reuse activities and the objects ,_ they are reused !21 i. For example, the I',_:_lsai_l,

object can be classified according to the characteristics of the unit itself, its ii_t,,_'-

faces, and its context. The model recognizes t,he need to assess tile ,-tuaiiti_:> ._1'

the reusable object based upon _he characteristics of the project in which ir wii[

be reused.

'_Ve are working on a language and support system that takes element:_rv

processes and generalizes them into more complex processes. Elementary

processes correspond to the "basic algorithms" used to perform small tasks, such

as the addition of two atomic units. Our goal is to identify useful sets of elementary processes, and then show how they can be combined and extended to perform more complex actions (such as the addition of a stream of atomie units.)

Using our language, abstract data structures may be mapped onto particular

structures (e.g., the addition process for streams eouM be mapped onto a process

for addition of arrays of numbers), and also composed with other structures (e.'4..

an array addition task could be _;omposed with a division task in order _o cre-/t_,

a module for computing means.) Finally the system will package _'esuitin_

processes into an ac_:eptal)le l:Ll/_l:tge oompollent, whet, her a proeeduro ,?r t',l[_'-

tAon. Our current Iang:_a_le supp<)rrs ,)nly ]'llI/t'tio/l:tl [)l'(sc'_Jb.-,_s. :_ I'lllllI'e sI,.[_ i.'- t_,

5642

2-64
- 23 -

support the creation of data abstractions or modules.

To study the issue of code reuse, the LASER project is currently building a

system that examines exising systems in order to study and extract code that can

be reused to seed a component repository. The system measures the various

components in the system and identifies candidate reusable components based

upon their tack of complexity, reusability within the existing system, independence, etc. These candidate components are then isolated (made independent)

and qualified. The qualification involves the eatagorization and classification

based upon a number of attributes, and the association of a f, lncdonal

specification with the component.

The approach expressed here provides a focus for further research issues.

Some of the questions for which work has begun are:

• How can process modeb be formally expressed so they can be comm_lnicate,i.

analyzed and tailored?

• How can various models be stored so they can be accessed by the GQ*I t_l

and help generate the automated collection of the appropriate me:Lsuros?

• How can a specific process model be developed that satisfies r,he _tetinition ai>t

storage of the prior two questions?

• How can we better capture and reuse experiences in the form of lessons learned

from previous efforts?

• What other measurement data can be automatically collected?

• How could the set of measurement tools defined above be developed s_) th:_r

they can be tailored t'or various types of measures, maximizing the I'o_lse of system components among the tools and the language independence']

• How can we classify experience so it can be appropriately reused?

• Based upon a specification, how can a component be devise_t ,l,dckly from ¢_l_:-

mentary processes:

• How can we transform existing components to make them more independent.

and measure the cost of reuse?

• How can we have confidence that the factory-provided modules will _to what

we want?

• How can we integrate aggregates of modules with their associated attributes so

that they can be analyzed, managed, and controlled?

• How can we verify properties of aggregates of modules, not just individu:d

modules?

• How can the test, plans for components be combine_t to provi,le :l test pl:ln

and oracle for aggregate application struet Ires?

5642

2-65
- 24 -

7. CONCLUSIONS

The approach expressedin thispaper has evolved over years of studying and

experimenting with software development and maintenance. It provides a compatable and consistentframework for both software development and software

engineering research. It recognizesand takes advantage of the experimental

nature of softwareengineering.

It allows us to understand how things are being done and where the problems are by studying the process and product in actual environments. It allows us

to formalize models of the process, product and knowledge. These models ,.'an

then be analyzed. They can be used to form a basis for research and at the same

time provide immediate input to project development.

From a research perspective, it provides a focus for research problems b_ed

upon problems that need to be solved. It provides a framework to rio together

existing pieces of research.

From a corporate perspective, the approach can be applied directly and 'he

organization can grow and build its own experience base. It supports "_chnoi_:%

transfer in a natural way and it ties the research and development organizations

closer together.

REFERENCES

il! W. Agresti, "SELAdaExperiment:Status and Design Experience." P!'o,'o-,iings of the Eleventh Annual Software Engineering Workshop. N\a.\ Go, I,lar_l

Space Flight Center, Greenbelt, biD. December 19_6.

•121J. Bailey, V. R. Basili, "A Meta-Model for Software D_+volopment R_:soul.,._

ExDenditures." Proceedings of the Fifth [nternatioual Conference _m .'5_.q'twar_

Engineering, San Diego, USA, March 1981. pp. 107-115.

[3i v. R.

N[odels and

Catalog No.

Basili, "Data Collection, Validation. and Analysis." in Tutorial

Metrics for Software Management and Engineering, [EEE

EHO-167-7, 1981, pp. 310-313.

()i1

[4] V. R. Basili, "Quantitative Evaluation of Software Engineering Methodology," Proc. of the First Pan Pacific Computer Conference, Melbourne, Australia.

September 1985 [also available as Technical Report, TR-1519, Dept. of Computer Science, University of Maryland, College Park, July 198.5].

'5] V. R. Basili, "Can We Measure Software Technology: Lessons Learne_[ from

S Years of Trying," Proceedings of the Tenth Annual Software Engineering

Workshop, NASA Goddard Space Flight Center, Greenl)elt. X[I). I),_'.ml),r

1_18.5.

5642

2-66
- 25 -

[6]V. R. Basili,"Evaluating Software Characteristics:Assessment of Software

Measures in the Software Engineering Laboratory,"Proceedingsof the Sixth

Annual Software EngineeringWorkshop, NASA Goddard Space FlightCenter,

Greenbelt,Nil),1081.

[7] Victor R. Basili, "Software Maintenance = Reuse-Oriented Software

Development," in Proc. Conference on Software Maintenance, Key-Note

Address, Phoenix. AZ, October 1988 [also available _ Technical Report. TR0244. Dept. of Computer Science, University of Maryland, College Park. July

19851 .

[81 V. R. Basili, J. Beane. "Can the Parr Curve help with the Manpower Distribution and Resource Estimation Problems," Journal of Systems and Software.

vol. 2, no. 1, 1981, pp. 47 -57.

[9] V. R. Basili. G. Caldiera. "Reusing Existing Software." Technical Report-

:2116, Institute for Advanced Computer Studies. University of Maryland. (_'o_leze

Park, Maryland. October 1988.

[10J V. R. Basili, K. Freburger. "Programming Measurement and Estimation in

the Software Engineering Laboratory." .Journal of Systems and Software. vol. 2.

no. i, 1981, pp. 47-57.

[11] V. R. Basili, D. H. Hutchens. "An Empirical Study of a Syntactic Xleasure

Family," IEEE Transactions on Software Engineering, vol. SE-9. no. 1i.

November 1983. pp. 664-672.

[12] "v. R. B_ili, E. E. Katz. "Metrics of Interest in an Ada Development."

Proc. of the [EEE Computer Society Workshop on Software Engineering Technology Transfer, April rosa, pp. 22-29.

[13] v. R. Basili, E. E. Katz, "Examining the Modularity of Ada Pro,_rams."

Proc. of the Joint Ada Conference, Arlington. Virginia, March 16-[9. 1!)87.

[14] V. R. Basili, E. E. Katz. N. M. Panlilio-Yap, C. Loggia Ramsey. S.

Chang, "Characterization of an Ada Software Development," IEEE Computer

Magazine, September 1985, pp..53-65.

[15] V. R. Basili, N. M. Panlilio--Yap, "Finding Relationships Between Effort

and Other Variables in the SEL," IEEE COMPSAC, October 1985.

[16] V. R. Basili, B. Perricone, "Software Errors and Complexity: An Empiric'll

Investigation," ACM Communications, vol. 27, no. 1, January 1984. pp. -t5-52.

[17] V. R. Basiti. R. Reiter, .lr., ":\C:ontrolled Experiment Quantitatively

Comparing Software Development Approaches," [EEE Transactions on Softxvnr,,

Engineering, vol. SE-7. no. 5. May l!)Sl, pp. 2.99-320.

5642

2-67
- 26 -

[18] V. R. Basili, H. D. Rombach, "Tailoring the Software Process to Project

Goals and Environments," Proc. of the Ninth International Conference on

Software Engineering, Monterey, CA, March 30- April 2, 1987. pp. 345-3.57.

[19 t V. R. Basili. H. D. Rombach "The TAME Project: Towards

Improvement-Oriented Software Environments," IEEE Transactions on

Software Engineering, vol. SE-14, no. 6, June 1988, pp. 758-773.

[20] V. R. Ba.sili, H. D. Rombaeh, "Software Reuse: A Comprehensive

Framework," CS-TR-21.58, Department of Cgmputer Science. University of

Maryland. College Park. Maryland.

[21] V. R. Basili, H. D. Rombaeh, J. Bailey, and B. G. Joo. "Software

Reuse: A Framework." Proc. of the Tenth Minnowbrook \Vorkshop on

Sot'tware Reuse. Blue Mountain Lake, New York, July 1987.

[:22] V. R. Basili. R. \V. Selby. Jr., "Data Collection and Analysis in Software

Research and Management." Proc. of the American Statistical Association :m,i

Biomeasure Society .Joint Statistical Meetings, Philadelphia, PA. August 1:3-1 f).

1984.

[23] Victor R. Basiti. R. W. Selby, "Comparing the Effectiveness or' <oft',v:ir,_

Testing Strategies," IEEE Transactions on Software Engineering. Vol.

SE-la, No. 12, December 1987, pp. I.'278-1'2,96.

[24} V. R. B_ksili. R. W. Selby. Jr., "Calculation and Use of an Environment'_

Characteristic Software Metric Set." Proceedings of the Eighth International

Conference on Software Engineering, London, [-I,,[. Aug_Lst 10SS.

[25] V. R. B_mili. R. W. Selby, D. H. Hutchens. "Experimentation in

Software Engineering," [EEE Transactions on Software Engineering. v<_i.Sl[

1:2. no.7. July 198(5. pp.733-743.

[:26] V. R. Basili. R. \V. Selby, and T.-Y. Phillips, "Metric Anal.vsis an, i D:,r:x

Validation Across Fortran Projects," IEEE Transactions on Software Engine,._l --

ing, vot. SE-9, no. 6, November 1_83, pp. 6.52-663.

[271 V. R. Basili, A. d. Turner, "Iterative Enhancement: A Practical Technique for Software Development," IEEE Transactions on Software Engineering.

vol. SE-1. no. 4. December 1975.

[28 t V. R. Basili. D. M. Weiss, "A Methodology for Collecting Valid Software

Engineering Data." IEEE Transactions on Software Engineering, vol. SE-Ii).

no.6. November 1984. pp. 728-738.

[29] V. R. Basili. D. N[. Weiss, "Evaluation of :t .%)ftwaro Requiroln<_f> I}_,,._iment by Analysis _>i'_ih:mge Data." Procee_tin_s _)t' the f'_il'ttl [ntel'n:tli_m:_]

5642

2-68
°),-

Conference on Software Engineering, San Diego, USA, March 1981, pp. 314-323.

[30] V. R. Basili, M. V. Zeikowitz, "Analyzing Medium Scale Software

Development, Proceedings of the Third International Conference on _oftware

Engineering, Atlanta, Georgia, USA, May 1978, pp. 116-123.

• • ¢1 '! [31] B. W. Boehm, "Software Engmeenno, IEEE Transactions on Computers,

vol. C-25, no. 12, December 1976, pp. 1226-1241.

[321 B. W. Boehm, "Software Engineering Economics." Prentice-Hall, Englewood Cliffs, N J, 1981.

[33j B. W. Boehm, "A Spiral Model of Software Development and Enhancement," ACM Software Engineering Notes, vol. ll, no. 4. August 1.986, pp. "22-

42.

[34! B. W. Boehm. J. R. Brown, and M. Lipow, "Quantitative Evaluation .of

Software Quality," Proceedings of the Second International Conference on

Software Engineering, 1976, pp. 502-60.5.

[35] C. Brophy, W. Agresti, and V. R. Basili. "Lessons Learned in Use of Ada

Oriented Design Methods," Proc. of the Joint Ada Conference, Arlington. \irginia, March 16-19, 1987.

[36] _V. J. Decker, W. A. Taylor. "Fortran Static Source Code Analyzer Program (SAP)," Technical Report SEL-82-002. NASA Goddard Space Flight

Center, August 1982.

[:371 C. W. Doerflinger, V. R. Basili. "klonitoring Software Dovelopment

Thro_tgh Dynamic Variables." IEEE Transactions on Software Engineering, vol.

SE-11. no. 9, September 1.985, pp. 978-985.

[38] D. L. Doubleday, "A&_P: An Ada Static Solfl'ee Code Analyzer Pt'ogram."

Technical Report, TR-t89.5, Deptartment of Computer Science. University of

Maryland, College Park, August 1987.

[39] M. Dyer, "Cleanroom Software Development Method," IBM Federal Systems Division, Bethesda, Maryland, October 14, 1982.

[40] J. Gannon, E. E. Katz, and V. R. Basili, "Measures for Ada Packages:

An Initial Study," Communications of the ACM, vol. 29, no. 7, July I986, pp.

616-623.

[41] M. H. Halstead, "Elements of Software Science," Elsevier North-Holland.

New York, 1977.

1421 D. H. Hutchens. V. R. Ba.sili, "System St,r,lct lro :\naly,_is: ('l_lsuwin_ with

5642

2-69
- 28-

DataBindings." IEEE Transactionson SoftwareEngineering,August198.5,pp.

749-757.

[43t D. R. Jefferey,V. R. Basili, "Validating the TAME ResourceDataModel."

Proceedingsof the Tenth InternationalConferenceon SoftwareEngineering,

Singapore,April, 1988,pp. 187-201.

[441E. E. Katz, H. D. Rombach,andV. R. Basili,"Structure and Maintainability of Ada Programs:CanWe Measurethe Differences?,"Prec. of the

Ninth MinnowbrookWorkshopon SoftwarePerformanceEvaluation.BlueNiounrain Lake.NewYork. August5-8, 1986.

[45]T. J. McCabe."A ComplexityMeasure,"IEEETransactionson Software

Engineering,December1976,pp. 308-320.

[46]F. E. McGarry. "RecentSELStudies."Proceedingsof tile TenthAnn_lai

SoftwareEngineeringWorkshop,NASA GoddardSpaceFlight Center.December

1985.

[471F. N. Parr. "An Alternative to the Rayleigh Curve Model for Software

Development Effort," [EEE Transactions on Software Engineering, vet. SE-6.

no. 3, .March 1980.

[48] J. Purtilo and J. Callahan, "Parse Tree Annotations", Communications of

the ACM, to appear.

[49] L. Putnam, "A General Empirical Solution to the .Macro Software Sizin_

and Estimating Problem." IEEE Transactions on Software Engineering. "oi.

SE-4. no. 4. April 1978, pp. 34.5-361.

[50] C. Loggia-Ramsey. V. R. Basiti, "An Evaluation of f_xpert Systems for

Software Engineering .Management." [EEE Transactions on Software Engineering, Vol. 1.5. no. 6. June 1989. pp. 747-7.597.

[.51J J. Ramsey, V. R. Basili, "Analyzing the Test Process Using Structural

Coverage." Proceedings of the Eighth International Conference on Software

Engineering, London, U-K, August 198.5.

[.521H. D. Rombach, "Software Design Metrics for Maintenance." Proceedings

of the Ninth Annual Software Engineering Workshop, NASA Goddard Space

Flight Center, Greenbelt, NED, November 1984.

[.53] H. D. Rombach, V. R. Basili, "A Quantitative Assessment of Software

Maintenance: An Industrial Case Study," Conference on Software .Maintenance.

Austin, Texas, September 1987.

[54] H. D. Rombaeh. V. R. B_ili. and R. \\'. Selby..Jr.. "The Role ol'('()(i_,

5642

2-70
- 29 -

Reading in the Software Life Cycle," Proc. of the Ninth Minnowbrook Workshop

on Software Performance Evaluation, Blue Mountain Lake, New York. August

5-8, 1986.

[55] H. D. Rombach, B. T. Ulery, "Establishing a Mea.surement-Bused Maintenance Environment Program: Lessons Learned in the SEL", Proceedings of the

IEEE Conference on Software Maintenance, Miami Beach, October, 1989.

[56] W. W. Royce, "Managing the Development of Large Software Systems:

Concepts and Techniques," Proceedings of the 'vVESCON, August 1970.

[57] R. W. Selby. Jr., "Incorporating Metrics into a Software Environment."

ProCeedings of the Joint Ada Conference, Arlington. VA, March 16-1!). [087. pp.

326-333.

[58] R. W. Setby, Jr., V. R. Basil[, "Analyzing Error-Prone System Co,lpling

and Cohesion." Technical Report TR-88-46, Institute for Advanced Comp_ter

Studies. University of Maryland. College Park, Maryland. June 1988.

[59] R. W. Selby, Jr., V. R. Basil[, and T. Baker, "CLEART_OO.\I Software

Developmentx An Empirical Evaluation." [EEE Transactions on Software

Engineering, Vol. 13 no. 9, September, 1987, pp. 1027-1037.

[601 J. D. Vale[t, "The Dynamic Management Information Tool

(DYN._\ilTE):Analysis of the Prototype, Requirements and Operational

Scenarios." M.Sc. Thesis, University of Maryland. 1987.

[61] C. E. WaIston, C. P. Felix, "A Method of Programming X[e'uCm','m,:n'_

and Estimation." IBM Systems Journal, vol. 16. no. t. 1977. pp. 54-7:;.

[621 D. M. Weiss. V. R. Basil[, "Evaluating Software Development by Analysis

of Changes: Some Data from the Software Engineering Laboratory." IEEE Transactions on Software Engineering, vol. SE-11. no. 2. February [98.5. pp. 157-

168.

[63] L. Wu, V. R. Basil[, and K. Reed, "A Structure Coverage Tool for Ada

Software Systems," Proc. of the Joint Ada Conference, Arlington, Virginia.

March 1(3-19, 1987.

[64] M. Zelkowitz, R. Yeh, R. Hamlet, J. Gannon, and V. R. Basili, "Software

Engineering Practices in the U.S. and Japan," IEEE Computer Magazine, .Iune

1984, pp. 57-66.

5642

2-71

SECTION 3--MEASUREMENT ENVIRONMENT

STUDIES

SECTION 3 - MEASUREMENT ENVIRONMENT STUDIES

The technical papers included in this section were originally

prepared as indicated below.

• Inteqratinq Automated Support for a Software Management C¥cle Into the TAME System, V. Basili and

T. Sunazuka, University of Maryland Technical

Report TR-2289, July 1989

• Towards A Comprehensive Framework for Reuse: A

Reuse-Enablinq Softwar_ Evolution Environment,

V. Basili and H. Rombach, University of Maryland,

Technical Report TR-2158, December 1988

5642

3-1
UMIACS-TR-89-75

CS-TR-2289

July, 1989

Integrating Automated Support for a Software

Management Cycle into the TAME Systemt

Toshihiko Sunazuka

NEC Corporation

Tokyo, Japan

Victor R. Basili

Institute for Advanced Computer Studies

Computer.Science Department

University of Maryland

College Park, MD 20742

5642

3-2
ABSTRACT

Software managers are interested in the quantitative management of software

quality, cost and progress. There have been many of models and tools developed, but

they are of limited scope. An integrated software management methodology, which

can be applied throughout the software life cycle for any number purposes, is required.

The TAME (Tailoring A Measurement Environment) methodology, developed at

the University of Maryland, is based on the improvement paradigm and the

Goal/Question/Metric (GQM) paradigm. This methodology helps generate a software

engineering process and measurement environment based on the project characteristics.

The SQMAR (Software Quality Measurement and Assurance Technology)

developed in NEC is a software quality metric system and methodology applied to the

development processes. It is based on the feed forward control principle. Quality target setting is carried out before the Plan-Do-Check-Action activities are performed.

These methodologies are integrated to realize goal--oriented measurement, process

control and visual management. The Software Management Cycle is a substantiation

of these concepts. Based on the TAME process model, development and management

environments can be generated. The SQMAT system helps target setting, data analysis

and visual display.

In this paper we discuss a metric setting procedure based on the GQM paradigm,

a management system called the Software Management Cycle (SMC), and its application to a case study based on NASA/SEL data. A method for evaluation Software

Management Cycle process is described. The expected effects of SMC are quality improvement, managerial cost reduction, accumulation and reuse of experience, and a

highly visual management reporting system.

KEYWORDS

TAME, improvement paradigm, Goal/Question/Metric paradigm, SQMAR.

Plan-Do-Check-Action activities, process control, visual management, software engineering process, goal-oriented measurement, software quality metrics.

% Research supported in part by NASA grant NSG-S123 and NEC (through the I.ndustnal Associates Program of the Department

of Computer Science.)

5642

3-3
1. Introduction

Management plays a key role in the software development process. In the end, it is

management's responsibility to produce and deliver a quality product productively and profitably

and to generate corporate credibility with the customer. Thus, effective management methodologiea are needed to support management in assessing the current status of the project and achieving delivery of the final system on-time, within budget, and with the specified product qualities.

It would also be useful if the methodology supported the improvement of quality and productivity

on the current project and on future projects. Many companies are working to provide such

methods for their managem.

However, it is difficult to assess the current status of a project precisely because of the lack of

visibility of the software during development. It is even more difficult to predict project progress

because of the lack of clearly defined goals, the lack of feedback in the achievement of those goals,

and the difficulties caused by the variation in personnel.

2. Supporting Methodologies

Thus, requirements for the management methodology include the abilityto make the software

as visible,quantifiable and objective as possible. Several methodologies and paradigms use

metrics to satisfythese management needs during development. There have been many software

metrics proposed in the literaturethat attempt to provide the visibility,quantificationand objectivity[Boeh76, McRW77, MuriS0, BaKa83 I.

From a customer perspective of product quality,a compreheusi_e set of quantifiablesoftware

characteristicswere proposed by Boehm, et al. [Boeh761 and laterrefined by McCall and Waiters

[McRW77]. Based on these studies,Software Quality Metrics (SQM) was developed by Murine

(METRIQS Incorporated) as a quantitativesoftware quality assessment technology {Muri80 I.

SQMAT

Based upon the SQM, the NEC Corporation has developed a Software Quality Measurement

and Assurance Technology (SQMAT) [AzSM87, AzSu86, SuAY85] and has been using it as one of

the support tools in their software quality control (SWQC) group activities [Mizu82]. Quality

control seminars are held periodically for every level of worker; programmer through general

manager. The seminars are used to motivate as well as educate everyone with respect to the

quality control technologies.

SQMAT is a software quality metric system and methodology applied to the development

processes, which takes experimental SQM results into consideration. SQMAT consists of a quality

measurement and evaluation method with three levels of quality criteria, and a support tool for a

visual display for management. Its most notable feature is that the feed forward control principle

is emp'loyed in addition to the feedback control principle. That is, quality target setting is carried out before the Plan-Do-Check-Action activities (Deming's PDCA cycle) are performed.

SQMAT procedures are defined as follows:

(1) In the TARGET phase, a quality priority ranking is established for the individual quality

characteristics, based on the users' requirements and the development policy. It is important to clarify the quality target, i.e., classify the quality characteristics into 3 categories

and set the target quantitatively.

(2) In the PLAN phase, Software Quality Measurement Criteria (SQMC), are set up and

methods for achieving the target quality are discussed in advance, primarily with the

5642

3-4
quality assurance people and managers.

(3) In the DO phase, high quality software is produced by complying with development standards and SQMC as guidelines. "Before the formal review, the developer executes a quality

self-check.

(4) In the CHECK phase, the software is checked and evaluated against the individual quality

criteriaset up in the PLAN phase. Quality is measured by a third party. If errors are

detected, problem reports are drawn up. After scoring, score sheets and quality graphs are

developed, and the achieved quality isjudged by comparing it to the target quality level.

(5) In the ACTION" phase, correctiveaction is taken, based on problem reports. Achieving the

quality target permits proceeding on to the next phase. SQMAT can be applicable not only

to large scalesoftware, but also to small projects. NEC's experience with the approach has

had measurable results. For example, based upon comparison with historicaldata, (i) a

number of errors have been eliminated during the design and implementation phases, and

(2) productivity (measured by lines-of-source-code/hour) has increased by 10%.

The Improvement Paradigm

The Quality Improvement Paradigm [Bas85a] for software engineering processes is a top level

paradigm that is based upon the scientificmethod as applied to software evaluation. It provides

the view of software evolution as an experimental process from which we must learn and improve

the current project as well as [uture projects (Characterize, Set Goals, Choose Methods, Build,

Analyze, Learn and Feed Back). It is a meta-life cycle model that aims at improving the

software quality and productivity based upon measurement and reuse of experience. It needs to

be instantiated for a variety of sub-activities,e.g. specificprocesses such as testing,product

reviews, managing. It consistsof six major steps:

(I) Characterize the current project environment.

(2) Set up goals and refinethem into quantifiablequestions and metrics for successfulproject

performance and improvement over previous project performances.

(3) Choose the appropriate software project execution model for this project and supporting

methods and tools.

(4) Execute the chosen processes and construct the products, collectthe prescribed data, validate it,and analyze the data to provide feedback in real-time for correctiveaction on the

current project.

(5) Analyze the data to evaluate the current practices,determine problems, record the findings

and make recommendations for improvement for future projects.

(6) Package the experience in the form of updated and refinedmodels and other forms of structured knowledge gained from this and previous projects and proceed to step i to start the

next project.

This paradigm is aimed at providing a basis for corporate learning and improvement

[BaRo87] and isbased upon experience with measurement and evaluation of software development

in a number of companies.

Goal Question/Metrlc Paradigm

The Goal/Question/_etric (GQ1V[) paradigm [BaWe85, BaSe84] is a mechanism for generating

measurement in a goal-directed manner. It represents a systematic approach for setting the project goals (tailored to the specificneeds of an organization), defining them in an operational,

tractable way by refiningthem into a set of quantifiablequestions that in turn imply a specificset

of metrics and data for collection(addresses the aspects related to step 2) of the improvement

5642

3-5
paradigm). Appropriate metrics are tailored to each project based on the G/Q/M templates and

past experience. It includes the development of data collection mechanisms, e.g., forms,

automated tools, the collection and validation of data, and the analysis and interpretation of the

collected data and computed metrics in the appropriate context of the questions and the original

goals.

In order to support the process of setting goals and refiningthem into quantifiablequestions,

a set of templates for setting goals, and a set of guidelinesfor deriving questions and metrics has

been developed [BaRo88]. These templates and guidelines reflectour experience from having

applied the GQM paradigm in a variety of environments [RoBa87, WeBa84, BaWe81 I.

Goals are defined in terms of purpose, perspective and environment. Different sets of guidelinesexist for defining product-related and process-related questions. Product--relatedquestions

are formulated for the purpose of defining the product (e.g.,physical attributes,cost,changes and

defects,user context), defining the quality perspective of interest(e.g.,functionality,reliability,

user friendliness),and providing feedback from the particular quality perspective. Processrelated questions are formulated for the purpose of defining the process (process conformance,

domain conformance), defining the quality perspective of interest(e.g.,reduction of defects,cost

effectivenessof use),and providing feedback from the particularquality perspective.

The TAME (Tailoring A Measurement Environments) system [BaRo88] is a measurement

environment that supports and integratesthe Quality Improvement and the Goal Question Metric

paradigms.

Based on the work at NEC, the TAME project, and the managerial requirements specified

above, a management methodology, called the Software Management Cycle (SMC), has been

developed. Its main concepts are goal oriented, process control and visual management. Management procedures, support tools and forms, and an evaluation method are provided as part of

SMC.

3. Relationship of the SQM, GQM, and SQMAT

The SQM and the GQM are both mechanisms for measuring software quality. Both models

are top-down and characterize quality characteristicsat three levels. In the SQM, these levelsare

Factor, Criteria, and Metric. For example, a high levelfactor such as correctness isdefined by

the set of criteriatraceability,completeness, and consistency which in turn are defined in terms of

a predefined set of metrics.

The GQM model consistsof a goal, which isspecifiedby a set of quantifiablequestions,which

in turn are defined by a set of metrics and data distributionstailoredto the specificenvironment.

Thus to define a high levelgoal like correctnessof the finalproduct, we must definea set of questions that characterize the product (with respect to its physical attributes,cost of development,

changes and defects,and customer base and operational profile),define a model for correctness

(which could include such concepts as traceability,completeness, and consistency), provide

insights into the validityof the model and the data within the particularenvironment, and the

resultsof the model along with some possiblesubstantiation of the model results.

The SQM model predates the GQM model, but the latter is more general. The GQM can be

used to characterize, evaluate, predict, or motivate a product, process, model or metric, with

respect to a variety of perspectives (e.g. customer, developer, user, manager, etc.) based upon an

open ended definition of quality. It takes into account the specific environment in which the product has been developed as well as assessment of such things as an evaluation of how well the

5642

3-6
particular methods were used, how well the domain of application was understood in order to

help interpret the resulting evaluation metrics appropriately. It also involves the feedback of

information for future development through learning.

The SQM model is written from the point of view of determining a set of quality characteristics of the final product from the point of view of the customer. It does not measure process for

developing that product and since its viewpoint is that of the customer, it provides limited support for learning, feedback and improvement within the development organization. Its measurement process tends to be passive and is not focussed on capturing the causes of the quality problems.

The measurement focus of SQM as used in SQMAT has evolved and widened over time and is

currently more consistent with the GQM. This wider view of SQM uses metrics to measure quality of an intermediate product from the point of user, developer and so on.

GQM SQM

narrow-sense I wide-sense

Objective Characterize, Assess, Assess (Quality)

Predict, Motivate

Structure Goal Factor

Question Criteria

Metric Metric

Usage Project & Quality Management Quality Management

Object Any Product, Process, Product Any Product

Model, or Metric Process

Viewpoint Developer, User, User (same as

Manager, Corporate GQM)

Establishmeat manner

of GQM or

SQM

G Select or Tailor

Q Select or Tailor

M Select or Tailor

F

C

M

Select

Select

Select Select

or Tailor

Table 1. Features of GQM and SQM.

5642

4

3-7
4. Software Management Cycle (SMC)

The Quality Improvement Paradigm provides a top level organizational perspective on the

software development and maintenance process. SMC is the management procedure and support

system under that paradigm. It emphasizes three concepts; goal-oriented measurement, process

control, and visual management. In response to each concept, several activities are necessary.

These activities, performed during the management procedure, make it possible for management

to achieve higher quality and productivity.

The management procedure used in SMC consistsof the following fivesteps:

(Step I) Define system/project characteristics

It is important to define the system characteristicsin detail to reflectthe user requirements

for development. A set of system/project characteristicsforms are prepared to gather information on the requirements and the current project status.

This is equivalent to the firststep of the Quality Improvement Paradigm. The system

engineer is responsible for understanding the customer requirements for the particular project

correctly. The development environment should be also clarified.This characterization permits

the comparison of the current project with prior projects with similar characteristics.This information isused in the next step.

(Step 2) Select Goals,Questions, and Metrics

To achieve high quality and productivity, itis necessary to set the specificobjectives. This is

the key step to the success of the project. Unless the goals are appropriate, the project will fail.

The GQM paradigm is used to do this. It satisfiesthe requirement for goal-oriented measurement. It helps both developers and managers clarify the objectives of the project prior to

development.

Guidelines and templates are used to establish the particular GQM used. Templates from

priorsystems can be used or modified for this project. For each metric, measurement instructions

are prepared, which includethe importance of metric, the collectionmethod and person responsible,the data presentation,the decisionsaffected etc.

Besides the set of goals and metrics for the particular project, a common set of managerial

metrics have been specifiedto be applied to all projects. We can gather the data for getting the

level of quality and productivity through projects and development phases. The metrics from

thiscommon set are shown below.

[Quality ]

- Number of detected errors at test phase (from

integration test through system test)

- Number of detected errorswithin six months afterrelease

[Productivity ]

- Number of specification pages

- Number of non-comment source statement

- Effort at each phase by man-hour

5642

3-8
(Step 3) Select activities

Methods to effectively achieve the objective are considered at this time. Appropriate activities for economically producing the software and managing the quality of the project can now be

chosen based on the specific objectives laid out in the GQM.

This step is criticalto achieve the objectives. Setting goals, without specifying the means to

achieve them, is meaningless. Sufficientdiscussion on the activityselection process is necessary

from various viewpoints; how they fitinto the development environment, how they integrate with

the management methods and training plans, etc. and how the help achieve the objectives,provide focus for the questions and affectthe definitionof the metrics.

For process control,a review checklistis prepared for each phase of development based on the

metrics specifiedby the GQM model and past history,e.g. prior fault data. Feedback to the process should also be performed as soon as possible after a review. Problems can be easily found

using the review checklist. Periodic checks; e.g. monthly, or at the finalreview of each development phase, are required to monitor the process. The earlier the phase at which monitoring

starts,the more effectiveit is for quality improvement. Audit and configuration management are

also process control methods governing quality.

(Step 4) Measure and assessthe process and the products

Project data will be collected periodically,at least at the end of each development phase.

Based on the metrics selected, the process and the products are measured. The results are

assessed by using specificrating criteria.It is helpful for manager to take proper action quickly.

Continuous measurement and assessment can produce high quality product.

For visual management, graphical displays of the appropriate management information can

be selected based on the graph selection form. The project's current status can be found by using

the visual display tool provided by the SMC system. It is helpful for software managers to see

the achieved quality level in a concrete form to support such activities as decision making, the

management of quality and scheduling of the next workload. For example, graphs provide the

manager with time series data indicating process and product changes, as well as comparative

data from Past projects.

(Step 5) Support correctiveaction

For low scoring metrics, some action should be taken. A corrective action list is prepared and

used to improve both the current and future process and products.

Based on the assessment of results at step 4, proper action is required quickly for problems or

the sign of any problems. If necessary, the activity plan can be revised. These experiences are

accumulated and used to future projects.

Guidelines necessary to perform project management, based on SMC, are as follows:

- Goal selection

- Question selection

- Metric selection

- Activity selection

- Management graph selection

- Project status diagnosis

- Corrective action recommendation

- Reliability prediction

5642
The SMC helps the manager in the definition of an appropriate software engineering process

during the GQM and activities selection phases (steps 2 and 3), by allowing the manager to tailor

goals, measures, methods and tools to the specific system/project characteristics. A data base can

be defined and built to support the measurement environment during the GQM selection phase

and to support both the development and management environments during the activities selection phase. After executing one whole cycle through the SMC process, the results of analyzing the

current project data can be fed back to each SMC phase. Updating the database and improving

each step of the SMC helps generate a software engineering process for future projects.

The SMC support system is currently a prototype built on top of existing software packages.

It consists of (1) a data base, (2) a set of statistical packages, and (3) a set of graphical types

(developed using Microsoft Excel), all integrated under a common user interface.

Accumulation of application information in a data base enables the organization to establish

guidelines for future projects. Therefore, the relation between the system characteristics and the

measurements associated with the particular GQM should be collected and saved in a data base.

Emphasis should be on the metrics common across several projects

5. An example G/Q/M

A simplified pair of GQM models, one for product and one for process are given. They are

written from the point of view of the manager (which may include some of the concerns of the

customer) for evaluating various components to improve quality, cost and usage of methods based

upon managerial data.

First we will define some terms and offer a model of the qualities of interest:

DEFINITIONS:

Size (NCSS) -_- the number of non-commentary source statement (NCSS)

Actual Effort (AEF) -----total number of staff hours to develop a component

Estimated Effort {EEF) ---- estimated number of staff hours based upon the software science

metric, E

Actual Errors (AER) -= the total number of errors reported

Estimated Errors (EER) = the estimated number of errors based upon the software science

metric, B

Actual Error Rate (AERR) -_-- AER / NCSS

Estimated Error Rate (EERR) = EER / NCSS

Changes (CH) = the total number of changes reported

Change Rate (CR) = CH / NCSS

Effort Distribution (PED) ---= the percent of staff hours for a particular component spent in each

phase

5642

3-10
Test Efficiency (PTE) = the percent of machine time spent testing a component

Work Rate (WR) = NCSS / AEF

Effort Variance (EFV) = AEF / EEF

Error Variance (ERV) = AER / EER

MODEL:

The objectives for management are cost, quality and the effectivenessof the methods.

Evaluation isperformed on the basisof improvement over some norm.

Cost can be assessed as the relationship between input, staffeffort,and output, the quantity

of documentation and program pr.oduced.In this case we will consider cost as demonstrated by

two factors:work rate (WR), which provides some measure of the cost of production for a lineof

code, and Effort Variance (EFV), which provides some measure of whether the effortis reasonable

relativeto some measure of the expected effort.

Quality is assessed in two categories, must-be quality and attractive quality. These terms,

must-be quality and attractive quality, are common Japanese quality perspectives. Must-be

quality means the fundamental qualities necessary for software to function, i.e., functionality and

reliability. Attractive quality means any additional quality characteristics for the software to

satisfy the users specific needs, e.g., usability, security, portability. In this case, we will consider

quality as demonstrated by two factors: error variance (ERV), which provides some measure of

whether the error rate is reasonable relative to some measure of expected errors, and change rate

(CR), which provides some measure of the entropy of the system.

Method characteristics axe assessed based upon their adherence to a set of standards. Project

manager experience is also assessed since the success of a project deeply depends on his ability. In

this case, we consider method evaluation using two factors: effort distribution (PED), which will

provides us some insight into whether the distribution of the effort was acceptable according to

standard baselines of effort distribution, and test efficiency (PTE) which when combined with test

time, will provide some insight into the effectiveness of the test process, and therefore the effectiveness of the methods used for development.

Note that the model uses the software science measures, E and B as a basis for estimating,

effort and bugs. It assumes these calculated values as basic estimates for the variables effort and

errors and uses them as norms when comparing the actual values for effort and errors.

In our proposed model, the values of these variables for any component are then compared to

the values for some normal population. All values within 2 sigma variation from the average are

considered acceptable. Those values with more than a 2 sigma variation in the "right" direction

are considered good; those with more than a two sigma variation in the "wrong = direction are

considered as not meeting the target goal. For example, the effort variance (EFV) for a component is considered bad ifit isgreater than two sigma above the norm determined by the average value of cost for the restof the population.

In the example given in the next section the baselines are determined by the the rest of the

component population in the particular project. In an environment where there is data from a

sufficient number of projects, the baselines could be determined by projects with similar characteristics from other projects.

5642

8

3-11
PRODUCTGOAL:

Purpose: Evaluate various software components within a project in order to assess them and

recommend areas for improvement.

Perspective: Examine the relative cost and quality from the point of view of the manager.

PRODUCT DEFIMTION:

Product Dimensions: A quantitative characterization of the physical attributes of the product.

QI. What is the size of each component in terms of non-commented source statements (NCSS)?

Q2. What is the value of the software science metrics for each component (E,B)?

Changes/Defects: A quantitative characterization of the enhancements, errors, faults, and failures.

Q3. What is the number of defects associated with each component (AER)?

Q4. What is the number of changes a_ociated with each component (CH)?

Q5. What is the fault rate, change rate (AERR, CR)?

Cost: A quantitative characterization of the resources expended•

Q6. What is the staff effort involved in the development of each component, i.e. design, code,

test?

QT. What is the distribution of effort spent in the design, code and test _>hase (PED)?

Context: A quantitative characterization of the customer community and their operational

profiles.

INo questions for this example]

In general, five viewpoints are necessary for process questions. Two of five, "Effort of Use"

and =Effect of Use', are actually used in a case study next section.

PROCESS GOAL:

Purpose: Evaluate the design, code and test processes in order to improve them.

Perspective: Examine the relative cost distribution and test efficiency from the point of view of

the manager.

PROCESS QUESTIONS:

Quality of Use: A quantitative characterization of the process and an assessment of how well it is

performed.

Q8. How much experience does the team have with respect to the methods and tools used?

Qg. How much experience does the manager have with respect to similar projects?

5642
DomainofUse:A quantitativecharacterization of the object to which the process is applied and

an analysisof the process performer's knowledge concerning this object.

Q10. How understandable are the requirements?

Effort of Use: A quantitative specificationof the quality perspective of interest. In this case, a

quantitative specificationof the costs.

Q6. What is the staff effortinvolved in the development of each component, i.e.design, code,

test?

QT. What is the distributionof effortspent in the design, code and test phase (PED)?

Q11. What is the machine time spent in the testphase for each component (PTE)?

Feedback from Use: This includes questions related to improving the process relativeto the quality perspective of interest.

Q12. What is the input to the design and code methods and tools,and the defect detection

methods and tools?

QI3. What should be automated?

O. Case Study

The concepts of SMC can be applied to a variety of project types because of the flexibilityof

this methodology. Metrics and development methodologi6s are tailored to each project. In this

section,we discuss several general issuesin applying SlVIC and provide a sample application to the

management of a specificproject based upon models and the goals, questions and metrics of the

previous section.

In executing SMC in a project, the software management procedure mentioned previously, the

templates, guidelines and some forms are used. A step by step approach based on this procedure

is demonstrated. A sufficient budget for managing these activities is required. It is also necessary to establish an organization to support the SQM process. Certainly, a seminar on SMC for

both managers and developers would have provided better results. It should be remembered that

the more experience the manager and the organization have with SMC, the better they will be

able to apply the method. The continuous application of the method provides a better support

for quality and productivity.

This example uses the NASA/SEL [McGa85, Bas85b] project data base. Thirteen newly

developed components for a particular project were selected. Size range of non-comment source

statements is from 60 to 299 LOC. Graphs for project management were made using Microsoft

Excel.

In step I, "System/Project Form" is filledout. This clarifyboth the software functional

requirements and the development environment. The profileof the system and the environment

are defined.

In step 2, the project goals are determined based on the system/project characteristicsfrom

step 1 and the managerial strategy; e.g. cost,quality levelto be achieved, methodologies to be

5642

IO

3-13
employed, etc. Each goal is extended to questions and metrics by means of the GQM template.

The "Quality Target" and "Managerial Metrics" are determined at this step. Cost and quality

improvement and better usage of various methods were chosen as goals. To support management, the "Graph Selection Form" is provided. Six graphs were selected; those are for work rate,

effort variance, error variance, change rate, effort distribution and test efficiency. Questions to

achieve these goals are shown-in Chap. 2.

In step 3, the best way to achieve GQM is discussed and appropriate activitiesare selected.

These depend on the pieces of information from previous steps. Development methodologies and

quality checkpoints are listedon a specificform. This form is used as a checklistduring development.

In step 4, the development process is monitored and managerial data are collected periodically. To make the project status visible,display graphs are very helpful. The graphs used were

selectedin step 2.

The following table shows the results of statistical analysis on the NASA/SEL project. Six

criteria on three categories are chosen. Regression analysis was executed for the "Error Variance = data. Analysis of variance was executed for the rest of data. Based on the graphs and this

table, the project's current status can be found. Comments for four of 13 components are

described below. Figure 2 shows some sample graphs.

Rules for interpreting the results

For each metric, there exists pattern to interpret the results. Consider the following examples.

[ Cost l

(I) Work Rate: Development speed measured by NCSS per man-hour

- In case of a low value, there are several potential problems

* low quality

* insufficientdevelopment environment

* looseprocess control

etc.

(2) Effort Variance: actual effortvs. estimated effort

- Evaluate the goodness by variation between estimated and

actual effort

J in the case that the actual effortis lower, the work

rate ishigh (or functions could be simple)

* in the case that actual effortishigh, the interpretation

of the resultsare the same as for Work Rate.

[Quality}

(I)Error Variation: the number of actual errors compared

with the estimated (a measure of complexity)

5642

11

3-14
Component

number

c29

c61

c6

c5

c46

c7

c9

c4

c49

c43

cll

c63

c50

C 0 S T I QUALITY

Work Effort Error Change

Rat.e Variance Variance Rate

METHOD

Effort Distribution

D C T

Test Efficiency

0

X

X XX O0

XX

XX

0

O0 0 O0

X XX

O0

0

30C

XX

0

X

XX

x XX

x

X

X

X

X

X

O

OO

O0

0

5642

D design phase

C code phase

T test phase

Assessment Criteria (except Cost Distribution)

O0 : excellent ( >= AVE +2 O" )

O: good ( >= +1 O" )

X: poor ( <= -I O" )

XX: bad ( <= -20-)

Assessment Criteria for Cost Distribution

XX: very high rate ( >=AVE+2 O-)

X: high rate ( >= +I O" )

x: low rate ( <= -1 0-)

xx : very low rate ( <_ -2 O" )

Table 2. Component Assessment Table.

- It assumes that the greater the complexity, the greater the

number of errors.

* Quality is high if the number of errors is low in

comparison with the estimated number based on complexity.

(2) Change Rate: the normalized magnitude of the number of

specificationchanges and error modifications

- In case that the number of specification change is large,

there is a problem in the development methods

* insufficient review

* less communication with user

* loose configuration control

- In case that the number of error modification is large,

12

3-15
quality isconsidered to be low.

- In both cases,degradation of the system can be assumed

due to entropy because of change

[ Methodology ]

(i) Effort distribution:effortratioof each phase

- Evaluate the percentage of effort in each phase (design /

coding / test).

* IS the effortin the design phase sufficient?

In the case of insufficienteffort,the degree of

specificationcompletion isconsidered to be low.

* Does itcost too much in coding phase?

In case of too much effort,it isassumed that the

specificationisinsufficientor the development

environment isnot so good.

* Is the effortappropriate in test phase?

In case of too littleeffort,it isassumed that testing

was insufficientand the system was delivered with errors.

In case of too much effort,it isassumed that the test

method isnot efficientand/or the qualityis low so

the testphase lastedtoo long.

(2) Test efficiency : percent of machine time in the test phase

- The ratio is high if the preparation of test cases is sufficient.

- The ratio is high if quality is high so error modification

effort is sm_ll.

Assumed activities in the test phase

[PreparationI MachineTest/ xed/ [ rror modification}

The pattern for interpretating of results can be made by combining the above heuristics.

Comments

c5 :

Quality is good, but cost is high. Because of the high cost

in design and code phases, product quality must be high.

Some changes may have caused the rise of both design and

code cost rate.

(good)

Quality ishigh.

(to be improved)

Work rate is low.

(diagnosis)

It isnecessary to monitor the early process to avoid

the slideof schedule. Quick feedback and effective

5642

13

3-16
reviews age nece_ary.

c7

There is a problem in the methods for development and management.

The number of changes is large. This caused the rate of

design and code to be too high. Because of insufficient test

instead of high test efficiency, number of errors is also large.

(to be improved)

- Though test efficiency is very high, preparation,

interpretationand errorcorrection must be insufficient,

because there axe stillmany errors.

- There axe many more changes than those of other components.

(diagnosis)

Design or review methodology must be improved. Be

more carefulin testphase. Try to find out the potential

errors based on the test results.More experience and

knowledge are required to do so.

c43 :

It'sa very good component. The only concern isthe percentage

effortof the code phase. Itis true that the difficulty

of this component islow, but both quality and productivity

are high.

cll:

There is a problem in methods for development and management.

(to be improved)

Because of poor design, the code phase coststoo much

and there are many errors.

(diagnosis)

Itisnecessary to improve design phase to be able to make

a better quality document. The test method should also be

reconsidered.

In total,the differencebetween the goals and resultscan be evaluated from Table I.

From the view of COST, only component 5 was well above the standard cost. This component, however, achieved a high qualityrating,so itsproject goal can be considered as achieved.

From the view of QUALITY, four of thirteen components (7,11,46,49) did not realize their

quality target. Error analysis indicates that most errors can be reduced by avoiding careless mistakes. Component 7 has an extra problem. An unusually high number of changes extended the

design phase and caused many errors. Further investigative action should be taken into the

causes of those changes and the manager should be encouraged to minimize change. The quality

target has not been achieved for these four components.

From the view of USAGE OF METHODS, two components (6, 29) had too high a cost in test

phase. They rated satisfactory for cost and quality however. Four component (9, 11, 29, 63)

rated poorly with respect to test efficiency. One component (20) did not meet target in both

14

5642

3-17
categories. There are several problems to be solved in test phase.

All three goals can not always be achieved sufficiently.However, avoiding carelessmistakes

and improving the test method should produce a betterpl'oduct.

In step 5, correctiveaction is taken based on the collected data and managerial graphs from

step 4. For unachleved items in Figs, 4 and 6, the cause of each problem is pursued and an

improvement method is discussed and executed. [f something is found wrong in a certain step,

the activitiesin that step are improved, quickly. In this way, a project can be managed systematically throughout the lifecycle.

The expected effectsof applying SMC are quality improvement, managerial cost reduction,

accumulation and reuse of experience and a highly visiblemanagement reporting system.

7. Evaluation of Software Management Cycle

We are interestedin evaluating and improving the SMC itself.Data collectedat each phase

and after releaseenable us to analyze the effectof the SMC. The followings are the GQM for

evaluation of SMC.

Goal: Evaluate the effectiveness of the SMC

Process Conformance:

QI. How much managerial training was given to the manager?

Q2. How well were the SMC methods applied?

Domain Conformance:

Q3. How well was the SMC procedure understood?

Q4. How well was how to interpret graphs understood?

Cost:

Q5. How many hours were spent to perform SMC?

Effect:

Q6. What was the distributionof the management time?

QT. Were graphs and forms helpful for the manager?

Feedback:

Q8. What changes need to be made in the methodology to

make it more effective?

Q9. What toolsor activitieswould make the use of SMC

more effective?

During development, quality/productivity metrics (set Q), methods metrics (set M) and

5642
feedbackmetrics (set F) axe necessary. Customer satisfactionmetrics (set C) are required after

release.

Table 2 shows the classificationof data.

Four evaluation methods are provided.

(I) How good axe goals?

Based on correlationanalysis between Q, M, F and C, it can be judged that a proiect must be

good ifthe metrics of the project include most of elements of Q, M or F which have high correlation coefficientto C.

(2) How good axe activities(methods, feedback)?

Based on correlation analysis between M, F and C, it must be good activityifan element of

M or F has high correlationcoefficient.

(3) How good are metrics?

Based on regression analysis between C and Q, M, F, the metrics of a project must be good or

predictable ifthe projecthas high regressioncoefficient.

(4) How good are products?

Based on significanttest of the difference between two population (past projects' C and

current C), the current projects'products must be good if the differenceisstatisticallysignificant.

The results of these analyses help to improve Software Management Cycle and update the

knowledge of management database.

8. Conclusion

The concepts and use of Software Management Cycle based on the Quality Improvement

Paradigm are described in this paper. This methodology can improve not only product quality

but also process quality. Three concepts; goal-oriented measurement, process control and visual

management, axe important to manage a project effectively,quantitatively and objectively.

Further plans for the SMC include:

(I) its application to a variety of projects,analyzing the processes and accumulating knowledge

for differentproject classes,and

(2) the development of a fullmanagement support tool which covers the whole process.

The authors are convinced that this methodology contributes to the building of an

appropriate software engineering process for improving both quality and productivity.

5642

16

3-19

(/.?
9. References

[AzSu86} M.Azuma, T.Sunazuka, "Software Quality Measurement and Assurance Technology

(SQMAT)*, Quality, Voi.16, No.I, pp.79-84, January 1986, (in Japanese).

[AzSM87] M.Azuma, T.Sunazuka, K.Minomura, "Software Quality Assessment Criteria and

Measurement Technology', Standardization and Quality Control, Vol.40, No.8, pp.63-75, August

1987, (in Japanese).

[Bas85a] V.R.Basili, "Quantitative Evaluation of Software Engineering Methodology," Proc. of

the First Pan Pacific Computer Conference, Melbourne, Australia, September 1985 [alsoavailable

as Technical Report, TR-1519, Dept. of Computer Science, University of Maryland, College

Park, July 1985].

[Bas85b] V.R.Basili, "Can We Measure Software Technology: Lessons Learned from 8 Years of

Trying, = Proceedings of the Tenth Annual Software Engineering Workshop, NASA Goddard

Space Flight Center, Greenbelt, MD, December 1985.

[BaKa83] V.R.Basili, E.E.Katz, "Metrics of Interest in an Ada Development," Proc. of the IEEE

Computer Society Workshop on Software Engineering Technology Transfer, April 1983, pp.

22-29.

[BaRo87] V.R.Basili H.D.Rombach, "Tailoring the Software Process to Project Goals and

Environments, = Proc. of the Ninth International Conference on Software Engi_ering, Monterey, CA, March 30 - April 2, 1987, pp. 345-357.

[BaRo88] V.R.Basili H.D.Rombach, "The TAME Project: Towards Improvement-Oriented

Software Environments, = IEEE Transactions on Software Engineering, vol. SE-14, no. 6,

June 1988, pp. 758-773.

[BaSe84] V.R.Basili R.W.Selby,Jr., "Data Collection and Analysis in Software Research and

Management, = Proc. of the American Statistical Association and Biomeasure Society Joint Statistical Meetings, Philadelphia, PA, August 13-16, 1984.

[BaSeS51V.R.Basili R.W.Selby,Jr., "Calculation and Use of an Environment's Characteristic

Software Metric Set,= Proceedings of the Eighth International Conference on Software Engineering, London, UK, August 1985.

[BaWe84] V.R.Basili D.M.Weiss, "A Methodology for Collecting Valid Software Engineering

Data, = IEEE Transactions on Software Engineering, vol. SE--10, no.6, November 1984, pp.

728-738.

[BaWeSl] V.R.Basili D.M.Weiss, "Evahiation of a Software Requirements Document by Analysis

of Change Data, = Proceedings of the Fifth International Conference on Software Engineering,

San Diego, USA, March 1981, pp. 314-323.

[BoBL76] B.W.Boehm, J.R.Brown, and M.Lipow, "Quantitative Evaluation of Software Quality,"

Proceedings of the Second Internatiopal Conference on Software Engineering, 1976, pp. 592-605.

[Grad87] R.B.Grady, =Measuring and Managing Software Maintenance", IEEE Software, Vol.4,

No.5, pp.35-45, September 1987.

[GrCa87] R.B.Grady, D.L.Caswell, "SOFTWARE METRICS: Establishing A Company-wide

5642
Program", Prentice-Hall, Englewood Cliffs, N J, 1987.

[McRW77] J.A.McCall, P.K.Richards, G.F.Walters, "Factors in Software Quality", RADC TR77-369, 1977.

[McGa85] F.E.McGarry, "Recent SEL Studies," Proceedings of the Tenth .Annual Software

Engineering Workshop, NASA Goddard Space Flight Center, December 1985.

[Mizu82] Y.Mizuno, "Software Quality Improvement', Proc. 6th compsac 82, 1982.

[Muri80] G.E.Murine, "Applying Software Quality Metrics in the Requirement Analysis Phase of

a Distributive System', Proc. Minnow Brook Conference, 1980.

[RoBa87] H.D.Rombach, V.R.Basili, "A Quantitative Assessment of Software Maintenance: An

Industrial Case Study," Conference on Software Maintenance, Austin, Texas, September 1987, pp

134-144.

[SuAY85J T.Sunazuka, M..A.zuma,N.Yamagishi, "Software Quality Assessment Technology",

Proc. 8th International Conference on Software Engineering, London, U'K, pp.142-148, August

1985.

[WeBa85] D.M.Weiss, V.R.Ba.sili, "Evaluating Software Development by Analysis of Changes:

Some Data from the Software Engineering Laboratory," IEEE Transactions on Software

Engineering, vo[. SE-11, no. 2, February 1985, pp. 157-168.

18

5642

3-21
UMIACS-TR-88-92

CS-TR-2158

December, 1988

Towards A Comprehensive Framework for Reuse:t

A Reuse-Enabling Software Evolution Environment

V. R. Basili and H.D. Rombach

Institute for Advanced Computer Studies

Department of Computer Science

University of Maryland

College Park, MD 20742

ABSTRACT

Reuse of products, processes and knowledge will be the key to enable the

software industry to achieve the dramatic improvement in productivity and quality required to satisfy _he anticipated growing demands. Although experience shows that

certain kinds of reuse can be successful, general success has been elusive. A software

life--cycle technology which allows broad and extensive reuse could provide the means

to achieving the desired order-of-magnitude improvements. This paper motivates and

outlines the scope of a comprehensive framework for understanding, planning, evaluating and motivating reuse practices and the necessary research activities. As a first step

towards such a framework, a reuse-enabling software evolution environment model is

introduced which provides a basis for the effective recording of experience, the generalization and tailoring of experience, the formalization of experience, and the (re-)use

of experience.

'_ Research for this study was supported m part by NASA grant nSG-5123, ONR grant NO0014-87-KA)307 and Airmics grant

DE-AC05--.OR21400 to the University of Maryland.

3-22

5642
TABLE OF CONTENTS:

IlNTRODUCTION .............................................................................................

2 SCOPE OF A COMPREHENSIVE REUSE FRAMEWORK ..............................

3 A REUSE-ENABLING ENVIRONMENT MODEL ...........................................

3.1 Implicit Learning and Reuse .......................................................................

3.2 Explicit Modeling of Learning and Reuse ....................................................

3.2.1 Recording Experience ...........................................................................

3.2.2 Generalizing & Tailoring Existing Experience Prior to it_ Potential

Reuse ..............................................................................................................

3.2.3 Formalizing Existing Experience Prior to its Potential Reuse ................

3.2.4 (Re-) Using Existing Experience ...........................................................

4 TAME: AN INSTANTIATION OF THE REUSE-ENABLING ENVIRONMENT MODEL ..............................................................................................

5 CONCLUSIONS ...............................................................................................

6 ACKNOWLEDGEMENTS ................................................................................

7 REFERENCES .................................................................................................

7 •

8

10

11

12

15

16

17

20

21

21

5642

3 -23
ORIGINAL PAGE IS

OF POOR QIJALIT¥

1. INTRODUCTION

The existing gap between the demand and our ability to produce high quality _of_war,,

co,t-effectively calls for improved soft, ware Iife-,:ycte technology A reuse-enabling -:of_war,. [ff,.

cycle technology is expected to contribute significantly to higher quality and productivit), ()u:itit_ ,.:an be expected to improve t)v reusing proven oxpermnce m the form ,,f pr(_,luct_, t,r,_,.,.>-.-

and knowledge. Productivzty can be ,'xpected to increase by ubing ,_xlstmg _.xp,.r>n,:,. rath,.r "i_:trl

developing it from scratch whenever needed.

Reusing existing experience is the key to progress in any area. Without reusv o.v,_ryrMn_

must be re-learned and re-,'reated: progress in :m ,,r'onomi,'al fa..shion is unlikely l)ur_n_ _h,

evolution _f software, we routinely reu.s_ ,-xp,,rl_.n,'e irl th, _ f,)rtn of ,'xi_tmg pro Jm't. .... z -.',._v.r:

.\do ,:omponents. design documents, nlathenl:_t_c:,l aubrout.lnesl, proc,_sses (-_. ,Je>.lg[1 irl-g_,.,'ri,,r_-

methods, compiler tools), and domam-specitic km)wi,:dge I,'g. cost models les_ons h'arn,.,i u,.,:_.--

urement data). Most reuse occurs implicitly m an ad-hoc fa.sMon rather than a.s lhe r,_-ult _.i_

explicit planning and support. While reuse is tes._ Institutionalized in software en_meerin_ _h:m _n

other engineering disciplines, there exist some "qlCCeb>iflll <a_es Of reuse te prr)(Juct r*m_e R,.u-,. :rt

_oftware engineering has been suceesbful whenev,,r the r,,u_.ed ,:xperienc- j_, -_qf--,t-<,'r_i,d.: , ,

mathematical subroutines, or the stabihtv ,)1 _h_ cont,_xt m wh2ch t.he -xp,'r_ence t-- r,'u>,-[ ,mLpensates for _he lack of self-description. ,. g.. reube of high-h.v,d ,leslgns :,cro>._ projo,:t. _vtth -Jnilfar characteristzcs regarding the application _h)maln. the ,h>_gn met.hods. :rod the p,,:'.c,rm,,[ [u

_oftware engineering, t,he potential productivity p:_v.,)ff fronl r,,uso ,:an b,. ,luite high -irl,',. Ir /"

inexpensive to store and reproduce _ot'tware englrleerl[_g expertence compared _.() oth,:r ,qlgllp.-rIng disciplines.

The goal of research in the area of reuse is t.he ach_evem,mt ()f systematic methods f_r _ffectivety reusing existing experience to maxinnze quality and ,:o.-,t benefits. Nuccessful reu>e ,b'l-m,l>

on the eharact.4ri_tics of the candidate reuse objects, the char:'_cteristics of the r,.us,, proc.>_

• The :_rm "_vrn!lLt,_rl" la _JSert _11 t, hls paper _,o ,'I)mprls_ l.h_ ,'ntlr_' ,_,ft',_'aro ]IF,' ,'y,'l _ (,l_v_lOprll,qlL t_d _IItl_tr, erl:_.:t,';

5642

2

3-24
O_._Ct_!AL P_GE 15

OF POOR QUALITY

itself, and the technical and managerial environment in which reuse takes place Int,:r_st _;l

reusability has re-emerged during the last couple of years 4. 9, II. 1 '_, la. 14. 15. 16. 17. l!J.

20, 211, due in part to the stimulus provided by Ada and in part to our mcre_ed understandin_

of the relation between software processes and products.

Our increased understanding tells i,s that m order to improve ,luality and produ,;_tvtl} '.1:t

reuse we need a framework which allows (a} the reuse of all kinds of softwar,. ,mgl_p,'rln_ ..x!,.r:-

ence. i.e., products, processes and knowledge, (b) the better understanding _)t' ril,. r,'u.-, :_r_'),,.>,

itself, and (c) the better understanding of the technical and managerial evolution environment _fl

which reuse is expected to be enabled.

This paper presents a reuse-,mab[ing software ew_lutmn ,mvtronment rn,_,i,_[. :_,- :ir-_ -r.:

towards a comprehensive framework for understanding, planning. ,evahJatln_ _[>t [tl(_r:',:LTlrt_"

reuse practices and the necessary' research activities Section :2 motivates the rlec**e_ar} -,'_,!_,-,H" ,

comprehensive reuse framework and the important role of a reuse-enabling software ,_volu_l_ll

environment model within such a framework. Section .3 introduces the reuse-enab[m_ <_>l.'_ar,.

evolution environment model and ,iiscusses its ability to explicitly' model the r-cordln_ ,_f,.×p_r_-

ence. the generalization and tailoring of experience, the formalizatton of ,.xper_ep.,',.. _m,I T:_...>-

_lse of experience. The TAME model :t _pecitic instanuation of the rmlse-enablin_ -,_tr,,_:Jr....'..,-

]ution environment model, is presented m Section .t This specific mstantiatloh > ,i-,.,t ,_, :i>,_.

specifically describe the ,nt,:grat_on of r.h, recordin_ and (re-tuse :_ctlvities inr¢_ m trnl,r_'.-:_i,':_v

oriented software evolution pro_'ess.

Before we proceed, we detine some crucial terms that will be used in this paper _o the r<_d,'r

understands what we mean by them in the software context. We have tailored \Vebsters e,,m,:r:d

definitions of these terms to the specific domain of software evolution. [mpro,,ezz_ent m<m.,

enhancing a software process or product with r#spect to quality and productivity [.earr_l >. the.

activity of acquiring experience by instruction (e.g., construction) or study (eg.. :malysis} [_'e;zse

is the activity of repeatedly _lsirlg Pxh>ting ,:xperience. :_fter reclailnmg t_1 with ,r wl*h_,ul

5642
ORIGINAL PAGE IS

OF POOR QUALITY

modificatton. Feedback means returning Lo the entry point, of some process a.rmed '.v_t.h d_,e

experience created during prior executions of the process. \Ve use the expression ezpe. r,ence base,

to mean a repository containing all kinds of experience .-\n ,xperience base can be implemented

m a variety of ways depending on the type of experience stored. An experience b,_se may cons>l

or one or more of ehe following: traditional datab,'_es containing factu;d piece_, of mi'orlnation.

informauon bases containing structured inforniacion, and knowledge hazes including mechanism._

for ,teducing new informauon :5, 247.

2. SCOPE OF A COMPREHENSIVE REUSE FRAMEWORK

Reuse in most environments is implicit :rod ;ut ho,: \Vh,:n it > -xi.Ii,.tt ,Jr i..l:tna_,i, t_

predominantly deals wit, h the reuse of code. [n -qegtlOll I. '.YO ,_xpre:->e,] OilF i,ciic_" _h:LI "['t'*'c_IV'*

reuse technology needs to be based on (a) the reuse of products, proces.-es :m,] knowle,ie,.. {b} :l

good understanding of t,he reuse process itself, and (c) a good understanding of the reu.-,e *'nablin_

software evolution environment.

To better justify these beliefs, we will descrl_e amd discus5 th,, r,eu_*, pra,'tw,. :II ri,,.

Software Engineering Laboratory (SEt) at N.-\S:\ (;od,h_r,i 5pace F!i_hl. ( ,.nr,.r _. 1" "I'hls :,

:m _xample ;vhere reuse has been qmt_e auccessl'ul ,tt ;t vg.rler'_ ,)f l_,_.,.i:- ;tit_,H! pr-,ivm,m:m_]}

implicit, Ground support soft.ware for sar_ellites has be,,n ,i,!x,,l,_pe,I t'{,r a aumb_'r ,_f _,,:trs ill

FOI{TR.\N _eused experience exists in the people rn,,rho_is, and i_,(_l> :L, ,.veil :L< in _h,' pr_r:_m

library and measurement database.

To explain reuse in t,his environment we must, tirst explain the management structure.

There are t,wo levels of management revolved m the technical project management The second

level managers tone from NASA and one from Computer ."ciencea ( 'orp(watmn. r.he ,'out, r:).ctor).

hay,. been managing t, his class of projects for several years .'5.'peeific proj,,ct, managers ar,' Lvp,-

,'ally promoted from within the ranks, on ,'ither side. from the b-tt,.r ,t,,v,.h)p,.r> on prior proj_!cl>

5642
This provides a conunual learning experience for the management tealn. "r,.chnioaI r_.vi_w 0.rid

discussion is informal but commonplace. Lessons learned from experience are used to irnprow

management's ability to monitor and control project developments.

The organizational structure has been relatively constant, from project r,_ proj,.ct Th,.r,.

have been minor variations due to m_provements m such things :us method-s :tnd rool_ .vhici, !_a_,.

evolved from experience or been motivated the literature and verified hv ,.xp-rilnont:tl ,tar:_

analysis on prior projects.

The basic systems have been relatively constant. This permits reuse of th- :_pptica[ion

knowledge as well as the requirements, and design. For example the requirement. ,tc.-,_m,,nr_ ,r,.

qmte mixed with regard to the lev*q of specificity. In some places they _re ,tulre pr-,'l>," i,u_ _r_

other cases the arc very incomplete, tel3 ing on the experience of the p,_ople from prior _,r,_!,.,'_>

Requirements documents have phrases similar to th,. following: Capability X l'_,r ne_ .at<-

lite S2 is similar to capabilit, y X for satellite S1 except for the following. This implicitly provides reuse of prior requirements documents .as well :1.5 implicitly allows for r_us,, of prior ,h.s*_n

,tocument.s and code.

Systems within a ,:la_s, all have a sirmtar design at _J_e top tev,'l and the _t_t.rt':w.< _ni_,u'_

subsystems are relatively wall defined :rod _end _o be relat_veh" error free. l)esi_n ts tm[dt,':r[._

reused from system to system as _pecified by _he experienced high b_v,q managers

_euse at the code level is mor_. expliciL. The _oftwar," development proce_-s used _.- :* r._,'s,"

oriented version of the waterfall model The coding ph:mo: heg,ns by' seeding the ,'ode library with

the appropriately specified elements from the appropriate prior pro.lects. These code components

are then examined for their ability to be reused Some are used as is. others modilied minimally.

others modified extensively, and yet others are eliminated and judged easier to ,Dvelop from

scratch. This is a reuse approach that has evolv,.d ow_r time and h_s been quite effective

A variety of tools have evolve, l that are qui_'e application spe?ilh- Th,'se iuctud,. ,.v,'ry_lnn_

from tools _hat generate ,li>pla.vs n,:,',t,:d for I,,stm< h_ :tl>plica¢lotl -p,.cili," -_.-I,.lll *l[illtl,,:_

5642
ORIGINAL PAGE w_.

=

OF POOR QUALITY

knowledge :_bot, t these _ools has been disseminated by guidance from more :enlor member.-. _)t' rh.

developmen_ t_am,

The SEL environment ]s a good example of strong reuse at a variety of lev,:l_. _n a variet_

of ways a_ part of the software dewlopment process. There has been a pattern of [<trntng and

r,_using kno_vled<e, processes and products. The use of the me_uremenr ,t:ttabase h:_.- il,q!J-,i

w_th project control and schedule _ well :_s quality ,_-ssessment and produetivir_ 2. t"

NASA is now considering changing to Ada. Several Ada projects have :_lr_ad._ b.*m ,:_..mpleted. This has involved an obvious loss in the reuse heritage at the e_ode lewl, _- wa_. :_nr_,_-

Dated t_;ut lr h:_s :d..o inv¢_lved a less obvious and unexpected loss of reu>. :it _'h. r-,l.,r,,rn,.r_r=

:lnd design h.wj, _n the (_rg:*mzatlonal structure, and even in the rtpplica_zon kn(,_ h:,i<,. ,tr,._

The mltiai Lmpact of .\da w_ staggering because of the implicit, r:lrh,:r rr_.. -._r,il,ir

understanding of reuse m the envtronment_. This understanding of reuse need_ r,, _,. i'_,r'a_t:tlt,..,:,i

Based upon the concept that reuse is more than just reuse of code and _hat tt u,,_d> t() },,,

explicitly modeled, _v,. need _o reconsider how we measure progress m reu._. Th- m,_._.-,.,,m<_r.

,:urrentl_ used m the qgI. :tr_ ba.sed upon lines of code reused from one proj_:r_ I(_ u_ i_,:_ _,a'..._

th_s view progre.-> rna_ ilo{ !_,. relaLed at all to the lines of code reus_:d \V,, tl,:,,¢J ",) :_,:t_-_;',. "_,.

_ffects of reuse (m _he resources .xpend,'d m the entire software life ,-veb_ m,I ,,n _i., !u.h_?. ,l'

the products produced u>_ng an ,:xptic_t reuse oriellted evohl{.ion lllode_ [II ,r:l_r r}l,, _,r,'.,,--

>tlOlll_J allow ils IneLkbtlre f_or i_._ly ._et i)f reuae-related goal_ :';, I. '_. t0, C'h:mgm_, ,_ur .>d,.l- md

our metrics ',viii help _s to better understand the effects of _he _radltlonal r.use pr:tct_,.,,> :tn,i

compare them with t.he effects of an explicit, reuse oriented reuse model.

In summary, we be[ieve tha_,acomprehensive reuse framework needs to include ra) _ i'_ms.

enabling software evolution environment model, [b) detailed models of reus_ :).lid I<xrn_ng, :tll,l {,'}

characterization schemes for r.use and learning based upon these models.

5642

3-28
_I:-_(_].!,_.L ̧ r.__,-_

C_,_rJ__'" *".... i_',*

3. A REUSE-ENABLING ENVIRONMENT MODEL

In the past,, reuse ha.s been discussed Independent of the software evolution environment

We believe reuse can only be an effective mechanism if [t is viewed a.s an integral par¢

paired with [earning, _Jf a reuse-enabIillg -;oftw3.ro ev¢-_llltion environment. Nnne ,_t" :i>

_radit,;onal ,engineering disciplines has ,.ver mtroduc,,d the retlse of t)l.li d ng blocks :_ tnde[..n-

,tent of t.he respect`ire building process. ["or example, m civil <_gineering people }_av,. :1_)_

created "reuse libraries" containing building blocks of all shapes and struct,ures, and then tried

to use them to build bridges, town houses, high-rises and ,'ot,ta_es [nstead, they ,te'.'ise,t t

_tandard r,,,:hnolo_y for huIl,tina,'ert:_ln ryf ..... ,f t.,llldln,_'>,,.g r(,_n h_,u>,,s! rhr_,gh t:_rl_ ;r_,-

,:ess of tmderstanding and learning This allo_ved them ,o t,.rin,: rh,_ needs for .,,rt:t,n -tan,iar, i

building blocks at well-defined stages ,}t :hmr ,:onstru,:t_on proc,,s_ [n _h,: -_flwar. :Lr,.n;_ .a,.

have not. followed this approach..

[f we accept the premise that effective reuse reqmres a good understandin_ ,)f t,h,. ,_nvlrr)nment in which it is expected to take place. _h_m we inubt model retlse in _h," cont,:xL ,;1_ a rml.-,,

-nabling -oftware -vohztiOll onvlronment Such a cont_,,xt will :flh>w us _,_ learn h,_ r,, r,':>,

hotter Th,' ,_ttlmat,? ,_xpectation is that, such irnpr(_v,,nl_,nt ,aoul,] h.a,t r_),.n -v,.r tn,'r,,:tet:__"

qsage of generator technology during snftwar,, ,,votut_on "l'h. :tbiJit_ to :tut,_[n:tt,, rh,, _en,.r:,tl,,n

_,t" products from other products r_ttects t_he ultimate ,t,._r .... ,,1" und,,rstandin_ th,. ,_n,b.rtyin_ ,,t>

StlUCtlOn processes Automated processes are ea.bv to r,,u..e l"<,r _.xalnpJr:. in iHllldiIl_ .'()[IID]L,'F

front-,:nds, we rarely reuse components of other ,:ontplh.rs: in_,tead, w,, r-use the ,'_>lnpLJer g_q/eratots which automate the ent,ire process of building compiler front,-,mds from formal language

specifications

In _ect,ion 31 we discuss how learning and r.us_. _mplicitly o,:cur m the ,'OrlIeXt r'H" tr:l, Jitional software ,_volution environment,s. In Section 3:2. we discuss how l,.arn_n< :u_d rm_s,, .':m h,.

,_xplictt, ly modeled in the ,.'ontext of a rouse-,mablin_ software ,_volulion .nvlr_mm,:nt

5642
ORIGINAL PAGE IS

OF POOR QUALITY

3.1. Implicit Learning and Reuse

During a workshop on "Requirements for .'5of_ware Development Environments"

held at t,he University of M:tryland in 1.q85, a view of a software evolutaon environment ',_:t,_

proposed that consist, ed of an informa_,)n ,ystem :_nd ¢hr,e informauon producers md ,'on>umers: people, inethod_, anti '()()Is :22 Th,. inf,_rmatlon system is ,telined b_ :_ 5_ftwar,, ,vo[urton process model describing t,he reformation the .:omnluntcatlon 3.F/IOllg [)eOp],*. lli,![!l(,,l>

and tools, and the activit, y sequence_ For ,teveloping and maintaining software

The t,raditional software ,_w_lution ,environment model in FigurP 1 is :_ refinemen_ _)f ri_ls

<lrlier model

l ]methods ! I

people I tools

"1' 'if "IT

Software Evolution Process

A

Y

- products

- management, plans

- schedules

- project data

-- o°.

°°.

PROJECT DA:TABASE

Figure 1: Traditional (non-reuse oriented) Software Evolution Environment Model

5642
C<';. - __,.__",_-, q;,;2,LITY

The purpose of the software evolution process is to produce output prodm_'_ - g.. ,iesign

documents, code, from input products, eg., requirement documents. P_'ople exe,'ute this proce_

manually or by utilizing available methods and tools. These methods and tools can t)e under tile

control of a project database. A'II or part of the infornla.tioll produced ,turin_ this process Ls

stored in a pro lect ,iatabase. e.g.. products, phms such ;ks managelnen_ plans ,_r -,'h,.,illl,_s pr,,-

ject data.

Typically. support for such a traditional software evolution ,.nvironment mo, i,:t includes :_

project database and means for the interaction of people w_th methods, tools, and the proj-ct

,]ataba_e during software evolution. The experience of people. :ks _41 '_-, -_In," _f :h- m,vho, ts

and tools. _s usually not controlled bv the project database .\_ a ,'on_,-,1u,m,'-. the> ,.,:!,,:rL,me,: _-

r_ot owned bv the organization (via the project databa-se) but rath,.r _,_vu,.d i_ tll,il','l,i_*:L_

human beings and lost entirely after the project has been ,'omple_ed

Although the ideas of learning and reuse are not explicitly' rpflectcd in the traditionat

software evolution environment model, they do exist implicitly. The exp,,rienc,' ,_f the t>,',)!,l,'

Involved in the software evolution process and the experience encoded _n rneth_,t- :m,l moi> >

reused. In many cases, prevmusly developed products _re reused :ks in!>u_ produc_.- [n :}>' -:_m,'

way. products developed during one actw_ty of the evohltion proce>s ,:nn },,, :-It>,.,i tn >uil-,.-

quent activities ,)f this same proc,:ss. People learn {gain exper_em-'ei from ;,,'rt',_rr=llIle _i_- :tct_x!-

ties of the evolution process. Another form of implicit learning oc,.urs wll_.r_,._-r pr_,iu,'t,, pint,>

or project data are stored in the project ,tataba.se

The basic problem in this traditional environment model _s not that learmn_ and feud,'

can not occur, but that learning and reuse are not explicitly supported and only because _f _ndividuat efforts or by accident.

5642

3-31
ORIGINAL PAGE tS

OF POOR QUALITY

3.2. Explicit Modeling of Learning and Reuse

Systemat, ic improvement of software evolution pratt.ices requires a reuse-enabling environment model which explicitly models learning, reu-_e and feedback activities, and integrate_ th,m

into tile software evoiut, lon _rocess. Figure '2 +]eplcts :-,uch ;_. reuse ..nabling ,'n'.'lronment mo, i,q

FB

Software Evolution Process

informal

-_Fj3__ --'

schematized _roductized

PROJEC r SPECIFIC

DOMAI '_ SPECIFIC

GE ERAI."

EXPERIENCE BASE

v

R

FB

++

Figure 2: Reuse-Enabling Software Evolution Environment .Model

All the potentially reusable experience, inch.ding softwart, ,vohmon Inrthods and Tools are

und,:r t.h+; control of an experience b_e. Improvement i.', b:u,_e<l on the f,','.th:u'k ()f,.xlst,ng,.xperi-

-no,. [labeled with "FB" for reuse in Figure 2). [&edbztck r,_(ttnr,:_ learnimz, :rod r,,us+. _ysu.matt,.

le:_.rnlng r,.,t,nr_._ '_upport I'or r,he rerordin_ (ff ,'xp.rwnc. ([:d..led with "[_" l'()r r,','-r,tin_z m [:'i_ur5642
• - , - ..j . ,J

O_" t'C __-_, ""......_......

2), the off-line generalizing or tailoring of experience (labeled with "G" :rod "T" i'_r gen,rah;,-

ing and tailoring in Figure 2), and the formalizing of experience [labeled with "F" for formaiizlng

in Figure 2), Off-line generalization is concerned with movement of experienc_ fr,_n_ proj,,ct

specific to domain-specific and general: off-line tailoring is concerned with mov,.m,mt ,_I" -xp,,rlence from general to domain-_.pecilic and proj,,ct-_p,ecilic Off-line f,)rmalizatl_m _ ,'_m,',,rlk,.,i

with movement of experience from informal to schemauzed and productized >v>_,.rua_*," >',>,"

requires support, for (re-)u_ing ex,sting ,experience (label,:d with "U" for u_e :n I:i_ir,. 2a, on,:

1,

on-line generalizing or tailoring of candidate ,experience (not explicitly rettecte,t in [:ieur. '2.

because it is assumed to be an integral [)art of the (re-)use activity'). .

Although reuse :rod l,'arntng are pos_qbht in both the reuse-enabling m,l r:_,. rF:t,iiT._:l_t

environment models, there are significant ,tifference5 _n the way experience _.. ,,_,,_,,,,t L.,i :L,,_

learning and reuse are ,_xpliciti.v integrated and supported The b:mic ,lifl,w,,nc,, i,,,T,a,,,r_ :_:,,

reuse-enabling model and the tradit.ionai model is that learning and reuse h,.com,:-×!HL,qlt':

modeled and are desired characteristics of software evolution

3.2.1. Recording Experience

The objective of recording experi.nce Is to cr,.'a_e a repository of '.v,.[l sp,-clli,,,l tn,l ,:r_::nized experience. This requlre_s a precise de_,:riptioll of Lhe _xperlence ro be reccw+.,I rh,. ,i_,>l<_t

and implementation of a ,:omprehensiv,- exp,'rwnc,' be.s,:, md ,_ffective rnechani>ms ',,_r ,,it ..... t::<

validating, storing and retri,:ving experience. \V,, r,el)lac,, the project datab:>e ,,f rh. _r:_,iiI_m:_k

environment rnodeJ by an the more compeehe,_s_ve concept of an experience t):Lqe' '¢¢}11,'h ',>

intended to capture the entire body of experience recorded during the planning and ,'xecutton {_(

all software projects within an organization. All information ttows between the soflwar,, ev,Hution process and the experience b:Lse rell_.:cting the recording of experience are [ab°ded with "f_" _n

Figure 2.

• The at.tributes '¢)n-line • an,{ "c, ff-[in_" in,_i,:at," whorh_r _ho ,'_rr,'_pon,hnK :tcLtVlLicS :tr_ _-rf,_rm_d _-, : trt ,,r r>*.'>,"l

dent of any pa.rI.icular_rHt,,,car. -vt_iilckorl i,re_j_el

5642
ORIGINAL P_!GV fS

OF POOR QUALITY

Examples of recording experience include such acuvlu,:.-, as (al storing of :_ppropri:_t,dy

documented. ,:atalogued and categorized code components from prior _?'s_ems in a product

library, (b) cataloguing of a set of lessons learned in applying a new t`echnology in _ knowie,ig.

ba._e, or (c) capturing of measurement data related to the cost of developin_ :l system m a In_:Lsurement d at, aba.se.

In the SEL example of Section :2, code from prior _ysrems is _v:ulai)l- ,_o rh,, i_r._)_rL[ii

library of the current project although no code object repository h_ been ,tev.lope,i .\b_:u;ur,_-

ment data characterizing a broad number of project aspects such az the project environm.,nt.

methods and tools used, deiects encountered, and resourc_:s sp.nt at. _xplicirly _,_or-,! _rl rtl,. Ft-I.

measurement databa__. 2. " I_ Re_lulre[neIIt, S alia tiest_n docuFneii/5 ;k_ _,L,_'[i t> l"'>*_[l- :,':It':L,',;

about the technical and managerial lmpiicat.ions ,)f various me_hods m,i rooi_ :Lr_ ,,m[,i!,'!_:<.

stored in humans or on paper

Today It Is possible, but not common, to find product, libraries. It is even le._s ,'ornmon _,

record process-relat_ed experience such :_ process plans or data which charact-rlze th. lnl[_m', ,_f

certain methods and Tools within an organizaUon TherP ,.x]st_ two mare r,,a-_on_ ,,_hv ,,_. _,,,.,i _,_

record more process-r_'tated ,,xperktmce: iaj _t is _eneraJty h:trd ro mo,ii(', ,'x_:qm,: !,[_,,',,a, refficiently without` any knowledge regarding the processes accordma ro whi,:i_ "h,:x t_,'r',' • _",1[[r , ai :

and (bl the-ffectl;'- r,ulS,, of process related ,'xperience _uch _._ process plans ,_r t:u._ ,'_u_,l _ i'llvide si_niiicantly rnor_. I,'verage for irnprow_mont than just the r-ils_, of product>

3.2.2. Generalizing & Tailoring Existing Experience Prior to its Potential Reuse

The objective of generalizing existing experience prior to its reuse is to make a ,'andidat_,

reuse object, useful in a larger set of pot, enttat target, applications. The objective of t,%lJorlll_ ,.xlsting experience prior to its [_otentlal reuse _s to fine tune a eandh]ate r_,us_ • ,)}_je,.t to lit ;i -p,.r_Ii,.

task or exhibit, spe_iat :urribut,..... *uch as size or p.rformano: These activm.s r,,,lutr,, _ ,,,ql

documented cataiog,:,t :rod ,:at,,gor_z,.,t _,,t ,_f rt'tlY_' o['_J,'ct,'-i, m_ch:mi._m_ thal -uppt,rr rh,,

5642

- 12-

3-34
_,.--=_ n_ ' ̧ |'- _ ,_-._ -

modification process, and an understanding of the potential t_arget application.> (;eneraiizatton

and tailoring are specifically concerned with movement across the boundaries of the "generalit.v"

dimension: from genera[ to domain-specific and project-specific and vice versa Objectiv_'s and

characteristics are different from project to project, and even more so from environment to

onvlronment. We cannot reuse pa.sr experience without mo, iif','ln< _t to rh_, ne,',is of _he ,'urr,.n_

project. Thestabili_y of t.he environment m which reuse takes place. :m '.veil :_-s rhe_>rl_lnat_oh ,,l"

the experience, determine the amount of tailoring required.

Examples of generalizing and tailoring experience include such activities a.s [a) ,]eveloptng :t

generic package from a specific package. {bl mstantiatin_ a _enert_" packag,, for a yp,.,'iti,: tSp,:. ".,

'generalizing lessons learned from a specilic ,tesign t,.chnoh)gy for :t spocilic :,pplicatlon t_, til_

,tesign for that application or any application. {,i) or parame_er_zmg a ,:o>t model t',_r L _[>-,.u{1,"

,n_ ironment.

[n the SEL. requirements and design documents have implicitly evolved to be applicable t,_

all FORTR._MN projects in the ground support software domain Nlea_ur_'ment ,tata tlav,, b,'-n

explicitly generalized into domain-specific ba.setines r-_ardine ,h.f,.cts and r_'s,mrce-xpon,iilur,,-

2. _.. 1_ Requirements and designs are impli,utly rallor.d towards the n,'_, is _,f a :>._ l,r,_j,..'r

b_ed on r.he manager's experience, and code *s ,.xplicitly hand-moditi,.d rn rh,, ne,-_is ,_i' .t n,.,a

project.

In general, recorded ,'xperience is proj,,ct >p,',uli,' hi ,_rd,.r ro r,'u>,: thi:, ,.xp,:rl,:nc_ tn :_

f_t_lre project within the same application domain, we h:tv,. _,_ if;t} generahzc the r,'ce_rri,.'_t pr,)jeet

specific experience into domain specific or general experience and (bl _hen tailor it again to ri_e

specific characteristics of the new project. We distinguish between off line and on-line generaliz-

_ng a.nd roadoring activities:

• Off-llne generalizing and 'tailoring is concerned w_th lncr_.:_ng the reus, _ pol'elltiaJ of oxl._ting proc,:ss and product-related experience before knowing the preci._,- reuse' ,'ont.ext (it. the

[;,rvqect wlthill which the ,-xp,,rle[ICl! IS I.,elllg r-Ii:_e,t) _)l'f lin," _,.I_,.r:,.hz:ttt_m .tn,t r:uh_rin< I_

-13-

3-35

5642
OP2G_NAL PAG_ IS

OF POOR QUALITY

concerned with movement across the boundaries of the specificity dimension wlthv_ rile ,_xp.r>

ence base: from general to domain-specific and then to project-specific, and visa versa. Th.s,"

activities are labeled with "G" and "T" in Figure 2. An example of off-line gener:thzatJ_2rl Js

the construction of baselines. The idea is t,o use project-specific measurement data ¢- _. F:m[T

profiles across development phasesi of several projeet.s within some application ,t,>m:L4f_ tn,i r,_

create the applicauon-domain specdic fault, profile baseline, gach new proj-,-t wlrhHt ri>. -:lNl,=

_tpplica_ion domain might reuse this baseline in order t.o control its deve[optnent [,r,>,--.- a.- :':tr

faults are concerned. An example of off-line tailoring is the adaptation ,,.f :_ g,_n,_raJ

sc,.entific paradigm such as "divide and conquer" to the softwar_ engineering ,i_mlalr,

• On-llne tailoring and generalizing i._ concerned with tailorin_ candidale !:r'>',..- _t,:

product-related experience re the specific needs :!.IlK_ ,-'haracterist.lcs of a proj,_,'l :tn,t ;n,. :; ..... .:;

,of[ware evolution environment These acuvlties are not explicitly reHect,.(I in ITi_lr, , 2 !_t.,.::,j-,.

they are integral part of the (re-)use activity An example of on-line _atloring is rh,, :ld:q,_a-

_.ion of a design inspection meehod re better detect the fault types anticipated _n r_,_ "u_r-:',t

project 6 _ .-kn example of on-line generalization Is _he inclusion of project _p.ciF, c -fforr .i:_i _

i'rom a pa._t project ip.to the ,Iomain >p,',:ific effort bz>,qine in ,_rdvr _,_ 5,,rt,,r pian :b,: !','.!_*zF,._

r,_sourees for the current prnject ()bvioll.-.lv [hls ktnd of generahzati(m ,v_,id h:t_ ._ i,.,.:_ ,,-rformed off-line too.

h is important to find a cost-effect[v,, balance" ber_-en off line :_nd ,m line 'alh_rinc r_.;

generalization. It can be expected that generalization is predominantly performed _,ff-tlrle. r_ul_rmg on-line.

A good developer is capabte of informally tailoring general and domain specific _xp+,r|,..nce

to [.he specific needs of his or her proieet. Performing these transformations on ,!xisting .Xl,,.rl-

_nc," assumes the ability to generalize experience to a broader context _han tfi. ,me *tudied.

_)r re _ailor experience to a speeilic proJeCt Th, _ }*etter Tills ,:xperience I.- pack:tged, rh,. _,,Ir,,r

,,,_r un,t-rstanding (_f the m'lvirorlment \lalI_l:uI_mtl :t [)odv of ,,x[),.rlrltC!. :t,'r{tlir,"_t .Jttrlttff i

5642
number of pro?_cts is one of tile prerequisites for learning and feedback across projects

A misunderstanding of the importance of tailoring exists in many organizations These

organizations have specific development guidebooks which are of limited value because t, he.v "are

written for some Meal project" which "has noihing in common with the current project and

therefore, ,to not apply" 23: All guidebooks (including standards such :_ D()D-<TI)-21_LI xr,,

general and need to be tailored to each project; m order to be effective.

3.2.3. Formalizing Existing Experience Prior to its Potential Reuse

The object:re of formalizing Pxisting experience prior to its potential reu_ _- t,, tncro:t>_, rh,,

reuse potential of a ,'andidate reuse obiect by encoding it in more precise, b_tr.r ,_mi,.r-_o,,,t ,._:r.-

This requires models of the various reuse objects, notations for rnaking the model> m<,r,, pr,-u-,,

notations for :_bstractmg reuse object characteristics, mechanisms for validaung rhes,' mo, ic]> _u,i

mechanisms for interpreting models in the appropriate context. Formalization acuvlties ar_ ,'¢mcerned with movement across the boundaries of the formality dimension within th,' ,_xp,_rt.nc ,'

b_e: From informal to schematized and then to productized. These ;_ctivitles are '.:tb,q,-,_ ,,_:th

"F" in Figure :2.

Examples of formalizing experience include such acuvitteb :t_ !al ,,vrirm_ {',,n,'r,_m_.i

specifications for a ,:od_ _ module. (hi turning a h:ssons learned document into :t m:tna_,,rn,,rll .v--

tern that supports decision making, {,:) budding ',t ,'ost model empirically t,;k_,,d IBf:,,m rh,. _J:_:_

available, {,1) developing evaluatmn criteria for _valuating the performance )f t [,:lrtl,'uk:n"

method, or (e) automating methods into tools.

[n the $EL, measurement dat, a have been explicitly formalized into cost models 1 and -rror

models enabling the better planning and control of software projects with r_gard to ,-os_ o_,ttln:ltion and the effectiveness of fault detection and isolation methods _2, 6, _. lS l.,>>on> J,,:_rn,,,t

have been integrated into expert systems aimed at supporting the managemen_ ,h,c>i,m proc.s.

5, 2t

5642

- I5-

3-37
OF FoOR QUALITY

The more we can formalize' experience, the better _t _;an be reused. Therefore. we try not

only to record experience, but over time to forrnalize experience from entirely informal (e.g.. concepts), to structured or schematized [e.g., methods), or even to completely formal {e.g., tools)

The potential for misunderstanding or misinterpretation decreases as experience is described more

formally To the same degree the experience can h,_ modilied more ea_ih', or in the ,:a_,_ of

processes, it may be executed automat, icaily (eg.. tool._) rather than manually (e.g. methodsL

3.2.4. (Re-) Using Existing Experience

The objective of reusing exisUng _xperlence zs ro maximize the effectivo use of p.roviou.-ly

recorded experience ,luring the planmng an,i execlltion ,>{" :1[1 projects within :m oraamzation

Th,s requires a precme characterizatmn of the avadable candidate rPuse objects, a precise characterization of the reuse-enabling environment tnehutin_ rip: evolution process that t_ .xpect,_d i_

enable reuse, and mechanisms that support the reuse of experlence We must support the {re-}use

of exisung experience during the specification of reuse needs in order to compare them with

,Jescriptmns of existing experience, the identification and understanding of candidate, the evalu._-

tion 0f ,'andidate reuse objects, the possible tailoring ,)r the r,_us_ c)i)J_'rt, the integra[Ion _f ,'i>.

r_ll-_e obj_'ct Into the ongoing _oftware project, an,l the -valuating, of the proj,.ct > "/IrCC?:S .\[l

reformation ilows between the experience base and the <,fl'rwarP ,.vojUtl,)n pro,',,ss r-tte,:_in_ rh_

{re-)use of experience are labeled with "[.'" in lPigur, :2

Examples of reusing experience include such ;tcllVllleS :t.- (el.} using co, te components from

the repository. (b) developing a risk management plan b:med upon the lessons learned from applylag a new technology, it) estimating the cost of a project based on data collected from past projects, or (d) using a development method created for a prior project.

In the SEL. reuse needs are informally specified :_s part o1' the r_'qmr,'men{s ,io,-unl,nt

},latching candidate requirements and design documents are idenufied bv managers who :trP

,'xp,'ri-n,-,:,t tn _his ,'nvlremrm.nt The ,waluatmn ,_1"rhos,, ,'andul:tte r,'us,, oI,j,.,'L, 1> tn part h:L,,.,I

5642
on human experience and Ln part ,_n measurement data. They are tailored b;L<,:,J on [ht

application-domain knowledge of the-personnel. They are integrated rote a very stable evolution

process based on human experience. All this reuse is implicit except for the reuse of ,:ode. which

although explicit, is informal It could only be successful because it evolved within a very stabie

environment The recent change from FORTRAN to ..\da h_s resulted In ,ir:_tic ,:hai_._ .... _l" _hl>

environment and as a consequence to lh,:loss m the implicit reuse heritage

Since the key for improvement of products is always improvement of the pro,>!_> ,:r,.atln_

those products, we need to put equal emphasis on the reuse of product and process oriented

experience. Ewn today, we have examples of reuse of process experlen,:, such c. prefer.-.

plans {standards such :m D()D-STI)-2Ib7. managemenl p[:ms..chedul,'s)or .uroc,.>> i:_t:_ ,.rrc, r

effort or reliability data that ,telin," b:_,qines regarding software ew_lut_on t,roc,:ss_.> .vlrhin t

specific organization) In most of these cases Ihe actual use of this ml'ormatlon :Vlthltl :t _[,,','llic

project context is not supported: _t ts up to the respe.ctive manager to lind the needed anl'ormation. and to make sense out of it in the context of the current project.

4. TAME: AN INSTANTIATION OF THE REUSE-ENABLING ENVIRONMENT

MODEL

The objective of the rellse-onahlillg soft_:_r_: ,.v,,lutiorl ,,nvlronment mo,b'l ,_1"5,.,'rlon ;;"2L>

t.o explicitly' model the learning and reuse .r_q:tt,:d actl_it,,.> ,)i" r,,,'or,ting oxperlenc,' <,.n,,r:tlnZill_

and tailoring experience, formalizing experience, and {re-)u_mg experlence so that ;hey ,'an t3_

understood, evaluated, predicted and motivated.

In order to instantiate a specific reuse-enabling environment, we need to choose :_ mode{ ,_f

the software evolution process itself [n general, such :m ,'vohition process me, let needs re t,e "apahie el describing the integration _f learning -tnd r,mse nnr,o the software evolution process In particular. _t needs to b,_ capabl,_ of m_d_e[irl R wh,,_ #'XI_,'FI,'[IC_! I_ ,:r,'at,'d :m,t i',,cc_rd,',t lilts) t}le

5642

- 17 -

3-39
O_._,,_ .... w',._ +-

O_ _-OG[_ QuALIfY

experlence haze ;_ well a.s when existing experience is used. It needs _.o provide anat.v_Ls for the

purpose of on-line feedback, evaluating the application of all reuse expemence, ami off-lin_ %odback for improving the experience base.

The reuse-enabling TAME environment model depicted m Fi_ur. :'_ _s :.t instantl:LLl(m ,)f

the reuse-enabling software environment model ,+1'Section 3.2. ba.+_,] _,n :L ',err ++'n,:r:_l +tnt+:'r,'+,.-

merit oriented evolution process mo,iel.

/

! dlk

FB

t

_oftware Evolution Proces_

. ! +s_l_t .__¢_onstruct! ./,+mpm._

character_ se_ _-*_ ..... i 7-'_ro luct_

en_irortrnent i goals i _'_ _ I

I

i • W

i " i

"_--+_F,,B__- -'

informal schematized)roductized

PROJEC r SPECIFIC

DOMAI q SPECIFIC

GE _'ERAL"

tr

'V

T

+ R

EXPERIENCE BASE

Figure 3: Reuse-Enabllng "TAME" Environment Model

Each soPtware project perform-d according to this improvement ori,mted ,,vc_hJt_<m pt'+>+',,_>

[ll_)(]_d t'o[Isis[',_ of :t [)[;t[l[litl_ :lll<l ;ill +,x,_CtltiOll ._|ra_e T}I- )i3.[lllll/_ _t:k_{' lIIChl+],'-_ .t .'h:tI';t,'l,.ft/;_.-

5642
0;: POOR QUALITY

uon of the current status of tile proiect environment, the setting of project and _mprov,.m.nt

goals, and the selection of construction and analysis methods and tools that promise to me-t tilt

stated goals in the context of the characterized environment. The execution stage includes T.he

construction of output products and the analysis of these construction processes and resuh, in_ ,mrput products.

The T.X.k,[E environment model giws us a b_is for discussing the ,nte_ration of ,n_ r,,,',_rding and (re-)use activities into the software evolution process. During the environment ::har:_.::t,:rization stage of the improvement oriented process model we (re-)use knowledge about the ne,_ds

and characteristics of previous projects :rod recorci v_he needs and characteristics ,/ 'ii .... _rr-n_

project into the .xperience base During r.he '_oal sorting stage we ire-ms- eXlstlrlg pJ t[l:_ ')q l:-

struction and analysis from smular projects and record the new plans which i/;lv-L,--:I !:ui_,r-.i ',

the needs of the current project into the experience base. During the metho, i :rod r,,,_i _.q-.':;,>._:

stage, we (re-)use as many of the constructive and analytic methods and tooi_ which had [,,_en

used successfully in prior projects of similar type as feasible and record possibly taiiore,i v_rslons

of _hese methods and tools into the experience base. During <:onstruction we :lpi,l ? Th,.-,']-,':-.I

methods and tools, and record the ,'onstructed products into the _xpermn,',' b:>, • [)ur:n_ :_n:.i'. _t-

',se use the selected methods and tools in ¢)rder to ,:otle,:t :rod validate ,tara :,.n,_ :Ln:J_;- :h-m i;L:

r'_cord _hp data. analysis r_sults and lessons learned into the exp,:ri,,nc- h;>-

The TA.\lE environment explicitly supports the eapturme: _)l" MI kind> _)1 ._xF,.rten,',, T!I,.

consistent application of the improvement oriented process mod,.I :_,:r,_-s .dl pr_j,.cr., _lrhin vl

organization provides a mechanism for evMuating the recorded experience, helping n._ to ,b.clde

what and how to reuse, tailoring and analyzing. TAME supports continuous learning. The ,xplicit and comprehensive modeling of the reuse-enabling evolution env_roument inchMing the experience b._e, the evolution process, and the various learning and reuse activities {>,'e l:i_ur, • :/I :_iI_

us to measure ,and evaluate all relevant aspects of reuse. The measurement methodology used :rod

supported within the Tek.ME _nvironment h:_s been publish.d m ,'arli.r pap.,r_ 7 *:

5642
OR_GI,'.%_L F,,-_,_E ;g

01: POOR _JAL.|[Y

5. CONCLUSIONS

[n this paper we have motivated and outlined the scope of a comprehensiv,- r,:use framework. introduced a reuse-enabling software environment model as a first step t,owards such :t

comprehensive reuse framework, and presented a first instantiatAOll of ,llch &n ,-'nv_r,>nmehl E;_ rh,.

context of the T:L\IE (Tailoring .-\ Measurement [rnvironment) project at the lnlv-rsl_v ,,r \l:_r_-

land T. 8i.

The reuse-enabling software evolutmn environment mode[ presented in Section ?, provhJe> :t

basic environment for supporting the recording of experience, t.he off-line _em'raiizata,m Lra,l

t.azloring of experience, the off-line formal,zation of ,'xperience. and the jr" i {1>.... I" ,._(:-l][jz

experience.

trurther _teps re,luired towards r.he ,}utlined rmlse framework are mor,e >p,:,qii, a_wl,,i- ._

each of these activities that differentiate the components of these activities and serv,- _ :t b:L-i_

for characterization, discussion and analysis \Ve are currently taking the reuse-,_nabling :_ofr,,varenvironment model of sect.ion 3:2 down one level and developing a model for lre-h>lng ,-x!,,r,-

-nce. Based on this reuse rnod,..l we will ,levolop a. rmlse t;txonomv allo_ln_ f,)r fh," .'il trgt,',..tt,.:ttlon of any instance of reus,_ The r,,L-e m,_d,q will [)r,)vl(],: insight into Th,* ,_[h,'r :t,'t_x _rl ..... I' !:,

reuse-enabling ,mvironment model ,rely m the way they interact with rhc [r,'-)u>-:,.,'rt',l_'.

Corresponding models for ,:ach ,_f the other activtttes need to be dev-h)ped an,i lnT,.<r:tt,.,i ,Iris,

the reuse-enabling software environment modet.

The reuse-enabling T.-L\IE environmelle model serves _ a basis for better lm,ierstandin_.

evaluating and motivating reuse practices and necessary research activities Performing projects

according to the T,_LXlE environment model requires powerful automated support for dealing wi_h

the large amounts of experience and performing the complicated activities of recordm,z. _*:ner:_ti_-

ing and tailoring, formalizing, and lr_'-hl_mg ,_xperience [ndispens'tbte ,Ionlp,'Jne[l[> Of ,ucil :m

automated support "_ysD!nl at*' :x pow,rfi,l ,'xperi,mo. b:Lse :tll_l ;t []le.'L'_llr_}Hl_'II[ .llt)[)_}rf -%-(*'1ii

\lanv of the r_llbe a[)[)r<_aches ill the [*;t>.l [I;tV," F_>:_ItlZle, l rh:_.l l'be ,i,'v,']_)[y,*r h:L. -,li'{i,'l,'Ilr IlllI.{I,'ll

- 20-

3-42

5642
knowledge of the characterisucs of the particular prop,ct environment. specific needs for

reuse, the candidate reuse objects, etc. It is'not trivial to have all this information available.

The institutionalized learning of an organization and the proper documentauon of that

knowledge is definitely one of the keys to effective reuse. This leads to even better specification

methods and tools (one of the frequently mentioned keys to effective reuse)

.-Ls part of the TAa\[E project at the University of Maryland we have been workin¢ ,m providing appropriate support for building such an experience base, and supporting learnin¢ :rod

(re-)use via measurement. We have completed several components towards a first prototype

T.XNIE system. These components include the definition of project goals and thetr r_4in,.rnent lnt_,

quantifiable questions and metrics, the collection and validation of ,lata. th,qr m:tiy-t> :_n,i rhstorage of all kinds of experience. One of the touChest r_se:_rch problems is to _e rne:u_ur,m,m_

not only for analysis, but also for feedback {learning and reuse} _nd plannin,4 [,_Jrr_)os,_, \\,' n_,,i

more understanding of how to support feedback and planning The T.L\IE system Is intended to

serve as a vehicle for our research towards the effective support of explicit learning and reuse :t>.

outlined in this paper.

8. ACKNOWLEDGEMENTS

We thank all our colleagues and graduate students who ,'ontribut,:d ro thts paper b_ ,.tri_,:r

working on the T.&ME or any other reuse-related proj,.ct or reviewing earlier v,rsions ,q' rh>

paper.

7. REFERENCES

tl J. Bailey, V R. Basili, A.Meta-Model for Software Development Resource Expenditur,> " i[_

Proc. Fifth International Conference on $oftwar,_ Engineering. San Die_o. ['S.-\. \[trch

1981. pp 107-116

5642
_,r_,N,_L PAGE IS

OF. POOR QUALITY

21 V R. Bazili, "Can We Measure Software Technology: Lessons Learned from Eight Years of

Trying," in Proc. Tenth .Annual Software Engineering Workshop, NASA Goddard Space

Flight Center, Greenbelt. MD December t985.

;ll V R. Ba.sili, "Quantitative Evaluation of Software Methodology," Dept. of Computer

Science, University of Maryland, College Park, TR-tS19. July 1985 ialso in Proc ,_f

the First Pan Pacific Computer Conference. Australia, September 19861

4 Victor R. Ba_ili, "Software Maintenance = Reuse-Oriented Software Development" in

Proc. Conference on Software Maintenance, Key-Note Address, Phoenix. AZ. O,:¢ober i9_ _

5 V R. Ba.sili, C. Loggia Ramsey, "ARROWSMITH-P - A Prototype Expert <v_r.em for

Software Engineering Management," IEEE Proceedings of the Expert S_._ten> _rl

Government Symposium, McLean, VA, October 1985. pp. 25.t-264.

tl V R. Basili, H. D. Rombach, "Tailoring the Software Process to Project Goals and

Environments," Proc. of the Ninth International Conference on Software Engineering, Monterey, CA, March 30 - April 2, 1987, pp. 3.t5-357

7 V R. Basili, H. D. Rombach, "T.M\II£: Integrating Me:_urement into Sol*war- Environments." Technical Report TR-1764 (or T.&\IE-TR.I-t9,*.7), Dept. of (omput,,r 5,'1,'u,',,

(niversity of Maryland. College Park. MD 207-12, hlne 1997

'_ V R. Ba_ili, H. D Rombach "The TA.\IE ProJeCt,: Towards [mprovement-()rl,_nr,_l

Software Environments," [EEE Transactions on Software Engineering. voi SE l_. ::_, _

hme 1988. pp. 758-773. is also available a.s Techmcai Report {I-.ML-\L'S-TI{ ",'_ - _"

TR-1983, or T,_fE-TR-2-1988), Department of Computer Science. Lniverslty of \I:,r:,-

land. College Park, M.D 207-t2!.

9, V R. Basili, H. D. Rombach, J Bailey, and B G Joo, "SoFtware Reuse: A Framework "

Proc, of the Tenth Minnowbrook Workshop on Software Reuse, Blue .Mount,am [.aRc,

New York, July 1987.

lO V R. Basili, R. W. Selby, D. H. Hutchens, "Expemmentatton tn Software Engmeertn_,"

IEEE Transactions on Software Engineering, voI.SE-12, no 7. July 19,¢5, pp 73:'-7 t:'

lI V R. Basdi and M. Shaw, "Scope of SoFtware Reuse." \Vhite pap.r, w,_rkine _,roup ,u

Scope of Software Reuse', Tenth Minnowbrook W(_rkshop ,m <ofr,,v:_r, }{,:u-- ['lta,.

Mountain Lake, New York, .July 1087 l in preparauoni

t2 Ted Biggerstaff, "Reusability Framework, .\sse.-sm.nt, /,11¢i Dir,-,:lions " IEE]-_ "-i/t.,v:_.r,,

Magazine, March 1987, pp tl-40.

I:' P Freeman, "Reusable Software Engineering: C.ncept_ :rod Re_earch I)lr,,ctlon_." ['rn,'.

oF _he Workshop on Reusability, September 1987,, pp. B?; 75

1.t R Prieto-Diaz, P Freeman, "Classifying Softw:trp for [___:u.>abillt? ," IEEE Software. ,,'o[ t.

no 1. ,January 1987, pp. 6-16.

15 IEEE Software, special issue on 'Reusing Software, vot.-t, no.l, January P,}s7

16, IEEE Software, special issue on 'Tools: Making Reuse a Reality', vol.4, no.7. Juiv 1987

17 G. A. Jones, R. Pr{eto-Diaz, "Building and Managing Software Libraries." Proc Compsac'88, Chicago, October 5-7. 1988, pp. 228-236.

18' F. E. McGarry, "Recent SEL Studies." in Proc Tenth Annual Software Engineering

Workshop, NASA Goddard Space Flight Center, Greenbelt, MD. Dee t:lS5

19 Mary Shaw, "Purposes'and Varieties of Software Reuse," Prore_'dings of the T,_uth

Minnowbrook Workshop on Software Reuse, Blue Mountain Lake. New York. .hlly

1987

20 T A Standish, ",.\n Essay on Software Reuse." Ha:El': "l'r:m:.:_ct.m.._ ,m ",,_ftx_:_r,,

Engine,-rmg, vol. SE-10. m_ 5 Y,.p_emher t!18,t, pp. 191 1!17

5642

- 22 -

3-44
. _ ";<<i _'_''..'..,_,,...,,....__::_IS

_,'< . <(:: O;.LIL_YY

2i \V. Tr:_cz. "T_itorial on 'Softwar, Reuse Kim:r;il_ T,;chnolo_) '° lEEK C_:flc,_ \L;m.i,.r

EHO2.S-2. I '.)8_

22 kt. V Zetkowt_.z led.!, "Procee, lin_._ o[' ;.he l+ni'+,.rsitv of \[:lr':[;u_d \V, ;,::.m_ .,ll

R.,!u,r,:ments _or :_. _I)t'[w3.FI. _ F:n_ine_!rin_ Envlr,",nrnetlr_' ,7 r,',-mb-[ . \',D ) '.. I!)'-.f; "

T,.'chnic:_.t R<.,porr TR-173;;. Depf of Compur_.r sclen,.< 1 niv,.rsi,tv ,.I X[:_r:.i.t::,J _ ,,[-

1__._ P:u'k. .kID .....0, t_._ D_cemb_r [985 _o be published _ :_ !)cok..%bbx t_uh,' l!]',-<

2:}! M. V Z<lko',:'il,: R. ",..:h R. [-{aml,..r ./ L].mr:on V P_ Br,...<iii. "<,._I ','.:_.i..... _,z:i_,.,:r:_z [r:,...:-

u,:,_s ,_ "h,_ U5 trJ. I:_p;m." lEEK ('ompul,:r \[:,4:Lzm,:. h_l_e [!).',t :>e ;. _i!}

24 •I \:q_'_:. B D-ck,,r• ] Buell. "The. 5of_,v.-._.r,_ \lnn.z_,m_,,nt K:_<ir,'_nm.m_ " .:_ [': .. !'h_r-

_een[h Annual 5,)tk,war,, Enzine_rln _ _,\orksh,..q). \As..\ (;,_d,b.rd :<bat. }"]'.z::: ' ,.m,.r

Cr,_enb_'lt..",[D .X_wemb_r ;:;0. 1985

5642

SECTION 4--ADA TECHNOLOGY

STUDIES

SECTION 4 - ADA TECHNOLOGY STUDIES

The technical papers included in this section were originally

prepared as indicated below.

• "Evolution of Ada Technology in a Production Software Environment," F. McGarry, L. Esker, and

K. Quimby, Proceedings of the Sixth Washinqton Ada

Symposium (WADAS), June 1989

• "Using Ada to Maximize Verbatim Software Reuse,"

M. Stark and E. Booth, Proceedings of TRI-Ada 1989,

October 1989

5642

4-1
EVOLUTION OF ADA TECHNOLOGY IN A PRODUCTION SOFTWARE ENVIRONMENT

Frank McGarry

National Aeronautics and Space

Administration

Goddard Space F]ight Center

Greenbelt, Md. 20771

Linda Esker

Computer Sciences Corporation

System Sciences Division

10110 Aerospace Rd.

Lanham-Seabrook, Md. 20706

Kelvin Quimby

Computer Sciences Corporation

System Sciences Division

I0110 Aerospace Rd.

Lanham-Seabrook, Md. 20?06

INTRODUCTION

The Ada programming language and the associated

software engineering disciplines have been described as one of the most significant developments in software technology in many years.

Although many claims have been made about its advantages and impacts, there have been very few

empirical studies that clarify the impact of Ada.

The Software Engineering Laboratory (SELl is an

organization sponsored by the National Aeronautics

and Space Administration (NASA) consisting of

three principal members: NASA/Goddard Space

Flight Center, the University of Maryland, and

Computer Sciences Corporation. The SEL was

Founded in 1976 to carry out studies and measurements related to evolving software technologies

[7]. The studies are aimed at understanding both

the software development process and the impacts

that evolving software practices may have on the

software process and product. Since 1976, the SEL

has conducted over 65 experiments by app]ying selected techniques to specific development efforts

and measuring the resulting process and product.

In early 1985, the SEL initiated an. effort to

study the characteristics, applications, and impacts of Ada. Beginning with a relatively small

practice problem (6000 source lines of Ada), the

SEL has collected detailed development data From a

total of eight Ada projects (some of which are

still ongoing). The projects range in size from

6000 lines to approximately 160,000 lines of code.

PROJECT BACKGROUND

Development Environment

All Ada projects studied were developed in a

DEC environment, using either a VAX 11/780 or a

VAX 8600. Both machines are shared with other

general users, and the support was average compared to other typlcal projects developed in

FORTRAN. As will be pointed out later, varying

degrees of use were made of the available tools

and methodologies.

COPYRIGHT 1989 BY THE ASSOCIATION FOR COMPUTING MACHINERY. INC.

Pe'rn_.lsslon to copy Wl|hO_l lee a@or part of this malenal Is ojrlmled Wovlded

that the COl_es are nol made or distributed for dlr_:l commefc_l advantaoje, the

ACM COpyright nottce and Ihe bile of the pubhcahon and its date appear, and

nobce is glve_ Ihat copying is by permission of the Assoclat_on for Compelling

Mach;,nefy To copy othefw*se, of Io fE*puD_lSh requires a fee and/o_r specJfi¢

permpsslon

In studying the series of Ada projects, the

goal/question/metric (GQM) paradigm [3] was followed. The goa] of the study was to _eterm_ne the

impact of Ada on productivity, reliability, reuse,

and general product characteristics. A second

interest was to study the use of Ada features

(such as generics and strong typing) over t_me.

Pro_ect Hi_gry

Information on six Ada projects was analyzed

for this study. The projects were developed over

a span of 4-I/2 years starting in late 1984 and

ending with two projects that will be completed in

]989. The study categorized the six projects into

three groups distinguished solely by approximate

start date: the first two are caI]ed the First

Ada projects, the next two are the second Ada

projects, and the most recent are the third Ada

projects. The time]ine for the six projects is

shown in Figure 1.

The first experiences wlth Ada in the SEL occurred wlth two projects that were initiated in

]ate 1984 and early 1985. A team of seven programmers was Formed In ]ate 1984 and began extensive training In December of 1985. The First

target project was a simu]ator that was required

to model an attitude contro] system of a particular NASA satellite, the Gamma Ray Observatory

(GRO). Comparable simulators had been developed

in the past by NASA, and this particular Ada project (GRODY) was deve]oped in parallel to the identical project being developed in FORTRAN. The

results of .that particular comparison are documented by Agresti et at. [l] and McGarry and

Nelson [14], It was estimated that the development of GRODY would probably require from 10 to

12 staff-years of effort, considering previous

experiences with similar FORTRAN projects.

The GRODY team had an average of nearly 5 years

of experience with software development; however,

none had any previous Ada experience. In Fact,

there had been no earIler Ada experience by anyone

In this environment, so no lessons learned and no

Ada experts were available to team members. The

team was experienced wlth filght dynamics problems

a]though it was, on the average, less experienced

than the typical development teams in this' environment.

TO prepare for the design and development of

GRODY, the team underwent 6 months of extensive

135

5642

4-2
I

I 128GRODY KSLOC

GROSS •

: PARALLEL FORTRAN EFFORT(46 KSLOC)

EMS _

1/85 1186 1/87

Figure 1. Ada Projects t n the Fllght

training, whtch covered Ada syntax as well as

principles of software engineering and detatled

destgn techniques [16]. The training included the

following:

e Alsys video tapes wtth discussion

• Lectures on Ada from University of

Haryland staff

• Lectures and workshops on PAHELA [8],

object-oriented design, and other design techniques

e Horkshops based on Booth's tratning

materials [5]

• Lectures on software engineering principles, including abstractlot; and Information hiding

Ourtng the 6 months of training, the electronic

mat1 system (F.MS) Ada pro_ect was developed. Thls

was the ftrst Ada effort completed by this organization. It was set up as a training problem, but

detailed statistics were kept so that the development process and product could be analyzed.

Nhen completed, EHS consisted of approximately

6000 lines of Ada code.

These first Ada pro_ects, GROOY and E'MS, were

developed by the s_e personnel tn &stmtlar software development environment. The EHS project was

developed on a VAX 11/780 ustng the DEC Ada Compilation System (ACS); the GROOY project was developed on the VAX 8600. also using the DEC ACS.

The two pro_ects classlfled as the second Ada

projects both began in early 1987 and were of

medium size with complexity typical of other efforts tn thts environment. 8oth pro_ects were

slmulators required to support the GOES mission.

GOAOA was a dynamics simulator stm_lar to GROOY;

I

I

J UARSTELS 7_ KSLOC

I 67 FDAS KSLOC I

_7 KSL(_X)

162GOAOA KSLOC I

PROJECTS

J 1

EUVEDS

135 KSLOC

EUVETELS J

75 KSLC_X)

t 3RD ADA PROJECTS

2ND ADA

PROJECTS

u_

1/88 1/89 1/90

Dynamics OJvtston of NASA/Goddard

GOESIH was a telemetry simulator that would be

used in testing the attttude ground support software.

The GOADA team consisted of approximately seven

people, some of whom were not asstgned full ttme

to this project. Of these seven, three had prevtous Ada experience and two had experience tn the

application area. The GOESIM team consisted of

four people, one o_ whom had previous Ada experience and one who had experience with thts type of

application. The tralnlng ccnsJsted of lectures

and video tapes on Ada, with parttcu]ar attention

to the Ada style guide that had recently been developed; lectures and classes tn Ada syntax and

Ada _ncepts wero also held. The tratntng lasted

a_2:oxln_ately _ weeks for each team.

Both of these second Ada pro_ects used the 0EC

ACE to complete developmnnt. Because there had

been previous Ada experlence from the flrst Ada

pro_ects, experienced Ada pr_raLmmers were avallable to these teams for consultation and guidance.

They proved to be heavily used commodltles. The

second Ada pro_ects'spanned approxlmately 18 months

each.

The final two pro_ects analyzed tn thts study,

the third Ada projects, were both started In early

1988. The UARSTELS project had requirements and

characteristics stmtlar to GOESIH. one of the second Ad_ projects. The other pro_ect. FDAS, was a

much difference type of system, a source code manager used for m_ntpulattng fltght dynamics software components.

The UARSTELS team consisted of three people,

one of whom had previous Ad& experience; the FDAS

team consisted of four people, none of whom had

previous Ada experience. ' Training for both teams

took the form of lectures and _orkshops based on

5642

4-3
the Ada style gdlde; additional tralnlng from personnel wlth previous Ada experience was also provided. All team members attended classes on Ada

syntax and Ada development.

The teams supporting the third Ada projects had

more personnel available to them who had previous

Ada experience, and they also had available several lessons-learned reports that had been developed by the earlier Ada projects. These two

projects spanned approximately 15 to 18 months

each.

Data Collection

Detailed data for this study were collected for

the six projects. As for all software developed

In the f11ght dynamics environment, the data are

co]letted from the following sources:

• Data collection forms

• Tools and accounting records

• Interviews and subjective information

The most extensive and detailed data were provided

on a series of data collection forms completed by

both the developers and managers and fncl_ding the

following:

• Effort data (hours spent weekly by

activity)

• Error data (for all changes and errors)

• Project estimation (managers' estimate

of final size, cost, schedules)

• Product origlnatlon (characteristics

of modules as they are designed)

These data were the major source of information

used tn the comparisons.

The tools used to record Information tnclude

the automated accounting system (for records of

computer time used, source code changes, and

source code stze history); configuration control

tools (Configuration Management System (CHS), used

to record changes to source code); and ASAP, the

Ada Static Source Code Analyzer [g], which calculates detailed counts and characteristics of Ada

source code. Information was also 9rovlaed

through Interviews with team developers and managers (to record lessons learned and general Impressions) and through subjective assessments made

by senior software engineers (on topics such as

methodoIogles applied). A11 thls informatlon ls

consistently collected, quality assured, and recorded on a data base where It Is used as the

basis for study or analysis.

EVOLVING CHARACTERISTICS OF THE SOFTHARE

Destan Charact_Flstlcs

The first Ada project dld not aut_tlcally

assume the reuse of the standard FORTRAN project

methodologies and products. During the predestgn

phase, the project decided on the products, revtiws, and methodology to be used. Project personnel decided to conform to the traditional

design review process that had been used In the

flight dynamics environment for many years [2,

15]. However, project members investigated sewera1 destgn methodologies and eventually applied a

modified version of object-oriented design tl. 12,

19].

Use of an object-oriented design required a

rethinking of the design, its documentation, and

Its presentation. This caused some inconsistencies with the traditional approach used In the

SEL. Project members were required to develop new

design products because the existing destgn documentation methods for structured and functionally

oriented software design did not apply to the

object-oriented design Cl7], To develop a design

notation used In the design documentatlon, they

combined concepts from George Cherry's process

abstraction method, PAMELA [8], and Grady Booch's

object-oriented design [S]. Design diagrams were

presented at the preliminary design review (POR),

and top-level package specifications were developed for the critical design review (CDR). These

components were then expanded and compiled during

Build 0 of the implementation phase,

The second and third Ada projects adopted and

built on the same methodology and design notation. They also conducted design reviews, but the

speclflc products generated at each stage were

somewhat modifled from the first Ada efforts. For

these later projects, package specifications were

developed for the PDR, and package bodies and subunlt program design language (PDL) were developed

for the CDR. In addition, the components were

compiled before the CDR. These efforts pointed

out the need to redefine the specific products

produced at key milestones of the design process;

however, the characterlstlcs of the products have

yet to be decided. Qulmby and Esker [17] present

a more detailed analysis of the evolving characteristics of both the design process and the products developed.

Software Size

Traditionally, software size has been described

In terms of the llnes of code deve]oped for the

system. Lines of code can, however, be expressed

by many measurements Ell], including the following:

• Tota. physical lines of code (carrlage

r_curns)

• Noncomment/nonblank physical lines of

code

e Executable llnes of code (ELOC) (not

including declarations)

e Statements (semicolons In Ada. which

include declarations)

Table I describes the slze of the Ada projects In

the flight dynamics environment ustng these four

measurements. For comparison to the Ada projects,

typical FORTRAN projects of slmllar applications

are also summrlzed.

Unless only Ada statements are counted, these

figures indicate that ust,_g Ada results tn many

more lines of code than using FORTRAN. The Increase tn lines of code IS not necessarily a negative result; rather, tt means that the size of the

system implemented tn Ada will be larger than an

equivalent system tn FORTRAN. It ts also clear

5642

4-4
Table I. Software Characteristics of Projects

APPLICATION

TOTAL LINES

(CARRIAGE RETURNS)

NONCOMMENT/

NONBLANK

EXECUTABLE LINES

(NO OECLARAllONS)

STATEMENTS

(SEMICOLON- INCLUDES

DECLARAllONS)

1ST ADA

PROJECTS

DYN SIM

128,000

60,000

40,250

22.500

2NO ADA

PROJECTS

OYN SIM TM SIM

162,200 87,500

79.900 42.300

49,000 25,500

29,200 16,300

3RD AOA

PROJECTS

CONFIG

FORTRAN

PROJECTS

67,700

36,000

19.700

12,700

TM SIM DYN SIM

75,000 45,500

41,400 26,000

22.150 22,500

15,200 22,300

TM SIM

28.000

15.0O0

12,500

12,000

Table 2. Effort Distribution (Percent of Total Effort) During Each

Life-Cycle Phase of Ada and FORTRAN Projects

PHASE

REQUIREMENTS

ANALYSIS

DESIGN

CODE

TEST

1ST ADA

PROJECTS

24

42

26

2NO ADA

PROJECTS

4.3

30.5

52_0

13.2

3RD ADA

PROJECTS

6.7

36.0

44.0

13.3

FORTRAN

PROJECTS

12.5

22.5

35.0

30

cO

LN

that a precise definition is needed of what constitutes altne of code in Ada and what types of

code are included tn that measurement.

Many factors contribute to the increased size

of the Ada projects. The style of Ada results tn

code growth because it encourages formatting,

blank lines, and longer, readable names for data

elaments and subuntts. The strong typing within

Ada also produces more code than in FORTRAN because each data element must be expltctt]y declared. In addition, the local style guide places

further requirements on the format for readabilIty. Among other requirements, the style guide

stipulates that each calling argument must be on a

separate physical llne. All these features have

increased the code size, but the increased size

also provides advancemeRts in the areas of capabil|ty, readability, and understanding.

Effort Distribution bv Phase Date_

Effort distributions can be described by the

effort expended during the key life-cycle phases

of a project and by the effort expended tn software development activities. Using the first approach, effort distribution by phase dates, the

typical FORTRAN life-cycle effort distribution

[15] In the flight dynamics envlronment shows

12.5 percent of the total effort expended during

the predesJgn or requirements analysis phase,

22.5 perc_P* during the design phase, 35 percent

during the code implementation phase, and 30 percent during the system test phase (Tab]e 2).

From the review of literature on Ada [18], It

was expected that the effort distributions would

be slgnlflcantly different for the Ada projects

due to the modified design and implementation approaches. It had been anticipated that the Ada

projects would require more effort during the detailed design phase and less effort during the

code and test phases. However, In the flight

dynamics environment, slgnlflcant changes to the

ltfe cycle have not been observed. The Ada projects were planned by managers experienced with

FORTRAN projects, and perhaps their plans were

Influenced by the FORTRAN life cycle.

Although the changes are not occurring as

quickly as anticipated , the Ada life cycle is

changing s11g,tly wtth each project and may soon

show a different life cycle than that expected for

a FORTRAN project. The life cycles for the second

and third Ada projects are shifting sltghtly to

show more design ttme required and less system

test time. The effort distributions of the Ada

5642

4-5
Tabl e 3. Effort Distribution (Percent of Total Effort) for Development Activities Across All Life-Cycle Phases of Add

and FORTRAN Projects

PHASE

REQUIREMENTS

ANALYSIS

DESIGN

CODE

TEST

1S÷AOA

PROJECTS

ill

3.2

36.5

42.7

17.6

2ND ADA

PROJECTS

i

3.7

275

52.0

16.8

3RD ADA

PROJECTS

6.5

35.7

44.5

13.3

FORTRAN

PROJECTS

6.0

24.0

45.0

25.0

p.

projects are showing that the FORTRAN 11re c_cle

cannot be auto_ttcaliy assumed for Add.

These observations probably Indicate that the

llfe-cycle definition Is not easily changed merely

because a different language or technique is applied. The evolution toward the expected characteristics of the new technology tsa slow, gradual

process.

Effort Distribution by Actlvttv

The second approach to effort distributions ts

to analyze the effort by activity. In addition to

collecting the effort expended between key phase

dates (e.g., between the CDR and the start of system test), the SEL also collects detailed effort

data Independent of phase dates.' Effort by phase

Is ttme driven and assumes, for example, that all

design activities are complete and cease at the

end of the design phase. In reality, many activities take place during each life-cycle phase and,

therefore, the effort distribution by activity can

be quite different from the distribution by phases.

This, indeed, was the case for the first Add

projects. Although only 24 percent of the total

effort was allocated to the design phase,

36.5 percent of the total project effort was spent

on design activities (Tables Z and 3). The extra

effort needed for Add design activities ts more

apparent tn the dlstrtbutton of effort by activity

than In the distribution by phase dates. The effort by development acttvlty again reinforces the

trend seen above. On the Add projects, more effort was requtred for design and less effort for

software testing than qn the FORTRAN projects.

Use of Add Features

In an effort to achieve some measurement of the

use of the features available In the Ada language,

the SEL Identified six Add features to monitor:

generic packages, type declarations, packages,

tasks, comptlable POL, and exception handling.

The SEL then examined the code to see how little

or how much these features were used. The purposes of this analysis were, ftrst, to determine

to what degree features of Add were used by the

Add project and. second, to determine whether the

use of Add features "w,ttured" as an environment

gained experience with the language. Data on the

use of these Add features were obtained using the

Ada Static Source Code Analyzer Program [9].

Analysis of the use of comptlable PDL and exception handling dld not show any trends. Perhaps tt

ts too early to see results In these areas; however, trends were observed in the use of the other

features.

The average size of packages (In source lines

of code (SLOC)) for the first Add projects Is much

larger than the average size of packages for the

second and third Add projects (Figure Z). This

increase ts due to a difference tn the structuring

method between the first Add projects and all subsequent Add projects [17]. The first Ada pro_ects

were designed with ond package at the root of each

subsystem, which led to a heavtly nested structure. In addition, nesting of package specifications wlthtn package bodies was used to control

package visibility. Current Add projects are

using the vtew of subsystems described by

Grady Booch [6. Ch. 17] as an abstract design entity whose interface ts defined by a number of

separately comptlable packages, and the only

nested Add packages are generic package tnstanttatlons.

The generic package tsa major too1 tn the Add

language contributing to software reusability.

Reports have shown the benefits of Add reusable

software [18] and, in the flight dynamics environment, use of generic packages has been Increasing

from the first to the current Ad& projects. Hore

than one-third of the packages on current projects

are generic packages. Although more analysls is

needed, thts higher use of genertcs possibly reflects both a stronger emphasis on the development

of verbattm reusable components and an Increased

understandlng of how to use generic Add packages

effectively In the f11ght dynamics environment.

The use of strong typing In these software systems was measured by the number of type de_larattons per thousand lines of code. Although the

measure itself ts not Intuitively meaningful, tt

provides a method of observing trends In the use

of Ada type declarations. In the fltght dynamics

environment, the amount of typing Is Increasing

over time. Thts _sy indicate that the developers

are becoming more comfortable wlth the strong typtng features of Add and are using Its capabilities

to a fuller extent.

5642

4-6
GENERICS

53

so,

4o,

GENERIC PACK 30 TOTAL TYPES

30. TOTAL PACK KSLOC

20. I5.7

tO.

0.

i STADA 2NDAOA 3RD ADA

PROJECTS PROJECTS PROJECTS

10

0.

STRONG TYPING

t ST ADA 2NOADA 3ROADA

PROJECTS PROJECTS PROJECTS

3

PACKAGES

KSLOC TOTAL m

PACKAGE 1_. TASKS

ISTAOA 2NOAOA 3RDAOA

PROJECTS PROJECTS PROJECTS

Figure 2. Use of Ada Features

lo.

8

8.

6,

4.

2.

0

1STAOA

PROJECTS

TASKING

o

2NO ADA 3ROAOA

PROJECTS PROJECTS

The tasking feature of Ada ts used in the

flight dynamics environment for the dynamics simulators. Eight tasks were used for GROOY, the

first dynamics simulator in Ada. For subsequent

dynamics simulators of approxl_tely the same stze

and functionality as GRODY, design personnel determined that Four tasks were sufficient to implement the interactive capability of the system. It

ts expected that future dynamics simulators will

continue to use tasking, however developers are

now using tasking more Judiciously. The third Ada

projects are telemetry simulators, which are seqbenttal systems that do not benefit from the feature of the language, and thus do n6t use Ada

tasks.

COST/RELIABILITY/REUSE

Productivity

Discussions on Ada productlvlty require careful

interpretation because so many definitions exist

for software size measures In Ada. Depending on

the measurement used, software developers using

Ada can be shown to be either as productive as or

not as productive as software developers using

FORTRAN. Using the total ltnes of delivered code

as a measure, the Ada pro_ects studied show an

improving productivity over time, and they show a

productivity greater than FORTRAN (Figure 3).

However. considering only code statements (semicolons for Ada or excluding a11 comments and continued lines of code tn FORTRAN), the results are

different. An increasing productivity trend remains tn the Ada projects over time, but the Ada

projects have not yet achieved the productivity

level of FORTRAN projects.

In the fltght dynamics environment, many software components are reused on FORTRAN projects.

Because no Ada components existed previously, the

first Ada projects were, tn fact, developing a

greater percentage of their delivered code than

the typical FORTRAN project. A past study by the

SEL and experience with FORTRAN projects indicated

that reused code costs approximately 20 percent of

the cost of new code [2]. Using this estimate,

reusability can be factored Into software size by

estimating the amount of _ code. Developed code I$ calculated as the amount of new

code plus 20 percent of the reused code. Ntth

software reusability factored in, the productivity

for developed statements on Ada projects ts approximately the same as that for FORTRAN projects

(Figure 4).

Hany objections are often made when computing

productivity tn terms of lines of code: it is

affected by style, there are many ways to code the

same function, etc. The most intuitive measure to

use In computing productivity is cost per "function." Some attempts have been made within the

SEL to compare the functionality of projects being

compared (e.g.. GRODY versus GROSS), and data seem

to indicate that comparing "statements" ts the

closest measure to comparing functionality.

The trends in Ada productivity are very positive In that the overall cost of producing an Ada

5642

4-7
LINES

PER

STAFFDAY

8o

6o

40

2o

(TOTAL LINES)

FORTRAN IST AOA 2ND AOA 3RO AOA

PROJECTS PROJECTS PROJECTS PROJECTS

Flgure 3.

12

146

(STATEMENTS)

119

FORTRAN 1ST ADA 2NO ADA 3RD ADA

PROJECTS PROJECTS PROJECTS PROJECTS

Ada CostlProducttv¶ty of Delivered Code

DEVELOPED UNES (NEW + 20% OLD)

46.8 441

FORTRAN 1ST AOA 2NON:)A 3ROADA

PROJECTS PROJECTS PROJECTS PROJECTS

12

10

8

6

4

2

0

10.3

(OEVELOPED STATEMENTS)

FORTI_ N 1ST AOA 2NO Ann 3ROAOA

PROJECTS PROJECTS PROJECTS PROJECTS

Figure 4. Ada Cost/Productivity of 0eveloped Code

System has quickly become equivalent to that of

producing a FORTRAN system. The fltght dynamtcs

FORTRAN environment ts stable, mature, and built

on a long and extended legacy of experience.

Although the first Ada pro_ect required 30 percent

more staff-hours to complete than a stmtlar

FORTRAN project, this overhead included effort to

develop new practices and processes and to learn a

new environment. Htth experience, the environment

is becoming more stable and productivity is increasing.

Reliability

As with productivity, the many ways of measurIng software size affect the results of rellabtlity studies. For example, will the error rate

normalized by the total system stze or by the number of language statements gtve the most accurate

reading of the rlllabiltty of Ada software compared to FORTRAN? In the flight dynamics environmint, changes to the software made after untt

testing when the software Is placed under configuration control are formally reported on change

report forms. The developer must supply the reason for the change (e.g., error, requirement

change) and, tf the change is due to an error, the

source and type of error.

In a very mature FORTRAN development environment. the GROSS project reported 3.4 errors per

thousand lines of source code (KSLOC) in the system. AS Table 4 shows, all the Ada projects

achieved an error rate lower than the rate on the

FORTRAN project. In addition, the error rate on

the Ada projects shows a decreasing trend over

tlme.

Hhen the error rate ts normalized by the number

of language statements, the first and second Ada

projects show a slightly higher error rate than

the FORTRAN project. However, the error rate

again shows a decreasing trend over time. On the

third Ada projects, the errors have decreased to

rate as good as, if not better than, the error

rate on the FORTRAN project. It is still too "

early to observe a definite difference from the

FORTRAN rates; however, the reliability of the Ada

pro)eras appears at least as good as that of

FORTRAN projects and Is improving wtth each Ada

project.

Classes of Errors

Errors reported are classified according to

source and type of error. Sources of errors can

be requirements, functional specifications, design, code, or previous changes. Types of errors

5642

4-8
Table 4. Ada and Error/Change Rate

ERRORSYKSLOC °

ERRORSYK STMTS

1STADA

PROJECTS

1.8

10.2

2ND ADA

PROJECTS"

1.7 1.4

9.4 7,5

3RDADA

PROJECTS"

1.0 1.0

5.3 5.6

FORTRAN

PROJECTS

3.4

6.9

• SLOC-TOTALLINES(I_K_LUDE$ COMMENT_REuSED)

"FIGURESE_kSEDONEST]MATES

are initialization, logic/control structure, Internal interface, external Interface, data value

or structure, and computational.

On a typ|cal FORTRAN project In the f11ght dynamlcs envlronment, design errors amount to only

3 percent of the total errors on the pro_ect (Figure S). For the first and second Ada projects, 25

to 3S percent of all errors were classified as

design errors, a substantial Increase. For the

thtrd Ada project, however, design errors dropped

significantly and are estimated to be approximately 7 percent. Thls rate Is close to that experienced on FORTRAN projects and clearly shows a

maturation process with growing expertise In Ada.

The literature on Ada reports that the use of

Ada should help reduce the number of interface

errors In the software [4]. Although the compiler

wtll catch most calling parameter consistency

errors, interface errors can also include errors

that wtll not be detected until run ttme_ Typically, these are errors In string parameters or

subtypes with different constraints and errors in

calling parameters due to the need for additional

or different types of parameters. Using guidelines and examples tn the data collection document

[13]. the errors are classified by the developer

reporting and correctlng the error.

In the flight dynamics FORTPJ_t4 environment,

about one-third of all errors on a project are

interface errors. On the flrst and second Ada

projects, the percentage of interface errors was

not greatly reduced (Figure 5), wlth approxl_tely

one-fourth of the errors belng interface errors.

Hith current projects, however, the SEL is now

observing a significant change: interface errors

are decreasing.

In the SEL, "errors due to a previous change"

categorizes errors caused by a previous modlflcatlon to the software. The flrst Ada projects

showed a large Jump In the percentage of these

errors compared to projects using FORTRAN (Figure S). However, all subsequent Ada projects show

a rate for these errors that Is very similar to

the FORTRAN rate. Thls initial Jump In .the error

rate can probably be attributed to inexperience

wlth Ada, inexperience with Ada design methodologies, and a nested software architecture that made

the software much more complex. Again, the error

profile Is evolving with the maturity of the Ada

environment.

Software Reuse

Throughout the years of developing similar systems In FORTRAN In the fllght dynamlcs environment, the average level of software reuse has been

between 15 and 20 percent [lO, 20]. FORTRAN projects that attained a software reuse rate of

35 percent or higher are rare. After the First

Ada projects and wtth only S to 6 years of maturing In the environment, Ada projects, have now

achieved a software reuse rate of over 25 percent,

already greater than the typical FORTRAN project.

The UARSTEL$ project ts expected to consist of

more than 35 percent reused code. This trena of

increasing software reuse Is very promising.

%

DESIGN ERRORS

%

INTERFACE ERRORS

FO_ 151'_A _ 3RDADA

PROJECTS PRIOJECTS PROJECTS PROJECTS

Figure 5. Error Characteristics

%

ERROR DUE TO PREVIOUS CHANG E

20.

14

|

FORTRAN let AOA a_O ADA 3RO AOA

PROJECT_ PROJECTS PROJEGI_) PROJECTS

5642

4-9
CONCLUSIONS/OBSERVATIONS/COHPARISONS

Many aspects of software development with Ado

have evolved as our Ado development environment

has matured and our personnel have become more

experienced In the use of Ado. The SEL has seen

differences In the areas of cost, reliability,

reuse, size, and use of Ado features.

A first-time Ado project can be expected to

cost about 30 percent more than an equivalent

FORTRAN p_oJect. However, the SEL has observed

significant improvements over ttme as a development environment progresses to second and third

uses of Ado.

The reliability of Ado projects ts Initially

similar to that expected in a mature FORTRAN environment. Ntth time, however, improvements can

be expected as experience with the language Increases.

Reuse ts one of the most promising aspects of

Ado. The prOpOrtion of reusable Ado software on

our Ado projects excl_ds the proportion of reusable FORTRAN software on our FORTRAN projects.

This result was noted fairly early tn our Ado

projects, and our experience shows an Increasing

trend over time.

The stze of an Ado system wtll be larger than a

stmtlar system In FORTRAN when considering SLOC.

Size measurements can be misleading because dtfferent measurements reveal different results.

Ratlos of Ado to FORTRAN range from 3 to 1 for

total physical lines to 1 to 1 for statements.

The use of Ado features definitely evolves wlth

experience. As more experience is gained, some

Ado features may be found to be Inappropriate for

spectftc applications. However, the lessons

learned on an earlier project play an invaluable

part in the success of later projects.

REFERENCES

;. Agrestl, N., et al. Designing wlth Ado for

satelllte simulation: A case study. Proceeding S

of the First [nternatlonal Svmooslum on Ado for

the NASA Snace Station, June 1986.

Z. Agrestl, N., NcGarry, F., et al.

Handbook for Software Oevelooment. Software Engineerlng Laboratory, SEL-8_O01, Aprll 1984.

3. Baslll, V. Ouantltatlve Evaluatlon of Software Methodoloqy. Unlverslty of Maryland, Techntcal Report TR-ISIg, 1985.

4. 8astlt, V., et al. Use of Ado ?or FAA's Advanced Automation System (AAS). The NITRE Corporation, Aprtl 1987.

S. Booth, G. Software Enolneerlna Hith Ado,

Menlo Park, CA: Benjamin/Cummings Publishing Cornpany, 1983.

6. Booch, G. Software Comuonents Hlth Ado --

Structures. Tools. and Subsystems. Menlo Park,

CA: Benjamin/Cummings Publishing Company, 1987.

7. Card, D., McGarry, F., Page, G., et al. The

Software Enatneertno Laboratgry. Software Engineering Laboratory, SEL-SI-104, February 1982.

8. Cherry, G. Advanced Software [nalneerlnq Ntth

Ado--Process Abstraction Method for Embedded LarqP

Aooltcattons. Language Automation Associates,

Reston, VA, 1985.

9. Doubleday, 0. ASAP: An Ado Static Sourc_

Code Analyzer Program. University of Maryland,

Oepart_nt of Computer Science, Technical Report 1895, August 1987.

10. Esker, L. Software Reuse Profile Study of Recent FORTRAN Protects in the Fllaht Dynamics Area.

Computer Sciences Corporation , IX-881083(59 283),

January 1989.

11. Flresmlth, D. Mlxlng apples and oranges: Or

what ts an Ado ltne of code anyway? Ado Letters,

September/October 1988.

12. Godfrey, S.. and Brophy, C. Assesslna the

Ado Oeslon Process and Its Imollcattons: A CasP

_dJ._- Software Engineering Laboratory,

SEL-87-O04, July Ig87.

13. Heifer, G. Data Collection Procedures ?or

the Rehosted SEL Database. Software Engineering

Laboratory, SEL-87-008, October 1987.

14. McC_rry, F., and Nelson, R. An Exoerlmen_

Nith Ado--The GRO Dynamics Simulator. NASAIGSFC.

April 1985.

15. NcGarry, F., Page. G., et al. Recommended

Aooroach to Software Develooment. Software Engineertng Laboratory, SEL-81-ZOS, April 1983.

16. Murphy, R., and Stark, M. Ado Tralnlnq Ev_luatlon and Recommendatlon_. Software Engineering

Laboratory, SEL-85-002, October 1985.

17. Quimby, K., and Esker. L. Evolution of Ado

Technoluov in the Fllaht Ovnamlcs Area: Desian

Phase Analvsls. Software Engineering Laboratory,

SEL-88-003, 1988.

18. Reifer, D. Ada's impact: A quantitative

assessment. Proceedings of the 1987 ACH SIG_d_

International Conference. December 1987.

19. Setdewltz, E., and StarK, M. General OblectOriented Software DeveloPment. Software Engineering Laboratory, SEL-86-002, August 1986.

20. Solomon, 0., and Agrestl, N. Profile of

Software Reuse in the Fllaht Ovnamlcs Envlronment

(Prellmlnarv). Computer Sciences Corpo:'atlon,"

CSC/TM-8716082, 1987.

5642

4-I0
USING ADA TO MAXIMIZE VERBATIM SOFTWARE REUSE

Michael E. Strut, HASA/Goddard SpaceFLigh_Cent_

Eric W. Booth. CompoterSciencesCorpor_on

1. INTRODUCTION

The reuse of software holds the Imm_se of increased productivity

and reliability. Experience has shown that making even the

alightmt elumge to • "reused" piece of mftware can result in cindy,

mq_lictable errors [Solomon, 1987]. For this reason, the Flight

Dynamics Division (FDD) of Godciard Space Flight Center

(GSF-C')is concer_atin 8 effort on developing "verbatim" reusable

software ccmxponen_ with Ada, where verbatim mem_ that no

c._mges whatever are made to the conrpon=_

This pape_ presents the lessons learned on several simulator

projec_ in the FDD environment that exploit features of the Ada

languaSe, such u packages m_t _rics, to achieve verbatim stone.

Thesesimulau_ aredividedintotwo sopwate,butreisted,l:roblem

domains. A dynmm_ an_nudmor is meal by the FDD mathemaficud

malysts to verify the a_l_le coatml laws that a spacecraft builder

developed. A W._ $/mu/a/orgenl_atastrot data sets for

other mission support software. FDD began using Ade in 1985

with the development of the Gamma Ray Ol_un'vatory attitude

dynamics simulator (GRODY). Since that _e, six additional

simula_ projects have been started. With each t_:_uive project,

• <xmcexm'atod effort is made to use the lessons learned from

p_vious Ade dmulator devdopment project.

This luzper focuses on the concepts used in the projects that have

had the most impact on verbatim software reuse in the FDD

environmertt: GRODY, the Upper A_nospha'e Research Satellite

Teleme_ Simulator (UARSTE_), and the Gmeric Dynamics md

Telemeu'y Simulator (GENSIM). This paper defines underlying

design principles, discusses how Ada features support these

principles for reuse in the small and shows how these principles

are used to achieve reuse in the large. _naUy, this paper presents

suR=or_g _,t- fi_n cuz_.._ reusabifi_ _

The FDD has been using a modi_ed vm_ion of the General ObjectOriented Development (GOOD) methodology [Seidewi_, 1986;

Stark. 1987; SeidewiZz, 1988] to develop its Ade software. Three

concepts that play • role in GOOD enhance verbal_n reuse:

abstraction, inheritance, end problem-specific architecmras. These

concepts support the reuse of successively larger components

within successively narrower doma_n.s. The next two .,tcctious

describehow theseconceptsare applied to simulator projec_ in the

FDD. Abstraction supports "reuse in the •mall," and inheritance

andpmblm-specificarchitectures s'uplx_"reuseinthelarge."The

current practice is to supporx reuse in the unall through component

libraries, as is done with a collection of componenLs by Grady

Booch [Booth, 1987] and EVB's Generic. Reusable Ada

Components for Engineering (GRACE) [Berard, 1989]. The

UARSTELS and GENSIM prop._s are cited to describe how reuse

on a large scale is accomplished and to demonstrate the potential in

cost savings and/or the ability to solve more complex problems.

2. REUSE IN THE SMALL:

USE OF ADA GENERICS

section presents design and implementation guidelines for

usin 8 Ade generim. It shows how the design principles mentioned

in the i_viom _ are implemmtod using AdL

Designing individual generic components is understood to the

point that such componenm are commenfially available. The Bcoch

taxonomy of ml_cua'escreates• family of components that satisfy

the same abstraction within the context of different problems, for

exarnple_ sequential versus corgm'r_t applications.

The design process becomes more intermling when a hierarchy of

generic cemponents is needed. This is the case when d_igni_ 8

generic subsystems with multiple levels of abstraction.

leveling of a subsystem _ be embodied in the Ada code by using

genericunitsand insumfiatiousin thefollowing threeways:

1. LAbt_ry_t instanfiafior_

2. Nested instsntiations

3. Nested genericdeFaddons

This section will define each of these approaches and provide

examples to demonstrate their application. Implications for

reusabifity and lessons learned are included for each approach.

library Unit Instantiations

The fu'st approach is to create m instantiation of a gene_ic chat is a

librsry UniL This approach is appealing for practical reasons. The

potentially broad scope provided by libr_ urn| instamiations may

be necessary for Ada compilers that do not implement code

5642

4-11
sharing I [Ganapathi. 1989]. Most implementations currently do

not support code sharing. Instead. each msumtiation creates a

complete object code copy of the generic template. The systemwide scope provided by library units often makm only one copy

(imuer, e) nece_y.

To describe the first method of library unit insiantiations, assume

that the generic package A depends on a set of rubpm_'am_ P.

provided by the generic package B. Instan_adn 8 B as • library unh

creates a copy of this generic package ca/led Instanc,e..B. The

generic package A may then be instantiated by using the

subprograms provided by Insumce_B as actual p&ametcm. Thh

allows the generic packages A and B to be designed and

imple_nented having no external dependances, w_ch makes reuse

simpler. This approach is depicted in the design diagnun 2 shown

in Figure I.

generic

with procedure Pl Is <>;

package A Is

end A;

generic

oo.

package B is

procedure Pl;

, ii3:

with A.Insumce_B;

use Inmnce B;

package Inmumce A is new A;

ce_B 0 A , ........

> - -,i ::i *'=-_,/ :. :i: : :':.i:::::-:": -.: .. :i !:!:::i.-_" </i.:':._:':'::,:!i!'i'!:::_:i::.i *

lit, me I.

As previously mmfioned, instanfiating gener_ as library units

has the advantage that insumtiafions of A sndB may be impo_d

(named inaw/th clause) by any compilafi_ trait in a system.

When fullvisibility is desired, this technique works well.

However, if B were an abstrect state machine _ and the design

required that B should be visible ordy to A, this poumfially broad

visibility of B is undesirable. It would be far better to use the

language to enforcethatdesigndecision.

Another drawback of h'brary unit ir_mntiations is that as generic

componen_ become more complex+ they require a longer lht of

more problem-specific generic formal parameters. Each

instantiation of the generic becomes long and complex. The

instantiations can be made simpler by specifying defaults for

f_mal sublxograms using the notation "with procedure Pl is <>2

Then (e._mmmg in this case that Ks only formal parameter is Pl),

the instance of A can be written as shown in Figure 2.

1 Code slm_g isatecJmiquethatallowseachinsum_ ofageneric

tosharethesame objectcode. The resultis_uallyasmaller

objectcode sizeandslowerexec_on speedforthesyste_

2 The nmafianfm thedesigndiagnms usesmtmded-4:em_

recumglestorepresentpackages,solida_fowstorepresent

dependencies, broke_ arrows to represent insta_fiations, and broken

package symbols to represent a generic unit.

3 A package that is an abstract state mech/ne is a package that

ma/ntain_ state information in the package body [Booch. 1983].

llt, ere 2.

technique works well when numerous simple ftmcdons are

used as formal parame_'s. This use ofAda .6m_dat_ the search of

an object code h'brm7 to resolve exmmal references. Figure 3, part

2 _ m example of mathematical packagesbeing used to

in_lantiate the package of flightdynamics absU'ectdatatypes4

shown in part I. Care should be used with this technique, since

the use clause does more than simply make objects and operations

directly visible. There are visibility md precedence rules of the

Isnguage that will affectwhat defaults wi]J.be used [Mendal, 1988].

generic

type REAL b dlglta o;

RaDIAN$ Is digits o;

VECTOR ISarray ( 12_rEGER range <>) of REAL;

type MATRIX IS array ( INTEGER range _,

INTEGER range <>) _ REAL;

with ruction sin ( Angle : in RADIAN$)

return REAL is _;

with function cos ( Angle : in RADIALS)

return REAL is <>;

with function Floor ( Item: in REAL)

remra REAL Is _;

p_.kaSe Genaic.Ammde_Tyr_ IS

Figure 3 (1 of 2).

4 A package that is m abstract data type exports objects, typos, and

operations but does not maintain state information in the body

[Booch. 19831.

5642

4-12
with Single_Math_Functions,

Single_I.Lne ar AIgebra;"

use SingleMath_Functions,

Sin gle_LL-_sr.._gebra;

with Ma_h_Types,

Ga_/c_Ammde_T_;

package Single_Auimde_Types is

new Generic_Aaimde_Types(

=> Math_Types.SINGLE,

RADIANS => Singlc..M_h_Functions.RADlANS,

VECFOR => SingleLinesr_AlgebrLVECTOR,

MA'rR]X => Single_Linear_Algebra.MATRIX);

Figure3 (2 of2).

A second method that uses library unit imun_ is to insum_ate B

as a library unit. which is then wlth-ed into the body of generic A

(Figaro 4). Since the procedures do not need to be pessed as actual

parameters,Otisoption allows A m have a shorter formal

pm-ameter list. However, method often requires the use of common .

types for A and B. For example, for a flight dynamics gpp_cation,

the generic package A would have to be coupled to the same

floating-point types as the generic or imumcs of package B. As

long as this sort of coupling is relatively simple, it can be

managed (in the simulator case by having • single package

_ontainin8thebasicfloating-poinz types).

Figure 4,

The disadvantage of this approach is the use of a common types

package to implicitly couple A and B. This defeats software

engineering principles of information hiding _d data abs_'action.

The following section describes how to exploit these software

engineeringprinciples bynestinggenericinstan_ion_

Nested Instantiations

Continuing the same example, the generic package B may be

instantiated in the body of generic A (Figure 5) using the generic

formalsofA. Thisoptionisidealforabstractinnand information

hiding because it San be extended m a seriesof nested

instantiations. Objects,types,md operationsfrom B may beused

as building blocks end specialized to raise the level of

abstraction [Stark.1987]. This is sometimesreferredtoas reexported.ImportingthepackageB inth_ m_mcr allowsobjects,

types._nd operationstobe hidden in thebody of the generic

p--kageA butstillusedtoinstantiate B.

................ -:.(A ;.

Figure 5.

Using nested instandafion.s is an appearing technique for software

engineering reasons. The limited scope provided by a package

boundary increases info_ hiding md protection. The amount

of information that the user of the oum_"package A needs to know

may be Limited to the package specification of A. The fact that en

instandadon of a lower level generic is used to implement the

package is irrelevant to the user. Ramifications of future changes

made during the rnalnte_nce phase will be much more limitedin

scope then the library trait insu_sfion q_roach.

The dlsadvmutgn of using nesth_g in.sundafions is the advantage of

using library unit insumtiations. That is, if the Aria compiler

being used does not _ort code timing and an instance of t

imrdaul& generic is neceassry in several locations, nesRng will

result in mu/fiple object code copies. For example, the executable

size of GRODY is less than 2 megabytes (Mb). This simulator

does not have multiple copies of nested instantiations.

UARSTEJ_, on the other hand, makes extensive use of nested

instant_ons, whichresulmdinan executable dzc of6 Mb.

On thesurface,the impficsdonof thesefindingsseems tosuggest

using librmry unit h_standations exclusively. The scmtl

implication, however, b that one should use nested insumdations

only when a few copies are necessary. One way tominimize the

number of copies is to implemeztt the generic package as an

abstract data type rather than an abstract state machine. This

approach_-may be possible when multiple instances of the

sbstraction use the same types and subprograms u actual

paxgmemn butuse differentobjects(Ads allowsobjectsto be

passed at ramtime, while types and subl_ograms must be passed at

instsntiat/on time). This approach allows copiestobe created with

object decimations insmadof generic instamiadons; it also has the

benefit ofincreasedflexibility, sinceobjectsmay be declared static

at compilation _me or creamd (dynamically) at run time.

When the c_Tect design calls for multiple instances of an abstract

state machine the long-term soluti_ is W acquire a compiler that

supportscodesharing.

5642

4-13
Nested Generic Definitions

Nested generic defm/uons is the third design spproach described

here. This technique is appealing when the problem calh for •

high degree of coupling between generics.

Changing the previous examples, ff instantiations of generic

package A end generic package B will have common types and

subprograms as actual perameten, the foUowing arcMtectmes ere

possible:

i. Make the types and subprograms visible to the generic

templates via with clm.tses.

2. Make each generic package a h'l_a_ mar madduplicate the

genaric fomud pmm_etm's in tim generic part of

3. Nest tim generic dofinitior_ within the specification of

another generic package, C. The generic part of C

timcommon formal pmsmetets (Figme 6).

: B .

"............. /

Figure 6.

Although the first option works, it suffers from a high degree of

coupling. Future instances of either A or B will always be using

the same common types and subprograms. This inflexibility

results in limiting the verbatim reusability.

The second option is a large tmprovement over the rust. Future

of packages A and B may now supply their own types and

subprograms for the generic formal parameters. However, this

architecture becomes tedious and error prone when the common

typ_ and sublx_grams are long and complex. Since generic

instamiabon becomes a large pgrt of the effort when maximizing

verbatim reuse, it is desirable to simplify this activity.

The third option accomplishes the same goals as thesecond option

but with less duplication. This opdon is useful when the number

of nested generics or the number of common generic formal

paramete,s becomes large. It is less error prone becausethe

common acmtl parameters are supplied only once. The

maintenance phase also benefits from the single location of

commmt actual imrameters.

Figure 7 shows an example of nested generic deFmitiom f_m

UARSTELS. The generic function FSS Digitize it declared

withim the genetic package Generic Sensor Digitization. The

genmic fmmal paramete_ of the ccmpos/m package (generic sensor

digitization), as well es the generic formal parameters of FS$

Digitize, ere referenced in the body of FS$ Digitize.

generic

type REAL It digltt <>;

type COUNTS It range o;

with procedure Log_Error

(Message : in String) is Text..IO.Put_Line_

package Generic_Sensor Digitization Is

foneeen Linear_Digitize

( Parameter: in REAI_

Dins - in REAI_

Seal,- in REAL ) return COUNTS;

generic

type COUNTER It range <>;

with function tan (X: REAL) return REAL Is o;

with bract/on sin ('X: REAL) return REAL is <>;

with function cos (X: REAL) return REAL is <>;

with function Floor (X: REAL) return REAL is o;

function F$S_Digitize

( Angle:tnRE_

Coelficiem: in REAL;

Tolerance : In REAI_

Maximum_Number of_Iterations : In COUNTER )

return COUNTS;

,.o

end Ganmc_Sensor_l)i_tizs_on;

Figure 7.

FinaUy. a practical benefit accrues from using the nested generic

definition approach. Most compilers, as previously pointed out. do

not m_pctt code sharing; instead, they expand the generic template

at instantiation time. The advantage for the designer needing only

I geaeric B'om a package containing 10 nested generic de/'mkions

is that only the object code for the 1 instantiation will be

genamed.

5642

4-14
3. REUSE IN THE LARGE:

PROJECT-SCALE REUSE

Thb section presenu the details of the. two projects w.FDD that

have had the most impact on verbatim reuse: UARSTELS and

GENSIM. For both projects, an overviewispresentedgivingthe

background information, the goals, and the motivating factors

involved during deve!opmenL Each system's architecture is

discussed using the concepts and notation from the previous

section of this paper. Finally, the lessons learned from each

project are discussed with their implications for fumm development

efforts.

UARSTELS Overview

The UARSTEL_ project was started in February 1988 and was the

fourth Ada simulator initiated at FDD. Previous simulators

included the GRODY experiment, the Geostafionary Operational

Environmental Satellite-I (GOES-I) dynamics simulator

(GOADA), and the GOES-I telemetry simulator (GOESIM). The

GOES-I simulators represent the first operational Ada soRware

developed at FDD.

The GRODY design team exploited the feature of nested uni_

which resulted in increased information hiding (L,dormation

protection m/gig be a bet_r phrase). The rationale for using

information hiding is increased reliability. Higher reliability

should,in_ increasereusebi_ty; that is, if a component is very

reliable, it is appealLng to reuse. This was the basis for the

GRODY design.

The lesson learned from this approach was that exumsiveuseof

nested packages acum//y _ reusability [Quimby, 1985]. In

addition,durLngthecodingand testingphases, the develcrprnerR

teamo_ed thehighcoeval/effort ov=head inctmedby thenested

archi_

Given theselessonsfrom GRODY, theGOADA and GOESIM

design teams developed a non-nested architecture with the twin

goals of increasing rensab/lity and ndu6.ng compilation overhead.

Both these goals were met. Individ_l software comlxmems could

be picked up by successive projects and reused with slight

modifications. The use of library packages, rather than nested

packages, kept the compilation costs to a minimmn.

One of the lessons learned from the implementation phases of the

GOADA and GOESIM projects was that using Ade was not

significantly decreasing the level of effort for integration tesling.

Tkis was unexpected. It was predicu_l that the integr_iun test

phase would require less effort than past FORTRAN integration

test phase. Some AcLadevelopmenm have claimed that system

integration took significantly less effort than for simi_, previous

non-Ade projec_ [Hudson. 1988].

Antlysis by the UARSTELS design team showed that by unnesting GRODY's packages, more objects and types heeL,he visible

at a high level in the C-OADA design. This increased the number

of components to integrate at each level.

UARSTELS Architecture

The architectureof UARSTELS was influencedfrom the sum with

theknowledge that another,very similarsimulator would follow:

the Extreme Ultraviolet Explorer (EUVE) telemetry simulator

(EUVETELS). A high level of reuse from UARSTELS to

EUVETELS was both desired and thought to be possible because

of reused functional specifications. However, because the two

spacecral't were themselves different, the telemetry simulators

would be diffenmc The design for UARSTELS needed to take into

sccotug these spacecraft dcpendencim and parametcnz_d_m.

Each dmign decision made on UARSTELS attemptedtosatisfythe

followingrequirements:

1. All UARSTELS requirements

2. Some known requirements from previous systems

3. Some possible future mission requiremen_

The goalsofthe UARSTEL_ team were tomaximize verbatim

reuseand aLlow thecompilertochecksystemintegTafionasmuch

as possible.To achievethis.the designteam took a hybrid

approach to the system'sarchitecture.Most packages were

developedash'brarypackager,,ratherthannestedunits;however,

these packages were designed as generic units. This allowed the

masons of these genericpackages to be he.fred in succmsively

h/gher level packages. The level of nes_g (or layering) within

UARSTELS is comparable tothat ofGRODY, with the important

difference being the use of generic m_its in UARSTELS. The

generic packages may be picked up and reused just as the

nongeneric packages in GOADA or GOESIM, with the important

difference again being the use of generics. The nested

instanfiations allow the language to perform integration checks

(whethertheprogrammer wantsthem performed or not)ateach

compilation just as in GRODY.

- A specific example of _ nested insumci_ion approach is the

design of each simulated r_or modal within UARSTEL.S. In

H_u'e 8, the Repon_W_u_, Data_Se_ and Plot_File generic

packages are librm7 units. As rach, they may be picked up and

reusedind_y ofeachotlm'.Inthecaseors sensormodal,

howev_, each oftheseobjectsisnecessary.To provideallthree

of the_ ahsm,ctions to each sensor model they are instanfiated

within the specification of the genericpackage Sensor_Output.

The eppfication-specific parameters are provided to the _ree lowlevel generics when they are instantiated. The sensor.specific

parameters are generic formal parameters to Sensor Omput.

Sem_r_Ouqmt is then insumtisted within the body of the generic

sensor model package, with thesensor-specific parameters being

provided at that time. The spacecrah-specific parameters are

genericformalparametersof the genericsensormodel package.

The inm_._on ofthe generic sensor model package resolvesall

the formalparametm's, resultinginthe Fine Sun_Smmor object.

As a rmuk of this architecmze. UARSTELS differs from previous

systems in its recompilation time and executable size. The

increaseduseofgenaicmits had• directeffecton the compilation

overhead. Ittakeslonger,in CPU time and elapsedtime,to

recompile UARSTELS than any of the othersimulators.In

addidon,while UARSTELS is significantly smaller m sourcelines

of code andin number of componen_ it is alsosignificantly larger

in executable image size. This is because the Ada compiler used in

FDD doesnot supportcode sharing.Instead,itexpandsgeneric

units for eachinstanLiabon.

5642

4-15
FineSun

Sensor-3

; Sensor I_

,%;=;;'_'--..

; Oumut ,

_1,/#j, ...,

Figm'e 8.

UARSTELS Lessons Learned

The lessons lem'ned may be summarized as follows:

• Nesting re_c_ inm_'mion tinting

• Ldbm'y units provide reusable softw_ om'trpomm_

• Generic librm-y units provide reusable components and

allow inknmation hiding via nested inmmti_aions

• Many Ada implementations inca_ • large compilalion

time ov_ for the use of r_sted and genmqc traits

• Many Ada compilers provide a simple implemenuuion of

generic maim

The nestingfeature of Ada must be exploited. The virtue of

nesting is information l_ding(protection) and higher reliability;

this is one of the promises of using Adi. However. like all of

Ad_'s promises, higher reliability does not i_ppen autormuically.

It must be engineered.

The strong typing feature of Ade must also be exploited. The

definition of distinct types that are relevant to the application

dom_i,a, saw.has fligM dyrtamics, needs to be engineerad. Ada can

help elimbmm the common mistakes (i.e.., mixing radians and

degrees or meters and kilometers), but this will not occur

automatically either. Thi_ugh the use of sWongly typed objects,

reliability can be further improved and integrationtesting can be

fim_ autonm_

S_-ong typing encourages and. in most cases, forces the use of

nesting. Operations on private types must be defined within the

sarne scope (p_ck•ge) that det'mes the type. Since the internal

structure of that type is not visible outside this scope, all

operationsmust be def'med within the scope or theoperations must

be imported with a generic instantiation.

Cu'cumventmg nesting, strong typing, and generics in order to

min_ize compilation time is a short-term fix with the long-term

ramification o[ decreascd reliability and reusability. If the

compilation overhead is unacceptable, then alternative Ada

development environments will be required.

G_NSIM Overview

The GENSI_I projectwas sara'tedin 1986 by a groupstudying

ways toincreasethereuseofsimulatorsoftwareand thepossibility

of integzatingthedynamicsand telnmeu'ysimulationcapabilities.

Th_ group consisted of both software developers and mathematical

analysts, all of whom had simulator project experience. At the

same time. reuse studies in the Softw_e Engineering Laboratory

(SEL) [Solomon. 1987] showed that reusing code without

modification (verbatim reuse) yields a tremendous reduction in

development cost.One ofthe early products ofthe GENSIM effort

wu a study that estimated that costs of software development for •

dynamics simulator could be cut in half by creating verbat/m

remable cmnlxma_,

The GENSIM teem beLieved that the best approach tomaxJmizin8

verbatim reuse was to reuse products from allphases of the

softwareengineering life cycle. This belief was based on developer

experle_ rather tlum any formal software engineering theory.

The simulator problemwss divided into "modules," each ofwhich

models an entity in the problem domain. The products associated

with each module include a speci/'w.ation, design documantatiort.

cod_ madtest cases; each follows project-widestandards. A module

sp_ificat_a consists of a complete definition of the inputs and

outputs needed, the algorithms m be implemented to model the

entity, and documentation of the mathematical analysis and

atmmpfiom underlyingthealgorithm.A module designisbuilt

_.anding m stand_ protocols for ini_c_ comput_ons, mi

thepig of pmTmeu_sbetwee_modules. These protocolsallow

theindivklualmodules tobe config_ed_tm a standardsimulator

arc_mc:_.

To detesminethefeasibility of a genez_simulator,• prototypeis

beingdevelopedand appLiedto• simplifiedmission. Afterthe

prototypedemonstratesthe abilityto configure• dynamics

s_nulatorfordifferentmissions,• fullsetof components and

moduleswillbe developed.

GENSIM Arch_ec_ure

This subsection In'st describes the GI_SIM dynamics simulator

architeomre, then discusses design Lssues for both modules and

standard sul_stems. The next subsection discusses lessons learned

from the initial GENSLM design that can be applied to the final

v_sien ofthesystem.

Figure 9 shows the stand_d dymunies simulator arehitectme. The

reusable modules are par_ of the spacecraft, hardware, and

environment models (SHEM) suhsystem. Typical SHEM

modules include Sun sensor modeling and geomagnetic field

modeling. The spacecraftcontrolsubsystem isalways mission

dependent became a dymmius simulatorisintended to trot at_tude

control _gorithms for a specific satellite.. The only reusable pm"a

in this subsystem are a simulatedgn_undcomm_d uplin_ interface

and a module that computes esd.rnation and conu'oi errors. The

user inun'face has the obvious capabilities of reading and editing

input parameters and producing reports and plots from analysis

results. The case interface subsystem is responsible for

maintaining sLmulationcues, including analysis resultsdata.

simulation input parameters, and suspended simulation cases. The

5642

4-16
caseinterfacealsostandardizescommurdcationsbetweentheSHEM

modules and the user interface.The simulationexecutive

subsystemhas two majorpurposes:tomanage requeststocontrol

thesimulatorand tocon_'olthesequencingand timingofmodule

execution.The lastsubsystem isa utilities subsystem,which

consistsofseveralgenericunitsand a setof standard,interrehaed

instantiations, as is describedin Section2. For example,the

inatsntiafion of a generic linear algebra package requires a square

root function, which is provided by the instantiation of a general

mathenlaticaJfunctionspackage.

Figure 9.

Figure 10 shows the implemenmLion for a typicLl SHEM module

(Module K). The package Module_K perforrr_ the actual

modeling. Itcan be initializedand invokedRum thesimulmon

executive, and communicates with other SHEM modules through

procedme and function calls. The generic module dambme and

generic module results objects are gen_c packages that implement

the standard communications between a module and the other

subsystems.These genericsarcinsumtiatedwithtypesmd values

from the Module_KTypes package. These instanti_ons are caUed

on by the Module_K package when the model itself is being

initialized or ac6vated, and they are called on by case interface

compon_ts whenever parameu_ror results data is required by some

othersubsystem.Usingthisstandardappre_h allowsadifferent

sexofSHEM modules tobe usedforeachmission.

Dm_m

2

_ TY1_s

E

_)HoduieK

r Results_anager

I_erf_= Types

Figure 10.

The GENSIM de.signjust de.sen'bed is a conservative extension of

existing simulator designs. In genenl, the dynamics simulation

capability remains the same, but the design has been reworked to

be more object oriented. For example, the case interface

subsym:m was added to tre_ the conceptofsimulationcase as m

object, rather than to dism'bute tho_ c_abifities between the user

interface,the SI-_.M, and thautilities.The utilitiessubsystem

was alsochanged from one giganticgenericp_kage to several

smaller, independent gener/c units. The ma/n effort in GENSIM

has been directed toward generalizing the design to make the

componen_ reconfigurable, rath_ than _fi.ng new capabilities.

The userinterface,caseinterface,_ thesimulationexecutive

subsystemsmust be implementedas genericsubsys(emsthatcan

be parameterizedby theselectionofmodulesfora givenmission.

The caseinto-facesubsystemcm be used todemonstratehow a

genericsubsystem is designed. The discussionof individual

modulesinthepreviousparagraphshows how thegenericmodule

databL_and module resultspackagesareused by modules.The

case interfacesubsystem must accessthesesame instancesto

communical_withtheuse_and tomaintainsimulationcases,

Figure 11 shows the design for the generic case interface

subsystem. The case manager object is responsible for managing

simulation caSes as a unit; and the ground command interface,

parameter interface, and results interface objects manage

components (such as analysis results) of s simulation case. The

parameter interface and the results interface also mlmage the

communicationswithSHEM modules. FigureII shows thatall

thesepackagesareinterrelated; however,theyareused one ata

time. For example, a procedurethat editsg_md commands would

me thegroundcommand interfacebut nottheotherobjects.

4-17

5642
Case

Manager

Su_)endod

Results

Inmrfa_

Results

Figure 11.

If these objects were not being implemented as generics.

object in this subsystem would be implemented as • librm7

package, which could be imported independently. With • generic

subsystem, each of these packages must be partmeterized, snd

many of the generic formal paramete_ are common to more than

one package. If each object were implemented as a separate generic

package, there would be multiple definitions of the same formal

parameters with all the maintenance problems such a redundant

structure entails. Since the packages in this subsysunn are coupled

anyway, the case interface subsystem is implemented using the

nestedgeneticdefinitionstechnique(described in Section2 ofthis

paper), which allows the common parameters to be placed in the

generic part of the composite package.

In the GENSIM design, even the parameters that apply to only one

package were placed.in the composite package. When this is done,

the nested packages=n)no longer ganeric. This approach reduces

any possible confusionbetween generic packages and their

insumces by allowingthe userto instantiate the entire subsystem.

Then the nested packages can be used without having to insum_ate

more generics. The cost is that the nested packages are now more

highly coupled. In the case interface, this inert, reed coupling is

justifiedbecause all the coupled packages arc part ofthe ahsm¢_on

"simulationcase."The utilities subsystemconsisu ofindependent

genericpackages,where the couplingisintroducedbetween

instancesof thesegenerics.This spprolchallowsthegeneric

packagesto be used outsidethecontextof dynamicssimulators,

Thereisno correspondingneed touse individualcomponentsof

casein)rrfaccoutsidethecontextofdynamicssimulatorsbecause

thccouplingbetweenthecomponentsisdefinedby thenatureofa

simulationcase_The degreeofcouplingallowedinthedesign

implementationof reusablecomponents is one of the key

judgmentsdevelope_mustmake.

•, Suspended cases

", Parameters

Results

'"_4 Ground Commands

Interface

P

f ooo.oooooooo..j

Mis_0_ C-m_c •

Iam.f,_. Case :

Package Interface

ii-1.....

i

Hgm'e 12.

GENSIM Contributions

The major benefits derived from the GKNSIM project are in the

categories of (1) gaining experience in the use of edv_czd Adz

feanm_ (2) gening ideas for improvements in simula_r design,

and (3) producing the reusable componertts themselves. This

sul_se_c_ will focus on the first two cazegories. In the f'wn area,

the redefinition of theutilities subsystem as a set of independent

genericpackagestiedtogetherasasetof interrelatedinsmrttiadorts

demonsu,tted theabilityto w_ite genericpackages as descn'bed in

Secuon 2.

When designing a subsysmn this way, the developer must make

sure thaz all the generic formal parameters designed are matched by

actual parameters provided by some other package. It is also e.ases

development ff the code is written md tested bottom up,so that the

lowerlevel instenfiation._ needed to instamiate the sen/or-level

generics are tested and, in t'm-n, can be relied on to support the

testing of other object. If care is taken to design a set of generic

packages with consistent naming conventions, defaults can be

provided by stande_ instantiations, allowing the rapid writing of

instmtiatimm fortestingpurposes.When alltheseconditionsam

met, thetechniquesdeathbedin Section2 work very weLl.The

decoupledgenerics/coupledinstantiatious techniqueisthebest

approach to develop packages that provide the abilityto use

problem domain absu_tions rather than predet'med Ada coustr_ts.

GENSLM hasalsoconn'ibutedtotheuseofstrongtypingby using

more privatetypesthin previoussimulators and by beginningto

focuson thedecisioncriteriafortheiruse.The criteriaforusing

privatetypesisthatthey shouldadd thcprotectionofdataintegrity.

For example, the attitude types package defines the private Wpe

COSINEMATRIX. This typeisidenticalindatadefinitionto

5642

4-18
any other 3-by-3 matrix but has a set of operations that guarantee

thata COSINE_MATRIX alwaysrepresentsamtetion.Inthecase

ofGENSIM's orbitdatatypes,aprivatetypedoesnotadd any such

dhta protection; the effect is to force a user to use operations

provided for the data type in precisely the same way as one would

use an assignmentstatemenL When thisisthecase.the type

_tould be made visible in a package specification.

Possible Improvements to GENSIM

The GENSIM prototypIng has been successful in generalizing

dynamics simulator designs, but some features intmfited firom past

simulators can be improved on. Curr_dy, simulator module state

dam typesarebuilt horn individual scalarobjectsorarraysof scalar

objects.A more obJect-oriented designwould define-bstractdata

types (ADTs) for the problem domain entities and then use these

ADTs to dafme module statm. This is particularly true when there

are multiple objects of a type, as is the case with spacecraft

semor_. As an example, the current designs of a fine Sun sensor

model defines the simplified module state as follows:

package body Fine_Sun Smasor is

- N is the number of Free San Sensors used for a mission

type STATE is reco_

m#m_Anglm

: Double_Linear .Algebra.VECTOR( 1..N);

Bem_AnSks

: Double_Lincar_Algebru.VECTOR( 1..N);

AZp_t_Limit

: Double_Linear_AlgeaxLVE£TOR(1..N);

Bem Limit

: Double Line_ AIgcbrLVEL'q'OR(1 ..N);

axt_

Module_Stare : STATE;

end F'me_Sun_Semor_Module;

A butter implementation is as follows:

with F'me_Sun Sersor_ADT;

package body Free_Sun_Sensor Module is

type STATE is array of (1..N)

of Fine._Sun_Semor_ADT.HNE S UN_S ENS OR;

ModuleState : STATE;

,,,

end Fme_Sun_Sensor_M odul_

In the second implementation. Fine_Sun_Sensor_ADT

.FINE_SUN_SENSOR is an abstract data type that encapsulates

all thenecessaryattributes fora singlesensorand providesboth

selector and constructor operations. One advantage of using

abstract data types is the tighter encapsulation of data. If a change

is made to the f'me Sun sensor modeling, the soopo of the change

isthenrestricted tothebody oftheshstract datatypepackagerather

than affectingthe entire module.

Another advantageof usingabstractdatatypesinthiscontextis

thesmct separationof theproblemdomain objectitselfand itsuse

withina softwaresystem.Inthefirstexample,thedeclarationof

Alpha_Limit mixes the problem domain concept of a Iimitcd

sensorfield of view with the fact that N sensorsare used for a

particularmission.The second implementationmakes itclearthat

N refers to the number of sensorsand that the abstract data type

encapsulates eachsensor's angles and limits.

When the problem domain object (or class) is implemented with a

distinct Ada library unit. it is possible m use the object-oriented

programming concept of inheritance to create • hierarchy of classes

and subclaases. Figure 13 shows how this could work when all

the details of a fine Sun sensor model are considered. This

inheritance tre_ which is implemented usingnmted instantiations.

shows four levels of increasing complexity, starting with the

superclass FSS_ADT and creating a chainof subclassesfrom there.

Each of these four generics can be instantiated either as a library

unit or nested within a module. Each subclass in the chain tailors

its superclass by ixcorporating the models provided by the

respec_ve utility packages, htherited operations can also be

specialized and new operations can be =ideal to a package [Stark,

1987]. For example, a fine Stm sensor engineering model needs

to decalibrate simulated data so that cafibration £gorithms can be

tested. Since this decah'bration is specific to fine Sun sensors, the

operationwould be addedto thegenericFSS engine_ingmodel

package, with thenoise., biases, md misaLignmemsbeingprovided

by the generic measurement utilities.

i_ !,

i

1

• : ",GenmcFSS_ [

....".... ,: : :_=_ j I

• * .,% •

• t* • .

•, , ., :

f + ,o" : I : ---_..,," ,..,;-,,.... 'c,maz _ , c,amc ":

f ADT _ ' Utilities o

•* ,o:

:_ ..." ; maCom_on

: .,. %

._ ,,:

_'i _'...........

: Genmc

' "°" • Fafltm_Models

,/ ../

• ; f"6gg_'"i.. _ C.,atmc I , _ _ _, r_ot_c,uias &

F_ADT I ;........ LMliri= "_" "

Figtae13.

This approach uses the nested generic instantiations in the sexne

way as UARSTELS. The difference is that all sensor-spccific

utilities, such as f'me Sun sensor deca]ibration, are part of the

sensor abstract state machine, not part of the utility packages.

5642

4-19
The use of inheritance_llows theselectionof an appropriatemodel

for a wider varietyof applications.A telemetrysimulator would

typicallypick lhc l_lemetry model, a dynmnics simulator would

pick the hardware model, and erroranalysissoftware would use the

engineering model. The use ofinherit_nc_reduces the redmldmcy

between the differentapplications,which saves effortin both

development and run,romance,

The other area in which the simulator _ be made more genial is

the module types packages. These packages are not defined as

generic units, but they contain a mix of mission-specific

parameters (such as default initialconditions) and mission

independent parameters. The nestingof a genericpackage within

thetypes package isone possibleway tomake the sy_ e_ier to

configure. The nested generic would be paramemrized by the

mission dependencies, with the restof the types package remaining

mission independent. A libraryinstandation of the nesmd generic

would then be created to define the module's use for a particular

mission rather than to exwusively modify the types packages.

Figure 14 shows how this would work for a f'mc Sun sensor

module. The cost of this is that the othmr packages in a module

now need to import both the types package and the instance of the

nested generic,whereas only thetypes package wm needed before.

The stria separ_on of the parameu_l partfi'omthe consistent

isworth the added complexity.

( Mission Sl_cific_FSS_Dcfaults ) i ......

[ -."1

Figure 14.

4. FUTURE DIRECTIONS

The experiences of the UARSTELS and GENSIM projects have

demonstrated that the Ada lmguage, and pmicularly generics, can

be used to produce verbatim reusable components that can be fit

into more thin one archi_ctuae. Some other Ada language fcenn'es

need to be examined more closely to see how useful they axe for

simulation software. There has been a u-end to using more swong

typing as more experience is gained, but the FDD's Ada sohware

has not gone as far as the Common Ada Missile Packages

(CAMP) packages in using dislmcttypes, The CAMP packages

use a separate type for each unit of me,more, both in generic

parameter lists and in nongeneric code [Herr, 1988]. The

edvantsge ofusing _ degree of smmg typingisthatthe compiler

isable to catch any dimensionally incorrectcomputation. The

disadvantage is that overloaded operators need to be defined

amywhere thattwo or more differenttypes can be correctly used ina

computahon. A balance needs tobe found between the exl_mes of

CAMP and ofusing a single floating-pointtype as the basisof all

calculations.To do this,criteriamust be del-medfor the proper use

of Ada's typing features. When to use or not to use types,

subtypes,derivedtypes,orprivatetypesneeds clelrdeI'wJtion.

This plpefs discussion of inheritancefocuses on nested generic

instantiationsas a means of implementing the concept. An

alternateapproach isto use derived types to simulate inheritance

[Perez,1988]. In thesimulatorsdiscussedearlier,genericsareused

forboth pm'arne_erizationand forinheritance.To use derivedtypes

for inheritancewould require the investigationof the interaction

between paramelarizafionand in_ritanc¢ when differentlanguage

fesuues areused.

Genera/Conceptsfor Large-Scale Verbatim Reuse

The lessons learned by the UARSTELS and the GENSIM projects

have led us to a general reuse modeL TI_ model defines different

levels of reuse end which reuse-in-the-small techniques should be

spplled at which level. Figure 15 shows the leveled reuse model

on the left end typical examples on the right. As in most layered

models, the hJsher layers depend on services provided by the lower

ARCHITECTURE

LEVELS

PROBLEM

LEVELS.

layer.

!

LEVELS

• Sysmn Templaz_

;• Component

Templams

• DomainObv_s

and Classes

• Language Exumd/ng

Objects and Classes

EXAMPLE

DoublePrecision

FSS Module

hs_. T

Line_ Algebra

Figure 15.

The lowest laym" of the model is the langmge extension layer. This

layer's purpose is to create a goblem-specific language by adding

reusable Ade components to the existing capabilities of the Ada

language. In the flight dynamics donuun, this means det'ming

types and operationsfor mathematical constructssuch as vectors,

m_ziom_ and orbits. Appl_ttions code can them be developed

using the specialized capabilities rather than predet-med Ada

consm_s. Th_ level can be considered the state of the practice for

software reuse. The Bnoch components and the EVB GRACE

components _e at this level.

The language extension layer itself uses a layered approach. The

domain-specific objects are usually built on top of more general

objects. T'_ orbit data described above is specific to flight

dynamics, butit is represented as two vectors representing position

md velocity. When carried into design, an orbit data types package

would depend on a more general Linear slgebra package that exports

vector types.

The other important distinctionat thislevel is between entity

abstractionsand action abs_actions. An object with action

abstraction is completely described by what it does. A sort

package provides operations to sort data; a random number

generator generates random numbers. An obJect with entity

abstraction has atu'/bules beyond i_ set of computations. For

example, a queue can be described as a set of homogcncous dam

thatisaccessed and modified using a FIFO protocol.

5642

4-20
Figure 16 shows how the level of abstraction and level of

generality can be used to characterize language extension

comlxments. Some typicalsimulatorcomponents are chaxactenzed

by these two characteristics.The scale from domain specificto

general is more continuous titanisshown on thisdiagram. For

example,,a line& algebrapackage isspecifictothe mathematical

domain, but itisconsid_ed a genaral-pmlx_e package inthe flight

dynamics domain. Thus, itwould fallsomewhere inthe middle of

the scale.The distinctionbetween entityand actionabstractionis

more clear cut. If the object has relevant propertiesbeyond the

actionsitperforms, ithas entityabs'action. These propertiesare

seen in Adz code as state information that can be re,eyed and

modified by a package'• operations.

ENTITY ACTION

ABSTRACTIONS ABSTRACTIONS

Quatenuon Telemetry Encoding

DOMAJN Orbit Hanlwm, e F_ure

Models

Stacks,Queu=.

GENERAL Vectors, Matrices

Sorts, integrators,

Rmdom Numb=.

Genenmrs

Figure 16.

The next level of the model is the doma/n level . This is the level

at which the majorprobl_n domain mddes reside. State-of-the.art

reuse libraries such as the CAMP contain components th_ are

reused at this level [Herr, 1988]. Both the domain level and

language extension level consist of objects and classes. The

diff=.ence isthatthe objectsatthe domain leveld_ thepmbleen

domain, and the objects at the language extension level are •

means of expressing the model for • givan problem domain object

The fine Sun sensor abstract data types described in the previous

section are all problem domain entities. They are des_bed in

teams of vector and matrix algebra, and in terms of standard en'or

sources, telemetry encoding, and sensor failuremodels. The

generic packages for t'me Son sertsor data types implement the

domain entities using capabilities provided at the language

extension level

Figure 13 shows how a mix of domain entitiesand generic

language extensions can be used tobuild a kierarchyof cluses and

subclasses. The measurement udlltles,hardware utilities,and the

telemetry utilities are all language extensions, but they are used in

building the problem domain inheritance model

The next level of reuse isthe component template level, the level

at which generic components are builtto fitinto a given system

archimctme. The GENSIM SHEM modules md the UARSTELS

sensor models are examples of componem templates. Componen_

can be l_lt dJ.recdy from problem domain objects, or they can

provide indirect support. In GENSIM, the SHEM modules wig be

built around abslract dam types, such as those provided for the f'me

Sun ser_sors, and the standard module database and module results

packages that are instemiated to support the module. In addition to

these packages, a standard screen format file is used by the user

interfaceto allow user inputs for each SHEM module. The key

distinctionis thatthe component template level defines a/l the

components needed to fita problem domain object into a given

system architecture, where the domain level consists of a set of

objects that are not conslzained by a particular system design, but

only by the problem being solved.

The component template ]cvel objecL_ are also pararnctcnzcd, but

the emphasis slufts somewhat. The Free Sun sensor abstract data

types areparamcterized by data types forvectors md matrices and

by operations needed to intm'face with other problem domain

objects. The t-me Sun sensor module isparamelerized by items

such as the numb=" of sensors, the default input values, and

selections of which inputs a user is allowed to modify. Some

value.= of problem domain parameter= may be constrained at this

,level. Figure 15 give• the example of Double_Precision_FSS_

Module. This module has been constrained to use a particular

floating-point dam type. but it is still parameterized by the

number of sensors and default value.

The top level is the system template level. A generic system is a

reusable design into which individual components can be fit.

Objects at this level are parameterized by the set of componenLs

being used in a particularconfiguration and .by any other values

that have a system.wide effect. In GENSIM, the parameterization

of the gen=.iccase interface is related to the particular setof SHEM

modules being used. The simulation executive is parameterized

both by the set of components being used and by the spacecraft's

control mode& which affect how often thesecomponents need to be

executed.

The two template levels provide the capability of quickly building

a soRware system, l._e the langtmge exumsion and domain levels,

the capabilities provided by the lower level are used by the higher

one. The key distinction is that the lower two levels give a

complete definition of the problem domain, and the upper two

levels give a complete definition of a gens=afized software system

architeczme; It is important that the problem domain objects be

completely independent of pa,-xicolar system arch.itectt_es. To

ach/eve this, the lower two levels from Figure 15 are grotrped as

problem domain levels end the upper two are grouped as

aw.k/m:_e levelL

The discussion in thissectionhas focused on design issues,not

how AdA should be used to realizethese designs. The principles

thatapply toreuse in the small can be extended m reuse in the

large.A develop=, must stillbe conr2med about a mix of generic

packages and their instamiations, and the coupling between

components remains a key issue.

In the problem domain level the only coupling between objects

should be defined by the problem. The preferred means of linking

objectstogeth=, is to restrict dependencies to those bexween h'brary

instentiations.One previously mentioned exception to thisisthe

simulation of inheritance. Other relationships can also be

simulated through nested generic instantiations or nested generic

declarations.An example wh=.e nested inst-ntiationsare useful is

in the case where one object is built from simpl=, components, as

m inertial rdenmce unit (IRU) is built from gyroscopes. The IRU

presents a somewhat diff=.ent interftum thin • gyroscope, although

they are strongly related. The nested generic declarations are useful

when alternate models depend on the same objects or types. For

example, an orbit types package is parameterized in terms of

simple mathematical functions, but they are used by a variety of

different models for propagating orbits over time. Rather than

nesting insumtiationsof the orbit types within several different

models, the design=, can present the models as a set of options

thatdepend on thestone orbittypes.

Importing other libraryunits into generic units isnot a _oblem

when used for component templates or system templates. Figure

17 shows where the generic case interface package tmports the

instantiation of s generic types package. This interface types

5642

4-21
package provides standard data types forcommunication between

the us_, the stored simulation data. and the SHEM modules. The

deslgner should try to minimize this sort of coupling. In

GENSIM, only interface types and a common types package are

imported intogenericsin thismanner.

f Generic 1

Case

Interface

Interface Types )

J

Figure 17.

S. MEASURING THE EFFECT OF LARGE-SCALE

VERBATIM SOFTWARE REUSE

This sectiondiscusses the impact of the verbatim reuse on ixoject

management by describinghow costsam affectedand the effectsof

the layered model. A recent SEL study [Solomon. 1987]

characterized software compommts as being new, mbuih (greater

than 25-percent modification), adapted (up to 25-percem modified),

end v_irn (unmodified). Expressed as a Ix'ceentageof thecost of

a new component, the costs of the different types of reused

components are approximately :

Ve_atim 10 %

Adapted 3O%

Rebuilt 50 %

(actually7.2 %)

To make conservative estimates the 10-percent figure is used for

verba_-n components, and any nonverbatim component is assumed

to be new.

The GENSIM cost study [Mendelsohn. 1988] shows that the

current levels of reuse for dynamics simulators save 15 to 20

percent over all-new systems. The study also determined that

dyno,mics simulators have a potentialfor about 70- to80-percem

verbatim reuse; only the spacecraft control system code is

developed from scratch for each mission. These verbatim reuse

levels translate to a cost savings of from 60 to 70 pe-rccnt over an

sJl-new system or at lea.st 50 percent from current systems.

The key to achieving high levels of verbatim reuse is-to reuse

specifications and design. The analysts who define the

requirements for FDD systems developed common mathcrnaticai

specificationsfor allsystems supporting EUVE and UARS. The

currentestimate for EUVETEL.S reuse from UARSTELS is87

percent,which tr_mslatesto approximately 80-percc'ntcost savings

over a new system. Even the FORTRAN software supporting

EUVE has a reuse level of from 60 to 70 percent from UARS.

whereas typical levels fall into the 20- to 30-percent range. The

increase from reusing the mathematical specifications is much

gre.a_ than the increatse observed as the result of using Ada as the

implementation language for simulators [Brechbiel. 1989]. These

dam confu'm tlm correcmess of GF,.NSIM's use of a set ofstand._

mathematical specifications.

In addition tomeasuring the levelof verbatim muse, the effectof

verbatim reuse can be divided into the reuse of problem domain

comlx_nents and the reuse of components ax the architecture levels.

No FDD simulators have been developed using the proposed reuse

model, so the estimate will be based on the fact that the user

intmfac¢ for the dynamics simulators typicallycontains40 percent

of the solace lines of code and no problem domain objects. Since

the capabilities of the GENSIM simulation executive and case

interface subsystems ate currently distributed among other

subsystans. 40 percent is a conservative estimate. It is probably

correct to assert thaz the benefits of reusable architectures equal or

exceed those of developing reusable problem domain components.

It is dear that these benefits m'e roughly equal

6. MANAGEMENT RECOMMENDATIONS

The primary management recommendation is to build the problem

domain levels first and to build them bottom up. The language

extension layer is a means of extrusion for domain objects and

classes. The domain objects and classes serve as the building

blocks for reusable system &clu'le,ctmes. Another advantage of

building the problem domain layers Rrst is the abili_ to build

multiple architecturesfrom the same set of problem domain

objects. For simulation applications, this means that the same

setof problem domain objectscould be used to build a dynamics

simulator, a telemetry simulator, or a combined dynamics and

telemetrysimulator.

The su-ictseparationofproblem layersfi'vmarchitecr2relayersalso

provides the means of keeping up with technology. The same

domain objectswould be usable on either an 8086 bas_ computer

with a monochrome textscreen or on an g0386-based computer

with high.resolution graphics. The architectureof the system

would be changexL although itwould probably not be rebuiltfrom

scratch. The archimcture of a system should bc driven by

technology, and the solution of flight dynamics problems should

not be. The separation of these considerat/ons in design makes -it

easiertomange mchnological change.

7. CONCLUSIONS

The currentstateof the artinsoftware reuse istoprovide problem

domain components and problem domain objects. This paper has

demonstrated thatdesigning vcrbatLm reusablecomponcnLs at the

architecture level can create approxonately the same savings as the

currentstateof the art. The new approach that needs tobe applied

5642

4-22
tofuturesystemsistosnictlyseparatetheproblemdomainobjects

from thepa.rtscular systemarchitectures_nd tobuildtheproblem

dommn layersfrom thebottom-up.When b_isappro-,'hLsusedto

developverbazimreusablesoftware,there.sourcessavedcan be

appliedto new problems (extendingtheproblem dormtin)or to

provide better solutions m existing problems by upgrading the

azr.hitecture.

Stark.M.. md E.Seid_witz."rowLrdsaGencrLlObjcct-Oricntcd

Ad= LifeCycle."Proceedingsoft_ JointAda Conference.Mm'ch

1987

REFERENCES

Ber_d. E.."ReusabilityTutorial."ProceedingaoftheWm_gton

Ada Syrnpoaium.1989

Booch, G.,SoftwareEngineeringWith Ada. Menlo Park.CA:

Benjamin Cummings, 1987

Booch. G.,SoftwareComponents With Ada. Menlo Park.CA:

BenjaminCummings, 1987

Brechbie.I, F. "Ada md Spec_cadan ReuseVersusSoftweaeCost,

ReLiability,and Reusabilityin a FlightDynamics Support

Environment."(m bepublished)

Ganapathi M., and G. Mendel, "Issuesof Ada Compiler

Technology,"IEEE Computer,February1989,voL 22,no.2,pp.

52-6O

Herr. C.. D. McNicholL and S. Cohen, "Compi]_ Validation ar-,d

Reusable Ada Parmfor ReJ.l-Tbn_ EmbeddedApplicatic¢_" ACM

SIGAda Aria Letters, Septembe_/octaber 1988, voL VI]J, no. 5,

pp. 75-86

Hudson. W., "Adz Compilea Development," Defense Science,

March 1988, pp. 59-64

MendLl, G., 'ThreeReasons To Avoid theUse Clame," ACM

SIGAda Ada Letters,January/February1988,voL _ no. I,pp.

52-57

Mendclsohn.C, "Impact Study ofGeneric SimPerer Software

(GENSIM) on Animde Dynamics Simulator Development Within

the Systems Development Brtnch," (unpublishedstudy of

sirnutmarreusedata)

Perez, E. "Simulating Inheritance With Adz," ACM SlGAda Ada

Letters, September/October 1988, voi. VIII, no. 5. pp. 37-46

Quimby, K., "Evolutionof Ada Technology in the Flight

Dynamics Area: Design Phase Analysis," Software Engineering

Laboratory,SEL-88-003, December 1988

Seidewitz. E.. and M. Stark. "Towards a General Object-Oriented

Development Methodology," Proceedings of the First International

Symposium on Ariafor t_ NASA SpaceStation,June 1986

Seidewitz, F-. "General Object-Oriented SoftwareDevelopment

With Aria: A L._fe Cycle Approach," Proceedings of th_ CASE

Technology Conference, April 1988

Solomon, D.,and W. Agresti."Profileof SoftwareReuse inthe

FlightDynamics Environment,"Computer SciencesCorporation,

CscfrM-87/6062,November 1987

5642

4-23

STANDARD BIBLIOGRAPHY OF SEL LITERATURE

STANDARD BIBLIOGRAPHY OF SEL LITERATURE

The technical papers, memorandums, and documents listed in

this bibliography are organized into two groups. The first

group is composed of documents issued by the Software Engineering Laboratory (SEL) during its research and development

activities. The second group includes materials that were

published elsewhere but pertain to SEL activities.

SEL-ORIGINATED DOCUMENTS

SEL-76-001, Proceedinqs From the First Summer Software Enqineerinq Workshop, August 1976

SEL-77-002, Proceedings From the Second Summer Software Enqineering Workshop, September 1977

SEL-77-004, A Demonstration of AXES for NAVPAK, M. Hamilton

and S. Zeldin, September 1977

SEL-77-005, GSFC NAVPAK Desiqn Specifications Lanquaqes

StudM, P. A. Scheffer and C. E. Velez, October 1977

SEL-78-005, Proceedings From the Third Summer Software Enqineerinq Workshop, September 1978

SEL-78-006, GSFC Software Enqineering Research Requirements

Analysis Study, P. A. Scheffer and C. E. Velez, November 1978

SEL-78-007, Applicability of the Rayleiqh Curve to the SEL

Environment, T. E. Mapp, December 1978

SEL-78-302, FORTRAN Static Source Code Analyzer Proqram

(SAP) User's Guide (Revision 3), W. J. Decker and

W. A. Taylor, July 1986

SEL-79-002, The Software Enqineerinq Laboratory: Relationship Equations, K. Freburger and V. R. Basili, May 1979

SEL-79-003, Common Software Module Repository (CSMR) System

Description and User's Guide, C. E. Goorevich, A. L. Green,

and S. R. Waligora, August 1979

SEL-79-004, Evaluation of the Caine, Farber, and Gordon Program Design Lanquage (PDL) in the Goddard Space Flight Cen

ter (GSFC) Code 580 Software Design Environment,

C. E. Goorevich, A. L. Green, and W. J. Decker, September

1979

B-I

5642
SEL-79-005, Proceedings From the Fourth Summer Software Enqineering Workshop, November 1979

SEL-80-002, Multi-Level Expression Design LanquageRequirement Level (MEDL-R) System Evaluation, W. J. Decker

and C. E. Goorevich, May 1980

SEL-80-003, Multimissi0n Modular Spacecraft Ground Support

Software System (MMS/GSSS) State-of-the-Art Computer Systems/

Compatibility Study, T. Welden, M. McClellan, and

P. Liebertz, May 1980

SEL-80-005, A Study of the Musa Reliability Model,

A. M. Miller, November 1980

SEL-80-006, Proceedings From the Fifth Annual Software Enqineerinq Workshop, November 1980

SEL-80-007, An Appraisal of Selected Cost/Resource Estimation Models for Software Systems, J. F. Cook and

F. E. McGarry, December 1980

SEL-81-008, Cost and Reliability Estimation Models (CAREM)

User's Guide, J. F. Cook and E. Edwards, February 1981

SEL-81-009, Software Engineering Laboratory Proqrammer Workbench Phase 1 Evaluation, W. J. Decker and F. E. McGarry,

March 1981

SEL-81-011, Evaluatinq Software Development by Analysis of

Chanqe Data, D. M. Weiss, November 1981

SEL-81-012, The Rayleiqh Curve as a Model for Effort Distribution Over the Life of Medium Scale Software Systems,

G. O. Picasso, December 1981

SEL-81-013, Proceedings From the Sixth Annual Software Enqineering Workshop, December 1981

SEL-81-014, Automated Collection of Software Enqineering

Data in the Software Engineering Laboratory (SEL),

A. L. Green, W. J. Decker, and F. E. McGarry, September 1981

SEL-81-101, Guide to Data Collection, V. E. Church,

D. N. Card, F. E. McGarry, et al., August 1982

SEL-81-104, The Software Enqineering Laborat6ry, D. N. Card,

F. E. McGarry, G. Page, et al., February 1982

5642

B-2
SEL-81-107, Software Engineering Laboratory (SEL) Compendium

of Tools, W. J. Decker, W. A. Taylor, and E. J. Smith,

February 1982

SEL-81-110, Evaluation of an Independent Verification and

Validation (IV&V) Methodology for Fliqht Dynamics, G. Page,

F. E. McGarry, and D. N. Card, June 1985

SEL-81-205, Recommended Approach to Software Development,

F. E. McGarry, G. Page, S. Eslinger, et al., April 1983

SEL-82-001, Evaluation of Management Measures of Software

Development, G. Page, D. N. Card, and F. E. McGarry,

September 1982, vols. 1 and 2

SEL-82-004, Collected Software Engineerinq Papers:

ume l, July 1982

Vo ISEL-82-007, Proceedings From the Seventh Annual Software

Engineering Workshop, December 1982

SEL-82-008, Evaluating Software Development by Analysis of

Chanqes: The Data From the Software Enqineerinq Laboratory,

V. R. Basili and D. M. Weiss, December 1982

SEL-82-I02, FORTRAN Static Source Code Analyzer Program

(SAP) System Description (Revision i), W. A. Taylor and

W. J. Decker, April 1985

SEL-82-I05, Glossary of Software Enqineerinq Laboratory

Terms, T. A. Babst, F. E. McGarry, and M. G. Rohleder,

October 1983

SEL-82-806, Annotated Biblioqraphy of Software Engineerinq

Laboratory Literature, M. Buhler and J. Valett, November 1989

SEL-83-001, An Approach to Software Cost Estimation,

F. E. McGarry, G. Page, D. N. Card, et al., February 1984

SEL-83-002, Measures and Metrics for Software Development,

D. N. Card, F. E. McGarry, G. Page, et al., March 1984

SEL-83-003, Collected Software Enqineering Papers:

ume II, November 1983

VolSEL-83-006, Monitorinq Software Development Through Dynamic

Variables, C. W. Doerflinger, November 1983

5642

B-3
SEL-83-007, Proceedinqs From the Eiqhth Annual Software Enqineerinq Workshop, November 1983

SEL-84-001, Manager's Handbook for Software Development,

W. W. Agresti, F. E. McGarry, D. N. Card, et al., April 1984

SEL-84-003, Investiqation of Specification Measures for the

Software Engineering Laboratory (SEL), W. W. Agresti,

V. E. Church, and F. E. McGarry, December 1984

SEL-84-004, Proceedings From the Ninth Annual Software Engineerinq Workshop, November 1984

SEL-85-001, A Comparison of Software Verification Techniques, D. N. Card, R. W. Selby, Jr., F. E. McGarry, et al.,

April 1985

SEL-85-002, Ada Traininq Evaluation and Recommendations From

the Gamma Ray Observatory Ada Development Team, R. Murphy

and M. Stark, October 1985

SEL-85-003, Collected Software Engineering Papers:

ume III, November 1985

VolSEL-85-004, Evaluations of Software Technologies: Testing,

CLEANROOM, and Metrics, R. W. Selby, Jr., May 1985

SEL-85-005, Software Verification and Testing, D. N. Card,

C. Antle, and E. Edwards, December 1985

SEL-85-006, Proceedinqs From the Tenth Annual Software Enqineering Workshop, December 1985

SEL-86-001, Programmer's Handbook for Fliqht Dynamics Software Development, R. Wood and E. Edwards, March 1986

SEL-86-002, General Object-Oriented Software Development,

E. Seidewitz and M. Stark, August 1986

SEL-86-003, Fliqht Dynamics System Software Development Environment Tutorial, J. Buell and P. Myers, July 1986

SEL-86-004, Collected Software Engineering Papers:

ume IV, November 1986

VOISEL-86-005, Measuring Software Design, D. N. Card, October

1986

5642

B-4
SEL-86-006, Proceedings From the Eleventh Annual Software

Enqineering Workshop, December 1986

SEL-87-001, Product Assurance Policies and Procedures for

Flight Dynamics Software Development, S. Perry et al., March

1987

SEL-87-002, Ada Style Guide (Version 1.1), E. Seidewitz

et al., May 1987

SEL-87-003, Guidelines for Applying the Composite Specification Model (CSM), W. W. Agresti, June 1987

SEL-87-004, Assessinq the Ada Desiqn Process and Its Implications: A Case Study, S. Godfrey, C. Brophy, et al.,

July 1987

SEL-87-008, Dat_ Collection Procedures for the Rehosted SEL

Database, G. Heller, October 1987

SEL-87-009, C011ected Software Engineering Papers:

S. DeLong, November 1987

Volume V,

SEL-87-010, Proceedings From the Twelfth Annual Software Enqineering Workshop, December 1987

SEL-88-001, System Testing of a Production Ada Project: The

GRODY Study, J. Seigle, L. Ester, and Y. Shi, November 1988

SEL-88-002, C011ected Software Engineering Papers:

ume VI, November 1988

VolSEL-88-003, Eyo!ution of Ada Technology in the Flight Dynamics Area: Desiqn Phase Analysis, K. Quimby and L. Esker,

December 1988

SEL-88-004, Proceeding of the Thirteenth Annual Software

EngineeriNg Workshop, November 1988

SEL-88-005, Proceedings of the First NASA Ada User's Symposium, December 1988

SEL-89-001, Software Engineering Laboratory (SEL) Data Base

Organization and User's Guide, M. So et al., May 1989

SEL-89-002, Implementation of a Production Ada Project:

GRODY Study, S. Godfrey and C. Brophy, May 1989

The

SEL-89-003, Software Management Environment (SME) Concepts

and Architecture, W. Decker and J. Valett, August 1989

5642

B-5
SEL-89-004, Evolution Of Ada Technolouv in the Flight Dynamics Area: ImDlementation/T@stinq Phase Analysis,

K. Quimby, L. Esker, L. Smith, M. Stark, and F. McGarry,

November 1989

SEL-89-005, Lessons Learned in th@ Tr_nzition tO Ada from

FORTRAN at NASA/Goddard, C. Brophy, November 1989

SEL-89-006, Collected SOftware Engineering Papers:

um_ VII, November 1989

VolSEL-RELATED LITERATURE

4Agresti, W. W., V. E. Church, D. N. Card, and P. L. Lo,

"Designing With Ada for Satellite Simulation: A Case Study,"

Proceedinqs of the First International Symposium on Ada for

the NASA Space Station, June 1986

2Agresti, W. W., F. E. McGarry, D. N. Card, et al., "Measuring Software Technology," Program Transformation and Proqramminq Environments. New York: Springer-Verlag, 1984

iBailey, J. W., and V. R. Basili, "A Meta-Model for Software Development Resource Expenditures," Proceedings of the

Fifth International Conference on Software Enqineering.

New York: IEEE Computer Society Press, 1981

7Basili, V. R., Maintenance = Reuse-0riented Software

Development, University of Maryland, Technical Report

TR-2244, May 1989

iBasili, V. R., "Models and Metrics for Software Management and Engineering," ASME Advances in Computer Technology,

January 1980, vol. 1

7Basili, V. R., Software Development; A Paradiqm for the

Future, University of Maryland, Technical Report TR-2263,

June 1989

Basili, V. R., Tutorial on Models and Metrics for Software

Management and Enqineering. New York: IEEE Computer Society

Press, 1980 (also designated SEL-80-008)

3Basili, V. R., "Quantitative Evaluation of Software Methodology," Proceedinas of the First Pan-Pacific Computer Conference, September 1985

iBasili, V. R., and J. Beane, "Can the Parr Curve Help

With Manpower Distribution and Resource Estimation Problems?," Journal of Systems _nd Software, February 1981,

vol. 2, no. 1

B-6

5642
iBasili, V. R., and K. Freburger, "Programming Measurement

and Estimation in the Software Engineering Laboratory,"

Journal of Systems and Software, February 1981, vol. 2, no. 1

3Basili, V. R., and N. M. Panlilio-Yap, "Finding Relationships Between Effort and Other Variables in the SEL,"

Proceedings of the International Computer Software and Applications Conference, October 1985

4Basili, V. R., and D. Patnaik, A Study on Fault Prediction

and Reliability Assessment in the SEL Environment, University

of Maryland, Technical Report TR-1699, August 1986

2Basili, V. R., and B. T. Perricone, "Software Errors and

Complexity: An Empirical Investigation," Communications of

the ACM, January 1984, vol. 27, no. 1

iBasili, V. R., and T. Phillips, "Evaluating and Comparing

Software Metrics in the Software Engineering Laboratory,"

Proceedings of the ACM SIGMETRICS Symposium/Workshop: Quality Metrics, March 1981

Basili, V. R., and J. Ramsey, Structural Coverage of Functional Testing, University of Maryland, Technical Report

TR-1442, September 1984

3Basili, V. R., and C. L. Ramsey, "ARROWSMITH-P--A Prototype Expert System for Software Engineering Management,"

Proceedinqs of the IEEE/MITRE Expert Systems in Government

Symposium, October 1985

Basili, V. R., and R. Reiter, "Evaluating Automatable Measures for Software Development," Proceedinqs of the Workshop

on Quantitative Software Models for Reliability, Complexity,

and Cost. New York: IEEE Computer Society Press, 1979

5Basili, V., and H. D. Rombach, "Tailoring the Software

Process to Project Goals and Environments," Proceedinqs of

the 9th International Conference on Software Enqineerinq,

March 1987

5Basili, V., and H. D. Rombach, "T A M E: Tailoring an Ada

Measurement Environment," Proceedinqs of the Joint Ada Conference, March 1987

5Basili, V., and H. D. Rombach, T A M E: Integratinq

Measurement Into Software Environments, University of

Maryland, Technical Report TR-1764, June 1987

5642

B-7
6Basili, V. R., and H. D. Rombach, "The TAME Project:

Towards Improvement-Oriented Software Environments," IEEE

Transactions on Software Engineering, June 1988

7Basili, V. R., and H. D. Rombach, Towards A Comprehensive

FramewQrk for Reuse: A Reuse-Enabling Software Evolution

Environment, University of Maryland, Technical Report

TR-2158, December 1988

2Basili, V. R., R. W. Selby, Jr., and T. Phillips, "Metric

Analysis and Data Validation Across FORTRAN Projects," IEEE

Transactions on Software Engineering, November 1983

3Basili, V. R., and R. W. Selby, Jr., "Calculation and Use

of an Environments's Characteristic Software Metric Set,"

Proceedings of the Eighth International Conference on Software Engineering. New York: IEEE Computer Society Press,

1985

Basili, V. R., and R. W. Selby, Jr., Comparing the Effectiveness of Software Testing Strategies, University of Maryland,

Technical Report TR-1501, May 1985

3Basili, V. R., and R. W. Selby, Jr., "Four Applications

of a Software Data Collection and Analysis Methodology," Proceedinqs of the NATO Advanced Study Institute, August 1985

4Basili, V. R., R. W. Selby, Jr., and D. H. Hutchens, "Experimentation in Software Engineering," IEEE Transactions on

Software Engineering, July 1986

5Basili, V. and R. Selby, Jr., "Comparing the

Effectiveness of Software Testing Strategies," IEEE

Transactions on Software Engineering, December 1987

2Basili, V. R., and D. M. Weiss, A Methodoloqy for Collecting

Valid Software Engineering Data, University of Maryland,

Technical Report TR-1235, December 1982

3Basili, V. R., and D. M. Weiss, "A Methodology for Collecting Valid Software Engineering Data," IEEE Transactions on

Software Engineering, November 1984

IBasili, V. R., and M. V. Zelkowitz, "The Software Engineering Laboratory: Objectives," Proceedings of the Fifteenth Annual Conference on Computer Personnel Research,

August 1977

Basili, V. R., and M. V. Zelkowitz, "Designing a Software

Measurement Experiment," Proceedings of the Software Life

Cycle Manaqement Workshop, September 1977

5642

B-8
iBasili, V. R., and M. V. Zelkowitz, "Operation of the Software Engineering Laboratory," Proceedings of the Second Software Life Cycle Manaqement Workshop, August 1978

iBasili, V. R., and M. V. Zelkowitz, "Measuring Software

Development Characteristics in the Local Environment," Computers and Structures, August 1978, vol. 10

Basili, V. R., and M. V. Zelkowitz, "Analyzing Medium Scale

Software Development," Proceedings of the Third International Conference on Software Engineerinq. New York: IEEE

Computer Society Press, 1978

5Brophy, C., W. Agresti, and V. Basili, "Lessons Learned

in Use of Ada-Oriented Design Methods," Proceedinqs of the

Joint Ada Conference, March 1987

6Brophy, C. E., S. Godfrey, W. W. Agresti, and V. R. Basili,

"Lessons Learned in the Implementation Phase of a Large Ada

Project," Proceedinqs of the Washinqton Ada Technical Conference, March 1988

2Card, D. N., "Early Estimation of Resource Expenditures and

Program Size," Computer Sciences Corporation, Technical Memorandum, June 1982

2Card, D. N., "Comparison of Regression Modeling Techniques

for Resource Estimation," Computer Sciences Corporation,

Technical Memorandum, November 1982

3Card, D. N., "A Software Technology Evaluation Program,"

Annais do XVIII Conqresso Nacional de Informatica,

October 1985

5Card, D., and W. Agresti, "Resolving the Software Science

Anomaly," The Journal of Systems and Software, 1987

6Card, D. N., and W. Agresti, "Measuring Software Design

Complexity," The Journal of Systems and Software, June 1988

Card, D. N., V. E. Church, W. W. Agresti, and Q. L. Jordan,

"A Software Engineering View of Flight Dynamics Analysis

System," Parts I and II, Computer Sciences Corporation,

Technical Memorandum, February 1984

4Card, D. N., V. E. Church, and W. W. Agresti, "An Empirical

Study of Software Design Practices," IEEE Transactions on

Software Engineerinq, February 1986

5642

B-9
Card, D. N., Q. L. Jordan, and V. E. Church, "Characteristics of FORTRANModules," Computer Sciences Corporation,

Technical Memorandum, June 1984

5Card, D., F. McGarry, and G. Page, "Evaluating Software

Engineering Technologies," IEEE Transactions on Software

Enqineerinq, July 1987

3Card, D. N., G. T. Page, and F. E. McGarry, "Criteria for

Software Modularization," Proceedinqs of the Eighth International Conference on Software Engineering. New York: IEEE

Computer Society Press, 1985

iChen, E., and M. V. Zelkowitz, "Use of Cluster Analysis

To Evaluate Software Engineering Methodologies," Proceedings

of the Fifth International Conference on Software Engineering. New York: IEEE Computer Society Press, 1981

4Church, V. E., D. N. Card, W. W. Agresti, and Q. L. Jordan,

"An Approach for Assessing Software Prototypes," ACM Software

Engineering Notes, July 1986

2Doerflinger, C. W., and V. R. Basili, "Monitoring Software

Development Through Dynamic Variables," Proceedings of the

Seventh International Computer Software and Applications

Conference. New York: IEEE Computer Society Press, 1983

5Doubleday, D., ASAP: An Ada Static Source Code Analyzer

Proqram, University of Maryland, Technical Report TR-1895,

August 1987 (NOTE: I00 pages long)

6Godfrey, S., and C. Brophy, "Experiences in the Implementation of a Large Ada Project," Proceedinqs of the 1988

Washington Ada Symposium, June 1988

Hamilton, M., and S. Zeldin, A Demonstration of AXES for

NAVPAK, Higher Order Software, Inc., TR-9, September 1977

(also designated SEL-77-005)

Jeffery, D. R., and V. Basili, Characterizing Resource

Data: A Model for Loqical Association of Software Data,

University of Maryland, Technical Report TR-1848, May 1987

6jeffery, D. R., and V. R. Basili, "Validating the TAME

Resource Data Model," Proceedinqs of the Tenth International

Conference on Software Enqineerinq, April 1988

5Mark, L., and H. D. Rombach, A Meta Information Base for

Software Engineering, University of Maryland, Technical Report TR-1765, July 1987

5642

B-10
6Mark, L., and H. D. Rombach, "Generating Customized Software

Engineering Information Bases From Software Process and Product Specifications," Proceedings of the 22nd Annual Hawaii

International Conference on System Sciences, January 1989

5McGarry, F., and W. Agresti, "Measuring Ada for Software

Development in the Software Engineering Laboratory (SEL),"

Proceedings of the 21st Annual Hawaii International Conference on System Sciences, January 1988

7McGarry, F., L. Esker, and K. Quimby, "Evolution of Ada

Technology in a Production Software Environment," Proceedinqs of the Sixth Washington Ada Symposium (WADAS), June 1989

3McGarry, F. E., J. Valett, and D. Hall, "Measuring the

Impact of Computer Resource Quality on the Software Development Process and Product," Proceedings of the Hawaiian International Conference on System Sciences, January 1985

National Aeronautics and Space Administration (NASA), NASA

Software Research Technology Workshop (Proceedings), March

1980

3page, G., F. E. McGarry, and D. N. Card, "A Practical Experience With Independent Verification and Validation,"

Proceedings of the Eighth International Computer Software

and Applications Conference, November 1984

5Ramsey, C., and V. R. Basili, An Evaluation of Expert Systems for Software Engineering Management, University of

Maryland, Technical Report TR-1708, September 1986

3Ramsey, J., and V. R. Basili, "Analyzing the Test Process

Using Structural Coverage," Proceedings of the Eighth International Conference on Software Engineering. New York:

IEEE Computer Society Press, 1985

5Rombach, H. D., "A Controlled Experiment on the Impact of

Software Structure on Maintainability," IEEE Transactions on

Software Enqineerinq, March 1987

6Rombach, H. D., and V. R. Basili, "Quantitative Assessment

of Maintenance: An Industrial Case Study," Proceedings From

the Conference on Software Maintenance, September 1987

6Rombach, H. D., and L. Mark, "Software Process and Product Specifications: A Basis for Generating Customized SE

Information Bases," Proceedings of the 22nd Annual Hawaii

International Conference on System Sciences, January 1989

5642

B-II
7Rombach, H. D., and B. T. Ulery, Establishinq a Measurement Based Maintenance Improvement Proqram: Lessons Learned

in the SEL, University of Maryland, Technical Report

TR-2252, May 1989

5Seidewitz, E., "General Object-Oriented Software Development: Background and Experience," Proceedinqs of the 21st

Hawaii International Conference on System Sciences, January

1988

6Seidewitz, E., "General Object-Oriented Software Development with Ada: A Life Cycle Approach," Proceedinqs of the

CASE Technoloqy Conference, April 1988

6Seidewitz, E., "Object-Oriented Programming in Smalltalk

and Ada," Proceedings of the 1987 Conference on ObjectOriented Proqramming Systems, Lanquaqes, and Applications,

October 1987

4Seidewitz, E., and M. Stark, "Towards a General ObjectOriented Software Development Methodology," Proceedinqs of

the First International Symposium on Ada for the NASA Space

Station, June 1986

7Stark, M. E. and E. W. Booth, "Using Ada to Maximize

Verbatim Software Reuse," Proceedinqs of TRI-Ada 1989,

October 1989

Stark, M., and E. Seidewitz, "Towards a General ObjectOriented Ada Lifecycle," Proceedinqs of the Joint Ada Conference, March 1987

7Sunazuka. T., and V. R. Basili, Inteqratinq Automated

Support for a Software Manaqement Cycle Into the TAME Syste____mm,University of Maryland, Technical Report TR-2289, July

1989

Turner, C., and G. Caron, A Comparison of RADC and NASA/SET.

Software Development Data, Data and Analysis Center for

Software, Special Publication, May 1981

Turner, C., G. Caron, and G. Brement, NASA/SEL Data Compendiu_mm, Data and Analysis Center for Software, Special Publication, April 1981

5Valett, J., and F. McGarry, "A Summary of Software Measurement Experiences in the Software Engineering Laboratory,"

Proceedinqs of the 21st Annual Hawaii International Conference on System Sciences, January 1988

5642

B-12
3Weiss, D. M., and V. R. Basili, "Evaluating Software Development by Analysis of Changes: Some Data From the Software Engineering Laboratory," IEEE Transactions on Software

Engineering, February 1985

5Wu, L., V. Basili, and K. Reed, "A Structure Coverage Tool

for Ada Software Systems," Proceedings of the Joint Ada Conference, March 1987

iZelkowitz, M. V., "Resource Estimation for Medium Scale

Software Projects," Proceedings of the Twelfth Conference on

the Interface of Statistics and Computer Science. New York:

IEEE Computer Society Press, 1979

2Zelkowitz, M. V., "Data Collection and Evaluation for Experimental Computer Science Research," Empirical Foundations

for Computer and Information Science (proceedings),

November 1982

6ZelkowitZ, M. V., "The Effectiveness of Software Prototyping: A Case Study," Proceedings of the 26th Annual Technical Symposium of the Washington, D. C., Chapter of the ACM,

June 1987

6Zelkowitz, M. V., "Resource Utilization During Software

Development," Journal of Systems and Software, 1988

Zelkowitz, M. V., and V. R. Basili, "Operational Aspects of

a Software Measurement Facility," Proceedings of the Software Life Cycle Management Workshop, September 1977

NOTES:

iThis article also appears in SEL-82-004, Collected Software Enqineerinq Papers: Volume I, July 1982.

2This article also appears in SEL-83-003, Collected Software Engineering Papers: Volume If, November 1983.

3This article also appears in SEL-85-003, Collected Software Enqineering Papers: Volume III, November 1985.

4This article also appears in SEL-86-004, Collected Software Engineering Papers: Volume IV, November 1986.

5This article also appears in SEL-87-009, Collected Software Enqineering Papers: Volume V, November 1987.

5642

B-13
6This article also appears in SEL-88-002, Collected Software Enqineerinq Papers: Volume VI, November 1988.

7This article also appears in SEL-89-006, Collected Software Enqineering Papers: Volume VII, November 1989.

5642

B-14