ENGINEERING LABORATORY SERIES S EL-92-003 

( *'A SA- ~~w/v6m N9 3-1 7 16 1 

50FTkAKE FNGINEFHING PAPtRS, VOLUME --THRU-- 

10 (NASA) 134 p N93-17172 

CD L t ET, T E 0 

Unclas 

G3/61 0136130 
-I 
SOFTWARE ENGINEERING LABORATORY SERIES SEL-92-003 

Eh 

COLLECTED SOFTWARE 

NEERING PAPERS: VOLUME X 

NOVEMBER 1992 

NnsA 

National Aeronautics and 

Space Administration 

Goddard Space Flight Center 

Greenbelt, Maryland 20771 

FOREWORD 

The Software Engintering Laboratory (SEL) is an organization sponsored by the National 

Aeronautics and Space Admb&mtion/Goddard Space Flight Center (NASNGSFC) and 

created to investigate the effectiveness of software engineering technologies when applied to 

the development of applications software. The SEL was created in 1976 and has three 

primary organizational members 

NASA/GSFC, Software Engineering Branch 

University of Maryland, Department of Computer Science 

Computer Sciences Corporation, Software Engineering Operation 

The goals of the SEL arc (1) to understad the software development process in the GSFC 

environment; (2) to measure the e&ct of various methodologies, tools, and models on this 

process; and (3) to identify and then to apply successful development practices. The 

activities, findings, and recomqendations of the SEL are recorded in the Software 

Engineering Laboratory Series, a continuing des of reports that includes this document. 

Single copies of this document can be obtained by writing to 

Software Engineering Branch 

Code 552 

Goddard Space Flight Center 

Greenbelt, Maryland 20771 

TABLE OF CONTENTS 

Sectionl-Intmduction ............................................ 1-1 

Section %“he Software Engineering Laboratory ....................... 

“fhs 

2-1 

arry,etal. ..... 

Section ?oftware Tools .......................................... 3-1 

“’Ibwards Automated Support for Extraction of Reusable Components,” 

“Automated Support for Eixperience-Based Software Management,” 

S. K AM-El-Habiz, V R. Bas& and G. Caldiera ............... 3-3 

J.D.Valett .............................................. 3-11 

Section Aftware Models ........................................ 4-1 

“Tbe Software-Cycle Model for Re-Enginccrhg and Reuse,” 

“On the Nature of Bias and Defects in the Software Specification 

“An Improved Classification Bee 

J. W. Bailey and V R. Basili ................................ 

Proas,” F! A. Straub and M. V Zelkuwitz .................... 

4-3 

4-19 

of High Cost Modules Based 

Upon an Axiomatic Definition of Complexity~ J. Tian, 

“Providing an Empirical Basis for Optimizing the Verification and 

%sting Phases of Software Development,” L. C. Briand, 

A. Porter, and M. V. Zekowitz .............................. 4-27 

V R. Basili, and C. J. Hetmanski ............................ 4-37 

‘IA Classification Procedure for the Effed Management of Changes 

During the Maintenance Process,” L. C. Briand and 

V.R.Basili.. ............................................. 4-49 

Section Coftware Measurement ................................... 5-1 

“’Ibward FUI Life Cycle control: Adding Maintenance Measurement 

to the SEI,,” H. D. Rombach, B. T. Ulery, and J. D. Valett ....... 5-3 

Section &Ada Technology ......................................... 6-1 

“Object-Oriented Programming with Mixins in Ada,” E. Seidewitz ......... 6-3 

“Software Engineering Laboratory Ada Performance Study-Results 

6-19 

Standard Bibliography of SEL Literature 

and Implications,” E. W. Booth and M. E. Stark ............... 

\ 

SECTION 1 - INTRODUCTION 

, p, , /,' ' 

SECTION 1-INTRODUCTION 

This document is a collection of selected technical papers produced by participants in 

the Software Engineering Laboratory (SEL) from October 1991 through November 

1992. The purpose of the document is to make available, in one reference, some results 

of SEL research that originally appeared in a number of different forums. This is the 

loth such volume of technical papers producedby the SEL. Although these papers cover several topics related td software engineering, they do not encompass the entire 

scope of SEL activities and interests. Additional information about the SEL and its 

research efforts may be obtained from the sources listed in the bibliography at the end 

of this document. 

For the convenience of this presentation, the 11 papers contained here are grouped into 

5 major sections: 

The Software Engineering Laboratory 

Software 'Ibols Studies 

Software Models Studies 

Software Measurement Studies 

Ada Rchnology Studies 

The first section (Section 2) presents a paper that characterizes the SEL as an experience factory and summarizes major lessons learned in the past 15 years. Studies on automated tools to aid in reuse and experience-based software management appear in 

Section 3. Section 4 includes studies on models for reuse, verification and testing phase 

optimization, effective management of maintenance phase changes, the software specification process, and the analysis of highest modules. Section 5 presents a study of 

maintenance measurement as it applies to the SEL. Finally, a study on the use of mixins 

in Ada and a summary of the performance of Ada within the SEL are included in 

Section 6. 

The SEL is actively working to understand and improve the software development 

process at Goddard Space Flight Center (GSFC). Future efforts will be documented in 

additional volumes of the Collected Sufbvare EngineehgPapets and other SELpublications. 

1-1 

looo5788L 

SECTION 2 - THE SOFTWARE 

ENGINEERING LABORATORY 

I 

/ ., I 

SECTION %TEE S0FIW.AR.E ENGINEERING LABORATORY 

The technical paper included in this don was originally prepared as indicated below. 

“The Software Engineering Laboratory-An Operational Software Experience Factory,” V. Bas& G. Caldiera, E McGq, et al., Aoceedings of the 

Fourteenth Intemaiional Conference on Sofnvmc Engineering (ICSE 92), 

May 1992 

2-1 

THE SOFIWARE ENGINEERING LABORATORY-AN OPERATIONAL SOFWARE 

EXPERIENCE FACTORY 

Victor Basili and Gianluigi Caldiua 

Univenity of Maryland 

Frank McG.ny and Rose Pajerdti 

Notional Acrvnautics and Spa Adminiantion/ 

Goddard Space Flight Center 

Gerald Page and Sharon Wpligora 

complter Sciences Corporation 

ABSTRACT 

For 15 years, the saftware Enghcexhg Lpbontory (Sm) has ban 

C-g Out 8lUdiU and expaimtnts fOr the purport Of lUldersbnding, assessing. and improving so- d sofrwue proc-se~ 

tiond Aeronautics and Space Adminhtmti~dbddd Spsoe plight 

Center (NASA/GSFC). 'he SEL can- three major orgrmiUti0ns: 

witbin a production software development cnvirarment 8ttheN.- 

~AsA/Gs~,plight~8~ 

University of Muykod. Deputmcnt of canputa Sciena 

COIltpUkX -8 -a, FLight 

TechDology Group 

Them organizations h8ve johtIy eked out reved hundred 

sofiwam studies, producing hundreds of reports, papers, and 

documents.~ofwhichdelcribe 8omca8pcctofthesoftwuce~ 

namics environment at NASA. "be rtudies range from small. 

controlled cxpc~hents (mch a8 andping the effectiveness of 

codertadingvenur thatof~ctiondtetting)toluge,mnltipieproject studies (such as assessing the impacts of Ada on a pm 

duction environment). The org&on's driving god is tohprove the software process continually, so that sustained 

improvement may be observed in the rcdbg prodUCt8. This 

paper dircusses tbe SEI.. as a fimctioning example of an opexationd boftwarc experience factory and aummuizes the charactenstics of and major lessons leaxned from 15 ysur of sa 

operations. 

1. TBE ExpERlENCE FAmORY CONCEPT 

Software engiucc~g has produced a fair amomt of reararch and 

technoiogy trpasfcr in the first 24 ycus of its existence. Pqk 

have buih tcchnologiCq methods. and todt that am lutd by many 

organizations in dcvclqmcot and maintamme d softarue 

v*- 

Unlike otha dihphcs, however, very little lwcuch huben 

done in the &dopmmt of modcls for the dw components d 

the dbciplinc. Models have btm developed prim;rrily for the 

softwrve product, providing mathanatid modela of b function 

and st" (e.& fulide state machitha h dbpct-orienred design). 

ghSIhg techaology that h88 been Mdpd h the flight dyol; in #nnC dvurced b-. Of b q- (e.&, & 
record fmding. and make ~ommendatims for 

future project improvements 

- Package the elq~ricnce gained in the form of updated 

and refined models and other forms of st"d 

knowledge gained from this and prior pjccts 

- Store the packagesin anexpcriennbasc mthey are 

available fa future project, 

The GonVQuestionlMetric Approarb is used to &he mment on the Jofrprarc project. process, and product in mch a way that 

Conceptual level (goal): A goal is defined for an object, 

foravarietyofruwons,with respect tovuiousmodelr of 

quality,from variouspointsofview,andrel.tivetoaparticlllar enviroD"t 

Opcntional level (quedon): A set of questions ir u.td 

to define models of the object of study and the focuses 

on that object to characterize the assessment or achi~cmcnt of a sptcific goal 

Quantitativelevel (mchic): A set ofmetrics,bascd onthe 

models, is wociated with mry question in order to answer it in a quantitative way 

The umccpt of the EqdcDce F8cbry was mhpduced to hthroot of continual impxovunent and compttitivc advantage. 

tiadize the collective laming of the olgrnizrrtioo that M at the 

Theexpaiearrf~orycanbealogicaland/orphysjcal~~~ 

but it is important that its activities axe scpanrted and made independent f" those of the project organization. The packaging of 

II 

1 I I 

Flgm 1. Project o%pnfulon Fnnctions 

I I 

tnf4EcrF-Y I 

I I 

Figure 2. Expcrknce Factory Fnadlons 

1 PROJECTORGANIZATION I EXPgRIENcBFAcMlRY 

2-4 
On the othcr hand, from the perspective of so- en-g 

research, there are the following goals: 

PROJECT ORGANEATION I EXPERIENCE FACTORY I 

rpecified by their ability to perfonn spocifk tasks and 

to interact with each other. 

Conceptual level This level represents the interface of 

the architectural agcnts and the flows of data and conlr~l 

whom. what is done ia tbe expaience hctory, and what 

isdoneintheprojcctorg.nizstiOn. 'hboundaryofthe 

CJEPerience factoay, Le., thc line that ~cpuater it from the 

pjcct orpkafion. is &6ned at this level based on the 

needs and chaactcristics of an ogmization. It can 

evolve as these needs lad chamctcms evolve. 

hplammcltimId: Thir kvel defines the actual 

among them. They specify who canmunicates with 

tochnicrl and cqanidd Mp~EPtion ofthe ar- 

~.gmtrandtbdrcoanech 'OlUatIbC~ 

'a,andoanmpnwtioaluly8Ddapp~ntional deputmenw ut iacluded m tbe rpecificatims 

Level lhey.rsurigaedproarsandproductm&ls, 

r- @sople arcanpufm). 0rhai"entatim &ail4 wh as mrrppine tbc agents ova organiza2-5 
quantify the existing softwarr process and associated products. let 

alone understand the impact of sp~~&cproccss methods. Thus, the 

SEL staff initiated efforts to develop some means by which the 

software process cdd be understood (through measu!=ment), 

qualified. and meamably improved through continually expanding 

understanding, experimentation, and process dinement. 

This working relationship has beat maintained continually since its 

nization. h general, these gds have mahued rather than changed; 

Undtrstd. Improve insight intothe softwa~~process and 

its poducts by characterizing a production environment. 

hu: Murmt the hprt thst avdlable technologies 

gierPtbawfici.ltothecnvironmentand,mostimpo~tly. how the tcchaol@es must be refined to best match the 

proceuwiththsenvironmcnt 

3. Pa&agt/Tnfure: Afkidentifjhgprocess improvements, 

package the technology in a form that allowsit tobe applied 

ThtJc gd arc .ddrersad qmtially, in an iterative fashion. as 

inception with rclatively little change to the overall gods of the orgathey afe as follows: 

1. 

2. 

h.veoathcra4rwueprocesa Dct~whidrtcchnol* 

intbeploQctiollorg~on. 

shown in Fip 4. 

The vh taken to attaining tbese goals has been to apply 

potentially beneficial techniques to tbe development of proaUctim 

soflu" andto IDC.III~C theproous andproduct in cnougb detail 

CQP, such as cost, diability, and/or maint.iolrbility, are defined as 

the orjpnization dem"e3 . the major near- and long-term ob* 

tivca for its software dcvclopment pm hppovemcnt program. 

menc that is, it &tines the puriculardata to be captwed and the 

questioar that must be addrrssed in each cxp&mcntal project. 

All of the "rpcrimeatr conducted by the SEL have occd within 

tbe pdnction cnvimnmcnt oftbe flight dynamics sahwart development facility at NASA/GSFC. The SEL production ~~~Vironmcnt consists of projects that me clded as midsized mftwarc 

systans. Tbe average propa lasts 2 to 3- l/2 yezux. with an avenge 

SM size of 15 software developers. The average software size is 

175 thousand SOUICC lines of code (KSLOC), couoting commatq. with about 25 percent reused from previous dcvelopmcnt 

to gpmtithbly uaws the 8pplie.d tochodogy. Me;mrrer of cob 

once ttlosc objectives arc known. the SEL ttrffdesips the cxpcricffor~s. Virtually all projects in this envlmnment are aientiljc 

ground-bed systems. although some embedded systems have 

bun developed. Most software is developed in FORTRAN. although Ada is starting to be used more frequently. Other lanpap, such as Pascal and Assembly, arc used occasiondy. Since 

this enviroament is relatively consistent, it is conducive to the 

experimentation process. In the SEL. there exists a homogeneous 

class of software. a stable development environment. and a controlled, consistent. management and development process. 

3. SELOPERATIONS 

Tht following threc major functional pups support the experimentatim and sfudies within the SEL (Figure 5): 

Software developers, who are responsible for producing 

thc flight dynamics application softwprr 

Software engineering analysts, who arc the researchers 

responmble for carrying out the experimentation process 

aod proaucing study results 

Data base support staff, who arc responsible for collccting, checking. and archiving all of the information collected from the development cffo~ 

D~ning tbe past 15 years. the SEL has cokcted and archived data 

m over 100 rodtwve development projects in the oganization. 

The dah me also used to build typical project profiles against 

which ong&g projects can be compared and evaluated. 'he SEL 

provides managus in this environment with tools (online and 

ppa) for monitoring and Usessing project status. 

?Lpicdly, thm arc 6 to 10projects sim- in progress in 

the flight dynamics cnvironmcnt AS was mentioned earlitr, they 

nrt~gc 175 BLOC, mging f" small (G8 BLOC) to 

(S 400 BLOC). with a few exceeding 1 million rolprr hes of 

code (MSLOC). Erb projuCt is considered an experiment within 

the SEL, and the goid is to extract detailed information to understand the pmces better and to provide guidance to future 

To suppi the studies and to support the gal of conhdly 

SEL rcgullrfy colbcts detailed data fram its dcvelopnent pmjats. 

Tbe types of data collected include cost (maswed in sort), 

proae4 and euct data Process data bclude information .bout 

&e project, mch as the methodology. tools and techniques used, 

and hfonnation about PQSonnei nperitncc and Wg. produa 

data include size (in SLOC). change and mr information. and the 

"Its of postdcvelopmcat static analysis of the dclivmd code. 

Thc datnmaybe somewhat dil[ratnt from om pmject to mother 

since the gods for a particular experiment may be merent betwear 

projects. Tbere is a basic set of infomation (such as effort and 

error data) that is colleded for every project. Howevcr, as changec 

am made to +c pmccssca (e.g., Ada projects), the detailed drtr 

collected may be modified. For cxample. Fip 6 shows thc 

standard error report fonn. used on all projects. and the m&ex 

Ada version, used for projects wbae Ada is being studied 

cenhal data bate. The analysts then use thwe data together witl 

OLha infomation. such as subjective lessons lamed, to anaiyze th4 

impact of a specific softwarc process and to measure and then fee 

='data are used to build predictive models for future projects an. 

to pruvide a rationale for refrning pa"lar softwprt ~UCCSSC 

being used As the data are analyzed.papen andrepom ~IC genu 

ated that reflect results of the nmemus studies. Additionally, tb 

results of the analysis arc padcaged as standards, policies. trPinin 

materials. and management tools. 

1. 

2. 

3. 

e 

iaar;lsingrrodastrndiagoftheooftwat.edcvelopmentproceu,the 

AS tbe inf~~~ati~a is collcctd, it is quality 194 and win i 

bpdr ~taulb to both ongoing pmjects and follow-m PIO~&S. 

2-6 
DEVELOPERS 

(DEVELOP FLIGHT DYNAMICS s/w) 

STAFF 275-300 (FTE') 

TYPICAL PRWECT 150-200 KSLW 

SIZE 

ACTIVE PROJECTS 6-10 

(AT ANY GIVEN TIME) 

WOJECT STAFF 1525 PEOPLE 

SIZE 

RfFWEMENlSTO 

DMLopMENTpAocEss 

19761992 loo PROJECTS 

SOFTWARE ANALYSTS 

(STUDY PROCESS) 

DEVELOPMENT MEASURES 1 FOREACHPRatECT STAFF 5-10 RESEARCHERS 

19761992 250 EPORTSOOCUMENTS 

FUNCTION - SETGOALYOUESTW 

MRRlCS 

- DESlGNSTUDlESl 

* ANALYSISRESEARCH 

- REFINE SW PROCESS 

EXPERIMENTS 

- PRODUCE REPORTS' 

FINDINGS 

QATA BASE SUPPORT (MAINTAMA SEL DATA) 

I 

STAFF 2-5 (FIE) I ELDATABASE 

0 

's 

FORMS LlBRARY 

*PROCESS FORMSDATA 

* RECOWUAFCHNE DATA j 

MAWAIN SEL DATA BASE 

OPERATE SEL LIBRARY 

P 

In tba suxd major stage &the arperiencc hctory, elements of the 

pmcus (such as roffwue developnart techniques) me assessed, and h evolving technologies arc tdod to the partrculu 

periment in which some software method is studied in detail. 

Garerplly.thcsllbjaa ofthe audy is rspcci6c modi&&- to the 

standard process, a process that obviously comprises numemus 

 SO^" methods. 

envimnment E.ch projsa in thc SEL ia Considered to be 8n ex2-7 
2-8 
' BY LIFE-CYCLE PHASE: BY ACTIVITY: 

DEPENDENT REPORflNG 

DATE PROGRAMMER 

Flgprc 7. Mort DIstribntion 

x, 84 

ooersent~~Yth.1exan~the~t~~involverthe 

.ppliedOatbICCpropctlwithintheSEL,uchpmvidingdclitioarrl 

clesnroan 10ftwac~odologp[9J. Thicmetfiodologyhubeca 

insight into the Qeplroom pmcea and each dding lol~t elanart 

of "t" to the mttbodology for this one ea-t. 

he SEL trained teams in the methodology. then deiined a 

modified set of Qu"-aptci6c data to be collected- l" 

projects wcxc studiedin an attempt to assess the impact that Clezw 

room had on the pzoccrr as well as on soch m-s as 

prodocti* and ~fiabsty. EgIm 9 depictt cbe cIm" '&s d 

the Clean" changes, .swell LI tbc rcIults of the tkc eXpaiments. 

The Cleanroom expaimcatr included significant changes to the 

teoaive tnining, pqmmtion, and careful exaxtion of the rtucfiu. 

mdies (aa they ~TC fa dl such cxpcrima~tr). and ach included a 

&scription of the pals. the qutsfianr that had to be addressed, and 

the metrics that had to be collected to answer thc questions. 

Since this methodology consistt of multiple spcci& methods (e+, 

box st" derigo, Meal testing. ngorauinspationr), -h 

Cleanroom methodology in gtmrrl. A. a result of the analysis. 

ckan" has been uasscsred" as 8 bencficid approach for the 

SEL (as measured by spacific goals of tbese studies), but apedk 

elements of the full mctbodology had to be Mod to better fit the 

parhdar SEL environment. The tailoring and modifying resulted 

in a revised Cleanroan pnmss model, written in the fonn of a 

PXQCCSS handbook [lo], for future applications to SEL projects 

-dad SEL. devdopnat mctbodology, tbtnby requiring CXDetailed cqla5"~al phr -re gcnartod for Uch of the 

~~ar~th~~tobe~alOag~~e~in~~~ 

That step is the "packaging" component of the experience factory 

process 

4.3. PACKAGING 

The hal stage of a complete experience factory is that of pskaging. After beneficial methods and technologiu are identified, the 

organization must provide fedback to ensuing projects by cap 

tudng the process in the form of stadads, tds. and training. Thc 

SEL has pmduced a set of standards for its own use that reflect the 

results of the studies it has ccmductd. It is lrppPllent that such 

standards must continually wobe to captum modified chamdcriatics of tht pNnxss. (The SEL typicauy upda!es its basic SmIdaNi 

mry 5 yurs) Examples of Jt.ndards that have bear produd as 

put of tbe packaging process includc: 

Manageri Hdbd for sgtworr Development 1111 

Raconvnrndrd Approoch to Sofhwm Development [12] 

onc ddirionai example of an cxleruive p.clging effat. in tbe 

SEL is a management tool called the Software hmt Envi- 

"at (SME). ThccaoccptsdtheS~ whicbisnow.rr opentional tool used locally in the Sm have evolved oyer 8 yean. 

This tod acm SEL project data, models. rtlntiondrips. iessoas 

tuirtlcs to the manager of an ongoing project Thir allows the 

managa to gain insight into the pws conrirteocy witb a devi- 

'Ilks example of 'plck.gtrg" rr&d. the e"& that mud bo 

lcmonsl~lefiwdmodelr.dgcacrrlondartPldiag, euily 

rv.il.blstootbafollowroadmlapacntpropctrinaprticulataleuncd, and matk8gd mlm Of thmb to pltrmt propCt Cbation hum tbc nonn for the eavirmment (Figurc lo> 

placed OD making ds of softwan? projsctr, in tbc fonn of 

m-iThc tool~estheoollectroa ' of1S~of~uchiVsd 

iotbesBLtolclea.ppmprute * ,rimil.tpropad.ermtb.tMnrg 

propctbuedon cherrulyledbidory ot rimilu roftwuo &orts. 

As~e;xllaple.doftbearwchrrrteristicsoftbc~*dynmnicr 

as can plm. monitor, @ct, and better understand the& awll 

psojectr have ded in the emr model depicted in Kgum 8, 

wberehiatmyhushowntypkdrdtwueemr~intb~~ 

phases dthe Iife qck. Asnswprojccts me dmlaped md ermr 

dseqanch 11c routinefy rqxntd and added to the sm data 

kse, the manager can easily cmpa~~ err01 mtes on his orberprojact with typical emor rata on completed, similnr pmjects 

vt, p" imyt, and ppcL.ging ~IC applic8blc 

to all c"mcnts. 

5. ADAANALYSIS 

A mom detailed cxampk of ooc technology that hu bat0 rtudied 

Ada. 

velopmmt proarr and hd 0rt.bliM nrly rehidpa, and 

modelrthrtimpmvcdthemanrgubilityoftheproarr hhdb 

Gnatuwditrpnmss by adding andre6ningtechniquerwithinb 

@mdard methodoiogy. Redizing that Ada and objcct~ricnted 

techniques OW p0tu1ti.l for major impnwmat in the flight 

with Ada. 

Tbe ht step was to set up ~g#aations and goals against wbicb 

results would be The SEL's wclksbblisbed baadine 

and set of measures pruvidd an exdent basb fa compuisOa. 

ExpcdaUons included a change in the effort distribution of developntnt activities (e.& incre;rsed duign and deQused testi.g); no 

grcatcrcost per new line of code, increased msc; dsclused mrrib 

fenancc costs; and increpstddiddity (Le., lower ctror Xatu. fewer 

interface em. and fewer &sign errors). 

obviauly. the dat. are cnvironmart dependensbut tbc CoDCcpEl d 

in the SEL within tbe COnfeB of the CxperiMCc factory is that of 

By 1985. thc SEL. had rbieved a good un- - gof 

how Mdtwuc wudtvelopad in the FDD, it hdbadintdtbe da 

dynamkrenvirmment , the SEL decided toplrrut Gxpaimcnt.riar 

2-9 
INSPECTIONS 

5 

AVERAGE NUMBER OF 

REVIEWERS 

AVERAGE 

DURATW 

EFFORT DISTRIBUTION 

UPWARD TREND IN DESIGN X 

401 I 

DESIGN CODE TEST OTHER 

SEL BASELINE 

1ST EXPERIMENT 

N2ND EXPERIMENT 

ERRORS (PER K DLOC) 

I 6.0 

SEL 1ST 2ND 

BASELINE CLEANROOM CLEANROOM 

EXPERIMENT EXPERIMENT 

TEST PROCESS 

PER SUBSYSTEM 

AVERAGE 

BUILD SIZE 5500 LOC 

UNCOVERING ERRORS 

NO SIGNIFICANT DIFFERENCES 

501 I - 

40 -J 

10 

0 

COMPUTA- DATA INITIAUZA- INTERFACE LOGIC 

TION TlON 

INSPECTIONS 

TESTING 

PRODUCTIVITY (DLOC PER DAY) 

40 

SEL 1 ST 2ND 

BASELINE CLEANROOM CLEANROOM 

EXPERIMENT EXPERIMENT 

F¶gure 9. ClcrnroomAsscssmmtintbcSEL 

2-10 
Software Management Environment 

EXPERIENCE BASE AUTOMATED TOOL (SME) MANAGEMENT AID 

1. COMPARUEXPLNN 1 

1. HISTORICAL DATA 

[Fj DATA BASE 

2. PROCESS MODELS 

SME 

3. KNOWLEDGE 

- LESSONS LEARNED 

- INTUITION 

I a QRRENT PRQlECT 

--L No. OF 

ERRORS 

I I 

\ 

2-1 1 
6 PROJECTS USING Ada AND OOD 

100 

decrcrstd ad is now well below the cost to deliver an equivalent 

Reliability of Ada systems has also improved as the environment 

has mrturtd Although the mor rates for Ada systems, shown in 

Fip 13. w~t signitkantly low from the start than th- for 

tb hi& level of reuse in tbe later systems is a major contriitorto 

FO" system (Fip 12). 

F". they have continped to d4cnase even futtha Aglris 

this~yimpvcd~. 

Druing thiadyarpaiOd,theSELwent thnmgb v8riow lmls of 

packaging be AdJooD mdoddogy. On tbo eadiast Projea in 

1985, whea OOD was still vay young in the industry, the SEL 

found it necessary to tailor and package their own G"l 

in the flight dynamics cavironmcnt This document @dud in 

1986) adjusted and extended the industry -dud for use io tbe 

local environment In 1987. tbe SEL also developed an Ada Styk 

Guide (141 that pmvided coding stadads for the local environited pmjsct-spi& b.ioing. cdtutd the urty training io these 

tschniques. The SEL rlso prodwed lessona-kar~~~d rtpom on the 

AWD experiences. ncanmcnding rehents to the methodRecently, because of the mtim and appent benefit to the 

ogpniutioa, AWD is being packaged as put of the baseline 

SEL methodology. Tbc atmdprd methodology handbooks [ll, 121 

idn& Ada and OOD as mainstream methods. In addition. a campkte and highly tailored training pmpm is being developed that 

taches Ada and OOD as an integrated part of the fight dyrmnks 

envirwunent. 

Although AnJooD will continue to be dined within the Sw it

has progressed through all stages of the apuknce factuy. mohg 

h a candidate trial methodology to a fully intcpatcd and padraged part of the standard methodology. Tbe SEL conaidus it base 

lined and ready far furtha itmemental improvement 

IMPLICATIONS FOR THE DEVELOPMENT ORGANIObpct-od~td Develwt (GOOD) mdh&lw (13) for U& 

ment. c0"ercl.l - Ada~gcounes.~l~tedwithiimology. 

6. 

ZATION 

For 15 yurs. NASA hubem funding thc efforts to carry out 

cant costs and a certain levd of overhead associpted with these efforts; a logical question to ask is "Has there bccn significant bene 

fit?" The historical information strongly supports a very positive 

answer. Not only has the expcndittue of resources been a wise 

investment for the NASA fight dynihcs en-ent. but member~ of the SEL strongly believe that such efforts should be 

expime d audicr within the Sm There haw been Jignificosr TO DEVELOP 

1.6 EFFORT PER OMLOPED STATEMENT 

l., - FORTRAN 

12 12 

. 12 ma 1.1 

COST 'TO DELIVER 

EFFORT PER OEUYERED STATEMEW 

I 1.0 1.0 

1.0 

E 

eo-@ 

: 

f.. 

0.4 

02 

0.0 

FIgurc l2. CostsToDevelopmdDeUver 

commonplace throughout both NASA and the software communit 

in gcoeral. The benefits far outweigh the costs. 

Since the SEL's inception in 1976, NASA has spent qq"ate1 

$14 million dollus (coohact Npport) in the duee major suppu 

ing studies and dyzing redts), technology (producin 

standards and policies). and data proceasing (collecting forms an 

maintaining data bases). Approximately 50 &-years of NAS 

ptrso~el effort have. bem expended on the SEL. During this sm 

period, tbe flight dynamics area has gptnt apmxbatdy $150 m' 

lion m building apcrPtiood software, all of which has been part I 

araa~bythttypcof~~environment: reseych(dtfir 

the study process. 

2-12 
1 

11 

I 1 

21 

E l. 

1. 

rtntcmenl, ihs jrmprovanent is mr& but when it meesurcl 

nificmt. 

2. Reliabilitydthe~huimp~wcdby35pacalt. As 

meaJuItd by the number of CPOIS per thousand lines of 

code (WKSJBC), fight dynamics software hsr Mpd 

fnnn an avenge of 8.4 E/KsLoc in the eady 1980s to 

approximately 5.3 E/KsM)c today. Tbese figures cover 

cxy to g?cdons. Although opem~ons and m8bteoPllce 

datnsnnotmdy soomsiveuthcdeveIopmentda@,the 

small amount of data available indicates significant 

OVenlu cort 8I.d pmddvity, the i"ent is rigthe-phLICsthfo118h reoeptuIcet&ngand&&vimprov-t in that arca 8s WelL 

3. Theu- -* "OfsofrwuehmimproveddrPnutidly. fn the late 1970s and cdy tbe envhmcnt 

expcncnccdwi&~onsin~vity,rdinbility. aod 

quality fnrm pjcct to project Today. however, the SEL 

has excellent models of tbe procus; it has well-deiincd 

methods. and managas rue Mer able to predid. 

and manage the cost and quality of the softwan being 

produced. This conclusion is substantiated by rtcent SEL 

data that &ow a continually improving set of models for 

planning. prediaing, and estimating all development 

projects in the flight dynamics environment. Thex no 

longer is the extreme uncertainty in estimating such 

common parameters as cost, stfig, size. and reliability. 

OthtrmePsurts include theeffort put forth in rcwodc (e.g.. 

changing and corrading) and in overall software rcusc. 

Thczemeasu~dsoindicateasignifi~t improvement to 

the software within this one environment. 

4. 

lo rdditim to the CO" mearrue~ Of mf- (e.&, COS and di- 

-t" such - the SEL'S. Not only has the Underability), the an many other major benefits derived fmm a "meastanding of ooftwnre significantly improd within the research 

camnunily, but this mde-ding is apparent throughout the 

mth &vdopcnt community within this cnvirornneat Not only 

have the xwmdwsbenebted, but the deveiopm and maaagcfl 

who haw kco expord to this e&ut am much bemr pqd to 

plan, coatrd, and, in graenl, *lop much bigher quality syxtemr Om view ofthisprognm is that it is a nuj0r"training" 

cxucise withia 8 large prododon envhnmcnl. and the 800 to 

loo0 dmlq~n and mrnogen who have puticipted in develop 

mmt effortr rhldicdby the SEL uz! much better trrriacd and effective rdrwue enginem. nis is dlw to thc extensive haining ard 

n4llvinw 

gtnaal~dl&velopmgetfromthe~~hetTatsC~tinJn coIIcIo.iQI. tb SELfuoctiars u an optIPtimd cxampk ofthe 

m sstim 1 mrp to Ute ftnaiaarrl groups discussed 

me2 is lralilrA by tbe SEI, data base md itr uchivcs of manrsrmaJmodeltmddaticnsbip8[16]. Theanalysisfuactionfrom 

pislpe2 ispufd by tbe SEL team of oattanre engineedng 

mdy- wbo mdppasses and- to understand tbs 

rehe the new technologier unda study. Fkdly, tbe synbair 

pckaghg lltw pmocm and technology in 8 form tdod qnsyntbcrir, arpdmging, an the guideher. -dads. sod tools the 

cxp"*frctorgcarapc 'Ibecanapuslmo&lforthesEL 

lmdersELapeneimrin~3. .raecxpkncebpreinFigenviroammt. tben plan 8I.d ex- expcrimentr to .sstBs .Dd 

fl" Oftheexpcri~ bxy mapa to meSELb.ctiviriesin 

Cifldy to rbe flight dynamics en- . Tbeproductrofthir 

SPL PraQlea to infuse its findings b8ck into the proiect ag, 

nimtioa lIbenplroduas an2 rha~Iialcc~~s of the expenaKxf.aorymodel. 

Cmrrnt SEL duits an focostd on addrrssing two major questions. 

The 6int is Wow long does it take for a new tcchndogy to mow 

throagh dl the stages of the expcrhce fwT' That is, fiom 

g pad bwlining the c"t envirwoleot. througb 

assessing the impacts of the tshudogy pad dining it. to packrging the pnmsr 8nd infusing it into the proita orgmizdcm. 

preliminup kndings hm SEL'r A& 8I.d Qai- indicate a cyck of roughly 6 to 9 ]rurs. but further data 

poiuts me needed. 'be secmd question tbe SEL is pursuing is 

Wow laup PO ogrniution c~ll adopt the e~riarc factory model?" " SELisintneatedinlcaming whatthe dleupissucs ue 

when the acqx of the experience factory is adendad beymd 8 

single environment. NASA is spoasoring m effoa to explorc tbe 

infusion of SELUre implementations of the cxperimce factoy 

concept YJOSS the entire Agency. 

ACKNOWLEDGMENT 

Matwid far this paper xcprcscnts wok not only of tbe antbors 

listed, but of many other SEL staE membcn. Special ackndcdgment L #ven to Geny HcUcr of CSC, who played a key rok in 

editing this PQPeL 

REFERENCES 

Numaws p.pcrs, reports, md des have bear genenrted oyer 

the SEL's Isyear existence. A complete listing of thue caa be 

found in the Annotated Bibliography of Software Engineering 

2-13 
Loboratory Literature. SEI-82-1006. L. MONS~CW~CZ and J. Vdctt, 

November 1991. 

This bibliography may be obtained by contacting: 

ThCSELLibrUy 

~rrcnbci~, m mn I 

code552 - 

NASAxiSPC 

A listing of references Specific to this paper follows. 

1. V. R Bd, 'Towards a Mature MePIlurement EnvirWment: 

Creating a Softwvt Engineering Research Envin"cnt,"b 

cecdings of the Fifteenth Annual Softwatt. Engineering Workshop, NASGSFC. Greenbelt, Maryland, SEL-90-006, Ne 

vembcr 1990. 

2 V.RBuili,"QuMtit.tiveEvpluofaSoftwucEnginsaing 

Metbodology."~ . 8s Of the First PM PrifiC kptCr 

Ccmfcxcnce, Melboumc. And% September 1985. 

V.R. Bd and D. M. Weiss. "A Methodology for (Mlecthg 

Valid Software Engineering Data," IEEE Tm"s on 

Software Engineering. November 1984. pp. 728-738. 

4. V. R Basili,"SoftwarrDevelopnent: APanmiigmfortheFbhuc 

(Keynote Address)." ' gs COMPSAC '89. <Mutdo, 

F%xida, September 1989. pp. 471485. 

5. V.R.BIsilimdH.D.RombPcL'lhiloringtbeSo~Rocers 

to hjcct Goah and hvhnments." Proccedingr oftbe Ninth 

Intemationd Carfatnce on So- EagiDeaing, Moabaey, 

6. V.R.BasiiiandH.D.Ranbac4~TAMEProjecr:TawPb 

actions 00 Solbare Engimering. Vd 14. Na 6.. June 1988. 

7. V.RBasili PndG. Caldicra,"M~l~dd~ 

3. 

c.lifomh Md 30- A@ 2.1987, p~. 345-39. 

bnprovcment-oriented Solhan3 ell^^"^^ 

pp. 758-773. 

IssuuintbeExpcne~xFactory,"pIocseQl . g8oftbssixteenm 

Annual sahwut Enmg Wo~op, NASmR2. 

Greeabclt, Marylurd,Softwprc~~gLabo~~ 

I)ecembu 1991. 

V. R Basili. G. Caldiem, and G. Cantane, "A Rdaenct 

Architecture for the Component Fbctoq," ACM TrrngctioaS on Software bginemhg and Methodology, Vi. 1. No. 1. 

Januay 1992. pp. 5-0. 

8. 

9. HD. Mills. M. Dyer. and RC. Linger. "Clcanroom Software 

Engineering." IEEE Software, November 1990, pp. 19-24. 

10. S. Green. Scftware Engineering Laboratory (SEL) Cleanroom 

Process Madel. SEL91-004, November 1991. 

11. L. Landis. F- E. McCany, S. Waligoxa. et al.. Manager's 

Handbook for Software Development (Revision I), 

SEL86101. November 1990. 

12 F.E. McGarry. G. Page, S. %linger, et al.. Recommended 

Appnwrch to Software Development. -81-205, Apd 1983. 

Revision 3 inprepdon: schedukdforpublicatiarion June 1992. 

13. E. Scidewitz and M. Stark, General ObJkct-Orieniui Software 

Development. SEL86-002, August 1986. 

14. E. Sdewitz et d, A&@ StyZe Guide (Version 1.1). 

SEL.47-002, May 1987. 

Is. D. Bohd et rl. A Srudy on Sue and Reuse Trrndr in Attitude 

Gmd Support Systems (AGSSs) Developed fa he Flight 

L$namics Division (FDD) (1976-1988). NASAJGSFC. CSc/ 

TM-89m31, pcbnury 1989. 

16. W. Decker. R Hendrick, and J. Val- Sofh.mc Engineering 

Labororay (Su) Relationships, hfodcls. and hfanagemenl 

Rules, SEL-91401, February 1991. 

2-14 
SECTION 3 - SOFTWARE TOOLS 
~~ . . . . .. ... . 
SECTION 3-SOFIWARE TOOLS 

The technical papers included in this section were originally prepared as indicated 

below. 

‘“lbwards Automated Support for Extraction of Reusable Components,” S. 

K. AM-El-Hafiz, I% R. Basili, and G. Caldiera, Bvaedings of the IEEE Conference on SoNare Maintenance, 1991 (CSM 91), October 1991 

“Automated Support for Experience-Based Software Management,” J. D. 

Valett, Proceedings of the Second Iwine Sojlwarr Symposium (ISS ’92), March 

1992 

3-1 

100057Ei8L 

Towards Automated Support for Extraction of Reusable 

Components 

S. K. Abd-El-Hafiz V. R. Basili G. Caldiera 

Institute for Advanced Computer Studies, 

Department of Computer Science, 

University of Maryland, College Park, MD 20742, U.S.A. 

Abstract 

A cosi effective introduction of software reuse techniques requires the reuse of etisting software devtloped 

in many cases wiihoui aiming ai reusability. This paper discusses the problems relaied io the analysis and 

reengineering of ezisiing software in order io reuse it. 

We introduce a process model for componeni ettraciion and focus on the problem of analyzing and qualifying sobware components which are candidates for 

reuse. A profoiype fool for supporting ihe edracfion 

of reusable componenis is presenied. One of ihe componenis of this tool aids in undersianding programs 

and is based on ihe funciional model of correciness. 

If can assist software engineers in the process of finding correci formal specificaiions for programs. A detailed descripiion of ihis componeni and an ezample io 

demonstrate a possible operafional scenano are given. 

1 Introduction 

Successful reuse of software resources can increase the overall quality and productivity in software 

projects by a large factor. Some of the problems that 

still limit software reuse are: 

1. The difficulty of understanding a given software 

product in the absence of its original developers. 

2. The scarce availability of reusable objects, even 

though there is a tremendous amount of available 

software. 

3. The difficulty of retrieving, from a large data 

base, software components which can best match 

the given semantics requirements. 

New process models for software development 

should substitute the existing ones that are not defined to benefit from or support reuse. These new 

models should take advantage of reuse, introduce more 

reusable resources, and overcome the existing problems that limit reuse. 

Developing reusable components is generally more 

expensive than developing specialized code, because 

of the overhead of designing for reusability and maintaining the component repository. A rich and wellorganized catalog of reusable components is the key 

to a successful component repository and a long term 

economic gain. Moreover, such a catalog will not be 

available to an organization unless it can reuse the 

same code it developed in the past. Mature application domains, where most of the functions that need to 

be used already exist in some form in earlier systems, 

should provide enough components for code reuse. For 

example, Lanergan and Grasso found rates of reuse of 

about 60% in business applications[l). A technique 

for extracting reusable components can improve pre 

ductivity since it provides the software developer with 

components that are ready for reuse or need minor 

adaptation. Moreover, it can improve the software 

quality as it helps in better understanding these components during the extraction process. 

In this paper, we use a process model[2] that serves 

not only to enhance the development of the project 

under consideration but also to organize and plan for 

better reuse technology in future projects. This model 

splits the traditional life-cycle model into two separate 

organizations, the project organization and the experience factory. In this framework we introduce a process model for component extraction and focus on the 

problem of qualifying candidate software components 

for reuse. 

4. The lack of extraction and adaptation techniques A prototype tool constituting one of the elements 

of an integrated system for extracting reusable compo- that facilitate the reuse process. 

1ooo6768L 
nents is described. This prototype tool helps in understanding programs by deriving their specifications and 

is based on the functional model of correctness[3, 41. 

The tool could be applied ta program fragments as 

well as to complete programs and it helps in simultaneously checking syntax, static semantics, and generating specifications. We conclude the paper with 

an example to demonstrate a possible operational scenario of the tool. 

2 Organizing the component extraction 

Currently, all reuse occurs in the project development, where there is a completion deadline and the 

top priority is to deliver the system on time. This 

makes the objective of developing reusable software, 

at best, a secondary concern. Besides, project personnel cannot recognize the pieces of software appropriate 

for other projects. 

We make use of a reuse-oriented model based on 

two separat.e organizations[2]: 

The project organization: Its goal is to deliver 

the systems required by the customer. The process model can be chosen based upon the characteristics of the application domain, taking advantage of prior software products and experience. 

0 The experience factory: It supports project 

development by analyzing and synthesizing all 

kinds of experience, acting as a repository for such 

experience, and supplying that experience to various projects on demand. Within the experience 

factory, we can identify various sub-organizations. 

One of them is the component factory which 

develops reuable components. extracts reusable 

components from existing systems, and generalizes or remodels any previously produced component. 

Different conceptual architectures can be used for 

the component factory[5]. At one extreme there is the 

clustered architecture in which all software development activities are concentrated in the project organization and the component factory is dedicated only to 

processing already existing software. At the other estreme there is the detached architecture in which the 

development activities are concentrated in the component factory and the project organization performs 

only high-level design and integration. The clustered 

architecture is much closer to the way software is currently implemented. The development of the cornpe 

nents is probably faster in the project organization 

since there is less communication overhead and more 

direct pressure for their delivery. On the other hand, 

the components developed are more context dependent. In the detached architecture, there is more emphasis on developing general purpose components in 

order to serve several project organizations more efficiently. On the other hand, there are more chances 

for bottlenecks and for periods of inactivity due to the 

lack of requests from the projects. The detached architecture is probably better suited for environments 

where the practice of reuse is formalized and mature. 

An organization that is just starting with reuse should 

probably instantiate its component factory using the 

clustered architecture and then, when it reaches a sufficient level of maturity and improvement with this 

architecture, start implementing the detached architecture in order to continue the improvement. 

In any case, the extraction of reusable components 

is a characteristic activity of the component factory. 

The next section will present in detail the features of 

this activity, in the framework of a component fackory. Cddiera and Basili[G] have proposed a process 

model for the extraction of reusable components in 

two phases: the identification phase and the qualification phase (see figure 1). The necessary human 

intervention in the second pliase is the main reason 

for splitting the process in two steps. The first phase, 

which can be fully automated, reduces the amount of 

expensive human analysis needed in the second phase 

by limiting analysis only to components that really 

look worth considering. 

3 The extraction process 

3.1 Identification 

Program units are automatically extracted and 

made to be independent compilation units. These independent units are measured according to observable 

properties related to their potential for reuse in three 

steps. These steps are summarized here: 

1. Definition of the reusability attribute model: 

A set of automatable measures that captures the characteristics of potentially reusable components is defined along with acceptable ranges of values for these 

metrics. 

2. Extraction of components: Modular units (e.g. 

C functions. Ada subprograms or blocks, or Fortran 

subroutines) are extracted from existing software and 

3-4 
completed so that they have all the external references 

needed to reuse them independently. 

3. Application of the model: The current reusability attribute model is applied to the extracted, completed components. Components whose measures are 

within the model's range of acceptable values become 

candidate reusable components to be analyzed in the 

qualification phase. 

A detailed description of the component identification phase, a definition of a basic reusability attribute 

model, and an application of this model on several 

case studies using a computer-based 'system have already been discussed in the literature(61. 

\Components I 

Figure 1: Component extraction. 

3.2 Qualification 

The extracted components are analyzed in order 

to understand them and record their meaning. The 

components are packaged by associating with them a 

reuse specification, a significant set of test cases, a set 

of attributes based on a reuse classification scheme, 

and a set of procedures for reusing the component. 

This phase consists of following steps: 

1. Formal specification: A precise description of 

what the component does is generated and some assurance is obtained that the component meets the requirements. 

Since formal specifications are based on mathematical notations, they help in understanding the software by removing the ambiguities which might be introduced by any informal notation. Formal specifications are different from the programs they specify 

since they only express the behavior of the program 

without stating how the program derives this behavior. So, formal specifications are the basis for selecting 

and storing software components as they improve understandability and assist in producing more reliable 

and higher quality software. Since the specification of 

complex tasks may in itself be complex, the process 

of specification construction must be formalized and 

supported by automated tools. In the next section, we 

will describe a prototype tool that aids in understanding programs. This tool provides automated support 

for deriving the functional specifications of programs 

and proving their partial correctness. In other words, 

it helps in proving that the program is consistent with 

its specification but does not prove its termination. 

Formally specifying a software component and 

proving its partial correctness do not mean that the 

component will pass this step. There are several other 

properties that should exist in the candidate components for the sake of understandability. We must not 

ignore other important features such as proper documentation, use of meaningful variable names, and the 

structured style of programming. The informal information that the software engineer deals with cannot be 

ignored relying on the fact that the automated specifications tools will supplement those features. The 

informal information is important in explaining some 

intuitive ideas that are hard to explain using formal 

specifications. 

Since we need both formal and informal information, a domain expert is needed to perform the specification step. This expert extracts the formal specification of each candidate reusable component, assisted by 

the automated tools available, and examines the other 

informal features that cannot be judged using automated tools. Components that are not relevant, not 

correct, or whose functional specification is not easy 

to extract are discarded. The expert reports reasons 

for discarding candidates and other insights that will 

he used to improve the reusability attributes model. 

2. Testing: Test cases are generated, executed and 

associated with components. Deriving the functional 

specification and proving the correctness of a program do not mean that it will not fail when compiled 

and/or executed. This might simply be due to the 

fact tliat termination of the program has not been 

proven. Moreover, in most verification and specification systems, arithmetic operations ignore things such 

as overflow, underflow, and round-off errors. 

Testing can take advantage of tlie functional specification generated by performing functional testing. 

Also, structural testing can be done using a coverage analyzer. If, as is likely, the component needs a 

3-5 
‘wrapping’ to be executed, the testing step generates 

this wrapping. If a component passes the testing then 

test cases, wrapping, and test results are stored in the 

component repository. Components that do not satisfy the test are discarded. Again, the reasons for discarding candidates are recorded and used to improve 

the reusability attributes model and possibly the process for extracting the functional specification. This 

is most likely the last step at which a component will 

be discarded. 

3. Packaging: The extracted candidates are stored 

in the component repository along with their functional specifications and test cases. The component 

repository is actually a data base of experience in 

which information on software products, processes, 

and measures of aspects of them is stored. That is 

why we organize this data base by classifying both the 

reusable components and their development histories 

according to several domain dependent criteria. 

Information for the future reuser is provided in a 

manual that contains a description of the component’s 

function and interfaces as identified during generation 

of its functional specification, directions on how to install and use it, information about its procurement 

and support. and information for component maintenance. 

At the end of each process cycle the reusability attribute model is updated by drawing on information 

from the qualification phase to add more measures, 

modify or remove measures that proved iceffective, or 

alter the range of acceptable values. This step requires 

analysis and possibly even further experimentation. 

The taxonomy is updated by adding new attributes 

or modifying the existing ones according to problems 

reported by the experts who classify the components. 

4 The CARE system 

4.1 Overview 

The CARE(G] system(CARE’ : Computer Aided 

Reuse Engineering) has been designed to support the 

proposed process model for estracting reusable components. As shown in figure 2, it consists of two main 

subparts: the component identifier and the component 

qualifier. The component identifier consists of the 

model editor. which helps in defining and modifying 

the reusability attributes model, and’ the component 

extractor which applies such model to the programs. 

’The CARE system is under development at the Computer 

Science Department of the University of Maryland 

The component qualifier consists of the specifier, the 

tester, and the packager. The current version of the 

CARE system consists of the component extractor and 

the specifier. It runs on a Sun Workstation and supports ANSI C and Ada. In the rest of this section we 

focus on the description of the specifier. 

smam MODEL EDITOR 

2.2 

TerreR 

1.2 

2.3 

PACKAGER EXTRACrOR 

COMPoNEhS 

REWSlTORY 

Figure 2: CARE system architecture. 

4.2 The component specification tool 

The prototype specifier included in the CARE tool 

is the second in a series of prototype tools developed at 

the Computer Science Department of the University of 

Maryland under the general name FSQ, for Functional 

Specification Qualifier. This prototype supports the 

derivation of programs specifications and the verification of whether or not the programs meet those specifications. It does not only help to specify and check 

the partial correctness of finished programs, but it also 

works on unfinished programs and program fragments. 

It is a program understanding tool that is based on 

a formal specification technique. CARE-FSQ2 uses 

Mills’ functional model of correctness[3, 41 in order to 

derive the specifications. This model requires the user 

to provide only the loop function and then a teclinique 

is provided to derive the program specification. Other 

techniques(7, 81 require the user to provide an entry assertion, an exit assertion, and a loop assertion. Those 

techniques are more useful in verifying that the program is consistent with its specification. The process 

of deriving specifications helps more in understanding 

the software. Moreover, the functional method pro- 
vides simple and intuitive notations that can be easily 

understood. 

The CARE-FSQz prototype helps in checking syntax, static semantics, and generating specifications at 

the same time. CARE-FSQ2 also provides the capability of carrying out some algebraic simplifications 

and enables the user to make use of some well defined 

mathematical functions in the specification of the loop 

function. 

4.2.1 Formal foundation: Each statement S is 

given a meaning as a function from a program state 

to another state. A state is a mapping from the variable names to their current values. The square bracket 

notation is used to denote the function represented by 

the program construct contained inside the brackets, 

i.e. [SI represents the function computed by the statement s. We use four basic structures[3, 4): 

1. Assignment 

a variable and e is an expression, is: 

The meaning of the assignment u := e, where u is 

We can define the meaning of variables and expressions as a mapping from a state to a value. 

2. Comnosition 

position, we have: 

If A and B are statements and o is functional com- 

[A;B] = [.4] 0 [B] 

3. Alternation 

[ if B then S fi ] = {(U, [qU) : [B](U) = irue}U 

{(U, U) : [B](U) = false} 

[if B then SI else S2 fi] = {(U, [Sl]U) : [B](U) 

= true} u (11, [S?]U) : [B](U) = false} 

4. Iteration 

(while B do S 04 = {(T.U) : 3k 2 0 : YO 5 i < k ( 

((B]([S]'(T)) = true A [B]((Slk(T)) = false 

A [SI"T) = U)) 

In other words. the loop function is undefined for a 

state T unless there is a natural number k which denotes the number of iterations after which the test first 

fails. T is then transformed to the k-fold composition 

of S on T. In order to carry out practical proofs, the 

following characterizing theorem is neededI9). 

Theorem 

Then f = [W] if and only if 

Let W be the program fragment while B do S od, 

1. domain(f) = domnin((W1) 

2. ([B](T) = false) e f(T) = T 

3. f = [if B then S fi] of 

This theorem provides a method for deriving the 

correct loop function f: 

1. Guess or work out a trial function f. 

2. Use the three conditions of the theorem to check 

that the trial function is correct. 

A trace table can be used to organize the derivation 

of program meanings (by a symbolic execution of the 

program)[4, 91. 

The strength and weakness of the functional 

method, in comparison with other specification techniques, originate from the fact that even though exact 

functions state accurately the meaning of a loop, they 

are harder to work with than the weak assertions that 

suffice when there is a loop initialization providing a 

precondition. 

4.2.2 The implementation: CARE-FSQ:, is implemented using the Synthesizer Generator[ 101 and 

Maple, an interactive algebraic symbolic executor(l11. 

An overview of the tool is shown in figure 3. The 

Synthesizer Generator requires as an input a description of an attribute grammar and generates from it 

a hybrid language-based editor that allows a combination of text editing and structure editing. As the 

user edits program text and annotations, the system 

creates and edits abstract syntax trees that represent 

pieces of programs and their specifications. The attributes of the nodes of this tree carry information 

about the static semantics of the program as well as 

its specifications, and they are evaluated incrementally. The basic feature of Maple is its ability to simplify expressions involving unevaluated elements. As 

each complete statement is entered by the user, it is 

evaluated and the results are printed on the output 

device. Maple enables carrying out algebraic simplifications during the synibolic execution. In order to 

overcome the limitations of Maple in the evaluation 

of boolean expressions, CAREFSQz has an interactive feature that allows the user, before writing the 

specifications, to simplify boolean expressions and the 

expressions containing array notations. 

3-7 
abstraction helps in making the specification process 

more general and easier. The following seven operations are defined for atomic and stream ports: 

Language 

Based Editor 

Figure 3: Overview of CARE-FSQz. 

In a typical CARE-FSQ2 session, the user derives 

the specifications of the program using step-wise abstractions. In other words, the user starts by trying to 

find the correct specification of every loop in the program as a separate entity. After succeeding in this, 

the correct specification of the whole program can be 

found. This methodology of step-wise abstraction enabies the software engineer to concentrate on small 

pieces of code, one at a time, and to mitigate in this 

way the difficulty of specifying the whole program. 

Currently, CARE-FSQ2 supports a subset of Ada 

with modifications on the input/output mechanism. 

The data types supported are integer, boolean, character, a restricted form of floating point, constrained 

arrays, and user defined data types. The basic control 

structures of Ada are supported except unconditional 

‘go to.’ statements, and case statements. Static semantic checking is also included. A brief description 

of the input/output mechanism and the specification 

language is given in the rest of this subsection. 

Input and output is done through atomic and 

stream ports(l21. A subprogram, called an elementary 

process, accepts input data from input ports, performs 

computation specified with ail Ada-like notation. and 

returns results through output ports. The input and 

output of single data items can be carried out through 

atomic ports. Stream ports are used as schemes for 

data types whose elements can be accessed in a linear 

order. The stream ports of one process can be bound 

to particular data types to produce the implementation. Input’ and output ports can be bound to files 

to communicate with the system. This form of data 

1. Receive(p): To Receive a value via the input port 

p from the source associated with the port. 

2. Send(p): To Send a value via the output port p 

to the destination associated with the port. 

3. Initialize(p): To open the stream associated with 

the stream port p for reading. 

4. Receive(p, u): To receive a value into a variable 

u from the stream associated with the input port 

P. 

5. Send(p, u): To send the value of variable t, to the 

stream associated with the output port p. 

6. isEOS(p): A boolean function to check if end of 

stream is reached in the input stream port p. 

7. Finalize(p): To close the stream associated with 

the port p. The effect of finalization for an output 

stream port is that the function isEOS becomes 

true at the consumer process. 

The specifications for CARE-FSQ? are written using guarded command sets whose syntax is: 

< guarded command set > ::= 

< guarded command > 

{ I c guarded command >} 

< guarded command > ::= 

< boolean expr > - 

< concurrent assignment > 

< concurrent assignment > ::= 

< var > := < etpr > I < var > , 

< concurrent assignment > , < expr > 

A concurrent assignment is an extension of the assignment statement where a number of different variables 

can be substituted simultaneously. The concurrent 

assignment statement is denoted by a list of different variables to be substituted at the left hand side 

of the assignment operator and an equally long list of 

expressions as its right hand side. The ith variable 

from the left hand list is to be replaced by the ith expression from the right hand list. The expressions can 

include calls to some mathematical functions such as 

min, mas, product, sum, factorial, igcd (greatest common divisor), irem (remainder), and iquo (quotient). 

3-8 

1ooo57Wl 
An array is considered to be a partial functio? from 

subscript values to the type of array elements. The 

command a(i) := e assigns a new function to a, a 

function that is the same as the old one except that at 

the argument i its value is e. The notation (a, i, e) is 

used to denote the array that is the same as a except 

when applied to the value i yields e. The notation 

(a,indez = m..n,e) is used to denote the array that 

is the same as u except when applied to index values 

between m and n, i.e. m 5 indez 5 n, it yields e. The 

expression e can be a function of the bound variable 

indez. To make the two notations consistent, (a, i, e) 

is written (a,indez = i,e) where indez is a bound 

variable. The notation defined for arrays are used for 

stream ports as well. A stream port is treated as an 

array whose subscript is of type integer with the first 

element subscript being one. 

- 

. --. y1 :-y; 

{me ->xl.yl :-idn(xl.yl).min(xI.yl)) Wwle x 1 /- yl loop irxi >y1 rhen Xl :-x1- 1; el.= 

end If; 

yl :-y1- 1; 

emd loop; b :- 1; 8 :-x1; 

Cm > 0 -5 a. b :-0, b * hc%orld(m) I e-0 -> I) vrmla m > 0 loop 

end loop; 

b :- b - 8; 

m :- m - 1; 

z :- b; 

ScndCz); 

Figure 4: The program to be specified. 

4.2.3 Example: We describe a short example, due 

to the space limitation, to demonstrate a sample result obtained using CAREFSQ?. In order to find the 

correct specification of a while loop, the user should 

annotate it with a trial loop function enclosed between 

two curly braces. CARE-FSQ? assists the user in verifying the correctness of the loop specification by calculating the composition [if 'B then S fi] o f. The 

user, on the other hand. must ensure that the three 

while loop verification conditions are satisfied. After 

verifying all the wiiile loops in the program. the user 

expr : (xi-yl < 0 or yl-xl < 0) and yl-xl < 0 

Uould you like to sinplify this expression? Cy/nl: y 

Enter the simplified expression: y1 < xl 

sxpr : (xlyl < 0 or yl-xl < 0) and not yl-xi < 0 

Uould you like to shplify this expression? Cy/nl: y 

Enter the sinplified expression: yl > xi 

sxpc : not (xl-yi < 0 or yl-xl < 0) bld 

Entar the sinplified expratsion: yl = xi 

like to sinplify this expression? Ly/nl: y 

The SrJnbOlic execution result is : ----72='=----z-=5 

yl < xl -> 

xi, gl := 

mirdxl-Lyl). nin<xl-l.yl) 

I 

yl > xl -> 

xl, yl := 

nin(xl,yi-l), nin<xl.yl-1) 

I 

yl = xi -> 

wi, yl := 

mid xi. 81) , min(x1,yl) 

Figure 5: Finding the specification of the first loop. 

sxpr : -a < 0 d -a4 < 0 

Would you like to simplify this expression? ty/nI: y 

Enter the simplified expression: a > 1 

Would yw like to simplify this expression? Cy/nl: y 

Enter the simplified expression: a = 1 

expr : not -a < 0 and 8 <= 0 

Uarld yau like to simplify this expression? Cy/nl: y 

Enter the siwlified expression: a <= 0 

GXW : -8 < 0 8d 8-1 <= 0 

Figure 6: Finding the specification of the second loop. 

can proceed to find the functional meaning of the 

whole program. 

Figure 4 shows a program that receives two integers 

as input , finds their minimum, calculates its factorial 

. 

3-9 
if it is positive, and then saves the result in z. First, 

the verification ronditions of the two while loop have 

to be checked. Hence, we let CAREFSQz print the 

composition [if B then S fi] of to assist us in this 

process. Before printing the results of the composition, the user is prompted to enter his simplifications 

for some expressions if he/she desires(see figures 5 and 

Since the three verification conditions are satisfied 

for both loops, we can therefore proceed to find the 

functional meaning of the whole program which is 

shown in figure 7. 

- 

6). 

Figure 7: Specification of the whole program. 

5 Conclusion 

In this paper, we have presented a process model 

for extracting reusable components. It first identifies 

these components using software metrics, then it qualifies them. We have focused on the qualification phase 

which generates their formal specifications, generates 

a significant set of test cases, and packages them for 

future reuse. We have then described the specification tool of the qualification phase, CARE-FSQ2, that 

helps in understanding programs by generating their 

correct formal specifications. Further research needs 

to be done in order to be able to qualify and tailor 

large programs for reuse. 

Acknowledgement 

Research for this study was supported in part by 

NASA (Grant NSG-5123), ONR (Grant N00014-87- 

k-0307), and Italsiel S.p.X. (IAP Grant). 

References 

(21 V. R. Basili, ”Software Development: A 

Paradigm for the Future”, Proc. Compsac’89, 

IEEE Computer SOC. Press, Los Alamitos, Calif., 

Order No. 1964, pp. 471-485. 

[3] H. D. Mills, ”The New Math of Computer Programming”, Communications of ACM, vol. 18, 

no. 1, Jan. 1975, pp. 43-48. 

[4] J. D. Gannon, R. B. Hamlet and H. D. Mills, 

”Theory of Modules”, IEEE ‘12ans. on Software 

Engineering, vol. SE-13, no. 7, July 1987, pp. 820- 

829. 

[5] V. R. Basili, G. Caldiera, G. Cantone, “A Reference Architecture for the Component Factory”, 

Technical Report CS-TR-2607, Institute for Advanced Computer Studies and Dept. of Computer 

Science, Univ. of Maryland, College Park, MD 

20742, March 1991. 

[6] G. Caldiera and V. R. Basili, ”Identifying 

and Qualifying Reusable Software Components”, 

IEEE Computer, Feb. 1991, pp. 61-70. 

(71 C. A. R. Hoare, “An Axiomatic Basis for Computer Programming”, Communications of ACM, 

vol. 12, no. 10, Oct. 1969, pp. 576-580, 583. 

[8] E. W. Dijkstra, ”A Discipline of Programming”, 

Prentice Hall, 1976. 

[9] H. D. Mills, V. R. Basili, J. D. Gannon, and R. G. 

Hamlet, “Principles of Computer Programming: 

A Mathematical Approach”, Boston, MA, Allyn 

and Bacon, 1987. 

(lo] T. W. Reps and T. Teitelbaum, The Synthesizer 

Generator Reference Manual, Springer-Verlag, 

1989. 

[ll] B. W. Char et al, Maple User’s Guide, Watcom 

Publication Limited, Waterloo, Ontario, 1985. 

[12] B. Joo, ”Adaptation and Composition of Program Components”, Ph.D. Dissertation, Dept. of 

Computer Science, Univ. of Maryland, College 

Park, Maryland, 1990. 

[l] R. G. Lanergaii and C. A. Grasso, ”Software Engineering with Reusable Design and Code”, IEEE 

Trans. on Software Engineering, vol. SE-10, no. 5, 

Sept. 1984, pp. 498-501. 

3-10 
Automated Support for Experience-Based Software Management /’- (3 IVgcf3- 1 : -: ,,i 0 1$ 4 Jon D. Valett 

NASA/Goddard Space Flight Center 

Software Engineering Branch Code 552 

Greenbelt, MD 20771 

internet jvalett@gsfcmail.nasa.gov 

phone: (301) 286-6564 

FAX: (301) 286-9183 

Abstract 

To effectively manage a software development project, the software manager must have access to key information concerning a project’s status. This information includes not only data 

relating to the project of interest, but also, the experience of past development efforts within the 

environment. This paper describes the concepts and functionality of a software management tool 

designed to provide this infomation. This tool, called the Software Management Environment 

(SME), enables the software manager to compare an ongoing development effort with previous 

efforts and with models of the “typical” project within the environment, to predict future project 

status, to analyze a project’s strengths and weaknesses, and to assess the project’s quality. In 

order to provide these functions the tool utilizes a vast corporate memory that includes a data 

base of software memcs, a set of models and relationships that describe the software development environment, and a set of rules that capture other knowledge and experience of software 

managers within the environment. Integrating these major concepts into one software management tool, the SME is a model of the type of management tool needed for all software developmen t organization s. 

Keywords: software management, measurement, reuse of experience, management tools 

1.0 Background 

Good software management is generally viewed as a critical ingredient in successful software projects. One key aspect of good management is having access to the data that are necessary to understand the strengths and weaknesses of an ongoing development effort. To provide 

such access, a myriad of management-oriented tools have been developed. These tools typically 

allow the software manager to perform cost and size estimation, to plan a development project, 

to set up work-breakdown structures, and to provide other planning needs. Such tools are certainly useful, yet they do not provide the full scope of functionality required for a manager to effectively evaluate a software project. 

Ideally, an experience-based software management tool would enable a manager to observe 

3-11 

1WO6788L 
a project’s progress, to compare that progress with other projects or with a madel of how a 

project “normally” behaves, to predict key projtct parameters such as Size, completion date, or 

mrs, to assess the project’s progress pointing out its strengths and wcahesses, and to analyze 

the quality of the software project and the software product. In order to provide this 

functionality, the tool would require access to key data relating to a project’s status and to the 

past experience necessary to understand and manage the ongoing project. Included in this 

knowledge and experience is a data base of software metrics, a set of models of a development 

environment, a set of management rules that provide insight into a project’s strengths and weaknesses, a set of quality definitions, and a set of relationships that help to define an environment’s 

characteristics. Such a management tool would integrate this experience into a single environment providing the functionality required to actively monitor a software project. 

A working model of the management tool described above is being developed within the 

Software Engineering Laboratory (SEL) at NASA’s Goddard Space Flight Center (GSFC). This 

tool, called the Software Management Environment (SME) uses software measurement and the 

experience acquired fiom software measurement as its basis. Other tools either are being or have 

been developed that utilize measuxement as a major component. These tools include TAME [ 11, 

Amadeus[2], and GINGER[% SME is a unique experience-based tool because it focuses on 

utilizing the measurement and the experience of a measurement program to automate support for 

project managers in actually monitoring the progress of their projects. While the SME has been 

constructed for a specific development environment, the concepts, architecture, and functionality 

of the tool, which are described in this paper, are general enough for any organization to build a 

similar tool. This paper will discuss the management activities that the SME addresses, the 

components needed to build an SME, and how these components are integrated to provide the 

management functions described. 

342 
2.0 Management Activities 

In order for the SME to be an effective tool, it must automate key management functions. 

While the current SME is not comprehensive in its coverage of all management functions, it does 

provide support for many important aspects of software management. The SME utilizes a 

measurement-based approach to software management. Within this approach reusing 

management experience is viewed as an important aspect of the management process. This 

experience-based approach to management includes the following activities: 

Observation and Comparison: The manager monitors the progress of a project by examining 

key project measures such as effort, size, and errors. The manager compares the status of the 

current project with past projects and with models of these measures that represent the nominal 

case within the environment. By observing and coniparing, the manager is able to determine the 

current project's status and the differences between the current project and the normal project 

within the environment. 

Prediction and Estimation: The manager estimates key project parameters such as project cost 

and size. The manager also, uses various models and relationships to continually update these 

predictions. These activities allow the manager to determine at-completion values for important 

measures and to estimate project schedule. 

Analysis: Based on the measurement data, past project experience, and subjective information 

about a project, the manager identifies potential project problems. 

Assessment: Using available measurement data and definitions of project quality, the manager 

assesses the overall quality of the ongoing project. For example, these quality assessments 

provide the manager with an idea of the project's maintainability, correctability, and stability. 
A software am1 should only attempt to avtomatc aspects of a process that axe understood 

well enough to perform manually; in the case of SME, all of the activities described above are 

carried out on projects within this development environment In fact, such activities an part of 

the nd management process. The SME integrates data and expcriensc into one tool that 

provides managers with functions that help them to perform these activities. 

3.0 The Software Management Environment (SME) 

The Software Engineering Laboratory (SEL) has actively been developing the management 

concepts that are the basis for the SME for the past 15 years. A prototype of the tcwl was develogm'. between 1984 and 1987; this prototype provided a set of recommendations rkr developing 

an accrlal version of the tool.[41 This set of recommendations was then incorporated into ihe ac- 

:ad development of the SME, which began in 1987. The remainder of this section will discuss 

*,e SEL and the concepts that are the underlying ideas fcr the SME. 

3.1 The Software EngineerihgLaboratory 

The SEL was established in 1976 and has three primary organizational members: 

NASNGSFC, Software Engineering Branch; The University of Maryland, Computer Science 

Department; and The Computer Sciences Corporation, Software Engineering Operation. The 

goals of the SEL are (1) to understand the software development process in the GSFC environment; (2) to measure the effEcts of various methodologies, tools, and models on this process; and 

(3) to identify and then LCI apply successful development practices.[S] During the SEL's 15 years 

it has collected data on over 100 software development projects. These data include such items 

as software development effort, software size, erwr data, change data, and computer utilization 

data and are stored in a lilrgc repository called the SEL data base.[6] This data base has been 

3-14 

1- 
used throughout the past 15 years to help the SEL to accomplish its three objectives. In the process of studying and measuring this particular development environment the SEL has produced 

numerous reports and papers which characterize this environment, evaluate various tools and 

methods, and capture experience and lessons learned in various software development efforts. 

(For a complete list of SEL documents and reports see the "Annotated Bibliography of Software 

Engineering Litemture".[71) 

Throughout the SEL's history, this software measurement program has been used extensively in the management of actual software projects. Such use of measurement data is common 

among companies that have instituted measurement programs (eg. reference [SI). As this use of 

measurement as a management tool evolved, the SEL began attempts to automate the process. 

Such automation is only possible through a comprehensive understanding of how to use software 

measurement data within a particular development environment. Within the SEL environment, 

software managers use not only the data collected on their current project, but also, the 

information and experience from past projects. The studies and reports characterizing the 

environment provide the manager with profiles of how particular measms behave, numerous 

relationships for estimation and prediction of such measures, and lessons learned concerning 

how to analyze measurement data. Automating the access to this vast corporate resource is the 

goal of the SME. 

3.2 SME Concepts 

Understanding the SME requires a firm understanding of the three major components that 

are the basis for the tool. The first is the SEL data base, it provides the historical data of past 

projects, as well as the dynamic data on projects that are currently being managed. The second, 

is a set of models and relationships that describe the development environment. These models 

and relationships provide the profile of a normal project, as well as the necessary information to 

predict and estimate key project parameters. Finally, experienced software managers analyze 

3-15 

lOoo5T88L 
measunment data to determine a project's strengths and weaknesses. The knowledge required to 

perform this analysis is captured in management rules that provide the expert analysis portion of 

the SME. These thrte SME concepts provide the experience base needed for an organization to 

construct an SME-like tool. 

An important aspect of these SME concepts is that the experience they represent continually 

evolves as the development environment and process changes. The SME packages the current 

level of experience; as it changes, the experience base is refined to reflect these changes. The 

representation of the experience, however, does not change. Therefore, the key aspect of the 

SME, from the perspective of someone who wishes to build a similar tool, is the concepts and 

the architecture of those concepts, not the experience itself. 

Software Mea suremen t Da 

Measurement of the software development process and its products is a necessary component of successful software management. Within the SME, data from the SEL data base is utilized to provide the underlying measurement data. The SEL data base captures information on 

* 

all software projects within one particular development environment. This data includes such 

items as the weekly effort expended on a project, the size of the ongoing software project (in 

both lines of code and number of modules), the amount of computer utilization on a project, and 

the number of emrs uncovered as well as the number of changes made to the source code. In 

addition to these basic measures, the SEL data base contains data on such items as number of 

modules designed, number of open problem reports, and the amount of time spent uncovering 

and repairing errors. While these lists of data are not complete, they do provide a snapshot of the 

types of data available to the SME. 

The SME uses the data from the SEL data base as a basis for all of its analysis, comparison, 

prediction and assessment. The data provide the information that characterize and describe the 

current software development project as well as past projects of interest. Having access to so 

much descriptive data allows the SME to provide its wide range of functionality. Thus, software 

3-16 

1ooo5786L 
meas”ent is the backbone of the SME. Measurement provides the basis for all other SME 

concepts; neither the management rules nor the models and relationships would be possible 

without it. 

The second component of the SME is the models and relationships that represent the software development process and its products. The models and relationships used within the SME 

and presented within this paper are derived from numerous previous SEL reports and studies. A 

summary of the types of models and relationships used can be found in the document “The 

Software Engineering Laboratory (SEL) Relationships, Models, and Management Rules”.[9] 

The term model is used to describe a pattern of how some measure or combination of 

measures normally behaves within a software development environment. Measurement models 

have been described in numerous SEL reports and papers, but they have generally all been 

developed using similar methods. Typically, a model for some particular measure is developed 

by examining the data for that measure over a set of similar projects. The data is then combined, 

usually using some type of averaging, to develop a model of the “normal” project. Since even 

within one environment all projects may not be homogeneous, different models for the same 

measure are developed for significantly different project types. Within the SME, there are 

currently two different modcl types, depending on the development mcthodology used on the 

projects. Other models may need to be developed depending on such parameters as project type, 

programming language, or development environment. Deciding what different factors constitute 

a distinct model type is an important research component of developing an SME. Certainly, each 

individual project is distinct, but usually projects within a development environment have many 

similarities that result in reasonable models. 

. 

As an example of a model that is used by SME, Figure 1 shows how source code grows 

within the SEL environment. (For the purposes of this paper, there is no need to distinguish 

between various model types.) It provides a representation of the typical growth of the number 
of source lines of code within a project’s controlled library. The wide band indicates a range of 

what is considered to be “normal” source code growth. (In this case the range is one standard 

deviation on either side of the actual model.) As another example, figure 2 is the model of error 

rate for the SEL environment. This model shows the typical errors uncovered and repaired per 

line of code within the environment throughout a project’s lifetime. Again, the band represents a 

range over which the error rate is considered “normal.” (In both Figuns 1 and 2, lines of code is 

defined as physical lines including commentary and blank lines. In Figure 2, error is defined as a 

conceptual error in the software.) Another kind of model used within the SME is of the amount 

of time spent in each phase of a project. This model is depicted in Figure 3; it provides a mechanism for determining how much calendar time a project normally spends in each phase of the 

software development life cycle. 

Relationships, on the other hand, provide the SME with a way to estimate critical project 

factors based on other estimates, or current status. Relationships are typically developed by 

using numerous software development projects’ data to determine if any correlation exists 

between various measures. Normally, such data analysis is done to test hypotheses that certain 

relationships exist between such measures. 

As an example, within the SEL environment, a relationship has been found between lines of 

code and the actual durarion of a project. This relationship is shown as the equation: 

D = 5.450 * L ** 0.203 

where, 

D is the duration of the project in months (from project start through acceptance test), and 

L is the total delivered lines of code in thousands. 

Such a relationship allows a manager to estimate the length of a project based on an estimate of 

the number of lines of code for that project. Other relationships have been established between 

computer use and lines of code, effort and number of modules, etc. Such relationships provide a 

software manager both a mechanism for estimating various parameters and a consistency check 

for sets of estimates. 

3-18 
w 

0 

8 

100. 

73. 

50. 

25 

mw 

..................................... 

..................................... 

..................................... 

..................................... 

Schedule 

100. ox 

Figure 1: Modal of Source Coda Growth 

.................... I 

....................... 

....................... 

Schedule 

Figure 2: bbdel of Errors/Line of Cob 

3-19 
- 

Capturing how experienced softwaxe managers use and evaluate measmment data has been 

investigated by the SEL.[lO] These studies show that using expert systems techniques for the 

capture and use of this experience is feasible in this domain. This knowledge about software 

measurement has been published in numerous SEL reports and it provides a foundation for creating an experience base for utilizing software measures in management[9] The concept of these 

software management rules is that interviewing software managers and capturing how they interpret certain conditions of a project provides reusable knowledge concerning the strengths and 

weaknesses of a project. These interpretations are then combined into specific management rules 

that describe the possible explanations for certain conditions. For example, figure 4 shows a 

graphic of a simple management rule. This figure shows how one might interpret a deviation 

from the normal pattern of computer use per line of source code (again represented as a model 

similar to those described in the previous section). For example, early in the project if the number of CPU hours per line of code is above normal one possible interpretation is that the design 

was not actually complete. Later in a project, if the measure is below normal, the possible explanations might be either low productivity, or insufficient testing. Such a figure provides a simple 

representation of a management rule. 

Actually, a number of simple management rules can be combined to form rules that describe 

the possibilities that certain explanations are true. For example, a rule such as 

If the number of programmer hours per software change is above normal and 

the project is early in the code phase then possible explanations are 

Good solid, reliable code (05) 

Poor testing (02s) 

Changes are hard to isolate (025) 

Changes are difficult to make (0.25). 

describes the possible explanations for a certain condition. This rule uses numbers to show the 

3-20 

1oomTBBL 
-t Cod. Sy8tmm t Tost Acaaptura \ Tast 

30% 20% 155 

t 

h8ip 

35% 

Figure 3: Modal of Project Sahedule 

o.ooi0 

0.0008 

0.0005 

0.0003 

DES ON RCCTE I I 

0.0008 

Schedule 

Figure 4: Rule for Analyzing Computer Uae 

3-21 
certainty that each of the possible consequents are me. Thus, it is more likely that good solid, 

reliable code is the explanation for the deviation then poor testing, although either explanation 

could be true. This rule is then combined with other rules for other measu~e deviations to increase the certainty that particular explanations are comct. Using this method of evaluating 

software measures provides a set of possible explanations describing a project’s strengths and 

weaknesses. By using sets of rules in this manner, an automated system can examine the 

empirical evidence about a project and provide some insight into the project’s status. 

4.0 Using the SME 

This section describes how the SME utilizes the concepts described above to provide its 

functionality. While the concepts of the SME are the most important aspect of the tool, 

understanding how to utilize those concepts to provide management support is also of interest. 

Attempting to build an SME-like tool requires knowledge of how to integrate the experience into 

a useful tool. The examples used are realistic in that they show the actual functionality of the 

SME, however, due to the inability to reproduce the color SME images, the graphics images are 

in black and white. 

ComDanso n 

One major function of the SME is the ability to observe data and compare it to models and 

previous development efforts. Figure 5, shows an example of using the SME to compare data to 

a model. In this example the manager is looking at the way error rate behaves on the project of 

interest. The current project is shown as the solid line and the model is shown as a band of what 

is considered “normal” for error rate. The x-axis shows the expected schedule for the project. 

That is, the start date and end date shown are the manager’s estimates, however, the other phase 

dates shown are the expected phase dates for the project (as calculated by the SME). The tool 

3-22 

1oO05~L 
il 21 12 is 09 Hodcl 

89 89 90 90 91 Sch 

Figure 5: Rate of 'Reported Errorr/Liner of Coda' for Project A 

3-23 
also shows the manager's estimates for all the phase dates on the top of the screen. The Y-axis 

shows the error rate in errors per line of source code in the controlled library. Note that the phases represent a typical watexfall life cycle, with the major phases being design, code and unit test, 

system test, and acceptance test. By using this comparison, the manager is able to track such key 

items as emr rate, productivity, and amount of computer time used. Additionally, the manager 

is able to overlay other projects' error rate patterns in order to compare the behavior of those 

projects to the current project. 

.. 

lCW 

Figure 6, provides a look at another function of the SME. This figure is similar to the comparison figure, except that it also shows a predicted final value for the measure. In this figure, 

the measure of interest is computer use (in number of CPU hours). This is shown in absolute 

terms on the Y-axis. That is, the actual amount of time used on the machine is shown (it is not 

normalized). The SME allows the user to predict where the project will be when it is completed. 

This function utilizes the model and a projection of the progress of the project based on the measures in SME (eg. the project is 50% of the way through the code and test phase), to predict the 

final values of the measure, and of the schedule. In this example, the number of CPU hours on 

the project is predicted to be 1255, while the current estimate is 990 hours. Also, the project is 

predicted to take longer then the manager has estimated. Such predictions enable the software 

manager to gain another perspective on the final values of project measuxes and on the projected 

end date of the project. 

Analvsls 

A key component of the SME is the utilization of expert systems technology for software 

management. Through experience, software managers are able to improve their ability to analyze software measurement data. Based on the measurement data and their experience, managers 

are able to identify the strengths and the weaknesses of a project. The SME utilizes a rule base 

3-24 

"L 
that captures managers’ knowledge of how to perform such analysis. This rule base is then used 

to analyze deviations from the normal project. An example of such analysis is found in figure 7. 

In this figure, the error rate of the current project is lower then nonnal for this particular point in 

the development life cycle. The SME uses this information, information about other measures, 

and subjective data about the project to provide possible reasons for such a deviation. The top 

two explanations arc then displayed for the user. In this case, the explanations are that insufficient testing is being performed and that an experienced development team is producing a superior project. Either of these two explanations might be correct, they only provide insight to the 

user as to possible explanations for the deviations. Other explanations are certainly possible; the 

user of the tool can obtain further data on why the system reached its conclusions and on the 

other conclusions. The user can also provide the system with more Subjective information about 

the project of interest, perhaps leading to changes in the conclusions that are inferred. 

Assessment 

A fmal function of the SME is to utilize software measures to provide an assessment of the 

overall quality of a software project. An example of such an assessment is shown in figure 8. In 

this figure the bar graph shows the SME’s rating of certain quality measures as they compare to 

the nonnal project in the environment at that point in its development. The quality factors shown 

are maintainability, reliability, and stability. Each of these factors can be determined by combining various software measurement data. For example, the quality factor of maintainability is calculated by adding the percentage of errors that are easy to isolate with the peIcentage of emrs 

that are easy to correct. Thus, as these percentages increase the maintainability of the project is 

said to increase. For each quality factor displayed, SME has a specific definition for how to 

compute that factor. These definitions, which are really a form of a relationship, use a specific . 

set of measures to compute the relative value of that quality indicator. Of course, SME also uses 

a model of how these factors behave over time in order to display the normal band on the graph. 

Quality assessment provides the software manager with an overall appraisal of how the project of 
8 

U 

w 

1250 

1000 

750 

500 

250 

03 09 05 09 02 

ll 09 05 is 

89 . 89 90 so 91 

o9 Muugrr's 

Schedule 

03 11 06 10 03 SHE 

il 04 09 20 30 Rmdicted 

89 89 90 90 91 Sch 

6: Prrdictrd Growth in *CPU Hours* for Rtojrct A 

03 09 05 09 02 

11 09 05 15 

89 89 90 90 91 

o9 Manager's 

Schrdulr 

03 10 05 09 02 SME 

11 21 12 15 09 Model 

89 89 90 90 91 Sch 

Figure 7: Analyds of *Reported Errors/Lincs of Code' for Projmct A 

3-26 
Mainkinrbilit y Rdirbility Stability 

Overall Aaaesamsnt for Project A 

Figure 8: Overall Assessment Function 

3-27 
interest is doing compared to the normal quality measures in the environment. 

5.0 SME as a Model Tool 

Currently, the SME is being used by numerous software managers in the SEL software 

development environment to assist them in monitoring actual software projects. The SEL, as an 

experience factory [5], has provided the concepts necessary to build an SME for this particular 

software development domain. Other organizations can develop an SME-like tool by beginning 

to capture the experience of their environment. While within the SEL environment all three of 

the major components of SME have been well developed, other organizations may have only 

limited parts of the components, Such limitations should not be viewed as detrimental to the 

development of an SME. Similar tools should be developed using the experience available; they 

can then evolve into more complete tools as the local experience base provides additional 

artifacts for reuse. 

The SME is an attempt to integrate a measurement process, the results of a longstanding 

software engineering research effort, and the expertise of software managers into a tool for managing and controlling software projects. As such, it provides for the utilization of corporate 

experience to manage ongoing software projects. An SME has been built for one particular software development organization. Other software development organizations should use the 

SME's concepts as a model for building similar tools for their environment. By providing the 

user with increased project awareness, predictions of key project parameters, expert analysis of 

software measures, and assessment of the overall quality of the development effort, an SME is 

extremely valuable to a software manager. Such a tool provides improved project management 

through the packaging of experience. 

3-28 
References 

133 

141 

r51 

[71 

PI 

PI 

Basili, V. R and H. D. Rombach, ‘The TAME Project: Toward Improvement-Oriented 

Software Environments,” IEEE Transacrwns on Sofhvare Engineering, June 1988, pp. 

758-773. 

Selby, R. W., et al., “Metric-Driven Analysis and Feedback Systems for Enabling 

Empirically Guided Software Development,” Proceedings of the 13th International 

Conference on Software Engineering, IEEE Computer Society Press, May 1991, pp. 288- 

298. 

Kusumoto, S., et al., “GINGER: Data Collection and Analysis System,” Technical 

Report, Osaka University, Osaka, Japan, June 1990, 

Valett, J., “The Dynamic ’Management Information Tool (Dynamite): Analysis of Prototype, Requirements, and Operational Scenarios,” Master’s Thesis, University of Maryland, May 1987. 

Basili, V.R., et al. “The Software Eiigineenng Laboratory - An Operational Software 

Experience Factory,“ Proceedings of the 14th International Conference on Software 

Engineering, IEEE Computer Society Press, May 1992. 

So, M. et al., “SEL Data Base Organization and User’s Guide (Revision 1);’ SEL-89- 

101, The Software Engineering Laboratory, NASA Goddard Space Flight Center, Greenbelt, Maryland, February 1990. 

Morusiewicz, L. and J. Valett, “Annotated Bibliogmphy of Software Engineering Laboratory Literature,” SEL-82- 1006, The Software Engineering Laboratory, NASA Goddard 

Space Flight Center, Greenbelt, Maryland, November 1991. 

Grady, R., “Work Product Analysis: The Philosopher’s Stone of Software?,” IEEE Software, March 1990, pp. 26-34. 

Decker, W., R. Hendrick, and J. Valett, “The Software Engineering Laboratory (SEL) 

Relationships, Models, and Management Rules,” SEL-9 1-00 I, The Software Engineering Laboratory, NASA Goddard Space Flight Center, Greenbelt, Maryland, February 

1991. 

Ramsey, C. and V. R. Basili, “An Evaluation of Expert Systems for Softwaxe Engineering Management,” IEEE Transactions on Sofrware Engineering, June 1989, pp. 747-759. 

3-29 

SECTION 4 - SOFTWARE MODELS 

SECI’ION AOFIWARE MODELS 

The technical papers included in this section were originally prepared as indicated 

below. 

“The Software-Cycle Model for Re-Engineering and Reuse,” J. W. Bailey 

andV R. Basili,Pmceedings of theACM Tn-Ada 91 Conference, October 1991 

“On the Nature of Bias and Defects in the Software Specification Process,” 

F! A. Straub and M. V. Zelkowitz, Proceedings of the Sixteenth Intemational 

CornputerSo@ulle undApprications Confeence (COMPSA C 92), September 

1992 

“An Improved Cldcation Bee Analysis of High Cost Modules Based 

Upon an Axiomatic Definition of Complexity,” J. Tian, A. Porter, and 

M. V Zelkowitz, proceedings of the Third IEEE International Symposium on 

Sofrware Reliability Engineering (ISSRE 92), October 1992 

“Providing an Empirical Basis for Optimizing the Verification and %sting 

Phases of Software Development,” L. C. Briand, V. R. Basili, and C. J. Hetmanski, Proceedings of the mud IEEE Intmtional Symposium on Sofrware 

Reliability Engineering (ISSRE 92), October 1992 

“A Classification Procedure for the Effective Management of Changes During the Maintenance Process,” L. C. Briand and V. R. Basili, hoceedings of 

the 1992 IEEE Conference on Sofrware Maintenance (CSM 92), November 

1992 

4-1 

loO05788L 

The Software-Cycle Model for Re-Engineering and Reuse 

/ --/ 21 John W. Bailey’ 

Victor R. Bass 

Depanmcnt of Computer Science 

College Park, Maryland 20742 

+also consultant with Rational. 6707 Democracy Blvd. Bethesda, Maryland 20817 

--d i 

The University of Maryland c 

Fi9SU1.71~~/S 

i 

Abstract 

This paper reports on the progress of a study which will 

contribute to our ability to perform high-level, 

component-based programming by describing means to 

obtain useful components, methods for the configuration 

and integration of those components, and an underlying 

economic model of the costs and benefits associated with 

this approach to reuse. One goal of the study is to develop 

and demonsmtc methods to recover reusable components 

from domain-specific software through a combination of 

tools, to perform the identification, extraceion. and 

reengineering of components. and domain experts, to 

direct the application of those tools. A second goal of the 

study is to enable the reuse of those components by 

identifying techniques for confguring and recombining the 

re-engineered software. This component-rtcovexy or 

software-cycle model addresses not only the selecuon and 

re-engineering of components, but also their recombination 

into new programs, Once a model of reuse activities has 

been developed, the quantification of the costs and benefits 

of various reuse options will enable the development of an 

adapt2ble economic model of reuse, which is the principal 

goal of the overall study. This paper reports on the 

conception of the software-cycle model and on stvd 

supponing techniques of software recovery, maiurement 

and reuse which will lead to the development of the desired 

economic model. 

Motivation nnd Scope 

. hlouvation for the devclopment of an expert-assistcd but 

highly structured and highly automatable model of software 

infarmation capture and reuse stcms in part from the 

‘1991 AChl 0-69791 45-719 ll1090JJ267 51 SO 

lbcognition of the difficulty of using purely programming 

component-based approaches to reuse libraries. For terrain 

kinds of objects and components a smct programming 

component-based library is adequate. ’Ihe success of 

object-onenled and object-based approaches have becn the 

most notable in this regard. However, the inability for such 

libraries to capture a sufficient amount howledge to 

dramatically reduke subsequent software development costs 

in a gend and problem-independent way has also been 

obsuved On the other hand, models of software reuse 

which utilize domain experts in pervasive and undirected 

ways arc aIso uniikely to provide a complete solution due 

to the large amount of responsibility and effon which is 

centralized in the conmbution of such experts. The present 

work provides a structured model of information 

identification and reuse which is both feasible and suitable 

for further development and refinement. 

Using the Ada language, this paper provides examples of 

techniques for choosing. re-engineering. and recombining 

components into programs. It also describes rudimentary 

methods for quantifying the effon to exhact reuszble 

components from existing programs as well as the effort to 

recombine them into new programs. It does not include the 

cataloging and retrieval of components, nor does it include 

a mechanism to quantify reusability based on 

empirically-derived frequency-of-use measures. Ir does 

model a prcposed cycle of software development, use, 

re-enginering. and reuse, but it does not attempt to model 

other aspects of reuse within a sofnvare development 

environment, such as pure knowledge and experience. 

Ocher recent research papers and technicd repons have 

covcrcd this larger scope [Basili and Rombach]. pasiii and 

Caldiera]. 

Introduction 

Any componcnt of software is sccn to k composed of 

many functional and declarative dctails. some of which 

pertain to thc spccific problem bcing solvcd by thc program 

conraining that componcnt, somc of which penain to the 

general application domain of thc containing progam, and 

some of v:hich pc~n u> ncithher he problem nor the 
domain, but rathcr dcfinc thc esscnce of the component's 

function in the abstract. Thercforc, to dircct the sclcction 

and n-engineering of componcnu of software, thrcc lcvcls 

of functional spicificity of Lhc software which constitulcs 

my componcnt are dcfmcd: 1) problcm-spccilic derails 

which would be likcly to diffcr bctwun this and another 

similar application in thc samc domain. 2) domain-spccific 

details which arc not likcly to differ bctwecn this &d 

~0th~ similar application in thc same domain but which 

would bc unlikely LO be appropfiatc outside of this domain, 

and 3) essential aspects which comprisc the absuact 

fimctionai core of the component and without which the 

component would be meaningleis. ...- - - 

-* 'herefore. an economic equation presents itself, which is 

Along with rhcsc examplcs. proposals are given for how to 

measurc rhc amount of rc-cnginecring rcquircd to derive 

componcnts suitable for thc diffcrcni mcthods of reuse. as 

well as the amount of effon required to rccombine 

componcnts using the diffmnt mcthods. As effon is 

wrpcndcd to make a componcnt more general, mort 

opportunitics to rcusc it bccomc availablc. Howcver, each 

of those rcuse opportunities will have to rcsupply the 

specifics required for the reusable component to perform its 

function in the ncw contcxi, implying an amount of reuse 

effort which is proportional to the degree of generality of 

the component 

The thne levels cannot be absolutely defined, nor can a 

given detail be deterministically assigned to a level, since 

from different points of view, a given detail could be 

thought of as belonging to different levels of specificity. 

Two analyses of a given component could possibly identify 

different sets of details at each of the three levels. 

However, an analysis of a candidate component for the 

purpose of directing the re-engineering and reuse processes 

must assign each identifiable detail to one of the three 

levels. 

Once specificity levels have been assigned to all details of 

a candidate componenL a measurement of the effort 

required to remove each of the problem-specific details is 

obtained in order to estimate the total effort to gene& 

the component for reuse within its domain. Further, a 

measurement of the effort required to remove each of the 

domain-specific delails is obtained in order to estimate the 

total effort to generalize the component for reuse in other 

domains. If these measurements show the 

cost-effectiveness of either of these generaiizations. then 

the candidate component is suitably generalized and placed 

in either a domain-specific or domain-independent 

repository, as is appropriate. 

In order to assign specificity levels to all the constituent 

details of a candidate component, domain experts may have 

to be consulted. However, automation to suppon the 

identification of the details and to support the component 

generalization through their removal can be used u) 

sixmline the proccss. Further, there m2y be ways to 

mpmre the domain expens' decisions and the reasons for 

them, in order to partially automate or support any 

subsequent decision malung which follows similar patterns. 

To support the generalization process and iu quantification, 

three styles of softwzre component reuse which arc 

currently being practiced are idendied and examined for 

their adaprabilicy 10 the model. These reuse styles are 

termed layered, failored. and generaled reuse. Examples 

illusnaung them, and demonstrating how they are related 

by an underlying dimension of gentrality. are shown. 

how to optimize the somelimes competiing factors of 

generalization effort, reuse effort, and breadth of utility. 

The solution to this equation will have to wait until more 

work is done on the probability of reuse for a given 

generalization, and other facton. Rather hard questions 

figure in to this equation, such as the cost-benefit of 

constraining a solution to fake advantage of an available 

component (which amounts to establishing and following 

standards) as opposed to developing a more suitable one, 

and even the cost of classifying, storing and retrieving 

components. Developing a framework for an economic 

model which captures these factors is the first step to a 

greater understanding of these issues. The last section 

relates the activities defined in the software-cycle process 

model to this economic model of reuse. 

The Software-Cycle Model 

This section describes the model of software development 

which underlies this srudy. The model proposes the 

recycling of existing software into components which can 

be combined into new programs. This proposed sofrwore 

cycle takes place in the context of a software development 

organizdon and allows effort already applied to the 

creation of previous programs to be recaptured and used ro 

reduce the effort needed to create new programs. This 

sofnvare-cycle model is consistent with models of 

experience capcure and flow within a development 

organization as described by Pasili and Rombach] and 

pasili and Caldiera]. It describes in detail, and proposes 

an implementation for. one aspcct of the more 

comprehensive experience factory described in those 

studies. 

The software-cycle model is so-named to describe the flow 

of information and experience, in the form of software, into 

newly developed programs where it can be recovered and 

packaged for efficient reuse in subsequently developed 

software programs. The capture and reuse of information 

af the delivery point of the conventional software lifecycle 

is clearly not the only time at which such information is 
zcessible. However. this approach is chosen because a~ 

& rime that software is delivered, the information is 

packaged in a concrete form (sofware prognrms) which 

can be analyzed and manipulated. Also, a substantial 

mount of information may be available from 

previously-developed programs which is not racordcd in 

any form other than the delivered softwart. Further, by 

instituting an approach which applies effort fo capturt 

Eusabic information at this stage, the software 

development organization has he choice to separate the 

information recovery and repackaging from the effon to 

develop the software, and to conduct those activities 

independently and in parallel. So. for pragmatic reasons. 

the present model of information flow in a software 

development organization uses developed software as the 

'main source for recoverable information. (Set also 

[Caldiera and Bass].) 

I t 

n 

As shown in Figure 1, existing programs are examined for 

candidate reusable components. For the purpose of this 

study, a component can be any definable portion of 

software. Obvious examples are individual, or sets of. 

subroutines, subprograms, functions. paragraphs. packages, 

OT othcr structuring fcaturcs of the software language in 

use.. A re-engincered component can be any of thest. 

although ir can also be nothing more lhan a template or a 

sit of instructions for a software generation routine. 

A re-enginecred component can be intnded either for 

reuse only within a panicular domain or reuse across many 

domains. If a component is only intended for reuse within 

a domain, its rt-engineering seck lo remove any 

problem-specific details from it, but u) allow any 

domain-specific details to remain. Such components arc 

tumed domain-specific components. If a component is 

intended for reuse across domains, however, then its 

re-engincexing would attempt to remove all 

domain-specific details as well as the problem-specific 

details. leaving only essential function. "his kind of 

component is tcrmed a domain-independent componenr. 

Leaving a component insufficiently general to be used 

across domains obviously limits the number of 

opportunities it might enjoy for reuse. However, there 

significant compensating advantages. A domain-specific 

component retains more details which hen do not have to 

be resupplied by the reuse client. Also, the generalization 

effort to reach only problem-independence is usually less 

rhan the generalization effort required to reach 

domain-independence. So. ' by accepting a constrained 

reuse scope, a component can be easier to generalize as 

well as easier fo reuse. 

A candidate component for re-engineering is one which has 

idcntifnble problem-specific or domain-specific details and 

which can be feasibly mcngineered to eliminate the 

presence of some or all of those details. A domain expert 

may be needed to differentiate between problem-specific 

and domain-specific delails, and measurement of the 

estimated generalization effort is needed to determine the 

feasibility of the re-engineering. Some components may be 

candidarcs to yield a domain-specific component after 

recnginetrinj but not a domain-independent component 

Other components may be candidam to yield 

domain-independent components (possibly in addition u) 

domain-specific components), while still others may not be 

good candidates IO yield either category of reusable 

component 

The goal of reuse ncngintcring is to be able to isolate and 

then 10 replace the problem-specific andor the 

domain-specific aspecls of a component so that it can be 

made to operate in different contexts. A component might 

be view4 as a blend of general function, which defines iu 

essence, and specific function which relates to the current 

context or declarations on which the general function is 

performed. This is shown graphically in Fig- 2a. The 

general function, shown in light grey, is that which is 

essential to the component or that which dcfmes the nature 

of the component. The specific function, shown.in dark 

my, can either be problem-specific or domain-specific. 

As mentioned, it may be necessary to consult domain 

experts to distinguish between a problem-specific detail 

and a domain-specific dedl. Howcver, given a sufficient 

body of experience. it may be possible to predict the 

specificity of a detail via a predictive function that is 

tailorcd by prcvious upert decisions, or by statislical 

4-5 
analyses of several similar componenls in the same domain. 

Figure 2b 

Ailer re.enginrering. the usen~ia~ iunc~ionali~y remains in me 

ieus8bIe component buI problem.specific or conleal specilic delalls 

ue alimineted 8na become (he responribihcy ol Iht reusel Io piOvde 

One possible inaati8lion coda rerun in the origin81 component 89.1" 

bul many olher instantia~ions are now possible 

Figurt 2b shows an imaginary candidate component which 

contains both essential function, which is general, and 

specific details which. if altered, could allow the 

component to conmbute its functionality in different 

contexts. These specific derails, shown in dark grey, have 

ken removed from the body of the component to signify 

that they are now viewed as only one of potentially many 

possible instantiations of the remaining, general 

component The re-enginering process of the 

saftware-cycle model seeks to locale and remove these 

non-general aspects (either only the problem-specific 

aspects or, possibly, the domain-spa5fk aspects as well) 

and to relegate them 1.0 the responsibiliry of the reuser as 

part of the component's instantiation. The techniques for 

the removal of these derails are discussed as part of the 

=tion on re-engineering techniques which follows. It will 

be shown the= that the re-engineered component dots not 

need to be expressed in the programming language of the 

original candidate component which was used to produce 

ir It might be a pre-processable component or a 

component generator which can be uscd to produce 

components when necessary. In hex: cases. it is the 

tunplate or the generator that is reusable, since any 

subsequently requircd componcnts would bc produccd on 

dunand and would not. Ihcmsclves, be considered reusable. 

Separated and re-engineered (generalized) components arc 

stored in a repository to bc made availnblc to the 

developcrs of new software. Similar to the process of 

consulting domain expens when categorizing the details 

which nacd to be gcnenlized out of candidate components, 

repository experts may have to bc consulmi to assist in the 

location and instantiation of required components in the 

repository. Repository experts could possibly choose from 

among various schemes to satisfy the needs of a developer. 

Certain choices might provide more utility but might come 

with more restrictions or limitations of options. Also, the 

repository expert might choose from different methods to 

arrive at functionally the same result to the requesting 

developer, for example by either generaling the software or 

by providing a tailoxable component 

Componenrs in the repository are amibuted with 

measurement information describing the expected effa to 

instantiate them for reuse. In many cases, this instantiation 

becomes the responsibility of the reusing developer, for 

example when the component is already a smcM 

component in the developer's language of choice and 

simply must be supplied with actual parameters to serve the 

developer's need. In other cases. the instantiation can be 

the responsibility of the repository expert, who might have 

m produce components for the developer from templam. 

rules. instance specifications, and generator programs. In 

either case, the measurement attribute of a component will 

guide its users when deciding whether to select it or not. 

and how much effort to expect to expend configuring it for 

reuse. 

A request for software components might be unfiillable 

given the cumnt state of a repository. In this case, the 

repsitory expem can work with the developer to design 

and crate a new component which will not only serve the 

cumnt need but which will become 2n instant candidate for 

insenion into the repository. with a minimum of 

recnginecring. Or, gaps in the capabilities of rhe 

repository can be idenufied by the experts prior to a 

specific need. and special developments can be guided, 

specifically for the purpose of supplying components to fill 

those gaps. In the software-cycle model, any new 

development is done with reuse in mind, specifically.with 

an eye toward funher populating the component repository. 

Neither of these last two topics, the selection of 

components from a repository and the direct development 

of components rather than through n-engineering, are 

currently part of the study. They are mendoned here in 

order tn complete the sofnvare cycle depicted in Figure 1. 

4-6 

1ooo6788L 
The major emphases of the study are the identification of 

candidate reusable components from among existing 

software. the Fgineering of those components lo 

improve their gcncrality. the measurement of those 

processes, and the development of an economic model 

which can assist an organbation in optimizing its software 

cycle costs. 

Reuse Modes and Methods 

By studying the dependencies among software elements, a 

detmhation can be made of the reusability of those 

elements in other contexts. For example, if a component of 

a program uses or depends upon another component, then 

the first component would not normally be reusable in 

another program where the second component was not also 

present On the other hand, a component of a software 

program which does not depend on any other software can 

be reused in any context (ignoring for the moment whether 

or not it performs any useful purpose in that context). The 

issue of software independence is at the hean of this study. 

It will be Seen that ind independence of a software 

component often comes at the cost of functionality. The 

ideal software reuse re-engineering process would provide 

B means of preserving alI of the function or utility of a 

component while also making it independent of 

problem-specific or domain-specific details. However, this 

is not possible in most cases since some of the desind 

functionality is likely to be captured by those sptcific 

details, and removing the details will remove thar 

functionality. This study describes a compromise solution, 

which is first to generalize a component. and then to 

systemarize the means to configure it in order to restore the 

specific function required in a panicular context of rcuse. 

A scheme to maintain generalized, reusable components in 

a repository, in addition to a means of configuring them in 

different ways for different domains or contexts, enables a 

repository with a manageable number of components to be 

described. Without the ability to instantiate a given 

component in different ways for different usages, a 

repository would have to contain many times as many 

assets in order to serve the same need. In order to avoid 

this problem, this work recommends storing fewer 

components, each of which is sufficiently general to be 

able to operate in various contexts. and then providing 

methods to instantiate them to provide functionality in 

those contexts. 

By examining existing successes in software reuse, it can 

be Seen that there are three different but related ways of 

making software components which are general and 

independcnt, and yet which remain wpable of being 

instantiated with problem-spccific details. An important 

prunise of this work is that software which is general in 

these ways does not necessarily necd to be developed 

directly. Instead, it is often possible to re-engineer existing 

software so that it achieves the necessary independence. 

For this study, the ttuu modes (vt termed layered. 

tailored. and generured. Each mode describes components 

which can be combined to develop larger programs. 

However, a tailored component can be made more flexible 

and general than a layered component and a generated 

component can be the most flexible and general of all. On 

the other hand, a layered component is the easiest to reuse, 

requiring the least effort on the part of the client to 

incorporate it into a program, while a generated component 

is the most difficult to reuse. 

What all of these techniques smve for is the absence of 

dependence f" the reused software on external 

declarations, which would hamper the generality of the 

software. In other words, a component of reusable 

software should ideally not be expected to "brow" about 

declarations and other components which RE 

problem-specific. A reusable resource which requires the 

reuser to also include other common denominator 

components. which contain needed declarations, is not as 

reusable as one whichhas no such requirements. 

Within the confines of a single domain, however, certain 

dependencies can be tolerated, since the users can be 

expected to guaranw the minimum rquirtd declaration 

spa= across all occurrences of reuse of a component This 

result opens up vast new ranges of possibilities, since the 

generality of a component need no longer be absolute but 

rather need only be general with respect to a cedn domain 

or domains. No expectation of generality within other 

domains is maintained Domain-specific reusability 

implies a certain amount of built-in de+ndence whereas 

wide-scale reusability or generality precludes this 

possibility. By allowing domain-specific conspaints, the 

possibilities for identifying reusable components expand 

enormously but the breadth of applicability for each 

component is limited to that d3main. 

Layered Reuse 

Layered reuse is used to describe the case where reusable 

functions or operations arc viewed simply as abstract 

primitives which are callable from within the language of 

the client A math library, probably the most commonly 

cited example of reuse, and onc which is often viewed as 

an ideal, is an example of layed reuse. Analogous to a 

math package, other common examples are packages of 

utilities which operate on universal types or concepts, such 

as string handing utilities and time utilities. Other 

succcsscs in layered software reuse include user interface 

4-7 

looo5788L 
or UO toolkits. graphid display toolkits. runtime kemels, 

and laycrcd nctwork protocol software. 

Layered rcusability is oftcn viewed as the goal for a library 

of reusable components, where a sufficienlly rich set of 

abstract operations would bc availablc to an applications 

programmcr in order to minimize thc effon rcquired to 

generate a new system. In addition to the previously 

mentioned indcpcndcnce from other componenrs, an 

additional recommendation for the success of a layered 

component is hat the dab on its interface bc expressed in 

terms of standard types. This restriction allows the client 

solware to communicate with the reusable component 

without the additional complexity of adhering to specific 

non-standard types. One reason that a math library is so 

inherently reusable, for instance, is that real numbers arc a 

universal way of expressing the values used by and 

returned by the mathematical functions in a library. Any 

language which suppons real numbcrs can make available 

a corresponding set of mathematical functions. 

However, unlike the portability enjoyed when restricting 

one's domain LO a universal concept such as real numbers, a 

considerable amount of software which might otherwise be 

available for reuse is written to operate on problem-specific 

types and data structures. This is the case whether those 

types are named and declared as in Pascal OT Smalltalk, arc 

common data artas as in Foman, or are merely locations in 

memory as in assembly language, Components can still be 

haen in a layered manner but in these cases they typically 

depend so heavily on specific data structures that they arc 

limited to being reused only where identical data structures 

or other operands are present. It is not always possible to 

pvamettrize a component with respect to all of its 

assumptions about context Because of these limitations on 

the applicability of a layered component. constructing 

comprehensive reusable libraries of them in languages such 

as Ada has becn harder than might have been expected. 

Tailored Reuse 

. Another category of successful reuse is tailored reuse. 

where configuration of the reusable sofwarc is requircd in 

order to allow it to interoperate properly with the client 

sofware. A familiar example of such nuse is seen with 

database management systems which requirt tailoring in 

order to handle records of the user-defined strucolre~. 

Simpler examples of tailored reuse art generic data 

sm~ctures which allow the client software to cmtt stacks, 

queues, lists, etc., of application-specific types or to sxch 

rhrough or son objects of those types. Stiil other examples 

of tailored reuse are forms management systems which arc 

customized by parameterization, expen systems which 

must be initialized with rules, spreadsheets which must bc 

supplied with formulas, ar,d statistics packages which must 

be provided with data sets and programs to achieve the 

desircd mults. 

Tailoring in this way is accomplishcd before the 

compncnt is called, but it happcns automatically at 

execution time as pan of the language behavior. Whereas 

in laycrcd reuse a client simply calls a component with the 

proper parameters, tailored reusc implies a two-step 

process wherc a component is first moldcd to the specific 

configuration required by the currcnt context and is then 

called to perform its function. 

The generic feature of Ada allows certain kinds of 

tailoring, in the form of generic parameterization, to be 

accomplished. Because of the static checking enforced by 

Ada, however, only a limited amount of parametenzitions 

arc possible. Other languages have different mechanisms 

for accomplishing this parameterization. Most nowbly, 

assembly languages employ very flexible macro 

expansions which can be quite powerful. However, 

object-orientcd languages have traditionally used a more 

flexible form of layering (full inheritance) while 

overlooking the possibility for component 

parameterization. (Future revisions to C++, however, are 

expected to include a template mechanism to dow 

within-language tailoring [Ellis and Snousmpl.) 

Generated Rege 

The third category of reuse, generated reuse, occurs when 

the reusable softivare is USA as a generator program rather 

than being incorporated directly into the final application. 

The required software is emined as a result of the generaror 

pr0gn-n operating on input tables or files. Typically, only 

the generator and not the generated software is reused. The 

gene& software is regenerattd. as opposed t~ being 

modified directly. if changes an required. Whenas 

Iayd and tailored reuse take advantage of 

language-suppod features (subprograms and generics in 

rhc of the Ada lanpge) generated rcuse rquires 

akiitional tooling to accomplish a kind of tailoring which is 

external to the implementalion language. 

A common example of generated reuse. which perhaps 

mtches the definition somewhat, is a compiler, which 

accepts files of a high-order language and emits software in 

a machine-executable form. One reason that it may seem 

unconventional to think of a compiler as reusable softwart 

is that its output is not ktly manipulated or even 

obsaved by the compiler's users. Nevertheless, it fits the 

definition here for generated reuse (which could be thought 

of as a batch form of tailored reuse). 

Other common examples, where the generated output is 

more likely to be manipulated ur at least observed by the 

4-8 

100(#7ssL 
users of the gentrator, are fourth-generation languages. 

user interface genuators. test ca~c gentraton, parser 

generators and table-driven forms management systems. 

At least one large Ada development is making substantial 

of generated reuse in an MIS systun development, 

mugh the use of a specially-developed gcncram [AIC). 

Table 1 is a Summary of the modes of software muse 

described and the examples mentioned for each 

LaYUtd: 

hbth libdes 

Common utilities packages 

User interface or UO toolkits 

Graphics kemel systems 

Runhe kernels 

Network layered software 

Tailored: 

Database management sysrems 

Forms management systems (runtime configured) 

Expert systems 

Spreadsheets 

Sratistics packages 

Generic data structures 

Generated: 

Forms management systems (fde driven) 

User interface generators 

Testcase generators 

Eighiorder languages 

Founh-generation languages 

Parser generators 

MIS sysrems 

Table 1. Reuse Modes and Examples 

The distinctions between these categories can sometimes 

become blurred. For example, whether a reusable package 

is configured at run time by parameterization (tailored) or 

' in advance by tables such that it emils a separate program 

Cqenerated) may not be of any real consequence. In fact, 

the examples given in one category oficn have analogs 

which exist in thc other category. For example, forms 

management systems already exist in both gcncrated and 

tzilored versions. Although parser gcncraiors are typically 

generated components, since they arc smd-alone 

grammar-driven programs which cmit desircd software, 

th~y could instead be incorporatcd into the cnd-product urd 

re-emit heir parsers on he fly. Thc obvious reason not to 

do this is for efficiency of repealed usc of thc same output. 

However, an intcrprcrcr for a language can bc thought of as 

a compiler which is configurcd to perform as tailorable 

sofw= In this CBK, the run-time efficiency is Wed off 

for the flexibility of being able ta alter the 

"paramererizarion' (the bltrprctcd program) quickly and 

&y. 

A Simple Example 

As a simple example of how a low-level component can be 

viewed as a generalizable layer of function, consider the 

following error-reporting routine. 

with Text-Io; 

procsdun Gyro-Spaed_Gror is 

kgin 

TextIo.ht-Line ("Error: The gyros are not up to speed."); 

This highly specific routine represents one end of the 

generality scale. It is easy to use, requiring a simple 

parameterless call. but might not be likely to be widely 

called upon within a program. There are three observable 

details within this unit 1) the use of Text-Io.ht-Line to 

report the error message, 2) the use of the standard output 

device to display the error. and 3) the choice of the literal 

suing to be displayed. 

uae Ilteral atflng 'Error: * 

Figure 3a 

In the example from Ihe lea procedure Repon-Error was seen Io 

bc composed ol tow decisions Two are "acre4 par! d thc esetnial 

lunaionaliry ma two are comidercd to be problem.speci(ic aetails 

R*-mnplnmered procedur* Report-Error 

L 

\ Prepend en lntro 

Put-Llne 

Figure 3b 

The re-engineered version of Repoil-Error shows the IWO problemspecific Oelails removed from the component. lo be supplied by lhe 

re.usef The mlrinsic lunclional aspeas 01 !he componenl iemain 

Other interpretations of lhe re-engineering decisions 10 be appliea 

could possibly remove one of Ihese. as well 

4-9 

1ooo57mL 
A consulmion with a domain expcrt might rault in OUT 

choice LO pmctcrize thc cxact error mcssagc to be 

reponed. which might yicld thc mon: sensiblc reporting 

routine, shown blow. 

with Text-Io; 

yedure Report-Enor (Message : Suing) is 

hw 

Tcxt-Io.ht-Line ("Error: k Mcssrgc); 

end Report-Error; 

This version of thc unit is dcpicted in Figure 3a Had we 

performed the transformation wilhout expcrt consultation 

we might have simply parameterized the entire message. 

However, in ob hypothetical problem domain we will 

assume that the expert recommended retaining a 

hard-coded standard prefix in order to facilitate the 

post-processing of the log file. Also, this generalization 

has cost US the part of the original functionality which 

spelled out the exact error message. Since the client must 

now supply this smng. we have increased the effort to use 

the unit by making it more general. 

The generalization of a value (a string value in this case) is 

the easiest kind of aansfmation since it can bc performed 

with a simple value parameter. Since the parameter type is 

lwguage-defined (type String) there is no funher 

complexity to exposing this parameter in the procedure 

interface. Also, the effort to configure the component 

amounts to simply defining the emr message string as a 

parameter. Again, this kind of reuse is the easiest. 

The procedure above still assumes that the user intends the 

message to be written to the current output device using 

Put-Line. That constitutes part of the retained functionality 

of this component. In the process, we have also added the 

derail that the standard prefrx "Error: " will always appear. 

Additional consultation wirh a domain expert might reveal 

that the assumed use of the standard output device is 

another problem-specific detail. A later =user of this 

component who was working on a different problem in the 

same domain might not want to be bound by that 

essumption. Again, Ada provides a simple way to 

parameterize the component so that users can specify the 

output devict. Again, however, this generalization comes 

er the cost of functionality. In this we, the functionality 

which is lost is the assumption is that the current output 

device is to be used Default parameters can sometimes 

provide an opportunity to restore such assumpaons while 

retaining the generality, as will be shown later. The 

parameterized version of the unit which follows removes 

the assumption of using the current output device but 

retains the function of writing the literal smng "Error. " 

followed by the caller's message. 

with Tcxt-Io; 

procedure Report-Error 

kgin 

cnd Rcport-Emr; 

(Marage : Suing: 

h-Dcvice : Tcxt-Io.File-Typc) is 

Text-Io.ht-Line (On-Device, "Error: " k Message); 

Notice that thc user is now requircd to do additional work. 

Instead of simply providing the error message. the dcsired 

output device or file must be providcd. That decision hu 

shiftcd from the component to the (re>uscr. Again, this is a 

form of value parameterization, he easiest form of both 

generalization and reuse configuration. 

An additional part of the functionality of the component is 

the literal suing prepended to the callcr's message. AS 

shown below, this could also be parameterized, again 

moving that specific functionality but generalizing the 

component on that behavior. This requires yet one molt 

piece of information from the user as part of the 

information needed for this component to perform its work, 

however once again it is a low-cost value parameterization. 

with Text-Io; 

procedure Repn-Error 

(Message : Suing; 

OnJevice : Text-Io.File-Type) is 

TcxtJo.Put-Line (On-Device, Inuo & Message); 

Intro : saing; 

begin 

cnd Rcpon-Error; 

This generalized component is depicted in Figure 3b. This 

might constitule a domain-independent version of the 

rcpomng routine, according to our domain expens, 

although the only way to be cerrain that a component is 

compatible with all domains is to ensure that it does not 

depend on any other components. In Ada any such 

dependencies an ~vealed by the context claw. A Larp 

transformation will eliminate the dependence on Text-Io. 

As noted, Ada affords us an oppomnity LO rtsmrc the 

assumpaon of using the specific smng "Error. " and the 

standard output device through the use of default 

paxamelers without reducing the generality. This is shown 

below. 

with Text-Io; 

procedure Report-hr 

(Message : Suing; 

HEO : Saing := "Error: "; 

On-Device : TextJo.File-Type := 

Text-1o.S randard-Output) is 

Text-Io.Put-Line (On-Device. Inuo & Message); 

begin 

end Report-Error; 

At this pinL two details rtmain (the use of 

4-10 
Text-IoPut-Line and the pepnding of a user suing). Thc 

UD of PutLine could be removed through tailoring 

@elow) but the removal of the choice IO concatenate M 

introductory st& could not be done within the language. 

For that degree of flcxibility, generated use would be 

required. Once a generalization is needed which is not 

language-suppofud, the costs arc considerably higher. One 

way to reduce thosc costs is to provide loo1 suppon for the 

generalization, a process which amounts to establishing a 

new language to accomplish the generlization. ?he MIS 

system described in [AICI has redud their software 

generation costs in this fashion. 

This points out the obvious conclusion that the cost of a 

generalization depends on the level of language or tool 

support for it One way to cstimate cost is to begin with an 

ordinal scale of difficulty and then to move to a more 

detailed scale after more analysis has bewr done. For 

example, it was noted that value parameterimtion is 

relatively straightfonvard. This would be at the lowest end 

of an ordinal effort scale. Above that would be tailoring 

paramettriration such as Ada's generic formal type and 

subprogram parameters. At the hardest end of the scale 

would be software generation, with tooI-supponed 

generation being easier than custom-built generation. A 

more detailed approach to effort would be to relate the cost 

m rhe number of lines of code that must be written. 

changed, or added. 

It can require a judgment call to choose what details to 

remove and what function to leave in the component For 

example, in the above example. tht fact that the original 

literal smng was broken up into a standard prefix and a 

user-supplied message was only one possibility for 

generalization. One guideline is to leave operational pans 

of a component intact and to allow the operands to be 

supplied by the reuser. A discussion of the separation of 

operations fmm operands can be found in [Bailey and 

B2siIiI. 

?he simple error-reponing example from before can also 

be re-engineered into a tailored component using the Ada 

lmguage. The difference between this result and the 

layered result is that the reusers will have to perform 

slightly more work in order to instantiale the component, 

but then subsequent calls can be simpler. As suggested, 

tailoring in Ada through the use of generics is Seen as a 

harder process than value parameterization but easier than 

software generation. A Word examplc of the component 

follows. 

with TextJo; 

generic , 

ho : String := "Enor: "; 

On-Dev ice : Text-Io. File-Type : = Text -1o.Cuncnt-Outpx; 

procedure Repm-Enor (Message : S-); 

Unfortunately, this is illegal in Ada since a Iimifcd tw 

~ext-Io.File-Typc) is not permitted as a generic value 

parameter. This is an example of where smng static 

checking can be at cross purposes wirh generalization and 

wsc. If it werc legal, nevenheless. the user would have 

the responsibility for providing the introductory string and 

the oup~t device one Lime (at the time of the generic 

instantiation) thus tailorin8 the component for further 

reue. From then on, the component would be no more 

difficult to use (from the standpoint of parameterization) 

than the original non-general version. 

To avoid this limitation of generic parameters, a solution 

could be obtained by generating the specific component 

desired, using tools outside of the Ada language. The 

generated component could look exactly like the original 

component but the reusable software would no longer be 

considered the component itself, but rather the generator 

which creates it. In this case, the generator would emit a 

Report-Error procedure which was hard-coded to write the 

error message on a given device. ?he value of that device 

would be given as a parameter to the generator. More 

examples of generation are shown later. 

A different tailoring would also be possible. As mentioned 

earlier, the dependence on Text-Io can bt eliminated by 

requiring that the client tailor the component to use a 

particular siring-processing routine. This makes the 

component completely indepenienf with the persistence of 

the use of a standard prefix as the only detail which is 

retained ffom the original version. 

gmeric 

Inn0 : soing := "Enor: "; 

with procedure Put (S : Suing); 

procedure Repon-Ekor (Message : Suing); 

procedure Repon-Error (hkssagc : Suing) is 

begin 

PUI fiao 6 Mcrsage); 

end Repon-Error, 

Note that this most gencral version is also the least 

functional. Nevenhcless, the ability to tairor the 

componcn1 once within a program and to then use it with 

the same level of effort as the first layered transformation 

makes it of some value. The reuscr has additional work to 

do with this solution. as well. For example, unless the error 

messages are to bc wriiten to standard outpu~ the 

subprogram LO be passed to the generic formal Put 
procedure has u, be written. This mCjm that thc cffon to 

reuse a lailorcd component could bc gram than the clfon 

u) reuse a companent gcncrator. So, thc cffon LO gcncralize 

is not always proponional ID the corresponding cffort to 

reuse. 

By examining existing systcms and by observing thc 

opportunities to gcncralizc thcir parts according to thcsc 

different melhods of rcusc, choices bccomc availablc in the 

ways in which the software can be re-cnginccrcd for futurc 

reuse. The next secuon dwcribes a simple mail systcm in 

ms of its conventional configuration as a custom-built 

application and thcn in terms of the various ways the parts 

of it can be generalized using he above methods. 

Re-Engineering a Simple Electronic Mail System 

This section fakes a simple electronic mail system through 

mnsformations to yield componenrs which can be 

combined using the three methods described above. In the 

interests of space, parts of the examples and some identifier 

names have been abbreviated, and no bodies are shown. 

Complete listings of the examples are available fiom the 

euthors. 

In a conventional design, one component, or package, of a 

mail system could be used to manage the mailboxes of the 

users and a second could manage the messages, or the 

constituents of a mailbox. This would reprcsent a 

conventional encapsulated or "object-based" design of the 

system whert the mailbox package would allow operations 

such as create, add a message, deletr a message. return a 

message, and perhaps displaying a directory of messages, 

maintaining the status of each message, and so on. The 

message package would allow message crcation and 

display, and possibly reply consauction, forwarding, etc. 

In a typical arrangement, using either Ada or an 

object-oriented language such as Smalltallc, the mailbox 

package (or object) would depend upon the message 

package to obtain the use of the declaration of message 

objects, in order to arrange those objects into mailboxes. In 

Ada the specifications for each of these two packages 

might reasonably be: 

peckage Mestagcs is 

cypc Uxmune is ... 

rypc Text is ... typc Message is private; 

procedure Set-Sender (M : in out Mcssrge; To : Uwmme); 

procedure Set-Receiver (M : in out Message; To : Useme); 

procedure Set-Subject (M : L7 out Message; To : Line); 

ploudure SetBody (M : in out Message; To : Text); 

function Sender-Of (Msg : Message) rem Usanme; 

function Receiver-Of (Msg : Message) return Usanme; 

rypc Line is ... 

hrnction Subjcct-Of (Msg : Messrge) retum Line; 

function Body-Of (Msg : Message) rcturn Text; 

type Mersage is 

pivale 

record 

Sender : Uscme; 

Rcceivcr : Utcmmc; 

Subject : Line; 

MsLBody : Tcxr 

end record; 

end Mcssngcs; 

with Mcssager; 

packrge Mailboxes is 

typc Mcrsagc is ncw MctsagesMcssage; 

- dcrive an equivalent type Message 

Mu-Mailbox-Size : Natural := 1OOO; 

rubtypc Box-Sue is Natural rmge 0 .. Mlx-Mailbox-Size; 

type Mailbox (Size : Box-Size := 0) is privare; 

procedure Store (Box : Mailbox; Owner : String); 

procedure Retrieve (Box : in out Mailbox: Ormer : String); 

function Size (Of-Box : Mailbox) return Boxsize; 

function MsgAt (Position : Natural; In-Box : Mailbox) 

procedure Remove (Num : Positive; InBox : in out Mailbox); 

procedure Append (Msg : Message; To-Box : in out Mailbox); 

procedure Mark-Read (N : Natural; In-Box : in out Mailbox); 

procedure Mark-Unread ... procedure Muk-Answered ... procedure Mark-Deleted ... procedure Mark-Undeleted ... function Is-Read 

function Is-Answered ... function Is-Deleted ... No-MsgAt-Position : exception; 

type Attributes k (Deleted Red, Ansuwed); 

type A~r-Seu is array (Amibuus) of Boolen; 

type Mail-Item is 

rerum Message; 

(MsgNumber : Natural; In-Box : Maiibox) rem Boolean; 

private 

record 

Item : Mcsrage; 

Sunu : Atu-Sets; 

end record; 

ryp Ircm-&ray is array (Positive rmgc 0) of Mail-Item; 

type Mailbox (Six : Box-Size := 0) is 

Items : Item-Array (1 .. Size); 

ncord 

end record; 

end Mailboxes; 

These packages are depicted in Figure 4a As shown, the 

Messages pachge is an example of an independently 

reusable layer, and the Mailboxes package consticum a 

layer on top of the Messages package. (Since the 

constiuent types of Username, Line, and Text arc not 

shown, it might be the case that they would be comprised 

of user-defmed types. making the Messages package 

dependent on other client software.) Realizing that the 

decision of how to implement the constituents of a message 

represents one of the opportunities for generalization of this 

package, the components of a message could be supplied as 

pmeters to a generic version of this package. Thrs 

would constitute a tailored version of the package: 

4-12 

1oO06788L 
This generalization is shown in Lhe top part of Fig- 4b. 

ne effort to perform this tailored generalization is in line 

wilh other tailoring efforts discussed in the previous 

stion. The declaration of three generic formal parameters 

is one measure of the WO& performed. Also, the rtusc 

effort implies the declaration of actual type parameters to 

be associated with these generic formal types. One way u, 

quantify the effon u, generalize, then, is to claim that three 

declarations are required. Three declarations arc also 

nquired of the client reuser. 

peckage Memeepee 

;'=\.\, peckape Mallboxwe .._.. .. . 

4 - c 0 m PO n e nl r e c 0 r d-I y pe e b e I r e c 11 on 

Export. type Millbox. en 110 array 

Figure 4. 

3-3 IM conventions snown ptewourly. lhls deplns me pocers d blormp tnc 

vyn)a and Mailboxes packages from tne lex( 

Going beyond this somewhat tailored version. nolie rhat 

even the suucture of a message could be a candidate 

genedizauon. In this case, tailoring would be difficult or 

impossible within the confines of the Ada language so 

generation is required. Generation is feasible since the 

contents of the Messages package could be 

deministically described if one were to specify the 

constituent components of a message. For example, if no 

subject line were wanted. the original package could 

insLead have been writen: 

package Messages u 

typc Umune is .I 

typt Text is ... type Message is privue; 

-- procedures Set-Sender. Set-Receiver, Set-Body 

-- functions Sender-Of, Receiver-Of. Body-Of 

rype Message is 

privarc 

-- no Subject component 

record 

scndn : UsemMlc; 

Receiver : Urcmame; 

MsgBody : Texr 

mdrecord; 

end Messagu; 

Or. if a message with a date and he stamp were desired, 

the abstraction could be augmented with an additional 

component, such as with the standard type Calendar.Time: 

with Cdendar; 

package Messages is 

rype Usemame is ... 

typc Text is ... type Message is privae; 

-- procedures Set-Sendcr, Set-Receiver. Set-Body. 

-- Set-Subject, md Set-Time 

-- functions Sender-Of. Receiver-Of. Body-Of. 

-- Subject-Of. Time-Of 

type Message is 

type Line k ... 

private 

record 

Sender : Usemame; 

Receiver : Usme; 

TieStamp : Calendar.Time; - new 

Subject : Line: 

MsgBody : Tex~ 

end record; 

end Messages; 

Allhough thc gcneric feature in Ada is not powerful enough 

u, allow thcx: variations as tailoring of a single common 

package, all of the Message package examples (as well as 

their corresponding bodies) could have been generated 

automatically, given he desired set of components for 

objects of type Message. This. therefore, becomes an 

example of gcnerated reuse, where he generator is the 

reusable software and not the actual message package 

software. For example, a simple editor-substitution 

generator has ken conslructcd which zccepts input such as 

4-13 

1oO06188L 
the following and emils Ada quivalcnt to thc example 

shown above. 

Gencr ate-Package 

(Context => "". 

Locd~Dccls => 

"subtype usemme is soing(l..lO);" k 

"subtype linc is suing( 1..60);" k 

"rubrypc tcxt is suing( L.80);". 

Package-Nunc => "mcssagcs". 

Private-Typc => "message". 

Set-1 => "set-sender", 

Set-2 => "set-receiver". 

Set3 => "set-subject". 

Set-4 => "set-body". 

Get-1 => "sender-of'. 

Get-2 => "rcccivcr-of", 

Get3 => "subject-of'. 

Get-4 => %ody-oT', 

Local-Type-1 => "usernune", 

Local-Type-2 => "username". 

Local-Tp-3 => "line", 

Local-Type-4 => "text"); 

The effon to construct this generalization amounted to the 

writing of about 20 lines of sofrware and the building of 

templates from the original unit. The effort to reuse the 

component is the construction of the above call. This could 

be seen as effort equivalent to declaring 17 smng constants. 

Note that, at this level of generalicy, which came at 

considerably higher cost than the previous tailoring, more 

than just a message package for a mail system could k 

generated. Any private type implemented as a record of 

components with set procedures and access functions could 

be generated with such a program. Therefore, this 

represents a domain-independent form of the component, 

wnere any mail system details are supplied by the reuser. 

So, the benefit of applying this substantial generalization 

effort is that the component can now be used by many 

domains. In fact, we will see that this same generator can 

be used to reptact pan of the Mailbox package, as well. 

Alrhough the style of rhe Mailbox package is not as general 

as the Messages package, there are several opportunities to 

. make it more general and therefore more reusable in other 

conttxts. For example, it could be tailored by maldng the 

constituent type Message and the maximum mailbox size 

generic formal parameters: 

gncric 

typ Message is private; 

Mex-Mailbox-Size : Nand := lo00. 

pxkrge General-Mailbox= is 

... -- same tu package Mdboxcs. above 

d Gcncral-Mailboxes; 

This arrangement of the blboxes pachge is shown in the 

bottom pan of Figure 3b. Fonu~tely, no operations on he 

:)'pe Message were needed by the package Mulboxes. 

othcnvk those operations would have had u) have bccn 

passed as gencric paramctcrs.* Thcrcfore, following the 

convention suggcstcd abovc, the generalization effort hcrr: 

is the effon to writc two gcncric formal parameter 

dcclantions. Rcuscr effort is the choice of a typc and a 

value to perform the instantiation. 

Beyond the rclativcly. simplc gcncralization shown above, 

it can bc observcd that the Mailbox abstraction is actually 

composcd of a four-componcnt record-type abstraction and 

an array. Reusing the previously described example of 

privalc record type abswctions, the package Mailboxes 

could be divided into two separalc abstractions as follows: 

generic 

type Message is private; 

package General-Mail-Items is 

type Mail-Item is private; 

procedure Set-Message 

procedure Set-Read 

procedure Set-Answered ... prcccdure Set-Deleted ... function Get-Mesragc (An-Item : MsiI-Item) re" Message; 

function Is-Read (h-Item : Mail-Item) rerum Boolem; 

function Is-Answered (An-Item : Mail-Item) rem Boolcm; 

function k-Deleted (An-Item : Mail-Item) rem Boolean; 

type MailJtem is 

(AnJtem : in out Mail-Item; To : Message); 

(An,Imn : in out MailJtem; To : Boolean); 

priVak 

-- a modified implementation 

record 

Item : Message; 

Rad : Boolean; 

Answered : Boolean; 

Dele:d : Boole~; 

end General-Mail-Iums; 

generic 

end record; 

type Mail-Item is private; 

Max-Mailbox-Size : Na~~ral:= 1ooO; 

package Genaal-Mailboxes is 

rubcype Box-Size is Natural range 0 :. Max-Mailbox-Size; 

type Item-hay is may (Positive rangc 0) of Mail-Iten; 

type Mailbox (Size : BoxSize := 0) is 

Item : Item-Array (1 .. Size); 

rccord 

end record; 

*If A& supponed full inheritance, it would be possible u) 

write the Mailbox abstraction so thar it relies on certain 

operations to be defined for the generic formal type 

Message. The user would then guarantee that any expected 

functions would be available for any actual type paramete: 

associated with the formal type Message, eliminating the 

syntactic complexiry of pasing them vi.? zdditional generic 

formal subprograms. This illustrates one of the advantages 

of late binding, something thar Ada disallows in order u) 

ensure thar required operadons are available prior to rhe 

compilation of any insatiations of the generic. 

4-14 

1ooo5T8BL 
+ure Stme (BOX : Mailbox: Owner : Shg); 

procedure Remcvc (Box : in out Mailbox: Owna : String); 

function Size (Of-Box : Mailbox) return Box-Sk; 

procedure Rcmove 

- procedure Append 

(Md-Ir~n-At : Positive; In-Box : in out Mailbox); 

(A-Mail-Msg : Mail-Xmn; To,Box : m out Mailbox); 

No-MsgAt-Position : exception; 

& General-Mailboxes; 

nese packages arc depicted by Figures 5b and 5c. In the 

above case, the client could obtain the functional quivalent 

tht original mailbox package via the following 

*tandati0IK: 

psckqe Mail-ItmS k 

pckllge mailbox^^ is 

new General-Mail-Items (MuragesMtrsage); 

new GenclalJvlailboxes (M ail-I~cms.MailJtcm); 

1 I tailoicd pbckagc Maaaagea 

@ 

-types lor urarname. ate. 

-(supplied by reuser) & 4 -: om p o n a nt I s cor d .I y pe b s I r e e I I on :<. 

exports typa U*..rg. 

Figure Sa. 

~0 add- changes are made during the second pass at ~ailoring the two 

ms. Only by generating the Uersages package can the docisions am 

3'~ -sure d the abrtract data type be ganeratized. since rwh a run-time 

6 no( psible within the Ada tangruge. 

lallored. factorid paelrage Mall-llama 

$om* mabaage type (i .e.. 

Meraagia.Maawa9e. above) 

d-com po na nt ree 0 I d-ly pe a bat I a ct 1 on 

Figure 5b. 

7he Mailboxes package is broken inlo two componenls. one which implements 

W-hrms as a record-type Cala abstraction. above. 

Figure Sc 

Cwf package lanored lrom the original Mailboxes package iinpl~"xS 

IL: L:! Isl of mail items 

t2mUn Olher lhan lmplemenl hsis so 11 can be replaced wilh a gCneral. 

r-':Jm lisl abslracllon. as shown in Ine texl 

This no longer conlains any problem. SPeCrliC 

Two tmdeoffs in this example are observed. First, the 

specific way in which package Mail-Ircm was structured 

originally was modified into the more gend 

multi-component mrd shown here. This uadeoff was 

accepted in order to allow this impiementaaon of 

Mail-Items to be similar to the implementation of 

Messages, which was previously shown to be highly 

generalizable. This is an example of how standardization 

limits the choices available to the implementer while 

increasing the generality of the resulting programs. For 

example, by adopling this approach, the generator program 

mentioncd before could be used to gcneratt an equivalent 

package to Mail-Items through the following input. thereby 

allowing the generation of both the Messages package and 

the Mail-Items package from the same reusable 

component: 

Gmeratc-Padcage 

(Conrext => "with messages;". 

tocal-Dsk =* 

"type message is new messages.mcssage;". 

Package-Name => "mail-items". 

Private-Type => "mail-item", 

Set-1 => "set-messqc", 

Set-2 =e "set-red". 

Set-3 => "set-answered", 

Set-4 => "set-dtleted". 

Get-1 => "get-message", 

Get-2 => "is-read". 

Get-3 => "is-answered". 

Get-4 => "is-deleted", 

heal-Type-1 => "message", 

Local,Type_2 => "bo~lcsm". 

LocSl-Typc-3 => "bo~le~", 

bd-Type-4 => "boolew"); 

* 

The second tradeoff was to make the type Mailbox visible. 

This was necessary since the client software will have to 

gain direct access to a h4ailJtem within a mailbox array in 

order m perfonn the operations from package Mail-Itrms 

on ir Simply returning a value of Mail-Item via a function 

call would not allow the user to set the components of a 

Mail-Item in a mailbox. An alternative solution would 

have been to implement the items in a mailbox as ~ccess 

values, each designating a Mail-Item. In this way, a 

function. returning an access value would provide the 

capability for the client to modify the designated objecr, a 

Mail-Item. This situations occurs frequently when 

factoring composite abstractions into their constituent 

abstractions, and suggests hat by presenting objects 

directly on the interfxe to an absmction, rather than just 

their values, an abstraction can be made more general and 

reusable. 

Further generalizations are not shown in detail in the 

interests of space. However, note that the above 

General-Mailboxes abstraction is the only remaining 

custom-made application code in the example. It amounts 
to an ordcrcd list of itcms of disccmiblc size. to which 

items can bc appcndcd and from which itcms can bc 

deleted. and which can bc storcd to and rcvieved from 

fdes. Exccpt for UIC ability to storc and rctricvc thc lists, 

such an abstraction would probably bc availjblc in a library 

of generic data smcturcs. Assuming the constitucnt 

objecls are privau: and not limitcd priva~e, it would bc 

possible to pcrform binary inpuvoutput on thcm. So, it is 

not unnasonablc to augmcnt an existing gcncric abstraction 

to include storagc and remicval. Such an augmcnution of a 

list rcsourcc could be accomplished by laycring somclhing 

like the following onto iL 

- Layering on a list abstraction: 

with Simple-LEU; 

generic 

type Item is privatc; 

type Item-Access is access Item; 

package Item-Lists is new Simple-Lists (Ikm, Ikm-Access); 

rype Mailbox is new 1tem-Lists.List; 

procedure Store (A-Box : Mailbox; To-Ele : String); 

procedure Retrieve 

end General-Mailboxes; 

package General-Mailboxes is 

(A-Box : in out Mailbox; From-File : String); 

To obtain the equivalent functionality as was provided by 

instances of the earlier package General-Mailboxes, the 

following declarations would now be required: 

package Mdl-Itcms is 

new General-Mail-Item (Messages.Message); -- same 

type Mail-I~~"Access is access Md-Ittm3vlailJrem; 

package Mailboxes is new General-Mailboxes 

(Item => Mail-Items.Mail-Item. 

Irtm-Access => Mail-Itan-Access); 

?he client can meat the above package Mailboxes similarly 

to the earlier version; it will have all the same operations 

due to the derivability of those already implemented by 

Simple-Lists. Also. note thal the mailbox implementation 

has been made private again by using designated objects to 

hold mail items. This would allow an hem-At function to 

return an access value to the actual mail-item and not just 

the value of that mail-item. This allows updates of the 

item via the operations that were defined in the Mail-Item 

package (Set-Message, Set-Deleted. ctc.). 

hieasurement Summary 

Measurement is required at two points of the software 

cycle. When candidate units are king identified and 

domain-specfic details are being distinguished' from 

problem-specific der& esdmztes of the generalization 

effon necessary to remove any give detd are required. At 

the time of reuse. estimates oi the configuiition effon 

necessary to adapt a component for reusc are rquired. 

Observations from conduc tins scveral generalualions have 

shown hat an initial cstimatc based on an ordinal scalc is 

possiblc. This scalc has valuc parametcrization as he 

easiest to puform for both generalization and reuse. 

Hardcr than this is typc or operation parameterization, 

which requires tailored generalization in the case of Ada. 

The hardest form of generalization is building a 

spccial-purpose componcnt gcncrator. This can bc mad: 

easier through the use of codc-gcneration support tools. 

After an initial evaluation of the generalization effon has 

becn made and an approach to generalization has been 

detcnnined, a more accurate assessment of the effort may 

be possible. The most direct indicator of the effort required 

is the number of lines of code that have IO be written, 

changed or added. In many cases, a generalization can be 

accomplished with just a few lines of new or changed code. 

However, in the case of unsupported component 

generation, the entire generator may have to be written. 

Reuse effon is easier to quantify since the component in 

question is already known. The effon to configure a 

generator or to instandate a generic can be estimated based 

on the number of inputs or parameters required. In most 

cases, the usage of a tailored or generated component is 

similar regardless of whether the component was developed 

from scratch or obtained hom a repository. However, even 

this step can be complicared by the fact that a development 

might choose to be constrained in some way in order to 

take advantage of an available component. The COSLS of 

such a decision can be especially difficult to esdmac. In 

the long run, however, it is expeccd that the adoption of a 

component, similar to the adoption of a standud, is a 

cost-effective c ho icz . 

Another meam Lhar is needed is an estimate of the future 

value of a unit in a repository. It may not be the besl 

approach to populatt a repository with many units which 

were inexpensive to generalize if they will rarely k 

needed. It would be better to spend the time performing a 

diffcult generaliration if the resulting unit will more than 

return that invesuncnt Here again, domain expem will 

have to assist in making this determination. 

Future Work 

Progress is needed on memcs to qumufy generalizanon 

and reuse effon. Effective memcs will open the way to 

establishing an economic model of reuse that could enable 

an organization to choose its optimzl approach to reuse 

engineering. Note that the Same approach or even the same 

specific model would not necessarily be best for two 

different orgmizations. One obvious reason for this is diZ 

one organization mzy concennate in a single application 

domain while motfier organizadon mag do work in mmy 

4-16 

1oO0578aL 
domains WiLh very litlle repetition. The fmt organization 

may find its optimal approach LO rtust is to develop a 

mature repository of domain-specific components while he 

second organization may find ha! only 

domain-independent components are likely to bc cost 

cfrecuve. 

In addition to the costs of generalization and Axe. an 

Eonomic view of the software cyclc suggested in this 

paper would have to deal with rtpository maintenance. 

component relrieval, component probabilities of reuse and 

cost savings, and the effon required of domain expens and 

repository expens. Current progress is being made in some 

of these arcas by interviewing expens at one branch of the 

NASA Goddard Space Flight Center where reuse has been 

practiced for many years, originally with Fortran and more 

rsently with Ada. The results of these interviews will 

assist us in formulating a more quantifiable model of the 

costs and benefits of reuse at that organization. It is hoped 

hat this experience can then be exnapolaltd into a broader 

model of reuse engineering that can be adapted for use at 

other organizations. 

References 

pailey and Basili] J. Bailey and V. Basili, "Software 

reclamation: Improving Post-Development Reusability," in 

Proceedings Eighth Annual Conference on Ada 

ieehnology, Atlanta, Ga., 1990. 

9asili and Caldera] V.R. Basili and G. Caldera, "A 

ikference hhitecture for the Component Factory," 

Computer Science Technical Reporr Series, University of 

biarylmd, College Park, MD. March 1991, 

UMIACS-Tp.-91-24 or CS-TR-2607. 

puili and Rombach] V.R. Basili and H.D. Rombach, 

'Suppon for Comprehensive Reuse," Software Engineering 

Joumal, July 1991, (also, Computer Science Technical 

Reporr Series, University of Maryland, College Park, MD. 

February 1991. CS-TR-2606 or vMTACS-TR-91-23). 

[Caldiera and Basili] G. Caldiera and VR. Basili. 

"!dcntifying and Qualifying Rcusable Software 

Components," IEEE Compurer. Vo1.24. N0.2, Fcb.1991. 

~9.61-70. 

!E!lis and Stroustrup] M. Ellis and B. Strousuup. "The 

.%!!notated C++ Rcfcrcnce Manual," Addison Wesley, 

1999, p. 341. 

:AIC] Ada Information Clearinghouse. "STA"S-R - 

COBOL and C Programmers Moving Successfully to Ada." 

Ada Informrion Clearingnouse h'ewsleiicr 8.2. June 1990. 

John W. Bailey is a Ph.D. candidate at the Universiry of 

Maryland Computer Science Department He has been 

consulting and reaching in the areas of Ada and softwart 

measurement for nine years. and is cumntly consulting to 

Rarional. He has an M.S. in computer science from the 

University of Maryland, where hc also earned a bachelor's 

and a master's degree in cello performance. He is a 

member of the ACM. 

Victor R. Basil; is a professor at the University of 

Maryland's Institute for Advanced Computer Studies and 

Computer Science Deparanent. FA research interests 

include measuring and evaluating software development 

and is a founder and principal of the Software Engineering 

Laboratory. a joint venture of NASA, the University of 

Maryland, and Computer Sciences Corporation. He 

meived a B.S. in mathematics from Fordham College, an 

M.S. in mathematics from Syracuse University and a Ph.D. 

in computer science from the University of Texas. He is a 

fellow of the IEEE Computer Society. 

4-17 

loood786L 

On the Nature of Bias and Defects in 

the Software Specification Process 

PABLO A.STRAUB 

COMPUTER SCIENCE DEPARTMENT 

CATHOLIC UNIVERSITY OF CHILE 

Abstract 

Implementation bias in a specification is an arbitmry 

constraint in the solution space. This paper describes the 

problem of bias and then presents a model of the specification and design processes describing individual subprocesses in terms of precision/detail diagmms, and a model 

of bias in multi-attribute software specifications. While 

studying how bias is introduced into a specification we realized that software dejects and bias ore dual problems of a 

single phenomenon. This has been used to ezplain the large 

proportion of faults found during the coding phase at the 

Software Engineering Labomtory at NASA Goddard Space 

Flight Center. 

1 Introduction 

Most informal software specifications are ambiguous, 

imprecise, and incomplete. Moreover, this is usually not 

evident by looking at a particular specification. This has 

prompted research on desirable and undesirable characteristics of specifications and specification languages. To 

make specifications precise, formal languages are used. 

Some of these languages are defined so that automatic 

compilation or execution is possible. However, much detail 

has to be included in executable speafications [SI. Th‘ is extra detail not only makes the specification harder to read 

[6], but also leads to ‘implementation bias’. 

Alas, implementation bias-an arbitrary constraint in 

the solution space-is a term often used but not well defined. This has resulted in two effects: Either (1) specifications are biased, or (2) they are incomplete, for fear 

of bias. In fact, what has been called ‘bias’ in the literature is sometimes the desirable record of design constraints 

and design decisions. The problem of bias is related to the 

more important problem of software defects, because both 

are manifestations of either misconceptions with respect to 

the problem or preconceptions with respect to the solution; 

hence, we study these two problems together. 

OVERVIEW OF THE PAPER. This paper presents a 

model to help understand bias in software specifications. 

MARVIN v. ZELKOWITZ 

DEPARTMENT OF COMPUTER SCIENCE AND 

INSTITUTE FOR ADVANCED COMPUTER STUDIES 

UNIVERSITY OF MARYLAND AT COLLEGE PARI< 

The remaining of this introduction presents our framework, the problem of bias and the concept of specification correctness. The next section presents our view of the 

process of specification and design. Section 3 presents our 

model of bias which is based both on the specification prccess and on a classification of requirements. Within this 

model, bias is not an absolute property of a specification, 

but depends on the process of creation of the specified 

requirements, that is bias depends on the process of specification and design. Section 4 presents the relationships 

that exist between bias and defects in a specification, and 

a study made at the Software Engineering Laboratory that 

explains the high relative incidence of coding faults in that 

environment. 

1.1 Specification F’ramework 

In this work we are considering multi-attribute specifications developed by starting from a description of requirements, and then refining it in several stages [3, Chapter 11. 

Each stage takes a specification and produces a product, 

which is a more refined specification, until a program (i.e., 

a specification for a computation) is obtained. This view is 

not an endorsement of any particular development method: 

it models top down development, the waterfall life-cycle 

model, Boehm’s spiral model, transformational programming, and other development methods. 

We first define some related concepts. 

Attribute: feature or dimension that characterizes software 

systems (e.g., average response time). 

Requirement: constraint in the values of attributes (e.g., 

average response time shall be 0.5 seconds). 

Preference measure: a measure of the goodness of the different values for a given attribute (e.g., smaller response time values are better). 

Specification: statement of attributes, requirements, and 

preference meaSures for a software system. 

tion. 

Specificand set: set of all systems that satisfy a specificaSolution set: set of all systems that solve a problem. 

lOoo5188L 

4-19 

PRECEDING PACE BLANK PjO‘T FELMED 
constraints in the solution set. 

1.3 Avoiding bias 

(4 (e) (f) 

Figure 1: Specificand S, solution P, and particular 

solutions 2 and d‘: (a) ideal, (b) acceptable initial 

specification, (c) successive specification stages, (d) incomplete specification, (e) bias, (f) usual case. 

Whereas the specificand set is defined in terms of a particular precise specification of a problem, the solution set 

is defined in terms of the problem itself without reference 

to any written Specification. That is, the specificand set 

comprises all systems that are correct with respect to the 

written specification, and the solution set comprises all 

systems that satisfy the user or customer. The differences 

between these sets are at the heart of our model; they are 

also the cause of defects in specifications. 

1.2 The problem of bias 

An ideal initial specification is general and precise 

enough so that a software system satisfies the specification 

if and only if it solves the problem at hand, that is, the 

specificand set equals the solution set (Figure la). This 

view is too optimistic, because there can be many solutions that do not even involve software. In practice, we 

only require software systems satisfying the specification 

to be solutions, and that no substantial class of solutions 

does not satisfy the specification, so that we can arrive at 

an optimal or nearly optimal solution (Figure Ib). An idealized development by staged specifications constrain the 

specificand set (Figure IC) by adding design decisionsand nothing else. Incomplete specifications (Figure Id) 

may lead to defects; for instance, z’ satisfies the specification but it does not solve the problem. .On the other hand, 

bias (Figure le) may lead to inefficiencies (e.g., optimal 

solution is really 2”) and other development problems because the developers are overconstrained. Unfortunately, 

most specifications suffer both problems (Figure If). 

A specification is biased if some of its requirements are 

arbitrary. Biased specifications overly constrain the specificand set, precluding some valid implementations as solutions to the problem at hand. Hence, the amount of bias is 

a common yardsti& to judge software specification methods: those that are considered biased are usually rejected. 

Unfortunately, bias is sometimes confused with intended 

A generally accepted rule to avoid bias is “A specification should describe only what is required of the system 

and not how it is achieved.”’ However, this rule does not 

solve the problem: it only shifts it, because whether some 

requirement is a what or a how depends on one’s point of 

view. For instance, the same requirement can be seen as 

a how by the designer and as a what by the implementor. During the process of refining the specification, some 

how’s become what’s: a design decision (Le., how to do 

something) made by a designer is a requirement (i.e., whot 

to do) for the implementor. A how becomes a what when 

a decision is made: a new requirement is incorporated into 

the current specification stage. 

Consider a specification for a subprogram. The external interface of the subprogram is considered a requirement 

by the programmer (it is a what), because he or she cannot change it. This same interface was previously a how 

for the designer of the whole program, because he or she 

could have chosen an alternative interface. On the other 

hand, internals of the subprogram (e.g., algorithms, data 

structures, local variable names) are mostly how’s for the 

programmer, because he or she can change them. 

There is no reason to include a how in a specification: 

specifications should describe what is desired and no more. 

However, often some attribute that is already fixed (i.e., 

it is a what) is not specified because of fear of bias. For 

instance, if within an institution there is a convention for 

local variable names for the purpose of easing maintenance, 

then the adherence to this convention is a what: It is already fixed, the programmer cannot change it, so it should 

be specified. We argue that this kind of constraint is not 

bias; in Section 3.3 we provide a definition of bias that is 

consistent with this view. 

1.4 Specification Correctness 

Specification bias and specification defects are intimately related. As can be seen from Figure 1, bias is 

related to the set difference of the solution set and the 

specificand set, P - S. That is, there is bias only if there 

are acceptable and preferred solutions outside the specificand set. Conversely, defects are related to the specificand set minus the solution set, S - P. That is, if an 

implementation i is unacceptable but is correct with respect to the specification, it is in the set difference (i.e., 

i # P /I i E S + : E S - P). In other words, bias and 

’ A common statement of this rule is “A specification should 

describe only what the system should do, not how it should 

do it.” This modified rule is only useful with functional specifications: it views a software system as a specification for a 

computation, rather than as a product. 
defects in the specification are dual problems. 

Assume that for a given specification, the specificand 

set is contained in the solution set. In this case, all correct 

implementations are acceptable. This motivates the notion of specification correctness with respect to a problem, 

which is similar to the more familiar notion of implementation correctness with respect to a specification. (The 

main difference between these two concepts is that specification correctness cannot be formally verified because it is 

defined relative to an abstract problem.) A speafication is 

correct if it is realizable (there is a correct implementation) 

and complete (all correct implementations solve the problem). That is, for a correct specification it is possible to 

derive an implementation and any implementation derived 

solves the problem. On the other hand, a specification is 

called impertinent to the problem if there is not a correct 

implementation that solves the problem. 

The above is formalized as follows: Let S be the specificand set of a specification and let P be the solution set 

of a problem. 

The specification is realizable iff S # 0. 

The specification is complete tu.r.t. the problem iff S E 

0 The specification is correct w.r.t. the problem iff it is 

The specification is pertinent to the problem iff S n 

P. 

realizable and complete. 

P # 0. 

The following relations between these concepts are immediate: correctness implies pertinence (S # 0 A S E P j 

S n P # 0); pertinence implies realizability (S n P # 0 j 

S # 0); completeness and pertinence imply correctness 

(because pertinence implies realizability); unrealizability 

implies completeness and impertinence (S = 0 

P A S n P = 0); there is no correct specification for a 

problem without a solution (P = 0 *,3S: S # 0 AS E P). 

To analyze the correctness of a specification with respect to a problem, consider the emptyness of the set S- P, 

related to the completeness of the specification, and of the 

set S n P, related to the pertinence of the specification. 

There are four cases: (a) The specification is unrealizable; 

(b) the specification is correct; (c) the specification is realizable but not pertinent; and (d) the specification is pertinent but incomplete, that is the specification can be made 

correct by adding more requirements. Figure 2 presents 

these cases, with case (d) comprising two subcases. In 

cases (a) and (c), the only choice is to backtrack, since 

at this point it is impossible to derive an acceptable solution. In cue (b) there are no problems of correctness, but 

there can be problems of specification bias, if the preferred 

solution lies outside the specificand set as in Figure le. 

In case (d), the specification is incomplete, so addition of 

S 

Figure 2: Specificand set S with respect to solution 

set P: (a) unrealizable, (b) correct, (c) realizable but 

impertinent, (d) pertinent but incomplete. 

problem-specific information is needed to achieve a correct 

specification. 

2 Specification Refinement 

The specification and design processes are complex processes in which technical knowledge, art and inspiration 

take part [lo]. Goel and Pirolli [4] describe the traditional view of design as a four-step process: '(1) an exploration and decomposition of the problem (that is, analysis); (2) an identification of the interconections; (3) the 

solution of the subproblems in isolation; and (4) the combination of the partial solutions taking into account the 

interconnections (that is, synthesis)." 

In this work we go beyond these general processes and 

describe the subprocesses that occur specifically in software design. We characterize these subprocesses by how a 

current specification is updated to produce the next specification within a series, and also by how precision and detail are added to the Specification. There is no assumption 

that all requirement analysis is done before design; on the 

contrary, requirements gathering and design are supposed 

to be intertwined [12]. 

2.1 Refinement S ubprocesses 

We assume that there is a written initial specification 

and that successive specifications will be created by a series 

of modifications to that specification. With respect to the 

subprocesses that perform these modifications-typically 

additions to the current specification-we postulate that 

there are four main kinds of activities that modify a specification: 

Ezplicotion: addition of a requirement by making explicit 

Design decision: addition of a requirement by choosing a 

a nonexplicit requirement. 

particular design. 

4-21 
Presentation chbnge: change in the notation, presenta- 

. tion, or structure of the s,pecification. 

Retruction: withdrawal of a requirement from a previous 

decision or explication. 

Even though we present these as discrete changes, actual 

changes to a specification usually involve a combination of 

them. For example, after finding an incorrect explication 

an analyst may replace the corresponding requirement by 

another one: a retraction followed by an explication. 

Explication 

Explication is one of the main activities during requirements gat hering. Explications make the specification more 

complete, that is, ensure that software systems satisfying 

the specifications are solutions. In Figure 1 the goal is 

to transform a specification like (d) into one like (a). This 

goal is achieved by making explicit either domain injormation, problem-specific information, or consequences of the 

specification, thus reducing the specificand set. 

Of course, the new requirement is not always a valid 

explication (e.g., something believed to be a consequence of 

the requirements might not be). This is intimately related 

to the concepts of specification correctness (Section 1.4) 

and bias (Section 3.3). 

Design Decisions 

As the name suggests, design decision is the most important process during design activities. Design decisions 

guide the implementation process towards a preferred set 

of solutions reducing the specificand set (as in Figure IC). 

The information needed to make design decisions comes 

mainly from the previous specification and the solution domain. For example, semantic-preserving transformations 

in transformational programming are design decisions, because they preserve the functionality while improving other 

attributes of the algorithm. 

We have identified several kinds of design decisions: decomposition, refinement, composition, abstraction, instantiation, reuse, creation of alternatives, and choice. Some of 

these are intimately related so we discuss them together. 

Decomposition and refinement. Decomposition consists 

of dividing the problem into subproblems. It is usually 

followed by refinement, which means defining unspecified 

concepts or objects. These two processes are the core of 

stepwise refinement. 

Composition. On the other hand, composition is the 

process of creating a solution to a problem by combining 

solutions to subproblems. That is, composition is the main 

process in bottom-up development. Composition is used 

most effectively in combination with reuse. 

Abstmction, instantiation, and reuse. Abstraction as a 

design decision consists of specifying a solution to a more 

general problem (i.e., a problem of which the problem of 

interest is an instance), usually defining a set of (formal) 

parameters to describe particular instances. The rationale 

for solving more general problems is that it is often easier 

to abstract away particulars of the problem of interest and 

solve a general problem. Furthermore, the more general 

solution can be reused in other contexts. 

Reuse as a design decision consists of prescribing the use 

of a particular solution to a subproblem. If the solution to 

be reused is parameterized (;.e., it has formal parameters) 

actual parameters must be provided to do the reuse. Instantiation is the process of defining actual parameters for 

a parameterized abstract solution. 

A solution to reuse need not be already implemented: it 

may be simply specified as the solution to another subproblem. When several subproblems in the current design are 

instances of a single general problem, abstraction, instantiation and reuse can be employed to “factor” the design. 

Cnotion oj alternatives and choice. When it is not immediate which kind of design is the best, it is possible to 

create several alternative designs using some of these techniques. A valid implementation must conform to one of the 

created designs. After more elaboration of these designs, 

some are discarded until one design prevails. Choice is the 

process of selecting among alternative designs; the choice 

process is more objective when it is based on preference 

measures [2]. 

Presentation Changes 

Presentation changes are intended to change the precision, formality, readability, modularity or other aspects 

of the specification itself, without affecting the specificand 

set, that is, without adding more information. For example, a condition written in Enghsh, referring to a collection 

of objects can be replaced by a logical predicate in which 

the collection is represented by a set. 

Ideally, a presentation change does not change the 

specificand set, that is, it does not create new requirements. However, restrictions in the specification languages 

or methods used may impose additional constraints. In the 

above example, should our specification language support 

lists but not sets, we might have specified a list as an implementation for a set. If we later coded this list in Pascal 

we might have coded our list specification into an array 

or linked structure rather than the more efficient set data 

type that actually was originally specified. That is, as a 

result of a specification language deficiency we have added 

an additional arbitrary constraint for the program that resulted in it being less efficient, that is, we have added bias. 

Retraction 

4-22 
I Explicit reqs. 1 

I Explicit I 

Figure 3: Classification of requirements: explicitness. 

Fictitious requirements are shown with segmented line 

because they are not real requirements. 

Retraction occurs when a designer realizes that the current design is incorrect or otherwise undesirable. The goal 

of retractions is to create a pertinent specification, as defined in Section 1.4. As we said before, the retraction process is usually done in conjunction with other processes 

that create a new Ureplacementn requirement. 

Inherited 

Figure 4: Classification of explicit requirements: origin. 

reader of the specification and not requirements at a!]: the 

reader believes that they are either implicit, implied or 

absent requirements. 

A real nonexplicit requirement is either an implicit, implied, or absent requirement. 

3.2 Origin 

3 A Model of Bias 

Presence of bias cannot be determined from the requirements alone, because it depends on the origins of requirements. For instance, if the origin of a particular requirement is in the problem, the requirement is not bias; if the 

origin is a misconception it may be. Hence, our definition 

of bias is based on a classification of requirements. 

Requirements are classified into several classes with 

subtle differences. These subtleties are what makes bias 

hard to define and even harder to find. The main classification criteria we consider are explicitness and origin, 

which depends on the process of creation of new requirements. 

3.1 Explicitness 

A requirement is ezplicit if it is present in the specification; otherwise, it is nonezplicit. 

Nonexplicit requirements are a recurring cause for misunderstandings in product development. They are further 

classified as follows (Figure 3). 

Implicit requirements are those that are understood to 

be part of every product in the application domain, and 

so they are left unstated. 

Implied requirements are logical consequences of other 

requirements. 

Absent requirements are requirements unintentionally 

omitted in the specification, but are required by the solution set. These are not part of every product in the 

application domain. 

Fictitious requirements [a] are assumptions made by the 

An expliat requirement is new with respect to a certain specification stage if it is first made explicit at that 

stage; otherwise, the reqliirement is inherited from previous stages. (When the specification stage is clear from context we will say simply ‘new’ or ‘inherited’ requirement.) 

Of course, every explicit requirement is new to one stage, 

namely the stage in which it is introduced. 

The discussion in Section 2 motivates the following classification of new requirements with respect to their origin 

(Figure 4). 

Designed requirements are the consequence of design 

decisions taken at the current specification stage. 

Ezplicatiue requirements are created by explication of 

implicit, implied, or absent requirements. 

Eztmneous requirements are created by explication of 

fictitious requirements. 

Imposed requirements are those imposed by the limitations of the specification method or language used, created 

as a side effect of a presentation change. 

This classification describes possible origins for the requirements, but it does not provide a method to determine 

the origin. For example, without a complete analysis of 

the application domain, there is no definite method to tell 

whether a requirement is extraneous or the explication of 

an implicit requirement. 

3.3 The Nature of Bias 

We define bias in terms of theqrigin of the requirements 

described in a specification: A specification containing extraneous or imposed requirements is biased. 

4-23 
This definition provides insight into the problem of bias, 

including both its origins and consequences. The origin 

of bias is either wrongful interpretation of nonexplicit requirements or the limitations imposed by the specification 

method. The consequences are that the specificand set 

can be overly constrained or that the solution adopted can 

be suboptimal. That is, a biased specification will lead 

the design towards particular implementations that are not 

necessarily the best possible. 

The definition does not provide a method to measure 

bias content in a specification, because bias is defined in 

terms of the origin of requirements and we cannot be completely sure of the origin of some requirements. Furthermore, bias is relative to the application domain and the 

software engineering environment, because the domain and 

environment define what is implicit. 

For example, in an environment in which all programs 

are written in a particular programming language, the 

presence of idioms of this language in a specification is 

not necessarily bias, unless another implementation language is introduced to the environment. This is what 

happened at the Software Engineering Laboratory (SEL) 

at the National Aeronautics and Space Administration 

(NASA).2 During the first experience with development 

in the Ada language they realized that software specifications for satellite dynamics simulators were “heavily biased 

toward FORTRAN. In fact the high level design for the 

simulators is actually in the specifications document” [I]. 

This was not a problem411 the contrary, it facilitated 

both development and reuse of specification and codeuntil the first development in Ada: the specifications had 

to be rewritten first. Given our definition of bias these 

FORTRAN-oriented specifications were not necessarily biased; they contained many designed requirements. Before 

Ada was introduced, the use of FORTRAN was an tmplicit requirement. After that, the choice of appropriate 

language became an ezplicit attribute, resulting in the assumption of FORTRAN as a fictitious requirement. 

The relative nature of bias is an essential characteristic. It stems from the existence of nonexplicit requirements 

and the inherent uncertainty with respect to those requirements. That does not imply that there is nothing to do: 

an obvious task is to make explicit as much as possible 

about the domain and environment. If this is done, we 

are reducing considerably the possibilities of bias. However, as long as there are nonexplicit requirements, there 

will be doubt about these requirements and hence possibility of bias. Making explicit the implicit requirements of 

a certain domain and environment still leaves two sources 

of bias: restrictions on the method and languages, and 

absent requirements. These two cannot be avoided completely: the first because any method that provides some 

’The SEL was created in 1976 to study and improve the 

software process at NASA Goddard Space Flight Center. 

guidance in the specification process will guide the design 

to some particular kind of solutions; the second because at 

the beginning of a project most requirements are absent. 

4 Software Defects 

Both bias and software defects are a consequence of 

problems in the development process. Section 1.4 shows 

the duality of bias and faults by analysing the differences 

of the specificand set and the solutions set. Here this comparison is extended further. We classify software defects in 

three classes [Ill: faults occur in documents, errors occur 

in human processes, and failures occur in automatic processes. There is an analogy between the problem of bias 

and defects: fictitious requirements are like errors (both 

during human processes), imposed and extraneous requirements like minor faults (both occur in documents), and 

inefficiencies like minor failures (both occur during automatic processes). The criticality of the attributes involved 

is related to whether something is considered a fault or 

simply bias. 

During software development, successive specifications 

are written, usually starting from an incomplete specification towards a correct specification. Every specification 

inherits from all previous specifications, so if there is a 

new requirement that contradicts an explicit previous requirement the new Specification is inconsistent and hence 

unrealizable. The only solution is to retract either the 

new requirement or previous requirements. Similarly, if 

a new requirement contradicts a nonexplicit real requirement the specification is made impertinent to the problem 

(Le., it solves another problem); again, the only solution 

is to retract. All too often a specification is unrealizable 

or impertinent but this is not evident to the developers so 

no retraction occurs and development continues. This is a 

secondary but important source of defects. 

We have studied these problems at the SEL. The software analyzed are ground support systems for unmanned 

spacecraft. Most systems are about lOOK source lines FORTRAN programs, but a sizable percentage are now in Ada. 

The SEL has a database describing systems and their development processes made in the last 15 years. The analysis that follows uses data from that database, but only 

considers relatively recent data (since January 1, 1986), 

because the software process has changed. 

Table 1 summarizes counts of change reports classified. 

by type of change (e.g., requirement changes, fault correction) in all SEL projects. From the table, 49.4% of the 

changes are due to faults, 12.3% correspond to planned enhancements and 10.6% are due to requirements changes. 

Table 2 summarizes counts of the changes due to the 

8074 faults of Table 1, classified by source of fault. From 

the table, 74.8% of faults are related to coding and 16.3% 
I TvDe of channe 

Fault source 

Requirements 

Functional specification 

Fault correction 

Environment change 

Improvement of user services 

Planned enhancement 

Presentation changes 

Requirement changes 

Other 

Total 

All faults 

Count % 

76 0.9 

242 3.0 

Count % 

8074 49.4 

533 3.3 

1205 7.4 

2018 12.3 

1464 9.0 

1730 10.6 

1327 8.1 

16351 100.1 

Design 

Subtotal specifications 

Code 

Table 1: Changes by type in SEL projects since 1986. 

996 12.3 

1314 16.3 

6043 74.8 

Previous change 

Other 

Total 

714 8.8 

3 0.0 

8074 99.9 

Table 2: Fault source in SEL projects since 1986. 

of the detected faults are directly related to incorrect specifications (our definition of ‘speafication’ includes three 

SEL phases: requirements, functional specifications, and 

design). This simple analysis demonstrates that up to 16% 

of all problems can be related to implementation bias in 

the specifications. 

However, because requirements documents and their 

changes originate outside the SEL and within some requirements generation group at NASA, these changes are 

not considered faults in the specifications. If we msume 

that the 1730 requirements changes in Table 1 were indeed fault corrections, the total number of faults would 

be 8074 + 1730 = 9804, the total number of specification 

faults would be 1314 + 1730 = 3044 and hence specification errors would account for up to 31.0% of all faults. 

This assumption is not as extreme as it looks, because 

predicted changes in the requirements, improvements and 

environment (hardware) changes are classified separately. 

In summary, considering all faults, between 1/6 and 1/3 

of all faults at the SEL are related to specifications, and 

potentially are related to implementation bias. 

Another source of faults related to specifications are 

faults of omission: when something is not specified it is not 

a problem of the code but of the specification. The fact 

that the problem shows up during coding or testing does 

not mean that the problem is coding. Table 3 shows counts 

of faults of omission, commission, omission/commission 

separated by fault source (the ‘Total’ column is not identical to the ‘All faults, Count’ column from Table 2 because 

Source 

specs. 

Design 253 550 159 996 

Code 2302 2334 921 482 6039 

Prev. chg. I 289 295 79 50 713 

Total I 2965 3297 1207 595 8064 

Percent I 36.8 40.9 15.0 7.4 100.0 

Table 3: Omission and commission faults in SEL 

projects. 

10 faults had invalid data). At the SEL 37% of all faults 

are faults of commission, 41% are faults of omission and 

15% are faults of omission/commission. Thus, about one 

half of the faults are of omission and potentially can be 

attributed to incompleteness in the specifications. 

In conclusion, even though coding appears to be by far 

the most important source of faults, a deeper analysis of 

the specification process reveals that many coding faults 

have roots in earlier stages. Implementation bias undoubtedly plays an important role in many of these 3000 faults 

that are related to changes due to specification issues. 

5 Conclusion 

Even though bias is widely recognized as an undesirable property of specifications, it has not been adequately 

studied. This has caused confusion with the related concept of design decision, so that the presence of designed 

requirements in specifications has been considered undesirable. This is in contrast with the use of specifications 

in other engineering disciplines, where a specification may 

include many designed requirements (e.g., materials, manufacturing methods). 

In this paper we presented a model to describe the nature of bias and distinguish bias from designed requirements and other requirements in a specification. This 

model is based on a classification of all the requirements 

described in a specification and also those that are not described (i.c., nonexplicit); it explains the nature of bias, 

but since it uses nonexplicit requirements it does not lead 

to any definite method to detect bias. However, the model 

does explain both the relative and unavoidable nature of 

bias. Because bias depends on the specification process we 

had to model that process. This modeling shed light on 

the problem of software defects, a relationship that in tum 

helped us to potentially explain the high relative number 

of coding faults found at the SEL. 

Although we have developed an explanatory model 

of the design process, quantification of these concepts is 

needed before we can develop practical procedures for ap 

plying them in large scale developments. Additional work 

4-25 
in this direction in continuing. 

Acknowledgements 

This research was supported in part by grant NSG-5123 

from NASA Goddard Space Flight Center to the University of Maryland. Thanks to Sergio Cirdenas-Garcia and 

Eduardo Ostertag for their helpful comments. P. Straub 

was partially supported by a scholarship from the Catholic 

University of Chile while he was at the University of Maryland. 

References 

[l] Carolyn E. Brophy, W.W. Agresti, and Victor R. 

Basili. Lessons learned in use of Ada-oriented design 

methods. In Proceedings of the Joint Ada Confennce, 

March 1987. 

[2] Sergio Cirdenas and Marvin V. Zelkowitz. Evaluation 

criteria for functional specifications. In Prucdings 

12th Int'l Conf. on Softwan Engineering, pages 26- 

33, Nice, France, March 1990. 

[3] Bernard Cohen, William T. Harwood, and Melvyn I. 

Jackson. The Specification of Complez Systems. 

Addison-Wesley, Reading, Massachusetts, 1986. 

[4] Vinod Goel and Peter Pirolli. Motivation the notion of 

generic design within information-processing theory: 

The design problem space. AI Magazine, 10(1), spring 

1989. 

[SI I. J. Hayes and C.B. Jones. Specifications are not (necessarily) executable. Softwan Engineering Journal, 

pages 330-338, November 1989. 

[6] C.A.R. Hoare. An overview of some formal methods 

for program design. IEEE Computer, pages 85-91, 

September 1987. 

[7] Cliff B. Jones. Systematic program development. In 

N. Gehani and A.D. McGettrick, editors, Software 

Specificution Techniques. Addison Wesley, Reading, 

Massachusetts, 1986. 

[8] Edward V. Krick. An Introduction to Engineering and 

Engineering Design. John Wiley and Sons, New York, 

N.Y., second edition, 1969. 

[9] Harlan D. Mills, Michael Dyer, and Richard C. Linger. 

Cleanroom software engineering. IEEE Software, 

pages 19-24, September 1987. 

[lo] Ellen Shoshkes. The Design Pmess. Whitney Library 

of Design, New York, 1989. 

[ll] Pablo A. Straub and Eduardo J. Ostertag. EDF: A 

formalism for describing and reusing software experience. In International Symposium on Software Reliability Engineering, pages 106-1 13, Austin, Texas, 

May 17-18 1991. 

' 

[12] William Swartout and Robert Balzer. On the inevitable intertwinning of specification and implementation. Communications of the ACM, 25(7):438-440, 

July 1982. 

4-26 
An Improved 

Modules 

0 f- i N93-"JJ19?' 0 >= 

Classification Tree Analysis of High Cost 

Based Upon an Axiomatic Definition 

of Complexity 

Jianhui Tian 

Soft. Eng. Process Group 

IBM Canada Laboratory 

North York Ontario, 

' Canada 

. Adam Porter Marvin V..Zelkowitz 

Computer Science Dept. 

University of Maryland 

College Park, Maryland 

Inst. for Advanced Computer Studies 

and Computer Science Dept. 

University of Maryland 

College Park, Maryland 

Abstract 

Ideniijicaiion of high cosi modules has been viewed 

as one mechanism io improve overall sysiem nliabiliiy, since such modules iend io produce more ihan iheir 

share of problems. A decision ine model has been 

used io identify such modules. In ihis cumni paper, a 

previously developed aziomaik model of program complen'iy is merged with ihe pnviously developed decision 

tree process for an improvemeni in ihe ability io identify such modules. This improvemeni has been iesied 

using data f" ihe NASA Softwan Engineering Lobomio y. 

1 Introduction 

Identification of high cost modules has been viewed. 

as one mechanism to improve overall system reliability, 

since such modules tend to produce more than their 

share of problems. In order to idelitify such modules, 

Selby and Porter (2, 3) developed a decision proce- 

.due based upon decision trees. With their technique, 

which we call Classification nee Analysis (CTA), they 

showed on a set of 16 large-scale programs containing over 4700 modules obtained from the NASA Software Engineering Laboratory, that they could identify 

which subset of the 74 measures obtained from each 

module would produce good estimators of high-cost 

modules. 

Recently Tian and Zelkowitz [4] developed an axiomatic model of program complexity. Based upon 

this model, the 74 measures kept on each of the 4700 

modules could be reduced to only 18 measures that 

represented valid complexity measures. Using these 

18 measures, the decision tree process results in an 

improvement over the original Selby-Porter method. 

In this paper we will first describe the original decision tree process, we then summarize the axiomatic 

complexity model, and then demonstrate that we can 

. improve on the previous model in identifying high-cost 

modules. 

2 Classification Tree Analysis 

In a series of earlier studies by Selby and Porter, 

a technique called classification tree analysis (CTA) 

was used to identify high cost components. Of critical 

importance to CTA is the selection of measures (or 

attributes) to construct the classification tree. 

We define a high cost component as one in the 

uppermost quartile (Le., 25 percent) relative to past 

data. The rationale for this definition is the so called 

"80:20 rule", which states that about 80 percent of a 

software system's cost is associated with roughly 20 

percent of the syskm. 

A classification tree is essentially a decision tree 

that branches on the range of values according to a 

measure at an inkrnal node repeatedly until 'a component can be identified as high or low cost, or until 

all measures are exhausted. 

The'classification tree method that was used, called 

the classification paradigm, consists of the following 

three integral parts: 

Classification tree generation is the central 

4-27 
I Modules I 

2.1 

activity of constructing clansification trees and 

preparing them for analysis and feedback; 

Data management and calibration are the 

activities that retain and manipulate historical 

data and tailor classification tree parametere to 

the development environment; and 

Analysis and feedback is the part that leverages the information resulting from the tree generation by applying it in the development process. 

The central piece of the application of classification tree is to develop remedial plans and take 

corrective actions. 

CTA Method 

The goal is to predict high cost modules in the current project with high cost being interpreted as the 

highest quartile. The historical data (or training set), 

consisting of one project immediately preceding the 

current one, are grouped into quartiles according to a , 

measure's value, with all measures being considered. 

Starting from the root, a measure is selected to sep 

arate modules into four subsets associated with each 

arc. The number to the left of an arc is the lower 

(inclusive) bound and the number to the right is the 

upper (non-inclusive) bound for the subset according 

to the measured value. So we have four subsets (quartiles). 

A set of modules associated with an arc is positively 

identified if more than a threshold (termination criterion) of modules are in the highest quartile of cost, and 

it is represented in the tree as a terminal node marked 

with a u+" sign. A set can be negatively identified 

similarly, and represented correspondingly by a u-n 

sign. If a set cannot be either positively or negatively 

identified, another measure is selected to further classify these modules into finer subsets. This process 

continues until either all modules are identified or all 

measures are exhausted without being able to make 

such a determination. In the latter case, the terminal node is marked with a I'?" sign, representing that 

CTA can not make a prediction for modules in this 

set. 

Notice that the generation of the classification tree 

depends solely on the training set and various.parameters selected for the technique. The current project 

will only use the tree but not affect the structure of 

the tree. 

I module+rctioncall I 1 4; ; 1 1; 1 

operators 30 18 10 33 58 

pr 'ction -?-- + 

module calls 

I-ytUal I- - 4- - +I 

Table 1: Predicting High Cost Modules 

As an example, consider the sample (fictitious) test 

data of Table 1, and the classification tree in Figure 1. 

This test set includes 5 modules and 4 measures. In 

this case, the CTA method predicts 3 out of 4 modules 

correctly (it misses module ms) and is unable to classify module m, through the classification tree. For 

example, module m5 follows the right most branch 

from the root (cyclomatic complexity of m5 is greater 

than 26) and again follows the right most branch from 

there (operator counts of ms is greater than 34). We 

can finally predict it to be of high cost because its 

module call counts falls between 4 and 10. 

2.2 CTA Cost 

There are two types of cost associated with the CTA 

technique: the cost of building classification trees and 

the cost of using them. The former is determined by 

the factors: 1) the CTA parameters, 2) the size of 

the available measure pool where measures are to be 

selected, and 3) the implementation efficiency of the 

CTA supporting tools. For the latter cost factor, the 

tree size is a good measure. Because the classification 

trees we are studying have fixed structure (there are 

4 branches from every internal node), we can effectively capture the cost of using classification trees by 

counting the number of internal nodes for them. 

2.3 CTA Performance 

According to the match between CTA predictions 

and actual cost data for the modules in a test set, 

various performance measures can be defined: 

Coverage: The percentage of modules (either positively or negatively) identified; 

Accumcy: The percentage of correct matches between 

predictions and actual data; 

Comsisicncy: The percentage of predicted high cost 

modules who are actually high cost. High consistency 

4-28 
cyclomatic 

complexity 

- - + ? ? + - module: u yfl/lo\.; 

- ? ? + 

Figure 1: Component Classification Tree 

indicates less “false alarms;” and 

Completeness: The percentage of actual high cost 

modules predicted correctly by CTA. It reveals the 

power of CTA to uncover high cost modules. 

3 Axiomatic Program Complexity 

Most program complexity studies define complexity 

as a numeric comparison between any two programs. 

However, we have come to realize that some programs 

are inherently incomparable. For example, it makes 

litttle sense to compare the complexity between a payroll system and a real-time emission control system in 

a car. They each come from radically different application domains. 

Instead we view complezity as a partial ranking 

among the set of programs and a wmplezity measun 

as a function applied to specific programs as an approximation of the attribute we are trying to determine. The following summarizes this model [4]. 

3.1 Axiomatic model 

Consider a program as a hierarchy of modulcs consisting of instructions, data, and the underlying execution control mechanism. We initially li& ourselves 

to a Pascal-like nested scope sequential control language. Programs are represented by their abstract 

syntax trees: 

0 U represents the set of all programs. 

0 AST(P) repreaents a binary abstract tree representation for progrm P. The root node of program P is given by toot(P), the left subtree of P 

is left(P) and the right subtree of P is given by 

tight (P) . 

. 

0 For programs P and Q, IN(P,Q) is true if P is 

a subprogram of Q (i.e., AST(P) is a subtree of 

AWQ)). 

If IN(P,Q) is true, then dist(P,Q) represents 

the path length in order to go from toot(P) to 

r4Q). 

0 P with all free occurrence of 2 replaced by y not 

in P is denoted as Py’. We use Pa” to mean the 

renaming is carried out for all corresponding oneto-one pairs in lists CY and p, where 

(uar(P) is the variable list of program f). 

A wmplcn’ty ranking R is a binary relation on the 

set of programs. The complexity ranking between‘programs P and Q is R(f, 0). We interpret R(P, Q) as P 

being no more complex than Q. P and Q are wmparuble, denoted C(P,Q) , if either R(P,Q) or R(Q,P) 

holds, i.e., C(P, Q) iff R(P, Q) V R(Q, P). 

4-29 
A wmpletiiy measure Y is a function that maps 

every program into a vector of real numbers: Y : U -., 

R" . 

Although simple definitions, we ate immediately 

confronted by a diflicult problem: 

Theorem T1: There exist complexity rankings that 

are undecidable.' 

Although the general problem of complexity ranking is undecidable, many practical rankings are not. 

In what follows we restrict ourselves to these more 

practical rankings. 

Axiom Al: (VP, Q) ( = 3 C(P, Q) ) where 

[xl is the function of program X. 

Given programs P and 8, the problem of = 

is unfortunately also undecidable. This axiom, then, 

is at the center of the problem of developing effective complexity measures on real programs. We certainly want to be able to compare equivalent programs 

in order to determine which is best; however, undecidability says that we cannot always do this. It is 

for this reason that most complexity measures have 

not achieved significant breakthroughs since the underlying models are rarely comparable. However, in 

many practical applications, such as described above, 

we know or can assume that two given programs have 

the same or similar functionality. 

Because of this, in practice we often use a weaker 

form of this axiom that only addresses the similarity 

of two programs: 

Axiom Al': (VP, Q) ( x 3 C(P, Q) ). 

A program in general consists of many hierarchically related components. As a result, we require that 

a program must be comparable with a subpart of itself. 

Axiom A2: (VP, Q) (IN(P, Q) 3 C(P, Q) ) 

~ 'Axiom and theorem references an hsd to [4], which ah 

contains the proofs of the theom. Some of the theorems 

given in that ,paper are not relevant to this present discusion 

and hence are not listed hue. 

Axiom A2 brings up the intuitive notion that we 

would like complexity to increase as programs become 

larger, i.e., if P is a component in Q (IN(P, Q)), then 

P is no more complex than 9. We left this out because 

there are cases where the opposite is true. Consider 

Q formed from P by addition of easily recognizable 

keywords or tags; Q might be more readable, thus 

easier to maintain as a result. Another case is that 

loops are often more easily understood if they include 

their initialization code than if presented without it. 

Contextual information might help to reduce the 

complexity of composite programs. But the degree 

of the reduction must be limited, otherwise infinitely 

large programs paradoxically might be the simplest. 

On the other hand, a periodic function such as cosine(z) as the complexity of a program, where z is 

some size measure of a program P, is clearly not acceptable. As a general trend, then, adding components must result in a more complex program: 

Since our goal is to compare the complexity of two 

different programs, define a predicate 7 such that 

I(V(P),V(Q)) is true if program P is no more complex than program Q. For V into R, we have the obvious definition that I(V(P),V(Q)) i. just (V(P) < 

Y(Q)). For higher dimensions, other results are possible (e.g., a dot product called the performance level 

measure which compares alternative software designs 

[11>. 

7 is our decision process which-determines how 

well V approximates our complexity ranking 72 between P and Q based on the measured complexity 

values Y(P) and V(Q). We would like the relationship to be 7(V(P), Y(Q)) - R(P, Q), and in fact it 

is an implied axiom in most other complexity models. 

However, we believe that this is the major weakness 

that has prevented most complexity models from being truly effective. Because of undecidability issues 

(e.g. theorem Tl), for all P and Q we cannot determine 7 for every 72. As a result, we use a weaker 

condition, namely: 

Axiom A4: (VP, Q) (72(P, Q) =+ V(P) 5 V(Q) ) 

Since for many useful applications, 72 defines a total 

ranking, we then have: 

4-30 
Theorem T5: When 'R is total, i.e., (VP, Q)C(P, 8) , we have: 

In order to be useful, we would lie our complexity measures to distribute programs across a range of 

values. If there is only a single "dominating" cluster 

point, we gain little information from the measure. 

Axiom A5 allows, for rough comparisons, bi-polar or 

multi-polar distributions: 

Axiom A5: (Vk E R)(36 > 0) (IU - {P : V(P) E 

[k - 6, k + all1 = IUD 

Axiom A5 forces our complexity measure to be 

nontrivial, as in: 

Theorem T7: (VP)(3Q) (V(P) # V(Q)) 

When V maps programs into a discrete bounded 

set S, axiom A5 requires that at least two points in S 

have infinitely many programs with such values: 

Theorem T8: If set S of complexity values is finite, 

then: 

3.2 A classification model 

Given these five axioms, we developed a classification model for categorizing the various complexity 

mesures depending upon the information they provide. A vertical classification uses a subset of the attributes for the entire program, while a hierarchical 

classification uses some functional relationship among 

the program's parts. 

Veriical classification 

A complexity ranking 'R is abstmct, denoted 

AB(R), if given P and Q with AST(P) = AST(Q), 

then R(P, Q)(and equivalently, R(Q, P)). 

If two programs are syntactically identical except 

for variable names, as long as two set of names are 

isomorphic, the only conceivable differences is interpretational (the meaning attached to each name). On 

the other hand, when considered functionally, each 

name is just a surrogate for the underlying data object. Thus we have the classification: 

A complexity ranking R is functional, denoted 

FN('R), if given P and Q with name,sets a and P 

such that A3T(PpO) = AST(Q), then R(P, Q). 

Haerarchieal classification 

Assessing complexity by using only the components 

while ignoring interactions (i.e. ignoring the context 

where the components are defined and used) results in 

a contezt free ranking: A complexity ranking 72 is contezt fne, denoted CF(R), if given PI its ranking with 

respect to any given Q can be uniquely determined 

by: (1) Q and (2) root(P), the complexity ranking of 

left(P), and the complexity ranking of right(P). 

As a special case of context free complexity where 

organizational information is completely ignored, we 

can have primitive complexity: A complexity ranking 7Z is primitive, denoted PR(R), if all programs 

P and Q with the same collection of AST(P) and 

AST(Q) nodes (same number of occurrences for each 

corresponding pair) then R(P, Q). 

Also, a complexity ranking R is inferactzonal, denoted IA(R), if it is not context free, i.e. lCF(72). 

Without considering interaction, the complexity of 

the composite complexity is the sum of all the components complexities. However, due to interaction 

among component parts, the total complexity may be 

greater than the sum. Such a complexity ranking is 

called oucnzll. 

If we are allowed to modify the internal structure, 

or reorganize the program according to some programming practices (such as modularization, data abstraction and information hiding), we may be able to cut 

down the interfacing complexity, thus the overall complexity. Since the two programs are functionally equivalent, they are comparable in complexity (A2). 

The relationship among different hierarchical 

classes can be summarized in the following tree: 

4-31 
4.1 Measure Screening 

Hiemnhicol 

Ovemll 

Not Ovemll 

-OA('R) 

OA('R) Intemctionol 

IA('R) 

Using this model, we have been able to develop 

Weyuker's 9 properties for complexity measures as 

special cases of our axioms [5]. Since those properties have been widely studied over the past 4 years, 

and since we can model her properties with our classification model, we believe that our axiom are a reasonable apprcximation of program complexity. 

4 Application of the Model 

Sixteen software system, ranging from 3000 to 

112,000 lines of FOKI'RAN source code, were selected 

from NASA ground support software for unmanned 

spacecraft control developed in the NASA/GSFC Software Engineering Laboratory. Each required between 

5 and 140 person-months to develop over a period of 5 

to 25 months by 4 to 23 persons. Each project contains 

from 83 to 531 modules, totalling over 4700 modules. 

There are 74 attributes, each quantified by a specific 

measure, for each module divided into three broad categories: fault, effort, and style (or complexity). 

For each application instance, one of the projects 

was used as a training project in order to develop the 

classification tree for the next project. This was repeated for the remainder of the 16 projects. 

Five of the projects were of a greatly different size 

than the others (by more than a factor of 3). We 

deemed these to not fulfill Axiom Al' on similarly 

of functionality. This reduced the set of projects to 

11 (and 10 data points) and are given as Group A in 

what follows. We used a different ordering of 6 of the 

projects in terms of training set to give us Group B 

(and 5 additional data points). CIA refers to the original Classification nee Analysis process, while ACT 

refers to the Axiomatic Classification 'Ikee process de 

veloped in this paper. 

Aom the set of 74 measures for each.module, we 

first eliminate all measures that are not directly measureable from the modules themselves. Thus effort 

data, e.g., number of hours to develop the module, 

are eliminated. We also eliminated change and error 

data since they represent interactions among program 

components and the operational environment. We can 

therefore reduce the number of measures to 40. 

All candidate measures satisfy axioms Axiom Al' 

(comparing functionally equivalent programs), Axiom A2 (comparing component-composite pairs), 

Axiom A4 (measures agree with their ranking), and 

Axiom A5 (no single cluster). However, many of 

the measures do not satisfy Axiom A3, the general 

monotonicity axiom. These measures are averaging 

measure such as assignment statements per 1000 ezecuiable siatemenis, which may be correlated with average effort per 1000 lines or 80, but not with the total 

development effort. Therefore these measures will be 

eliminated. This reduces the candidate measures from 

40 to 18, with the candidate measure set S being the 

left half of lhble 2. 

Both abstract and non-abstract aspects contribute 

to cost, so measures from any vertical class are potentially acceptable. On the other hand, as we are only 

considering cost and complexity at the module level, 

the hierarchical classification is not relavent. The 

analysis based on the measure classification scheme 

does not eliminate any measure for CTA in this case. 

4.2 Aggregate Evaluation 

Given 18 remaining messures that meet the boundary conditions based on the axioms and measure classifications, we next determine which of them best predicts total effort. The underline distribution, as we 

assumed, is a four region distribution (grouped into 

four quartiles) determined by historical data. A quartile of modules is positively identified if more than 

75% of the modules (tolerance level: 25%) have the 

upper most quartile of effort. The negative sets can 

be similarly identified. 

Let m(V) (i = 1,2,3,4) be the number of modules 

in quartile i using measure V; pi(V) be the proportion 

of modules in m(V) belonging or to the upper most 

quartile of effort; and nj(V) be the rest proportion in 

m(V) (therefore pi(V) + ni(V) = 1). As a result, 

a quartile is positively identified if pi(V) 2 0.75, and 

4-32 
Meets Axiom A8 

assignment statements 

inpuboutput statements 

input-output parameters 

source hes 

comments 

source lines minus comments 

executable statements 

function calls 

module calls 

function plus module calls 

cyclomatic complexity 

operators 

operands 

total operators 

total operands 

decisions statements 

format statements 

origin 

Fails Axiom A3 

assignment statements per 1000 executable statements 

input-output statement per comment 

input-output parameters per comment 

inputoutput statements per 1000 executable statements 

input-output statements per input-output parameter 

inputoutput statements per 1000 source lines 

function calls per comment 

function calls per inputoutput statement 

function calls per function plus module call 

function calls per input-output parameter 

function calls per module call 

.module calls per comment 

module calls per input-output parameter 

module calls per function plus module call 

module calls per input-output statement 

function plus module calls per 1000 source lines 

function plus module calls per inpuboutput statement 

function plus module calls per inpuboutput parameter 

function plus module calls per 1000 executable statements 

function plus module calls per comment 

cyclomatic complexity per 1000 source lines 

cydomatic complexity per 1000 executable statements 

Table 2: Attributes passing initial screening 

negatively identified if %(V) 3 0.75. 

To formulate the objective function for the aggro 

gated selection, we need to evaluate the contribution 

of each quartile. We can weight them by the number of modules falling into the quartile. Therefore, we 

formulate our selection criteria as: 

for i ranging from 1 to pi(V) 2 0.75 V ni(V) 2 0.75 

This selection criterion maximizes the number of 

modules in positively or negatively identified quartiles. 

For each of the quartiles neither positively nor negatively identified, another measure is selected using the 

same criterion. The process continues until all modules are identified or all measuru are exhausted. 

5 Results 

We applied both the original CTA process and the 

modified ACT process to the 16 NASA projects broken 

down into the 11 projects of groups A and six projects 

of B. The following sections describe the results of this 

analysis. 

Size of generated trees 

One measure of the efficiency of the technique is 

the size of the classification trees that are generated. 

Figure 2 shows that the axiomatic model (ACT) reduces tree size approximately 27% over the original 

CTA model from 188 nodes to 136 nodes in the 15 

programs with average tree size dropping from 12.5 to 

9.1 nodes. 

The smaller the tree the more desirable (less costly 

to use to navigate through the tm, fewer measures to 

collect), thus a point in the upper left region represents 

an improvement over the original CTA. 

Performance coverage 

Table 3 compares the coverage based on the original 

and modified classification trees. In all the projects 

except one, near 100% coverage is achieved by both 

methods. Thus the decision tree analysis method almost always will predict a cost for a module and will 

4-33 

loO05788L 
(CTA 

/ACT/ 9 11 2 5 74 4 3 3 3 9 13 3 2 4 1112.3 2.6 9.1 I 

individual dbta points average 

group A I group B (A B ali 

66 76 78 63 53 67 71 85 73 71170 50 81 77 58170 68 69 

individual data point8 average 

PUP A 1 group €3 ABd 

ICTA 70 66 31 54 52 63 30 16 50 101 7 100 33 17 65 39 35 38 

[group A group B d 

ICTAI w 99 97 

ICTA 

b. average comparison 

Table 3: Coverage Comparison 

individual data points average 

group A I group B A B ali 

26 60 54 62 42 21 42 33 6 471 7 4 40 63 47 38 28 35 

rarely leave modules unclassified. So, we can conclude 

that the CTA technique using either selection method 

achieves fairly good and consistent coverage, with an 

average of 97% coverage for both. 

individual data points 

group A IgroupB 

Performance accuracy 

average 

A B all 

Accuracy improved about 5% with the ACT process, as given in Table 4. 

rCTA 

Performance consistency 

17 15 7 8 86 4 9 3 4 1813 6.3 4 1 17.7 3.4 12.5 

Table 5 gives the consutency comparison. This is 

the measure that drives the whole process, being that 

identification of high cost modules is the major goal 

CTA 

ACT 

group A gronp B 

98 98 99 98 91 93 97 100 100 98 100 98 97 97 100 

99 100 97 100 82 93 100 98 100 99 98 98 100 97 100 

IACT167 73 80 66 50 67 81 83 73 89179 54 86 85 58175 74 741 

Table 4: Accuracy Comparison 

\ACT167 61 37 57 56 63 50 15 50 23143 85 40 29 65150 50 501 

Table 5: Consistency Comparison 

LACT(30 46 73 59 49 21 14 33 6 30171 13 40 63 47135 39 35) 

Table 6: Completeness Comparison 

of the prediction process. 

The performance level between the two selection 

methods is significantly different, with the modified 

ACT selection method outperforming the original 

CTA method by a margin of 50% to 38%. 

Performance completeness 

While ACT generates many fewer “false darmsIJ’ 

(ix., predicting high cost modules which really are 

not high cost - the above consistency measure), both 

methods are comparable in actually identifying the 

high cost modules, i.e., the completeness measure of 

Table 6. That is, both will fail to indicate high cost 

modules in aver half the cases. 

6 Conclusions 

Classification Bees are a method to use measureable quantities from program modules in order to determine desireable attributes from the development 

process. Identification of high cost modules should 

correlate closely with other process measures such as 

reliability. 

In this paper, we presented a Classification nee 

Analysis (CTA) method and a modification to it, 

the Axiomatic Classification Pee Analysis (ACT) 

method, where an axiomatic model of program complexity was used to develop the candidate measures in 

the classification tree. 

4-34 
In all important measures, the ACT was either M 

good as or improved upon the original CTA model: 

(1) Classification tms were smaller; (2) Caverage was 

[4] Tim J. and M. V. Zelkowitz, “A formal program 

complexity model and ita application,” J. of Sysiemr and Softwan 17,3 (March, 1992) 253-266. 

the same; (3) Accuracy improved; (4) Consistency improved and (5) Completeness was the same. We therefore believe that we have a candidate process that improves upon the original model. 

15) E. J, Weyuker, ‘Evaluating Software Complexity 

Measures: IEEE %-. on Software Engineering, 14, 9 (1988) 1357-1365. 

Using an axiomatic basis for classification trees has 

two 

1. 

2. 

- important economic benefits: . 

By eliminating unnecessary measures from the 

classificaiton tree (e.g., reducing the list from 74 

to 18 in the NASA SEL experiment), we eliminate the n&d to collect such data. This would 

imply less overhead on the development process. 

The axiomatic classification tree analysis technique generates improved results, allowing management to better control and evaluate the development process and allow for more informed 

decision making with less risk involved. 

Of course there is still much more to be done. ACT 

is only right on 50% of the modules it calls high cost, 

and only finds accurately over one third of these modules. However, the method is improving, and is inexpensive to use since it is available as a byproduct of 

static analysis of the developing code. Further work 

will continue on developing these models. 

Acknowledgements 

This research was supported in part by National 

Science Foundation grant CCR-8819793 and National 

Aeronautics and Space Administration grant NSG5123 to the University of Maryland. 

References 

Cirdenas S. and M. V. Zelkowitz, “A management tool for the evaluation of software designs,” IEEE %ns. on Soflwan Engineering 17, 

9 (September, 1991) 961-971. 

Porter A. A. and R. W. Selby, “Empirically 

Guided Software Development Ueing MetricBased Classification ‘Ikees” , IEEE Soflwan, 

(March, 1990) 4654. 

Selby R. W. and A. A. Porter, “Learning from 

example: Generation and evaluation of decision 

trees for software resource analysis,” IEEE %ns. 

on Soflwan Engineering 14,12 (1990) 1743-1757. 

4-35 

Providing an Empirical Basis for Optimizing the Verification and 

Testing Phases of Software Development 

I 

Lionel C. Briand, Victor R. Basili and Christopher J. Hetmanski 

Institute for Advanced Computer Studies, 

Computer Science Department, 

University of Maryland, College Park, MD, 20742 

A bstract: 

Applying equal testing and verijication effort to all parts 

of a software system is not wry eficient, especially when 

resources are limited and scheduling is tight. Therdore, 

one needs to be able to differentiate low I high fault 

density conlponents so that testing I ver#htion cfort can 

be concentrated where nee&d Such a strategy is upccred 

to detect more faults and thus improve rhe resulting 

reliability of the overall system. This'paper presents an 

alternative approach for constructing such models that is 

intended to fuflll spec@ii sofhare engineering ace&, (ie. 

dealing with partial I incomplete information and creathg 

models that are easy to interpret). Our approach to 

classification is to (I) measure the sofware system to be 

considered and (2) build multivariate stochastic models for 

prediction. We present experimental results obtained by 

classifying FORTRAN components developed at the 

NASA Goddard Space Flight Center into two fault density 

classes: low and high. Also, we evaluate the accuracy of 

the model and the insights it provides iwo the software 

process. 

Key work: fault-prone software components, stochastic 

modeling, machine learning. 

1. Introduction 

In this paper, we address the issue of identifying high fault 

density software components via empirical stochastic 

modeling. If we can identify components that produce a 

great deal of faults relative to their size, then we can 

concentrate the verification and testing processes on them 

and thereby optimize the resulting reliability of the 

developed software system. However, building such 

Research this study was supported in part by NASA grant NSG 

5123 and by AFOSR 90-0031 

stochastic models is a difficult task. The data collected is 

often incomplete andb hcteropcous and presents many 

problems with respect to model construction (e.g. 

mtcrdepd~ics, outliers, complex relationships). In this 

paper, we present an alternative modeling process based on 

both Statistics and machine learning principles W831. We 

show how the process facilitates the identification of high 

fault density components based on memcs obtainable at 

the end of the coding phase. 

The modeling approach presented in this paper, called 

Optimized Set Reduction (OSR). has been developed at the 

University of Maryland PBT91 J in the framework of the 

TAME project [BR88] . It is derived from the ID3 model 

[Q79, Q86, BRW] which was originally developed for 

automatic generation of classification/decision trees. As 

discussed in [CE87,BBT91], the use of ID3 has several 

inherent problems and leaves mom for improvement with 

respect to many data analysis and modeling issues (i.e. 

small data sets, missing data values, noisy data, 

heteroscedasticity). Our motivation for developing OSR 

and a tool to support it was to design a data analysis 

technique matching, to the extent possible, the specific 

needs of building multivariate empirical models for 

software engineering. The issue of using OSR for 

predicting on a continuous range is addressed in PBT91J. 

In this paper, we discuss using OSR to classify software 

components into two fault density classes (low. high). 

In Section 2. we present the basic principles of the OSR 

algorithm and formally define the approach. This 

formalism is intended to give an unambiguous 

presentation of some of the features of OSR rather than a 

complete definition of it. Saction 3 discusses the issue of 

building models based on partial information (i.e. missing 

data for technical or cost reasons). Section 4 presents a 

process called "pattem merging" whose goal is to facilitate 

interpretation and leaning based on the generated models. 

Sections 5 and 6 present some of the resulls obtained via 

4-37 

1oO05788L 
I Distribution: I 

Probablllty 

CPLX = Nominal 

RELY 

DATA 

= 

= 

LOW 

High 

L 

Productlviti 

Figure 1: Example of a Pattern and its Associated Probability Distribution 

experimentation using OSR. Bascd on these results, we 

can determine the accuracy of the model. Also, we can 

compare OSRs outputs with those of a logistic regression 

based model. which is one of the most standard statistical 

techniques for classification [HL89, AG90J. Finally. 

Section 7 underlines the major conclusions and directions 

for futm d. 

2. Optimized Set Reduction 

2.1 Basic Principles 

Let us assume we want to 8sscss a Mcular characteristic 

of an object (e.g. the fault density of a component). We 

will refer to this characteristic as the Dependent Variable 

(Y). The object is represented by a set of explanatory 

variables which describe the software component (called 

Xs). These variables can be eithm disc- or continuous. 

For example, a software component may be described by 

two Xs, its cyclomatic complexity (continuous) and the 

type of its function (discrete). Also, assume we have a 

historical data set containing a set of pattern vectors that 

contain the previously cited Xs plus an associated actual Y 

value. We will call the Xs portion of the pattem veclor a 

meoswenunt vector. 

The goal of the OSR algorithm is to determine which 

subsets of experiences (Le. patlem vectors) from the 

historical data set provide the best characterizations of the 

object lo be assessed. In other words. we lry to determine 

which subsets of the data set yield the "best" probability 

distributions on the Y range. A good probability 

distribution is a probability distribution concentrating a 

large number of pattun vectors in either a small part of 

the range (Y is continuous) or in a small number of 

dependent variable categories (Y is discrete). One of the 

commonly used probability distribution evaluation 

functions is the information theory entropy (H). 

Altcmative probability distribution evaluation functions 

aredlxmsed in [QM, SP88, M89J. Each of the subsets of 

the historical data set yielding "optimal" distributions, 

refend to as opW subsets, are characterized by a sa of 

conditions (rcfud to as predicutes) which arc me for all 

pattern vectors in that subset. Each set of predicates 

characterizing a subset is called a patfern. Figure 1 shows 

an example of a pattern and its associated probability 

distribution in the data set. The pattem is composed of 

three predicates when thedepudent variable tobe assessed 

is "development productivity". Figure 1 shows that if 

these predicates (Le. ComPLeXity = Nominal, 

RELiabilitY=Low, DATA base size = High) are me for a 

project, then its productivity is most likely to be in the 

second productivity class. 

2.2 Formal Definition of the OSR Process 

We want to identify oprimul subsets in the historical data 

set. We can formalize the process using set theory and 

predicate calculus by defining the function Opt. Let us 

assume we have a set of m explanatory variables 

{xl.X2. ... .Xm} and a corresponding set of explanatory 

variable value domains (EV1.EV2. ... .EVm}. Let us define 

the measurement vector domain to be MV = , . F- II EV, . The dependent variable value domain @V) may be Seen as 

a set of classes which can be either intervals or categories. 

Therefore, the value domain of the paaem vectors in the 

datasetcanberepresentedasPV = DV x MV. Let 

PVS be a set of patiem vectors representing the historical 

data set (PVS E PV). A predicate is a variable value pair 

(i.e. an Xi and its corresponding explanatory variable 

value). 

4-38 
Definition 1: Let PSS be a subset of PVS and let the 

measurement vector mv describe the object to assessed. 

VALID(PSS, mv) is true if mv is composed by at last 

one predicate which is true for all the pauern vectors in the 

set PSS. 

PSS E; PVS A mv E MV A SiE(l..m) 

such that Vpv E PSS (mv(i) = pv(i) ) 

j VALID( PSS, mv) 

Definition 2: TC(PSS, PVS) is true if the two data sets 

PSS and PVS do not show a statistic$ly significant 

difference in distribution on the DV range. This is may be 

evaluated by performing statistical inference tests for 

comparing distributions. We currently use a binomial test 

for proportions since it does not have any applicable 

restraints (e.g. minimum expected frequencies like the 

Chi-squared test of independence)[CA88]. For each 

dependent variable class, the probability that proportions 

in PSS and PVS differ by chance is calculated. If for a! 

least one of the classes, this probability is below a level 

of significance TC defined by the user, then we reject the 

hypothesis that the two distributions 81c identical. TC 

stands for Tetmination Criterion because the OSR pmss 

will be terminated if the condition defined by TC is true. 

Definition 3: EMIN(PSS1, PVS) is true if PSSl is one 

of the subsets of PVS yielding a minimal normolited 

entropy H upon all subsets of 

pamm vectors (e.g. a one vector subset has a minimal 

entropy but it is not a statistically significant subset and 

therefore is not relevant here). 

.. .. 

where 

H(PSS) = C- p(PSS.d)logiap(PSS,d) dcDV 

where 

p(PSS. d) is the a priori probability that a vector 

which is an element of PSS has a dependent 

variable value belonging to the dependent variable 

class d 

Definition 4: Opt(PVS, mv) is a function yielding a set 

of optimal pattern vector subsets. 

Opt(PVS, mv) = (PSS s PVS I VALID(PSS, mv) 

A Em(PSS, Pvs) ) 

-The number of possible predicate combinations 

makes the scarch execution time prohibitive. 

*We want the patterns to contain a minimal set of 

predicates. i.e., we want all the predicates in the 

pamm to have a significant impact on the resulting 

pauan enmy- =We loost some information about the relative impact 

of the various predicates in the entropy reduction 

Proc-. 

.The contexts in which the various predicates appear 

relevant me undettrmmed. 

Therefore, we implement a greedy algorithm using the 

function Opt which addresses the issues mentioned above. 

The Optimized Set Reduction algorithm can be roughly 

described by a three step recursive algorithm. 

Step 1: If the dependent variable is continuous, its range 

is divided into a set of classes according to two main 

factors: the necessary model accuracy and the size of the 

data set Then, the ranges / categories of the explanatory 

variables ~fe divided / clu~ted into classes (e.g. Classil 

... Classij for the explanatory variable Xi) based on 

meaningful class creation techniques. For example, a 

Complexity range can be divided in three classes: low, 

avaage, high. Numerous techniques can be used in order 

to create meaningful classes (e.g. cluster analysis) lDG841. 

However, this issue will not be addressed in this paper. 

Step 2: Select all the pattem vectors in the data set 

having a value for the explanatory variable Xi belonging 

to ClaSSik, where the Xi for the object to be assessed 

belongs to the same class, and where the subset 

characterized by the predicate Xi E Class, yields the 

minimum value for H. However, 

several subsets (characterized by different predicates) , 

yielding ”similar” minimal entropies (Le. the similarity 

criterion has to be defined by the user of the algorithm) 

can be extracted at once. Let us call PSSi the extracted 

subsets of pauern vectors. 

Step 3: Step 2 is repeated in a recursive manner on each 

subset PSSi and each successive subset until the user 

defined termination criteria crc) is reached. 

This OSR algorithm can be formally specified as a two 

parameter recursive function where PVS is the historical 

data set and mv the vector describing the object to be 

assessed: 

.. .. 

However, the function Opt as defined cannot be used as an 

algorithm to extract the optimal subsets. The most 

important xea!jons are: 

4-39 
Subsetl.1 SubSctl2 Subset2.1 subaet2.2 

I "Subset of' relationship \ Extracted subset 

. . Figure 2: Example of OSR Hierarchy 

OSR(PVS. mv) = ifOp(PVS, mv) f 0 

then 

The whole subset extraction process can be represented as 

a hierarchy (set Figure 2). Note that this representation 

should not be confused with a partition tree since: (1) the 

extracted subsets are not exclusive and (2) a subset can 

have several parent subsets. Each path of the hierarchy 

represents a generated pattem (e.g. Figure 2: 

x, E class,, AND x, E as,, defines Subsetl.1) which 

is relevant to the particular prediction to be prformed. As 

shown in Figure 2. two pauems may yield exactly the 

Same subset (e.g. Subset 2.2). The extracted subsets (Le. 

leaves of the hierarchy) which form various probability 

distributions across the dependent variable range may show 

different trends. For each leaf probability distribution, if 

the dependent variable is discrete. the dependent variable 

class containing the largest number of pattem vectors may 

be selected as the most likely class for the new object 

(characterized by mv) to lie in. Using an alternative 

Bayesian approach, a loss I risk function could be defined 

by the user [BBT91]. In this case, the dependent variable 

class yielding the minimum expected loss is selected. 

Each pattean prediction (Le. hierarchy l@ is used to make 

a fd global prediction based on predefined decision rules. 

In order to perform such decisions effectively. we need to 

be able to evaluate the accuracy of the identified pauerns. 

This issue is treated in Section 3. 

3. Handling Partial Information with OSR 

3.1 Definition of the Problem 

As mentioned above, analyzing complex data sets and 

variable relationships is a very difficult task for several 

xeasons (i.e. incomplete / heterogeneous / small data sets, 

missing data, complex interdependencies). The most 

common of these is the problem of partial information. 

Our lack of understanding of software processes (due to our 

lack of experience and the wide variability from one 

development environment to another) makes experience 

difficult to reuse. Also, because of cost and schedule 

related constraints, necessary data cannot always be 

collected. All of these issues contribute to the 

incompleteness of our data. 
Missing information reduces our ability to predict and 

understand. However, we have to establish whether or not 

the lack of a piece of data is an obstacle to prediction. This 

means that we need a model that both generates predictions 

and provides some insight into the reliability of each 

individual prediction. A goodness indication at the model 

level such as the coefficient of determination in Ieastsquares regression analysis is not sufficient since it fails to 

yield an individual retiability meawe fa each prediction. 

For example, let us say we wish to predict project 

productivity according to collected physical features of the 

system and predefined quality requirements. Suppose we 

do not have any information about the team experience 

related to the programming environment and the 

application domain. This information might be somewhat 

irrelevant, i.e. if the structural complexity of the software 

and the required system reliability are low, then the 

variance of the prediction is small. However, if high 

reliability on a complex software system is expected, them 

people rated as having low experience are likely to 

generate schedule and/or budget slippages. This will make 

any prediction based exclusively on other criteria 

meaningless. Therefore, we need a modeling approach 

that can answer the question: Do I have enough 

information to make a reliable prediction? 

3.2 Solutions to Partial Information within 

the OSR Framework 

However, a pauem is not only a logical proposition. The 

order in which the predicates appear in the hierarchy 

(Figure 2) is relevant from an understanding perspective. 

A predicate is relevant only when the conditions defined by 

its preceding / parent predicates in the hierarchy (i.e. 

referred as to the conrext of a predicate in a panicular 

pattem) are me. For example. Predicate 1 significantly 

reduces entropy by itself. Also, in the context of ' 

Predicatel, Predicate2 significantly reduces entropy. 

However, based on this pa", there is no evidence that 

predicate2 significantly reduces enuopy by itself. 

The notion of pattern reliability and significance, as 

mentioned above, can be more formally defined as follows: 

the reliability of a pattern with respect to a particular 

dependent variable class is the probability that the pattern 

will predict the cOrrect value for the dependent variable. 

Let DVclass; be dependent variable class i. Let T equal the 

number of generated pauems (Pj) that predict DVClasSi. 

Let C equal the number of paterns which correctly predict 

DVclassi (based on the actual DV value of the pattern 

vector for which the pattern was produced during DEA). 

Then we define the reliability of Pj with respect to the 

dependent variable class DVClassi as: 

R [DVClasSi ; Pj] = C / T 

The probability that a pattern appears T times yielding a 

particular classification DVclassi times by 

chance (P(C*T,P) Can be expressed by the binomial 

distribution: 

For each meaSurement vector in the historical data set we 

run the OSR algorithm using as an initial data set (i.e. 

set the top of the OSR hierarchy) the historical data set 

minus the measurement vector to be predicted. It is 

moved from the data set in order to avoid any bias in the 

results. We therefore extract specific pattems for each 

measurement vector and form a set of pattems representing 

the trends observable on this mcular data set 

This resulting set of patterns, or Specific Pattern Set 

(SPS) may be Seen as a model of the historical data set. 

Many of these patterns will be the same or "similar" and 

will therefore form classes of patterns. For each of these 

classes, based on the SPS, we can evaluate statistics such 

as pattern reliability (i.e. percentage of correct 

classification) or patiern significance (Le. the probability 

that the reliability is greater than or equal to the one 

observed by chance) by comparing he predicted DV values 

with the actual ones. These statistics can then be used to 

evaluate predictions as explained in the subsequent 

paragraphs. The process of generating a SPS will be 

referred as to Development Environment Analysis @EA). 

In the text below, we assume the produced patterns have 

the following conjunctive normal form: 

predicate1 AND Predicare2 AND ... AND PredicaleN 

where, p = p@VClasSi) , Le. the a priori probability that 

the value of the dependent variable is in DVClassi. 

If the pattern reliability R is equal to 1.0, then the 

binomial equation can be simplified and the level of 

significance is simply p . If R is below one, then the 

pattern significance S can be calculated by using the 

T 

following formula: 

T-C S= &P( C+i;T;p) 1 

Since we are able to differentiate significant, reliable 

patterns from the n6n-significant and/or unreliable ones, 

we can assess the reliability of the prediction when we 

make it. A prediction based on a reliable pattern with a 

sufficient level of significance (e.g. S c 0.05) is 

441 
believable, whereas. one based on a reliable pattern with a 

poor level of significance is not. A poor reliability means 

that a pam is not robust to "noise" (Le. the dependent 

variable variations created by unknown or non-measured 

explanatory variables). A poor significance may mean 

that the pattem is a rcsult of noise or more complex 

phenomena which are kyoM the scop~ of this paper. 

4. A Process for Merging Patterns 

pattws arc useful both for predicting variables of interest 

(e.g. fault density) and providing understandable / 

interpretable models. However, interpreting the patterns 

generated by a DEA would force the user to deal with 

useless complexity. Many of these patterns are similar and 

should not be differentiated. This can prevent the user hm 

getting a cleat picture of the model trends. Therefore, the 

patms generated by the OSR process need to be grouped 

in order to make them more easily understandable and 

interpretable. This can be done using a 

(described below) where the user fixes 

the desired level of "similarity" between pattern by 

assigning values to a smaU set of parameters. 

Let us defie two pattems FTl and PT2: 

.. 

m: xj E C18ssjy AND xi E classi, 

Pr2 xj E Classjy ANDXk E Classk, 

Suppose in the context where x, E class,. the pattem 

vector set for which X, E Class, happens to show a 

strong ussociation with the one for which X, E Class,. 

This implies that these predicates capture basically the 

same phenomenon. The strength of the association can be 

assessed by using normalized Chi-squared based statistic 

such as Pearson's phi [CA88]. A Chi-squared test can be 

performed in order to assess the statistical level of 

significance of such an association. The two panems will 

be merged into one signifying that the selection of one 

predicate, or the other, during the OSR process, occurs by 

random. This is a result of slight differences between the 

two predicates and therefore distinguishing between them 

does not help to understand the object of study. This 

phenomenon is mainly due to complex interdependencies 

between Xs that are often underlying the software 

engineering data sets. 

The notion of a "slight difference" is rather subjective and 

therefore must be defined by the user. Thus, he / she 

declares either a Phi value (actually Phi * which better 

represents in this case the degree of association [CAP88]) 

or a level of significance which represents the minimal 

degree of association necessary to assume two pedi~ite~ as 

similar. This process of merging patterns based on the 

similar predicates principle yields the resulting pattern 

PT [ 1.2) which contains the composite predicate 

(Xi E Classa ORXk E Class,), implicitly meaning 

that its two component predicates are interchangeable in 

this Context. 

PT(1.2): X, E clrs~,AND (X, E Class,, OR X, E Ch) 

Automated merging of similar patterns can be performed if 

the user provides either a Phi value or a level of 

significance that would comspond to an unambiguous 

deftnition of portem similarity. 

In a similar manner, we can define a second merging 

principle. Let us suppose we have the following patterns: 

m xj E class, AND xi E Class, 

m x, E classjr AND xi E Class, 

Let us assume that Class, is a neighbor class of class, 

on the Xi range. In this particular case. if the two 

pattems characterize subsets with no statistically 

significant difference in distribution on the DV range, then 

they can be merged. This is because the variation from one 

class to the other seems to have a non-relevant effect on 

the dependent variable in the context where X, E CIpss, . Therefore, in order to assess if merging is possible, the 

probability that differences between distributions are due to 

random is calculated. For each dependent variable class, the 

proportions of pattem vectors are compared between the 

two distributions by calculating the probability that 

difference in proportion is due to random. If for all 

dependent variable classes, the resulting minimum 

probability is above a user-defined critical probability 

value, we accept the hypothesis that there is no significant 

difference between the two distributions. In the current 

tool, this is calculated through a binomial test in order to 

avoid the assumptions related to other more 

computationally effective tests (e.g. Chi-squared test of 

independence) [CAPMI. 

Both of the merging principles defined above can be used 

simultaneously in order to obtain more general patterns. 

However, the merging process using both of them must 

be carefully defined. In a tool, such mechanisms can be 

completely automated. The user would have to define 

some thresholds / criteria allowing the algorithm to declare 

two predicates similur (Le., a level of significance, Phi 

value) and/or two classes similnr (i.e., critical probability 

value). Before the merging process starts, the tool will 

calculate the matrix containing all the phi values and 

levels of significance between all predicates. Then, the 

merging process for the fmt position predicates starts: it 

is a several pass process where only two predicates can be 

merged at a time. First, predicates are merged according to 

the similar class principle. Then, the pairs of predicates 

4-42 

1"L 
showing the strongest significant associations are merged 

(similar predicate principle). During the next passes, 

predicates can be merged to composite predicates and 

composite predicates to Composite predicates. The process 

stops when no merging is possible according to the 

criteria defined by the user. Once finished, association 

 trice,^ are calculated within the contexts defined by each 

unique first position pndicate (composite or not) resulting 

from the first pass. Then, the merging process for second 

pition predicates begins within each context following 

the rules defined above. This is repeated successively on 

increasing predicate positions until a predefined (Le. by the 

user) maximum merging level is reached. Thus, the user 

defines the number of predicate positions he / she wants to 

look at and therefore set the maximum merging depth of 

the algorithm. 

5. Experiment Design 

Our goal in this article is to describe a technique to 

distinguish between low and high risk components. 

The notion of risk has multiple dimensions. We focus 

here on the identification of lowhigh fault density 

components, If we can distinguish between these two 

types of components, then we can concentrate on the high 

fault density ones during the verification and testing 

process. Moreover, if we can build this kind of model for 

each kind of fault, we can apply fault specific testing 

techniques to localize and correct faults. Basili and Selby 

showed in pS87 ] that the effectiveness of three of the 

most well known testing approaches could vary 

significantly according to the type of fault considered. 

Although more experiments are needed to beacr understand 

the issue, this study supports the idea of building different 

models for each type of fault 

The collected data set is based on fifteen FORTRAN 

Projects which were developed at the NASA Goddard Space 

Flight Center in the early eighties. On all of these project, 

static measures at the component level were collected 

using a static code analyzer. Fault report forms were filled 

out during the test phases of the development process. 

Faults were identified, classified according to a predefined 

taxonomy and localized in the system. 

Our definition of fault density is the ratio of the number of 

faults over the number of executable statements. In this 

experiment we will look, as a first step, to faults related to 

incorrect data structure readings or writings ( called "data 

value" faults in the NASA Software Engineering 

Laboratory). This type of fault represents about 50 percent 

of the total number of faults collected on the projects 

studied in this experiment 

6. Experimental Results 

6.1 Prediction Results 

4- 

We used the OSR technique to build classification models 

that wen intended to provide an answer to the question: Is 

this component likely to be in the lowest / highest 

quartilcs on the "data value" fault density range? This was 

done by pedming a DEA on the data set which contained 

399 pattern vectors. Each pattern vector was comprised of 

a list of static measures which describe a software 

component (i.e. the measurement vector), plus, the fault 

density of that component Thereby, we were able to 

calculate an average classification correctness (i.e. 

percentage of components correctly classified) of the OSR 

model . Also. we try to demonstrate through examples 

that reliable pattems can be differentiated from misleading 

"s. 

For the sake of simplicity. we will look only at the two 

first predicates (the most relevant according to the OSR 

selection mechanism) of each of the generated patterns. R, 

0 and S are respectively the Reliability, number of 

Occurrence (the number of times a pattern appeared), and 

the Significance of the pattem. The explanatory variable 

ranges were divided into quaniles. This method is the 

simplest technique for class creation but most likely the 

least effective. The class creation process is one of the 

issues that remains to be investigated (See Conclusion). 

OSR suggested that low and high fault density 

components were partly characterized by the following 

significant (e 0.05 level of significance) and nonsignificant pawns: 

Low Fault Density Components 

Assume that Fq, Sq. T4 and 4 represent respectively the 

First quartile, Second quartile and so forth, on the 

explanatory variable ranges. 

Examples of Highly-Significant Reliable Patterns: 

PT1: # stmts E Lq AND # calls E Fq, 

R = 1.0, 0 = 18, S = O.OO0 

R = 1.0, 0 = 17, S = O.OO0 

R = 1.0, 0 = 10, S = O.Oo0 

R = 1.0, 0 = 15, S = O.OO0 

R = 1.0, 0 = 8, S = 0.004 

R = 1.0, 0 = 11, S = 0.005 

R = 1.0, 0 = 24, S = O.Oo0 

m # stmts E 4m#calls€ sq, 

pT3: # stmts E Lq AND # format/sunt E Fq. 

FT4: # stmts E Lq AND # i/o stmt / stmt E Fq, 

PT5: # mts E 4 AND # assigdstmt E Fq, 

pT6: # stmts E 4 AND # decis-node/stmt E F4, 

Pl7: # stmts E 4 AND #funcr/stmt E Tq 

PT8: # decision nodes E 4 AND # calls E Fq. 

F"l9: # decision nodes E 4 AND # calls E Sq. 

PTlO # decision nodes E 4 AND # i/o stmts E Fq, 

R = 1.0, 0 = 14. S = O.OO0 

R = 1.0, 0 = 15, S = O.OO0 

43 
R = 1.0, 0 = 11. S = 0.001 

PTl1: # opemmdstmt E Fq AND# calls E Fq, 

PT12 # operators/stmt E Fq AND # format/stmt E Fq, 

PT13: # op”l/sont E Fq AND # functions E 4, 

Examples of Non-Significant Reliable Pattenrs 

PT14: # stmts E Tq AND # format/stmt Fq, 

R = 1.0, 0 = 9, S = 0.002 

R = 1.0, 0 = 6, S = 0.016 

R = 1.0, 0 = 8, S = 0.004 

R = 1.0, 0 = 2, S = 0.25 

R = 1.0, 0 = 2, S = 0.25 

R = 1.0, 0 = 2, S = 0.25 

R = 1.0, 0 = 4, S = 0.0625 

PT15: # ~tmts E Tq AND # i/o sImt/stmt E Fq, 

PT16: # stmts E Tq AND # i/o stmts E Fq, 

PT17: # stmts E Tq AND # i/o stmts E Sq, 

PT18: # ope”/stmt E Fq AND # funct/stmt E 4, 

R = 1.0, 0 = 4, S = 0.0625 

Example of a Non-Significant Non-Reliable Pattan 

PT19 # stmu E Tq AND # functions E Tq, 

R-0.0, O=l.S=l.m 

High Fault Density Components 

Examples of Significant Reliable Panems 

PT1: # lines E Fq AND # comment/stmt E Tq, 

R = 1.0, 0 = 11. s = 0.001 

PT2: # stmts E Fq AND # comment/stmt E Tq. 

R = 0.94, 0 = 17. S = 0.OOO 

PT3: # format/stmt E Lq AND # commcnthtmt E Tq, 

R = 1.0, 0 = 10, S = 0.001 

pT4: # decisions nodes E Fq AND # Wstmt E Lq, 

R = 0.95. 0 = 21. S = O.OO0 

pT5:Usunts~FqANDd CallSESq. 

R = 0.94, 0 = 18, S = 0.OOO 

PT6 # stmts E Fq AND # i/o stmt/stmt E Sq, 

R = 1.00, 0 = 13, S = 0.OOO 

PT7: # stmts E Fq AND # operandhine E Sq, 

R= 1.00, 0=20,S=O.OOO 

PT8: # stmts E FqAND# operand/stmt E Sq, 

R = 1.00, 0 = 18, S = O.OO0 

PT9: # sunrs E Fq AND # i/o variablelline E Fq. 

R = 1.00, 0 = 27, S = 0.OOO 

PTlO: # stmts E Fq AND # operators E Sq, 

R = 0.91, 0 = 11. S = 0.006 

PTll: # operator/sunt E 4 AND # assign/stmt E 4, 

R = 1.0, 0 = 6, S = 0.015 

As shown in the above results, significant reliable patms 

can be recognized and diffenntiaied from the non-reliable / 

non-significant ones. Therefore, significant reliable 

pat” can be identified and used with confidence for both 

prediction and interpretation. For instance, if we take 

pattem PT1 for low density components. we observe a 

retiability of 100% based an 18 occumnces. This produces 

a vay good significance. lhe pedictions generated 

by this pttun can therefore be considered vuy reliable and 

used with confidence. Both the OSR patterns and the 

logistic regnSSion model yield an average classification 

correctness of 82%. This result is very encouraging 

considering that the class creation process used (i.e. 

dividing the range in quaniks) was primitive and that the 

explanatory variables available are all continuous (which 

is an important advantage for the logistic regression 

model). Moreover, note that the OSR process is entirely 

automated. 

The patterns produced by OSR are not always easy to 

model) requires expert knowledge. However, in the next 

subsections, we provide some rules for reading and 

interpreting the above pauerns. Some pattem merging 

IWUltSiUCalSOproVided 

invrprct Intapretation of patterns (or any other stochastic 

6.2 Pattern Interpretation Rules 

Interprccation of patterns is much easier than interpreting 

regmsion coefficients. First, OSR takes into account the 

fact that an explanatory variable can have a strong impact 

in a CMtain context (defined by the predicates in preceding 

positions) and not be relevant in another one. Second, if 

strong associations exist in a given context, then the 

pattern merging process makes it apparent by creating 

composite predicates (see examples in section 6.3). The 

variation of reliability generated by a particular predicate 

can help assess the significance of the impact of an 

explanatory variable (on the dependent variable) when the 

explanatory variable belongs to a certain class of values 

within a certain context. Let us take the following pattern 

as an example: #sun& E Lq AND #calls E Fq which 

yields a reliability of 10096. However, #stmts E Lq 

alone only yields a reliability of 88%. 

This result suggests that #calls E Fq is a relevant 

predicate in the context where #sun& E Lq because it 

shows a significant impact on the fault density. 

However, a pattern must always be interpreted in context 

In some contexts (e.g. #stmts E Fq), a variable (e.g. 

#operators) may not rake on the full range of values. The 

interpretation of patterns like pattern PTlO for high 

density components must be done carefully: #operators E 

Sq may be interpreted as a “rather large” number of 

operators because in the context #sun& E Fq, very few 

components show either #operatam E Tq or #operators E 

4 (i.e. # stmts is smngly associated with # operadrs). 

Therefore. the OSR process did not select patterns like 

#sunts E Fq AND #operators E Tq since they yielded 

subsets that met the termination criteria. This example 

shows that even though interpreting patterns is always 
simple, it requires the support of a tool . 

6.3 Pattern Merging Results and Interpretation 

of Recognized Patterns 

In this section. we intend to show how the merging 

process can help to group similsr raw patterns into 

composite patterns and therefore provide more easily 

inteqmtable information. If we simplify the raw pauems 

generated by OSR using the merging criteria: Phi = 0.40 

and critical probability value of 0.0005, we get a set of 

composite patterns for each of the dependent variable 

classes. In order to illustrate the point, we first show some 

of the intermediate steps of the merging process. Then we 

give two composite patterns: CP1 and CP2 (formed by the 

merging process), which characterize low fault density 

components. 

For example, low density component pattems PTl and 

Pn can be merged based on the similar classes principle. 

They both show the same fit predicate: # sunts E Lq. 

Their second position predicate shows the same variable # 

calls and two neighboring classes (Fq and Sq). Since they 

do not show a statistically significant difference is 

dishbution (critical probability value = O.OOOS), then they 

can be merged in: #stmts E Lq AND # calls <MEDIAN. 

Similarly, low density component patterns PT3 and PT4 

can be merged based on the similar predicate principle, 

They both show the same first position predicate and their 

second position predicates are strongly associated (Phi2 = 0.57). Therefore, they can be merged in: #stmts E 4 

AND (#formats/stmt E Fq OR #YO stmts/stmt E Fa. 

This merging process is repeated until no more merging is 

possible according to the user's criteria. CPl and CP2 are 

the final resulting composite pattems which characterize 

low fault density components: 

CP1: SIZE-HIGH AND CALLS & VO_LoW, 

R = 99% , 0 = 169. S = O.Oo0 

CP2: SIZE-HIGH AND FUNCT-HIGH, 

R = 865.0 = 43, S = O.OO0 

where the composite predicate SEE-HIGH is defined as: 

I statemenu E Fq OR I s~ancnu E Sg 

OR t fanuts E 4 OR t decision nodes E 4 

OR I ~~~IOIS / smt~ Fq 

and, in the context where SIZE-HIGH is true, the 

following mposite predicates are formed: 

I frnaimr E Tq OR t funclia~ E I4 

OR funaionrlstrnt E Tq OR funaiaulsmt E Lq 

FUNff-HIGH e 

CPl and CP2 actually define classes of raw patterns that 

are assessed equivalent according to the user-defined 

criteria. Some of the low density patterns presented in 

section 6.1 belong to CP1: PTl. PT2, PT3, PT4, PT8, 

PT9, PTlO, PT11, pT12. PT14, PT15, PT16. PT17 and 

others to CP2: PT7, PT13, PT18. PT19. Both of the 

composite pauems suggest that large components are 

likely to have low fault densities. This agrees with a study 

conducted by Basili and Penicone [BP84]. This may be 

partially explained by the fact that low operator densities 

seem to be strongly associated with large components. 

CPl suggests that a low number of function calls or a low 

number of UO statements increase the probability of 

having a low fault density. cp2 indicates that a large 

component showing a high density of functions is likely 

to show a low fault density. 

Merging patterns is alwavs desirable. It allows us to 

combine related. rare, isolated patterns to more significant 

pattems and thereby group together trends which capture 

essentially the same phenomenon. This makes the 

generated composite pattems easier to interpret and gives 

the usa a more abstract and general view of the results. 

Also, as we have seen, patterns with a small number of 

occurrences cannot be frusted (even though they show 

good reliabilities) because of their weak level of 

significance. However, if these patterns are shown to be 

strongly associated with other reliable patterns, then the 

significance of the generated composite pattem increases. 

This allows us to gain more trust in rare reliable patterns 

based on the calculated composite pattem's level of 

significance. However, this should be used very carefully 

and needs further investigation. 

7. Conclusion 

Based on the above experimental results. building useful 

models for assessing the fault density of software 

components. based upon early available simple metrics in 

the presence of noisy data appears possible. Whenever 

OSR generates a very reliable and significant pattem, the 

prediction can be used with confidence. To the contrary, if 

the pattern is not a reliable and significant one, an 

alternative modeling method such as logistic regression 

may give a more believable prediction. We have Sfen that 

problems such as partial information in the data set can be 

accommodated for by assigning a relative goodness to each 

prediction. Also, the patterns appear to be easier to 

interpret than regression coefficients and correlation 
mauices which are the usual outputs of regression 

analysis. "lis is due mainly to the fact that OSR produces 

symbolic / logical expressions whtn the notion of context 

is introduced by considering the order of the predicates. 

Also, the merging process helps the user look at the 

model at various kvel of abslraction. From a mare general 

perspective, based on previous [BBDl, BP92) and cmnt 

experimental results, OSR is a data analysis framework 

that successfully integrates statistical and machine lcamhg 

approaches in empirical modeling with respect to specific 

software engineering needs. However, while the 

experimental results thus far have been encouraging, many 

aspects of the processes involved in OSR are still to be 

optimized. Such processes include, by order of importance. 

EV class definition. the refinement and automation of the 

merging process, support for pattern interpretation, the 

atmbute selection process and the selection of termination 

critaia 

8 Acknowledgments 

We thank Gianluigi Caldiaa, Denis Okkampf, William 

Thomas and especially Sandro Morasca for their excellent 

suggestions. We also thank the referees for their insightful 

comments. 

References 

[AG90] Alan Agesti. "Categorical Data Analysis", wileyintascience. 1990 

[BP84] V. Basili and B.T. Paricone, "Software Errors and 

Complexity: An Empirical Investigation," Communications 

of the ACM. vol. 27. no. 1. January 1984. 

[BR88] V. Basili and H. Rombach. "The TAME Project: 

Towards lmprovement-oriented Software Environments". 

IEEE Trans. Software Engineering 14 (6). 

[BS87] V. Basili and R. Selby. " Comparing the Effectiveness 

of Software Testing Strategies", IEEE Trans. on Software 

Engineering 13 (12). 

[BR84] 

Trees". Wadworth & Brook, 1984. 

[BBIPl] L. Briand. V. Basili and W. Thomas. "A Pattem 

Recognition Approach to Software Engineering Data 

Analysis", IEEE trans. Software Eng.. Special issue on 

software measurement principles, techniques and 

environments, November 1992. 

[BF92] L. Briand and A. Porta, "An Alternative Modeling 

Approach for Predicting Error Profiles in Ada Systems", 

European conference on quantitative evaluation of software 

and systems (EUROMETRICS92). Brussels, Belgium, April 

1992. 

(CA881 J. Capon, "Statistics for the Social Sciences", 

Wadworth publishing company, 1988. 

L. Brei" et al. "Classification and Regression 

[CE87] J. Cendrowska, "PRISM: An Algorithm for Inducing 

Modular Rules". Journal of Man-Machine Studies. 27. 

pp .349 

[DGM] W. Dillon ad M. Goldstein. "Multivariate Analysis", 

John Wiley & sons. 1984. 

[HL89] D. Hosma and S. Lemeshow, "Applied Logistic 

Regression", John Wiley & sons. 1989 

[M83] R. Michrlski, 'Theory and Methodology of Inductive 

Laming." In R. Michalski. J. Carbonell & T. Mitchell 

(Eds.), Machine learning (Vol. 1). Los Altos, CA: Morgan 

Kauhann. 

[M89] J. Mingers, "An Empirical Comparison of Selection 

Measures for Decision-tree Induction". Machine learning 3. 

pp.319, 1989. 

[Q79] J. Quinlm. "Discovering Rules by Induction from Large 

 collection^ of Examples". In D. Michie (Ed.), Expert System 

in the microelectronic age. Edinburg University Press, 1979. 

[Q86] J. Quinlm. "Induction of Decision Trees". Machine 

learning 1, Number 1. pp.81, 1986. 

[SP88] R. Selby and A. Porter. "Learning from Examples: 

Generation and Evaluation of Decision trees for Software 

Resource hdysis.., IEEE trans. Software Eng., 1988. 

4-46 
Appendix: Definition of the Generalization 

Algorithm (notation consistent with section 

2.2) 

This generalization process can be formalized using the 

following definitions and algorithms: 

Definition Al: We define a composite predicate (cp) as 

cp = Up, p E PD, which the set of all predicates. 

Composite predicates can be combined to form other 

composite predicates. 'zhus, we define cp,,, = cpi u cpj. 

Pss Definition A2 An association c0efZcien.t a, is an 

assigned statistical degree of association between Cpi and 

cpj where PSS is the data set used to determine this 

association. Let us assume the two following data subsets: 

PSS, = {pv E PSS(cp,istrue) 

PSS, = {pv E PSq cpj is true) 

A two row-two column contingency table is defined, 

where the subsets characterizing each row and column a~ 

respectively PSSi, PVS - PSSi, PSSj, PVS - PSSj. 

Based on this table, a Chi-square based statistic (Le. 

Pearson's Phi) defining the degree of association between 

the two subsets is calculated and assigned to ar. 

Definition A3: A context is a conjunction of a set of 

composite predicates that defines PSS E PVS. This 

defines the data subset on which an association coefficient 

is calculated and therefore its domain of validity. 

Definition A4: An ussociation matrix A& is a square 

matrix of association coefficients calculated in a context 

C. where the rows / columns represent all possible 

predicates 

PSS examde: A:: contains all 

where V PV E PSS, cpt A cp, is me. 

Definition A5: TWO composite ~edicatcs Cpi and Cpj are 

said to be associated in the context of C if ay 2 some 

minimal level of association.This will be denoted as 

cPI e cPJ' 

Definition A6: A predicate tree is a tree repmentation of 

the pattems generated during the Development 

Environment Analysis (i.e. DEA) process. As mentioned 

is Section 3.2, DEA produces a set of patterns 

representing the observed trends in the historical data set. 

It is expected that a significant number of these pattems 

will be duplicated or similar. This representation is a 

compact way of representing the specific pattern set 

(SPS). Each path of a predicate uee represent a pattem 

genemted by DEA. (see Figure 3) 

~ 

PATTERN SET 

Figure 3: Example Predicate Tree 

Notice that the mot of the predicate tree is a "dummy" 

predicate which can be thought of as the identity predicate 

cp~ (Le. cpi A cp~ e Cpi ). Note that in the above 

example, all of the predicates are singleton. This 

represents a predicate tree before any generalization. 

Branches will be merged and composite predicates created 

at the nodes during the generalization process. 

Definition AI: The maximum merging depth (user 

defined) is the depth in the predicate tree to which 

generalization is to be performed. It defines the 

observation depth of the patterns by the user. 

Definition A8: Two composite predicates Cpi. Cpj are 

said to be "mergeable neighboring composite predicates" if 

the following conditions are fulfilled: 

(1) There exist two predicates px: Xi E ClaSSik. py: 

Xi E Classit such that px and py are one of the 

disjunctive mcats of cpi and qj, respeztively. 

(2) ChSSik ad Classit are neighboring ClasSes on 

variable Xi range. 

(3) Cpi and Cpj ykld the same classification, show a 

difference of rellability below DR and a maximum 

e47 

"788L c' '-2 
pauem level of significance S (it. DR and S are fixed 

by the user). 

If these three conditions are true. then mncp(qi, qj, s, , 

DR) is me. 

In ordet to define the genaalization algorithm based on the 

above definitions, we assume that it starts with the 

procedure call: GeneraliZe@redicate tree, roof cp~, 0, PHI, 

DR) 

We can now define the Gentralize algorithm as follows: 

Generalize (predicate tree, node, context, 

current depth, PHI. DF) 

(1) If the node is a “inaI node of the pedicate tree 

OR if depth > maximum merging depth then 

lz” 

(4) while 3 vi, cpj such that cpi = cpj do 

EQItn 

. select cpi and Cpj such as ai,j 

association in A= 

. merge(predicate tree, node. Q, Cpj) 

. recalculate AXm-, , the association matrix for 

cpi. ..., cpi-1, CPi+l. --.. CPj-1. CPj+l, ..., CPm. 

Cpiuj in context 

is the strongest 

(5) for each successor of node in predicate tree 

Generalizc@redicatctree,successor,context”~, 

depth+l, PHI, DF) 

dIn step (4). a call is made to procedure merge defined 

merge as follows: 

IZUX&E merge @redicate tree, node, qi. qj) 

qi and qj a~. SUCC~SSO~S Of node 

(1) Combine Cpi and Cpj to form a single node 

CPiUj 

(2) Combine all like Sub- rooted at qiuj 

&merge 
A Classification Procedure for the Effective Management of Changes c 

Lionel C. Briand and Victor R. Basili ' 

Computer. Science Department and Institute for Advanced Computer Studies 

College Park, MD, 20742 

+- / during the Maintenance Process --- / 

/" 5 / 2 

University of Maryland '>4 / ,>- Y92- py i$s 

- 

To be published in the proceedings of the IEEE Conference on software 

maintenance, Orlando, Florida, USA, November 1992. 

Abstract 

During sofhvare operation, maintainers are often faced with nwnerous change requests. Given available resources such as @art and calendar time, changes, if approved, have to 

be planned to fit within budget and schedule consnaints. In this paper, we address the 

issue of assessing the difficulty of a change hed on known or predictable data. This 

paper slzould be considered as a first step towards the construction of customized 

economic models for maintainers. In it, we propose a modeling approach, based on 

regular statistical techniques, that can be used in a variety of soware maintenance 

environments. This approach can be easily ructomated, and is sirrlple for people with 

iimited Statistical experience to Me. Moreover, is deals effectively with the uncertainry 

usually associated with both model inputs and ourputs. Tht modeling approach is 

validated on a data set provided by the NASA Goddard Space Flight Center which shows 

it hus been effective in classifLing changes with respect to the effort involved in 

implementine them. Other ndvwges of the qproach are discussed along with addirionul 

steps to improve the results. 

Key words: mainteMnceprocess, change &jjkulv, change request management. 

' Research this study was supported in part by NASA grant NSG 5123 and the Wuo Corporation (XAP Member) 

449 

1ooo5766L 
’ 1 Introduction 

Given the limited r~sources (Le. effort and calendar time) available to the maintenance 

activity within software organizations and the number of change requests proposed, 

difficult decisions need to be made These decisions include: which changes to implement, 

how much optional functionality to provide in enhancements. A large amount of total 

software effort is spent on maintenance [LS80, GRA871. Changes in the form of 

corrections, enhancements or adaptations effect the software source code and/or the 

documentation. Some of these changes arc crucial, others arc less important. Therefore, 

when one considers the global cost and variety of ”Ee activities, management of 

changes becomes an important and complex task It requires the support of models so we 

may perform systematic comparison of the Costs and benefits of changes before 

implementing them fRW921. One approach is to build such models based upon past 

proj ec t experiences. 

To this end, effort models have to be designed to predict resource usage and optimize the 

cost-effectiveness of the maintenance process. Well defined modeling procedures need to 

be established so they can be repeated and refined, allowing the model to evolve 

consistently as new data arc collected 

This paper describes a modeling procedure for constructing a predictive effort model for 

changes during the maintenance phase. This technique is intended to handle small data sets 

and the mc#rainty (Le. for cost or technical reasons) usually associated with model inputs 

and outputs (Le. is this particular predication believable?). We assess the feasibility of 

building such a model using a data set that describes several projects in the SEL 

environment at the NASA Goddard Space Flight Center. Based upon the results of the 

analysis, we also make recommendations for improving the data collection process. 

2 Context of Study and Experiment Design 

In this study, we use a data set consisting of 163 changes collected on four different 

maintenance projects. Each change is represented by a vector consisting of a variety of 

metrics associated with the change. The four projects arc referred to in the paper as projects 

pl, p2, p3, p4. These projects arc from the same application domain: satellite ground 

support softsvare written in FORTRAN. 

The change process in the SEL environment has two main phases: an “understanding” 

phase whert the change is determined and isolated in the system and an “implementation” 

phase, where the change is designed, implemented and tested. 

The effort associated with both the understanding and implementation phases is collected 

on discrete scales (i.e. ordinal) in order to facilitate the data collection from a maintainer’s 

perspective. The effort range is divided into five intcrvals: below one hour, between one 

hour and one day, between one day and one week, between one week and one month, 

above one month. For each change performed, the appropriate understanding effort and 

implementation effort intervals are recorded by the person making the change. These effort 

intervals are indexed from 1 to 5 and will be referred to as dificulty indices in the paper. 

All the change-related data used in this paper was collected on a standard fonn (see 

Appendix). The memcs collected range from measures on a continuous scales (e.& 

number of components added, number of lines of code added) to categorical measures 

(e.g., source of the change, technical description of the change). Some of these memcs are 

predictable before starting the design of the change, others can only be assessed after the 

implementation of the change has begun. 

In this paper, we focus exclusively on the effort spent to implement (i.e design, code, test) 

a change. There are two reasons for this: I) Almost no information is available to the 

4-50 

looo6788L 
maintainer before the understanding phase. Therefore, no prediction model can be built. 2) 

In this environment, the effort expended in the understanding phase is generally somewhat 

smaller than the effort expended during the implementation. It is thus more essential to use 

a @ctive model for the implementation phase. 

The available memcs are defined as follows: 

Type of modifcation (cdon, enhancement, adaptation). 

origin of the mor in the software He cycle. This is nfd toas source in the text 

( requirements, sptcifications, design, code, prcVious change). 

Software products effected by the change (code only, code and design). This is rcfed 

to as objects in the text 

Number of components added, changed, deleted. They arc rcfd to as comp.add, compxh., comp.del., nspccrively. 

Number of lines of code added, changed, deleted. They are referred to as Zoc. add., 

LOC. ch., LOC. del., respectively. 

Change technical description (initialization, logic/conuol structure, user interface, 

module interface, data structures, computational). This metric is nfd to as ch.desc. 

During the understanu'ingphe, estimates can be made of the first three memcs. The 

number of components involved in a change can also be approximated since the change is 

isolated in the system architecture. But any prdction in tums of Iines of code to be added, 

deleted or changed is still complex at this point and can only be predicted at a coarse level 

of precision. 

3 The Modeling Approach 

Considering the discrete nature of the effort data reported during maintenance, the 

prediction issue becomes a classification issue, ix. in which effort class will the change 

probably lie? The maintainer can only predict values for most input memcs with a cdn 

degree of uncertainty. It is important that the modeling process takes this constraint into 

account. This help to make the genmted model easy to use. Also, our data set is small and 

contains discrete explanatory variables. Thenfore, we need a modeling approach which is 

both effective on small samples and which handles discrete and continuous explanatory 

variables in a consistent way. 

3.1 The Modeling Process Steps 

A high level view of the model consuuction process can be defined as follows: 

1- IdenrifL Predictable Menics. Identify the memcs, among those available, that are 

predictable before the implementation phase. For ratio and interval memcs that are 

predictable early but only with a certain degree of uncertainty, the range is recoded as an 

ordinal range with a set of ordered classes. These classes reflect a reasonable level of 

prediction granularity. For example, a ratio level memc range like "number of 

components added" could be divided into three intervals forming the three memc classes 

low, average, and high. 

2- Idenrifr Signijicant Predictable Menics. Identify a subset of the predicable memcs that 

appear to be' good predictors of the difficulty index, using a consistent evaluation 

technique for all candidates. 

4-51 
3- Generate a Classflcation Funcnbn. Associate the resulting metrics in a classification 

function which has the following form: 

PredictcdJXfficulty = Classification-Function (Sigmficant~Prcdictable-Memcs) 

whm Predicted-Difficulty = some classification scheme based on the difficulty indices, 

e.g., {easy, difficult) and Significant_Predictable-Memcs = (some of the predictable 

memcs collected on the Maintenance Change Report Form which appear as good 

4- Validate the Model. Conduct an experiment on a representative (Le. in terms of size 

and quality) set of data. Two mtasurcs that can be used to validate the model art: Average 

Classificarion Comctness (i.e. ratio of number of corrtct classification / total number of 

paformed classifications), and Indecision Rate (ie. ratio of number of undecidable cases 

/ total number of changes to be classified). The latter reflects the nced for such a model to 

deal with output uncertainty, therefore warning the user whenever a certain level of 

codidence is not reached far a specltic prediction. 

@-) 

3.2 

This section presents a possible implementation of the previously described process. Our 

goal in defining such a procedure can be described by the following points: 

An Implementation of the Modeling Process 

We want the generated model to be as simple to use as possible. 

The uncgtainty associated with the model inputs at the timc of prediction must be taken 

into account by the model, Le., intervals rather than values should be used as model 

The model should be able to provide some estimated risk of error associated with each 

classification. Thus, the user would be able to select a minimal level of confidence (i.e. 

maximum risk) that would differentiate the model classifications as believable or nonbelievable. 

inputs. 

. The steps of the procedure arc: 

1- Idennfi Predictuble Mezrics. The input is a set of available memcs. The output is a set of 

memcs whose values arc either known or prtdrctable, with a certain degree of accuracy, 

before the change implementarion phase. 

There are several processes for selecting the set of prdctable memcs. The determination 

of predictability can be either based on interviews with people with a good knowledge of 

the maintenance process (and then refined with experience) or observed through controlled 

experiments [BSP83, BW84J. Both help to determine the average estimation accuracy that 

can be reasonably expected for a given memcs. 

The range of each continuous / ordinal medictable memc is divided into intervals (e.n., 

percendes, natural clusters pIL84J). Thk more accurately predictable the metric, the more 

numerous and nmow the intervals can be. We recode the memc ranges according to their 

respective predictability so the maintainer can easily select the right hemal and use some 

of the predictive power of memcs not measurable before the implemenzation phase. These 

intervals are called mem'c classes in the paper. 

Our need to define these metric classes for predictable memcs stems from the impossibility 

of relying exclusively on measurable (at the time of prediction) memcs, e.g. building an 

accunte model for predicting change effort is likely to require measures of change size that 

are not available before the implementation phase. We have no choice other than taking into 

consideration memcs that cannot be measured but only approximated with a certain degree 

4-52 

loo05788L 
of prwkion-by the maintainer after the undcrstondingpke of the change process. 

2- Idenrifi Signifscant Predictable Menics. The input to the second step is the set of 

predictable metrics from the first step and the outputs are a subset of significant predictors 

and their conesponding association table. This association table distributes the difficulty 

&iices across the metric classes defined on each predictor value domain. 

Consider as an example Table 1 which shows the association table of the metric umber of 

lines of code added across the four difficulty classes (class 5 has so few changes that we 

merge it to class 4). This table is calculated based on the actual dismbutions in the data set 

considered for modeling. Each column represents a mcmc class (e.g. > 30 implies that the 

number of loc added is more than 30) and each row an index of difficulty. With respect to 

each predictable memc and using its calculated distribution of difficulty indices, an average 

difficulty index (i-e ADI) is calculated for each metric class (shown in the bottom row of 

Table 1). The calculation of a meaningful and statistically significant AD1 requires us to set 

up the memc classes in a way that guarantees a minimum number of changes in each of 

them. 

4 

. I2 I 48% I 36% I 9.5% 1 

1 I 

3% 4% 50% 

I I I I I 

I3 I 42% 60% I 40.5% I 

1 I I AD1 1 2-40 I 2.68 I 3.40 I 

Table 1: "number of lines of code added" distribution 

Taking the association table Table I as an example, the calculated index averages look 

consistent with what was expected. The AD1 seems to increase substantially with the 

number of lines of code added. In general, with respect to the ratio and intend level 

memcs whose the value domains have been rccoded in successive memc classes (see step 

l), significant differences should exist between class ADIs. Based on a F-test, a one-way 

analysis of variance (ANOVA) can be performed and the statistical level of significance of 

the metric class AD1 differences may be estimated [CAPSS). Whenever the 0.05 level of 

significance is not reached, the boundaries should be recoded in a way that minimizes the 

level of significance. Since all the continuous memc ranges have been recoded into an 

ordinal scale, we have to calculate the degree of association between the difficulty indices 

and the memc classes in order to assess the predictive power of each memc. One approach 

consists of computing the Chi-square statistic (which is valid at the nominal level 

[CAP88]) for each memc association table. A statistical level of significance characterking 

the association between the difficulty indices and the metric classes is calculated based on 

the generated Chi-square value. Thus, the top ranked metrics showing sufficient degree of 

association are selected as parameters potentially usable to build a multivariate prediction 

model. Some more sophisticated measures of association (Le. PRE-measures of 

associations [CAP88]) can provide more intuitiodinformation about the associations and. 

therefore allow an easier selection. However, this issue is beyond the scope of this paper. 

3- Generate a Clarsification Function. The input to the third step is the set of association 

tables of significant predictable memcs and the output is a classification model that predicts 

an expected difficulty index associated with changes. Note that although five difficulty 

indices are defined on the change form, a small minority of the changes (5%) actually lie in 

the extreme intervals (i.e. intervals 1,5). 

4-53 

loo05788L 
This makes classification into these intervals extremely difficult. Also, since 80% of the 

chances belong to classes 2 and 3, we will first build a classification model intended to 

differentiate these two classes: less than one day (Le. referred as easy), more than one day 

(i.e. refemd as difficult). In section 4.2, we will refine our classification by dealing with a 

“more than one week” class (Le. indices 4 and 5). Thus, based on the generated 

classifications the user wi be able to make decisions with respect to the requested 

available res0m.c~ and the cxpecttd gains. 

The process of building a classification function is composed of two steps: 

1- Paform a regression: Based on all the available association tables and the corresponding ADIs for each change in the. dam set, we perfom a stepwise linear regression [DL841 of 

the following form: 

Actual-difficulty-index = W1* ADI-memcl+ ... + WN * ADI-metricN 

implementations of changes. a, ‘s is done by comparing the predicted difficulty to both the 

Due to interdependencies between memcs, only a subset of the pnseltcted metrics remains 

in the generated prediction function (Le. only the one showing, based on a F-partial test, a 

level of sig” below 0.05). In order to make the model easier and less costly to use, 

the number of parameters in the regression equation can be minimized. In this case, one or 

several parameters are nmvd (especially when they show a statistical significance close 

to 0.05) and the resulting models arc evaluated. Then, the user has to assess the loss of 

correlation against the ease of use gained by removing parameters from the model. If the 

tradeoff appears reasonable, then the new model is adopted. Weights arc calculated for each 

remaining parameters and the resulting optimized linear function allows us to calculate an 

difficulty index expected value. This may be used to classify the change based on the 

realistic assumption that: the closer the expected value of the di&culty index to an actual 

Miculty index, the more likely the corresponding change belongs to the matching effort 

class. Therefore the following interval-based decision rule is used to make classifications. 

2- Define a decision rule for classification: the predicted difficulty index range is divided 

into three intervals (i.e. easy change predicted, undecidable, difficult change predicted) in a 

way that guarantees a maximal average classification correctness. For example, the 

boundaries for classifying a change as either less or more than one work day can be 

defined as in Figure 1. The classification of future changes will be performed according to 

the intend in which their calculated difficulty index will lie. 

I EASY Undecidable DIFFICULT 

t I I 1 

1 2.4 2.6 5 

I I 

Predicted Dificulty Index Range 

Figure 1 : Example of decision intervals 

4-54 
The process for creating these decision boundaries can be described as follows: 

1- The user defines a risk I loss function having the following form: 

Expected-loss = Wtightl*MRl+ Weight2*MR2 

where MRn is the "isclassification rate &.lculatcd for changes actually in class n. 

The loss function weights can be defined according to the respective costs of 

misclassification associated with each class. Most of the time, this weight will be set to 

one. A search algorithm can then be used to determine the interval between two 

neighboring changes on the predicted index range that provides the best decision 

boundaries ( i.e. that minimizes the risk / loss function). These two neighboring changes 

form the boundaries of the smallest possible undecidable interval on the range. 

2- In a stepwise manner, this interval can be widened on both sides of the index range 

according to some automatable process. For instance, the interval can be expanded in a 

stepwise manner, including one more change at a time on each side of the interval, until a 

maximal expected loss value (Le. pndefmed by the user) is reached. Based on this 

process, the user will be able to determine the boundaries of the decidable intervals 

corresponding to the desired level of risk 

4 A Validation Study 

According to the procedure defined above and based upon the previously described four 

project data set, the significance of each available metric as a predictor is assessed. Table 2 

shows the Chi-square-based levels of Significance. Then, in order to build the needed 

classification models, the metics yielding a good level of sigmficance arc selected. First, 

we build a general model usable for any project in the same environment. This model is 

intended to be useful at the start of a maintenance process when not enough data are 

available to mate a project specific model. 

Then, we build an independent classification model for each project which is expected to be 

more accurate with respect to future changes for each specific system, respectively. The 

various results will be compared in order to assess the validity of cross-project models in 

this environment. The ranges of the continuous memcs were recoded according to the 

previously described prucedure. Two or three memc classes were defined for each of the 

metrics. according to the urtdictabilitv level of the memc and the distribution of the 

changes on their &ective*mge. In 0th words, the intcrval boiindaries were chosen in a 

way that reflected their predicability, optimized the classification power of the metric (i.e. 

ophized the chi-squak) and guaranteed, to the extent possibl& a sufficient number of 

changes within each metric class. 

4-55 
Type I 0.W 

m Requirements. specifications, Source O.oo00 &sign, code, prtvious change 

Chdesc initialization, logic, interface, O.oo00 data smrcrurt, compuultional 

I Correction, enhancemau, I adamaon 1. 

Comp.add. 

Comp. ch. 

Objects 

Locadd. I O.oo00 

. O.OOO2 o,>o 

O.oo00 Q, [Z 51 , >5 

0.0005 code only, code and design 

I c10, [lo, 301 , >30 I 

-Del. I o.ooo6 I 

Table 2: Level of significance and class boundaries / categories of metrics 

4.1 A General Model 

This model is intended to be specific to the NASA SEL environment It has been built 

based on systems belonging to the same application domain and therefore may not 

represent necessarily other domains accurately. Table 3 shows for each selected metric, the 

class ADIs and the corresponding result of the one way analysis of variance [CAP881 that 

assessed the statistical significance of the AD1 variations across memc classes. They all 

appear below 0.05 and we can therefore say that the meaic classes with respect to 

continuous memcs have been adequately defined because they show significant AD1 

differences. 

Table 4 shows two distinct regression-based classification functions (PI stands for 

Predicted difficulty Index). Note that the parameters of the regression equations are the 

memc association table-based ADIs and not the metric values themselves. For the sake of 

simplification, the names of the memcs areshown in the equations. For each function, the 

calculated regression equations are given with the respective level of signXicance of each 

memc (i.e.shown between brackets above the equations and based on partial F-tests). 

If the memc does not appear significant at a 0.05 level, then they are excluded of the 

equation. The global coefficient of determination R2 is also given. The first one was 

obtained by performing a stepwise regression using the class ADIs of the significant 

predictable memcs. Only one of the lines of code (i.e. loc) based memcs was retained in 

the equation: luc.ch. Then, in an attempt to avoid the use of this memc (i.e. which is still 

the most difficult to assess despite the coarse defined memc classes), we recalculated the 

equation parameters when ignoring it. The coefficient of correlation did not appear much 

affected by the change. This can be explained by the higher significance of the remaining 

parameters and their stronger calculared coefficients that show a strong interdependence 

with 1oc.ch. In other words, they partially compensated the loss of explanatory power due 

to the removal of 1oc.ch.. Thus; the generated model becomes even easier to use and does 

not loose much of its accuracy (seo, Table 5). 

4-56 
Table 3: Metric class ADIs 

I Description of the Models R-XI 

0.50 

Model 1 (0.0) (0.0) (0.0) (0.~) (0.04) (0.a) 

PI - - 4.22 + 0.59 Swrce + 0.62 Ch.desc + 0.58 locch + 0.38 Comp.rdd + 0.36 Comp.ch 

(0.0) (0.0) (0.01) (0.0) 

+ 0.68 Swrce + 0.69 Chdesc + 0.49 Comp.add + 0.56 Cunp.ch 

I 0.46 I 

Table 4: General models 

Table 5 shows the clarsificarion correms (Le. rate of comct classification ) obtained 

when using the above models (Table 4). The decision boundaries have been optimized to 

yield the best results. First, they ha;re been selected to yield a 0% indecision rate (column 

IR = 0% in Table 5). Then the undecidable interval has been widened in order to 

demonstrate the possibility of selecting decision intervals that fit the user’s need in terms of 

classification comcmess (column IR > 0% in Table 5). In this case, the selected interval 

boundaries are arbitmy and are shown for the sake of example. The row “classification” 

indicates the classification performed (i.e. easy changes = [l-21 or [l-31). Each cell 

contains, for all models, the undecidable interval boundaries between brackets and the 

corresponding classification correctness. Whenever the undecidable interval has been 

widened (Le. IR > O%), the corresponding indecision rate is given. 

Despite the mediocre coefficient of determination, a pdcularly good correctness C~ES ken 

obtained when the interval [ 1-31 represents easy changes. However, the results appear 

much less satisfactory for the other classification performed. Nonetheless, this can be 

substantially improved by widening the undecidable interval. Thus, the model appears 

usable for at least a subset of .the changes. However, when possible (Le. enough project 

data are available), project specific models should be used as demonstrated in the next 
4.2 Project Specific Models 

Table 6 shows optimal equations resulting from stepwise regressions performed 

independently for each of the four projects. The format used is qe same as in Table 5. 

Differences between models are observable with respect to the variables selected. This does 

not necessarily mcan a real variation in the impact of the explanatory variables across 

projects. It may be due to a lack of variation of a variable within a project specific data set 

Description of the Models 

Model Pl (0.03) (0.0023) (o.-) 

PI - - 1.56 + 0.71 Source + 0.80 Comp.ch 

R-sq 

0.68 

(0.0) (0.004 (0.003) 

Mode' p2 I PI I - 3.62 + 0.65 Source + 0.80 Ch.desc 

L 

PROJECT MODEL RESULTS 

Indecision IR=O% 

Z:lWiTcath clas. [1-2]/ [3-5] clas. [1-3] / (4-51 

Model P1 [2.39 2-80] : 88% [3.35 3.741 : 88% 

- 

- -- 

' Model pt r2.31 2.521 : 7496 [3.42 3-61 1 : 93% 

I 0.45 1 I 1 

I i I 0.75 I (0.001) (O.oO01) (0.001 6) (0.009) 

PI = - 1.34 + 0.59 Ch.desc + 0.50 Loc.add + 0.44 Comp.ch ModdP3 I 

(0.002) (0.003) (0.001) Model P4 I 

PI I - 2-95 + 0.65 Loc.add + 1.02 Loc.ch I 0.50 I 

Table 6: Project specific regression equations 

The commess is shown to improve substantially (see Table 7), compared to the general 

model results whenever easy changes = [l-21 (except for project P2). The results are only 

presented for a minimal undecidable interval. However, the interval could be widened as 

shown in the previous section in order to get even better correctness in the decidable 

intervals. 

Table 7: Classification results 

4-58 
5 Conclusions, Lessons Learned and Future Research 

This modeling approach provides a simple and flexible way of classifying changes 4uring 

the maintenance process. The classification power of continuous explanatory variables can 

be optimized by changing the class boundaries until the chi-square statistic reaches a 

maximum (this can be automated). This is performed while minimizing the number of 

metric classes and thereby facilitating the prdction process. It allows for an optimal use of 

the available explanatory variables by considering the uncertainty associated with each of 

them at the time of prediction. 

A user defined loss function (Le. risk model) can be while selecting the decision boundaries on the predicted index range until a predched expected loss is reached. 

This allows the construction of a Classification model optimal and customized, for specific 

user needs. Thus, by tuning the undecidable interval, he / she can handle in an appropriate 

and simple way the uncertainty associated with the model output. Also, the modeling 

process has shown many opportunities for a high extent of automation that would help 

optimize the memc class definitions and select the most suitable decision boundaries. 

Despite the fact that collecting change effort data on a discrete range (i.e. ordinal level) 

makes the data analysis more difficult and the usable statistical techniques less powerful, 

valuable information can still be extracted from the data while taking into account the 

constraints associated with a software development environment. As presented, effective 

classification has been performed among three effort classes with nspect to changes within 

the maintenance process. 

Despite organizational issues and data collection accuracy problems, it would be better to 

collect effort data at a ratio level. This would allow the use of more effective statistical 

techniques. The gains in terms of management efficiency are likely to be substantial. 

However, if effort data are collected in a discrete manner, each class should contain, to the 

extent possible, the same number of changes. When the dismbution is not uniform, 

classification for small proportion classes may be difficult 

Subsystem and component characteristics that are collectible in an automated way through 

code static analyzers (i.e. data binding between components, code complexity, ...) are 

likely to help refine the classification models. Maintainer skills and experience with respect 

to the maintained system should also be considered in the analysis in order to better select 

the required level experience for minimizing the cost of maintenance. Despite encouraging 

average results in the above experiments, a more complete data collection process is 

required in order to refine these change difficulty @ction models. 

.. . 

6 Acknowledgements 

We would like to thank Jon Valett hm the NASA Goddard Space Flight Center, Adam 

Poner and Chris Hetmanski for their suggestions that helped improve both the content and 

the form of this paper, 

7 References 

[BSP83] V. Basili, R. Selby and T. Phillips. “Metric Analysis and Data Validation across 

FORTRAN Project?. IEEE Transactions on Software Engineering, SE-9(6):652-663, 

November 1983 

pW841V. Basili and D. Weiss. “A Methodology for Collecting Valid Software 

Engineering Data”. IEEE Transactions on Software Engineering, SE-10(6):728-738, 

4-59 
November 1984 

[CAP881 J. Capon, “Statistics for the Social Sciences”, Wadworth publishing company, 

1988. 

PL841 W. Dillon and M. Goldstein, “Multivariate Analysis”, John Wiley & sons, 1984(, 

[GRA87] R. Grady, “Software Mctrics: Establishing a Company-Wide Program”, 

Prtntice-hall, 1987. 

LSSO] B. Lientz and E. Swanson, “Software maintenance management“, AddisonWesley, 1980. 

fRUV921 D. Rombach, B. Ulcq and J. Valett, “Toward Full Cycle Control: Adding 

Maintenance Measurement to the SEL”, Journal of systems and software, May 1992. 

4-60 
SECTION 5 - SOFTWARE 

MEASUREMENT 

SECIlON 5-SOFFWARE MEASUREMENT 

The technical paper included in this section was origindly prepared as indicated below. 

“’lbward Full Life Cycle Control: Adding Maintenance Measurement to the 

SEL,” H. D. Rombach, B. T. Ulery, and J. D. Vdett, Journal of Systems and 

Software, May 1992 

5-1 

1. SYSTEMS SOFTWARE 125 

1991: 18:125-138 

<- 

dLj -6/ 

,/j ,’.,-% / Toward Full Life Cycle Control: Adding c 

Maintenance Measurement to the SEL 

fY Ng2- 390 7r-f -. - (r 

H. Dieter Rombach and Bradford T. Ulery 

Computer Science Department and Umiaa, University of Maryland, College Park, Matyiond 

Jon D. Valett 

NASA, Goddard Space Flight Center, Greenbelt, Maryland 

Organization-wide measurement of software products 

and processes is needed to establish full life 

cycle control over software products. The Software 

Engineering Laboratory (SELI-a joint venture 

between NASA’s Goddard Space Flight Center, the 

University of Maryland, and Computer Sciences Corporation-started measurement of software development 

more than 15 years ago. Recently, the measurement 

of maintenance has been added to the scope of the 

SEL. In this article, the maintenance measurement program is presented as an addition to the already existing 

and well-established SEL development measurement 

program and evaluated in terms of its immediate benefits and long-term improvement potential. Immediate 

benefits of this program for the SEL include an increased understanding of the maintenance domain, the 

differences and commonalities between development 

and maintenance, and the cause-effect relationships 

between development and maintenance. Initial results 

from a sample maintenance study are presented to 

substantiate these benefits. The long-term potential of 

this program includes the use of maintenance baselines to better plan and manage future projects and to 

improve development and maintenance practices for 

future projects wherever warranted. 

1. INTRODUCTION 

Most software organizations lack satisfactory control 

over their development and maintenance projects. This 

lack of control is exemplified by the absence of explicit 

models enabling the identification of ambiguous prodAddress correspbndence to Pmfmor H. Dieter Rombach, AG 

S t W Eng., Fachbereich Informarik. Univenitaer Kaircrsloutern, 

Pactfach 3iU9, D-67SO Kaisenlauiern. Germany. 

uct requirements, the selection of practices best suited 

to achieve given requirements, or the prediction of the 

impact early project decisions may have on the quality 

of the resulting products. Each organization has its own 

set of control problems and reasons standing in the way 

of improvement. Comprehensive measurement programs are needed as a first step toward improvement 

[l]. Such programs can help identify the specific problems of an organization in quantitative terms, pinpoint 

possible causes, motivate improvements, and assess 

alternatives considered for improvement. 

The Software Engineering Laboratory (SEL)-a 

joint venture including government, industry, and 

university- began measurement of satellite ground 

support software development projects in 1976. The 

three primary organizational members of the SEL are 

the Systems Development Branch at NASA’s Gaddard 

Space Flight Center, the Computer Science Department 

at the University of Maryland, and the Systems Development Operation at Computer Sciences Corporation. 

This collaboration has produced numerous case studies 

and controlled experiments [2-61. Results from these 

case studies and experiments motivated several 

improvements within the SEL [7-91. 

In 1988. the SEL incorporated maintenance into its 

scope of measurement. The result is an even more 

comprehensive measurement program in which data is 

now being collected during development and maintenance of all software systems. In the SEL, pre- and 

postlaunch maintenance activities are performed by 

separate organizational entities. Cumntly , maintenance 

data are only collected from prelaunch maintenance 

activities. In the remainder of this article, the term 

“maintenance” shall refer to this prelaunch phase 

@ Elsevier Science Publishing Co., hc. 

655 Avenue of che Americas. New York. NY 10010 

loo06788L 

5-3 
126 J. SYSTEMS SOFTWARE 

1992: 18:125-138 

H. D. Rombach. B. T. Ulery. and J. D. Valen 

between delivery of a completed software system and 

the actual launch of the related spacecraft. This maintenance measurement program is customized to the peninent SEL characteristics, including the definition of 

maintenance, the maintenance improvement goals, and 

other product, process. and people factors. 

Empirical research in the SEL is based on the idea of 

continuous improvement. This idea has been formulated as the quality improvement paradigm [ 11. According to this paradigm, improvement is the result of 

continuously understanding current practices, changing 

them, and empirically validating the impact of these 

changes. Improvement requires measurement. 

In the SEL, measurement goals define the data to be 

collected and provide the context for data interpretation. This goal-oriented approach to measurement has 

been formulated as the goal/question/metric paradigm 

[ 1, 10, 1 11. It suggests defining each goal by developing a set of analysis questions, which in turn lead to a 

set of metrics and data. The short-term goals of our 

maintenance measurement program have been to 

increase the understanding of maintenance within the 

SEL; the long-term goals are to stimulate improvements in the SEL’s ability to plan and manage future 

maintenance projects and-whenever needed- to motivate the use of different development and maintenance 

practices. 

Specific characteristics of the SEL maintenance environmect 3 well 32 the ccrr.;rehtr.si:.: scope of vir 

measurement approach make this program unique. The 

study results presented here may not be directly comparable to those from other maintenance environments, 

yet they do show how a comprehensive measure program can be used to better understand and improve an 

organization’s development and maintenance process 

and products. Few comprehensive maintenance studies 

have been published [12- 141. Most empirical maintenance studies report on laboratory-style controlled 

experiments [ 15, 161, isolated case studies [13, 171. or 

project surveys [18]. A survey of maintenance studies 

has been published by Hale and Haworth [19]. 

The purpose of this article is to state our initial 

maintenance study goals and questions, present the 

related results, and propose-based on what we have 

learned-a revised set of goals and questions for hture 

studies. 

The study results are organized according to the 

types of data used to address the goals and questions: 

quantitative maintenance baselines, comparisons 

between quantitative development and maintenance 

baselines, and qualitative information regarding the 

cause-effect relationships between development and 

maintenance. These results have increased our understanding of maintenance processes and maintained 

products in the SEL. commonalities and differences 

between development and maintenance, and development characteristics affecting maintenance. On occasion, our results are carefully compared with results 

from other published studies or widely believed maintenance myths. 

We begin our presentation with a background discussion of the SEL and the new maintenance measurement program (sections 2 and 3, respectively). We then 

present the results of our study (section 4). We conclude with an assessment of the SEL maintenance 

measurement program and a revised set of goals and 

questions for future maintenance studies. 

2. THE SEL 

The goals of the SEL are to understand its software 

development processes, to measure the effects of various methods and tools on these processes, and to 

identify and then apply new, improved development 

practices. Improved understanding within this panicular environment provides the basis for better planning 

and management as well as a rationale for adopting new 

practices [4]. 

Development in the SEL supports satellite missions. 

SEL studies generally focus on attitude ground support 

systems and their associated simulators. These product 

lim are very stable: the system architecture. documentation standards, and organizational responsibilities do 

not change significantly from one mission to another. 

Attitude ground support systems have 130-240K lines 

of FORTRAN source code (where a line of code is 

measured as a physical line, including comment lines) 

and require 15-30 staff years to develop. Simulators 

have 25-75K lines and require 3-10 staff years to 

develop. 

Research in this environment is guided by two basic 

paradigms: the quality improvement paradigm (QIP) 

and the goal/question/metric paradigm (GQM). The 

QIP, which applies the principle of continuous improvement to software engineering, defines the context 

for measurement within the SEL [I]. Accordingly, 

software development can be improved by iterating the 

following steps for each project: (1) characterize the 

corporate environment: (2) state improvement goals in 

quantitative terms; (3) plan the appropriate development practices and methodologies together with measurement procedures for the project at hand; (4) perform 

the development and measure, analyze. and provide 

feedback; and (5) perform postmortem analysis and 

provide recommendations for future projects. Each QIP 

iteration is characterizea by its own set of goals. These 

5-4 
Toward Full Life Cycle Control J. SYSTEMS SORWARF. 127 

IW2; 11:125-138 

goals reflect-and evolve with-the maturity of the 

investigated organization. 

Measurement in the SEL is guided by the GQM 

paradigm [lo]. Measurement is used to characterize 

current development practices, monitor and manage 

development projects, identify strengths and weaknesses of the current practices, and evaluate promising 

new technologies in a controlled environment. The 

GQM paradigm describes a goal-oriented approach 

to measurement in which metrics are tied to specific measurement goals. According to the GQM paradigm, each measurement goal is listed explicitly, 

a set of specific questions is posed to address each 

goal, and specific metrics and measurement procedures are defined to support the questions. The resulting data collection procedures and interpretations are 

tailormade to the study's goals and local environment 

characteristics. For instance, in the SEL, this generally 

means that metrics and measurement procedures reflect 

the use of SEL-specific development practices, fit the 

organizational structure, and permit comparisons with 

historical data. Goals, questions, and metrics provide a 

context that helps ensure that data are interpreted correctly and are compared only to data and results from 

similar contexts. 

Two types of measurement are common in the SEL: 

routine monitoring and exploratory studies. Routine 

monitoring is used to characterize the local environnmr Lroadiy . The resulting qrlwititative and qualitative 

baselines are used to plan and manage new projects and 

to compare the effects of newly introduced tools or 

methods against [6]. Objective and subjective data are 

routinely gathered for each project [20]. Objective data 

include staff hours, computer utilization, source code 

growth, and the number and kinds of changes made to 

the source code. Subjective data characterize the software development process and software product characteristics. The data for over 100 projects monitored 

over the last 15 years is maintained in the SEL 

database [21]. 

Exploratory studies are used when the SEL is in the 

initial phase of understanding a process or methodology. For example. the SEL is currently studying three 

projects following the cleanroom methodology (221. 

Special data collection procedures were designed for 

these projects to permit researchers to monitor the 

effort spent in reading and reviewing designs and code. 

Measurement in the SEL has provided a rauonale for 

making evolutionary changes to NASA's development 

practices, including stricter use of code-reading techniques [SJ. guidelines for Ada projects [23], and the 

adoption of the cleanroom development approach (241. 

With the addition of maintenance measurement; the 

SEL is attempting to lay the foundation for similar 

improvements in maintenance. 

3. THE SEL MAINTENANCE MEASUREMENT 

PROGRAM 

The following subsections describe the SEL maintenance environment and the specific goals and procedures of our measurement program. A more detailed 

description of this environment. its products. and maintenance processes appeared in the proceedings of the 

1989 IEEE Conference on Software Maintenance [XI. 

3.1 Maintenance Environment 

In the SEL, maintenance is partly defined by organizational responsibility and schedule. As depicted in 

Figure 1, each product passes throush three different 

organizational units during its lifetime: analysts produce 

the initial functional specifications used by the developers and remain responsible for these specifications throughout development and until launch: 

operations assumes complete responsibility after 

launch. During the period between development 

and launch, the analysts have complete responsibility 

for the system, including the implementation of any 

changes. 

In this study, maintenance refers specifically to software change activities performed by the analysts during 

the postdevelopment, prelaunch phase. By nature of 

these constraints, the maintenance phase is typically 

shorter in the SEL than in other environments (one to 

two years), and the maintenance changes are not triggered by operational failures but by failures detected 

during simulated uses of the software by prospective operators and externally triggered changes of the 

overall satellite mission. 

~~ 

Figure 1. Organizational structure of the SEL environment. 

5-5 

1"L 
128 J. SYSTEMS SOFTWARE 

1992; 18:12S-138 

H. D. Rombach, B. T. Ulery. and J. D. Valett 

The products maintained are the same simulators and 

attitude ground support systems described in section 2. 

Typically, the effort expended during the one- to twoyear time frame that these systems are in maintenance 

is approximately 5 % of the development effort. Maintenance procedures vary from project to projeg depending on the typ of system being maintained, the size of 

the maintenance team (2-10 people on the projects 

studied), the specific methods and tools elected by the 

individual programmers, and other factors. In general, 

formal change control procedures are followed; changes 

are implemented one at a time, but may be tested 

in groups; and one maintainer is responsible for 

implementing each change. 

3.2 Maintenance Measurement Goals 

Consistent with the overall directions of the SEL. we 

chose three general goals for the maintenance measurement program: (1) to understand maintenance processes 

and products better; (2) to improve our ability to 

manage current maintenance projects and plan future 

ones; and (3) to establish a sound basis for in >roving 

development from a maintenance perspective. 

Following the QIP, the initial goals focus on understanding maintenance. Representative measurement 

goals and questions selected for this study are summarized in Figures 2, 5, and 11. Analysis results related to 

these goals and questions are presented in section 4. 

3.3 Maintenance Measurement Procedures 

The data collection procedures used in this study were 

designed according to the principles of the GQM 

paradigm. Data were collected via exploratory interviews and routine data collection forms [20]. The routine data collection forms used during maintenance 

include the Weekly Maintenance Effon Form and the 

Maintenance Change Report Form (Appendix A). The 

effort form is filled out once per week per maintainer 

per system; one change form is filled out per completed 

change. The weekly effort forms record the distribution 

of effort (in staff hours) by type of change (correction, 

enhancement, adaptation, or other ') and by engineering 

activity (designing, coding, etc.). The change forms 

record the distribution of changes by type of change, 

sue of change, changed objects (e.g., code, user's 

guide), expended staff time, fault type (if applicable), 

and more. All data are validated through a series of 

'All t"Icnance effon that cannot k aNnbuttd to an individual 

miuntcnance change IS classified as "other." Th~s includes cffon 

related to management. mccungs, and training. 

checks by the data entry personnel. project managers. 

and SEL researchers. Data are stored and made available to researchers and developers through the SEL 

database [2 11. 

4. MAINTENANCE MEASUREMENT BENEFITS 

The maintenance measurement program has already 

increased understanding of maintenance in the SEL. 

Previously, much of this understanding was at best 

intuitive and approximate. In this section we demonstrate what we have learned as a result of our initial 

study. The results are separated into baseline characterizations of maintenance, a comparative analysis of 

development and maintenance, and an analysis of how 

development decisions affect maintenance. 

In this study, we restrict our analyses to three large 

attitude ground support systems for which we have 

complete and valid data: the Gamma Ray Observatory, 

the Geostationary Operational Environmental Satellite, 

and the Cosmic Background Explorer. Maintenance of 

these systems was performed between 1988 and 1991. 

A total of 90 changes and over 10,OOO hours of effort 

serve as the basis for all quantitative analyses of maintenance presented here. 

Examining the data on these three projects has provided valuable insight into the maintenance process 

within this environment. The results presented here are 

intciidcd io demonstraie the iucreased understanding 

of the maintenance process that can result from a 

measurement program. 

4.1 Maintenance Baselines 

The first step toward understanding any environment is to develop baselines describing that environment 

[ 12, 14). The goals and questions related to this part of 

the SEL study are listed in Figure 2. They are intended 

GOAL 1 : Characterize the changes performed during 

maintenance. 

QUESTION 1 

QUESTION 2 

How many changes of each type are completed? 

How much effort is spent on changes of each type? 

GOAL 2: Characterize product evolution during 

maintenance. 

QUESTION 3 

QUESTION 4 

How much code is affected by each change? 

Is code added, changed or deleted? 

GOAL 3: Characterize the maintenance process stability 

QUESTION 5 

How do maintenance processes differ across projects? 

Figure 2. Measurement goals for understanding maintenance 

in the SEL. 

5-6 

1"L 
Toward Full Life Cycle Control 1. SYSTEMS SOFTWARE 129 

1992; 18:125-138 

to characterize what kinds of changes are performed 

during maintenance, which parts of the systems change 

and how, and what maintenance processes are followed. In the long term, the resulting baselines are 

expected to provide a basis for determining whether 

new techniques or process adjustments have any measurable impact on the SEL maintenance processes or 

products. Any comparison between SEL baselines and 

baselines from other environments must take environmental differences into account. 

Each maintenance change in this environment is well 

defined by a formal change request. There are several 

key steps in the change process: changes must be 

approved, implemented, tested, and released. In general, more changes are approved than can be implemented. This poses the difficult management problem 

of selecting which changes to implement. This decision 

is based on the importance of the changes approved as 

well as the budget available to make changes. The 

implementation of a change is performed by one programmer; there is no standard, formal methodology. 

Testing, beyond debugging by the programmer, is performed for several changes at once. One important 

implication is that the associated effort measured cannot 

be ascribed to a particular change. In fact, testing is 

typically performed at two levels: the first level provides internal checkpoints for configuration management; the second level occurs before each release. 

Each idnicnance change pcrforiiieri in the: SEL is 

classified as an enhancement, adaptation, or correction 

[26]. A simple count of changes suggests that maintenance is primarily corrective; however, the effort distribution reveals that most effort is actually related to 

enhancements (Figure 3). Either way, adaptations do 

not seem to contribute significantly (Figure 2, questions 

1 and 2). Note that the average enhancement requires 

just over twice the effort of the average correction. 

This phenomenon could be caused by the fact that 

enhancements are typically larger than corrections. 

that enhancements are inherently more difficult to 

accommodate into an existing system, or both. 

As early as 1976, Belady and Lehman [14] demonstrated the benefits of program evolution models for the 

purpose of understanding the decay of software undergoing change. Figure 4 summarizes how many modules 

and lines of source code have been added, changed, or 

deleted per change (Figure 2, questions 3 and 4). On 

average, three lines of code are added for every existing line changed or deleted. Entire modules are rarely 

added and never deleted. In the SEL. maintainers do 

not significantly alter the system’s architecture to make 

changes. We hypothesize that the high number of lines 

added reflects the high proportion of enhancements, 

and that architectural stability reflects an “if it ain’t 

broke don’t fix it” attitude. Such an attitude could be 

explained by the general lack of understanding of overall system architecture. The observed growth pattern 

also suggests that module functionality increases during 

maintenance, leading to a decrease in module cohesion. 

Decreased cohesion may not be a problem during the 

short lifespan of a satellite system, but may reduce 

the reuse potential of modules in future developments. 

Our most striking observation about SEL maintenance is the extent to which the maintenance processes 

vary across similar projects (Figure 2, question 5). 

Some of the variability reflects the size and composition 

of the maintenance teams (2-10 programmers). One 

particular area where the processes differ appears to be 

in the approach to testing. The projects studied have 

not established well-defined criteria for when system or 

integration testins should be performed during maintenance. Such variability in the process reflects the relatively ad hoc nature of the maintenance environment as 

compared to the development environment. In fact, 

. 

Enhuwvmnt 

3% 

Effort Number of Changes 

Figure 3. Distributions of effort and number of changes by type. 

5-7 

loO05788L 
J. SYSTEMS SOFTWARE 

1992; 18:125-138 

130 H. D. Rombach, B. T. Ulery, and J. D. Valett 

1- 

0.0 

Lines of Code 

Figure 4. Lines of code and modules per change. 

studies such as this one aim at increasing the maturity 

of the maintenance process within this environment. By 

identifying which aspects of the process are most successful, a single consistent process will be identified. 

4.2 Maintenance vs. Development 

Applying experience from past development studies to 

maintenance requires an understanding of the similarities and differences between maintenance and development. The goals of this part of the study were to 

compare changes made during development and maintenance. types of changes. and change processes 

(Figure 5). These comparisons are possible because 

both development and maintenance data are available 

for the three systems studied. 

Throughout development and maintenance, the effort 

spent on each change is recorded. Effort is classified 

as easy when it takes less than an hour to complete 

a change, medium when it takes between an hour and a 

~~~ 

GOAL 4: Compare changes made during development and 

maintenance. 

QUESTION 6 

How does the effon per change compare? 

GOAL 5: Compare the types of changes made to products 

at both phases. 

QUESTION 7 

Are the faults found during maintenance different than 

those found during development? 

QUESTION 8 

How do the distributions of errors by class 

compare? 

QUESTION 9 

How docs the distribution of effort by activity type 

compare? 

Figure 5. Measurement goals for understanding the similarities and differences beween development and maintenance. 

GOAL 6: Compare change processes at both phases. 

Modules 

day, and hard otherwise. A distinction is made between 

the effort to isolate a change (understand the request 

and locate the affected modules) and the effort to complete the change (design. code, test). Figure 6 shows 

that changes performed during maintenance generally 

require more effort than those performed during development (Figure 5, question 6). We consider two 

hypotheses that might account for this pattern: changes 

requested during maintenance are inherently harder 

than those requested during development; and it is 

more difficult to perform the same change during maintenance than it would be during development. While we 

cancot dete.mir.e whether partied;; mzdu!c; are easy 

or difficult to change during maintenance based on our 

data, we are able to examine both hypotheses further at 

the level of the individual change. 

Regarding the first hypothesis, we find no obvious 

difference between the effort distribution patterns for all 

changes (Figure 6) and corrections only (Figure 7). We 

conclude that the increased effort is not primarily due to 

differences in the distributions of types of changes 

requested. 

Regarding the second hypothesis, various characteristic differences between development and maintenance 

are commonly thought to explain why the same change 

might be more difficult to perform during maintenance. 

These include product factors (such as increased complexity and missing or out-of-date documentation), process factors (such as schedule constraints, methods, and 

tools), and people factors (such as a lack of familiarity 

with the software). In the SEL, we cannot attribute the 

maintenance difficulties to product factors because there 

is already a sharp increase in change effort during 

acceptance test, but little change in the products. 

Instead, we suspect some combination of process and 

people factors. Although we are unaware of any significant methodological differences between the 

5-8 
Toward Full Life Cycle Control 1. SYSTEMS SOFIWARE 131 

1992: 18:125-138 

0% 

Ely Wium Hud 

Effort to Isolate Changes 

60% 1 bnto- I I s D.v.iopnunt I 

Ely Modium Hud 

Effort to Complete Changes 

Figure 6. Effort to isolate and complete changes: maintenance vs. development. Easy, (1 hour; medium, ) 1 hour and 

(1 day; hard, )I day. 

way a change is implemented during development 

or maintenance, development has a much higher 

rate of change activity: these systems average 

over 1,OOO changes during testing. Although the high 

number of changes may increase certain costs (e.g., 

configuration control), it may actually reduce others 

(e.g., testing is not repeated once for every change). 

Maintainers are not only generally unfamiliar with the 

systems they maintain. but the volume of maintenance 

may be insufficient to develop such familiarity. We 

expected the unfamiliarity with the maintained systems 

to have a more dramatic impact on the isolation activity 

(which might require an understanding of the entire 

system) than the completion activity (which typically 

requires only an understanding of individual modules). 

40%- 

20%- 

Modi" nard 

Effort to Isolate Faults 

Ely Modium Hud 

Effort to Complete Faults 

Figure 7. Effort to isolate and complete faults: maintenance 

vs. development. Easy. (1 hour; medium, ) 1 hour and (1 day; 

hard, )1 day. 

Instead, we discovered a proportional increase in both 

isolation and completion efforts (Figure 6). This may 

be explained by the fact that SEL maintainers are 

experts in the application domain, not sofnuare development; therefore, they may be expected to readily 

understand the change specifications, but not the code. 

Both during development and maintenance a significant fraction of the changes are corrections (Figure 3). 

Figure 8 shows that the types of faults corrected during 

development and maintenance are similarly distributed 

(Figure 5, question 7). During maintenance, more corrections are related to incorrect initialization (21 vs. 

17%) and logic (25 vs. 19%), but fewer are related to 

incomct interface (19 vs. 22%), data (26 vs. 28%), 

and computation (9 vs. 14%) as compared to develop5-9 
132 J. SYSTEMS SORWARE H. D. Rombach, B. T. Ulery. and J. D. Valen 

1992; 18:12!5-138 

Initialization Logic lnlertace Data Computational 

Figure 8. Number of faults per class: maintenance vs. development. 

ment (see [20] for definition of classification scheme). 

Some of the differences seem to be related to the 

organizational structure of the environment. Maintenance is performed by people more familiar with the 

application domain and less familiar with the solution 

domain. The opposite is true for the developers. In this 

environment, many application-specific parameters are 

iefiectel in the sohare as iiikidization parmeters. ?.s 

such, they require a clear understanding of the application, and faults are more easily found by maintainers. 

The opposite is true for typical solution faults such as 

interface and computational faults. 

Figure 9 shows that the distributions of errors differ 

significantly between maintenance and development 

(Figure 5, question 8). During maintenance. many 

more faults are attributed to inappropriate requirements 

or specifications (26 vs. 3%). and a few more are 

attributed to inappropriate design (1 1 vs. 8%): fewer 

are attributed to inappropriate implementation (55 vs. 

73%) or previous change; (7 vs. !e%). !n sttempting !c 

explain these differences, the following hypotheses have 

been formulated. Few faults are attributed to previous 

changes during maintenance because maintainers are 

unaware of changes made during development and the 

SpecifiAions Design code Previous Change 

Figure 89. Number of faults per source: maintenance vs. development. 

5-10 

1ooo6788l 
Toward Full Life Cycle Control J. SYSTEMS SOFTWARE 133 

1992: 18: 125- 138 

total number of changes during this phase is low. The 

high proportion of faults attributed to the requirements 

or specifications reflects the nature of the testing: applications experts are now using the systems to prepare 

for the missions, whereas during development most 

testing is performed by the developers themselves. 

During development and maintenance, effort data 

is collected according to the following process model: 

isolation (understanding a requested change and 

identifying the affected modules), design (proposing 

a change), implementation (implement the proposed 

change), unit and system test (testing the changed modules and system), and acceptance test (testing a set of 

related changes). The development data include all 

effort; it is not limited to changes. 

Figure 10 shows that during maintenance, more effort 

is spent on design activities, about the same amount 

of effort is spent on implementation activities, and less 

effort is spent on testing activities (Figure 5, question 

9). The increase in design effort may be explained by a 

lack of familiarity with the system structure, resulting 

in increased effort to isolate changes. The decrease in 

testing effort may be explained by different testing 

procedures. During maintenance, integration testing is 

almost absent because the system structure doesn’t 

change much, and acceptance testing is performed for 

groups of changes together. 

How do these results compare with similar findings 

published in the literature’? W hue comparing baseline 

data across environments is difficult. some patterns are 

evident. The increased cost of maintenance changes and 

corrections has been noted previously by many authors 

[22, 271. This lends support to the claim that faults 

introduced during design but discovered during maintenance may cost significantly more than if discovered 

and corrected earlier in the life cycle [27]. As has been 

noted in other environments [28], we find that maintenance changes in the SEL require more “upstream” 

(i.e., design) than “down-stream’’ (i.e.. testing) effort). 

4.3 Development for Maintenance 

As a final result of the maintenance measurement program, the SEL has enhanced its understanding of the 

impact of development decisions on maintenance 

(Figure 11). This increased understanding is illustrated 

by our initial findings concerning the complexity of 

delivered products and the quality of their documentation. The qualitative results of this section are based 

primarily on subjective data from exploratory interviews. Nevertheless, they are essential during the early 

phases of a measurement program for guiding future 

improvement cycles. 

Our initial inquiries have revealed complexity problems related to intermodule structure and the encoding 

global information (Figure 11, question 10). Maintainers reported major problems related to the fact 

that global information was encoded redundantly. For 

example, constants were encoded in multiple 

FORTRAN common blocks. Software modification frequently resulted in inconsistent representations of global 

information. 

Two recurrent documentation problems have been 

identified (Figure 1 1, question 11). These concern the 

30 

- 

s 

E 

W - 

Figure 10. Effon per activity: maintenance vs. 20 Ol development. m 

ti 

L 

a. 

10 

0 

Dergn Implement Test Other 

Aclivily 

5-1 1 
J. SYSTEMS SOFTWARE 

1992: 18:125-138 

134 H. D. Rombach, B. T. Ulery, and J. D. Valett 

GOAL 7: Characterize the impact of the delivered product 

on maintenance. 

What structural product characteristics have positive/ 

negative effects on maintenance? 

What product documentation standards have positive/ 

negative effects on maintenance? 

Figure 11. Measurement goal for understanding the effects 

of development on maintenance. 

QUESTION 10 

QUESTION 11 

use of program design language (PDL) and debug 

statements. PDL descriptions of each module are 

included in the source code as a header. Most maintainers regard PDL as redundant. Furthermore, the delivered PDL is usually outdated. In the SEL environment, 

developers are required to keep their design PDL as 

part of the software module. Unfortunately, this PDL is 

frequently obsolete by the time the module reaches the 

maintenance phase; thus, it is useless to the maintainers. Also, the majority of people maintaining the software suggested that this practice be stopped entirely, 

since the same level of abstraction is provided to them 

in the code structure and comments. 

Many maintainers suggested that the debug interface 

of the code be improved. Because attitude ground 

support software is highly computational, an extensiuc CIeb~g ktrrfice is provided with each system. The 

problem with the current debug interface is that frequently it assumes intimate familiarity with the code in 

that the output was of the form (variable) = (value). 

Maintainers suggested that future debug interfaces 

provide a more descriptive explanation of the output 

printed. 

As we learn more about the problems maintainers 

have with the software delivered from development and 

identify solutions to these problem, the guidelines and 

standards for development [7-91 will be modified to 

reflect these recommendations. 

5. SUMMARY AND CONCLUSIONS 

In this section, we summarize the benefits of the maintenance measurement study for the SEL, outline future 

maintenance measurement directions within SEL, and 

package some of the general lessons learned about 

establishing measurement programs for use in other 

maintenance environments. 

5.1 SEL Maintenance Study Benefits 

The most immediate benefit of this program has been 

an enhanced understanding of the SEL maintenance - 

environment. The quantitative baselines presented in 

the preceding section resulted in a better understanding 

of maintenance requests, maintained products, and 

maintenance processes. They enabled us to identify 

weaknesses in the SEL maintenance environment. 

The comparison between changes performed during 

development and maintenance has helped us understand 

where we may benefit from existing development baselines. For example, whereas the distributions of faults 

corrected during development and maintenance are similar, effort distributions are not. This suggests that 

reuse of lessons learned from development is more 

justified when they pertain to faults than when they 

pertain to effort. 

Baselines may also be used to compare the effects of 

new development technologies on maintenance. For 

example, both cleanroom and an Ada/object-oriented 

design approach have been applied on recent development projects with the expectation that "more reliable" 

systems will result. We are now in a position to validate these expectations by comparing the effects of the 

new approaches to traditionally run projects. 

In the long term, development and maintenance are 

expected to improve as a result of our increased understanding. At this point, recommendations for improvement are based predominantly on qualitative feedback 

from maintainers (rather than quantitative measurement 

baselines). Most of these suggestions have to do with 

the separation of the deveiopnient anri analysis or@- 

zations (Figure I) and the absence of standard maintenance processes. The separation of development and 

maintenance means that a maintainer is entirely dependent on the code and documentation acquired at the 

time of delivery [29]. Consequently, inadequacies in 

the code or documentation are much more of an obstacle to maintenance than in an organization where maintenance and development are more closely related. 

Each maintenance change is performed by one individual without much guidance regarding the maintenance process itself. The ad hoc nature of the 

maintenance processes makes it hard to measure. compare measurements, and make recommendations. We 

expect our measurement program to contribute to the 

standardization of maintenance processes over time. 

Overall, the SEL maintenance measurement program 

is perceived as successful and beneficial to this particular environment. The lessons learned from our study 

have resulted in changes and additions to the SEL 

standards and policies for software development [8]. 

Because numerous new projects are always under 

development in the SEL. we will be able to examine 

whether the revised standards have a measurable impact 

on the quality of the development product. 

5-12 
Toward Full Life Cycle Control 1. SYSTrMS SOFTWARE 135 

1992; 18:125-138 

5.2 Future Maintenance Research 

As we continue to learn about the SEL maintenance 

environment, numerous future measurement directions 

become evident. Some directions reflect changes in 

the environment itself, others reflect changes in our 

understanding of the environment. We must continually revise our goals, questions, meuics, and procedures to reflect the current priorities and understanding. 

Figure 12 contains an example set of revised questions 

for each of our seven maintenance goals to guide hture 

maintenance studies. 

We must continue to revise our measurement program in response to previous misconceptions inherent 

in our initial qualitative models of maintenance process. 

For example, our current effort classification scheme 

does not explicitly recognize configuration management 

as a discrete activity. This effort is grouped together 

with nontechnical activities such as meetings and management. In the future, we may want to update our data 

collection forms to include configuration management 

as a separate activity, since it seems to represent a 

significant portion of current maintenance effort. 

GOAL 1 : Characterize the changes performed during 

maintenance. 

QUESTION 1 

How many changes of each type are requested by 

different sources (e.g., analyst, operator)? 

GOAL 1: Charxterie pmduct evolution d~ring 

maintenance. 

QUESTION 2 

How docs coupling/cohesion change during 

maintenance? 

GOAL 3: Characterize the maintenance process stability. 

QUESTION 3 

Which process factors determined process stability 

(e.g., staffing level. familiarity with system)? 

GOAL 4: Compare changes made during development and 

maintenance. 

QUESTION 4 

What is the average change effon per module during 

each phase? 

GOAL 5: Compare changes made to products at both 

phases. 

QUESTION 5 

What are the distributions of requirements changes by 

type? GOAL 6: Compare development and maintenance processes. 

QUESTION 6 

What are the distributions of change effon by activity. 

GOAL 7: Characterize the impact of the delivered product 

on maintenance. 

QUESTION 7 

What product characteristics resulting from reuse have 

- positive/negative effecrs on maintenance? . 

Figure 12. Revised measurement questions for future 

maintenance improvement cyclu: 

When our empirical investigations identify important 

phenomena, we must refocus our measurement goals 

and questions in order to study the phenomena. For 

example, one hypothesized implication of the stable 

architecture of the maintained systems (very few 

modules are being added or deleted) is that module 

cohesion within these systems may be deteriorating. 

Such deterioration may lead to weaker and weaker 

system architecture, and ultimately lead to even more 

difficult maintenance. Such a hypothesis needs much 

closer investigation before it can be presented as a 

potential problem. 

When measurement does identify specific problems, 

the next step is to analyze the problems and attempt to 

identify viable solutions. For instance, we have quantified the types and kinds of faults uncovered during 

maintenance. Next, we might begin to analyze their 

causes in development. Such analysis may lead us to 

mechanisms for preventing faults, or it may help us 

identify better ways of detecting them. 

Finally, the maintenance environment itself is continually changing. Transitions to the use of Ada and 

Cleanmm development in the SEL will require periodic adjustments to our measurement procedures. Such 

changes are not unexpected; in fact, measurement by 

nature must continue to evolve as the environment 

evolves. 

5.3 Measurement Lessons Learned 

The extension of the SEL into maintenance not only 

enabled us to gain experience with maintenance 

but also with establishing a maintenance measurement 

program [25]. 

Our first lesson is that there is a distinction, at least 

conceptually, between start-up and routine phases of 

measurement. During the start-up phase, there is considerable freedom to reevaluate measurement goals and 

redesign the metrics and procedures as our understanding of the local priorities and what is feasible grows. 

Once data collection forms have been designed and 

reflected in the data base and once people have been 

instructed in the procedures, it becomes expensive to 

introduce further changes. It is therefore critical that 

the start-up phase proceed cautiously. We suggest validating all measurement procedures through pilot studies. 

Our second lesson concerns which questions are 

suitable for routine measurement. It may be tempting to 

use routine measurement as a mechanism for answering 

questions that could be resolved more efficiently by 

other means. For example. if the sofnvare design docu- 

"ation is never maintained, it would be wasteful to 

discover this via routine data collection. Routine mea5-13 
- 

136 J. SYSTEMS SORWARE 

1992: 18:125-138 

H. D. Rombach, B. T. Ulery, and J. D. Valett 

surement is appropriate for monitoring large-scale and 

historical trends, but it is not needed to ascertain simple 

facts. Many of the questions we would like to pursue 

are risky, i.e., we cannot be sure that the resulting data 

will prove useful. 

Third, we have found the establishment of a 

measurement program in a new environment to be 

a timeconsuming and sensitive task. Getting the program started requires building initial models of the 

maintenance organization, the maintained products, the 

maintenance processes. and the specific maintenance 

problems at hand. These models are used to design the 

measurement procedures. but must be validated during 

the start-up phase. Special care must also be taken to 

establish the creditability of measurement and win the 

cooperation needed to make the program a success. To 

collect valid data, the people providing most of the data 

need to be well motivated and instructed. Motivation 

requires addressing measurement goals of direct interest to the people providing cooperation and an opportunity for these people to review and comment on the 

resulting data and analyses. 

Our analysis results demonstrate the immediate 

returns possible from investment in a measurement 

program. A measurement program provides invaluable 

insight into the processes and products within the given 

environment. As long as measurement is performed 

within a context of well defined goals and questions, 

such a program can be a success for any software 

organization. 

ACKNOWLEDGMENTS 

Research for this study was supported by National 

Aeronautics and Space Administration grant NSG-5123 to 

the University of Maryland. 

We thank the CSC and GSFC personnel who have participated in our maintenance measurement program-the project 

personnel, the SEL data librarians, and those who reviewed 

earlier versions of this paper-for their tremendous support. 

We also thank Bruce Blum. the editor in charge of our paper, 

and the anonymous referees for their excellent suggestions. 

REFERENCES 

1. V. R. Basili. Software development: a paradigm for the 

future, in Proceedings of the 13th Annual International Computer Soft ware and Applications Conference, 1989, pp. 471-485. 

2. M. Buhler and J. Valett. Annotated Bibliography of 

Software Engineering Laboratory Literature. SEL-82- 

906, NASA/GSFC, Gr&nbelt, Maryland, 1990. 

3. V. R. Basili. Measuring the software process and product: lessons learned in the SEL, in Proceedings of the 

10th Annual Software Engineering Workshop, 1985. 

4. F. E. McGany, Studies and experiments in the SEL. in 

5. 

6. 

7. 

8. 

9. 

10. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 

21. 

22. 

23, 

Proceedings of the 10th Annual Software Engineering 

Workshop. 1985. 

R. W. Selby, Jr.. and V. R. Basili, Comparing the 

Effectiveness of Software Testing Strategies. IEEE 

Tram. Software Eng. 13: 1278- 1296 (1987). 

J. D. Valett and F. E. McGarry, A summary of software 

measurement experience in the Software Engineering 

Laboratory, in Proceedings of the 2Ist Annual Hawaii 

International Conference on System Sciences. 1988. 

F. McGarry, G. Page, S. Eslinger. V. Church. and 

P. Merwarth. Recommended Approach to Software 

Development, SEL-81-205, NASA/GSFC. Greenbelt, 

Maryland, 1983. 

Manager’s Handbook for Software Development. rev. 1. 

SEL-84-101, NASA/GSFC, Greenbelt. Maryland, 1990. 

R. Wood and E. Edwards. Programmer’s Handbook for 

Flight Dynamics Sofnvare Development, SEL-86-00 1. 

NASA/GSFC. Greenbelt Maryland, 1986. 

V. R. Basili and D. M. Wciss. A Methodology for 

Collecting Valid Software Engineering Data. IEEE 

Trans. Software Eng. SE-10. 728-738 (1984) 

V. R. Basili and H. D. Rombach, The TAME Project: 

Towards Improvement-Oriented Software Environments, 

IEEE Trans. Software Eng. 758-773 (1988). 

R. B. Grady, Measuring and Managing Software Maintenance, IEEE Software 4. 35-45 (1987). 

R. Arnold and D. Parker, The dimensions of healthy 

maintenance, in 6th Internarional Conference on Software Engineering, 1982, pp. 10-27. 

L. Belady and M. Lehman, A Model of Large Program 

Deveiopment. IkM Syst. J. 3, 225-252 (irib). 

H. D. Rombach, A Controlled Experiment on the Impact 

of Software Structure on Maintainability. IEEE Trans. 

Software Eng. SE-12, 344-354 (1987). 

C. K. S. Chong Hok Yuen, An empirical approach to the 

study of errors in large software under maintenance. in 

Conference on Software Muintenance - 1985. 1985, 

H. D. Rombach and V. R. Basili, Quantitative assessment of maintenance: an industrial case study, in IEEE 

Conference on Software Maintenance - 1987, 1987, 

B. W. Boehm and P. N. Papaccio, Understanding and 

Controlling Software Costs, IEEE Trans. Software 

Eng. SE-14, 1462- 1477 (1988). 

D. P. Hale and D. A. Haworth. Software maintenance: a 

profile of past empirical research, in Conference on 

Software Maintenance - 1988, 1988. pp. 236-240. 

G. Heller. Data Collection Procedures for the Rehosted 

SEL Database, SEL-87-008, NASA/GSFC. Greenbelt, 

,Maryland, 1987. 

M. So. SEL Database Organization and User’s Guide, 

SEL-89401. NASA/GSFC, Greenbelt, Maryland, 1989. 

H. D. Mills, Software Development. IEEE Trans. Soft: 

ware Eng. 2:265-273 (1976). 

C. Brophy, Lessons Learned in the Transition to Ada 

from FORTRAN at NASA/Goddard, SEL-89-005, 

NASAIGSFC, Greenbelt, Maryland, 1989. 

pp. 96-105. 

pp. 13-144. 

5-14 
Toward Full Life Cycle Control 

Othor 

J. SYSTEMS SOFIWARE 137 

1992; 18:125-138 

Othor hours spont on tho projd (nktd to nuintonma) not 

eovord abovo. includos nunrg.nwnt, mootings. 00. 

24. S. Green, A. Kouchakdjian, V. Basili, and D. Weidow. 

The Cleanroom Case Study in the Software Engineering 

Laboratory: Project Description and Early Analysis, 

SEL-90-002, NASA/GSFC, Greenbelt, Maryland, 1990. 

25. H. D. Rombach and B. T. Ulery, Establishing a mcasurement based maintenance improvement program: 

lessons learned in the SEL, Conference on Software 

Maintenance - 1989. 1989. pp. 50-57. 

26. E. B. Swanson, The dimensions of software maintenance, in Proceedings of the 2nd IEEE Inrernationol 

Unlt tuff 

Syrtom Tnt 

AccaptaW 

Bonchmarlc Toet 

0th.r 

APPENDIX A: Data Collection Forms 

Hours spont tuting tho dungod or addod components. 

lncludu hours spnt tostlng tho Intogrstkn of tho componontr. 

Hours spont mceoptanco to.tlng or bonchnurk tostlng tho 

modltkd systom. 

Other hours spont on tho projut (rohtod to nuintonma) not 

cowd abovo. includu nurug.nwnt. mthgs, otc. 

Conference on Soff ware Engineering. 1976. pp. 492 - 497. 

27. B. Boehm. Software Engineering Economics. 

Prcntice-Hall. Englewood Cliffs, New Jersey, 1981. 

28. N. Chapin, Software Maintenance Life Cycle, in Conference of Software Maintenance - 1988, 1988, pp. 

29. E. B. Swanson and C. M. Beath, Departmentalization in 

Software Development and Maintenance. Commun. 

6-13. 

ACM 33. 658-667 (1990). 

FQ-rurw WEEKLY MAINTENANCE EFFORT FORM 

N.m: 

ProJ.ct: 

FrkLy Dab: -w 

Section A -Total Houn Spent on Malntenance (Inc(udr(km.p.n(md nul- n 

I C1.u ~ ___ I Detlnitlon I Houn 1 I I I IComction I Hours smnt on all nulntanrnco usocbtd with a rvstom 1 1 

II Ihllum. . I I 11 I 

Adqptatbn 

Houn sponl on dI nulntonnco ...oclrtod with modlfylng tho 

rystom duo to a roqulmnmnb chango. includ.. adding, 

dMlng, or modlfylng syrtam hatuns as a nsult of 

mqulnmon~ chango. 

%c?z rp~! on all ~r..lfitanrncC .uochlod wilh mociilying a 

systwn to dapt to a chango in lureban. syrtrm softwan. or 

onvlronmntal char.ctorlstkt. 

IF I 

Hours spont undustrnding tho klluro or mquost for 

Mh.ncwn"o or adaptation. 

Chango 

Doslgn 

Hours rpont .ctu.lly mdoslgning tho ry.1.m b.4 on an 

udorstanding of tho ncrrury ehngo. 

Hours spont chmglng tho systom to compkta tho noc.rury 

chango. Thls inciudu changlng not only tho cod., but tho 

urocktddotwmnt.tlon. 

5-15 
138 J. SYSTEMS SOFTWARE 

1992: 18:125-138 

H. D. Rombach, B. T. Ulery, and I. D. Valen 

For "MI useMAINTENANCE CHANGE REPORT FORM - 

OSMR Nu- m 

SECTION A: Change Request lntormatlon 

Functiorvl Dowrlptlon of Chengo: 

What was tho typo ot modlfiutlon? 

- Comdlon 

- Enhncwmnt 

- Adeputbn 

What causod th. ch.ng.7 

- Roqulmmont~s.p.clflutbns 

- Soltmmduign 

- Pmviocuchango 

- 0th.r 

-cod. 

SECTION 8: Change Impknnntrtlon Informatlon 

Comoononts ChanaodAddo&Dol.t.d: 

lhrl0 16.yto Irc..klO 

+lhr 16.y 1-k lmonth >lmonth 

Eatlnute offort spont ird.Ung/dotormlning tho chengo: 

EstImato offort to dosign, Impknmnt, and tort tho chango: 

Chock ail chengd ob).as: 

- RoqulmmenUSp.clticatbns Documont 

- Doaign Documont 

- Cod. 

- Systom Docrlption 

- Usds Guide 

- Othor 

If cod. chongod, ch"trrixo tho chmgo !chC most 

eppllubk) 

- Inltlaltzatkn 

- Logidcontrol struetun 

- Intorhco (intorrul) 

- lntorf.co (eaomal) 

- Data (valm or rtructuro) 

- Computational 

- Othor (nono of tho abovo apply) 

(e.9.. changod flow of control) 

(moduk to moduk communkatlon) 

(moduk to oxtoml communicatbn) 

(0.g.. variabk of velue changd) 

(0.9.. clung. of math oxpr.ubn) 

Estlmato tho numkr of 11110s ot cod. (including comments): - 

Entor tho numkr ot compononts: - - 

addod chengd dktd 

Entor tho numkr of tho addad componnts that aro twty - 

now totdly musod mw Wfih 

oddd 

.-, 

changod 

- 

delolod 

modlflutlons 

AAY 1989 

Tigure A2. Maintenance Change Report Form. 

5-16 
I 

E 

I 

i 

SECTION 6 - ADA TECHNOLOGY 

SECTION &ADA TECHNOLOGY 

The technical papers included in this section were originally prepared as indicated 

below. 

“Object-Oriented Programming with Mixins in Ada,” E. Seidewitz, Ada 

Letters, March/Aprill992 

“Software Engineering Laboratory Ada Performance Study-Results and 

Implications,” E. W. Booth and M. E. Stark, A.oceedings ofthe Fourth Annual 

NASA A& User’s Symposirun, April 1992 

6-1 

/ 

-"? - NTED PROGRAWG WITH WINS IN ADA /SG/YD * 

Ed Seidewitz 

Goddard Space Fli ht Center 

&de 552.2 

Greenbelt MD 20777 

(30 11286- 763 1 

eseidewitz@gs fcmail. nasa.gov 

My guess is that object-oriented programming will be in the 

1980s what structured programming was in the 1970s. 

Everyone will be in fmr of it. Every "ufacturer will 

promote his products os supporting it. Every manager will 

pay lip semce to it. Ewry programmer will practice it. And 

no one will know just what it is. 

IRcn-h 821 

ming language feam in 

Ada 83, why one might desire them in Ada and how they might be added in Ada X [seidevia91]. The approach I 

rook in this paper was to build the new objcct-oriented features of Ada 9X as much as possible on $he basic 

constructs and philosophy of Ada 83. The object-oriented fatuns proposed for Ada 9X IAda9X 91b1, while different 

in detail, are based on the same kind of approach. 

Further consideration of this approach led me on a long dlectbn on the nature of object-crknted programming 

and its application to Ada The results of this reflection, presented in this ppe?, show how a fairly natural objectoriented style can indeed be devclqcd even in Ada 83. The exacise of developing this style is useful for at least 

three reasons: 

1. It pvides a useful style for programming Object-Oritnted applications in Ada 83 until new features become 

available with Ada 9X: 

T Recently, I wrote a paper discussing the lack of "true" object-oriented 

2. It demystifies many of the mechanisms that seem to be "magic" in most object-oriented programming languages 

by making them explicic and 

3. It points out areas that are and are not in need of change in Ada 83 to make object-oriented programming more 

natural in Ada 9X. 

In the next four sections I will address in tm the issues of object-oriented classes. mixins. self-reference and 

supertyping. The presentation is through a sequence of examples, similar to those in [seidewirz911. This results in 

some overlap with that paper, but all the examples in the present paper are written entirely in Ada 83. I will retm 

to considerations for Ada 9X in the last section of the paper. 

An object represents a component of.. .[a] software system.. . An object consists of some private memory and a set of 

operatwns.. A crucial properly o an object is that its private 

describes the implementation of a set of objects that all 

represent the same kind of system component. 

[Goldbe= ud Robson 831 

In Ada, an object is a variable or a constant that contains a value. ?he declared type of the object determines 

the set of possible values for the object and the set of operations that may be applied to the object. If this type is a 

private type, then the value of the object may only be changed through application of an operation. This 

corresponds to the object-oriented notion of a cfars. 

memory can be manipulated o d y by its operatwns.. A class 

Consider, for example, a simple fmmial account class implemented as a private type: 

with Finance-Types; use Finance-Types; 

package Finance is 

type ACCOUNT is limited private; 

procedure Open (The-Account : h out ACCOUNT; 

With Balance : h MONEY) ; - 

6-3 

loO05788L 
procsdurr Deposit (Into-Account : ia out ACCOUNT; 

The-Amount : fn MONEY) ; 

procedure Withdraw (From-Account : h out ACCOUNT; 

The-Amount : in MONEY 1 ; 

function Balance-Of (The-Account : ACCOUNT) return MONEY; 

private 

type ACCOUNT is 

record 

and record; 

Balance : MONEY := 0.00; 

end Finance; 

'Iht type Finance .ACCOUNT npresents a class of BCCOunt objects. The subpgrams defined in package 

Finance are the allowable operations on objects of this class. 'Iht body of this paclrage is straightfonvard. Note 

that for simplicity I will assume that a number of Simple types (such as MONEY) are defined in a Finance-Types 

package. 

The class defd by package Finance ppovides a Simple but very genaal abstraction. In an objezt-oriented 

approach, such general classes ~fc used as the basis for implementing mort specialized classes. For example, a 

savings account is a specific kind of account that holds savings that earn intereSt Other than some new operations 

Bssociated with earning intenst, a savings account is the same as the original general financial account. Thus we 

should be able to implement a savings account in terms of a general account 

with Finance-Types: um Finance-Types; 

package Savings is 

type ACCOUNT is limited private; 

procedure Open (The-Account : b out ACCOUNT; 

With-Balance : in MONEY) ; 

procedure Set-Rate (Of-Account : out ACCOUNT; 

To-Ra t e :in RATE) ; 

procedure Deposit (Into-Account : in Out ACCOUNT; 

The-Amount : in MONEY) ; 

procedure Withdraw (From-Account : Out ACCOUNT; 

The-Amount : in MONEY) ; 

procedure Earn-Interest (On-Account : Out ACCOUNT; 

Over-Time :in INTERVAL) ; 

function Balance-Of (The-Account : ACCOUNT) return MONEY; 

Zunction Interest-On (The-Account : ACCOUNT) return MONEY; 

private 

type ACCOUNT i8 

record 

Parent : Finance.ACCOUNT; 

Rate : RATE := 0.06; 

Interest : MONEY := 0.00; 

end record; 

end Savings; 

While this may not seem to gain us a lot in this simple example, such incremental extension of abswctions is 

fundamental to object-oriented techniques. The class of financial accounts is said to be the superclaw of the class of 

savings accounts. Each savings account (of type Savings. ACCOUNT) has a unique pent financial account (of 

type Finance. ACCOUNT). 

Now, three of the seven savings account operations (Open, Deposit and Withdraw) are syntacbcally and 

semantically the same as the comsponding financial account operations. Thus, we wouid to inherit these 

financial account opaatims. Ada 83 has no direct way of doing this. Nevertheless. we can achieve the effect of 

inheritance far our present purposes by using call-though subprograms. For example. the Savings .Deposit 

6-4 
@on can easily be implemented as follow= 

procedure Deposit (Into-Account : in out ACCOUNT; 

The-Amount : in MONEY) is 

begin Finance.Deposit (Into-Account.Parent, The-Amount) 

end Deposit; 

The expense of such call-throughs may be minimized through the use of pragma Inline. 

Three other Savings Bccount OQeratiOns (Set-Rate, Earn Interest and Interest-On) provide the 

incremental new functionality of the savings account subclass. “‘Ea operations are implemented in terms of the 

additional components of the representation of type Savings .ACCOUNT. For example: 

procedure Earn-Interest (On-Account : out ACCOUNT; 

Over-The : in INTERVAL) is 

Balance : constant MONEY :- Balance-Of (On-Account); 

begin 

if Balance > 0.00 then 

On-Account.Interest := On-Account.Interest 

+ Balance*On-Account.Rate*Over-Time; 

end if; 

end Earn-Interest; 

Note that the Balance-Of operation used hen 

The remaining savings account operation. Balance Of, is syntactidy the same 85 the financial account 

operation, but it is semantically different. The balance 9 a savings account includes interest earned up to the 

present point in time: 

function Balance-Of (The-Account : ACCOUNT) return MONEY i8 

begin 

the subclass opedon Savings. Balance-Of. 

return Finance.Balance-Of (The-Account.Parent) 

+ The-Account . Interest; 

end Balance-Of; 

Note that while Balance Of is not a call-through operalion, the superclass operation Finance. Balance - Of is 

used in its implementation, 

The usefulness of a superclass like the financial account class comes from the fact that it can provide a common 

basis far a nwnber of subclasses. For example, a class of checking accounts may provide anotKer subclass of 

fmancial accounts: 

with Finance; 

with Finance-Types; use Finance-Types; 

package Checking is 

type ACCOUNT is limited private; 

procedure Open (The-Account : in out ACCOUNT; 

procedure Set-Fee (Of-Account : in out ACCOUNT; 

procedure Deposit (Into-Account : in out ACCOUNT; 

With-Balance : in MONEY) ; 

To-Fee :in MONEY) ; 

The-Amount : in MONEY) ; 

The-Amount : in MONEY) ; 

procedure Withdraw (From-Account : b out ACCOUNT; 

function Balance-Of (The-Account : ACCOUNT) return MONEY; 

private 

6-5 
tw ACCOUNT i8 

record 

Parent : Finance.ACCOU?JT; 

Overdraft-Fee : MONEY := 10.00; 

end record; 

end Checking; 

In this simple example, the only difference behvm checking accounts and financial aCcountS is that 

overdrawing a checking account is not permitted. Further, each overdraft auempt (Le, a reamed check) is penalized 

by deducting 8 fee fr" the munt Thus. the impk"tati0n of Withdraw muSt be changed for checking 

accounts: 

procedure Withdraw (From-Account : in out ACCOUNT; 

The-Amount : in MONEY) ia 

begin 

if The-Amount <= Balance-Of(From-Account) then 

else 

Finance.Withdraw (From-Account.Parent, The - Amount); 

Finance.Withdraw (From-Account.Parent, 

From-Account.Overdraft-Fee); 

end if; 

end Withdraw; 

The savings account and checking account subclasses of the fwial account class may themselves act as 

superclasses for even more specialized classes. Thus, a genual class may be the root of a quite extended class 

hierarchy. Each subclass in the hiaarchy incrementally extends the capabilities of its superclass, while inheriting 

common functionality. 

NS 

A mixin is..a subclass dcfintwn that may be applied to 

superclasses to create a related famiry qf modified 

[Bradu and Cook 901 

A superclass may be used as the base for many subclasses. However, as described so far, a subclass is tied to 

one superclass. For instance, savings accounts are based specifically on the class defined by package Finance. 

There may be other types of accounts to which we want to added interest-bearing functionality such as that defied 

for savings accounts. For example, an interest-bearing checking account is basically a checking account with 

interest-bearing functionality added to it (or, alternatively, a savings account with checking functionality added). 

Mer than recoding essentially the same intenst-bearing functionality each time it is needed, we can capture 

this functionality in a generic package that takes a specific superclass as a parameter: 

with Finance-Types; use Finance-Types; 

generic 

type SUPERCLASS is limited private; 

with function Balance-Of (The-Account : SUPERCLASS) return MONEY is <>; 

package Interest is 

type MIXIN is limited private; 

tp ACCOUNT is 

record 

Parent : SUPERCLASS; 

Extension : MIXIN; 

end record; 

procedure Set-Rate (Of-Account : in Out ACCOUNT; 

To-Rate :in RATE) ; 

prOCedUr8 Earn-Interest (On-Account : b Out ACCOUNT; 

Over-Time : ia INTERVAL) ; 

function Balance-Of (The-Account : ACCOUNT) return MONEY; 

6-6 
function Interest-On (The-Account : ACCOUNT) return MONEY; 

private 

type MIXIN is 

record 

Rate : RATE := 0.06; 

Interest : MONEY := 0.00; 

end record; 

end Interest; 

A generic @cage such as this is called a mixin because it provides an increment of functionality which may be 

“mixed-into“ any superclass that has the operations required to fill in the generic parameters. Typically, mixins are 

used within a framework of mulripk inhcrirancc. For exam~le, we can mnsmct the savings account class by 

inheriting from borh the financial account class and an appropriate instantiation of the interest mixin: 

with Finance, Interest; 

with Finance-Types; U80 Finance-Types; 

package Savings i8 

type ACCOUNT is limited private; 

procedure Open (The-Account : in out ACCOUNT; 

With-Balance : in MONEY) ; 

... 

function Interest-On (The-Account : ACCOUNT) return MONEY; 

private 

package Savings-Interest is 

type ACCOUNT is new Savings-Interest-ACCOUNT; 

new Interest (Finance.ACCOUNT, Finance-Balance-Of); 

end Savings; 

superclass Finance via a call-through. For example: 

procedure Open (The-Account : in out ACCOUNT; 

begin 

end Open; 

The record type Savings Interest .ACCOUNT is defined as a visible. rather than a private, type in the mixh to 

allow access to the Pare5 component. Note that it would not be possible to replace this use of a visible record 

component with a function that retums the parent object, because we need to use the parent as an in out 

parameter. The type MIXIN is never used itself outside of the mixin package. 

Call-through subprograms are also needed to inherit from the mixin instantiation Savings-Interest. This 

is because the equivalent derived subprograms obtained from the derived type definition of Savings. ACCOUNT 

are hidden by the operations declared in the package specification, and in Ada 83 there must be a full subprogram 

body for each of these declarations. For example: 

function Interest-On (The-Account : ACCOUNT) return MONEY is 

begin 

end Interest-On; 

functionality of the checking account class: 

The Parent component of the ACCOUNT type defined in mixin Interest is used to inherit from the parent 

With-Balance : in MONEY) is 

Finance.Open (The-Account.Parent, With-Balance); 

return Savings-1nterest.Interest-On 

(Savings-Interest.ACCOUNT(The-Account)); 

Having introduced the concept of mixins, we can. of coutse, also create a mixin that embodies the overdraft 

6-7 
with Finance-Types; use Finance-Types; 

generic 

type SUPERCLASS is limited private; 

with procedure Withdraw (From-Account : 

The-Amount : 

with function Balance-Of (The-Account : 

package Draft is 

type MIXIN is limited priv8te; 

type ACCOUNT is 

record 

Parent : SUPERCLASS; 

Extension : MIXIN; 

end record; 

out SUPERCLASS ; 

in MONEY) is <>; 

SUPERCLASS) return MONEY is <>; 

procedure Set-Fee (Of-Account : in out ACCOUNT; 

procedure Withdraw (From-Account : in out ACCOUNT; 

To-Fee :in MONEY) ; 

The-Amount : in MONEY) ; 

private 

type MIXIN is 

record 

end record; 

Overdraft-Fee : MONEY :- 10.00; 

end Draft; 

Even our original fm& account class can be wnvd to a mixin: 

with Finance-Types; use Finance-Types; 

generic 

type SUPERCLASS is limited private; 

package Monetary is 

type MIXIN is limited private; 

type ACCOUNT is 

record 

Parent : SUPERCLASS; 

Extension : MIXIN; 

end record: 

procedure Open (The-Account : in out ACCOUNT; 

With - Balance : in MONEY) ; 

1. 

function Balance-Of (The-Account : ACCOUNT) return 

private 

type MIXIN is 

record 

end record; 

Balance : MONEY := 0.00; 

end Monetary; 

MONEY; 

Of course, this mixin does not require any supedass functionality to implement its operations. However, use of the 

mixin construct allows monerary account functionality to be mixed into any class. 

The use of mixins causes fraditional class hiemhies to collapse into pieces. Each iece is a mixin that 

instantiating a number of mixins and inheriting all necessary functionality from them. To provide a definite starting 

provides a well-defined increment of functionality. We can then form specific classes LthescpMctsby 

6-8 
point for this process, we can defiie a root class that basically does nothing more than provide an empty record to 

which we can add mixins: 

package Root is 

type CLASS is limited private; 

private 

type CLASS is 

=cord 

end record; 

null; - 

end Root; 

While this root class seems a bit pointless, the concept will prove useful in the next section. 

At last we are ready to consmct an interest-bearing checking Bccount class without Fewriung any savings 

account or checking account functionality. To do this, we simply mix together interest, draft and monetary account 

functionality. All Interest Bearing Checking.ACCOUNT operations are imph"ed as call-throughs to 

various mixin operations. mus, f" fie thr# mixins Monetary, Interest and Draft, we can easily 

construct an interest-bearing checking account class, as well as reconseucting our original financial, savings and 

checking account classes. 

Of course. in the actual Interest Bearing-Checking package. the three mixin generics must be 

instan- in a specific sequential order. In the p"t case, we must first establish the basic monetary account 

functionality, then mix in interest and draft functionality. This results in the following implementation: 

with Root, Monetary, Interest, Draft; 

with Finance-Types; use Finance-Types; 

package Interest-Bearing-Checking is 

type ACCOUNT is lutd private; 

procedure Open (The-Account : in out ACCOUNT; 

With-Balance : in MONEY 1 ; 

... 

function Interest-On (The-Account : ACCOUNT) roturn MONEY; 

private 

package Checking-Finance is -- Basic financial account 

new Monetary (Root.Class); 

package Checking-Interest is -- Mix in interest functionality 

new Interest (Checking-Finance.ACCOUNT, Checking-Finance.Ba1ance-Of); 

procedure Withdraw (From-Account : in out Checking-1nterest.ACCOUNT; 

T he-Amoun t :in MONEY) ; 

-- call-through to Finance.Withdraw 

package Checking-Draft is -- Mix in overdraft fee functionality 

new Draft (Checking-Interest.ACCOUNT, 

Withdraw, 

Checking-1nterest.Balance-Of); 

type ACCOUNT is new Checking-Draft.ACCOUNT; 

-- Private type representation 

end Interest-Bearing-Checking; 

Note that all the mixins instantiated in the private part of the specification. Each instantiation uses the type and 

subprograms from the previous instantiation as arguments. The intermediate procedure Withdraw for type 

Checking 1nterest.ACCOUNT is necessary because the instantiated mixin Checking Interest Only 

provides the%terest-rehd operations on Checking-Interest .ACCOUNT. It is implemena as simply a callthrough to Finance. Withdraw. 

6-9 
When an object of a given class is created its state components 

include those of the class and all its superclasses and it can 

perform operations of the class and its superclasses on the 

component state. References to "serf' in operations of a 

superclass refer w the composite object on behalf of which the 

operation is w be performed. 

Wegner 811 

In the inmest-bearing checking Bccount package at the end of the last section. the Interest mixin was 

instantiatad before the Draft mixin. It would seem that we could equally well have instantiated them in the 

opposite der: 

with Root, Monetary, Interest, Draft; 

with Finance-Types; use Finance-Types; 

package Interest-Bearing-Checking i8 

type ACCOUNT ia limited private; 

procedure Open (The-Account : in out ACCOUNT; 

With-Balance : in MONEY) ; 

... 

function Interest-On (The-Account : ACCOUNT) return MONEY; 

private 

package Checking-Finance is 

p8ckage Checking-Draft ia -- Mix in overdraft fee functionality 

-- Basic financial account 

new Monetary (Root.Class); 

new Draft (Checking-Finance.ACCOUNT, Checking-Finance.Withdraw, 

Checking-Finance.Ba1ance-Of); 

function Balance-Of (The-Account : Checking-Draft.ACCOUT) 

-- call-through to Finance.Balance-Of 

package Checking-Interest is -- Mix in interest functionality 

type ACCOUNT is new Checking-1nterest.ACCOUNT; 

return MONEY; 

new Interest (Checking-Draft.ACCOUNT, Balance-Of); 

-- Private type representation 

end Interest-Bearing-Checking; 

Unfortunately. it tums out that this introduces a subtle mr, as follows: 

In the new implementation, the Draft mixin iS instantiated before the Interest mixin, using the 

Checking-Finance . Balance-Of operation. 

The implementation of the Withdraw operation in the Draft mixin uses the Balance Of operation given 

as a generic formal superclass operation to determine if there is an overdraft. 

subprogram used is Checking-Finance . Balance-Of, which does not add in any earned interest 

Checking Draft of the Draft mixin, so as to include the overdraft functionality. 

accumulatedinterest is ignored when checking for an overdraft. This is clearly unfair to the customer! 

The problem is that we do not dly want to use the superck7ss Balance-Of operation in the Draft mixin' 

instantiation. Ratha, we need to use the Balance-Of OperatiOn from the composite subclass being constructed. 

However, we cannot use the subclass type Interest Bearing Checking.ACCOUNT in the instantiation of 

the Draft mixin. becaw that type cannot be fully d&xd yet. nus, we must instead be sure to instantiate the 

Interest mixin first, so that the interest-bearing functidty is mixed into the Balance-Of operation before 

Draft is instantiated. 

Such orda dependencies arc at best annoying sources of potential errors. At worst, they can introduce circular 

dependencies that make it impossible to mix together certain mixiins. To avoid this, we nd a mechanism that 

allows mixins to call subclass operations in addition to superclass operations. Following the parameterization 

Vh that led us to mixins in the fit place, we can include a second generic fonnal type parameter in mixins to 

6-10 

Inihis case, the actual 

The Interest-Bearing-Checking. Withdraw operation is inherited from the instantiation 

This means that 

1ooo5786L 
represent the subclass. 

For example, we want the Draft mixin to use the subclass Balance-Of opedon: 

with Finance-Types; use Finance-Types; 

generic 

type SUPERCLASS is limited private; 

with procedure Withdraw (From-Account : in out SUPERCLASS; 

type SUBCLASS is limited private; 

with function Balance-Of(The-Account : SUBCLASS) return MONEY is <>; 

with function Self (Parent : SUPERCLASS) return SUBCLASS is <>; 

The-Amount : in MONEY) i8 <>; 

package Draft is 

type MIXIN is limited private; 

type ACCOUNT is 

record 

Parent : SUPERCLASS; 

Extension : MIXIN; 

end record; 

procedure Set-Fee (Of-Account : Out ACCOUNT; 

To-Fee :in MONEY) ; 

The-Amount : in MONEY 1 i 

procedure Withdraw(From-Account : b out ACCOUNT; 

function Self (This-Account : ACCOUNT) miiturn SUBCLASS; 

private 

type MIXIN is 

record 

end record; 

Overdraft-Fee : MONEY := 10.00; 

end Draft; 

The Withdraw operation for this mixin is then implemented as follows: 

procedure Withdraw (From-Account : in out ACCOUNT; 

The-Amount : MONEY) is 

bagin 

if The-Amount <= Balance-Of(Self(Froxn-Amount)) then 

else 

Finance-Withdraw (From-Account.Parent, The-Amount); 

Finance-Withdraw (From-Account.Parent, 

From-Account.Extension.Overdraft-Fee); 

end if; 

end Withdraw ; 

Note the use of the function Self to convert an object of type Draft. ACCOUNT to the appropriate object of type 

SUBCLASS. These odd little Self functions are the key to this approach. They allow us to use the subclass 

The question is, of course, how can we implement such a Self function? Strangely enough, we can implement 

operations as required. 

it in teams of the superclass Self function given as a genaic fmal parameter. 

function Self (This-Account : ACCOUNT) return SUBCLASS is 

bwia 

end Self; 

return Self (This-Account.Parent); 

6-11 

tOoo578sL 
Obviously. this passing of the buck must end somepke. it ends with the root class. which we reimplement as 

follows: 

genezic 

type SUBCLASS is limited private; 

package Root is 

type CLASS is limited private; 

procedure Initialize (The-Object : in OutCLASS; 

function Self (This-Object : CLASS) return SUBCLASS; 

To-Self : in SUBCLASS); 

private 

type CLASS is 

record 

end record; 

Self : SUBCLASS; 

end Root; 

Thus the mystery is resolved: the Self functions all ultimately access a Self component defmed in the root class. 

Now, the astute nadn may have noticed that wc have inaoduced a mange sort of circularity here. The 

representation of any class built on the root class will include a compoimt a€ the subclass type. However, when we 

finish constructing a class from the mot class and mixinn, the “it is the very subclass with which we need to 

instantiate the root class to begin with! To achieve this circularity, we must require that the subclass type be an ~ccess type. ’Iht Self component is then intended to be a infer back to the complete, composite subclass object. 

would othawise have in out parametas.) 

account class using either order of mixin instantiation: 

with Finance-Types; use Finance-Types; 

package Interest-Bearing-Checking is 

type ACCOUNT is limited private; 

procedure Open (The-Account : in out ACCOUNT; 

procedure Close (The-Account : in out ACCOUNT); 

procedure Set-Rate (Of-Account : in ACCOUNT : 

(Actually, axas types arc also accdcd to allow the Self & tions to work propaly with subclass procedures that 

With inclusion of subclass parameters in mixins, we can now conectly implement the interest-bearing checking 

With-Balance : in MONEY) ; 

To-Rate : in RATE) : 

. .. 

private 

type ACCOUNT-FtECORD; 

type ACCOUNT is access ACCOUNT-RECORD; 

end Interest-Bearing-Checking: 

An advantage of implementing a private type as an access type is that the derails of the type representation can be 

deferred to the package body by Using an incomplete type def~tion for ACCOUNT-RECORD in the private part of 

the specification. The use of an access type also allows the use of in rather than in out parameters in procedures 

such as Set-Rate, which is necessary for the use of Self functions. 

defmition is also achieved using the incomplete type definition for ACCOUNT-RECORD. The 

circle circ%Jr is cl y compkting the definition of ACCOUNT-RECORD after all the nlixin instantiations in the package 

body. ’Iht figure on the next page shows the structure of an Interest-Bearing-Checking .ACCOUNT object 

resulting from the following implementation: 

6-12 

1oo05188L 
An-Account : Interest-Bearing-Checking.ACCOUNT 

I 

Checking-DrafLACCOUNT 

C hecking-Finance.ACCOUNT 

I 1 

Balance 

Rate :0.06 

Interest :o.oo 

i 

with Root, Monetary, Interest, Draft; 

package body Interest-Bearing-Checking is 

package Checking-Root is 

we Checking-Root; 

package Checking-Finance is 

new Root (SUBCLASS => Interest - Bearing-Checking.ACCOUNT1; 

new Monetary 

(SUPERCLASS => Checking-Root.CLASS, 

SUBCLASS => Interest-Bearing-Checking-ACCOUNT); 

use Checking-Finance; 

package Checking-Draft is 

new Draft 

(SUPERCLASS -> Checking-Finance.ACCOUNT, 

SUBCLASS => Interest-Bearing-Checking.ACCOUNT); 

function Balance-Of (The-Account : in Checking-Draft.ACCOUNT) 

-- call-through to Finance.Balance-Of 

use Checking-Draft; 

package Checking-Interest is 

return MONEY; 

new Interest 

(SUPERCLASS => Checking Draft.ACCOUNT, 

SUBCLASS => Interest-Bearing-Checking.ACCOUNT); - 

type ACCOUNT - RECORD is new Checking-1nterest.ACCOUNT; 

... 

end Interest-Bearing-Checking; 

(Note that to simplify the instantiations, I have taken advantage of the box defaults on the generic formal 

subprogram parameters of the mixins.) 

A disadvantage of using an access type is that interest-bearing checking accounts must be explicitly allocated. 

We can do this as part of the Open operation: 

6-13 
procedure open (The-Account : ha out ACCOUNT; 

With-Balance : h MONEY) 58 

-gh 

if The-Account /= null then 

end if; 

Close (The-Account 1 ; 

The-Account :- new ACCOUNT-RECORD; 

Checking-Root.Initialize 

Checkhg-Finance.Open (The-Account.Parent.Parent, With-Balance) ; 

(The-Object => The-Account.Parent.Parent.Parent, 

To Self => The-Account); 

end Open; 

Note the use of the root Initialize omon to set the Self component. The figure on the previous page 

shows the structure of nestad rccords and self reference that results from the allocation and initialization of an 

Interest-Bearing-Checking.ACCOUNTObjCCt 

We also need to provide a way to deallocate intenst-bearing checking accounts: 

procedure Free is new Unchecked-Deallocation (ACCOUNT-RECORD, ACCOUNT); 

procedure Close (The-Account : in out ACCOUNT) i8 

end Close; 

begin 

Free (The-Account) ; 

All the rest of the intaest-bearing checking BCcOunt opaations are Mtcd from OM ar the other of the mixin 

instantiations. 

Sub ing is a substitutability relationship, i.e., an instance of 

the subtype is implemented is totally irrelevant; all that 

matters is that it have the right behavior so that it can be 

substituted. 

a s 2 type can stand in for an instance of its supertype. How 

[Uonde and Pugh 911 

Typically, the customex of a bank will have several accounts at that bank. Each bmJc account may be, say, a 

savings account, a checking account or an interest-bearing checking account. To manage all the bank accounts of 

one customer, we would like to create a bank munt type that is the supertype of the types that represent the 

various classes of accounts. We could then mate lists of bank accounts. define bank account operations. etc. 

As discussed in the previous scctiOns, each class is implemented in Ada by a private type that is distinct from all 

other class types. Nevertheless, we can still explicitly create a bank account supertype: 

with Savings, Checking, Interest-Bearing-Checking; 

with Finance-Types; uae Finance-Types; 

package Bank is 

type ACCOUNT-TYPE is (SAVINGS, CHECKING, INTEREST-CHECKING); 

type ACCOUNT (Kind : ACCOUNT-TYPE :- SAVINGS) is 

record 

case Kind is 

when SAVINGS => A-Savings-Account : Savings.ACCOUNT; 

when CHECKING -> A Checking-Account : Checking-ACCOUNT; 

when INTEREST-CHECKING => G-Interest-Checking-Account 

: Interest-Bearing-Checking.ACCOUNT; 

end caae; 

end record; 

procedure Open (The-Account : in out ACCOUNT; 

procedure Close (The-Account : out ACCOUNT) ; 

With-Balance : in MONEY) ; 

6-14 

1ooo5788L 
procedure Deposit (Into-Account : in out ACCOUNT; 

The-Amount : in MONEY) ; 

The-Amount : in MONEY) ; 

procedure Withdraw (From-Account : in out ACCOUNT; 

function Balance-Of (The-Account : ACCOUNT) return MONEY; 

end Bank; 

The type Bank-ACCOUNT defines a Supatype with sublypes Bank.ACCOUNT(SAVINGS), 

Bank.ACCOUNT(CHECKING)Bnd Bank.ACCOUNT(INTEREST-CHECKING). Each Subtype correspondst0 

one of the classes defined in previous sections. Note that a private type is unnecessary here, because we wish to be 

able to freely convert between the Bank. ACCOUNT subtypes and the class types. 

The five opedons Mined in package Bank refkt the opedons that are common to all the BCCOunt types. 

Semantically, we wish each supertype operation to "X the implanentation of the epproPriate subtype operation. 

For example, the statement: 

Bank. Withdraw (From-Account => A, The-Amount -> X) ; 

should be equivalent to either Savings .Withdraw, Checking-Withdraw or 

Interest Bearing Checking. Withdraw, -g on the subtype of A. Since the subtype of A can, in 

general, onlybe determkd at run-hme. we an? effecuvely asking that Bank. Withdraw be dynumicdly bound to 

the appropriate subtype operation. 

We can achieve the effect of dynamic binding in Ada by implementing the bank account operations as 

dispatching or case-selection subprograms. For example: 

procedure Withdraw (From-Account : in out ACCOUNT; 

The-Amount : in MONEY) i8 

begin 

case Kind i8 

when SAVINGS => 

Savings.Withdraw (From-Account-A-Finance-Account, The-Amount); 

Checking.Withdraw (From-Acc0unt.A-Checking-Account, The - Amount); 

Interest-Bearing-Checking.Withdraw 

when CEECKING => 

when INTEREST-CHECKING -> 

(From-Account.An-Interest-Checking-Account, The-Amount); 

end case; 

end Withdraw; 

handle all kinds of bank 8ccounts. For example: 

Once we have the bank account supertype. we can create polymorphic data structures and operations that can 

type CUSTOMER-ACCOUNTS is array(POSIT1VE range <>) Of Bank.ACCOUNT; 

function Total-Assets-Of (The-Accounts : CUSTOMER-ACCOUNTS) return MONEY is 

Total : MONEY := 0.00; 

begin 

for I in The-Accounts'range loop 

end loop; 

Total := Total + Bank.Balance-Of (The-Accounts(1)); 

return Total; 

end Total-Assets-Of; 

The function defmed above finds the total assets a customer has in his aocwnts. regardless of what kinds of 

accounts they arc. 

6-15 
It is important to note that to be included in a supatypc. a class need only provide implementations for all the 

operations defined for the supmypc. The ways in which various subtype classes implement these opemtions do not 

have to be related a! all. For example, the Bank. ACCOUNT supertype is Constructed from a number of classes 

implemented by various combinations of the mixins Monetary, Interest and Draft. These classes thus share 

some common implementation, but this is not at all important to the construction of the supertype. 

Thus. supertypes and suplasses are really distinct concepts. Looking at it another way, the supertype 

provides a set of dispatching operations for those opemuions which am common to all its subtypes, regardless of how 

those operations may be implemented by the subtype. classes or how the subtypes may be repmented. A supertype that is constructed in this way from a given list of subtype classes is said to be the union typc of those classes. Thus 

we have cmsmted a bank account supextype that is the union of the savings, checking and interest-bearing 

checking account classes. 

It was noted earlier that the use of mixins causes a collapse of the original class hierarchy. Using union types, 

however, we can still form a type hierarchy by appropriately grouphg classes. As well as the Bank. ACCOUNT 

union type, such a rype hierarchy for account classes could include the union of the savings and interest-bearing 

checking account classes (an investment supertype Ecathg interest-bearing checking accounts as savings accounts) 

and the union of the checking and interest-bearing checking 8CCOunt classes (a cash aCCOunt supertype treating 

interest-bearing checking accounts as checking accounts). Note how it is possible for a class to be included in more 

than one union type. 

ATlONS FOR ADA 9X 

There is a recognized need for improving Adas support for 

data obszraction, and the construction of progrimasfrom preexisting components. 

[A&9X 9181 

The mixin-based style described in this papex combines the bcnefifs of objectsiented mixins with the 

advantages of explicit parmeterhation through generics. With superclass and subclass parameterization, mixins are 

completely independent software components that can be mixed and matched in many combinations. This leads to a 

powerful paradigm known as parameterized programming that promotes highly reusable code (see, for example, 

Unfortunately, as the reader can see from the examples in this paper, this style is awkward in places with 

Ada 83. In particular. the following areas especially need to be addressed in Ada 9X: 

1. There needs to be a way to create a subclass type by simple extension of a class type and to parameterize this extension with a mixin. The proposed Ada 9X record extension mechanism [A&9X91bl fills this need admirably 

well. 

[Gogum W Seidewitz md St.k 911). 

2. There needs to be a simpler way to achieve self-reference during the combination of mixins. This need seems 

to be filled by the proposed mechanism in Ada 9X to allow type. extensions as generic formal type parameters 

IAdr9X91bl. This would probably necessitate the use of nested generics to allow the mixin type to be an 

extension of the SUPERCLASS type parameter and the SUBCLASS type panmeter to be an extension of the 

mixin type. Such a construction would, however, eliminate the need for Self functions. 

3. There needs to be a mechanism for consaucting supert~ without having to explicitly code watch 

operations. Ada 9X does provide an automatic dispatchmg capability using "tagged records" [~&9~91b]. 

However, this capability can only be used if the subtypes are implemented as subclasses (type extensions) of the 

supertype. This perpetuates the confusion of superclass and supertype. 

Thus the proposed object-oriented features for Ada 9X largely suppart the mixin style described in this paper. 

Unfortunately, the tagged record mechanism confuses type extension and dispatching. This is analogous to the 

equation of superclasses and supertypes in most typed object-oriented programming languages (such as Ci+ 

[Smwutrup 861). 

Requiting supertypes to be superclasses is inconvenient when we are using generic mixins to construct classes. 

and wish to create a type hierarchy after the fact. perhaps a better model far Ada 9X would be the "absmct type" 

mechanism of the languages Emerald [Black u .I 871 and POOL-I [heriu md WII der Linden 901. Even With the currently proposed Ada 9X features, however, a generics-based approach to mixins, such as that presented in this paper, could 

be an important contribution of Ada 9X back to the object-oriented programming community. 

T 

I would like to thank my colleague Mike StarL for a number of good suggestions that greatly improved the 

clarity of prtsentation of this paper. 

6-16 

loo0578BL 
Ada9X 91a DRAFT Mapping Ratwnale Document, Ada 9X Project Report, February 

1991 

Ada9X 91b Ada 9X Mapping Document, Draft Ada 9X Project Report (2 volumes). 

August 1991 

America and van der Linden 90 Pierre America and Franlr van der Linden, "A Parallel Object-Oriented 

Language with Inheritance and Subtyping". Proceedings of the Conference 

on Object-Oriented Programming System, Languages, and Applications 1 

European Conference on Object-Oriented Programming, SIGPLAN 

Notices, October 1990 

Black et al. 87 

Bracha and Cook 90 

Goguen 84 

Goldberg and Robson 83 

Lalonde and Pugh 91 

Rentsch 82 

Seidewitz 91 

Seidewirz and Stark 91 

Stroustrup 86 

Wegner 87 

Andrew Black, Noman Hutchinson, Eric Jul, Henry Levy and Larry Carter, 

"Disaibution and Abslr8ct Types in Emerald", IEEE Transactions on 

Sofnvorc Engineering, January 1987 

Gilad Bracha and William Cook, "Mixin-Based Inheritance", Proceedings 

of the Conference on Object-Oriented Programming System, Languages, 

and Applications I European Conference on Object-Oriented Programming, 

SIGPLAN Notices, Octobex 1990 

Joseph A. Goguen, "Paramettrized Programming". IEEE Transactions on 

Sofware Engineering, September 1984 

Adele Goldberg and David Robson, Smalltalk#: The Language and its 

Implementation, Addison-Wesley, 1983 

Wilf LaLonde and John Pugh. "Subclassing # Subtyping # Is-a", Journal of 

Object-Orien&d Programming, January 1991 

"Object~nted hgra"ing", SICPUN Notices, September 1982 

Ed Seidewia, "Object-Oriented Programming through Type Extension in 

Ada 9X", Ada Lctters, March/Apd 1991 

Ed Seidewitz and Mike Stark "An Object-Oriented Approach to 

Parameterized Software in Ada". Proceedings of the Eighth Washington 

Ada Symposium, June 1991 

Bjarne Strousuup. The C++ Programming Language, Addison-Wesley, 

1986 

Peter Wegner, "The Object-Oriented Classification Paradigm", in Research 

Directions in Object-Oriented Programming, ed. by Bruce Shriver and 

Peter Wegner. The MIT Press. 1987 

6-17 

1"L 

/x y/ 

SOFIWARE ENGINEERING LABORATORY 

ADA PERFORMANCE STUDY-RESULTS AND IMPJJCATIONS 

Eric W. Booth 

Computer Sciences Corporation 

Lanham-Seabrook, Maryland 

(301) 794-1277 

r. * * 7 .a 

v93)- 4, i;2.’3 

Michael E. Stark 

NASAKioddard Space Flight Center 

Greenbelt, Maryland 

(301) 286-5048 

SUMMARY 

The Ada Language Reference Manual (LRM) (Reference 1) states: 

“Ada was designed with three ovemding concerns: program reliability and maintenance, 

programming as a human activity, and efficiency.” 

Initial implementations of Ada compilers and development environments tended to favor the first two 

concerns over the concern for efficiency. Similarly, initial (non-real-time, nonembedded) applications 

development using Ada as the programming language tended to favor maintainability, readability, and 

reusability. 

As software systems become more sophisticated the need to predict, measure, and control the run time 

performance of systems in the Flight Dynamics Division (FDD) is a growing concern. The transition to 

Ada introduces performance issues that were previously nonexistent. More-over, this transition is often 

accompanied by the transition to object-oriented development (OOD), which has performance implications, 

independent of the programming language, that must be considered. To better understand the implications 

of new design and implementation approaches, the Software Engineering Laboratory (SEL) conducted an 

Ada performance study. 

The SEL is an organization sponsored by the National Aeronautics and Space AdministratiodGoddard 

Space Right Center (NASNGSFC) to investigate the effectiveness of software engineering technologies 

applied to the development of applications software. The SEL was created in 1977 and has three . , organizational members: NASNGSFC, Systems Development Branch; The University of Maryland, 

Computer Sciences Department; and Computer Sciences Corporation, Systems Development Operation. 

The goals of the SEL are (1) to understand the software development process in the GSFC 

environments; (2) to measure the effect of various methodologies, tools, and models on this process; and 

(3) to identify and then to apply successful development practices. The activities, findings, and 

recommendations of the SEL are recorded in the Software Engineering Laboratory Series, a continuing 

series of reports that includes the Ada Pelfonnance Study Report (Reference 2). 

This paper describes the background of Ada in the FDD, the objectives and scope of the Ada 

Performance Study, the measurement approach used, the performance tests performed, the major test 

results, and the implications for future FDD Ada development efforts. 
APPROACH TO MEASUREMENT 

To measure the run-time performance of design altematives and language features, two fundamental 

approaches were used. The fust approach measured the run-time improvement of existing systems after 

an alternative had been incorporated into a baseline version of the system. The second approach used the 

ACM SIGAda PIWG test suite and added tests specific to the flight dynamics environment. 

Overview 

Benchmark programs are commonly used to evaluate the performance of design alternatives and 

language features. Such benchmark programs include (1) sample applications such as sorting programs 

or, as in the FDD, simulators, (2) programs to measure the overhead associated with a design alternative 

or language feature, and (3) synthetic benchmarks designed to measure the time needed to execute a 

representative mix of statements (e.g., Whetstone, Dhrystone) (Reference 6). The fust approach used by 

this study falls into the first benchmark category, and the second approach falls into the last two. 

To measure the overhead of a design altemative or language feature, the dual-loop approach is used to 

subtract the overhead associated with control statements that aid in performing the measurement. This 

approach uses a control loop and a test loop; the test loop contains everything contained in the control loop 

and the alternative being measured. A major factor in designing a dual-loop benchmark is compiler 

optimization. It is critical that the code generated by the compiler for both loops be identical except for the 

quantity being measured (Reference 7). In addition, it is necessary to ensure that the statement or 

sequence of statements being tested does not get optimized away. 

Although the dual-loop approach can be used for synthetic benchmarks and applications, this technique 

is not required if the run time of the program is long in comparison to the system clock resolution 

(Reference 7). Instead, the CPU time can be sampled at the beginning of the program and again after a 

number of iterations of the program. The time for the benchmarklapplication is then (CPU-Stop - 

CPU-Start)/Number-Iterations. The same measurement can be achieved by submitting the test program to 

run as a batch job and obtaining the CPU time from the batch log file. This CPU time can then be divided 

by the number of times the sequence of statements being measured is executed in the main control loop of 

the test program. 

It is important to understand the run-time environment in which the benchmarks are run when 

interpreting test results. VMS checks the timer queues once per second, which can affect measurement 

accuracy. Under VMS, the Ada run-time system is bundled with the release of the operating system and 

installed as a shareable executable image. Consequently, DEC Ada performance is directly dependent on 

the installed version of VMS. There is also a degree of uncertainty when using CPU timers provided in 

time-shared systems like VMS. In the presence of other jobs, CPU timers charge ticks to the running 

process when the wall clock is updated. It is therefore possible for time to be charged to active processes 

inaccurately because context switches can occur at any time. Finally, it cannot be assumed that running 

benchmarks for a hosted system in batch during low usage (such as, at 11 pm) guarantees standalone 

conditions (References 7 and 8). Therefore, the FD benchmarks to test individual design altematives were 

run on the weekend to minimize these effects. 

6-20 
THE FIRST APPROACH -- SIMULATOR 

Several of the design alternatives examined by this study were tested and analyzed in the context of 

two FDD simulators. Alternatives were chosen to be implemented in the context of these simulators for 

the following reasons: 

1. They were simulator-specific, e.g., diffemnt ways of implementing the scheduler. 

2. They could be implemented in an isolated part of the simulator where their impact could easily be 

measured using the VAX PCA. 

3. They could be implemented in an isolated part of the simulator and still have a measurable effect on 

the time required for a 20-minute simulation run. 

Baselined versions of the simulators were used to test each of the design alternatives. CPU times were 

obtained for 20-minute simulation runs of the baselined versions from the log files created by batch runs. 

PCA was used to obtain a profile of the simulators. These pmfdes showed what percentage of the CPU 

time was spent in each Ada package of the simulator. The VAX manual Guide to VAX Pelformance and 

Coverage Analyzer (Reference 9) contains more infomation on PCA. 

Design alternatives were incorporated into the baselined versions of the simulators. New CPU times 

were obtained for 20-minute simulation runs from the log files created by batch runs and new profiles 

obtained using PCA. The following two figures show the accounting infomation contained in a batch log 

file and a sample of PCA output. From these two pieces of information, the impact of each design 

alternative was assessed. 

Sample PCA Output 

VAX Performance and Coverage Analyzer 

CPU Sampling Data (11219 data points total) - .*" 

Bucket Name +----+----+----+----+----+----+----+----+----+---- + 

PROGRAM-ADDRESS\ I 

UTILITIES- . . . . I************************************************** 

SEARCH-STRING . . I**************** 

SIMULATION-SCHEDULEI*********************** 

SPACECRAFT-ATTITUDEI**************** 

DATABASE-MANAGER . ADDING-UTILITIES . EARTH-SENSOR . . . UTILITIES-LONG- . DATABASE-TYPES- . SPACECRAFT-WHEELS 

AOCS-PROCESSOR . . SPACECRAFT-EPHEMERI 

ENVIRONMENTAL-TORQU 

*************** 

********** 

******** 

****** 

****** 

****** 

****** 

***** 

***** 

THRUSTERS I ***** . . . . GEOMAGNETIC-FIELD I***** 

DEBUG-COLLECTOR . I   

MAGNETOMETER . . . I ** ** 

SADA I *** . . . . . . . SOLAR-SYSTEM . . . I *** 

+----+----+----+----+----+----+----+----+----+---- + 

23.2% 

10.7% 

7.5% 

7.5% 

7.08 

4.7% 

3.7% 

3.0% 

2.9% 

2.9% 

2.6% 

2.5% 

2.3% 

2.2% 

2.2% 

2.1% 

1.7% 

1.4% 

1.2% 

6-21 
Sample Batch Log File Accounting Information 

Accounting information: 

Buffered 1/0 count: 109 Peak working set size: 4096 

Direct 1/0 count: 1132 Peak page file size: 15304 

Charged CPU time: 0 00:06:45.08 Elapsed time: 0 00:09:02.47 

Page faults: 11766 Mounted volumes: 0 

THE SECOND APPROACH -- PIWG 

Design alternatives not isolated to a particular part of either of the simulators were tested using the 

PWG structure of measurements. The PIWG structure of measurements is based on the concept of a 

control loop and a test loop. The test loop contains everything in the control loop and one alternative to be 

measured. The CPU time is sampled before the execution of each loop and after many iterations of each 

loop. If the test loop time duration is not considered stable, the process is repeated with a greater number 

of iterations; this is accomplished through an outer loop surrounding the test and control loops. To be 

considered stable, the test loop time duration must be greater than a predefmed minimum time. If this 

condition is met, the test loop time duration is compared against the control loop time duration, and the 

number of iterations is compared against a predefined minimum number of iterations. If the test loop time 

is greater than the control loop time or the minimum number of iterations has been exceeded, the results are 

considered stable, and the CPU time for the design alternative is calculated. The time for the alternative is 

the difference between the amount of CPU time taken for the control loop and the amount of CPU time 

taken for the test loop, divided by the total number of iterations performed. Collecting control loop and 

test loop CPU times, calculating design altemative times, and testing for stability were done using PIWG's 

Iteration package in the test drivers for this study. 

All test drivers used in this study were called three times from a main driver routine so that the CPU 

time for a given design alternative could be averaged for more accuracy. All results were averaged and 

recorded using PIWG's YO package and report generator procedure. The following is a sample PWG 

report. 

Sample PIWG Report 

Test Name: Generic-A 

CPU Time: 117.2 microseconds 

Wall Time: 117.2 microseconds 

Test Description: 

Use of generic matrix processing - Generic package for 3x3 matrix 

Test Name: Generic-C 

CPU Time: 117.2 microseconds 

Wall Time: 117.2 microseconds 

Class Name: Matrix - Gen 

Iteration Count: 128 

Number of samples: 3 

Class Name: Matrix - Gen 

Iteration Count: 128 

Number of samples: 3 

Test Description: 

use of generic matrix processing - NonGeneric package for 3x3 matrix 

6-22 
TESTOVERVIEW 

Ten test groups were developed. Each test group represented a design or implementation issue 

relevant to current FDD applications. The test groups were chosen as a result of an in-depth analysis of 

several FCA runs with two FDD simulators. If a certain design alternative or language feature appeared to 

consume a relatively large portion of central processing unit (CPU) time or memory, it was analyzed, 

measured, and quantified in this study. The design alternatives or language features consuming a 

relatively small portion of CPU time or memory were not studied further. Therefore, the test groups 

presented here are intended to be a representative sampling, rather than an exhaustive sampling, of current 

design and implementation approaches. The test groups are presented in two categories: design-oriented 

tests and implementation-oriented tests. 

Design-Oriented Tests 

Following is a brief description of the purpose of each design test group performed on the Ada 

performance study. 

Group I: Scheduling. This test group contained three tests that addressed the run-time cost of various 

scheduling altematives. This test compared a event-driven design against a time-driven design and a hardcoded design. The event-driven design maintains a prioritized (sorted) queue of event identifiers that 

specifies the time-step and next simulation event. The time-driven design iterates over an array of event 

identifiers for each fmed time-step. The hard-coded design contains the event (procedure) calls in the 

source code. With the event-driven design the user may vary the order and frequency of each event. In 

the time-driven design the user may only vary the order of the event. In the hard-coded design there are 

not options available to the user. The implications of the different approaches were analyzed and 

contrasted. The results of this test group provided the applications designers with information necessary 

to make trade-off decisions among flexibility, accuracy, and performance. 

Group 2: Unconstrained Structures. Leaving data structures unconstrained allows greater user 

flexibility and enhances future reusability. However, the additional run-time code that may be generated 

can impose a significant run-time and memory overhead. This group measured the expense of 

unconstrained records and arrays and proposed viable altematives. 

Group 3: Initialization and Elaboration. This test group addressed initialization of static and dynamic 

data using various combinations of elaboration-time and execution-time altematives. This test group was 

relevant for applications requiring minimal initialization time. 

Group 4: Generic Units. The benefits of using generic units are reduced source-code size, 

encapsulation, information hiding, decoupling, and increased reuse (Reference 10). However, many Ada 

compilers implement this language feam poorly. This test group addressed the options available with the 

compiler implementation and how well these options were implemented. 

Group 5: Conditional Compilation. The ability to include additional "debug code" in the delivered 

system adds to the system size and imposes a run-time penalty even if it is never used. The test group 

analyzed the current approach and proposed flexible alternatives for future systems. The results of this test 

group can have applications beyond "debug code" elimination. 

Group 6: Object-Oriented Programming. Two of the fundamental principles of object-oriented 

programming (OOP) are polymorphism and inheritance. Ada does not directly support these principles. 

However, the designer may simulate the effect of inheritance and polymorphism through the use of variant 

6-23 

loo05788L 
records and enumeration types. These OOP principles, whether direct or indirect, incur certain run-time 

overhead and problems (Reference 11). 

Implementation-Oriented Tests 

Following is a brief description of the purpose of each implementation test group performed on the 

Ada performance study. 

Group 7: Matrix Storage. The most basic, and perhaps the most common, mathematical expressions 

in flight dynamics applications involve matrix manipulations. This group addressed row-major versus 

column-major algorithms to quantify the performance implications. 

Group 8: Logical Operators. The Ada LRM clearly defines the behavior of logical expression 

evaluation. The Ada Style Guide (Reference 12) recommends avoiding the short-circuit forms of logical 

operators for performance reasons. The implications of this recommendation in the flight dynamics 

environment were measured and analyzed. 

Group 9: Prugma Inline. Flight dynamics simulators contain a large number of procedure and 

function calls to simple call-throughs and selectors. The overhead of making these calls can slow the 

performance of any simulator. This test measured the use of pragma INLINE as an alternative to calling 

a routine. 

Group IO: Smng-to-Enumerution Conversion. Current flight dynamics simulators contain a central 

logical data base. The physical data are distributed throughout the simulator in the appropriate packages. The logical data base provides keys (strings) that map into the physical data. The logical data base 

converts these strings to the appropriate enumeration type to retrieve the corresponding data. This test 

assessed the performance implications of this approach. 

Test Documentation 

Each performance test in this report is described in this section in the following format: 

Purpose. Each test was designed with a specific design or implementation alternative in mind. The 

rationale for the choice of the alternatives tested results from analysis of existing Ada systems developed in 

the FDD. 

Method. Some tests were performed as changes to an existing system, while other tests were 

performed by creating new, special-purpose software. The basis for each method was one of two 

approaches: DEC's FCA measurement tool or the PIWG structure of measurements. The details of the 

method(s) used for each test are described. 

Results. The result of executing a test is some combination of CPU time and object code size. Most 

tests were designed to measure the CPU run time in microseconds (p). In some cases the object code 

size in bytes is relevant. The data resulting from each test run are provided. 

Analysis. In many cases, detailed analysis of the test results is necessary to understand'the 

implications for future projects. The analysis performed is summarized, and the implications are 

highlighted. 

6-24 
RESULTS 

1. Looping scheduler 

2. Bypass logical data base 

3. Conditional compile debug code 

4. Use static data structures 

As a result of this performance study, more accurate estimation of run-time performance for future 

FDD simulators is possible. Assuming future dynamics simulators are similar in function to GOADA, a 

more accurate perfomance estimation is possible given the following information: 

1. The run-time performance for a typical run of the GOADA simulator is 6 minutes, 45 seconds for a 

20-minute simulation. This yields a 1:3 simulation time to real-time ratio. 

10.7% 2.2% 8.5% 

14.5% 1.8% 12.7% 

2.1% 0.0% 2.1% 

45.0% 13.0% 32.0% 

2. The performance proffle generated by PCA of a typical GOADA run shows the distribution of the 

CPU run time resource throughout the simulator. 

3. The measured results of this study that lead to more efficient design and implementation 

alternatives. 

The following table combines the results of the Ada Performance Study with the PCA performance 

profile of GOADA. Each row of the table is measured against the baseline of 6 minutes and 45 seconds of 

CPU time to perform a 20 minuter simulation. 

Impact of Measured Performance Results on Dynamics Simulators 

Altemative I GOADA I Study Results I Difference 1 

5. Optimized utility packages I 26.6% I 5.3% I 21.3% I 

Total Percentages I 98.9% I 22.3% I 76.6% I 

The first row of the table shows the performance difference between the baseline scheduler in GOADA 

and the looping scheduler alternative (see test group 1, scheduling). Another option is to use the “hardcoded“ approach for the scheduler. However, the hard-coded approach sacrifices all flexibility in the 

interest of performance. For this reason, the more flexible “looping” alternative is recommended. 

The second row highlights the difference between accessing the logical data base and accessing the 

physical data directly (see test group 10, string- to-enumeration conversion). This striking improvement 

came from removing one string-toenumeration type conversion from the main simulation loop. The third 

row recommends the conditionally compiled debug code (see test group 5, conditional compilation). The 

fourth row is the estimated result of using a static record structure instead of a dynamic structure in all 

simulator packages (see test group 2, unconstrained structures). 

The fifth row is based on the result of comparing GOADA’s baseline matrix multiply function to the 

optimized matrix multiply function (Reference 13). Since the FDD deals with mainly three dimensions, an 

optimized set of utilities can be developed on that basis. The fully optimized version required less than 

one-fifth of the CPU time required for the baseline version. 

6-25 
As this table shows, a dynamics simulator that is similar to GOADA and is implemented with the 

results of this study would consume 76.6 percent less CPU than the current version, more than 

quadrupling the speed. This would yield an upper bound estimate of 95 seconds to perform a 20-minute 

simulation run, or approximately a 1 : 13 simulation-time-to-real-time ratio. 

This estimate is an upper bound for three reasons. First, this study examined a representative, rather 

than an exhaustive, list of design and implementation alternatives. That is, only those altematives that held 

the most promise of a large performance difference were studied. There may be many other alternatives 

that offer only minor gains. However, the combined performance gain of all may be significant. 

Second, coding optimizations to GOADA, or any simulator, were not studied. The goal of the study 

was to identify those design and implementation alternatives that lead to optimal systems. Line-by-line 

micro-optimizations on a simulator only provide information on final efficiency and lack the needed 

information on how to systematically predict and achieve that level of efficiency. 

Finally, the DEC Ada 1.5-44 compiler is a relatively error-free fmt attempt at an Ada compilation 

system. The next generation of Ada compilers, which includes DEC Ada 2.0, are now available. These 

second-generation compilation system includes improvements to the optimizer and code-generator. For 

example, simply compiling GOADA using DEC Ada 2.0 improved the simulator's performance by 7.4%. 

CONCLUSIONS 

The following statements summarize the results of the Ada Performance Study: 

Design- and implementation decisions that favored fidelity over efficiency were the largest contributor 

to poor run-time performance. The design should continually be reevaluated against evolving user 

requirements and specifications. 

Ada simulators in the FDD can be designed and implemented to achieve run times comparable to 

those of existing FDD FORTRAN simulators. Inefficient systems indicate problems in the system 

design or the compiler being used. 

Current Ada compilation systems still have inconvenient features that may contribute to poor 

performance. Organizations using Ada should use available performance-analysis tools to assess 

their compilation systems. 

Design changes are much more expensive than coding changes during final system testing. Often due 

to schedule and budget constraints, design changes are impossible. Therefore the important implication of 

the Ada performance study results is that new technology (in this case Ada and OOD) requires 

performance prototyping and benchmarking early in the design phase even in seemingly simple or 

straightforward cases. 

The Ada Peflormunce Study Report (Reference 2) contains a detailed analysis of each alternative 

studied and summarizes the results of this analysis with specific performance recommendations for future 

OOD/Ada development efforts in the FDD. Different application domains may be able to apply these 

results and recommendations. However, this does not preclude the necessity for application domain 

specific prototyping and benchmarking to determine the application specific performance issues. 

1 

6-26 
REFERENCES 

1. Ada Programming Language, American National Standards Institutehiilitary Standard 18 15A, 

1983 (ANSVMILSTD-1815A- 1983) 

2. Goddard Space Flight Center, SEL91-003, Sojlware Engineering Laboratory Ada Performunce 

Study Report, E. Booth and M. Stark, July 1991 

3. Goddard Space Flight Center, SEL-88-003, Evolution of Ada Technology in the Flight Dynamics 

Area: Design Phase Analysis, K. Quimby, et al., prepared by Computer Sciences Corporation, 

December 1988 

4. Goddard Space Flight Center, FDD/552-90/010, Sofhvare Engineering Luboratory (SEL) Study of 

the System and Acceptance Test Phases of Four Telemetry Simulator Projects, D. Cover, prepared 

by Computer Sciences Corporation, September 1990 

Goddard Space Flight Center, SEL-89-005, Lessons Learned in the Transition to Ada from 

FORTRAN at NASMGoddard, C. Brophy, University of Maryland, November 1989 

5 

6. Clapp, R., and T. Mudge, Rationale, Chapter I - Introduction, Ada Performance Issues, Ada 

Letters, A Special Issue, Association of Computing Machinery, New York, NY, ISBN 0-89791- 

354-x, vol. 10, no. 3, Winter 1990 

7. Clapp, R., and T. Mudge, Rationale, Chapter 3 - The Time Problem, Ada Performance Issues, 

Ada Letters, A Special Issue, Association of Computing Machinery, New York, NY, ISBN 0- 

89791-354-x, vol. 10, no. 3, Winter 1990 

8. Gaumer, D. and D. Roy, Results, Reporting Test Results, Ada Performance Issues, Ada Letters, 

A Special Issue, Association of Computing Machinery, New York, NY, ISBN 0-89791-354-x, 

vol. 10, no. 3, Winter 1990 

9. Digital Equipment Corporation, Guide to VAX Performance and Coverage Analyzer, June 1989 

10. Goddard Space Flight Center, FDD/552-90/045, Extreme Ultraviolet Explorer (EWE) Telemetry Simulator (EUVETELS) Sofrware Development History, E. Booth and R. Luczak, prepared by 

Computer Sciences Corporation, 1990 

1 1. Booch, G., Object Oriented Design, The BenjaminKummings Publishing Company, Inc., 

Redwood City, CA, ISBN 0-8053-0091-0, 1991 

12. Goddard Space Flight Center, SEL-87-006, Ada Style Guide, E. Seidewitz et al., June 1986 

13. Burley, R., “Some Data from Ada Performance Study” internal FDD memorandum, September 

1990 

6-27 

i 

STANDARD BIBLIOGRAPHY OF SEL 

LITERATURE 

I 

i 

STANDARD BIBLIOGRAPHY OF SEL LITERATURE 

The technical papers, memorandums, and documents listed in this bibliography are organized into two groups. The first group is composed of documents issued by the Software Engineering Laboratory (SEL) during its research and development activities. 

The second group includes materials that were published elsewhere but pertain to SEL 

activities. 

SEGORIGINA’lZD DOCUMENTS 

SEG76-001, Proceedings From the First Summet &@ware Engineering Wonkrhop, 

August 1976 

SEG77-002, Proceedings Fmm the Second Summer So@m Engineering Workshop, 

September 1977 

SEG78-005, Proceedings F” the Third Summet Software Enginem’ng Workshop, 

September 1978 

SEG78-006, GSFC So@are Engineering Research Requkments Anabsis Study, 

I? A. Scheffer and C. E. Velez, November 1978 

SEL78-007, Applicability of the Raykigh Curve to the SEL Environment, T E. Mapp, 

December 1978 

SEG78-302, FORTUN Static Some Code Anaw hg” (SAP) User’s Guide 

(Revision 3), W. J. Decker, W. A. Thylor, et id., July 1986 

SEG79-002, The Soware Engineering Labomtory Relationship Equations, 

K. Ereburger and V. R. Basili, May 1979 

SEG79-004, Evaluation of the Cainc, Fe and Gordon Bq” Design hnguage 

(PDL) in the Goddad Space Fhght Center (GSFC) Code 580 Sofim Design Envimnment, C. E. Goorevich, A. L. Green, and W. J. Decker, September 1979 

SEG79-005, Piweedings Iiom the Fourth Summer Sofiare Engineering Wonkhop, 

November 1979 

SEGS0-002, Multi-Level Expression &sign Language-Rkquiwment Level (MEDL-R) 

System Evaluation, W. J. Decker and C. E. Goorevich, May 1980 

SEGSW5, A Study of the Musa Reliabibty M&l, A. M. Miller, November 1980 

SEGS-, Roceedings hm the Fiji% Annual Sofian? Engincerng Wonkhop, 

November 1980 

SEGW7, An Appraisal of Selected CostIRe”e Estimation Modelr for So@m 

System, J. E Cook and E E. McGarry, December 1980 

BI-1 
SEG80-008, %torial on Models andMetrics for SofhuareManagement and Engineering, 

V. R. Basili, 1980 

SEGs1-011, Evaluating Software Development by Anarysis of Change Data, 

D. M. Weiss, November 1981 

SEG81-012, The Ruykigh Cwe as a Model for Eflort Distriburion Over the Life of 

Mtdium Scale So@m %stems, G. 0. Picasso, December 1981 

SEG81-013,Aoceedingsofthe SixthAnnualSojlwam Engineerbag Workshop, December 

1981 

SEG81-014,Automated Collection of Sofiare Engineering Data in the Sofiware Engineering Laboratory (SEL), A. L Green, W. J. Decker, and E E. Mffiarry, September 

1981 

SEG81-101, Guide to Data Collection, V. E. Church, D. N. Card, E E. McGarry, et al., 

August 1982 

SEG81-104, lk So" Engineering Laboratory, D. N. Card, E E. McGarry, 

G. Page, et al., February 1982 

SEL81-110, EvalwtionofanZ~ent VmFcation und 'validation (N&v) Methodology for Flight Dynamics, G. Page, E E. MCGarry, and D. N. Card, June 1985 

SEL81-305,RecommendedAppmch to &@are Development, L. Landis, S. Wdigora, 

E E. McGarry, et al., June 1992 

SEG82-001, Evaluation of Management Measures of Soware Development, G. Page, 

D. N. Card, and E E. McGarry, September 1982, vols. 1 and 2 

SEG82-004, Collected Sojhvare Engineering Paper: Volume 1, July 1982 

SEG82M)7, Ruceedinp of the Seventh Annual Somare Engineering Worhhop, 

December 1982 

SEG82-008, Evaluating Soware Development by Analysis of Changes: The Data hm 

the So@are Engineering Labonrtory, V. R. Basili and D. M. Wed, December 1982 

SEG82-102, F0RY"Static Source CodeAna&erPmgram (SAP) System Description 

(Revirion I), W. A. nylor and W. J. Decker, April 1985 

SEL82-105, Glossary of Soware Engineering Laboratory Tm, T. A. Babst, 

M. G. Rohleder, and E E. McGarry, October 1983 

SEL82-1106, Annotated Bibliography of Software Engineering Luboratoty Literahue, 

L. Morusiewicz and J. Valett, November 1992 

SEL-83401, An Appmach to Sofiare Cost Estimation, E E. Mffiarry, G. Page, 

D. N. Card, et al., February 1984 

BI-2 
SEG83-002, Measures and Metrics for Sojlware Devebpment, D.N.Card, 

E E. McGany, G. Page, et al., March 1984 

SEG83-003, Collected Soean? Engineering Papm: Volume 11, November 1983 

SEG83-006, Monitoring Sojbvare Development kugh Qv"ic Wiizbles, 

C. W. Doerflinger, November 1983 

SEG83-007, heedbags of the Eighth Annual Sojlwure Engineering Workrhop, 

November 1983 

SEG83-106, Monitoring Soji~are Devebpmnt kugh Qvnamk Kznizbles (Revision I), C. W. Doerflinger, November 1989 

SEG84-003, Investigation of Speijkation Measures forthe Sojbvare EngineerngLaboratory (SEL), W. W. Agresti, I? E. Church, and E E. McGarry, December 1984 

SEG84-004, Rvceedings of the Nith Annual S@wm Engineering Workrhop, 

November 1984 

SEG84-101, Manager's Handbook for So- Devebpment (.on l), L. Landis, 

E E. McGany, S. Waligora, et al., November 1990 

SEG85-001, A Compmison of Sofiware Vk$ieation Techniques, D.N.Card, 

R. W. Selby, Jr., E E. McGarxy, et al., April 1985 

SEG85-002, Ada Tmining Evaluation and Recommendations Iiom the Gamma Ray 

Observatory Ada Development Team, R. Murphy and M. Stark, October 1985 

SEG85-003, Collected Soware Engineering Bpm: Volume HI, November 1985 

SEG85-004, Evaluations of Sojbvare Technologes: Testing, CLEANRO.OM, and 

Metrics, R. W. Selby, Jr., and V. R. Basiii, May 1985 

SEG85-005, Software Vmpcation and Tdg, D. N. Card, E. Edwards, E McGany, 

and C. Antle, December 1985 

SEG85-006, Roeeedings of the Tenth Annual Sojbvam Engineering Workrhop, 

December 1985 

SEG86-001, Rogrammer's Handbook for Flight Dynamics software Development, 

R. Wood and E. Edwards, March 1986 

SEL86-002, Geneml Object-Oriented Software Devebpment, E.Seidewitz and 

M. Stark, August 1986 

SEG86-003, mght Dynumics System &@ware Devebpment Envimnment (FDSISDE) 

nCtOria1, J. Buell and R Myers, July 1986 

SEL86-004, collected Sojhare Engineeg &pm: Volume November 1986 

b1-3 
SEL-86-005, Measuring Sofhvme Design, D. N. Card et al., November 1986 

SEL86-006, hceedings of the Eleventh Annual Sojhvare Engineering Workshop, 

December 1986 

SEG87-001, Muct Assurance policies and Rocedurw for Flight Dynamics Sofiare 

Development, S. Perry et al., March 1987 

SEL87402, Ada@ Styk Guide (version l.l), E. Seidewitz et al., May 1987 

SEL87-003, Guidelines for Applyhg the Composite Specification Model (CSM), 

W. W. Agresti, June 1987 

SEC87-004, Assessing the Ada@ Design A.ocesr and I& Implications: A Care Study, 

S. Godfkey, C. Brophy, et al., July 1987 

SEG87-009, Collected Sojhvam Engineering Papers= Volume t: November 1987 

SEG87-010, Rvceedhgs of the lbeljlh Annual &@are Engineering Wonkrhop, 

December 1987 

SEG88-001, System Testing of a Proriuction A& Project: The GRODY Study, J. Seigle, 

L. Esker, and Y. Shi, November 1988 

SEG88-002, Collected Sofiam Engineering Pam: Volume W, November 1988 

SEG88-003, Evolution of A& Technology in the Flight Dynamics Area: Design Phase 

Analysis, K. Quimby and L. Esker, December 1988 

SEG88-004, Aioceedings of the ntirteenth Annual Sofrware Engineering Workrhop, 

November 1988 

SEc88-005, Proceedings of the First MSA Ada User's Symposium, December 1988 

SEG89-002, Implementation of a A.oduction A& Project: rite GRODY Study, 

S. Godfrey and C. Brophy, September 1989 

SEG89-004, Evolution ofA& Technology in the Flight DynamicsArea: Impkmentationl 

Testing Phase AM&& K. Quimby, L Esker, L. Smith, M. Stark, and E McGarry, 

November 1989 

SEG894Q5, Lessons Leamed in the Tmnsition to Ada From FORTRAN at NASAI 

Goddard, C. Brophy, November 1989 

SEG89-006, Collected Sojhvm Engineering Papers: Volume WI, November 1989 

SEG89-007, Aioceedings of the Fourteenth Annual &@ware Engineering Wonkrhop, 

November 1989 

SEG89-008, hceedings of the Second NASA Ada Users' Symposium, November 1989 

b1-4 
SEG89-103, Sofrware Management Environment (SME) Concepts and Architecture 

(Revision I), R. Hendrick, D. Kistler, and J. Wlett, September 1992 

SEG89-201, Sofrware Engineering Laboratory (SEL) Database Organizution and User’s 

Guide (R” 2), L. Morusiewicz, J. Bristow, et al., October 1992 

SEG90-001, Database Access Manuger for the So@m Engineering Laboratory 

(WSEL) User’s Guide, M. Buhler, K. Fb~phrey, and D. Spiegel, March 1990 

SEG90-002, The Cleanmom Case Study in the &@are EngineeringLaboratory: Project 

Desc@tion and EadyAnalysis, S. Green et al., March 1990 

SEG90-003,A Study of the Portability of anAdu System in thesoware EnginemhgLaboratory (SEL), L. 0. Jun and S. R. Valett, June 1990 

SEG90-004, Gamma Ray Observatory Dynamics Simulator in Ada (GRODV Experment Summary, T. McDermott and M. Stark, September 1990 

SEG90-005, Collected Sofrware Engineering Pap: Volume WII, November 1990 

SEG90-006, Aoceedingr of the Fifteenth Annual SopVare Engineering Wonkshop, 

November 1990 

SEG91-001, sofrware Engineering Labomtory (SEL) Relationships, M&h, and Management Rules, W. Decker, R. Hendrick, and J. Valett, February 1991 

SEG91-003, Sofrware Engineering Labomtory (SEL) Ada P4o”ce Study Report, 

E. W. Booth and M. E. Stark, July 1991 

SEG91-004, Sofrware Engineering Labomtory (SEL) Cleamm ptocess Model, 

S. Green, November 1991 

SEG91-005, Collected Sojhvare Engineering Papers: Volume E, November 1991 

SEG91-006, lkceedings of the Sixteenth Annual So@m Engineering Wot4lclrhop, 

December 1991 

SEG91-102, So@are EngineeringLaboratov (SEL) Data and Information Policy (&vision I), E McGany, August 1991 

SEG92-001, Sofrware Management Environment (SME) Installation Guide, D. Kistler 

and K. Jeletic, January 1992 

SEG92-002, Data Collkction Procedures for the Sojbvare Engineeiing Laboratory (SEL) 

Database, G. Heller, J. Wett, and M. Wild, March 1992 

SEG92-003, Collected Software Engineering Papers: Volume X, November 1992 

SEGRELATED LITERATURE 

‘OAbd-El-Hafiz, S. K., V. R. Basili, and G. Caldera, “lbwards Automated Support for 

Extraction of Reusable Components,”Rvceedings of the IEEE Conference on Soware 

Maintenance-1991 (CSM 91), October 1991 

BI-5 
4Agresti, W. W., V. E. church, D. N. Card, and P. L. Lo, “Designing With Ada for Satellite Simulation: A Case Study,” Proceedings of the First International SVmposium on 

Ada for the NASA Space Starion, June 1986 

2Agresti, W. W., E E. McGarry, D. N. Car4 et al., “Measuring Software T~c~o~o~Y,~’ 

Rq” Tmnsfomration and Rqpmming Environments. New York S pringer-Verlag, 

1984 

‘Bailey, J. W., and V. R. Bad& ‘A Meta-Model for Software Development Resource 

Expenditures,“ Rvceedkgs of the Fifi Intmuztiwull Cbnfmmee on So@are Engineming. New York IEEE Computer Society Press, 1981 

BBdey, J. W., and V. R. Basili, “Software Reclamation: Improving Post-Development 

Reusability,” Proceedings of the Eight;hAnnual National Confmnce on Ada Technology, 

March 1990 

l@ailey, J. W., and V. R. Basili, “The Software-Cycle Model for Re-Engineering and 

Reuse,” Proceedings of the ACM Ti-Ada 91 Conference, October 1991 

‘Basili, V. R., “Models and Metria for Software Management and Engineering,” 

ASMEAdvances in Computer Zkchnobgy, January 1980, vol. 1 

Basili, V. R., lltttorial on Models and Mehics for Sopktate Management and Engineering. 

New York IEEE Computer Society Press, 1980 (also designated SEL80-008) 

3Basili, V. R., “Quantitative Evaluation of Software Methodology,” Roceedings of the 

Fitst Pan-Pac@c Computer Confwence, September 1985 

’Basili, V R., Maintenance = Reuse-Oriented &@are Development, University of 

Maryland, ’Mhnical Report TR-2244, May 1989 

’Basili, V R., Sofnvare Development: A Paradigm for the Future, University of Maryland, 

’Tkchnical Report “R-2263, June 1989 

8Basili, V. R., “Viewing Maintenance of Reuse-Oriented Software Development,” 

IEEE Soware, January 1990 

‘Basili, V. R., and J. Beane, “Can the Parr Curve Help With Manpower Distribution 

and Resource Estimation Problems?,”Journal of Systemand Sofrware, February 1981, 

vol. 2, no. 1 

%asili, V. R., G. Caldiera, and G. Cantone, “A Reference Architecture for the Component Factory,”ACM Tmnsactions on Software Engineering and Metbdoloa, January 

1992 

l@asili, V., G. Caldiera, E McGarry, et al., “The Software Engineering LaboratoryAn Operational Software Experience Factory,” Bvceedinp of the Fourteenth Intemational Conference on Sofnvare Engineering (ICSE 92), May 1992 

BI-6 
IBasili, V. R, and K. Freburger, “Programming Measurement and Estimation in the 

Software Engineering Laboratory,” Joumal of System and Sojlwam, February 1981, 

vol. 2, no. 1 

3Basili, V R., and N. M. Panlilio-Yap, “Finding Relationships Between Effort and 

Other Variables in the SEI,,” Rvceedhgs of the Intemationul Computer Sofhvare and 

Applications Confmnce, October 1985 

4Basili, V R, and D. Patnaik, A Study on Fault prediction and Reliability At in 

the SEL, Environment, University of Maryfand, Report TR-1699, August 1986 

*Baa, V. R., and B. T. Perricone, “Software Errors and Complexity: An Empirical 

Investigation,” Communications of ticCe ACM, January 1984, vol. 27, no. 1 

‘Basili, V. R., andT. Phillips, “Evaluating and Comparing Software Metria in the Software Engineering Laboratory,” Proceedings of the ACM SIGMETUICS JLmposiuml 

Wotkhop: Qual@ Meirics, March 1981 

3Basili, V. R., and C. L. Ramsey, “ARROWSMI[TH-P-A Prototype Eprt System for 

Software Engineering Management,” Proceedings of the LEEE/” System 

in Gin”ent JL”, October 1985 

Basili, V. R., and J. Ramsey, Stiuctrrml Covmge of Fmfionul Tag, University of 

Maryland, llxhdcal Report TR-1442, September 1984 

Basili, V. R, and R Reiter, “Evaluating Automatable Measures for Software Development,” Rvceedings of the Workshop on Quantitative Soem Models for ReliabiZity, 

Comp-, and Cost. New York IEEE Computer Society Press, 1979 

5Basili, V. R., and H. D. Rombach, “Thiloring the Software Process to Project Goals 

and Environments,” Aoceedingr of the 9th International Confmnce on Sofnvare Engineering, March 1987 

5Basili, V. R, and H. D. Rombach, “‘I‘ A M E: Woring an Ada Measurement Emrironment,” Rxeedingr of the Joint Ada Conference, March 1987 

5Basili, V R., and H. D. Rombach, “TAME Integrating Measurement Into Software 

Environments,” University of Maryland, ’It=chnical Report TR-1764, June 1987 

6Basili, V R., and H. D. Rombach, “The TAME Project: Tbwards ImprovementOriented Software Environments,” IEEE Tmnsactions on Sojlware Engineehg, June 

1988 

’Basili, V R., and H. D. Rombach, TowardrA ComprehenrieI;mmework forReure: A 

Reuse-Enabling Sojhvure Evolution Envhnment, University of Maryland, Tkchnical 

Report TR-2158, December 1988 . 

8Basili, V R., and H. D. Rombach, Towanis A Conp&msive Fnzmework for Reuse: 

Model-Based Reuse Chamcterization Schemes, University of Maryland, ’kchnical 

Report TR-2446, April 1990 

BI-7 
Qasili, V. R., and H. D. Rombach, “Support for Comprehensive Reuse,” Sojbvare Engineering Jozunal, September 1991 

3Basili, V. R, and R. W. Selby, Jr., ‘‘Calculation and Use of an Environment’s Characteristic Software Metric Set,”Proceedingsof the Eighth International Conference on Sopware Engineering. New York IEEE Computer Society Res, 1985 

Basili, V. R., and R. W. Selby, “Comparing the EfEectiveness of Software Testing Strategies,” IEEE Tranractionr on So@m Engineerin, December 1987 

3Basili, V. R., and R. W. Selby, Jr., “Four Applications of a Software Data Collection 

and Analysis Methodology,”Rvceedings of rhe NATOAdvanced Study Inrtirute, August 

1985 

5Basili, V. R., and R. Selby, “Comparing the Effectiveness of Software lksting Strategies,” IEEE Eansacfions on Sofhare Engineering, December 1987 

Qasili, V. R., and R. W. Selby, ”Paradigms for bpenmentation and Empirical Studies 

in Software Engineering,” Reliability Engineering and @stem Safq, January 1991 

4Basili, V. R., R. W. Selby, Jr., and D. H. Hutchens, “Experimentation in Software 

Engineering,” IEEE Tmnsactions on Sofhare Engineering, July 1986 

2Basili, V. R., R. W. Selby, and ‘I: Phillips, “Metric Analysis and Data Validation Across 

FOKI” Projects,” IEEE Tmnsactiom on Sojbvare Engineering, November 1983 

2Basili, V. R., and D. M. Weiss, A Methodologv for Collecting Wid SoNare Engineering 

Data, University of Maryland, lbchnical Report TR-1235, December 1982 

3Basili, V. R., and D. M. Weiss, ‘!A Methodology for Collecting Valid Software Engineering Data,’’ IEEE Transactions on Sofrware Engineen’ng, November 1984 

‘Basili, V. R., and M. V Zekowitz, “The Software Engineering Laboratory: Objectives,” Proceedings of the Fifteenth Annual Conference on Computer Personnel Research, 

August 1977 

Basili, V. R., and M. V Zelkowitz, “Designing a Software Measurement Experiment,” 

Proceedings of the Software Life %le Management Workshop, September 1977 

lBasili, V. R., and M. V. Zelkowitz, “Operation of the Software Engineering Laboratory,” Rxeedings of the Second Sofhvare Life ock Management Wonkrhop, August 

1978 

lBasili, V. R., and M. V. Zelkowitz, “Measuring Software Development Characteristics 

in the Local Environment,” Computers and Sauctures, August 1978, vol. 10 

Basili, V. R., and M. V Zelkowitz, “Analyzing Medium Scale Software Development,” 

Proceedings of the Third International Conference on Sojbvam Engineering. New York 

IEEE Computer Society Press, 1978 

BI-8 
bth, E. W., and M. E. Stark “Designing Configurable Software: COMPASS Implementation Concepts,” hee&g,s of Tn’-Ada 1991, October 1991 

lhth, E. W., and M. E. Stark, “Software Engineering Laboratory Ada Performance 

Study-Results and Implications,“proCeedings of the Fourth Annual NASA Ada User’s 

Symposium, April 1992 

%nand, L. C., and V. R Basili, ‘A Classification Procedure for the Effective Management of Changes Duringthe Maintenance F%cess,”Rvceedingsof the 1992IEEE Conference on Soware Maintenance (CSM 92), Nuvember 1992 

%xiand, L C., V R. Basih, and C. J. Hetmanski, “Rovidhg an Empirical Basis for 

Optimizing the Verification and %sting Phases of Software Development,” Proceedings of the Third IEEE Intematiunal Symposium on Software Reliizbdity Engineering 

(ISSRE 92), October 1992 

%riand, L C., V. R Basili, and W. M. Thomas,A”n Recognition Approach forSofiware Engineering Data Analysis, University of Maryland, ’kchnical Report “€2-2672, 

May 1991 

5Brophy, C. E., W W Agresti, and V R. Basili, “Lessons Learned in Use of AdaOriented Design Methods,” A.oceedingr of the Joint Ada Conference, March 1987 

6Brophy, C. E., S. Godfrey, W. W. Agresti, and V R Basili, “Lessons Learned in the 

Implementation Phase of a Large Ada Project,” Proceedings of the Washington Ada 

Technical Conference, March 1988 

2Card, D. N., “Early Estimation of Resource Expenditures and Program Size,” 

Computer Sciences Corporation, lbchnical Memorandum, June 1982 

2Card, D. N., “Comparison of Regression Modeling ”iques for Resource-Estimation,” Computer Sciences Corporation, 

3Card, D.N., ‘A Software Bchnology Evaluation Program,” Annais do MII 

Congress0 Nacwnal de Informaiica, October 1985 

D. N., and W W. Agresti, ‘‘Resolving the Software Science Anomaly,” Joumal 

6Card, D. N., and W. W. Agresti, “Measuring Software Design Complexity,”Journal of 

System and Sojiware, June 1988 

4Card, D. N., V. E. Church, and W. W. Agresti, “An Empirical Study of Software Design 

Practices,” IEEE Transactions on Sofrware Engineering, February 1986 

Card, D. N., V E. Church, W. W. Agresti, and Q. L. Jordan, “A Software Engineering 

View of Flight Dynamics Analysis System,” Parts Iand II, Computer Sciences Corporation, %chnical Memorandum, February 1984 

Memorandum, November 1982 

of stems 41cd SO@are, 1987 

BI-9 
Card, D. N., Q. L. Jordan, and V E. Church, “Characteristics of FO- Modules,” 

Computer Sciences Corporation, ’kchnical Memorandum, June 1984 

5Card, D. N., E E. McGarry, and G. T Page, “Evaluating Software Engineering 

khnologie~,” IEEE Tmnsactions on Soware Engineering, July 1987 

3Card, D. N., G. T. Page, and E E. McGarry, “Criteria for Software Modularization,” 

Proceedings of the Eighth Intemational Conference on software Engineering. New York 

IEEE Computer Society Press, 1985 

lChen, E., and M. V Zelkowitz, “Use of Cluster Analysis ’Ib Evaluate Software Engineering Methodologies,” Roceedings of the Fifth Intemational Confetnce on Sojhare 

Engineering. New York IEEE Computer Society Press, 1981 

4Church, V E., D. N. Card, W. W. Agresti, and Q. L. Jordan, “An Approach for 

Assessing Software Prototypes,”ACMSo@are Engineering Notes, July 1986 

2Doerflinger, C. W., and V. R Basili, “Monitoring Software Development Through 

Dynarmc Wables,” hceedings of the Seventh Intemational Computer Sofnvare and 

Applications Conference. New York IEEE Computer Society Press, 1983 

Doubleday, D., AW An A& Static Source Code Anakr Progmm, University of 

Maryland, Ihhnical Report TR-1895, August 1987 (NOTE: 100 pages long) 

6Go&ey, S., and C. Brophy, “Experiences in the Implementation of a Large Ada 

Project,” Proceedings of the 1988 Wmhington Ada Syrnposium, June 1988 

5Jeffery, D. R., and V Basili, Chamcterizing Resource Data: A Model for Logrcal 

Association of Sojhare Data, University of Maryland, khnical Report TR-1848, May 

1987 

6Jeffery, D. R., and V R. Basili, ‘Vdidating the ‘MIME Resource Data Model,” hceedings of the Tenth Intemational Confermce on So@at Engineering, April 1988 

5Mark, L., and H. D. Rombach, A Mda Information Base for Soware Engineering, 

University of Maryland, ’Rchnical Report TR-1765, JLI~~ 1987 

6Mark, L, and H. D. Rombach, “Generating Customized Software Engineering 

Infoxmation Bases From Software Process and Product Specifications,”Proceedings of 

the 22ndAnnual Hawaii Intenrational Conference on System Sciences, January 1989 

5McGarry, E E., and W. W. Agresti, “Measuring Ma for Software Development in the 

Software Engineering Laboratory (SEL),” hceedings of the 21st Annual Hawaii . 

International Conference on System Sciences, January 1988 

’McGarry, E, L. Esker, and K. Quimby, “Evolution of Ada khnology in a Production 

Software Environment,” hcedngs of the Sixth Washington Ada symposium 

(WADAS), June 1989 

BI-10 
3McGarry, E E., J. Vdett, and D. Hall, ‘‘Measuring the Impact of Computer Resource 

Quality on the Software Development pn>cess and Product,” Proceedings of the 

Hawaiiun Internarional Conference on System Sciences, January 1985 

3Page, G., E E. McGany, and D. N. Card, ‘A Practical Experience With Independent 

Verikation andValidation,”Proceedings of the Eighth International Computer Software 

and Applications Conference, November 1984 

5Ramsey, C. L., and V. R Basili, “An Evaluation of Expert Systems for Software Engineering Management,” IEEE Tmnsactions on So@m Engineering, June 1989 

3Ramsey, J., and V. R. Basili, “Analyzing the Rst Process Using Structural Coverage,” 

Proceedings of the Eighth Intmtional Conference on Softwan? Engineering. New York: 

IEEE Computer Society Press, 1985 

5Rombach, H. D., ‘% Controlled Experiment on the Impact of Software Structure on 

Maintainability,” IEEE Tmnsactions on Somar Engineering, March 1987 

8Rombach, H. D., “Design Measurement: Some Lessons Learned,” IEEE Soj%van?, 

March 1990 

Qombach, H. D., “Software Reuse: A Key to the Maintenance Problem,”Buttenvorfh 

Journal of Infomation and Sojbvar Technology, JanuaryFebruary 1991 

6Rombach, H. D., and V. R. Basili, “Quantitative Assessment of Maintenance: An 

Industrial Case Study,” Proceedings From the Conference on So” Maintenance, 

September 1987 

6Rombach, H. D., and L. Mark, “Software Process and Product Specifications: A Basis 

for Generating Customized SE Information Bases,” Proceedhgs of the 22nd Annul 

Hawaii Intenurtional Conference on System Sciences, January 1989 

’Rombach, H. D., and B. T. Ulery, Ertabbshing a Measurement Based Maintenance 

Improvement Pmgram: Lessons Leamed in the SEL, University of Maryland, lkchnical 

Report TR-2252, May 1989 

l%mbach, H. D., B. T. Ulery, and J. D. Valett, ‘“Ibward Full Life Cycle Control: 

Adding Maintenance Measurement to the SEL,” Journal of System and Sowam, 

May 1992 

%eidewitz, E., “Object-Oriented Programming in Smalltalk and Ada,” heedings 

ofthe 1987 Conference on Object-Oriented Programming System, hnguages, and 

Applications, October 1987 

%eidewitz, E., “General Object-Oriented Software Development: Background and 

Experience,” heedings of the 21st Hawaii International Confernce on System 

Sciences, January 1988 

BI-11 
6Seidewitz, E., “General Object-Oriented Software Development with Ada: A Life 

Cycle Approach,” Proceedings of the CASE Technology Confmnce, April 1988 

gseidewitz, E., “Object-Oriented Programming Through Extension in Ada 9X,” 

Ada Letten, March/Aprill991 

1OSeidewit.q E., “Object-Oriented Programming With Mixins in Ada,” Ada Letters, 

March/Aprill992 

4Seidewitz, E., and M. Stark, ‘‘l[bWards a General Object-Oriented Software Development Methodology,” Proceedings of the Fht International sLmposium on Ada for the 

MSA Space Station, June 1986 

gSeidewitz, E., and M. Stark, “An Object-Oriented Approach to Parameterized Software in Ada,” Proceedings of the Eighth Wwhin-n Ada Symposium, June 1991 

8Stark, M., “On Designing Parametrized Systems Using Ada,” heedings of the 

Seventh Washingon Ada Symposium, June 1990 

’Stark, M. E. and E. W. Booth, “Using Ada to Maximize Verbatim Software Reuse,” 

hceedings of l?U-Ada 1989, October 1989 

’Stark, M., and E. Seidewitz, “?bwards a General Object-Oriented Ada Lifecycle,” 

Proceedings of the Joint Ada Conference, March 1987 

‘OStraub, I? A, and M. V Zelkowitz, “On the Nature of Bias and Defects in the Software Specification Process,” Rvceedings of the Sixteenth International Computer Sopwaie and Applications Conference (COMPSAC 92), September 1992 

gstraub, I? A., and M. V Zelkowitz, “PUC A Functional Specification Language for 

Ada,” Proceedings of the Tenth Intemational Conference of the Chilean Computer Science 

Society, July 1990 

7Sunazuka, T, and V. R. Basili, IntegratingAutomated Suppolt for a software Management *le Into the U.ME System, University of Maryland, “kal Report TR-2289, 

July 1989 

”?Iian, J., A. Porter, and M. V Zelkowitz, “An Improved Classification ’Ree Analysis of 

High Cost Modules Based Upon an Axiomatic Definition of Complexity,”A.oceedings 

of the lhitd IEEE International JLmposium on Softwme Reliability Engineering 

(ISSRE 92), October 1992 

ker, C., and G. Caron, A Comparison of RADC and NASAISEL Sojlwate Development Data, Data and Analysis Center for Software, Special Publication, May 1981 

%dett, J. D., ‘Automated Support for Experience-Based Software Management,” 

Proceedings of the Second Irvine S0fh.m Symposium (ISS ‘92), March 1992 

’Valett, J. D., and E E. McGarry, ‘A Summary of Software Measurement meriences 

in the Software Engineering Laboratory,” Rvceedings of the 21st Annual Hawaii 

Intemational Conference on System Sciences, January 1988 

BI-12 
3Weiss, D. M., and V. R. Basili, “Evaluating Software Development by Analysis of 

Changes: Some Data From the Software Engineering Laboratory,” IEEE Fansuctions 

on So@m Engineeting, February 1985 

L., V. R. Basili, and K. Reed, “A Structure Coverage ’bo1 for Ada Software Systems,” Proceedingr of the Joint Ada Conference, March 1987 

*Zekowitz, M. V., “Resource Estimation for MediumScale Software Projects,” Rvceedings of the llvewh Confmnce on the Iizter$ace of Statistics and Computer Science. 

New York IEEE Computer Society Press, 1979 

2Zelkowitz, M.V., “Data Collection and Evaluation for Experimental Computer 

Science Research,” Empirical Foundatiom for Computer and Information Science (Proceedings), November 1982 

6Zekowitz, M. V., “The EfEectiveness of Software Prototyping: A Case Study,” Proceedings of the 26thAnnual Technical SVmposium of the Warhingtn, D. C., Chapter of the 

ACM, June 1987 

6Zekowitz, M. V., “Resource Utilization During Software Development,” Journal of 

Systems and So@are, 1988 

8Zekowitz, M. V., “Evolution ’Ibwards Spedcations Environment: Fhperiences With 

Syntax Editors,” Information and Sofnm Technology, April 1990 

BI-13 

1000022) 

111~ 
NOTES: 

'This article also appears in SEG82-004, CoUected Sojhvare Engineering Papm: 

Volume I, July 1982. 

*This article also appears in SEL83-003, Collected Software Engineering Papers: 

Volume 11, November 1983. 

3This article also appears in SEG85-003, Collected Soman? Engineering Papm: 

Volume III, Nwember 198's. 

"his article also appears in SEL86-004, CoUected Sofi'ware Engineering Papm: 

Volume Nwember 1986. 

SThis article also appears in SEG87-009, CoUected So@m Engineering Papm: 

Volume November 1987. 

SThiS article also appears in SEL88-002, Colkcted Sowm Engineering Papm: 

Volume V, November 1988. 

'This article also appears in SEL89-006, Collected So@am Engineering Papm: 

Volume VI, November 1989. 

8This article also appears in SEG90-005, Collected Software Engineering Papm: 

Volume VH, November 1990. 

%s article also appears in SEG91-005, Collected Sopare Engineering Papm: 

Volume E, November 1991. 

article also appears in SEG92-003, collected Sojhvare Engineering Papm: 

Volume X, November 1992. 

b1-14 
I 

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the lime for reviewing instructions. searching existing data sourcas, 

gathering and maintaining the data needed. and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 

cOllectiOfl Of information, including suggestions for reducing this burden, lo Washington Headquarters Services. Directorate for Information Operations and Reports. 1215 Jefferson 

Davis Highway. Suite 1204, Arlington. VA 22202-4302, and to the Office of Management and Budget. Papemork Reduction ProJect (0704-0188). Washington, DC 20503. 

1. AGENCY USE ONLY (Leave blank) 

4. TITLE AND SUBTITLE 

3. REPORT TYPE AND DATES COVERED 

Contractor Report (UM&CSCxGSFC 

14. SUBJECT TERMS 

12-1P page bibliography 

17. SHXlRlTY CLASSlFICAl" 18. SECURITY CLASSIFICAm 19. SECURIM CLAQQlflCATlON 

OF REPORT OF THIS PAGE OF ABSTRACT 

Unclassified Unclassified Unclassified 

collected Softwm Engineering Papers: VoL X 

6. AUTHOR(S) 

15. NUMBER Of PAGES 

am. '00 

16. PRICE CODE 

20. UMITATDN OF ABSTRACT 

2YL 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS (ES) 

NASAK;SFC Softwm Engineering Branch 

Univ. of MD Dept. Computer Science 

Computersciences, SohmEngineering operation 

National Amnautics and Space Administration 

Washington, DC 20546-0001 

9. SPONSORING I MONITORING ADGENCY NAME(S) AND ADDRESS (ES) 

11. SUPPLEMENTARY NOTES 

12.. DISTRIBUTION I AVAILABILITY STATMENT 

5. FUNDING NUMBERS 

ASG5445 

8. PEFORMING ORGANIZATION 

REPORT NUMBER 

SEL 92 003 

10. SPONSORING / MONITORING 

ADGENCY REPORT NUMBER 

SEL 92 0 

12b. DISTRIBUTION CODE 

1 

13. ABSTRACT (Maximum 200 words) 

This document is a collection of technical papers produced 10/91-11/92. The purpose is to make available 

results of SEL mh that originally appeared in different forums. These papers cover topics in software engineering, but do not encompass entire mpe of SEL activities. ?ha are 5 sections: 

ThesoftwmEngineeringLabaratory 

So- Tools Studies 

Software Models Studies 

Software Measurement Studies 

Ada Technology Studies