From: bwatt@csi.uottawa.ca (Bill Watt)
Subject: Debugging the process
Organization: Dept. of Computer Science, University of Ottawa
Date: Tue, 22 Dec 92 16:45:08 GMT


Hypothesis: 
  Software quality improvement should be done by 
  inspecting products: first, to find and fix defects, and second, 
  to gather data in order to identify and prevent defects.


1. It is not necessary to follow the SEI model to benefit from 
statistical quality control in software development. Inspections 
can be done on any product (specs, design, code, test cases, 
documentation). One can start any time. One does not have to wait 
until SEI level 5. Inspections serve two purposes: first, to find 
and fix defects; second, to accumulate data on the kinds of 
defects encountered and then guide management in removing the 
causes of the defects. As Dr. Demming has shown, using inspections 
to remove causes of defects reduces the cost of rejects, reduces 
the cost of rework, and eventually reduces the cost of inspections.


2. Dr. Humphrey emphasizes in "Managing the Software Process" that 
the SEI model is for the development of large, complex software 
products. One example he uses is a product (an IBM operating 
system, I believe) developed at 13 laboratories spread over 8 
countries. For this kind of project the process should probably be 
as complex as the SEI model. But not necessarily. One should start 
with inspections and add management practices from the SEI list as 
required to eliminate the causes of defects.


3. If software workers (are unionized and) object to inspections 
because the findings may be used to substantiate terminations, an 
independent auditor should moderate the inspections and provide a 
periodic impersonal statistical summary of the defects to 
management. These should be posted and employees encouraged to 
offer suggestions on changing the process to eliminate causes of 
defects. Defects arise mostly from problems in the way the work is 
organized, not from the way individuals perform.


4. Eliminating defects is a 4-step process: first, identify the 
defects; second, figure out what's causing them; third, figure out 
how to stop causing them; and fourth, get people to stop causing 
them. Not easy. Steps two and three can be addressed with a group 
problem-solving method such as Kepner-Tregoe. (History is full of 
horror stories about people using wrong solutions to problems.) 
Software engineering isn't yet at the point where there are known 
ways to prevent all types of defects. Inspections are used for 
feedback.


5. After starting inspections, wait until the stats settle down 
before making any management changes. Starting inspections will 
tend to decrease defects without any management changes as workers 
use the feedback to change their individual processing. After they 
have done so take the remaining defects to management for changes 
to the organization of the work. Also, as Dr. Demming points out, 
if the process isn't stable you can't measure the effect of changes.


6. The SEI model is the process equivalent of the waterfall model. 
As such it has limited application. Also, it is the wrong way 
around in the sense that it leaves process improvement by 
statistical quality control to the final level. The insistence 
that the whole process be defined and in place in order to benefit 
from statistical quality control is in error because the 
individual worker (the indivisible atomic process) and and every  
part of a process producing something tangible can benefit from 
statistical methods.


7. Of the two approaches to quality, a) defined as having 
desirable properties, b) defined as having no undesirable 
properties (defects), the second is more promising in that it is 
measurable and controlable with statistical methods.


8. The software development process is not the problem. The 
quality of the input to the process is the greater problem. More 
benefit will come from client capability assessment than from 
software developer capability assessment. Software engineering 
has been through this before as it was attempted to prove program 
correctness. It should have been evident then that programmers 
were not making the mistakes, customers and their systems analysts 
were making the mistakes in not specifying requirements correctly. 
One can't build systems for clients who a) don't know what they're 
doing, b) can't agree among themselves on what they want, and c) 
don't have any time to devote to the project.



Would like to hear from those studying cause and effect 
relationships in software defects, and categorizing ways of 
preventing different kinds of defects in different kinds of 
systems.


Wm. R. Watt
Newsgroups: comp.software-eng