Testing by the numbers
In order for an organization to reach SEI level 4 capability, quantitative process management is essential. As a step towards SEI level 4 the GSM BSS software organization has developed a metric analysis programme which is being used during the system test phase. This programme focuses on metrics analysis which allows quantitative metric information to be correlated with non-metric knowledge and used real-time. It has allowed us to take a giant leap from simply collecting pass/fail data during testing to analysing the data, so they can be used by the project. Quantitative information is used to plan and monitor test activities and evaluate the product at the end of testing. Not only has this programme provided a technique for managing the test phase, it has provided a mechanism for improving the quality of our releases and we are beginning to use it as a prediction of the quality of our product as release. The metric analysis programme also provides a mechanism for improving the test process and is also being utilized to indicate areas requiring improvement within development. Our paper is a case study of how we use test metrics. It discusses the corrective actions that resulted from the analysis of the metrics. We have found this approach invaluable as we improve the quality of our processes and product.
metrics test metrics test analysis test improvement