Software testing using model programs
A strategy described as 'testing using M model programs' (abbreviated to 'M-mp testing') is investigated as a practical alternative to software testing based on manual outcome prediction. A model program implements suitably selected parts of the functional specification of the software to be tested. The M-mp testing strategy requires that M (M greater than or equal to 1) model programs as well as the program under test, P, should be independently developed. P and the M model programs are then subjected to the same test data. Difference analysis is conducted on the outputs and appropriate corrective action is taken. P and the M model programs jointly constitute an approximate test oracle. Both M-mp testing and manual outcome prediction are subject to the possibility of correlated failure. In general, the suitability of M-mp testing in a given context will depend on whether building and maintaining model programs is likely to be more cost effective than manually pre-calculating P's expected outcomes for given test data. In many contexts, M-mp testing could also facilitate the attainment of higher test adequacy levels than would be possible with manual outcome prediction. A rigorous experiment in an industrial context is described in which M-mp testing (with M = 1) was used to test algorithmically complex scheduling software. In this case, M-mp testing turned out to be significantly more cost effective than testing based on manual outcome prediction. Copyright (C) 2001 John Wiley & Sons, Ltd.
m-mp testing test oracle n-version programming back-to-back testing comparison testing