Therefore an exhaustive testing isn't efficient. Nevertheless testing is still necessary, but the focus of testing moves from error detection to certification of quality. The efficiency of tests shall be better than conventional testing. In order to certify and to improve the reliability of software by testing, it is necessary to regard the customers' view on reliability. The experiment will introduce usage modelling and statistical testing for a subsystem of a quite large customer project, and will involve the customer in the early phases.
Customers count failures per execution, whereas developers count defects per kilostatement.
Statistical testing means random selection of representative test cases from the infinite number of possible executions of the software. The expected usage of the system is modelled with Markov-Chains. Test cases are generated from the usage model and run on test equipment. The test results are evaluated in order to predict the reliability of the system. The reliability of the delivered system is monitored and compared with the prediction.