Patent ID: 8793659

Claim:
A computer implemented method, comprising: receiving, by a computing system, a user request to run a test of an operating system, the test being defined by a script identifying a plurality of phases comprising a setup phase for configuring a test environment, a test phase to detect errors in the operating system, and a cleanup phase to restore the test environment to an initial state, wherein each of the plurality of phases comprises one or more functions, and the script comprises conditions pertaining to each function; detecting a failure when running the test of the operating system, wherein the failure occurs when one of the conditions in the script is not satisfied; identifying, by the computing system and based on the conditions in the script, one of the plurality of phases in which the failure has occurred; if the failure occurred during the setup phase, discontinuing to run the test of the operating system, and reporting the failure during the setup phase to a user; if the failure occurred during the test phase, determining performance measurements at a time of the failure, reporting the failure during the test phase and the performance measurements to the user, and continuing to run the test; and if the failure occurred during the cleanup phase, reporting the failure during the cleanup phase to the user.