A Specifications-Based Mutation Engine for Testing Programs in C#

Abstract ² This paper presents a simple and efficient engine which produces mutations of source code written in C# . The novelty of this engine is that it produces mutations that do not contradict with the specifications of the program. The latter are described by a set of pre - and post -conditions and invariants. The engine comprises two parts, a sta tic analysis and syntactic verification component and a mutation generation component. Preliminary experiments showed that the proposed engine is more efficient than a simple mutations generator in terms of producing only valid mutations according to the s pecifications posed, thus saving time and effort during testing activities. Keywords -m utation testing; mutation engine; specifications; I. I NTRODUCTION Technology advancements nowadays lead to the automation of a large number of activities within the software development process. The exploitation of computing power drives the need for producing better, faster and more reliable software systems. Nevertheless, the aforementioned targets increase software complexity and size, making this need hard to be satisfied. The competition in the software development market pushes companies to increase their productivity, developing software in tighter time limits usually sacrificing the quality of the resulting software . One of the most significant reasons for the inadequate quality control in software development is the lack of efficient software testing. The latter is a way for verifying the correctness and appropriateness of a software system, or, alternatively, for ensuring that a program meets its specific ations ([1 ], [2] ). Software testing is not a simple process; on the contrary, it consumes a large percentage of the time and budget of the whole development process. In some cases it even surpasses the time needed for the creation of the software product. Its main purpose is to reveal and locate faults so as to assist developers improving the functional behavior of the system under development. Software testing consists of two main processes, the identification of faults ( testing) and the ir correction (debugging). Indentify ing faults is the most time consuming process as it can take up to 95% of the time of software testing. Having this in mind, we can safely conclude that there will be a constant need to develop tools that will assist in accelerating and automating the testing process, guiding developers to locate and debug faults faster and more efficiently. The aim of the present paper is to introduce a mutation engine for source code written in C#, which is the basic ele ment of a novel mutation testing technique that takes into consideration the specifications of the program for creating only valid mutants . The engine is implemented in V isual S tudio 2010 and consist s of two components: The first offers the ability to vali date the grammatical correctness of the source code and provide s a form of statistical analysis for exporting useful information that can be used to process/modify the source code. The second involves the production of mutations of the original source code and facilitates the identification of faults , as well as the assessment of the quality of test data. The rest of the paper is structured as follows: Section II describes briefly the basic concepts that form the necessary technical background of this work. Section III presents the mutation engine, its architecture and key elements ruling the generation of mutations , as along with a brief demonstration of the supporting software tool. Section IV describes a set of preliminary experiments and the correspondin g results that indicate the correctness and efficiency of the proposed approach. Finally, Section V concludes the paper and suggests some steps for future work. II. T