Mutation Testing in Java (Using Jumble)

08 May, 2017
Mutation testing is a white-box testing technique, which changes certain portions of the code to reveal possible faults. On a very high level, it is the process of rewriting the source code with some known or possible bugs or differences to the actual code in small ways in order to remove the redundancies in the source code. The goal of mutation testing is to evaluate the coverage of your tests. There are many mutation testing tool for Java.
JumblePITμJavaJester
Jumble
PIT
μJava
Jester
This post deals with Jumble Mutation testing Tool in Java. Jumble is a mutation testing tool which changes the Java code at byte code level. It runs a unit test case and applies certain set of mutations.
Passed Mutations: Changes in code which do not affect the final output
Failed Mutations: Changes in code which change the final output
Score : It is the percentage of passed mutations out of the total mutations. It is used to assess the validity of our test suite.InstallationDownload Eclipse IDE and install Jumble to Eclipse.
Simple Code Snippet in Java
int index = 0;
while(true)
{
   index++;
   if (index == 10) 
        break;
}

Mutated Code in JAVA
int index = 0;
while (true)
{
   index++;
   if (index >= 10) 
        break;
}

Explanation: The mutant code above will pass the Jumble test because change in == to >= does not affect the output of the code. Execution will stop when index == 10 and since we are increasing value by 1 and index starts from 0 so the output will remain same.
Example with JumbleThe code written below has been tested with Jumble plugin in Eclipse. The code detects the first occurrence of a duplicate and returns the value to the calling function. The program is flawed in many ways which you can try figuring out.
// Java program to illustrate mutation Testing// The code detects the first occurrence of a // duplicate and returns the value to the calling// functionpackage testPackage;  import java.util.Arrays;import java.util.List;public class SampProg{    protected int repeatedNumber(final List a)    {        int len = a.size(),i,dup = -1;        int[] arr = new int[len];        for (i=0; i<len; i++)        {            arr[i] = a.get(i);        }          Arrays.sort(arr);        try        {            for (i=1; i<len; i++)            {                if(arr[i] == arr[i-1])                {                    dup = arr[i];                    break;                }            }        }        catch(Exception e)        {            System.out.println(e.getMessage());        }        return dup;    }}
After writing the program we create test cases using JUnit in Java. The output obtained on performing Jumble Analysis is given below:
Mutating testPackage.SampProg
Tests: testPackage.SampProgTest
Mutation points = 11, unit test time limit 2.94s
M FAIL: (testPackage.SampProg.java:8): -1 -> 1
M FAIL: (testPackage.SampProg.java:10): 0 -> 1
.M FAIL: (testPackage.SampProg.java:10): negated conditional
M FAIL: (testPackage.SampProg.java:16): 1 -> 0
M FAIL: (testPackage.SampProg.java:18): 1 -> 0
M FAIL: (testPackage.SampProg.java:18): - -> +
M FAIL: (testPackage.SampProg.java:18): negated conditional
M FAIL: (testPackage.SampProg.java:16): += -> -=
M FAIL: (testPackage.SampProg.java:16): negated conditional
.
Jumbling took 7.595s
Score: 18%
Interpreting the Result
In the first line of M FAIL in the output -1 was changed to 1. In the main program, if there are no duplicates we expect the value to be returned as -1. but instead we get the value 1. We can fix the mutation by enabling checks for value mutation.Next, we can see “negated conditional” on the 3rd, 7th and 9th line where condition has been negated. Here however we can’t fix the error for this program.Other cases where the mutation failed to give proper output is when operators are changed to other operators. In this and many algorithmic codes, mutations related to operator changes cannot be fixed. However mutation testing gives brief idea in some mutations as to where the test case is weak or there is an error in the source code. This link gives mutations performed by jumble
In the first line of M FAIL in the output -1 was changed to 1. In the main program, if there are no duplicates we expect the value to be returned as -1. but instead we get the value 1. We can fix the mutation by enabling checks for value mutation.
Next, we can see “negated conditional” on the 3rd, 7th and 9th line where condition has been negated. Here however we can’t fix the error for this program.
Other cases where the mutation failed to give proper output is when operators are changed to other operators. In this and many algorithmic codes, mutations related to operator changes cannot be fixed. However mutation testing gives brief idea in some mutations as to where the test case is weak or there is an error in the source code. This link gives mutations performed by jumble
This article is contributed by Ipsit. If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.
Java
Java
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Stream In Java
Introduction to Java
Constructors in Java
Exceptions in Java
Generics in Java
Functional Interfaces in Java
Java Programming Examples
Strings in Java
Differences between JDK, JRE and JVM
Abstraction in Java