Code Coverage with JaCoCo and SonarQube
By: Priyanka Gour
Tags: Jacoco, Automation, Sonarqube, Quality Assurance, Airtel

At Airtel X Labs, We, Quality Assurance engineers, are responsible for ensuring that softwares are released with zero defects and adhere to the highest quality standards. To improve our efficiency, we implement different technologies, tools and techniques and make sure that the automated test suite is robust enough to discover all loop holes before the code goes live. Adequate code coverage is one of the key milestones that we follow as a practice.

JaCoCo and SonarQube are two important tools necessary to implement this practice. When I researched online, I realised that the information is too scattered and there is no single source for starters to learn the basics of code coverage implementation. Hence I have written this blog to provide all the information required for JaCoCo and SonarQube integration at one place. I have gathered information from other blogs and integrated it with my own experience at Airtel X Labs, to give a consolidated account and make life easier for other Engineers.

What is Code Coverage?

1. Code Coverage is a measurement of how many lines, statements, or blocks of your code are tested using your suite of automated tests.

2. It’s an essential metric to understand the quality of QA efforts.

3. Code coverage shows you how much of your application is not covered by automated tests and is therefore vulnerable to defects.

4. Code coverage is typically measured in percentage values — the closer to 100%, the better.

How to Choose a Code Coverage Tool ?

When choosing a code coverage tool, there are many things to consider.

1. First, you should consider your programming language (on which language does your application built on).

2. You should also consider whether an open source tool or a commercial tool is right for you.

3. A code coverage tool should be well-integrated with a broad range of development and QA tools that you already use so that your team is likely to adopt it readily and the code coverage metrics it provides are useful.

4. When using a code coverage tool, you need easy access to reports and metrics.

5. Finally, look at code coverage as one aspect of your overall QA Strategy.

What is JaCoCo and why you need this?

JaCoCo is an abbreviation for Java Code Coverage. It’s free to open source library developed by EclEmma. This tool helps us in finding out the piece of code which is not used at all while running the system, it can be manual or by some sort automatic test.Attaching JaCoCo to your unit test makes sure that you haven’t missed out any unwanted untested code.It gives you an estimate of how many times your code got executed.

Setting up SonarQube:

Use below link for SonarQube installation:

Setting up JaCoCo:

There are multiple ways of running JaCoCo for your project:

Install it on Eclipse.

Install using maven.

Install standalone JaCoCo java agent on any running java process or server.

Install it on Eclipse

Install from update sites.

Go to Help-> Install new software and put in below URL.

2. Install using maven.

If you are already using maven with your project and want to generate reports using simple maven commands then you can go with this installation.

Steps to install JaCoCo and run reports.

Open pom.xml and add following lines.

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<version>0.8.1-SNAPSHOT</version>undefined</plugin>undefined<reporting>

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<reportSets>

<reportSet>

<reports>

<report>report</report>

</reports>

</reportSet>

</reportSets>undefined</plugin>undefined</plugins>undefined</reporting>undefined</project>

Run the maven goals to generate reports.

mvn jacoco:report

3. Install standalone JaCoCo java agent on any running java process or server.

This suits you well when you want to run JaCoCo on your live or test environments directly. Also in case, you do not have test setups or local workspace setup.

There is standalone JaCoCo jar which you can directly plugin in your remote Java process using command line arguments.

Once the java agent is attached to your server process you can open up a port to fetch reports or you can directly go to your server and ask for report dump. This report dump can be in any format CSV, XML, *.exec, HTML or zip. In order to use it with Sonarqube, we have export the report in *.exec format.

Here are the commands you to attach JaCoCo to a running Java process.

-javaagent:C:\dev\servers\jacoco-agent.jar=destfile=< path to jacoco C:\lib>jacoco.exec,append=true,includes=some.application.*

This Java argument will attach the JaCoCo agent. Once you have enabled the JaCoCo agent you need to run tests manually or automated.

This will create jacoco.exec on given location but dump data only on server stop.

To avoid server interruption for the report you need to enable command interface using following option.

output=<cpserver>,address=*,port=<some port 6300>

Below are the step by step process :

Step 1: Add below properties/Dependencies/Plugins in all the pom of your application for code coverage:

Properties: <sonar.language>java</sonar.language>

<sonar.sourceEncoding>UTF-8</sonar.sourceEncoding>

<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>

<jacoco.version>0.7.9</jacoco.version>

<sonar-jacoco-listeners.version>3.2</sonar-jacoco-listeners.version>

<jacoco.outputDir>${project.build.directory}</jacoco.outputDir>

<jacoco.out.ut.file>jacoco.exec</jacoco.out.ut.file>

<sonar.jacoco.reportPath>${jacoco.outputDir}/${jacoco.out.ut.file}</sonar.jacoco.reportPath>

<org.json.version>20180813</org.json.version>

<sonar.coverage.exclusions></sonar.coverage.exclusions> Note:- You can add files for exclusions in -Dsonar.coverage.exclusions argument or in<sonar.coverage.exclusions></sonar.coverage.exclusions>(only folder level and class level exclusion is available in jacoco, we cannot exclude methods) Dependency: <dependency>

<groupId>org.codehaus.sonar-plugins.java</groupId>

<artifactId>sonar-jacoco-listeners</artifactId>

<version>${sonar-jacoco-listeners.version}</version>

<scope>test</scope>

</dependency> Plugins: <plugins>

<plugin>

<groupId>org.sonarsource.scanner.maven</groupId>

<artifactId>sonar-maven-plugin</artifactId>

<version>3.4.0.905</version>

</plugin>

<plugin>

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<version>${jacoco.version}</version>

<executions>

<! — Prepares a variable, jacoco.agent.ut.arg, that contains the info

to be passed to the JVM hosting the code being tested. →

<execution>

<id>prepare-ut-agent</id>

<phase>process-test-classes</phase>

<goals>

<goal>prepare-agent</goal>

</goals>

<configuration>

<propertyName>jacoco.agent.ut.arg</propertyName>

<append>true</append>

</configuration>

</execution>

</executions>

</plugin>

<plugin>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-surefire-plugin</artifactId>

<configuration>

<argLine>${jacoco.agent.ut.arg}</argLine>

<! — Specific to generate mapping between tests and covered code →

<properties>

<property>

<name>listener</name>

<value>org.sonar.java.jacoco.JUnitListener</value>

</property>

</properties>

<! — test failure ignore →

<! — <testFailureIgnore>true</testFailureIgnore> →

</configuration>

</plugin>

</plugins>

Step 2: Copy JaCoCo agent jar on the server (https://mvnrepository.com/artifact/org.jacoco/org.jacoco.agent)

Step 3: Add below arguments in the application service (Spring boot) and Catalina.sh (Tomcat)

and in Environments->Servers -> NAME -> Start Server -> Arguments (Webogic)

-javaagent:/app/jacoco-agent/lib/jacocoagent.jar=destfile=/tmpjacoco.exec,append=true,includes=com.airtel.*,jmx=true,dumponexit=true

Example of Spring boot Service with JaCoCo configuration:

[Unit]

Description=EXAMPLE

After=syslog.target network.target

[Service]

User=root

EnvironmentFile=/etc/profile.d/proxy.sh

ExecStart=/usr/bin/java -Dcom.sun.management.jmxremote.port=9911 -Dcom.sun.management.jmxremote.rmi.port=9124 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dserver.port=8001 -Dlog_path=/app/logs/ -Dspring.profiles.active=qa -Dspring.config.location=/data/example/properties/ -Dspring.config.name=application -javaagent:/app/jacoco-agent/lib/jacocoagent.jar=output=file,destfile=/tmp/jacoco.exec,append=true,includes=com.airtel.*,jmx=true,dumponexit=true -jar /app/releases/example.jar

ExecStop=/bin/kill -TERM $MAINPID

Group=root

[Install]

WantedBy=multi-user.target

Step 4: Deploy the application testing branch on the server

Step 5: Restart Tomcat/Weblogic/Springboot application service

Step 6: Execute the tests (integration/Automation/junit)

Step 7: Execute below command on the server

Change the name of the file every time(e.g. jacocoLog.exec)

java -jar -Djacoco.dest=/tmp/ jacocoLog.exec /app/jacoco-agent/lib/runtime-code-coverage.jar

Example:

java -Djacoco.dest=/tmp/testJacoco.exec -Djmx.port=9911 -Dservice.url=$SERVER_IP -jar /app/solace-app-jar-with-dependencies.jar

Step 8: Copy the above generated file (jacocoLog.exec) to the Jenkins server where the target folder of the code resides,

using below command

scp /tmp/ jacocoLog.exec jenkins@$SERVER_IP:/root/jenkins/workspace/workspace_name/target

(Where pom.xml of application code resides)

Step 9: Now the execute below command to find the code coverage (jacocoLog.exec should be replace by the name of your file):

mvn — settings /opt/maven/conf/settings.xml sonar:sonar -Dsonar.jacoco.reportPaths=/root/jenkins/workspace/workspace_name/target jacocoLog.exec -Dsonar.projectKey=project-name -Dsonar.coverage.exclusions=**/model/*.java,**/enums/*.java,**/constants/*.java

Example:

mvn — settings $mavenSettingsFile sonar:sonar -Djmx.port=$JMX_PORT -Dsonar.host.url=http://$SONARQUBESERVER_IP:9000 -Dsonar.jacoco.reportPaths=$deploymentDirectory/testJacoco.exec -Dsonar.projectName=projectName -Dsonar.projectKey=projectName -Dsonar.coverage.exclusions=**/dto/**,**/entity/**

You can check the results on below link:

http://sonarQubeIp:9000/dashboard?id=projectName

Sonar report will look like:

Thanks to Anirudh Bhardwaj for guiding us and Airtel X Labs co-workers who have collaborated to make this happen.