Intro to JaCoCo

1. Overview

Code coverage is a software metric used to measure how many lines of our code are executed during automated tests.

In this tutorial, we’re going to stroll through some practical aspects of using JaCoCo, a code coverage reports generator for Java projects.

2. Maven Configuration

In order to get up and running with JaCoCo, we need to declare this maven plugin in our pom.xml file:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.7.7.201606060606</version>
    <executions>
        <execution>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
        <execution>
            <id>report</id>
            <phase>prepare-package</phase>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
    </executions>
</plugin>

The link provided here will always go to the latest version of the plugin in the maven central repository.

3. Code Coverage Reports

Before we start looking at JaCoCo’s code coverage capabilities, we need to have a code sample. Here’s a simple Java function that checks whether a string reads the same backward and forward:

public boolean isPalindrome(String inputString) {
    if (inputString.length() == 0) {
        return true;
    } else {
        char firstChar = inputString.charAt(0);
        char lastChar = inputString.charAt(inputString.length() - 1);
        String mid = inputString.substring(1, inputString.length() - 1);
        return (firstChar == lastChar) && isPalindrome(mid);
    }
}

Now all we need is a simple JUnit test:

@Test
public void whenEmptyString_thenAccept() {
    Palindrome palindromeTester = new Palindrome();
    assertTrue(palindromeTester.isPalindrome(""));
}

Running the test using JUnit will automatically set in motion the JaCoCo agent. It will create a coverage report in binary format in the target directory, target/jacoco.exec.

Obviously we can’t interpret the output single-handedly, but other tools and plugins can, e.g. Sonar Qube.

The good news is that we can use the jacoco:report goal in order to generate readable code coverage reports in several formats, like HTML, CSV, and XML.

For example, now we can take a look at the target/site/jacoco/index.html page to see what the generated report looks like:

Following the link provided in the report, Palindrome.java , we can drill through a more detailed view for each Java class:

Note that we can straightforwardly manage code coverage using JaCoCo inside Eclipse with zero configuration, thanks to the EclEmma Eclipse plugin.

4. Report Analysis

Our report shows 21% instructions coverage, 17% branches coverage, 3/5 for cyclomatic complexity, and so on.

The 38 instructions shown by JaCoCo in the report refer to the byte code instructions, as opposed to ordinary Java code instructions.

JaCoCo reports help us visually analyze code coverage by using diamonds with colors for branches, and background colors for lines:

• Red diamond means that no branches have been exercised during the test phase.
• Yellow diamond shows that the code is partially covered – some branches have not been exercised.
• Green diamond means that all branches have been exercised during the test.

The same color code applies to the background color, but for lines coverage.

JaCoCo mainly provides three important metrics:

• Lines coverage reflects the amount of code that has been exercised based on the number of Java byte code instructions called by the tests.
• Branches coverage shows the percent of exercised branches in the code, typically related to if/else and switch statements.
• Cyclomatic complexity reflects the complexity of code by giving the number of paths needed to cover all the possible paths in a code through linear combination.

To take a trivial example, if there are no if or switch statements in the code, the cyclomatic complexity will be 1, as we only need one execution path to cover the entire code.

Generally, the cyclomatic complexity reflects the number of test cases we need to implement in order to cover the entire code.

5. Concept Breakdown

JaCoCo runs as a Java agent. It’s responsible for instrumenting the byte code while running the tests. JaCoCo drills into each instruction, and shows which lines are exercised during each test.

To gather coverage data, JaCoCo uses ASM for code instrumentation on the fly, receiving events from the JVM Tool Interface in the process:

It’s also possible to run the JaCoCo agent in server mode. In this case, we can run our tests with jacoco:dump as a goal in order to initiate a dump request.

We can follow the official documentation link for more in-depth details about JaCoCo design.

6. Code Coverage Score

Now that we know a bit about how JaCoCo works, let’s improve our code coverage score.

In order to achieve 100% code coverage, we need to introduce tests that cover the missing parts shown in the initial report:

@Test
public void whenPalindrom_thenAccept() {
    Palindrome palindromeTester = new Palindrome();
    assertTrue(palindromeTester.isPalindrome("noon"));
}
    
@Test
public void whenNearPalindrom_thenReject(){
    Palindrome palindromeTester = new Palindrome();
    assertFalse(palindromeTester.isPalindrome("neon"));
}

Now we have enough tests to cover our the entire code, but to make sure of that, let’s run the Maven command mvn jacoco:report to publish the coverage report:

As we can see, all the lines/branches/paths in our code are fully covered:

In a real world project, as developments go further, we need to keep track of the code coverage score.

JaCoCo offers a simple way of declaring minimum requirements that should be met, otherwise the build will fail.

We can do that by adding the following check goal in our pom.xml file:

<execution>
    <id>jacoco-check</id>
    <goals>
        <goal>check</goal>
    </goals>
    <configuration>
        <rules>
            <rule>
                <element>PACKAGE</element>
                <limits>
                    <limit>
                        <counter>LINE</counter>
                        <value>COVEREDRATIO</value>
                        <minimum>0.50</minimum>
                    </limit>
                </limits>
            </rule>
        </rules>
    </configuration>
</execution>

As we can see, we’re limiting the minimum score for lines coverage to 50%.

The jacoco:check goal is bound to verify, so we can run the Maven command mvn clean verify to check whether the rules are respected or not. The logs will show something like:

[ERROR] Failed to execute goal org.jacoco:jacoco-maven-plugin:0.7.7.201606060606:check 
  (jacoco-check) on project mutation-testing: Coverage checks have not been met.

7. Conclusion

In this article, we learned how to make use of the JaCoCo maven plugin to generate code coverage reports for Java projects.

Keep in mind though, 100% code coverage doesn’t necessarily reflect effective testing, as it only reflects the amount of code exercised during tests. In a previous article, we talked about mutation testing as a more sophisticated way to track the effectiveness of tests compared to ordinary code coverage.