Introduction to JsonPath

1. Overview

One of the advantages of XML is the availability of processing — including XPath — which is defined as a W3C standard. For JSON, a similar tool called JSONPath has emerged.

This tutorial will give an introduction to Jayway JsonPath, a Java implementation of the JSONPath specification. It describes setup, syntax, common APIs and a demonstration of use cases.

2. Setup

To use JsonPath, we simply need to include a dependency in the Maven pom:

<dependency>
    <groupId>com.jayway.jsonpath</groupId>
    <artifactId>json-path</artifactId>
    <version>2.9.0</version>
</dependency>

3. Syntax

We’ll use the following JSON structure to demonstrate the syntax and APIs of JsonPath:

{
    "tool": 
    {
        "jsonpath": 
        {
            "creator": 
            {
                "name": "Jayway Inc.",
                "location": 
                [
                    "Malmo",
                    "San Francisco",
                    "Helsingborg"
                ]
            }
        }
    },

    "book": 
    [
        {
            "title": "Beginning JSON",
            "price": 49.99
        },

        {
            "title": "JSON at Work",
            "price": 29.99
        }
    ]
}

3.1. Notation

JsonPath uses special notation to represent nodes and their connections to adjacent nodes in a JsonPath path. There are two styles of notation: dot and bracket.

Both of the following paths refer to the same node from the above JSON document, which is the third element within the location field of creator node, that is a child of the jsonpath object belonging to tool under the root node.

First, we’ll see the path with dot notation:

$.tool.jsonpath.creator.location[2]

Now let’s look at bracket notation:

$['tool']['jsonpath']['creator']['location'][2]

The dollar sign ($) represents root member object.

3.2. Operators

We have several helpful operators in JsonPath:

• Root node ($) denotes the root member of a JSON structure whether it is an object or array. We included usage examples in the previous subsection.
• Current node (@) represents the node being processed. We mostly use it as part of input expressions for predicates. Suppose we are dealing with book array in the above JSON document; the expression book[?(@.price == 49.99)] refers to the first book in that array.
• Wildcard (*) expresses all elements within the specified scope. For instance, book[*] indicates all nodes inside a book array.

3.3. Functions and Filters

JsonPath also has functions that we can use at the end of a path to synthesize that path’s output expressions: min(), max(), avg(), stddev() and length().

Finally, we have filters. These are boolean expressions to restrict returned lists of nodes to only those that calling methods need.

A few examples are equality (==), regular expression matching (=~), inclusion (in) and check for emptiness (empty). We mainly use filters for predicates.

For a full list and detailed explanations of different operators, functions and filters, please refer to the JsonPath GitHub project.

4. Operations

Before we get into operations, a quick sidenote: This section makes use of the JSON example structure we defined earlier.

4.1. Access to Documents

JsonPath has a convenient way to access JSON documents. We do this through static read APIs:

<T> T JsonPath.read(String jsonString, String jsonPath, Predicate... filters);

The read APIs can work with static fluent APIs to provide more flexibility:

<T> T JsonPath.parse(String jsonString).read(String jsonPath, Predicate... filters);

We can use other overloaded variants of read for different types of JSON sources, including Object, InputStream, URL and File.

To make things simple, the test for this part does not include predicates in the parameter list (empty varargs). But we’ll discuss predicates in later subsections.

Let’s start by defining two sample paths to work on:

String jsonpathCreatorNamePath = "$['tool']['jsonpath']['creator']['name']";
String jsonpathCreatorLocationPath = "$['tool']['jsonpath']['creator']['location'][*]";

Next, we will create a DocumentContext object by parsing the given JSON source jsonDataSourceString. The newly created object will then be used to read content using the paths defined above:

DocumentContext jsonContext = JsonPath.parse(jsonDataSourceString);
String jsonpathCreatorName = jsonContext.read(jsonpathCreatorNamePath);
List<String> jsonpathCreatorLocation = jsonContext.read(jsonpathCreatorLocationPath);

The first read API returns a String containing the name of the JsonPath creator, while the second returns a list of its addresses.

And we’ll use the JUnit Assert API to confirm the methods work as expected:

assertEquals("Jayway Inc.", jsonpathCreatorName);
assertThat(jsonpathCreatorLocation.toString(), containsString("Malmo"));
assertThat(jsonpathCreatorLocation.toString(), containsString("San Francisco"));
assertThat(jsonpathCreatorLocation.toString(), containsString("Helsingborg"));

4.2. Predicates

Now that we have the basics, let’s define a new JSON example to work on and illustrate how to create and use predicates:

{
    "book": 
    [
        {
            "title": "Beginning JSON",
            "author": "Ben Smith",
            "price": 49.99
        },

        {
            "title": "JSON at Work",
            "author": "Tom Marrs",
            "price": 29.99
        },

        {
            "title": "Learn JSON in a DAY",
            "author": "Acodemy",
            "price": 8.99
        },

        {
            "title": "JSON: Questions and Answers",
            "author": "George Duckett",
            "price": 6.00
        }
    ],

    "price range": 
    {
        "cheap": 10.00,
        "medium": 20.00
    }
}

Predicates determine true or false input values for filters to narrow down returned lists to only matched objects or arrays. We can easily integrate a Predicate into a Filter by using it as an argument for its static factory method. The requested content can then be read out of a JSON string using that Filter:

Filter expensiveFilter = Filter.filter(Criteria.where("price").gt(20.00));
List<Map<String, Object>> expensive = JsonPath.parse(jsonDataSourceString)
  .read("$['book'][?]", expensiveFilter);
predicateUsageAssertionHelper(expensive);

We could also define our customized Predicate and use it as an argument for the read API:

Predicate expensivePredicate = new Predicate() {
    public boolean apply(PredicateContext context) {
        String value = context.item(Map.class).get("price").toString();
        return Float.valueOf(value) > 20.00;
    }
};
List<Map<String, Object>> expensive = JsonPath.parse(jsonDataSourceString)
  .read("$['book'][?]", expensivePredicate);
predicateUsageAssertionHelper(expensive);

Finally, a predicate can be directly applied to read API without the creation of any objects, which is called an inline predicate:

List<Map<String, Object>> expensive = JsonPath.parse(jsonDataSourceString)
  .read("$['book'][?(@['price'] > $['price range']['medium'])]");
predicateUsageAssertionHelper(expensive);

All three of the Predicate examples above are verified with the help of the following assertion helper method:

private void predicateUsageAssertionHelper(List<?> predicate) {
    assertThat(predicate.toString(), containsString("Beginning JSON"));
    assertThat(predicate.toString(), containsString("JSON at Work"));
    assertThat(predicate.toString(), not(containsString("Learn JSON in a DAY")));
    assertThat(predicate.toString(), not(containsString("JSON: Questions and Answers")));
}

5. Configuration

5.1. Options

Jayway JsonPath provides several options to tweak the default configuration:

• Option.AS_PATH_LIST returns paths of the evaluation hits instead of their values.
• Option.DEFAULT_PATH_LEAF_TO_NULL returns null for missing leaves.
• Option.ALWAYS_RETURN_LIST returns a list even when the path is definite.
• Option.SUPPRESS_EXCEPTIONS makes sure no exceptions are propagated from path evaluation.
• Option.REQUIRE_PROPERTIES requires properties defined in the path when an indefinite path is evaluated.

Here is how to apply Option from scratch:

Configuration configuration = Configuration.builder().options(Option.<OPTION>).build();

and how to add it to an existing configuration:

Configuration newConfiguration = configuration.addOptions(Option.<OPTION>);

5.2. SPIs

JsonPath’s default configuration with the help of Option should be enough for the majority of tasks. However, users with more complex use cases can modify the behavior of JsonPath according to their specific requirements — using three different SPIs:

• JsonProvider SPI lets us change the ways JsonPath parses and handles JSON documents.
• MappingProvider SPI allows for customization of bindings between node values and returned object types.
• CacheProvider SPI adjusts the manners that paths are cached, which can help to increase performance.

6. Example Use Cases

We now have a good understanding of the JsonPath functionality. So, let’s look at an example.

This section illustrates dealing with JSON data returned from a web service.

Assume we have a movie information service that returns the following structure:

[
    {
        "id": 1,
        "title": "Casino Royale",
        "director": "Martin Campbell",
        "starring": 
        [
            "Daniel Craig",
            "Eva Green"
        ],
        "desc": "Twenty-first James Bond movie",
        "release date": 1163466000000,
        "box office": 594275385
    },

    {
        "id": 2,
        "title": "Quantum of Solace",
        "director": "Marc Forster",
        "starring": 
        [
            "Daniel Craig",
            "Olga Kurylenko"
        ],
        "desc": "Twenty-second James Bond movie",
        "release date": 1225242000000,
        "box office": 591692078
    },

    {
        "id": 3,
        "title": "Skyfall",
        "director": "Sam Mendes",
        "starring": 
        [
            "Daniel Craig",
            "Naomie Harris"
        ],
        "desc": "Twenty-third James Bond movie",
        "release date": 1350954000000,
        "box office": 1110526981
    },

    {
        "id": 4,
        "title": "Spectre",
        "director": "Sam Mendes",
        "starring": 
        [
            "Daniel Craig",
            "Lea Seydoux"
        ],
        "desc": "Twenty-fourth James Bond movie",
        "release date": 1445821200000,
        "box office": 879376275
    }
]

where the value of release date field is milliseconds since the Epoch, and box office is revenue of a movie in the cinema in US dollars.

We are going to handle five different working scenarios related to GET requests, supposing that the above JSON hierarchy has been extracted and stored in a String variable named jsonString.

6.1. Getting Object Data Given IDs

In this use case, a client requests detailed information on a specific movie by providing the server with the movie’s exact id. This example demonstrates how the server looks for requested data before returning to the client.

Say we need to find a record with id equaling to 2.

The first step is to pick up the correct data object:

Object dataObject = JsonPath.parse(jsonString).read("$[?(@.id == 2)]");
String dataString = dataObject.toString();

The JUnit Assert API confirms the existence of several fields:

assertThat(dataString, containsString("2"));
assertThat(dataString, containsString("Quantum of Solace"));
assertThat(dataString, containsString("Twenty-second James Bond movie"));

6.2. Getting the Movie Title Given Starring

Let’s say we want to look for a movie starring an actress called Eva Green. The server needs to return title of the movie that includes Eva Green in the starring array.

The succeeding test will illustrate how to do that and validate the returned result:

@Test
public void givenStarring_whenRequestingMovieTitle_thenSucceed() {
    List<Map<String, Object>> dataList = JsonPath.parse(jsonString)
      .read("$[?('Eva Green' in @['starring'])]");
    String title = (String) dataList.get(0).get("title");

    assertEquals("Casino Royale", title);
}

6.3. Calculation of the Total Revenue

This scenario makes use of a JsonPath function called length() to figure out the number of movie records in order to calculate the total revenue of all the movies.

Let’s look at the implementation and testing:

@Test
public void givenCompleteStructure_whenCalculatingTotalRevenue_thenSucceed() {
    DocumentContext context = JsonPath.parse(jsonString);
    int length = context.read("$.length()");
    long revenue = 0;
    for (int i = 0; i < length; i++) {
        revenue += context.read("$[" + i + "]['box office']", Long.class);
    }

    assertEquals(594275385L + 591692078L + 1110526981L + 879376275L, revenue);
}

6.4. Highest Revenue Movie

This use case exemplifies the usage of a non-default JsonPath configuration option, namely Option.AS_PATH_LIST, to find out the movie with the highest revenue.

First, we need to extract a list of all the movies’ box office revenue. Then we convert it to an array for sorting:

DocumentContext context = JsonPath.parse(jsonString);
List<Object> revenueList = context.read("$[*]['box office']");
Integer[] revenueArray = revenueList.toArray(new Integer[0]);
Arrays.sort(revenueArray);

We can easily pick up the highestRevenue variable from the revenueArray sorted array and then use it for working out the path to the movie record with the highest revenue:

int highestRevenue = revenueArray[revenueArray.length - 1];
Configuration pathConfiguration = 
  Configuration.builder().options(Option.AS_PATH_LIST).build();
List<String> pathList = JsonPath.using(pathConfiguration).parse(jsonString)
  .read("$[?(@['box office'] == " + highestRevenue + ")]");

Based on that calculated path, we’ll determine and return the title of the corresponding movie:

Map<String, String> dataRecord = context.read(pathList.get(0));
String title = dataRecord.get("title");

The whole process is verified by the Assert API:

assertEquals("Skyfall", title);

6.5. Latest Movie of a Director

This example will illustrate how to figure out the last movie directed by a director named Sam Mendes.

To begin with, we create a list of all the movies directed by Sam Mendes:

DocumentContext context = JsonPath.parse(jsonString);
List<Map<String, Object>> samMendesMovies = context.read("$[?(@.director == 'Sam Mendes')]");

We can also define a Filter on the root node director and use it as a predicate:

Filter directorSamMendesFilter = Filter.filter(Criteria.where("director")
    .contains("Sam Mendes"));
List<Map<String, Object>> samMendesMovies = JsonPath.parse(jsonString)
    .read("$[?]", directorSamMendesFilter);

We then use that list for the extraction of release dates. Those dates will be stored in an array and then sorted:

List<Object> dateList = new ArrayList<>();
for (Map<String, Object> item : samMendesMovies) {
    Object date = item.get("release date");
    dateList.add(date);
}
Long[] dateArray = dateList.toArray(new Long[0]);
Arrays.sort(dateArray);

We use the lastestTime variable (the last element of the sorted array) in combination with the director field’s value to determine the title of the requested movie:

long latestTime = dateArray[dateArray.length - 1];
List<Map<String, Object>> finalDataList = context.read("$[?(@['director'] 
  == 'Sam Mendes' && @['release date'] == " + latestTime + ")]");
String title = (String) finalDataList.get(0).get("title");

The following assertion proves that everything works as expected:

assertEquals("Spectre", title);

7. Conclusion

This article covered fundamental features of Jayway JsonPath — a powerful tool to traverse and parse JSON documents.

Although JsonPath has some drawbacks, such as a lack of operators for reaching parent or sibling nodes, it can be highly useful in a lot of scenarios.