Geospatial Support in ElasticSearch

1. Introduction

In this article, we’ll learn how to use the geospatial capabilities of Elasticsearch.

We won’t delve into how to set up an Elasticsearch instance and the Java client. Instead, we’ll describe how to save geo-data and how we can search for it using geo queries.

Let’s dive into the available geo data types.

2. Geo Data Type

In Elasticsearch, we can work with two main types of geo data: a geo_point consists of latitude and longitude coordinates, and a geo_shape can depict different shapes like rectangles, lines, and polygons. We must manually create the index mapping and explicitly set the field mapping to use geo-queries to proceed. Furthermore, it’s important to note that dynamic mapping won’t work while setting mapping for geo types.

Moving forward, let’s delve into the specific details of each geo data type.

2.1. Geo Point Data Type

The simplest type is the geo_point, which represents a pair of latitude and longitude on a map. We can use a point in various ways, such as looking if it is inside a box or searching for objects in a specific range represented by a distance. Additionally, we could search for the indexed point within a query representing a complex geo_shape. As an illustration, we can view a geo_point as a pinpoint on a map.

Moreover, geo points allow us to group documents by location, including within specific regions or by distance from a specified point, and then sort the documents accordingly. For example, objects from near to our point to further away.

The first thing to remember is to create the mapping of geo_point using the properties of our index’s data type:

PUT /index_name
{
    "mappings": {
        "TYPE_NAME": {
            "properties": {
                "location": { 
                    "type": "geo_point" 
                } 
            } 
        } 
    } 
}

So, we are ready to send our data points to Elasticsearch.

2.2. Geo Shape Data Type

Unlike geo-point, geo-shape provides the functionality to save and search complex shapes like polygons and rectangles. To search documents that contain shapes other than geopoints, we must use a geo_shape data type.

Similarly, let’s map a geo shape in our index’s data type:

PUT /index_name
{
    "mappings": {
        "TYPE_NAME": {
            "properties": {
                "location": {
                    "type": "geo_shape"
                }
            }
        }
    }
}

Elasticsearch represents a geo_shape as a triangular mesh that allows it to provide a very high spatial resolution.

Next, we’ll look at how we can save the data inside our index.

3. Different Ways to Save Geo Point Data

Let’s assume we mapped a type location as a geo_point in our index.

3.1. Latitude Longitude Object

We can explicitly define the latitude and longitude of the points, providing them as keys to the location type:

PUT index_name/_doc
{
    "location": { 
        "lat": 23.02,
        "lon": 72.57
    }
}

This is the most readable method that doesn’t add any ambiguity.

3.2. Latitude Longitude Pair

We can reduce the verbosity of the previous method and define the latitude-longitude pair in plain string format:

{
    "location": "23.02,72.57"
}

To emphasize, string geopoints are ordered as lat, lon, while array geopoints, GeoJSON, and WKT are ordered as the reverse: lon, lat.

3.3. Longitude Latitude Array

Alternatively, we can provide the point as an array:

{
    "location": [72.57, 23.02]
}

It’s important to realize that the sequence of latitude-longitude is reversed when latitude and longitude are supplied as an array.

Initially, the latitude-longitude pair was used in both a string and an array, but later, it was reversed to match the format used by GeoJSON.

3.4. Geo Hash

At last, we can use geo hash instead of the explicit pair values to represent our point:

{
    "location": "tsj4bys"
}

Even though hash values are concise and great for proximity search, they are not very readable. For example, we can use the online tool to convert latitude-longitude to geo hash.

4. Different Ways to Save Geo Shape Data

Let’s assume we mapped a type region as a geo_shape in our index.

4.1. Point

We create first the most simple shape, which is a point:

POST /index/_doc
{
    "region" : {
        "type" : "point",
        "coordinates" : [72.57, 23.02]
    }
}

In short, inside the region field, we have a nested object consisting of field type and coordinates. In particular, these meta-fields help Elasticsearch identify the data.

4.2. LineString

Then, we are going to insert a linestring:

POST /index/_doc
{
    "region" : {
        "type" : "linestring",
        "coordinates" : [[77.57, 23.02], [77.59, 23.05]]
    }
}

In short, the coordinates for a linestring are two points that represent the starting point and the end of the line segment. Aggregating many LineString is helpful when creating navigation systems.

4.3. Polygon

Next, we’ll insert a polygon geo shape:

POST /index/_doc
{
    "region" : {
        "type" : "polygon",
        "coordinates" : [
            [ [10.0, 0.0], [11.0, 0.0], [11.0, 1.0], [10.0, 1.0], [10.0, 0.0] ]
        ]
    }
}

With attention to this example’s first and last coordinates, we must ensure the match for a closed polygon.

4.4. Other GeoJSON/WKT Formats

Finally, the list of GeoJSON/WKT structures supported by Elasticsearch is very rich:

• MultiPoint
• MultiLineString
• MultiPolygon
• GeometryCollection
• Envelope (This is not valid GeoJSON, but Elasticsearch and WKT support it)

Moreover, we can check all the supported formats on the official ES site.

To sum up, we must provide the inner type and coordinates fields to index documents correctly. Additionally, sorting and retrieving geo-shape fields are currently impossible in Elasticsearch due to their complex structure. Therefore, the only way to retrieve geo fields is from the source field.

5. Insert Geo Data in ElasticSearch

Now, let’s insert some documents and learn how to fetch them using geo queries. To begin with, we must add the Elastic search’s Java Client:

<dependency>
    <groupId>co.elastic.clients</groupId>
    <artifactId>elasticsearch-java</artifactId>
    <version>8.9.0</version>
</dependency>

5.1. Create an Index With Explicit Mappings

Before we can insert data, we need to define the mappings for our index:

client.indices().create(builder -> builder.index(WONDERS_OF_WORLD)
  .mappings(bl -> bl
    .properties("region", region -> region.geoShape(gs -> gs))
    .properties("location", location -> location.geoPoint(gp -> gp))
  )
);

Here, the client is an instance of the ElasticsearchClient object. In short, we are creating two data types. The first is a geo_shape named region, and the second is a geo_point named location.

5.2. Insert geo_point Documents

To begin with, we create a Java Class that represents will represent our geo_point data:

@Data
@AllArgsConstructor
@NoArgsConstructor
public class Location {
    private String name;
    private List<Double> location;
}

In detail, the name will refer to the location’s name, and the list location will be the two values representing the position. In addition, we are using Lombok to keep the code clean and concise.

Now, we can index a new document using the index() method:

Location pyramidsOfGiza = new Location("Pyramids of Giza", List.of(31.1328, 29.9761));
IndexResponse response = client.index(builder -> builder
  .index(WONDERS_OF_WORLD)
  .document(pyramidsOfGiza));

Also, the .document() will automatically convert the Location object to a valid JSON. Alternatively, we can work directly with JSON strings and use .withJson() and provide the string as a StringReader:

String jsonObject = """
    {
        "name":"Lighthouse of alexandria",
        "location":{ "lat": 31.2139, "lon": 29.8856 }
    }
    """;
IndexResponse response = client.index(idx -> idx
  .index(WONDERS_OF_WORLD)
  .withJson(new StringReader(jsonObject)));

5.3. Insert geo_shape Documents

Next, to insert geo_shape documents, we can work with JSON strings directly:

String jsonObject = """
    {
        "name":"Agra",
        "region":{
            "type":"envelope",
            "coordinates":[[75,30.2],[80.1,25]]
        }
    }
    """;
IndexResponse response = client.index(idx -> idx
    .index(WONDERS_OF_WORLD)
    .withJson(new StringReader(jsonObject)));

Now, we are ready to make some queries to search for our data.

6. Query Geo Data in ElasticSearch

6.1. Geo Bounding Box Query

To begin with, suppose we have a bunch of geo points on a map, and we want to find them in a rectangular area. Then, we must use a bounding box query to fetch all the points:

{
   "query":{
      "geo_bounding_box":{
         "location":{
            "top_left":[30.0,31.0],
            "bottom_right":[32.0,28.0]
         }
      }
   }
}

Further, we can create a SearchRequest in our project for this purpose:

SearchRequest.Builder builder = new SearchRequest.Builder().index(WONDERS_OF_WORLD);
builder.query(query -> query
  .geoBoundingBox(geoBoundingBoxQuery ->
    geoBoundingBoxQuery.field("location")
      .boundingBox(geoBounds -> geoBounds.tlbr(bl4 -> bl4
        .topLeft(geoLocation -> geoLocation.coords(List.of(30.0, 31.0)))
        .bottomRight(geoLocation -> geoLocation.coords(List.of(32.0, 28.0))))
      )
  )
);

Moreover, the Geo Bounding Box query supports similar formats as we have in the geo_point data type. Additionally, sample queries for supported formats can be found on the official site.

Finally, we use the SearchRequest to query ElasticSearch:

SearchResponse<Location> searchResponse = client.search(build, Location.class);
log.info("Search response: {}", searchResponse);

6.2. Geo Shape Query

Next, to query geo_shape documents, we must use GeoJSON.

For example, we might want to find all the documents that fall within specific coordinates:

{
  "query":{
    "bool":{
      "filter":[
        {
          "geo_shape":{
            "region":{
              "shape":{
                "type":"envelope",
                "coordinates":[[74.0,31.2],[81.1,24.0]]
              },
              "relation":"within"
            }
          }
        }
      ]
    }
  }
}

In detail, the relation field in the query determines spatial relation operators used at search time. So, we can choose from a list of operators:

• INTERSECTS – (default) returns all documents whose geo_shape field intersects the query geometry
• DISJOINT – retrieves all documents whose geo_shape field has nothing in common with the query geometry
• WITHIN – gets all documents whose geo_shape field is within the query geometry
• CONTAINS – returns all documents whose geo_shape field contains the query geometry

Similarly, we can query using different GeoJSON shapes.

For example, the query above can be implemented as the following SearchRequest:

StringReader jsonData = new StringReader("""
    {
        "type":"envelope",
        "coordinates": [[74.0, 31.2], [81.1, 24.0 ] ]
    }
    """);

SearchRequest searchRequest = new SearchRequest.Builder()
  .query(query -> query.bool(boolQuery -> boolQuery
    .filter(query1 -> query1
      .geoShape(geoShapeQuery -> geoShapeQuery.field("region")
        .shape(
          geoShapeFieldQuery -> geoShapeFieldQuery.relation(GeoShapeRelation.Within)
            .shape(JsonData.from(jsonData))
  ))))).build();

Similarly, to query the data, we can call search() using the SearchRequest:

SearchResponse<Object> search = client.search(searchRequest, Object.class);
log.info("Search response: {}", search);

To point out the source of the SearchResponse maps to a generic Object class. Given that, geo_shapes can have various forms, and we don’t know what the query might return beforehand.

6.3. Geo Distance Query

Next, to find all the documents that come within a specified range of a point, we use a geo_distance query:

{
    "query":{
        "geo_distance":{
          "location":{
              "lat":29.976,
              "lon":31.131
          },
          "distance":"10 miles"
        }
    }
}

Likewise, we can implement the query above in Java using a SearchRequest:

SearchRequest searchRequest = new SearchRequest.Builder().index(WONDERS_OF_WORLD)
  .query(query -> query
    .geoDistance(geoDistanceQuery -> geoDistanceQuery
      .field("location").distance("10 miles")
      .location(geoLocation -> geoLocation
        .latlon(latLonGeoLocation -> latLonGeoLocation
          .lon(29.88).lat(31.21)))
    )
  ).build();

Like geo_point, geo distance query supports multiple formats for location coordinates.

6.4. Geo Polygon Query

Next, we’ll create a polygon within which we want to find all the points. In particular, we’ll create a geo_shape query of shape geo_polygon:

{
    "query":{
        "bool":{
          "filter":[
            {
                "geo_shape":{
                    "location":{
                        "shape":{
                            "type":"polygon",
                             "coordinates":[[[68.859, 22.733],[68.859, 24.733],[70.859, 23]]]
                        },
                        "relation":"within"
                    }
                }
            }
          ]
        }
    }
}

Again, we can rewrite the query in Java:

JsonData jsonData = JsonData.fromJson("""
    {
        "type":"polygon",
        "coordinates":[[[68.859,22.733],[68.859,24.733],[70.859,23]]]
    }
    """);

SearchRequest build = new SearchRequest.Builder()
  .query(query -> query.bool(
    boolQuery -> boolQuery.filter(
      query1 -> query1.geoShape(geoShapeQuery -> geoShapeQuery.field("location")
        .shape(
          geoShapeFieldQuery -> geoShapeFieldQuery.relation(GeoShapeRelation.Within)
            .shape(jsonData)))))
 ).build();

Only the geo_point data type is supported with this query.

7. Conclusion

In this article, we discussed different mapping options for indexing geo data, i.e. geo_point and geo_shape.

We also went through different ways to store geo-data, and finally, we observed geo-queries and Java API to filter results using geo queries.