Elasticsearch How to Optimize Nested Aggregations in Elasticsearch

By Opster Team

Updated: Jun 18, 2023

| 2 min read

Introduction

Nested aggregations in Elasticsearch are a powerful tool for analyzing and summarizing complex, nested data structures. They allow you to perform aggregations on nested documents within a single query, providing valuable insights into your data. In this article, we will discuss how to optimize nested aggregations in Elasticsearch for better performance and scalability. If you want to learn about reverse_nested nested path ” + path + ” is not nested – how to solve this Elasticsearch error, check out this guide.

Understanding Nested Aggregations

Nested aggregations are used when dealing with nested data structures, where a document contains a list of other documents as a field. This is common in scenarios such as e-commerce, where an order document may contain a list of products as a nested field. To perform aggregations on these nested fields, you need to use the `nested` aggregation type.

Here’s an example of a nested aggregation that calculates the average price of products within an order:

GET /orders/_search
{
  "size": 0,
  "aggs": {
    "orders": {
      "nested": {
        "path": "products"
      },
      "aggs": {
        "average_price": {
          "avg": {
            "field": "products.price"
          }
        }
      }
    }
  }
}

Optimizing Nested Aggregations

1. Use Filtered Aggregations

Filtering the data before performing nested aggregations can significantly improve performance. By reducing the number of documents that need to be processed, you can minimize the overhead of the aggregation. Use the `filter` aggregation to apply a filter before performing the nested aggregation:

GET /orders/_search
{
  "size": 0,
  "aggs": {
    "filtered_orders": {
      "filter": {
        "range": {
          "order_date": {
            "gte": "now-30d"
          }
        }
      },
      "aggs": {
        "orders": {
          "nested": {
            "path": "products"
          },
          "aggs": {
            "average_price": {
              "avg": {
                "field": "products.price"
              }
            }
          }
        }
      }
    }
  }
}

2. Limit the Number of Buckets

Creating a large number of buckets can lead to high memory usage and slow performance. Limit the number of buckets by using the `size` parameter in the `terms` aggregation:

GET /orders/_search
{
  "size": 0,
  "aggs": {
    "orders": {
      "nested": {
        "path": "products"
      },
      "aggs": {
        "product_categories": {
          "terms": {
            "field": "products.category",
            "size": 10
          },
          "aggs": {
            "average_price": {
              "avg": {
                "field": "products.price"
              }
            }
          }
        }
      }
    }
  }
}

3. Use the `doc_count` Metric

Instead of calculating the count of documents in each bucket using a `sum` aggregation, use the `doc_count` metric provided by Elasticsearch. This metric is more efficient and can improve the performance of your nested aggregations:

GET /orders/_search
{
  "size": 0,
  "aggs": {
    "orders": {
      "nested": {
        "path": "products"
      },
      "aggs": {
        "product_categories": {
          "terms": {
            "field": "products.category"
          },
          "aggs": {
            "total_products": {
              "sum": {
                "field": "_doc_count"
              }
            }
          }
        }
      }
    }
  }
}

4. Use the `composite` Aggregation

The `composite` aggregation allows you to paginate through the results of a multi-bucket aggregation, reducing the memory usage and improving performance. This is particularly useful when dealing with large datasets:

GET /orders/_search
{
  "size": 0,
  "aggs": {
    "orders": {
      "nested": {
        "path": "products"
      },
      "aggs": {
        "product_categories": {
          "composite": {
            "size": 100,
            "sources": [
              {
                "category": {
                  "terms": {
                    "field": "products.category"
                  }
                }
              }
            ]
          },
          "aggs": {
            "average_price": {
              "avg": {
                "field": "products.price"
              }
            }
          }
        }
      }
    }
  }
}

Conclusion

Nested aggregations in Elasticsearch provide a powerful way to analyze and summarize complex, nested data structures. By optimizing your nested aggregations using the techniques discussed in this article, you can improve the performance and scalability of your Elasticsearch queries. Always consider filtering your data, limiting the number of buckets, using the `doc_count` metric, and leveraging the `composite` aggregation for better performance.