Briefly, this error occurs when Elasticsearch tries to sort on a field that doesn’t exist in the mapping. The mapping is a process that defines how a document and its fields are stored and indexed. To resolve this issue, you can either create a new index with the correct mapping that includes the field you want to sort on, or you can update the mapping of the existing index to include the new field. Remember, updating the mapping will not affect the existing documents, so you may need to reindex your data.

Before you dig into reading this guide, have you tried asking OpsGPT what this log means? You’ll receive a customized analysis of your log.

Try OpsGPT now for step-by-step guidance and tailored insights into your Elasticsearch/OpenSearch operation.

In addition we also recommend you try running the Elasticsearch Error Check-Up which can resolve issues that cause many errors.
This guide will help you understand the origin of the log “No mapping found for [field name] in order to sort on”, why it happens, with which concepts of Elasticsearch it is related and then give you some ideas on how to debug and avoid it.

Overview

Elasticsearch informs you with this error message that it could not fulfil your search request because one of the fields listed on the sort part of your request is not mapped in the index.

What it means

When you create a new index, you have two choices on how the fields of that index will be mapped:

• Dynamic mapping: allows you to experiment with and explore data when you’re just getting started. Elasticsearch adds new fields automatically, just by indexing a document.
• Explicit mapping: allows you to precisely choose how to define the mapping definition.

Any field that was not explicitly mapped and that was not yet included in any of the documents already indexed will not have been mapped, meaning Elasticsearch couldn’t know its data type. In that case, Elasticsearch throws an error when this unmapped field is included as one of the fields by which the results should be sorted.

We can easily replicate this by creating an index, indexing a document and then sending a search request sorting by some field not included in that document:

PUT test
 
POST test/_doc
{
  "id": "1",
  "i_value": -1,
  "d_value": -1.1
}
 
POST test/_search
{
  "sort": {
    "some_field": "desc"
  }
}
 
Result:
{
  "error" : {
    "root_cause" : [
      {
        "type" : "query_shard_exception",
        "reason" : "No mapping found for [some_field] in order to sort on",
        "index_uuid" : "Y-er-CNcRKiRMF5Ywd6PHA",
        "index" : "test"
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "test",
        "node" : "h4JFNQwSSvypf9vCF0ZLvQ",
        "reason" : {
          "type" : "query_shard_exception",
          "reason" : "No mapping found for [some_field] in order to sort on",
          "index_uuid" : "Y-er-CNcRKiRMF5Ywd6PHA",
          "index" : "test"
        }
      }
    ]
  },
  "status" : 400
}

On the other hand, notice that if the field was mapped, Elasticsearch would have no problem fulfilling our request:

PUT test
{
  "mappings": {
    "properties": {
      "some_field": {
        "type": "keyword"
      }
    }
  }
}
 
POST test/_doc
{
  "id": "1",
  "i_value": -1,
  "d_value": -1.1
}
 
POST test/_search
{
  "sort": {
    "some_field": "desc"
  }
}
 
 
Result:
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "BONmRXsBcaxskpj9pIrZ",
        "_score" : null,
        "_source" : {
          "id" : "1",
          "i_value" : -1,
          "d_value" : -1.1
        },
        "sort" : [
          null
        ]
      }
    ]
  }
}

Why it occurs

This error occurs because the sort part of your search request includes a field that is not yet mapped in the index (either dynamically, when the first document containing that field gets indexed, or explicitly, by defining the index’s schema beforehand).

One very common scenario in which this occurs is when you have several indices with possible different schemas hidden behind an alias. For example, this could be a time series observability solution that receives metrics from different subjects being monitored, each of them providing a different set of metrics. If your search use case expects all of them to comply with a common schema, but you actually never defined an index template that would map all the possible fields for every different sub-schema, then it may occur that some fields exist (and therefore are mapped) for some of those indices, but not for the others. If your user wants to sort for that field, the search request would fail.

In the example below, we created a couple of indices without mapping their fields first. We then hid them behind an alias and ran some queries on it trying to sort by fields that both exist and don’t exist in our source indices.

PUT house-sensors-doors-00001/_doc/1
{
  "sensor_id": 1,
  "subject_id": 1,
  "measured_at": "2021-08-28T10:42:00Z",
  "where": "hall",
  "description": "front door",
  "locked": true
}
 
PUT house-sensors-appliances-00001/_doc/2
{
  "sensor_id": 2,
  "subject_id": 2,
  "measured_at": "2021-08-28T10:43:00Z",
  "where": "kitchen",
  "description": "oven",
  "on": true,
  "temperature": 375,
  "temperature_scale": "Farenheit"
}
 
POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "house-sensors*",
        "alias": "house-sensors"
      }
    }
  ]
}

If you sort by a field that exists in both indices, the request will not fail:

GET house-sensors/_search
{
  "sort": {
    "measured_at": "desc"
  }
}
 
Response:
 
{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "house-sensors-appliances-00001",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : null,
        "_source" : {
          "sensor_id" : 2,
          "subject_id" : 2,
          "measured_at" : "2021-08-28T10:43:00Z",
          "where" : "kitchen",
          "description" : "oven",
          "on" : true,
          "temperature" : 375,
          "temperature_scale" : "Farenheit"
        },
        "sort" : [
          1630147380000
        ]
      },
      {
        "_index" : "house-sensors-doors-00001",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : null,
        "_source" : {
          "sensor_id" : 1,
          "subject_id" : 1,
          "measured_at" : "2021-08-28T10:42:00Z",
          "where" : "hall",
          "description" : "front door",
          "locked" : true
        },
        "sort" : [
          1630147320000
        ]
      }
    ]
  }
}

But if you try to sort by a field that does not exist in both indices, the request will partially fail:

GET house-sensors/_search
{
  "sort": {
    "temperature": "desc"
  }
}
 
Response:
 
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 1,
    "failures" : [
      {
        "shard" : 0,
        "index" : "house-sensors-doors-00001",
        "node" : "h4JFNQwSSvypf9vCF0ZLvQ",
        "reason" : {
          "type" : "query_shard_exception",
          "reason" : "No mapping found for [temperature] in order to sort on",
          "index_uuid" : "Vmp6BImnTnqOP6ALF8IxNQ",
          "index" : "house-sensors-doors-00001"
        }
      }
    ]
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "house-sensors-appliances-00001",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : null,
        "_source" : {
          "sensor_id" : 2,
          "subject_id" : 2,
          "measured_at" : "2021-08-28T10:43:00Z",
          "where" : "kitchen",
          "description" : "oven",
          "on" : true,
          "temperature" : 375,
          "temperature_scale" : "Farenheit"
        },
        "sort" : [
          375
        ]
      }
    ]
  }
}

How to resolve it

If the sort part of your search request includes an unmapped field, chances are it’s because:

1. You misspelled the field’s name.
2. It’s a field that used to exist, but was removed.
3. It’s a field that you expect to exist at some point, but since no document actually included it yet, and since the mapping of your index was probably not statically created, it remains unmapped.
4. It’s a new field present in recent indexes of a time series, but wasn’t present in older indices of the same time series and the time series is queried over an alias.

If you misspelled or if the field does not exist anymore, the fix should be easy: just correct it in your application. For the third and fourth scenarios, if you expect the field to exist at some point and want to keep as a sortable field in your application, you can get around it by explicitly mapping:

PUT test/_mapping
{
  "properties": {
    "some_field": {
      "type": "keyword"
    }
  }
}

That will prevent Elasticsearch from throwing the “No mapping found for [field name] in order to sort on”.

By the way, it’s always a good practice to explicitly map your indices and not rely on the dynamic mapping feature, because you sure know more about your data than Elasticsearch, as stated in its official documentation.   

Log Context

It might help knowing where in Elasticsearch code this error log is thrown. If we take a look at the FieldSortBuilder.java class for the codebase for 7.14 version of Elasticsearch we can easily see that it is thrown by the resolveUnmappedType() method as shown below:

   private MappedFieldType resolveUnmappedType(SearchExecutionContext context) {
        if (unmappedType == null) {
            throw new QueryShardException(context, "No mapping found for [" + fieldName + "] in order to sort on");
        }
        return context.buildAnonymousFieldType(unmappedType);
    }

In the FieldSortIT.java we can find the integration tests related to field sorting. If we take a look at the test described by the testIgnoreUnmapped() we can see that it expects that the “No mapping found for [field name] in order to sort on” is thrown when the sort part of your search request asks the cluster to sort by a field that was not mapped. In the case of this integration test they create an index, index a document with some fields and then execute a search request sorting by a field that was not mapped neither explicitly, nor by being included in the document indexed.

   public void testIgnoreUnmapped() throws Exception {
        createIndex("test");
 
        client().prepareIndex("test", "type1", "1").setSource(jsonBuilder().startObject()
                .field("id", "1")
                .field("i_value", -1)
                .field("d_value", -1.1)
                .endObject()).get();
 
        logger.info("--> sort with an unmapped field, verify it fails");
        try {
            SearchResponse result = client().prepareSearch()
                    .setQuery(matchAllQuery())
                    .addSort(SortBuilders.fieldSort("kkk"))
                    .get();
            assertThat("Expected exception but returned with", result, nullValue());
        } catch (SearchPhaseExecutionException e) {
            //we check that it's a parse failure rather than a different shard failure
            for (ShardSearchFailure shardSearchFailure : e.shardFailures()) {
                assertThat(shardSearchFailure.toString(), containsString("[No mapping found for [kkk] in order to sort on]"));
            }
        }
...