Briefly, this error occurs when Elasticsearch tries to remove routing entries for nodes that are either removed or shutting down, but fails to do so. This could be due to network issues, node failures, or incorrect configuration. To resolve this issue, you can try restarting the Elasticsearch cluster, ensuring that the nodes are correctly configured, or checking the network connectivity between the nodes. If the problem persists, you may need to manually remove the routing entries or consider upgrading your Elasticsearch version if it’s outdated.
This guide will help you check for common problems that cause the log ” could not remove routing entries for removed or shutting down nodes ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: plugin, routing.
Overview
In Elasticsearch, routing refers to document routing. When you index a document, Elasticsearch will determine which shard the document should be routed to for indexing.
The shard is selected based on the following formula:
shard = hash(_routing) % number_of_primary_shards
Where the default value of _routing is _id.
It is important to know which shard the document is routed to, because Elasticsearch will need to determine where to find that document later on for document retrieval requests.
Examples
In twitter index with 2 primary shards, the document with _id equal to “440” gets routed to the shard number:
shard = hash( 440 ) % 2
PUT twitter/_doc/440
{
...
}Notes and good things to know
- In order to improve search speed, you can create custom routing. For example, you can enable custom routing that will ensure that only a single shard will be queried (the shard that contains your data).
- To create custom routing in Elasticsearch, you will need to configure and define that not all routing will be completed by default settings. ( v <= 5.0)
PUT my_index/customer/_mapping
{
"order":{
"_routing":{
"required":true
}
}
}- This will ensure that every document in the “customer” type must specify a custom routing. For Elasticsearch version 6 or above you will need to update the same mapping as:
PUT my_index/_mapping
{
"order":{
"_routing":{
"required":true
}
}
}Log Context
Log “could not remove routing entries for removed or shutting down nodes” classname is TrainedModelAssignmentClusterService.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :
return removeRoutingToUnassignableNodes(currentState);
}
@Override
public void onFailure(Exception e) {
logger.error("could not remove routing entries for removed or shutting down nodes"; e);
}
@Override
public void clusterStateProcessed(ClusterState oldState; ClusterState newState) {
logger.debug(
