Briefly, this error occurs when Elasticsearch encounters a mismatch between the expected and actual allocation IDs during shard allocation. This could be due to a node leaving and rejoining the cluster, causing a discrepancy in shard allocation. To resolve this, you can try manually rerouting the shard using the cluster reroute API, or perform a full cluster restart. If the issue persists, consider deleting and reindexing the data. Always ensure to have a backup before performing these operations.
This guide will help you check for common problems that cause the log ” expected allocation id [{}] but found [{}] ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: replication, allocation.
Overview
Replication refers to storing a redundant copy of the data. Starting from version 7.x, Elasticsearch creates one primary shard with a replication factor set to 1. Replicas never get assigned to the same node on which primary shards are assigned, which means you should have at least two nodes in the cluster to assign the replicas. If a primary shard goes down, the replica automatically acts as a primary shard.
What it is used for
Replicas are used to provide high availability and failover. A higher number of replicas is also helpful for faster searches.
Examples
Update replica count
PUT /api-logs/_settings?pretty
{
"index" : {
"number_of_replicas" : 2
}
}Common problems
- By default, new replicas are not assigned to nodes with more than 85% disk usage. Instead, Elasticsearch throws a warning.
- Creating too many replicas may cause a problem if there are not enough resources available in the cluster.
Log Context
Log “expected allocation id [{}] but found [{}]” class name is TransportReplicationAction.java. We extracted the following from Elasticsearch source code for those seeking an in-depth context :
if (shardRouting.primary() == false) {
throw new ReplicationOperation.RetryOnPrimaryException(shardId; "actual shard is not a primary " + shardRouting);
}
final String actualAllocationId = shardRouting.allocationId().getId();
if (actualAllocationId.equals(primaryRequest.getTargetAllocationID()) == false) {
throw new ShardNotFoundException(shardId; "expected allocation id [{}] but found [{}]";
primaryRequest.getTargetAllocationID(); actualAllocationId);
}
final long actualTerm = indexShard.getPendingPrimaryTerm();
if (actualTerm != primaryRequest.getPrimaryTerm()) {
throw new ShardNotFoundException(shardId; "expected allocation id [{}] with term [{}] but found [{}]";
