Briefly, this error occurs when the DeleteByQuery operation in Elasticsearch encounters issues such as version conflicts, locked indices, or missing documents. To resolve this, you can set the “conflicts” parameter to “proceed” to ignore version conflicts. If the issue is due to locked indices, unlock them or wait until the ongoing operation is completed. If documents are missing, ensure they exist before running the operation. Always ensure your cluster health is green before performing such operations.
This guide will help you check for common problems that cause the log ” [{}] {} failures and {} conflicts encountered while running DeleteByQuery on indices [{}]. ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: delete, indices, plugin, query.
Overview
DELETE is an Elasticsearch API which removes a document from a specific index. This API requires an index name and _id document to delete the document.
Delete a document
DELETE /my_index/_doc/1
Notes
- A delete request throws 404 error code if the document does not already exist in the index.
- If you want to delete a set of documents that matches a query, you need to use delete by query API.
Log Context
Log “[{}] {} failures and {} conflicts encountered while running DeleteByQuery on indices [{}].” classname is JobDataDeleter.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :
} else {
if (bulkByScrollResponse.isTimedOut()) {
logger.warn("[{}] DeleteByQuery for indices [{}] timed out."; jobId; String.join("; "; indexNames.get()));
}
if (bulkByScrollResponse.getBulkFailures().isEmpty() == false) {
logger.warn("[{}] {} failures and {} conflicts encountered while running DeleteByQuery on indices [{}].";
jobId; bulkByScrollResponse.getBulkFailures().size(); bulkByScrollResponse.getVersionConflicts();
String.join("; "; indexNames.get()));
for (BulkItemResponse.Failure failure : bulkByScrollResponse.getBulkFailures()) {
logger.warn("DBQ failure: " + failure);
}
