Briefly, this error occurs when the Elasticsearch Bulk Processor reaches its maximum capacity and can’t accept more data, leading to data loss as it starts dropping events. This could be due to high data ingestion rate or slow processing speed. To resolve this, you can increase the bulk size or the number of concurrent requests allowed in the Bulk Processor. Alternatively, optimize your Elasticsearch cluster by adding more nodes or increasing hardware resources. Also, consider improving your data ingestion strategy by using a queue or buffer system to handle peak loads.
This guide will help you check for common problems that cause the log ” Bulk processor is full. Start dropping events. ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: plugin, bulk.
Overview
In Elasticsearch, when using the Bulk API it is possible to perform many write operations in a single API call, which increases the indexing speed. Using the Bulk API is more efficient than sending multiple separate requests. This can be done for the following four actions:
- Index
- Update
- Create
- Delete
Examples
The bulk request below will index a document, delete another document, and update an existing document.
POST _bulk
{ "index" : { "_index" : "myindex", "_id" : "1" } }
{ "field1" : "value" }
{ "delete" : { "_index" : "myindex", "_id" : "2" } }
{ "update" : {"_id" : "1", "_index" : "myindex"} }
{ "doc" : {"field2" : "value5"} }Notes
- Bulk API is useful when you need to index data streams that can be queued up and indexed in batches of hundreds or thousands, such as logs.
- There is no correct number of actions or limits to perform on a single bulk call, but you will need to figure out the optimum number by experimentation, given the cluster size, number of nodes, hardware specs etc.
Log Context
Log “Bulk processor is full. Start dropping events.” classname is AnalyticsEventEmitter.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :
listener.onFailure(
new ElasticsearchStatusException("Unable to add the event: too many requests."; RestStatus.TOO_MANY_REQUESTS)
);
if (dropEvent.compareAndSet(false; true)) {
logger.warn("Bulk processor is full. Start dropping events.");
}
}
}
private IndexRequest createIndexRequest(AnalyticsEvent event) throws IOException {
