Briefly, this error occurs when Elasticsearch cannot find the latest audit template, which is essential for logging audit events. This could be due to misconfiguration or deletion of the template. To resolve this issue, you can 1) Check your Elasticsearch configuration to ensure it’s correctly set up for auditing. 2) If the template was accidentally deleted, you need to recreate it. 3) Ensure that the Elasticsearch cluster has sufficient resources, as lack of resources can sometimes cause this error. 4) If the problem persists, consider restarting your Elasticsearch cluster.
This guide will help you check for common problems that cause the log ” Latest audit template missing and audit message cannot be added to the backlog ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: template, plugin.
If you want to learn more about Elasticsearch templates, check out this guide.
Overview
A template in Elasticsearch falls into one of the two following categories and is indexed inside Elasticsearch using its dedicated endpoint:
- Index templates, which are a way to define a set of rules including index settings, mappings and an index pattern. The template is applied automatically whenever a new index is created with the matching pattern. Templates are also used to dynamically apply custom mapping for the fields which are not predefined inside existing mapping.
- Search templates, which help in defining templates for search queries using mustache scripting language. These templates act as a placeholder for variables defined inside the search queries.
Examples
Create a dynamic index template
PUT /_template/template_1?pretty
{
"index_patterns": [
"logs*",
"api*"
],
"settings": {
"number_of_shards": 2
},
"mappings": {
"dynamic_templates": [
{
"strings": {
"match_mapping_type": "string",
"mapping": {
"type": "keyword"
}
}
}
],
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date"
}
}
}
}Create a search template
POST /_scripts/search_template_1?pretty
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"description": "{{query_string}}"
}
}
}
}
}Executing a search query using search template
GET /_search/template?pretty
{
"id": "search_template_1",
"params": {
"query_string": "hello world"
}
}The search request will be executed by default on all the indices available in the cluster and can be limited to particular indices using an index parameter.
Notes
- A dynamic index template is always useful when you do not know the field names in advance and want to control their mapping as per the business use case.
Log Context
Log “Latest audit template missing and audit message cannot be added to the backlog” classname is AbstractAuditor.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :
if (backlog.size() >= MAX_BUFFER_SIZE) {
backlog.remove();
}
backlog.add(toXContent);
} else {
logger.error("Latest audit template missing and audit message cannot be added to the backlog");
}
// stop multiple invocations
if (putTemplateInProgress.compareAndSet(false; true)) {
MlIndexAndAlias.installIndexTemplateIfRequired(
