Briefly, this error occurs when Elasticsearch is unable to find the latest audit template, but the backlog has already been written. This could be due to a misconfiguration or an issue with the audit logging system. To resolve this issue, you can try the following: 1) Check and correct the audit logging configuration, 2) Ensure that the audit template is correctly installed and accessible, 3) Restart Elasticsearch to refresh the system and clear any temporary issues. If the problem persists, you may need to investigate deeper into the system for any underlying issues.
This guide will help you check for common problems that cause the log ” Latest audit template missing but the back log has been written ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: template, plugin.
If you want to learn more about Elasticsearch templates, check out this guide.
Overview
A template in Elasticsearch falls into one of the two following categories and is indexed inside Elasticsearch using its dedicated endpoint:
- Index templates, which are a way to define a set of rules including index settings, mappings and an index pattern. The template is applied automatically whenever a new index is created with the matching pattern. Templates are also used to dynamically apply custom mapping for the fields which are not predefined inside existing mapping.
- Search templates, which help in defining templates for search queries using mustache scripting language. These templates act as a placeholder for variables defined inside the search queries.
Examples
Create a dynamic index template
PUT /_template/template_1?pretty
{
"index_patterns": [
"logs*",
"api*"
],
"settings": {
"number_of_shards": 2
},
"mappings": {
"dynamic_templates": [
{
"strings": {
"match_mapping_type": "string",
"mapping": {
"type": "keyword"
}
}
}
],
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date"
}
}
}
}Create a search template
POST /_scripts/search_template_1?pretty
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"description": "{{query_string}}"
}
}
}
}
}Executing a search query using search template
GET /_search/template?pretty
{
"id": "search_template_1",
"params": {
"query_string": "hello world"
}
}The search request will be executed by default on all the indices available in the cluster and can be limited to particular indices using an index parameter.
Notes
- A dynamic index template is always useful when you do not know the field names in advance and want to control their mapping as per the business use case.
Log Context
Log “Latest audit template missing but the back log has been written” classname is AbstractAuditor.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :
if (backlog.size() >= MAX_BUFFER_SIZE) {
backlog.remove();
}
backlog.add(toXContent);
} else {
logger.error("Latest audit template missing but the back log has been written");
}
// stop multiple invocations
if (putTemplateInProgress.compareAndSet(false; true)) {
MlIndexAndAlias.installIndexTemplateIfRequired(
