Briefly, this error occurs when there’s an issue adding an ingest pipeline in Elasticsearch. An ingest pipeline is a sequence of processors that perform transformations on data. The error could be due to incorrect pipeline configuration, insufficient permissions, or a system error. To resolve this, ensure the pipeline configuration is correct and valid. Check if the user has the necessary permissions to add a pipeline. If it’s a system error, restarting Elasticsearch might help. Also, ensure that the Elasticsearch version supports the pipeline feature.
This guide will help you check for common problems that cause the log ” adding ingest pipeline {} ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: plugin, template.
If you want to learn more about Elasticsearch templates, check out this guide.
Overview
A template in Elasticsearch falls into one of the two following categories and is indexed inside Elasticsearch using its dedicated endpoint:
- Index templates, which are a way to define a set of rules including index settings, mappings and an index pattern. The template is applied automatically whenever a new index is created with the matching pattern. Templates are also used to dynamically apply custom mapping for the fields which are not predefined inside existing mapping.
- Search templates, which help in defining templates for search queries using mustache scripting language. These templates act as a placeholder for variables defined inside the search queries.
Examples
Create a dynamic index template
PUT /_template/template_1?pretty
{
"index_patterns": [
"logs*",
"api*"
],
"settings": {
"number_of_shards": 2
},
"mappings": {
"dynamic_templates": [
{
"strings": {
"match_mapping_type": "string",
"mapping": {
"type": "keyword"
}
}
}
],
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date"
}
}
}
}Create a search template
POST /_scripts/search_template_1?pretty
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"description": "{{query_string}}"
}
}
}
}
}Executing a search query using search template
GET /_search/template?pretty
{
"id": "search_template_1",
"params": {
"query_string": "hello world"
}
}The search request will be executed by default on all the indices available in the cluster and can be limited to particular indices using an index parameter.
Notes
- A dynamic index template is always useful when you do not know the field names in advance and want to control their mapping as per the business use case.
Log Context
Log “adding ingest pipeline {}” classname is IndexTemplateRegistry.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :
"error adding ingest pipeline [{}] for [{}]; request was not acknowledged";
pipelineConfig.getId();
getOrigin()
);
} else {
logger.info("adding ingest pipeline {}"; pipelineConfig.getId());
}
}
@Override
public void onFailure(Exception e) {
