Briefly, this error occurs when Elasticsearch cannot find the segments file in the specified directory. This file is crucial as it contains information about the segments in an index. The absence of this file can be due to accidental deletion or corruption. To resolve this issue, you can restore the segments file from a backup. If a backup is not available, you may need to rebuild the index. Additionally, ensure that Elasticsearch has the necessary read/write permissions for the directory. Regularly backing up your data can prevent such issues in the future.
This guide will help you check for common problems that cause the log ” no segments* file found in ” + directory + “: files: ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: plugin, lucene.
Overview
Lucene or Apache Lucene is an open-source Java library used as a search engine. Elasticsearch is built on top of Lucene.
Elasticsearch converts Lucene into a distributed system/search engine for scaling horizontally. Elasticsearch also provides other features like thread-pool, queues, node/cluster monitoring API, data monitoring API, Cluster management, etc. In short, Elasticsearch extends Lucene and provides additional features beyond it.
Elasticsearch hosts data on data nodes. Each data node hosts one or more indices, and each index is divided into shards with each shard holding part of the index’s data. Each shard created in Elasticsearch is a separate Lucene instance or process.
Notes and good things to know
When an index is created in Elasticsearch, it is divided into one or more primary shards for scaling the data and splitting it into multiple nodes/instances.
- As each shard is a separate instance of Lucene, creating too many shards will consume unnecessary resources and damage performance.
It takes proper planning to decide the number of primary shards for your index, taking into account the index size, max growth, and the number of data nodes.
- Previous versions of Elasticsearch defaulted to creating five shards per index. Starting with 7.0.0, the default is now one shard per index.
Log Context
Log “no segments* file found in ” + directory + “: files: ” class name is OldSegmentInfos.java. We extracted the following from Elasticsearch source code for those seeking an in-depth context :
if (infoStream != null) {
message("directory listing gen=" + gen);
} if (gen == -1) {
throw new IndexNotFoundException("no segments* file found in " + directory + ": files: " + Arrays.toString(files));
} else if (gen > lastGen) {
String segmentFileName = IndexFileNames.fileNameFromGeneration(IndexFileNames.SEGMENTS; ""; gen); try {
T t = doBody(segmentFileName);
