Quick Links
Upgrade Elasticsearch 6.8.23 to Elasticsearch 7.17.5
In this article, we will focus on upgrading Elasticsearch 6.8.23 to Elasticsearch 7.17.5. For more information on version upgrades in general, the different methods available, and everything you need to know prior to initiating the upgrade, you can see our complete guide here: How to Upgrade Elasticsearch Versions.
Rolling upgrades are only supported from Elasticsearch version 6.8.23 to Elasticsearch version 7.17.5. From earlier versions of 6 (6.1,6.2, etc.) to version 7, you need a full cluster restart or you must upgrade first to version 6.8.23 before upgrading to version 7.
Preparing to upgrade Elasticsearch nodes
Elasticsearch nodes cannot be downgraded after upgrading. Before starting the upgrade process you should do the following:
1. Use the Kibana Upgrade Assistant
The Kibana Upgrade Assistant is very helpful because it can indicate:
- Indices that are not compatible with version 7 and must be re-indexed.
- Settings in elasticsearch.yml and/or kibana.yml that are deprecated and must be updated/replaced.
- Cluster settings that are not compatible with version 7.
2. Check the deprecation log
You should read and resolve any issues highlighted in the deprecation log. These logs are usually located in:
/var/log/elasticsearch/Your-Cluster-Name_deprecation.log
3. Review the breaking changes
Breaking changes documentation is published with each new version of Elasticsearch to make you aware of any functionality that may change or disappear. You should always check that documentation to ensure that none of those settings, configurations, or mappings are being used in your setup.
The main things that could be affected are:
- Node configuration (elasticsearch.yml)
- Index mappings and templates
- Cluster settings
- Code used in Elasticsearch clients (applications and indexing scripts)
The most important breaking changes in version 7 are:
- Hits.total is now an object, so you need to replace it with hits.total.value
- Changes in the way clusters are formed: Elastic.yml’s zen discovery settings have been replaced by discovery.seed_hosts and others
4. Check Elasticsearch plugins’ compatibility
If you are using any Elasticsearch plugins, you should check the availability and compatibility of those plugins with the new version.
5. Set up a test environment
You should first try the upgrade process in a test or staging environment in order to check and resolve all issues before upgrading your production cluster.
6. Backup your data
Remember that it is not possible to downgrade an Elasticsearch node, so the only way to practically reverse a failed upgrade is to create a new cluster with the old version and recover your data from snapshots. Therefore, it is essential to take snapshots of all Elasticsearch indices before starting the upgrade process.
As an extra precaution, it is also advisable to export any Kibana visualizations to file.
Upgrading nodes in a rolling upgrade
The process for upgrading your nodes is as follows, upgrading all NON-master-eligible nodes first.
How to upgrade nodes in a rolling upgrade:
- Make sure your cluster is stable, green
You need to make sure that all replicas are available in order to ensure that shutting down the node will not result in a loss of data.
- Disable unnecessary indexing
Wherever it is practically feasible to do so, you should stop all indexing processes because that will increase cluster stability.
- Disable shard allocation
It is important to stop shards reallocating so that when you stop a node for the upgrade the cluster does not reallocate shards to another node (see the command below).
- Stop Elasticsearch
- Remove old plugins
- Upgrade Elasticsearch
The method used to upgrade will depend upon the installation method used to install.
- Upgrade plugins
Elasticsearch will not start if the plugin is not exactly the same version of Elasticsearch.
- Start Elasticsearch
- Re-enable shard allocation
Using the command given below.
- Check that the upgraded node has rejoined the cluster
Using the command below, you can check how many nodes are in the cluster.
- Wait for cluster status to turn green
The command provided below will also show you the progress of the shard recovery process on the upgraded node until the cluster reaches a green state.
Do not be in a hurry to upgrade your nodes, wait for the cluster to fully recover before moving on. If the cluster does not go green, look in the logs to find any issues that may indicate problems with the upgrade or configuration. - Repeat
Repeat the full process above for each node.
To stop a node:
systemctl stop elasticsearch
To disable shard allocation, run:
PUT _cluster/settings
{
"persistent": {
"cluster.routing.allocation.enable": "primaries"
}
}To remove plugins, e.g. s3:
sudo bin/elasticsearch-plugin remove repository-s3
To reload the daemon after upgrading:
systemctl daemon-reload systemctl enable elasticsearch
To re-enable shard allocation, run:
PUT _cluster/settings
{
"persistent": {
"cluster.routing.allocation.enable": null
}
}To get cluster status and see how many nodes are in the cluster, use:
GET _cluster/health
