Elasticsearch server setup#

plans-img Available on Enterprise plans

deployment-img Cloud and self-hosted deployments

Elasticsearch allows you to search large volumes of data quickly, in near real-time, by creating and managing an index of post data. The indexing process can be managed from the System Console after setting up and connecting an Elasticsearch server. The post index is stored on the Elasticsearch server and updated constantly after new posts are made. In order to index existing posts, a bulk index of the entire post database must be generated.

Deploying Elasticsearch includes the following two steps: setting up Elasticsearch, and configuring Mattermost.

Set up Elasticsearch#

We highly recommend that you set up Elasticsearch server on a dedicated machine separate from the Mattermost Server.

  1. Download and install the latest release of Elasticsearch v8, or Elasticsearch v7.17+. See the Elasticsearch documentation for installation details.

  2. Set up Elasticsearch with systemd by running the following commands:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service

  1. Confirm Elasticsearch is working on the server by running the following command:

curl localhost:9200

  1. Get your network interface name by running the following command:

ip addr

  1. Edit the Elasticsearch configuration file in vi by running the following command:

vi /etc/elasticsearch/elasticsearch.yml

  1. In this file, replace the network.host value of _eth0_ with your network interface name, and save your changes.

  2. When using Elasticsearch v8, ensure you set action.destructive_requires_name to false in elasticsearch.yml to allow for wildcard operations to work.

  3. Restart Elasticsearch by running the following commands:

sudo systemctl stop elasticsearch
sudo systemctl start elasticsearch

  1. Confirm the ports are listenings by running the following command:

netstat -plnt

You should see the following ports, including the ones listening on ports 9200 and 9300. Confirm these are listening on your server’s IP address.

  1. Create an Elasticsearch directory and give it the proper permissions.

  2. Install the icu-analyzer plugin to the /usr/share/elasticsearch/plugins directory by running the following command:

sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-icu

  1. Test the connection from Mattermost to Elasticsearch by running the following command:

curl 172.31.80.220:9200

Configure Mattermost#

Follow these steps to configure Mattermost to use your Elasticsearch server and to generate the post index:

  1. Go to System Console > Environment > Elasticsearch.

  2. Set Enable Elasticsearch Indexing to true to enable the other the settings on the page. Once the configuration is saved, new posts made to the database are automatically indexed on the Elasticsearch server.

  3. Ensure Backend type is set to elasticsearch.

  1. Set the server connection details for Elasticsearch or AWS OpenSearch:

  1. Enter Server Connection Address for the enterprise search server you set up earlier.

  2. (Optional) Enter Server Username used to access the enterprise search server.

  3. (Optional) Enter Server Password associated with the username.

  4. Set Enable Cluster Sniffing (Optional). Sniffing finds and connects to all data nodes in your cluster automatically.

Tip

From Mattermost v7.8, optional CA and client certificate configuration settings are available for use with basic auth credentials or to replace them. See the Enterprise search configuration settings documentation for details.

  1. Select Test Connection and then select Save. If the server connection is unsuccessful you won’t be able to save the configuration or enable searching with Elasticsearch or AWS OpenSearch.

  2. Select Build Index to build the post index of existing posts. This process can take up to a few hours depending on the size of the post database and number of messages. The progress percentage can be seen as the index is created. To avoid downtime, set Enable Elasticsearch for search queries to false so that database search is available during the indexing process.

Important

Complete bulk indexing before enabling Elasticsearch or AWS OpenSearch in the next step. Otherwise, search results will be incomplete.

  1. Enable enterprise search by setting Enable Elasticsearch for search queries to true, and setting Enable Elasticsearch for autocomplete to true.

  2. Save your configuration updates and restart the Mattermost server.

Note

For high post volume deployments, we strongly encourage you to read and properly configure the Mattermost LiveIndexingBatchSize configuration setting.

Limitations#

  1. Elasticsearch and AWS OpenSearch uses a standard selection of “stop words” to keep search results relevant. Results for the following words will not be returned: “a”, “an”, “and”, “are”, “as”, “at”, “be”, “but”, “by”, “for”, “if”, “in”, “into”, “is”, “it”, “no”, “not”, “of”, “on”, “or”, “such”, “that”, “the”, “their”, “then”, “there”, “these”, “they”, “this”, “to”, “was”, “will”, and “with”.

  2. Searching stop words in quotes returns more results than just the searched terms (ticket).

  3. Search results are limited to a user’s team and channel membership. This is enforced by the Mattermost server. The entities are indexed in Elasticsearch or AWS OpenSearch in a way that allows Mattermost to filter them when querying, so the Mattermost server narrows down the results on every Elasticsearch or AWS OpenSearch request applying those filters.