Setting up a multi-tiered log infrastructure Part 4 -- Elasticsearch Setup

Setup Elasticsearch cluster nodes

Install Elasticsearch

In this example we are building out a three node cluster but this can scale up to fit whatever cluster size you choose. View Elasticsearch setup and configuration docs

Install Java

yum install java-1.8.0-openjdk-headless.x86_64

Import signing key from

rpm --import

Create repo file

vi /etc/yum.repos.d/Elasticsearch.repo

Insert this text

name=Elasticsearch repository for 2.x packages

Install elasticsearch

yum install elasticsearch

Set ES to start on boot

systemctl enable elasticsearch.service

Configure Elasticsearch

Edit the ES config before starting elasticsearch on the nodes

vi /etc/elasticsearch/elasticsearch.yml

Change the setting for

set the name on all three nodes to graylog (it won’t work otherwise)

Change the setting for

set to the individual hostname of the node

Change the setting for node.master

leave node.master = false and do not uncomment it on the node that will be used as the master node for the cluster. the default setting is to perform as a master. This node will also be used for the graylog server and web interface

set node.master = false on the other two nodes. These nodes will be used for data storage and shard replication only

Change the setting for

set = false on the node that will be used as the master node for the cluster. This node will not store data, as it will only function as the master.

set = true on the other two nodes. These nodes will be used for data storage and shard replication only

Setup Note: the default configuration is setup for multicast. To disable multicast, make the next two changes.

Change the setting for

uncomment false

Change the setting for

set [“node-master- hostname”] this should be set to the name of the node that will function as the master node. Set on all three ES nodes

Configure firewalld rules

Now that the config file is edited, let’s make some firewall rule changes. if for some reason you aren’t using a firewall then you can skip this.

Configure a default zone with firewalld (The default zone is assumed to already be set as “Internal”)

Create a new service file for our new elasticsearch nodes

vi /etc/firewalld/services/es-transport.xml

Use this as the contents for es-transport.xml

<?xml version="1.0" encoding="utf-8"?>
    <description>transport for elasticsearch nodes.</description>
    <port protocol="tcp" port="9300"/>

Permanently create an selinux context label

semanage fcontext -a -t firewalld_etc_rw_t -s system_u /etc/firewalld/services/es-transport.xml

Apply the new selinux label

restorecon -vF /etc/firewalld/services/es-transport.xml

Add rich rules to allow connections from our other nodes. (this should be the list of all elasticsearch nodes that need to talk with each other)

firewall-cmd --zone=internal --add-rich-rule="rule family="ipv4" source address="" service name="es-transport" accept" --permanent

Reload the current firewall config

firewall-cmd --reload

Check the interface and verify the services

firewall-cmd --zone=internal --list-services

Verify the config

Start elasticsearch on all of your clustered nodes. You will have to do this on each server.

systemctl start elasticsearch.service

Check that the nodes have created a cluster and then we can move on to the next step

curl ‘http://localhost:9200/_cluster/health?pretty’

You must be logged in to post a comment.

Proudly powered by WordPress   Premium Style Theme by