Setting up a multi-tiered log infrastructure Part 9 -- Rsyslog HA Setup

  1. Setting up a multi-tiered log infrastructure Part 1 -- Getting Started
  2. Setting up a multi-tiered log infrastructure Part 2 -- System Overview
  3. Setting up a multi-tiered log infrastructure Part 3 -- System Build
  4. Setting up a multi-tiered log infrastructure Part 4 -- Elasticsearch Setup
  5. Setting up a multi-tiered log infrastructure Part 5 -- MongoDB Setup
  6. Setting up a multi-tiered log infrastructure Part 6 -- Graylog Setup
  7. Setting up a multi-tiered log infrastructure Part 7 -- Graylog WebUI Setup
  8. Setting up a multi-tiered log infrastructure Part 8 -- Rsyslog Setup
  9. Setting up a multi-tiered log infrastructure Part 9 -- Rsyslog HA Setup
  10. Setting up a multi-tiered log infrastructure Part 10 -- HA Cluster Setup
  11. Setting up a multi-tiered log infrastructure Part 11 -- Cluster Tuning

Setup rsyslog aggregator nodes (Optional)

Setup Note:  As part of the overall design, an HA cluster allows aggregating logs to the Central Log Repository with as little loss of logs as possible due to downtime or maintenance. Below are steps for building an HA cluster and setting up rsyslog for CENTOS 7.

Install/upgrade to the latest rsyslog

yum update rsyslog

Create an rsyslog spool directory (this will be needed later)

mkdir /var/lib/rsyslog

Setup Note: A custom rsyslog.conf is available for the aggregator nodes that allows receiving logs on tcp and udp port 514 by default. Copy the content from the appendixes into the appropriate files.

Path: /etc/rsyslog.conf rsyslog.conf for aggregator nodes

Edit the rsyslog config

vi /etc/rsyslog.conf

Uncomment the lines for the action and change server.domain.tld to the name of your CLR node (don’t do this until you are ready to ship logs to the CLR node)

#action(type="omfwd"
# target="server.domain.tld"
# port="514"
# protocol="tcp"
# queue.filename="wait_queue"
# queue.size="1000000"
# queue.type="LinkedList"
# )

Settings to edit

Nothing to do if using the proper config

Restart rsyslog and enable start on boot

systemctl enable rsyslog.service

systemctl restart rsyslog.service

Setup Note: rsyslog offers many input options, which are too extensive to cover in depth. It is assumed that endpoint devices will be using either tcp or udp for syslog services. On CENTOS 7 the concept of using service files and zones makes configuration different than using iptables. While multiple port service files can be created the extensive options that rsyslog allows made the concept of one service file per port more appealing. This scheme can be extended to incorporate any port rsyslog supports allowing easier management of the firewall rules.

Configure firewalld rules

Now that the config file is edited, let’s make some firewall rule changes. If for some reason you aren’t using a firewall then you can skip this.

Configure a default zone with firewalld (The default zone is assumed to already be set as “Internal”)

Create new service files for the rsyslog node, one for tcp and one for udp

vi /etc/firewalld/services/rsyslog-tcp.xml

Use this as the contents for rsyslog-tcp.xml

<?xml version="1.0" encoding="utf-8"?>
  <service>
    <short>rsyslog-tcp</short>
    <description>rsyslog server access for default tcp port.</description>
    <port protocol="tcp" port="514"/>
  </service>

Edit /etc/firewalld/services/rsyslog-udp.xml

vi /etc/firewalld/services/rsyslog-udp.xml

Insert this text

<?xml version="1.0" encoding="utf-8"?>
  <service>
    <short>rsyslog-udp</short>
    <description>rsyslog server access for default udp port.</description>
    <port protocol="udp" port="514"/>
  </service>

Permanently create selinux context labels

semanage fcontext -a -t firewalld_etc_rw_t -s system_u /etc/firewalld/services/rsyslog-tcp.xml
semanage fcontext -a -t firewalld_etc_rw_t -s system_u /etc/firewalld/services/rsyslog-udp.xml

Apply the new selinux labels

restorecon -vF /etc/firewalld/services/rsyslog-tcp.xml
restorecon -vF /etc/firewalld/services/rsyslog-udp.xml

Add services (or ports) to allowed rules

firewall-cmd --permanent --zone=internal --add-service=rsyslog-tcp

firewall-cmd --permanent --zone=internal --add-service=rsyslog-udp

You can just define ports if you like

firewall-cmd --zone=internal --add-port=514/tcp

firewall-cmd --zone=internal --add-port=514/udp

Reload the current firewall config

firewall-cmd --reload

Check the interface and verify the services

firewall-cmd --zone=internal --list-services

You must be logged in to post a comment.

Proudly powered by WordPress   Premium Style Theme by www.gopiplus.com