Header Shadow Image


Install RabbitMQ in High Availability

In this post we'll install RabbitMQ in High Availability on 3 nodes.  We'll do this to share out the instance with third party applications that need it while providing fault tolerance. We will reference the following post but instead on CentOS 7.

So let's get started.

HOSTS COMMANDS DESCRIPTION
rmq01 / rmq02 / rmq03 CentOS 7 Create 3 seperate VM's to add to your cluster.
rmq01 / rmq02 / rmq03 echo "net.ipv4.ip_nonlocal_bind = 1" >> /etc/sysctl.conf; echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf; sysctl -p Set no local bind and ip forward for RabbitMQ,
HAProxy and Keepalived.
rmq01 / rmq02 / rmq03

[root@rmq01 ~]# cat /etc/firewalld/zones/public.xml
<?xml version="1.0" encoding="utf-8"?>
<zone>
  <short>Public</short>
  <description>For use in public areas. You do not trust the other computers on networks to not harm your computer. Only selected incoming connections are accepted.</description>
  <service name="ssh"/>
  <service name="dhcpv6-client"/>
  <port protocol="tcp" port="24007-24008"/>
  <port protocol="tcp" port="49152"/>
  <port protocol="tcp" port="38465-38469"/>
  <port protocol="tcp" port="111"/>
  <port protocol="udp" port="111"/>
  <port protocol="tcp" port="2049"/>
  <port protocol="tcp" port="4501"/>
  <port protocol="udp" port="4501"/>
  <port protocol="udp" port="20048"/>
  <port protocol="tcp" port="20048"/>
  <port protocol="tcp" port="22"/>
  <port protocol="udp" port="22"/>
  <port protocol="tcp" port="10000"/>
  <port protocol="udp" port="49000-59999"/>
  <port protocol="tcp" port="49000-59999"/>
  <port protocol="udp" port="9000"/>
  <port protocol="tcp" port="9000"/>
  <port protocol="udp" port="137"/>
  <port protocol="udp" port="138"/>
  <port protocol="udp" port="2049"/>
  <port protocol="udp" port="2049"/>
  <port protocol="tcp" port="4369"/>
  <port protocol="tcp" port="5671-5672"/>
  <port protocol="tcp" port="25672"/>
  <port protocol="tcp" port="35672-35682"/>
  <port protocol="tcp" port="15672"/>
  <port protocol="tcp" port="61613-61614"/>
  <port protocol="tcp" port="1883"/>
  <port protocol="tcp" port="8883"/>
  <port protocol="tcp" port="15674"/>
  <port protocol="tcp" port="15675"/>

</zone>
[root@rmq01 ~]#

[root@rmq01 ~]# scp /etc/firewalld/zones/public.xml rmq02:/etc/firewalld/zones/public.xml
Password:
public.xml                                                                                       100% 1509     1.0MB/s   00:00
[root@rmq01 ~]# scp /etc/firewalld/zones/public.xml rmq03:/etc/firewalld/zones/public.xml
Password:
public.xml                                                                                       100% 1509   921.3KB/s   00:00
[root@rmq01 ~]#

[root@rmq01 ~]# systemctl restart firewalld; systemctl status firewalld -l
[root@rmq02 ~]# systemctl restart firewalld; systemctl status firewalld -l
[root@rmq03 ~]# systemctl restart firewalld; systemctl status firewalld -l

Prepare the firewall
and distribute the
configuration to the
other nodes. NOTE:
The ports that are
needed for this are
below the line with 4369.

rmq01 / rmq02 / rmq03

RMQ01

[root@rmq01 ~]# yum install rabbitmq*
(Optional) [root@rmq01 ~]# rabbitmq-server -detached
[root@rmq01 audit]# systemctl restart rabbitmq-server

[root@rmq01 ~]# rabbitmqctl cluster_status
[root@rmq01 ~]# rabbitmqctl add_user roboconf roboconf
[root@rmq01 ~]# rabbitmqctl set_permissions roboconf ".*" ".*" ".*"
[root@rmq01 ~]# rabbitmqctl set_policy ha-all "" '{"ha-mode":"all","ha-sync-mode":"automatic"}'

COOKIE:

[root@rmq01 audit]# cat /var/lib/rabbitmq/.erlang.cookie
OMYNAUNDNDCMEOBIYFLU[root@rmq01 audit]#
[root@rmq01 audit]#
[root@rmq01 audit]# scp /var/lib/rabbitmq/.erlang.cookie rmq02:/var/lib/rabbitmq/.erlang.cookie
[root@rmq01 audit]# scp /var/lib/rabbitmq/.erlang.cookie rmq03:/var/lib/rabbitmq/.erlang.cookie

RMQ02

[root@rmq02 ~]# yum install rabbitmq*
(Optional) [root@rmq02 ~]# rabbitmq-server -detached

[root@rmq02 ~]# systemctl restart rabbitmq-server
[root@rmq02 ~]# rabbitmqctl stop_app
[root@rmq02 ~]# rabbitmqctl join_cluster rabbit@rmq01
[root@rmq02 ~]# rabbitmqctl start_app

RMQ03

[root@rmq03 ~]# yum install rabbitmq*
(Optional) [root@rmq03 ~]# rabbitmq-server -detached

[root@rmq03 ~]# systemctl restart rabbitmq-server
[root@rmq03 ~]# rabbitmqctl stop_app
[root@rmq03 ~]# rabbitmqctl join_cluster rabbit@rmq01
[root@rmq03 ~]# 
rabbitmqctl start_app
 

Check:

[root@rmq01 audit]# rabbitmqctl status

Install and configure RabbitMQ on nodes
as to your left.
rmq01 / rmq02 / rmq03

At the end of configuring RabbitMQ, ensure service start and stop correctly using the following:

[root@rmq01 audit]# systemctl restart rabbitmq-server
[root@rmq01 audit]# systemctl status rabbitmq-server -l

You may receive errors.  In such case, see below.

Once you're done
above, you'll stop
and start the
service in this manner.
rmq01 / rmq02 / rmq03

Run any of the following command, or a combination of, on deny entries in /var/log/audit/audit.log that may appear as you stop, start or install above services:

METHOD 1:
grep AVC /var/log/audit/audit.log | audit2allow -M systemd-allow;semodule -i systemd-allow.pp

METHOD 2:
audit2allow -a
audit2allow -a -M ganesha_<NUM>_port
semodule -i ganesha_<NUM>_port.pp

USEFULL THINGS:

ausearch –interpret
aureport

Configure selinux. Don't disable it.

 

 

 

 

 

 

 

 

 

 

 

This actually  makes your host safer and is actually
easy to work with using just these commands.

rmq01 / rmq02 / rmq03

PACKAGES:
yum install haproxy     # ( 1.5.18-6.el7.x86_64 used in this case )

On all the nodes add the following:  

# cat /etc/haproxy/haproxy.cfg

global
    log         127.0.0.1 local0
    stats       socket /var/run/haproxy.sock mode 0600 level admin
    user        haproxy
    group       haproxy
    daemon
    debug
    maxconn 1024

defaults
    mode tcp
    log global
    option                  dontlognull
    option                  redispatch
    retries 2
    timeout client 30m
    timeout connect 4s
    timeout server 30m
    timeout check 5s

listen stats
    bind :9000
    mode http
    stats enable
    stats hide-version
    stats realm Haproxy\ Statistics
    stats uri /haproxy-stats
    stats auth admin:secretPassword

frontend rmq-in
    mode tcp
    bind rmq-c01:15672
    option tcplog
    default_backend             rmq-back


backend rmq-back
    log         /dev/log local0 debug
    option      tcplog
    mode        tcp
    balance     source
    timeout client  3h
    timeout server  3h
    server      rmq01.nix.mds.xyz    rmq01.nix.mds.xyz:5672 check fall 3 rise 2
    server      rmq02.nix.mds.xyz    rmq02.nix.mds.xyz:5672 check fall 3 rise 2
    server      rmq03.nix.mds.xyz    rmq03.nix.mds.xyz:5672 check fall 3 rise 2
#

Set logging settings for HAProxy:

# cat /etc/rsyslog.d/haproxy.conf
$ModLoad imudp
$UDPServerAddress 127.0.0.1
$UDPServerRun 514
local6.* /var/log/haproxy.log
local0.* /var/log/haproxy.log

Configure rsyslogd (/etc/rsyslog.conf):

local0.*             /var/log/haproxy.log
local3.*             /var/log/keepalived.log

Install and Configure HAPROXY. great source that helped with this part.
rmq01 / rmq02 / rmq03

PACKAGES:

yum install keepalived    # ( Used 1.3.5-1.el7.x86_64 in this case )

RMQ01:

vrrp_script chk_haproxy {
        script "killall -0 haproxy"             # check the haproxy process
        interval 2                              # every 2 seconds
        weight 2                                # add 2 points if OK
}

vrrp_instance rmq-c01 {
        interface eth0                          # interface to monitor
        state MASTER                            # MASTER on haproxy1, BACKUP on haproxy2
        virtual_router_id 57                    # Set to last digit of cluster IP.
        priority 101                            # 101 on haproxy1, 100 on haproxy2

        authentication {
                auth_type PASS
                auth_pass 
s3cretp@s$w0rd
        }

        virtual_ipaddress {
                delay_loop 12
                lb_algo wrr
                lb_kind DR
                protocol TCP
                192.168.0.57                    # virtual ip address
        }

        track_script {
                chk_haproxy
        }
}

RMQ02:

vrrp_script chk_haproxy {
        script "killall -0 haproxy"             # check the haproxy process
        interval 2                              # every 2 seconds
        weight 2                                # add 2 points if OK
}

vrrp_instance rmq-c01 {
        interface eth0                          # interface to monitor
        state BACKUP                            # MASTER on haproxy1, BACKUP on haproxy2
        virtual_router_id 57                    # Set to last digit of cluster IP.
        priority 102                            # 101 on haproxy1, 100 on haproxy2

        authentication {
                auth_type PASS
                auth_pass 
s3cretp@s$w0rd
        }

        virtual_ipaddress {
                delay_loop 12
                lb_algo wrr
                lb_kind DR
                protocol TCP
                192.168.0.57                    # virtual ip address
        }

        track_script {
                chk_haproxy
        }
}

RMQ03:

vrrp_script chk_haproxy {
        script "killall -0 haproxy"             # check the haproxy process
        interval 2                              # every 2 seconds
        weight 2                                # add 2 points if OK
}

vrrp_instance rmq-c01 {
        interface eth0                          # interface to monitor
        state BACKUP                            # MASTER on haproxy1, BACKUP on haproxy2
        virtual_router_id 57                    # Set to last digit of cluster IP.
        priority 103                            # 101 on haproxy1, 100 on haproxy2

        authentication {
                auth_type PASS
                auth_pass 
s3cretp@s$w0rd
        }

        virtual_ipaddress {
                delay_loop 12
                lb_algo wrr
                lb_kind DR
                protocol TCP
                192.168.0.57                    # virtual ip address
        }

        track_script {
                chk_haproxy
        }
}

Configure keepalived. A great source that helped with this as well.
Any External Node

In the first window, create the following python code and start it:

[root@awx-mess01 ~]# cat ./receive-mq.py
#!/usr/bin/env python
import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('rmq-c01'))
channel = connection.channel()

channel.queue_declare(queue='hello')

def callback(ch, method, properties, body):
    print(" [x] Received %r" % body)

channel.basic_consume(callback,
                      queue='hello',
                      no_ack=True)

print(' [*] Waiting for messages. To exit press CTRL+C')
channel.start_consuming()

[root@awx-mess01 ~]#

In the second window start the following code:

[root@linsrvj01 ~]# cat ./send-mq.py
#!/usr/bin/env python
import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('rmq-c01'))
channel = connection.channel()
channel.queue_declare(queue='hello')

channel.basic_publish(exchange='',
                      routing_key='hello',
                      body='Hello World!')
print(" [x] Sent 'Hello World!'")

connection.close()
[root@linsrvj01 ~]#

Now keep rerunning the send-my.py code and notice how the receiver retrieves each message:

[root@linsrvj01 ~]# ./send-mq.py
[x] Sent 'Hello World!'
[root@linsrvj01 ~]# ./send-mq.py
[x] Sent 'Hello World!'
[root@linsrvj01 ~]#

On second terminal, watch the messages appear:

[root@awx-mess01 ~]# ./receive-mq.py
 [*] Waiting for messages. To exit press CTRL+C
 [x] Received 'Hello World!'
 [x] Received 'Hello World!'


 

Using the following
external tutorial,
use the following
python snippets
to test the RabbitMQ setup.  

PROBLEM:

If you get the below issue:

[root@rmq01 audit]# systemctl status rabbitmq-server -l
â rabbitmq-server.service – RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sat 2018-11-24 20:48:47 EST; 8min ago
  Process: 4593 ExecStop=/usr/lib/rabbitmq/bin/rabbitmqctl stop (code=exited, status=0/SUCCESS)
  Process: 4558 ExecStart=/usr/lib/rabbitmq/bin/rabbitmq-server -detached (code=exited, status=1/FAILURE)
 Main PID: 4558 (code=exited, status=1/FAILURE)

Nov 24 20:48:47 rmq01.nix.mds.xyz systemd[1]: Starting RabbitMQ broker…
Nov 24 20:48:47 rmq01.nix.mds.xyz rabbitmq-server[4558]: Warning: PID file not written; -detached was passed.
Nov 24 20:48:47 rmq01.nix.mds.xyz systemd[1]: rabbitmq-server.service: Got notification message from PID 4571, but reception only permitted for main PID 4558
Nov 24 20:48:47 rmq01.nix.mds.xyz rabbitmq-server[4558]: ERROR: node with name "rabbit" already running on "rmq01"
Nov 24 20:48:47 rmq01.nix.mds.xyz systemd[1]: rabbitmq-server.service: main process exited, code=exited, status=1/FAILURE

Nov 24 20:48:47 rmq01.nix.mds.xyz rabbitmqctl[4593]: Stopping and halting node rabbit@rmq01 …
Nov 24 20:48:47 rmq01.nix.mds.xyz systemd[1]: Failed to start RabbitMQ broker.
Nov 24 20:48:47 rmq01.nix.mds.xyz systemd[1]: Unit rabbitmq-server.service entered failed state.
Nov 24 20:48:47 rmq01.nix.mds.xyz systemd[1]: rabbitmq-server.service failed.
[root@rmq01 audit]# ps -ef|grep -Ei rabbit
rabbitmq  1801     1  0 19:31 ?        00:00:00 /usr/lib64/erlang/erts-5.10.4/bin/epmd -daemon
rabbitmq  5102     1  1 20:54 ?        00:00:02 /usr/lib64/erlang/erts-5.10.4/bin/beam.smp -W w -K true -A30 -P 1048576 — -root /usr/lib64/erlang -progname erl — -home /var/lib/rabbitmq — -pa /usr/lib/rabbitmq/lib/rabbitmq_server-3.3.5/sbin/../ebin -noshell -noinput -s rabbit boot -sname rabbit@rmq01 -boot start_sasl -config /etc/rabbitmq/rabbitmq -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,"/var/log/rabbitmq/rabbit@rmq01.log"} -rabbit sasl_error_logger {file,"/var/log/rabbitmq/rabbit@rmq01-sasl.log"} -rabbit enabled_plugins_file "/etc/rabbitmq/enabled_plugins" -rabbit plugins_dir "/usr/lib/rabbitmq/lib/rabbitmq_server-3.3.5/sbin/../plugins" -rabbit plugins_expand_dir "/var/lib/rabbitmq/mnesia/rabbit@rmq01-plugins-expand" -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@rmq01" -kernel inet_dist_listen_min 25672 -kernel inet_dist_listen_max 25672 -noshell -noinput
rabbitmq  5146  5102  0 20:54 ?        00:00:00 inet_gethost 4
rabbitmq  5147  5146  0 20:54 ?        00:00:00 inet_gethost 4
root      5389  1671  0 20:56 pts/0    00:00:00 grep –color=auto -Ei rabbit
[root@rmq01 audit]#

It's probably because you need to run the following on all the nodes:

[root@rmq01 audit]# echo "net.ipv4.ip_nonlocal_bind = 1" >> /etc/sysctl.conf; echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf; sysctl -p
net.ipv4.ip_nonlocal_bind = 1
net.ipv4.ip_forward = 1
[root@rmq01 audit]#

PROBLEM:

[root@rmq02 ~]# rabbitmqctl join_cluster rabbit@rmq01.nix.mds.xyz
Clustering node rabbit@rmq02 with 'rabbit@rmq01.nix.mds.xyz' …
Error: unable to connect to nodes ['rabbit@rmq01.nix.mds.xyz']: nodedown

=ERROR REPORT==== 24-Nov-2018::21:19:50 ===
** System NOT running to use fully qualified hostnames **
** Hostname rmq01.nix.mds.xyz is illegal **

DIAGNOSTICS
===========

attempted to contact: ['rabbit@rmq01.nix.mds.xyz']

rabbit@rmq01.nix.mds.xyz:
  * connected to epmd (port 4369) on rmq01.nix.mds.xyz
  * epmd reports node 'rabbit' running on port 25672
  * TCP connection succeeded but Erlang distribution failed
  * suggestion: hostname mismatch?
  * suggestion: is the cookie set correctly?

current node details:
- node name: rabbitmqctl2977@rmq02
- home dir: /var/lib/rabbitmq
- cookie hash: Cv6uyhSSoLwjp6RqeyCv2Q==

[root@rmq02 ~]#

The cookie was not distributed correctly as per above.  Distribute the cookie and try again.

PROBLEM:

[root@rmq03 audit]# systemctl start rabbitmq-server
Job for rabbitmq-server.service failed because the control process exited with error code. See "systemctl status rabbitmq-server.service" and "journalctl -xe" for details.
[root@rmq03 audit]# systemctl status rabbitmq-server -l
â rabbitmq-server.service – RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sat 2018-11-24 22:29:01 EST; 7s ago
  Process: 3405 ExecStop=/usr/lib/rabbitmq/bin/rabbitmqctl stop (code=exited, status=1/FAILURE)
  Process: 3371 ExecStart=/usr/lib/rabbitmq/bin/rabbitmq-server (code=exited, status=1/FAILURE)
 Main PID: 3371 (code=exited, status=1/FAILURE)

Nov 24 22:29:00 rmq03.nix.mds.xyz rabbitmqctl[3405]: {error_logger,{{2018,11,24},{22,29,0}},crash_report,[[{initial_call,{auth,init,['Argument__1']}},{pid,<0.19.0>},{registered_name,[]},{error_info,{exit,{“Error when reading /var/lib/rabbitmq/.erlang.cookie: eacces”,[{auth,init_cookie,0,[{file,"auth.erl"},{line,285}]},{auth,init,1,[{file,"auth.erl"},{line,139}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,328}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[net_sup,kernel_sup,<0.10.0>]},{messages,[]},{links,[<0.17.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,27},{reductions,635}],[]]}
Nov 24 22:29:00 rmq03.nix.mds.xyz rabbitmqctl[3405]: {error_logger,{{2018,11,24},{22,29,0}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{"Error when reading /var/lib/rabbitmq/.erlang.cookie: eacces",[{auth,init_cookie,0,[{file,"auth.erl"},{line,285}]},{auth,init,1,[{file,"auth.erl"},{line,139}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{offender,[{pid,undefined},{name,auth},{mfargs,{auth,start_link,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
Nov 24 22:29:00 rmq03.nix.mds.xyz rabbitmqctl[3405]: {error_logger,{{2018,11,24},{22,29,0}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{shutdown,{failed_to_start_child,auth,{"Error when reading /var/lib/rabbitmq/.erlang.cookie: eacces",[{auth,init_cookie,0,[{file,"auth.erl"},{line,285}]},{auth,init,1,[{file,"auth.erl"},{line,139}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}}}},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
Nov 24 22:29:00 rmq03.nix.mds.xyz rabbitmqctl[3405]: {error_logger,{{2018,11,24},{22,29,0}},crash_report,[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.9.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,auth,{“Error when reading /var/lib/rabbitmq/.erlang.cookie: eacces”,[{auth,init_cookie,0,[{file,"auth.erl"},{line,285}]},{auth,init,1,[{file,"auth.erl"},{line,139}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}}}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,133}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[<0.8.0>]},{messages,[{'EXIT',<0.10.0>,normal}]},{links,[<0.8.0>,<0.7.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,27},{reductions,149}],[]]}
Nov 24 22:29:00 rmq03.nix.mds.xyz rabbitmqctl[3405]: {error_logger,{{2018,11,24},{22,29,0}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,auth,{"Error when reading /var/lib/rabbitmq/.erlang.cookie: eacces",[{auth,init_cookie,0,[{file,"auth.erl"},{line,285}]},{auth,init,1,[{file,"auth.erl"},{line,139}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}}}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
Nov 24 22:29:01 rmq03.nix.mds.xyz rabbitmqctl[3405]: {“Kernel pid terminated”,application_controller,”{application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,auth,{\”Error when reading /var/lib/rabbitmq/.erlang.cookie: eacces\”,[{auth,init_cookie,0,[{file,\"auth.erl\"},{line,285}]},{auth,init,1,[{file,\"auth.erl\"},{line,139}]},{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,304}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,239}]}]}}}}},{kernel,start,[normal,[]]}}}"}
Nov 24 22:29:01 rmq03.nix.mds.xyz systemd[1]: rabbitmq-server.service: control process exited, code=exited status=1
Nov 24 22:29:01 rmq03.nix.mds.xyz systemd[1]: Failed to start RabbitMQ broker.
Nov 24 22:29:01 rmq03.nix.mds.xyz systemd[1]: Unit rabbitmq-server.service entered failed state.
Nov 24 22:29:01 rmq03.nix.mds.xyz systemd[1]: rabbitmq-server.service failed.
[root@rmq03 audit]#

If you get the above, notice the cookie access error above in red.  Need to ensure the cookie is set to rabbitmq:rabbitmq :

[root@rmq03 audit]# ls -altri /var/lib/rabbitmq/.erlang.cookie
67885078 -r——–. 1 root root 20 Nov 24 21:24 /var/lib/rabbitmq/.erlang.cookie
[root@rmq03 audit]# chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie
[root@rmq03 audit]# 

 

Enjoy!

REF: https://www.rabbitmq.com/ha.htmlhttp://roboconf.net/en/user-guide/clustered-rabbitmq.htmlhttps://www.rabbitmq.com/networking.html#portshttps://insidethecpu.com/2014/11/17/load-balancing-a-rabbitmq-cluster/ , https://github.com/ansible/awx/issues/574https://github.com/MrMEEE/awx-build/issues/26


Cheers,
TK

Comments are closed.


     
  Copyright © 2003 - 2013 Tom Kacperski (microdevsys.com). All rights reserved.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License