Header Shadow Image


Patroni: FATAL:  could not connect to the primary server: server closed the connection unexpectedly

Getting this?

[root@psql04 ~]# tail -f /data/patroni/log/postgresql-Wed.log
2022-03-09 20:07:40.890 EST [27627] FATAL:  could not connect to the primary server: server closed the connection unexpectedly
                This probably means the server terminated abnormally
                before or while processing the request.
^C
[root@psql04 ~]#

Check on the targe primary cluster that it is not getting blocked via Haproxy:

[root@psql07 patroni]# tail -f /var/log/audit/audit.log|grep -Ei denied
type=AVC msg=audit(1646874430.882:1393): avc:  denied  { name_connect } for  pid=1045 comm="haproxy" dest=5432 scontext=system_u:system_r:haproxy_t:s0 tcontext=system_u:object_r:postgresql_port_t:s0 tclass=tcp_socket permissive=0

Both logs should scroll as the Standby Cluster tries to make a connection to the Primary Cluster.  Resolve using:

grep AVC /var/log/audit/audit.log* |grep -Ei denied >/var/log/audit/audit.previous; cat /var/log/audit/audit.previous |  audit2allow -M systemd-allow; semodule -i systemd-allow.pp

Better yet, allow all haproxy traffic:

semanage permissive -a haproxy_t

As soon as this is set, Haproxy connections work perfectly and all of a sudden, the Patroni cluster is able to replicate just fine:

[root@psql04 ~]# tail -f /data/patroni/log/postgresql-Wed.log
                before or while processing the request.
2022-03-09 20:28:01.807 EST [30690] FATAL:  could not connect to the primary server: server closed the connection unexpectedly
                This probably means the server terminated abnormally
                before or while processing the request.
2022-03-09 20:28:06.816 EST [30703] LOG:  fetching timeline history file for timeline 1226 from primary server
2022-03-09 20:28:06.838 EST [30703] LOG:  started streaming WAL from primary at 32/55000000 on timeline 1225
2022-03-09 20:28:06.854 EST [30703] LOG:  replication terminated by primary server
2022-03-09 20:28:06.854 EST [30703] DETAIL:  End of WAL reached on timeline 1225 at 32/5508DE08.
2022-03-09 20:28:06.857 EST [27428] LOG:  new target timeline is 1226
2022-03-09 20:28:06.859 EST [30703] LOG:  restarted WAL streaming at 32/55000000 on timeline 1226
^C
[root@psql04 ~]#  patronictl –config-file=/etc/patroni.yml list
+————-+——————–+—————-+———+——+———–+—————–+
| Member      | Host               | Role           | State   |   TL | Lag in MB | Pending restart |
+ Cluster: postgres (6617627977882355208) ———-+———+——+———–+—————–+
| postgresql0 | psql04.nix.mds.xyz | Standby Leader | running | 1226 |           | *               |
| postgresql1 | psql05.nix.mds.xyz | Replica        | running | 1226 |         0 | *               |
| postgresql2 | psql06.nix.mds.xyz | Replica        | running | 1226 |         0 | *               |
+————-+——————–+—————-+———+——+———–+—————–+
[root@psql04 ~]#

Cheers,
 

Subscribe
Notify of
0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments

     
  Copyright © 2003 - 2025 Tom Kacperski (microdevsys.com). All rights reserved.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License

 

0
Would love your thoughts, please comment.x
()
x
The IT Development and Technology Mini Vault | MicroDevSys.com
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.