Cloudera: Ncat: Connection refused.
Getting the following connecting from your Azure instance back to your Cloudera CM + CDH Cluster?
[root@cm-awn01 ~]# nc -v 123.123.123.123 7183
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connection refused.
The corresponding Cloudera Agent error looks like this:
[17/May/2020 13:34:53 +0000] 5306 Thread-13 https ERROR Failed to retrieve/store URL: https://cm-r01nn01.mws.mds.xyz:7183/cmf/par cel/download/CDH-6.3.0-1.cdh6.3.0.p0.1279813-el7.parcel.torrent -> /opt/cloudera/parcel-cache/CDH-6.3.0-1.cdh6.3.0.p0.1279813-el7.parcel.to rrent
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 193, in fetch_to_file
resp = self.open(req_url)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 188, in open
return self.opener(url, *pargs, **kwargs)
File "/usr/lib64/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 179, in https_open
return self.do_open(opener, req)
File "/usr/lib64/python2.7/urllib2.py", line 1214, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 111] Connection refused>
If you're using HAProxy for traffic routing to various instances, you'll need the following stanza in the HAProxy configuration file:
# CM GUI
listen cm
bind cm-c01:80
mode http
redirect scheme https if !{ ssl_fc }
frontend cmin
bind cm-c01:443 ssl crt /etc/haproxy/certs/cm-c01.mws.mds.xyz-haproxy.pem no-sslv3
default_backend cmback
frontend cm7183in
bind cm-c01:7183 ssl crt /etc/haproxy/certs/cm-c01.mws.mds.xyz-haproxy.pem no-sslv3
default_backend cmback
backend cmback
mode http
balance roundrobin
server cm-r01nn01.mws.mds.xyz cm-r01nn01.mws.mds.xyz:7183 ssl check verify none port 7183 inter 12000 rise 3 fall 3
server cm-r01nn02.mws.mds.xyz cm-r01nn02.mws.mds.xyz:7183 ssl check verify none port 7183 inter 12000 rise 3 fall 3
Otherwise the following port won't be open:
[root@cm-r01xs01 .ssh]# netstat -pnltu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 192.168.0.120:7183 0.0.0.0:* LISTEN 15001/haproxy
[root@cm-r01xs01 .ssh]#
And you'll receive the subject error. If you're routing traffic through your external facing router, ensure you have proper firewall rules configured for NAT traffic to and from the Azure instances:
iptables -t nat -I PREROUTING -s 103.192.131.145 -p tcp --dport 7183 -j DNAT --to 192.168.0.120:7183
iptables -I FORWARD -p tcp -d 192.168.0.53 --dport 7183 -j ACCEPT
iptables -t nat -I PREROUTING -s 103.192.131.145 -p udp --dport 7183 -j DNAT --to 192.168.0.120:7183
iptables -I FORWARD -p udp -d 192.168.0.53 --dport 7183 -j ACCEPT
Allowing traffic from said external IP 103.192.131.145. Furthermore, your HAproxy or Cloudera servers also contain the valid port. For instance:
[root@cm-r01xs01 .ssh]# cat /etc/firewalld/zones/public.xml |grep -Ei 7183
<port protocol="tcp" port="7183"/>
<port protocol="udp" port="7183"/>
[root@cm-r01xs01 .ssh]#
With that, you can expect the following result from your Azure instances.
[root@cm-awn01 ~]# nc -v 123.123.123.123 7183
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 123.123.123.123:7183.
HTH,
BD