Cloudera and Azure: WrongHost: Peer certificate subjectAltName does not match host, expected , got DNS:host01.dom.com, DNS:host02.dom.com, DNS:host03.dom.com
So you're getting this while trying to connect Cloud Hosts to your local Cloudera Infrastructure?
WrongHost: Peer certificate subjectAltName does not match host, expected dhcp-100-0-0-100.remote.user.isp.com, got DNS:srv-c01.cdh.local.hst, DNS:cm-r01nn01.cdh.local.hst, DNS:cm-r01nn02.cdh.local.hst
The reason for this is that Cloudera reconfigured your Agent config.ini to use the hostname derived from the IP address that is used to login to the remote machine. This is done while adding a new node. The script that does this is below:
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# ls -altri
total 152
67410958 -rwxr-xr-x 1 cdhroot cdhroot 49327 May 25 00:27 scm_prepare_node.sh
67410959 -rw-r–r– 1 cdhroot cdhroot 3023 May 25 00:27 US_export_policy.jar.8
67410960 -rw-r–r– 1 cdhroot cdhroot 3035 May 25 00:27 local_policy.jar.8
67410961 -rw-r–r– 1 cdhroot cdhroot 13155 May 25 00:27 customGPG
67410962 -rw-r–r– 1 cdhroot cdhroot 63 May 25 00:27 packages.scm
67410963 -rw-r–r– 1 cdhroot cdhroot 63 May 25 00:27 always_install.scm
67410964 -rw-r–r– 1 cdhroot cdhroot 15 May 25 00:27 x86_64_packages.scm
101634639 drwxrwxr-x 3 cdhroot cdhroot 19 May 25 00:27 repos
34689507 drwx—— 2 cdhroot cdhroot 83 May 25 00:27 gnupg.vNKKLRISC9
67410966 -rw-rw-r– 1 cdhroot cdhroot 1670 May 25 00:27 F36A89E33CC1BD0F71079007327574EE02A818DD.pub
67410967 -rw-rw-r– 1 cdhroot cdhroot 1691 May 25 00:27 5F14D39EF0681ACA6F044A43F90C0D8FE8F86ACD.pub
67410968 -rw-rw-r– 1 cdhroot cdhroot 1735 May 25 00:27 9543951160C284C0E7CA254573985D43B0B19C9F.pub
67410969 -rw-rw-r– 1 cdhroot cdhroot 3104 May 25 00:27 CECDB80C4E9004B0CFE852962279662784415700.pub
67410970 -rw-rw-r– 1 cdhroot cdhroot 5416 May 25 00:27 DF2C4DD7629B1AC08A0966E00F65552736F57F35.pub
67410965 -rw-rw-r– 1 cdhroot cdhroot 20819 May 25 00:27 scm_prepare_node.log
67410957 drwx—— 4 cdhroot cdhroot 4096 May 26 21:53 .
33554504 drwxrwxrwt. 16 root root 8192 May 26 22:01 ..
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
Below is a sample of it's log file:
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# cat scm_prepare_node.log
using SSH_CLIENT to get the SCM hostname: 100.0.0.100 42572 22
opening logging file descriptor
###CLOUDERA_SCM### SCRIPT_START
###CLOUDERA_SCM### TAKE_LOCK
BEGIN flock 4
END (0)
###CLOUDERA_SCM### DETECT_ROOT
effective UID is 1000
BEGIN which pbrun
END (1)
BEGIN sudo -S id
which: no pbrun in (/usr/local/bin:/usr/bin)
uid=0(root) gid=0(root) groups=0(root)
END (0)
Using 'sudo ' to acquire root privileges
###CLOUDERA_SCM### DETECT_DISTRO
BEGIN grep Tikanga /etc/redhat-release
END (1)
BEGIN grep 'Scientific Linux release 5' /etc/redhat-release
END (1)
BEGIN grep Santiago /etc/redhat-release
END (1)
BEGIN grep 'CentOS Linux release 6' /etc/redhat-release
END (1)
BEGIN grep 'CentOS release 6' /etc/redhat-release
END (1)
BEGIN grep 'Scientific Linux release 6' /etc/redhat-release
END (1)
BEGIN grep Maipo /etc/redhat-release
END (1)
BEGIN grep 'CentOS Linux release 7' /etc/redhat-release
END (0)
/etc/redhat-release ==> CentOS 7
###CLOUDERA_SCM### DETECT_SCM
CentOS Linux release 7.5.1804 (Core)
BEGIN host -t PTR 100.0.0.100
100.0.0.100.in-addr.arpa domain name pointer dhcp-100-0-0-100.cable.user.start.ca.
END (0)
using dhcp-100-0-0-100.cable.user.start.ca as scm server hostname
BEGIN which python
END (0)
.
.
.
Notice how the reverse lookup of the ISP IP is being shown. We want to ensure the hostname returned is what we have in our SSL / TLS Certificates.
To circumvent this, dnsmasqerade can be installed to intercept the reverse lookup requests and present the correct hostname the TLS certificate expects. Here is the full config:
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# host -t PTR 100.0.0.100
100.0.0.100.in-addr.arpa domain name pointer srv-c01.mws.mds.xyz.
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
31.25.100.45 cm-awn01.nix.mds.xyz cm-awn01
10.0.0.6 cm-awn01.nix.mds.xyz cm-awn01
100.0.0.100 srv-c01.mws.mds.xyz
100.0.0.100 srv-c01.mws.mds.xyz srv-c01
100.0.0.100 cm-r01nn01.mws.mds.xyz cm-r01nn01
100.0.0.100 cm-r01nn02.mws.mds.xyz cm-r01nn02
100.0.0.100 cm-r01en01.mws.mds.xyz cm-r01en01
100.0.0.100 cm-r01en02.mws.mds.xyz cm-r01en02
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# cat /etc/resolv.conf
# Generated by NetworkManager
search 4iktk4qpujwufjkjx511w5ourh.bx.internal.cloudapp.net nix.mds.xyz
nameserver 127.0.0.1
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# cat /etc/dnsmasq.conf
conf-dir=/etc/dnsmasq.d,.rpmnew,.rpmsave,.rpmorig
listen-address=::1,127.0.0.1,10.0.0.6
interface=eth0
# expand-hosts
# domain=nix.mds.xyz
# Google's nameservers
server=123.123.123.123
server=8.8.8.8
server=8.8.4.4
address=/nix.mds.xyz/127.0.0.1
address=/nix.mds.xyz/10.0.0.6
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# systemctl status dnsmasq
? dnsmasq.service – DNS caching server.
Loaded: loaded (/usr/lib/systemd/system/dnsmasq.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2020-05-26 22:23:45 EDT; 1min 55s ago
Main PID: 4841 (dnsmasq)
CGroup: /system.slice/dnsmasq.service
??4841 /usr/sbin/dnsmasq -k
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-c…inotify
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 8.8.4.4#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 8.8.8.8#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 123.123.123.123#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: reading /etc/resolv.conf
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 8.8.4.4#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 8.8.8.8#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 123.123.123.123#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: ignoring nameserver 127.0.0.1 – local interface
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: read /etc/hosts – 10 addresses
Hint: Some lines were ellipsized, use -l to show in full.
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
Thx,
TK