Header Shadow Image


Cloudera and Azure: WrongHost: Peer certificate subjectAltName does not match host, expected , got DNS:host01.dom.com, DNS:host02.dom.com, DNS:host03.dom.com

So you're getting this while trying to connect Cloud Hosts to your local Cloudera Infrastructure?

WrongHost: Peer certificate subjectAltName does not match host, expected dhcp-100-0-0-100.remote.user.isp.com, got DNS:srv-c01.cdh.local.hst, DNS:cm-r01nn01.cdh.local.hst, DNS:cm-r01nn02.cdh.local.hst

The reason for this is that Cloudera reconfigured your Agent config.ini to use the hostname derived from the IP address that is used to login to the remote machine. This is done while adding a new node.  The script that does this is below:  

[root@cm-awn01 scm_prepare_node.pFWkTK2i]# ls -altri
total 152
 67410958 -rwxr-xr-x   1 cdhroot cdhroot 49327 May 25 00:27 scm_prepare_node.sh
 67410959 -rw-r–r–   1 cdhroot cdhroot  3023 May 25 00:27 US_export_policy.jar.8
 67410960 -rw-r–r–   1 cdhroot cdhroot  3035 May 25 00:27 local_policy.jar.8
 67410961 -rw-r–r–   1 cdhroot cdhroot 13155 May 25 00:27 customGPG
 67410962 -rw-r–r–   1 cdhroot cdhroot    63 May 25 00:27 packages.scm
 67410963 -rw-r–r–   1 cdhroot cdhroot    63 May 25 00:27 always_install.scm
 67410964 -rw-r–r–   1 cdhroot cdhroot    15 May 25 00:27 x86_64_packages.scm
101634639 drwxrwxr-x   3 cdhroot cdhroot    19 May 25 00:27 repos
 34689507 drwx——   2 cdhroot cdhroot    83 May 25 00:27 gnupg.vNKKLRISC9
 67410966 -rw-rw-r–   1 cdhroot cdhroot  1670 May 25 00:27 F36A89E33CC1BD0F71079007327574EE02A818DD.pub
 67410967 -rw-rw-r–   1 cdhroot cdhroot  1691 May 25 00:27 5F14D39EF0681ACA6F044A43F90C0D8FE8F86ACD.pub
 67410968 -rw-rw-r–   1 cdhroot cdhroot  1735 May 25 00:27 9543951160C284C0E7CA254573985D43B0B19C9F.pub
 67410969 -rw-rw-r–   1 cdhroot cdhroot  3104 May 25 00:27 CECDB80C4E9004B0CFE852962279662784415700.pub
 67410970 -rw-rw-r–   1 cdhroot cdhroot  5416 May 25 00:27 DF2C4DD7629B1AC08A0966E00F65552736F57F35.pub
 67410965 -rw-rw-r–   1 cdhroot cdhroot 20819 May 25 00:27 scm_prepare_node.log
 67410957 drwx——   4 cdhroot cdhroot  4096 May 26 21:53 .
 33554504 drwxrwxrwt. 16 root    root     8192 May 26 22:01 ..
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#

Below is a sample of it's log file:

[root@cm-awn01 scm_prepare_node.pFWkTK2i]# cat scm_prepare_node.log
using SSH_CLIENT to get the SCM hostname: 100.0.0.100 42572 22
opening logging file descriptor
###CLOUDERA_SCM### SCRIPT_START
###CLOUDERA_SCM### TAKE_LOCK
BEGIN flock 4
END (0)
###CLOUDERA_SCM### DETECT_ROOT
effective UID is 1000
BEGIN which pbrun
END (1)
BEGIN sudo -S id
which: no pbrun in (/usr/local/bin:/usr/bin)
uid=0(root) gid=0(root) groups=0(root)
END (0)
Using 'sudo ' to acquire root privileges
###CLOUDERA_SCM### DETECT_DISTRO
BEGIN grep Tikanga /etc/redhat-release
END (1)
BEGIN grep 'Scientific Linux release 5' /etc/redhat-release
END (1)
BEGIN grep Santiago /etc/redhat-release
END (1)
BEGIN grep 'CentOS Linux release 6' /etc/redhat-release
END (1)
BEGIN grep 'CentOS release 6' /etc/redhat-release
END (1)
BEGIN grep 'Scientific Linux release 6' /etc/redhat-release
END (1)
BEGIN grep Maipo /etc/redhat-release
END (1)
BEGIN grep 'CentOS Linux release 7' /etc/redhat-release
END (0)
/etc/redhat-release ==> CentOS 7
###CLOUDERA_SCM### DETECT_SCM
CentOS Linux release 7.5.1804 (Core)
BEGIN host -t PTR 100.0.0.100
100.0.0.100.in-addr.arpa domain name pointer dhcp-100-0-0-100.cable.user.start.ca.
END (0)
using dhcp-100-0-0-100.cable.user.start.ca as scm server hostname
BEGIN which python
END (0)
.
.
.

Notice how the reverse lookup of the ISP IP is being shown.  We want to ensure the hostname returned is what we have in our SSL / TLS Certificates.

To circumvent this, dnsmasqerade can be installed to intercept the reverse lookup requests and present the correct hostname the TLS certificate expects.  Here is the full config:

[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# host -t PTR 100.0.0.100
100.0.0.100.in-addr.arpa domain name pointer srv-c01.mws.mds.xyz.
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

31.25.100.45            cm-awn01.nix.mds.xyz cm-awn01
10.0.0.6                cm-awn01.nix.mds.xyz cm-awn01
100.0.0.100         srv-c01.mws.mds.xyz

100.0.0.100         srv-c01.mws.mds.xyz srv-c01
100.0.0.100         cm-r01nn01.mws.mds.xyz cm-r01nn01
100.0.0.100         cm-r01nn02.mws.mds.xyz cm-r01nn02
100.0.0.100         cm-r01en01.mws.mds.xyz cm-r01en01
100.0.0.100         cm-r01en02.mws.mds.xyz cm-r01en02
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# cat /etc/resolv.conf
# Generated by NetworkManager
search 4iktk4qpujwufjkjx511w5ourh.bx.internal.cloudapp.net nix.mds.xyz
nameserver 127.0.0.1
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# cat /etc/dnsmasq.conf
conf-dir=/etc/dnsmasq.d,.rpmnew,.rpmsave,.rpmorig

listen-address=::1,127.0.0.1,10.0.0.6

interface=eth0
# expand-hosts
# domain=nix.mds.xyz

# Google's nameservers
server=123.123.123.123
server=8.8.8.8
server=8.8.4.4

address=/nix.mds.xyz/127.0.0.1
address=/nix.mds.xyz/10.0.0.6
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#
[root@cm-awn01 scm_prepare_node.pFWkTK2i]# systemctl status dnsmasq
? dnsmasq.service – DNS caching server.
   Loaded: loaded (/usr/lib/systemd/system/dnsmasq.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2020-05-26 22:23:45 EDT; 1min 55s ago
 Main PID: 4841 (dnsmasq)
   CGroup: /system.slice/dnsmasq.service
           ??4841 /usr/sbin/dnsmasq -k

May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-c…inotify
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 8.8.4.4#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 8.8.8.8#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 123.123.123.123#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: reading /etc/resolv.conf
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 8.8.4.4#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 8.8.8.8#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: using nameserver 123.123.123.123#53
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: ignoring nameserver 127.0.0.1 – local interface
May 26 22:23:45 cm-awn01.nix.mds.xyz dnsmasq[4841]: read /etc/hosts – 10 addresses
Hint: Some lines were ellipsized, use -l to show in full.
[root@cm-awn01 scm_prepare_node.pFWkTK2i]#

 

Thx,
TK

Leave a Reply

You must be logged in to post a comment.


     
  Copyright © 2003 - 2013 Tom Kacperski (microdevsys.com). All rights reserved.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License