Header Shadow Image


Cloudera: Azure: URLError:

Receiving the following when connecting external cloud servers from Azure, AWS, or GCP?

[17/May/2020 13:29:50 +0000] 4894 Thread-13 https        ERROR    Failed to retrieve/store URL: https://cm-r01nn01.mws.mds.xyz:7183/cmf/pa    rcel/download/CDH-6.3.0-1.cdh6.3.0.p0.1279813-el7.parcel.torrent -> /opt/cloudera/parcel-cache/CDH-6.3.0-1.cdh6.3.0.p0.1279813-el7.parcel.    torrent Traceback (most recent call last):
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 193, in fetch_to_file
    resp = self.open(req_url)
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 188, in open
    return self.opener(url, *pargs, **kwargs)
  File "/usr/lib64/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 179, in https_open
    return self.do_open(opener, req)
  File "/usr/lib64/python2.7/urllib2.py", line 1214, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno -2] Name or service not known>

Solve it by adding entries into your /etc/hosts file like this:

[root@cm-awn01 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

100.100.100.10   cm-awn01.nix.mds.xyz cm-awn01
10.0.0.6        cm-awn01.nix.mds.xyz cm-awn01

123.123.123.123 srv-c01.mws.mds.xyz
123.123.123.123 cm-r01nn01.mws.mds.xyz
123.123.123.123 cm-r01nn02.mws.mds.xyz
[root@cm-awn01 ~]#

Thanks
TK

Cloudera: WrongHost: Peer certificate subjectAltName does not match host

Getting the following when configuring remote workers on Azure:

[17/May/2020 13:09:38 +0000] 3529 MainThread agent        ERROR    Heartbeating to 123.123.123.123:7182 failed.
Traceback (most recent call last):
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/agent.py", line 1387, in _send_heartbeat
    self.cfg.max_cert_depth)
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 139, in _init_
    self.conn.connect()
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/httpslib.py", line 69, in connect
    sock.connect((self.host, self.port))
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/SSL/Connection.py", line 313, in connect
    if not check(self.get_peer_cert(), self.addr[0]):
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/SSL/Checker.py", line 125, in _call_
    fieldName='subjectAltName')
WrongHost: Peer certificate subjectAltName does not match host, expected 123.123.123.123, got DNS:srv-c01.mws.mds.xyz, DNS:cm-r01nn01.mws.mds.xyz, DNS:cm-r01nn02.mws.mds.xyz

Because locally visible hostnames aren't externally, /etc/hosts modifications are necessary in this case to make the self signed certificates happy:

[root@cm-awn01 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

100.100.100.10   cm-awn01.nix.mds.xyz cm-awn01
10.0.0.6        cm-awn01.nix.mds.xyz cm-awn01

123.123.123.123 srv-c01.mws.mds.xyz
123.123.123.123 cm-r01nn01.mws.mds.xyz
123.123.123.123 cm-r01nn02.mws.mds.xyz
[root@cm-awn01 ~]#

Ensure your Cloudera Agent Config matches:

[root@cm-awn01 ~]# cat /etc/cloudera-scm-agent/config.ini|grep server
server_host=srv-c01.mws.mds.xyz

Hackish but works.

GL,
TK

Cloudera: Ncat: Connection timed out.

Getting this error when connecting Azure instances back to a Cloudera CM + CDH installation?

[root@cm-awn01 ~]# nc -vz srv-c01.mws.mds.xyz 7191
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connection timed out.
[root@cm-awn01 ~]#

Ensure ports are defined correctly in the local firewall configuration:

[root@cm-r01xs01 .ssh]# grep -Ei 7191 /etc/firewalld/zones/public.xml
  <port protocol="tcp" port="7191"/>
  <port protocol="udp" port="7191"/>

[root@cm-r01xs01 .ssh]#

Ensure NAT rules are also in place in case connectivity occurs through an external facing router.

iptables -t nat -I PREROUTING -s 123.123.123.123 -p tcp --dport 7191 -j DNAT --to 192.168.0.120:7191
iptables -I FORWARD -p tcp -d 192.168.0.120 --dport 7191 -j ACCEPT
iptables -t nat -I PREROUTING -s 123.123.123.123 -p udp --dport 7191 -j DNAT --to 192.168.0.120:7191
iptables -I FORWARD -p udp -d 192.168.0.120 --dport 7191 -j ACCEPT

Good Luck!
TK

Cloudera: Ncat: Connection refused.

Getting the following connecting from your Azure instance back to your Cloudera CM + CDH Cluster?

[root@cm-awn01 ~]# nc -v 123.123.123.123 7183
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connection refused.

The corresponding Cloudera Agent error looks like this:

[17/May/2020 13:34:53 +0000] 5306 Thread-13 https        ERROR    Failed to retrieve/store URL: https://cm-r01nn01.mws.mds.xyz:7183/cmf/par   cel/download/CDH-6.3.0-1.cdh6.3.0.p0.1279813-el7.parcel.torrent -> /opt/cloudera/parcel-cache/CDH-6.3.0-1.cdh6.3.0.p0.1279813-el7.parcel.to   rrent Traceback (most recent call last): 
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 193, in fetch_to_file
    resp = self.open(req_url)
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 188, in open
    return self.opener(url, *pargs, **kwargs)
  File "/usr/lib64/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 179, in https_open
    return self.do_open(opener, req)
  File "/usr/lib64/python2.7/urllib2.py", line 1214, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno 111] Connection refused>

If you're using HAProxy for traffic routing to various instances, you'll need the following stanza in the HAProxy configuration file:

# CM GUI
listen cm
        bind cm-c01:80
        mode    http
        redirect scheme https if !{ ssl_fc }

frontend cmin
        bind    cm-c01:443 ssl crt /etc/haproxy/certs/cm-c01.mws.mds.xyz-haproxy.pem no-sslv3
        default_backend cmback

frontend cm7183in
        bind    cm-c01:7183 ssl crt /etc/haproxy/certs/cm-c01.mws.mds.xyz-haproxy.pem no-sslv3
        default_backend cmback

backend cmback
        mode http
        balance roundrobin

        server cm-r01nn01.mws.mds.xyz    cm-r01nn01.mws.mds.xyz:7183 ssl check verify none port 7183 inter 12000 rise 3 fall 3
        server cm-r01nn02.mws.mds.xyz    cm-r01nn02.mws.mds.xyz:7183 ssl check verify none port 7183 inter 12000 rise 3 fall 3

Otherwise the following port won't be open:

[root@cm-r01xs01 .ssh]# netstat -pnltu 
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 192.168.0.120:7183       0.0.0.0:*               LISTEN      15001/haproxy
[root@cm-r01xs01 .ssh]# 

And you'll receive the subject error.  If you're routing traffic through your external facing router, ensure you have proper firewall rules configured for NAT traffic to and from the Azure instances:

iptables -t nat -I PREROUTING -s 103.192.131.145 -p tcp --dport 7183 -j DNAT --to 192.168.0.120:7183
iptables -I FORWARD -p tcp -d 192.168.0.53 --dport 7183 -j ACCEPT
iptables -t nat -I PREROUTING -s 103.192.131.145 -p udp --dport 7183 -j DNAT --to 192.168.0.120:7183
iptables -I FORWARD -p udp -d 192.168.0.53 --dport 7183 -j ACCEPT

Allowing traffic from said external IP 103.192.131.145.  Furthermore, your HAproxy or Cloudera servers also contain the valid port.  For instance:

[root@cm-r01xs01 .ssh]# cat /etc/firewalld/zones/public.xml |grep -Ei 7183
  <port protocol="tcp" port="7183"/>
  <port protocol="udp" port="7183"/>
[root@cm-r01xs01 .ssh]#

With that, you can expect the following result from your Azure instances.

[root@cm-awn01 ~]# nc -v 123.123.123.123 7183
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 123.123.123.123:7183.

HTH,
BD

Importing Certificates Into the Truststore: Connection not protected

Getting this message?

Connection not protected

The security of your connection is reduced. Criminals can attempt to steal your data from the website. You are advised to leave this website.

And you know this is a trusted site, such as a local HTTPS web server?  Then export the certificate and add it to your trusted certificates stash by following these steps.

  1. Click on View Certificate
  2. From the panel that opens up, click on the Details tab.
  3. Next, click on Copy to File….
  4. Leave default settings.  In our case, it was DER encoded binary X.509 (. CER )
  5. Following that select the location and the name of the file to copy the certificate too.
  6. Next, start mmc.exe by searching for this program on your Windows 10 machine or executing it from the command line. 
  7. Click on File then  Add / Remove Snap In
  8. Click on Certificates followed by clicking the Add > button.
  9. Save or click Finish
  10. Next, navigate to Console Root -> Certificates (Local Computer) -> Trusted Root Certificatiom Authorities -> Certificates
  11. On the panel that opens up, right click Certificates and All Tasks -> Import
  12. Once imported, File -> Save and Exit.
  13. Reload your site. 

Steps 1,2,3,4,5:

https://i2.wp.com/www.microdevsys.com/WordPressImages/ImportTrustedCertificates-CertificateExportWizard.JPG?ssl=1

Steps 6, 7 and 8:

https://i2.wp.com/www.microdevsys.com/WordPressImages/ImportTrustedCertificates-MMC-Add-Certificates-Snap-In.JPG?ssl=1

Steps 10 and 11:

https://i2.wp.com/www.microdevsys.com/WordPressImages/ImportTrustedCertificates-MMC-Import-Certificate-Wizard.JPG?ssl=1

Regards,
AS

Connection not protected: Kaspersky Antivirus

Getting this?

Connection not protected

The security of your connection is reduced. Criminals can attempt to steal your data from the website. You are advised to leave this website.

For regular web traffic, this is normal. But if you want to avoid this for a local HTTPS server you're running, then these are the steps you want to take to disable this:

  1. In Kacpersky Total Security, click the Cog Wheel near the bottom left.
  2. Next, select Protection then Application Control from the right side. 
  3. Click on Manage Applications
  4. Type the browser you would like to modify in the top right search box.
  5. Double click the browser app and click the Exclusions tab
  6. Click on Do not scan all traffic 
  7. Select the checkbox Only for specific IP addresses
  8. Type in the addresses.
  9. Save the settings

Example of what this look like:

https://i2.wp.com/www.microdevsys.com/WordPressImages/KasperskyAntivitus-AllowSites.JPG?ssl=1

NOTE: Above screenshots shows a sample application.  It should really be Microsoft iExplorer, Firefox or Chrome if those browsers are being used. 

Thx,
EJ

This site is not secure

Getting this?

This site is not secure

This might mean that someone’s trying to fool you or steal any info you send to the server. You should close this site immediately

For regular web traffic, this is a valid warning message.  And you should be wary of sites that don't have valid certificates.  It's inadvisable to purchase anything from sites that don't conform to the expected security standards to keep your money safe.  Simply leave the site. 

However, when running a home-based HTTPS server, this can get annoying.  The site is known.  It's yours.  So why not just trust it and avoid the hassle. Despite IE giving the option to trust a set of sites by adding them to the trusted site's panel in internet options, that still will not work fully.  However, if you plan to use IE only for your local development work, then the steps below are fine. In this case, updates IE settings by:

  1. Click the IE Settings (Cog Wheel) at the top right corner of your IE browser.
  2. Select Internet Options
  3. Uncheck: Warn about certificate address mismatch
  4. Uncheck: Check for publisher's certificate revocation
  5. Uncheck: Check for server certificate revocation*
  6. Uncheck: Check for signatures on downloaded programs

Apply and save the settings.  Restart Internet Explorer.  The warning should now go away. 

Cheers,
TK

etcd: error validating peerURLs or etcd: request sent was ignored ( cluster ID mismatch )

Getting the following?

etcd: health check for peer 83f149dc6ec1b00a could not connect: dial tcp 10.3.0.124:2380: connect: connection refused (prober "ROUND_TRIPPER_SNAPSHOT")

or

etcd: request sent was ignored (cluster ID mismatch: remote[a82b23223d9f684e]=314e5a8f7a211a07, local=47f62724bd585a9)

or

etcd: publish error: etcdserver: request timed out

or

etcd: error validating peerURLs {ClusterID:314e5a8f7a211a07 Members:[&{ID:c470debdfc4607fe RaftAttributes:{PeerURLs:[http://192.168.0.108:2380]} Attributes:{Name:etcd01 ClientURLs:[]}} &{ID:83f149dc6ec1b00a RaftAttributes:{PeerURLs:[http://10.3.0.124:2380]} Attributes:{Name:etcd02 ClientURLs:[http://10.3.0.124:2379]}} &{ID:a82b23223d9f684e RaftAttributes:{PeerURLs:[http://10.3.0.118:2380]} Attributes:{Name:etcd03 ClientURLs:[http://10.3.0.118:2379]}}] RemovedMemberIDs:[]}: unmatched member while checking PeerURLs ("http://10.3.0.118:2380"(resolved from "http://10.3.0.118:2380") != "http://10.3.0.108:2380"(resolved from "http://10.3.0.108:2380"))

or

Error:  client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: connect: connection refused
; error #1: dial tcp 127.0.0.1:4001: connect: connection refused

error #0: dial tcp 127.0.0.1:2379: connect: connection refused
error #1: dial tcp 127.0.0.1:4001: connect: connection refused

or

etcd: request sent was ignored (cluster ID mismatch: peer[83f149dc6ec1b00a]=314e5a8f7a211a07, local=47f62724bd585a9)

You can solve this by reinitializing the entire cluster. Set the cluster state to new on each node:

[root@psql03 etcd]# cat /etc/etcd/etcd.conf
ETCD_LISTEN_PEER_URLS="http://10.3.0.118:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://10.3.0.118:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.3.0.118:2380"
ETCD_INITIAL_CLUSTER="etcd01=http://10.3.0.108:2380,etcd02=http://10.3.0.124:2380,etcd03=http://10.3.0.118:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://10.3.0.118:2379"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-c01"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_NAME="etcd03"
ETCD_HEARTBEAT_INTERVAL=250
ETCD_ELECTION_TIMEOUT=1250
[root@psql03 etcd]#

 

[root@psql02 etcd]# cat /etc/etcd/etcd.conf
ETCD_LISTEN_PEER_URLS="http://10.3.0.124:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://10.3.0.124:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.3.0.124:2380"
ETCD_INITIAL_CLUSTER="etcd01=http://10.3.0.108:2380,etcd02=http://10.3.0.124:2380,etcd03=http://10.3.0.118:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://10.3.0.124:2379"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-c01"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_NAME="etcd02"
ETCD_HEARTBEAT_INTERVAL=250
ETCD_ELECTION_TIMEOUT=1250
[root@psql02 etcd]#

 

[root@psql01 snap]# cat /etc/etcd/etcd.conf
ETCD_LISTEN_PEER_URLS="http://10.3.0.108:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://10.3.0.108:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.3.0.108:2380"
ETCD_INITIAL_CLUSTER="etcd01=http://10.3.0.108:2380,etcd02=http://10.3.0.124:2380,etcd03=http://10.3.0.118:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://10.3.0.108:2379"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-c01"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_NAME="etcd01"
ETCD_HEARTBEAT_INTERVAL=250
ETCD_ELECTION_TIMEOUT=1250
[root@psql01 snap]#

Then start each node:

systemctl start etcd

and verify:

[root@psql03 etcd]# etcdctl  cluster-health
member 83f149dc6ec1b00a is healthy: got healthy result from http://10.3.0.124:2379
member 93200353704b2d19 is healthy: got healthy result from http://10.3.0.108:2379
member a82b23223d9f684e is healthy: got healthy result from http://10.3.0.118:2379
cluster is healthy
[root@psql03 etcd]#

Next, go back and change the configuration of each ETCD member back to existing:

ETCD_INITIAL_CLUSTER_STATE="existing"

Restart all nodes by stopping ETCD first then starting it up on all nodes in close succession:

[root@psql01 etcd]# systemctl start etcd
[root@psql02 etcd]# systemctl start etcd
[root@psql03 etcd]# systemctl start etcd

At this point you should be good.  Now if you also get this message:

etcd[16312]: the clock difference against peer a82b23223d9f684e is too high [1.20986188s > 1s] (prober "ROUND_TRIPPER…MESSAGE")

Check your NTP configuration, run ntpdate <NTP SERVER> and restart NTPD: systemctl restart ntpd .   This should resolve it. 

Thx,
TK

Conflicting collector combinations in option list; please refer to the release notes for the combinations allowed

Getting this?

Conflicting collector combinations in option list; please refer to the release notes for the combinations allowed

It could be because of this parameter in conjunction with the rest of the GC options specified.

ExecStart=/usr/bin/java -Xmx4096M -Xms2048M -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+AlwaysPreTouch -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSIncrementalPacing -XX:ParallelGCThreads=4 -XX:+AggressiveOpts nogui -XX:+UnlockExperimentalVMOptions -XX:ActiveProcessorCount=4 -jar server.jar

But that wasn't the only thing.  It was also this option:

-XX:+UseParNewGC

Turns out that:

-XX:+UseG1GC

is a newer option that may conflict with the rest of the options as well. 

Thx,

Windows 10: Cannot ping other VLAN’s / Subnets : Request timed out.

Getting this?

C:\Users\tom>ping 10.0.0.1

Pinging 10.0.0.1 with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 10.0.0.1:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

Read the rest of this entry »


     
  Copyright © 2003 - 2013 Tom Kacperski (microdevsys.com). All rights reserved.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License