Header Shadow Image


Decomission or Recomission a host using Cloudera 6.X API Calls: /api/v3/cm/commands/hostsOfflineOrDecommission

Need to decomission a host?  Just call this:

curl -u admin:pAsS –insecure -X POST –header 'Content-Type: application/json' –header 'Accept: application/json' -d '{"items":[“cm-r01wn02.mws.mds.xyz”]}'    'https://cm-c01.mws.mds.xyz:7183/api/v3/cm/commands/hostsOfflineOrDecommission'
{
  "id" : 17256,
  "name" : "HostsDecommission",
  "startTime" : "2021-01-05T02:49:37.220Z",
  "active" : true,
  "children" : {
    "items" : [ ]
  }

Need to recomission a host?  Just call this:

curl -u admin:pAsS –insecure -X POST –header 'Content-Type: application/json' –header 'Accept: application/json' -d '{"items":[“cm-r01wn02.mws.mds.xyz”]}'    'https://cm-c01.mws.mds.xyz:7183/api/v33/cm/commands/hostsRecomission'

How this maps to the ApiHostNameLlst specification: https://archive.cloudera.com/cm6/6.2.0/generic/jar/cm_api/apidocs/json_ApiHostNameList.html

Cheers,

REF: https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/swagger-html-sdk-docs/java/docs/ClouderaManagerResourceApi.html#hostsDecommissionCommand
REF: https://cm-c01.mws.mds.xyz:7183/static/apidocs/ui/index.html#!/ClouderaManagerResource/hostsDecommissionCommand
 

WrongHost: Peer certificate subjectAltName does not match host, expected 1.2.3.4, got DNS: host1.domain, DNS: host2.domain, DNS: host3.domain

Another form of this error is when the certificate validation produced an IP instead of a host, such as this:

WrongHost: Peer certificate subjectAltName does not match host, expected 1.2.3.4, got DNS:srv-c01.earth.water.fire, DNS:cm-r01nn01.earth.water.fire, DNS:cm-r01nn02.earth.water.fire
[02/Jan/2021 03:15:59 +0000] 32309 Thread-13 downloader   ERROR    Failed fetching torrent: Peer certificate subjectAltName does not match host, expected 1.2.3.4, got DNS:srv-c01.earth.water.fire, DNS:cm-r01nn01.earth.water.fire, DNS:cm-r01nn02.earth.water.fire

In our software stack, Cloudera Manager is sitting behind an HAproxy / Keepalived VIP:

Cloudera CM <- HAproxy <- Keepalived <- Cloudera Node

In this case, the error was seen on the Cloudera Node.  So what could be the issue?

Verifying using forward and reverse lookups also produced expected results.

# dig -x 1.2.3.4
;; QUESTION SECTION:
;4.3.2.1.in-addr.arpa.       IN      PTR

;; ANSWER SECTION:
4.3.2.1.in-addr.arpa. 86400  IN      PTR     cm-r01nn01.earth.water.fire.


# dig cm-r01nn01.earth.water.fire
;; QUESTION SECTION:
;cm-r01nn01.earth.water.fire.                IN      A

;; ANSWER SECTION:
cm-r01nn01.earth.water.fire. 1200    IN      A       1.2.3.4


# nslookup 1.2.3.4
Server:         192.168.0.100
Address:        192.168.0.100#53

4.3.2.1.in-addr.arpa name = cm-r01nn01.earth.water.fire.


# nslookup cm-r01nn01
Server:         192.168.0.100
Address:        192.168.0.100#53

Name:   cm-r01nn01.earth.water.fire
Address: 1.2.3.4

Troubleshooting done revealed that pointing the node directly to the Cloudera CM server, bypassing the HAProxy and Keepalived VIP's worked well. A further investigation revealed that Selinux / Auditd were blocking HAProxy and Keepalived communication.  

type=AVC msg=audit(1609572407.005:1253694): avc:  denied  { name_bind } for  pid=3533 comm="haproxy" src=8084 scontext=system_u:system_r:haproxy_t:s0 tcontext=system_u:object_r:luci_port_t:s0 tclass=tcp_socket  

Running the following several times helped create the correct rules to allow communication:  

grep AVC /var/log/audit/audit.log* >> /var/log/audit/audit-denied.log; cat /var/log/audit/audit-denied.log | audit2allow -M systemd-allow;semodule -i systemd-allow.pp  
systemctl restart haproxy keepalived  

Initially, this did not appear to fully resolve the issue.  A full restart of the Cloudera SCM server however did, apparently confirming this problem was made up of two issues.  Communication to or from the CM server was easily analyzed by using tcpdump to verify that no traffic was being received or that no valid replies were being sent, from the Cloudera SCM server.  The underlying behaviour of the logic to detect the correct hostname isn't known without diving into the java source code, however, regardless of the logic, what could be said is that the returned hostname from the query produced no results, defaulting to an IP.  

Regards,

init_smb_request: invalid wct number 255 (size 248)

Getting this SMB error?

init_smb_request: invalid wct number 255 (size 248)

Solve it using this parameter in the SMB conf file on the server:

# grep -Ei "max protocol" /etc/samba/smb.conf; cat messages|grep -Ei smb|grep 255|tail
        max protocol = SMB2

Cheers,

Kerberos authentication failed: kinit: Cannot read password while getting initial credentials

Sometimes for messages like this:  

Kerberos authentication failed: kinit: Cannot read password while getting initial credentials

There is a simple solution.  Reset the user's password, because it probably expired or the user account used was just created without the user having set a new password on it.  In our case, running the following FreeIPA command produced the above issue:  

ipa-client-install –force-join -p autojoin -w "SecretPass" –fixed-primary –server=$IPA01.$NDOMAIN –server=$IPA02.$NDOMAIN –domain=$NDOMAIN –realm=$UNDOMAIN -U

Cheers,
TK

User is not authorized to read Azure subscriptions. Permission elevation is required to proceed.

Getting this while trying to delete Azure Active Directory Tenants?

{"errorCode":"PermissionsElevationRequiredToReadSubscriptions","localizedErrorDetails":{"permissionsElevationRequiredToReadSubscriptions":"User is not authorized to read Azure subscriptions. Permission elevation is required to proceed."},"operationResults":null,"timeStampUtc":"2020-11-23T02:38:42.————-","clientRequestId":"—————","internalTransactionId":"——————–","tenantId":"——————–","userObjectId":"—————————","exceptionType":"UnauthorizedAccessException"}

Switch Directories to another one.  Then from there, click on Overview of this Active Directory, then click on Switch Tenant.  Delete the Tenant from here.  Deleting a Tenant whilst selected won't work. Once you do this, refresh the pages.  Your Tenant should now be gone. 

Cheers,
BV

C:\Program Files\WindowsApps\Microsoft.Darwin_100.1.38862.0_x64__8weekyb3d8bbwe\InputSystem_w32.dll is either not designed to run on Windows or it contains an error.

Receiving the following when trying to start Age of Empires: Definitive Edition?

C:\Program Files\WindowsApps\Microsoft.Darwin_100.1.38862.0_x64__8weekyb3d8bbwe\InputSystem_w32.dll is either not designed to run on Windows or it contains an error. Try installing the program again using the original installation media or contact your system administrator or the software vendor for support. Error status 0xc0000022.

Read the rest of this entry »

OpenWRT: Microsoft Azure to Cloudera CDH via VPN Gateway

In this post, we'll show you how to create and connect your local home network to the Azure space network.  We'll take this a step further by connecting this Microsoft Azure VM instances defined to an on premise Cloudera CDH cluster.  Together, the on-prem cluster will be extended with compute capacity from Azure while the workloads are running.  Once workloads are done, the extra compute can be turned off or destroyed no the Azure side. This will provide some cost savings while also reducing the overall IaaS and PaaS costs normally associated with on-prem infrastructures.  The below steps are essentially a learning LAB or POC type of setup.  This is not meant for a PROD type of setup.  For PROD, Expressroute or a higher end configuration will be needed.  Or entirely Cloud based solutions would take the place of this setup. 

Read the rest of this entry »

Asus Merlin Firmware: Wrong date shows set to Sat May  5 01:07:40 DST 2018

Do you end up with the wrong date when using Asus or Asus Merlin software?

admin@ASUS-MERLIN-INTERNET:/tmp/home/root# date
Sat May  5 01:07:40 DST 2018
admin@ASUS-MERLIN-INTERNET:/tmp/home/root#

Not yet clear what is really causing this but a temporary workaround is shown below using the Asus Merlin startup scripts:

admin@ASUS-MERLIN-INTERNET:/tmp/home/root# cat /jffs/scripts/init-start
#!/bin/sh

NTP0=$(nvram show 2>/dev/null | awk -F'=' '/ntp_server0/{ print $2 }')
NTP1=$(nvram show 2>/dev/null | awk -F'=' '/ntp_server1/{ print $2 }')
PSV=$(ps|grep -Ei "ntpd_[s]ynced"|wc -l)

echo "Using the following NTP servers: NTP0 ($NTP0) and NTP1 ($NTP1).  Number of running NTP servers right now is $PSV";

if [[ $NTP0 != “” && $NTP1 != “” && $PSV == 0 ]]; then
        /usr/sbin/ntp -d -n -t -S /sbin/ntpd_synced -p $NTP0 -p $NTP1 &
        if $? == 0; then
                echo "SUCCESS: Started the NTPD server."
        else
                echo "FAILED to start the NTPD server.  Non 0 exit code detected."
        fi
else
        echo "ERROR:  Either NTP0($NTP0) or NTP1($NTP1) was empty.  Or NTPD was already started.  No action taken.";
fi
admin@ASUS-MERLIN-INTERNET:/tmp/home/root# ls -altri /jffs/scripts/init-start
   9640 -rwxr-x—    1 admin    root           716 Oct 17 18:50 /jffs/scripts/init-start
admin@ASUS-MERLIN-INTERNET:/tmp/home/root#

It uses the nvram variables to retireve your configured NTP servers.  

Hope this helps!

Thx,

iPhone Bricked: Update or Recovery

Had the misfortune of experiencing of doing the Apple equivalent of bricking my iPhone while doing an iOS update.  Why did I do an iOS update?  Well here's how I did this without any data loss. 

Read the rest of this entry »

Lost Thunderbird Settings and Folders

Lost Thunderbird settings and folders?  All settings in our Thunderbird were reset after a string of events including recent upgrade and some random reboots from apparent hardware issues.  Irrespective, file corruption occurred. 

Read the rest of this entry »


     
  Copyright © 2003 - 2013 Tom Kacperski (microdevsys.com). All rights reserved.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License