If you are receiving the following when postgresql ( w/ Patroni ) is starting up: 2019-04-04 14:59:15.715 EDT [26025] FATAL:  could not receive data from WAL stream: ERROR:  requested WAL segment 000000390000000000000008 has already been removed 2019-04-04 14:59:16.420 EDT [26029] FATAL:  the database system is starting up consider running the individual postgres line separately in […]

To fix: May 22 00:29:31 psql01 etcd: read wal error (walpb: crc mismatch) and cannot be repaired Do the following.  First copy the old wal files out of the way: [root@psql01 wal]# ls -altri total 375092 201347741 -rw——-. 1 etcd etcd 64000056 Mar 30 15:44 0000000000000027-000000000181bbc9.wal 201347715 -rw——-. 1 etcd etcd 64000104 Apr  1 18:46 […]

Linux LVM: Adding Disk Space to Virtual of Physical Drives In this writeup, we will aim to increase the size of the root drive that has: 1) Standard drive partitioning using fdisk and no LVM: /dev/sda1 2) Has LVM for the OS and files: /dev/sda2 This procedure will help to avoid the can't find the […]

DNS issue: Can't ping but nslookup works You can do several things in this case. Start Services then recycle DHCP Client.  ipconfig /flushdns and netsh int ip reset resettcpip.txt can fix this temporarily as well. I've elected to simply stop DHCP Client and let the system do all lookups against my internal DNS servers. This […]

We will setup a serial null modem cable for administering and connecting to a physical machine via another in the event that:   1) We want to capture kernel crashes and dumps.   2) Login to the machine machine remotely via another linux box to do things like restart the network.   For this we will need:   1) […]

Getting this? com.cloudera.cmf.service.CommandException: java.io.IOException: Cannot create command directory: /var/lib/cloudera-scm-server/temp/commands/114 it's because we blow the folder away.  Reinstall the packages: [root@cm-r01nn01 ~]# yum reinstall cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server -y Thx, TK

Recent kernel panics have pointed to an issue with the HPET timer on some motherboards.  To disable, add the following to the kernel line: nohpet clocksource=rtc Then also disable HPET from BIOS. Thx, TK  

Getting this? # locale locale: Cannot set LC_CTYPE to default locale: No such file or directory locale: Cannot set LC_MESSAGES to default locale: No such file or directory locale: Cannot set LC_ALL to default locale: No such file or directory   Check this: [root@cm-r01nn02 yum.repos.d]# cat /etc/locale.conf LANG=en_EN.UTF-8 [root@cm-r01nn02 yum.repos.d]#  should be: [root@cm-r01nn01 ~]# cat […]

When receiving this error, look into the worker cloudera–scm-agent to determine why.  In our case it was:   [29/Mar/2019 00:11:47 +0000] 800 MainThread agent        ERROR    Error, CM server guid updated, expected f2f1e171-d20d-4425-afe9-58b567b51397, received 18757343-bd5c-4b15-a104-91fd432ebc82 Then follow this page to resolve the above.   Thx, TK

If you are getting the following: 2019-03-21 01:04:17,021 FATAL main:org.hsqldb.cmdline.SqlFile: SQL Error at 'UTF-8' line 6: "alter table SETTINGS     add column LDAP_USER_SEARCH_BASE varchar(1024),     add column LDAP_USER_SEARCH_FILTER varchar(1024),     add column LDAP_GROUP_SEARCH_BASE varchar(1024),     add column LDAP_GROUP_SEARCH_FILTER varchar(1024)" Row size too large. The maximum row size for the used table […]