Header Shadow Image

David Korn, where are you?

Folks are asking where you are?

I've wondered the same and hope to see David Korn back in action again.  Thread's been quiet.  Did some reasearch earlier and David Korn was hired by Google after departing from AT&T (http://www.unix.com/what-is-on-your-mind-/237119-david-korn-glenn-fowler-laid-off.html). No idea what's he's working on but his profile is: 


And some chat about this is at:

There was some GitHub work on it:

I haven't seen much progress at all on KSH in the last 3 years to be honest.  But I did see a general industry wide movement away from the traditional UNIX systems over to Linux where BASH, Python, NodeJS(Yes even for scripting on the Linux command line) is preferred over KSH.  (This is a bit odd given that BASH is derived from KSH, if I'm not mistaken.)

Seems like the biggest factor in determining these changes in 2013 was IBM's, Oracle's, HP's lack of any major announcements for the AIX, Solaris and HP-UX platforms that was being used as the last remaining major UNIX versions.  For example, there was supposed to be AIX 8.1 but they downgraded that to AIX 7.2 with a future release of AIX seemingly planned for 2019 which likely may be more Linux based then anything else.  Perhaps the industry is interpreting the lack of updates to the UNIX platforms as 'The End' and so anything tightly coupled with it like KSH is considered deprecated?

In the least, if KSH is on it's way out as a mainstream scripting language, I would have expected KSH to move towards the newer realms of development areas such as in Configuration Management and the likes of Salt, Ansible, Puppet, Chef etc.  We see alot of new languages sprout up that are about 20% as powerful or mature as KSH is but are all DSL's (Domain Specific Languages) so no one knows these languages nor has anyone considered the limitations of these languages yet nor portability of transfer ability of these languages to other DSL's. Vendors are very reluctant to make their DSL's cross platform as well, and for good reason.

While the industry is struggling a bit with a shortage of folks able to write code for Salt, Ansible, Puppet, Chef etc.  a significant number of scripters stand by the sidelines with proven languages.  The templating nature of these DSL's is supposed to make them better at transferring the knowledge over to new folks that join, which is true (less of a learning curve then the equivalent 500 line traditional shell script written by an expert) but at the same time the DSL's are leaving behind infrastructures that no one is able to support and code that has to be rewritten when they leave or code that is simply a mish-mash of the DSL's and traditional languages like Bash, Perl, Python, KSH etc.  

Perl popularity isn't what I would have expected as well with Python moving up quite quickly.

Going back to my earlier comment, if KSH was given templates for common tasks across any flavor such as changing an IP, adding a DHCP, managing a config file or config entry etc. across these platforms it would certainly help here, if there's still apetite for that.  Also having one platform, Linux, vs 4-5 makes automation way easier: I would not need to work with the lowest common denominator to support everything.  

All in all, hope David has a few surprises left up his sleeve.  :)

My 5c …. 

Tom K.

Cloudera Manager Installation Issues

When getting the following errors below on the Cloudera Manager Installation on RHEL 7.2+, try both a Date and Time Configuration and stopping then starting the agent service with these commands:

systemctl stop cloudera-scm-agent
systemctl start cloudera-scm-agent

If you get the following messages.  In the absence of a DNS server, also check and configure the /etc/hosts for the hostname and IP similar to this:        host.domain.xyz host

Or if you are using DD-WRT use records such as this:


Exact error message received:

Installation failed. Failed to receive heartbeat from agent.

  • Ensure that the host's hostname is configured properly.
  • Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules).
  • Ensure that ports 9000 and 9001 are not in use on the host being added.
  • Check agent logs in /var/log/cloudera-scm-agent/ on the host being added. (Some of the logs can be found in the installation details).
  • If Use TLS Encryption for Agents is enabled in Cloudera Manager (Administration -> Settings -> Security), ensure that/etc/cloudera-scm-agent/config.ini has use_tls=1 on the host being added. Restart the corresponding agent and click the Retry link here.

>>[14/Sep/2016 15:24:13 +0000] 16330 Dummy-14 agent ERROR Failed to kill process with pid 16358
OSError: [Errno 3] No such process
>>[14/Sep/2016 15:24:13 +0000] 16330 Dummy-14 agent ERROR Shutdown callback failed.
>>OSError: [Errno 9] Bad file descriptor
>>[14/Sep/2016 15:24:13 +0000] 16330 Dummy-14 agent ERROR Shutdown callback failed.
KeyError: 15

The proper response given is:

# nslookup
Address:      name = mds-host1.      name = mds-host1.mds.xyz.


Once installation progressed, we got these messages:

Transparent Huge Page Compaction is enabled and can cause significant performance problems. Run "echo never > /sys/kernel/mm/transparent_hugepage/defrag" to disable this, then add the same command to an init script such as /etc/rc.local so it will be set upon system reboot. The following hosts are affected: 
mds-host05; mds-host[01-04]

Cloudera recommends setting /proc/sys/vm/swappiness to a maximum of 10. Current setting is 30. Use the sysctl command to change this setting at run time and edit /etc/sysctl.conf for this setting to be saved after a reboot. You can continue with installation, but Cloudera Manager might report that your hosts are unhealthy because they are swapping. The following hosts are affected: 
mds-host05; mds-host[01-04]

The following failures were observed in checking hostnames… 
Host mds-host01 expected to have name mds-host01 but resolved (InetAddress.getLocalHost().getHostName()) itself to mds-host01.mds.xyz.


Resolve these accordingly to above instructions and continue.  To set the swappiness run the following:

sysctl -w vm.swappiness=10

And continue with the installation.  If you get this error:

/usr/lib64/cmf/service/zookeeper/zkserver.sh: line 41: /var/lib/zookeeper/myid: Permission denied
Supervisor returned FATAL. Please check the role log file, stderr, or stdout.
Completed only 0/1 steps. First failure: Command (85) has failed
Failed to start role.
Completed only 1/2 steps. First failure: Failed to execute command Start on service ZooKeeper

Simply change permissions like this:

101947599 d———.  2 root root    6 Sep 14 23:52 /var/lib/zookeeper

chmod 755 /var/lib/zookeeper

101947599 drwxr-xr-x. 2 root root 6 Sep 14 23:52 /var/lib/zookeeper

And continue the install.  Possibly /usr/lib64/cmf/service/zookeeper/zkserver.sh might not be setting permissions correctly.  Also set the folder to zookeeper.zookeeper as well:

chmod 755 /var/lib/zookeeper; chown zookeeper.zookeeper /var/lib/zookeeper; ls -altrid /var/lib/zookeeper

And continue.  Also set the ACL's on the folder:

# setfacl -m "u:zookeeper:rwx,g:zookeeper:rwx" /var/lib/zookeeper/
# getfacl zookeeper
# file: zookeeper
# owner: zookeeper
# group: zookeeper


Yet still did not work.  Digging deeper we see:

[root@mds-host01 zookeeper]# pwd
[root@mds-host01 zookeeper]# tail -f zookeeper-cmf-zookeeper-SERVER-mds-host01.log -n 10
2016-09-15 01:48:59,395 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:user.home=/var/lib/zookeeper
2016-09-15 01:48:59,396 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:user.dir=/run/cloudera-scm-agent/process/31-zookeeper-server
2016-09-15 01:48:59,396 ERROR org.apache.zookeeper.server.ZooKeeperServerMain: Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Missing data directory /var/lib/zookeeper/version-2, automatic data directory creation is disabled (zookeeper.datadir.autocreate is false). Please create this directory manually.
        at org.apache.zookeeper.server.persistence.FileTxnSnapLog.<init>(FileTxnSnapLog.java:102)
        at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:109)
        at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:91)
        at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:121)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:79)


So let's create it manually and retry the installation.  So command was:

chmod 755 /var/lib/zookeeper; chown zookeeper.zookeeper /var/lib/zookeeper; ls -altrid /var/lib/zookeeper; setfacl -m "u:zookeeper:rwx,g:zookeeper:rwx" /var/lib/zookeeper/; getfacl /var/lib/zookeeper/; mkdir /var/lib/zookeeper/version-2; chown zookeeper.zookeeper /var/lib/zookeeper/version-2;

This time above fixed the issue.  Retry and continue with installation. 


Date and Time Configuration on RHEL 7, CentOS 7, Scientific Linux 7 or Fedora

yum install ntp -y; timedatectl set-ntp yes; timedatectl set-timezone America/Toronto

Use timedatectl list-timezones to get your timezone.


Wrap lines in journalctl output

journalctl -b –no-pager | less

Easy peasy!

Cheers, TK

Creating a Minecraft PE Server

Basically it boils down to this running on Ubuntu:

minecraft@minecraftpe01:~$ wget -q -O – https://raw.githubusercontent.com/PocketMine/php-build-scripts/master/installer.sh | bash -s -

If the above doesn't work, try this:

wget -q -O compile.sh https://raw.githubusercontent.com/PocketMine/php-build-scripts/master/compile.sh

then install any dependencies it complains about.

To ensure you are compatible with the latest protocols, ensure your .phar file is updated. Direct link is here:

wget -O PocketMine-MP_1.6dev-27_ef8227a0_API-2.0.0.phar https://bintray.com/pocketmine/PocketMine/download_file?file_path=PocketMine-MP_1.6dev-27_ef8227a0_API-2.0.0.phar

minecraft@minecraftpe01:~$ cp -ip PocketMine-MP_1.6dev-27_ef8227a0_API-2.0.0.phar PocketMine-MP.phar
cp: overwrite 'PocketMine-MP.phar'? y
minecraft@minecraftpe01:~$ ls -altri *phar
694443 -rw-rw-r– 1 minecraft minecraft 3966174 Jun 15 18:50 PocketMine-MP.phar
701448 -rw-rw-r– 1 minecraft minecraft 3966174 Jun 15 18:50 PocketMine-MP_1.6dev-27_ef8227a0_API-2.0.0.phar
minecraft@minecraftpe01:~$ ls -altri *phar*
700731 -rw-rw-r– 1 minecraft minecraft     287 May  2 00:34 PocketMine-MP.phar.sig
700960 -rw-rw-r– 1 minecraft minecraft 3961710 May  2 00:38 PocketMine-MP.phar-old
694443 -rw-rw-r– 1 minecraft minecraft 3966174 Jun 15 18:50 PocketMine-MP.phar
701448 -rw-rw-r– 1 minecraft minecraft 3966174 Jun 15 18:50 PocketMine-MP_1.6dev-27_ef8227a0_API-2.0.0.phar

You can also download the latest .phar file to allow your latest client to work with this server.  The .phar file can be found here: https://ci.itxtech.org/job/Genisys/lastSuccessfulBuild/

 Source documentation.  Further sources available here and here.


Failed to execute operation: Access denied

When running you get this message:

# systemctl enable nfs-server.service
Failed to execute operation: Access denied

Solution is to disable selinux or to allow for proper SELinux rules to allow for the above action:

# vi /etc/sysconfig/selinux


# setenforce 0
# getenforce

Alternately see the following post for details on how to tailor SELinux rules for similar cases.


Error: A specified parameter was not correct vim.host.diskpartitioninfo.spec

SSH to esxi (enable ssh via console if not enabled) then:

esxcfg-scsidevs -l


partedUtil delete <DISK> <PARTITION>


partedUtil delete (Gave invalid partition number) :

partedUtil delete /vmfs/devices/disks/naa.600508b1001c7ef7f736cd3dbeb5915d vml.0200010000600508b1001c7ef7f736cd3dbeb5915d4c4f47494341


OpenNebula VM Password Prompt

When instantiating VM's through OpenNebula, when logging into those VM's, the VM's may ask you for a password.  The solution here is to edit the template in the GUI and add a context to it including the public SSH key.  Click on the template then Update from the top right hand side buttom row then enter the key in:


SSH Password Prompt - Missing SSH Contextualization

If the above field is empty, ensure it has the proper passless SSH key.  Further reading on contextualization.

The other reason for this issue is that you are trying to get into your machine as opennebula.  Try as root instead:

[oneadmin@opennebula01 .ssh]$ ssh
oneadmin@'s password:

[oneadmin@opennebula01 .ssh]$ ssh root@
[root@localhost ~]#



How to tail two files simultaneously

[oneadmin@opennebula01 one]$ tail -f oned.log sched.log

==> sched.log <==

Tue Apr 12 21:09:36 2016 [Z0][SCHED][I]: Getting scheduled actions information. Total time: 0.00560014s

==> oned.log <==
Tue Apr 12 21:09:36 2016 [Z0][ReM][D]: Req:8160 UID:0 VirtualMachinePoolInfo invoked , -2, -1, -1, -1
Tue Apr 12 21:09:36 2016 [Z0][ReM][D]: Req:8160 UID:0 VirtualMachinePoolInfo result SUCCESS, "<VM_POOL><VM><ID>40<…"

==> sched.log <==
Tue Apr 12 21:09:36 2016 [Z0][SCHED][I]: Getting VM and Host information. Total time: 0.00360117s

==> oned.log <==
Tue Apr 12 21:09:52 2016 [Z0][InM][D]: Host mdskvm-p01 (2) successfully monitored.
Tue Apr 12 21:09:52 2016 [Z0][VMM][D]: VM 40 successfully monitored: STATE=a USEDCPU=0.0 USEDMEMORY=524288 NETRX=344326 NETTX=1360



VNC Failed to connect to server (code: 1006)

When receiving the following on OpenStack or OpenNebula:

VNC Failed to connect to server (code: 1006)

try the below steps to resolve the issue on either hypervisor manager:

Read the rest of this entry »

  Copyright © 2003 - 2013 Tom Kacperski (microdevsys.com). All rights reserved.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License