Getting this adding hosts to XCP-ng?

write EPROTO 140708195170112:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Read the rest of this entry »

Getting this?

bash: /usr/local/bin/node: No such file or directory

Fix with:

[root@xoa-org01 bin]# node -v
-bash: /usr/local/bin/node: No such file or directory
[root@xoa-org01 bin]# . ~/.bash_profile
[root@xoa-org01 bin]# node
> .exit
[root@xoa-org01 bin]# node -v
v8.16.2
[root@xoa-org01 bin]#

Thx,
TK

If you're getting this error:  

Oct 14 01:43:26 xcpng01 xapi: [error|xcpng01.nix.mds.xyz|3179 UNIX /var/lib/xcp/xapi|dispatch:SR.get_uuid D:3c84eaa48cb2|backtrace] SR.get_uuid D:855756122ab7 failed with exception Db_exn.DBCache_NotFound("missing row", "SR", "OpaqueRef:NULL")

Read the rest of this entry »

To create a plugin entry for XCP-ng 8.0.1, follow the following procedure ( We will use the GlusterFS plugin for this example. ):

Create the GlusterFS repo:

[19:40 xcpng02 sm]# cat /etc/yum.repos.d/gluster63.repo
[gluster63]
name=Gluster 6.3
baseurl=http://mirror.centos.org/centos/7/storage/x86_64/gluster-6/
gpgcheck=0
enabled=1
[19:41 xcpng02 sm]#

Read the rest of this entry »

HTTPS pages are slow to load.  If you have this issue, try the following:

Visit Services -> Cryptsvc (Cryptographic Services) -> Properties -> Logon Tab -> Select Local System Account

Cheers,
TK

Create local storage under XCP-ng as follows.  

Read the rest of this entry »

Creating PV's, VG's and LV's under XCP-ng.
 

[06:47 xcpng02 ~]#
[06:47 xcpng02 ~]# pvremove /dev/sdb
  pvremove /dev/sdb: Command not permitted while global/metadata_read_only is set.
[06:47 xcpng02 ~]# pvremove /dev/sdb –config global{metadata_read_only=0}
  Labels on physical volume "/dev/sdb" successfully wiped.
[06:47 xcpng02 ~]# pvcreate /dev/sdb –config global{metadata_read_only=0}
  Physical volume "/dev/sdb" successfully created.
[06:47 xcpng02 ~]# vgcreate vmsrvg /dev/sdb
  vgcreate vmsrvg /dev/sdb: Command not permitted while global/metadata_read_only is set.
[06:48 xcpng02 ~]# vgcreate vmsrvg /dev/sdb –config global{metadata_read_only=0}
  Volume group "vmsrvg" successfully created
[06:48 xcpng02 ~]# lvcreate -L 128G -n vmsrlv vmsrvg –config global{metadata_read_only=0}
  Logical volume "vmsrlv" created.
[06:52 xcpng02 ~]#

Rest of the commands don't need special handling:

   40  pvremove /dev/sdb
   41  pvremove /dev/sdb –config global{metadata_read_only=0}
   42  pvcreate /dev/sdb –config global{metadata_read_only=0}
   43  vgcreate vmsrvg /dev/sdb
   44  vgcreate vmsrvg /dev/sdb –config global{metadata_read_only=0}
   45  lvcreate -L 128G -n vmsrlv vmsrvg –config global{metadata_read_only=0}
   46  mkfs.ext4
   47  yum install xfs
   48  yum search xfs
   49  yum install xfsprogs.x86_64 xfsprogs-devel.x86_64
   50  mkfs.xfs
   51  mkfs.xfs /dev/mapper/vmsrvg-vmsrlv
 

Cheers,
TK

Getting this?

[root@mdskvm-p01 ~]# gluster volume delete mdsgv01
Deleting volume will erase all information about the volume. Do you want to continue? (y/n) y
volume delete: mdsgv01: failed: Some of the peers are down
[root@mdskvm-p01 ~]#

[root@mdskvm-p01 ~]# gluster volume remove-brick mdsgv01  mdskvm-p01.nix.mds.xyz:/mnt/p01-d01/glusterv01 force
Remove-brick force will not migrate files from the removed bricks, so they will no longer be available on the volume.
Do you want to continue? (y/n) y
volume remove-brick commit force: failed: Deleting all the bricks of the volume is not allowed
[root@mdskvm-p01 ~]#

[root@mdskvm-p01 ~]# gluster volume info

Volume Name: mdsgv01
Type: Distribute
Volume ID: f5b57076-dbd4-4d77-ae13-c1f3ee3adbe0
Status: Stopped
Snapshot Count: 0
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: mdskvm-p01.nix.mds.xyz:/mnt/p01-d01/glusterv01
Options Reconfigured:
diagnostics.client-log-level: DEBUG
diagnostics.brick-sys-log-level: INFO
diagnostics.brick-log-level: DEBUG
performance.readdir-ahead: on
server.allow-insecure: on
nfs.trusted-sync: on
performance.cache-size: 1GB
performance.io-thread-count: 16
performance.write-behind-window-size: 8MB
client.event-threads: 8
server.event-threads: 8
cluster.quorum-type: none
cluster.server-quorum-type: none
storage.owner-uid: 36
features.shard: on
features.shard-block-size: 512MB
performance.low-prio-threads: 32
cluster.data-self-heal-algorithm: full
storage.owner-gid: 36
[root@mdskvm-p01 ~]#

Solve it by checking the peers and removing them first:

[root@mdskvm-p01 ~]# gluster peer status
Number of Peers: 2

Hostname: opennebula01
Uuid: 94d11cc5-2a8b-4583-97f3-5890cbd7d624
State: Peer Rejected (Disconnected)

Hostname: mdskvm-p02
Uuid: ad7d956a-a121-422e-8c5c-56765bdf6a62
State: Peer in Cluster (Connected)
[root@mdskvm-p01 ~]#

Removing the peers:

[root@mdskvm-p01 ~]# gluster peer detach opennebula01
All clients mounted through the peer which is getting detached need to be remounted using one of the other active peers in the trusted storage pool to ensure client gets notification on any changes done on the gluster configuration and if the same has been done do you want to proceed? (y/n) y
peer detach: success
[root@mdskvm-p01 ~]#

[root@mdskvm-p01 ~]# gluster peer detach mdskvm-p02
All clients mounted through the peer which is getting detached need to be remounted using one of the other active peers in the trusted storage pool to ensure client gets notification on any changes done on the gluster configuration and if the same has been done do you want to proceed? (y/n) y
peer detach: success
[root@mdskvm-p01 ~]#

Now delete the last volume:

[root@mdskvm-p01 ~]# gluster volume remove-brick mdsgv01 replica 1 mdskvm-p02.nix.mds.xyz:/mnt/p02-d01/glusterv02 force
Remove-brick force will not migrate files from the removed bricks, so they will no longer be available on the volume.
Do you want to continue? (y/n) y
volume remove-brick commit force: success
[root@mdskvm-p01 ~]#

[root@mdskvm-p01 ~]# gluster volume delete mdsgv01
Deleting volume will erase all information about the volume. Do you want to continue? (y/n) y
volume delete: mdsgv01: success
[root@mdskvm-p01 ~]#

Now set everything up clean!

Cheers,
TK

NOTE: Experimental steps.  Use at your own discretion.  Also note that ultimately, these steps didn't resolve the subject errors though they were successful in updating the VDSM certs.  I ended up blowing the cluster away (as I didn't have much on it anyway).

Getting this with oVirt?

VDSM mdskvm-p01.nix.mds.xyz command Get Host Capabilities failed: General SSLEngine problem

You may need to regenerate the certificates:

[root@ovirt01 ~]# openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem -enddate -noout ; date
notAfter=Apr  8 17:12:07 2019 GMT
Wed Sep 25 18:22:02 EDT 2019
[root@ovirt01 ~]#

Process is (use mypass as the pass):

# cp -a /etc/pki/ovirt-engine "/etc/pki/ovirt-engine.$(date "+%Y%m%d")"
# SUBJECT="$(openssl x509 -subject -noout -in /etc/pki/ovirt-engine/certs/apache.cer | sed 's/subject= //')"
# /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh --name=apache --password="@PASSWORD@" --subject="${SUBJECT}"
# openssl pkcs12 -passin "pass:@PASSWORD@" -nokeys -in /etc/pki/ovirt-engine/keys/apache.p12 > /etc/pki/ovirt-engine/certs/apache.cer
# openssl pkcs12 -passin "pass:@PASSWORD@" -nocerts -nodes -in /etc/pki/ovirt-engine/keys/apache.p12 > /etc/pki/ovirt-engine/keys/apache.key.nopass
# chmod 0600 /etc/pki/ovirt-engine/keys/apache.key.nopass

Certificate renewals:

https://access.redhat.com/solutions/3028811

Steps to renew other certs:

# cd /usr/share/ovirt-engine/bin/
# SUB="ovirt01.nix.mds.xyz"; SUBA="ovirt01.nix.mds.xyz";

# ./pki-create-ca.sh –subject=/C=US/O=$SUB/CN=$SUBA –keystore-password=mypass

# ./pki-enroll-pkcs12.sh –name=engine –password=mypass –subject=/C=US/O=$SUB/CN=$SUBA
# ./pki-enroll-pkcs12.sh –name=jboss –password=mypass –subject=/C=US/O=$SUB/CN=$SUBA
# ./pki-enroll-pkcs12.sh –name=apache –password=mypass –subject=/C=US/O=$SUB/CN=$SUBA
# ./pki-enroll-pkcs12.sh –name=websocket-proxy –password=mypass –subject=/C=US/O=$SUB/CN=$SUBA

# ./pki-enroll-request.sh –name=engine –subject=/C=US/O=$SUB/CN=$SUBA –days=1800
# ./pki-enroll-request.sh –name=jboss –subject=/C=US/O=$SUB/CN=$SUBA –days=1800
# ./pki-enroll-request.sh –name=apache –subject=/C=US/O=$SUB/CN=$SUBA –days=1800
# ./pki-enroll-request.sh –name=websocket-proxy –subject=/C=US/O=$SUB/CN=$SUBA –days=1800

# openssl pkcs12 -in /etc/pki/ovirt-engine/keys/apache.p12 -nocerts -nodes > /etc/pki/ovirt-engine/keys/apache.key.nopass
# openssl pkcs12 -in /etc/pki/ovirt-engine/keys/websocket-proxy.p12 -nocerts -nodes > /etc/pki/ovirt-engine/keys/websocket-proxy.key.nopass
  enter import password  —> mypass

# ls -al /etc/pki/ovirt-engine/keys/*.key.nopass
# ls -al /etc/pki/ovirt-engine/keys/*.p12
# chown ovirt.root /etc/pki/ovirt-engine/keys/engine.p12
# chown ovirt.root /etc/pki/ovirt-engine/keys/jboss.p12
# chown ovirt.ovirt /etc/pki/ovirt-engine/keys/websocket-proxy.key.nopass
# chmod 600 /etc/pki/ovirt-engine/keys/*
# ls -al /etc/pki/ovirt-engine/keys/*.key.nopass
# ls -al /etc/pki/ovirt-engine/keys/*.p12

A handy script for this is:

[root@ovirt01 bin]# cat /root/ovirt-renew.sh
#!/bin/bash

cd /usr/share/ovirt-engine/bin/
SUB="ovirt01.nix.mds.xyz"; SUBA="ovirt01.nix.mds.xyz";

cp -a /etc/pki/ovirt-engine "/etc/pki/ovirt-engine.$(date "+%Y%m%d")"
SUBJECT="$(openssl x509 -subject -noout -in /etc/pki/ovirt-engine/certs/apache.cer | sed 's/subject= //')"
/usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh –name=apache –password="@PASSWORD@" –subject="${SUBJECT}"
openssl pkcs12 -passin "pass:@PASSWORD@" -nokeys -in /etc/pki/ovirt-engine/keys/apache.p12 > /etc/pki/ovirt-engine/certs/apache.cer
openssl pkcs12 -passin "pass:@PASSWORD@" -nocerts -nodes -in /etc/pki/ovirt-engine/keys/apache.p12 > /etc/pki/ovirt-engine/keys/apache.key.nopass
chmod 0600 /etc/pki/ovirt-engine/keys/apache.key.nopass

sleep 5;

./pki-create-ca.sh –subject=/C=US/O=$SUB/CN=$SUBA –keystore-password=mypass

./pki-enroll-pkcs12.sh –name=engine –password=mypass –subject=/C=US/O=$SUB/CN=$SUBA
./pki-enroll-pkcs12.sh –name=jboss –password=mypass –subject=/C=US/O=$SUB/CN=$SUBA
./pki-enroll-pkcs12.sh –name=apache –password=mypass –subject=/C=US/O=$SUB/CN=$SUBA
./pki-enroll-pkcs12.sh –name=websocket-proxy –password=mypass –subject=/C=US/O=$SUB/CN=$SUBA

./pki-enroll-request.sh –name=engine –subject=/C=US/O=$SUB/CN=$SUBA –days=1800
./pki-enroll-request.sh –name=jboss –subject=/C=US/O=$SUB/CN=$SUBA –days=1800
./pki-enroll-request.sh –name=apache –subject=/C=US/O=$SUB/CN=$SUBA –days=1800
./pki-enroll-request.sh –name=websocket-proxy –subject=/C=US/O=$SUB/CN=$SUBA –days=1800

openssl pkcs12 -in /etc/pki/ovirt-engine/keys/apache.p12 -nocerts -nodes > /etc/pki/ovirt-engine/keys/apache.key.nopass
openssl pkcs12 -in /etc/pki/ovirt-engine/keys/websocket-proxy.p12 -nocerts -nodes > /etc/pki/ovirt-engine/keys/websocket-proxy.key.nopass

ls -al /etc/pki/ovirt-engine/keys/*.key.nopass
ls -al /etc/pki/ovirt-engine/keys/*.p12
chown ovirt.root /etc/pki/ovirt-engine/keys/engine.p12
chown ovirt.root /etc/pki/ovirt-engine/keys/jboss.p12
chown ovirt.ovirt /etc/pki/ovirt-engine/keys/websocket-proxy.key.nopass
chmod 600 /etc/pki/ovirt-engine/keys/*
ls -al /etc/pki/ovirt-engine/keys/*.key.nopass
ls -al /etc/pki/ovirt-engine/keys/*.p12

[root@ovirt01 bin]#

Next run engine-setup:

[root@ovirt01 ovirt-engine]# engine-setup
          –== PKI CONFIGURATION ==–

          One or more of the certificates should be renewed, because they expire soon, or include an invalid expiry date, or do not include the subjectAltName extension, which can cause them to be rejected by recent browsers and up to date hosts.
          See https://www.ovirt.org/develop/release-management/features/infra/pki-renew/ for more details.
          Renew certificates? (Yes, No) [No]: Yes

At some point above it will ask to renew certificates.  Do so.  To venew the VDSM certs, use the following command:

[root@ovirt01 ovirt-engine]#
[root@ovirt01 ovirt-engine]# /usr/libexec/vdsm/vdsm-gencerts.sh /etc/pki/vdsm/certs/cacert.pem /etc/pki/vdsm/keys/vdsmkey.pem  /etc/pki/vdsm/certs/vdsmcert.pem
[root@ovirt01 ovirt-engine]# ls -altri /usr/libexec/vdsm/vdsm-gencerts.sh /etc/pki/vdsm/certs/cacert.pem /etc/pki/vdsm/keys/vdsmkey.pem  /etc/pki/vdsm/certs/vdsmcert.pem
137408334 -rwxr-xr-x. 1 root root 2362 Jul  9 05:36 /usr/libexec/vdsm/vdsm-gencerts.sh
 68603465 -rw——-. 1 vdsm kvm  5823 Sep 28 14:49 /etc/pki/vdsm/keys/vdsmkey.pem
   508026 -rw——-. 1 vdsm kvm  1127 Sep 28 14:49 /etc/pki/vdsm/certs/cacert.pem
   492099 -rw——-. 1 vdsm kvm  1237 Sep 28 14:49 /etc/pki/vdsm/certs/vdsmcert.pem
[root@ovirt01 ovirt-engine]# openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem -noout   -dates
notBefore=Sep 27 18:49:14 2019 GMT
notAfter=Sep 27 18:49:15 2020 GMT
[root@ovirt01 ovirt-engine]# vi /usr/libexec/vdsm/vdsm-gencerts.sh
[root@ovirt01 ovirt-engine]#
[root@ovirt01 ovirt-engine]#
 

Do the same on the host:

[root@mdskvm-p01 vdsm]# /usr/libexec/vdsm/vdsm-gencerts.sh /etc/pki/vdsm/certs/cacert.pem /etc/pki/vdsm/keys/vdsmkey.pem  /etc/pki/vdsm/certs/vdsmcert.pem
[root@mdskvm-p01 vdsm]#
[root@mdskvm-p01 vdsm]# ls -altri /usr/libexec/vdsm/vdsm-gencerts.sh /etc/pki/vdsm/certs/cacert.pem /etc/pki/vdsm/keys/vdsmkey.pem  /etc/pki/vdsm/certs/vdsmcert.pem
 67445862 -rwxr-xr-x. 1 root root 2362 Jul  9 05:36 /usr/libexec/vdsm/vdsm-gencerts.sh
    45255 -rw——-. 1 vdsm kvm  5816 Sep 28 17:48 /etc/pki/vdsm/keys/vdsmkey.pem
203185926 -rw——-. 1 vdsm kvm  1127 Sep 28 17:48 /etc/pki/vdsm/certs/cacert.pem
203185790 -rw——-. 1 vdsm kvm  1241 Sep 28 17:48 /etc/pki/vdsm/certs/vdsmcert.pem
[root@mdskvm-p01 vdsm]#

Additional notes from the oVirt community:

Have this page but it's for version 3.

https://access.redhat.com/solutions/2409751

I wasn't aware of this page. It's quite old, but mostly correct.
However, if you do not mind host downtime, it's much easier to re-enroll
certificates for all hosts, instead of the manual steps mentioned there
(that are quite old, perhaps not up-to-date).

Thinking the process didn't change much but wanted to ask if there's
anything more recent floating around.

I am not aware of anything specifically doing what you want.

Related pages you might want to check:

1. Section "Replacing SHA-1 Certificates with SHA-256 Certificates" of:

https://www.ovirt.org/documentation/upgrade-guide/chap-Post-Upgrade_Tasks.html

2. Only now I noticed that it does not mention the option --san for
setting SubjectAltName. It does appear here:

https://www.ovirt.org/documentation/admin-guide/chap-Utilities.html

See also:

https://www.ovirt.org/develop/release-management/features/infra/pki-renew.html

So I guess (didn't try recently) that if you follow the existing procedures
and generate pki without --san, a later engine-setup will prompt you to renew.

Best regards,

— Didi

If all of the above fails, force remove a host and add it back in:

https://raw.github.com/dougsland/misc-rhev/master/engine_force_remove_Host.py

Cheers,
TK

Getting this?

/var/log/glusterfs/bricks/mnt-p01-d01-glusterv01.log
[2019-09-25 10:53:37.847426] I [MSGID: 100030] [glusterfsd.c:2847:main] 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 6.5 (args: /usr/sbin/glusterfsd -s mdskvm-p01.nix.mds.xyz –volfile-id mdsgv01.mdskvm-p01.nix.mds.xyz.mnt-p01-d01-glusterv01 -p /var/run/gluster/vols/mdsgv01/mdskvm-p01.nix.mds.xyz-mnt-p01-d01-glusterv01.pid -S /var/run/gluster/defbdb699838d53b.socket –brick-name /mnt/p01-d01/glusterv01 -l /var/log/glusterfs/bricks/mnt-p01-d01-glusterv01.log –xlator-option *-posix.glusterd-uuid=f7336db6-22b4-497d-8c2f-04c833a28546 –process-name brick –brick-port 49155 –xlator-option mdsgv01-server.listen-port=49155)
[2019-09-25 10:53:37.848508] I [glusterfsd.c:2556:daemonize] 0-glusterfs: Pid of current running process is 23133
[2019-09-25 10:53:37.858381] I [socket.c:902:__socket_server_bind] 0-socket.glusterfsd: closing (AF_UNIX) reuse check socket 9
[2019-09-25 10:53:37.865940] I [MSGID: 101190] [event-epoll.c:680:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2019-09-25 10:53:37.866054] I [glusterfsd-mgmt.c:2443:mgmt_rpc_notify] 0-glusterfsd-mgmt: disconnected from remote-host: mdskvm-p01.nix.mds.xyz
[2019-09-25 10:53:37.866043] I [MSGID: 101190] [event-epoll.c:680:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2019-09-25 10:53:37.866083] I [glusterfsd-mgmt.c:2463:mgmt_rpc_notify] 0-glusterfsd-mgmt: Exhausted all volfile servers
[2019-09-25 10:53:37.866454] W [glusterfsd.c:1570:cleanup_and_exit] (–>/lib64/libgfrpc.so.0(+0xf1d3) [0x7f9680ee91d3] –>/usr/sbin/glusterfsd(+0x12fef) [0x55ca25710fef] –>/usr/sbin/glusterfsd(cleanup_and_exit+0x6b) [0x55ca2570901b] ) 0-: received signum (1), shutting down
[2019-09-25 10:53:37.872399] I [socket.c:3754:socket_submit_outgoing_msg] 0-glusterfs: not connected (priv->connected = 0)
[2019-09-25 10:53:37.872445] W [rpc-clnt.c:1704:rpc_clnt_submit] 0-glusterfs: failed to submit rpc-request (unique: 0, XID: 0x2 Program: Gluster Portmap, ProgVers: 1, Proc: 5) to rpc-transport (glusterfs)
[2019-09-25 10:53:37.872534] W [glusterfsd.c:1570:cleanup_and_exit] (–>/lib64/libgfrpc.so.0(+0xf1d3) [0x7f9680ee91d3] –>/usr/sbin/glusterfsd(+0x12fef) [0x55ca25710fef] –>/usr/sbin/glusterfsd(cleanup_and_exit+0x6b) [0x55ca2570901b] ) 0-: received signum (1), shutting down

Read the rest of this entry »