Header Shadow Image

FreeIPA Quick Setup Guide w/ Replication HA, AD DC Trust, Sudo, Ganesha NFS

In this post, we are setting up an IPA server on a separate domain than the one we had configured earlier ( nix.mds.xyz ) .   We do so because IPA comes not only with Authentication and DNS but also with a built in KDC to which we will be connnecting various pieces of software that will make changes to our KDC.  For this reason we prefer to separate our KDC on a secondary domain while allowing the same AD users to authenticate via both IPA servers.  

Add Windows Delegation in the Windows DNS ( MWS = Middle Ware Software ): 

    mws.mds.xyz -> idmipa03/04 /

Configure firewalld as follows or via commands:    

    for KEY in $(echo "80/tcp 443/tcp 389/tcp 636/tcp 88/tcp 464/tcp 53/tcp 88/udp 464/udp 53/udp 123/udp 445/udp 445/tcp 138/tcp 138/udp 139/udp 139/tcp 389/udp"); do firewall-cmd –zone=dmz –permanent –add-port=$KEY; done

OR add the following by manually editing the zone file:

# cat /etc/firewalld/zones/public.xml
<?xml version="1.0" encoding="utf-8"?>
  <description>For use in public areas. You do not trust the other computers on networks to not harm your computer. Only selected incoming connections are accepted.</description>
  <service name="ntp"/>
  <service name="dhcpv6-client"/>
  <service name="ssh"/>
  <port protocol="tcp" port="53"/>
  <port protocol="tcp" port="443"/>
  <port protocol="tcp" port="80"/>
  <port protocol="tcp" port="464"/>
  <port protocol="tcp" port="138"/>
  <port protocol="udp" port="88"/>
  <port protocol="udp" port="464"/>
  <port protocol="tcp" port="88"/>
  <port protocol="tcp" port="135"/>
  <port protocol="udp" port="123"/>
  <port protocol="tcp" port="139"/>
  <port protocol="tcp" port="389"/>
  <port protocol="tcp" port="445"/>
  <port protocol="udp" port="389"/>
  <port protocol="tcp" port="1024-1300"/>
  <port protocol="udp" port="445"/>
  <port protocol="udp" port="139"/>
  <port protocol="udp" port="138"/>
  <port protocol="udp" port="53"/>
  <port protocol="udp" port="67"/>
  <port protocol="udp" port="68"/>
  <port protocol="udp" port="54915"/>
  <port protocol="tcp" port="636"/>


firewall-cmd –zone=public –list-all

Install Free IPA:

yum install -y "*ipa-server" "*ipa-server-trust-ad" bind bind-dyndb-ldap ipa-server-dns ipa-adtrust-install ipa-server-dns
ipa-server-install –setup-dns –forwarder= -p "SECRET" -a "SECRET" -r MWS.MDS.XYZ -n mws.mds.xyz –hostname idmipa03.mws.mds.xyz

Disable DNSSEC validation:

# cat /etc/named.conf|grep -Ei dnssec
        # dnssec-enable yes;
        # dnssec-validation yes;
        dnssec-enable no;
        dnssec-validation no;

AD DC Trust

Let's get the trust going:

ipa-adtrust-install –netbios-name=MWS -a "SECRET"

ipa trust-add –type=ad mds.xyz –admin Administrator –password –two-way=true

NOTE: ipa trust-add may use the tag ipa-ad-trust-posix instead of ipa-ad-trust . This means the UID's and GID's will be taken from the UNIX Attributes you have defined in AD.  If the trust is created with the former and you don't want that, recreate the trust using the latter option.  Also see about modifying these LDAP settings here.  If you need to disable POSIX / UNIX Attributes that are defined in windows and allow SSSD to assign unique UID's based on the SID in Windows, then use the following line:

( Optional ) ipa -vv trust-add –type=ad mds.xyz –admin Administrator –password –two-way=true –range-type=ipa-ad-trust 

If you get:

ipa: ERROR: Cannot find specified domain or server name

add and search mds.xyz to the /etc/resolv.conf and restart IPA:

systemctl restart ipa
ipactl restart

[root@idmipa03 network-scripts]# ipa -vv trust-add –type=ad mds.xyz –admin Administrator –password –two-way=true
Added Active Directory trust for realm "mds.xyz"
  Realm name: mds.xyz
  Domain NetBIOS name: MDS
  Domain Security Identifier: S-1-5-21-1803828911-4163023034-2461700517
  Trust direction: Two-way trust
  Trust type: Active Directory domain
  Trust status: Established and verified
[root@idmipa03 network-scripts]#

Check the AD server to see the trust created:

Auto Create AD DC Trust via Free IPA commands.

Create the replica:

03: ipa-replica-prepare idmipa04.mws.mds.xyz –ip-address -p "PASSWORD"

03: scp /var/lib/ipa/replica-info-idmipa04.mws.mds.xyz.gpg root@idmipa04.mws.mds.xyz:/var/lib/ipa/

04: ipa-replica-install –setup-ca –setup-dns –forwarder= /var/lib/ipa/replica-info-idmipa04.mws.mds.xyz.gpg

If you get the below error, just follow the below commands to continue:

[root@idmipa03 network-scripts]# ipa-replica-prepare idmipa04.mws.mds.xyz –ip-address -p "SECRET"

Replica creation using 'ipa-replica-prepare' to generate replica file
is supported only in 0-level IPA domain.

The current IPA domain level is 1 and thus the replica must
be created by promoting an existing IPA client.

To set up a replica use the following procedure:
    1.) set up a client on the host using 'ipa-client-install'
    2.) promote the client to replica running 'ipa-replica-install'
        *without* replica file specified

'ipa-replica-prepare' is allowed only in domain level 0
The ipa-replica-prepare command failed.
[root@idmipa03 network-scripts]#

Commands to run:


ipa-replica-install –setup-ca –setup-dns –forwarder= /var/lib/ipa/replica-info-idmipa04.mws.mds.xyz.gpg

Ensure your hosts file has the following as well in case you get ERROR Unable to resolve host name, check /etc/hosts or DNS name resolution:

[root@idmipa04 network-scripts]# cat /etc/hosts   idmipa04.mws.mds.xyz idmipa04
[root@idmipa04 network-scripts]#

In case you get this:

ipaserver.install.installutils: ERROR    Unable to resolve the IP address to a host name, check /etc/hosts and DNS name resolution
ipaserver.install.server.replicainstall: ERROR    Reverse DNS resolution of address (idmipa03.mws.mds.xyz) failed. Clients may not function properly. Please check your DNS setup. (Note that this check queries IPA DNS directly and ignores /etc/hosts.)

Try adding the first configured IPA servers IP to /etc/resolv.conf like this (highlighted) :

[root@idmipa04 network-scripts]# cat /etc/resolv.conf|grep -Ei
[root@idmipa04 network-scripts]#

In case that does nto work, force add a reverse zone addition.  If you get the below:

IPA Error 3000: InvocationError
DNS zone 0.168.192.in-addr.arpa. already exists in DNS and is handled by server(s): winad02.mds.xyz., winad01.mds.xyz.

Check the Skip overlap check text box.  Your IPA server should now look like this:

Reverse Zone Creation

If you get this message:

ipaserver.install.server.replicainstall: ERROR    Reverse DNS resolution of address (idmipa04.mws.mds.xyz) failed. Clients may not function properly. Please check your DNS setup. (Note that this check queries IPA DNS directly and ignores /etc/hosts.)

then twist the /etc/resolv.conf a bit like this adding in the IP of the first IPA server (highlighted):

[root@idmipa04 network-scripts]# cat /etc/resolv.conf
; generated by /usr/sbin/dhclient-script
search mds.xyz nix.mds.xyz m01.mds.xyz
[root@idmipa04 network-scripts]#

And verifying with a dig -x should yield:  

;; ANSWER SECTION: 86400 IN    PTR     idmipa03.mws.mds.xyz.

Now ipa-replica-install succeeded but complained about CA only being setup on idmipa03.  So we fall back to the original commands after running ipa-server-install –uninstall  (You may need to readd the PTR records above):

01: ipa-client-install
02: ipa-replica-install –setup-ca –setup-dns –forwarder=

But we run into yet another issue:

2019-02-05T06:48:54Z DEBUG The ipa-replica-install command failed, exception: CalledProcessError: Command '/usr/sbin/ipa-getkeytab -k /etc/dirsrv/ds.keytab -p ldap/idmipa04.mws.mds.xyz@MWS.MDS.XYZ -H ldaps://idmipa03.mws.mds.xyz' returned non-zero exit status 9
2019-02-05T06:48:54Z ERROR Command '/usr/sbin/ipa-getkeytab -k /etc/dirsrv/ds.keytab -p ldap/idmipa04.mws.mds.xyz@MWS.MDS.XYZ -H ldaps://idmipa03.mws.mds.xyz' returned non-zero exit status 9

So we'll start fresh removing the host from the IPA Server -> Topology section / menu when we got this error:

Joining realm failed: Host is already joined.

Once reconfigured successfully, check the replica and master via Chrome HTTPS to ensure things like the Trusts are showing up on the replica.  Seems all good now!


If you created your reverse zone manually, you will want to enable dynamic DNS updates on the reverse zone:

  1. Visit DNS Zones -> 0.168.192.in-addr.arpa.
  2. Browse to Dynamic Update
  3. Select True

FreeIPA Dynamic DNS Configuration

And that wraps up your fully redundant IPA Cluster.  Feel free to add further nodes for added redundancy.  On to configuring our first client.  


Your resolv.conf should look as follows since all resolutions and lookups would go directly against this server, and failing over to the secondary when needed:

# cat /etc/resolv.conf
search mws.mds.xyz mds.xyz


NOTE On Private Groups

Be mindful of the private group creation option:

GID for AD Users
SSSD 1.16.1 Change Log


       auto_private_groups (string)
           If this option is enabled, SSSD will automatically create user private groups based on user's UID number. The GID number is ignored in
           this case.

           For POSIX subdomains, setting the option in the main domain is inherited in the subdomain.

           For ID-mapping subdomains, auto_private_groups is already enabled for the subdomains and setting it to false will not have any effect
           for the subdomain.

           NOTE: Because the GID number and the user private group are inferred from the UID number, it is not supported to have multiple entries
           with the same UID or GID number with this option. In other words, enabling this option enforces uniqueness across the ID space.

           Default: False

This is a new option.  Decide if you want this.  If you don't, private group lookups won't work yielding instead:

(Mon Feb 18 13:56:46 2019) [sssd[be[mws.mds.xyz]]] [sysdb_search_by_name] (0x0400): No such entry
(Mon Feb 18 13:56:46 2019) [sssd[be[mws.mds.xyz]]] [sysdb_delete_group] (0x0400): Error: 2 (No such file or directory)

[root@idmipa03 sssd]# getent group tom@mds.xyz
[root@idmipa03 sssd]# echo $?
[root@idmipa03 sssd]#

Likewise, you'll want to set your home folder to the correct Primary Group as defined in AD on this new version of sssd / IPA:

[root@idmipa03 mds.xyz]# chown tom@mds.xyz:nixadmins@mds.xyz tom
33554560 drwxr-xr-x. 2 tom@mds.xyz nixadmins@mds.xyz 26 Feb 13 03:34 tom

Unlike in older versions of sssd / IPA:

67149952 drwxr-xr-x. 3 tom@mds.xyz tom@mds.xyz 4096 Feb  4  2018 /home/mds.xyz/tom


Mapping AD DC groups to IPA groups:

Issue the following.  This assumes that nixadmins, cdhadmins and domain-users all exist on the AD DC side:

ipa group-add –desc='AD DC UNIX Admins External Map' nixadmins_external –external
ipa group-add-member nixadmins_external –external 'MDS\nixadmins'
ipa group-add –desc='AD DC UNIX Admins' nixadmins
ipa group-add-member nixadmins –groups nixadmins_external


ipa group-add –desc='AD DC CDH Admins External Map' cdhadmins_external –external
ipa group-add-member cdhadmins_external –external 'MDS\cdhadmins'
ipa group-add –desc='AD DC CDH Admins' cdhadmins
ipa group-add-member cdhadmins –groups cdhadmins_external


ipa group-add –desc='AD DC UNIX Admins External Map' 'domain-users_external' –external
ipa group-add-member 'domain-users_external' –external 'MDS\domain users'
ipa group-add –desc='AD DC UNIX Admins' 'domain-users'
ipa group-add-member domain-users –groups domain-users_external


NFS Home Directories Configuration

Now that we have our own NFS servers, we want to configure NFS mounted home directories on hosts directly connected to our two new IPA servers.  For this we used will connect to our NFS Ganesha Cluster.  To configure NFS home directories, we issue the following (NOTE: The different domain in the below commands):  

ipa automountlocation-del UserHomeDir01
ipa automountlocation-add UserHomeDir01
ipa automountlocation-import UserHomeDir01 /etc/auto.master
ipa automountmap-add UserHomeDir01 auto.share
ipa automountmap-add-indirect UserHomeDir01 –mount=/mds.xyz auto.n
ipa automountkey-add UserHomeDir01 auto.n –key=mds.xyz –info="-rw,hard,int,timeo=10,sec=sys,rsize=8192,wsize=8192 ldap:auto.share"
ipa automountkey-add UserHomeDir01 auto.master –key=/n –info=auto.share
ipa automountkey-add UserHomeDir01 auto.share –key=mds.xyz –info="-rw,hard,intr,timeo=10,sec=sys,rsize=8192,wsize=8192 nfs-c01.nix.mds.xyz:/n/mds.xyz"


SUDO Setup and Configuration

We begin by ensuring our Sudoers uses sss on our IPA servers only:

# grep -Ei "^sudoers" /etc/nsswitch.conf
sudoers: files sss

Set the sudo password in IPA:

[root@idmipa03 ~]# ldappasswd -Y GSSAPI -S uid=sudo,cn=sysaccounts,cn=etc,dc=mws,dc=mds,dc=xyz
New password:
Re-enter new password:
SASL/GSSAPI authentication started
SASL username: admin@MWS.MDS.XYZ
SASL data security layer installed.
[root@idmipa03 ~]#


Use the SECRET from above in the below configuration and ensure /etc/sudo-ldap.conf contains the following additional lines:

# grep -Eiv "^#" /etc/sudo-ldap.conf
uri ldap://idmipa03.mws.mds.xyz ldap://idmipa04.mws.mds.xyz
ssl start_tls
binddn uid=sudo,cn=sysaccounts,cn=etc,dc=mws,dc=mds,dc=xyz
bindpw <SECRET>
sudoers_base ou=sudoers,dc=mws,dc=mds,dc=xyz
tls_cacert /etc/ipa/ca.crt

You can get further details for above by issuing the following:

# ldapsearch -Y GSSAPI -b "dc=mws,dc=mds,dc=xyz" dn |grep -Ei sudo

Also ensure that the following sudo related lines appear in your sssd.conf (Items in green are new):



sudo_provider = ldap
ldap_sudo_search_base = ou=sudoers,dc=mws,dc=mds,dc=xyz

id_provider = proxy
proxy_lib_name = files
proxy_pam_target = system-auth-ac
sudo_provider = ipa
ldap_uri = ldap://idmipa03.mws.mds.xyz
ldap_sudo_search_base = ou=sudoers,dc=mws,dc=mds,dc=xyz

services = nss, sudo, ifp, pam, ssh
config_file_version = 2
domains = mws.mds.xyz, sudoproxy

Ensure the NIS Domain is configured ( on both hosts idmipa03 and idmipa04 ):

[root@idmipa03 ~]# cat /etc/sysconfig/network
[root@idmipa03 ~]#

NOTE: It is also possible, and perhaps better, to use the following lines in the domain section of sssd.conf (in place of the above ones):

sudo_provider = ipa
ldap_sudo_search_base = ou=sudoers,dc=mws,dc=mds,dc=xyz

In other words, use ipa instead of ldap.

Test from a client:

tom@mds.xyz@cm-r01en01:~] 🙂 $ sudo -l
[sudo] password for tom@mds.xyz:
Matching Defaults entries for tom@mds.xyz on cm-r01en01:
    !visiblepw, always_set_home, match_group_by_gid, env_reset, env_keep="COLORS DISPLAY HOSTNAME HISTSIZE KDEDIR LS_COLORS",
    _XKB_CHARSET XAUTHORITY", secure_path=/sbin\:/bin\:/usr/sbin\:/usr/bin

User tom@mds.xyz may run the following commands on cm-r01en01:
    (ALL : ALL) ALL

tom@mds.xyz@cm-r01en01:~] 🙂 $

Additional debugging hints can be found at this location.

Forwarder Setup to ( Windows ) MDS.XYZ

In order to resolve the MDS.XYZ domain from the command line and elsewhere, we need to forward on it.   However before you do that, disable the DNSSEC verifications or you'll get a warning message and according to some reports, interference when resolving AD users and groups. 

# grep dnssec /etc/named.conf
        # dnssec-enable yes;
        # dnssec-validation yes;
        dnssec-enable no ;
        dnssec-validation no;


This panel outlines how to set this up:

Take note that there is additional forwarding definitions on this panel:

Additional Forwarders

Alternative Forwarder Details


Sudo rules are easily configured from the UI though you can optionally do so from the command line as well.  In this case, we'll show you the UI method of configuring this.  Do the following

  1. Browse to Policy -> Sudo
  2. Choose a group to make sudo rules for.  We will choose the AD group nixadmins@mds.xyz that we mapped earlier.
  3. Using the AD group name (not the map name) create the sudo rule NixAdminsAll
  4. Under Who, select Specified Users and Groups.  Choose group nixadmins.
  5. Under Access this host select Any Host
  6. Under Run Commands choose Any Command
  7. For As Whom select Anyone and Any Group
  8. Save!
  9. Test (See at end)



Configuring the client is very straightforward. Issue all of the following commands to complete the configuration on the client side:

  • ipa-client-install –force-join -p admin -w '<SECRET>' –fixed-primary –server=idmipa03.mws.mds.xyz –server=idmipa04.mws.mds.xyz –domain=mws.mds.xyz –realm=MWS.MDS.XYZ -U
  • authconfig –enablesssd –enablesssdauth –enablemkhomedir –updateall –update
  • ipa-client-automount –location=UserHomeDir01 -U

The configuration files, for comparison, should look as follows ( particularly items in green ):

# cat /etc/sssd/sssd.conf

cache_credentials = True
krb5_store_password_if_offline = True
ipa_domain = mws.mds.xyz
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ipa_hostname = cm-r01en01.mws.mds.xyz
chpass_provider = ipa
ipa_server = idmipa03.mws.mds.xyz, idmipa04.mws.mds.xyz
ldap_tls_cacert = /etc/ipa/ca.crt
autofs_provider = ipa
ipa_automount_location = UserHomeDir01
dyndns_update = True
dyndns_update_ptr = True
ldap_schema = ad
ldap_id_mapping = True

sudo_provider = ipa
ldap_uri = ldap://idmipa03.mws.mds.xyz, ldap://idmipa04.mws.mds.xyz
ldap_sudo_search_base = ou=sudoers,dc=mws,dc=mds,dc=xyz

override_homedir = /n/%d/%u
# fallback_homedir = /n/%d/%u
# ldap_user_home_directory = unixHomeDirectory

services = nss, sudo, pam, autofs, ssh

domains = mws.mds.xyz
homedir_substring = /home









And the krb5.conf:

# cat /etc/krb5.conf
#File modified by ipa-client-install

includedir /etc/krb5.conf.d/
includedir /var/lib/sss/pubconf/krb5.include.d/

  default_realm = MWS.MDS.XYZ
  dns_lookup_realm = false
  dns_lookup_kdc = true
  rdns = false
  dns_canonicalize_hostname = true
  ticket_lifetime = 24h
  forwardable = true
  udp_preference_limit = 0
  default_ccache_name = KEYRING:persistent:%{uid}

    kdc = idmipa03.mws.mds.xyz:88
    master_kdc = idmipa03.mws.mds.xyz:88
    admin_server = idmipa03.mws.mds.xyz:749
    kpasswd_server = idmipa03.mws.mds.xyz:464
    kdc = idmipa04.mws.mds.xyz:88
    master_kdc = idmipa04.mws.mds.xyz:88
    admin_server = idmipa04.mws.mds.xyz:749
    kpasswd_server = idmipa04.mws.mds.xyz:464
    default_domain = mws.mds.xyz
    pkinit_anchors = FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem
    pkinit_pool = FILE:/var/lib/ipa-client/pki/ca-bundle.pem


  MDS.XYZ = {
    kdc = ad.mds.xyz
    default_domain = mds.xyz

  .mws.mds.xyz = MWS.MDS.XYZ
  mws.mds.xyz = MWS.MDS.XYZ
  cm-r01en01.mws.mds.xyz = MWS.MDS.XYZ
  .mds.xyz = MDS.XYZ
  mds.xyz = MDS.XYZ

Above are the samples.  Of course these can be automated fully.  Some of the above will be created automatically.  


Using username "mds.xyz\tom".
Using keyboard-interactive authentication.
Last login: Wed Feb 20 00:21:10 2019 from
-sh-4.2$ sudo -l
[sudo] password for tom@mds.xyz:
Matching Defaults entries for tom@mds.xyz on cm-r01en01:
    !visiblepw, always_set_home, match_group_by_gid, env_reset, env_keep="COLORS DISPLAY HOSTNAME
    _XKB_CHARSET XAUTHORITY", secure_path=/sbin\:/bin\:/usr/sbin\:/usr/bin

User tom@mds.xyz may run the following commands on cm-r01en01:
    (ALL : ALL) ALL
-sh-4.2$ sudo su –
Last login: Wed Feb 20 00:26:31 EST 2019 on pts/2
[root@cm-r01en01 ~]#



Configure log rotation for the ganesha log files.  They can get big.

[root@nfs03 ganesha]# cat /etc/logrotate.d/ganesha
/var/log/ganesha/ganesha.log {
    rotate 8

/var/log/ganesha/ganesha-rgw.log {
    rotate 8
    maxsize 100M

[root@nfs03 ganesha]#

Manually rotate the log file:

[root@nfs03 ganesha]# logrotate /etc/logrotate.d/ganesha -v

Distribute the log rotate file to the other nodes.

Configure extended logging and corresponding log rotation for keepalived:

[root@nfs03 log]# cat /etc/sysconfig/keepalived
[root@nfs03 log]# 

[root@nfs03 log]# cat /etc/logrotate.d/keepalived
/var/log/keepalived.log {
    rotate 8
    maxsize 100M

[root@nfs03 log]#

Logrotate the log file:

[root@nfs03 log]# logrotate /etc/logrotate.d/keepalived -v

Again, distribute the above configuration to all the nodes.


Comments are closed.

  Copyright © 2003 - 2013 Tom Kacperski (microdevsys.com). All rights reserved.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License