gpmidi
28 Feb 2023

The Problem

UPDATE

I'm in the middle of using a Rocky 8 box as an intermediate to try to get to the latest Fedora version. Will edit the post when I know if that works or not. 

Overview

Can't create a new replica of an older FreeIPA server (v4.6.8 on c7) to a new FreeIPA server (v4.9 on f36 and v4.10 on f37). The error is during the `Configuring certificate server (pki-tomcatd)` phase. 

Example ipa-replica-install error

# kinit <MY PERSONAL ADMIN USERNAME>
# ipa-replica-install --setup-adtrust --setup-ca --setup-dns  --no-forwarders  --skip-conncheck --add-sids

...

Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
  [1/30]: creating certificate server db
  [2/30]: setting up initial replication
Starting replication, please wait until this has completed.
Update in progress, 11 seconds elapsed
Update succeeded

  [3/30]: creating ACIs for admin
  [4/30]: creating installation admin user
Unable to log in as uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca on ldap://ipam.i.gpmidi.net:389
[hint] tune with replication_wait_timeout
  [error] NotFound: uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca did not replicate to ldap://ipam.i.gpmidi.net:389
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca did not replicate to ldap://ipam.i.gpmidi.net:389
The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

From Installer Log

2023-03-01T18:01:02Z DEBUG   [4/30]: creating installation admin user
2023-03-01T18:01:02Z DEBUG Waiting 30 seconds for uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca to appear on ldap://ipam.i.gpmidi.net:389
2023-03-01T18:01:32Z ERROR Unable to log in as uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca on ldap://ipam.i.gpmidi.net:389
2023-03-01T18:01:32Z INFO [hint] tune with replication_wait_timeout
2023-03-01T18:01:32Z DEBUG Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/ipaserver/install/service.py", line 686, in start_creation
    run_step(full_msg, method)
  File "/usr/lib/python3.11/site-packages/ipaserver/install/service.py", line 672, in run_step
    method()
  File "/usr/lib/python3.11/site-packages/ipaserver/install/dogtaginstance.py", line 789, in setup_admin
    raise errors.NotFound(
ipalib.errors.NotFound: uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca did not replicate to ldap://ipam.i.gpmidi.net:389

2023-03-01T18:01:32Z DEBUG   [error] NotFound: uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca did not replicate to ldap://ipam.i.gpmidi.net:389
2023-03-01T18:01:32Z DEBUG The ipa-replica-install command failed, exception: NotFound: uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca did not replicate to ldap://ipam.i.gpmidi.net:389

While Waiting For User Sync/Validation...

tl;dr The user seems to exist on both sides!

[root@ipa0 ~]# ldapsearch -x -D "cn=Directory Manager" -W -b "uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca" ldap://ipam.i.gpmidi.net:389
Enter LDAP Password:
# extended LDIF
#
# LDAPv3
# base <uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca> with scope subtree
# filter: (objectclass=*)
# requesting: ldap://ipam.i.gpmidi.net:389
#

# admin-ipa0.i.gpmidi.net, people, ipaca
dn: uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1
[root@ipa0 ~]# ldapsearch -x -D "cn=Directory Manager" -W -b "uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca" ldap://localhost
Enter LDAP Password:
# extended LDIF
#
# LDAPv3
# base <uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca> with scope subtree
# filter: (objectclass=*)
# requesting: ldap://localhost
#

# admin-ipa0.i.gpmidi.net, people, ipaca
dn: uid=admin-ipa0.i.gpmidi.net,ou=people,o=ipaca

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

The Environment

Source

Distro: CentOS 7.9.2009
FreeIPA: 4.6.8

Target

Originally

Distro: Fedora Server 36
FreeIPA: 4.9.11

Later

Distro: Fedora Server 37
FreeIPA: 4.10.1

Install Commands
Step 1 - Client
ipa-client-install --ssh-trust-dns --mkhomedir --realm=I.GPMIDI.NET --ntp-pool=0.pool.ntp.org --force-join --enable-dns-updates --subid --hostname=ipa0.i.gpmidi.net --ntp-server=1.pool.ntp.org
Step 2 - kinit
kinit <MY PERSONAL USER>
Step 3 - Replica Install
ipa-replica-install --setup-adtrust --setup-ca --setup-dns  --no-forwarders  --skip-conncheck --add-sids

Sometimes the `--debug` flag was also used. 

The installer would ask about trusted domain support - answered "no" via no entry unless noted otherwise. 

Enable trusted domains support in slapi-nis? [no]:
Cleanup Commands

Used after a failure to reset the environment. 

Step 1 - Uninstall
/usr/sbin/ipa-server-install --uninstall
Step 2 - Validated Server Removed

Browsed to https://ipam.i.gpmidi.net/ipa/ui/#/e/server/search and validated that the new server, ipa0, wasn't listed. Deleted if it was. 

Related Links

Attempted Fixes

Changed Replication Wait Time

Created ` /etc/ipa/installer.conf` (see below) and changed the time in seconds. 

# cat /etc/ipa/installer.conf
[global]
replication_wait_timeout=30
Result

30s = No change
300s = No change
600s = No change

Left at 30s for further testing - keeps it quick - provides more than enough time since my ldap db is small. 

Update Source IPA Box From C7 To C8

Upgrade from CentOS 7->CentOS 8->Rocky 8. 

Result

The upgrade of the source system from CentOS 7 to CentOS 8 failed badly. Might try again later. 

Update Source IPA Box 389 `root` Password Hash Type

# /usr/bin/pwdhash -D /etc/dirsrv/slapd-YOUR-DOMAIN-NET -s PBKDF2_SHA256 '<Current DirSrv Root Password>'
{PBKDF2_SHA256}xxxxxxxxxxxxxxxxxxxxxxxx
Result

No change

Updated Target IPA Box To Fedora Server 37

Updated target IPA box from f36 to f37. This changed the IPA version from 4.9.11 to 4.10.1. 

Result

No change

Changing Password Storage Scheme On Source

# dsconf -D "cn=Directory Manager" -W ldaps://ipam.i.gpmidi.net config replace passwordStorageScheme=PBKDF2_SHA256
Enter password for cn=Directory Manager on ldaps://ipam.i.gpmidi.net: <ENTERED ROOT PW>
Successfully replaced "passwordStorageScheme"
Result

No change

Trusted Domains Answer = Yes

Answered 'yes' to trusted domains. 

Enable trusted domains support in slapi-nis? [no]: yes
Result

No change

Restarted IPA On Source

Since the `dsconf` change above to the password storage scheme the IPA server on the source box hasn't been restarted. Restarted it via...

# ipactl restart
Result

No change

Replicated To Rocky 8 Machine First

  1. Created a temporary Rocky 8 box (IPA1)
  2. Added IPA1 as a client to IPAM
  3. Replicated IPAM to IPA1
  4. Added IPA0 as a client to IPA1
  5. Replicated IPA1 to IPA0
Result

TBD

Tags