top of page
  • Shawn McKinney

Configuring Apache Fortress-OpenLDAP For High Availability

Updated: Aug 5, 2021

This post describes how Apache Fortress and OpenLDAP can be placed into a highly-available configuration. Apache Fortress provides Identity and Access Management APIs over HTTP using either JSON or REST formats. OpenLDAP is where the data is stored and maintained.

There are two identically configured machines, each with an instance of Fortress (Java) and OpenLDAP (native) processes running on it. A Virtual IP address will be used to route traffic to the designated primary node. In the event of a failure on the primary, the routing will be to the other.

WHAT’S A VIRTUAL IP ADDRESS

From Wikipedia, the free encyclopedia: A virtual IP address (VIP or VIPA) is an IP address that doesn’t correspond to an actual physical network interface. Uses for VIPs include network address translation (especially, one-to-many NAT), fault-tolerance, and mobility.

NOT LOAD-BALANCED

All traffic to either Fortress or OpenLDAP gets routed to a single machine. The virtual IP determines which of the two identically configured machines is active and which is standby. In the event of failure on the primary node, the Virtual IP is then pointed to the secondary.

ADVANTAGES

  • Simple to understand, automate/setup, maintain and use.

  • Efficiency: No additional network hops (through a load-balancer) are required for round trips.

  • Reliability: No risk of a ‘split-brain’ occurrence, when one node falls out of synch of the other, i.e. differing views on the data / inconsistent results.

  • Safety: OpenLDAP running in multi-master replication mode will ensure both node’s data is kept in synch. Either node can function as the primary, at any time.

  • Flexibility: Upgrade to a load-balanced configuration when needed.

  • This configuration works with either Apache Fortress and/or OpenLDAP.

Section 1


INTRODUCTION

This document describes how to setup a VIP (Virtual IP Address) over 2 servers (fortress1 and fortress2). The concepts should work on any platform but were tailored for Redhat7+.

Here, the two servers can be seen from the clients as one single server, with one single IP address. However, only one server will be reachable, until the VIP switches from one server to the other.

In any case, server names aren't all that relevant when it comes to configuring a VIP.


GETTING NETWORK INFORMATION

We first need to list the existing network devices on each node. This is done using the ifconfig command :

[myuser@FORTRESS1 ~]$ ifconfig
ens160: flags=4163 mtu 1500
inet 10.71.6.25 netmask 255.255.255.0 broadcast 10.71.6.255
inet6 fe80::20c:29ff:feb0:e2f6 prefixlen 64 scopeid 0x20
ether 00:0c:29:b0:e2:f6 txqueuelen 1000 (Ethernet)

lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)

Do the same on the other machine.

CREATING A NEW VIRTUAL IP

This command creates a new IP associated with the existing one. This will not survive a reboot or network restart.

[myuser@FORTRESS1 ~]$ sudo ifconfig ens160:0 10.71.6.100
[myuser@FORTRESS1 ~]$ ifconfig
ens160: flags=4163 mtu 1500
inet 10.71.6.25 netmask 255.255.255.0 broadcast 10.71.6.255
inet6 fe80::20c:29ff:feb0:e2f6 prefixlen 64 scopeid 0x20
ether 00:0c:29:b0:e2:f6 txqueuelen 1000 (Ethernet)

ens160:0: flags=4163 mtu 1500
inet 10.71.6.100 netmask 255.0.0.0 broadcast 10.255.255.255
ether 00:0c:29:b0:e2:f6 txqueuelen 1000 (Ethernet)

lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)

Do the same on the other machine.

CREATING A NEW PERMANENT VIRTUAL IP

If we want this VIP to be up when the server is started, we need to create a network script for it.

Assuming the physical network device is ens160, and its associated network script is /etc/sysconfig/network-scripts/ifcfg-ens160, we need to copy this file and give it a new name ending with :0 :

[myuser@FORTRESS1 ~]$ sudo cp /etc/sysconfig/network-scripts/ifcfg-ens160 /etc/sysconfig/network-scripts/ifcfg-ens160:0
[myuser@FORTRESS1 ~]$ sudo chmod 644 /etc/sysconfig/network-scripts/ifcfg-ens160:0

Now change the DEVICE and IPADDR values :

[myuser@FORTRESS1 ~]$ sudo vi /etc/sysconfig/network-scripts/ifcfg-ens160:0

DEVICE=ens160:0
BOOTPROTO=none
HWADDR=
ONBOOT=yes
TYPE=Ethernet
NETMASK=255.255.255.0
IPADDR=10.71.6.100
GATEWAY=10.71.6.1

Finally, restart the network :

[myuser@FORTRESS1 ~]$ sudo service network restart
Restarting network (via systemctl): [ OK ]

We can check that the VIP is up and running :

[myuser@FORTRESS1 ]$ ifconfig
ens160: flags=4163 mtu 1500
inet 10.71.6.25 netmask 255.255.255.0 broadcast 10.71.6.255
inet6 fe80::20c:29ff:feb0:e2f6 prefixlen 64 scopeid 0x20
ether 00:0c:29:b0:e2:f6 txqueuelen 1000 (Ethernet)

ens160:0: flags=4163 mtu 1500
inet 10.71.6.100 netmask 255.255.255.0 broadcast 10.71.6.255
ether 00:0c:29:b0:e2:f6 txqueuelen 1000 (Ethernet)

lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)

Do the same on the other server.

NOTE: two servers can't have the same IP network address at the same time, which means the VIP will always be associated to only one server.

ACTIVATE/DE-ACTIVATE A SERVER

If we want to change the server that is seen as the VIP, we just have to shutdown the active server's interface and activate the new server's interface :

[myuser@FORTRESS1 ]$ sudo ifconfig ens160:0 down

and on the other server :

[myuser@FORTRESS2 ]$ sudo service network restart

Section 2

PREREQS TAILORED FOR REDHAT

  • Apache Fortress-OpenLDAP Virtual IP Setup complete

  • Two time-synched machines.

  • 1 Core, 1 GB RAM, 20 GB HD (minimum)

  • root access (or a dedicated user with sudo access)

  • Recent version of OpenLDAP client and server installed

  • Installed rsyslog

INTRODUCTION

This document describes the preparation of two OpenLDAP server instances, each running on separate machines, into a multi-master configuration, suited for Apache Fortress.


DIRECTORY INFORMATION TREE (DIT) LAYOUT

A DIT's a hierarchical structure that organizes its data under separate categories. If new to LDAP, it helps to think of a DIT as volume of files on a typical computer system. Each of the ou's under the suffix represent a category of data, sort of like folders in our typical machine's file manager. The data nodes themselves reside below their parents, and are stored using keys like uid and cn (depending on the type). This DIT recommendation supports Apache Fortress requirements and its data replicates between both masters.



OPENLDAP SERVER SETUP

To be performed on each machine... 1. Download and extract config package: fortress-openldap-ha-config-v3 It contains three files referenced below:

  • slapd.conf - OpenLDAP's configuration file

  • bootstrap.ldif - Seeds the OpenLDAP directory structure and data.

  • fortress.schema - Contains object definitions to store RBAC policies in OpenLDAP.

Run these steps as root.

2. Navigate to the config folder and copy these files from the config package:

$ cp fortress.schema $OPENLDAP_HOME/etc/openldap/schema
$ cp slapd.conf $OPENLDAP_HOME/etc/openldap

Where OPENLDAP_HOME matches machine's installation location, e.g. /opt/symas 3. Edit the slapd.conf file, make mods: a. Set serverid on the first line, which must be unique across all the servers.

# Server Number 1
serverid 1

b. In each subsection of syncrepl, modify "servernameXX" to your server name. Host name or IP can be used:

provider=ldap://servername01provider=ldap://servername02

c. Verify the credentials passwords in each section of the syncrepl section.

credentials=myslapdserverpw

d. Verify the rootpw in the default database section AND in the log database section.

rootpw myrootpw

e. Save the slapd.conf file. 4. Create two folders:


$ mkdir $DB_HOME/openldap-data/dflt
$ mkdir $DB_HOME/openldap-data/accesslog

Where DB_HOME matches local machine's OpenLDAP data home. This is specified in the slapd configuration. For example, default DB:

#-----------------------------------------------------------------
# Default LMDB database definitions
#-----------------------------------------------------------------
database        mdb

...

directory       "/var/symas/openldap-data/dflt"

and, accesslog DB:

#-----------------------------------------------------------------
# AccessLog database
#-----------------------------------------------------------------
database     mdb

...

directory    /var/symas/openldap-data/accesslog

5. Test the configuration:

$ slaptest -f $OPENLDAP_HOME/etc/openldap/slapd.conf -u

6. Import data using the supplied .ldif file.

a. Test the import with -u option:

$ slapadd -v -u -c -f $OPENLDAP_HOME/etc/openldap/slapd.conf -l bootstrap.ldif

b. Perform the import:

$ slapadd -v -c -f $OPENLDAP_HOME/etc/openldap/slapd.conf -l bootstrap.ldif

7. Create user for the slapd process.

$ adduser openldap

8. Create the slapd log file, then change owner from root to the new openldap user on slapd's files.

$ touch /var/log/openldap.log
$ chown openldap.openldap -R $OPENLDAP_HOME /var/log/openldap.log DB_HOME

9. Configure the slapd logger under rsyslog. a. edit rsyslog conf file

$ vi /etc/rsyslog.conf

b. Add the following to the file

local4.* /var/log/openldap.log

c. restart the rsyslog daemon

$ service rsyslog restart

10. Start the server under the openldap user.

SERVER 2 SPECIFIC INSTRUCTIONS

1. Extract the data from Server 1.

$ slapcat -b "dc=example,dc=com" -f $OPENLDAP_HOME/etc/openldap/slapd.conf -l output.ldif

2. Copy the Server 1 slapd.conf and output.ldif files over to Server 2.

3. Make the following mods to Server 2's slap.conf:

a. Set serverid on the first line, which must be unique across all the servers.


# Replica Server
serverid 2

Everything else should remain the same in the slapd.conf file on the second server.

b. Replace Server 2's slapd.conf with this new file.

4. Run the steps under OpenLDAP Server Setup again, on second machine. Be sure to use the extracted ldif, output.ldif, in place of the bootstrap.ldif on the import step.


VERIFY

1. slapd process is running as openldap user.

ps -ef | grep slapd

2. Tail the log

tail -f -n10000 /var/log/openldap.log

3. Connect and browse with Apache Directory Studio.

4. Add test data, verify it replicates.

5. Examine the Context CSN node on LDAP server #1.

ldapsearch -x -LLL -H ldap://fortress1 -D "cn=Manager,dc=example,dc=com" -w secret -s base -b 'dc=example,dc=com' contextCSN dn: dc=example,dc=com

dn: dc=example,dc=com
contextCSN: 20181123194724.769203Z#000000#001#000000
contextCSN: 20181123194743.991925Z#000000#002#000000

Where fortress1 is hostname or IP address of first instance.

6. Example on LDAP server #2.

ldapsearch -x -LLL -H ldap://fortress2 -D "cn=Manager,dc=example,dc=com" -w secret -s base -b 'dc=example,dc=com' contextCSN dn: dc=example,dc=com

dn: dc=example,dc=com
contextCSN: 20181123194724.769203Z#000000#001#000000
contextCSN: 20181123194743.991925Z#000000#002#000000

Where fortress2 is hostname or IP address of second instance.

7. Compare the two.


Section 3

PREREQS TAILORED FOR REDHAT

  • Java v8 installed

  • Apache Fortress-OpenLDAP Virtual IP Setup complete

  • Steps under OpenLDAP HA Installation complete

INTRODUCTION

Apache Fortress software will be installed to:

  • /opt/fortress: The root folder for the Apache Fortress runtime.

  • /opt/fortress/lib: Apache Fortress Realm proxy component.

  • /opt/fortress/webapps: Apache Fortress Rest component.

  • /opt/fortress/logs/catalina.out: The process logfile.

  • /etc/systemd/system/fortress.service: The service configuration is placed here.

FORTRESS SERVER SETUP

To be performed on each machine.

1. Prep Env

a. Download Tomcat 9 to local machine

b. Create Installation folder, extract tar, goto folder:

$ mkdir /opt/fortress
$ tar -zxvf apache-tomcat-[VERSION].tar.gz -C /opt/fortress --strip-components=1
$ cd /opt/fortress

Where [VERSION] matches downloaded latest Apache Fortress Tomcat. As of today, 9.0.13.

2. Install Fortress

a. Deploy Apache Fortress Realm Proxy:

$ wget http://repo.maven.apache.org/maven2/org/apache/directory/fortress/fortress-realm-proxy/[VERSION]/fortress-realm-proxy-[VERSION].jar -P /opt/fortress/lib

Where [VERSION] matches latest Apache Fortress Realm. As of today, 2.0.3.

b. Deploy Apache Fortress Rest:

$ wget https://repository.apache.org/content/repositories/releases/org/apache/directory/fortress/fortress-rest/[VERSION]/fortress-rest-[VERSION].war -P /opt/fortress/lib

Where [VERSION] matches latest Apache Fortress Rest. As of today, 2.0.3.

3. Configure Fortress-as-a-Service

a. Create a group and user for Apache Fortress process to run under:

$ groupadd fortress
$ useradd -s /bin/nologin -g fortress -d /opt/fortress fortress

b. Edit systemd file

$ vi /etc/systemd/system/fortress.service

c. Add the following. Change JAVA_OPTS for hostname, admin user/pw for your env.

[Unit]
Description=Apache Fortress is powered by Apache Tomcat
After=syslog.target network.target

[Service]
Type=forking

Environment=JAVA_HOME=/usr/lib/jvm/jre
Environment=CATALINA_PID=/opt/fortress/temp/fortress.pid
Environment=CATALINA_HOME=/opt/fortress
Environment=CATALINA_BASE=/opt/fortress
Environment='CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC'
Environment="JAVA_OPTS=-Djava.awt.headless=true 
                       -Dfortress.host=localhost
                       -Dfortress.port=389
                       -Dfortress.admin.user=cn=manager,dc=example,dc=com
                       -Dfortress.admin.pw=secret
                       -Dfortress.config.root=ou=config,dc=example,dc=com"
                       -Dfortress.min.admin.conn=1
                       -Dfortress.max.admin.conn=10
                       -Dfortress.ldap.server.type=openldap
                       -Dfortress.config.realm=DEFAULT
ExecStart=/opt/fortress/bin/startup.sh
ExecStop=/bin/kill -15 $MAINPID

User=fortress
Group=fortress

[Install]
WantedBy=multi-user.target
  • Here, we'll point each instance of Apache Fortress to slapd running localhost.

  • Defaults given for JAVA_HOME, service accounts, password, host, port, etc, change to match your env.

d. Enable the service to startup automatically.

$ chown fortress.fortress -R /opt/fortress
$ systemctl start fortress.service
$ systemctl enable fortress.service

e. Verify it's running and no errors in the logs.

$ ps -ef | grep tomcat
$ tail -f -n10000 /opt/fortress/logs/catalina.out

4. Testing

PREQUISITES FOR THESE TESTS

  • Curl installed onto local machine

USE APACHE FORTRESS QUICKSTART TO TEST SERVICES

1. Download and extract.

$ wget https://github.com/shawnmckinney/apache-fortress-quickstart/archive/master.zip

2. Open a system prompt and navigate to folder in quickstart package containing test files.

$ cd apache-fortress-quickstart-master/src/test/resources

a. Run the sample curl commands. Password for tests, corresponding with adminuser: $3cret b. Point the commands to IP address of the VIP, setup earlier.

4. Verify replication

Use a preferred LDAP client to verify that data is being replicated between the masters as you run the tests.


Developed by: Emmanuel Lécharny

341 views0 comments

Recent Posts

See All

OpenLDAP & LMDB Sizing Guide

Jan 17, 2022 Introduction Symas OpenLDAP configured with LMDB has been extensively tested and its performance characteristics are well...

Comments


bottom of page