GitSwarm-EE 2017.1-1 Documentation

GitSwarm Geo database replication

This document describes the minimal steps you have to take in order to replicate your GitSwarm database into another server. You may have to change some values according to your database setup, how big it is, etc.

Table of Contents

PostgreSQL replication

The GitSwarm primary node where the write operations happen will connect to primary database server, and the secondary ones which are read-only will connect to secondary database servers (which are read-only too).

Note: In many databases' documentation you will see primary being referenced as master and secondary as either slave or standby server (read-only).

Prerequisites

The following guide assumes that:

Step 1. Configure the primary server

  1. SSH into your GitSwarm primary server and login as root:

    sudo -i

  2. Omnibus GitSwarm has already a replication user called gitlab_replicator. You must set its password manually. Replace thepassword with a strong password:

    sudo -u gitswarm-psql /opt/gitswarm/embedded/bin/psql -h /var/opt/gitswarm/postgresql \
     -d template1 \
     -c "ALTER USER gitlab_replicator WITH ENCRYPTED PASSWORD 'thepassword'"

  3. Edit /etc/gitswarm/gitswarm.rb and add the following:

    postgresql['listen_address'] = "1.2.3.4"
postgresql['trust_auth_cidr_addresses'] = ['127.0.0.1/32','1.2.3.4/32']
postgresql['md5_auth_cidr_addresses'] = ['5.6.7.8/32']
postgresql['sql_replication_user'] = "gitlab_replicator"
postgresql['wal_level'] = "hot_standby"
postgresql['max_wal_senders'] = 10
postgresql['wal_keep_segments'] = 10
postgresql['hot_standby'] = "on"

    Where 1.2.3.4 is the public IP address of the primary server, and 5.6.7.8 the public IP address of the secondary one. If you want to add another secondary, the relevant setting would look like:

    postgresql['md5_auth_cidr_addresses'] = ['5.6.7.8/32','11.22.33.44/32']

    Edit the wal values as you see fit.

  4. Save the file and reconfigure GitSwarm for the changes to take effect.
  5. Now that the PostgreSQL server is set up to accept remote connections, run netstat -plnt to make sure that PostgreSQL is listening to the server's public IP.

  6. Continue to set up the secondary server.

Step 2. Configure the secondary server

  1. SSH into your GitSwarm secondary server and login as root:

    sudo -i

  2. Test that the remote connection to the primary server works:

    sudo -u gitswarm-psql /opt/gitswarm/embedded/bin/psql -h 1.2.3.4 -U gitlab_replicator -d gitlabhq_production -W

    When prompted enter the password you set in the first step for the gitlab_replicator user. If all worked correctly, you should see the database prompt.

  3. Exit the PostgreSQL console:

    \q

  4. Edit /etc/gitswarm/gitswarm.rb and add the following:

    postgresql['wal_level'] = "hot_standby"
postgresql['max_wal_senders'] = 10
postgresql['wal_keep_segments'] = 10
postgresql['hot_standby'] = "on"
  5. Reconfigure GitSwarm for the changes to take effect.

  6. Continue to initiate the replication process.

Step 3. Initiate the replication process

Below we provide a script that connects to the primary server, replicates the database and creates the needed files for replication.

The directories used are the defaults that are set up in Omnibus. If you have changed any defaults or are using a source installation, configure it as you see fit replacing the directories and paths.

Warning: Make sure to run this on the secondary server as it removes all PostgreSQL's data before running pg_basebackup.

  1. SSH into your GitSwarm secondary server and login as root:

    sudo -i

  2. Save the snippet below in a file, let's say /tmp/replica.sh:

    #!/bin/bash

PORT="5432"
USER="gitlab_replicator"
echo ---------------------------------------------------------------
echo WARNING: Make sure this scirpt is run from the secondary server
echo ---------------------------------------------------------------
echo
echo Enter the IP of the primary PostgreSQL server
read HOST
echo Enter the password for $USER@$HOST
read -s PASSWORD

echo Stopping PostgreSQL and all GitSwarm services
gitswarm-ctl stop

echo Backing up postgresql.conf
sudo -u gitswarm-psql mv /var/opt/gitswarm/postgresql/data/postgresql.conf /var/opt/gitswarm/postgresql/

echo Cleaning up old cluster directory
sudo -u gitswarm-psql rm -rf /var/opt/gitswarm/postgresql/data
rm -f /tmp/postgresql.trigger

echo Starting base backup as the replicator user
echo Enter the password for $USER@$HOST
sudo -u gitswarm-psql /opt/gitswarm/embedded/bin/pg_basebackup -h $HOST -D /var/opt/gitswarm/postgresql/data -U gitlab_replicator -v -x -P

echo Writing recovery.conf file
sudo -u gitswarm-psql bash -c "cat > /var/opt/gitswarm/postgresql/data/recovery.conf <<- _EOF1_
  standby_mode = 'on'
  primary_conninfo = 'host=$HOST port=$PORT user=$USER password=$PASSWORD'
  trigger_file = '/tmp/postgresql.trigger'
_EOF1_
"

echo Restoring postgresql.conf
sudo -u gitswarm-psql mv /var/opt/gitswarm/postgresql/postgresql.conf /var/opt/gitswarm/postgresql/data/

echo Starting PostgreSQL and all GitSwarm services
gitswarm-ctl start

  3. Run it with:

    bash /tmp/replica.sh

    When prompted, enter the password you set up for the gitlab_replicator user in the first step.

The replication process is now over.

Next steps

Now that the database replication is done, the next step is to configure GitSwarm.

➤ GitSwarm Geo configuration

MySQL replication

We don't support MySQL replication for GitSwarm Geo.