Migrate RevDeBug Server

Moving RevDeBug instances to a new VM

In the instructions, variable names are provided. Replace them with the ones used by you.

Backup PostgreSQL

Setting Up Environment Variables for PostgreSQL Login

If you are using the default variables to log in to RevDeBug PostgreSQL, you can use the following commands:

Change to the 'revdebug-server-docker-compose' directory:

cd ./revdebug-server-docker-compose

Export the environment variables for PostgreSQL credentials using the following commands:

export $(grep 'DB_.*:' docker-compose.yml | sed 's/.*{\(REVDEBUG_POSTGRES_.*\):-\(.*\)}/\1=\2/')
export $(eval cat .env | grep -v "^#")

These commands will set up the necessary environment variables for PostgreSQL. Make sure you are in the correct directory containing the 'docker-compose.yml' file and the '.env' file before executing the commands.

Performing a PostgreSQL database backup

Create a folder for database backups and navigate to it:

mkdir db && cd db

Execute the command to create a backup of the PostgresSQL for the databases revdebug_db and keycloak:

sudo docker compose -p revdebug exec -e PGPASSWORD="$REVDEBUG_POSTGRES_PASSWORD" postgres  pg_dump -U $REVDEBUG_POSTGRES_USER revdebug_db > ./pg-backup.sql

sudo docker compose -p revdebug exec -e PGPASSWORD="$REVDEBUG_POSTGRES_PASSWORD" postgres  pg_dump -U $REVDEBUG_POSTGRES_USER keycloak > ./pg-backup-keycloak.sql

Backup OpenSearch

Go to the directory named revdebug-server-docker-compose

cd ./revdebug-server-docker-compose

Add path.repo configuration to docker-compose.yml

In the docker-compose.yml file, add the following configuration to the environment section of the opensearch container:

environment:
  - path.repo=/usr/share/opensearch/snapshots

Recreate the OpenSearch container to apply the changes

sudo docker compose -p revdebug up -d

Enter the OpenSearch container

Run the following command to access the terminal within the OpenSearch container:

sudo docker exec -it revdebug-opensearch-1 bash

Create a snapshot repository using a request

Issue a PUT request to create a repository for snapshots with the following command:

curl -X PUT localhost:9200/_snapshot/opensearch-snapshots --header 'Content-Type: application/json' --data '{ "type": "fs", "settings": { "location": "/usr/share/opensearch/snapshots" }}'

Verify if the repository has been created

You can check if the repository has been successfully created by using the following command:

curl -X GET localhost:9200/_cat/repositories

List all snapshots taken so far

To list all the snapshots taken up to this point, run the following command:

curl -X GET localhost:9200/_snapshot/opensearch-snapshots/_all?pretty

Take a snapshot of the current OpenSearch database state

Create a snapshot of the current state of the OpenSearch database using the following command:

curl -X PUT localhost:9200/_snapshot/opensearch-snapshots/snapshot-name

Note: Replace snapshot-name in previous step with a name of your choice for the snapshot (or use the default).

Verify if the snapshot was successful

Check if the snapshot was executed successfully using the following command:

curl -X GET localhost:9200/_snapshot/opensearch-snapshots/_all?pretty

Exit the container.

Copy the snapshot folder from the OpenSearch container

sudo docker cp revdebug-opensearch-1:/usr/share/opensearch/snapshots ../db/snapshots

Shut down the RevDeBug server

To shut down the RevDeBug server, you can use the following command:

sudo docker compose -p revdebug down

This command will stop and remove the containers, networks, and volumes associated with the RevDeBug server specified in the docker-compose.yml file. The -p option allows you to specify a project name, and in this case, it's set to revdebug.

Create a backup of RevDeBug server data, configuration, and databases

Navigate to the parent directory and create a new folder named backup:

Create a backup of the server data

cd ..
mkdir backup
cd backup

Exclude the directories apm_database and revdebug_database and compress the contents of /var/revdebug/ into a tarball named backup.tar.gz:

sudo tar --exclude='apm_database' --exclude='revdebug_database' -czvf backup.tar.gz -C /var/revdebug/ .

Create a backup of the server configuration

Compress the contents of the revdebug-server-docker-compose directory into a tarball named configuration.tar.gz:

sudo tar -czvf configuration.tar.gz -C ../revdebug-server-docker-compose .

Create a backup of the databases

Compress the contents of the db directory into a tarball named db.tar.gz:

sudo tar -czvf db.tar.gz -C ../db .

After executing these commands, you should have three tarball files (backup.tar.gz, configuration.tar.gz, and db.tar.gz) containing the backups of the RevDeBug server data, configuration, and databases, respectively.

Please ensure that you have the necessary permissions to execute these commands and access the directories you want to back up.

Transfer backup files to the new server

Login to the new virtual machine.

If you already have a fresh instance of RevDeBug Server running, follow these steps to remove it data

Stop the RevDeBug server:

sudo docker compose -p revdebug down

Remove all RevDeBug data:

sudo rm -rf /var/revdebug

Remove the RevDeBug server configuration files (optional, but steps below restore those from the old server):

sudo rm -rf ./revdebug-server-docker-compose

Transfer backup files from the old server to the new server

Assuming you have access to both servers, use scp to securely copy the BACKUP_FOLDER from the old server to the BACKUP folder on the new server. Replace YOUR_OLD_SERVER_ADDRESS, HOME, and USER with appropriate values:

scp -r USER@YOUR_OLD_SERVER_ADDRESS:/home/USER/backup /home/USER/backup

Unpack the backup files

Unpack the configuration backup (optional if you have a fresh RevDeBug Server configured already)

Create a new directory named revdebug-server-docker-compose and extract the contents of the configuration.tar.gz backup file inside it:

mkdir revdebug-server-docker-compose && sudo tar --same-owner -xzvf /home/USER/backup/configuration.tar.gz -C ./revdebug-server-docker-compose

Unpack the server data backup

Create a new directory at /var/revdebug and extract the contents of the backup.tar.gz backup file into it:

sudo mkdir /var/revdebug && sudo tar --same-owner -xzvf /home/USER/backup/backup.tar.gz -C /var/revdebug

Unpack the database backup

Extract the contents of the db.tar.gz backup file into the ./db directory:

sudo tar -xzvf /home/USER/backup/db.tar.gz -C ./db

Please ensure you replace YOUR_OLD_SERVER_ADDRESS, HOME, and USER in step 5 with the appropriate values specific to your old server. Similarly, in step 6, ensure that you use the correct paths for unpacking the backups on your new server.

Always be cautious when performing operations that may modify or overwrite data on your server. Make sure to have backups and verify the paths before executing the commands.

Set the given variables in the .env configuration file

Go to ./revdebug-server-docker-compose

Open the .env file using a text editor and set the REVDEBUG_SERVER_NAME variable to your new server address. For example:

REVDEBUG_SERVER_NAME=YOUR_SERVER_ADDRESS

Save the changes to the .env file.

Add path.repo configuration (optional if configuration is also moved from the old server)

In the docker-compose.yml file, add the following configuration to the environment section of the opensearch container:

environment:
  - path.repo=/usr/share/opensearch/snapshots

Start the containers with the databases

To start the containers for the databases (PostgreSQL and OpenSearch) in the background, use the following command:

sudo docker compose -p revdebug up -d postgres opensearch

This command will start the PostgreSQL and OpenSearch containers as specified in the docker-compose.yml file with the project name revdebug.

Please make sure you have the required permissions to execute Docker commands.

Restore PostgreSQL database

Setting Up Environment Variables for PostgreSQL Login

If you are using the default variables to log in to RevDeBug PostgreSQL, you can use the following commands:

Change to the 'revdebug-server-docker-compose' directory:

cd ./revdebug-server-docker-compose

Export the environment variables for PostgreSQL login using the following commands:

export $(grep 'DB_.*:' docker-compose.yml | sed 's/.*{\(REVDEBUG_POSTGRES_.*\):-\(.*\)}/\1=\2/')
export $(eval cat .env | grep -v "^#")

These commands will set up the necessary environment variables for logging into PostgreSQL. Make sure you are in the correct directory containing the 'docker-compose.yml' file and the '.env' file before executing the commands.

Navigate to the uncompressed 'db' folder

Change to the directory where the database backup files are located:

cd ../db

Restore the Keycloak database using the following command:

sudo docker exec -i revdebug-postgres-1 sh -c 'PGPASSWORD="$REVDEBUG_POSTGRES_PASSWORD" psql -U $REVDEBUG_POSTGRES_USER keycloak' < /home/USER/db/pg-backup-keycloak.sql

Restore the revdebug_db database using the following command:

sudo docker exec -i revdebug-postgres-1 sh -c 'PGPASSWORD="$REVDEBUG_POSTGRES_PASSWORD" psql -U $REVDEBUG_POSTGRES_USER revdebug_db' < /home/USER/db/pg-backup.sql

Please make sure to replace /home/USER/db/pg-backup-keycloak.sql and /home/USER/db/pg-backup.sql with the correct paths to your PostgreSQL backup files.

These commands will restore the databases 'keycloak' and 'revdebug_db' in the PostgreSQL container named 'revdebug-postgres-1'.

Restore OpenSearch database

Copy the snapshot folder to the OpenSearch container

Copy the snapshots folder to the OpenSearch container using the following command:

sudo docker cp ./snapshots revdebug-opensearch-1:/usr/share/opensearch/

Enter the OpenSearch container as root

Access the OpenSearch container as the root user with the following command:

sudo docker exec -it --user root revdebug-opensearch-1 bash

Set the user 'opensearch' as the owner of the 'snapshots' folder

Change the owner of the snapshots folder to the 'opensearch' user using the following command inside the container:

chown -R opensearch:opensearch snapshots

Execute the request to create the snapshot repository

Issue the request to create the snapshot repository with the following command:

curl -X PUT localhost:9200/_snapshot/opensearch-snapshots --header 'Content-Type: application/json' --data '{ "type": "fs", "settings": { "location": "/usr/share/opensearch/snapshots" }}'

Verify if the repository has been created

Check if the repository has been successfully created using the following command:

curl -X GET localhost:9200/_cat/repositories

You can check the current structure of the database using the following command:

curl -X GET localhost:9200/_cat/indices

(Optional) Delete data from OpenSearch

If needed, you can delete all data from OpenSearch using the following command:

curl -X DELETE localhost:9200/_all

Perform the restore using the request

Issue the request to perform the restore with the following command:

curl -X POST localhost:9200/_snapshot/opensearch-snapshots/snapshot-name/_restore --header 'Content-Type: application/json' --data '{"indices": "-.opendistro*"}'

Note: Replace snapshot-name in the previous step with a name assigned when doing the snapshot during backup or leave it as is if you have used the default.

Verify the database structure after the restore

You can check the structure of the database after the restore using the following command:

curl -X GET localhost:9200/_cat/indices

Start all RevDeBug Server containers

Navigate to the 'revdebug-server-docker-compose' directory

Change to the 'revdebug-server-docker-compose' directory using the following command:

cd ../revdebug-server-docker-compose

Start all containers in the project

To start all the containers defined in the docker-compose.yml file, run the following command:

sudo docker compose -p revdebug up -d

The -p option specifies the project name, and in this case, it's set to revdebug. The -d option runs the containers in the background (detached mode).

After running this command migrated RevDeBug Server should get running.