Proxedo API Security: Migration manual 3.3.1 to 3.4.0

Copyright © 2019 Balasys IT Ltd.. All rights reserved. This document is protected by copyright and is distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this document may be reproduced in any form by any means without prior written authorization of Balasys.

This documentation and the product it describes are considered protected by copyright according to the applicable laws.

This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/). This product includes cryptographic software written by Eric Young (eay@cryptsoft.com)

Linux™ is a registered trademark of Linus Torvalds.

Windows™ 10 is registered trademarks of Microsoft Corporation.

The Balasys™ name and the Balasys™ logo are registered trademarks of Balasys IT Ltd.

The Zorp™ name and the Zorp™ logo are registered trademarks of Balasys IT Ltd.

The Proxedo™ name and the Proxedo™ logo are registered trademarks of Balasys IT Ltd.

AMD Ryzen™ and AMD EPYC™ are registered trademarks of Advanced Micro Devices, Inc.

Intel® Core™ and Intel® Xeon™ are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.

All other product names mentioned herein are the trademarks of their respective owners.

DISCLAIMER

Balasys is not responsible for any third-party websites mentioned in this document. Balasys does not endorse and is not responsible or liable for any content, advertising, products, or other material on or available from such sites or resources. Balasys will not be responsible or liable for any damage or loss caused or alleged to be caused by or in connection with use of or reliance on any such content, goods, or services that are available on or through any such sites or resources.

2022-02-09 .Copyright

Introduction

This guide describes the necessary steps and commands to upgrade your PAS instance from version 3.3.1 to 3.4.0.

There are four main parts to describe different phases and scenarios of the upgrade.

How to create a backup for safety
Single node upgrade process
Multi node upgrade process
How to restore pre-upgrade state in case of upgrade failure

Backup

Before starting the upgrade process, make sure you have a backup from which you can restore the current state of your PAS setup.

The following steps describe how to create a backup of an actual configuration manually.

All instructions need to be executed on the management node even in case of a multi node setup.

The password for the management component’s admin user will be necessary to be able to backup the running configuration.

At the end, save both files (bootstrap-config.zip and config.zip in the examples) to a backup server.

Bootstrap configuration

Install the zip package by running apt install zip.
Log in to pas user by running sudo -iu pas.
Save the following configuration files on the node in a zip file.
- /opt/balasys/etc
- /opt/balasys/.ssh for a multi node setup
- Parts of the automated core deployment tool
  - /opt/balasys/usr/share/automation/deploy-core.yml
  - /opt/balasys/usr/share/automation/host_vars
  - /opt/balasys/usr/share/automation/inventory.yml
  - /opt/balasys/usr/share/automation/roles/deploy-core/vars/main.yml

Example command to compress bootstrap files in a single node setup

zip --recurse-paths bootstrap-config.zip --symlinks \
    /opt/balasys/etc/ \
    /opt/balasys/usr/share/automation/{deploy-core.yml,host_vars,inventory.yml} \
    /opt/balasys/usr/share/automation/roles/deploy-core/vars/main.yml

Example command to compress bootstrap files in a multi node setup

zip --recurse-paths bootstrap-config.zip --symlinks \
    /opt/balasys/.ssh/ \
    /opt/balasys/etc/ \
    /opt/balasys/usr/share/automation/{deploy-core.yml,host_vars,inventory.yml} \
    /opt/balasys/usr/share/automation/roles/deploy-core/vars/main.yml

Running configuration

Log in to the web UI as admin user and navigate to the Config Backup page from the top bar.
On the Config Backup page select Running from the Export Config dropdown menu.

Figure 1. Backup and restore services with Proxedo API Security configuration
To export the running configuration, press the Download button. This will save the running configuration to a file named config.zip in the working directory.

To restore the backup, follow the instructions in Restoring the pre-upgrade state.

Single node upgrade process

This section describes how to upgrade PAS in a single node setup. In case any problem occurs during the upgrade and the version 3.3.1 needs to be restored, follow the instructions in Restoring the pre-upgrade state.

Prerequisites

The following requirements need to be met before carrying out the upgrade process:

The management, storage and core components of version 3.3.1 are running and healthy.
The password for the admin user of the management component is available.
The new Debian packages are downloaded and available for installation on the node:
- proxedo-api-security_3.4.0_all.deb
- proxedo-api-security-mgmt_3.4.0_all.deb
- proxedo-api-security-storage_3.4.0_all.deb

Upgrade steps

Install the new Proxedo API Security packages as root user. This will not restart systemd services.

Use the simplified installer windows for a directed and easier way of installing the PAS packages.

When the installer asks whether the existing configuration files shall be overwritten, select 'yes'. This question will be asked for each package.

By agreeing the installer to overwrite the existing files, the installer creates a working, single node configuration, consequently, there is no need to complete any manual updates. The newly created configuration is saved in a backup subdirectory next to its original location.

/opt/balasys/etc/infrastructure/storage/backups for storage infrastructure configuration
/opt/balasys/etc/storage/backups for storage configuration
/opt/balasys/etc/infrastructure/mgmt/backups for management infrastructure configuration
/opt/balasys/etc/mgmt/backups for management configuration
/opt/balasys/etc/infrastructure/core/backups for core infrastructure configuration

Figure 2. Question about overriding existing files

When the Debian installer asks what to do with the new versions of configuration files, select N for keeping installed versions:

Example question from the Debian installer

Installing new version of config file /opt/balasys/etc/infrastructure/storage/docker-compose.conf ...

Configuration file '/opt/balasys/etc/storage/config.yml'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
  What would you like to do about it ?  Your options are:
   Y or I  : install the package maintainer's version
   N or O  : keep your currently-installed version
     D     : show the differences between the versions
     Z     : start a shell to examine the situation
The default action is to keep your current version.
*** config.yml (Y/I/N/O/D/Z) [default=N] ?

Log in to pas user by executing sudo -iu pas. Carry out the following operations as pas user.
Run pas-mgmt-upgrade-config pre-upgrade. This will guide you through the pre-upgrade process. Once the pre-upgrade finishes, you will find a pas-config-3.3.1.zip in your working directory.

The old administrator password needs to be used, even if you provided a new one during the installation.
Run pas-mgmt-upgrade-config convert-config in the same directory. It will convert your old running configuration to be compatible with the new version. The new configuration will be saved as pas-config-3.3.1.upgraded-to-3.4.0.zip.
Run the update command for each component
- pas-update for core
- pas-mgmt-update for management
- pas-storage-update for storage
Run the check config command for each component for which it is available
- pas-mgmt-checkconfig for management
- pas-storage-checkconfig for storage
Restart the PAS management and storage components.

Stop the PAS core component.

By stopping the core component, the proxy functionality is hung up, and network traffic going through is interrupted. Traffic will be served again when the core component is restarted in the last step.

Run pas-mgmt-upgrade-config post-upgrade in the same directory where the pre-upgrade and configuration conversion steps were performed. Follow the instructions of the script to complete.

If a new administrator password was provided in the installer, that one needs to be used.
Start the PAS core component.

Multi node upgrade process

This section describes how to upgrade PAS in a multi node setup. In case any problem occurs during the upgrade and the version 3.3.1 needs to be restored, follow the instructions in Restoring the pre-upgrade state.

Prerequisites

The following requirements need to be met before carrying out the upgrade process:

The management and storage components of version 3.3.1 are running and healthy on the management node.

If the core component is also run on the management node, it needs to be running and functioning too.
The storage and core components of version 3.3.1 are running and healthy on the core node.
The password for the admin user of the management component is available.
The new Debian packages are downloaded and available for installation on the management node:
- proxedo-api-security_3.4.0_all.deb
- proxedo-api-security-mgmt_3.4.0_all.deb
- proxedo-api-security-storage_3.4.0_all.deb

Upgrade steps

Execute all steps on the management node.

Make sure local PAS management and storage components of version 3.3.1 are running.
Make sure remote PAS storage and core components of version 3.3.1 are running.

Install the new Proxedo API Security packages locally as root user. This will not restart systemd services.

The simplified installer is designed to help single node installation. Ignore all questions asked by the installer during the upgrade and select "No" where possible. Ignore any questions asked twice and any error prompts.

When the Debian installer asks what to do with the new versions of the configuration files, select N for keeping the installed versions.

Example question from the Debian installer

Installing new version of config file /opt/balasys/etc/infrastructure/storage/docker-compose.conf ...

Configuration file '/opt/balasys/etc/storage/config.yml'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
  What would you like to do about it ?  Your options are:
   Y or I  : install the package maintainer's version
   N or O  : keep your currently-installed version
     D     : show the differences between the versions
     Z     : start a shell to examine the situation
The default action is to keep your current version.
*** config.yml (Y/I/N/O/D/Z) [default=N] ?

Log in to pas user by executing sudo -iu pas. Carry out the following operations as pas user.
Run pas-mgmt-upgrade-config pre-upgrade. This will guide you through the pre-upgrade process. Once the pre-upgrade finishes, you will find a pas-config-3.3.1.zip in your working directory.
Run pas-mgmt-upgrade-config convert-config in the same directory. It will convert your old running configuration to be compatible with the new version. The new configuration will be saved as pas-config-3.3.1.upgraded-to-3.4.0.zip.
Update the configuration of the automated deployment tool.
1. At /opt/balasys/etc/automation/common_vars.yml
  - Update the storage_deb_path to the new .deb file.
  - Update the core_deb_path to the new .deb file.
  - Update the common.docker.PAS_IMAGE_TAG to 3.4.0.
    
    Example extract of values for the updated attributes
    
    storage_deb_path: /tmp/proxedo-api-security-storage_3.4.0_all.deb core_deb_path: /tmp/proxedo-api-security_3.4.0_all.deb common: docker: PAS_IMAGE_TAG: 3.4.0
Run the update command for each component:
- pas-update for core if you also run core on the management node
- pas-mgmt-update for management
- pas-storage-update for storage
Run the check config command for each component for which it is available
- pas-mgmt-checkconfig for management
- pas-storage-checkconfig for storage

Restart the local PAS management and storage components.

By stopping the local core component, the proxy functionality is hung up, and the network traffic going through is interrupted. The traffic will only be served again when the local core component is restarted after the post-upgrade command.

This step will inherently restart the HA component if it was running.

Restart the remote core component by running pas-mgmt-deploy-core --restart-core. This will restart the remote nodes with PAS version 3.3.1 but that is expected. Upgrading the remote components will be done in a later step.

By stopping the remote core component, the proxy functionality is hung up, and the network traffic going through is interrupted. The traffic will only be served again when the remote core component is restarted after the post-upgrade command.

This step will inherently restart the HA component if it was running.

Run pas-mgmt-upgrade-config post-upgrade in the same directory where you performed the pre-upgrade and convert steps. Follow the instructions of the script to complete.
Restart the local PAS core component.
If you are running HA, also restart the local HA component.
Upgrade and restart the remote PAS core component by running pas-mgmt-deploy-core --deploy-core --restart-core.
If you are running HA, also upgrade and restart the remote HA component by running pas-mgmt-deploy-core --deploy-ha --restart-ha.
Open the UI as admin user and go to the Status Dashboard. If you see FAILED in the Reload success column, go to the Changes Dashboard, and click Config Apply even if there are no pending changes.

Restoring the pre-upgrade state

Cleanup

Stop all PAS services on all nodes.
- systemctl stop proxedo-api-security for core
- systemctl stop proxedo-api-security-mgmt for management
- systemctl stop proxedo-api-security-storage for storage
Remove PAS packages as root user on all nodes. Remove packages only from those nodes where they are installed.
- apt remove --purge proxedo-api-security for core
- apt remove --purge proxedo-api-security-mgmt on management
- apt remove --purge proxedo-api-security-storage on storage
Remove the pas user by running userdel --force --remove pas

Restore

All instructions need to be executed on the management node.

Reinstall PAS version 3.3.1 packages.
Log in to pas user by running sudo -iu pas.
Copy the files saved during the backup to the pas user’s home directory.

Bootstrap configuration

It is important to run all commands as pas user to prevent from accidentally overwriting system files.

Unzip the saved bootstrap configuration files in the /opt/balasys directory as pas user by running unzip -u -o bootstrap-config.zip -d /.

Example bootstrap configuration restore command and output

$ unzip -u -o bootstrap-config.zip -d /
Archive:  bootstrap-config.zip
   creating: /opt/balasys/etc/
   creating: /opt/balasys/etc/ha/
  inflating: /opt/balasys/etc/ha/config.yml
   creating: /opt/balasys/etc/mgmt/
 extracting: /opt/balasys/etc/mgmt/users.htpass
  inflating: /opt/balasys/etc/mgmt/config.yml
   creating: /opt/balasys/etc/storage/
[...]

Start all PAS services including the HA component if you previously ran a HA setup.
If you are restoring a multi node setup, also deploy the remote node by running the remote deployment command.
```
pas-mgmt-deploy-core --deploy-core
```
If you run an HA setup, also start the HA service on the remote node.
```
pas-mgmt-deploy-core --deploy-ha
```

Running configuration

Log in to the web UI as admin user and navigate to the Config Backup page from the top bar.
To import the running configuration, on the Config Backup page choose a config file from the computer and press Upload to upload the configuration.

Figure 3. Backup and restore services with Proxedo API Security configuration
Apply the configuration.