Chapter 1. Backing up the undercloud node To back up the undercloud node, you configure the backup node, install the Relax-and-Recover tool on the undercloud node, and create the backup image. You can create backups as a part of your regular environment maintenance. In addition, you must back up the undercloud node before performing updates or upgrades. You can use the backups to restore the undercloud node to its state if an error occurs during an update or upgrade. 1.1. Supported backup formats and protocols The undercloud and backup and restore process uses the open-source tool Relax-and-Recover (ReaR) to create and restore bootable backup images. ReaR is written in Bash and supports multiple image formats and multiple transport protocols. The following list shows the backup formats and protocols that Red Hat OpenStack Platform supports when you use ReaR to back up and restore the undercloud and control plane. Bootable media formats ISO File transport protocols SFTP NFS 1.2. Configuring the backup storage location Before you create a backup of the control plane nodes, configure the backup storage location in the bar-vars.yaml environment file. This file stores the key-value parameters that you want to pass to the backup execution. Procedure In the bar-vars.yaml file, configure the backup storage location. Follow the appropriate steps for your NFS server or SFTP server. If you use an NFS server, add the following parameters the bar-vars.yaml file: tripleo_backup_and_restore_server: <ip_address> tripleo_backup_and_restore_shared_storage_folder: <backup_server_dir_path> tripleo_backup_and_restore_output_url: "nfs://{{ tripleo_backup_and_restore_server }}{{ tripleo_backup_and_restore_shared_storage_folder }}" tripleo_backup_and_restore_backup_url: "nfs://{{ tripleo_backup_and_restore_server }}{{ tripleo_backup_and_restore_shared_storage_folder }}" Replace <ip_address> and <backup_server_dir_path> . The default value for tripleo_backup_and_restore_server parameter value is 192.168.24.1 . If you use an SFTP server, add the tripleo_backup_and_restore_output_url parameter and set the values of the URL and credentials of the SFTP server: tripleo_backup_and_restore_output_url: sftp://<user>:<password>@<backup_node>/ tripleo_backup_and_restore_backup_url: iso:///backup/ Replace <user> , <password> , and <backup_node> with the backup node URL and credentials. 1.3. Installing and configuring an NFS server on the backup node You can install and configure a new NFS server to store the backup file. To install and configure an NFS server on the backup node, create an inventory file, create an SSH key, and run the openstack undercloud backup command with the NFS server options. Important If you previously installed and configured an NFS or SFTP server, you do not need to complete this procedure. You enter the server information when you set up ReaR on the node that you want to back up. By default, the Relax and Recover (ReaR) IP address parameter for the NFS server is 192.168.24.1 . You must add the parameter tripleo_backup_and_restore_server to set the IP address value that matches your environment. Procedure On the undercloud node, source the undercloud credentials: On the undercloud node, create an inventory file for the backup node: (undercloud) [stack@undercloud ~]USD cat <<'EOF'> ~/nfs-inventory.yaml [BackupNode] <backup_node> ansible_host=<ip_address> ansible_user=<user> EOF Replace <ip_address> and <user> with the values that apply to your environment. Copy the public SSH key from the undercloud node to the backup node. (undercloud) [stack@undercloud ~]USD ssh-copy-id -i ~/.ssh/id_rsa.pub <backup_node> Replace <backup_node> with the path and name of the backup node. Configure the NFS server on the backup node: (undercloud) [stack@undercloud ~]USD openstack undercloud backup --setup-nfs --extra-vars /home/stack/bar-vars.yaml --inventory /home/stack/nfs-inventory.yaml 1.4. Installing ReaR on the undercloud node Before you create a backup of the undercloud node, install and configure Relax and Recover (ReaR) on the undercloud. Prerequisites You have an NFS or SFTP server installed and configured on the backup node. For more information about creating a new NFS server, see Section 1.3, "Installing and configuring an NFS server on the backup node" . Procedure On the undercloud node, source the undercloud credentials: [stack@undercloud ~]USD source stackrc If you use a custom stack name, add the --stack <stack_name> option to the tripleo-ansible-inventory command. If you have not done so before, create an inventory file and use the tripleo-ansible-inventory command to generate a static inventory file that contains hosts and variables for all the overcloud nodes: (undercloud) [stack@undercloud ~]USD tripleo-ansible-inventory \ --ansible_ssh_user heat-admin \ --static-yaml-inventory /home/stack/tripleo-inventory.yaml Install ReaR on the undercloud node: (undercloud) [stack@undercloud ~]USD openstack undercloud backup --setup-rear --extra-vars /home/stack/bar-vars.yaml --inventory /home/stack/tripleo-inventory.yaml If your system uses the UEFI boot loader, perform the following steps on the undercloud node: Install the following tools: USD sudo dnf install dosfstools efibootmgr Enable UEFI backup in the ReaR configuration file located in /etc/rear/local.conf by replacing the USING_UEFI_BOOTLOADER parameter value 0 with the value 1 . 1.5. Creating a standalone database backup of the undercloud nodes If you are upgrading your Red Hat OpenStack Platform environment from 13 to 16.2, you must create a standalone database backup after you perform the undercloud upgrade and before you perform the Leapp upgrade process on the undercloud nodes. You can optionally include standalone undercloud database backups in your routine backup schedule to provide additional data security. A full backup of an undercloud node includes a database backup of the undercloud node. But if a full undercloud restoration fails, you might lose access to the database portion of the full undercloud backup. In this case, you can recover the database from a standalone undercloud database backup. Procedure Create a database backup of the undercloud nodes: openstack undercloud backup --db-only The db backup file is stored in /home/stack with the name openstack-backup-mysql-<timestamp>.sql . Additional resources Framework for Upgrades (13 to 16.2) Section 1.7, "Creating a backup of the undercloud node" Section 3.5, "Restoring the undercloud node database manually" 1.6. Configuring Open vSwitch (OVS) interfaces for backup If you use an Open vSwitch (OVS) bridge in your environment, you must manually configure the OVS interfaces before you create a backup of the undercloud or control plane nodes. The restoration process uses this information to restore the network interfaces. Procedure In the /etc/rear/local.conf file, add the NETWORKING_PREPARATION_COMMANDS parameter in the following format: Replace <command_1> and <command_2> with commands that configure the network interfaces. For example, you can add the ip link add br-ctlplane type bridge command to create the control plane bridge or add the ip link set eth0 up command to change the state of eth0 to up. You can add more commands to the parameter based on your network configuration. For example, if your undercloud has the following configuration: The NETWORKING_PREPARATION_COMMANDS parameter is formatted as follows: 1.7. Creating a backup of the undercloud node To create a backup of the undercloud node, use the openstack undercloud backup command. You can then use the backup to restore the undercloud node to its state in case the node becomes corrupted or inaccessible. The backup of the undercloud node includes the backup of the database that runs on the undercloud node. If you are upgrading your Red Hat OpenStack Platform environment from 13 to 16.2, you must create a separate database backup after you perform the undercloud upgrade and before you perform the Leapp upgrade process on the overcloud nodes. For more information, see Section 1.5, "Creating a standalone database backup of the undercloud nodes" . Prerequisites You have an NFS or SFTP server installed and configured on the backup node. For more information about creating a new NFS server, see Section 1.3, "Installing and configuring an NFS server on the backup node" . You have installed ReaR on the undercloud node. For more information, see Section 1.4, "Installing ReaR on the undercloud node" . If you use an OVS bridge for your network interfaces, you have configured the OVS interfaces. For more information, see Section 1.6, "Configuring Open vSwitch (OVS) interfaces for backup" . Procedure Log in to the undercloud as the stack user. Retrieve the MySQL root password: [stack@undercloud ~]USD PASSWORD=USD(sudo /bin/hiera -c /etc/puppet/hiera.yaml mysql::server::root_password) Create a database backup of the undercloud node: [stack@undercloud ~]USD sudo podman exec mysql bash -c "mysqldump -uroot -pUSDPASSWORD --opt --all-databases" | sudo tee /root/undercloud-all-databases.sql Source the undercloud credentials: [stack@undercloud ~]USD source stackrc If you have not done so before, create an inventory file and use the tripleo-ansible-inventory command to generate a static inventory file that contains hosts and variables for all the overcloud nodes: (undercloud) [stack@undercloud ~]USD tripleo-ansible-inventory \ --ansible_ssh_user heat-admin \ --static-yaml-inventory /home/stack/tripleo-inventory.yaml Create a backup of the undercloud node: (undercloud) [stack@undercloud ~]USD openstack undercloud backup --inventory /home/stack/tripleo-inventory.yaml 1.8. Scheduling undercloud node backups with cron You can schedule backups of the undercloud nodes with ReaR by using the Ansible backup-and-restore role. You can view the logs in the /var/log/rear-cron directory. Prerequisites You have an NFS or SFTP server installed and configured on the backup node. For more information about creating a new NFS server, see Section 1.3, "Installing and configuring an NFS server on the backup node" . You have installed ReaR on the undercloud and control plane nodes. For more information, see Section 2.3, "Installing ReaR on the control plane nodes" . You have sufficient available disk space at your backup location to store the backup. Procedure To schedule a backup of your control plane nodes, run the following command. The default schedule is Sundays at midnight: openstack undercloud backup --cron Optional: Customize the scheduled backup according to your deployment: To change the default backup schedule, pass a different cron schedule on the tripleo_backup_and_restore_cron parameter: openstack undercloud backup --cron --extra-vars '{"tripleo_backup_and_restore_cron": "0 0 * * 0"}' To define additional parameters that are added to the backup command when cron runs the scheduled backup, pass the tripleo_backup_and_restore_cron_extra parameter to the backup command, as shown in the following example: openstack undercloud backup --cron --extra-vars '{"tripleo_backup_and_restore_cron_extra":"--extra-vars bar-vars.yaml --inventory /home/stack/tripleo-inventory.yaml"}' To change the default user that executes the backup, pass the tripleo_backup_and_restore_cron_user parameter to the backup command, as shown in the following example: openstack undercloud backup --cron --extra-vars '{"tripleo_backup_and_restore_cron_user": "root"}

Commands:
tripleo_backup_and_restore_server: <ip_address> tripleo_backup_and_restore_shared_storage_folder: <backup_server_dir_path> tripleo_backup_and_restore_output_url: "nfs://{{ tripleo_backup_and_restore_server }}{{ tripleo_backup_and_restore_shared_storage_folder }}" tripleo_backup_and_restore_backup_url: "nfs://{{ tripleo_backup_and_restore_server }}{{ tripleo_backup_and_restore_shared_storage_folder }}"
tripleo_backup_and_restore_output_url: sftp://<user>:<password>@<backup_node>/ tripleo_backup_and_restore_backup_url: iso:///backup/
[stack@undercloud ~]USD source stackrc (undercloud) [stack@undercloud ~]USD
(undercloud) [stack@undercloud ~]USD cat <<'EOF'> ~/nfs-inventory.yaml [BackupNode] <backup_node> ansible_host=<ip_address> ansible_user=<user> EOF
(undercloud) [stack@undercloud ~]USD ssh-copy-id -i ~/.ssh/id_rsa.pub <backup_node>
(undercloud) [stack@undercloud ~]USD openstack undercloud backup --setup-nfs --extra-vars /home/stack/bar-vars.yaml --inventory /home/stack/nfs-inventory.yaml
[stack@undercloud ~]USD source stackrc
(undercloud) [stack@undercloud ~]USD tripleo-ansible-inventory --ansible_ssh_user heat-admin --static-yaml-inventory /home/stack/tripleo-inventory.yaml
(undercloud) [stack@undercloud ~]USD openstack undercloud backup --setup-rear --extra-vars /home/stack/bar-vars.yaml --inventory /home/stack/tripleo-inventory.yaml
sudo dnf install dosfstools efibootmgr
openstack undercloud backup --db-only
NETWORKING_PREPARATION_COMMANDS=('<command_1>' '<command_2>' ...')
ip -4 addr ls br-ctlplane 8: br-ctlplane: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 inet 172.16.9.1/24 brd 172.16.9.255 scope global br-ctlplane valid_lft forever preferred_lft forever sudo ovs-vsctl show Bridge br-ctlplane Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port eth0 Interface eth0 Port br-ctlplane Interface br-ctlplane type: internal Port phy-br-ctlplane Interface phy-br-ctlplane type: patch options: {peer=int-br-ctlplane}
NETWORKING_PREPARATION_COMMANDS=('ip link add br-ctlplane type bridge' 'ip link set br-ctlplane up' 'ip link set eth0 up' 'ip link set eth0 master br-ctlplane' 'ip addr add 172.16.9.1/24 dev br-ctlplane')
[stack@undercloud ~]USD PASSWORD=USD(sudo /bin/hiera -c /etc/puppet/hiera.yaml mysql::server::root_password)
[stack@undercloud ~]USD sudo podman exec mysql bash -c "mysqldump -uroot -pUSDPASSWORD --opt --all-databases" | sudo tee /root/undercloud-all-databases.sql
[stack@undercloud ~]USD source stackrc
(undercloud) [stack@undercloud ~]USD tripleo-ansible-inventory --ansible_ssh_user heat-admin --static-yaml-inventory /home/stack/tripleo-inventory.yaml
(undercloud) [stack@undercloud ~]USD openstack undercloud backup --inventory /home/stack/tripleo-inventory.yaml
openstack undercloud backup --cron
openstack undercloud backup --cron --extra-vars '{"tripleo_backup_and_restore_cron": "0 0 * * 0"}'
openstack undercloud backup --cron --extra-vars '{"tripleo_backup_and_restore_cron_extra":"--extra-vars bar-vars.yaml --inventory /home/stack/tripleo-inventory.yaml"}'
openstack undercloud backup --cron --extra-vars '{"tripleo_backup_and_restore_cron_user": "root"}