Chapter 4. Adopting Red Hat OpenStack Platform control plane services Adopt your Red Hat OpenStack Platform 17.1 control plane services to deploy them in the Red Hat OpenStack Services on OpenShift (RHOSO) 18.0 control plane. 4.1. Adopting the Identity service To adopt the Identity service (keystone), you patch an existing OpenStackControlPlane custom resource (CR) where the Identity service is disabled. The patch starts the service with the configuration parameters that are provided by the Red Hat OpenStack Platform (RHOSP) environment. Prerequisites Create the keystone secret that includes the Fernet keys that were copied from the RHOSP environment: Procedure Patch the OpenStackControlPlane CR to deploy the Identity service: Create an alias to use the openstack command in the Red Hat OpenStack Services on OpenShift (RHOSO) deployment: Remove services and endpoints that still point to the RHOSP control plane, excluding the Identity service and its endpoints: Verification Verify that you can access the OpenStackClient pod. For more information, see Accessing the OpenStackClient pod in Maintaining the Red Hat OpenStack Services on OpenShift deployment . Confirm that the Identity service endpoints are defined and are pointing to the control plane FQDNs: 4.2. Adopting the Key Manager service To adopt the Key Manager service (barbican), you patch an existing OpenStackControlPlane custom resource (CR) where Key Manager service is disabled. The patch starts the service with the configuration parameters that are provided by the Red Hat OpenStack Platform (RHOSP) environment. The Key Manager service adoption is complete if you see the following results: The BarbicanAPI , BarbicanWorker , and BarbicanKeystoneListener services are up and running. Keystone endpoints are updated, and the same crypto plugin of the source cloud is available. Note This procedure configures the Key Manager service to use the simple_crypto back end. Additional back ends, such as PKCS11 and DogTag, are currently not supported in Red Hat OpenStack Services on OpenShift (RHOSO). Procedure Add the kek secret: Patch the OpenStackControlPlane CR to deploy the Key Manager service: Verification Ensure that the Identity service (keystone) endpoints are defined and are pointing to the control plane FQDNs: Ensure that Barbican API service is registered in the Identity service: List the secrets: 4.3. Adopting the Networking service To adopt the Networking service (neutron), you patch an existing OpenStackControlPlane custom resource (CR) that has the Networking service disabled. The patch starts the service with the configuration parameters that are provided by the Red Hat OpenStack Platform (RHOSP) environment. The Networking service adoption is complete if you see the following results: The NeutronAPI service is running. The Identity service (keystone) endpoints are updated, and the same back end of the source cloud is available. Prerequisites Ensure that Single Node OpenShift or OpenShift Local is running in the Red Hat OpenShift Container Platform (RHOCP) cluster. Adopt the Identity service. For more information, see Adopting the Identity service . Migrate your OVN databases to ovsdb-server instances that run in the Red Hat OpenShift Container Platform (RHOCP) cluster. For more information, see Migrating OVN data . Procedure Patch the OpenStackControlPlane CR to deploy the Networking service: Note: If neutron-dhcp-agent was used in the OSP-17.1 deployment and should still be used after adoption, please enable also dhcp_agent_notification for neutron-api service. This can be done with patch: Verification Inspect the resulting Networking service pods: Ensure that the Neutron API service is registered in the Identity service: Create sample resources so that you can test whether the user can create networks, subnets, ports, or routers: 4.4. Adopting the Object Storage service If you are using Object Storage as a service, adopt the Object Storage service (swift) to the Red Hat OpenStack Services on OpenShift (RHOSO) environment. If you are using the Object Storage API of the Ceph Object Gateway (RGW), skip the following procedure. Prerequisites The Object Storage service storage back-end services are running in the Red Hat OpenStack Platform (RHOSP) deployment. The storage network is properly configured on the Red Hat OpenShift Container Platform (RHOCP) cluster. For more information, see Preparing Red Hat OpenShift Container Platform for Red Hat OpenStack Services on OpenShift in Deploying Red Hat OpenStack Services on OpenShift . Procedure Create the swift-conf secret that includes the Object Storage service hash path suffix and prefix: USD oc apply -f - <<EOF apiVersion: v1 kind: Secret metadata: name: swift-conf namespace: openstack type: Opaque data: swift.conf: USD(USDCONTROLLER1_SSH sudo cat /var/lib/config-data/puppet-generated/swift/etc/swift/swift.conf | base64 -w0) EOF Create the swift-ring-files ConfigMap that includes the Object Storage service ring files: USD oc apply -f - <<EOF apiVersion: v1 kind: ConfigMap metadata: name: swift-ring-files binaryData: swiftrings.tar.gz: USD(USDCONTROLLER1_SSH "cd /var/lib/config-data/puppet-generated/swift/etc/swift && tar cz *.builder *.ring.gz backups/ | base64 -w0") account.ring.gz: USD(USDCONTROLLER1_SSH "base64 -w0 /var/lib/config-data/puppet-generated/swift/etc/swift/account.ring.gz") container.ring.gz: USD(USDCONTROLLER1_SSH "base64 -w0 /var/lib/config-data/puppet-generated/swift/etc/swift/container.ring.gz") object.ring.gz: USD(USDCONTROLLER1_SSH "base64 -w0 /var/lib/config-data/puppet-generated/swift/etc/swift/object.ring.gz") EOF Patch the OpenStackControlPlane custom resource to deploy the Object Storage service: USD oc patch openstackcontrolplane openstack --type=merge --patch ' spec: swift: enabled: true template: memcachedInstance: memcached swiftRing: ringReplicas: 1 swiftStorage: replicas: 0 networkAttachments: - storage storageClass: local-storage 1 storageRequest: 10Gi swiftProxy: secret: osp-secret replicas: 1 passwordSelectors: service: SwiftPassword serviceUser: swift override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer networkAttachments: 2 - storage ' 1 Must match the RHOSO deployment storage class. 2 Must match the network attachment for the Object Storage service configuration from the RHOSP deployment. Verification Inspect the resulting Object Storage service pods: Verify that the Object Storage proxy service is registered in the Identity service (keystone): Verify that you are able to upload and download objects: Note The Object Storage data is still stored on the existing RHOSP nodes. For more information about migrating the actual data from the RHOSP deployment to the RHOSO deployment, see Migrating the Object Storage service (swift) data from RHOSP to Red Hat OpenStack Services on OpenShift (RHOSO) nodes . 4.5. Adopting the Image service To adopt the Image Service (glance) you patch an existing OpenStackControlPlane custom resource (CR) that has the Image service disabled. The patch starts the service with the configuration parameters that are provided by the Red Hat OpenStack Platform (RHOSP) environment. The Image service adoption is complete if you see the following results: The GlanceAPI service up and running. The Identity service endpoints are updated, and the same back end of the source cloud is available. To complete the Image service adoption, ensure that your environment meets the following criteria: You have a running director environment (the source cloud). You have a Single Node OpenShift or OpenShift Local that is running in the Red Hat OpenShift Container Platform (RHOCP) cluster. Optional: You can reach an internal/external Ceph cluster by both crc and director. If you have image quotas in RHOSP 17.1, these quotas are transferred to Red Hat OpenStack Services on OpenShift (RHOSO) 18.0 because the image quota system in 18.0 is disabled by default. For more information about enabling image quotas in 18.0, see Configuring image quotas in Customizing persistent storage . If you enable image quotas in RHOSO 18.0, the new quotas replace the legacy quotas from RHOSP 17.1. 4.5.1. Adopting the Image service that is deployed with a Object Storage service back end Adopt the Image Service (glance) that you deployed with an Object Storage service (swift) back end in the Red Hat OpenStack Platform (RHOSP) environment. The control plane glanceAPI instance is deployed with the following configuration. You use this configuration in the patch manifest that deploys the Image service with the object storage back end: Prerequisites You have completed the adoption steps. Procedure Create a new file, for example, glance_swift.patch , and include the following content: Note The Object Storage service as a back end establishes a dependency with the Image service. Any deployed GlanceAPI instances do not work if the Image service is configured with the Object Storage service that is not available in the OpenStackControlPlane custom resource. After the Object Storage service, and in particular SwiftProxy , is adopted, you can proceed with the GlanceAPI adoption. For more information, see Adopting the Object Storage service . Verify that SwiftProxy is available: Patch the GlanceAPI service that is deployed in the control plane: 4.5.2. Adopting the Image service that is deployed with a Block Storage service back end Adopt the Image Service (glance) that you deployed with a Block Storage service (cinder) back end in the Red Hat OpenStack Platform (RHOSP) environment. The control plane glanceAPI instance is deployed with the following configuration. You use this configuration in the patch manifest that deploys the Image service with the block storage back end: Prerequisites You have completed the adoption steps. Procedure Create a new file, for example glance_cinder.patch , and include the following content: Note The Block Storage service as a back end establishes a dependency with the Image service. Any deployed GlanceAPI instances do not work if the Image service is configured with the Block Storage service that is not available in the OpenStackControlPlane custom resource. After the Block Storage service, and in particular CinderVolume , is adopted, you can proceed with the GlanceAPI adoption. For more information, see Adopting the Block Storage service . Verify that CinderVolume is available: Patch the GlanceAPI service that is deployed in the control plane: 4.5.3. Adopting the Image service that is deployed with an NFS back end Important This content in this section is available in this release as a Technology Preview , and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information, see Technology Preview . Adopt the Image Service (glance) that you deployed with an NFS back end. To complete the following procedure, ensure that your environment meets the following criteria: The Storage network is propagated to the Red Hat OpenStack Platform (RHOSP) control plane. The Image service can reach the Storage network and connect to the nfs-server through the port 2049 . Prerequisites You have completed the adoption steps. In the source cloud, verify the NFS parameters that the overcloud uses to configure the Image service back end. Specifically, in yourdirector heat templates, find the following variables that override the default content that is provided by the glance-nfs.yaml file in the /usr/share/openstack-tripleo-heat-templates/environments/storage directory: Note In this example, the GlanceBackend variable shows that the Image service has no notion of an NFS back end. The variable is using the File driver and, in the background, the filesystem_store_datadir . The filesystem_store_datadir is mapped to the export value provided by the GlanceNfsShare variable instead of /var/lib/glance/images/ . If you do not export the GlanceNfsShare through a network that is propagated to the adopted Red Hat OpenStack Services on OpenShift (RHOSO) control plane, you must stop the nfs-server and remap the export to the storage network. Before doing so, ensure that the Image service is stopped in the source Controller nodes. In the control plane, the Image service is attached to the Storage network, then propagated through the associated NetworkAttachmentsDefinition custom resource (CR), and the resulting pods already have the right permissions to handle the Image service traffic through this network. In a deployed RHOSP control plane, you can verify that the network mapping matches with what has been deployed in the director-based environment by checking both the NodeNetworkConfigPolicy ( nncp ) and the NetworkAttachmentDefinition ( net-attach-def ). The following is an example of the output that you should check in the Red Hat OpenShift Container Platform (RHOCP) environment to make sure that there are no issues with the propagated networks: Procedure Adopt the Image service and create a new default GlanceAPI instance that is connected with the existing NFS share: Replace <ip_address> with the IP address that you use to reach the nfs-server . Replace <exported_path> with the exported path in the nfs-server . Patch the OpenStackControlPlane CR to deploy the Image service with an NFS back end: Verification When GlanceAPI is active, confirm that you can see a single API instance: Ensure that the description of the pod reports the following output: Check that the mountpoint that points to /var/lib/glance/images is mapped to the expected nfs server ip and nfs path that you defined in the new default GlanceAPI instance: Confirm that the UUID is created in the exported directory on the NFS node. For example: On the nfs-server node, the same uuid is in the exported /var/nfs : 4.5.4. Adopting the Image service that is deployed with a Red Hat Ceph Storage back end Adopt the Image Service (glance) that you deployed with a Red Hat Ceph Storage back end. Use the customServiceConfig parameter to inject the right configuration to the GlanceAPI instance. Prerequisites You have completed the adoption steps. Ensure that the Ceph-related secret ( ceph-conf-files ) is created in the openstack namespace and that the extraMounts property of the OpenStackControlPlane custom resource (CR) is configured properly. For more information, see Configuring a Ceph back end . Note If you backed up your Red Hat OpenStack Platform (RHOSP) services configuration file from the original environment, you can compare it with the confgiuration file that you adopted and ensure that the configuration is correct. For more information, see Pulling the configuration from a director deployment . This command produces the difference between both ini configuration files. Procedure Patch the OpenStackControlPlane CR to deploy the Image service with a Red Hat Ceph Storage back end: 4.5.5. Verifying the Image service adoption Verify that you adopted the Image Service (glance) to the Red Hat OpenStack Services on OpenShift (RHOSO) 18.0 deployment. Procedure Test the Image service from the Red Hat OpenStack Platform CLI. You can compare and ensure that the configuration is applied to the Image service pods: If no line appears, then the configuration is correct. Inspect the resulting Image service pods: If you use a Red Hat Ceph Storage back end, ensure that the Red Hat Ceph Storage secrets are mounted: Check that the service is active, and that the endpoints are updated in the RHOSP CLI: Check that the images that you previously listed in the source cloud are available in the adopted service: 4.6. Adopting the Placement service To adopt the Placement service, you patch an existing OpenStackControlPlane custom resource (CR) that has the Placement service disabled. The patch starts the service with the configuration parameters that are provided by the Red Hat OpenStack Platform (RHOSP) environment. Prerequisites You import your databases to MariaDB instances on the control plane. For more information, see Migrating databases to MariaDB instances . You adopt the Identity service (keystone). For more information, see Adopting the Identity service . Procedure Patch the OpenStackControlPlane CR to deploy the Placement service: Verification Check that the Placement service endpoints are defined and pointing to the control plane FQDNs, and that the Placement API responds: 4.7. Adopting the Compute service To adopt the Compute service (nova), you patch an existing OpenStackControlPlane custom resource (CR) where the Compute service is disabled. The patch starts the service with the configuration parameters that are provided by the Red Hat OpenStack Platform (RHOSP) environment. The following procedure describes a single-cell setup. Prerequisites You have completed the adoption steps. You have defined the following shell variables. Replace the following example values with the values that are correct for your environment: Procedure Patch the OpenStackControlPlane CR to deploy the Compute service: Note This procedure assumes that Compute service metadata is deployed on the top level and not on each cell level. If the RHOSP deployment has a per-cell metadata deployment, adjust the following patch as needed. You cannot run the metadata service in cell0 . USD oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' spec: nova: enabled: true apiOverride: route: {} template: secret: osp-secret apiServiceTemplate: override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true metadataServiceTemplate: enabled: true # deploy single nova metadata on the top level override: service: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true schedulerServiceTemplate: customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true cellTemplates: cell0: conductorServiceTemplate: customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true cell1: metadataServiceTemplate: enabled: false # enable here to run it in a cell instead override: service: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true conductorServiceTemplate: customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true ' If you are adopting the Compute service with the Bare Metal Provisioning service (ironic), append the following novaComputeTemplates in the cell1 section of the Compute service CR patch: cell1: novaComputeTemplates: standalone: customServiceConfig: | [DEFAULT] host = <hostname> [workarounds] disable_compute_service_check_for_ffu=true Replace <hostname> with the hostname of the node that is running the ironic Compute driver in the source cloud. Wait for the CRs for the Compute control plane services to be ready: Note The local Conductor services are started for each cell, while the superconductor runs in cell0 . Note that disable_compute_service_check_for_ffu is mandatory for all imported Compute services until the external data plane is imported, and until Compute services are fast-forward upgraded. For more information, see Adopting Compute services to the RHOSO data plane and Performing a fast-forward upgrade on Compute services . Verification Check that Compute service endpoints are defined and pointing to the control plane FQDNs, and that the Nova API responds: Compare the outputs with the topology-specific configuration in Retrieving topology-specific service configuration . Query the superconductor to check that cell1 exists, and compare it to pre-adoption values: The following changes are expected: The cell1 nova database and username become nova_cell1 . The default cell is renamed to cell1 . RabbitMQ transport URL no longer uses guest . Note At this point, the Compute service control plane services do not control the existing Compute service workloads. The control plane manages the data plane only after the data adoption process is completed. For more information, see Adopting Compute services to the RHOSO data plane . Important To import external Compute services to the RHOSO data plane, you must upgrade them first. For more information, see Adopting Compute services to the RHOSO data plane , and Performing a fast-forward upgrade on Compute services . 4.8. Adopting the Block Storage service To adopt a director-deployed Block Storage service (cinder), create the manifest based on the existing cinder.conf file, deploy the Block Storage service, and validate the new deployment. Prerequisites You have reviewed the Block Storage service limitations. For more information, see Limitations for adopting the Block Storage service . You have planned the placement of the Block Storage services. You have prepared the Red Hat OpenShift Container Platform (RHOCP) nodes where the volume and backup services run. For more information, see RHOCP preparation for Block Storage service adoption . The Block Storage service (cinder) is stopped. The service databases are imported into the control plane MariaDB. The Identity service (keystone) and Key Manager service (barbican) are adopted. The Storage network is correctly configured on the RHOCP cluster. The contents of cinder.conf file. Download the file so that you can access it locally: Procedure Create a new file, for example, cinder_api.patch , and apply the configuration: Replace <patch_name> with the name of your patch file. The following example shows a cinder_api.patch file: spec: extraMounts: - extraVol: - extraVolType: Ceph mounts: - mountPath: /etc/ceph name: ceph readOnly: true propagation: - CinderVolume - CinderBackup - Glance volumes: - name: ceph projected: sources: - secret: name: ceph-conf-files cinder: enabled: true apiOverride: route: {} template: databaseInstance: openstack databaseAccount: cinder secret: osp-secret cinderAPI: override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer replicas: 1 customServiceConfig: | [DEFAULT] default_volume_type=tripleo cinderScheduler: replicas: 0 cinderBackup: networkAttachments: - storage replicas: 0 cinderVolumes: ceph: networkAttachments: - storage replicas: 0 Retrieve the list of the scheduler and backup services: Remove services for hosts that are in the down state: Replace <service_binary> with the name of the binary, for example, cinder-backup . Replace <service_host> with the host name, for example, cinder-backup-0 . Deploy the scheduler, backup and volume services. Create another file, for example, cinder_services.patch , and apply the configuration. Replace <patch_name> with the name of your patch file. The following example shows a cinder_services.patch file for an RBD deployment: spec: cinder: enabled: true template: cinderScheduler: replicas: 1 cinderBackup: networkAttachments: - storage replicas: 1 customServiceConfig: | [DEFAULT] backup_driver=cinder.backup.drivers.ceph.CephBackupDriver backup_ceph_conf=/etc/ceph/ceph.conf backup_ceph_user=openstack backup_ceph_pool=backups cinderVolumes: ceph: networkAttachments: - storage replicas: 1 customServiceConfig: | [tripleo_ceph] backend_host=hostgroup volume_backend_name=tripleo_ceph volume_driver=cinder.volume.drivers.rbd.RBDDriver rbd_ceph_conf=/etc/ceph/ceph.conf rbd_user=openstack rbd_pool=volumes rbd_flatten_volume_from_snapshot=False report_discard_supported=True Check if all the services are up and running. Apply the DB data migrations: Note You are not required to run the data migrations at this step, but you must run them before the upgrade. However, for adoption, you can run the migrations now to ensure that there are no issues before you run production workloads on the deployment. Verification Ensure that the openstack alias is defined: Confirm that Block Storage service endpoints are defined and pointing to the control plane FQDNs: Replace <endpoint> with the name of the endpoint that you want to confirm. Confirm that the Block Storage services are running: Note Cinder API services do not appear in the list. However, if you get a response from the openstack volume service list command, that means at least one of the cinder API services is running. Confirm that you have your volume types, volumes, snapshots, and backups: To confirm that the configuration is working, perform the following steps: Create a volume from an image to check that the connection to Image Service (glance) is working: Back up the attached volume: Replace <backup_name> with the name of your new backup location. Note You do not boot a Compute service (nova) instance by using the new volume from image or try to detach the volume because the Compute service and the Block Storage service are still not connected. 4.9. Adopting the Dashboard service To adopt the Dashboard service (horizon), you patch an existing OpenStackControlPlane custom resource (CR) that has the Dashboard service disabled. The patch starts the service with the configuration parameters that are provided by the Red Hat OpenStack Platform environment. Prerequisites You adopted Memcached. For more information, see Deploying back-end services . You adopted the Identity service (keystone). For more information, see Adopting the Identity service . Procedure Patch the OpenStackControlPlane CR to deploy the Dashboard service: Verification Verify that the Dashboard service instance is successfully deployed and ready: Confirm that the Dashboard service is reachable and returns a 200 status code: 4.10. Adopting the Shared File Systems service The Shared File Systems service (manila) in Red Hat OpenStack Services on OpenShift (RHOSO) provides a self-service API to create and manage file shares. File shares (or "shares"), are built for concurrent read/write access from multiple clients. This makes the Shared File Systems service essential in cloud environments that require a ReadWriteMany persistent storage. File shares in RHOSO require network access. Ensure that the networking in the Red Hat OpenStack Platform (RHOSP) 17.1 environment matches the network plans for your new cloud after adoption. This ensures that tenant workloads remain connected to storage during the adoption process. The Shared File Systems service control plane services are not in the data path. Shutting down the API, scheduler, and share manager services do not impact access to existing shared file systems. Typically, storage and storage device management are separate networks. Shared File Systems services only need access to the storage device management network. For example, if you used a Red Hat Ceph Storage cluster in the deployment, the "storage" network refers to the Red Hat Ceph Storage cluster's public network, and the Shared File Systems service's share manager service needs to be able to reach it. The Shared File Systems service supports the following storage networking scenarios: You can directly control the networking for your respective file shares. The RHOSO administrator configures the storage networking. 4.10.1. Guidelines for preparing the Shared File Systems service configuration To deploy Shared File Systems service (manila) on the control plane, you must copy the original configuration file from the Red Hat OpenStack Platform 17.1 deployment. You must review the content in the file to make sure you are adopting the correct configuration for Red Hat OpenStack Services on OpenShift (RHOSO) 18.0. Not all of the content needs to be brought into the new cloud environment. Review the following guidelines for preparing your Shared File Systems service configuration file for adoption: The Shared File Systems service operator sets up the following configurations and can be ignored: Database-related configuration ( [database] ) Service authentication ( auth_strategy , [keystone_authtoken] ) Message bus configuration ( transport_url , control_exchange ) The default paste config ( api_paste_config ) Inter-service communication configuration ( [neutron] , [nova] , [cinder] , [glance] [oslo_messaging_*] ) Ignore the osapi_share_listen configuration. In Red Hat OpenStack Services on OpenShift (RHOSO) 18.0, you rely on Red Hat OpenShift Container Platform (RHOCP) routes and ingress. Check for policy overrides. In RHOSO 18.0, the Shared File Systems service ships with a secure default Role-based access control (RBAC), and overrides might not be necessary. If a custom policy is necessary, you must provide it as a ConfigMap . The following example spec illustrates how you can set up a ConfigMap called manila-policy with the contents of a file called policy.yaml : spec: manila: enabled: true template: manilaAPI: customServiceConfig: | [oslo_policy] policy_file=/etc/manila/policy.yaml extraMounts: - extraVol: - extraVolType: Undefined mounts: - mountPath: /etc/manila/ name: policy readOnly: true propagation: - ManilaAPI volumes: - name: policy projected: sources: - configMap: name: manila-policy items: - key: policy path: policy.yaml The value of the host option under the [DEFAULT] section must be hostgroup . To run the Shared File Systems service API service, you must add the enabled_share_protocols option to the customServiceConfig section in manila: template: manilaAPI . If you have scheduler overrides, add them to the customServiceConfig section in manila: template: manilaScheduler . If you have multiple storage back-end drivers configured with RHOSP 17.1, you need to split them up when deploying RHOSO 18.0. Each storage back-end driver needs to use its own instance of the manila-share service. If a storage back-end driver needs a custom container image, find it in the Red Hat Ecosystem Catalog , and create or modify an OpenStackVersion custom resource (CR) to specify the custom image using the same custom name . The following example shows a manila spec from the OpenStackControlPlane CR that includes multiple storage back-end drivers, where only one is using a custom container image: spec: manila: enabled: true template: manilaAPI: customServiceConfig: | [DEFAULT] enabled_share_protocols = nfs replicas: 3 manilaScheduler: replicas: 3 manilaShares: netapp: customServiceConfig: | [DEFAULT] debug = true enabled_share_backends = netapp host = hostgroup [netapp] driver_handles_share_servers = False share_backend_name = netapp share_driver = manila.share.drivers.netapp.common.NetAppDriver netapp_storage_family = ontap_cluster netapp_transport_type = http replicas: 1 pure: customServiceConfig: | [DEFAULT] debug = true enabled_share_backends=pure-1 host = hostgroup [pure-1] driver_handles_share_servers = False share_backend_name = pure-1 share_driver = manila.share.drivers.purestorage.flashblade.FlashBladeShareDriver flashblade_mgmt_vip = 203.0.113.15 flashblade_data_vip = 203.0.10.14 replicas: 1 The following example shows the OpenStackVersion CR that defines the custom container image: apiVersion: core.openstack.org/v1beta1 kind: OpenStackVersion metadata: name: openstack spec: customContainerImages: cinderVolumeImages: pure: registry.connect.redhat.com/purestorage/openstack-manila-share-pure-rhosp-18-0 The name of the OpenStackVersion CR must match the name of your OpenStackControlPlane CR. If you are providing sensitive information, such as passwords, hostnames, and usernames, use RHOCP secrets, and the customServiceConfigSecrets key. You can use customConfigSecrets in any service. If you use third party storage that requires credentials, create a secret that is referenced in the manila CR/patch file by using the customServiceConfigSecrets key. For example: Create a file that includes the secrets, for example, netapp_secrets.conf : USD cat << __EOF__ > ~/netapp_secrets.conf [netapp] netapp_server_hostname = 203.0.113.10 netapp_login = fancy_netapp_user netapp_password = secret_netapp_password netapp_vserver = mydatavserver __EOF__ Replace <secret> with the name of the file that includes your secrets, for example, netapp_secrets.conf . Add the secret to any Shared File Systems service file in the customServiceConfigSecrets section. The following example adds the osp-secret-manila-netapp secret to the manilaShares service: spec: manila: enabled: true template: < . . . > manilaShares: netapp: customServiceConfig: | [DEFAULT] debug = true enabled_share_backends = netapp host = hostgroup [netapp] driver_handles_share_servers = False share_backend_name = netapp share_driver = manila.share.drivers.netapp.common.NetAppDriver netapp_storage_family = ontap_cluster netapp_transport_type = http customServiceConfigSecrets: - osp-secret-manila-netapp replicas: 1 < . . . > 4.10.2. Deploying the Shared File Systems service on the control plane Copy the Shared File Systems service (manila) configuration from the Red Hat OpenStack Platform (RHOSP) 17.1 deployment, and then deploy the Shared File Systems service on the control plane. Prerequisites The Shared File Systems service systemd services such as api , cron , and scheduler are stopped. For more information, see Stopping Red Hat OpenStack Platform services . If the deployment uses CephFS through NFS as a storage back end, the Pacemaker ordering and collocation constraints are adjusted. For more information, see Stopping Red Hat OpenStack Platform services . The Shared File Systems service Pacemaker service ( openstack-manila-share ) is stopped. For more information, see Stopping Red Hat OpenStack Platform services . The database migration is complete. For more information, see Migrating databases to MariaDB instances . The Red Hat OpenShift Container Platform (RHOCP) nodes where the manila-share service is to be deployed can reach the management network that the storage system is in. If the deployment uses CephFS through NFS as a storage back end, a new clustered Ceph NFS service is deployed on the Red Hat Ceph Storage cluster with the help of Ceph orchestrator. For more information, see Creating a Ceph NFS cluster . Services such as the Identity service (keystone) and memcached are available prior to adopting the Shared File Systems services. If you enabled tenant-driven networking by setting driver_handles_share_servers=True , the Networking service (neutron) is deployed. The CONTROLLER1_SSH environment variable is defined and points to the RHOSP Controller node. Replace the following example values with values that are correct for your environment: Procedure Copy the configuration file from RHOSP 17.1 for reference: Review the configuration file for configuration changes that were made since RHOSP 17.1. For more information on preparing this file for Red Hat OpenStack Services on OpenShift (RHOSO), see Guidelines for preparing the Shared File Systems service configuration . Create a patch file for the OpenStackControlPlane CR to deploy the Shared File Systems service. The following example manila.patch file uses native CephFS: USD cat << __EOF__ > ~/manila.patch spec: manila: enabled: true apiOverride: route: {} template: databaseInstance: openstack databaseAccount: manila secret: osp-secret manilaAPI: replicas: 3 1 customServiceConfig: | [DEFAULT] enabled_share_protocols = cephfs override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer manilaScheduler: replicas: 3 2 manilaShares: cephfs: replicas: 1 3 customServiceConfig: | [DEFAULT] enabled_share_backends = tripleo_ceph host = hostgroup [cephfs] driver_handles_share_servers=False share_backend_name=cephfs 4 share_driver=manila.share.drivers.cephfs.driver.CephFSDriver cephfs_conf_path=/etc/ceph/ceph.conf cephfs_auth_id=openstack cephfs_cluster_name=ceph cephfs_volume_mode=0755 cephfs_protocol_helper_type=CEPHFS networkAttachments: 5 - storage extraMounts: 6 - name: v1 region: r1 extraVol: - propagation: - ManilaShare extraVolType: Ceph volumes: - name: ceph secret: secretName: ceph-conf-files mounts: - name: ceph mountPath: "/etc/ceph" readOnly: true __EOF__ 1 Set the replica count of the manilaAPI service to 3. 2 Set the replica count of the manilaScheduler service to 3. 3 Set the replica count of the manilaShares service to 1. 4 Ensure that the names of the back ends ( share_backend_name ) are the same as they were in RHOSP 17.1. 5 Ensure that the appropriate storage management network is specified in the networkAttachments section. For example, the manilaShares instance with the CephFS back-end driver is connected to the storage network. 6 If you need to add extra files to any of the services, you can use extraMounts . For example, when using Red Hat Ceph Storage, you can add the Shared File Systems service Ceph user's keyring file as well as the ceph.conf configuration file. Patch the OpenStackControlPlane CR: Replace <manila.patch> with the name of your patch file. Verification Inspect the resulting Shared File Systems service pods: Check that the Shared File Systems API service is registered in the Identity service (keystone): Test the health of the service: Check existing workloads: 4.10.3. Decommissioning the Red Hat OpenStack Platform standalone Ceph NFS service Important This content in this section is available in this release as a Technology Preview , and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information, see Technology Preview . If your deployment uses CephFS through NFS, you must decommission the Red Hat OpenStack Platform(RHOSP) standalone NFS service. Since future software upgrades do not support the NFS service, ensure that the decommissioning period is short. Prerequisites You identified the new export locations for your existing shares by querying the Shared File Systems API. You unmounted and remounted the shared file systems on each client to stop using the NFS server. If you are consuming the Shared File Systems service shares with the Shared File Systems service CSI plugin for Red Hat OpenShift Container Platform (RHOCP), you migrated the shares by scaling down the application pods and scaling them back up. Note Clients that are creating new workloads cannot use share exports through the NFS service. The Shared File Systems service no longer communicates with the NFS service, and cannot apply or alter export rules on the NFS service. Procedure Remove the cephfs_ganesha_server_ip option from the manila-share service configuration: Note This restarts the manila-share process and removes the export locations that applied to the NFS service from all the shares. USD cat << __EOF__ > ~/manila.patch spec: manila: enabled: true apiOverride: route: {} template: manilaShares: cephfs: replicas: 1 customServiceConfig: | [DEFAULT] enabled_share_backends = cephfs host = hostgroup [cephfs] driver_handles_share_servers=False share_backend_name=cephfs share_driver=manila.share.drivers.cephfs.driver.CephFSDriver cephfs_conf_path=/etc/ceph/ceph.conf cephfs_auth_id=openstack cephfs_cluster_name=ceph cephfs_protocol_helper_type=NFS cephfs_nfs_cluster_id=cephfs networkAttachments: - storage __EOF__ Patch the OpenStackControlPlane custom resource: Replace <manila.patch> with the name of your patch file. Clean up the standalone ceph-nfs service from the RHOSP control plane nodes by disabling and deleting the Pacemaker resources associated with the service: Important You can defer this step until after RHOSO 18.0 is operational. During this time, you cannot decommission the Controller nodes. Replace <VIP> with the IP address assigned to the ceph-nfs service in your environment. 4.11. Adopting the Bare Metal Provisioning service Important This content in this section is available in this release as a Technology Preview , and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information, see Technology Preview . Review information about your Bare Metal Provisioning service (ironic) configuration and then adopt the Bare Metal Provisioning service to the Red Hat OpenStack Services on OpenShift control plane. 4.11.1. Bare Metal Provisioning service configurations Important This content in this section is available in this release as a Technology Preview , and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information, see Technology Preview . You configure the Bare Metal Provisioning service (ironic) by using configuration snippets. For more information about configuring the control plane with the Bare Metal Provisioning service, see Customizing the Red Hat OpenStack Services on OpenShift deployment . Some Bare Metal Provisioning service configuration is overridden in director, for example, PXE Loader file names are often overridden at intermediate layers. You must pay attention to the settings you apply in your Red Hat OpenStack Services on OpenShift (RHOSO) deployment. The ironic-operator applies a reasonable working default configuration, but if you override them with your prior configuration, your experience might not be ideal or your new Bare Metal Provisioning service fails to operate. Similarly, additional configuration might be necessary, for example, if you enable and use additional hardware types in your ironic.conf file. The model of reasonable defaults includes commonly used hardware-types and driver interfaces. For example, the redfish-virtual-media boot interface and the ramdisk deploy interface are enabled by default. If you add new bare metal nodes after the adoption is complete, the driver interface selection occurs based on the order of precedence in the configuration if you do not explicitly set it on the node creation request or as an established default in the ironic.conf file. Some configuration parameters do not need to be set on an individual node level, for example, network UUID values, or they are centrally configured in the ironic.conf file, as the setting controls security behavior. It is critical that you maintain the following parameters that you configured and formatted as [section] and parameter name from the prior deployment to the new deployment. These parameters that govern the underlying behavior and values in the configuration would have used specific values if set. [neutron]cleaning_network [neutron]provisioning_network [neutron]rescuing_network [neutron]inspection_network [conductor]automated_clean [deploy]erase_devices_priority [deploy]erase_devices_metadata_priority [conductor]force_power_state_during_sync You can set the following parameters individually on a node. However, you might choose to use embedded configuration options to avoid the need to set the parameters individually when creating or managing bare metal nodes. Check your prior ironic.conf file for these parameters, and if set, apply a specific override configuration. [conductor]bootloader [conductor]rescue_ramdisk [conductor]rescue_kernel [conductor]deploy_kernel [conductor]deploy_ramdisk The instances of kernel_append_params , formerly pxe_append_params in the [pxe] and [redfish] configuration sections, are used to apply boot time options like "console" for the deployment ramdisk and as such often must be changed. Warning You cannot migrate hardware types that are set with the ironic.conf file enabled_hardware_types parameter, and hardware type driver interfaces starting with staging- into the adopted configuration. 4.11.2. Deploying the Bare Metal Provisioning service Important This content in this section is available in this release as a Technology Preview , and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information, see Technology Preview . To deploy the Bare Metal Provisioning service (ironic), you patch an existing OpenStackControlPlane custom resource (CR) that has the Bare Metal Provisioning service disabled. The ironic-operator applies the configuration and starts the Bare Metal Provisioning services. After the services are running, the Bare Metal Provisioning service automatically begins polling the power state of the bare metal nodes that it manages. Note By default, newer versions of the Bare Metal Provisioning service contain a more restrictive access control model while also becoming multi-tenant aware. As a result, bare metal nodes might be missing from a openstack baremetal node list command after you adopt the Bare Metal Provisioning service. Your nodes are not deleted. You must set the owner field on each bare metal node due to the increased access restrictions in the role-based access control (RBAC) model. Because this involves access controls and the model of use can be site specific, you should identify which project owns the bare metal nodes. Prerequisites You have imported the service databases into the control plane MariaDB. The Identity service (keystone), Networking service (neutron), Image Service (glance), and Block Storage service (cinder) are operational. Note If you use the Bare Metal Provisioning service in a Bare Metal as a Service configuration, you have not yet adopted the Compute service (nova). For the Bare Metal Provisioning service conductor services, the services must be able to reach Baseboard Management Controllers of hardware that is configured to be managed by the Bare Metal Provisioning service. If this hardware is unreachable, the nodes might enter "maintenance" state and be unavailable until connectivity is restored later. You have downloaded the ironic.conf file locally: Note This configuration file must come from one of the Controller nodes and not a director undercloud node. The director undercloud node operates with different configuration that does not apply when you adopt the Overcloud Ironic deployment. If you are adopting the Ironic Inspector service, you need the value of the IronicInspectorSubnets director parameter. Use the same values to populate the dhcpRanges parameter in the RHOSO environment. You have defined the following shell variables. Replace the following example values with values that apply to your environment: Procedure Patch the OpenStackControlPlane custom resource (CR) to deploy the Bare Metal Provisioning service: USD oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' spec: ironic: enabled: true template: rpcTransport: oslo databaseInstance: openstack ironicAPI: replicas: 1 override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer ironicConductors: - replicas: 1 networkAttachments: - baremetal provisionNetwork: baremetal storageRequest: 10G customServiceConfig: | [neutron] cleaning_network=<cleaning network uuid> provisioning_network=<provisioning network uuid> rescuing_network=<rescuing network uuid> inspection_network=<introspection network uuid> [conductor] automated_clean=true ironicInspector: replicas: 1 inspectionNetwork: baremetal networkAttachments: - baremetal dhcpRanges: - name: inspector-0 cidr: 172.20.1.0/24 start: 172.20.1.190 end: 172.20.1.199 gateway: 172.20.1.1 serviceUser: ironic-inspector databaseAccount: ironic-inspector passwordSelectors: database: IronicInspectorDatabasePassword service: IronicInspectorPassword ironicNeutronAgent: replicas: 1 rabbitMqClusterName: rabbitmq secret: osp-secret ' Wait for the Bare Metal Provisioning service control plane services CRs to become ready: Verify that the individual services are ready: Update the DNS Nameservers on the provisoning, cleaning, and rescue networks: Note For name resolution to work for Bare Metal Provisioning service operations, you must set the DNS nameserver to use the internal DNS servers in the RHOSO control plane: Verify that no Bare Metal Provisioning service nodes are missing from the node list: Important If the openstack baremetal node list command output reports an incorrect power status, wait a few minutes and re-run the command to see if the output syncs with the actual state of the hardware being managed. The time required for the Bare Metal Provisioning service to review and reconcile the power state of bare metal nodes depends on the number of operating conductors through the replicas parameter and which are present in the Bare Metal Provisioning service deployment being adopted. If any Bare Metal Provisioning service nodes are missing from the openstack baremetal node list command, temporarily disable the new RBAC policy to see the nodes again: USD oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' spec: ironic: enabled: true template: databaseInstance: openstack ironicAPI: replicas: 1 customServiceConfig: | [oslo_policy] enforce_scope=false enforce_new_defaults=false ' After you set the owner field on the bare metal nodes, you can re-enable RBAC by removing the customServiceConfig section or by setting the following values to true : After this configuration is applied, the operator restarts the Ironic API service and disables the new RBAC policy that is enabled by default. After the RBAC policy is disabled, you can view bare metal nodes without an owner field: Assign all bare metal nodes with no owner to a new project, for example, the admin project: Re-apply the default RBAC: USD oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' spec: ironic: enabled: true template: databaseInstance: openstack ironicAPI: replicas: 1 customServiceConfig: | [oslo_policy] enforce_scope=true enforce_new_defaults=true ' Verification Verify the list of endpoints: Verify the list of bare metal nodes: 4.12. Adopting the Orchestration service To adopt the Orchestration service (heat), you patch an existing OpenStackControlPlane custom resource (CR), where the Orchestration service is disabled. The patch starts the service with the configuration parameters that are provided by the Red Hat OpenStack Platform (RHOSP) environment. After you complete the adoption process, you have CRs for Heat , HeatAPI , HeatEngine , and HeatCFNAPI , and endpoints within the Identity service (keystone) to facilitate these services. Prerequisites The source director environment is running. The target Red Hat OpenShift Container Platform (RHOCP) environment is running. You adopted MariaDB and the Identity service. If your existing Orchestration service stacks contain resources from other services such as Networking service (neutron), Compute service (nova), Object Storage service (swift), and so on, adopt those sevices before adopting the Orchestration service. Procedure Retrieve the existing auth_encryption_key and service passwords. You use these passwords to patch the osp-secret . In the following example, the auth_encryption_key is used as HeatAuthEncryptionKey and the service password is used as HeatPassword : Log in to a Controller node and verify the auth_encryption_key value in use: Encode the password to Base64 format: Patch the osp-secret to update the HeatAuthEncryptionKey and HeatPassword parameters. These values must match the values in the director Orchestration service configuration: Patch the OpenStackControlPlane CR to deploy the Orchestration service: Verification Ensure that the statuses of all the CRs are Setup complete : Check that the Orchestration service is registered in the Identity service: Check that the Orchestration service engine services are running: Verify that you can see your Orchestration service stacks: 4.13. Adopting Telemetry services To adopt Telemetry services, you patch an existing OpenStackControlPlane custom resource (CR) that has Telemetry services disabled to start the service with the configuration parameters that are provided by the Red Hat OpenStack Platform (RHOSP) 17.1 environment. If you adopt Telemetry services, the observability solution that is used in the RHOSP 17.1 environment, Service Telemetry Framework, is removed from the cluster. The new solution is deployed in the Red Hat OpenStack Services on OpenShift (RHOSO) environment, allowing for metrics, and optionally logs, to be retrieved and stored in the new back ends. You cannot automatically migrate old data because different back ends are used. Metrics and logs are considered short-lived data and are not intended to be migrated to the RHOSO environment. For information about adopting legacy autoscaling stack templates to the RHOSO environment, see Adopting Autoscaling services . Prerequisites The director environment is running (the source cloud). The Single Node OpenShift or OpenShift Local is running in the Red Hat OpenShift Container Platform (RHOCP) cluster. adoption steps are completed. Procedure Patch the OpenStackControlPlane CR to deploy cluster-observability-operator : Wait for the installation to succeed: Patch the OpenStackControlPlane CR to deploy Ceilometer services: Enable the metrics storage back end: Verification Verify that the alertmanager and prometheus pods are available: Inspect the resulting Ceilometer pods: Inspect enabled pollsters: Optional: Override default pollsters according to the requirements of your environment: steps Optional: Patch the OpenStackControlPlane CR to include logging : 4.14. Adopting autoscaling services To adopt services that enable autoscaling, you patch an existing OpenStackControlPlane custom resource (CR) where the Alarming services (aodh) are disabled. The patch starts the service with the configuration parameters that are provided by the Red Hat OpenStack Platform environment. Prerequisites The source director environment is running. A Single Node OpenShift or OpenShift Local is running in the Red Hat OpenShift Container Platform (RHOCP) cluster. You have adopted the following services: MariaDB Identity service (keystone) Orchestration service (heat) Telemetry service Procedure Patch the OpenStackControlPlane CR to deploy the autoscaling services: Inspect the aodh pods: Check whether the aodh API service is registered in the Identity service: Optional: Create aodh alarms with the PrometheusAlarm alarm type: Note You must use the PrometheusAlarm alarm type instead of GnocchiAggregationByResourcesAlarm . Verify that the alarm is enabled: 4.15. Pulling the configuration from a director deployment Before you start the data plane adoption workflow, back up the configuration from the Red Hat OpenStack Platform (RHOSP) services and director. You can then use the files during the configuration of the adopted services to ensure that nothing is missed or misconfigured. Prerequisites The os-diff tool is installed and configured. For more information, see Comparing configuration files between deployments . Procedure Update your ssh parameters according to your environment in the os-diff.cfg . Os-diff uses the ssh parameters to connect to your director node, and then query and download the configuration files: Ensure that the ssh command you provide in ssh_cmd parameter is correct and includes key authentication. Enable the services that you want to include in the /etc/os-diff/config.yaml file, and disable the services that you want to exclude from the file. Ensure that you have the correct permissions to edit the file: The following example enables the default Identity service (keystone) to be included in the /etc/os-diff/config.yaml file: # service name and file location services: # Service name keystone: # Bool to enable/disable a service (not implemented yet) enable: true # Pod name, in both OCP and podman context. # It could be strict match or will only just grep the podman_name # and work with all the pods which matched with pod_name. # To enable/disable use strict_pod_name_match: true/false podman_name: keystone pod_name: keystone container_name: keystone-api # pod options # strict match for getting pod id in TripleO and podman context strict_pod_name_match: false # Path of the config files you want to analyze. # It could be whatever path you want: # /etc/<service_name> or /etc or /usr/share/<something> or even / # @TODO: need to implement loop over path to support multiple paths such as: # - /etc # - /usr/share path: - /etc/ - /etc/keystone - /etc/keystone/keystone.conf - /etc/keystone/logging.conf Repeat this step for each RHOSP service that you want to disable or enable. If you use non-containerized services, such as the ovs-external-ids , pull the configuration or the command output. For example: Note You must correctly configure an SSH configuration file or equivalent for non-standard services, such as OVS. The ovs_external_ids service does not run in a container, and the OVS data is stored on each host of your cloud, for example, controller_1/controller_2/ , and so on. 1 The list of hosts, for example, compute-1 , compute-2 . 2 The command that runs against the hosts. 3 Os-diff gets the output of the command and stores the output in a file that is specified by the key path. 4 Provides a mapping between, in this example, the data plane custom resource definition and the ovs-vsctl output. 5 The edpm_ovn_bridge_mappings variable must be a list of strings, for example, ["datacentre:br-ex"] . Compare the values: For example, to check the /etc/yum.conf on every host, you must put the following statement in the config.yaml file. The following example uses a file called yum_config : Pull the configuration: Note The following command pulls all the configuration files that are included in the /etc/os-diff/config.yaml file. You can configure os-diff to update this file automatically according to your running environment by using the --update or --update-only option. These options set the podman information into the config.yaml for all running containers. The podman information can be useful later, when all the Red Hat OpenStack Platform services are turned off. Note that when the config.yaml file is populated automatically you must provide the configuration paths manually for each service. The configuration is pulled and stored by default in the following directory: Verification Verify that you have a directory for each service configuration in your local path: 4.16. Rolling back the control plane adoption If you encountered a problem and are unable to complete the adoption of the Red Hat OpenStack Platform (RHOSP) control plane services, you can roll back the control plane adoption. Important Do not attempt the rollback if you altered the data plane nodes in any way. You can only roll back the control plane adoption if you altered the control plane. During the control plane adoption, services on the RHOSP control plane are stopped but not removed. The databases on the RHOSP control plane are not edited during the adoption procedure. The Red Hat OpenStack Services on OpenShift (RHOSO) control plane receives a copy of the original control plane databases. The rollback procedure assumes that the data plane has not yet been modified by the adoption procedure, and it is still connected to the RHOSP control plane. The rollback procedure consists of the following steps: Restoring the functionality of the RHOSP control plane. Removing the partially or fully deployed RHOSO control plane. Procedure To restore the source cloud to a working state, start the RHOSP control plane services that you previously stopped during the adoption procedure: If the Ceph NFS service is running on the deployment as a Shared File Systems service (manila) back end, you must restore the Pacemaker order and colocation constraints for the openstack-manila-share service: Verify that the source cloud is operational again, for example, you can run openstack CLI commands such as openstack server list , or check that you can access the Dashboard service (horizon). Remove the partially or fully deployed control plane so that you can attempt the adoption again later: Note After you restore the RHOSP control plane services, their internal state might have changed. Before you retry the adoption procedure, verify that all the control plane resources are removed and that there are no leftovers which could affect the following adoption procedure attempt. You must not use previously created copies of the database contents in another adoption attempt. You must make a new copy of the latest state of the original source database contents. For more information about making new copies of the database, see Migrating databases to the control plane .

Commands:
oc apply -f - <<EOF apiVersion: v1 data: CredentialKeys0: USD(USDCONTROLLER1_SSH sudo cat /var/lib/config-data/puppet-generated/keystone/etc/keystone/credential-keys/0 | base64 -w 0) CredentialKeys1: USD(USDCONTROLLER1_SSH sudo cat /var/lib/config-data/puppet-generated/keystone/etc/keystone/credential-keys/1 | base64 -w 0) FernetKeys0: USD(USDCONTROLLER1_SSH sudo cat /var/lib/config-data/puppet-generated/keystone/etc/keystone/fernet-keys/0 | base64 -w 0) FernetKeys1: USD(USDCONTROLLER1_SSH sudo cat /var/lib/config-data/puppet-generated/keystone/etc/keystone/fernet-keys/1 | base64 -w 0) kind: Secret metadata: name: keystone namespace: openstack type: Opaque EOF
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: keystone: enabled: true apiOverride: route: {} template: override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer databaseInstance: openstack secret: osp-secret '
alias openstack="oc exec -t openstackclient -- openstack"
openstack endpoint list | grep keystone | awk '/admin/{ print USD2; }' | xargs USD{BASH_ALIASES[openstack]} endpoint delete || true for service in aodh heat heat-cfn barbican cinderv3 glance gnocchi manila manilav2 neutron nova placement swift ironic-inspector ironic; do openstack service list | awk "/ USDservice /{ print \USD2; }" | xargs -r USD{BASH_ALIASES[openstack]} service delete || true done
openstack endpoint list | grep keystone
oc set data secret/osp-secret "BarbicanSimpleCryptoKEK=USD(USDCONTROLLER1_SSH "python3 -c \"import configparser; c = configparser.ConfigParser(); c.read('/var/lib/config-data/puppet-generated/barbican/etc/barbican/barbican.conf'); print(c['simple_crypto_plugin']['kek'])\"")"
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: barbican: enabled: true apiOverride: route: {} template: databaseInstance: openstack databaseAccount: barbican rabbitMqClusterName: rabbitmq secret: osp-secret simpleCryptoBackendSecret: osp-secret serviceAccount: barbican serviceUser: barbican passwordSelectors: service: BarbicanPassword simplecryptokek: BarbicanSimpleCryptoKEK barbicanAPI: replicas: 1 override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer barbicanWorker: replicas: 1 barbicanKeystoneListener: replicas: 1 '
openstack endpoint list | grep key-manager
openstack service list | grep key-manager
openstack endpoint list | grep key-manager
openstack secret list
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: neutron: enabled: true apiOverride: route: {} template: override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer databaseInstance: openstack databaseAccount: neutron secret: osp-secret networkAttachments: - internalapi '
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: neutron: template: customServiceConfig: | [DEFAULT] dhcp_agent_notification = True '
NEUTRON_API_POD=`oc get pods -l service=neutron | tail -n 1 | cut -f 1 -d' '` exec -t USDNEUTRON_API_POD -c neutron-api -- cat /etc/neutron/neutron.conf
openstack service list | grep network
openstack endpoint list | grep network | 6a805bd6c9f54658ad2f24e5a0ae0ab6 | regionOne | neutron | network | True | public | http://neutron-public-openstack.apps-crc.testing | | b943243e596847a9a317c8ce1800fa98 | regionOne | neutron | network | True | internal | http://neutron-internal.openstack.svc:9696 |
openstack network create net openstack subnet create --network net --subnet-range 10.0.0.0/24 subnet openstack router create router
oc apply -f - <<EOF apiVersion: v1 kind: Secret metadata: name: swift-conf namespace: openstack type: Opaque data: swift.conf: USD(USDCONTROLLER1_SSH sudo cat /var/lib/config-data/puppet-generated/swift/etc/swift/swift.conf | base64 -w0) EOF
oc apply -f - <<EOF apiVersion: v1 kind: ConfigMap metadata: name: swift-ring-files binaryData: swiftrings.tar.gz: USD(USDCONTROLLER1_SSH "cd /var/lib/config-data/puppet-generated/swift/etc/swift && tar cz *.builder *.ring.gz backups/ | base64 -w0") account.ring.gz: USD(USDCONTROLLER1_SSH "base64 -w0 /var/lib/config-data/puppet-generated/swift/etc/swift/account.ring.gz") container.ring.gz: USD(USDCONTROLLER1_SSH "base64 -w0 /var/lib/config-data/puppet-generated/swift/etc/swift/container.ring.gz") object.ring.gz: USD(USDCONTROLLER1_SSH "base64 -w0 /var/lib/config-data/puppet-generated/swift/etc/swift/object.ring.gz") EOF
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: swift: enabled: true template: memcachedInstance: memcached swiftRing: ringReplicas: 1 swiftStorage: replicas: 0 networkAttachments: - storage storageClass: local-storage 1 storageRequest: 10Gi swiftProxy: secret: osp-secret replicas: 1 passwordSelectors: service: SwiftPassword serviceUser: swift override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer networkAttachments: 2 - storage '
oc get pods -l component=swift-proxy
openstack service list | grep swift | b5b9b1d3c79241aa867fa2d05f2bbd52 | swift | object-store |
openstack endpoint list | grep swift | 32ee4bd555414ab48f2dc90a19e1bcd5 | regionOne | swift | object-store | True | public | https://swift-public-openstack.apps-crc.testing/v1/AUTH_%(tenant_id)s | | db4b8547d3ae4e7999154b203c6a5bed | regionOne | swift | object-store | True | internal | http://swift-internal.openstack.svc:8080/v1/AUTH_%(tenant_id)s |
openstack container create test +---------------------------------------+-----------+------------------------------------+ | account | container | x-trans-id | +---------------------------------------+-----------+------------------------------------+ | AUTH_4d9be0a9193e4577820d187acdd2714a | test | txe5f9a10ce21e4cddad473-0065ce41b9 | +---------------------------------------+-----------+------------------------------------+ openstack object create test --name obj <(echo "Hello World!") +--------+-----------+----------------------------------+ | object | container | etag | +--------+-----------+----------------------------------+ | obj | test | d41d8cd98f00b204e9800998ecf8427e | +--------+-----------+----------------------------------+ openstack object save test obj --file - Hello World!
.. spec glance: customServiceConfig: | [DEFAULT] enabled_backends = default_backend:swift [glance_store] default_backend = default_backend [default_backend] swift_store_create_container_on_put = True swift_store_auth_version = 3 swift_store_auth_address = {{ .KeystoneInternalURL }} swift_store_endpoint_type = internalURL swift_store_user = service:glance swift_store_key = {{ .ServicePassword }}
spec: glance: enabled: true apiOverride: route: {} template: secret: osp-secret databaseInstance: openstack storage: storageRequest: 10G customServiceConfig: | [DEFAULT] enabled_backends = default_backend:swift [glance_store] default_backend = default_backend [default_backend] swift_store_create_container_on_put = True swift_store_auth_version = 3 swift_store_auth_address = {{ .KeystoneInternalURL }} swift_store_endpoint_type = internalURL swift_store_user = service:glance swift_store_key = {{ .ServicePassword }} glanceAPIs: default: replicas: 1 override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer networkAttachments: - storage
oc get pod -l component=swift-proxy | grep Running swift-proxy-75cb47f65-92rxq 3/3 Running 0
oc patch openstackcontrolplane openstack --type=merge --patch-file=glance_swift.patch
.. spec glance: customServiceConfig: | [DEFAULT] enabled_backends = default_backend:cinder [glance_store] default_backend = default_backend [default_backend] rootwrap_config = /etc/glance/rootwrap.conf description = Default cinder backend cinder_store_auth_address = {{ .KeystoneInternalURL }} cinder_store_user_name = {{ .ServiceUser }} cinder_store_password = {{ .ServicePassword }} cinder_store_project_name = service cinder_catalog_info = volumev3::internalURL cinder_use_multipath = true
spec: glance: enabled: true apiOverride: route: {} template: secret: osp-secret databaseInstance: openstack storage: storageRequest: 10G customServiceConfig: | [DEFAULT] enabled_backends = default_backend:cinder [glance_store] default_backend = default_backend [default_backend] rootwrap_config = /etc/glance/rootwrap.conf description = Default cinder backend cinder_store_auth_address = {{ .KeystoneInternalURL }} cinder_store_user_name = {{ .ServiceUser }} cinder_store_password = {{ .ServicePassword }} cinder_store_project_name = service cinder_catalog_info = volumev3::internalURL cinder_use_multipath = true glanceAPIs: default: replicas: 1 override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer networkAttachments: - storage
oc get pod -l component=cinder-volume | grep Running cinder-volume-75cb47f65-92rxq 3/3 Running 0
oc patch openstackcontrolplane openstack --type=merge --patch-file=glance_cinder.patch
GlanceBackend: file GlanceNfsEnabled: true GlanceNfsShare: 192.168.24.1:/var/nfs
oc get nncp NAME STATUS REASON enp6s0-crc-8cf2w-master-0 Available SuccessfullyConfigured oc get net-attach-def NAME ctlplane internalapi storage tenant oc get ipaddresspool -n metallb-system NAME AUTO ASSIGN AVOID BUGGY IPS ADDRESSES ctlplane true false ["192.168.122.80-192.168.122.90"] internalapi true false ["172.17.0.80-172.17.0.90"] storage true false ["172.18.0.80-172.18.0.90"] tenant true false ["172.19.0.80-172.19.0.90"]
cat << EOF > glance_nfs_patch.yaml spec: extraMounts: - extraVol: - extraVolType: Nfs mounts: - mountPath: /var/lib/glance/images name: nfs propagation: - Glance volumes: - name: nfs nfs: path: <exported_path> server: <ip_address> name: r1 region: r1 glance: enabled: true template: databaseInstance: openstack customServiceConfig: | [DEFAULT] enabled_backends = default_backend:file [glance_store] default_backend = default_backend [default_backend] filesystem_store_datadir = /var/lib/glance/images/ storage: storageRequest: 10G glanceAPIs: default: replicas: 3 type: single override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer networkAttachments: - storage EOF
oc patch openstackcontrolplane openstack --type=merge --patch-file glance_nfs_patch.yaml
oc get pods -l service=glance NAME READY STATUS RESTARTS glance-default-single-0 3/3 Running 0 ```
Mounts: nfs: Type: NFS (an NFS mount that lasts the lifetime of a pod) Server: {{ server ip address }} Path: {{ nfs export path }} ReadOnly: false
oc rsh -c glance-api glance-default-single-0 sh-5.1# mount {{ ip address }}:/var/nfs on /var/lib/glance/images type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=172.18.0.5,local_lock=none,addr=172.18.0.5)
oc rsh openstackclient openstack image list sh-5.1USD curl -L -o /tmp/cirros-0.5.2-x86_64-disk.img http://download.cirros-cloud.net/0.5.2/cirros-0.5.2-x86_64-disk.img sh-5.1USD openstack image create --container-format bare --disk-format raw --file /tmp/cirros-0.5.2-x86_64-disk.img cirros sh-5.1USD openstack image list +--------------------------------------+--------+--------+ | ID | Name | Status | +--------------------------------------+--------+--------+ | 634482ca-4002-4a6d-b1d5-64502ad02630 | cirros | active | +--------------------------------------+--------+--------+
ls /var/nfs/ 634482ca-4002-4a6d-b1d5-64502ad02630
cat << EOF > glance_patch.yaml spec: glance: enabled: true template: databaseInstance: openstack customServiceConfig: | [DEFAULT] enabled_backends=default_backend:rbd [glance_store] default_backend=default_backend [default_backend] rbd_store_ceph_conf=/etc/ceph/ceph.conf rbd_store_user=openstack rbd_store_pool=images store_description=Ceph glance store backend. storage: storageRequest: 10G glanceAPIs: default: replicas: 0 override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer networkAttachments: - storage EOF
os-diff diff /tmp/collect_tripleo_configs/glance/etc/glance/glance-api.conf glance_patch.yaml --crd
oc patch openstackcontrolplane openstack --type=merge --patch-file glance_patch.yaml
os-diff diff /etc/glance/glance.conf.d/02-config.conf glance_patch.yaml --frompod -p glance-api
GLANCE_POD=`oc get pod |grep glance-default | cut -f 1 -d' ' | head -n 1` exec -t USDGLANCE_POD -c glance-api -- cat /etc/glance/glance.conf.d/02-config.conf [DEFAULT] enabled_backends=default_backend:rbd [glance_store] default_backend=default_backend [default_backend] rbd_store_ceph_conf=/etc/ceph/ceph.conf rbd_store_user=openstack rbd_store_pool=images store_description=Ceph glance store backend.
oc exec -t USDGLANCE_POD -c glance-api -- ls /etc/ceph ceph.client.openstack.keyring ceph.conf
oc rsh openstackclient openstack service list | grep image | fc52dbffef36434d906eeb99adfc6186 | glance | image | openstack endpoint list | grep image | 569ed81064f84d4a91e0d2d807e4c1f1 | regionOne | glance | image | True | internal | http://glance-internal-openstack.apps-crc.testing | | 5843fae70cba4e73b29d4aff3e8b616c | regionOne | glance | image | True | public | http://glance-public-openstack.apps-crc.testing |
openstack image list +--------------------------------------+--------+--------+ | ID | Name | Status | +--------------------------------------+--------+--------+ | c3158cad-d50b-452f-bec1-f250562f5c1f | cirros | active | +--------------------------------------+--------+--------+
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: placement: enabled: true apiOverride: route: {} template: databaseInstance: openstack databaseAccount: placement secret: osp-secret override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer '
alias openstack="oc exec -t openstackclient -- openstack" openstack endpoint list | grep placement Without OpenStack CLI placement plugin installed: PLACEMENT_PUBLIC_URL=USD(openstack endpoint list -c 'Service Name' -c 'Service Type' -c URL | grep placement | grep public | awk '{ print USD6; }') exec -t openstackclient -- curl "USDPLACEMENT_PUBLIC_URL" With OpenStack CLI placement plugin installed: openstack resource class list
alias openstack="oc exec -t openstackclient -- openstack"
oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' spec: nova: enabled: true apiOverride: route: {} template: secret: osp-secret apiServiceTemplate: override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true metadataServiceTemplate: enabled: true # deploy single nova metadata on the top level override: service: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true schedulerServiceTemplate: customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true cellTemplates: cell0: conductorServiceTemplate: customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true cell1: metadataServiceTemplate: enabled: false # enable here to run it in a cell instead override: service: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true conductorServiceTemplate: customServiceConfig: | [workarounds] disable_compute_service_check_for_ffu=true '
cell1: novaComputeTemplates: standalone: customServiceConfig: | [DEFAULT] host = <hostname> [workarounds] disable_compute_service_check_for_ffu=true
oc wait --for condition=Ready --timeout=300s Nova/nova
openstack endpoint list | grep nova openstack server list
. ~/.source_cloud_exported_variables echo USDPULL_OPENSTACK_CONFIGURATION_NOVAMANAGE_CELL_MAPPINGS rsh nova-cell0-conductor-0 nova-manage cell_v2 list_cells | grep -F '| cell1 |'
USDCONTROLLER1_SSH cat /var/lib/config-data/puppet-generated/cinder/etc/cinder/cinder.conf > cinder.conf
oc patch openstackcontrolplane openstack --type=merge --patch-file=<patch_name>
spec: extraMounts: - extraVol: - extraVolType: Ceph mounts: - mountPath: /etc/ceph name: ceph readOnly: true propagation: - CinderVolume - CinderBackup - Glance volumes: - name: ceph projected: sources: - secret: name: ceph-conf-files cinder: enabled: true apiOverride: route: {} template: databaseInstance: openstack databaseAccount: cinder secret: osp-secret cinderAPI: override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer replicas: 1 customServiceConfig: | [DEFAULT] default_volume_type=tripleo cinderScheduler: replicas: 0 cinderBackup: networkAttachments: - storage replicas: 0 cinderVolumes: ceph: networkAttachments: - storage replicas: 0
openstack volume service list +------------------+------------------------+------+---------+-------+----------------------------+ | Binary | Host | Zone | Status | State | Updated At | +------------------+------------------------+------+---------+-------+----------------------------+ | cinder-scheduler | standalone.localdomain | nova | enabled | down | 2024-11-04T17:47:14.000000 | | cinder-backup | standalone.localdomain | nova | enabled | down | 2024-11-04T17:47:14.000000 | | cinder-volume | hostgroup@tripleo_ceph | nova | enabled | down | 2024-11-04T17:47:14.000000 | +------------------+------------------------+------+---------+-------+----------------------------+
oc exec -t cinder-api-0 -c cinder-api -- cinder-manage service remove <service_binary> <service_host>
oc patch openstackcontrolplane openstack --type=merge --patch-file=<patch_name>
spec: cinder: enabled: true template: cinderScheduler: replicas: 1 cinderBackup: networkAttachments: - storage replicas: 1 customServiceConfig: | [DEFAULT] backup_driver=cinder.backup.drivers.ceph.CephBackupDriver backup_ceph_conf=/etc/ceph/ceph.conf backup_ceph_user=openstack backup_ceph_pool=backups cinderVolumes: ceph: networkAttachments: - storage replicas: 1 customServiceConfig: | [tripleo_ceph] backend_host=hostgroup volume_backend_name=tripleo_ceph volume_driver=cinder.volume.drivers.rbd.RBDDriver rbd_ceph_conf=/etc/ceph/ceph.conf rbd_user=openstack rbd_pool=volumes rbd_flatten_volume_from_snapshot=False report_discard_supported=True
openstack volume service list +------------------+------------------------+------+---------+-------+----------------------------+ | Binary | Host | Zone | Status | State | Updated At | +------------------+------------------------+------+---------+-------+----------------------------+ | cinder-volume | hostgroup@tripleo_ceph | nova | enabled | up | 2023-06-28T17:00:03.000000 | | cinder-scheduler | cinder-scheduler-0 | nova | enabled | up | 2023-06-28T17:00:02.000000 | | cinder-backup | cinder-backup-0 | nova | enabled | up | 2023-06-28T17:00:01.000000 | +------------------+------------------------+------+---------+-------+----------------------------+
oc exec -it cinder-scheduler-0 -- cinder-manage db online_data_migrations
alias openstack="oc exec -t openstackclient -- openstack"
openstack endpoint list --service <endpoint>
openstack volume service list
openstack volume type list openstack volume list openstack volume snapshot list openstack volume backup list
openstack volume create --image cirros --bootable --size 1 disk_new
openstack --os-volume-api-version 3.47 volume create --backup <backup_name>
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: horizon: enabled: true apiOverride: route: {} template: memcachedInstance: memcached secret: osp-secret '
oc get horizon
PUBLIC_URL=USD(oc get horizon horizon -o jsonpath='{.status.endpoint}') curl --silent --output /dev/stderr --head --write-out "%{http_code}" "USDPUBLIC_URL/dashboard/auth/login/?next=/dashboard/" -k | grep 200
spec: manila: enabled: true template: manilaAPI: customServiceConfig: | [oslo_policy] policy_file=/etc/manila/policy.yaml extraMounts: - extraVol: - extraVolType: Undefined mounts: - mountPath: /etc/manila/ name: policy readOnly: true propagation: - ManilaAPI volumes: - name: policy projected: sources: - configMap: name: manila-policy items: - key: policy path: policy.yaml
spec: manila: enabled: true template: manilaAPI: customServiceConfig: | [DEFAULT] enabled_share_protocols = nfs replicas: 3 manilaScheduler: replicas: 3 manilaShares: netapp: customServiceConfig: | [DEFAULT] debug = true enabled_share_backends = netapp host = hostgroup [netapp] driver_handles_share_servers = False share_backend_name = netapp share_driver = manila.share.drivers.netapp.common.NetAppDriver netapp_storage_family = ontap_cluster netapp_transport_type = http replicas: 1 pure: customServiceConfig: | [DEFAULT] debug = true enabled_share_backends=pure-1 host = hostgroup [pure-1] driver_handles_share_servers = False share_backend_name = pure-1 share_driver = manila.share.drivers.purestorage.flashblade.FlashBladeShareDriver flashblade_mgmt_vip = 203.0.113.15 flashblade_data_vip = 203.0.10.14 replicas: 1
apiVersion: core.openstack.org/v1beta1 kind: OpenStackVersion metadata: name: openstack spec: customContainerImages: cinderVolumeImages: pure: registry.connect.redhat.com/purestorage/openstack-manila-share-pure-rhosp-18-0
cat << __EOF__ > ~/netapp_secrets.conf [netapp] netapp_server_hostname = 203.0.113.10 netapp_login = fancy_netapp_user netapp_password = secret_netapp_password netapp_vserver = mydatavserver __EOF__
oc create secret generic osp-secret-manila-netapp --from-file=~/<secret> -n openstack
spec: manila: enabled: true template: < . . . > manilaShares: netapp: customServiceConfig: | [DEFAULT] debug = true enabled_share_backends = netapp host = hostgroup [netapp] driver_handles_share_servers = False share_backend_name = netapp share_driver = manila.share.drivers.netapp.common.NetAppDriver netapp_storage_family = ontap_cluster netapp_transport_type = http customServiceConfigSecrets: - osp-secret-manila-netapp replicas: 1 < . . . >
CONTROLLER1_SSH="ssh -i <path to SSH key> root@<node IP>"
CONTROLLER1_SSH cat /var/lib/config-data/puppet-generated/manila/etc/manila/manila.conf | awk '!/^ *#/ && NF' > ~/manila.conf
cat << __EOF__ > ~/manila.patch spec: manila: enabled: true apiOverride: route: {} template: databaseInstance: openstack databaseAccount: manila secret: osp-secret manilaAPI: replicas: 3 1 customServiceConfig: | [DEFAULT] enabled_share_protocols = cephfs override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer manilaScheduler: replicas: 3 2 manilaShares: cephfs: replicas: 1 3 customServiceConfig: | [DEFAULT] enabled_share_backends = tripleo_ceph host = hostgroup [cephfs] driver_handles_share_servers=False share_backend_name=cephfs 4 share_driver=manila.share.drivers.cephfs.driver.CephFSDriver cephfs_conf_path=/etc/ceph/ceph.conf cephfs_auth_id=openstack cephfs_cluster_name=ceph cephfs_volume_mode=0755 cephfs_protocol_helper_type=CEPHFS networkAttachments: 5 - storage extraMounts: 6 - name: v1 region: r1 extraVol: - propagation: - ManilaShare extraVolType: Ceph volumes: - name: ceph secret: secretName: ceph-conf-files mounts: - name: ceph mountPath: "/etc/ceph" readOnly: true __EOF__
oc patch openstackcontrolplane openstack --type=merge --patch-file=~/<manila.patch>
oc get pods -l service=manila
openstack service list | grep manila
openstack endpoint list | grep manila | 1164c70045d34b959e889846f9959c0e | regionOne | manila | share | True | internal | http://manila-internal.openstack.svc:8786/v1/%(project_id)s | | 63e89296522d4b28a9af56586641590c | regionOne | manilav2 | sharev2 | True | public | https://manila-public-openstack.apps-crc.testing/v2 | | af36c57adcdf4d50b10f484b616764cc | regionOne | manila | share | True | public | https://manila-public-openstack.apps-crc.testing/v1/%(project_id)s | | d655b4390d7544a29ce4ea356cc2b547 | regionOne | manilav2 | sharev2 | True | internal | http://manila-internal.openstack.svc:8786/v2 |
openstack share service list openstack share pool list --detail
openstack share list openstack share snapshot list
cat << __EOF__ > ~/manila.patch spec: manila: enabled: true apiOverride: route: {} template: manilaShares: cephfs: replicas: 1 customServiceConfig: | [DEFAULT] enabled_share_backends = cephfs host = hostgroup [cephfs] driver_handles_share_servers=False share_backend_name=cephfs share_driver=manila.share.drivers.cephfs.driver.CephFSDriver cephfs_conf_path=/etc/ceph/ceph.conf cephfs_auth_id=openstack cephfs_cluster_name=ceph cephfs_protocol_helper_type=NFS cephfs_nfs_cluster_id=cephfs networkAttachments: - storage __EOF__
oc patch openstackcontrolplane openstack --type=merge --patch-file=~/<manila.patch>
sudo pcs resource disable ceph-nfs sudo pcs resource disable ip-<VIP> sudo pcs resource unmanage ceph-nfs sudo pcs resource unmanage ip-<VIP>
USDCONTROLLER1_SSH cat /var/lib/config-data/puppet-generated/ironic/etc/ironic/ironic.conf > ironic.conf
alias openstack="oc exec -t openstackclient -- openstack"
oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' spec: ironic: enabled: true template: rpcTransport: oslo databaseInstance: openstack ironicAPI: replicas: 1 override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer ironicConductors: - replicas: 1 networkAttachments: - baremetal provisionNetwork: baremetal storageRequest: 10G customServiceConfig: | [neutron] cleaning_network=<cleaning network uuid> provisioning_network=<provisioning network uuid> rescuing_network=<rescuing network uuid> inspection_network=<introspection network uuid> [conductor] automated_clean=true ironicInspector: replicas: 1 inspectionNetwork: baremetal networkAttachments: - baremetal dhcpRanges: - name: inspector-0 cidr: 172.20.1.0/24 start: 172.20.1.190 end: 172.20.1.199 gateway: 172.20.1.1 serviceUser: ironic-inspector databaseAccount: ironic-inspector passwordSelectors: database: IronicInspectorDatabasePassword service: IronicInspectorPassword ironicNeutronAgent: replicas: 1 rabbitMqClusterName: rabbitmq secret: osp-secret '
oc wait --for condition=Ready --timeout=300s ironics.ironic.openstack.org ironic
oc wait --for condition=Ready --timeout=300s ironicapis.ironic.openstack.org ironic-api oc wait --for condition=Ready --timeout=300s ironicconductors.ironic.openstack.org ironic-conductor oc wait --for condition=Ready --timeout=300s ironicinspectors.ironic.openstack.org ironic-inspector oc wait --for condition=Ready --timeout=300s ironicneutronagents.ironic.openstack.org ironic-ironic-neutron-agent
openstack subnet set --dns-nameserver 192.168.122.80 provisioning-subnet
openstack baremetal node list
oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' spec: ironic: enabled: true template: databaseInstance: openstack ironicAPI: replicas: 1 customServiceConfig: | [oslo_policy] enforce_scope=false enforce_new_defaults=false '
customServiceConfig: | [oslo_policy] enforce_scope=true enforce_new_defaults=true
openstack baremetal node list -f uuid,provision_state,owner
ADMIN_PROJECT_ID=USD(openstack project show -c id -f value --domain default admin) for node in USD(openstack baremetal node list -f json -c UUID -c Owner | jq -r '.[] | select(.Owner == null) | .UUID'); do openstack baremetal node set --owner USDADMIN_PROJECT_ID USDnode; done
oc patch openstackcontrolplane openstack -n openstack --type=merge --patch ' spec: ironic: enabled: true template: databaseInstance: openstack ironicAPI: replicas: 1 customServiceConfig: | [oslo_policy] enforce_scope=true enforce_new_defaults=true '
openstack endpoint list |grep ironic
openstack baremetal node list
[stack@rhosp17 ~]USD grep -E 'HeatPassword|HeatAuth|HeatStackDomainAdmin' ~/overcloud-deploy/overcloud/overcloud-passwords.yaml HeatAuthEncryptionKey: Q60Hj8PqbrDNu2dDCbyIQE2dibpQUPg2 HeatPassword: dU2N0Vr2bdelYH7eQonAwPfI3 HeatStackDomainAdminPassword: dU2N0Vr2bdelYH7eQonAwPfI3
[stack@rhosp17 ~]USD ansible -i overcloud-deploy/overcloud/config-download/overcloud/tripleo-ansible-inventory.yaml overcloud-controller-0 -m shell -a "grep auth_encryption_key /var/lib/config-data/puppet-generated/heat/etc/heat/heat.conf | grep -Ev '^#|^USD'" -b overcloud-controller-0 | CHANGED | rc=0 >> auth_encryption_key=Q60Hj8PqbrDNu2dDCbyIQE2dibpQUPg2
echo Q60Hj8PqbrDNu2dDCbyIQE2dibpQUPg2 | base64 UTYwSGo4UHFickROdTJkRENieUlRRTJkaWJwUVVQZzIK
oc patch secret osp-secret --type='json' -p='[{"op" : "replace" ,"path" : "/data/HeatAuthEncryptionKey" ,"value" : "UTYwSGo4UHFickROdTJkRENieUlRRTJkaWJwUVVQZzIK"}]' secret/osp-secret patched
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: heat: enabled: true apiOverride: route: {} template: databaseInstance: openstack databaseAccount: heat secret: osp-secret memcachedInstance: memcached passwordSelectors: authEncryptionKey: HeatAuthEncryptionKey service: HeatPassword stackDomainAdminPassword: HeatStackDomainAdminPassword '
oc get Heat,HeatAPI,HeatEngine,HeatCFNAPI NAME STATUS MESSAGE heat.heat.openstack.org/heat True Setup complete NAME STATUS MESSAGE heatapi.heat.openstack.org/heat-api True Setup complete NAME STATUS MESSAGE heatengine.heat.openstack.org/heat-engine True Setup complete NAME STATUS MESSAGE heatcfnapi.heat.openstack.org/heat-cfnapi True Setup complete
oc exec -it openstackclient -- openstack service list -c Name -c Type +------------+----------------+ | Name | Type | +------------+----------------+ | heat | orchestration | | glance | image | | heat-cfn | cloudformation | | ceilometer | Ceilometer | | keystone | identity | | placement | placement | | cinderv3 | volumev3 | | nova | compute | | neutron | network | +------------+----------------+
oc exec -it openstackclient -- openstack endpoint list --service=heat -f yaml - Enabled: true ID: 1da7df5b25b94d1cae85e3ad736b25a5 Interface: public Region: regionOne Service Name: heat Service Type: orchestration URL: http://heat-api-public-openstack-operators.apps.okd.bne-shift.net/v1/%(tenant_id)s - Enabled: true ID: 414dd03d8e9d462988113ea0e3a330b0 Interface: internal Region: regionOne Service Name: heat Service Type: orchestration URL: http://heat-api-internal.openstack-operators.svc:8004/v1/%(tenant_id)s
oc exec -it openstackclient -- openstack orchestration service list -f yaml - Binary: heat-engine Engine ID: b16ad899-815a-4b0c-9f2e-e6d9c74aa200 Host: heat-engine-6d47856868-p7pzz Hostname: heat-engine-6d47856868-p7pzz Status: up Topic: engine Updated At: '2023-10-11T21:48:01.000000' - Binary: heat-engine Engine ID: 887ed392-0799-4310-b95c-ac2d3e6f965f Host: heat-engine-6d47856868-p7pzz Hostname: heat-engine-6d47856868-p7pzz Status: up Topic: engine Updated At: '2023-10-11T21:48:00.000000' - Binary: heat-engine Engine ID: 26ed9668-b3f2-48aa-92e8-2862252485ea Host: heat-engine-6d47856868-p7pzz Hostname: heat-engine-6d47856868-p7pzz Status: up Topic: engine Updated At: '2023-10-11T21:48:00.000000' - Binary: heat-engine Engine ID: 1011943b-9fea-4f53-b543-d841297245fd Host: heat-engine-6d47856868-p7pzz Hostname: heat-engine-6d47856868-p7pzz Status: up Topic: engine Updated At: '2023-10-11T21:48:01.000000'
openstack stack list -f yaml - Creation Time: '2023-10-11T22:03:20Z' ID: 20f95925-7443-49cb-9561-a1ab736749ba Project: 4eacd0d1cab04427bc315805c28e66c9 Stack Name: test-networks Stack Status: CREATE_COMPLETE Updated Time: null
oc create -f - <<EOF apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: cluster-observability-operator namespace: openshift-operators spec: channel: stable installPlanApproval: Automatic name: cluster-observability-operator source: redhat-operators sourceNamespace: openshift-marketplace EOF
oc wait --for jsonpath="{.status.phase}"=Succeeded csv --namespace=openshift-operators -l operators.coreos.com/cluster-observability-operator.openshift-operators
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: telemetry: enabled: true template: ceilometer: passwordSelector: ceilometerService: CeilometerPassword enabled: true secret: osp-secret serviceUser: ceilometer '
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: telemetry: template: metricStorage: enabled: true monitoringStack: alertingEnabled: true scrapeInterval: 30s storage: strategy: persistent retention: 24h persistent: pvcStorageRequest: 20G '
oc get pods -l alertmanager=metric-storage -n openstack NAME READY STATUS RESTARTS AGE alertmanager-metric-storage-0 2/2 Running 0 46s alertmanager-metric-storage-1 2/2 Running 0 46s oc get pods -l prometheus=metric-storage -n openstack NAME READY STATUS RESTARTS AGE prometheus-metric-storage-0 3/3 Running 0 46s
CEILOMETETR_POD=`oc get pods -l service=ceilometer -n openstack | tail -n 1 | cut -f 1 -d' '` exec -t USDCEILOMETETR_POD -c ceilometer-central-agent -- cat /etc/ceilometer/ceilometer.conf
oc get secret ceilometer-config-data -o jsonpath="{.data['polling\.yaml\.j2']}" | base64 -d
oc patch openstackcontrolplane controlplane --type=merge --patch ' spec: telemetry: template: ceilometer: defaultConfigOverwrite: polling.yaml.j2: | --- sources: - name: pollsters interval: 100 meters: - volume.* - image.size enabled: true secret: osp-secret '
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: telemetry: template: logging: enabled: false ipaddr: 172.17.0.80 port: 10514 cloNamespace: openshift-logging '
oc patch openstackcontrolplane openstack --type=merge --patch ' spec: telemetry: enabled: true template: autoscaling: enabled: true aodh: passwordSelector: aodhService: AodhPassword databaseAccount: aodh databaseInstance: openstack secret: osp-secret serviceUser: aodh heatInstance: heat '
AODH_POD=`oc get pods -l service=aodh -n openstack | tail -n 1 | cut -f 1 -d' '` oc exec -t USDAODH_POD -c aodh-api -- cat /etc/aodh/aodh.conf
openstack endpoint list | grep aodh | d05d120153cd4f9b8310ac396b572926 | regionOne | aodh | alarming | True | internal | http://aodh-internal.openstack.svc:8042 | | d6daee0183494d7a9a5faee681c79046 | regionOne | aodh | alarming | True | public | http://aodh-public.openstack.svc:8042 |
openstack alarm create --name high_cpu_alarm --type prometheus --query "(rate(ceilometer_cpu{resource_name=~'cirros'})) * 100" --alarm-action 'log://' --granularity 15 --evaluation-periods 3 --comparison-operator gt --threshold 7000000000
openstack alarm list +--------------------------------------+------------+------------------+-------------------+----------+ | alarm_id | type | name | state | severity | enabled | +--------------------------------------+------------+------------------+-------------------+----------+ | 209dc2e9-f9d6-40e5-aecc-e767ce50e9c0 | prometheus | prometheus_alarm | ok | low | True | +--------------------------------------+------------+------------------+-------------------+----------+
ssh_cmd=ssh -F ssh.config standalone container_engine=podman connection=ssh remote_config_path=/tmp/tripleo
chown ospng:ospng /etc/os-diff/config.yaml
service name and file location services: # Service name keystone: # Bool to enable/disable a service (not implemented yet) enable: true # Pod name, in both OCP and podman context. # It could be strict match or will only just grep the podman_name # and work with all the pods which matched with pod_name. # To enable/disable use strict_pod_name_match: true/false podman_name: keystone pod_name: keystone container_name: keystone-api # pod options # strict match for getting pod id in TripleO and podman context strict_pod_name_match: false # Path of the config files you want to analyze. # It could be whatever path you want: # /etc/<service_name> or /etc or /usr/share/<something> or even / # @TODO: need to implement loop over path to support multiple paths such as: # - /etc # - /usr/share path: - /etc/ - /etc/keystone - /etc/keystone/keystone.conf - /etc/keystone/logging.conf
services: ovs_external_ids: hosts: 1 - standalone service_command: "ovs-vsctl list Open_vSwitch . | grep external_ids | awk -F ': ' '{ print USD2; }'" 2 cat_output: true 3 path: - ovs_external_ids.json config_mapping: 4 ovn-bridge-mappings: edpm_ovn_bridge_mappings 5 ovn-bridge: edpm_ovn_bridge ovn-encap-type: edpm_ovn_encap_type ovn-monitor-all: ovn_monitor_all ovn-remote-probe-interval: edpm_ovn_remote_probe_interval ovn-ofctrl-wait-before-clear: edpm_ovn_ofctrl_wait_before_clear
os-diff diff ovs_external_ids.json edpm.crd --crd --service ovs_external_ids
services: yum_config: hosts: - undercloud - controller_1 - compute_1 - compute_2 service_command: "cat /etc/yum.conf" cat_output: true path: - yum.conf
will only update the /etc/os-diff/config.yaml os-diff pull --update-only
will update the /etc/os-diff/config.yaml and pull configuration os-diff pull --update
will update the /etc/os-diff/config.yaml and pull configuration os-diff pull
/tmp/tripleo/
▾ tmp/ ▾ tripleo/ ▾ glance/ ▾ keystone/
ServicesToStart=("tripleo_horizon.service" "tripleo_keystone.service" "tripleo_barbican_api.service" "tripleo_barbican_worker.service" "tripleo_barbican_keystone_listener.service" "tripleo_cinder_api.service" "tripleo_cinder_api_cron.service" "tripleo_cinder_scheduler.service" "tripleo_cinder_volume.service" "tripleo_cinder_backup.service" "tripleo_glance_api.service" "tripleo_manila_api.service" "tripleo_manila_api_cron.service" "tripleo_manila_scheduler.service" "tripleo_neutron_api.service" "tripleo_placement_api.service" "tripleo_nova_api_cron.service" "tripleo_nova_api.service" "tripleo_nova_conductor.service" "tripleo_nova_metadata.service" "tripleo_nova_scheduler.service" "tripleo_nova_vnc_proxy.service" "tripleo_aodh_api.service" "tripleo_aodh_api_cron.service" "tripleo_aodh_evaluator.service" "tripleo_aodh_listener.service" "tripleo_aodh_notifier.service" "tripleo_ceilometer_agent_central.service" "tripleo_ceilometer_agent_compute.service" "tripleo_ceilometer_agent_ipmi.service" "tripleo_ceilometer_agent_notification.service" "tripleo_ovn_cluster_north_db_server.service" "tripleo_ovn_cluster_south_db_server.service" "tripleo_ovn_cluster_northd.service") PacemakerResourcesToStart=("galera-bundle" "haproxy-bundle" "rabbitmq-bundle" "openstack-cinder-volume" "openstack-cinder-backup" "openstack-manila-share") echo "Starting systemd OpenStack services" for service in USD{ServicesToStart[*]}; do for i in {1..3}; do SSH_CMD=CONTROLLERUSD{i}_SSH if [ ! -z "USD{!SSH_CMD}" ]; then if USD{!SSH_CMD} sudo systemctl is-enabled USDservice &> /dev/null; then echo "Starting the USDservice in controller USDi" USD{!SSH_CMD} sudo systemctl start USDservice fi fi done done echo "Checking systemd OpenStack services" for service in USD{ServicesToStart[*]}; do for i in {1..3}; do SSH_CMD=CONTROLLERUSD{i}_SSH if [ ! -z "USD{!SSH_CMD}" ]; then if USD{!SSH_CMD} sudo systemctl is-enabled USDservice &> /dev/null; then if ! USD{!SSH_CMD} systemctl show USDservice | grep ActiveState=active >/dev/null; then echo "ERROR: Service USDservice is not running on controller USDi" else echo "OK: Service USDservice is running in controller USDi" fi fi fi done done echo "Starting pacemaker OpenStack services" for i in {1..3}; do SSH_CMD=CONTROLLERUSD{i}_SSH if [ ! -z "USD{!SSH_CMD}" ]; then echo "Using controller USDi to run pacemaker commands" for resource in USD{PacemakerResourcesToStart[*]}; do if USD{!SSH_CMD} sudo pcs resource config USDresource &>/dev/null; then echo "Starting USDresource" USD{!SSH_CMD} sudo pcs resource enable USDresource else echo "Service USDresource not present" fi done break fi done echo "Checking pacemaker OpenStack services" for i in {1..3}; do SSH_CMD=CONTROLLERUSD{i}_SSH if [ ! -z "USD{!SSH_CMD}" ]; then echo "Using controller USDi to run pacemaker commands" for resource in USD{PacemakerResourcesToStop[*]}; do if USD{!SSH_CMD} sudo pcs resource config USDresource &>/dev/null; then if USD{!SSH_CMD} sudo pcs resource status USDresource | grep Started >/dev/null; then echo "OK: Service USDresource is started" else echo "ERROR: Service USDresource is stopped" fi fi done break fi done
sudo pcs constraint order start ceph-nfs then openstack-manila-share kind=Optional id=order-ceph-nfs-openstack-manila-share-Optional sudo pcs constraint colocation add openstack-manila-share with ceph-nfs score=INFINITY id=colocation-openstack-manila-share-ceph-nfs-INFINITY
oc delete --ignore-not-found=true --wait=false openstackcontrolplane/openstack oc patch openstackcontrolplane openstack --type=merge --patch ' metadata: finalizers: [] ' || true while oc get pod | grep rabbitmq-server-0; do sleep 2 done while oc get pod | grep openstack-galera-0; do sleep 2 done oc delete --ignore-not-found=true --wait=false pod mariadb-copy-data oc delete --ignore-not-found=true --wait=false pvc mariadb-data oc delete --ignore-not-found=true --wait=false pod ovn-copy-data oc delete --ignore-not-found=true secret osp-secret