Chapter 5. Configuring the Object Storage service (swift) Red Hat OpenStack Platform Object Storage (swift) stores its objects, or data, in containers. Containers are similar to directories in a file system although they cannot be nested. Containers provide an easy way for users to store any kind of unstructured data. For example, objects can include photos, text files, or images. Stored objects are not compressed. 5.1. Object Storage rings The Object Storage service (swift) uses a data structure called the ring to distribute partition space across the cluster. This partition space is core to the data durability engine in the Object Storage service. With rings, the Object Storage service can quickly and easily synchronize each partition across the cluster. Rings contain information about Object Storage partitions and how partitions are distributed among the different nodes and disks. When any Object Storage component interacts with data, a quick lookup is performed locally in the ring to determine the possible partitions for each object. The Object Storage service has three rings to store the following types of data: Account information Containers, to facilitate organizing objects under an account Object replicas 5.1.1. Checking cluster health The Object Storage service runs many processes in the background to ensure long-term data availability, durability, and persistence. For example: Auditors constantly re-read database and object files and compare them using checksums to make sure there is no silent bit-rot. Any database or object file that no longer matches its checksum is quarantined and becomes unreadable on that node. The replicators then copy one of the other replicas to make the local copy available again. Objects and files can disappear when you replace disks or nodes or when objects are quarantined. When this happens, replicators copy missing objects or database files to one of the other nodes. The Object Storage service includes a tool called swift-recon that collects data from all nodes and checks the overall cluster health. Procedure Log in to one of the Controller nodes. Run the following command: Optional: Use the --all option to return additional output. This command queries all servers on the ring for the following data: Async pendings: If the cluster load is too high and processes cannot update database files fast enough, some updates occur asynchronously. These numbers decrease over time. Replication metrics: Review the replication timestamps; full replication passes happen frequently with few errors. An old entry, for example, an entry with a timestamp from six months ago, indicates that replication on the node has not completed in the last six months. Ring md5sums: This ensures that all ring files are consistent on all nodes. swift.conf md5sums: This ensures that all configuration files are consistent on all nodes. Quarantined files: There must be no, or very few, quarantined files for all nodes. Time-sync: All nodes must be synchronized. 5.1.2. Increasing ring partition power The ring power determines the partition to which a resource, such as an account, container, or object, is mapped. The partition is included in the path under which the resource is stored in a back end file system. Therefore, changing the partition power requires relocating resources to new paths in the back end file systems. In a heavily populated cluster, a relocation process is time consuming. To avoid downtime, relocate resources while the cluster is still operating. You must do this without temporarily losing access to data or compromising the performance of processes, such as replication and auditing. For assistance with increasing ring partition power, contact Red Hat support. 5.1.3. Custom rings As technology advances and demands for storage capacity increase, creating custom rings is a way to update existing Object Storage clusters. When you add new nodes to a cluster, their characteristics might differ from those of the original nodes. Without custom adjustments, the larger capacity of the new nodes may be underused. Or, if weights change in the rings, data dispersion can become uneven, which reduces safety. Automation might not keep pace with future technology trends. For example, some older Object Storage clusters in use today originated before SSDs were available. The ring builder helps manage Object Storage as clusters grow and technologies evolve. For assistance with creating custom rings, contact Red Hat support. 5.2. Customize the Object Storage service Depending on the requirements of your Red Hat OpenStack Platform (RHOSP) environment, you might want to customize some of the default settings of the Object Storage service (swift) to optimize your deployment performance. 5.2.1. Configuring fast-post By default, the Object Storage service copies an object whole whenever any part of its metadata changes. You can prevent this by using the fast-post feature. The fast-post feature saves time when you change the content types of multiple large objects. To enable the fast-post feature, disable the object_post_as_copy option on the Object Storage proxy service. Procedure Edit swift_params.yaml : Include the parameter file when you deploy or update the overcloud: 5.2.2. Enabling at-rest encryption By default, objects uploaded to the Object Storage service (swift) are unencrypted. Because of this, it is possible to access objects directly from the file system. This can present a security risk if disks are not properly erased before they are discarded. You can use OpenStack Key Manager (barbican) to encrypt at-rest swift objects. For more information, see Encrypt at-rest swift objects . 5.2.3. Deploying a standalone Object Storage cluster You can use the composable role concept to deploy a standalone Object Storage service (swift) cluster with the bare minimum of additional services (for example Keystone, HAProxy). The Creating a roles_data File section has information on roles. Procedure Copy the roles_data.yaml from /usr/share/openstack-tripleo-heat-templates . Edit the new file. Remove unneeded controller roles, for example Aodh*, Ceilometer*, Ceph*, Cinder*, Glance*, Heat*, Ironic*, Manila*, Mistral*, Nova*, Octavia*, Swift*. Locate the ObjectStorage role within roles_data.yaml . Copy this role to a new role within that same file and name it ObjectProxy . Replace SwiftStorage with SwiftProxy in this role. The example roles_data.yaml file below shows sample roles. Deploy the overcloud with your regular openstack deploy command, including the new roles. 5.2.4. Using external SAN disks By default, when the Red Hat OpenStack Platform (RHOSP) director deploys the Object Storage service (swift), Object Storage is configured and optimized to use independent local disks. This configuration ensures that the workload is distributed across all disks, which helps minimize performance impacts during node failure or other system issues. In similar performance-impacting events, an environment that uses a single SAN can experience decreased performance across all LUNs. The Object Storage service cannot mitigate performance issues in an environment that uses SAN disks. Therefore, Red Hat strongly recommends that you use additional local disks for Object Storage instead to meet performance and disk space requirements. For more information, see Configuration recommendations for the Object Storage service (swift) in the Deployment Recommendations for Specific Red Hat OpenStack Platform Services guide. Using an external SAN for Object Storage requires evaluation on a per-case basis. For more information, contact Red Hat Support. Important If you choose to use an external SAN for Object Storage, be aware of the following conditions: The Object Storage service stores telemetry data and Image service (glance) images by default. Glance images require more disk space, but from a performance perspective, the impact of storing glance images impacts performance less than storing telemetry data. Storing and processing telemetry data requires increased performance. Red Hat does not provide support for issues related to performance that result from using an external SAN for Object Storage. Red Hat does not provide support for issues that arise outside of the core Object Storage service offering. For support with high availability and performance, contact your storage vendor. Red Hat does not test SAN solutions with the Object Storage service. For more information about compatibility, guidance, and support for third-party products, contact your storage vendor. Red Hat recommends that you evaluate and test performance demands with your deployment. To confirm that your SAN deployment is tested, supported, and meets your performance requirements, contact your storage vendor. Procedure This template is an example of how to use two devices ( /dev/mapper/vdb and /dev/mapper/vdc ) for Object Storage: 5.3. Adding new Object Storage nodes To add new Object Storage (swift) nodes to your cluster, you must increase the node count, update the rings, and synchronize the changes. The following example procedure increases three nodes to four. Procedure Log in to the undercloud node as the stack user and source the stackrc credentials file: Use the ObjectStorageCount parameter to increase the Object Storage count by 1. This parameter is usually located in the node-info.yaml file, which is the environment file that contains your node counts: Add the IP address of the new node to the ObjectStorageIPs parameter in the node-info.yaml file: Create an environment file, for example hostname-map.yaml , and add the host name of the new storage node, for example, overcloud-objectstorage-4 to the HostnameMap parameter: Add the hardware and power management information about the new node to the node definition template. The node definition template is a file that was manually created to register overcloud nodes during initial overcloud configuration. For example, the template might be: /home/stack/nodes.json . The following example uses JSON format, although if your template is a YAML file, add the information according to YAML formatting and attributes. For more information about the node definition template, see Registering nodes for the overcloud in the Director Installation and Usage guide. Add the capabilities information about the new node to the node definition template: Import the node to the overcloud and perform introspection on the node: Add the root disk serial number to the new node: Replace <disk_serial_num> and <node_UUID> according to your new node. Include the files you created that contain your new node configurations in the deployment command with any other environment files that are relevant to your environment: 5.3.1. Updating and rebalancing the Object Storage rings The Object Storage service (swift) stores a copy of the ring files on the undercloud to deploy new Storage nodes and replace existing Controller and Object Storage nodes. Use this copy to modify overcloud ring files and distribute them across your nodes. Procedure Log in to the undercloud as the stack user and source the stackrc credentials file: Create the following directory: Download the overcloud rings builder file to the new directory: Extract the rings: Create a directory to back up the original version of the rings: Object Storage ring files are located in the etc/swift folder. Collect the values for the following variables according to your device details: <device_name> : <node_ip> : <port> The default port is 600x . If you altered the default, use the applicable port. <builder_file> The builder file name from step 3. <weight> and <zone> variables are user-defined. Use swift-ring-builder to add and update the new node to the existing rings. Replace the variables according to the device details. Upload the modified ring files to the Object Storage service on the undercloud: 5.3.2. Syncing node changes and migrating data You must deliver the changed ring files to the Object Storage (swift) containers after you copy new ring files to their correct folders. Important Do not migrate all of the data at the same time. Migrate the data in 10% increments. For example, configure the weight of the source device to equal 90.0 and the target device to equal 10.0. Then configure the weight of the source device to equal 80.0 and 20.0. Continue to incrementally migrate the data until you complete the process. During the migration, if you move all of the data at the same time, the old data is on the source device but the ring points to the new target device for all replicas. Until the replicators have moved all of the data to the target device, the data is inaccessible. Important During migration, the Object Storage rings reassign the location of data, and then the replicator moves the data to the new location. As cluster activity increases, the process of replication slows down due to increased load. The larger the cluster, the longer a replication pass takes to complete. This is the expected behavior, but it can result in 404 errors in the log files if a client accesses the data that is currently being relocated. When a proxy attempts to retrieve data from a new location, but the data is not yet in the new location, swift-proxy reports a 404 error in the log files. When the migration is gradual, the proxy accesses replicas that are not being moved and no error occurs. And when the proxy attempts to retrieve the data from an alternative replica, 404 errors in log files resolve. To confirm that the replication process is progressing, refer to the replication logs. The Object Storage service issues replication logs every 5 minutes. Procedure Use a script similar to the following example to distribute ring files from the undercloud node to a specified Controller node and restart the Object Storage service containers on those nodes: To confirm that the data is being moved to the new disk, run the following command on the new storage node: 5.4. Removing Object Storage nodes There are two ways to remove an Object Storage (swift) node. Choose one of the following options depending on the complexity of your cluster or the quantity of data that it contains: Simple removal: This method removes the node in one action and is appropriate for an efficiently-powered cluster with smaller quantities of data. See Removing an Object Storage node in one action . Incremental removal: Alter the rings to decrease the weight of the disks on the node that you want to remove. This method is appropriate if you want to minimize impact on storage network usage or if your cluster contains larger quantities of data. See Altering rings to incrementally remove an Object Storage node . 5.4.1. Removing an Object Storage node in one action Use this method to remove a node in an efficiently-powered cluster with small quantities of data. The following example procedure decreases four nodes to three. Procedure Log in to the undercloud as the stack user and source the stackrc credentials file: Use the ObjectStorageCount parameter to decrease the Object Storage count by 1. This parameter is usually located in node-info.yaml , which is the environment file that contains your node counts: Delete the IP address of the node that you removed from the ObjectStorageIPs parameter in the node-info.yaml file: Create an environment file, for example, remove-object-node.yaml . This file identifies and removes the Object Storage node that you specify. The following example content specifies the removal of overcloud-objectstorage-1 : Include both the node-info.yaml and remove-object-node.yaml files in the deployment command: Director deletes the Object Storage node from the overcloud and updates the rest of the nodes on the overcloud to accommodate the node removal. List active nodes to verify that you removed the correct node: On the undercloud node, delete the remove-object-node.yaml file so that it is not included in future redeployments: 5.4.2. Altering rings to incrementally remove an Object Storage node Use this method if you need to minimize the impact on the storage network while you remove a node, or if your cluster contains large quantities of data. To alter the storage rings to decrease the weight of the disks on the node that you want to remove, first complete the procedures listed in Prerequisites . The following example decreases the nodes from four to three. Prerequisites Updated and rebalanced Object Storage rings. See Updating and rebalancing the Object Storage rings . Changes in the Object Storage rings are synchronized. See Syncing the changes and migrating data . Procedure Log in to the undercloud as the stack user and source the stackrc credentials file: Use the ObjectStorageCount parameter to decrease the Object Storage count by 1. This parameter is usually located in node-info.yaml , which is the environment file that contains your node counts: Delete the IP address of the node that you removed from the ObjectStorageIPs parameter in the node-info.yaml file: Create an environment file, for example, remove-object-node.yaml . This file identifies and removes the Object Storage node that you specify. The following example content specifies the removal of overcloud-objectstorage-1 : Include both the node-info.yaml and remove-object-node.yaml files in the deployment command: Director deletes the Object Storage node from the overcloud and updates the rest of the nodes on the overcloud to accommodate the node removal. List active nodes to verify that you removed the correct node: On the undercloud node, delete the remove-object-node.yaml file so that it is not included in future redeployments: 5.5. Replacing Object Storage nodes Follow the instructions in this section to understand how to replace Object Storage nodes without impact to the integrity of the cluster. This example involves a three-node Object Storage cluster in which you want to replace the node overcloud-objectstorage-1 node. The goal of the procedure is to add one more node and then remove the overcloud-objectstorage-1 node. The new node replaces the overcloud-objectstorage-1 node. Procedure Increase the Object Storage count using the ObjectStorageCount parameter. This parameter is usually located in node-info.yaml , which is the environment file that contains your node counts: The ObjectStorageCount parameter defines the quantity of Object Storage nodes in your environment. In this example, scale the quantity of Object Storage nodes from 3 to 4 . Run the deployment command with the updated ObjectStorageCount parameter: After the deployment command completes, the overcloud contains an additional Object Storage node. Replicate data to the new node. Before you remove a node, in this case, overcloud-objectstorage-1 , wait for a replication pass to finish on the new node. Check the replication pass progress in the /var/log/swift/swift.log file. When the pass finishes, the Object Storage service should log entries similar to the following example: To remove the old node from the ring, reduce the ObjectStorageCount parameter to omit the old node. In this example, reduce the ObjectStorageCount parameter to 3 : Create a new environment file named remove-object-node.yaml . This file identifies and removes the specified Object Storage node. The following content specifies the removal of overcloud-objectstorage-1 : Include both the node-info.yaml and remove-object-node.yaml files in the deployment command: Director deletes the Object Storage node from the overcloud and updates the rest of the nodes on the overcloud to accommodate the node removal. Important Include all environment files and options from your initial overcloud creation. This includes the same scale parameters for non-Compute nodes. 5.6. Basic container management in the Object Storage service To help with organization in the Object Storage service (swift), you can use pseudo folders. These folders are logical devices that can contain objects and be nested. For example, you might create an Images folder in which to store pictures and a Media folder in which to store videos. You can create one or more containers in each project, and one or more objects or pseudo folders in each container. 5.6.1. Creating a container in the Object Storage service Use the dashboard to create a container. Procedure In the dashboard, select Project > Object Store > Containers . Click Create Container . Specify the Container Name , and select one of the following in the Container Access field. Type Description Private Limits access to a user in the current project. Public Permits API access to anyone with the public URL. However, in the dashboard, project users cannot see public containers and data from other projects. Click Create Container . Optional: New containers use the default storage policy. If you have multiple storage policies defined, for example, a default policy and another policy that enables erasure coding, you can configure a container to use a non-default storage policy: Replace <policy> with the name or alias of the policy that you want the container to use, and replace <container_name> with the name of the container. 5.6.2. Creating pseudo folders for containers in the Object Storage service Use the dashboard to create a pseudo folder for a container. Procedure In the dashboard, select Project > Object Store > Containers . Click the name of the container to which you want to add the pseudo folder. Click Create Pseudo-folder . Specify the name in the Pseudo-folder Name field, and click Create . 5.6.3. Deleting a container in the Object Storage service Use the dashboard to delete a container. Procedure In the dashboard, select Project > Object Store > Containers . Browse for the container in the Containers section, and ensure that all objects are deleted. For more information, see Deleting an object . Select Delete Container in the container arrow menu. Click Delete Container to confirm the container removal. 5.6.4. Uploading an object to the Object Storage service If you do not upload an actual file, the object is still created as a placeholder that you can use later to upload the file. Procedure In the dashboard, select Project > Object Store > Containers . Click the name of the container in which you want to place the uploaded the object. If a pseudo folder already exists in the container, you can click its name. Browse for your file, and click Upload Object . Specify a name in the Object Name field: You can specify pseudo folders in the name by using a / character,for example, Images/myImage.jpg . If the specified folder does not already exist, it is created when the object is uploaded. A name that is not unique to the location, that is, the object already exists, overwrites the contents of the object. Click Upload Object . 5.6.5. Copying an object within the Object Storage service Use the dashboard to copy an object. Procedure In the dashboard, select Project > Object Store > Containers . Click the name of the object's container or folder (to display the object). Click Upload Object . Browse for the file to be copied, and select Copy in its arrow menu. Specify the following: Field Description Destination container Target container for the new object. Path Pseudo-folder in the destination container; if the folder does not already exist, it is created. Destination object name New object's name. If you use a name that is not unique to the location (that is, the object already exists), it overwrites the object's contents. Click Copy Object . 5.6.6. Deleting an object from the Object Storage service Use the dashboard to delete an object. Procedure In the dashboard, select Project > Object Store > Containers . Browse for the object, and select Delete Object in its arrow menu. Click Delete Object to confirm the object is removed.

Commands:
sudo podman exec -it -u swift swift_object_server /usr/bin/swift-recon -arqlT --md5 ======================================================================--> Starting reconnaissance on 3 hosts (object) ======================================================================[2018-12-14 14:55:47] Checking async pendings [async_pending] - No hosts returned valid data. ======================================================================[2018-12-14 14:55:47] Checking on replication [replication_failure] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%, no_result: 0, reported: 3 [replication_success] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%, no_result: 0, reported: 3 [replication_time] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%, no_result: 0, reported: 3 [replication_attempted] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%, no_result: 0, reported: 3 Oldest completion was 2018-12-14 14:55:39 (7 seconds ago) by 172.16.3.186:6000. Most recent completion was 2018-12-14 14:55:42 (4 seconds ago) by 172.16.3.174:6000. ======================================================================[2018-12-14 14:55:47] Checking load averages [5m_load_avg] low: 1, high: 2, avg: 2.1, total: 6, Failed: 0.0%, no_result: 0, reported: 3 [15m_load_avg] low: 2, high: 2, avg: 2.6, total: 7, Failed: 0.0%, no_result: 0, reported: 3 [1m_load_avg] low: 0, high: 0, avg: 0.8, total: 2, Failed: 0.0%, no_result: 0, reported: 3 ======================================================================[2018-12-14 14:55:47] Checking ring md5sums 3/3 hosts matched, 0 error[s] while checking hosts. ======================================================================[2018-12-14 14:55:47] Checking swift.conf md5sum 3/3 hosts matched, 0 error[s] while checking hosts. ======================================================================[2018-12-14 14:55:47] Checking quarantine [quarantined_objects] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%, no_result: 0, reported: 3 [quarantined_accounts] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%, no_result: 0, reported: 3 [quarantined_containers] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%, no_result: 0, reported: 3 ======================================================================[2018-12-14 14:55:47] Checking time-sync 3/3 hosts matched, 0 error[s] while checking hosts. ======================================================================
cat > swift_params.yaml << EOF parameter_defaults: ExtraConfig: swift::proxy::copy::object_post_as_copy: False EOF
openstack overcloud deploy [... previous args ...] -e swift_params.yaml
- name: Controller description: | Controller role that has all the controller services loaded and handles Database, Messaging and Network functions. CountDefault: 1 tags: - primary - controller networks: - External - InternalApi - Storage - StorageMgmt - Tenant HostnameFormatDefault: '%stackname%-controller-%index%' ServicesDefault: - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Clustercheck - OS::TripleO::Services::Docker - OS::TripleO::Services::Ec2Api - OS::TripleO::Services::Etcd - OS::TripleO::Services::HAproxy - OS::TripleO::Services::Keepalived - OS::TripleO::Services::Kernel - OS::TripleO::Services::Keystone - OS::TripleO::Services::Memcached - OS::TripleO::Services::MySQL - OS::TripleO::Services::MySQLClient - OS::TripleO::Services::Ntp - OS::TripleO::Services::Pacemaker - OS::TripleO::Services::RabbitMQ - OS::TripleO::Services::Securetty - OS::TripleO::Services::Snmp - OS::TripleO::Services::Sshd - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Vpp - name: ObjectStorage CountDefault: 1 description: | Swift Object Storage node role networks: - InternalApi - Storage - StorageMgmt disable_upgrade_deployment: True ServicesDefault: - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::Docker - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::Kernel - OS::TripleO::Services::MySQLClient - OS::TripleO::Services::Ntp - OS::TripleO::Services::Securetty - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Sshd - OS::TripleO::Services::SwiftRingBuilder - OS::TripleO::Services::SwiftStorage - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - name: ObjectProxy CountDefault: 1 description: | Swift Object proxy node role networks: - InternalApi - Storage - StorageMgmt disable_upgrade_deployment: True ServicesDefault: - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::Docker - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::Kernel - OS::TripleO::Services::MySQLClient - OS::TripleO::Services::Ntp - OS::TripleO::Services::Securetty - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Sshd - OS::TripleO::Services::SwiftRingBuilder - OS::TripleO::Services::SwiftProxy - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages
openstack overcloud deploy --templates -r roles_data.yaml -e [...]
parameter_defaults: SwiftMountCheck: true SwiftUseLocalDir: false SwiftRawDisks: {"vdb": {"base_dir":"/dev/mapper/"}, "vdc": {"base_dir":"/dev/mapper/"}}
source ~/stackrc
parameter_defaults: ObjectStorageCount: 4
ObjectStorageIPs: <ip_address>
parameter_defaults: HostnameMap: overcloud-objectstorage-4: overcloud-objectstorage-4
"ports":[ "bb:bb:bb:bb:bb:bb" ], "name":"node01", "cpu":"4", "memory":"6144", "disk":"40", "arch":"x86_64", "pm_type":"pxe_ipmitool", "pm_user":"admin", "pm_password":"p@55w0rd!", "pm_addr":"192.168.24.205"
"capabilities": "profile:swift-storage,boot_option:local,boot_mode:uefi,node:objectstorage-4",
openstack overcloud node import ~/nodes.yaml openstack overcloud node introspect objectstorage-4 --provide
openstack baremetal introspection data save objectstorage-4 | jq ".inventory.disks" openstack baremetal node set --property root_device='{"serial": "<disk_serial_num>"}' <node_UUID>
openstack overcloud deploy --templates ... -e <existing_overcloud_environment_files> -e node-info.yaml -e hostname-map.yaml \
source ~/stackrc
mkdir temp && cd temp/
openstack object save overcloud-swift-rings swift-rings.tar.gz
tar xzvf swift-rings.tar.gz
mkdir backup && cp swift-rings.tar.gz backup/
openstack baremetal introspection data save <node_name> | jq ".inventory.disks"
openstack server show <node_name>
swift-ring-builder etc/swift/<builder_file> add <zone>-<node_ip>:<port>/<device_name> <weight>
tar cvzf swift-rings.tar.gz etc/ openstack object create overcloud-swift-rings swift-rings.tar.gz
cd etc/swift/ for j in 8 11 23; do ssh heat-admin@192.168.24."USDj" "rm *.ring.gz" for i in account.ring.gz container.ring.gz object.ring.gz; do scp USDi heat-admin@192.168.24."USDj":~/ ssh heat-admin@192.168.24."USDj" "sudo cp -f "USDi" /var/lib/config-data/puppet-generated/swift/etc/swift/" ssh heat-admin@192.168.24."USDj" "sudo chown root:root /var/lib/config-data/puppet-generated/swift/etc/swift/"USDi"" ssh heat-admin@192.168.24."USDj" "sudo restorecon /var/lib/config-data/puppet-generated/swift/etc/swift/"USDi"" done ssh heat-admin@192.168.24."USDj" "rm *.builder" for i in account.builder container.builder object.builder; do scp USDi heat-admin@192.168.24."USDj":~/ ssh heat-admin@192.168.24."USDj" "cat "USDi" | sudo tee /var/lib/config-data/puppet-generated/swift/etc/swift/"USDi" >/dev/null" done ssh heat-admin@192.168.24."USDj" 'for k in `sudo podman ps --format "{{.Names}}" | grep swift`; do sudo podman restart USDk; done' done cd ../../
sudo grep -i replication /var/log/container/swift/swift.log
source ~/stackrc
parameter_defaults: ObjectStorageCount: 3
ObjectStorageIPs: <ip_address>
parameter_defaults: ObjectStorageRemovalPolicies: [{'resource_list': ['1']}]
openstack overcloud deploy --templates ... -e <existing_overcloud_environment_files> -e node-info.yaml -e remove-object-node.yaml ...
openstack server list
rm <file_path>/remove-object-node.yaml
source ~/stackrc
parameter_defaults: ObjectStorageCount: 3
ObjectStorageIPs: <ip_address>
parameter_defaults: ObjectStorageRemovalPolicies: [{'resource_list': ['1']}]
openstack overcloud deploy --templates ... -e <existing_overcloud_environment_files> -e node-info.yaml -e remove-object-node.yaml ...
openstack server list
rm <file_path>/remove-object-node.yaml
parameter_defaults: ObjectStorageCount: 4
source ~/stackrc (undercloud) USD openstack overcloud deploy --templates -e node-info.yaml <environment_files>
Mar 29 08:49:05 localhost *object-server: Object replication complete.* Mar 29 08:49:11 localhost *container-server: Replication run OVER* Mar 29 08:49:13 localhost *account-server: Replication run OVER*
parameter_defaults: ObjectStorageCount: 3
parameter_defaults: ObjectStorageRemovalPolicies: [{'resource_list': ['1']}]
(undercloud) USD openstack overcloud deploy --templates -e node-info.yaml <environment_files> -e remove-object-node.yaml
swift post -H "X-Storage-Policy:<policy>" <container_name>