Chapter 6. Advanced configuration As a storage administrator, you can configure some of the more advanced features of the Ceph Object Gateway. You can configure a multi-site Ceph Object Gateway and integrate it with directory services, such as Microsoft Active Directory and OpenStack Keystone service. Prerequisites A healthy running Red Hat Ceph Storage cluster. 6.1. Configure LDAP and Ceph Object Gateway Perform the following steps to configure the Red Hat Directory Server to authenticate Ceph Object Gateway users. 6.1.1. Installing a Red Hat Directory Server Red Hat Directory Server should be installed on a Red Hat Enterprise Linux 8 with a graphical user interface (GUI) in order to use the Java Swing GUI Directory and Administration consoles. However, Red Hat Directory Server can still be serviced exclusively from the command line interface (CLI). Prerequisites Red Hat Enterprise Linux (RHEL) is installed on the server. The Directory Server node's FQDN is resolvable using DNS or the /etc/hosts file. Register the Directory Server node to the Red Hat subscription management service. A valid Red Hat Directory Server subscription is available in your Red Hat account. Procedure Follow the instructions in Chapter 1 and in Chapter 2 of the Red Hat Directory Server Installation Guide . Additional Resources See the Red Hat Director Server Installation Guide for more details. 6.1.2. Configure the Directory Server firewall On the LDAP host, make sure that the firewall allows access to the Directory Server's secure ( 636 ) port, so that LDAP clients can access the Directory Server. Leave the default unsecure port ( 389 ) closed. 6.1.3. Label ports for SELinux To ensure SELinux does not block requests, label the ports for SELinux. For details see the Changing Directory Server Port Numbers section in the Administration Guide for Red Hat Directory Server 10. 6.1.4. Configure LDAPS The Ceph Object Gateway uses a simple ID and password to authenticate with the LDAP server, so the connection requires an SSL certificate for LDAP. To configure the Directory Server for LDAP, see the Configuring Secure Connections chapter in the Administration Guide for Red Hat Directory Server 11. Once the LDAP is working, configure the Ceph Object Gateway servers to trust the Directory Server's certificate. Extract/Download a PEM-formatted certificate for the Certificate Authority (CA) that signed the LDAP server's SSL certificate. Confirm that /etc/openldap/ldap.conf does not have TLS_REQCERT set. Confirm that /etc/openldap/ldap.conf contains a TLS_CACERTDIR /etc/openldap/certs setting. Use the certutil command to add the AD CA to the store at /etc/openldap/certs. For example, if the CA is "msad-frog-MSAD-FROG-CA", and the PEM-formatted CA file is ldap.pem , use the following command: Example Update SELinux on all remote LDAP sites: Example Note This still has to be set even if SELinux is in permissive mode. Make the certs database world-readable: Example Connect to the server using the "ldapwhoami" command as a non-root user. Example The -d 9 option will provide debugging information in case something went wrong with the SSL negotiation. 6.1.5. Check if the gateway user exists Before creating the gateway user, ensure that the Ceph Object Gateway does not already have the user. Example The user name should NOT be in this list of users. 6.1.6. Add a gateway user Create a Ceph Object Gateway user to user LDAP. Procedure Create an LDAP user for the Ceph Object Gateway, and make a note of the binddn . Since the Ceph object gateway uses the ceph user, consider using ceph as the username. The user needs to have permissions to search the directory. The Ceph Object Gateway binds to this user as specified in rgw_ldap_binddn . Test to ensure that the user creation worked. Where ceph is the user ID under People and example.com is the domain, you can perform a search for the user. On each gateway node, create a file for the user's secret. For example, the secret might get stored in a file entitled /etc/bindpass . For security, change the owner of this file to the ceph user and group to ensure it is not globally readable. Add the rgw_ldap_secret option: Syntax Example Patch the bind password file to the Ceph Object Gateway container and reapply the Ceph Object Gateway specification: Example Note /etc/bindpass is not shipped automatically with Red Hat Ceph Storage and you need to ensure that the content is available on all the possible Ceph Object Gateway instance nodes. 6.1.7. Configure the gateway to use LDAP Change the Ceph configuration with the following commands on all the Ceph nodes: Syntax Example Restart the Ceph Object Gateway. Note Use the output from the ceph orch ps command, under the NAME column, to get the SERVICE_TYPE . ID information. To restart the Ceph Object Gateway on an individual node in the storage cluster: Syntax Example To restart the Ceph Object Gateways on all nodes in the storage cluster: Syntax Example 6.1.8. Using a custom search filter You can create a custom search filter to limit user access by using the rgw_ldap_searchfilter setting. There are two ways to use the rgw_ldap_searchfilter setting: Specifying a partial filter: Example The Ceph Object Gateway generates the search filter with the user name from the token and the value of rgw_ldap_dnattr . The constructed filter is then combined with the partial filter from the rgw_ldap_searchfilter value. For example, the user name and the settings generate the final search filter: Example User joe is only granted access if he is found in the LDAP directory, has an object class of inetorgperson , and specifies a valid password. Specifying a complete filter: A complete filter must contain a USERNAME token which is substituted with the user name during the authentication attempt. The rgw_ldap_dnattr setting is not used in this case. For example, to limit valid users to a specific group, use the following filter: Example 6.1.9. Add an S3 user to the LDAP server In the administrative console on the LDAP server, create at least one S3 user so that an S3 client can use the LDAP user credentials. Make a note of the user name and secret for use when passing the credentials to the S3 client. 6.1.10. Export an LDAP token When running Ceph Object Gateway with LDAP, the access token is all that is required. However, the access token is created from the access key and secret key. Export the access key and secret key as an LDAP token. Export the access key: Syntax Export the secret key: Syntax Export the token. For LDAP, use ldap as the token type ( ttype ). Example For Active Directory, use ad as the token type. Example The result is a base-64 encoded string, which is the access token. Provide this access token to S3 clients in lieu of the access key. The secret key is no longer required. Optional: For added convenience, export the base-64 encoded string to the RGW_ACCESS_KEY_ID environment variable if the S3 client uses the environment variable. Example 6.1.11. Test the configuration with an S3 client Test the configuration with a Ceph Object Gateway client, using a script such as Python Boto. Procedure. Use the RGW_ACCESS_KEY_ID environment variable to configure the Ceph Object Gateway client. Alternatively, you can copy the base-64 encoded string and specify it as the access key. Following is an example of the configured S3 client: Example Note The secret key is no longer required. Run the aws s3 ls command to verify the user: Example Optional: You can also run the radosgw-admin user command to verify the user in the directory: Example 6.2. Configure Active Directory and Ceph Object Gateway Perform the following steps to configure an Active Directory server to authenticate Ceph Object Gateway users. 6.2.1. Using Microsoft Active Directory Ceph Object Gateway LDAP authentication is compatible with any LDAP-compliant directory service that can be configured for simple bind, including Microsoft Active Directory. Using Active Directory is similar to using RH Directory server in that the Ceph Object Gateway binds as the user configured in the rgw_ldap_binddn setting, and uses LDAPs to ensure security. The process for configuring Active Directory is essentially identical to configuring LDAP and Ceph Object Gateway, but may have some Windows-specific usage. 6.2.2. Configuring Active Directory for LDAPS Active Directory LDAP servers are configured to use LDAPs by default. Windows Server 2012 and higher can use Active Directory Certificate Services. Instructions for generating and installing SSL certificates for use with Active Directory LDAP are available in the following MS TechNet article: LDAP over SSL (LDAPS) Certificate . Note Ensure that port 636 is open on the Active Directory host. 6.2.3. Check if the gateway user exists Before creating the gateway user, ensure that the Ceph Object Gateway does not already have the user. Example The user name should NOT be in this list of users. 6.2.4. Add a gateway user Create a Ceph Object Gateway user to user LDAP. Procedure Create an LDAP user for the Ceph Object Gateway, and make a note of the binddn . Since the Ceph object gateway uses the ceph user, consider using ceph as the username. The user needs to have permissions to search the directory. The Ceph Object Gateway binds to this user as specified in rgw_ldap_binddn . Test to ensure that the user creation worked. Where ceph is the user ID under People and example.com is the domain, you can perform a search for the user. On each gateway node, create a file for the user's secret. For example, the secret might get stored in a file entitled /etc/bindpass . For security, change the owner of this file to the ceph user and group to ensure it is not globally readable. Add the rgw_ldap_secret option: Syntax Example Patch the bind password file to the Ceph Object Gateway container and reapply the Ceph Object Gateway specification: Example Note /etc/bindpass is not shipped automatically with Red Hat Ceph Storage and you need to ensure that the content is available on all the possible Ceph Object Gateway instance nodes. 6.2.5. Configuring the gateway to use Active Directory Add the following options after setting the rgw_ldap_secret setting: Syntax Example For the rgw_ldap_uri setting, substitute FQDN with the fully qualified domain name of the LDAP server. If there is more than one LDAP server, specify each domain. For the rgw_ldap_binddn setting, substitute BINDDN with the bind domain. With a domain of example.com and a ceph user under users and accounts , it should look something like this: Example For the rgw_ldap_searchdn setting, substitute SEARCHDN with the search domain. With a domain of example.com and users under users and accounts , it should look something like this: Restart the Ceph Object Gateway: Note Use the output from the ceph orch ps command, under the NAME column, to get the SERVICE_TYPE . ID information. To restart the Ceph Object Gateway on an individual node in the storage cluster: Syntax Example To restart the Ceph Object Gateways on all nodes in the storage cluster: Syntax Example 6.2.6. Add an S3 user to the LDAP server In the administrative console on the LDAP server, create at least one S3 user so that an S3 client can use the LDAP user credentials. Make a note of the user name and secret for use when passing the credentials to the S3 client. 6.2.7. Export an LDAP token When running Ceph Object Gateway with LDAP, the access token is all that is required. However, the access token is created from the access key and secret key. Export the access key and secret key as an LDAP token. Export the access key: Syntax Export the secret key: Syntax Export the token. For LDAP, use ldap as the token type ( ttype ). Example For Active Directory, use ad as the token type. Example The result is a base-64 encoded string, which is the access token. Provide this access token to S3 clients in lieu of the access key. The secret key is no longer required. Optional: For added convenience, export the base-64 encoded string to the RGW_ACCESS_KEY_ID environment variable if the S3 client uses the environment variable. Example 6.2.8. Test the configuration with an S3 client Test the configuration with a Ceph Object Gateway client, using a script such as Python Boto. Procedure. Use the RGW_ACCESS_KEY_ID environment variable to configure the Ceph Object Gateway client. Alternatively, you can copy the base-64 encoded string and specify it as the access key. Following is an example of the configured S3 client: Example Note The secret key is no longer required. Run the aws s3 ls command to verify the user: Example Optional: You can also run the radosgw-admin user command to verify the user in the directory: Example 6.3. The Ceph Object Gateway and OpenStack Keystone As a storage administrator, you can use OpenStack's Keystone authentication service to authenticate users through the Ceph Object Gateway. Before you can configure the Ceph Object Gateway, you need to configure Keystone first. This enables the Swift service, and points the Keystone service to the Ceph Object Gateway. , you need to configure the Ceph Object Gateway to accept authentication requests from the Keystone service. Prerequisites A running Red Hat OpenStack Platform environment. A running Red Hat Ceph Storage environment. A running Ceph Object Gateway environment. 6.3.1. Roles for Keystone authentication The OpenStack Keystone service provides three roles: admin , member , and reader . These roles are hierarchical; users with the admin role inherit the capabilities of the member role and users with the member role inherit the capabilities of the reader role. Note The member role's read permissions only apply to objects of the project it belongs to. admin The admin role is reserved for the highest level of authorization within a particular scope. This usually includes all the create, read, update, or delete operations on a resource or API. member The member role is not used directly by default. It provides flexibility during deployments and helps reduce responsibility for administrators. For example, you can override a policy for a deployment by using the default member role and a simple policy override, to allow system members to update services and endpoints. This provides a layer of authorization between admin and reader roles. reader The reader role is reserved for read-only operations regardless of the scope. Warning If you use a reader to access sensitive information such as image license keys, administrative image data, administrative volume metadata, application credentials, and secrets, you might unintentionally expose sensitive information. Hence, APIs that expose these resources should carefully consider the impact of the reader role and appropriately defer access to the member and admin roles. 6.3.2. Keystone authentication and the Ceph Object Gateway Organizations using OpenStack Keystone to authenticate users can integrate Keystone with the Ceph Object Gateway. The Ceph Object Gateway enables the gateway to accept a Keystone token, authenticate the user, and create a corresponding Ceph Object Gateway user. When Keystone validates a token, the gateway considers the user authenticated. Benefits Assigning admin , member , and reader roles to users with Keystone. Automatic user creation in the Ceph Object Gateway. Managing users with Keystone. The Ceph Object Gateway will query Keystone periodically for a list of revoked tokens. 6.3.3. Creating the Swift service Before configuring the Ceph Object Gateway, configure Keystone so that the Swift service is enabled and pointing to the Ceph Object Gateway. Prerequisites A running Red Hat Ceph Storage cluster. Access to the Ceph software repository. Root-level access to OpenStack controller node. Procedure Create the Swift service: Creating the service will echo the service settings. Table 6.1. Example Field Value description Swift Service enabled True id 37c4c0e79571404cb4644201a4a6e5ee name swift type object-store 6.3.4. Setting the Ceph Object Gateway endpoints After creating the Swift service, point the service to a Ceph Object Gateway. Prerequisites A running Red Hat Ceph Storage cluster. Access to the Ceph software repository. A running Swift service on a Red Hat OpenStack Platform 17.1 environment. Procedure Create the OpenStack endpoints pointing to the Ceph Object Gateway: Syntax Replace REGION_NAME with the name of the gateway's zone group name or region name. Replace URL with URLs appropriate for the Ceph Object Gateway. Example Field Value adminurl http://radosgw.example.com:8080/swift/v1 id e4249d2b60e44743a67b5e5b38c18dd3 internalurl http://radosgw.example.com:8080/swift/v1 publicurl http://radosgw.example.com:8080/swift/v1 region us-west service_id 37c4c0e79571404cb4644201a4a6e5ee service_name swift service_type object-store Setting the endpoints will output the service endpoint settings. 6.3.5. Verifying Openstack is using the Ceph Object Gateway endpoints After creating the Swift service and setting the endpoints, show the endpoints to ensure that all settings are correct. Prerequisites A running Red Hat Ceph Storage cluster. Access to the Ceph software repository. Procedure List the endpoints under the Swift service: Verify settings for the endpoints listed in the command: Syntax Showing the endpoints will echo the endpoints settings, and the service settings. Table 6.2. Example Field Value adminurl http://radosgw.example.com:8080/swift/v1 enabled True id e4249d2b60e44743a67b5e5b38c18dd3 internalurl http://radosgw.example.com:8080/swift/v1 publicurl http://radosgw.example.com:8080/swift/v1 region us-west service_id 37c4c0e79571404cb4644201a4a6e5ee service_name swift service_type object-store Additional Resources For more information on getting the details about endpoints, see Show endpoints in the Red Hat OpenStack guide. 6.3.6. Configuring the Ceph Object Gateway to use Keystone SSL Converting the OpenSSL certificates that Keystone uses configures the Ceph Object Gateway to work with Keystone. When the Ceph Object Gateway interacts with OpenStack's Keystone authentication, Keystone will terminate with a self-signed SSL certificate. Prerequisites A running Red Hat Ceph Storage cluster. Access to the Ceph software repository. Procedure Convert the OpenSSL certificate to the nss db format: Example Install Keystone's SSL certificate in the node running the Ceph Object Gateway. Alternatively set the value of the configurable rgw_keystone_verify_ssl setting to false . Setting rgw_keystone_verify_ssl to false means that the gateway will not attempt to verify the certificate. 6.3.7. Configuring the Ceph Object Gateway to use Keystone authentication Configure the Red Hat Ceph Storage to use OpenStack's Keystone authentication. Prerequisites A running Red Hat Ceph Storage cluster. Access to the Ceph software repository. Have admin privileges to the production environment. Procedure Do the following for each gateway instance. Set the nss_db_path setting to the path where the NSS database is stored: Example Provide authentication credentials: It is possible to configure a Keystone service tenant, user, and password for keystone for the OpenStack Identity API, similar to the way system administrators tend to configure OpenStack services. Providing a username and password avoids providing the shared secret to the rgw_keystone_admin_token setting. Important Red Hat recommends disabling authentication by admin token in production environments. The service tenant credentials should have admin privileges. The necessary configuration options are: Syntax Example A Ceph Object Gateway user is mapped into a Keystone tenant . A Keystone user has different roles assigned to it on possibly more than a single tenant. When the Ceph Object Gateway gets the ticket, it looks at the tenant, and the user roles that are assigned to that ticket, and accepts or rejects the request according to the rgw_keystone_accepted_roles configurable. Additional Resources See the Users and Identity Management Guide for Red Hat OpenStack Platform. 6.3.8. Restarting the Ceph Object Gateway daemon Restarting the Ceph Object Gateway must be done to active configuration changes. Prerequisites A running Red Hat Ceph Storage cluster. Access to the Ceph software repository. admin privileges to the production environment. Procedure Once you have saved the Ceph configuration file and distributed it to each Ceph node, restart the Ceph Object Gateway instances: Note Use the output from the ceph orch ps command, under the NAME column, to get the SERVICE_TYPE . ID information. To restart the Ceph Object Gateway on an individual node in the storage cluster: Syntax Example To restart the Ceph Object Gateways on all nodes in the storage cluster: Syntax Example

Commands:
firewall-cmd --zone=public --add-port=636/tcp firewall-cmd --zone=public --add-port=636/tcp --permanent
certutil -d /etc/openldap/certs -A -t "TC,," -n "msad-frog-MSAD-FROG-CA" -i /path/to/ldap.pem
setsebool -P httpd_can_network_connect on
chmod 644 /etc/openldap/certs/*
ldapwhoami -H ldaps://rh-directory-server.example.com -d 9
radosgw-admin metadata list user
ldapsearch -x -D "uid=ceph,ou=People,dc=example,dc=com" -W -H ldaps://example.com -b "ou=People,dc=example,dc=com" -s sub 'uid=ceph'
ceph config set client.rgw OPTION VALUE
ceph config set client.rgw rgw_ldap_secret /etc/bindpass
service_type: rgw service_id: rgw.1 service_name: rgw.rgw.1 placement: label: rgw extra_container_args: - -v - /etc/bindpass:/etc/bindpass
ceph config set client.rgw OPTION VALUE
ceph config set client.rgw rgw_ldap_uri ldaps://:636 ceph config set client.rgw rgw_ldap_binddn "ou=poc,dc=example,dc=local" ceph config set client.rgw rgw_ldap_searchdn "ou=poc,dc=example,dc=local" ceph config set client.rgw rgw_ldap_dnattr "uid" ceph config set client.rgw rgw_s3_auth_use_ldap true
systemctl restart ceph- CLUSTER_ID @ SERVICE_TYPE . ID .service
systemctl restart ceph-c4b34c6f-8365-11ba-dc31-529020a7702d@rgw.realm.zone.host01.gwasto.service
ceph orch restart SERVICE_TYPE
ceph orch restart rgw
"objectclass=inetorgperson"
"(&(uid=joe)(objectclass=inetorgperson))"
"(&(uid=@USERNAME@)(memberOf=cn=ceph-users,ou=groups,dc=mycompany,dc=com))"
export RGW_ACCESS_KEY_ID=" USERNAME "
export RGW_SECRET_ACCESS_KEY=" PASSWORD "
radosgw-token --encode --ttype=ldap
radosgw-token --encode --ttype=ad
export RGW_ACCESS_KEY_ID="ewogICAgIlJHV19UT0tFTiI6IHsKICAgICAgICAidmVyc2lvbiI6IDEsCiAgICAgICAgInR5cGUiOiAibGRhcCIsCiAgICAgICAgImlkIjogImNlcGgiLAogICAgICAgICJrZXkiOiAiODAwI0dvcmlsbGEiCiAgICB9Cn0K"
cat .aws/credentials [default] aws_access_key_id = ewogICaGbnjlwe9UT0tFTiI6IHsKICAgICAgICAidmVyc2lvbiI6IDEsCiAgICAgICAgInR5cGUiOiAiYWQiLAogICAgICAgICJpZCI6ICJjZXBoIiwKICAgICAgICAia2V5IjogInBhc3M0Q2VwaCIKICAgIH0KfQo= aws_secret_access_key =
aws s3 ls --endpoint http://host03 2023-12-11 17:08:50 mybucket 2023-12-24 14:55:44 mybucket2
radosgw-admin user info --uid dir1 { "user_id": "dir1", "display_name": "dir1", "email": "", "suspended": 0, "max_buckets": 1000, "subusers": [], "keys": [], "swift_keys": [], "caps": [], "op_mask": "read, write, delete", "default_placement": "", "default_storage_class": "", "placement_tags": [], "bucket_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "user_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "temp_url_keys": [], "type": "ldap", "mfa_ids": [] }
radosgw-admin metadata list user
ldapsearch -x -D "uid=ceph,ou=People,dc=example,dc=com" -W -H ldaps://example.com -b "ou=People,dc=example,dc=com" -s sub 'uid=ceph'
ceph config set client.rgw OPTION VALUE
ceph config set client.rgw rgw_ldap_secret /etc/bindpass
service_type: rgw service_id: rgw.1 service_name: rgw.rgw.1 placement: label: rgw extra_container_args: - -v - /etc/bindpass:/etc/bindpass
ceph config set client.rgw OPTION VALUE
ceph config set client.rgw rgw_ldap_uri ldaps://_FQDN_:636 ceph config set client.rgw rgw_ldap_binddn "_BINDDN_" ceph config set client.rgw rgw_ldap_searchdn "_SEARCHDN_" ceph config set client.rgw rgw_ldap_dnattr "cn" ceph config set client.rgw rgw_s3_auth_use_ldap true
rgw_ldap_binddn "uid=ceph,cn=users,cn=accounts,dc=example,dc=com"
rgw_ldap_searchdn "cn=users,cn=accounts,dc=example,dc=com"
systemctl restart ceph- CLUSTER_ID @ SERVICE_TYPE . ID .service
systemctl restart ceph-c4b34c6f-8365-11ba-dc31-529020a7702d@rgw.realm.zone.host01.gwasto.service
ceph orch restart SERVICE_TYPE
ceph orch restart rgw
export RGW_ACCESS_KEY_ID=" USERNAME "
export RGW_SECRET_ACCESS_KEY=" PASSWORD "
radosgw-token --encode --ttype=ldap
radosgw-token --encode --ttype=ad
export RGW_ACCESS_KEY_ID="ewogICAgIlJHV19UT0tFTiI6IHsKICAgICAgICAidmVyc2lvbiI6IDEsCiAgICAgICAgInR5cGUiOiAibGRhcCIsCiAgICAgICAgImlkIjogImNlcGgiLAogICAgICAgICJrZXkiOiAiODAwI0dvcmlsbGEiCiAgICB9Cn0K"
cat .aws/credentials [default] aws_access_key_id = ewogICaGbnjlwe9UT0tFTiI6IHsKICAgICAgICAidmVyc2lvbiI6IDEsCiAgICAgICAgInR5cGUiOiAiYWQiLAogICAgICAgICJpZCI6ICJjZXBoIiwKICAgICAgICAia2V5IjogInBhc3M0Q2VwaCIKICAgIH0KfQo= aws_secret_access_key =
aws s3 ls --endpoint http://host03 2023-12-11 17:08:50 mybucket 2023-12-24 14:55:44 mybucket2
radosgw-admin user info --uid dir1 { "user_id": "dir1", "display_name": "dir1", "email": "", "suspended": 0, "max_buckets": 1000, "subusers": [], "keys": [], "swift_keys": [], "caps": [], "op_mask": "read, write, delete", "default_placement": "", "default_storage_class": "", "placement_tags": [], "bucket_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "user_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "temp_url_keys": [], "type": "ldap", "mfa_ids": [] }
openstack service create --name=swift --description="Swift Service" object-store
openstack endpoint create --region REGION_NAME swift admin " URL " openstack endpoint create --region REGION_NAME swift public " URL " openstack endpoint create --region REGION_NAME swift internal " URL "
openstack endpoint create --region us-west swift admin "http://radosgw.example.com:8080/swift/v1" openstack endpoint create --region us-west swift public "http://radosgw.example.com:8080/swift/v1" openstack endpoint create --region us-west swift internal "http://radosgw.example.com:8080/swift/v1"
openstack endpoint list --service=swift
openstack endpoint show ENDPOINT_ID
mkdir /var/ceph/nss openssl x509 -in /etc/keystone/ssl/certs/ca.pem -pubkey | certutil -d /var/ceph/nss -A -n ca -t "TCu,Cu,Tuw" openssl x509 -in /etc/keystone/ssl/certs/signing_cert.pem -pubkey | certutil -A -d /var/ceph/nss -n signing_cert -t "P,P,P"
ceph config set client.rgw nss_db_path "/var/lib/ceph/radosgw/ceph-rgw.rgw01/nss"
ceph config set client.rgw rgw_keystone_verify_ssl TRUE / FALSE ceph config set client.rgw rgw_s3_auth_use_keystone TRUE / FALSE ceph config set client.rgw rgw_keystone_api_version API_VERSION ceph config set client.rgw rgw_keystone_url KEYSTONE_URL : ADMIN_PORT ceph config set client.rgw rgw_keystone_accepted_roles ACCEPTED_ROLES_ ceph config set client.rgw rgw_keystone_accepted_admin_roles ACCEPTED_ADMIN_ROLES ceph config set client.rgw rgw_keystone_admin_domain default ceph config set client.rgw rgw_keystone_admin_project SERVICE_NAME ceph config set client.rgw rgw_keystone_admin_user KEYSTONE_TENANT_USER_NAME ceph config set client.rgw rgw_keystone_admin_password KEYSTONE_TENANT_USER_PASSWORD ceph config set client.rgw rgw_keystone_implicit_tenants KEYSTONE_IMPLICIT_TENANT_NAME ceph config set client.rgw rgw_swift_versioning_enabled TRUE / FALSE ceph config set client.rgw rgw_swift_enforce_content_length TRUE / FALSE ceph config set client.rgw rgw_swift_account_in_url TRUE / FALSE ceph config set client.rgw rgw_trust_forwarded_https TRUE / FALSE ceph config set client.rgw rgw_max_attr_name_len MAXIMUM_LENGTH_OF_METADATA_NAMES ceph config set client.rgw rgw_max_attrs_num_in_req MAXIMUM_NUMBER_OF_METADATA_ITEMS ceph config set client.rgw rgw_max_attr_size MAXIMUM_LENGTH_OF_METADATA_VALUE ceph config set client.rgw rgw_keystone_accepted_reader_roles SwiftSystemReader
ceph config set client.rgw rgw_keystone_verify_ssl false ceph config set client.rgw rgw_s3_auth_use_keystone true ceph config set client.rgw rgw_keystone_api_version 3 ceph config set client.rgw rgw_keystone_url http://<public Keystone endpoint>:5000/ ceph config set client.rgw rgw_keystone_accepted_roles 'member, Member, admin' ceph config set client.rgw rgw_keystone_accepted_admin_roles 'ResellerAdmin, swiftoperator' ceph config set client.rgw rgw_keystone_admin_domain default ceph config set client.rgw rgw_keystone_admin_project service ceph config set client.rgw rgw_keystone_admin_user swift ceph config set client.rgw rgw_keystone_admin_password password ceph config set client.rgw rgw_keystone_implicit_tenants true ceph config set client.rgw rgw_swift_versioning_enabled true ceph config set client.rgw rgw_swift_enforce_content_length true ceph config set client.rgw rgw_swift_account_in_url true ceph config set client.rgw rgw_trust_forwarded_https true ceph config set client.rgw rgw_max_attr_name_len 128 ceph config set client.rgw rgw_max_attrs_num_in_req 90 ceph config set client.rgw rgw_max_attr_size 1024 ceph config set client.rgw rgw_keystone_accepted_reader_roles SwiftSystemReader
systemctl restart ceph- CLUSTER_ID @ SERVICE_TYPE . ID .service
systemctl restart ceph-c4b34c6f-8365-11ba-dc31-529020a7702d@rgw.realm.zone.host01.gwasto.service
ceph orch restart SERVICE_TYPE
ceph orch restart rgw