Chapter 4. Identity and Access Management Red Hat Ceph Storage provides identity and access management for: Ceph Storage Cluster User Access Ceph Object Gateway User Access Ceph Object Gateway LDAP/AD Authentication Ceph Object Gateway OpenStack Keystone Authentication 4.1. Ceph Storage Cluster User Access To identify users and protect against man-in-the-middle attacks, Ceph provides its cephx authentication system to authenticate users and daemons. For additional details on cephx , see Ceph user management . Important The cephx protocol DOES NOT address data encryption in transport or encryption at rest. Cephx uses shared secret keys for authentication, meaning both the client and the monitor cluster have a copy of the client's secret key. The authentication protocol is such that both parties are able to prove to each other they have a copy of the key without actually revealing it. This provides mutual authentication, which means the cluster is sure the user possesses the secret key, and the user is sure that the cluster has a copy of the secret key. Users are either individuals or system actors such as applications, which use Ceph clients to interact with the Red Hat Ceph Storage cluster daemons. Ceph runs with authentication and authorization enabled by default. Ceph clients may specify a user name and a keyring containing the secret key of the specified user, usually by using the command line. If the user and keyring are not provided as arguments, Ceph will use the client.admin administrative user as the default. If a keyring is not specified, Ceph will look for a keyring by using the keyring setting in the Ceph configuration. Important To harden a Ceph cluster, keyrings SHOULD ONLY have read and write permissions for the current user and root . The keyring containing the client.admin administrative user key must be restricted to the root user. For details on configuring the Red Hat Ceph Storage cluster to use authentication, see the Configuration Guide for Red Hat Ceph Storage 8. More specifically, see Ceph authentication configuration . 4.2. Ceph Object Gateway User Access The Ceph Object Gateway provides a RESTful application programming interface (API) service with its own user management that authenticates and authorizes users to access S3 and Swift APIs containing user data. Authentication consists of: S3 User: An access key and secret for a user of the S3 API. Swift User: An access key and secret for a user of the Swift API. The Swift user is a subuser of an S3 user. Deleting the S3 'parent' user will delete the Swift user. Administrative User: An access key and secret for a user of the administrative API. Administrative users should be created sparingly, as the administrative user will be able to access the Ceph Admin API and execute its functions, such as creating users, and giving them permissions to access buckets or containers and their objects among other things. The Ceph Object Gateway stores all user authentication information in Ceph Storage cluster pools. Additional information may be stored about users including names, email addresses, quotas, and usage. For additional details, see User Management and Creating an Administrative User . 4.3. Ceph Object Gateway LDAP or AD authentication Red Hat Ceph Storage supports Light-weight Directory Access Protocol (LDAP) servers for authenticating Ceph Object Gateway users. When configured to use LDAP or Active Directory (AD), Ceph Object Gateway defers to an LDAP server to authenticate users of the Ceph Object Gateway. Ceph Object Gateway controls whether to use LDAP. However, once configured, it is the LDAP server that is responsible for authenticating users. To secure communications between the Ceph Object Gateway and the LDAP server, Red Hat recommends deploying configurations with LDAP Secure or LDAPS. Important When using LDAP, ensure that access to the rgw_ldap_secret = PATH_TO_SECRET_FILE secret file is secure. 4.4. Ceph Object Gateway OpenStack Keystone authentication Red Hat Ceph Storage supports using OpenStack Keystone to authenticate Ceph Object Gateway Swift API users. The Ceph Object Gateway can accept a Keystone token, authenticate the user and create a corresponding Ceph Object Gateway user. When Keystone validates a token, the Ceph Object Gateway considers the user authenticated. Ceph Object Gateway controls whether to use OpenStack Keystone for authentication. However, once configured, it is the OpenStack Keystone service that is responsible for authenticating users. Configuring the Ceph Object Gateway to work with Keystone requires converting the OpenSSL certificates that Keystone uses for creating the requests to the nss db format. Additional Resources See The Ceph Object Gateway and OpenStack Keystone section of the Red Hat Ceph Storage Object Gateway Guide for more information. 4.5. Ceph Dashboard SSO using OpenID Connect (OIDC) protocol (Technology Preview) The Ceph Management stack is built by using a modular, service-based architecture that relies on nginx and oauth2-proxy components. Important Technology Preview features are not supported with Red Hat production service level agreements (SLAs), might not be functionally complete, and Red Hat does not recommend using them for production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. See the support scope for Red Hat Technology Preview features for more details. The Ceph Management gateway ( mgmt-gateway ) provides unified access, improved user experience, and high availability for both the Ceph Dashboard and monitoring. The OAuth2 Proxy ( oauth2-proxy ) service provides enhanced security, seamless SSO, and centralized authentication management. Enable the mgmt-gateway service by itself to provide a single-entry point to the cluster, high availability for monitoring and dashboard, as well as for improved security. Enable the service together with the oauth2-proxy to use SSO. With the Ceph Management gateway ( mgmt-gateway ) and the OAuth2 Proxy ( oauth2-proxy ) in place, nginx automatically directs the user through the oauth2-proxy to the configured Identity Provider (IdP), when single sign-on (SSO) is configured. The IdP then runs the authentication and login. This configuration provides the following advantages: Unified access. Improved user experience. High availability for the Ceph Dashboard and monitoring. Enhanced security. Seamless SSO. Centralized authentication management. For more information about this modular SSO solution and implementation with the Ceph Dashboard, see Enabling OAuth2 single sign-on . 4.5.1. Using the Ceph Management gateway (mgmt-gateway) (Technology Preview) The Ceph Management gateway ( mgmt-gateway ) service provides a modular, service-based architecture design. Cephadm manages the `mgmt-gatewa`y service, which is built on top of nginx. This gateway acts as the front-end and single entry point to the Ceph cluster, providing unified access to all Ceph applications, including the Ceph Dashboard and the monitoring stack. Important Technology Preview features are not supported with Red Hat production service level agreements (SLAs), might not be functionally complete, and Red Hat does not recommend using them for production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. See the support scope for Red Hat Technology Preview features for more details. Using nginx enhances security and simplifies access management due to its robust community support and high-security standards. The mgmt-gateway service acts as a reverse proxy that routes requests to the appropriate Ceph application instances. Use the mgmt-gateway either by itself or with the OAuth2 Proxy service for authentication to enable single-sign on (SSO) access to the Ceph Dashboard. For more information about the OAuth2 Proxy service, see Using the OAuth2 Proxy (oauth2-proxy) service . Using the Ceph Management gateway provides multiple benefits. Unified access Using the Ceph Management gateway provides multiple benefits. Improved user experience Users no longer needs to keep track of where each application is running (IP or host). High availability for Ceph Dashboard nginx high availability (HA) mechanisms are used to provide high availability for the Ceph Dashboard. High availability for monitoring nginx high availability mechanisms are used to provide high availability for monitoring. Deploying the mgmt-gateway provides enhanced security. Once the mgmt-gateway service is deployed users cannot access monitoring services without authentication through the Ceph Dashboard. After the service is deployed, Prometheus, Grafana, and Alertmanager applications are accessible through links on the Ceph Dashboard. To get to the application links, go to Administration->Services on the Ceph Dashboard. Note With the mgmt-gateway mode, Prometheus and Alertmanager require the user must define a unique user and password. The default is admin:admin. nginx HA mechanisms are used to provide high availability for all the Ceph management applications including the Ceph Dashboard and monitoring stack. The mgmt-gateway service handles manager failover transparently and redirects the user to the active manager. With monitoring, the Ceph Management service takes care of handling HA when several instances of Prometheus, Alertmanager or Grafana are available. The reverse proxy automatically detects healthy instances and uses them to process user requests. 4.5.1.1. Ceph Management gateway limitations Using the mgmt-gateway service has the following limitations: High-availability configurations and clustering for the mgmt-gateway service itself are currently not supported. Services must bind to the appropriate ports based on the applications being proxied. Ensure that there are no port conflicts that might disrupt service availability. The mgmt-gateway service internally makes use of the nginx reverse proxy. Use the following to find the default container image is used by default: Storage administrators can change the image being used by changing the container_image_nginx cephadm module option. If other daemons were running, you must redeploy the daemons to use the new image. 4.5.1.2. Enabling the Ceph Management gateway with the command-line interface Enable the Ceph Management gateway for SSO access to the Dashboard and the Ceph cluster. Use this information to enable the mgmt-gateway service with the cephadm CLI commands. Procedure Deploy the mgmt-gateway service. Syntax Note The --enable-auth=true parameter is mandatory to enable SSO with the oauth2-proxy . Example Verify that the service was correctly deployed. 4.5.1.3. Enabling the Ceph Management gateway with a service specification file Enable the Ceph Management gateway for SSO access to the Dashboard and the Ceph cluster. Use this information to enable the mgmt-gateway service with a service specification file. Prerequisites Before enabling the Ceph Management gateway, be sure that the following are on each Ceph node that the mgmt-gateway service will run on. The port for gateway service use. A running Red Hat Ceph Storage cluster. (Optional) SSL protocols being used. (Optional) SSL ciphers. (Optional) SSL certificates and certificate keys. For more information about SSL protocols, ciphers, certificates, and certificate keys, see the Deploying web servers and reverse proxies in the Red Hat Enterprise Linux documentation. Procedure Create a YAML file for the mgmt-gateway service. Example Edit the YAML file to include the following details. Important The following fields are optional: ssl_protocols , ssl_ciphers , and ssl_certificate . When omitted, the mgmt-gateway service uses a safe and secure configuration. Changing these fields are permitted but should be done with care as this can compromise the security of your system. Syntax Important Use the TLSv1.3 SSL protocol. TLSv1.3 includes a set of secure ciphers by default. While TLSv1.2 is supported, it is crucial to use only a subset of secure ciphers when using this protocol. Using weak or outdated ciphers can significantly compromise the security of your system. Example The following table lists fields that are specific to the mgmt-gateway service section of the spec file ( ceph.deployment.service_spec.MgmtGatewaySpec ). Table 4.1. mgmt-gateway specific fields in the spec file Field Description disable_https Is a flag to disable HTTPS. If True, the server will use unsecure HTTP. enable_auth Is a flag to enable SSO auth. Requires oauth2-proxy to be active for SSO authentication. networks A list of network identities instructing the daemons to only bind on the particular networks in that list. In case the cluster is distributed across multiple networks, you can add multiple networks. placement For the orchestrator to deploy a service, it needs to know where to deploy daemons, and how many to deploy. This is the role of a placement specification. Placement specifications can either be passed as command line arguments or in a YAML files. For more information, see Management of services using the Ceph Orchestrator . port The port number on which the server will listen. server_tokens Flag control server tokens in responses: on , off , build , string . ssl_certificate A multi-line string that contains the SSL certificate. ssl_certificate_key A multi-line string that contains the SSL key. ssl_ciphers List of supported secure SSL ciphers. Changing this list may reduce system security. ssl_prefer_server_ciphers Prefer server ciphers over client ciphers: on , off . ssl_protocols A list of supported SSL protocols (as supported by nginx). ssl_session_cache Duration an SSL/TLS session is cached: off , none , [builtin[:size]] , [shared:name:size] . ssl_session_tickets A multi-option flag to control session tickets: on , off . ssl_session_timeout The duration for SSL session timeout. Syntax: time (for example, 5m ). ssl_stapling Flag to enable or disable SSL stapling: on , off . ssl_stapling_verify Flag to control verification of SSL stapling: on , off . Apply the specification file. Syntax 4.5.2. Using the OAuth2 Proxy (oauth2-proxy) service (Technology Preview) The OAuth2 Proxy service provides an advanced method for managing authentication and access control for Ceph applications. The oauth2-proxy service integrates with external Identity Providers (IdPs) to provide secure, flexible authentication with the OpenID Connect (OIDC) protocol. oauth2-proxy acts as an authentication gateway, ensuring that access to Ceph applications are tightly controlled. Important Technology Preview features are not supported with Red Hat production service level agreements (SLAs), might not be functionally complete, and Red Hat does not recommend using them for production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. See the support scope for Red Hat Technology Preview features for more details. Using the OAuth2 Proxy service with authentication requires using the Ceph Management gateway ( mgmt-gateway ). For more information see Using the Ceph Management gateway (mgmt-gateway) . Using the oauth2-proxy service provides multiple benefits. Enhanced security Provides robust authentication through integration with external IdPs by using the OIDC protocol. Seamless single sign-on (SSO) Enables seamless SSO across all Ceph monitoring applications, improving user access control. For more information about enabling OAuth2 SSO for the Ceph Dashboard, see Enabling OAuth2 single sign-on . Centralized authentication Centralizes authentication management, reducing complexity and improving control over access. Deploying the oauth2-proxy service provides enhanced security. Once the oauth2-proxy service is deployed all access to Ceph applications must be authenticated through the external IdP by using OIDC. Authentication prevents unauthorized users from accessing sensitive information. Users are redirected to the IdP for login and then returned to the requested application. This setup ensures secure access and integrates seamlessly with the Ceph management stack. The Prometheus, Alertmanager, and Grafana Ceph applications all require authentication through the required IdP. The oauth2-proxy service uses the OAuth Provider open-source project, enabling easier service integration with a variety of external IdPs, providing a secure and flexible authentication mechanism. For a full list of valid OAuth providers, see OAuth Provider Configuration on OAuth2 Proxy Docs . 4.5.2.1. OAuth2 Proxy limitations The oauth2-proxy service does not support high availability configurations. Important Proper configuration of the IdP and OAuth2 Proxy parameters is crucial to avoid authentication failures. Misconfiguration can lead to access issues. 4.5.2.2. Creating an admin account with Red Hat Single Sign-On 7.6.0 Create an admin account to synchronize OAuth2 service to the Ceph dashboard. Use Red Hat Single Sign-on (SSO) to synchronize users to the Ceph dashboard. See Syncing users to the Ceph dashboard using Red Hat Single Sign-On . For more information, see the following: Creating roles on the Ceph dashboard . Creating users on the Ceph dashboard . Prerequisites Before you begin, make sure that you have the following prerequisites in place: A running Red Hat Ceph Storage cluster. Dashboard is installed. Admin-level access to the dashboard. Users are added to the dashboard. Root-level access on all the hosts. Java OpenJDK installed. For more information, see the Installing a JRE on RHEL by using yum section of the Installing and using OpenJDK 8 for RHEL guide for OpenJDK on Red Hat Documentation . Red Hat Single Sign-On installed from a ZIP file. For more information, see Installing RH-SSO from a ZIP file in the Red Hat Single Sign-On 7.6 Server Installation and Configuration Guide on Red Hat Documentation . Procedure Download the Red Hat Single Sign-On 7.6.0 Server on the system where IBM Storage Ceph is installed.. For download package and information, see Software Details for Red Hat Single Sign-On 7.6.0 Server on the Red Hat Customer Portal . Extract the folder: Syntax Go to the standalone/configuration directory under the newly created rhsso-7.6.0 directory and open the standalone.xml file for editing Syntax Replace the localhost and 127.0.0.1 with the IP address of the server where the Red Hat Single-Sign On is installed. From the bin directory of the newly created rhsso-7.6.0 folder, run the add-user-keycloak script to add the initial administrator user: Syntax Optional: Users with Red Hat Enterprise Linux 8 or later might get Certificate Authority (CA) issues. Import the custom certificates from CA and move them into the keystore with the exact Java version. Syntax Start the server. From the bin directory of rh-sso-7.6 folder, run the standalone boot script: Log in to the admin account in https:IP_ADDRESS:8080/auth with the new username and password created. Note You have to create an admin account only the first time that you log into the console. 4.5.2.2.1. Setting up IdP for OAuth2-proxy using Red Hat SSO with OAuth2 protocol Complete these steps from the Red Hat Single Sign-On console. Procedure Create a realm. The realm must be Enabled . Go to Realm Settings adn complete the required settings. Toggle Enabled to ON . Toggle User-Managed Access to ON . Click the link address of the OpenID Endpoint Configuration to retrieve the required endpoint for OAuth2 configuration. The issuer endpoint is used during OAuth2 service configuration. Create a client. Set the Client Protocol as openid-connect . Update the client setting. Set the Access Type as confidential . Enter the Valid Redirect URIs . For example: https://<ip|hostname>/oauth2/callback and https://<ip|hostname>/oauth2/sign_out Verify the client credentials. Check that the Client Authenticator is set as Client Id and Secret . The secret is used as the client_secret parameter when configuring the oauth2-proxy service. Create a new administrator role. Enter administrator as the role name. Update the administrator information. Go to Manage->Users->Add User . Complete the Username , Email , First Name , Last Name , and toggle Email Verified to ON . From Credentials , set a new password credentials for the administrator user. From Role Mappings , set the Client Roles as the client that was created in step 3 and click Add selected for administrator role created in step 6. 4.5.2.3. Enabling the OAuth2 Proxy service Enable the OAuth2 Proxy service for SSO access to the Dashboard and the Ceph cluster. Enabling the oauth2-proxy service either with the cephadm CLI commands or by using a service specification file. Find the oauth2-proxy container image by running the ceph config get command. Storage administrators can specify a custom image by changing the container_image_oauth2_proxy cephadm module option. If other daemons were running, you must redeploy the daemons to use the new image. Prerequisites Before you begin, make sure that you have the following prerequisites in place: Ceph Management gateway ( mgmt-gateway ) service is enabled and --enable-auth is set to true. For more information, see Using the Ceph Management gateway (mgmt-gateway) . An admin account with Red Hat Single-Sign on 7.6.0. For more information, see Creating an admin account with Red Hat Single Sign-On 7.6.0 . 4.5.2.3.1. Enabling the OAuth2 Proxy service with the command-line interface Procedure Deploy the oauth2-proxy service. Syntax Example Verify that the service was correctly deployed. 4.5.2.3.2. Enabling the OAuth2 Proxy service with a service specification file Enable the OAuth2 Proxy service for SSO access to the Dashboard and the Ceph cluster. Use this information to enable the oauth2-proxy service with a service specification file. Prerequisites Before enabling the OAuth2 Proxy service, make sure that the following are on each Ceph node that the oauth2-proxy service will run on. Your client ID. The OIDC issuer URL. Client secret. The relevant domain addresses to allow. Note The domain can be the same or different from the oidc_issuer_url . A running Red Hat Ceph Storage cluster. (Optional) The host HTTPS address and host port. (Optional) Cookie secret. (Optional) SSL certificates and certificate keys. For more information about SSL protocols, ciphers, certificates, and certificate keys, see Deploying web servers and reverse proxies in the Red Hat Enterprise Linux documentation. Procedure Create a YAML file for the oauth2-proxy service. Example Edit the YAML file to include the following details. Syntax Example The following table lists fields that are specific to the oauth2-proxy service section of the spec file ( ceph.deployment.service_spec.OAuth2ProxySpec ). Table 4.2. oauth2-proxy specific fields in the spec file Field Description allowlist_domains List of allowed domains for safe redirection after login or logout, preventing unauthorized redirects. This can be the same or different from the oidc_issuer_url . client_id The client ID for authenticating with the identity provider. client_secret The client secret for authenticating with the identity provider. cookie_secret The secret key is used for signing cookies. Its length must be 16 bytes, 24 bytes, or 32 bytes to create an AES cipher. https_address The address for HTTPS connections, which are formatted as host:port . networks A list of network identities instructing the daemons to only bind on the particular networks in that list. In case the cluster is distributed across multiple networks, you can add multiple networks. oidc_issuer_url The URL of the OpenID Connect (OIDC) issuer. placement For the orchestrator to deploy a service, it needs to know where to deploy daemons, and how many to deploy. This is the role of a placement specification. Placement specifications can either be passed as command line arguments or in a YAML files. For more information, see Management of services using the Ceph Orchestrator . provider_display_name The display name for the identity provider (IDP) in the UI. redirect_url Optional. The URL oauth2-proxy will redirect to after a successful login. If not provided, the cephadm automatically calculates the value of this URL. ssl_certificate A multi-line string that contains the SSL certificate. ssl_certificate_key A multi-line string that contains the SSL key. Apply the specification file. Syntax

Commands:
DEFAULT_NGINX_IMAGE = 'quay.io/ceph/NGINX_IMAGE'
ceph config set mgr mgr/cephadm/container_image_nginx NEW_NGINX_IMAGE ceph orch redeploy mgmt-gateway
ceph orch apply mgmt-gateway [--placement= DESTINATION_HOST ] [--enable-auth=true]
ceph orch apply mgmt-gateway --placement=host01
touch mgmt-gateway.yaml
service_type: mgmt-gateway placement: hosts: - ceph-node-1 spec: port: 9443 ssl_protocols: # Optional - TLSv1.3 ssl_ciphers: # Optional - AES128-SHA - AES256-SHA - RC4-SHA ssl_certificate: | # Optional -----BEGIN CERTIFICATE----- < YOU CERT DATA HERE > -----END CERTIFICATE----- ssl_certificate_key: | -----BEGIN RSA PRIVATE KEY----- < YOU PRIV KEY DATA HERE > -----END RSA PRIVATE KEY-----
service_type: mgmt-gateway service_id: gateway placement: hosts: - ceph0 spec: port: 5000 ssl_protocols: - TLSv1.3 - ssl_ciphers: - AES128-SHA - AES256-SHA - ssl_certificate: | -----BEGIN CERTIFICATE----- MIIDtTCCAp2gAwIBAgIYMC4xNzc1NDQxNjEzMzc2MjMyXzxvQ7EcMA0GCSqGSIb3 DQEBCwUAMG0xCzAJBgNVBAYTAlVTMQ0wCwYDVQQIDARVdGFoMRcwFQYDVQQHDA5T [...] -----END CERTIFICATE----- ssl_certificate_key: | -----BEGIN PRIVATE KEY----- MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC5jdYbjtNTAKW4 /CwQr/7wOiLGzVxChn3mmCIF3DwbL/qvTFTX2d8bDf6LjGwLYloXHscRfxszX/4h [...] -----END PRIVATE KEY-----
ceph orch apply -i mgmt-gateway.yaml
unzip rhsso-7.6.0.zip
cd standalone/configuration vi standalone.xml
./add-user-keycloak.sh -u admin
keytool -import -noprompt -trustcacerts -alias ca -file ../ca.cer -keystore /etc/java/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.272.b10-3.el8_3.x86_64/lib/security/cacert
./standalone.sh
ceph config get mgr mgr/cephadm/container_image_oauth2_proxy
ceph config set mgr mgr/cephadm/container_image_oauth2_proxy NEW_OAUTH2_PROXY_IMAGE ceph orch redeploy oauth2_proxy
ceph orch apply oauth2-proxy [--placement= DESTINATION_HOST ]
ceph orch apply oauth2-proxy [--placement=host01]
touch oauth2-proxy.yaml
service_type: oauth2-proxy service_id: auth-proxy placement: hosts: - ceph-node-1 spec: https_address: HTTPS_ADDRESS:PORT provider_display_name: MY OIDC PROVIDER client_id: CLIENT_ID oidc_issuer_url: OIDC ISSUER URL allowlist_domains: - HTTPS_ADDRESS:PORT client_secret: CLIENT_SECRET cookie_secret: COOKIE_SECRET ssl_certificate: | -----BEGIN CERTIFICATE----- < YOU CERT DATA HERE > -----END CERTIFICATE----- ssl_certificate_key: | -----BEGIN RSA PRIVATE KEY----- < YOU PRIV KEY DATA HERE > -----END RSA PRIVATE KEY-----
service_type: oauth2-proxy service_id: auth-proxy placement: hosts: - ceph0 spec: https_address: "0.0.0.0:4180" provider_display_name: "My OIDC Provider" client_id: "your-client-id" oidc_issuer_url: "http://192.168.100.1:5556/realms/ceph" allowlist_domains: - 192.168.100.1:8080 - 192.168.200.1:5000 client_secret: "your-client-secret" cookie_secret: "your-cookie-secret" ssl_certificate: | -----BEGIN CERTIFICATE----- MIIDtTCCAp2gAwIBAgIYMC4xNzc1NDQxNjEzMzc2MjMyXzxvQ7EcMA0GCSqGSIb3 DQEBCwUAMG0xCzAJBgNVBAYTAlVTMQ0wCwYDVQQIDARVdGFoMRcwFQYDVQQHDA5T [...] -----END CERTIFICATE----- ssl_certificate_key: | -----BEGIN PRIVATE KEY----- MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC5jdYbjtNTAKW4 /CwQr/7wOiLGzVxChn3mmCIF3DwbL/qvTFTX2d8bDf6LjGwLYloXHscRfxszX/4h [...] -----END PRIVATE KEY-----
ceph orch apply -i oauth2-proxy.yaml