vania-janet/multiturn-rag-retrieval-data huggingface dataset:
A [demo video](https://mediacenter.ibm.com/media/1_e5gk6ktn) is also available for your reference.





 Managing IBM Cloud Object Storage regulated workloads with Hyper Protect Crypto Services 

IBM Cloud Object Storage is an IBM Cloud service for you to store unlimited amounts of data in the assigned bucket so that the data can be accessed anywhere from the cloud. Hyper Protect Crypto Services helps you protect encryption keys for data that is stored in Object Storage with the highest security level in the industry and gives only you the access to these keys.

For more information about the integration, see [the Object Storage reference doc](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-encryption). A [demo video](https://mediacenter.ibm.com/media/0_mgxwp16v) is also available for your reference.





 Encrypting Red Hat OpenShift on IBM Cloud routes with keys stored in Hyper Protect Crypto Services 

With Hyper Protect Crypto Services, you can securely create, store, and manage encryption keys in IBM Cloud. A private key that is stored in an Hyper Protect Crypto Services instance can be used by an Red Hat OpenShift on IBM Cloud router in TLS session establishment and in Certificate Signing Request (CSR) signing. To access Hyper Protect Crypto Services, an Red Hat OpenShift on IBM Cloud router needs to use the OpenSSL Engine grep11 to make calls to the Enterprise PKCS #11 over gRPC (GREP11) API.

For more information about the detailed steps, see [Encrypting routes with keys stored in Hyper Protect Crypto Services](https://cloud.ibm.com/docs/openshift?topic=openshift-hpcs-router).







 Getting started to manage your regulated workloads with Hyper Protect Crypto Services 

To manage your regulated workloads with Hyper Protect Crypto Services, see the [Getting started tutorial](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-get-started).





 Reference
Security and compliance - Standard Plan 

IBM Cloud® Hyper Protect Crypto Services has data security strategies in place to meet your security and compliance needs and ensure that your data remains protected in the cloud.



 Security readiness 

Hyper Protect Crypto Services ensures security readiness by adhering to IBM best practices for systems, networking, and secure engineering.

To learn more about security controls across IBM Cloud, see [How do I know that my data is safe](https://cloud.ibm.com/docs/overview?topic=overview-securitysecurity).



 Data encryption 

Hyper Protect Crypto Services offers a dedicated

hardware security module (HSM)to generate key material that you manage and perform [envelope encryption](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-envelope-encryption) operations. Hyper Protect Crypto Services also supports the management of your own HSMmaster keys. Built on FIPS 140-2 Level 4-certified HSMs, Hyper Protect Crypto Services offers the highest security level for cloud-based HSMs and stores cryptographic key material without exposing keys outside of a cryptographic boundary.

Access to the service takes place over HTTPS, and internal Hyper Protect Crypto Services communication uses the Transport Layer Security (TLS) 1.2 protocol to encrypt data in transit.





 Data deletion 

When you delete a key from Hyper Protect Crypto Services, the service marks the key as deleted, and the key moves to the Destroyed state. Keys in this state can no longer decrypt data that is associated with the key. Therefore, before you delete a key, review the data that is associated with the key and ensure that you no longer require access to it. Do not delete a key that is actively protecting data in your production environments.

Within 30 days after you delete a key, you can restore the key to reverse the deletion. For more information, see [Restoring keys](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-restore-keys).

Note that even if the key is not restored, your data remains in those services in the encrypted form.
[Overview - Standard Plan](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-overviewoverview)



* [Why IBM Cloud Hyper Protect Crypto Services?](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-overviewwhy_hpcs)
* [How does Hyper Protect Crypto Services work?](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-overviewhow-hpcs-work)
* [Key features](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-overviewkey-features)
* [What's next](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-overviewoverview-next)



[Service architecture - Standard Plan](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-architecture-workload-isolationarchitecture-workload-isolation)



* [Hyper Protect Crypto Services architecture](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-architecture-workload-isolationarchitecture)
* [Hyper Protect Crypto Services workload isolation](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-architecture-workload-isolationworkload-isolation)
* [Service dependencies](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-architecture-workload-isolationservice_dependencies)



[Use cases - Standard Plan](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-use-casesuse-cases)



* [Pervasively protecting data at rest in the cloud](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-use-casesdata-at-rest-encryption)
* hs-crypto.crypto.getmechanismlist
* hs-crypto.crypto.rewrapkeyblob
* hs-crypto.crypto.setattributevalue
* hs-crypto.crypto.sign
* hs-crypto.crypto.unwrapkey
* hs-crypto.crypto.verify
* hs-crypto.crypto.wrapkey
* hs-crypto.ep11.use
* hs-crypto.keystore.deletekey
* hs-crypto.keystore.listkeysbyattributes
* hs-crypto.keystore.listkeysbyids
* hs-crypto.keystore.listkeystoresbyids
* hs-crypto.keystore.storenewkey
* hs-crypto.keystore.updatekey
* hs-crypto.discovery.listservers



8. Click Create when you're done adding actions.







 2. Create a custom role for operating keystores 

This role is used to create and delete EP11 keystores but does not have permissions to manage keys.



1. In the IBM Cloud® console, go to Manage > Access (IAM), and select Roles.
2. Click Create.
3. Enter a name for your role; for example, keystore operator. This name must be unique within the account. Users see this role name in the console when they assign access to the service.
4. Enter an ID for the role. This ID is used in the CRN, which is used when you assign access by using the API. The role ID must begin with a capital letter and use alphanumeric characters only; for example, KeystoreOperator
5. Optional: Enter a succinct and helpful description that helps the users who are assigning access know what level of access this role assignment gives a user. This description also shows in the console when a user assigns access to the service.
6. From the list of services, select Hyper Protect Crypto Services.
7. Select Add for the following actions:
* UI
* API







 Managing key aliases 

You can use Hyper Protect Crypto Services to manage key aliases with the Hyper Protect Crypto Services API.

Key aliases are unique human-readable names that can be used to identify a key. Aliases enable your service to refer to a key by recognizable custom names, rather than the auto-generated identifier provided by Hyper Protect Crypto Services. Assume that you create a key that has the ID 02fd6835-6001-4482-a892-13bd2085f75d and it is aliased as US-South-Test-Key. You can use US-South-Test-Key to refer to your key when you make calls to the Hyper Protect Crypto Services API to [retrieve a key](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-retrieve-key) or the [metadata](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-retrieve-key-metadata).

Before you manage key alias for keys in Hyper Protect Crypto Services, keep in mind the following considerations:



* An alias is independent from a key.

An alias is its own resource and any actions that are taken on it do not affect the associated key. For example, deleting an alias does not delete the associated key.
* An alias can be associated with only one key at a time.

An alias can be associated with only one key that is located in the same instance and region. If you want to change the key that the alias is associated with, you need to perform the following steps:



1. Delete the alias.
2. Wait up to 10 minutes.
3. Re-create the alias and map it to the key.



* You can create an alias with the same name in a different instance or region.

Each alias is associated with a different key in each instance or region, with which, your service's application code can be reusable in different instances or regions.
Use cases - Standard Plan 

IBM Cloud® Hyper Protect Crypto Services can be used as a key management service to pervasively protect data at rest in the IBM Cloud as well as a cloud HSM for general-purpose cryptographic operations.



 Pervasively protecting data at rest in the cloud 

With the integration in the IBM Cloud Security Architecture, you can use Hyper Protect Crypto Services to encrypt your data at rest for cloud data and storage services in the highest security level with your own keys. The service uses the same key-provider API as Key Protect to provide a consistent envelope encryption and file system encryption approach to adopting IBM Cloud services.

IBM Cloud Hyper Protect Crypto Services supports Keep Your Own Key (KYOK) so that you have more control and authority over your data with encryption keys that you can bring, control, and manage. At the same time, you still maintain the control over the HSM that the encryption key is located in. With Hyper Protect Crypto Services, you fully leverage the proven technology that is co-developed and operated by large enterprises for managing their most sensitive data.

Encryption keys that are generated and protected by Hyper Protect Crypto Services can be used to provide application record-level or field-level encryption to protect data from other insider threats such as from database administrators.



 Data at rest encryption with KYOK 

Keys are protected by customer-managed dedicated HSMs, which means, only you have access to your data. The cryptographic capabilities of Hyper Protect Crypto Services are built on top of a FIPS 140-2 Level 4 Certified Hardware Security Module. You can benefit from the cryptographic capabilities of Hyper Protect Crypto Services for both your new and existing workloads. The IBM Key Protect for IBM Cloud API is integrated for generating and protecting encryption keys.
This value can be any valid URL such as an organization-specific URL that is related to your organization's HashiCorp Vault server Web UI console or documentation.
6. Type the path to the secrets on the HashiCorp Vault server that is accessible when you use the configured authentication method and the associated authentication credentials. This value must include only the secrets path and cannot include a secrets name at the end of the secrets path. If your authentication credentials contain the relevant LIST permission within the specified secrets path, you can view the list of secrets within that path.
7. Click Create Integration.
8. On the Toolchain's Overview page, on the Third-Party tools card, click HashiCorp Vault to use the HashiCorp Vault Secrets Picker and Pusher components to select secrets for use within your toolchain or delivery pipeline.





 Configuring HashiCorp Vault by using the API 

The HashiCorp Vault tool integration supports the following configuration parameters that you can use with the [Toolchain HTTP API and SDKs](https://cloud.ibm.com/apidocs/toolchain) when you [create](https://cloud.ibm.com/apidocs/toolchaincreate-tool), [read](https://cloud.ibm.com/apidocs/toolchainget-tool-by-id), and [update](https://cloud.ibm.com/apidocs/toolchainupdate-tool) tool integrations.

You must specify the tool_type_id property in the request body with the hashicorpvault value.



Table 1. HashiCorp Vault tool integration parameters

 Parameter Usage Type Terraform argument Description 

 authentication_method required, updatable String authentication_method The authentication method for your HashiCorp Vault instance. 
 dashboard_url required, updatable String dashboard_url The URL of the HashiCorp Vault server dashboard for this tool integration. In the graphical UI, the browser goes to this dashboard when you click the HashiCorp Vault tool integration card. 
 default_secret optional, updatable String default_secret The default secret name that is used if your HashiCorp Vault instance does not return a list of secret names.
"algorithmType": "AES",
"algorithmMetadata": {
"bitLength": "256",
"mode": "CBC_PAD"
},
"algorithmBitSize": 256,
"algorithmMode": "CBC_PAD",
"lastUpdateDate": "2020-03-05T16:39:25Z",
"keyVersion": {
"id": "436901cb-f4e4-45f4-bd65-91a7f6d13461",
"creationDate": "2020-03-05T16:39:25Z"
},
"dualAuthDelete": {
"enabled": false
},
"deleted": false
}
]
}

The payload or key material for a root key stays within the bounds of a hardware security module and cannot be retrieved.

For a detailed description of the response parameters, see the Hyper Protect Crypto Services key management [REST API reference doc](https://cloud.ibm.com/apidocs/hs-crypto).
The unique identifier that is assigned to your Hyper Protect Crypto Services service instance. For more information, see [Retrieving an instance ID](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-retrieve-instance-ID). 



A successful sync API request returns an HTTP 204 No Content response, which indicates that the IBM cloud service that is associated with the specified key is notified.



The sync API can be initialized only when it is longer than an hour since the last notification to the associated cloud services of the key. If you send a request to this API and the key is synced or a key lifecycle action is taken within the past hour, the API returns a 409 Conflict response.
The service also helps your business meet regulatory compliance with the technology that provides exclusive controls on the external and privileged user access to data and keys.





 How does Hyper Protect Crypto Services work? 

The following items are a few highlights of the Hyper Protect Crypto Services architecture:



* Applications connect to Hyper Protect Crypto Services through the PKCS #11 API or the GREP11 API.
* Dedicated keystore in Hyper Protect Crypto Services is provided to ensure data isolation and security. Privileged users are locked out for protection against abusive use of system administrator credentials or root user credentials.
* Secure Service Container (SSC) provides the enterprise level of security and impregnability that enterprise customers expect from [IBM LinuxONE](https://www.ibm.com/it-infrastructure/linuxone) technology.
* FIPS 140-2 Level 4 compliant cloud HSM is enabled for highest physical protection of secrets.



For an architectural diagram of Hyper Protect Crypto Services, see [Service architecture, workload isolation, and dependencies](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-architecture-workload-isolation).





 Key features 

Hyper Protect Crypto Services provides both key management and cloud HSM functions:



 Key management service 



* Key lifecycle management

Hyper Protect Crypto Services provides a single-tenant key management service to create, import, rotate, and manage keys with the standardized API. After the encryption keys are deleted, you can be assured that your data is no longer retrievable.
* Encryption for IBM Cloud data and workload services

By integrating with other IBM Cloud services, Hyper Protect Crypto Services offers the capability of bringing your own encryption to the cloud. The service provides double-layer protection for your cloud data by wrapping the encryption keys that are associated with your cloud services.
* Access management and auditing

Hyper Protect Crypto Services integrates with Cloud Identity and Access Management (IAM) to enable your granular control over user access to service resources. For more information, see [Managing user access](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-manage-access).
* UI
* CLI
* API







 Importing root keys 

You can use IBM Cloud® Hyper Protect Crypto Services to secure your existing root keys by using the IBM Cloud console, or programmatically with the Hyper Protect Crypto Services key management service API.

Root keys are symmetric key-wrapping keys that are used to protect the security of encrypted data in the cloud. For more information about importing root keys, see [Bringing your encryption keys to the cloud](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-importing-keys).

Plan ahead for importing keys by [reviewing your options for creating and encrypting key material](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-importing-keysplan-ahead). For added security, you can enable the secure import of the key material by using an [import token](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-importing-keysusing-import-tokens) to encrypt your key material before you bring it to the cloud.



 Importing root keys with the IBM Cloud console 

After you [create an instance of the service](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-provision), complete the following steps to add your existing root key with the Hyper Protect Crypto Services GUI.



1. [Log in to the IBM Cloud console](https://cloud.ibm.com/).
2. Go to Menu > Resource list to view a list of your resources.
3. From your IBM Cloud resource list, select your provisioned instance of Hyper Protect Crypto Services.
4. To import a key, select the KMS keys tab in the side menu.
5. In the Keys table, click Add key and select Import a key.

Specify the key's details:



Table 1. Describes the settings for importing a root key

 Setting Description 

 Key type The type of key that you would like to manage in Hyper Protect Crypto Services.
* To learn more about Hyper Protect Crypto Services concepts and terminologies, check out [Components and concepts](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-understand-concepts).
* If you are using the Standard Plan, manage your keys with [Hyper Protect Crypto Services key management service API](https://cloud.ibm.com/apidocs/hs-crypto) and [Key Protect CLI](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-set-up-cli).
* If you are using the Unified Key Orchestrator Plan, manage your keys and keystores with [manage your keys with Unified Key Orchestrator API](https://cloud.ibm.com/apidocs/uko).
* Encrypt your data and perform cryptographic operations with the

[PKCS #11 API](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-pkcs11-api-ref) or the [GREP11 API](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-grep11-api-ref).
Using a TransferManager class, the SDK handles all the necessary logic for downloading objects in parallel parts.





 Extended bucket listing 

This extension to the S3 API returns a list of buckets with their LocationConstraint.
All buckets in a service instance are always returned on a list request, not just the subset that is located in the region of the target endpoint. This API is useful for finding where a bucket is located.





 Version 2 object listing 

Version 2 listing allows for more powerful scoping of object listings.





 Key Protect 

[IBM® Key Protect for IBM Cloud®](https://cloud.ibm.com/docs/services/key-protect?topic=key-protect-about) helps you create encrypted keys for apps across IBM Cloud® services. Keys are secured by FIPS 140-2 Level 3 cloud-based hardware security modules (HSMs) that protect against the theft of information. [Hyper Protect Crypto Services](https://cloud.ibm.com/docs/services/hs-crypto?topic=hs-crypto-overview) is a single-tenant, dedicated HSM that is controlled by you. The service is built on FIPS 140-2 Level 4 hardware, the highest offered by any cloud provider in the industry.





 SSE-C 

IBM Cloud® Object Storage provides several options to encrypt your data. By default, all objects that are stored in IBM Cloud Object Storage are encrypted by using randomly generated keys and an all-or-nothing-transform (AONT). While this default encryption model provides at-rest security, some workloads need full control over the data encryption keys used. You can manage your keys manually by supplying your own encryption keys - referred to as Server-Side Encryption with Customer-Provided Keys (SSE-C).





 Archive rules 

IBM Cloud® Object Storage Archive is a low-cost option for data that is rarely accessed. You can migrate data from any of the storage tiers (Standard, Vault, Cold Vault, and Flex) to a long-term offline archive.





 Retention policies 

Immutable Object Storage maintains data integrity in a WORM (Write-Once-Read-Many) manner.
* UI
* API







 Creating root keys 

You can use IBM Cloud® Hyper Protect Crypto Services to create

root keysby using the Hyper Protect Crypto Services console, or programmatically with the Hyper Protect Crypto Services key management service API.

Root keys are symmetric key-wrapping keys that are used to protect the security of encrypted data in the cloud. For more information about root keys, see [Envelope encryption](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-envelope-encryption).



 Creating root keys with the console 

[After you create an instance of the service](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-provision), complete the following steps to create a root key with the IBM Cloud console.

If you enable [dual authorization settings for your Hyper Protect Crypto Services instance](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-manage-dual-auth), keep in mind that any keys that you add to the service require an authorization from two users to delete keys.



1. [Log in to the IBM Cloud console](https://cloud.ibm.com/login).
2. Go to Menu > Resource list to view a list of your resources.
3. From your IBM Cloud resource list, select your provisioned instance of Hyper Protect Crypto Services.
4. To create a new key, select the KMS keys tab in the side menu.
5. In the Keys table, click Add key, and select Create a key.

Specify the key's details:



Table 1. Describes the settings to create a key

 Setting Description 

 Key type The type of key that you would like to manage in Hyper Protect Crypto Services. From the list of key types, select [Root key](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-understand-conceptsroot-key-concept). 
 Key name A unique, human-readable name for easy identification of your key.
oc get secrets -A | grep cos







 Optional: Setting up encryption by using Hyper Protect Crypto Services 

If you want to set up encryption, create an instance of Hyper Protect Crypto Services or Key Protect. Then, create a root key, and a Kubernetes secret that uses your Hyper Protect Crypto Services or Key Protect credentials.

Storage class encryption is available only for versions 4.10.0 and later of OpenShift Data Foundation.



1. Create an [Hyper Protect Crypto Services](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-provision&interface=ui) or [Key Protect](https://cloud.ibm.com/docs/key-protect?topic=key-protect-provision) service instance.
2. Create root key



* [Hyper Protect Crypto Services](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-create-root-keys&interface=ui).
* [Key Protect](https://cloud.ibm.com/docs/key-protect?topic=key-protect-create-root-keys&interface=ui).



3. After creating your instance and root key, make a note of your Hyper Protect Crypto Services or Key Protect instance name, instance ID, root key ID, and public endpoint.
4. Create a [service ID](https://cloud.ibm.com/docs/account?topic=account-serviceids), [API key](https://cloud.ibm.com/docs/account?topic=account-serviceidapikeys), and [access policy](https://cloud.ibm.com/docs/account?topic=account-assign-access-resources) that allows access to either Hyper Protect Crypto Services and Red Hat OpenShift on IBM Cloud or Key Protect and Red Hat OpenShift on IBM Cloud. Make a note of the API that you create.



[Access your Red Hat OpenShift cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-access_cluster).



1.