vania-janet/multiturn-rag-retrieval-data huggingface dataset:
99.999999999% <br>(11 9's) 99.999% <br>(5 9's) Provider-managed AES-256 encryption, Customer-managed encryption 
 5 IOPS per GB tier It is designed for high I/O intensity workloads that are characterized by a large percentage of active data, such as transactional and other performance-sensitive databases. 99.999999999% <br>(11 9's) 99.999% <br>(5 9's) Provider-managed AES-256 encryption, Customer-managed encryption 
 10 IOPS per GB tier It is designed for demanding storage workloads such as data-intensive workloads created by NoSQL databases, data processing for video, machine learning, and analytics. 99.999999999% <br>(11 9's) 99.999% <br>(5 9's) Provider-managed AES-256 encryption, Customer-managed encryption 
 custom Customers can specify capacity between 10 - 16000 MB with IOPS ranging 100 - 48000. 99.999999999% <br>(11 9's) 99.999% <br>(5 9's) Provider-managed AES-256 encryption, Customer-managed encryption 



File Storage for VPC shares are encrypted by using provider-managed encryption or customer-managed encryption keys.



 Durability 

Think of durability as a measurement of how healthy and resilient your data is. Durability in VPC storage means that your data is stored consistent and intact without any signs of data decay, influence of drive failures, or any other form of corruption. 99.999999999% (11 nines) durability means that if you store 10 million files, then you expect to lose one file every 10000 years.

When people hear the word durability, most of them think of hardware failures of storage, compute, and network components that might cause data loss. In VPC storage, your data is protected against drive failures and numerous type of disk errors that otherwise might negatively impact data durability and data integrity.
correlation_ID Optional.The unique identifier that is used to track and correlate transactions. 
 additional_data Optional.The additional authentication data (AAD) that is used to further secure the key. Each string can hold up to 255 characters. If you supply AAD when you make a wrap call to the service, you must specify he same AAD during the unwrap call. 
 encrypted_data_key Required. The ciphertext value that was returned during a wrap operation. 



The original key material is returned in the response entity-body. The response body also contains the ID of the key version that was used to unwrap the supplied ciphertext.

The plaintext that is returned is base64 encoded. For more information on how to decode your key material, see [Decoding your key material](https://cloud.ibm.com/docs/key-protect?topic=key-protect-unwrap-keyshow-to-decode-key-material). The following JSON object shows an example returned value.

{
"plaintext": "Rm91ciBzY29yZSBhbmQgc2V2ZW4geWVhcnMgYWdv",
"keyVersion": {
"id": "02fd6835-6001-4482-a892-13bd2085f75d"
}
}

If Key Protect detects that you rotated the root key that is used to unwrap and access your data, the service also returns a newly wrapped data encryption key (ciphertext) in the unwrap response body. The latest key version (rewrappedKeyVersion) that is associated with the new ciphertext is also returned. Store and use the new ciphertext value for future envelope encryption operations so that your data is protected by the latest root key.





 Decoding your key material 

When you unwrap a data encryption key, the key material is returned in base64 encoding. You will need to decode the key before encrypting it.



 Using OpenSSL to decode key material 



1.
When the embedding application provides a data source module specification, then the calling application provides a module to Cognos Dashboard Embedded, for exampling with the methods addSources() or updateModuleDefinitions(), then you can choose to provide any of the following fields in an encrypted form:



* user
* password
* schema
* jdbcUrl
* sourceUrl
* value of a header property for CSV urls



Before sending the module, your application uses the public key to encrypt the sensitive fields of the module.

If you have a string to encrypt that is longer than 245 bytes, then do the following steps:



1. Divide the string that you want to encrypt in parts that are a maximum of 245 bytes.
2. Encrypt the parts from step 1 and make sure the are base64 encoded.
3. Concatenate the encode parts from step 2 with the colon ":" character.
4. Use the value from step 3 in the encrypted field.



Example of a JDBCurl:
EncryptUpdateResponse, err := cryptoClient.EncryptUpdate(context.Background(), EncryptUpdateRequest2)

ciphertext = append(ciphertext, EncryptUpdateResponse.Ciphered...)
Show more
* JavaScript code snippet

client.EncryptUpdate({
State: state,
Plain: Buffer.from(message.substr(20))
}, (err, data={}) => {
cb(err, data.State, Buffer.concat([ciphertext, data.Ciphered]));
});







 EncryptFinal 

The EncryptFinal function finishes a multiple-part encryption operation.

Enterprise PKCS #11 over gRPC

Enterprise PKCS #11

PKCS #11





 Description Binds to EP11 m_EncryptFinal, which is an implementation of PKCS #11 C_EncryptFinal. 
 Parameters message EncryptFinalRequest { bytes State = 1; } message EncryptFinalResponse { bytes Ciphered = 2; } 
 Return values Wraps EP11 error into message Grep11Error. 



Code snippets



* Golang code snippet

EncryptFinalRequest := &pb.EncryptFinalRequest {
State: EncryptUpdateResponse.State,
}

EncryptFinalResponse, err := cryptoClient.EncryptFinal(context.Background(), EncryptFinalRequest)
* JavaScript code snippet

client.EncryptFinal({
State: state
}, (err, data={}) => {
cb(err, Buffer.concat([ciphertext, data.Ciphered]));
});







 EncryptSingle 

The EncryptSingle function processes data in one pass with one call.
Db2 default encryption uses a two-tiered key hierarchy: Data is encrypted with a data encryption key (DEK). The DEK is encrypted with a master key and is stored in encrypted form with the database or the backup image. A unique DEK is generated by Db2 for each encrypted database and for each encrypted backup. A master key is used to encrypt a DEK. Each encrypted database is associated with one master key at one time.

Zoom

![IBM Db2 default encryption by using the standard PKCS #11 API](https://cloud.ibm.com/docs-content/v1/content/217f108cc44e2ce15fb692d4b57b4c628464e908/hs-crypto//images/pkcs-db2.svg)

Figure 6. IBM Db2 default encryption by using the standard PKCS #11 API



With the PKCS #11 library integration, Hyper Protect Crypto Services supports the industry-standard PKCS #11 API. The Hyper Protect Crypto Services PKCS #11 library connects your database to Hyper Protect Crypto Services to perform cryptographic operations. The database system can invoke operations to manage the TDE master encryption keys or the master keys in the Hyper Protect Crypto Services PKCS #11 library. The Hyper Protect Crypto Services PKCS #11 library then interacts with your Hyper Protect Crypto Services instance to provide the highest level of security for storing and managing your TDE master encryption keys or your master keys in the cloud. It, in turn, provides the highest level of security to your data encryption keys and your data.



* For a tutorial on how to use TDE with Hyper Protect Crypto Services, see

[Tutorial: Using Oracle Transparent Database Encryption with Hyper Protect Crypto Services PKCS #11](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-tutorial-tde-pkcs11).
* For a tutorial on how to use Db2 default encryption with Hyper Protect Crypto Services, see
* Managers create keys and control their lifecycle (in addition to being able to do everything Readers and Writers can do).





 Architecture overview 

Key Protect uses the Advanced Encryption Standard algorithm in Galois/Counter Mode (AES GCM) to wrap and unwrap DEKs. CRKs that are not imported are created with 256-bit key material. Imported CRKs can be have 128, 192, or 256-bit key material.

The following architecture diagram shows how Key Protect components work to protect your sensitive data and keys.

Zoom

![The diagram shows how Key Protect components protect sensitive data and keys.](https://cloud.ibm.com/docs-content/v1/content/0bc8db34a3d90b2a5103c9c71b08a96de9525bbc/key-protect/images/kp-architecture.svg)

Figure 2. Key Protect architecture

Access to the Key Protect service takes place over HTTPS. All communication uses the Transport Layer Security (TLS) protocol to encrypt data in transit. For more information about TLS and the ciphers supported by Key Protect, check out [Data encryption](https://cloud.ibm.com/docs/key-protect?topic=key-protect-security-and-compliancedata-security).



Table 2. Key Protect service components

 Components Description 

 Key Protect REST API The Key Protect REST API drives encryption key creation and management across IBM Cloud services. 
 IBM-managed hardware security module IBM Cloud data centers provide the hardware to protect your keys. Hardware security modules (HSMs) are tamper-resistant hardware devices that store and use cryptographic key material without exposing keys outside of a cryptographic boundary. All cryptographic operations, such as key creation and key rotation, are performed within the HSM. IBM periodically rotates the HSM's master keys, providing an extra layer of security. 
 Customer-managed encryption keys Root keys are symmetric keys that protect data encryption keys with [envelope encryption](https://cloud.ibm.com/docs/key-protect?topic=key-protect-envelope-encryption). Root keys never leave the boundary of the HSM.
* Access and data management: Access to the customer-owned IBM Cloud Object Storage service instance is controlled by IAM.
* etcd Backup: Every 8 hours
* Location control plane Backup: Every hour
* Encryption: Data is automatically encrypted by using the default built-in encryption mechanisms in IBM Cloud Object Storage. You can further choose to protect your data by using a root key in IBM Key Protect and use the key to encrypt the data in your bucket. For more information, see the [IBM Cloud Object Storage documentation](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-encryption).



The time to recover a location or cluster is dependent on the size of the location or cluster and the network latency between IBM Cloud and your host infrastructure.





 Where can I find information about the data that my managed services store? 

You can find information about the data that is stored by your managed service by reading the data security topic for each service.





 Which IBM Cloud region is my information stored in? 

Where your Satellite information is stored depends on the IBM Cloud region that manages the control plane of your Satellite location. By selecting the IBM Cloud region that is closest to the infrastructure provider for your Satellite location, your data is automatically spread across zones in that region for high availability. Because the zones of an IBM Cloud region might be in a different city or country than the infrastructure hosts that you bring to your Satellite location, make sure that your data can be stored in the selected IBM Cloud region.





 How can I remove my information? 

Review your options to remove your personal and sensitive information from IBM Cloud Satellite.

Removing personal and sensitive information is permanent and non-reversible. Make sure that you want to permanently remove your information before you proceed.

[Deleting a location](https://cloud.ibm.com/docs/satellite?topic=satellite-host-removelocation-remove-console) does not remove all information from IBM Cloud Satellite. When you delete a location, location-specific information is removed from the etcd instance that is managed by IBM. However, your information still exists in the following places.
[10]: Flex is used for dynamic workloads where access patterns are more difficult to predict.

Objects that are subject to a bucket's Immutable Object Storage retention policy will have expiration actions deferred until the retention policy is no longer enforced.





 Encrypt data with your own key 

By default, all objects that are stored in COS are encrypted by using randomly generated keys and an all-or-nothing-transform (AONT). When you archive data into your COS instance, your objects are automatically encrypted with data encryption keys (DEKs). When you need to access a bucket, the service checks your user permissions and decrypts the objects within the bucket for you. This encryption model is called provider-managed encryption.

While this default encryption model provides at-rest security, some workloads need full control over the data encryption keys used.

COS provides several options to encrypt your data. This encryption model is called customer-managed encryption:



* You can manage your keys manually by providing your own encryption keys - referred to as Server-Side Encryption with Customer-Provided Keys (SSE-C).
* You can choose to use the integration capabilities with IBM Cloud® Key Management Services like IBM® Key Protect and Hyper Protect Crypto Services.

You can use IBM® Key Protect to provision encrypted keys for apps across IBM Cloud services. As you manage the lifecycle of your keys, you can benefit from knowing that your keys are secured by FIPS 140-2 Level 3 certified cloud-based hardware security modules (HSMs) that protect against the theft of information. When you integrate COS with the Key Protect service, you can add envelope encryption to your DEKs. In Key Protect, you can provision highly secure root keys, which serve as primary keys that you control in the service. When you create a bucket, you can configure envelope encryption for the bucket at its creation. This added protection wraps (or encrypts) the DEKs associated with the bucket by using a root key that you manage in Key Protect. The practice, called key wrapping, uses multiple AES algorithms to protect the privacy and the integrity of your DEKs, so only you control access to their associated data.
$ ibmcloud kp key wrap 807eb0a6-cc10-4bfe-8331-41a6f712c4ea

Wrapping key...
OK
Ciphertext
eyJjaXBoZXJ0ZXh0IjoiKzhjbHVqcUNP ...<redacted>... NmY3MTJjNGViIn0=

 capture the ciphertext
$ CIPHERTEXT=eyJjaXBoZXJ0ZXh0IjoiKzhjbHVqcUNP ...<redacted>... NmY3MTJjNGViIn0=

 unwrap the ciphertext to reveal the plaintext (DEK)
$ ibmcloud kp key unwrap 807eb0a6-cc10-4bfe-8331-41a6f712c4ea $CIPHERTEXT

Unwrapping key...
OK
Plaintext Rewrapped Plaintext
mXW4GmBhkRG1+Dzkx1D6dMX99a4KyYwtwbghaaLEQZ0=





 Example 2 

Generate a base64-encoded data encryption key (DEK) and wrap it with the root key.

 create a root key
$ KEY_ID=$(ibmcloud kp key create my-root-key --output json | jq -r '.["id"]')

$ echo $KEY_ID

9052d914-bebe-4177-90e9-6073a167b507

 create a random, base64-encoded, 32-byte data encryption key (DEK)
$ PLAINTEXT=$(openssl rand -base64 32)

$ echo $PLAINTEXT

tEqoiCu9CeUsbqRe6yBUkrJg80QKKG/z46OkX0YQMV8=

 wrap the plaintext key with the root key, creating a DEK (this is the ciphertext)
You own the KEK, which you create as a root key in the Key Protect service. The Data Engine service never sees the root (KEK) key. Its storage, management, and use to wrap and unwrap the DEK is performed entirely within the key management service. If you disable or delete the key, the data can no longer be decrypted.

Key Protect keys are secured by FIPS 140-2 Level 3 certified cloud-based

hardware security modules (HSMs). For more information, see [Bringing your encryption keys to the cloud](https://cloud.ibm.com/docs/key-protect?topic=key-protect-importing-keys).



 Working with customer-managed keys 

You can use IBM Cloud® Activity Tracker to audit the lifecycle events of your keys, such as creating a key, deleting a key, rotating a key, and more. For more information, see [IBM Cloud Activity Tracker events for Key Protect](https://cloud.ibm.com/docs/key-protect?topic=key-protect-at-events).







 Setting up Key Protect encryption 



1. Provision [Key Protect](https://cloud.ibm.com/docs/services/key-protect?topic=key-protect-about) on your IBM Cloud account.
2. Go to your instance of Key Protect and generate or enter a [root key](https://cloud.ibm.com/docs/services/key-protect?topic=key-protect-getting-started-tutorial).
3. Create a new Standard plan instance of Data Engine and select the Key Protect key for encrypting your queries. For performance reasons, create both the Key Protect and Data Engine instances in the same IBM Cloud region.
4. Give the new Data Engine instance [access](https://cloud.ibm.com/docs/account?topic=account-serviceauth) to your Key Protect key.



* Open your IBM Cloud dashboard.
* From the menu bar, click Manage > Access (IAM), and select Authorizations.
* Click Create.
* In the source service menu, select IBM Cloud Data Engine.
Rewrapping keys 

Reencrypt your data encryption keys by using the IBM® Key Protect API.

When you [rotate a root key in Key Protect](https://cloud.ibm.com/docs/key-protect?topic=key-protect-key-rotation), new cryptographic key material becomes available for protecting the data encryption keys (DEKs) that are associated with the root key. With the rewrap API, you can reencrypt or rewrap your DEKs without exposing the keys in their plaintext form.

To learn how envelope encryption helps you control the security of at rest data in the cloud, see [Protecting data with envelope encryption](https://cloud.ibm.com/docs/key-protect?topic=key-protect-envelope-encryption).



 Rewrapping keys by using the API 

You can reencrypt a specified data encryption key (DEK) with a root key that you manage in Key Protect, without exposing the DEK in its plaintext form.

Rewrapping keys works by combining unwrap and wrap calls to the service. For example, you can emulate a rewrap operation by first calling the unwrap API to access a DEK, and then calling the wrap API to reencrypt the DEK by using the newest root key material.

[After you rotate a root key in the service](https://cloud.ibm.com/docs/key-protect?topic=key-protect-rotate-keys), rewrap a data encryption key that is associated with the root key by making a POST call to the following endpoint.

https://<region>.kms.cloud.ibm.com/api/v2/keys/<keyID_or_alias>/actions/rewrap



1. [Retrieve your service and authentication credentials to work with keys in the service](https://cloud.ibm.com/docs/key-protect?topic=key-protect-set-up-api).
2. Copy the ID of the rotated root key that you used to perform the initial wrap request.
IBM Key Protect docs 

IBM® Key Protect is a full-service encryption solution that allows data to be secured and stored in IBM Cloud™ using the latest envelope encryption techniques. Encryption keys can be provisioned or imported for use with applications for many IBM Cloud™ services and then managed from a central location.

 Developer tools 

[API & SDK reference](https://cloud.ibm.com/apidocs/key-protect)[CLI reference](https://cloud.ibm.com/docs/key-protect?topic=key-protect-key-protect-cli-reference)[Terraform reference](https://registry.terraform.io/modules/terraform-ibm-modules/kms/ibm/latest/submodules/key-protect)

 Recommended content 

[Getting started with Key Protect Learn how to create or import existing cryptographic keys using the Key Potect dashboard.](https://cloud.ibm.com/docs/key-protect?topic=key-protect-getting-started-tutorial)[Protecting data with envelope encryption Learn how envelope encryption, which forms the backbone of Key Protect, works to secure your data.](https://cloud.ibm.com/docs/key-protect?topic=key-protect-envelope-encryption)[Grouping keys together using key rings Learn how to use use Key Protect to create a group of keys for a target group of users that require the same IAM access permissions.](https://cloud.ibm.com/docs/key-protect?topic=key-protect-grouping-keys)

 Learn more 

[IBM Developer<br><br>Visit IBM Developer for technical articles, code patterns, tutorials, and more.<br><br>![112-Developer-tools.svg](https://cloud.ibm.com/media/docs/images/homepage/112-Developer-tools.svg)](https://developer.ibm.com/depmodels/cloud/)[Architecture Center<br><br>Discover the architecture references available for this product.<br><br>!
Integrating with IBM Cloud Object Storage 

IBM® Key Protect for IBM Cloud® and IBM Cloud® Object Storage work together to help you own the security of your at rest data. Learn how to add advanced encryption to your IBM Cloud® Object Storage resources by using the IBM Key Protect service.



 About IBM Cloud Object Storage 

IBM Cloud Object Storage provides cloud storage for unstructured data. Unstructured data refers to files, audio/visual media, PDFs, compressed data archives, backup images, application artifacts, business documents, or any other binary object.

To maintain data integrity and availability, IBM Cloud Object Storage slices and disperses data to storage nodes across multiple geographic locations. No complete copy of the data resides in any single storage node, and only a subset of nodes needs to be available so you can fully retrieve the data on the network.

Provider-side encryption is provided, so your data is secured at rest and in flight. To manage storage, you create buckets and import objects with the IBM Cloud console, or programmatically by using the [IBM Cloud Object Storage REST API](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-compatibility-api).

For more information, see [About COS](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-about-cloud-object-storageabout-cloud-object-storage).





 How the integration works 

Key Protect integrates with IBM Cloud Object Storage to help you achieve full control of the security of your data.

As you move data into your instance of IBM Cloud Object Storage, the service automatically encrypts your objects with data encryption keys (DEKs).

Within IBM Cloud Object Storage, DEKs are stored in the service securely, near the resources that they encrypt. When you need to access a bucket, the service checks your user permissions and decrypts the objects within the bucket for you. This encryption model is called provider-managed encryption.

To enable the security benefits of customer-managed encryption, you can add envelope encryption to your DEKs in IBM Cloud Object Storage by integrating with the Key Protect service.
{region}.secrets-manager.appdomain.cloud/api/v2/secrets/arbitrary/{id}" | jq --raw-output '.payload | sub(".,"; "")'

If you inspect the contents of $ARBITRARY_SECRET, you see base64 encoded data. The following snippet shows an example output.

echo $ARBITRARY_SECRET
eUdB68klDSrzSKgWcQS5...(truncated)

To view the secret in its original form (binary file), you can use base64 decoding. The following example uses the base64 macOS utility to base64 decode the $ARBITRARY_SECRET contents.

echo $ARBITRARY_SECRET | base64 --decode > my-secret.png

The data is converted back to a binary file that you can open from your local computer.





 Downloading the previous version of a certificate 

After you rotate a certificate, you can programmatically access its previous version by using the Secrets Manager API.

The following example request retrieves a secret and its contents. When you call the API, replace the ID variables and IAM token with the values that are specific to your Secrets Manager instance.

curl -X GET
--header "Authorization: Bearer {iam_token}" --header "Accept: application/json" "https://{instance_ID}.{region}.secrets-manager.appdomain.cloud/api/v2/secrets/{id}/versions/previous"