vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Planning data encryption 

When you're planning a data encryption strategy for your Block Storage for VPC volumes, snapshots, File Storage for VPC shares, or custom images, you might find this checklist helpful.



 Planning for data encryption 

Consider the following prerequisites before you set up data encryption for your VPC resources.



Table 1. Checklist for planning data encryption

 Considerations 

 __ Evaluate the amount of control that you want over your data encryption. [IBM-managed encryption](https://cloud.ibm.com/docs/vpc?topic=vpc-vpc-encryption-aboutvpc-provider-managed-encryption) is provided by default for boot volumes, data volumes, and file shares. With [customer-managed encryption](https://cloud.ibm.com/docs/vpc?topic=vpc-vpc-encryption-aboutvpc-customer-managed-encryption), you own the encryption keys and control the encryption process. 
 __ For encrypted custom images, review the image requirements, supported operating systems, and learn about creating and importing QCOW2 custom image files. For more information, see [Planning for custom images](https://cloud.ibm.com/docs/vpc?topic=vpc-planning-custom-images). 
 __ Choose the UI, CLI, or API for provisioning customer-managed encryption. 
 __ Evaluate which [key management service](https://cloud.ibm.com/docs/vpc?topic=vpc-vpc-encryption-planningbyok-encryption-prereqs) best meets your needs. Determine the availability of these services in your region and zone. 
 __ Determine whether your account can authorize access: <br>For Cloud Block Storage as the [source service](https://cloud.ibm.com/docs/vpc?topic=vpc-vpc-encryption-planningbyok-volumes-prereqs), Lite accounts must [upgrade to a Pay-As-You-Go](https://cloud.ibm.com/docs/account?topic=account-upgrading-accountupgrade-paygo) account or a Subscription account.
In fact, the system can lose multiple disks in the cluster simultaneously without degrading customer performance or adverse risk of data loss. Redundant network ports and paths protect against network failures across the cloud connections.





 Encryption 

IBM Cloud® provides full-disk encryption without compromising storage application performance. For more information about encryption of File Storage for Classic, see [Securing your data in File Storage](https://cloud.ibm.com/docs/FileStorage?topic=FileStorage-mng-data). For more information about provider- and customer-managed encryption in a VPC, see [Data encryption for VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-vpc-encryption-aboutvpc-customer-managed-encryption).
They are symmetric key-wrapping keys used as roots of trust for wrapping (encrypting) and unwrapping (decrypting) other keys stored in a data service. Root keys contain key material that is used to wrap and unwrap DEKs. The key material of the root key can either be imported or generated by the Key Protect service.
: With Key Protect, you can create, store, and manage the lifecycle of root keys to achieve full control of other DEKs stored in the cloud. Unlike a standard key, the plaintext of a root key can never leave the bounds of the Key Protect service.

Standard keys
: Standard keys, which can be used as DEKs, can store encrypted data in Key Protect. Once your plaintext standard key is generated and protecting underlying data, destroy the plaintext and safely store the encrypted standard key.





 Wrapping keys with envelope encryption 

You can wrap one or more DEKs with advanced encryption algorithms by designating a root key in Key Protect that you can fully manage.

Complete the following steps to encrypt data via envelope encryption:



1. Generate or provision a DEK from an IBM Cloud service, such as [COS](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-encryption), to encrypt your sensitive data. The DEK will encrypt your sensitive data.
2. [Generate](https://cloud.ibm.com/docs/key-protect?topic=key-protect-create-root-keys) or [import](https://cloud.ibm.com/docs/key-protect?topic=key-protect-import-root-keys) a root key that will be used to protect the DEK from step 1.
3. Use the root key to encrypt, or [wrap](https://cloud.ibm.com/docs/key-protect?topic=key-protect-wrap-keys) the DEK. To provide maximum encryption security, you can specify additional authenticated data (AAD) during your wrap request.
The information provided in this section is used to mount the attached data volume (provided by the user) and is later encrypted using the "seeds" provided in the workload and env sections. You can provide any path of your choice for the "mount" field. The path provided by the user is used internally to mount the data volume. The mount path provided in the contract must match the path provided under the volumes section of the docker-compose.yaml file, so that all the data associated with the container workload is stored in this data volume.

The volumes subsection has support for auto encryption of the data volume with user-provided seeds. If a data volume is attached to the Hyper Protect Virtual Servers instance, it is encrypted automatically with the seeds that are provided through the "seed" field in the volumes subsections of the contract. Thus two seeds must be provided, one through the workload section (by the workload persona) and the other through the env section (by the deployer persona). These two seeds are internally converted to UTF8 sequences and then concatenated. Later, the hash (SHA256) of the concatenated sequence is computed as a hexdigest, which is used as the LUKS passphrase to encrypt the data volume. You can use the following command to validate the hexdigest:

echo -n "seed1seed2" | sha256sum

Here you can learn how the "seed" can be provided in the workload section of the contract. For more information about how the "seed" input can be provided through the env section, see [The env section](https://cloud.ibm.com/docs/vpc?topic=vpc-about-contract_sehpcr_contract_env). It is mandatory to provide both the seeds for encryption. Encryption fails if only one of the seeds is provided instance shuts down.

You can add a higher level of encryption protection and control to your data at rest by integrating with Hyper Protect Crypto Services.
Unwrapping a data encryption key (DEK) decrypts and authenticates the contents within the key, returning the original key material to your data service.

If your business application needs to access the contents of your wrapped DEKs, you can use the [Hyper Protect Crypto Services key management service API](https://cloud.ibm.com/apidocs/hs-cryptoactiononkey) to send an unwrap request to the service. To unwrap a DEK, you specify the ID value of the root key and the ciphertext value returned during the initial wrap request.

The following diagram shows key unwrapping in action.

Zoom

![Unwrapping data](https://cloud.ibm.com/docs-content/v1/content/217f108cc44e2ce15fb692d4b57b4c628464e908/hs-crypto//images/unwrapping-keys.svg)

Figure 3. Unwrapping data

After you send the unwrap request, the system reverses the key wrapping process by using the same AES algorithms. A successful unwrap operation returns the base64 encoded plaintext value to your IBM Cloud data at rest service.





 What's next 

Hyper Protect Crypto Services supports the integration with other services. With envelope encryption, Hyper Protect Crypto Services provides advanced protection to your data stored in the integrated services.



* For the full list of supported services for integration, see [Integrating services](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-integrate-services).
* For how to create a root key for envelope encryption, see [Creating root keys](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-create-root-keys).
* For how to bring your own root keys for envelope encryption, see [Importing root keys](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-import-root-keys).
baef972e58d4362d97f822d8ff4c5339c5898a0dc184d88d29e5b010b9835ed6 cidata/vendor-data





 attestationPublicKey 

The attestationPublicKey is the public key that you provide which is used to encrypt the attestation document. The attestationPublicKey is part of the user-data file. Encrypting the attestation document is optional.

d388326d90583b2140831e821311aedaee1ad4b4e721b458f8769d3f9267b0dc attestationPublicKey





 Decrypting the attestation document 

If user data contains a public RSA key (attribute: attestationPublicKey), then the attestation document (se-checksums.txt) is encrypted with the given key. The encryption is done by the same process as that of contract encryption. For more information, see [Contract encryption](https://cloud.ibm.com/docs/vpc?topic=vpc-about-contract_sehpcr_contract_encrypt). The public RSA key itself can also be encrypted like the contract.

The encrypted attestation document is then named se-checksums.txt.enc. The decrypt-attestation.sh file can be accessed by mounting the /var/hyperprotect in the docker container. For example,

volumes:
- "/var/hyperprotect/:/var/hyperprotect/:ro"







 Understanding attestation flows 

The following diagram shows two scenarios for attestation from the point of view of the auditor to validate that the deployment is the expected one. The left side of the diagram shows the establishment of trust by the auditor who is rooted on a third-party certificate authority. Any key used is kept in a Hyper Protect Crypto Service and signed in a certificate chain based on the third-party authority.
Managing encryption 

By default, message payload data in Event Streams is encrypted at rest by using a randomly generated key. Although this default encryption model provides at-rest security, you might need a higher level of control. For these use cases, Event Streams supports customer-managed encryption with the following IBM Cloud® Key Management Services:



* IBM® Key Protect for IBM Cloud® (Bring Your Own Key - BYOK) helps you provision encrypted keys for apps across IBM Cloud services. As you manage the lifecycle of your keys, you benefit from knowing that your keys are secured by FIPS 140-2 Level three certified cloud-based hardware security modules (HSMs) that protect against the theft of information. For more information about using Key Protect, see the [Getting Started tutorial](https://cloud.ibm.com/docs/key-protect?topic=key-protect-getting-started-tutorial).
* Hyper Protect Crypto Services (Keep Your Own Key - KYOK) is a single-tenant, dedicated HSM that is controlled by you. The service is built on FIPS 140-2 Level 4-certified hardware, the highest offered by any cloud provider in the industry. For more information about using Hyper Protect Crypto Services, see the [Getting Started tutorial](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-get-started).



These services allow the use of a customer-provided key to control encryption. By disabling or deleting this key, you can prevent any further access to the data that is stored by the service, because it is no longer possible to decrypt it.

Consider to use customer-managed keys, if you require the following features:



* Encryption of data at rest controlled by your own key.
* Explicit control of the lifecycle of data that is stored at rest.



Customer-managed keys are available on the Enterprise plan and only on clusters that were created after October 2019.

Deletion of the customer-managed key is unrecoverable and results in the loss of any data that is stored in your Event Streams instance.



 What is not covered by customer-managed encryption
* UI
* API







 Wrapping keys 

You can manage and protect your encryption keys with a root key by using the IBM® Key Protect API and Console.

When you wrap a

data encryption keywith aroot key(DEK), Key Protect combines the strength of multiple algorithms to protect the privacy and the integrity of your encrypted data.

To learn how key wrapping helps you control the security of at rest data in the cloud, see [Protecting data with envelope encryption](https://cloud.ibm.com/docs/key-protect?topic=key-protect-envelope-encryption).



 Wrapping a key using the console 

If you already have an instance of Key Protect and wish to encrypt your DEK by using a graphical interface, you can use the IBM Cloud console.

After you import or create your [own keys](https://cloud.ibm.com/docs/key-protect?topic=key-protect-create-root-keys), complete the following steps to wrap your data using the key:



1. Log in to the [IBM Cloud console](https://cloud.ibm.com/login/).
2. From the Menu, choose the Resource List to view a list of your resources.
3. From your IBM Cloud resource list, select your provisioned instance of Key Protect.
4. Choose the Root key from the list of your keys that you want to use to wrap your data.
5. Click the ⋯ icon and then click the Envelope encryption option to open the side panel. Make sure the "Wrap key" option is selected.
6. You have a choice of whether to allow Key Protect to wrap your data or whether you want to provide your own base64 encoded plaintext and additional authentication data (AAD), as needed, to wrap the data. Note that if you choose to provide your own plaintext and AAD, you will have to provide it when unwrapping your data. If you do not have experience creating AAD and base 64 encoded plaintext, the recommended practice is to allow Key Protect to wrap your data by leaving the Wrap key for me option selected.
7. Click Wrap Key button.







 Wrapping keys by using the API
About data encryption for VPC 

IBM Cloud® takes security seriously and understands the importance of encrypting data to keep it safe. Block Storage for VPC volumes, snapshots, and File Storage for VPC file shares are automatically encrypted by using IBM-managed encryption. You can also choose to manage your own encryption for volumes, file shares, and custom images by using customer-managed encryption.



 IBM-managed encryption 

By default, VPC volumes and file shares are encrypted at rest with IBM-managed encryption. The service comes with no additional cost.

IBM-managed encryption uses the following industry standard protocols:



* AES-256 encryption.
* Keys are managed in-house with Key Management Interoperability Protocol (KMIP).
* The self-encrypting drives in the Storage architecture are validated for Federal Information Processing Standard (FIPS) Publication 140-2 Level 2.
* Storage architecture is validated for Federal Information Security Management Act (FISMA), and the Health Insurance Portability and Accountability Act (HIPAA)
* Storage architecture is also validated for Payment Card Industry (PCI), Basel II, California Security Breach Information Act (SB 1386), and EU Data Protection Directive 95/46/EC compliance.







 Customer-managed encryption 

For end-to-end encryption in the IBM Cloud, you can use customer-managed encryption. Your data is protected while at rest, and also in transit from the storage to the hypervisor and host. You are responsible for encrypting your data outside the VPC.

With customer-managed encryption, you can bring your own customer root key (CRK) to the cloud or have a [key management service](https://cloud.ibm.com/docs/vpc?topic=vpc-vpc-encryption-aboutkms-for-byok) (KMS) generate a key for you. Root keys are used to encrypt volume, file share, and custom image passphrases with [envelope encryption](https://cloud.ibm.com/docs/vpc?topic=vpc-vpc-encryption-aboutvpc-envelope-encryption-byok), a process that wraps a key with another key.
A similar flow is available for [Block Storage for VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-block-storage-vpc-encryption) to encrypt VPC boot and data volumes.





 Example: Watson services 

Many solutions are based on services with artificial intelligence (AI). At IBM they are offered as [Watson services](https://cloud.ibm.com/developer/watson/services). By default, all data is encrypted. In the Premium plans, you can enhance security by [taking control of the encryption keys (BYOK)](https://cloud.ibm.com/docs/watson?topic=watson-keyservice). After authorizing the service type, e.g., Watson Assistant, to access Key Protect, the following additional option is offered when creating an instance with Premium plan.

Zoom

![Control encryption in Watson services](https://cloud.ibm.com/docs-content/v1/content/f84e2238288b7f0355715a4c355609ce7ac48fb0/solution-tutorials/images/solution51-extended-app-security/Sol51_WatsonBYOK.png)

Control encryption in Watson services







 Organize and control access 

IBM Cloud includes many capabilities for fine-grained access control. Depending on your type of application, project, and account, the following features help you to organize who has access to the application resources and with what set of privileges.



* [Service IDs](https://cloud.ibm.com/docs/account?topic=account-serviceids): Similar to how a user ID identifies a user, a service ID can identify a specific service or application, even a task. The service ID could be considered a "technical user". You can assign privileges to a service ID. Moreover, a service ID has its own IAM API keys to authenticate. Thus, a service ID can be used instead of a regular user ID, thereby simplifying resource management and increasing security.
[Encrypt Data](https://cloud.ibm.com/docs-content/v1/content/57ab3ec034294d8a6de510b97bec3035d4654761/security-broker/images/encrypt.svg)

Encrypt Data
3. In the left navigation menu, navigate to a database and schema and select the table for which you want to assign or change a policy. The table appears on the right.
4. Click the checkbox for a column and then expand the Data Protection drop-down list. By default, CTR Policy is selected.

Zoom

![Data Protection Policies](https://cloud.ibm.com/docs-content/v1/content/57ab3ec034294d8a6de510b97bec3035d4654761/security-broker/images/data_protection.svg)

Data Protection Policies
5. Perform one of the following operations:



* Accept the default policy or select a different default policy.
* Specify a new default policy and add an encryption or masking mode.



Default Policy rules are overwritten by individual mode selections. However, because the Default FPE policy contains only encryption modes, and the Default Masking Policy contains only Masking modes, selecting the opposite mode will not affect the default policy. For example:



* If you select Default FPE and any Mask Mode, both are applied to that column.
* If you select Default FPE and any other FPE mode, (like FPE-cc), the FPE-cc option overwrites the default FPE option for that column.



6. Select a Key ID from the drop-down menu or accept the default.
7. Click REVIEW at the bottom of the left panel to review your selections.
8. Review your policy and select a Migration Plan. The migration plans are listed below: Save Policy: Defines a policy for future migration. Deploy Policy: Establishes a data schema for an environment that was not processed through Data Security Broker migration and the data type conversion but does not migrate the data.
IBM-managed encryption on the storage system is always applied, even when you use customer-managed encryption. This key protects your data while in transit and while at rest.

Encryption of the network link between your workload outside of IBM Cloud and a workload inside IBM Cloud is your responsibility. For more information, see [Encryption in Security and regulation compliance](https://cloud.ibm.com/docs/vpc?topic=vpc-responsibilities-vpcsecurity-compliance).
* A data encryption key (DEK) encrypts data within the QCOW2 file and secures the block data clusters in the virtual disk. The DEK is managed by open source QEMU technology and auto-generated when a QCOW2 file is created.

For Block Storage for VPC volumes that are created from stock images, the DEK is generated by QEMU running on IBM-provided KVM hypervisors. For Block Storage for VPC volumes that are created from custom images, it is generated by QEMU that runs on your on-premises node. The DEK (an AES-256 key) is encrypted with a LUKS passphrase and stored encrypted in the QCOW2 file.
* A LUKS passphrase (also called a "key encryption key") encrypts and decrypts the DEK. This key is managed by the VPC generation 2 infrastructure and is encrypted by your root key. It is stored as metadata that is associated with the Block Storage for VPC volume that contains the QCOW2 file.
* A customer root key that encrypts volume, share, and custom image passphrases with envelope encryption, which creates a wrapped DEK or WDEK. Root keys are customer-managed from KMS instances (Key Protect or Hyper Protect Crypto Services), and stored and managed securely within the KMS instance. The root key also unwraps (decrypts) the WDEK, providing access to your encrypted data.







 Supported key management services
[Transparent Database Encryption by using the standard PKCS #11 API](https://cloud.ibm.com/docs-content/v1/content/217f108cc44e2ce15fb692d4b57b4c628464e908/hs-crypto//images/pkcs-database.svg)

Figure 4. Transparent Database Encryption by using the standard PKCS #11 API
* IBM Db2 default encryption protects key database files and database backup images from inappropriate access while they are stored on external storage media. The database system automatically encrypts and decrypts data when it is used by authorized users and applications. Typically, database users do not need to be aware of default encryption and database client applications do not need to be adapted specifically.

Db2 default encryption uses a two-tiered key hierarchy: Data is encrypted with a data encryption key (DEK). The DEK is encrypted with a master key and is stored in encrypted form with the database or the backup image. A unique DEK is generated by Db2 for each encrypted database and for each encrypted backup. A master key is used to encrypt a DEK. Each encrypted database is associated with one master key at one time.

Zoom

![IBM Db2 default encryption by using the standard PKCS #11 API](https://cloud.ibm.com/docs-content/v1/content/217f108cc44e2ce15fb692d4b57b4c628464e908/hs-crypto//images/pkcs-db2.svg)

Figure 5. IBM Db2 default encryption by using the standard PKCS #11 API



With the PKCS #11 library integration, Hyper Protect Crypto Services supports the industry-standard PKCS #11 API. The Hyper Protect Crypto Services PKCS #11 library connects your database to Hyper Protect Crypto Services to perform cryptographic operations.
* All the data that is hosted in an auditing instance is encrypted at rest using AES 256.
* When an IBM Cloud service sends data to an auditing instance, data is encrypted in transit over HTTPS.
* When a user requests an export, the data is encrypted during transit, and is also encrypted at rest in IBM Cloud Object Storage (COS).



The service plan that you choose for an IBM Cloud Activity Tracker instance defines the number of days that data is stored and retained. For example, if you choose the Lite plan, data is not stored at all. However, if you choose the 7-day plan, data is stored for 7 days and you have access to it through the UI.

You can archive events from an IBM Cloud Activity Tracker instance into a bucket in an IBM Cloud Object Storage (COS) instance. [Learn more](https://cloud.ibm.com/docs/services/activity-tracker?topic=activity-tracker-archiving). When data is archived, data going to IBM Cloud Object Storage (COS) is encrypted in transit over HTTPS. You must configure archiving to a COS instance if you want to backup your events for long term storage. You can use the Data Engine service to query data in archive files that are stored in an Object Storage (COS) bucket in your account. You can run queries from the IBM Cloud UI, or programmatically.

You can export data in JSONL format locally, write data to your terminal, or request an email with a link to the data. [Learn more](https://cloud.ibm.com/docs/services/activity-tracker?topic=activity-tracker-export). Consider the following information when you export log data:



* You export a set of event entries. To define the set of data that you want to export, you can apply filter and searches. You can also specify the time range.
* From the Web UI, when you export events, you get an email that is sent to your email address, with a link to a compressed file that includes the data. To get the data, you must click the link and download the compressed file.