vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Encrypting a bucket with Key Protect 

While all data stored in Cloud Object Storage is automatically encrypted using randomly generated keys, some workloads require that the keys can be rotated, deleted, or otherwise controlled by a key management system (KMS) like Key Protect.



 Before you begin 

Before you plan on using Key Protect with Cloud Object Storage buckets, you need:



* An [IBM Cloud™ Platform account](http://cloud.ibm.com/)
* An [instance of IBM Cloud Object Storage](http://cloud.ibm.com/catalog/services/cloud-object-storage)



You will also need to ensure that a service instance is created by using the [IBM Cloud catalog](https://cloud.ibm.com/catalog) and appropriate permissions are granted. This tutorial does not outline the step-by-step instructions to help you get started. This information is found in section [Server-Side Encryption with IBM Key Protect (SSE-KP)](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-kp)





 Step 1: Create a bucket 

When your key exists in Key Protect and you authorize a service for use with IBM COS, associate the key with a new bucket:



1. Navigate to your instance of Object Storage.
2. Click Create bucket. ![Navigate to COS](https://cloud.ibm.com/docs-content/v1/content/63322f2f3a72050206eb05ba7f590fdcb741105c/cloud-object-storage/images/kp-tut-1.png)
3. Select Custom bucket. ![Navigate to COS](https://cloud.ibm.com/docs-content/v1/content/63322f2f3a72050206eb05ba7f590fdcb741105c/cloud-object-storage/images/kp-tut-2.png)
4.
From the list of key types, select [Standard key](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-understand-conceptsstandard-key-concept). 
 Key name A unique, human-readable alias for easy identification of your key. To protect your privacy, ensure that the key name does not contain personally identifiable information (PII), such as your name or location. 
 Key alias (Optional) One or more unique, human-readable aliases that you want to assign to your key for easy recognition. Alias size can be 2 - 90 characters. You can set up to five key aliases for the key, with each separated by a comma.<br><br>Note: Each alias must be alphanumeric, case-sensitive, and cannot contain spaces or special characters other than dashes (-) or underscores (_). The alias cannot be a version 4 UUID and must not be a Hyper Protect Crypto Services reserved name: allowed_ip, key, keys, metadata, policy, policies, registration, registrations, ring, rings, rotate, wrap, unwrap, rewrap, version, versions. 
 Key ring ID Select a key ring from the list that contains the existing key rings. If you don't assign a key ring, the key will be added to the default key ring. For more information about key rings, see [Managing key rings](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-managing-key-rings). 
 Expiration date (Optional) Set the date and time when the key gets expired. After the expiration date, the key moves into the Deactivated state. For more information about key state, see [Monitoring the lifecycle of encryption keys](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-key-states). 
 Description (Optional) Add an extended description for your key. It needs to be 2 to 240 characters in length. 



6. When you finish filling out the key's details, click Create key to confirm.







 Creating standard keys with the API
Optional: Add labels to help you to search for similar secrets in your instance.
8. Optional: Add metadata to your secret or to a specific version of your secret.



1. Upload a file or enter the metadata and the version metadata in JSON format.



9. Click Next.
10. Select a certificate authority configuration.

The configuration that you select determines the certificate authority to use for signing and issuing the certificate. To view the configurations that are defined for your instance, you can go to Secrets engines > Public certificates.
11. Select the key algorithm to be used to generate the public key for your certificate.

The key algorithm that you select determines the encryption algorithm (RSA or ECDSA) and key size to use to generate keys and sign certificates. For longer living certificates, it is recommended to use longer key lengths to provide more encryption protection. Options include RSA2048, RSA4096, ECDSA256, and ECDSA384.
12. Optional: Enable advanced options for the certificate.



1. To bundle your issued certificate with intermediate certificates, switch the bundle toggle to On. After your certificates are bundled, they can no longer be unbundled.
2. To enable automatic rotation for the certificate, switch the rotation toggle to On. Your certificate is rotated 31 days before it expires.
3. To request a new private key with the certificate on each rotation, switch the rekey toggle to On.



13. Select Manual as your DNS provider.
14. Add the domains to include in your request.

You can include up to 100 domains, subdomains, or wildcards. The Common Name, or fully qualified domain name of the certificate, can't exceed 64 characters in length. A wildcard can be selected as the Common Name.



1. In the Common name section, from your list of domains, select the Common Name of the certificate.



15. Click Next.
16. Review the details of your certificate.
17. Click Add.
18. Check the issuance details of your certificate by clicking the Actions icon !
You must also have been added to the allowlist, which can be achieved through your contact with the service representative.

Because the HSM information is not verified until after Key Protect has been provisioned (an operation that can take more than an hour), it is important to ensure these fields are filled out correctly when first attempting to create a Key Protect on Satellite location.



* HSM IP address: The IP address of your HSM. This is needed in order to connect to the worker nodes that you assigned to Key Protect.
* HSM server certificate: The NTLS communications used by the Thales HSM requires certificate exchanges between the HSM and Key Protect. You must create a TLS certificate in your HSM and provide the certificate Key Protect will use to verify communications from the HSM.
* Partition label: The name of the partition that you created for Key Protect to use.
* Partition crypto officer password: The credential Key Protect needs to login to the relevant partition on the HSM to perform key operations.
* Master key label: Key Protect uses a Master Key Encryption Key in your partition. A label or name is assigned to this key and is used by Key Protect in PKCS#11 API to refer to the master key.
* Signing key label: A label or name for a key in a partition, which is used for data authentication such as data signing and verification.
* Import key label: Key Protect supports importing Bring Your Own Key (BYOK) by a DES3 encryption Key. A label or name is assigned to this key.
* Secure import key label prefix (TKEK): Key Protect supports a secure mechanism to Bring Your Own Key (BYOK) into an HSM. The prefix used by the HSM for these keys must be known to Key Protect.
* Activity Tracker ingestions key: To receive audit logs, you must create an IBM Cloud Activity Tracker instance and an ingestion key.



There are two additional credentials you must share before your service can be deployed, the HSM client cert and the HSM client key, but these are not shared as part of deploying the UI itself. These credentials enable NTLS communications used by the Thales HSM for exchanges between the HSM and Key Protect running on your worker node.
Refer to the [Configure User groups](https://cloud.ibm.com/docs/security-broker?topic=security-broker-sb_user_group) section to configure or add user groups.
4. Refer to the [Apply RBAC policy](https://cloud.ibm.com/docs/security-broker?topic=security-broker-sb_apply_rbac_policy) section to a RBAC policy to user groups.
5. Perform a simple masking by following the instructions in the [Apply simple masking](https://cloud.ibm.com/docs/security-broker?topic=security-broker-sb_apply_mask) section to a RBAC policy to user groups.
* [Re-encrypt package before you forward it to upstream apps](https://cloud.ibm.com/docs/containers?topic=containers-comm-ingress-annotationsssl-services-support): The ALB decrypts HTTPS requests before forwarding traffic to your apps. If you have apps that require HTTPS and need traffic to be encrypted before it is forwarded to those upstream apps, you can use the ssl-services annotation. If your upstream apps can handle TLS, you can optionally provide a certificate that is contained in a one-way or mutual-authentication TLS secret.









 Persistent storage 

Review supported options for encrypting and protecting your data on persistent storage in IBM Cloud.

By default, all IBM Cloud storage solutions automatically encrypt your data at rest with an IBM-managed encryption key at no additional cost. For more information, see the following links.



* [Classic NFS File Storage for Classic](https://cloud.ibm.com/docs/FileStorage?topic=FileStorage-mng-dataencryptvolume)
* [Classic Block Storage](https://cloud.ibm.com/docs/BlockStorage?topic=BlockStorage-mng-data)
* [IBM Cloud Object Storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-encryption)



Depending on the type of storage that you choose, you can set up additional encryption with IBM Key Protect to protect your data in transit and at rest with your own encryption key.



* [IBM Cloud Object Storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-encryption)
* [Portworx](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_encryption)
To allow private traffic requests from outside the cluster to your apps, you can create private Kubernetes networking services, such as private NodePorts, NLBs, and Ingress ALBs. You can then use Calico pre-DNAT policies to block traffic to public NodePorts of private networking services. For more information, see [Planning private external load balancing](https://cloud.ibm.com/docs/openshift?topic=openshift-cs_network_planningprivate_access).





 Public traffic to cluster apps 

To make your apps externally accessible from the public internet, you can create public NodePorts, network load balancers (NLBs), and Ingress application load balancers (ALBs). Public networking services connect to this public network interface by providing your app with a public IP address and, depending on the service, a public URL. When an app is publicly exposed, anyone that has the public service IP address or the URL that you set up for your app can send a request to your app. You can then use Calico pre-DNAT policies to control traffic to public networking services, such as allowing traffic from only certain source IP addresses or CIDRs and blocking all other traffic. For more information, see [Planning public external load balancing](https://cloud.ibm.com/docs/openshift?topic=openshift-cs_network_planningopenshift_routers).

For additional security, you can isolate networking workloads to edge worker nodes.
: Edge worker nodes can improve the security of your cluster by allowing fewer worker nodes that are connected to public VLANs to be accessed externally and by isolating the networking workload. When you [label worker nodes as edge nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-edgeedge_nodes), NLB and ALB pods are deployed to only those specified worker nodes. Router pods remain deployed to the non-edge worker nodes. Additionally, to prevent other workloads from running on edge nodes, you can [taint the edge nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-edgeedge_workloads).
Key management by Key Protect 

This topic describes managing column encryption keys by using IBM® Key Protect for IBM Cloud® (Key Protect). It explains how to create a Key Protect instance and to provide master keys and how to write and read encrypted data using these master keys.



 Creating a Key Protect instance and master keys 

To create a Key protect instance and master keys:



1. Create a Key Protect service instance. See [Provisioning the service](https://cloud.ibm.com/docs/key-protect?topic=key-protect-provision).
2. Create customer root keys in this instance. The customer root keys serve as master keys for column encryption. See [Creating root keys](https://cloud.ibm.com/docs/key-protect?topic=key-protect-create-root-keys).

If you want to use existing root keys, you can import those. See [Importing root keys](https://cloud.ibm.com/docs/key-protect?topic=key-protect-import-root-keys).
3. Configure user access rights to the master keys by using the IBM IAM service. See [Granting access to master keys](https://cloud.ibm.com/docs/key-protect?topic=key-protect-grant-access-keysgrant-access-key-level).







 Writing encrypted data 

To write encrypted data:



1. Pass the following parameters to IBM Analytics Engine Serverless:



* "parquet.crypto.factory.class": the class implementing EncryptionPropertiesFactory. Set to "com.ibm.parquet.key.management.IBMKeyToolsFactory".

sc.hadoopConfiguration.set("parquet.crypto.factory.class","com.ibm.parquet.key.management.IBMKeyToolsFactory")
* "parquet.encryption.kms.instance.id": The ID of your KeyProtect instance, for example:
Setting and editing network access policies 

After you set up your IBM® Key Protect service instance, you manage network access policies by using the service API.



 Managing network access policy settings 

For the best results now available, use [context-based restrictions (CBR)](https://cloud.ibm.com/docs/account?topic=account-context-restrictions-whatis) instead of an [allowed IP policy](https://cloud.ibm.com/docs/key-protect?topic=key-protect-manage-allowed-ip) to define and enforce access policies for IBM Cloud resources based on the network location of access requests.

A network access policy for Key Protect instances is an extra policy that customers can use to block a Key Protect instance from getting API requests from public or private networks.

The network access policy applies to newly provisioned and existing instances. For existing instances the network access policy is enforced after it is set.

The network access policy capability is available only through the Key Protect API. To find out more about accessing the Key Protect APIs, check out [Setting up the API](https://cloud.ibm.com/docs/key-protect?topic=key-protect-set-up-api).



 Understanding network access policies 

Two options control network access to Key Protect instances:



* Public and private network access - this is the default
* Private network access only





 Public and private network access 

The Key Protect instance accepts API requests from both public and private endpoints.

Public and private network access is the default setting and is used if a policy is not set.

For example, multiple teams are testing a solution that uses Key Protect instances. Development and test teams issue API requests from both outside (public endpoints) and inside (private endpoints) the IBM Cloud. You allow public and private API requests to ensure each team has access to Key Protect instances during this phase of the project.





 Private network access only 

The Key Protect instance accepts API requests from only private endpoints.

To connect to Key Protect by using a private network connection, you must use the Key Protect API or the Key Protect CLI plug-in. This capability is not available from the Key Protect GUI.
If you're using Hyper Protect Crypto Services, see the [Hyper Protect Crypto Services information](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-get-started) for corresponding instructions.



1. Provision the [Key Protect](https://cloud.ibm.com/docs/key-protect?topic=key-protect-provision) service.

Provisioning a new Key Protect service instance ensures that it includes the most recent updates that are required for customer-managed encryption.
2. [Create](https://cloud.ibm.com/docs/key-protect?topic=key-protect-create-root-keys) or [import](https://cloud.ibm.com/docs/key-protect?topic=key-protect-import-root-keys) a customer root key (CRK) in IBM Key Protect.

Plan ahead for importing keys by [reviewing your options for creating and encrypting key material](https://cloud.ibm.com/docs/key-protect?topic=key-protect-importing-keysplan-ahead). For added security, you can enable the secure import of the key material by using an [import token](https://cloud.ibm.com/docs/key-protect?topic=key-protect-importing-keysusing-import-tokens) to encrypt your key material before you bring it to the cloud.
3. Use your customer root key (CRK) to wrap, or protect, the unique passphrase that you used to encrypt your image with LUKS encryption. In the image encryption example, we used the passphrase abc123.



1. Make sure that the Key Protect CLI plug-in is installed so that you can generate a WDEK. For more information, see [Installing the Key Protect CLI plug-in](https://cloud.ibm.com/docs/key-protect?topic=key-protect-set-up-cliinstall-cli).
2. Encode your passphrase with Base64 encoding to prepare it for wrapping with your CRK.
Also, make sure to mark the parameters that contain sensitive information like passwords, API, and ssh private keys as "sensitive". These parameters are marked as “sensitive” in the readme file, under “Input parameter file”.
7. On the workspace Settings page, click Generate plan. Wait for the plan to complete.
8. Click View log to review the log files of your Terraform execution plan.
9. Apply your Terraform template by clicking Apply plan.
10. Review the log file to ensure that no errors occurred during the provisioning, modification, or deletion process.







 Related information 

For more information about Terraform on IBM Cloud, see [Getting started with Terraform on IBM Cloud](https://cloud.ibm.com/docs/ibm-cloud-provider-for-terraform?topic=ibm-cloud-provider-for-terraform-getting-started) tutorial.

For more information about using Terraform for creating only a VPC for SAP, without the SAP architecture, see [Creating single-tier virtual private cloud for SAP by using Terraform](https://cloud.ibm.com/docs/sap?topic=sap-create-terraform-single-tier-vpc-sap).

SAP One Support Notes that apply to this document:



* [SAP Note 84555 - Windows Server, Linux®, and UNIX: Certified hardware](https://launchpad.support.sap.com//notes/84855)
* [SAP Note 2927211 - SAP Applications on IBM Cloud Virtual Private Cloud (VPC) Infrastructure environment](https://launchpad.support.sap.com//notes/2927211)
* [SAP Note 2923773 - Linux® on IBM Cloud (IaaS): Adaption of your SAP License](https://launchpad.support.sap.com//notes/2923773)
* [SAP Note 2414097 - SAP Applications on IBM Cloud Classic Infrastructure environment](https://launchpad.support.sap.com//notes/2414097)
1. In the Security and Compliance Center navigation, click Settings.
2. On the Storage tile, click Connect.
3. Ensure that the service-to-service policy between Cloud Object Storage and Security and Compliance Center is configured. If a policy is already in place, this screen is not shown and you can skip to the next step.
4. Select an instance of Cloud Object Storage.
5. From the table, select the bucket that you want to use.
6. Click Connect.







 Step 4: Scan your resources 

An attachment is how you target a specific grouping of your resources to evaluate against a specific profile. To start scanning your resources, create an attachment.



1. In the Security and Compliance Center navigation, click Dashboard Then, click Get started.

Alternatively, you can create an attachment through the Profiles page. On the Attachments tab of the profile details page, click Create and then continue with step 2. Or, you can navigate to the Attachments page where a flat list of all of your attachments is shown.
2. Provide a name and description for your attachment. Be sure to be as descriptive as possible so that it's easy for other members of your team to understand what is being evaluated. Then, click Next.
3. Select the Profile and Profile version that you want to use for your evaluation.
4. Customize the underlying evaluations in your scan by editing the default parameters to match your specific use case.
5. Define a Scope to identify the resources that you want to evaluate and those that you want to Exclude. Then, click Next.
6. Select the frequency at which you want to evaluate your attachment.

Options include every day, every 7 days, and every 30 days. Additionally, you can now pause your scans if you need to.
7. Optional: Configure notifications.



1. If you want to receive notifications, toggle Notify me to On.
2. By default, when notifications are enabled, you are alerted when 15% or more of your controls fail in a single scan. You can change this by adjusting the Threshold percentage.
Use of the allowlist function helps prevent unauthorized scanning activities. For more information, see [Creating an allowlist for the private cloud service endpoint](https://cloud.ibm.com/docs/containers?topic=containers-access_clusterprivate-se-allowlist).





 Worker communication to other services or networks 

Allow your worker nodes to securely communicate with other IBM Cloud services, on-premises networks, other VPCs, and IBM Cloud classic infrastructure resources.





 Communication with other IBM Cloud services over the private or public network 

Your worker nodes can automatically and securely communicate with other [IBM Cloud services that support private cloud service endpoints](https://cloud.ibm.com/docs/account?topic=account-vrf-service-endpoint), such as IBM Cloud® Container Registry, over the private network. If an IBM Cloud service does not support private cloud service endpoints, worker nodes can securely communicate with the services over the public network through the subnet's public gateway.

Note that if you use [access control lists (ACLs)](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-acls) for your VPC subnets, you must create inbound or outbound rules to allow your worker nodes to communicate with these services.





 Communication with resources in on-premises data centers 

To connect your cluster with your on-premises data center, you can use the IBM Cloud® Virtual Private Cloud VPN or IBM Cloud® Direct Link.



* To get started with the Virtual Private Cloud VPN, see [Configure an on-prem VPN gateway](https://cloud.ibm.com/docs/vpc?topic=vpc-vpn-onprem-example) and [Create a VPN gateway in your VPC, and create the connection between the VPC VPN gateway and your local VPN gateway](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-a-vpc-using-the-ibm-cloud-consolevpn-ui). If you have a multizone cluster, you must create a VPC gateway on a subnet in each zone where you have worker nodes.