vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Instead, their work's isolated in pockets across the globe due to regional compliance regulations and centralized databases.

Red Hat OpenShift on IBM Cloud delivers secure compute that can host sensitive and data processing on an open platform. That global platform is hosted in near-by regions. So it's tied to local regulations that inspire patients’ and researchers’ confidence that their data is both protected locally and makes a difference in better health outcomes.



 Context 

Securely hosting and sharing disease data for Research Nonprofit



* Disparate groups of researchers from various institutions don’t have a unified way to share data, slowing down collaboration.
* The security concern adds to the collaboration burden that causes even less shared research.
* Developers and Researchers are spread across the globe and across organizational boundaries, which make PaaS and SaaS the best option for each user group.
* Regional differences in health regulations require some data and data processing to remain within that region.







 Solution 

Securely hosting and sharing disease data for Research Nonprofit.

The research nonprofit wants to aggregate cancer research data across the globe. So they create a division that is dedicated to solutions for their researchers.



* INGEST - Apps to ingest research data. Researchers today use spreadsheets, documents, commercial products, and proprietary or home-grown databases to record research results. This situation is unlikely to change with the nonprofit's attempt to centralize data analysis.
* ANONYMIZE - Apps to anonymize the data. SPI must be removed to comply with regional health regulations.
* ANALYZE - Apps to analyze the data. The basic pattern is to store the data in a regular format and then to query and process it by using AI and machine learning (ML) technology, simple regressions, and so forth.



Researchers need to affiliate with a regional cluster, and apps ingest, transform, and anonymize the data.



1. Syncing the anonymized data across regional clusters or shipping them to a centralized data store
2. Processing the data, by using ML like PyTorch on bare metal worker nodes that provide GPUs



INGEST
: IBM Cloudant is used at each regional cluster that stores researchers’ rich data documents and can be queried and processed as needed.
And this is why it's so important to establish a centralized place to manage all these things as we build out more applications and microservices. As we have more of these, the problem becomes more complex. If it falls into the wrong hands, how is protected? How do we block it from getting to that point? How was that data isolated?

When we look at the damage that this can cause, we're looking in the millions of dollars, for example, for a data breach. In terms of developer operations, if you're not properly managing these, forget even the case in which a bad actor hasn't been involved, but it can be confusing for teams to use this. So what we need to do is make sure that we have it, again, centralized improperly stored so that we can leverage these secrets in the correct way and they can properly communicate with services.

OK, now, let's look at the next layer of the onion in a more complicated example.

Now let's go back to Jane, the enterprise developer. Jane – our "E dev" here – needs again to have access to that development repo that she's referring to earlier. Let's go ahead and call this, maybe it's GitHub. So what she needs to be able to do is have write access and so she's going to need to request the information that's going to give her access to that role. So that's where a secrets manager service comes in in a perfect, complementary fashion.

A secrets manager service can securely store these credentials, along with other types of secrets, in a centralized way for her to be able to access or maybe other services to access, like we'll see in a second. But when it's done, it's given her the peace of mind that her user credentials are securely stored and now she can worry about authenticating with the service and getting her job done because secrets manager is going to take care of that for her.

However, what's really important for secrets manager service to do is interact with the cloud service provider's IAM, or identity and access management service.
Information security 

IBM is committed to providing our clients and partners with innovative data privacy, security and governance solutions.

Notice: Clients are responsible for ensuring their own compliance with various laws and regulations, including the European Union General Data Protection Regulation. Clients are solely responsible for obtaining advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulations that may affect the clients' business and any actions the clients may need to take to comply with such laws and regulations.

The products, services, and other capabilities described herein are not suitable for all client situations and might have restricted availability. IBM does not provide legal, accounting, or auditing advice or represent or warrant that its services or products ensure that clients are in compliance with any law or regulation.

If you need to request GDPR support for IBM Cloud® Watson resources that are created, see [GDPR Subject Access Request](https://cloud.ibm.com/docs/watson?topic=watson-gdpr-sar).



 European Union General Data Protection Regulation (GDPR) 

IBM is committed to providing our clients and partners with innovative data privacy, security and governance solutions to assist them on their journey to GDPR compliance.

Learn more about IBM's own GDPR readiness journey and our GDPR capabilities and offerings to support your compliance journey [here](https://www.ibm.com/data-responsibility/gdpr/).





 Labeling and deleting data in Discovery 

Discovery includes an API to label data per call.

With this API you can:



* Label your data with a customer ID.
* Delete all data for a specific customer ID, including related notices.



Data is labeled by adding a customer_id of your choice (see restrictions in [How to label data](https://cloud.ibm.com/docs/discovery?topic=discovery-information-securitylabeling)) to the optional X-Watson-Metadata header. Discovery can then delete it by customer_id.

On any REST call, an optional header X-Watson-Metadata can be sent with semicolon separated field=value pairs, where currently only customer_id is persisted. By adding that customer_id in X-Watson-Metadata header, the request indicates that it contains data that belongs to this customer_id.
Make sure that your information can reside in a different country before you create your cluster in a non-multizone metro area.

The data that you create and own is always stored in the same location as the cluster. For more information about what data you can create in your cluster, how this data is encrypted, and how you can protect this data, see [Protecting sensitive information in your cluster](https://cloud.ibm.com/docs/containers?topic=containers-encryption).





 How can I remove my information? 

Review your options to remove your information from Red Hat OpenShift on IBM Cloud.

Removing personal and sensitive information is permanent and not reversible. Make sure that you want to permanently remove your information before you proceed.

Is my data removed when I remove the cluster?
: Deleting a cluster does not remove all information from Red Hat OpenShift on IBM Cloud. When you delete a cluster, cluster-specific information is removed from the etcd instance that is managed by IBM. However, your information still exists in the IBM Cloud Object Storage backup and can still be accessed by the IBM service team by using the account-specific customer root key in IBM Key Protect that IBM owns and manages.

What options do I have to permanently remove my data?
: To remove that data that IBM stores, choose between the following options. Note that removing your personal and sensitive information requires all your clusters to be deleted as well. Make sure that you backed up your app data before your proceed.



* Open an IBM Cloud support case: Contact IBM Support to remove your personal and sensitive information from Red Hat OpenShift on IBM Cloud. For more information, see [Getting support](https://cloud.ibm.com/docs/get-support?topic=get-support-using-avatar).
* End your IBM Cloud subscription: After you end your IBM Cloud subscription, Red Hat OpenShift on IBM Cloud removes the customer root key in IBM Key Protect that IBM created and managed for you as well as all the personal and sensitive information from the etcd data store and Object Storage backup.







 Does Red Hat collect information about my cluster? 

Yes.
Development and test teams issue API requests from both outside (public endpoints) and inside (private endpoints) the IBM Cloud. You allow public and private API requests to ensure each team has access to Key Protect instances during this phase of the project.





 Private network access only 

The Key Protect instance accepts API requests from only private endpoints.

To connect to Key Protect by using a private network connection, you must use the Key Protect API or the Key Protect CLI plug-in. This capability is not available from the Key Protect GUI.

For example, development and testing is complete and the solution that uses Key Protect instances is in production. You want to limit API requests to private networks for security reasons. All Key Protect API requests must originate from within the IBM Cloud.

In the Regions and endpoints section there is a section that explains how to [enable private endpoints](https://cloud.ibm.com/docs/key-protect?topic=key-protect-regionsconnectivity-options).

After the network access policy is set to private-only you cannot make any Key Protect API calls from the public network, including the API to change the policy. Make sure the private environment is set up before setting the network access policy to private-only. See [using private endpoints](https://cloud.ibm.com/docs/key-protect?topic=key-protect-private-endpoints).





 Delete or deprovision a Key Protect instance 

The network access policy is not enforced when a Key Protect instance is deleted or deprovisioned.

The instance access policy, which controls access to the instance from either public or private IP addresses, is not enforced when the following command to delete the instance is issued.

$ ibmcloud resource service-instance-delete (NAME | ID)







 Enabling network access to your Key Protect instance with the console 

If you prefer to enable a dual authorization policy on your instance by using a graphical interface, you can use the IBM Cloud console.

After the network access policy is set to private-only the UI cannot be used for any Key Protect actions.

Keys in a private-only instance will not be shown in the UI and any Key Protect actions in the UI will return an unauthorized error (HTTP status code 401).
Protecting sensitive information in your Watson service 

IBM Cloud

You can add a higher level of encryption protection and control to your data at rest (when it is stored) and data in motion (when it is transported) by enabling integration with IBM® Key Protect for IBM Cloud®.

The data that you store in IBM Cloud is encrypted at rest by using a randomly generated key. If you need to control the encryption keys, you can integrate Key Protect. This process is commonly referred to as Bring your own keys (BYOK). With Key Protect you can create, import, and manage encryption keys. You can assign access policies to the keys, assign users or service IDs to the keys, or give the key access only to a specific Watson service. The first 20 keys are free.

Data encryption of Watson services requires a new Premium plan instance. You cannot encrypt an existing service instance or instances not in the Premium plan. Not all Watson services support Premium plans. For more information about Premium plans, [contact a Watson representative](https://ibm.biz/contact-wdc-premium)



 About customer-managed encryption 

Watson uses

envelope encryptionto implement customer-managed keys. Envelope encryption describes encrypting one encryption key with another encryption key. The key used to encrypt the actual data is known as adata encryption key (DEK). The DEK itself is never stored but is wrapped by a second key that is known as the key encryption key (KEK) to create a wrapped DEK. To decrypt data, the wrapped DEK is unwrapped to get the DEK. This process is possible only by accessing the KEK, which in this case is your root key that is stored in Key Protect.

Key Protect keys are secured by FIPS 140-2 Level 3 certified cloud-based

hardware security modules (HSMs).





 Enabling customer-managed keys with Watson 

Some Watson services have additional details about how to work with Key Protect. For more information, see the [docs](https://cloud.ibm.com/docs?tab=all-docs&category=ai) for the service.
You will need to [retrieve](https://cloud.ibm.com/docs/key-protect?topic=key-protect-manage-allowed-ipretrieve-allowed-ip-port) the port and [specify](https://cloud.ibm.com/docs/key-protect?topic=key-protect-manage-allowed-ipsend-private-allowed-ip-traffic) the port number during each request via a private endpoint.
* There is limited allowed IP policy support for Key Protect instances that are integrated with other IBM Cloud services. Service to service calls will bypass allowed IP policy enforcement. To find out more information about how allowed IP policies affect Key Protect instances that are integrated with other IBM Cloud services, see [Using an allowed IP policy on an instance that is integrated with other IBM Cloud services](https://cloud.ibm.com/docs/key-protect?topic=key-protect-manage-allowed-ipallowed-ip-s2s).
* When you enable an allowed IP policy for your Key Protect instance, the resources in your instance will not be displayed in the UI. After enabling an allowed IP policy, you will only be able to view and access the keys and associated resources in your Key Protect instance through the Key Protect API. Before making a request, make sure that you are assigned the correct access policy for your Key Protect instance and use the correct endpoints and port (for private endpoints).
* If both a network policy and an allowed IP policy are enabled at the same time, only traffic aligning with both policies will be allowed. When assigning an allowed IP policy to a Key Protect instance that has an existing allowed network policy, the IP addresses that are listed on the policy must access the instance from the network specified in the allowed network policy.
* ipWhitelist has been deprecated and replaced with allowedIP. Due to the deprecation of the previous ipWhitelist term, if you don't specify a query param while making a GET instance policies request, you will receive a duplicate allowed IP record when you retrieving your list of instance policies.





 Enabling and updating an allowed IP policy for your Key Protect instance
When you have data privacy requirements, [Private Data collections](https://hyperledger-fabric.readthedocs.io/en/release-2.2/private-data/private-data.htmlwhat-is-private-data) provide a way to further isolate specific data from the rest of the channel members. The combination of the use of channels and private data offer various solutions for achieving [Data Residency](https://cloud.ibm.com/docs/blockchain?topic=blockchain-console-icp-about-data-residency).





 GDPR 

In order to be GDPR compliant, it is recommended that you store PII data off chain.







 Hyperledger Fabric Security 

Because IBM Blockchain Platform is based on Hyperledger Fabric, you can leverage the secure features included in a Fabric network.



* TLS v1.2 communications TLS is embedded in the trust model of Hyperledger Fabric. By default, server-side TLS is enabled for all communications using TLS certificates. TLS is used to encrypt the communication between your nodes and between your nodes and your applications. TLS prevents man-in-the-middle and session hijacking attacks. All IBM Blockchain Platform components use TLS to communicate with each other.
* Transaction integrity: Fabric uses the cryptographic ECDSA standard to guarantee transaction integrity. With ECDSA, the transaction originator, such as a client application, signs their message by using their private key, and the recipient, such as a peer, uses the originator’s public key to verify the authenticity of the message. If a transaction is tampered with on its way to the recipient, the signature verification fails.
* Channels: While Fabric channels are a powerful mechanism for partitioning and isolating data, they also provide the primary foundation for data privacy. Only members of the same channel can access the data of this channel. To ensure channel security, the channel configuration update policy is configured to define the number of channel operators who need to agree on the channel update request before a channel can be updated. Therefore, a clear process must exist for defining the organizations that are allowed to join and update channel.
No user data is written to disk.

You can choose to opt out of logging at either the account level or the API request level. For more information, see [Controlling request logging for Watson services](https://cloud.ibm.com/docs/watson?topic=watson-gs-logging-overview).





 Data separation and encryption 

IBM Cloud

The service's Standard and Premium pricing plans offer different levels of data separation and encryption for users:



* Standard plans are multi-tenant solutions that provide logical separation of data by using common encryption keys.
* Premium plans are single-tenant solutions that provide physical separation of data. Premium plans provide dedicated data storage accounts that use unique encryption keys.



Users of Premium plans can also integrate with IBM® Key Protect for IBM Cloud® to create, import, and manage their encryption keys. This process is commonly referred to as Bring your own keys (BYOK). For more information about using Key Protect, see [Protecting sensitive information in your Watson service](https://cloud.ibm.com/docs/text-to-speech?topic=watson-keyservice).





 Network endpoints 

IBM Cloud

IBM Cloud supports both public and private network endpoints with certain plans. Connections to private network endpoints do not require public internet access. Private network endpoints support routing services over the IBM Cloud private network instead of the public network. A private network endpoint provides a unique IP address that is accessible to you without a VPN connection.

Private network endpoints are supported only for paid plans. Check the plan information for your service to learn about the plans that support private network endpoints. For more information, see [Public and private network endpoints](https://cloud.ibm.com/docs/text-to-speech?topic=watson-public-private-endpoints).





 Virtual private endpoints 

IBM Cloud

IBM Cloud® Virtual Private Endpoints for Virtual Private Cloud (VPC) are available with certain plans. Virtual private endpoints enable you to connect to supported IBM Cloud services from your VPC network by using the IP addresses of your choosing, allocated from a subnet within your VPC. Virtual private endpoints are an evolution of the private connectivity to IBM Cloud services.
While it is likely that users with rights at the account level will also have rights over the various instances (and perhaps, though not always, the other way around), note that the names given to account roles are different than those for the roles within the service instances, reflecting this difference between account roles and service instance roles. For more information on those roles, their names, and their permissions, check out [IAM roles and actions](https://cloud.ibm.com/docs/account?topic=account-iam-service-roles-actions).



 How IAM access works 

After you set up and organize resource groups in your account, you can take advantage of a couple of strategies to streamline the access management process:

Access groups
: You can minimally manage the number of assigned policies by giving the same access to all identities in an access group instead of assigning the same access multiple times per individual user, service ID, or trusted profile. Users must be invited to your account before you can add them to an access group. If a user qualifies for a trusted profile that is a member of the access group, you don't need to invite them to your account.

Trusted profiles
: If your organization has an enterprise directory, trusted profiles can reduce the time and effort to manage access. It simplifies the login process to your IBM Cloud account for federated users in your enterprise. You can automatically grant federated users or compute resources access to your account by creating trusted profiles. For federated users, add conditions based on SAML attributes to define which federated users can apply a profile. For compute resources, specify specific resources, or add conditions based on resource attributes to define which compute resources can apply a profile. For both entity types, the level of access that is granted is determined by the access policies that are specified within each trusted profile, or the access groups that the trusted profile is a member of. However, trusted profiles don't require federated users to be invited to an account, and only users that are federated by an external identity provider (IdP) can apply a trusted profile.

When you're a member of multiple access groups, all policies apply at once when you access an account.
IBM Maximo Software Development Life Cycle (SDLC): [https://www.ibm.com/support/pages/ibm-maximo-software-development-life-cycle](https://www.ibm.com/support/pages/ibm-maximo-software-development-life-cycle)





 Data Security & Privacy (DS&P) 

Manage application logging when configured with certain verbose options can allow for extensive information being gathered. This logging could include Personally Identifiable Information (PII) or Sensitive Personal Information (SPI). This information is generated and stored in plain text files on the application server. These logs are often made available to the customer upon request via SFTP. Application administration, including the logging configuration, are the responsibility of the customer, and it is highly recommended that logging PII/SPI not be configured unless absolutely required. The following document describes how to configure logs to exclude any data classified as PII or SPI:

[https://www.ibm.com/support/pages/node/2801463](https://www.ibm.com/support/pages/node/2801463)

IBM Data Security and Privacy Principles for IBM Cloud services can be found at the link below: [https://www.ibm.com/support/customer/csol/terms/?cat=data-security](https://www.ibm.com/support/customer/csol/terms/?cat=data-security)

IBM Privacy Shield Privacy Policy for Certified IBM Cloud Services can be found below. This is applicable to EU-US and Swiss-US customers: [https://www.ibm.com/privacy/details/us/en/privacy_shield.html](https://www.ibm.com/privacy/details/us/en/privacy_shield.html)

Data Responsibility at IBM [https://www.ibm.com/blogs/policy/dataresponsibility-at-ibm/](https://www.ibm.com/blogs/policy/dataresponsibility-at-ibm/) If a government wants access to data held by IBM on behalf of a SaaS client, IBM would expect that government to deal directly with that client