vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Identity and access IBM provides the function to restrict access to resources through the IBM Cloud console and REST APIs. The Customer is responsible for managing access to resources through IBM Cloud Identity and Access Management (IAM). 







 Security and regulation compliance 

Security and regulation compliance includes tasks, such as security controls implementation and compliance certification.



Table 5. Responsibilities for security and regulation compliance

 Task IBM Responsibilities Your Responsibilities 

 Encryption IBM does not provide encryption capabilities. The Customer is responsible for encryption of data on disk, in motion, and in backups. The Customer is also responsible for choosing and managing appropriate additional security features. If the Customer uses Key Protect (Bring Your Own Key), or another form of encryption, the Customer is responsible for managing the service authorization and keys. 
 Security IBM is responsible for ensuring the security of data on disk and data in motion within its infrastructure. The Customer is responsible for restricting user access to the appropriate resources and resource groups. 







 Disaster recovery 

Disaster recovery includes tasks, such as providing dependencies on disaster recovery sites, provision disaster recovery environments, data and configuration backup, replicating data and configuration to the disaster recovery environment, and failover on disaster events.



Table 6. Responsibilities for disaster recovery

 Task IBM Responsibilities Your Responsibilities 

 Diversity IBM provides diverse network options for consumption. The Customer must ensure diversity of Direct Link is deployed. 
 Redundancy IBM provides diverse network options for consumption. Direct Link is not a redundant service. The Customer is responsible for establishing redundancy, as needed, via BGP schema. The Customer must also understand that Direct Link is not a redundant service. While IBM Cloud supplies Diverse Router (XCR) options, failover must be built into the BGP scheme a customer deploys between multiple Direct Links. 
 Host service in multiple regions IBM is responsible for hosting this service in multiple regions. The Customer is responsible for designing and deploying their workload in a way that achieves the wanted availability and Disaster Recovery capabilities by using provided tools. For example, deploy in different zones of a region, use at least two load balancers that are located in different zones, and either use DNS records to point to the load balancers, or ensure that your application can handle a list of IP addresses that it can connect to.
High availability and disaster recovery 

The IBM Cloud® Hyper Protect Virtual Servers service runs on reliable IBM LinuxONE infrastructure. The IBM Cloud® Hyper Protect Virtual Servers service will not be available during regular maintenances. A single virtual server can still have an outage in a disaster scenario. Therefore, deploy your workload in active-active mode across multiple virtual server instances, which are running in different data centers (for example, Dallas 10,Dallas 12, and Dallas 13). This setup ensures an operable workload with fault tolerance on the underlying virtual servers.

Example workloads that you can deploy this way are databases (PostgreSQL, MongoDB, or MySQL) or applications with no local state.

If the latency requirements or types of workload do not allow to run an active-active configuration across data centers, you can perform regular backups from one virtual server to another instance in a different data center. In a disaster, the amount of lost data depends on the frequency of the backups and the time to restore a backup.



 Backup 

Back up your data from business-relevant virtual servers (primary virtual servers) to recovery virtual server instances. A virtual server that is created with Hyper Protect Virtual Servers is configured with two disks. One for the operating system and the other one for your data (mounted as /data/). One possible way to back up the data disk is to set up a cron job, which copies content of the disk to a recovery virtual server instance. Choose the appropriate backup frequency for your workload. For example, if you accept a maximum loss of data changes of 1 hour, add the following text file (for example, cron_backup) to /etc/cron.hourly:

!/bin/sh

rsync -a /data/ <public HPVS IP or internal HPVS IP>:/data

On Ubuntu, you install the file-copying tool rsync on the primary virtual server in two steps:

Initially, on the primary virtual server, use the following command to synchronize the package index files from their sources. This is required to retrieve information about available packages, versions, and dependencies.

apt-get update
When you create a backup job, you must enter a password for the encrypted data. The password is case-sensitive. To recover the data, you must provide the encryption password that was entered when the files were backed up.

You can also enter a password hint. When you want to restore data, you can view the password hint to remind you of the encryption password for this job.

If you forget the encryption password, you lose access to the data. You cannot retrieve the password from the system.





 Performance Options 

Bandwidth throttling settings specify the amount of bandwidth that is used by an IBM Cloud Backup for Classic Agent for backups. For example, you might want to restrict the amount of bandwidth that is used for daytime backups so that online users are not affected. At the same time, you might allow unlimited bandwidth usage at night so that scheduled backups run as fast as possible.

Bandwidth settings include the following parameters.



* Maximum bandwidth, in megabits per second, to be used by the IBM Cloud Backup for Classic Agent for all backups and restores.
* Length of time during the day that throttling is in effect. Only one time window can be specified. Outside the window, no throttling takes place.
* Days of the week that throttling is in effect.



If the bandwidth-throttling time period begins when a backup is underway, the maximum bandwidth is applied dynamically to the running backup. Similarly, if the bandwidth-throttling time period ends when a backup is running, bandwidth throttling is ended for the backup.

If you edit an IBM Cloud Backup for Classic Agent’s bandwidth settings while a backup is running, the new settings do not affect the backup that is running. Bandwidth settings are applied when a backup starts, not during its operation.







 Downloading the user guide 

Connect to the IBM Cloud® network with [IBM Cloud® VPN](https://www.ibm.com/cloud/vpn-access) so that you can download the user guides from the [Downloadable IBM Cloud Backup for Classic Documentation](http://downloads.service.softlayer.com/evault/Documentation/).
If you create a duplicate from a replica volume, the new volume is created in the same data center as the replica volume.

Duplicate volumes can be accessed by a host for read/write as soon as the storage is provisioned. However, snapshots and replication aren't allowed until the data copy from the original to the duplicate is complete.

For more information, see [Creating a duplicate File Volume](https://cloud.ibm.com/docs/FileStorage?topic=FileStorage-duplicatevolume).





 Using replicas to fail over when disaster strikes 

When you fail over, you’re "flipping the switch" from your storage volume in your primary data center to the destination volume in your remote data center. For example, your primary data center is London and your secondary data center is Amsterdam. If a failure event occurs, you’d fail over to Amsterdam – connecting to the now-primary volume from a compute instance in Amsterdam. After your volume in London is repaired, a snapshot is taken of the Amsterdam volume to fail back to London and the once-again primary volume from a compute instance in London.



* If the primary location is experiencing problem but the storage and host are still online, see [Fail over with an accessible primary volume](https://cloud.ibm.com/docs/FileStorage?topic=FileStorage-dr-accessible).
* If the primary location is down, see [Fail over with an inaccessible primary volume](https://cloud.ibm.com/docs/FileStorage?topic=FileStorage-dr-inaccessible).
If you require a higher level of service for automatic disaster recovery, see IBM's [Cloud disaster recovery solutions](https://www.ibm.com/cloud/disaster-recovery).
* How many copies of my backup can I create in other regions?

 How many copies of my backup can I create in other regions? 

You can copy a backup snapshot from one region to another region, and later use that snapshot to restore a volume in the new region. Only one copy of the backup snapshot can exist in each region. You can't create a copy of the backup snapshot in the source (local) region.
* What's the difference between Bare Metal Servers (Classic infrastructure) and Bare Metal Servers for VPC?

Compared to the classic bare metal infrastructure, Bare Metal Servers for VPC provides the following advantages:



* Security and performance of the private cloud with the flexibility and scalability that you expect of the public cloud.
* Integrates with other VPC services such as virtual server, security groups, networking, and more.
* Faster provisioning and deployment and simple hourly billing.



Keep in mind that Bare Metal Servers for VPC is less customizable than classic bare metal servers.

For more information about the differences between the Classic infrastructure and VPC, see [ Comparing IBM Cloud Classic and VPC infrastructure environments](https://cloud.ibm.com/docs/cloud-infrastructure?topic=cloud-infrastructure-compare-infrastructure).
* What operating systems are supported?

Currently, VMware ESXi, Windows, RHEL, RHELfor SAP, Debian GNU, SUSE Linux Enterprise, Ubuntu Linux, and custom images are supported.
* What storage options are available?

Two storage options are available that include secondary local NVMe drives.



* The bx2-metal-192x768 profile provides mirrored 960 GB SATA M.2 drives as boot storage only.
* The bx2d-metal-192x768 profile provides mirrored 960 GB SATA M.2 drives as boot storage and 16 3.2 TB U.2 NVMe SSDs as secondary local storage to support vSAN, or user-managed RAID.



VPC Block Storage is not supported.
Open a support ticket with details if you have time periods longer than a minute with no connectivity so that the interruptions are investigated.





 Disaster Recovery Strategy 

IBM® watsonx.data provides mechanisms to protect your data and restore service functions. Business continuity plans are in place to achieve targeted recovery point objective (RPO) and recovery time objective (RTO) for the service. The following table outlines the targets for watsonx.data.



Table 2. Disaster Recovery Strategy

 Disaster recovery objective Target Value 

 RPO <= 24 hours 
 RTO < 24 hours 







 Locations 



 AWS Regions 



1. Oregon (us-west-2)
2. N. Virginia (us-east-1)
3. Frankfurt (eu-central-1)







 IBM Regions 



1. Dallas (us-south)
2. Washington (us-east)
* [MongoDB](https://cloud.ibm.com/docs/databases-for-mongodb?topic=databases-for-mongodb-high-availabilityhigh-availability)
* [RabbitMQ](https://cloud.ibm.com/docs/messages-for-rabbitmq?topic=messages-for-rabbitmq-high-availability)



* Each cluster is spread over multiple zones.
* Data is replicated across the zones.
* Users can scale up storage and memory resources for an instance. See the example in the [documentation on scaling for Databases for Redis](https://cloud.ibm.com/docs/databases-for-redis?topic=databases-for-redis-resources-scaling) for details.
* Backups are taken daily or on demand. Details are documented for each service. Here is an example of [backup documentation for Databases for PostgreSQL](https://cloud.ibm.com/docs/databases-for-postgresql?topic=cloud-databases-dashboard-backups).
* Data at rest, backups, and network traffic are encrypted.
* Each [service can be managed using the Cloud Databases CLI plug-in](https://cloud.ibm.com/docs/databases-cli-plugin?topic=databases-cli-plugin-cdb-reference)









 Continuous Availability profile application - IBM Cloud Object Storage 

Zoom

![Object Storage architecture](https://cloud.ibm.com/docs-content/v1/content/adcecdbe4e86955f88aac7f3c1e7ec3ff44992ef/overview/images/BCDR-Architecture-Diagram-image10.svg)

Figure 9. An architecture diagram showing a Object Storage architecture

In this scenario, the data is always available in both MZRs through cross-regional buckets. Therefore, the only change that is needed is to update the DNS routing to point to the backup MZR.







 Advanced recovery profile
Understanding high availability and disaster recovery for Cloud Shell 

As an IBM Cloud® platform service, IBM® Cloud Shell follows the practices that are described in [How IBM Cloud ensures high availability and disaster recovery](https://cloud.ibm.com/docs/overview?topic=overview-zero-downtime), with the exception that IBM Cloud Shell is regionally available rather than globally available. Because Cloud Shell is region-dependent, automated global failover isn't supported. You should always maintain a backup of your data outside of Cloud Shell.

IBM Cloud Shell is available in the Dallas (us-south), Frankfurt (eu-de), and Tokyo (jp-tok) regions, and each location has three different data centers for redundancy. The data for each location is kept in the data centers near that location. If all the data centers in a location or region fail, IBM Cloud Shell becomes unavailable in that location or region. When this happens, you can change to another location or region as described in [Changing locations](https://cloud.ibm.com/docs/cloud-shell?topic=cloud-shell-shell-uichange-location).

For more information, see [How IBM Cloud ensures high availability and disaster recovery](https://cloud.ibm.com/docs/overview?topic=overview-zero-downtime).



 Backing up your data 

Your Cloud Shell workspace storage is temporary, and it's not intended to be used as the primary storage location for your files. Don't store business-critical or sensitive data in your workspace, and create a backup of your files outside of IBM Cloud Shell. You're responsible for your data, and your backup can help you recover in case an outage or data loss occurs for any reason. For more information about how to transfer files to create a backup, see [Transferring multiple files](https://cloud.ibm.com/docs/cloud-shell?topic=cloud-shell-filestransfer-many-files).





 What to do if a region is not available 

If a region is not available, Cloud Shell indicates that the current region or location is not available. Files and data from that region are not available unless you manually restore them from a previous backup.
While this design has only one backup server instance, it is recommended to deploy Enterprise Manager when encryption is used for backup or backup copy jobs. It is advised to install the Enterprise Manager server on the recovery site so it is available for disaster recovery.
* Proxy server - This proxy is used for management components that are located in the recovery region.
* Repository - A location used to store backup files for the management components in the recovery region and also the target for backup copy jobs from the protected region.
* SFTP/SMB server - These servers are not Veeam services but native Windows services that are used for file-level backups of some of the management components. A Veeam file copy job copies files to the protected region for extra protection.



Review the following Veeam design decisions:



* For optimal performance and availability, placing the Veeam components on separate virtual and physical servers is considered best practice. However, this practice increases complexity in smaller environments. Therefore, the all-in-one deployment scenario for use case 1 is selected.
* As the total number of protected VMs is low, the embedded database option for the database for use case 1 is selected.
* The bare metal servers with direct attached storage option are used as it provides a backup infrastructure that is separated from the virtualized infrastructure compute and storage.
* In a two-site environment, it is best practice to install the Veeam Backup server component in the DR site. In a disaster situation, Veeam Backup server is available to start the recovery.
* Deploy Enterprise Manager to use password loss protection. Enterprise Manager administrators can unlock backup files by using a challenge-response mechanism.
* It is recommended that the proxy is as close as possible to the source data with a high-bandwidth connection. The traffic from the source to the proxy is not yet optimized, meaning that 100% of the backup data is transferred over this link. A good connection is required between proxy and repository as optimized data (normally 50% of the source data size) is transferred across this link. Therefore, place proxies in both the protected and recovery regions.
* Proxies can be hosted on Windows Server or Linux OS with almost no performance differences. For the all-in-one deployment scenario, a Windows OS is used.
Managing your data in Container Registry 

Information about your data and how it is stored in IBM Cloud® Container Registry.

The IBM Cloud platform provides layered security controls across network and infrastructure. IBM Cloud provides a group of security services that can be used by application developers to secure their mobile and web apps. For more information, see [How do I know that my data is safe?](https://cloud.ibm.com/docs/overview?topic=overview-security)



 How your data is stored 



 Image data 

Image data is stored in IBM Cloud Object Storage, which is encrypted at rest, and encrypted in transit between IBM Cloud Container Registry and IBM Cloud Object Storage. For more information about IBM Cloud Object Storage, see [About IBM Cloud Object Storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-about-cloud-object-storage).

Data that is stored in IBM Cloud Container Registry is backed up regularly. For more information, see [High availability and disaster recovery](https://cloud.ibm.com/docs/Registry?topic=Registry-ha-dr).





 Scanning data 

To scan images and containers in your account for security issues, IBM Cloud Container Registry collects, stores, and processes the following information:



* Free-form fields, including IDs, descriptions, and image names ([registry](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_overviewoverview_elements_registry), [namespace](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_overviewoverview_elements_namespace), [repository](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_overviewoverview_elements_repository) name, and image [tag](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_overviewoverview_elements_tag))
* Metadata about the file modes and creation timestamps of the configuration files
* The content of system and application configuration files in images and containers
* Installed packages and libraries (including their versions)
Disaster recovery testing 

Disaster recovery (DR) testing is what you must do to keep your ability to be resilient, hoping not to need to do it for real. There are different types of disaster recovery testing that you can do to verify your resiliency capabilities:



* DR dry test
* DR simulation
* Switch-over





 Disaster recovery dry test 

A DR dry test is performed by checking all of the resource availability and runbooks on paper, without running a real DR simulation or switch-over.

This type of testing is normally run with higher frequency compared to the other testing flavors as no real activities are performed, but it does require the same effort in terms of skills and people.

The adoption of a recovery orchestrator software improves the whole dry testing process and reduces the time and effort to be run because most of the operations are performed by the software itself. This also increases the preparedness checking of a real DR simulation or switch-over by periodic scanning that all of the components of the DR solutions are in place and works as expected.





 Disaster recovery simulation 

DR simulation is a way to verify or audit the emergency runbooks and check the

recovery time objectives(RTO) andrecovery point objectives(RPO) provided by the solution by simulating as much as possible in the same conditions as a real emergency.

This means introducing disruptions on replication network connections before interrupting the communications among the sites, hence simulating the sudden loss of the primary site.

You can do this only if your data replication solution is resilient and not providing impact on the production, which continues on the primary site. If this is not the case, then look at the documentation for [plan and design for the worst conditions](https://cloud.ibm.com/docs/overview?topic=overview-understanding-drworst-conditions).

DR simulation deploys a duplicate of your production environment on the DR site that you can use to perform validation and checking. This environment is cleaned at the end of the simulation and updates that happened to the DR test environment are discarded, as the real production has continued on the primary site.

It is important thus that network streams flowing to the DR test environment are copies of the real production environment.
* System documentation - External documentation, internal architecture documentation, run books, and so on.



Other requirements:



* Backups must be at least incremental daily and full weekly.
* Backups must be monitored. Failed backups must be investigated and corrective action that is documented as necessary to maintain the service's RPO.
* Backups and backup logs should be accessible only to authorized individuals who need access to do their jobs.
* Transaction-based systems must implement transaction recovery. Transaction-based systems include database management systems and transaction processing systems. Mechanisms supporting transaction recovery include transaction rollback and transaction journaling.







 Information system recovery and reconstitution 

You should enable recovery and reconstitution of the system to a known state after a disruption, compromise, or failure. Contingency plans must include procedures for validating successful recovery and reconstitution. Recovery and reconstitution activities include resuming operational capabilities at the original location.







 Backup and disaster recovery for the reference architectures 

See the following resources depending on which reference architecture you are using:



* [Business continuity and disaster recovery for VPC reference architecture](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-vpc-architecture-bcdr)
* [Business continuity and disaster recovery for Satellite reference architecture](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-satellite-architecture-bcdr)







 Backup and disaster recovery for IBM Cloud services 

In addition to backing up your workloads and having the capability to restore service in the face of a disaster, your BCDR strategy needs to consider all of the IBM Cloud services in your deployment. The following table provides references to additional information regarding BCDR for each service in the reference architecture.

The following table provides references for more information about BCDR for each service in the reference architecture.



Table 1. Backup and disaster recovery information for IBM Cloud services in the reference architectures

 Category VPC reference architecture Satellite reference architecture Optional for both