vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* UI
* CLI







 Disaster Recovery - Fail over with an accessible primary volume 

If a catastrophic event occurs on the primary site and the primary storage is still accessible, customers can take the following steps to quickly access their data on the secondary site.

Before you start the failover, make sure that all host-authorization is in place.

Authorized hosts and volumes must be in the same data center. For example, you can't have a replica volume in London and the host in Amsterdam. Both must be in London or both must be in Amsterdam.



 Authorizing the host in the UI 

You can authorize a host to access the IBM Cloud® File Storage for Classic volume through the [IBM Cloud® console](https://cloud.ibm.com/cloud-storage/file).



1. Log in to the [IBM Cloud® console](https://cloud.ibm.com/login) and click the menu icon on the upper left. Select Classic Infrastructure![Classic icon](https://cloud.ibm.com/docs-content/v1/content/04e9937a86546143babfc65ea21fdd4ea2d12d13/icons/classic.svg).
2. Locate the source or destination volume in the File Storage for Classic list. Its replica volume is listed under the source volume in the inactive status.
3. Click the replica name and on the next screen, click Actions![Actions icon](https://cloud.ibm.com/docs-content/v1/content/04e9937a86546143babfc65ea21fdd4ea2d12d13/icons/action-menu-icon.svg). From the menu, select Authorize Hosts.
4. Select a host type and then choose a host from the list that is available for the volume. Filter the available host list by the device type, subnet, or IP address.

When the list is filtered by subnet, the subnets that are displayed are subscribed subnets in the same data center as the storage volume.
5.
You can create a full database backup in the SQL Server by using SQL Server Management Studio. Table-level backups cannot be created. For more information, see [Backup and restore with SQL Server Management Studio](https://docs.microsoft.com/en-us/sql/relational-databases/backup-restore/quickstart-backup-restore-database?view=sql-server-ver15).





 Transactional replication 

Transactional replication is a method for copying and distributing data and database objects from one database to another and then synchronizing between databases to maintain consistency. This enables changes to be transferred between one database and another, which includes data, tables, stored procedures, views, and so on. Transactional replication is typically used in database clustering or server-to-server environments and is used for high availability of database and high throughput. However, this method can be used for database migration. For more information, see [Transactional replication](https://cloud.ibm.com/docs/microsoft?topic=microsoft-mssql-migrationmssql-migration-rep).





 Database mirroring 

Database mirroring in SQL Server allows you to keep a copy, or mirror, of an SQL Server database on a standby server. Mirroring ensures two separate copies of the data always exist, providing high availability and complete data redundancy. This method is usually used for increased availability of a database, as well as during production database upgrades. This method can also be used for database migration. For more information, see [Database mirroring](https://cloud.ibm.com/docs/microsoft?topic=microsoft-mssql-migrationmssql-migration-mirror).





 Always On availability groups 

SQL Server Always On availability groups provide high availability and disaster recovery solutions and are available in versions of SQL Server 2012 and newer. This feature can be used to migrate your existing SQL Server databases to IBM Cloud with minimal downtime. If you have an existing Windows® Server Failover Cluster with Always On availability groups, you are able to extend the cluster temporarily during migration by creating an additional secondary replica with asynchronous replication.
A multizone region (MZR) consist of 3 or more availability zones that are independent from each other to ensure that single failure events affect only a single zone. By default, Log Analysis hosted event search is deployed across 3 zones, one primary zone and two secondary zones:



* Each zone is located in a different data center in the region.
* The data in your primary zone is automatically replicated to the secondary zones with low latency. You don't need to do anything to enable the replication.
* The service is designed to withstand a single zone failure with no interruption.



The MZR architecture offers automatic failover between zones within the region, and high availability for a auditing instance withing a region.





 SZR 

The SZR architecture offers failover across 3 distinct systems within the single datacenter so that you get high availability from a system failure, but not from a datacenter failure.





 Data availability for Activity Tracker hosted event search offerings 

When you provision an auditing instance, you select the location where the instance is created. The region determines where the auditing data is processed and the data is hosted.





 Considerations 

IBM Log Analysis follows IBM Cloud requirements for [planning and recovering from disaster events](https://cloud.ibm.com/docs/overview?topic=overview-zero-downtimedisaster-recovery).

When a regional disaster occurs, consider the following information:



* Data and the auditing metadata such as dashboards, alerts, views, screens, templates are backed up every 24 hours. In the event of an un-recoverable disaster, up to 24 hours of data and metadata changes to the auditing instance in the failure region can be lost.
* The estimated recovery time for rebuilding the regional site and restoring the service at another location is 24 hours.
* Due to the large volume of data, older data might not be available at the time the service is restored, as this process requires additional time to recover data from the backups.
* When the auditing instance in the DR region is available in the new location, you will be able to use it while the data is restored into the newly constructed region.
Backing up and restoring storage data 

Depending on the storage solution that you use for your Red Hat® OpenShift® on IBM Cloud® cluster, you have different options to back up and restore your data.

If you have not provisioned storage, review the [Kubernetes storage basics](https://cloud.ibm.com/docs/openshift?topic=openshift-storage-plan) to understand your storage options for single zone and multizone clusters.



*  [File Storage for Classic](https://cloud.ibm.com/docs/openshift?topic=openshift-file_storage)
*  [Block Storage](https://cloud.ibm.com/docs/openshift?topic=openshift-block_storageblock_backup_restore)
*  [Block Storage for VPC](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-blockvpc-block-backup-restore)
*  [Object Storage](https://cloud.ibm.com/docs/openshift?topic=openshift-storage-cos-understand)
*  [Portworx disaster recovery](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_recovery)
* [Encrypting your data](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-encryption)









 Storage in Satellite location 

Within the Satellite location, Satellite storage uses Satellite Config to provide a convenient way to install various storage drivers in Red Hat OpenShift on IBM Cloud clusters, by using storage templates. The storage templates are provided and tested by the vendors. After you install Satellite storage, your cluster users can use Kubernetes persistent volume claims (PVCs) to order and save their application data in persistent storage. For more information, see [Understanding Satellite storage](https://cloud.ibm.com/docs/satellite?topic=satellite-sat-storage-template-ov).

You are responsible for providing the underlying virtual and physical storage that will be accessed by Satellite storage in your on-premises Satellite location. You should ensure your underlying physical storage is encrypted using keys managed by an on-premises FIPS 140-2 level 2 or higher hardware security module (HSM).





 Related controls in IBM Cloud Framework for Financial Services 

More details on some of the major IBM Cloud Framework for Financial Services controls related to storage can be found in the following articles:



* [Encryption at rest](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-encryption-at-rest)
* [Backup and recovery](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-bcdr)







 Next steps 



* [Audit logging of IBM Cloud events](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-logging-audit)
You can incrementally add audio resources and train your custom acoustic models rather than adding all data before training the models. For example, you can add your audio resources one at a time or in groups, training your models on subsets of the audio resources rather than on all of the audio at one time.







 Disaster recovery for asynchronous speech recognition jobs 

For speech recognition with the asynchronous HTTP interface, you need to maintain the following information:



* All callback URLs that you allowlist for use with the asynchronous interface. If a failure occurs, you might need to use the POST /v1/register_callback method to re-register the URLs. The method returns an appropriate response if a URL is already allowlisted.
* Copies of the audio files that you submit to the asynchronous interface for speech recognition. If a failure occurs before you receive or retrieve the results of a completed asynchronous job, you need to use the POST /v1/recognitions method to resubmit the audio files when the service is restored. Once you have the results of a completed asynchronous job, you no longer need to maintain the audio files.



For more information, see [The asynchronous HTTP interface](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-async). As with the backup data for custom models, you can actively preserve this information and be prepared to reissue the necessary requests ahead of time.
Disaster recovery for custom acoustic models 

For custom acoustic models, you need to maintain and be prepared to re-create and restore your custom acoustic models and their audio resources. You also need to be prepared to retrain your custom acoustic models on their audio resources.



 Backing up custom acoustic models 

Preserve the following information about your custom acoustic models:



* A list of all of your custom acoustic models and their definitions. To list information about your custom models:



* Use the GET /v1/acoustic_customizations method to list information about all custom models.
* Use the GET /v1/acoustic_customizations/{customization_id} method to list information about a specified custom model.



For more information, see [Listing custom acoustic models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-manageAcousticModelslistModels-acoustic).
* Copies of all audio resources, both individual audio files and archive files, that you add to your custom acoustic models. To list information about the audio resources for your custom models:



* Use the GET /v1/acoustic_customizations/{customization_id}/audio method to list information about all audio resources for a custom model.
* Use the GET /v1/acoustic_customizations/{customization_id}/audio/{audio_name} method to list information about a specified audio resource for a custom model.



For more information, see [Listing audio resources for a custom acoustic model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-manageAudiolistAudio).



It is a best practice to preserve this information in a format that you can use to re-create your custom acoustic models in the event of a failure. Actively maintaining the information about your custom models and their audio resources, and preparing the calls listed in the following section ahead of time, can enable you to recover as quickly as possible.





 Restoring custom acoustic models 

If you need to recover from a disaster, you can use the backup information to re-create your custom acoustic models and their data:



1.
Components in a three-tier highly available architecture on an IBM Cloud VPC 

This tutorial covers the configuration of all the components of a three-tier architecture that includes the load balancers. It is for administrators who are creating three tier applications in IBM next-gen MZRs. Working knowledge of IBM Cloud VPC, Linux® administration, three-tier applications like nginx, tomcat and mysql might be required. While it incorporates Nginx, Apache Tomcat and MySQL servers, it can be extended to any other components, such as Apache, PHP, and other database systems. The underlying infrastructure setup and layout will remain the same. The replication, configuration, and clustering components might change depending on the software you choose for your application.



 Overview 

Whether you are running a mission critical application or a development and testing environment, having a backup plan to restore your systems promptly is crucial to increase productivity and reduce the impact to your business.

With a production system, you can’t afford downtime and must plan an HA and a DR strategy. You need to be able to bring your systems back up to business whether you experience a catastrophic failure or because of human error, and in the shortest possible time. In some cases, systems are dispensable and you can easily re-create them with scripts while in others, a few minutes to hours of downtime is acceptable.

Your strategy depends on:



* Your business requirements
* Available hardware and software redundancy
* The geography and latency requirements of your busines application
* Your financial affordability to implement such policy



The two main factors that are associated with HA architectures are RPO and RTO.

RPO, the Recovery Point Objective, refers to the amount of data loss that you can tolerate. The higher the frequency of your backups, the better your RPO.

RTO, the Recovery Time Objective, refers to the time required to bring the system back online after a failure or disaster occurs. The time taken to restore the files and roll forward (or backward) with transaction logs is your RTO.

You might need to design your HA strategy based on your RPO and RTO objectives. A more stringent RPO and RTO might also be cost prohibitive.





 Architecture diagram
* Cyber-recovery backup proxy is a Veeam VMware backup proxy that is used to transport data from the production environment to the cyber-recovery site. It is an architecture component that sits between the cyber-backup server and other components of the cyber-backup infrastructure. While the cyber-backup server administers tasks, the cyber-recovery backup proxy processes jobs and delivers backup traffic. Basic backup proxy tasks include retrieving VM data, compression, de-duplicating, encrypting, and forwarding to the cyber-recovery backup repository. For more information, see [Veeam backup proxy](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-veeam-cr-sa-vp).







 vCenter Server instance 

The vCenter Server instance provides the compute and network resources to host VMs and sandboxes. If the vSAN option in the vCenter Server instance ordering is used, then the vCenter Server instance also provides the storage resources.

The vCenter Server instance consists of a consolidated cluster with a minimum of three vSphere ESXi hosts. This cluster hosts the following VMs:



* vCenter server appliance - The VCSA manages the vSphere resources and provides a single management plane for vSphere.
* NSX-T™ manager cluster - The NSX-T manager cluster consists of three manager appliances that provide the management and control plane for the virtualized network, also known as the overlay network.
* Active Directory™ and domain name service - The ADDNS VMs are two Microsoft® Windows 2019 VMs configured for Active Directory and DNS. This option is selected in the order process by selecting the "Two highly available dedicated Windows Server VMs on the management cluster" option. By selecting this option, you must provide your own licenses.
* Veeam service instance - For more information, see [Veeam components](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-veeam-cr-sa-components).
* Gateway cluster - A gateway cluster consists of two vSphere ESXi hosts provisioned on IBM Cloud gateways.
A region contains [multiple availability zones](https://www.ibm.com/cloud/data-centers/) to meet local access, low latency, and security requirements for the region. Each [multizone region (MZR)](https://cloud.ibm.com/docs/overview?topic=overview-locationstable-mzr) is composed of 3 or more zones that are independent from each other to ensure that single failure events affect only a single zone.





 Availability of Code Engine instances 

When you provision a Code Engine project, you select the location (MZR) where the instance is created. The region determines where your workloads, such as apps and jobs, are hosted. By default, your workload is deployed within a zone. If a failure of the hosting zone occurs, the workload is automatically re-created in one of the remaining zones.

For more information about deploying apps across multiple regions with a custom domain name, see [Deploying an application across multiple regions](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regions).





 Disaster Recovery for Code Engine instances 

In a major regional disaster, such as an earthquake, flood, or tornado, an entire region might be impacted. To ensure that your workloads are resilient to such events, deploy your workloads across multiple MZRs and implement an automatic failover mechanism by leveraging an Edge Proxy service. For example, you can use the service that is provided by [IBM Cloud® Internet Services](https://cloud.ibm.com/docs/cis?topic=cis-getting-started). For more information about deploying an application across multiple regions, see [Deploying an application across multiple regions with a custom domain name](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regions).





 Backing up your Code Engine instances 

IBM Cloud performs a system backup of service instances metadata and stores this backup data in a cross-regional storage instance.



Table 2. Cross-regional endpoints

 Code Engine region Cross-regional endpoint 

 au-syd AP 
 br-sao BR
Specify a Service name if you want something other than the default name.
6. Click Create.



When you provision an Event Streams service instance, a service cluster is automatically deployed into your Satellite location. You can verify the start of the deployment of the service cluster with the following steps:



1. From the left Navigation menu, select Satellite, then Locations.
2. Select your Satellite location.
3. Select Services.
4. Verify that a service named messagehub is listed. If it is not yet listed, refresh the page until it is listed before you move to the next step.



While the service instance and cluster are provisioned, create the storage assignment. Proceed to the next step and complete the instructions.





 Step 5: Create the block storage configuration assignment (by using Satellite Storage UI) 

During the Event Streams service instance provision, block storage configuration is automatically queued for confirmation and assignment. This confirmation and assignment requires acknowledgment from the Satellite location administrator.

The following steps use the Storage user interface for Satellite. If you prefer to use the CLI, you can use it to create the storage configuration from templates, and then assign that configuration to the Event Streams messagehub service cluster.



1. Go to Satellite, by clicking Satellite > Locations in the navigation bar.
2. Select your Satellite location.
3. Select the Services tab.
4. Look for the acknowledgment window.



1. Complete the storage configuration setup.
2. Complete assignment of the storage configuration to the Event Streams service cluster.





After the storage assignment is created, allow up to 60 minutes for the Event Streams service instance to be ready for use.





 Step 6: (Optional) Enable the schema registry API 

The schema registry API is not automatically enabled when you provision an Event Streams Satellite instance. You must provide an IBM Cloud Object Storage bucket as the backend storage for schema registry to enable this API. You are responsible for managing this bucket, including but not limited to: data encryption, data backup, and disaster recovery.



1. Create a Cloud Object Storage instance either on IBM Cloud or on a Satellite location.
In the event of a catastrophic failure impacting an entire region, the transit gateway service management plane, which is global, continues to be available. Transit gateway instances are restored when the failing region recovers. However, in some rare cases, you may need to recreate specific instances of your transit gateways, as well as their connections.

Your disaster recovery plan should include knowing, preserving, and being prepared to restore all data that is maintained on IBM Cloud. This includes data about all deployed transit gateways and their connections.



 Backing up transit gateways for disaster recovery 

You must be prepared to recreate your transit gateways and connections. This section helps you ensure you have all the data needed for this purpose.

IBM Cloud Transit Gateway backups are cross-regionally durable. They are stored across multiple regions, and are restorable to other regions.

The following steps use the IBM Cloud CLI, but you can also use the IBM Cloud console or API.

Preserve a list of all of your transit gateways and their connections. To do so, follow these steps:



1. Use the ibmcloud tg gateways command to list details about all your transit gateways. Save this output.
2. For each gateway, use the ibmcloud tg connections GATEWAY_ID command to list information about its connections. Save this output.



For more information, see the [Transit Gateway CLI reference](https://cloud.ibm.com/docs/transit-gateway?topic=tg-cli-plugin-transit-gateway-cli).

Saving the information returned from these commands will help you recover from a failure quickly. In the event of a failure, use the saved information and run the ibmcloud tg gateway-create and ibmcloud tg connection-create commands to recreate your transit gateways and connections.