vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Database restore can only be done to one of the previous daily backups (cannot restore to point in time).





 Disaster Recovery 

In the event of a DR issue with the Maximo Application Suite Dedicated Service offering for a specific customer, IBM's focus will be in the following order:



1. Recover the existing infrastructure in place
2. Recover within the same IBM Cloud data center to a new infrastructure
3. Recover to a secondary IBM Cloud data center



In the event a disaster is declared, the base parameters are:

Recovery Time Objective (RTO) - 72 hours Recovery Point Objective (RPO) - 24 hours

RTO is the longest possible time needed to make the application available. RPO is the longest possible time since data was last backed up.

These are Service Level Objectives (SLO's) and not Service Level Agreements.

For information on IBM business continuity, please refer to IBM's Business Continuity Management Position Paper:

[https://www.ibm.com/downloads/cas/81M5VGP7](https://www.ibm.com/downloads/cas/81M5VGP7)
* UI
* CLI
* API
* Terraform







 Point-in-time Recovery (PITR) 

IBM Cloud® Databases for MongoDB Enterprise Edition offers PITR using any timestamp greater than the earliest available recovery point. To discover the earliest recovery point through the API, use the [point-in-time-recovery timestamp endpoint](https://cloud.ibm.com/apidocs/cloud-databases-api/cloud-databases-api-v5capability).

{
"point_in_time_recovery_data": {
"earliest_point_in_time_recovery_time": "2019-09-09T23:16:00Z"
}
}

In this phase, the point-in-time-recovery timestamp endpoint always returns current time - approximately one week.



 Recovery 

Backups are restored to a new deployment. After the new deployment finishes provisioning, your data in the backup file is restored into the new deployment. Backups are also restorable across accounts, but only by using the API and only if the user that is running the restore has access to both the source and destination accounts.

The new deployment is automatically sized to the same disk and memory allocation as the source deployment at the time of the backup from which you restore. Especially with PITR, that might not be the current size of your deployment. If you need to adjust the resources that are allocated to the new deployment, use the optional fields in the UI, CLI, or API to resize the new deployment. If the deployment is not given enough resources the restore fails, so allocate enough for your data and workload.

While storage and memory are restored to the same as the source deployment, specific instance configurations are not automatically set for the new instance. In this case, rerunning the configuration after a restore might be needed. Note any instance modifications before running the restore (parameters like shared_buffers, max_connections, deadlock_timeout, archive_timeout) to ensure accurate settings for the instance after the restore is complete.

Do not delete the source deployment while the backup is restoring.
Disaster recovery 

IBM is responsible for the recovery of Red Hat OpenShift on IBM Cloud components in case of disaster. You are responsible for the recovery of the workloads that run the cluster and your application data. If you integrate with other IBM Cloud services such as file, block, object, cloud database, logging, or audit event services, consult those services' disaster recovery information.



Table 6. Responsibilities for disaster recovery

 Resource IBM responsibilities Your responsibilities 

 General <br><br> * Maintain service availability across [worldwide locations](https://cloud.ibm.com/docs/openshift?topic=openshift-regions-and-zones) so that customers can deploy clusters across zones and regions for higher DR tolerance.<br> * Provision clusters with three replicas of master components for high availability.<br> * In multizone regions, automatically spread the master replicas across zones.<br> * Continuously monitor to work to ensure the reliability and availability of the service environment by site reliability engineers.<br> * Update and recover operational Red Hat OpenShift on IBM Cloud and Kubernetes components within the cluster, such as the Ingress application load balancer and file storage plug-in.<br> * Back up and recover data in etcd, such as your Kubernetes workload configuration files<br> * Provide the ability to integrate with other IBM Cloud services such as storage providers so that data can be backed up and restored.<br><br><br> <br><br> * Set up and maintain disaster recovery capabilities for your apps and data. For example, to prepare your cluster for HA/DR scenarios, follow the guidance in [High availability for Red Hat OpenShift on IBM Cloud](https://cloud.ibm.com/docs/openshift?topic=openshift-ha). Note that persistent storage of data such as application logs and cluster metrics are not set up by default.<br><br><br> 







 Applications and data 

You are completely responsible for the applications, workloads, and data that you deploy to IBM Cloud. However, IBM provides various tools to help you set up, manage, secure, integrate and optimize your apps as described in the following table.



Table 7. Applications and data
An architecture diagram showing a Code Engine example application architecture

Any configuration that is mentioned on the Code Engine project is performed on both projects to maintain consistency between the primary and backup instances.

In the event that a failover needs to be triggered, these are the basic steps to follow.



1. Make sure that the app is deployed in Code Engine in the backup MZR.
2. Make sure that any data that is needed by the app is available in the backup MZR. This could be accomplished in different ways, depending on the service. IBM Cloudant has bidirectional replication, PostgreSQL uses read-only replicas that can be promoted to be leaders, MongoDB and Redis rely on backup and restore procedures.
3. If necessary, scale up the application in the backup MZR to handle the expected load.
4. When the data is ready and the Code Engine application is appropriately scaled, start the application.
5. Update the DNS CNAME record to point to the Code Engine application endpoint of the Code Engine application in the backup MZR.







 Continuous availability profile application - IBM Cloudant 

Zoom

![IBM Cloudant example application architecture](https://cloud.ibm.com/docs-content/v1/content/adcecdbe4e86955f88aac7f3c1e7ec3ff44992ef/overview/images/BCDR-Architecture-Diagram-Image8.svg)

Figure 7. IBM Cloudant example application architecture

In this scenario, IBM Cloudant will replicate the data automatically. In the event of a disaster, the only change that is needed will be to reconfigure DNS to point to the cluster in US East.

Benefits
: Fastest recovery time, dependent only on the time it takes to verify that the data was replicated and make the DNS switch.

Impacts
: Cost. To support the RTO, the only viable option is the active/hot-standby model.
: Development teams need to update their deployment pipelines to also deploy their apps to the standby IBM Cloud Code Engine project every time they deploy to production.
Avoiding common disaster recovery pitfalls 

To ensure that your plans provide the results that you require in the event of a disaster, be aware of the following common disaster recovery (DR) pitfalls when planning your solution.



 Plan and design for the “best conditions” 

It is important that the solution and the testing methodology are planned and designed to mimic what the real conditions might be.

A DR plan is a set of policies, tools, and procedures that enable organizations to recover from and continue the operation of vital technology systems following a natural or human-induced disaster. The business continuity and disaster recovery (BCDR) infrastructure capacity should be able to manage the workloads of the production sites. Assuming that this condition will not occur might seem attractive from a cost perspective, but it can make the effects of a disaster much more severe when there is not enough capacity to manage normal business.

Avoid reusing dismissed production components or lower grade and quality assets to build the DR solution because it affects the reliability of the solution. The amount of time that this solution needs to manage production is not predictable, and this variable introduces more risks.





 Avoiding single-point-of-failure (SPOF) 

SPOF can be anywhere in a solution, not only on the technology side. Your solution might depend on people, vendors, providers, and other external dependencies. You must know and identify clearly in advance most of your SPOFs and have a plan to mitigate your dependencies. Be prepared to discover SPOFs during the first sessions of your [disaster recovery test](https://cloud.ibm.com/docs/overview?topic=overview-dr-testing).

Among SPOFs, provider risk is a condition to consider in your DR plan. When you have both production and DR on the same provider, your risk condition is increased and must be carefully considered.





 Having only a plan A 

Risks can't be eliminated; they can be only mitigated. The more risk that you mitigate with a single solution, the more the solution would cost. Thus, any solutions have a residual risk, as budget for disaster recovery is normally a finite or limited number.

Having a plan B with a different
Air-gap An air gap is a term that is used to denote network isolation where the Linux backup repository is not accessible directly from the production networks. For backup transfer, the air gap is selectively unlocked and then locked when the transfer is complete. 
 Production environment The vSphere environment that hosts the production data. For the isolated recovery environment, the production environment can be in IBM Cloud or elsewhere. 
 Cyber-recovery site The location of the isolated recovery environment. 
 Production data Production data refers specifically to the VM images (VMDKs) of the VMs running in production environment. 
 Cyber-recovery data Copies of the production VM images (VMDKs) in the isolated recovery environment. 
 Cyber-recovery backup The backup of production data to the isolated recovery environment. A cyber-recovery backup is different from a production backup. 
 Cyber-recovery toolsets Cyber-recovery toolsets are applications that are provided by the client for use within the cyber-recovery site to scan, analyze, or clean cyber-recovery data. 
 Sandbox A sandbox is an isolated zone on a network that enables the OS and applications to be started without impacting production systems. Specifically, in these solution architectures, it is one or more NSX-T segments that are protected with firewall rules where copies of backups can be accessed or used to start VMs. 







 Related links 



* [About Veeam Backup and Replication](https://helpcenter.veeam.com/docs/backup/vsphere/overview.html?ver=120)
* [Veeam architects site](https://www.veeambp.com/)
* [Overview of VMware Solutions](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-solution_overview)
* [Veeam on bare metal server introduction](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-veeam-bms-archi-intro)
Designing an architecture for your application resiliency objectives 

It is important to ensure that you're meeting your defined recovery (DR) objectives and the DR metric thresholds for your application. Review the following sample application architecture examples to understand how you can meet your recovery objectives by using IBM Cloud®.

The resiliency options, proposed profiles, and associated information are presented so that you can define your application's DR requirement levels. Information that is stated is not a warranty and IBM® will not issue credits for failure to meet an objective. These

recovery time objective(RTO) andrecovery point objective(RPO) examples are presented as a reference for additional steps that can be taken to achieve different levels of resiliency. Refer to the Service Level Agreements (SLAs) for any commitments and credits that are issued upon failure to meet any committed SLAs. For more information about the recovery strategy classes, see [Planning your applications recovery strategy objectives](https://cloud.ibm.com/docs/overview?topic=overview-understanding-drplanning-your-applications-recovery-strategy-objectives).



 Hybrid disaster recovery: Microservices 

Business continuity is a top priority for modern applications. The following example shows patterns and best practices to implement DR for hybrid microservice applications in an active/passive configuration.

Zoom

![Disaster recovery for hybrid microservice applications](https://cloud.ibm.com/docs-content/v1/content/adcecdbe4e86955f88aac7f3c1e7ec3ff44992ef/overview/images/BCDR-Architecture-Diagram-image1_architecture_diagram.svg)

Figure 1. An architecture diagram for disaster recovery for hybrid microservice applications

Some of the significant challenges that are unique to the hybrid DR microservices implementation architecture include the following:

Business continuity
: Business processes can continue to be processed despite man-made or natural disasters.

Operational flexibility
: Having a well-designed solution with code and data in multiple sites allows for flexibility for where applications and data are deployed based on user need and traffic.

Cost reduction
Maintain regular backups of toolchain and pipeline definitions, Git repos, and any other toolchain integration data that is stored and managed by IBM. To support global failover, create and maintain alternative toolchain and pipeline definitions, including tool integration data and your Git repos, in another IBM region. 
 Restore toolchain data. Restore all toolchain and Git repos to the original IBM Cloud region, when that region is available. To support global failover, manually switch to using the alternative toolchains and repos in another region. 



To learn more about disaster recovery, see [Continuous Delivery high availability and disaster recovery](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-ha-dr).
Currently, Data Security Broker Manager supports a high availability deployment that is active-passive. To configure and manage Data Security Broker Shields across all regions for a specific application, administrators can only use a single instance of Data Security Broker Manager.

The metadata kept by the Data Security Broker Manager instance must regularly be backed up from the primary instance and restored to the standby instance in order to guarantee that the metadata for the primary and standby Data Security Broker Managers remain consistent. To perform the backup and recovery procedures required to synchronize the Data Security Broker Manager instances, see [Disaster recovery for IBM Cloud Data Security Broker](https://cloud.ibm.com/docs/security-broker?topic=security-broker-sb_dr).





 Synchronization Frequency 

In order to configure data protection policies or audit current policies, service availability requirements determine how frequently the active and standby Data Security Broker Managers synchronize. Typically, these are rare occurrences, and a job that automatically synchronizes metadata between the primary and standby must be adequate.

The synchronization job between the primary and standby Data Security Broker Managers must be run after each configuration change if you do not wish for configuration loss.





 Deploying Across Regions 

The standby Data Security Broker Manager instance must be deployed in a different geographic region than the primary Data Security Broker Manager instance in order to maximize availability of Data Security Broker Manager in the event of any outage. It is not necessary for the two Data Security Broker Manager instances to be directly connected to one another, but they must both have access to a common backup repository where the backup files used to synchronize the two instances are kept.

To ensure that Data Security Broker Shields are aware of both the primary and standby Data Security Broker Manager instance, follow the steps below:



1. The IP address of the deployment of the standby Data Security Broker Manager must be configured in the primary Data Security Broker Manager's Proxy_Access parameter.
2. The primary Data Security Broker Manager deployment's IP address must be entered in the BM_IP parameter for the Helm Chart that is used to deploy Data Security Broker Shield.



Ensure that you add and configure the Application in Data Security Broker Manager after completing High availability configuration.
An alert indicating a change or update on a mobile app icon.







 Q 



 quota 

The number of resources that can be consumed at an account or service instance level.







 R 



 Raft 

A crash fault tolerant ordering service implementation based on the etcd library of the Raft protocol. Raft follows a leader-and-follower model, where a leader node is elected per channel, and its decisions are replicated by the followers.





 read-mostly 

Pertaining to data that changes dynamically.





 recall 

A measurement that specifies the percentage of relevant results returned, out of all available relevant results. Recall, which is a measure of sensitivity, is determined by the number of correct positive results divided by the number of positive results that should have been returned. Accuracy is best measured by using both precision and recall. See also [precision](https://cloud.ibm.com/docs/overview?topic=overview-glossaryx2003831), [accuracy](https://cloud.ibm.com/docs/overview?topic=overview-glossaryx3125742).





 recovery point objective (RPO) 

In disaster recovery planning, the point at which data is restored to in the event of a disaster. See also [disaster recovery](https://cloud.ibm.com/docs/overview?topic=overview-glossaryx2113280).





 recovery time objective (RTO) 

The maximum duration of time within which an application should be restored after any type of disaster. See also [disaster recovery](https://cloud.ibm.com/docs/overview?topic=overview-glossaryx2113280).





 red-black deployment 

A deployment technique that drives continuous delivery by enabling synchronized test, development, and deployment. Initially, development is done on an inactive environment (black) while the active environment continues to take traffic (red). Once deployment starts, both environments go live (red-red) until routing is disabled on the formerly active, previous version environment, then subsequently removed (black) while the new environment serves as the only active environment.
Notice that a single data center distributes data across devices within a single site only.

Create a custom COS bucket with the storage features and the policies that you identify.[Learn more](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-archiving).

Do not configure a long-term retention policy on a COS bucket if you need access to the data and query it with the Data Engine service.

Use the following table to help you identify the features that you should consider when you create a bucket:

Disaster Recovery or compliance requirements

Backup or access to data beyond the service plan days requirements



Table 6. Disaster recovery or compliance requirements

 Requirement Storage class: Standard[7] Storage class: Vault[8] Storage class: Cold Vault[9] Storage class: Flex[10] 

 Type of workload Continuous access Data isn't accessed frequently Accessed every 90 days or less Dynamic workloads where access patterns are difficult to predict 
 Data resiliency Cross region Cross region Cross region Cross region 
 Retention policy ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/checkmark-icon.svg) !
Notice that a single data center distributes data across devices within a single site only.

Create a custom COS bucket with the storage features and the policies that you identify. [Learn more](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-archiving).

Do not configure a long-term retention policy on a COS bucket if you need access to the data and need to query it with the Data Engine service.

Use the following table to help you identify the features that you should consider when you create a bucket:

Disaster Recovery or compliance requirements

Backup or access to data beyond the service plan days requirements



Table 6. Disaster recovery or compliance requirements

 Requirement Storage class: Standard[7] Storage class: Vault[8] Storage class: Cold Vault[9] Storage class: Flex[10] 

 Type of workload Continuous access Data isn't accessed frequently Accessed every 90 days or less Dynamic workloads where access patterns are difficult to predict 
 Data resiliency Cross region Cross region Cross region Cross region 
 Retention policy ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker/images/checkmark-icon.svg) !
Select a data center for the DR node and click Enable disaster recovery.

Zoom

![Select a DR data center](https://cloud.ibm.com/docs-content/v1/content/2a59ee244ed61eb35ab50e45757b42d22f720132/Db2onCloud/images/dr_2_v2.jpg)

Figure 2. Select a data center for the recovery node
3. The new DR node is displayed on the Disaster recovery page along with a notification indicating a successful deployment.

Zoom

![DR Deployed](https://cloud.ibm.com/docs-content/v1/content/2a59ee244ed61eb35ab50e45757b42d22f720132/Db2onCloud/images/dr_3_v2.jpg)

Figure 3. Recovery node sucessfully created







 Forcing a failover to the DR site 



1. To force a failover to the DR site, open the web console for the recovery site from the IBM Cloud dashboard.

Zoom

![Recovery node console](https://cloud.ibm.com/docs-content/v1/content/2a59ee244ed61eb35ab50e45757b42d22f720132/Db2onCloud/images/dr_4_v2.jpg)

Figure 4. Recovery node console
2. To initiate a takeover, click Promote on the Disaster recovery page.

Zoom

![Recovery node takeover](https://cloud.ibm.com/docs-content/v1/content/2a59ee244ed61eb35ab50e45757b42d22f720132/Db2onCloud/images/dr_5_v2.jpg)

Figure 5. Recovery node takeover
3. Click Promote to confirm takeover.

Zoom

!