vania-janet/multiturn-rag-retrieval-data huggingface dataset:
High availability and disaster recovery 

IBM® watsonx.data service instances are deployed in IBM Cloud multi-zone region (MZR) and AWS MZR . The availability of watsonx.data components is Active-Active and Active-Only.



 Active-Active 

Multi-tenant components support multiple customers and are configured with multiple replicas across Availability Zones (AZs) to ensure availability. This category consists of most of the watsonx.data components.





 Active-Only 

Single-tenant components in this category are dedicated to a single customer. This category consists of the Presto engine and metastore. These components restart in a new zone if there is a failure.

In Multi Zone Regions (MZRs), Presto and HMS are distributed across different zones.

When a single availability zone fails in an MZR, or a hardware failure occurs in any region, the workloads automatically fail and restart in other zones within that region. Every watsonx.data instance comes with a default cross-regional Metadata bucket and an optional Trial bucket(10 GB). Both the buckets are enabled with IBM Cloud® Object Storage Versioning. The data is backed up by enabling replication to a separate IBM Cloud Object Storage Account. However, for any external bucket that the customer brings into watsonx.data instance, the customer is responsible for those backups.

In case of a regional disaster, you will receive an email that includes all the steps that you need to follow. See responsibilities for watsonx.data. Single-tenant components operate on an 'Active Only' model, ensuring immediate restart on new nodes that provide the same service if there is a failure.

Single-tenant components are strategically distributed across 3 AZs to enhance reliability. When an AZ fails, sufficient capacity to initiate the required services on the available AZs is ensured. This minimizes any impact that is caused by an AZ outage.





 Responsibilities 



Table 1. Responsibilities

 Task IBM Responsibilities Your Responsibilities 

 Backups watsonx.data is responsible for automatic daily backups, of all watsonx.data provided resources. The Client is responsible for:1) Create a new instance of IBM watsonx.data to restore the backups and validate that the IBM backups that are restored properly.
[Express](https://expressjs.com/) to enable a simple web server that allows interaction with the data.



There are five main files:



1. server.js: This runs the web server and communicates with Redis. It is getting regular calls from the front end to the /data route. It connects to Redis, downloads an object with the ID and position of every truck and returns that to the front end.
2. index.html: This is the one and only page of the front end application. When it loads, it shows a map of the US from the awesome [Open Street Map](https://www.openstreetmap.org/copyright) service. Then it is on a five-second loop where it makes an HTTP request to the backend (server.js), retrieves all the trucks and their positions and loops through them to plot their latest position on the map. Each truck has a map marker whose lat/long gets updated with every cycle.
3. producer.js: This takes in a location file and a truck_id from the script invocation and then loops through all the data points in the location file. Each data point is turned into a message with the truck ID and lat/long and posted once a second to Event Streams. For added fun, it picks a random starting point along the route and turns around and comes back when it reaches the end of the location file.
4. cloudantConsumer.js: This script creates a trucktracker database in Cloudant if it does not already exist and then polls Event Streams at regular intervals for more data. When data comes in, it gets written in bulk to Cloudant.
5. redisConsumer.js: This script polls Event Streams at regular intervals for data. When data comes in, it gets written into a Redis "hash" where the hash key is the truck ID and the value is the truck data (lat/long). This hash arrangement allows all the truck data to be pulled down by the server.js script in one go.
When you perform a recovery, your data is restored from the most recent incremental backup and any outstanding transactions from the WAL log are used to catch your database up to the time you recovered to. After the recovery is finished, and the transactions are run, the logs display a message. You can check that your logs have the message,

LOG: last completed transaction was at log time 2019-09-03 19:40:48.997696+00

There are two scenarios where recovery does not show up in the logs.



1. Your deployment has a recent full backup and there is no activity after the backup was taken that needs to be replayed.
2. If you entered a time to recover to that is after the current time or is past latest available point-in-time recovery point.



In both cases the recovery is usually still successful, but there won't be an entry in the logs to check the exact time that the database was restored to.
* First, make sure the Zendesk Agent Workspace is not enabled for your account: From the Zendesk navigation pane, go to Settings, and then click Agents. Deselect the Enable the Zendesk Agent Workspace checkbox.
* Then download the app. From the Chat dashboard navigation pane, expand Settings, and then click Account.
* Open the App tab.
* Click Upload private app, and then browse for the application file that you downloaded earlier.



![Screen capture of the Zendesk Account page where you can upload a private app.](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/zd-upload-app.png)

For more information, see [Uploading and installing a private app in Zendesk Chat](https://developer.zendesk.com/documentation/apps/getting-started/uploading-and-installing-a-private-app/).



5. Click Save and exit to finish setting up the connection to the Zendesk Chat service desk.



When you test the service desk integration, make sure there is at least one agent with Online status. Agent status is set to Invisible unless it is explicitly changed.

Watch a 4-minute video that provides an overview of setting up a connection to a Zendesk service desk:

The product user interface is slightly different from the interface that is shown in the video. However, the main steps are the same.





 Securing the transfer to Zendesk 

When you add security to your Zendesk integration, you ensure that the visitors you are helping are legitimate customers. Enabling visitor authentication also enables support for cross-domain traffic and cross-browser identification. For more information, see the [Zendesk documentation](https://support.zendesk.com/hc/en-us/articles/360022185314-Enabling-authenticated-visitors-in-the-Chat-widget).

Before you can secure the Zendesk connection, complete the following required tasks:



1.
* Is a VPE created in high-availability mode?

A VPE is not created in high-availability (HA) mode, by default. HA comes primarily from the IBM Cloud service.
* Can I access an IBM Cloud service by using a private service endpoint IP address?

When an IBM Cloud service is created, IBM Cloud DNS Services are automatically set up to resolve the IBM Cloud service FQDN to the IBM Cloud private service address.

When a VPE is created, VPE assigns a reserved IP with which you can access the IBM Cloud service. It is recommended to use the reserved IP instead of the IBM Cloud private service endpoint.
* Does mapping an IBM Cloud service to an IP address on a VPC network make the service private?

Mapping an IBM Cloud service to an IP address on a VPC network does not make the service private. For example, if a service has a public endpoint, you can still access the public endpoint after the service is mapped.
* Does controlling access to an IP on a VPC network that is mapped to a service control access to the mapped service?

Controlling access to an IP address on a VPC network that is mapped to an IBM Cloud service does not control the access to the mapped service itself.
* Is there a limit to the number of IP addresses I can bind to an endpoint gateway?

When the reserved IP address that is bound to the endpoint gateway is source NATed on the VPC gateway, it is done by using IP masquerading on the port. As the number of IP addresses bound to the endpoint gateway grows, the number of available ports to masquerade might become a concern.
* How many IP addresses can I use for NAT operations on the VPC gateway?

A finite pool of IP addresses is used for NAT operations on the VPC gateway. One IP address is required per VPC per zone.
* What does a VPN health status indicate?

Status descriptions are as follows:



* Heathy - Indicates that your VPN server is operating correctly.
* Degraded - Indicates that your VPN server has compromised performance, capacity, or connectivity.
* Faulted - Indicates that your VPN server is completely unreachable and inoperative.
For more information on this issue, see [(SRX) IPSec comes UP when SRX-A is the Initiator, but fails when SRX-A becomes the responder](https://kb.juniper.net/InfoCenter/index?page=content&id=KB22239) in the Juniper Knowledge Base. Consult [vpn (Security)](https://www.juniper.net/documentation/en_US/junos/topics/reference/configuration-statement/security-edit-vpn.html) for more details on these settings.





 Correcting warning 1177 

One or more security zone policy rules with the dynamic-application any configuration was detected. If the vSRX is not installed with the Content Security Bundle (CSB) license and application signature database, then this configuration might cause traffic disruption due to changes in newer vSRX releases, such as 19.4R2-S3.

For example, the following security policy configuration contains the dynamic-application any label:

set security policies from-zone untrust to-zone untrust policy DYNAMIC-APPLICATION-POLICY-LOCAL match dynamic-application any
set security policies from-zone untrust to-zone untrust policy DYNAMIC-APPLICATION-POLICY-LOCAL match source-address SL8
set security policies from-zone untrust to-zone untrust policy DYNAMIC-APPLICATION-POLICY-LOCAL match destination-address SL8
set security policies from-zone untrust to-zone untrust policy DYNAMIC-APPLICATION-POLICY-LOCAL then permit

If you are using a similar configuration, it is recommended that you either install the CSB license and the application signature database or remove the dynamic-application any rule.





 Correcting warning 1179 

The vSRX version running on the Gateway is not certified on IBM Cloud and is unsupported. Operations such as OS Reload and Rebuild Cluster will overwrite the current unsupported vSRX version with the version currently listed on the Gateway Details page.
Why is my RHEL virtual server created from a Classic Infrastructure custom image in failed state? 

After importing a Red Hat Enterprise Linux custom image from IBM Cloud® Classic Infrastructure to IBM Cloud VPC, you use it to create a new virtual server instance in VPC. However, the new virtual server instance remains in failed state.

 What’s happening 

You use a Red Hat Enterprise Linux custom image from IBM Cloud Classic Infrastructure that was imported to IBM Cloud VPC to provision a new virtual server instance in VPC. However, the new virtual server instance does not start successfully; it is in a failed state.

 Why it’s happening 

When the new VPC virtual server that was created from a Classic Infrastructure custom image attempts to start, the Red Hat Enterprise Linux license activation fails.

 How to fix it 

To resolve the licensing issue for the Red Hat Enterprise Linux image in the VPC environment, you must log in to the failed virtual server, rearrange the network routes, and re-register the operating system. When the licensing issue is fixed, the virtual server instance should start successfully.



1. Log in to the failed virtual server instance by using SSH. Locate the fabric IP address for the virtual server instance in the IBM Cloud console on the Virtual server instance page. Log in to the failed virtual server instance by using SSH in a terminal. The following example shows how to log in to the virtual server instance from a terminal by using SSH.

 % ssh -i .ssh/id_rsa_vsikey1 root@52.116.206.42
Warning: Permanently added '52.116.206.42' (ECDSA) to the list of known hosts.
[root@skv-vsirh2x7 ]
2. Rearrange the network routes for the Red Hat Enterprise Linux operating system that is in the failed state.



1. Determine the right gateway from the ip/route table. Use the ip show/ route command to get the entire list of routes that are present. Use the ping command to determine the right gateway. See the following example:

[root@skv-vsic1x7im ] route -n
Kernel IP routing table
The stage runner tries to access and mount the configmap named config-name.
* $prop-name: The indirect configmap name syntax. The configmap name is looked up from the environment-properties configmap, which contains every environment value that is set on the pipeline UI. For example, if there is a prop: my-config entry in the environment properties, then my-config is used for $prop.



* secret: Pull key-value pairs from a provided configmap that is accessible by PipelineRun. The following values are supported:



* secret-name: The direct Secret name syntax. The stage runner tries to access and mount the Secret named secret-name.
* $prop-name: The indirect Secret name syntax. The Secret name is located in the environment-properties configmap that contains every environment value that is set on the pipeline UI. For example, if the environment-properties configmap contains the my-secret entry property, my-secret is used for $prop.







 Example configuration 

version: '1' fixed, this value is used to track schema changes

 setup runs right after the app repo is cloned
setup:

 the pipeline will break in case setup fails (default is true)
abort_on_failure: true

 any docker image can be used which is accessible by the private worker
image: ibmcom/pipeline-base-image:2.7

 you can reference configmaps, each key in the configmap is going to be available as /config/{key}
configmap: my-config

 $prop is the indirect config map syntax, the concrete configmap is looked
 up from the environment-properties configmap

 Eg. if there's a prop: my-config entry in the environment properties,
 then my-config is going to be used for $prop
configmap: $prop

 the mechanism described works for secrets as well!
secret: $my-secrets

 the script is executed inside the checked out app repo
script: 
!/bin/sh
Observing Kubernetes Ingress 

You can monitor the Kubernetes Ingress ALBs by using IBM Cloud Monitoring.



  Setting up monitoring with IBM Cloud Monitoring 

Gain operational visibility into the performance and health of your Kubernetes Ingress ALBs by deploying a Monitoring agent to your worker nodes to forward metrics to IBM Cloud® Monitoring. When you deploy a monitoring agent to the worker nodes in your cluster, Monitoring is already automatically enabled to detect and scrape the data and then display them in your IBM Cloud Monitoring dashboard.



1.  [Provision an instance of IBM Cloud Monitoring](https://cloud.ibm.com/observe/monitoring/create).
2.  Configure a monitoring agent in your cluster.



1.  Select your monitor from the [Monitoring](https://cloud.ibm.com/observe/monitoring) page.
2.  Click Monitoring sources.
3.  Follow the instructions for installing a monitoring agent to your cluster.



Note that it may take a few minutes for your agent to finish installing.
3.  In the [Monitoring console](https://cloud.ibm.com/observe/monitoring), click Open Dashboard for the instance that you provisioned.
4.  In the Monitoring UI, click Dashboards > Applications > NGINX Ingress. Note that the NGINX Ingress dashboard does not appear until there is data available. This process can take some time. The NGINX Ingress dashboard is visible only when the monitoring agent finds nginx_ingress_controller_requests metrics value. These metrics are exposed when you send queries to the hosts that are defined in your Ingress resources.

The default Ingress monitoring filters can produce a high volume of metrics, which can incur additional billing. Add the filter exclude: nginx_ingressbucket to reduce costs by excluding some metrics while retaining the default metrics needed for the NGINX dashboard.



For more information about IBM Cloud Monitoring, see [Getting started tutorial](https://cloud.ibm.com/docs/monitoring?topic=monitoring-getting-started).
When the core is in Hyper-Threading Technology mode, you need twice as many licenses as you do in single-threaded mode, even though the application is not getting two times better performance over single-threaded mode. As a result, EDA and other practitioners disable Hyper-Threading Technology and prefer to run their software in single-threaded mode.

The second reason for disabling Hyper-Threading Technology is to avoid load balancing issues and operating system overhead that comes with Hyper-Threading Technology. High performance computing (HPC) applications typically create as many message passing interface (MPI) tasks as the number of physical cores and let the operating system manage each MPI task on a distinct physical core. When Hyper-Threading Technology is enabled, the operating system sees twice as many CPUs. While the operating system usually does a great job at mapping the MPI tasks to one of the two threads of a core, the OS occasionally might place both tasks on a physical core and leave other cores free. Having both tasks on a physical core with other cores free results in sub-optimal system utilization and variable application performance. In addition, the OS kernel allocates data structures for each of the active CPUs and updates them continuously so that more kernel space is allocated in Hyper-Threading Technology mode and management overhead is alleviated. Alleviating operating system overhead by disabling Hyper-Threading Technology helps many HPC applications achieve optimal performance.

The third reason for disabling Hyper-Threading Technology is security. Some users prefer to disable Hyper-Threading Technology to avoid any side channel attack possibilities that are associated with Hyper-Threading Technology execution. IBM Cloud does not share a physical core between two customer's virtual machines so this kind of attack is not possible.

If you determine that disabling Hyper-Threading Technology is right for your configuration, you can use one of the following methods to disable it on your guest virtual machines (VMs) in IBM Cloud.





 Checking Hyper-Threading Technology status and changing it in my guest VM: Basics 

When a virtual machine is up and running, you can check the Hyper-Threading Technology status by using the lscpu command on Linux operating systems. You see output similar to the following:

Architecture: x86_64
: In the context of Code Engine, any incoming HTTP request (even the request to load a web page) or a REST API call is considered an event. The concept of being event-driven is often the key factor when you choose between an app or a job because, by definition, apps are run because of an HTTP request while jobs are run as of result of an invocation.
: If you know that your workload responds to incoming HTTP requests, then app is the right choice. However, if your workload is run and then done, a job is a better fit.

How does your code scale?
: Both apps and jobs are scalable. Apps are scaled in response to measurable real-time criteria such as the number of active incoming requests because each instance of your app might be able to process only a certain number of concurrent requests at one time. Jobs are scaled based on how many instances are specified when the job is created.
: If you know that you want a specific number of instances of your code to run, and each instance can be run without an incoming HTTP request, then a job is the right choice. However, if the number of instances must scale dynamically based on the incoming HTTP load, apps are a better fit.





 Common scenarios for Code Engine 

Read through some of these common scenarios to gain understanding of when to choose an app and when to choose a job.

Does your workload require low latency or is it interactive?
: If your workload requires a client or user to wait synchronously for the response of the request, and the response must be available within a few milliseconds, use an application. Applications provide an externally reachable endpoint and respond synchronously to the request. Examples of such workloads are websites, chatbots, and mobile applications. Use [applications](https://cloud.ibm.com/docs/codeengine?topic=codeengine-application-workloads).

Is your computation lightweight and does it require low CPU, memory, and I/O?
: If your workload is lightweight and requires low CPU, memory, and I/O, then the concurrency option, available for applications might be helpful.
Each NetScaler appliance configuration can be a single Netscaler or a pair of Netscalers in an HA pair, depending on your requirements, providing local load balancing services for the servers behind them. The devices are configured to talk to one another so they exchange state information on each server assigned to global load balancer rotation. Any DNS request that comes to these configured NetScalers can return a proper record for a server that is online and responsive. Any server not responsive is removed from the rotation and another is selected.

You must already have load balancing set up, even if it is only one server being balanced. You will need additional IP addresses for some services, namely the GSLB site IP. This IP is used by the NetScaler to communicate with the other NetScalers in the global load balancing protocol.

The following global balancing procedure uses:



 VPX1 

IP 50.97.235.236 is named VPX1Vserver, and is the local load balancing VIP for that device. 50.23.66.52 will be called VPX1site, and is the local IP for that device's GSLB.





 VPX2 

IP 208.43.241.249 is used for VPX2Vserver, and the GSLB IP is 208.43.224.4, called VPX2site.



1. Go to Traffic Management > GSLB and right click to enable the feature. Then, select Sites and Add.
2. On the first device, enter the Name as VPX1, the Type as local, and the IP as 50.23.66.52, then select Close.

You should see the site listed with a status of green. Do not add a remote site yet.
3. Go to Traffic Management > GSLB and select the GSLB Wizard. Click Next. Enter the hostname that you will be load balancing (in this example, gslb.tsstesting.com) Leave the Record Type as A, and the Service Type ANY. The virtual server name will populate itself. Click Next.
4. Choose your form of balancing and persistence method just like you would with regular load balancing. Click Next.
5.