vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Why is my cluster missing the public containers.appdomain.cloud subdomain? 

Virtual Private Cloud

Classic infrastructure

 What’s happening 

When you expose an app through a Ingress controller subdomain, you get a local subdomain instead of a public route, in the format: <service_name>-<project_name>.router.default.svc.cluster.local.

When you try to open the Red Hat OpenShift web console or another app route in your browser, you might see an error similar to the following.

Application is not available
The application is currently not serving requests on this endpoint.

 Why it’s happening 

After the cluster is created and enters a normal state, the Ingress controller subdomain networking and load balancing components still take some time to deploy.

If you expose your app before the networking components fully provision, or if the components experience an error, your apps can only be exposed internally with the default Ingress controller's svc.cluster.local domain.

When the components fully provision, a public Ingress controller subdomain is available for your apps, in the format <cluster-name>-<accountID-hashed>-<ssll>.<region>.containers.appdomain.cloud.

 How to fix it 



1. After you create a cluster, wait some time before you expose your apps, even after the cluster enters a normal state.
2. Check the Master Status. If the Master Status is not Ready, [review its status](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_master) and follow any troubleshooting information to resolve the issue.

ibmcloud oc cluster get -c <cluster_name_or_ID>
3. Check that your cluster has public connectivity so that the networking components can talk to the master as they deploy.



* VPC clusters with public and private cloud service endpoints enabled: [Ensure that a public gateway is enabled on each subnet](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-subnetscreate_vpc_subnet) that your cluster is attached to.
Configuring recommended backup flows 

A backup schedule is to perform the following types of backups.



*  Monthly full backups of all the databases.
*  Weekly full backups of all the databases.
*  Daily differential backups of all the databases.



To set up a recommended backup schedule, follow the steps.



1.  Log in to the web console.
2.  Go to Backup and restore > Backup.
3.  Make sure that destination buckets for your backups are created.

For more information, see [Creating destination buckets](https://cloud.ibm.com/docs/netezza?topic=netezza-bnr-overviewcreate-destinations).
4.  Configure your monthly schedule:



1.  Select a destination bucket for your backup.
2.  Select Full for the backup type.

The first backup must be a full one so that any subsequent differential backup has a reference.
3.  Select the databases that you want to back up.
4.  Check the Schedule for a later time box.
5.  Specify the starting date.
6.  Specify the starting hour.
7.  Select Monthly when you specify the frequency.
8.  Click Create schedule.



5.  Configure your weekly schedule:



1.  Select a destination bucket for your backup.
2.  Select Full for the backup type.
3.  Select the databases that you want to back up.
4.  Check the Schedule for a later time box.
5.  Specify the starting date.

Select a date that is a week after the monthly start date and time.
6.  Specify the starting hour.
7.  Select Weekly when you specify the frequency.
8.  Click Create schedule.



6.  Configure your daily schedule:



1.  Select a destination bucket for your backup.
2.  Select Differential for the backup type.
3.  Select the databases that you want to back up.
4.  Check the Schedule for a later time box.
5.  Specify the starting date.

Select a date that is a day after the monthly start date and time.
6.  Specify the starting hour.
7.  Select Daily when you specify the frequency.
8.  Click Create schedule.
When an AZ fails, sufficient capacity to initiate the required services on the available AZs is ensured. This minimizes any impact that is caused by an AZ outage.





 Responsibilities 



Table 1. Responsibilities

 Task IBM Responsibilities Your Responsibilities 

 Backups watsonx.data is responsible for automatic daily backups, of all watsonx.data provided resources. The Client is responsible for:1) Create a new instance of IBM watsonx.data to restore the backups and validate that the IBM backups that are restored properly. 2) Restore backups of external components that they brought into watsonx.data. 
 Restore watsonx.data handles the restoration of backups for provided resources. The Client is responsible for:1) Create a new instance of watsonx.data to restore the backups and validate that the IBM backups that are restored properly. 2) Restore backups of external components that they brought into watsonx.data. 







 Application-level high availability 

Applications that communicate over networks and cloud services are subject to transient connection failures. Design your applications to retry connections when a temporary loss in connectivity to your deployment or to IBM Cloud, causes errors. As watsonx.data is a managed service, regular updates and maintenance occur as part of normal operations. Such maintenance occasionally causes a temporary service interruption.

Your applications must be designed to handle temporary interruptions to the service, implement error handling for failed commands, and implement retry logic to recover from a temporary interruption.

Following are some of the error codes that might be expected during the temporary service interruptions:

If a Presto coordinator node restarts, be it for maintenance purposes or due to a system failure, applications are required to reestablish their connection with the Presto engine.

Several minutes of unavailability or connection interruptions are not expected. Open a support ticket with details if you have time periods longer than a minute with no connectivity so that the interruptions are investigated.





 Disaster Recovery Strategy 

IBM® watsonx.data provides mechanisms to protect your data and restore service functions. Business continuity plans are in place to achieve targeted recovery point objective (RPO) and recovery time objective (RTO) for the service. The following table outlines the targets for watsonx.data.



Table 2. Disaster Recovery Strategy
Why is my agent not showing up in the list of Active Agents? 

  What’s happening 

My agent is running but it does not show up in the list of Active Agents in the UI.

  Why it’s happening 

This can be caused by several reasons. Check the following list to determine the cause.

  How to fix it 



*  There is a small delay (approximately 2 minutes) before a running Connector agent shows up in the list. Ensure you’ve waited long enough for the Connector agent to show up in the list.
*  Make sure the Connector ID your agent is using matches the Connector ID you are connecting to.
*  Make sure the API Key is valid and has the right permissions to the Connector instance.
*  Check the logs of the agent to determine if there are any errors.
*  Make sure the region specified for your Connector is accurate and matches one of the values in [Supported IBM Cloud regions](https://cloud.ibm.com/docs/satellite?topic=satellite-sat-regions).
Point-in-time Recovery 

IBM Cloud® Databases for MySQL offers Point-In-Time Recovery (PITR) for any time in the last 7 days. The deployment continuously backs up incrementally and can replay transactions to bring a new deployment that is restored from a backup to any point in that 7-day window you need.

The Backups tab of your deployment's UI keeps all your PITR information under Point-in-Time.

Zoom

![PITR section of the Backups tab](https://cloud.ibm.com/docs-content/v1/content/99dae60bd0de1987ffcce4822e29b2bcdefe3a43/databases-for-mysql/images/pitr-backups-tab.png)

Figure 1. PITR section of the Backups tab

Included information is the earliest time for a PITR. To discover the earliest recovery point through the CLI, use the [cdb mysql earliest-pitr-timestamp](https://cloud.ibm.com/docs/databases-cli-plugin?topic=databases-cli-plugin-cdb-referenceMySql-earliest-pitr-timestamp) command.

ibmcloud cdb mysql earliest-pitr-timestamp <deployment name or CRN>

To discover the earliest recovery point through the API, use the [/deployments/{id}/point_in_time_recovery_data](https://cloud.ibm.com/apidocs/cloud-databases-api/cloud-databases-api-v5getpitrdata) endpoint to find the earliest PITR time.

{
"point_in_time_recovery_data": {
"earliest_point_in_time_recovery_time": "2019-09-09T23:16:00Z"
}
}



 Recovery 

Backups are restored to a new deployment. After the new deployment finishes provisioning, your data in the backup file is restored into the new deployment.
The mappings can be found in the image-map.tf file and the scale-image-map.tf file in this public [GitHub repository](https://github.com/IBM-Cloud/hpc-cluster-symphony).





 Why does the egosh command not work if tried immediately after Schematics provisions the cluster? 

This is expected behavior. Even after the Schematics web console shows that the cluster is successfully provisioned, there are still some tasks that run in the background for several minutes. Allow a few minutes (typically 2 minutes is sufficient) after the cluster gets provisioned for egosh to be available.





 Why does cluster creation using dedicated hosts fail sometimes with the error status of dedicated host is failed? 

In some regions, dedicated hosts have a limitation on the number of virtual server instances that can be placed on them at one time. You can try to provision the cluster with a smaller number of virtual server instances to overcome this.





 Why does Spectrum Scale not allow use of the default value of 0.0.0.0/0 for security group creation? 

For security reasons, Spectrum Scale does not allow you to provide a default value that would allow network traffice from any external device. Instead, you can provide the address of your user system (for example, by using [https://ipv4.icanhazip.com/](https://ipv4.icanhazip.com/)) or a range of multiple IP addresses.





 Does the Spectrum Symphony offering support multiple key pairs to establish SSH to all of the nodes? 

Yes, the Spectrum Symphony offering supports multiple single key pairs that can be provided for access to all of the nodes that are part of the cluster. In addition, Spectrum Symphony has a feature where each node of the cluster can be accessed through passwordless SSH.





 What storage types are available through this offering? 

In the Spectrum Symphony offering, you can use Spectrum Scale scratch storage or persistent storage. A scratch storage configuration uses virtual server instances with instance storage. A persistent storage configuration uses bare metal servers with locally attached NVMe storage.





 Which Linux operating system is supported for worker nodes?
Backup and restore with the web console 

With the web console, you can run on demand backups, set, modify, and delete backup schedules, and restore backups.

You can run differential and/or cumulative backups only after you run a full backup first. You can restore differential and/or cumulative backups only after you restore a full backup first.

Schedule times of full, differential and cumulative backups of the same set of databases cannot overlap. You must schedule these backups in 30 minutes intervals.

If you want to back up and restore a specific database, you must ensure that:



* From the Users and groups view, you select Users > Object privileges > Database > GLOBAL > List to show this database.
* From the Users and groups view, you select Users > Admin privileges > Database > GLOBAL > Backup to back up this database.
* From the Users and groups view, you select Users > Admin privileges > Database > GLOBAL > Restore and Users > Admin privileges > Database > the name of the database you want to restore > Restore to restore this database.





 Running on demand backups 



1. Go to Backup and restore > Backup.
2. Make sure that a destination bucket for the backups is created.

For more information, see [Creating destination buckets](https://cloud.ibm.com/docs/netezza?topic=netezza-bnr-overviewcreate-destinations).
3. Select a destination bucket for your backup.
4. Select the backup type.

You can choose a full, differential or cumulative backup type.
5. Specify whether you want to back up all databases or select databases.
6. Click Back up now.







 Scheduling backups 



1. Go to Backup and restore > Backup.
2. Make sure that a destination bucket for the backups is created.

For more information, see [Creating destination buckets](https://cloud.ibm.com/docs/netezza?topic=netezza-bnr-overviewcreate-destinations).
3. Select a destination bucket for your backup.
4. Select the backup type.
5.
After you have all of the information, the following create request is a POST to the https://resource-controller.cloud.ibm.com/v2/resource_instances endpoint:

curl -X POST 
https://resource-controller.cloud.ibm.com/v2/resource_instances 
-H 'Authorization: Bearer <>' 
-H 'Content-Type: application/json' 
-d '{
"name": "my-instance",
"target": "bluemix-us-south",
"resource_group": "5g9f447903254bb58972a2f3f5a4c711",
"resource_plan_id": "dash
}'

The parameters name, target, resource_group, and resource_plan_id are all required. If needed, you can send [additional parameters](https://cloud.ibm.com/docs/Db2onCloud?topic=Db2onCloud-provisioningprov_add_parms) in the request body.

More information on the Resource Controller API is found in its [API Reference](https://cloud.ibm.com/docs/Db2onCloud?topic=Db2onCloud-provisioning).





 List of additional parameters 



* backup_id - A CRN of a backup resource to restore from. The backup must have been created by a database deployment with the same service ID. The backup is loaded after provisioning and the new deployment starts up that uses that data. A backup CRN is in the format crn:v1:<...>:backup:<uuid>. If omitted, the database is provisioned empty.





* backup_location - The location of the deployment's backups.
* disk_encryption_key_crn - The CRN of a [KMS key](https://cloud.ibm.com/docs/Db2onCloud?topic=Db2onCloud-provisioning), which is then used for disk encryption.
If you do not use your own key, the deployment automatically creates and manages its own disk encryption key using the Key Protect service. If you would like to use HPCS for encryption, you must [provision an HPCS instance](https://cloud.ibm.com/docs/docs/hs-crypto?topic=hs-crypto-provision&interface=ui) and generate or import a key. Currently, HPCS is not EU-Cloud enabled.
6. Backup Encryption Key - If you use Backup Encyrption Key, you can provide your own KMS instance and key in order to encrypt your backups. This is an optional parameter, and if not provided the default KMS instance and key will be used.
7. CPU allocation - Choose dedicated compute resources for your deployment. With dedicated cores, your resource group is given a single-tenant host with a guaranteed minimum reserve of cpu shares. Your deployments are then allocated the number of CPUs you specify. This defaults to the standard plan if not specified in the provisioning request by using the API or CLI.
8. Endpoints - You can configure the types Service Endpoints on your deployment. The default is that connections to your deployment can be made from the public network.
9. High Availability - whether the services should be Highly Available
10. Oracle compatibility - whether the service instance should have Oracle compatibility enabled



After selecting the appropriate settings, click Create to start the provisioning process.





 Using the command line 

The IBM Cloud CLI tool is what you use to communicate with IBM Cloud from your terminal or command line. For more information, see [Download and install IBM Cloud CLI](https://cloud.ibm.com/docs/cli?topic=cli-getting-started).

To create a Cloud Databases deployment, you use the CLI to request a service instance with the service ID of the database (or messaging queue) that you want to provision.

Run the following command template:

ibmcloud resource service-instance-create <service-name> <service-id> <service-plan-id> <region> --service-endpoints <SERVICE_ENDPOINTS_TYPE>
For more information, see [Getting started with monitoring](https://cloud.ibm.com/docs/monitoring?topic=monitoring-getting-started).

For IBM Cloud Hyper Protect Virtual Servers for IBM Cloud® Virtual Private Cloud instances, the memory metrics cannot be collected because IBM Cloud Hyper Protect Virtual Servers for IBM Cloud® Virtual Private Cloud instances are created using Secure Execution images and the memory of a secure execution instance is not accessible.





 Create a VPN gateway 

Create a VPN gateway on the subnet if you want to securely connect your VPC to another private network.

To create a VPN gateway, run the following command:

ibmcloud is vpn-gateway-create my-vpn-gateway $subnet

From the output that's returned, save the ID of the VPN gateway in a variable so you can use it later, for example:

vpn_gateway="0757-7e91085b-dc11-4707-aa4d-66e735e9a2bc"

The status of the VPN gateway is pending when it's first created. Before you can proceed, the VPN gateway needs to move to the available status, which takes a few minutes. To check the status of the VPN gateway, run this command:

ibmcloud is vpn-gateway $vpn_gateway

To create a VPN connection on the VPN gateway to peer address 169.61.161.150 and pre-shared key mykey, run the following command:

ibmcloud is vpn-gateway-connection-create my-vpn-conn $vpn_gateway 169.61.161.150 mykey

The status of the VPN connection is down when it is first created and becomes up after the connection is established. To check the status of VPN connections on a VPN gateway, run this command:

ibmcloud is vpn-gateway-connections $vpn_gateway





 Congratulations! 

You've successfully created and configured your VPC using the IBM Cloud CLI.
When you create a subnet, you must specify an IP range that's within one of the address prefixes of the selected zone.

A subnet cannot be resized after it is created.

ibmcloud is subnet-create my-subnet $vpc us-south-3 --ipv4-cidr-block "10.0.1.0/24"

From the output that's returned, save the ID in a variable so you can use it later, for example:

subnet="0738-658756a4-1106-4914-969a-3b43b338524a"

The status of the subnet is pending when it's first created. Before you can create resources in the subnet, the subnet needs to move to the available status, which takes a few seconds. To check the status of the subnet, run this command:

ibmcloud is subnet $subnet





 Attach a public gateway 

Attach a public gateway to the subnet if you want to allow all attached resources to communicate with the public internet.

To create a public gateway, run the following command:

ibmcloud is public-gateway-create my-gateway $vpc us-south-3

From the output that's returned, save the ID in a variable so you can use it later, for example:

gateway="0738-446c0c63-f0b1-4043-b30d-644f55fde391"

To attach the public gateway to your subnet, run the following command:

ibmcloud is subnet-update $subnet --pgw $gateway

Only one public gateway per zone is allowed in a VPC, but that public gateway can be attached to multiple subnets in the zone. To find the public gateway for a zone, run the ibmcloud is public-gateways command and look for the particular VPC and Zone values.





 Adding an SSH key 

Add your public SSH key to your IBM Cloud account. This key is specified when you create the instance, and you need it later to log in to the instance. You can use one key to provision multiple instances.
For more information, see [Ordering Snapshots](https://cloud.ibm.com/docs/BlockStorage?topic=BlockStorage-orderingsnapshots).

To create a snapshot schedule, use the following command.

 slcli block snapshot-enable --help
Usage: slcli block snapshot-enable [OPTIONS] VOLUME_ID

Enables snapshots for a given volume on the specified schedule

Options:
--schedule-type TEXT Snapshot schedule [INTERVAL|HOURLY|DAILY|WEEKLY]
[required]
--retention-count TEXT Number of snapshots to retain [required]
--minute INTEGER Minute of the day when snapshots should be taken
--hour INTEGER Hour of the day when snapshots should be taken
--day-of-week TEXT Day of the week when snapshots should be taken
-h, --help Show this message and exit.

You can also see the list of your snapshot schedules through the SLCLI with the following command.

 slcli block snapshot-schedule-list --help
Usage: slcli block snapshot-schedule-list [OPTIONS] VOLUME_ID

Options:
-h, --help Show this message and exit.





 Managing a Snapshot schedule with Terraform 

To create a snapshot schedule, use the ibm_storage_block resource and specify information in the snapshot_schedule argument. The following example defines two different schedules. One schedule is for weekly snapshots that are taken on Sundays at 1:20 PM. Twenty snapshots are kept before the oldest one is deleted to make space for a new one. The second schedule is for hourly snapshots.

resource "ibm_storage_block" "test1" {
type = "Endurance"
datacenter = "dal05"
capacity = 20
iops = 0.25
os_format_type = "Linux"

 Optional fields
allowed_virtual_guest_ids = [ 27699397 ]
allowed_ip_addresses = ["10.40.98.193", "10.40.98.200"]
Verify that all details in the New configuration section are correct.
10. Click Confirm OS reload to confirm and initiate the OS reload. Click Cancel to cancel the action.







 What's next? 

After you initiate the OS reload process, the device is taken offline and the OS reload process begins. The time period in which an OS reload takes place varies based on the current and new configuration of the device. Throughout the configuration process, the minimum time for the OS reload is displayed on each screen. The timeframe that is displayed is an estimate that is generated by the system. If the reload takes longer than 24 hours, contact [IBM Cloud® Support](https://cloud.ibm.com/docs/virtual-servers?topic=virtual-servers-gettinghelp).

When the device returns online, it functions as specified in the new configuration for the OS reload. All data that was saved to the device is lost, but can be restored if you created a backup of the device before the OS reload. If data wasn't backed up, it cannot be retrieved.

You need to reregister your device with [IBM Cloud Backup](https://cloud.ibm.com/docs/Backup?topic=Backup-getting-started).