vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Responsibilities for Cloud Databases 

Learn about the management responsibilities and terms and conditions that you have when you use IBM Cloud® Cloud Databases. For a high-level view of the service types in IBM Cloud and the breakdown of responsibilities between the client and IBM for each type, see [Shared responsibilities for IBM Cloud offerings](https://cloud.ibm.com/docs/overview?topic=overview-shared-responsibilities).

This document covers all the IBM Cloud Databases that include Databases for DataStax, Databases for Elasticsearch, Databases for EnterpriseDB, Databases for etcd, Databases for MongoDB, Databases for PostgreSQL, Databases for Redis, and Messages for RabbitMQ.

Review the following sections for the specific responsibilities for you and for IBM when you use Cloud Databases. For the overall terms of use, see [IBM Cloud Terms and Notices](https://cloud.ibm.com/docs/overview/terms-of-use?topic=overview-terms).



 Incident and operations management 



Table 1. Responsibilities for incident and operations

 Task IBM Responsibilities Your Responsibilities 

 Monitoring Cloud Databases is responsible for hosting monitoring and health services. The Client is responsible for integrating with the [IBM Cloud® Monitoring](https://cloud.ibm.com/docs/monitoring?topic=monitoring-platform_metrics_enabling), [IBM Cloud® Activity Tracker](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-activity-tracker), or [IBM Cloud® Log Analysis](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-logging). 
 High Availability Cloud Databases is responsible for deploying databases across availability zones in a Multi-Zone Region (MZR), or across hosts in a single-campus multizone region, and storing backups in cross-region Cloud Object Store instances. Cloud Databases provides replication, fail-over features, and infrastructure maintenance or updates. High availability varies based on each database type, refer to database-specific documentation for details. The Client is responsible for designing application logic to retry connections caused by temporary connection failures (during regular database maintenance and updates). 
 Database performance Cloud Databases is responsible for hosting and maintaining database infrastructure.
Performance 

IBM Cloud® Databases for PostgreSQL deployments can be both manually [scaled to your usage](https://cloud.ibm.com/docs/databases-for-postgresql?topic=databases-for-postgresql-resources-scaling), or configured to [autoscale](https://cloud.ibm.com/docs/databases-for-postgresql?topic=databases-for-postgresql-autoscaling) under certain resource conditions. There are a few factors to consider when you are tuning the performance of your deployment.



 Monitoring your deployment 

Databases for PostgreSQL deployments offer an integration with the [IBM Cloud® Monitoring service](https://cloud.ibm.com/docs/databases-for-postgresql?topic=databases-for-postgresql-monitoring) for basic monitoring of resource usage on your deployment. Many of the available metrics, like disk usage and IOPS, are presented to help you configure [autoscaling](https://cloud.ibm.com/docs/databases-for-postgresql?topic=databases-for-postgresql-autoscaling) on your deployment. Observing trends in your usage and configuring the autoscaling to respond to them can help alleviate performance problems before your databases become unstable due to resource exhaustion.





 Disk IOPS 

The number of input/output operations per second (IOPS) is limited by the type of storage volume. Storage volumes for Databases for PostgreSQL deployments are provisioned on [Block Storage Endurance Volumes in the 10 IOPS per GB tier](https://cloud.ibm.com/docs/BlockStorage?topic=BlockStorage-orderingBlockStorage&interface=ui). If your operational load saturates or exceeds the IOPS limit, database requests and operations are delayed until the disk can catch up. Extended periods of heavy-load can cause your deployment to be unable to process queries and become effectively unavailable. If you experience delayed responses and failing operations, you might be exceeding the disk's IOPS limit.
The cityname is passed as value to a prepared SQL statement using a parameter marker for security reasons. The rows are fetched from the database and passed to an HTML template for rendering.





 Step 8: Cleanup 

To clean up resources used by the tutorial, follow these steps:



1. Visit the [IBM Cloud® Resource List](https://cloud.ibm.com/resources).
2. In the Code Engine section locate the project sqldatabase. Click on the three dots and select Delete to delete the project and its app.
3. Locate the database sqldatabase under Databases. Again, click on the three dots and select Delete to delete the database.



Depending on the resource it might not be deleted immediately, but retained (by default for 7 days). You can reclaim the resource by deleting it permanently or restore it within the retention period. See this document on how to [use resource reclamation](https://cloud.ibm.com/docs/account?topic=account-resource-reclamation).





 Expand the tutorial 

Want to extend this app? Here are some ideas:



1. Offer a wildcard search on the alternate names.
2. Search for cities of a specific country and within a certain population values only.
3. Change the page layout by replacing the CSS styles and extending the templates.
4. Allow form-based creation of new city information or allow updates to existing data, for example population.







 Related Content 



* Documentation: [API Description for the ibm_db Python driver](https://github.com/ibmdb/python-ibmdb/wiki/APIs)
* [Db2 Data Management Console](https://www.ibm.com/products/db2-data-management-console)
* [Db2 on Cloud](https://cloud.ibm.com/catalog/services/db2)
Activity Tracker Integration 

IBM Cloud Databases deployments are integrated with Activity Tracker events in [IBM Cloud® Activity Tracker](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-getting-started), so you can view service-level events.

This document covers the integration of Activity Tracker with Cloud Databases, which includes Databases for DataStax, Databases for Elasticsearch, Databases for EnterpriseDB, Databases for etcd, Databases for MongoDB, Databases for PostgreSQL, Databases for Redis, IBM Cloud® Databases for MySQL, and Messages for RabbitMQ.

Currently, Activity Tracker integration is available for Cloud Databases deployments according to the following table.



Table 1. Activity Tracker regions

 Deployment Region Activity Tracker Region 

 us-south us-south 
 jp-tok jp-tok 
 jp-osa jp-tok 
 eu-gb eu-gb 
 che01 che01 
 eu-de eu-de 
 au-syd au-syd 
 us-east us-east 
 ca-tor ca-tor 
 par01 eu-de 



Events from your deployments appear in an Activity Tracker instance in the same region, except in the case of jp-osa. Events for deployments in jp-osa are forwarded to jp-tok. If you have deployments in multiple regions, you must set up the Activity Tracker in multiple regions.



 Activity Tracker 

When you provision the service, events are automatically forwarded from all your Cloud Databases deployments in the same region.

The service can be provisioned from [its catalog page](https://cloud.ibm.com/catalog/ibm-cloud-activity-tracker) or from an existing [Observability Dashboard](https://cloud.ibm.com/observe/activitytracker).

The Activity Tracker service has a lite plan that is free to use, but it only offers streaming events. To take advantage of the tagging, export, retention, and other features, you need to use one of the [paid plans](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-service_plan).



 Using the Activity Tracker
Microsoft SQL Server database migration 

Database migration is defined as moving a database to and from the database server. There are various ways to perform the migration of your Microsoft® SQL Server database between servers or instances whether you need to move a single database, multiple databases, logins, or stored procedures and views. You can use any of the following methods or combinations of methods to migrate Microsoft® SQL Server databases.



 Why migrate? 



1. Moving to an entirely new server.
2. Moving to a different instance of SQL.
3. Restoring databases from a backup.
4. Creating a development server or going live to a production server.
5. Versioning - A specific database or its version has features that offer more benefits than their existing database or its version.







 Migration overview diagram 

Zoom

![Migration Overview Diagram](https://cloud.ibm.com/docs-content/v1/content/8aa0853d34cf2349f1d2db8e5c61e6422a580169/cloud-infrastructure/images/migration-overview-diagram.svg)

Figure 1. Migration overview diagram





 Use cases 



1. On-premises to IBM Cloud VPC.
2. Other cloud service providers to IBM Cloud VPC.
3. IBM Cloud classic bare metal or virtual server instance to IBM Cloud VPC.







 Prerequisites 

Before you begin your Microsoft® SQL Server database migration, review and complete the following prerequisites:



1. Conduct an inventory of the databases that needs to be migrated.
2. Assess the databases for potential migration issues or blockers, and then resolve any items that you might have uncovered.
3. Ensure that enough disk space is available for the target server of the new database.
4. Ensure that the user has appropriate rights or permissions to perform migration activity.
5. SQL Server Management Studio needs to be installed on each server for few methods.
6. Define a plan for the migration.
7. Back up all the data.







 Migration considerations 



* Migrating a server during nonpeak times.
"source" : "https://$USERNAME1:$PASSWORD1@example.com/db",
"target" : "https://$USERNAME2:$PASSWORD2@$ACCOUNT2.cloudant.com/db",
}

See the following example error response if one of the requested databases for a replication does not exist.

{
"error" : "db_not_found",
"reason" : "could not open https://example.com/db/"
}





 Continuous replication 

By default, the synchronization of a database during replication happens one time when the replicate request is made. To ensure that replication from the source database to the target database takes place continually, set the continuous field of the JSON object within the request to true.

With continuous replication, changes in the source database are replicated to the target database forever until you specifically cancel the replication.

Changes are replicated between the two databases while a network connection is available between the two instances.

When in operation, the replication process does not stop when it finishes processing all current updates. Instead, the replication process continues to wait for further updates to the source database, and applies them to the target.

Continuous replication forces checks to be made continuously on the source database. These checks result in an increasing number of database accesses, even if the source database content did not change. Database accesses are counted as part of the work that is done within a multi-tenant database configuration.





 The _replicator database 

The _replicator database is a special database within your account, where you can PUT or POST replication documents to specify the replications you want.

Before you start a replication, you must create the _replicator database. To create a database, send a PUT request to:

https://$ACCOUNT.cloudant.com/_replicator

For more information, see [Databases](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-databases).

To cancel a replication, you DELETE the replication document.
Performance 

IBM Cloud® Databases for EnterpriseDB deployments can be both manually [scaled to your usage](https://cloud.ibm.com/docs/databases-for-enterprisedb?topic=databases-for-enterprisedb-resources-scaling), or configured to [autoscale](https://cloud.ibm.com/docs/databases-for-enterprisedb?topic=databases-for-enterprisedb-autoscaling) under certain resource conditions. There are a few factors to consider if you are tuning the performance of your deployment.



 Monitoring your deployment 

Databases for EnterpriseDB deployments offer an integration with the [IBM Cloud® Monitoring service](https://cloud.ibm.com/docs/databases-for-enterprisedb?topic=databases-for-enterprisedb-monitoring) for basic monitoring of resource usage on your deployment. Many of the available metrics, like disk usage and IOPS, are presented to help you configure [autoscaling](https://cloud.ibm.com/docs/databases-for-enterprisedb?topic=databases-for-enterprisedb-autoscaling) on your deployment. Observing trends in your usage and configuring the autoscaling to respond to them can help alleviate performance problems before your databases become unstable due to resource exhaustion.





 Disk IOPS 

The number of Input-Output Operations Per Second (IOPS) is limited by the type of storage volume. Storage volumes for Databases for EnterpriseDB deployments are provisioned on [Block Storage Endurance Volumes in the 10 IOPS per GB tier](https://cloud.ibm.com/docs/BlockStorage?topic=BlockStorage-orderingthroughConsoleorderingthroughConsoleEndurance). If your operational load saturates or exceeds the IOPS limit, database requests and operations are delayed until the disk can catch up. Extended periods of heavy-load can cause your deployment to be unable to process queries and become effectively unavailable. If you experience delayed responses and failing operations, you might be exceeding the disk's IOPS limit. You can increase the number IOPS available to your deployment by increasing disk space.





 Memory Usage
Understanding high availability and disaster recovery for Cloud Databases 

This document covers all the IBM Cloud® Databases, which include Databases for DataStax, Databases for Elasticsearch, Databases for EnterpriseDB, Databases for etcd, Databases for MongoDB, Databases for PostgreSQL, Databases for Redis, Messages for RabbitMQ, and Databases for MySQL.

IBM Cloud® Databases instances are deployed in either a multi-zone region (MZR) (for example, Dallas, Frankfurt, London, Sydney, Tokyo, and Washington), or a single-campus multizone region (for example, Chennai). Each instance is deployed in a highly available configuration; that is, data is replicated by each database onto one or more servers, making the data highly available during normal operations.



* In MZRs, database members are distributed across different data centers, or zones.
* In single-campus multizone regions, database members are distributed across different hosts.



If a single-campus multizone region failure in an MZR or a hardware failure in any region occurs, your data is still accessible as it is replicated onto other fully functioning database servers. Such issues are addressed by IBM Cloud® Specialists in place.

You can consult your Cloud Databases documentation for more details on how your specific database replicates data among each of its members.

In addition to the high-availability configuration, for deployments in IBM Cloud Multi-Zone Regions, your data is snapshotted and backed up daily by the IBM Cloud® Databases platform and stored in [cross-region Cloud Object Storage buckets](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-endpointsendpoints-geo). For most IBM Cloud single-campus multizone regions, your data is backed up locally in [Single-campus multizone region Cloud Object Storage buckets](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-endpointsendpoints-zonee).

If a complete region failure occurs, the database servers in the region might not be accessible, but the backup data remains available. You can initiate a restore from these backups into an available region from the service management console. Consult your Cloud Databases backups page for more details.
Performance 

IBM Cloud® Databases for Elasticsearch deployments can be [scaled to your usage](https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-resources-scaling), configured to [autoscale](https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-autoscaling) under certain resource conditions, or [horizontally scaled](https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-horizontal-scaling) with more Elasticsearch nodes. If you are tuning the performance of your deployment, consider a few factors.



 Monitoring your deployment 

Databases for Elasticsearch deployments offer an integration with the [IBM Cloud® Monitoring service](https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-monitoring) for basic monitoring of resource usage on your deployment. Many of the available metrics, like disk usage and IOPS, are presented to help you configure [autoscaling](https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-autoscaling) on your deployment. Observing trends in your usage and configuring the autoscaling to respond to them can help alleviate performance problems before your databases become unstable due to resource exhaustion.





 Elasticsearch sharding 

When you add an index to Elasticsearch, it splits the data into shards and spreads those shards across the nodes in the cluster. The sharded configuration allows for Elasticsearch to run concurrent operations on your data across all the nodes. To gain extra concurrency and performance, [add nodes to Elasticsearch cluster](https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-horizontal-scaling). When you add nodes, your shards are automatically rebalanced across the cluster to spread resource usage across all the nodes and increasing performance.





 Memory Management
* UI
* CLI
* API







 x86 instance profiles 

When you provision IBM Cloud® Virtual Servers for Virtual Private Cloud, you can select from six families of profiles: Balanced, Compute, Memory, Very High Memory, Ultra High Memory, Storage Optimized, and GPU.

A profile is a combination of instance attributes, such as the number of vCPUs, amount of RAM, network bandwidth, and default bandwidth allocation. The attributes define the size and capabilities of the virtual server instance that is provisioned. In the IBM Cloud console, you can select the most recently used profile or click View All Profiles to choose the profile that best fits your needs.

For more information about profiles for IBM Z (s390x processor architecture), see [s390x instance profiles](https://cloud.ibm.com/docs/vpc?topic=vpc-vs-profiles).

The following profile families are available:



Table 2. Virtual server family selections

 Families Description 

 [Balanced](https://cloud.ibm.com/docs/vpc?topic=vpc-profiles&interface=uibalanced) Balanced profiles offer a core to RAM ratio 1 vCPU to 4 GiB of RAM ratio and are best for midsize databases and common cloud applications with moderate traffic. 
 [Compute](https://cloud.ibm.com/docs/vpc?topic=vpc-profiles&interface=uicompute) Compute profiles offer a core to RAM ratio 1 vCPU to 2 GiB of RAM ratio and are best for moderate to high web traffic workloads. Compute profiles are best for workloads with intensive CPU demands, such as high web traffic workloads, production batch processing, and front-end web servers. 
 [Memory](https://cloud.ibm.com/docs/vpc?topic=vpc-profiles&interface=uimemory) Memory profiles offer a core to RAM ratio 1 vCPU to 8 GiB of RAM ratio and are best for memory caching and real-time analytics workloads. Memory profiles are best for memory intensive workloads, such as large caching workloads, intensive database applications, or in-memory analytics workloads.
Performance 

IBM Cloud® Databases for etcd deployments can be both manually [scaled to your usage](https://cloud.ibm.com/docs/databases-for-etcd?topic=databases-for-etcd-resources-scaling), or configured to [autoscale](https://cloud.ibm.com/docs/databases-for-etcd?topic=databases-for-etcd-autoscaling) under certain resource conditions. Consider a few factors when you are tuning the performance of your deployment.



 Monitoring your deployment 

Databases for etcd deployments offer an integration with the [IBM Cloud® Monitoring service](https://cloud.ibm.com/docs/databases-for-etcd?topic=databases-for-etcd-monitoring) for basic monitoring of resource usage on your deployment. Many of the available metrics, like disk usage and IOPS, are presented to help you configure [autoscaling](https://cloud.ibm.com/docs/databases-for-etcd?topic=databases-for-etcd-autoscaling) on your deployment. Observing trends in your usage and configuring the autoscaling to respond to them can help alleviate performance problems before your databases become unstable due to resource exhaustion.





 Storage Limits and Disk IOPS 

etcd has a [storage limit](https://coreos.com/etcd/docs/latest/dev-guide/limit.html) of 8 GB of data on a single deployment. It is not possible to store more than 8 GB of data in an etcd deployment, and exceeding the limit makes your deployment read-only.

Databases for etcd deployments over-allocate space because disk allocation affects the performance of the disk. Larger disks have higher performance with baseline input/output operations per second (IOPS) of 10 IOPS for each GB. etcd relies on writing data to disk to maintain its consensus algorithm, so exceeding your IOPS deployment's limits can cause cluster instability. Scaling your deployment's disk resources increases the IOPS available to your deployment.





 Memory Usage 

etcd uses memory to cache data and can benefit from increasing the amount of memory to at least the size of the data set.
[The transition between states is , , , and .](https://cloud.ibm.com/docs-content/v1/content/522c6f62358b063f921283400a601c3f9bc66f08/Cloudant/images/fb85704.svg)

Figure 1. Replication Scheduler states

The scheduler introduces two new endpoints:



* [/_scheduler/docs](https://cloud.ibm.com/apidocs/cloudantgetschedulerdocs)
* [/_scheduler/jobs](https://cloud.ibm.com/apidocs/cloudantgetschedulerjobs)



You can manage and determine replication status more quickly and easily by using these endpoints.

See the typical process for using the replication scheduler to manage and monitor replications:



1. Create a [replication document](https://cloud.ibm.com/apidocs/cloudantputreplicationdocument) that describes the needed replication, and store the document in the [replicator database](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-replication-apithe-replicator-database).
2. Monitor the status of the replication by using the /_scheduler/docs endpoint.







 Authentication during replication 

In any production application, security of the source and target databases is essential. In order for replication to continue, authentication is necessary to access the databases. Checkpoints for replication are [enabled by default](https://cloud.ibm.com/apidocs/cloudantpostreplicate), which means that replicating the source database requires write access.

To enable authentication during replication, include a username and password in the database URL. The replication process uses the supplied values for HTTP Basic Authentication.

See the following example of specifying username and password values for accessing source and target databases during replication:

{
"source": "https://$USERNAME:$PASSWORD@example.com/db",