vania-janet/multiturn-rag-retrieval-data huggingface dataset:
For more information, see the [IAM guide](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-managing-access-for-cloudant) or the legacy [Authentication document](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-work-with-your-accountauthentication). 
 Authorization IBM Cloudant supports both legacy and IAM access controls. The IBM Cloudant team recommends that you use IAM access controls for authentication whenever possible. If you're using IBM Cloudant legacy authentication, it is recommended that you use [API keys](https://cloud.ibm.com/apidocs/cloudantintroduction) rather than account-level credentials for programmatic access and replication jobs. For more information, see the [IAM guide](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-managing-access-for-cloudant) or the legacy [Authentication document](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-work-with-your-accountauthentication) and the legacy [Authorization document](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-work-with-your-accountauthorization). 
 At-rest encryption All data that is stored in an IBM Cloudant instance is encrypted at rest by using LUKS1 with 256-bit Advanced Encryption Standard (AES-256). By default, IBM Cloudant manages the encryption keys for all environments. If you require bring-your-own-key (BYOK) encryption for encryption-at-rest, you enable it by using your encryption key that is stored in an IBM Cloud Key Protect instance. IBM Cloudant supports the BYOK feature for new IBM Cloudant Dedicated Hardware plan instances that are deployed in all regions.
IBM Cloudant Search and IBM Cloudant Query indexes of type text (both of which are Apache Lucene under the hood) provide you with a way to index any number of fields into the index. Some examples exist where this type of indexing is abused either deliberately, or mostly by mistake. Plan your indexing to comprise only the fields required by your actual queries. Indexes take up space and can be costly to rebuild if the number of indexed fields are large.

We also have the issue of which fields that you store in an IBM Cloudant Search. Stored fields are retrieved in the query without doing include_docs=true so the tradeoff is similar to the [Understand the tradeoffs in emitting data or not into a view](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-indexing-and-queryingtradeoffs-emit-data-or-not-in-view) section. For more information, see IBM Cloudant Search [docs](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-cloudant-search).





 Design document management requires some flair 

As your data set grows, and your number of views goes up, sooner or later you want to ponder how you organize your views across design documents. A single design document can be used to form a so-called view group: a set of views that belong together by some metric that makes sense for your use case. If your views are static, that makes your view query URLs semantically similar for related queries. It’s also more performant at index time because the index loads the document once and generates multiple indexes from it.

Design documents themselves are read and written by using the same read/write endpoints as any other document. With these endpoints, you can create, inspect, modify, and delete design documents from within your application. However, even small changes to design documents can have significant effects on your database. When you update a design document, all views in it become unavailable until indexing is complete. This lag can be problematic in production.
Document versioning and MVCC 

[Multi-version concurrency control (MVCC)](https://en.wikipedia.org/wiki/Multiversion_concurrency_control) is how IBM® Cloudant® for IBM Cloud® databases ensures that all of the nodes in a database's cluster contain only the [newest version](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-documents) of a document.

Since IBM Cloudant databases are [eventually consistent](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-cap-theorem), it is necessary to prevent inconsistencies from arising between nodes as a result of synchronizing between outdated documents.

Multi-Version Concurrency Control (MVCC) enables concurrent read and write access to an IBM Cloudant database. MVCC is a form of [optimistic concurrency](https://en.wikipedia.org/wiki/Optimistic_concurrency_control). It makes both read and write operations on IBM Cloudant databases faster because the database locks on either read or write operations isn't necessary. MVCC also enables synchronization between IBM Cloudant database nodes.



 Revisions 

Every document in an IBM Cloudant database has a _rev field that indicates its revision number.

A revision number is added to your documents by the server when you insert or modify them. The number is included in the server response when you make changes or read a document. The _rev value is constructed by using a combination of a simple counter and a hash of the document.

The two main uses of the revision number are to help in the following cases:



1. Determine what documents must be replicated between servers.
2. Confirm that a client is trying to modify the latest version of a document.



You must specify the previous _rev when you [update a document](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-documentsupdate) or else your request fails and returns a [409 error](https://cloud.ibm.com/apidocs/cloudantlist-of-http-codes).
/whisk.system/cos shared
/whisk.system/messaging shared
/whisk.system/alarms shared
2. Get a list of entities in a package.

ibmcloud fn package get --summary <package_name>

Example

ibmcloud fn package get --summary /whisk.system/cloudant

Example output

package /whisk.system/cloudant: Cloudant database service
(parameters: apihost, bluemixServiceName, dbname, host, iamApiKey, iamUrl, overwrite, password, username)
action /whisk.system/cloudant/delete-attachment: Delete document attachment from database
(parameters: attachmentname, dbname, docid, docrev, params)
action /whisk.system/cloudant/update-attachment: Update document attachment in database
(parameters: attachment, attachmentname, contenttype, dbname, docid, docrev, params)
action /whisk.system/cloudant/read-attachment: Read document attachment from database
(parameters: attachmentname, dbname, docid, params)
action /whisk.system/cloudant/create-attachment: Create document attachment in database
(parameters: attachment, attachmentname, contenttype, dbname, docid, docrev, params)
action /whisk.system/cloudant/read-changes-feed: Read Cloudant database changes feed (non-continuous)
(parameters: dbname, params)
...
...
feed /whisk.system/cloudant/changes: Database change feed
(parameters: dbname, filter, iamApiKey, iamUrl, query_params)

This output shows that the IBM Cloudant package includes several actions and a feed.

The IBM Cloudant package also defines the parameters username, password, host, and port. These parameters must be specified for the actions and feeds to be meaningful.
3. Get a description of an action or feed to see the parameters that are required.

Example
You can create a package of local code or a clone of any GitHub repository.

Before you begin



* [Install the Cloud Functions plug-in for the IBM Cloud CLI](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-cli_install).
* Create a manifest.yaml or manifest.yml file for your app and store it in the root directory. The manifest.yaml file specifies the overall structure of the package, including any metadata that must be included with the ibmcloud fn deploy command. To learn more about manifest.yaml files, see the [wskdeploy documentation](https://github.com/apache/openwhisk-wskdeploy/blob/master/docs/programming_guide.md).



To add a package:



1. Clone the package repo.

git clone https://github.com/ORG_NAME/REPOSITORY_NAME
2. Navigate to the directory that contains the manifest.yaml file.

cd <filepath>/<package_name>
3. Deploy the package.

ibmcloud fn deploy -m manifest.yaml

Some packages require certain environment variables to enable the package to function properly. If so, include the environment variables with the deploy command. For example, you can choose a name for the package and specify it with the PACKAGE_NAME variable.

PACKAGE_NAME=CUSTOM_PACKAGE_NAME VARIABLE_NAME=VARIABLE_VALUE ibmcloud fn deploy -m manifest.yaml





 IBM Cloud Object Storage package example 

To see an example of how to install a package, check out the [IBM Cloud Object Storage package](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_obstorage). IBM Cloud® Object Storage is a service that allows users to store all types of files, such as images, videos, music, and text. To interact with the files, a Cloud-based datastore of key-value pairs is stored in a bucket.
* From the [CLI](https://cloud.ibm.com/docs/vpc?topic=vpc-create-ifvimage-from-volume-vpc-cli), you can create a custom image at the same time as you create a new instance, or you can create an image from an existing instance. Issue the ibmcloud is image-create command and specify the boot volume's ID.







 Image from volume encryption 

When you create an image from a volume, you have the following encryption choices.



* If you want to create an instance and boot volume with default IBM-managed encryption, then the image from that boot volume inherits the IBM-managed encryption. You can keep this encryption, or you can specify your own root key when you create the image. This key is called a [customer root key](https://cloud.ibm.com/docs/vpc?topic=vpc-vpc-encryption-aboutvpc-customer-managed-encryption), or CRK.
* If you want to create an image from an encrypted boot volume on an existing instance, and that boot volume uses a CRK, then the image from that volume inherits that root key. You can keep this encryption, or you can specify another CRK to be used for the image.







 Image from volume lifecycle 

You can use the UI, CLI, API, and Terraform to manage the lifecycle of your custom images with three statuses. You can move the image back and forth through all the statuses and set dates to automatically change an image status. All status changes are tracked as an Activity Tracker event. You can filter your list of images based on status to aid in clean-up or tracking of your images. For more information about making status changes, see [Managing custom images](https://cloud.ibm.com/docs/vpc?topic=vpc-managing-custom-images&interface=ui).



Table 1. Image lifecycle status

 Image status Description 

 available You can use the image to create an instance. 
 deprecated You can use the image to create an instance. Using the deprecated status can discourage use of the image before changing the status to obsolete.
* Create an instance of [IBM Cloud Object Storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-getting-started-cloud-object-storage) and add your image to a bucket.









 Importing a version to your private catalog 

Complete the following steps to import a version of your software to your private catalog. Your private catalog was created for you as part of [Getting set up to sell software](https://cloud.ibm.com/docs/sell?topic=sell-sw-getting-started).



1. In the IBM Cloud console, click the Navigation Menu icon ![Navigation Menu icon](https://cloud.ibm.com/docs-content/v1/content/ff5860bfede49db3197c4bbba7f7123769bdc068/icons/icon_hamburger.svg) > Partner Center > Sell > My Products.
2. Select the product that you're onboarding.
3. From the Software tab, click Import a version.
4. Select your deployment method.
5. Select whether you are adding your product from a private or public repository. For Operators, select your source repository and then choose private or public repository.
6. Enter your source URL.

You can review the following list of supported formats per software type:



* Helm chart: https://raw.githubusercontent.com/IBM/charts/master/repo/ibm-helm
* Node-RED Operator: https://github.com/IBM-Cloud/operator-bundle-sample/archive/refs/tags/v0.0.3.tar.gz
* Operator bundle from a Red Hat OpenShift on IBM Cloud registry: For an example, select the Akka Cluster Operator from the list of available Operators in the Certified repository.
* OVA image: https://github.com/gcatalog/OVA-sample/blob/main/ova-sample.yaml
ibmcloud is image-export-jobs 

List all export jobs of an image.

ibmcloud is image-export-jobs IMAGE [--output JSON] [-q, --quiet]



 Command examples 



* ibmcloud is image-export-jobs 72251a2e-d6c5-42b4-97b0-b5f8e8d1f479
* ibmcloud is image-export-jobs my-image







 Command options 



* IMAGE: ID or name of the image.
* --output: Specify output format, only JSON is supported. One of: JSON.
* -q, --quiet: Suppress verbose output.



--------------------







 ibmcloud is image-export-job-create 

Create an image export job.

ibmcloud is image-export-job-create IMAGE --bucket BUCKET [--name NEW_NAME] [--format qcow2 | vhd] [--output JSON] [-q, --quiet]



 Command examples 



* ibmcloud is image-export-job-create 72251a2e-d6c5-42b4-97b0-b5f8e8d1f479 --name my-export-job --format qcow2 --bucket my-bucket
* ibmcloud is image-export-job-create my-image --name my-export-job --format qcow2 --bucket my-bucket







 Command options 



* IMAGE: ID or name of the image.
* --name: New name of the image export job.
* --format: The format to use for the exported image. If the image is encrypted, only qcow2 is supported. One of: qcow2, vhd. (default: qcow2).
* --bucket: The Cloud Object Storage bucket to export the image to. The bucket must exist and an IAM service authorization must grant Image Service for VPC of VPC Infrastructure Services writer access to the bucket.
Generally, the revision number increases by one each time, but more complex scenarios are possible (we cover these scenarios later). Older document bodies are discarded or compacted (don't rely on being able to get them back). All IBM Cloudant operations that change a document need the document's _id and its _rev (this scenario is unlike most databases).

That's the end of this part. The next part is called Authentication.







 Authentication video 

Learn how Legacy authentication and IAM authentication work. You can also learn how IBM Cloudant generates credentials, and how the three official IBM Cloudant libraries handle authentication.



* Authentication video script

Welcome to the Introduction to IBM Cloudant course, an 17-part video series that gives you an overview of the IBM Cloudant database-as-a-service.

This video is part 5 - Authentication.

We said earlier that IBM Cloudant is a web-based service on the public internet. How can we be sure that our data is safe and that only our code can access it? This scenario is where authentication comes in.

IBM Cloudant supports two types of authentication.

Legacy authentication is where a username or api-key and password are supplied with each request that uses HTTP Basic Authentication or exchanged for a cookie that uses a one-off session API call. A session cookie is cycled regularly, so your client code needs to capture the refreshed cookie and store it for subsequent requests. IAM authentication is the access management system that underpins all of the IBM Cloud services. To authenticate with IAM, you need an IAM API key and the hostname of the IBM Cloudant service. The API key is exchanged for a bearer token by using the IAM API and the bearer token is passed to IBM Cloudant with each request. The bearer token lasts only an hour, so must be renewed with the IAM service periodically. When an IBM Cloudant service is provisioned, you can generate IAM only credentials, or both IAM and Legacy credentials - you decide.

How are credentials generated?

In the IBM Cloud Dashboard under your IBM Cloudant service, in the Service Credentials tab, click New Credential.
Delete one or more images.

ibmcloud is image-delete (IMAGE1 IMAGE2 ...) [--output JSON] [-f, --force] [-q, --quiet]



 Command options 



* IMAGE1: ID or name of the image.
* IMAGE2: ID or name of the image.
* --output: Specify output format, only JSON is supported. One of: JSON.
* --force, -f: Force the operation without confirmation.
* -q, --quiet: Suppress verbose output.



--------------------







 ibmcloud is image-export-job 

View details of an image export job.

ibmcloud is image-export-job IMAGE JOB [--output JSON] [-q, --quiet]



 Command examples 



* ibmcloud is image-export-job 72251a2e-d6c5-42b4-97b0-b5f8e8d1f479 6451a2e-d6c5-42b4-97b0-b5f8e8d1f479
* ibmcloud is image-export-job my-image my-export-job







 Command options 



* IMAGE: ID or name of the image.
* JOB: ID or name of the image export job.
* --output: Specify output format, only JSON is supported. One of: JSON.
* -q, --quiet: Suppress verbose output.



--------------------







 ibmcloud is image-export-jobs 

List all export jobs of an image.

ibmcloud is image-export-jobs IMAGE [--output JSON] [-q, --quiet]



 Command examples 



* ibmcloud is image-export-jobs 72251a2e-d6c5-42b4-97b0-b5f8e8d1f479
* ibmcloud is image-export-jobs my-image
When you've completed these steps so that Image Service for VPC can access images in IBM Cloud Object Storage or export images to IBM Cloud Object Storage, continue with one of the following topics:



* [Creating a Linux custom image](https://cloud.ibm.com/docs/vpc?topic=vpc-create-linux-custom-image)
* [Creating a Windows custom image](https://cloud.ibm.com/docs/vpc?topic=vpc-create-windows-custom-image)
* [Importing a custom image](https://cloud.ibm.com/docs/vpc?topic=vpc-importing-custom-images-vpc)
* [Exporting a custom image](https://cloud.ibm.com/docs/vpc?topic=vpc-managing-custom-imagescustom-image-export-to-cos)
Working with files 

Your IBM® Cloud Shell workspace includes a home directory where you can temporarily work with files in your Cloud Shell sessions. You can upload or download files one at a time to your workspace through the UI, or use command-line tools to work with many files at once.



 Before you begin 

Enable File upload and download in the IBM Cloud console. For more information about how to enable file upload and download, see [Enabling or disabling Cloud Shell features for an account](https://cloud.ibm.com/docs/account?topic=account-shell-settingsshell-features-enable).





 Temporary workspace storage 

Your Cloud Shell workspace includes 500 MB of temporary secure storage, which you can access through your personal home directory, /home/<user-name>. Only you can access your workspace storage, and your storage is separate for each account that you access or region where you work in IBM Cloud Shell.

Your workspace storage is shared across all of your sessions, so you can work with the same files in different session tabs. The storage persists only while your workspace is active. If you're idle in Cloud Shell for over an hour, your files and data are removed. Similarly, if you reach the usage quota or you restart Cloud Shell, Cloud Shell closes and removes your data. If you make any changes to files that you want to keep, be sure to download the files at the end of your session.

There is a known issue where your connection to Cloud Shell is lost if your reach the temporary storage limit. If this happens, the only way to fix the connection is to restart Cloud Shell, which removes all of your files. While you work in IBM Cloud Shell, avoid uploading large files and remove any unused files by using standard Linux™ commands, such as rm.



 Backing up your data 

Your workspace storage is temporary, and it's not intended to be used as the primary storage location for your files. Don't store business-critical or sensitive data in your workspace, and create a backup of your files outside of IBM Cloud Shell. You're responsible for your data, and your backup can help you recover in case an outage or data loss occurs for any reason.
What is replication? 

Data can be copied from one database to another in the same IBM® Cloudant® for IBM Cloud® account, across accounts and across data centers.

Data can even be replicated to and from an IBM Cloudant account and a mobile device by using [IBM Cloudant Sync](https://www.ibm.com/cloud/learn/offline-first) or [PouchDB](https://pouchdb.com/). Replication can run in one direction or in both directions, as a "single shot" or continuous operation, and can be finely tuned by using parameters.

IBM Cloudant’s replication protocol is compatible with a range of other databases and libraries, making it a great fit for Internet of Things (IoT) and mobile applications.

IBM Cloudant is a distributed JSON data store with an HTTP API. IBM Cloudant can be run as a service on multiple clouds, or in your server rack. Documents are stored in databases and can grow to any size as IBM Cloudant shards its data across many nodes. Replication is the copying of data from a source database to a target database. The source and target databases do not need to be on the same IBM Cloudant account, or even in the same data center.

Zoom

![The graphic shows a picture of database a and b. Database b has one document. After replication, the documents from database a are show in database b.](https://cloud.ibm.com/docs-content/v1/content/522c6f62358b063f921283400a601c3f9bc66f08/Cloudant/images/replication_guide_1.svg)

Figure 1. Replication in pictures

Replication is complete when the most recent version of each document in the source transfers to the destination database. Transfers include new documents, updates to existing documents, and deletions. Only the most recent version of a document remains after replication; older versions are omitted.

The source database remains unaltered by replication, apart from checkpoint data that is written to it to allow partial replications to resume from the last known position. Any pre-existing data in the destination database remains.
* API for VM capture - [Capture a PVM instance and create a deployable image (version 2)](https://cloud.ibm.com/apidocs/power-cloudpcloud-v2-pvminstances-capture-post).
* API for image export - [Add image export job to the jobs queue (version 2)](https://cloud.ibm.com/apidocs/power-cloudpcloud-v2-images-export-post).



CLI enhancements include changes to the existing [ibmcloud pi instance-capture](https://cloud.ibm.com/docs/power-iaas-cli-plugin?topic=power-iaas-cli-plugin-power-iaas-cli-referenceibmcloud-pi-instance-capture) command and a new [ibmcloud pi jobs](https://cloud.ibm.com/docs/power-iaas-cli-plugin?topic=power-iaas-cli-plugin-power-iaas-cli-referenceibmcloud-pi-jobs) command.

The new VM capture, image export, and image import features are restricted to one operation at a time per Power Systems Virtual Server workspace. If one of these operations is submitted successfully, then another new operation (VM Capture, Image Export, and Image Import) cannot be submitted until the previous operation is complete.

You can use the Power Systems Virtual Server legacy REST APIs for VM Capture, Image Export, and Image Import until 1 October 2022. You must plan to transition to the new IBM® Power Systems™ Virtual Server REST APIs or CLI commands before the earlier APIs are withdrawn from service.

You are charged different rates based on whether you export to the image catalog or COS.

Before you capture an IBM i VM, ensure that any buffer I/O memory is flushed (written) to the disk by running the following command:

CHGASPACT OPTION(FRCWRT)



 Using the Power Systems Virtual Server user interface to capture and export a VM 



1. Click the Capture and export icon in your Virtual server instances view. The icon is to the right of the trash icon.