vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Required.

Allowable list items are: management, data. The minimum length is 1 item.





 Example 

Example request

ibmcloud cloudant events-config-update --types management,data





 Example output 

Example Ok response.

{
"ok" : true
}







 ibmcloud cloudant throughput 

View the current consumption of provisioned throughput capacity for an IBM Cloudant instance. The current consumption shows the quantities of reads, writes, and global queries conducted against the instance for a given second.

ibmcloud cloudant throughput



 Example 

Example request

ibmcloud cloudant throughput





 Example default output 

Example CurrentThroughputInformation response.

query	13
read	133
write	42





 Example full output 

Example CurrentThroughputInformation response.

{
"throughput" : {
"query" : 13,
"read" : 133,
"write" : 42
}
}





 Default JMESPath 

A JMESPath query is applied to the output of this command by default. The default JMESPath is:

throughput

If a custom JMESPath query is provided, it will replace the default JMESPath.
Files that include media, such as images, videos, and audio, are called BLOBs (Binary Large Objects). BLOBs can be stored as attachments associated with documents.

More information about JSON can be found in the [JSON Guide](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-how-json-works).





 Distributed systems 

By using IBM Cloudant's API, you can interact with a collaboration of numerous machines, called a cluster. The machines in a cluster must be in the same datacenter, but can be within different "pods" in that datacenter. Using different pods helps improve the High Availability characteristics of IBM Cloudant.

An advantage of clustering is that when you need more computing capacity, you add more machines. This method is often more cost-effective and fault-tolerant than scaling up or enhancing an existing single machine.

For more information about IBM Cloudant and distributed system concepts, see the [CAP Theorem](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-cap-theorem) guide.





 Replication 

[Replication](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-replication-api) is a procedure followed by IBM Cloudant, [CouchDB](https://couchdb.apache.org/), [PouchDB](https://pouchdb.com/), and other distributed databases. Replication synchronizes the state of two databases so that their contents are identical.

You can replicate continuously. Continuous replication means that a target database updates every time the source database changes. Continuous replication can be used for backups of data, aggregating data across many databases, or for sharing data.

However, continuous replication means testing continuously for any source database changes. This testing requires continuous internal calls, which might impact performance or the cost of using the database.

Continuous replication can result in many internal calls. These calls might affect costs for multi-tenant users of IBM Cloudant systems. Continuous replication is disabled by default.





 Using the proper tool for the job
"repldoc-example",
replicationDoc,
)

documentResult, response, err := service.PutReplicationDocument(putReplicationDocumentOptions)
if err != nil {
panic(err)
}

b, _ := json.MarshalIndent(documentResult, "", " ")
fmt.Println(string(b))

The previous Go example requires the following import block:

import (
"encoding/json"
"fmt"
"github.com/IBM/cloudant-go-sdk/cloudantv1"
)

All Go examples require the service object to be initialized. For more information, see the API documentation's [Authentication section](https://cloud.ibm.com/apidocs/cloudant?code=goauthentication-with-external-configuration) for examples.

See the following example of a JSON document that defines a continuous replication:

{
"_id": "weekly_continuous_backup",
"source": "https://$USERNAME:$PASSWORD@$SOURCE_SERVICE_DOMAIN/source",
"target": "https://$USERNAME:$PASSWORD@$TARGET_SERVICE_DOMAIN/target",
"continuous": true
}





 Other replication use cases 

IBM Cloudant’s replication protocol is compatible with other databases and libraries for various real-world applications.



 Apache CouchDB 

[Apache CouchDB](https://couchdb.apache.org/) is an open source database that can communicate with IBM Cloudant, and that requires minimal setup. The following applications are included:



* Backup - Replicate your data from IBM Cloudant to your own CouchDB databases and take nightly snapshots of your data for archiving purposes. Send the data to a backup service such as [Amazon Glacier](https://aws.amazon.com/glacier/) for safe keeping.
* Local-first data collection - Write your data to local Apache CouchDB first, then replicate it to IBM Cloudant for long-term storage, aggregation, and analysis.
21 October 2019 

Multiple restrictions employed for performance gains
: IBM released exciting new capabilities for IBM Cloudant that are available now. IBM Cloudant documented the deprecation of some functionality, and also, new restrictions for other processes. A communication was released that details the exact timeline when these restrictions go into effect. If you use any of the following functions or are concerned about how to make the necessary application changes, reach out to support by email. The following functions are affected by this deprecation:



* Remove the offset field from the response body of an all_docs request. The IBM Cloudant team recommends that you use paging with start_key / end_key and limit.
* The _sorted field has no functional effect because all responses are sorted automatically.
* Duration of operations has a 5-second limit. Transactions lasting more than 5 seconds fail.
* Limitations on keys (10 KB) and values (100 KB) that are emitted from a map function are shown in the following list:



* The sum of all keys that are emitted for a document cannot exceed 100 KB.
* Emitted keys cannot exceed 10 KB.
* Values cannot exceed 100 KB.
* In rare cases, the number of key-value pairs that are emitted for a map function can affect database performance or violate IBM Cloudant rules. For example, the number of key-value pairs might lead to a transaction either exceeding 10 MB, which isn’t allowed, or exceeding 5 MB, which impacts the performance of the database. In this situation, IBM Cloudant returns an error.



* The stable = true option is no longer supported, and the stale = "ok" option is converted to update = false.







 15 October 2019 

The following changes were made in build 8106:

Improvements
: Internal bug fixes.





 1 October 2019 

The following changes were made in build 8103:

X-Cloudant-Action HTTP response header
: New X-Cloudant-Action HTTP response header that returns the IBM Cloud® IAM actions that are associated with a request.

Search requests
: Previously, search requests would return a 400 status code both on a bad request and on internal server errors. Now, internal server errors correctly return a 500 response status code.
For more information, see the following website:



* [IBM Cloudant Logging with LogDNA blog](https://blog.cloudant.com/2019/09/16/Cloudant-Logging-with-LogDNA.html)







 Compress your HTTP traffic 

IBM Cloudant compresses its JSON responses to you if you supply an HTTP header in the request that indicates that your code can handle data in this format:

Request:

> GET /cars/_all_docs?limit=5&include_docs=true HTTP/2
> Host: myhost.cloudant.com
> Accept: /
> Accept-Encoding: deflate, gzip

Response:

< HTTP/2 200
< content-type: application/json
< content-encoding: gzip

Compressed content occupies a fraction of the size of the decompressed equivalent, meaning that it takes a shorter time to transport the data from IBM Cloudant’s servers to your application.

You might also choose to compress HTTP request bodies by using the Content-encoding header. This practice helps reduce data transfer times when you write documents to IBM Cloudant.
This value is optional.

--extension, --ext
: Set CloudEvents extensions to send to the destination. Must be in NAME=VALUE format. This action adds a new CloudEvents extension or overrides an existing CloudEvent attribute. Specify one extension per --extension option; for example, --ext extA=A --ext extB=B. This value is optional.

--extension-rm, --ext-rm
: Remove CloudEvents extensions to send to the destination by specifying the name of the key. Specify one extension per --ext-rm option; for example, --ext-rm extA --ext-rm extB. This value is optional.

--output, -o
: Specifies the format of the command output. Valid values are json, yaml, jsonpath=JSONPATH_EXPRESSION, and jsonpath-as-json=JSONPATH_EXPRESSION. Use jsonpath to specify the path to an element of the JSON output. This value is optional.

--path
: The path within the destination application where events are forwarded; for example, /events. The default path is the root URL of the destination application. This option can only be specified if destination-type is app. This value is optional.

--prefix, -p
: Prefix of the IBM Cloud Object Storage object. This value is optional.

--quiet, -q
: Specify this option to reduce the output of the command. This value is optional. The default value is false.

--suffix, -s
: Suffix of the IBM Cloud Object Storage object. Consider the file type (extension) of your file when specifying the suffix. This value is optional.





 Example 

The following example updates a COS subscription called mycosevent to listen for only write events.

ibmcloud ce subscription cos update --name mycosevent --event-type write





 Example output 

Updating COS source 'mycosevent'...
* and the credentials for secure-file-storage-cloudant.



When using Cloud Shell, you can use nano credentials.env to edit the file.
3. Set the environment variables required for secure-file-storage.template.yaml file to generate secure-file-storage.yaml in the next step.



1. Start by setting the cluster name by replacing <YOUR_CLUSTER_NAME>:

export MYCLUSTER=<YOUR_CLUSTER_NAME>
2. Set the ingress subdomain and ingress secret using ibmcloud ks commands:

export INGRESS_SUBDOMAIN=$(ibmcloud ks cluster get --cluster $MYCLUSTER --output json | jq -r 'try(.ingressHostname) // .ingress.hostname')
export INGRESS_SECRET=$(ibmcloud ks cluster get --cluster $MYCLUSTER --output json | jq -r 'try(.ingressSecretName) // .ingress.secretName')
3. Set the image repository name to the pre-built image icr.io/solution-tutorials/tutorial-cloud-e2e-security:

export IMAGE_REPOSITORY=icr.io/solution-tutorials/tutorial-cloud-e2e-security
4. Set additional environment variables by replacing the default values:

export BASENAME=secure-file-storage
export TARGET_NAMESPACE=default
5. Optionally set $IMAGE_PULL_SECRET environment variable only if you are using another Kubernetes namespace than the default namespace and the IBM Cloud Container Registry for the image. This requires additional Kubernetes configuration (e.g. [creating a container registry secret in the new namespace](https://cloud.ibm.com/docs/containers?topic=containers-registryother)).



4. Run the below command to generate secure-file-storage.yaml. It will use the environment variables you just configured together with the template file secure-file-storage.template.yaml.

./generate_yaml.sh
SELECT * FROM crn:v1:bluemix:public:messagehub:us-south:a/2383fabbd90354d33c1abfdf3a9f35d5:4d03d962-bfa5-4dc6-8148-f2f411cb8987::/jsontopic STORED AS JSON
EMIT cos://us-south/mybucket/events STORED AS PARQUET
EXECUTE AS crn:v1:bluemix:public:kms:us-south:a/33e58e0da6e6926e09fd68480e66078e:5195f066-6340-4fa2-b189-6255db72c4f2:key:490c8133-5539-4601-9aa3-1d3a11cb9c44

The difference from a batch query is that the FROM clause points to a Event Streams topic by using the CRN of an Event Streams instance. The allowed format of the events is JSON. The EMIT specifies the Cloud Object Storage bucket and the required format is Parquet. The last option to specify is a valid Key Protect key that holds an API key with the permissions to read from Event Streams and write to Cloud Object Storage. The API key is needed, as theoretically, the job can run forever.

Data Engine uses private endpoints when available, which is only the case in Enterprise plans. Public endpoints are only used if private endpoints are not available or enabled.





 Data on Cloud Object Storage 

In addition to the objects that are written when you do a [batch query](https://cloud.ibm.com/docs/sql-query?topic=sql-query-overviewresult=), the following two objects are also created:



* _checkpoint: Do not delete or modify the objects in this structure as it stores the last offsets of the topic.
* _schema_as_json: Do not delete or modify the objects in this structure as it stores the schema of the Kafka events.



The objects are written to Cloud Object Storage in micro batches.
After generating a [Service Credential](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-service-credentials), the resulting JSON document can be saved to /.bluemix/cos_credentials. The SDK will automatically source credentials from this file unless other credentials are explicitly set during client creation. If the cos_credentials file contains HMAC keys the client authenticates with a signature, otherwise the client uses the provided API key to authenticate by using a bearer token.

If migrating from AWS S3, you can also source credentials data from /.aws/credentials in the format:

[default]
aws_access_key_id = {API_KEY}
aws_secret_access_key = {SERVICE_INSTANCE_ID}

If both /.bluemix/cos_credentials and /.aws/credentials exist, cos_credentials takes preference.

For more details on client construction, [see the Javadoc](https://ibm.github.io/ibm-cos-sdk-java/com/ibm/cloud/objectstorage/client/builder/AwsClientBuilder.html).





 Code Examples 

In your code, you must remove the angled brackets or any other excess characters that are provided here as illustration.

Let's start with an complete example class that will run through some basic functionality, then explore the classes individually. This CosExample class will list objects in an existing bucket, create a new bucket, and then list all buckets in the service instance.



 Gather required information 



* bucketName and newBucketName are [unique and DNS-safe](https://cloud.ibm.com/docs/cloud-object-storage/api-reference?topic=cloud-object-storage-compatibility-api-bucket-operationscompatibility-api-new-bucket) strings. Because bucket names are unique across the entire system, these values need to be changed if this example is run multiple times. Note that names are reserved for 10 - 15 minutes after deletion.
For more information, see the API documentation's [Authentication section](https://cloud.ibm.com/apidocs/cloudant?code=goauthentication-with-external-configuration) for examples.

See the example response of requesting the records in reverse sort order:

{
"total_rows": 2,
"offset": 0,
"rows": [
{
"id": "abc123",
"key": "bob.smith@aol.com",
"value":
"Bob Smith",
true,
"2019-01-24T10:42:59.000Z"
]
},
{
"id": "abc125",
"key": "amelie.smith@aol.com",
"value":
"Amelie Smith",
true,
"2020-04-24T10:42:59.000Z"
]
}
]
}
Show more





 Specifying start and end keys 

The start_key and end_key query arguments can be used to specify the range of values that are returned when querying the view.

The sort direction is always applied first. Next, filtering is applied by using the start_key and end_key query arguments. It is possible that no rows match your keyrange if sort and filter plans don't make sense when combined.

See the example of using HTTP to make a global query that includes start_key and end_key query arguments:

GET $SERVICE_URL/$DATABASE/_design/$DDOC/_view/$VIEW_NAME?start_key="alpha"&end_key="beta" HTTP/1.1

See the example of a global query that includes start_key and end_key query arguments.

Client libraries use POST method instead of GET because they have the same behavior.



* Curl
* Java
* Node
* Python
* Go
The data that IBM Cloudant holds can be viewed and updated through the IBM Cloudant Dashboard.

If you have an account that is provisioned by IBM Cloud (including a dedicated instance), IBM Cloudant does not collect the personal data that was discussed earlier. This data is held by IBM Cloud.

Do not use sensitive data for IBM Cloudant instance names when you provision by using IBM Cloud, such as: Personal Information (PI), Personal Identifying Information (PII), and Customer-specific Data.

IBM Cloudant processes limited customer PI in the course of running the service and optimizing the user experience of it. IBM Cloudant uses email for contacting customers. Monitoring customer interactions with the IBM Cloudant Dashboard is the other way IBM Cloudant processes PI.



 Restriction of processing 

IBM Cloudant sends dashboard interaction data to Segment. It's possible to ask IBM Cloudant to restrict processing of customer PI in this way with an IBM Cloudant support request through the [IBM Cloud Support portal](https://www.ibm.com/cloud/support). Upon receipt of such a request, IBM Cloudant deletes information that is associated with the customer as sent to Segment, and prevents further data from being sent. IBM Cloudant needs to retain the ability to contact dedicated customers by email. IBM Cloudant provides an interface for customers to keep this information up to date either directly, or by using customer configuration of their contact details with their IBM Cloud account details.







 Is the IBM Cloudant database encrypted? 

All clusters have an encrypted file system (encryption at rest) that uses Linux™ Unified Key Setup (LUKS). Data in the database is visible to the operations and support teams (see the following paragraph).

For sensitive data, that you determine must remain invisible to IBM Cloudant, you must encrypt or otherwise protect (pseudonymize) your data before you send it to us. Do not use PI as a document _id in your URLs, for example, https://$ACCOUNT.cloudant.com/$DATABASE/$DOCUMENT_ID, since PI is always visible and written to the access logs.





 Data locations 

Locations where IBM Cloudant processes personal data are made available, and kept up to date, through the DSA.
Cloudant service = Cloudant.newInstance();

GetSearchInfoOptions infoOptions =
new GetSearchInfoOptions.Builder()
.db("foundbite")
.ddoc("app")
.index("description")
.build();

SearchInfoResult response =
service.getSearchInfo(infoOptions).execute()
.getResult();

System.out.println(response);

const { CloudantV1 } = require('@ibm-cloud/cloudant');

const service = CloudantV1.newInstance({});

service.getSearchInfo({
db: 'foundbite',
ddoc: 'app',
index: 'description'
}).then(response => {
console.log(response.result);
});

from ibmcloudant.cloudant_v1 import CloudantV1

service = CloudantV1.new_instance()

response = service.get_search_info(
db='foundbite',
ddoc='app',
index='description'
).get_result()

print(response)

getSearchInfoOptions := service.NewGetSearchInfoOptions(
"foundbite",
"app",
"description",
)

searchInfoResult, response, err := service.GetSearchInfo(getSearchInfoOptions)
if err != nil {
panic(err)
}

b, _ := json.MarshalIndent(searchInfoResult, "", " ")
fmt.Println(string(b))

The JSON structure includes the following individual fields:

name
: Name or ID of the Search within the design document.

search_index
: The Search Index

pending_seq
IBM Cloudant’s filtered replication allows the definition of a JavaScript function that uses the return value to determine whether each document in a database is to be filtered or not. [Filter functions](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-design-documentsfilter-functions) are stored in [design documents](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-design-documentsdesign-documents).

See the following example filter function for replicating non-deleted documents:

function(doc, req) {
if (doc._deleted) {
return false;
}
return true;
}

When a replication job starts, a filter function’s name is specified as a combination of the design document where it is stored, and the filter function’s name. You can also specify a query_params value. This value is an object that contains properties that are passed to the filter function in the query field of its second (req) argument.

See the following example that uses HTTP to start a filtered replication:

POST /_replicator HTTP/1.1
Content-Type: application/json
Host: $SERVICE_URL
Authorization: ...

See the following example that uses the command line to start a filtered replication:



* Curl
* Java
* Node
* Python
* Go



curl -X POST -H "Content-type: application/json" "$SERVICE_URL/_replicator" -d @filtered-replication.json

import com.ibm.cloud.cloudant.v1.Cloudant;
import com.ibm.cloud.cloudant.v1.model.DocumentResult;
import com.ibm.cloud.cloudant.v1.model.PutReplicationDocumentOptions;
import com.ibm.cloud.cloudant.v1.model.ReplicationDatabase;
import com.ibm.cloud.cloudant.v1.model.ReplicationDatabaseAuth;
import com.ibm.cloud.cloudant.v1.model.ReplicationDatabaseAuthIam;
We limit documents to a maximum size of 64 MB.





 No JavaScript reducers when options.partitioned is true 

Design documents with options.partitioned set to true can't contain JavaScript reduce functions, only built-ins Erlang reducers such as _stats.







 Storing the view definition 

Each view is a JavaScript function. Views are stored in design documents. So, to store a view, IBM Cloudant simply stores the function definition within a design document. A design document can be [created or updated](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-design-documentscreating-or-updating-a-design-document) just like any other document.

To store a view definition, PUT the view definition content into a _design document.

In the following example, the getVerifiedEmails view is defined as a map function, and is available within the views field of the design document.

Use the PUT method to add a view into a design document:

PUT $SERVICE_URL/$DATABASE/_design/$DDOC HTTP/1.1
Content-Type: application/json

The following sample adds a new getVerifiedEmails named view function to the allusers design document with view definition:

{
"views": {
"getVerifiedEmails": {
"map": "function(user) { if(user.email_verified === true){ emit(doc.email, {name: user.name, email_verified: user.email_verified, joined: user.joined}) }} "
}
}
}

See the request examples:



* Curl
* Java
* Node
* Python
* Go



curl -X PUT "$SERVICE_URL/users/_design/allusers" --data '{
"views": {
"getVerifiedEmails": {
"map": "function(user) { if(user.email_verified === true){ emit(doc.email, {name: user.name, email_verified: user.email_verified, joined: user.joined}) }}"
Additionally, [HMAC credentials](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-uhc-hmac-credentials-main) make it possible to allow [temporary public access that uses pre-signed URLs](https://cloud.ibm.com/docs/services/cloud-object-storage/hmac?topic=cloud-object-storage-presign-url).



 Upload a public object 

curl -X "PUT" "https://{endpoint}/{bucket-name}/{object-name}" -H "x-amz-acl: public-read" -H "Authorization: Bearer {token}" -H "Content-Type: text/plain; charset=utf-8" -d "{object-contents}"





 Allow public access to an existing object 

Using the query parameter ?acl without a payload and the x-amz-acl: public-read header allows public access to the object without needing to overwrite the data.

curl -X "PUT" "https://{endpoint}/{bucket-name}/{object-name}?acl" -H "x-amz-acl: public-read" -H "Authorization: Bearer {token}"





 Make a public object private again 

Using the query parameter ?acl without a payload and an empty x-amz-acl: header revokes public access to the object without needing to overwrite the data.

curl -X "PUT" "https://{endpoint}/{bucket-name}/{object-name}?acl" -H "Authorization: Bearer {token}" -H "x-amz-acl:"







 Static websites 

While IBM Cloud Object Storage doesn't support automatic static website hosting, it's possible to manually configure a web server and use it to serve publicly accessible content hosted in a bucket. For more information, see the [overview of static website options](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-static-website-options).