vania-janet/multiturn-rag-retrieval-data huggingface dataset:
For sensitive data, that you determine must remain invisible to IBM Cloudant, you must encrypt or otherwise protect (pseudonymize) your data before you send it to us. Do not use PI as a document _id in your URLs, for example, https://$ACCOUNT.cloudant.com/$DATABASE/$DOCUMENT_ID, since PI is always visible and written to the access logs.





 Data locations 

Locations where IBM Cloudant processes personal data are made available, and kept up to date, through the DSA.

For more information about data locations, see the [DSA under 7. IBM Hosting and Processing Locations](https://www.ibm.com/software/reports/compatibility/clarity-reports/report/html/softwareReqsForProduct?deliverableId=2EBB5860B34311E7A9EB066095601ABB).





 Service security 



 Using IBM Cloudant securely 

As a user of IBM Cloudant, you must follow these guidelines:



* Use the default CORS configuration to prevent unexpected access.
* Use API keys liberally, since components can have least privileged access, which is coupled with the audit log. This practice helps you understand who accessed which data.
* Encrypt or otherwise protect (pseudonymize) sensitive data that you determine must remain invisible to IBM Cloudant.







 Physical and environmental security measures 

Physical security of our data centers is handled by the infrastructure providers: IBM Cloud®, AWS, and 21Vianet. All hold externally audited certifications for their physical security. IBM Cloudant doesn't provide further details of the physical security controls in place at our data centers.

Physical security of the office locations that are used by our personnel is handled by IBM Corporate. Certification details and attestation reports (that is, ISO and SOC2) can be provided to the customer upon request.





 Technical and Organizational Measures 

Technical and Organizational Measures (TOMs) are employed by IBM Cloudant to ensure the security of Personal Data. IBM Cloudant holds externally audited certifications for the controls IBM Cloudant employs. Certification details and attestation reports (that is, ISO and SOC2) can be provided to the customer upon request.





 Service access to data
query: ':',
includeDocs: true,
limit: 1
}).then(response => {
console.log(response.result);
});

from ibmcloudant.cloudant_v1 import CloudantV1

service = CloudantV1.new_instance()

response = service.post_search(
db='<db-name>',
partition_key='<partition-key>',
ddoc='<ddoc>',
index='<index-name>',
query=':',
include_docs=True,
limit=1
).get_result()

print(response)

postPartitionSearchOptions := service.NewPostPartitionSearchOptions(
"<db-name>",
"<partition-key>",
"<ddoc>",
"<index-name>",
":",
)
postPartitionSearchOptions.SetIncludeDocs(true)
postPartitionSearchOptions.SetLimit(1)

searchResult, _, err := service.PostPartitionSearch(postPartitionSearchOptions)
if err != nil {
panic(err)
}

b, _ := json.MarshalIndent(searchResult, "", " ")
fmt.Println(string(b))

The previous Go example requires the following import block:

import (
"encoding/json"
"fmt"
"github.com/IBM/cloudant-go-sdk/cloudantv1"
)

See the following example that uses HTTP to query a global index:

GET /$DATABASE/_design/$DDOC/_search/$INDEX_NAME?include_docs=true&query=":"&limit=1 HTTP/1.1
Content-Type: application/json
Host: $ACCOUNT.cloudant.com
If JSON is not specified, output will be returned in a tabular format.help | --help | -h: List options available for the command.<-- </section "id="section-target-get-options-cos" "> --><-- <section "id="section-target-get-example-cos" "> --> Example The following is an example using the ibmcloud atracker target get --target new-target-name command showing a COS target. Target
Name: 		new-target-name
ID: 		xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
CRN: 		crn:v1:staging:public:atracker:us-south:a/xxxxxxxxxx:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx
Type: 		cloud_object_storage
COS Endpoint: 		s3.us-west.cloud-object-storage.appdomain.cloud
COS Target CRN: 		crn:v1:staging:public:cloud-object-storage:global:a/xxxxxxxxxx:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx:
COS Bucket: 		cloud-object-my-target
Service to Service Enabled:	true
Write Status: 		success
Created: 		2021-07-21T16:04:15.174Z
Updated: 		2021-07-21T17:49:56.452Z
<-- </section "id="section-target-get-example-cos" "> --><-- </section "class="ph" id="section-target-get-cli-cos" "> --><-- <section "class="ph" id="section-target-list-cli-cos" "> --> Listing all targets in a region Use this command to list the configured targets for an IBM Cloud Activity Tracker Event Routing region. ibmcloud atracker target ls --output FORMAT] ! !
For more information, see the API documentation's [Authentication section](https://cloud.ibm.com/apidocs/cloudant?code=goauthentication-with-external-configuration) for examples.

See the example to retrieve a list of the first 10 documents from a database, applying the user-created getVerifiedEmails view.

Client libraries use POST method instead of GET because they have the same behavior.



* Curl
* Java
* Node
* Python
* Go



curl -X GET "$SERVICE_URL/users/_design/allusers/_view/getVerifiedEmails?limit=10"

import com.ibm.cloud.cloudant.v1.Cloudant;
import com.ibm.cloud.cloudant.v1.model.PostViewOptions;
import com.ibm.cloud.cloudant.v1.model.ViewResult;

Cloudant service = Cloudant.newInstance();

PostViewOptions viewOptions = new PostViewOptions.Builder()
.db("users")
.ddoc("allusers")
.view("getVerifiedEmails")
.limit(10)
.build();

ViewResult response =
service.postView(viewOptions).execute()
.getResult();

System.out.println(response);

const { CloudantV1 } = require('@ibm-cloud/cloudant');

const service = CloudantV1.newInstance({});

service.postView({
db: 'users',
ddoc: 'allusers',
view: 'getVerifiedEmails',
limit: 10
}).then(response => {
console.log(response.result);
});

from ibmcloudant.cloudant_v1 import CloudantV1

service = CloudantV1.new_instance()

response = service.post_view(
db='users',
"use_index": "_design/query-demo-index"
}
Show more
2. Run this query:

acurl "https://$ACCOUNT.cloudant.com/databasedemo/_find" -X POST -H "Content-Type: application/json" -d @query3.json
3. Review the query results:

{
"docs": [{
"descriptionField": "hot",
"temperatureField": 97
},
{
"descriptionField": "hot",
"temperatureField": 75
}
],
"bookmark": "g1AAAABbeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYorJBoYmKWaWVokpZkamJtaGJskm5slmSenGhulWpibJhqC9HHA9OWATErUAGlkzsgvycoCAEsUF_A"
}



Now you know how to extract data from your database by using IBM Cloudant Query. For more information, see the [IBM Cloudant documentation](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-ibm-cloud-public).
* Uses HTTP basic authentication with credentials that are not bound to an individual user or service.
* IBM Cloudant API keys can be granted permissions only at a database level.







 API key notes 

In this document, wherever API keys are mentioned it refers to IAM API keys. IBM Cloudant legacy access controls also have a concept of API keys, and any discussion about IBM Cloudant legacy credentials or username and password combinations also includes IBM Cloudant API keys.







 Enabling IAM with IBM Cloudant 

All IBM Cloudant service instances provisioned July 2018 or later are provisioned in Resource Groups and are enabled with IBM Cloud IAM. Optionally, you can choose to also enable the IBM Cloudant legacy authentication mechanism. When you provision a new IBM Cloudant instance from the IBM Cloud catalog, select one of the following authentication methods.

Use both legacy credentials and IAM
: This mode means that both IAM and legacy credentials can be used to access the account. In particular, both IAM and legacy sets of credentials are provided to all applications bound to the account and service credentials generated.

Use only IAM
: This mode means that only IAM credentials are provided by using Service binding and credential generation.

When you use IAM roles other than Manager, such as Reader, Writer, Monitor, or Checkpointer, you must use Use only IAM to avoid supplying users with legacy credentials that include greater access permissions.

IBM Cloudant service instances that are provisioned previously in a Cloud Foundry org and space can be migrated to a Resource Group. After you migrate to a Resource Group, the instance is enabled with IBM Cloud IAM. For more information, see the [Resource Groups FAQ](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-faq-ibm-cloud-resource-groups) about how to migrate.



 IBM Cloudant API keys and Use only IAM 

Use of IBM Cloudant API keys alongside IAM is possible but not recommended. This recommendation is made because IBM Cloudant API keys and permissions are not visible or manageable by using the IAM policy interface, rendering holistic access management impossible.

The choice between Use only IAM or Use both legacy credentials and IAM affects the following factors:



1.
Currently only JSON file format is supported. 



Example

ibmcloud schematics blueprint get --id Blueprint_Basic.eaB.5cd9

Output

BLUEPRINT

Name Blueprint Basic Json
ID blueprint_Basic.eaB.435a
Description Deploys a simple two module blueprint Updated
Status create_success
Location us-east
Creator test@in.ibm.com
Last modified 2022-11-21T08:24:23.197Z

MODULES

SNO Workspace Name Workspace ID Status Location Updated
1 basic-resource-group us-east.workspace.basic-resource-group.3a89a5f1 INITIALISED us-east
2 basic-cos-storage us-east.workspace.basic-cos-storage.c775f80a INITIALISED us-east

OK





 ibmcloud schematics blueprint list 

You can list all the blueprints.

Syntax

ibmcloud schematics blueprint list [--profile PROFILE] [--limit LIMIT] [--offset OFFSET] [--output OUTPUT]

Command options



Schematics blueprint list flags

 Flag Required / Optional Description 

 --limit or -l Optional The maximum number of blueprints to list. Ignored if a negative number is set. Maximum number to list is 200, the default is -1. 
 --offset or -m Optional Offset in list. Ignored if a negative number is set. The default value is -1. 
 --output or -o Optional Returns the command-line output in JSON format. Currently only JSON file format is supported. 



Example

ibmcloud schematics blueprint list

Output

Name ID Source Type Status Location Creator Last modified
Blueprint Basic blueprint_Basic.eaB.435a GitHub job_finished us-east test@in.ibm.com 2022-11-21T10:45:57.329Z

OK





 ibmcloud schematics blueprint destroy 

Destroys all the resources associated with the modules in a blueprint. This action cannot be reversed.
It is recommended that you create an index.

To create an index, follow these steps:



1. Copy the following sample JSON data into a file named query-demo-index.json:

{
"index": {
"fields": [
"descriptionField",
"temperatureField"
],
"partial_filter_selector": {
"descriptionField": {
"$eq": "hot"
},
"temperatureField": {
"$gt": 50
}
}
},
"ddoc": "query-demo-index",
"type": "json"
}
Show more
2. Run the following command to create an index:

acurl "https://$ACCOUNT.cloudant.com/databasedemo/_index" -X POST -H "Content-Type: application/json" -d @query-demo-index.json
3. Review the results:

{
"result": "created",
"id": "_design/query-demo-index",
"name": "490441584f9eddb8d09ef234d636b5f3b18e4ce6"
}



You aren't required to create an index to run a query. However, if you don't, the following warning is included with your results as an indicator that creating an index reduces processing and makes your queries more effective. "Warning": "No matching index found, create an index to optimize query time."





 Step 2: Running a simple query 

This example demonstrates how IBM Cloudant Query finds documents based on the descriptionField with the value boiling.

To run the query, follow these steps:



1. Copy the following sample JSON into a data file named query1.json:

{
"selector": {
"descriptionField": "boiling"
}
}
2. Run the following command to query the database:
* --json or -j Results as JSON.







 Examples 

Set a PostgreSQL deployment named "MyPGSQL" with a "member" group to have a total memory to 4096 MB.

ibmcloud cdb deployment-groups-set MyPGSQL member --memory 4096









 Autoscaling 

The Autoscaling configuration represents the various conditions that control autoscaling for a deployment.



 ibmcloud cdb deployment-autoscaling 

Short version - autoscaling

Retrieve of all autoscaling conditions for a particular deployment.

ibmcloud cdb deployment-autoscaling <deployment name or CRN> GROUP_ID [--json]

Autoscaling currently only applies to the data members on your deployment, so the GROUP_ID is member.



 Command options 



* --json or -j Return the results as JSON.







 Examples 

ibmcloud cdb deployment-autoscaling elasticsearch-preproduction member







 ibmcloud cdb deployment-autoscaling-set 

Short version - autoscaling-set

Enable, disable, or set the conditions for autoscaling on your deployment.

ibmcloud cdb deployment-autoscaling-set (NAME|ID) GROUP_ID (@JSON_FILE|JSON_STRING) [--json] [--nowait]

Autoscaling currently only applies to the data members on your deployment, so the GROUP_ID is member. The autoscaling parameters that you would like to be set or unset are defined in a JSON object.



 Command options 



* --json or -j Return the results as JSON.
* --nowait or -n Do not wait for command completion.







 Examples 

This command sets memory to autoscale when I/O utilization hits a certain threshold for a deployment named elasticsearch-preproduction.
These partitioned queries exercise only one shard of the database. This practice makes them faster to execute than global queries. For billing purposes, they are classified as “read” requests instead of the more expensive “query” requests, which provides you with more usable capacity from the same IBM Cloudant plan.

Not all data designs lend themselves to a partitioned design, but if your data can be molded into a <partition key>:<document key> pattern, then your application can benefit in terms of performance and cost.



* [Partitioned Databases documentation](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-database-partitioning)
* [Partitioned Databases - Introduction blog](https://blog.cloudant.com/2019/03/05/Partition-Databases-Introduction.html)







 Treat the primary index as a free search index 

A default IBM Cloudant document _id is a 32-character string, encoding 128 bits of random data. The _id attribute is used to construct the database’s primary index, which is used by IBM Cloudant to retrieve documents by _id or ranges of keys when the user supplies a startkey/endkey pair. We can leverage this fact to pack our data into the _id field and use it as a “free” index that can query for ranges of values.

See some examples in the following list:



* Use time-sortable document IDs so that your documents are sorted into rough date and time order. This sorting makes it easy to retrieve recent additions to the database. For more information, see [Time-sortable -IDs](https://blog.cloudant.com/2018/08/24/Time-sortable-document-ids.html).
* Pack searchable data into your _id field, for example, <customerid><date><orderid> can be used to retrieve data by customer, customer/date, or customer/date/orderid.
* In a partitioned database, the judicious choice of partition key allows an entire database to be winnowed down to a handful of documents for a known partition key. Make sure that your partitioning schema solves your most common use case.
DocumentResult response =
service.putDocument(documentOptions).execute()
.getResult();

System.out.println(response);

from ibmcloudant.cloudant_v1 import Document, CloudantV1

service = CloudantV1.new_instance()

event_doc = Document(
type='event',
userid='abc123',
eventType='addedToBasket',
productId='1000042',
date='2019-01-28T10:44:22.000Z'
)
response = service.put_document(
db='events',
doc_id='0007241142412418284',
document=event_doc
).get_result()

print(response)

eventDoc := cloudantv1.Document{}
eventDoc.SetProperty("type", "event")
eventDoc.SetProperty("userid", "abc123")
eventDoc.SetProperty("eventType", "addedToBasket")
eventDoc.SetProperty("productId", "1000042")
eventDoc.SetProperty("date", "2019-01-28T10:44:22.000Z")

putDocumentOptions := service.NewPutDocumentOptions(
"events",
"0007241142412418284",
)
putDocumentOptions.SetDocument(&eventDoc)

documentResult, response, err := service.PutDocument(putDocumentOptions)
if err != nil {
panic(err)
}

b, _ := json.MarshalIndent(documentResult, "", " ")
fmt.Println(string(b))

const { CloudantV1 } = require('@ibm-cloud/cloudant');
The API key is exchanged for a bearer token by using the IAM API and the bearer token is passed to IBM Cloudant with each request. The bearer token lasts only an hour, so must be renewed with the IAM service periodically. When an IBM Cloudant service is provisioned, you can generate IAM only credentials, or both IAM and Legacy credentials - you decide.

How are credentials generated?

In the IBM Cloud Dashboard under your IBM Cloudant service, in the Service Credentials tab, click New Credential. A JSON document is created that contains the IAM key, the basic auth username and password, and the IBM Cloudant hostname.

See the example set of credentials:

For IAM, you need the apikey and host. For either Legacy or Basic-Auth or both, you need the URL (which contains the username and password that is embedded in the URL).

IBM Cloudant has three official client libraries: Java™, Node.js, and Python.

All three handle authentication automatically. You don't need to worry about how it exchanges the API key for a session token or how IAM authentication works - it is handled for you.

When we discuss the API in the documentation, we use Basic Auth as a convenience. However, we recommend that you use IAM authentication if possible as it allows better integration with the IBM Cloud platform and finer-grained permissions.

Time for our next practical exercise.

Log in to IBM Cloud and locate the IBM Cloudant Lite service that we created last time. In the Service Credentials tab, click the New Credential button to generate a set of IAM+Legacy credentials. Make a note of the JSON that it returns - we'll need that for the next exercise.

Then, visit the URL specified in the credentials JSON - what do you see?

To summarize, credentials are generated from the IBM Cloud Dashboard. You can have IAM or both IAM + legacy credentials. Both authentication methods involve exchanging your credentials for a time-limited token (authentication) - the token is then updated periodically as you use the service. The official libraries handle all of these tasks for you.

That's the end of this part.
If you choose to, you can replace random-route: true with host: <myChosenHostName>, supplying a host name of your choice.





 Step 4: Deploy to IBM Cloud 

You can use the IBM Cloud CLI to deploy apps.



1. Log in to your IBM Cloud account, and select an API endpoint.

ibmcloud login

If you have a federated user ID, instead use the following command to log in with your single sign-on ID. See [Logging in with a federated ID](https://cloud.ibm.com/docs/account?topic=account-federated_id) for more information.

ibmcloud login --sso
2. Target a Cloud Foundry org and space:

ibmcloud target --cf

If you don't have an org or a space set up, see [Adding orgs and spaces](https://cloud.ibm.com/docs/account?topic=account-orgsspacesusers).
3. From within the get-started-java directory, push your app to IBM Cloud.

ibmcloud cf push



Deploying your app can take a few minutes. When deployment completes, a message shows that your app is running. View your app at the URL listed in the output of the push command with /GetStartedJava appended to the end, or view both the app deployment status and the URL by running the following command:

ibmcloud cf apps

You can also go to the IBM Cloud [resource list](https://cloud.ibm.com/resources) to view your app. If you click Visit App URL, remember to append /GetStartedJava to the URL.

You can troubleshoot errors in the deployment process by using the ibmcloud cf logs GetStartedJava --recent command.





 Step 5: Add a database 

Next, we'll add an IBM Cloudant NoSQL database to this app and set up the app so that it can run locally and on IBM Cloud.



1. In your browser, log in to IBM Cloud and go to the Dashboard. Select Create resource.
2.
Search for IBM Cloudant, and select the service.
3. For Available authentication methods, select Use both legacy credentials and IAM. You can leave the default settings for the other fields. Click Create to create the service.
4. In the navigation, go to Connections, then click Create connection. Select your app, and click Connect.
5. Using the default values, click Connect & restage app to connect the database to your app. Click Restage when prompted.

IBM Cloud will restart your app and provide the database credentials to your app using the VCAP_SERVICES environment variable. This environment variable is available to the app only when it is running on IBM Cloud.



Environment variables enable you to separate deployment settings from your source code. For example, instead of specifying a database password in your source code, you can store it in an environment variable that you reference in your source code.





 Step 6: Use the database 

We're now going to update your local code to point to this database. We'll create a .env file that will store the credentials for the services the app will use. This file will get used ONLY when the app is running locally. When running in IBM Cloud, the credentials will be read from the VCAP_SERVICES environment variable.



1. Create a file called .env in the get-started-ruby directory with the following content:

CLOUDANT_URL=
2. Find your app in the IBM Cloud [resource list](https://cloud.ibm.com/resources). On the Service Details page for your app, click Connections in the sidebar. Click the IBM Cloudant menu icon (…) and select View credentials.
3. Copy and paste just the url from the credentials to the CLOUDANT_URL field of the .env file and save the changes. The result will be something like:

CLOUDANT_URL=https://123456789 ... bluemix.cloudant.com
4. Run your app locally.

rails server

View your app at: http://localhost:3000.
index_field1 = IndexField(deviceID='asc')
index_field2 = IndexField(ts='asc')

index = IndexDefinition(
fields=[index_field1, index_field2]
)

response = service.post_index(
db='readings',
name='deviceID-readings',
index=index,
type='json',
partitioned=True
).get_result()

print(response)

var indexField1 cloudantv1.IndexField
var indexField2 cloudantv1.IndexField
indexField1.SetProperty("deviceID", core.StringPtr("asc"))
indexField2.SetProperty("ts", core.StringPtr("asc"))

postIndexOptions := service.NewPostIndexOptions(
"readings",
&cloudantv1.IndexDefinition{
Fields: []cloudantv1.IndexField{
indexField1,
indexField2,
},
},
)
postIndexOptions.SetName("deviceID-readings")
postIndexOptions.SetType("json")
postIndexOptions.SetPartitioned(true)

indexResult, response, err := service.PostIndex(postIndexOptions)
if err != nil {
panic(err)
}

b, _ := json.MarshalIndent(indexResult, "", " ")
fmt.Println(string(b))

The previous Go example requires the following import block:

import (
"encoding/json"
"fmt"
"github.com/IBM/cloudant-go-sdk/cloudantv1"
"github.com/IBM/go-sdk-core/v5/core"
)

All Go examples require the service object to be initialized.