vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Combining IBM Cloudant with other IBM services 

Each database technology has its own strengths and weaknesses. Some are built for high availability and data durability (at the expense of more hardware and extra cost). Others favor speed and can churn out blazingly fast queries (but might lose data in a sudden power failure).

In this tutorial, you combine two IBM® services, IBM® Cloudant® for IBM Cloud® and IBM Cloud® Databases for Redis to optimize for speed and cost by implementing a caching system for database queries.

This tutorial takes less than an hour to complete. It is not entirely cost-free because the Databases for Redis service does not come with a free tier. However, if you deprovision the services after you finish with them, you should not have to pay more than a few dollars.



 The Project - Team Directory 

Team Directory is a web app that contains the details of all divisional employees. These employees are assigned to six different color-coded teams (Red, Orange, Green, Blue, Yellow, and Purple). When you visit the app, you can select one team and see the names of all the members of that team. The app also shows you whether this list of team members was obtained from the IBM Cloudant database or from the Databases for Redis cache and how long the query took. You can see just how much quicker cached data is to retrieve.





 Objectives 



1. Learn how to provision multiple services on the IBM Cloud® by using Terraform.
2. Learn how to use the IBM Cloudant NodeJS SDK to access your IBM Cloud® services.







 Prerequisites 

You need the following items:



1. An [IBM Cloud®](https://cloud.ibm.com/login) pay-as-you-go account
2. Access to a Mac or Linux™ terminal
3. [Git](https://git-scm.com/downloads)
4. [Node.js and npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm)
5.
Watson IoT Platform Connection and Analytics Service - Capacity 1 and Capacity 2 The Connection and Analytics Service plans offer a ready-to-run, pre-integrated, and fully managed SaaS IoT platform with extended connectivity, data management, and advanced analytics features. For more information about the features that are offered with these service plans, see [the Watson IoT Platform Knowledge Center ![External link icon](https://cloud.ibm.com/docs-content/v1/content/icons/launch-glyph.svg)](https://www.ibm.com/support/knowledgecenter/SSQP8H/iot/overview/overview.html) 



For more information about purchasing these plans, see [Internet of Things Platform in the IBM Cloud catalog ![External link icon](https://cloud.ibm.com/docs-content/v1/content/icons/launch-glyph.svg)](https://cloud.ibm.com/catalog/services/internet-of-things-platform).



 Plans that are withdrawn from marketing 

The following Standard and Advanced security plans are available only for existing customers of those plans. If you are a new customer, you must use the Lite plan or purchase one of the Watson IoT Platform Connection and Analytics Service plans.



 Plan Description 

 Watson IoT Platform Standard The Standard service plan includes the free data-transfer limit that is offered in the Lite plan and then usage-based billing beyond the free quote for data exchanged, data analyzed, and edge data analyzed. 
 Watson IoT Platform Advanced Security The Advanced Security service plan includes the free data-transfer limit that is offered in the Lite plan and then usage-based billing beyond the free quote for data exchanged, data analyzed, and edge data analyzed. This plan also provides advanced Risk and Security Management features. 









 Upgrading service plans 

If you are an existing customer and want to take advantage of the full [Watson IoT Platform feature set !
VPC landing zone - Standard variation 

The Standard variation of the VPC landing zone deployable architecture uses two Virtual Private Clouds (VPC), a Management VPC, and a Workload VPC to manage the environment and the deployed workload. Each VPC is a multi-zoned, multi-subnet implementation that keeps your workloads secure. A transit gateway connects the VPCs to each other and Virtual Private Endpoints are used connect to IBM Cloud services.

IBM Cloud Flow Logs for VPC enables the collection and storage of information about the internet protocol (IP) traffic that is going to and from network interfaces within your VPC. In addition, Activity Tracker logs events from enabled services. IBM Cloud Flow Logs for VPC and Activity Tracker are included in this deployable architecture. You can add more security services, such as Hyper Protect Crypto Services.



 Architecture diagram 

Zoom

![Architecture diagram for the Standard variation of VPC landing zone](https://cloud.ibm.com/docs-content/v1/content/9610604e9f8dcd5f7b6b138dd22ba445300be7f5/deployable-reference-architectures/terraform-ibm-landing-zone/vpc.drawio.svg)

Figure 1. Standard variation of VPC landing zone





 Design requirements 

Zoom

![Design requirements for VPC landing zone](https://cloud.ibm.com/docs-content/v1/content/9610604e9f8dcd5f7b6b138dd22ba445300be7f5/deployable-reference-architectures/terraform-ibm-landing-zone/heat-map-deploy-arch-slz-vpc-standard.svg)

Figure 2. Scope of the design requirements





 Components 



 VPC architecture decisions 



Table 1. Architecture decisions

 Requirement Component Reasons for choice Alternative choice 

 <br><br> * Provide infrastructure/application administration access to monitor, operate, and maintain the environment.<br> * Limit the number of infrastructure/application administration entry points to ensure security audit.<br><br><br> Management VPC service
You cannot create new service instances that use the Lite or Advanced plan types in the Dallas, Frankfurt, London, Sydney, Tokyo, and Washington DC locations. Any existing Lite and Advanced plans continue to function properly and continue to be supported. You can upgrade from a Lite plan to an Advanced plan. Use the new Plus plan and its associated 30-day free trial to explore new features and a simpler way to build that is available with the latest version of the product.





 24 September 2021 

New scoring for NLU enrichments
: Relevance and confidence scores are displayed for NLU enrichments that are returned by search. For example, when you open the JSON view of the document preview from a query result, you can see confidence scores for Entities mentions and relevance scores for Keyword mentions.





 9 September 2021 

New location for Plus plan
: The Plus plan is now available from the Sydney location. Use the new Plus plan and its associated 30-day free trial to explore new features and a simpler way to build that is available with the latest version of the product. For more information, see [Getting the most from Discovery](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-version-choose).

Change to Lite and Advanced plans in most locations
: Lite and Advanced plans are discontinued. You cannot create new service instances that use the Lite or Advanced plan types in the Dallas, Frankfurt, London, Sydney, Tokyo, or Washington DC locations. Any existing Lite and Advanced plans continue to function properly and continue to be supported. You can upgrade from a Lite plan to an Advanced plan.





 26 August 2021 

New locations for the Plus plan
: The Plus plan is now available from the London and Washington DC locations, in addition to Dallas, Frankfurt, and Tokyo.

Change to Lite and Advanced plans in some locations
: You cannot create new service instances that use the Lite or Advanced plan types in the Dallas, Frankfurt, London, Tokyo, or Washington DC locations. Any existing Lite and Advanced plans continue to function properly and continue to be supported. You can upgrade from a Lite plan to an Advanced plan.

New answer finding feature
: Answer finding is now generally available for managed deployments.
You cannot use an instance on the Lite plan to receive streamed data. This plan has a 0-day retention period.



 Features by plan 

The following tables outline the different features that are included in each service plan:



Table 2. List of features available that are available for each service plan

 Feature HIPAA 30 day Event Search plan 30 day Event Search plan 14 day Event Search plan 7 day Event Search plan Lite plan 

 Live streaming tail ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker/images/checkmark-icon.svg) 
 Events are stored and searchable !
See the following table for the API differences between classic and VPC.



Table 5. API comparison

 Classic infrastructure VPC infrastructure 

 Existing SoftLayer API (SLAPI) New developer-friendly, REST-based API 







 Next steps 

To review all our VPC infrastructure capabilities, see [About virtual private cloud](https://cloud.ibm.com/docs/vpc?topic=vpc-about-vpc). To start exploring infrastructure overall, see [Building your infrastructure](https://cloud.ibm.com/docs/overview?topic=overview-first-steps-it-ops).
Upgrading your plan 

IBM Watson™ Discovery offers three plans that provide different levels of resources and capabilities to suit your needs.

See [Discovery pricing plans](https://cloud.ibm.com/docs/discovery?topic=discovery-discovery-pricing-plans) and [Discovery catalog](https://cloud.ibm.com/catalog/services/discovery) for details.

You cannot directly upgrade your plan from Lite to XS because XS plans are not suitable for production usage. If you have the Lite plan but want the XS plan, you must create a new instance by using the XS size and then manually recreate your collections. In addition, you cannot directly downgrade from S to XS. If you have the S plan but want the XS plan, follow the same steps in the case of moving to an XS plan from a Lite plan.

You cannot upgrade from a Lite (v1) plan to a Plus (v2) plan. And you cannot upgrade from an Advanced (v1) plan to a Premium (v2) plan. To start using v2, create a new Plus or Premium plan. For more information about Premium plan instances that were created on or after 16 July 2020 or about Plus (including Plus Trial) plan instances, see [these docs](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-upgrade).



 Upgrading your service 

To resize your plan from Lite to Advanced:



1. From the [resource list](https://cloud.ibm.com/resources/), click on your Discovery instance to go to the Discovery dashboard page.
2. From the Manage page of Discovery, click Upgrade to choose the Advanced plan. Follow the steps on the Plan page to complete your upgrade.
3. Return to the Manage page and click Launch Watson Discovery to open the Discovery tooling.



* If you never created an environment for your Lite plan before the upgrade to Advanced, click the !
No, Lite Plan instances can only be upgraded to the Cloud Object Storage Standard plan.
* Are there any minimum object size or minimum duration requirements for objects stored with the One-Rate plan?

There are no minimum object size or minimum duration requirements for the One-Rate plan.
* What is the cost of data retrieval from One Rate Active buckets?

There is no data retrieval charge for the One Rate Active buckets.
* What happens if I exceed my monthly allowance for Outbound bandwidth and Operational requests?

For any usage (Outbound bandwidth or Operational requests) that exceeds the allowance determined by aggregated monthly capacity, a small overage fee applies based on the One Rate pricing regions. See [One Rate pricing plan details](https://cloud.ibm.com/objectstorage/createpricing).
* Is the overage pricing tiered for Outbound bandwidth and Operational requests?

No, the overage pricing for the One Rate plan has flat rates regardless of excess usage. See [One Rate pricing plan details](https://cloud.ibm.com/objectstorage/createpricing).
* I already have a Cloud Object Storage Standard plan in my IBM Cloud account. Can I add a One Rate plan for my new workloads?

Yes, you can add a One Rate plan to your existing account in addition to the Standard plan. If you are a new to Cloud Object Storage, you can add either Standard or One Rate plan (or both) based on your workload requirements.
* Why can I not create or delete a service instance?

A user is required to have have at a minimum the platform role of editor for all IAM enabled services, or at least for Cloud Object Service. For more information, see the [IAM documentation on roles](https://cloud.ibm.com/docs/account?topic=account-iam-service-roles-actions).
* Which one of my instances uses a Lite plan?

An account is limited to a single instance of IBM Cloud Object Storage that uses a Lite plan. You can find this instance three different ways:



1.
"language": "javascript",
"purge_seq": 10,
"waiting_commit": false,
"waiting_clients": 0,
"signature": "fc65594ee76087a3b8c726caf5b40687",
"update_seq": 375031,
"disk_size": 16491
}
}





 The _search_info endpoint 

The _search_info endpoint returns information about a specified search that is defined within a specific design document.

Method
: GET /db/_design/$DDOC/_search_info/yourSearch

Request
: None

Response
: JSON that contains information about the specified search.

Roles permitted*
: _reader

See the following example of getting information about the description search, which is defined within the app design document that is stored in the foundbite database, by using HTTP:

GET /foundbite/_design/app/_search_info/description HTTP/1.1

See the following examples of getting information about the description search index, which is defined within the app design document that is stored in the foundbite database:



* Curl
* Java
* Node
* Python
* Go



curl "$SERVICE_URL/foundbite/_design/app/_search_info/description"

import com.ibm.cloud.cloudant.v1.Cloudant;
import com.ibm.cloud.cloudant.v1.model.GetSearchInfoOptions;
import com.ibm.cloud.cloudant.v1.model.SearchInfoResult;

Cloudant service = Cloudant.newInstance();

GetSearchInfoOptions infoOptions =
new GetSearchInfoOptions.Builder()
.db("foundbite")
.ddoc("app")
.index("description")
.build();

SearchInfoResult response =
service.getSearchInfo(infoOptions).execute()
.getResult();
Managing access for IBM Cloudant 

IBM Cloud® Identity and Access Management provides a unified approach to managing user identities, services, and access control.

The following text describes the integration of IBM® Cloudant® for IBM Cloud® with IBM Cloud Identity and Access Management. The following topics are discussed:



* Differences between IBM Cloudant legacy access controls and IBM Cloud IAM access controls.
* Advantages and disadvantages of each to help you decide which to use.
* How to use IAM within IBM Cloudant's client libraries by using HTTP calls.
* Description of the IAM actions and roles available within IBM Cloudant.



For more information, see an overview of [IAM](https://cloud.ibm.com/docs/account?topic=account-iamoverview) that includes the following topics:



* Manage user and service IDs.
* Manage available credentials.
* Use IAM access policies that allow and revoke access to IBM Cloudant service instances.





 Differences between IBM Cloudant legacy access controls and IAM access controls 

The following section provides a brief overview of the differences between IBM Cloudant legacy access controls and IBM Cloud IAM's access control mechanisms.



 IBM Cloud Identity and Access Management 



* Centrally managed access management across IBM Cloud.
* Allow a user or service to access many different resources by using the same set of credentials (for example, same username and password or IAM API key).
* IAM API keys can be granted access to account management functions, like creating new databases.







 IBM Cloudant legacy access controls 



* Unique to IBM Cloudant.
* Access to each service instance requires its own set of credentials.
* Uses HTTP basic authentication with credentials that are not bound to an individual user or service.
* IBM Cloudant API keys can be granted permissions only at a database level.







 API key notes 

In this document, wherever API keys are mentioned it refers to IAM API keys. IBM Cloudant legacy access controls also have a concept of API keys, and any discussion about IBM Cloudant legacy credentials or username and password combinations also includes IBM Cloudant API keys.







 Enabling IAM with IBM Cloudant
* Simulated Qubits: 63









 Standard plan 

A pay-as-you-go plan for accessing IBM Quantum systems and simulators. Build your own programs and access all the benefits of Qiskit Runtime by running on real quantum hardware.





 Pricing overview 

The Lite plan is free. The Standard plan charges you per Qiskit Runtime second when running on physical systems. The following diagram illustrates what is included in a QR second. Any time spent waiting for results or in the queue for the quantum computer are excluded.

Zoom

![This diagram shows that everything before the program starts (such as queuing) is free. After the job starts, it costs $1.60 per second.](https://cloud.ibm.com/docs-content/v1/content/5f399fa794584e8a662591b494b9a99aa927c74c/quantum-computing/images/Runtime_Accounting_Diagram.png)

Figure 1. Runtime second accounting

Qiskit Runtime usage is the time counted by Qiskit Runtime to process a job, and is determined by the use of internal resources.





 Next steps 



* See [Manage costs](https://cloud.ibm.com/docs/quantum-computing?topic=quantum-computing-cost) to learn how to determine and minimize your costs.
Security 

Protecting application data for large-scale web and mobile apps can be complex, especially with distributed and NoSQL databases.

Just as it reduces the effort of maintaining your databases to keep them running and growing nonstop, IBM® Cloudant® for IBM Cloud® also ensures your data stays secure and protected.



 Tier one physical platforms 

The IBM Cloudant DBaaS is physically hosted on Tier-1 cloud infrastructure providers such as IBM Cloud® and Amazon. Therefore, your data is protected by the network and physical security measures that are employed by those providers, including (but not limited to):



* Certifications - Compliance with SSAE16, SOC2 Type 1, ISAE 3402, ISO 27001, CSA, and other standards.
* Access and identity management.
* General physical security of data centers and network operations center monitoring.
* Server hardening.
* IBM Cloudant gives you the flexibility to choose or switch among the different providers as your SLA and cost requirements change.



More details about the certifications are available in the [Compliance information](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-compliancecompliance).





 Secure access control 

IBM Cloudant has a multitude of built-in security features for you to control access to data:



Table 1. IBM Cloudant security features

 Feature Description 

 Authentication IBM Cloudant is accessed by using an HTTPS API. Where the API endpoint requires it, the user is authenticated for every HTTPS request IBM Cloudant receives. IBM Cloudant supports both legacy and IAM access controls. For more information, see the [IAM guide](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-managing-access-for-cloudant) or the legacy [Authentication API document](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-work-with-your-accountauthentication). 
 Authorization IBM Cloudant supports both legacy and IAM access controls. The IBM Cloudant team recommends that you use IAM access controls for authentication whenever possible.
It features a simpler pricing model (document hours are no longer part of the calculation) and tiered pricing for IBM Watson™ Discovery News queries. In addition, the Standard plan is retired, and the Lite plan has reduced document and Discovery News query limits. The pricing changes require no action by current Discovery News users. For details, see [Discovery Pricing Plans](https://cloud.ibm.com/docs/discovery?topic=discovery-discovery-pricing-plans).

Update to version string : The version date of the API is updated to 2018-08-01. To take advantage of the new environment sizing options (LT, XS, S, MS, M, ML, L, XL, XXL, XXXL), you must use this version date when creating environments, using the API. The environment sizes now have the type of string (previously the type was integer.)





 27 July 2018 

New Japanese language support for Discovery News : Released [IBM Watson™ Discovery News](https://cloud.ibm.com/docs/discovery?topic=discovery-watson-discovery-news) in one additional language: Japanese (collection_id: news-ja). IBM Watson™ Discovery News is also available in English, German, Korean, and Spanish.





 25 June 2018 

New connect and sync with Box, Microsoft SharePoint Online, and Salesforce : Added the option to connect to and sync with Salesforce, Microsoft SharePoint Online, and Box data sources. These data sources are not available in Premium environments. : Released the [Source Credential](https://cloud.ibm.com/apidocs/discoverylist-credentials) and [Configuration](https://cloud.ibm.com/apidocs/discoveryadd-configuration) APIs for these data sources. : IBM Watson™ Discovery supports only English language collections when connecting and syncing to Box, Salesforce, and SharePoint Online with the Discovery tooling. [Resolved](https://cloud.ibm.com/docs/discovery?topic=discovery-release-notes2aug18) : The individual document file size limit for Box, Salesforce, and SharePoint Online is 10MB.

New Performance Dashboard : Added a new Performance Dashboard in Discovery tooling.
From the [resource list](https://cloud.ibm.com/resources/), click on your Discovery instance to go to the Discovery dashboard page.
2. From the Manage page of Discovery, click Upgrade to choose the Advanced plan. Follow the steps on the Plan page to complete your upgrade.
3. Return to the Manage page and click Launch Watson Discovery to open the Discovery tooling.



* If you never created an environment for your Lite plan before the upgrade to Advanced, click the ![Cog](https://cloud.ibm.com/docs-content/v1/content/ded4adc3ea0bd2a81b113c579f2b1183926da211/discovery/images/icon_settings.png) icon, and choose Create environment. Choose the options for the Advanced plan that fit your needs. (X-Small, Small, Medium-Small, Medium, Medium-Large, Large, X-Large, XX-Large).
* If you created an environment for your Lite plan before the upgrade to Advanced, your new Advanced plan environment is set to Small by default.









 Switching from one Advanced tier to another 

If you already have an Advanced plan, and would like to upgrade it to a larger plan size, you can do so using the [API](https://cloud.ibm.com/apidocs/discoveryupdateenvironment).

For detailed information on Advanced plan storage limits and pricing, see [Advanced pricing plans](https://cloud.ibm.com/docs/discovery?topic=discovery-discovery-pricing-plansadvanced).

Available Advanced plan sizes are:



 Plan size Label 

 X-Small XS 
 Small S 
 Medium-Small MS 
 Medium M 
 Medium-Large ML 
 Large L 
 X-Large XL 
 XX-Large XXL 





* Querying and indexing can proceed during upgrading. The time required for upgrading depends on a number of factors. You can poll your environment, using the API, while the upgrade completes.
* Moving from one level of Advanced to another does not require the creation of new instances.