vania-janet/multiturn-rag-retrieval-data huggingface dataset:
IBM Cloud® Databases for DataStax docs 

IBM Cloud® Databases for DataStax is a scale-out NoSQL database built on Apache Cassandra, designed for high-availability and workload flexibility.

 Developer tools 

[API & SDK reference](https://cloud.ibm.com/apidocs/cloud-databases-api/cloud-databases-api-v5introduction)[CLI reference](https://cloud.ibm.com/docs/databases-cli-plugin)[Terraform reference](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/database)

 Recommended content 

[Deprecation of IBM Cloud® Databases for DataStax IBM Cloud® Databases for DataStax is deprecated and no longer supported as of 30 June 2024.](https://cloud.ibm.com/docs/databases-for-cassandra?topic=databases-for-cassandra-deprecation)[Setting the Admin Password The Databases for DataStax service is provisioned with an admin user. You must set the admin password before you use it to connect.](https://cloud.ibm.com/docs/databases-for-cassandra?topic=databases-for-cassandra-admin-password&interface=cli)[Creating and managing users, roles, and privileges IBM Cloud® Databases for DataStax uses a system of roles to manage database permissions. Roles are used to give a single user or a group of users a set of privileges.](https://cloud.ibm.com/docs/databases-for-cassandra?topic=databases-for-cassandra-user-management&interface=cli)

 Learn more 

[IBM Developer<br><br>Visit IBM Developer for technical articles, code patterns, tutorials, and more.<br><br>!
Databases <br><br> * [Hyper Protect DBaaS for MongoDB](https://cloud.ibm.com/docs/hyper-protect-dbaas-for-mongodb?topic=hyper-protect-dbaas-for-mongodb-manage-security-compliance)<br> * [Hyper Protect DBaaS for PostgreSQL](https://cloud.ibm.com/docs/hyper-protect-dbaas-for-postgresql?topic=hyper-protect-dbaas-for-postgresql-manage-security-compliance)<br><br><br> 



For more information, see:



* [Managing security and compliance in IBM Cloud](https://cloud.ibm.com/docs/overview?topic=overview-manage-security-compliance) - provides details on security and compliance within platform services.







 Related controls in IBM Cloud Framework for Financial Services 

The following [IBM Cloud Framework for Financial Services controls](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-aboutframework-control-requirements) are most related to this guidance. However, in addition to following the guidance here, do your own due diligence to ensure you meet the requirements.



Table 2. Related controls in IBM Cloud Framework for Financial Services

 Family Control 

 Change Management (CM) [CM-2 (2) Baseline Configuration | Automation Support for Accuracy and Currency](https://cloud.ibm.com/docs/framework-financial-services-controls?topic=framework-financial-services-controls-cm-2.2) <br>[CM-6 Configuration Settings](https://cloud.ibm.com/docs/framework-financial-services-controls?topic=framework-financial-services-controls-cm-6) <br>[CM-6 (1) Configuration Settings | Automated Management, Application, and Verification](https://cloud.ibm.com/docs/framework-financial-services-controls?topic=framework-financial-services-controls-cm-6.1)
They can filter the list by selecting a specific type of event, or by using keyword searches. For more information, see [Viewing notifications](https://cloud.ibm.com/docs/get-support?topic=get-support-viewing-notifications).





 01 February 2021 

Managing product availability in catalogs by location
: As the account owner or administrator, you can manage access to products in both the IBM Cloud catalog and the private catalogs in your account based on the location in which the products are deployed. For instance, if you want to limit access to products that are deployed in the Dallas 1 (us-south-1) zone, you can set a filter to include only those products. For more information, see [Managing catalog settings](https://cloud.ibm.com/docs/account?topic=account-filter-account).







 January 2021 



 11 January 2021 

Pay as you go with Committed Use pricing model
: Customers with a Subscription account can use the new pricing model, IBM Cloud Pay as you go with Committed Use. The new pricing model provides you with additional benefits as you navigate and build on IBM Cloud.

With this pricing model, you commit to spend a certain amount and receive discounts across the entire platform. You are billed monthly based on your usage, and unlike a subscription, you continue to receive a discount even after you reach your committed amount. For more information, see [Pay as you go with Committed Use pricing model](https://cloud.ibm.com/docs/account?topic=account-accountscommitment-model).







 December 2020 



 18 December 2020 

Managing features and other updates to IBM Cloud Shell
: Account owners or users with Cloud Shell administrator access can enable or disable Cloud Shell features for an account. The available features in this release are File upload and download and Web preview. The feature settings apply only to the enabled Cloud Shell locations. For more information, see [Enabling or disabling Cloud Shell features for an account](https://cloud.ibm.com/docs/account?topic=account-shell-settingsshell-features-enable).
Windows File System 

Crawl documents that are stored in a Microsoft Windows file system.

IBM Cloud Pak for Data

IBM Cloud Pak for Data only

This information applies only to installed deployments.



 What documents are crawled 



* Only documents that are supported by Discovery in your file path are crawled; all others are ignored. For more information, see [Supported file types](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collectionssupportedfiletypes).
* Document-level security is supported. When this option is enabled, your users can crawl and query the same content that they can access when they access the file system directly.
* When a source is recrawled, new documents are added, updated documents are modified to the current version, and deleted documents are deleted from the collection's index.
* All Discovery data source connectors are read-only. Regardless of the permissions that are granted to the crawl account, Discovery never writes, updates, or deletes any content in the original data source.







 Data source requirements 

In addition to the [data source requirements](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collection-typesrequirements) for all installed deployments, your Windows File System data source must meet the following requirements:



* The connector supports Microsoft Windows Server 2012 R2, 2016, 2019, and 2022.
* The remote agent server and the file servers to be crawled must belong to the same Windows domain. The crawler can gather access control list (ACL) data from a single Windows domain only.



Support for Microsoft Windows Server 2022 was added with the 4.6 release. Starting with the 4.7 release, you can secure traffic that is sent between the Windows Agent service and its crawler by enabling support for the transport layer security (TLS) protocol.





 Prerequisite steps 



* If you want to enable document-level security, you must take some steps to set it up. For more information, see [Supporting document-level security](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collection-typesconfiguredls).
You can see from the first line that standard JavaScript objects can be used in your code and sent to IBM Cloudant with no conversion, as they turn into JSON natively in JavaScript.

Writing a document is simply a matter of calling db.insert, which maps to a PUT/POST API call or to _bulk_docs.

To summarize, the official libraries for IBM Cloudant are Java™, Python, and Nodejs. They are thin wrappers around the IBM Cloudant HTTP API - so it's worth understanding the underlying API to understand all the parameters.

The libraries handle two things for you, which is useful:

Authentication
: Exchanging your keys for tokens, whether it be legacy authentication or IAM.

Retry logic
: The libraries can be configured to retry API calls that exceeded your provisioned capacity. If configured this way, they pause and reattempt the API call multiple times with exponential back-off.

Retrying such API calls is sensible if you have a temporary and unexpected elevation in traffic. If you are routinely exceeding your provisioned capacity, no amount of retrying gets the database work done - you need more capacity!

That's the end of this part. The next part is called Querying.







 Querying video 

Learn the different ways to query data in IBM Cloudant.



* Querying video script

Welcome to the Introduction to IBM Cloudant course, an 17-part video series that gives you an overview of the IBM Cloudant database-as-a-service.

This video is part 10 - Querying.

So far we performed CRUD (create, retrieve, update, and delete) operations from the command line, the dashboard, and from code. These operations use the document's _id:



* Fetch document by _id.
* Update document whose _id = 'x'.
* Delete document whose _id = 'x'.
* Get documents in the _id range 'a' to 'z'.



These operations are the building blocks of a database, but they get you only so far. What if you need to return a subset of documents that match on fields within the document? A person's birth date?
Improved query limits for Advanced and Premium plans : [Query expansion](https://cloud.ibm.com/docs/discovery?topic=discovery-query-conceptsquery-expansion) limits are increased for Advanced and Premium plans to 5,000 query expansions and 25,000 total terms. See [Discovery pricing plans](https://cloud.ibm.com/docs/discovery?topic=discovery-discovery-pricing-plans) for details.





 28 February 2018 

Deprecated AlchemyLanguage enrichments : AlchemyLanguage enrichments are deprecated, effective 1 March 2018.





 23 February 2018 

New document similarity query feature : Added the ability to query by document similarity. You can query for similar documents by document ids, and optionally further refine the similarity by specifying fields. See [Document similarity](https://cloud.ibm.com/docs/discovery?topic=discovery-query-conceptsdoc-similarity) for more information.

Improved highlight parameter : The [highlight parameter](https://cloud.ibm.com/docs/discovery?topic=discovery-query-parametershighlight) in query results is enhanced. Query results return complete sentences, ordered by their score.





 21 February 2018 

Fixed PDF file type : Previously, when ingesting PDF documents, the file_type returned when ingestion notices were queried, in the extracted_metadata object, and from the document details API was html. This is no longer the case. The file_type returned is now pdf.





 26 January 2018 

New Korean and Spanish accessibility in Discovery News : Added the ability to access Korean and Spanish collections to the [IBM Watson™ Discovery News](https://cloud.ibm.com/docs/discovery?topic=discovery-watson-discovery-news) tile in the tooling. Previously, these collections could only be queried via the API.





 23 January 2018 

New query expansion : Added the ability to expand the scope of a query - for example, you can expand a query for "car" to include "automobile" and "motor vehicle".
You can view the number of documents stored and the total amount of storage used by either using the [Environments](https://cloud.ibm.com/apidocs/discoveryget-environment-info) or [Collections](https://cloud.ibm.com/apidocs/discoveryget-collection-details) API, or by using the tooling. If your documents are, on average, larger than 100KB on disk, you will hit the storage limit of a plan before the maximum document limit. If you perform [document segmentation](https://cloud.ibm.com/docs/discovery?topic=discovery-configservicedoc-segmentation) on your documents, each segment counts as a separate document.

** The [IBM Watson™ Natural Language Understanding enrichments](https://cloud.ibm.com/docs/discovery?topic=discovery-configserviceadding-enrichments) are: Entity Extraction, Sentiment Analysis, Category Classification, Concept Tagging, Keyword Extraction, Relation Extraction, Emotion Analysis, Element Classification, and Semantic Role Extraction. Only the first 50,000 characters of each document are enriched.

*** Element Classification is an enrichment that parses through governing documents to convert, identify, and classify elements of importance. It uses Natural Language Processing to extract the following elements from PDF documents: party (who it refers to), nature (type of element), and category (specific class).

To take advantage of the new string environment sizing values (LT, XS, S, MS, M, ML, L, XL, XXL, XXXL), use the version date of 2018-08-01 or later in your API request when you create an environment. Before version 2018-08-01, the value was an integer.





 Premium 

Premium plans offer developers and organizations a single tenant instance of one or more Watson services for better isolation and security. These plans offer compute-level isolation on the existing shared platform, as well as end-to-end encrypted data while in transit and at rest.

For more information, or to purchase a Premium plan, contact [Sales](https://ibm.biz/contact-wdc-premium).





 Additional information
All requests and responses are made over HTTPS and all requests support the use of hash-based integrity checks using a Content-MD5 header. If the provided MD5 hash does not match the checksum computed by the storage service, the object is discarded and an error is returned. All GET and HEAD requests made to objects return an Etag value with the MD5 hash of the object to ensure integrity on the client side.

Developers use APIs to interact with their object storage. IBM Cloud Object Storage supports a subset of the S3 API for reading and writing data, as well as for bucket configuration. Additionally, there is a Object Storage Resource Configuration API for reading and configuring bucket metadata. Software development kits (SDKs) are available for the Python, Java, Go, and the Node.js framework. A plug-in is available for the [IBM Cloud Command Line Interface](https://cloud.ibm.com/docs/cli?topic=cli-getting-started).

The [IBM Cloud console](https://cloud.ibm.com/) provides a user interface for most operations and configuration as well.



 Cloud Object Storage on IBM Cloud Satellite 

Workloads that require object storage on-premise, or in a geographic location not supported by IBM Cloud data centers, can make use of IBM Cloud Satellite. For more information, see [the documentation](https://cloud.ibm.com/docs/satellite).





 Cloud Object Storage IaaS (Legacy) 

Legacy IaaS users are encouraged to migrate workloads to the IAM-enabled service. IaaS users can reference this documentation, but any features that relate to cloud services (IAM, Key Protect, Activity Tracking, Metrics Monitoring, Notifications, etc) are not supported for COS IaaS.





 Next Steps 

Documentation on the best way to [get started](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-getting-started-cloud-object-storage) provides support to provision accounts, to create buckets, to upload objects, and to use a reference of common operations through API interactions.
In addition to the IBM Cloud documentation site, there is a wide range of information about the underlying Db2 engine functionality in the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSFMBX/com.ibm.swg.im.dashdb.kc.doc/welcome.html). Updates to the service are posted on our [What's New](https://www.ibm.com/support/pages/whats-new-ibm-db2-cloud) page.

You can find pricing information and deploy a Db2 on Cloud instance through the IBM Cloud [catalog](https://cloud.ibm.com/catalog/services/db2) page for IBM Cloud. To learn more, contact [IBM Sales](https://www.ibm.com/contact/us/en/).





 Where can I find help for a problem that I'm having? 

For information about posting questions on a forum or opening a support ticket, see [Help & support](https://cloud.ibm.com/docs/Db2onCloud?topic=Db2onCloud-help_support).

Only community support is available for the free Lite plan.
[IBM Cloud Databases for etcd](https://cloud.ibm.com/docs/databases-for-etcd) IBM Cloud Databases for etcd is an enterprise-ready and fully managed etcd service that is built with native integration into the IBM Cloud. [Hyper Protect Crypto Services integration](https://cloud.ibm.com/docs/databases-for-etcd?topic=databases-for-etcd-hpcs) 
 [IBM Cloud Databases for MongoDB](https://cloud.ibm.com/docs/databases-for-mongodb) IBM Cloud Databases for MongoDB is an enterprise-ready and fully managed MongoDB service that is built with native integration into the IBM Cloud. [Hyper Protect Crypto Services integration](https://cloud.ibm.com/docs/databases-for-mongodb?topic=databases-for-mongodb-hpcs) 
 [IBM Cloud Databases for PostgreSQL](https://cloud.ibm.com/docs/databases-for-postgresql) IBM Cloud Databases for PostgreSQL is an enterprise-ready and fully managed PostgreSQL service that is built with native integration into the IBM Cloud. [Hyper Protect Crypto Services integration](https://cloud.ibm.com/docs/databases-for-postgresql?topic=databases-for-postgresql-hpcs) 
 [IBM Cloud Databases for Redis](https://cloud.ibm.com/docs/databases-for-redis) IBM Cloud Databases for Redis is an open source, in-memory key value store designed for the modern application stack. With Databases for Redis, you can use counters, queues, lists, and HyperLogLogs to handle complex data issues simply. [Hyper Protect Crypto Services integration](https://cloud.ibm.com/docs/databases-for-redis?topic=databases-for-redis-hpcs)
The supported media types are images, videos in MP4 or WebM file format, and videos hosted on YouTube or Vimeo.

Some examples of effective media include an introductory walkthrough of your product, an explanation of what your product is and why users might want to use it, or a comparison of certain features.





 Providing your documentation link 

Your documentation link is used to direct users to your product's warranted documentation.







 Defining your catalog entry by using the API 

You can programmatically provide certain details about your product that are displayed in the IBM Cloud® catalog by calling the Partner Center Sell API.



 Providing your company or product logo 

You can programmatically provide your company or product logo by calling the [Partner Center Sell API](https://cloud.ibm.com/apidocs/partner-center-sellupdate-catalog) as shown in the following sample request. In the example the URL of the company or product logo is https://www.ibm.com/contact/us/en/images/bee.svg:

curl --request PATCH --url https://product-
lifecycle.api.cloud.ibm.com/openapi/v1/products/9fab83da-98cb-4f18-
a7ba-b6f0435c9673/catalog --header 'Authorization: Bearer TOKEN' --header 'Content-Type: application/json' --data '{
"iconUrl": "https://www.ibm.com/contact/us/en/images/bee.svg"
}'

Make sure that your logo image is an SVG or PNG file that's a square image roughly sized at 32 x 32 pixels and doesn't appear blurry or pixelated.





 Providing a short description of your product 

You can programmatically provide a short description of your product by calling the [Partner Center Sell API](https://cloud.ibm.com/apidocs/partner-center-sellupdate-catalog) as shown in the following sample request. The example adds the Create and deploy a virtual server with ease by using a custom image. description to the product:
[IBM Developer<br><br>Visit IBM Developer for technical articles, code patterns, tutorials, and more.<br><br>![112-Developer-tools.svg](https://cloud.ibm.com/media/docs/images/homepage/112-Developer-tools.svg)](https://developer.ibm.com/depmodels/cloud/)[Architecture Center<br><br>Discover the architecture references available for this product.<br><br>![190-Application-Platform.svg](https://cloud.ibm.com/media/docs/images/homepage/190-Application-Platform.svg)](https://www.ibm.com/cloud/architecture)[IBM Training<br><br>Build your skills with IBM Cloud training.<br><br>![83-Elements-of-the-cloud.svg](https://cloud.ibm.com/media/docs/images/homepage/83-Elements-of-the-cloud.svg)](https://www.ibm.com/training/cloud)[Community<br><br>Join the IBM Cloud community and collaborate with others.<br><br>![35-Connectivity.svg](https://cloud.ibm.com/media/docs/images/homepage/35-Connectivity.svg)](https://community.ibm.com/community/user/publiccloud/home)

 Latest updates 

[View more](https://cloud.ibm.com/docs/databases-for-redis?topic=databases-for-redis-redis-relnotes)19 October 2022 | Deploying and Connecting an IBM Cloud® Databases Instance Tutorial

This tutorial guides you through the process of deploying a Cloud Databases instance and connecting it to a web front end by creating a webpage that allows visitors to input a word and its definition. These values are then stored in a database running on Cloud Databases. You install the database infrastructure by using Terraform and your web application uses the popular Express framework. The application can then be run locally, or by using Docker.
<-- </section "id="section-aggfilter-syntax" "> --><-- <section "id="section-aggfilter-example" "> --> Example The following example filters the matching document set to include only documents that mention IBM. filter(enriched_text.entities.text:IBM)
When combined with other aggregations, filters the matching documents set to include only those documents that meet the condition you specify. {
"query":"brake",
"aggregation": "filter(VEH_SPEED>50).term(field:STATE,count:3).term(field:CITY,count:2)"
}
The query response shows cities where incidents happen that involve the brakes and the vehicle speed is over 50. {
"matching_results": 9064,
"retrieval_details": {
"document_retrieval_strategy": "untrained"
},
"aggregations":
{
"type": "filter",
"match": "VEH_SPEED>50",
"matching_results": 1075,
"aggregations":
{
"type": "term",
"field": "STATE",
"count": 3,
"results":
{
"key": "CA",
"matching_results": 176,
"aggregations":
{
"type": "term",
"field": "CITY",
"count": 2,
"results":
{
"key": "FONTANA",
"matching_results": 6
},
{
"key": "ALTA LOMA",
"matching_results": 5
}
]
}
] ! ! ! !
}
Show more
New release for Premium instances
: This release is available for Premium instances of Discovery on IBM Cloud created after 16 July 2020. For Premium instances created before that date and for all Lite and Advanced plans, see [Getting started with Discovery](https://cloud.ibm.com/docs/discovery?topic=discovery-getting-started).

Change to IBM Cloud Premium
: The Premium plan is now generally available.

New Project-based interface
: The project-based UI includes configurations optimized for three common use cases: Document Retrieval, Conversational Search, and Content Mining. For more information, see [Creating projects](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-projects).

New Content Mining app
: This entirely new capability of Watson Discovery allows you to find insights in your data when you may not even know the question to ask. The powerful correlation tooling will help you unlock value from large unstructured data sets. For details, see [Analyzing your data with the Content Mining application](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-contentminerapp).

New tables as answers
: Snippets of text aren't helpful if they are found in a table, so Discovery instead returns a formatted table as an answer if your question is best answered by a table. For more information, see [Table retrieval](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-query-parameterstable_retrieval).

New dynamic faceted search feature
: Underspecified queries are common. Dynamic Faceted Search automatically categorizes your search results into intelligence facets without training by understanding how they are used in the sentences. See [Facets in Document retrieval projects](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-facetsfacetdr).

New reusable components
: You no longer need to build a Discovery application from scratch. We now ship out of the box with reusable, open source, React components.