vania-janet/multiturn-rag-retrieval-data huggingface dataset:
[IBM Cloud Databases for Redis](https://cloud.ibm.com/docs/databases-for-redis?topic=databases-for-redis-getting-started) databases-for-redis-group [Location-based events](https://cloud.ibm.com/docs/databases-for-redis?topic=databases-for-redis-activity-tracker) 
 [IBM Cloud Databases for etcd](https://cloud.ibm.com/docs/databases-for-etcd?topic=databases-for-etcd-getting-started) databases-for-etcd-group [Location-based events](https://cloud.ibm.com/docs/databases-for-etcd?topic=databases-for-etcd-activity-tracker) 
 [IBM Cloud Databases for Elasticsearch](https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-getting-started) databases-for-elasticsearch-group [Location-based events](https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-activity-tracker) 
 [IBM Cloud® Messages for RabbitMQ](https://cloud.ibm.com/docs/messages-for-rabbitmq?topic=messages-for-rabbitmq-getting-started) messages-for-rabbitmq-group [Location-based events](https://cloud.ibm.com/docs/messages-for-rabbitmq?topic=messages-for-rabbitmq-activity-tracker) 
 [IBM Cloud Databases for MongoDB](https://cloud.ibm.com/docs/databases-for-mongodb) databases-for-mongodb-group [Location-based events](https://cloud.ibm.com/docs/databases-for-mongodb?topic=databases-for-mongodb-activity-tracker)
Translating documents 

IBM is announcing the deprecation of the IBM Watson® Language Translator service for IBM Cloud® in all regions. As of 10 June 2023, the Language Translator tile will be removed from the IBM Cloud Platform for new customers; only existing customers will be able to access the product. As of 10 June 2024, the service will reach its End of Support date. As of 10 December 2024, the service will be withdrawn entirely and will no longer be available to any customers.

You can use the IBM Watson® Language Translator service to translate files from one language to another while preserving the original document format. The service supports translation of many file formats, including Microsoft® Office®, Open Office, subtitles, and many other common formats such as HTML, JSON, XML, and Adobe® PDF.



 Before you begin 

Make sure that you have the following information and meet the following requirements:



* You need your Language Translator service credentials (apikey and url).
* The document that you want to translate must not exceed the following size limits:



* 2 MB for service instances on the Lite plan
* 20 MB for service instances on the Standard plan
* 50 MB for service instances on the Advanced plan
* 150 MB for service instances on the Premium plan



* The document must be in one of the supported file formats. Use the correct file extension for the format of the document or specify the content type (MIME type) of the format with the request. For more information, see [Supported file formats](https://cloud.ibm.com/docs/language-translator?topic=language-translator-document-translator-tutorialsupported-file-formats).
* The source and target languages must be among the [List of supported languages](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelslist-languages-supported).
* The service correctly translates from and to bidirectional languages that are written left-to-right and right-to-left (for example, Arabic, Hebrew, and Urdu).



This tutorial walks you through translating documents from the command line with curl. You can also use the Watson SDKs to translate documents with a number of programming languages.
Analyzing data on demand with the Analyze API 

Use the Analyze API to process text documents through the enrichment pipeline of the Discovery service without storing any data from the source documents.

The Analyze API is supported by Enterprise plan deployments and installed deployments only.

This approach is ideal for business automation purposes. For example, if you want to classify emails, you can use the Analyze API to synchronously call Discovery to get a classification of the email. Then, you can use the output of that classification in your business logic.

The Analyze API supports JSON documents only.

When you analyze a document with the API, you indicate how you want the document to be processed by specifying the collection to associate with the analysis. The document is not stored in the collection. Instead, the configuration settings of the collection are applied to the document. For example, if you want to find entity references in a document, run the Analyze API against a collection where the Entities enrichment is applied. The resulting document analysis identifies any entity mentions in the document.

Submit a request for analysis against only one collection that is configured with the enrichments that you want to use to analyze your document on demand. Remember, the documents in the collection are not significant. It is the enrichments that are defined for the collection that matter. If you submit requests to several collections, then several models are initiated at the same time, which can cause request failures.

The following enrichments are supported in the Analyze API:



* [Advanced rules models](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain-mladvanced-rules)
* [Contracts](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-contracts-schema)
* [Custom entities](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-entity-extractor)
* [Dictionary](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain-dictionary)
Migration overview 

The [IBM® Cloudant® for IBM Cloud®](https://www.ibm.com/cloud/cloudant) database-as-a-service offering is a JSON document store that runs on multi-tenant clusters. The service is available with a choice of geographical locations with predictable costs, scalability, and a service-level agreement (SLA).

You can migrate to an IBM Cloudant Lite or Standard plan instance on IBM Cloud from one of the following plans:



Table 1. IBM Cloudant migration paths

 Plan Description 

 IBM Cloudant Enterprise Dedicated, single-tenant clusters 
 Apache CouchDB The self-hosted, open source database on which IBM Cloudant is based. 





 Benefits of the IBM Cloudant Lite and Standard plans 

With the Standard plan, you can reserve throughput capacity for your database service, that is, to specify how much throughput your application's database is going to need to handle demand. The Standard plan also charges for the amount of storage you use. Capacity is measured with the following metrics:



Table 2. Capacity metrics

 Metric Description 

 Reads per second The rate when simple document fetches are performed, for example, retrieving a document by its _id, or querying against a partitioned database that uses a partition key. 
 Writes per second The rate when data is written to the database. API calls dealing with document creation, update, or deletion count as "writes". 
 Global Queries per second The rate when the database is queried by global indices, typically by accessing the _find endpoint, secondary MapReduce, or search indices. 
 Storage The amount of disk space occupied by your JSON data, attachments, and secondary indices. 



As an example, the Lite plan offers 20 reads per second, 10 writes per second, 5 global queries per second, and 1 GB of storage for free. This plan is ideal when you're evaluating the product and during product development. When your application goes into QA or production, switch to the Standard plan to scale the instance. The Standard plan's smallest capacity has 100 reads per second, 50 writes per second, 5 global queries per second, and 20 GB of storage for USD$76.65 per month.
What types of off-chain databases are supported with the IBM Blockchain Platform? 

As a best practice it is recommended that you do not query the entire blockchain ledger for the purpose of aggregation or reporting. If you want to build a dashboard or collect large amounts of data as part of your application, you can query an off chain database that replicates the data from your blockchain network. This allows you to understand the data on the blockchain without degrading the performance of your network or disrupting transactions.

You can use block or smart contract events from your application to write transaction data to an off-chain database or analytics engine. For each block received, the block listener application would iterate through the block transactions and build a data store by using the key/value writes from each valid transaction's read-write set. The [Peer channel-based event services](https://hyperledger-fabric.readthedocs.io/en/release-2.2/peer_event_services.htmlpeer-channel-based-event-services) provide replayable events to ensure the integrity of downstream data stores. For an example of how you can use an event listener to write data to an external database, see the [Off chain data sample](https://github.com/hyperledger/fabric-samples/tree/release-1.4/off_chain_data) in the Fabric samples.

Blockchain solutions can use any RDBMS or NoSQL DB such as IBM Cloudant for offchain data storage. Hyperledger Fabric does not govern, interact with, or manage off-chain databases. In most cases, the off-chain database is used for reference data and non-transactional data. IBM has successfully built blockchain products and solution accelerators with Hyperledger Fabric and NoSQL databases such as OrientDB.





 If service discovery is on, will an endorsement request be routed to any peer on the network? 

It depends on whether your endorsement policy is set to "ANY", in which any peer can sign an endorsement request, or whether the policy is bound directly to an organization's peers. The service discovery information provided by the peer supplies two pieces of information, Layouts and EndorsersByGroup.
[35-Connectivity.svg](https://cloud.ibm.com/media/docs/images/homepage/35-Connectivity.svg)](https://community.ibm.com/community/user/publiccloud/home)

 Latest updates 

[View more](https://cloud.ibm.com/docs/databases-for-enterprisedb?topic=databases-for-enterprisedb-enterprisedb-relnotes)19 October 2022 | Deploying and Connecting an IBM Cloud® Databases Instance Tutorial

This tutorial guides you through the process of deploying a Cloud Databases instance and connecting it to a web front end by creating a webpage that allows visitors to input a word and its definition. These values are then stored in a database running on Cloud Databases. You install the database infrastructure by using Terraform and your web application uses the popular Express framework. The application can then be run locally, or by using Docker.

11 October 2022 | Protecting IBM Cloud® Databases for EnterpriseDB resources with context-based restrictions

Context-based restrictions (CBR) give account owners and administrators the ability to define and enforce access restrictions for IBM Cloud® resources based on the context of access requests. Access to Cloud Databases resources can be controlled with CBR and identity and access management (IAM) policies.

29 March 2022 | IBM Cloud® Databases for EnterpriseDB supports synchronous replication

Available security and compliance goals updated.
For more information, see [Deploying and Connecting a Cloud Databases Instance](https://cloud.ibm.com/docs/databases-for-redis?topic=cloud-databases-create-instance-tutorial).





 11 October 2022 

Protecting IBM Cloud® Databases for Redis resources with context-based restrictions
: Context-based restrictions (CBR) give account owners and administrators the ability to define and enforce access restrictions for IBM Cloud® resources based on the context of access requests. Access to Cloud Databases resources can be controlled with CBR and identity and access management (IAM) policies. For more information, see [Protecting Cloud Databases resources with context-based restrictions](https://cloud.ibm.com/docs/databases-for-redis?topic=cloud-databases-cbr&interface=ui).





 25 March 2022 

IBM Cloud® Databases for Redis Security Goals Updated
: Available security and compliance goals updated. See updated goals [here](https://cloud.ibm.com/docs/databases-for-redis?topic=databases-for-redis-manage-security-compliance).





 09 November 2021 

IBM Cloud® Databases for Redis 4.0 End of Life in May 2022
: After May 27, 2022, all IBM Cloud® Databases for Redis instances on version 4.0 that are still active will be upgraded in-place to the next major version, Redis 5.0. See blog post announcement [here](https://www.ibm.com/cloud/blog/announcements/ibm-cloud-databases-for-redis-4-end-of-life-in-march-2022).





 30 June 2021 

General Availability of IBM Cloud® Databases for Redis support for IBM Cloud Databases enabled by IBM Cloud Satellite.
: A distributed cloud provides consistent security and services across environments, centralized workload visibility, reduced latency, easier compliance and higher application development velocity. See blog post announcement [here](https://www.ibm.com/cloud/blog/announcements/deploy-managed-cloud-native-databases-anywhere-with-ibm-cloud-satellite).





 13 April 2020 

IBM Cloud® Databases for Redis autoscaling
IBM Cloud® Databases for Redis docs 

Redis is a blazingly fast, in-memory data structure store.

 Developer tools 

[API & SDK reference](https://cloud.ibm.com/apidocs/cloud-databases-api/cloud-databases-api-v5introduction)[CLI reference](https://cloud.ibm.com/docs/databases-cli-plugin)[Terraform reference](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/database)

 Recommended content 

[Getting Started A brief tutorial to introduce using an IBM Cloud® Databases for Redis deployment.](https://cloud.ibm.com/docs/databases-for-redis?topic=databases-for-redis-getting-started)[Setting the Admin Password The IBM Cloud® Databases for Redis service is provisioned with a admin user. You must set the admin password before you use it to connect.](https://cloud.ibm.com/docs/databases-for-redis?topic=databases-for-redis-admin-password&interface=ui)[Getting Connection Strings To connect to IBM Cloud® Databases for Redis, you need some users and connection strings.](https://cloud.ibm.com/docs/databases-for-redis?topic=databases-for-redis-connection-strings&interface=cli)

 Learn more 

[IBM Developer<br><br>Visit IBM Developer for technical articles, code patterns, tutorials, and more.<br><br>![112-Developer-tools.svg](https://cloud.ibm.com/media/docs/images/homepage/112-Developer-tools.svg)](https://developer.ibm.com/depmodels/cloud/)[Architecture Center<br><br>Discover the architecture references available for this product.<br><br>!
[35-Connectivity.svg](https://cloud.ibm.com/media/docs/images/homepage/35-Connectivity.svg)](https://community.ibm.com/community/user/publiccloud/home)

 Latest updates 

[View more](https://cloud.ibm.com/docs/databases-for-elasticsearch?topic=databases-for-elasticsearch-elasticsearch-relnotes)19 October 2022 | Deploying and Connecting an IBM Cloud® Databases Instance Tutorial

This tutorial guides you through the process of deploying a Cloud Databases instance and connecting it to a web front end by creating a webpage that allows visitors to input a word and its definition. These values are then stored in a database running on Cloud Databases. You install the database infrastructure by using Terraform and your web application uses the popular Express framework. The application can then be run locally, or by using Docker.

11 October 2022 | Protecting IBM Cloud® Databases for Elasticsearch resources with context-based restrictions

Context-based restrictions (CBR) give account owners and administrators the ability to define and enforce access restrictions for IBM Cloud® resources based on the context of access requests. Access to Cloud Databases resources can be controlled with CBR and identity and access management (IAM) policies.

25 March 2022 | IBM Cloud® Databases for Elasticsearch Security Goals Updated

Available security and compliance goals updated.
* [Contracts](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-contracts-schema)
* [Custom entities](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-entity-extractor)
* [Dictionary](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain-dictionary)
* [Document classifier](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-cm-doc-classifier)
* [Entities (NLP)](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-nlunlu-entities)
* [Keywords (NLP)](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-nlunlu-keywords)
* [Machine learning and Watson Explorer Content Analytics Studio models](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain-ml)
* [Regular expressions](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain-regex)
* [Patterns](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain-pattern) (Enterprise plan only)
* [Sentiment of documents](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-nlunlu-sentiment)
* [Table understanding](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-understanding_tables)[1]
* [Text classifier](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain-classifier)



For the complete list of the enrichments that are supported in each language, see [Language support](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-language-support).
" digital and <em>hybrid</em> <em>cloud</em> transformation.</p>n<p>URL: http://www.ibm.com/press/us/en/pressrelease/50837.wss</p>nnnn</body></html>",
" continuity software for nenterprise data centers and <em>cloud</em> infrastructure. Adding these ncapabilities"
] ! !
}
}
Show more
<-- </section "id="section-highlight" "> --><-- <section "id="section-deduplicate" "> --> deduplicate A beta capability that excludes duplicate documents from IBM Watson™ Discovery News collection query results based on the title field. See Excluding duplicate documents from query results]] ! .<-- <section "id="section-deduplicate_field" "> --> deduplicate.field A beta capability that excludes duplicate documents from your query results based on the specified {field}. See Excluding duplicate documents from query results]] ! ! .<-- </section "id="section-deduplicate_field" "> --><-- <section "id="section-deduplication" "> --> Excluding duplicate documents from query results If you are querying the IBM Watson™ Discovery News collection, or your private data collection contains multiple identical (or near-identical) documents, you can exclude most of them from your query results using document deduplication.Document deduplication is currently supported only as a beta capability. For more information, see Beta features]] in the release notes. This beta feature is currently supported in English only. For details, see Language support]] ! ! ! ! ! .Each query is deduplicated independently, so deduplication across offsets is not supported.Deduplication is performed after passages are extracted and aggregations are calculated, so if you include the passages parameter in your query, passages are returned from all documents in the query results, before deduplication.
Databases <br><br> * [Hyper Protect DBaaS for MongoDB](https://cloud.ibm.com/docs/hyper-protect-dbaas-for-mongodb?topic=hyper-protect-dbaas-for-mongodb-install-dbaas-cli-plugin)<br> * [Hyper Protect DBaaS for PostgreSQL](https://cloud.ibm.com/docs/hyper-protect-dbaas-for-postgresql?topic=hyper-protect-dbaas-for-postgresql-install-dbaas-cli-plugin)<br><br><br> 







 Application programming interfaces 

IBM Cloud offers a rich set of [APIs](https://cloud.ibm.com/docs?tab=api-docs) for working with your resources. The following table provides links for the APIs that can be used for each service in the reference architectures.



Table 3. API information for services in reference architectures

 Category VPC reference architecture Satellite reference architecture Optional for both 

 Core <br><br> * [VPC infrastructure services](https://cloud.ibm.com/docs/vpc?topic=vpc-set-up-environment&interface=api)[7]<br><br><br> <br><br> * [Satellite](https://containers.cloud.ibm.com/global/swagger-global-api//satellite-cluster)<br><br><br>
Query overview 

IBM Watson® Discovery offers powerful content search capabilities through search queries.

To retrieve data from Discovery after it is ingested, indexed, and enriched, submit a query.

As data is added to Discovery, a representation of each file is stored in the index as a JSON-formatted document. Enrichments that are applied to your collections identify meaningful information in the data and store it in new fields in these documents. To search your data, submit a query to return the most relevant documents and extract the information you're looking for.



 Query types 

Discovery accepts one of the following supported query types:

Query
: Finds documents with values of interest in specific fields in your documents. Queries of this type use Discovery Query Language syntax to define the search criteria.

Parameter name: query

Natural Language Query (NLQ)
: Finds answers to queries that are written in natural language. NLQ requests accept a text string value.

Parameter name: natural_language_query

Along with the query that you specify by using one of the supported query types, you can include one or both of the following parameters. The values for these parameters are also specified by using the Discovery Query Language (DQL) syntax:



* filter
* aggregation



For more information about the Discovery Query Language, see [DQL overview](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-query-dql-overview).

Queries that are submitted from the product user interface are natural language queries. A few other supported parameters are specified and given default values based on the project type in use. For more information, see [Default query settings](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-query-defaults).

Discovery does not log query request data. You cannot opt in to request logging.





 Choosing the right query type 

The following table summarizes the capabilities that are supported for each query type. Use it to help you determine which type of query to submit.



Query types comparison
This table has row and column headers. The row headers identify query types.
PNG, TIFF, and JPEG images embedded in PDF, Word, PowerPoint, and Excel files are also scanned, and any text is extracted.

View the following table to see the objects that a data source can crawl and which data sources support crawling new and modified documents during a refresh:



Table 1. Data sources that support crawling new and modified documents during refresh and objects that can be crawled

 Data source Crawls new and modified documents during refresh? Compatible objects that can be crawled 

 Box (Application level access) No Files, folders 
 Box (Enterprise level access) Yes Files, folders 
 Salesforce Yes Any default and custom objects that you have access to, accounts, contacts, cases, contracts, knowledge articles, attachments 
 Microsoft SharePoint Online Yes SiteCollections, websites, lists, list items, document libraries, custom metadata, list item attachments 
 Microsoft SharePoint OnPrem Yes SiteCollections, websites, lists, list items, document libraries, custom metadata, list item attachments 
 Web Crawl No Websites, website subdirectories 
 IBM Cloud Object Storage Yes Buckets, files 







 Available data sources 

You can use Discovery to crawl from the following data sources:



* [Box](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesconnectbox)
* [Salesforce](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesconnectsf)
* [Microsoft SharePoint Online](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesconnectsp)
* [Microsoft SharePoint OnPrem](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesconnectsp_op)
* [Web Crawl](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesconnectwebcrawl)
* [IBM Cloud Object Storage](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesconnectcos)



You can connect to a data source using the Discovery tooling or the API.