vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Connection properties (except for the CRN) can be specified as part of the URL, separated by &, or through the Java connection properties object. The following properties are supported:



* password (required): IBM Cloud API key for running the queries. The property name must start with a lowercase letter.
* user (optional): A username is not required and is ignored if given. It must start with a lowercase letter.
* targetcosurl (optional, but usually needed): Cloud Object Storage URI in SQL query style, where the results are stored. If this property is not specified, you cannot run queries that return a JDBC result set. The JDBC connection can still be used to retrieve database metadata and run DDL and [ETL-type statements](https://cloud.ibm.com/docs/sql-query?topic=sql-query-jdbcetl-type-statements). It must start with a lowercase letter.
* loggerFile (optional, default none): file to write driver logs to.
* loggerLevel (optional, default set by JDK: java.util.logging level for the driver. JDK default is usually INFO.



* DEBUG/FINER or TRACE/FINEST are the most useful values.



* filterType (optional, default none):



* Only tables are returned if filterType value is set to table.
* Only views are returned if filterType value is set to view.



* appendInto (optional, default true):



* If it is set to false, no INTO clause is appended, and results are not available through the driver. It is used with [ETL-type statements](https://cloud.ibm.com/docs/sql-query?topic=sql-query-jdbcetl-type-statements), where the INTO options are provided as part of the statement.









 Driver functions 

This driver is designed as a facade to JVM applications for easy access to Data Engine service. The driver uses the REST API of the service to run queries, stores the results in Cloud Object Storage, and makes these results accessible through JDBC interfaces.
Related references - sort item clause 

A sort item clause is referenced by the following clauses:



* [fullselect](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencefullselect)
* [windowSpec](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencewindowSpec)









 Relations 

A relation is a component of a simpleselect or a relation primary (which itself is a component of a relation). It is usually referred to as a table.

A relation is an entity that represents input data that is either stored physically on Cloud Object Storage or taken from virtual intermediate results. Such an intermediate result can be, for example, the result of a subselect, a values clause, a common table expression, or a table-valued function. Relations can be queried or joined with other relations.

Specifying the physical data stored on Cloud Object Storage as a relation is done by using the [externalTableSpec](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referenceexternalTableSpec) syntax. An example of a valid table URI is cos://us-geo/sql/orders.parquet, which references one of the sample tables that are provided by Data Engine out of the box.

Multiple relations can be composed by using join operators. The syntax for joining relations is defined by the following syntax diagrams.



 relation 





 joinClause 





 naturalJoinClause 

Relations can be joined by using several types of joins that are described in detail in section [joinType](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencejoinType).

Apart from the join type, the following two different types of joins exist:



* Joins that specify a join condition by using a booleanExpression or a USING clause.
* NATURAL joins that make an implicit guess on which columns to use for joining relations. Use natural joins carefully.







 relationPrimary 





 externalTableSpec
: Introduce new action, DELETE, on the /_iam_session endpoint, which invalidates the IAM session cookie.

Improvements
: Remove outdated dependencies.







 April 2018 



 26 April 2018 

The following changes were made in build 6909:

Improvements
: Improve compatibility with IAM.

http multipart requests
: Fix badarg error in the module that parses http multipart requests.





 25 April 2018 

The following changes were made in build 6900:

New! Support replication
: Support replication with IAM.

Validation
: Improve validation of password schemes.

State field
: In _scheduler/docs, set the value of the state field to crashing when the last event in the history was a crash.

_design/$DDOC/_rewrite and _design/$DDOC/_update
: Disallow _design/$DDOC/_rewrite and _design/$DDOC/_update endpoints with IAM.





 17 April 2018 

The following changes were made in build 6895:

Pluggable storage engine
: Fix a regression that is introduced with pluggable storage engine.





 6 April 2018 

Replaced queries parameter
: The queries parameter for performing multiple view queries in a single request is no longer accepted as a URL parameter for GET /{db}/_design/{ddoc}/_view/{view} or a request body parameter for POST /{db}/_design/{ddoc}/_view/{view}. The parameter was replaced with the endpoint POST /{db}/_design/{ddoc}/_view/{view}/queries and is supplied as a queries request body parameter. You can also make multiple queries with the following new endpoints:



* POST /{db}/_all_docs/queries
* POST /{db}/_design_docs/queries



Sending several queries to a view
To execute this task on an environment, follow the instructions for [Submitting a Service Request](https://cloud.ibm.com/docs/mas-ms?topic=mas-ms-sspsubmitting-a-service-request), and select one of the following options from the drop down list in Step 5.

![SSP-Reorg](https://cloud.ibm.com/docs-content/v1/content/bd177462727ef0a849ddcf8c842a6203b41f0625/mas-ms/images/SSP-Reorg-Chooser.png)

Runstats only operation to improve SQL performance. This option updates statistics in the system catalog about tables, view, and indexes. DB2’s query optimizer uses these statistics to improve query performance.

Offine reorganization of tablespaces along with runstats operation for the target asset to improve SQL performance and reclaim fragmented disk space.

The environment will be unavailable during the execution of this task, and depending on the size of the database, this task may take up to several hours complete.

Online reorganization of tablespaces along with runstats operation for the target asset to improve SQL performance and reclaim fragmented disk space. The environment is not stopped for this option.

Most cases require only the Runstats operation. Reorganization of tablespaces need only be run if a significant amount of data has been deleted from the database since the last tablespace reorganization.





 Run MAS MustGather and upload to ECuRep 

This task will collect application logs from the target environment and upload the logs to the ECuRep (IBM Support) server.

To execute this task on an environment, follow the instructions for [Submitting a Service Request](https://cloud.ibm.com/docs/mas-ms?topic=mas-ms-sspsubmitting-a-service-request), and fill-in the Support Case ID in Step 5.

!
couch_key_tree algorithm
: Rewrite the couch_key_tree algorithm to reduce its computational complexity and avoid calling stemming when unnecessary.

Allocation strategy
: Change the allocation strategy for the message queue for each important process, so it's not stored on the heap of that process.

Internal audit facility
: Improvements to internal audit facility.

Constant fields
: Any constant fields that are in the selector, and are part of the index. For example, {A: {$eq: 10}} is inserted into the sort list if they aren't already included. This method increases the chance that the best index is selected for the query, for example, index = [A, B], sort = [B], and selector = {A: 1}. The sort then becomes [A, B].

Only the fields that are in front of the current sort fields in the list are added.







 June 2018 



 29 June 2018 

The following changes were made in build 7051:

Forward compatibility clause
: Add forward compatibility clause for _stats disk format change.

Compatibility clause
: Add compatibility clause for attachment receiver to facilitate Erlang upgrade.

Audit facility
: Improvements to internal audit facility.





 12 June 2018 

The following changes were made in build 7014:

Query selector
: Remove the requirement to cover _id or _rev in the query selector in order to use a JSON index that explicitly contains them.







 May 2018 



 29 May 2018 

The following changes were made in build 6979:

Audit facility
: Improve internal audit facility.





 14 May 2018 

The following changes were made in build 6919:

New! Action, DELETE
: Introduce new action, DELETE, on the /_iam_session endpoint, which invalidates the IAM session cookie.

Improvements
: Remove outdated dependencies.







 April 2018 



 26 April 2018 

The following changes were made in build 6909:

Improvements
: Improve compatibility with IAM.

http multipart requests
: Fix badarg error in the module that parses http multipart requests.





 25 April 2018 

The following changes were made in build 6900:
: In architectures that are focused on container and microservices, you can use Secure to protect, monitor, and enhance forensic analysis of your pipeline and runtime components. [Learn more](https://cloud.ibm.com/docs/monitoring?topic=monitoring-getting-started-secure)





 19 November 2020 

Enhancements to Explore
: A new grouping editor helps you create and manage infrastructure groupings.

Dashboard ownership
: You can now transfer dashboard ownership to another user.

Dashboard enhancements
: The following dashboard enhancements are now available:



* The ability the dashboard menu to the sidebar.
* The ability to specify your preference for open or closed categories.







 27 July 2020 

Hosts overview
: The hosts overview provides a view of the performance and health of the physical hosts in your infrastructure.





 17 June 2020 

General availability of new dashboards
: New functionality to improve the editing experience and add new ways to visualize and consume data. Functional changes include:



* Addition of a form-based and PromQL method of building dashboards.
* Adding the ability to share dashboards with team members.
* Customizing time series names and exes values on the legend.
* The ability to configure multiple queries in a single panel.
* Adding the ability to create a unified view of both metrics and events.
* Availability of dashboard templates.
* The ability to map values to text.



Limited availability of PromQL support
: PromQL support for querying Prometheus metrics has been added for a subset of users.

Major functional changes
: The following major changes are made in this release:



* Topology maps are no longer available in dashboards. Topology maps can be accessed through the Explore option.
* My Dashboards are no longer available under Explore.
* The dashboard wizard has been removed. Use the dashboard templates in place of thd dashboard wizard.
* History and summary metrics have been deprecated.
* Ability to configure scope variables on a dashboard to quickly filter metrics.
* Addition of intelligent auto-completion and syntax highlighting.
Release notes for Discovery 

The release notes provide information about updates and improvements to IBM Watson™ Discovery since the previous release.



 Service API Versioning 

API requests require a version parameter that takes a date in the format version=YYYY-MM-DD. Whenever we change the API in a backwards-incompatible way, we release a new minor version of the API.

Send the version parameter with every API request. The service uses the API version for the date you specify, or the most recent version before that date. Don't default to the current date. Instead, specify a date that matches a version that is compatible with your app, and don't change it until your app is ready for a later version.

The current version is 2019-04-30.





 Discovery v1 deprecation announcement 

Watson Discovery v1 is being deprecated. Existing clients who use Watson Discovery v1 are asked to migrate to Watson Discovery v2 before the end-of-support date of 11 July 2023. End of Support means that no v1 instance will work on or after 11 July 2023. For more information about migration, see [Getting the most from Discovery](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-version-choose).





 25 January 2022 

FAQ extraction beta feature is deprecated : Support for the FAQ extraction beta feature was removed. The beta feature was available only for Web crawl connectors that were used from Watson Assistant Search skills. FAQ extraction will stop being supported in v1 Discovery service instances on 1 March 2022. As a consequence of the end of FAQ extraction support, the COVID-19 kit data collection will stop being supported also.

Known issue for training. You might encounter the following issue when you try to train your data collection:



* Error: If you are training your collection, you might see the message, Your query can not be run because you have exceeded the rate limit for your plan. Your current plan limits the number of complex queries you can run simultaneously, so you can fix this by upgrading or reducing the number of queries.
(https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/gs-sample-entities-facet.png)

Figure 15. Top Entities facet results

Notice that a Top Entities section is displayed. You can expand the entities and click one of them to filter the query results to show only those results in which the entity is mentioned. The Top Entities section is a built-in facet. It uses information that was added to the documents by the Entities enrichment.

You will add your own facet that uses the Keywords enrichment that you applied to the collection in a previous step.
3. On the Improvement tools panel, expand Customize display, and then click Facets.

Zoom

![Shows the expanded Customize display section in the Improvement tools pane.](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/gs-facet-sidebar.png)

Figure 16. Customize display options
4. Click New facet, and then click the From existing fields in a collection button.

Zoom

![Shows the options for the types of facets you can add.](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/gs-facet-add-options.png)

Figure 17. Facet options
5. Choose enriched_text.keywords.mentions.text, change the label to Keywords, and then click Apply.

Zoom

![Shows the facet panel where a keyword facet is being added.]
Blogs and Webinars 

Learn more about Discovery from these blog posts and recorded webinars.



 Blog Posts 



* [Improve your natural language query results from Watson Discovery](https://developer.ibm.com/blogs/improving-your-natural-language-query-results-from-watson-discovery/).
* [How Watson Discovery Can Be Used to Build Prediction Models for Movie Reviews](https://www.topcoder.com/blog/how-ibm-discovery-can-be-used-to-build-prediction-models-for-movie-reviews/)
* [Using Watson Discovery’s Document Segmentation Feature](https://medium.com/ibm-watson/using-ibm-watson-discoverys-new-document-segmentation-feature-7a58b44d32c2)







 Webinars 



* [Building with Watson: How to build a chatbot with Assistant and Discovery](https://www.youtube.com/watch?v=0zMM0lfIdnI&list=PLZDyxLlNKRY_GJskIreh9sQgExJ4z8oZO&index=7&t=0s)
* [Building with Watson: How to connect the dots in your domain specific content](https://www.youtube.com/watch?v=iZcO0pAHYlE&list=PLZDyxLlNKRY_GJskIreh9sQgExJ4z8oZO&index=8&t=0s)
* [Building with Watson: Passage Retrieval and Relevancy Training with Watson Discovery](https://www.youtube.com/watch?v=8BiuQKPQZJk&list=PLZDyxLlNKRY_GJskIreh9sQgExJ4z8oZO&index=9&t=0s)
* [Building with Watson: Network Visualizations with Watson Discovery Service](https://www.youtube.com/watch?v=pcNwV9prfmY&list=PLZDyxLlNKRY_GJskIreh9sQgExJ4z8oZO&index=10&t=0s)
Change label or cross-reference for example [PUT /v1/environments/{environment_id}/collections/{collection_id}/training_data/{query_id}/examples/{example_id}](https://cloud.ibm.com/apidocs/discoveryupdatetrainingexample) [POST /v2/projects/{project_id}/training_data/ queries/{query_id}](https://cloud.ibm.com/apidocs/discovery-dataupdatetrainingquery) <br>Use the v2 training_data update method. 
 Get details for a training data example [GET /v1/environments/{environment_id}/collections/{collection_id}/training_data/{query_id}/examples/{example_id}](https://cloud.ibm.com/apidocs/discoverygettrainingexample) Not available. Use the Read all examples call to get all examples associated with a query and find the example that you need in the returned list. 







 User data 

The user data API is the same in v2 and v1.



User data API support details

 Action v1 API v2 API 

 Delete [DELETE /v1/user_data](https://cloud.ibm.com/apidocs/discoverydeleteuserdata) [DELETE /v2/user_data](https://cloud.ibm.com/apidocs/discovery-datadeleteuserdata) <br>Similar to v1. Use customer_id to delete the data associated with that customer ID. 







 Events and feedback 

The v1 events and feedback API (/v1/events) is not available in v2.





 Credentials 

The v1 credentials API (/v1/environments/{environment_id}/credentials) is not available in v2. The function is available from the v2 product user interface.





 Gateway configuration 

The v1 gateways API (/v1/environments/{environment_id}/gateways) is not available in v2. The function is available from the v2 product user interface.
getCapacityThroughputInformationOptions := service.NewGetCapacityThroughputInformationOptions()

capacityThroughputInformation, response, err := service.GetCapacityThroughputInformation(getCapacityThroughputInformationOptions)
if err != nil {
panic(err)
}

b, _ := json.MarshalIndent(capacityThroughputInformation, "", " ")
fmt.Println(string(b))

The previous Go example requires the following import block:

import (
"encoding/json"
"fmt"
"github.com/IBM/cloudant-go-sdk/cloudantv1"
)

All Go examples require the service object to be initialized. For more information, see the API documentation's [Authentication section](https://cloud.ibm.com/apidocs/cloudant?code=goauthentication-with-external-configuration) for examples.

The returned structure includes the following fields:

current
: Details the current capacity that is allocated and shows both the number of capacity blocks and breakdown of throughput request classes.

target
: Details the target capacity that is allocated and shows both the number of capacity blocks and breakdown of throughput request classes.

blocks
: Number of provisioned throughput capacity blocks, where a block is 100 reads/sec, 50 writes/sec, and 5 global queries/sec.

throughput
: Breakdown of the specific number of reads/sec, writes/sec, and global queries/sec.

See the following example JSON response with the current and target capacity:

{
"current": {
"throughput": {
"read": 500,
"write": 250,
"blocks": 5,
"query": 25
}
},
"target": {
"throughput": {
"read": 1000,
"write": 500,
"blocks": 10,
Action v1 API v2 API 

 List training data [GET /v1/environments/{environment_id}/collections/{collection_id}/training_data](https://cloud.ibm.com/apidocs/discoverylisttrainingdata) [GET /v2/projects/{project_id}/training_data /queries](https://cloud.ibm.com/apidocs/discovery-datalisttrainingqueries) 
 Add query to training data [POST /v1/environments/{environment_id}/collections/{collection_id}/training_data](https://cloud.ibm.com/apidocs/discoveryaddtrainingdata) [POST /v2/projects/{project_id}/training_data /queries](https://cloud.ibm.com/apidocs/discovery-datacreatetrainingquery) 
 Delete all training data [DELETE /v1/environments/{environment_id}/collections/{collection_id}/training_data](https://cloud.ibm.com/apidocs/discoverydeletealltrainingdata) [DELETE /v2/projects/{project_id}/training_data /queries](https://cloud.ibm.com/apidocs/discovery-datadeletetrainingqueries) 
 Get details about a query [GET /v1/environments/{environment_id}/collections/{collection_id}/training_data/{query_id}](https://cloud.ibm.com/apidocs/discoverygettrainingdata) [GET /v2/projects/{project_id}/training_data /queries/{query_id}](https://cloud.ibm.com/apidocs/discovery-datagettrainingquery)
For additional training guidance, see [Relevancy training tips](https://cloud.ibm.com/docs/discovery?topic=discovery-relevancy-tips).





 Performing other training-data query operations 

You can administer and maintain training-data queries by using other API methods as described in the [API reference](https://cloud.ibm.com/apidocs/discovery):



* [List the training data for a collection](https://cloud.ibm.com/apidocs/discoverylist-training-data)
* [Delete all training data for a collection](https://cloud.ibm.com/apidocs/discoverydelete-all-training-data)
* [Display the contents of a specified training-data query](https://cloud.ibm.com/apidocs/discoveryget-details-about-a-query)
* [Delete a training-data query from the collection](https://cloud.ibm.com/apidocs/discoverydelete-example-for-training-data-query)
* [Change a training-data query example's relevance label or cross reference](https://cloud.ibm.com/apidocs/discoverychange-label-or-cross-reference-for-example)
* [Delete an example document from a training-data query](https://cloud.ibm.com/apidocs/discoverylist-training-data)

Error monitoring: Training-data errors appear in the notices, which you can monitor using the [query notices API](https://cloud.ibm.com/apidocs/discoveryquery-system-notices) ( GET /v1/environments/{environment_id}/collections/{collection_id}/notices).
Delete a document [DELETE /v1/environments/{environment_id}/collections /{collection_id}/documents/{document_id}](https://cloud.ibm.com/apidocs/discoverydeletedocument) [DELETE /v2/projects/{project_id}/collections/{collection_id}/documents/{document_id}](https://cloud.ibm.com/apidocs/discovery-datadeletedocument) <br>Segments of an uploaded document cannot be deleted individually. Delete all segments with a DELETE request that includes the parent_document_id of a segment result. 



v2 introduces a custom header that is named X-Watson-Discovery-Force that is not available in v1. You must include the header when you perform an operation on data that is shared across many collections to indicate that you want to perform the operation in each collection. If you do not include the header, a 403 error is returned.

Fields from JSON files that are added to a collection are converted differently during ingestion between v1 and v2. For more information about how JSON files are stored in the v2 index, see [JSON files](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-index-overviewfile-type-json).





 Queries 



Documents API support details

 Action v1 API v2 API 

 Query a collection Supports a GET or POST request. <br>[GET or POST /v1/environments/{environment_id}/collections/{collection_id}/query](https://cloud.ibm.com/apidocs/discoveryqueryusingget) Queries a project. To specify a single collection, include the {collection_id} parameter. Supports a POST request only. <br>[POST /v2/projects/{project_id}/query](https://cloud.ibm.com/apidocs/discovery-dataquery)