vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Related references - named window clause 

A named window clause is referenced by the following clauses:



* [fullselect](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencefullselect)
* [simpleselect](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencesimpleselect)



With the keyword OVER, you can define an unnamed window specification in a [functionOrAggregate](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencefunctionOrAggregate).







 SQL expressions 



 Expressions 

In the context of an SQL query statement, an expression is always a Boolean expression.



 expression 





 More topics - Boolean expression 

For more information, see [booleanExpression](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencebooleanExpression).





 Related references - expression 

An expression is referenced by the following clauses:



* [caseExpression](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencecaseExpression)
* [castExpression](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencecastExpression)
* [frameBound](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referenceframeBound)
* [fullselect](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencefullselect)
* [functionOrAggregate](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencefunctionOrAggregate)
* [groupByClause](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencegroupByClause)
Valid values: no-query-string, query-string-independent, query-string-dependent.



* no-query-string: Only delivers resources from cache when there is no query string.
* query-string-independent: Delivers the same resource to everyone independent of the query string.
* query-string-dependent: Delivers a different resource each time the query string changes.



--browser-expiration
: Specify how long you want the user's browser to store cached assets.



* Valid values are: respect-existing-header, 30s, 1M, 5M, 20M, 30M, 1h, 2h, 4h, 8h, 16h, 1d3d, 8d, 16d, 1m, 6m, 1y.
* 30s, 1M, 5M, and 20M are only available for Enterprise or Security plan instance.
* 30s means 30 seconds
* 30M means 30 minutes
* 1h means 1 hour
* 1d means 1 day
* 1m means 1 month
* 1y means 1 year



--development-mod
: Bypass all edge caches and send traffic toward your origin servers.

--serve-stale-content
: Continue serving cached content to users when origin servers are offline, even if the content has expired.

`--query-string-sort**: In the cache, CIS treats files with the same query strings as the same file, regardless of the order of the query strings.

-i, --instance
: Instance name or ID. If not set, the context instance specified by ibmcloud cis instance-set is used.

--output
: Specify output format, only JSON is supported.





 Examples 

Update caching settings for domain 31984fea73a15b45779fa0df4ef62f9b.

ibmcloud cis cache-settings-update 31984fea73a15b45779fa0df4ef62f9b --caching-level no-query-string --browser-expiration 1h -i "cis-demo"









 Custom page
Aggregation queries return a count of documents matching a set of data values; for example, top keywords, overall sentiment of entities, and more. For the full list of aggregation options, see the [Aggregations table](https://cloud.ibm.com/docs/discovery?topic=discovery-query-aggregations). These aggregations are written using the [Discovery Query Language](https://cloud.ibm.com/docs/discovery?topic=discovery-query-operators).





 natural_language_query 

A natural language query enables you to perform queries expressed in natural language, as might be received from an end user in a conversational or free-text interface - for example: "IBM Watson in healthcare". The parameter uses the entire input as the query text. It does not recognize operators. The natural_language_query parameter enables capabilities such as passage search and relevancy training. All private collections return a confidence score in the query results in most cases. For details, see [Confidence scores](https://cloud.ibm.com/docs/discovery?topic=discovery-improving-result-relevance-with-the-toolingconfidence). The maximum query string length for a natural language query is 2048.

Structure parameters

Structure parameters define the content and organization of the documents in the returned JSON. This includes the number of results returned, where in the results set to start returning documents, how the result set is sorted, which fields to return for each documents, if duplicate documents must be removed, and if relevant passages must be extracted from the results set. Structure parameters don't affect which documents are part of the entire results set.





 count 

The number of documents that you want returned in the response. The default is 10. The maximum for the count and offset values together in any one query is 10000.





 offset 

The number of query results to skip at the beginning. For example, if the total number of results that are returned is 10, and the offset is 8, it returns the last two results. The default is 0. The maximum for the count and offset values together in any one query is 10000.





 return
[similar.fields](https://cloud.ibm.com/docs/discovery?topic=discovery-query-parameterssimilar_fields) Which fields to compare when finding similar documents similar.fields=text,title 





 Query limitations 

You cannot query on field names that contain the following:



* Numerical characters (0 - 9) in the suffix of the field name (for example extracted-content2).
* The characters _, +, and - in the prefix of the field_name (for example +extracted-content).
* The characters ., ,, and : in the field_name (for example new:extracted-content)









 Operators 

Operators are the separators between different parts of a query. These are the available operators:



 Operator Description Example 

 [.](https://cloud.ibm.com/docs/discovery?topic=discovery-query-operatorsdelimiter) JSON delimiter enriched_text.concepts.text 
 [:](https://cloud.ibm.com/docs/discovery?topic=discovery-query-operatorsincludes) Includes text:computer 
 [::](https://cloud.ibm.com/docs/discovery?topic=discovery-query-operatorsmatch) Exact match title::"Query building" 
 [!:](https://cloud.ibm.com/docs/discovery?topic=discovery-query-operatorsnotinclude) Does not include text:!computer 
 [!::](https://cloud.ibm.com/docs/discovery?topic=discovery-query-operatorsnotamatch) Not an exact match text::!winter
curl -u "4ba1624f-48b6-484a-8e32-18d1c205c1fa":"qUy3B0CbGf9G" "{url}/v1/environments/a56dd9b4-040b-4ea3-a736-c1e7a467e191/collections/99040100-fe6a-4782-a4f5-28f9eee30850/training_data?version=2019-04-30"

The JSON that is returned is in the following format:

{
"query_id": "484baad4-d440-44b7-a0dd-70bb2ac77baa",
"natural_language_query": "why is the sky blue",
"filter": "text:meteorology",
"examples": [
{
"document_id": "9afcf06e-2ef7-438e-b2a5-e4d9432dc877",
"relevance": 0
},
{
"document_id": "54f95ac0-3e4f-4756-bea6-7a67b2713c81",
"relevance": 1
},
{
"document_id": "01bcca32-7300-4c9f-8d32-33ed7ea643da",
"cross_reference": "my_id_field:1463",
"relevance": 4
}
]
}
Show more
2. Add the new example by using the same syntax you use to define examples when creating a new training-data query. The following command example does not include a cross reference.

curl -u -X POST -u "4ba1624f-48b6-484a-8e32-18d1c205c1fa":"qUy3B0CbGf9G" -H "Content-Type: application/json" -d
However, any implementation of token-based pagination SHOULD attempt to ensure that resources are not duplicated or omitted

[3]if all pages in a collection are requested in sequence.



 Page token query parameter 

The page token query parameter, which SHOULD be named start but MAY be named token, MUST be used to determine what resource to start the page on or after. If the page token query parameter is unspecified, the logical first page of results MUST be returned.

Page tokens SHOULD be provided to the client in the page token field, having the same name as the page token query parameter, of the next, previous, or last object in a collection.





 Limit 

The collection MAY support the limit query parameter, which MUST be an integer. When supported, the limit parameter MUST be used to determine the maximum number of resources returned. Further, the number of resources returned SHOULD be the same as limit except for the last page of the collection.

If limit is unspecified or unsupported by the collection, a default limit MUST be used. The default limit MUST be consistent across all requests to the collection. It should be determined for each collection based on a reasonable default request time and payload size. The default limit should be highlighted in the collection's documentation.

If limit is supported, a collection MUST impose a maximum value for the limit. The maximum limit MUST be consistent across all requests to the collection. The maximum limit should be determined for each collection based on reasonable maximums for request time and payload size. The maximum limit should be highlighted in the collection's documentation.

If the value provided for limit is not a positive integer or is greater than the maximum limit, the server SHOULD return a 400 status code and appropriate error response model.





 Additional response fields 

Collections using token-based pagination MUST return a page token field, having the same name as the page token query parameter, in any next, previous, and last fields whenever these are present according the conditions defined for [Link-based pagination](https://cloud.ibm.com/docs/api-handbook?topic=api-handbook-paginationlink-based-pagination).
The search skill sends a natural language query to Discovery automatically. If you want to use the Discovery query language instead, you can specify it. To do so, open the JSON editor for the node response.

Edit the JSON code snippet to replace natural_language with discovery_query_language. For example:

{
"output": {
"generic": [
{
"query": "",
"filter": "enriched_text.sentiment.document.label:positive",
"query_type": "discovery_query_language",
"response_type": "search_skill"
}
]
}
}



Test this response type from the assistant Preview. You cannot test it from the dialog skill's "Try it out" pane. For more information about testing dialog and search skills together, see [Testing your assistant from a web page](https://cloud.ibm.com/docs/assistant?topic=assistant-deploy-web-link).





 Conditional responses 

A single dialog node can provide different responses, each one triggered by a different condition. Use this approach to address multiple scenarios in a single node.

The node still has a main condition, which is the condition for using the node and processing the conditions and responses that it contains.

In this example, your assistant uses information that it collected earlier about the user's location to tailor its response, and provide information about the store nearest the user. See [Context variables](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-runtime-context) for more information about how to store information collected from the user.

![Shows a node that shows a user ask, Where are you located, and the dialog has three different responses depending on conditions that use info from the $state context variable to specify locations in those states.](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/multiple-responses.png)
New query operator : The ::! operator is added as a unary not-equals operator. For example, you can now run query=field::!value (not equals). Previously the only exclusionary operator was :! for the not-contains operator (for example, query=field:!value).

Fixed defects : Applied security updates. : Improved status messages for search alerts.





 1 February 2017 

New release of Data Crawler now available : These notes apply specifically to the Data Crawler 1.3.0 release. [Update](https://cloud.ibm.com/docs/discovery?topic=discovery-adding-content-with-data-crawler) : The Data Crawler records the document_id values used to upload documents, and the status of the upload. Conversion notices are not persisted outside of the log. There is not presently a tool to interact with that data, but such tools are expected to be developed as time permits. The data is accessible via H2 database, which could be configured to use a remote DBMS.





 16 January 2017 

New release of Data Crawler now available : These notes apply specifically to the Data Crawler 1.2.5 release. [Update](https://cloud.ibm.com/docs/discovery?topic=discovery-adding-content-with-data-crawler) : The Data Crawler can optionally poll for document status immediately after uploading a file. This check is a part of the Crawler's concept of "uploading a document", so when this check is enabled, it is virtually impossible for the Crawler to upload concurrently more documents than what Discovery can process concurrently for the user. : A side effect of the check_for_completion feature is that the Crawler also can expose to the user why a document failed, when it failed. Any notices attached to a document that was successfully uploaded, but failed to process, are displayed in the Crawler log. The notices are not exported to a processable file, but IBM would welcome a feature suggestion for that.





 5 January 2017 

Known issues identified after the GA release on 15 December 2016.
Find answers fast with Cloud docs and solutions





Refine with search operators

Refine your searches with operators: * Use an asterisk (*) in a search string as a placeholder for any missing or wildcard words in a phrase. "" Use quotation marks to search for a phrase (for example, "command line interface"). - Use minus to exclude a word or phrase (for example, account cli -api). Your search must have at least one included word.[Solutions](https://cloud.ibm.com/docs?tab=solutions)Solutions Discover compliant solutions for your unique use case. New[Product guides](https://cloud.ibm.com/docs?tab=all-docs)Product guides Browse product documentation.[Tutorials](https://cloud.ibm.com/docs?tab=tutorials)Tutorials Check out our step-by-step tutorials.[API & SDK reference](https://cloud.ibm.com/docs?tab=api-docs)API & SDK reference Review the API and SDK references available for our services.[FAQs](https://cloud.ibm.com/docs?tab=faqs)FAQs Find answers to popular questions.

 Get started with IBM Cloud 

The IBM Cloud platform is composed of multiple components that work together to provide a consistent and dependable cloud experience.



* ![tab icon](https://cloud.ibm.com/media/docs/images/homepage/globe.svg) Take a tour
* ![tab icon](https://cloud.ibm.com/media/docs/images/homepage/account.svg) Manage your account
* ![tab icon](https://cloud.ibm.com/media/docs/images/homepage/ibm-cloud--security-compliance-center.svg) Security and compliance
* ![tab icon](https://cloud.ibm.com/media/docs/images/homepage/development.svg) Get developing
* !
Default query settings 

Learn about how the search query is configured for each project type by default.

When you submit a search from the product user interface, your text is passed as a natural language query value to the Query API. Other parameters that you can define when you use the API are assigned default values for queries that are made from the user interface. The following tables explain which values are specified by default for each project type. For more information about the Query API, see [Query reference](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-query-reference).

You can override some of the default values by using improvement tools in the user interface. For example, you can use the Search results tool to change parameters such as passages.enabled. For more information, see [Changing the result content](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-query-resultsquery-results-content).

The enrichments that are applied to your data autmoatically differ by project type. For more information, see [Default project settings](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-projectsprojects-include-project-defaults-reuse).



 Default query settings 



Default query settings

 Query default Document Retrieval Document Retrieval for Contracts 

 aggregation [term(enriched_text.entities.text,name:entities)] [term(enriched_html.contract. elements.categories.label,count:25,name:categories)] 
 count 10 10 
 highlight false false 
 passages.characters 200 200 
 passages.count 10 10 
 passages.enabled true true 
 passages.fields ["text", "title"] ["text", "title"] 
 passages.find_answers false false 
 passages.max_answers_per_passage 1 1 
 passages.max_per_document 1 1 
 passages.per_document true true 
 return [] [] 
 spelling_suggestions false true 
 sort "" "" 
 table_results.count 10 10 
 table_results.enabled false true 
 table_results.per_document 0 0
* If your documents are already indexed with the default stopwords and you then add a custom stopword list, the new stopwords are still included in the index. In that case, queries that contain these new stopwords filter them during querying.
* If a user searches for a word that was a stopword at one point in time but is later removed from the custom stopword list, they cannot find documents that match the original stopword because the term was removed during indexing. To fix this issue, delete the documents in your collection, and reupload all the documents so that they are indexed with the updated custom stopword list.



* Each set of stopwords is associated with a collection. If you query across multiple collections, each collection uses the custom stopword list associated with that collection.
* If you make significant changes to your custom stopword list, delete the documents in your collection and reupload all the documents so that they are indexed with the updated custom stopword list.



Stopword and query expansion lists cannot contain overlapping terms. If a [stopword](https://cloud.ibm.com/docs/discovery?topic=discovery-query-conceptsstopwords) in your stopwords list is also included within your query expansion list, the query expansion does not work. For example, if on is included in your stopword list and you specify in your query expansion list that rotfl must expand to rolling on the floor laughing, that expansion does not return the expected results. A list of stopwords is enabled by default (file name: custom_stopwords_{language}.txt) in Discovery, so compare the words in that file to your query expansion file (expansions.json) and adjust accordingly.





 Creating custom tokenization dictionaries 

Tokenization breaks text into units called tokens. A standard tokenization dictionary is applied to your collections, but you can improve the search accuracy for your domain or language by uploading a custom tokenization dictionary. Your custom dictionary appends the standard dictionary. You can upload your dictionary, using the Discovery API.
Improved query and add functions at top level : id, score, and highlight at the top level (You can continue to add documents to your collection using document IDs with the add a document function. See the [API Reference](https://cloud.ibm.com/apidocs/discoveryadd-a-document) for details. : _ prefixed field names at the top level (as a result, when querying for a document by ID, you can query for id instead of _id.) : and , in the field name : + and - prefixed field names : "" empty values for a field name : If your JSON documents include these characters in the field names, or id, score, and highlight at the top level, you need to remove them before adding the documents to your collection, or those fields are empty. You can create a custom configuration and normalize your JSON before adding documents to your collection to avoid this issue. See the [API reference](https://cloud.ibm.com/apidocs/discoveryadd-configuration) for details. In addition, documents that include the punctuation characters ?, :, or in the file name cause errors during ingestion. Before ingesting them, rename any documents that include these characters.

Improved 'natural_language_query' retrieval methods : The retrieval methods for natural_language_query are updated to improve the relevance of results by matching words with related semantics. This update only affects collections that did not undergo relevance training. If you are using natural_language_query and did not conduct relevance training, you might see improvement in the order of results returned.

Improved query builder navigation : Changes to the query builder to make it easier to toggle between the Discovery Query Language and Natural Language query options, as well as among query, filter, and aggregation.





 25 August 2017 

Improved 'passages' array : The passages array now includes field, start_offset, and end _offset. field is the name of the field the passage was extracted from. start_offset is the starting character of the passage text within the field. end_offset is the ending character of the passage text within the field.
[query-relations](https://cloud.ibm.com/apidocs/discoveryquery-relations) Action username, password, iam_access_token, iam_apikey, iam_url, headers, headers[X-Watson-Learning-Opt-Out], url, environment_id, collection_id, entities, context, sort, filter, count, evidence_count Query a Knowledge Graph relationship. 
 [add-training-data](https://cloud.ibm.com/apidocs/discoveryadd-training-data) Action username, password, iam_access_token, iam_apikey, iam_url, headers, headers[X-Watson-Learning-Opt-Out], url, environment_id, collection_id, natural_language_query, filter, examples, Add a query to training data. 
 [create-training-example](https://cloud.ibm.com/apidocs/discoverycreate-training-example) Action username, password, iam_access_token, iam_apikey, iam_url, headers, headers[X-Watson-Learning-Opt-Out], url, environment_id, collection_id, query_id, document_id, cross_reference, relevance Add an example to training data query. 
 [delete-all-training-data](https://cloud.ibm.com/apidocs/discoverydelete-all-training-data) Action username, password, iam_access_token, iam_apikey, iam_url, headers, headers[X-Watson-Learning-Opt-Out], url, environment_id, collection_id Delete all training data. 
 [delete-training-data](https://cloud.ibm.com/apidocs/discoverydelete-training-data) Action username, password, iam_access_token, iam_apikey, iam_url, headers, headers[X-Watson-Learning-Opt-Out], url, environment_id, collection_id, query_id Delete a training data query.
"document_id": "54f95ac0-3e4f-4756-bea6-7a67b2713c81",
"relevance": 1
},
{
"document_id": "01bcca32-7300-4c9f-8d32-33ed7ea643da",
"cross_reference": "my_id_field:1463",
"relevance": 4
}
]
}
Show more

The following command example adds the previous training-data query to a collection named 99040100-fe6a-4782-a4f5-28f9eee30850 in an environment named a56dd9b4-040b-4ea3-a736-c1e7a467e191:

curl -X POST -u "4ba1624f-48b6-484a-8e32-18d1c205c1fa":"qUy3B0CbGf9G" -H "Content-Type: application/json" -d
'{
"natural_language_query": "why is the sky blue",
"filter": "text:meteorology",
"examples": [
{
"document_id": "9afcf06e-2ef7-438e-b2a5-e4d9432dc877",
"relevance": 0
},
{
"document_id": "54f95ac0-3e4f-4756-bea6-7a67b2713c81",
"relevance": 1
},
{
"document_id": "01bcca32-7300-4c9f-8d32-33ed7ea643da",
"cross_reference": "my_id_field:1463",
"relevance": 4
}
]
}'