vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* Try the search bar that is located in the IBM Cloud® console menu bar.
* You can filter your resource list by the tag you are searching for.
* From the application detail page, you can locate the tags that are associated with that resource.



The same tag can be attached to multiple resources and service IDs by different users in the same billing account, and not all users have visibility on all resources on the account.





 Searching for tags by using the CLI 

To search for tags by using the CLI, see [Searching for resources](https://cloud.ibm.com/docs/account?topic=account-searching-for-resourcessearching-cl).





 Searching for tags by using the API 

To search for tags by using the API, see [Searching for resources](https://cloud.ibm.com/docs/account?topic=account-searching-for-resourcessearching-api).

A common use case for searching for tags by using the API is retrieving the tags that are attached to a resource. There are a couple of options for doing that.

The preferred option is the [Search API](https://cloud.ibm.com/apidocs/search), as shown in the following example.

curl -s -H'Content-type:application/json' -H'Accept:application/json' -H"Authorization:Bearer $IAM_TOKEN" -X POST 'https://api.global-search-tagging.cloud.ibm.com/v3/resources/search' -d'{"query":"crn:"crn:v1:bluemix:public:codeengine:us-south:a/aabbccddeeff00112233445566778899:aabbccdd-1234-4321-aaaa-001122334455::"", "fields":["crn","name","tags", "access_tags", "service_tags"]}'

If the resource with the given CRN exists and you can read it, then you get the following example output, which includes all of the tag types attached to the resource:

{
"chapters": {
"css_selector": ".chapter",
"type": "array"
},
"authors": {
"css_selector": ".author"
},
"authors_str": {
"css_selector": ".author",
"type": "string"
},
"comments": {
"css_selector": "[^comments-content]"
}
}
}
}
...
}
Show more

If you load the following HTML document into a collection that uses an updated configuration, the CSS selectors match the appropriate attributes and values in the HTML.

<html>
<body>
<div id="authors">
<div class="author"><span class="first_name">Jane</span> <span class="last_name">Rain</span></div>
<div class="author">Joe Snow</div>
</div>
<div class="chapter"><h1>The Rain in Spain</h1><p>falls mainly on the plain.</p></div>
<div class="chapter"><h1>How I Learned to Stop Worrying</h1><p>and love my snowblower.</p></div>
<span id="comments-section">
<h4>Comments:</h4>
<span id="comments-content-1">Rain gives me pain.</span>
<span id="comments-content-2">All snow must go!</span>
</span>
</body></html>

After the preceding HTML is ingested and enhanced, Discovery returns the following JSON:

{
"extracted_metadata": { ... },
"html": "...",
"text": "...",
Authorization: 'AWS4-HMAC-SHA256 Credential={access-key}/{date}/{region}/s3/aws4_request,SignedHeaders=host;x-amz-date;,Signature={signature}'
x-amz-date: {timestamp}
Content-Type: text/plain
Host: s3.us.cloud-object-storage.appdomain.cloud
Content-Length: 128

<Tagging>
<TagSet>
<Tag>
<Key>string</Key>
<Value>string</Value>
</Tag>
</TagSet>
</Tagging>

Example response

HTTP/1.1 200 OK
Date: Wed, 5 Oct 2020 15:39:38 GMT
X-Clv-Request-Id: 7afca6d8-e209-4519-8f2c-1af3f1540b42
Accept-Ranges: bytes
Content-Length: 0

--------------------





 Read an object's tags 

A GET issued to an object with the proper parameters returns the set of key-value tags associated with the object.

Not all operations are supported in Satellite environments. For details, see [supported Satellite APIs](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-apis-cos-satellite)

Syntax

GET https://{endpoint}/{bucket-name}/{object-name}?tagging path style
GET https://{bucket-name}.{endpoint}/{object-name}?tagging virtual host style

Example request

This is an example of reading a set of object tags.

GET /apiary/myObject?tagging HTTP/1.1
Authorization: Bearer {token}
Content-Type: text/plain
Host: s3.us.cloud-object-storage.appdomain.cloud
Content-Length: 0
For example, !hello would match the text !hello. 
 "" Literal phrase. Use to encloses a query term that contains spaces or other special characters. No special characters within the quotation marks are parsed by the API. 
 Approximate match. Append this operator followed by a 1 or 2 to the end of the query term to specify the allowed number of single-character differences between the query string and a match in the response object. For example, car1 would match car, cat, or cars, but it would not match cats. This operator is not valid when filtering on log_id or any date or time field, or with fuzzy matching. 
 * Wildcard operator. Matches any sequence of zero or more characters. This operator is not valid when filtering on log_id or any date or time field. 
 (), [] Grouping operators. Use to enclose a logical grouping of multiple expressions using Boolean operators. 
 Boolean or operator. 
 , Boolean and operator. 





 Filtering by intent or entity 

Because of differences in how intents and entities are stored internally, the syntax for filtering on a specific intent or entity is different from the syntax used for other fields in the returned JSON. To specify an intent or entity field within an intents or entities collection, you must use the : match operator instead of a dot.

For example, this query matches any logged event where the response includes a detected intent named hello:

response.output.intents:intent::hello

Note the : operator in place of a dot (intents:intent)

Use the same pattern to match on any field of a detected intent or entity in the response. For example, this query matches any logged event where the response includes a detected entity with the value soda:

response.output.entities:value::soda

Similarly, you can filter on intents or entities sent as part of the request, as in this example:

request.input.intents:intent::hello





 Filtering by other fields 

To filter on another field in the log data, specify the location as a path identifying the levels of nested objects in the JSON response from the /logs API. Use dots (.) to specify successive levels of nesting in the JSON data.
* body_cells: An array of cells that are not table header or column header or row header cells, of the current table with corresponding row and column header associations. Each body cell is defined as a collection of the following items:



* cell_id: The unique ID of the cell in the current table.
* location: The location of the cell in the input document as defined by its begin and end indexes.
* text: The textual contents of the cell from the input document without associated markup content.
* row_index_begin: The begin index of this cell's row location in the current table.
* row_index_end: The end index of this cell's row location in the current table.
* column_index_begin: The begin index of this cell's column location in the current table.
* column_index_end: The end index of this cell's column location in the current table.
* row_header_ids: An array of values, where each value is the cell ID value of a row header that is associated with this body cell.
* row_header_texts: An array of values, where each value is the text from a row header for this body cell.
* row_header_texts_normalized: An array of values, where each value is the normalized text from a row header for this body cell.
* column_header_ids: An array of values, where each value is the cell ID value of a column header that is associated with this body cell.
* column_header_texts: An array of values, where each value is the text from a column header for this body cell.
* column_header_texts_normalized: An array of values, where each value is the normalized text from a column header for this body cell.
* attributes: An array that identifies document attributes. Each object in the array consists of three elements:



* type: The type of attribute. Possible values are Address, Currency, DateTime, Duration, Location, Number, Organization, Percentage, and Person.
* text: The text that is associated with the attribute.
* location: The location of the attribute as defined by its begin and end indexes.
* The language sets that contain German (de), Spanish (es), English (en), French (fr), Italian (it), and the unspecified language (x_unspecified)









 Usage notes 



* The semantics of an extract pattern statement is driven by the pattern specification. Each match constructs an output result according to the return clause of the pattern specification and the select list at the top of the extract statement. The results are filtered and consolidated according to the having, consolidate, and limit clauses of the extract statement. If multiple overlapping matches exist for the pattern specification, a pattern extraction outputs all possible matches. Use consolidation to filter redundant outputs.
* The semantics of the from clause of an extract pattern statement are different from other forms of extract statements that do not have a pattern specification. The general semantics of an extract statement require that the extraction specification is evaluated over each combination of the views that are defined in the <from list>. If at least one of the views in the <from list> does not contain any results on a particular document, then the output of the extract statement is empty because the set of all combinations of results in the input views is empty. In the special case of extract pattern statements, the from clause is a placeholder that declares the names of relations that are involved in the pattern specification. The semantics of the statement are driven only by the pattern specification. In particular, the output of the statement can be non-empty even when some of the input views are empty.
* An extract statement that uses sequence pattern extraction can carry forward the columns of any view in the from list, but only if the view name does not appear in a repeat element of the pattern specification. For example, the statement CapsWordOneToThree results in a compilation error. The error occurs because the carried forward column CW.type at the top of the extract statement belongs to the view name CW, which is in the repeat element <CW.word>{1,3} of the pattern specification.

create view CapsWord as
extract 'UpperCase' as type,
regex /[A-Z]./ on 1 token in D.text as word
from Document D;

---> This results in and error due to the repeating element CW.word
create view CapsWordOneToThree as
--rd or --rd-path is required when you use a self-provided image.

-i IMAGE-TAG
: The image tag for the BYOI server image. Required if you're using your own image.

-t TAG
: Use tags to organize your resources. Tags are visible account-wide. Optional. Multiple tags are permitted, for example, -t tag1 -t tag2.

-g RESOURCE-GROUP-ID | RESOURCE-GROUP-NAME
: The resource group to which your Hyper Protect Virtual Servers instance belongs for access control and billing purposes, for example, Default. To list all of your resource groups, run ibmcloud resource groups. Optional.

-e ENV-CONFIG
: Specify environment variables if you are using a self-provided image. You must specify the variables in your registration definition first. You can set one or more environment variables as key value pairs by using the -e flag, for example, -ibmcloud hpvs instance-update CRN -i latest -e k1=v1 -e k2='v2 v3'. Environment variable names can have a maximum length of 64 characters and can be numbers, chars, and underscore. Environment variable values can have a maximum length of 12288.

--outbound-only
: If this parameter is set, only outbound connections are allowed from your Hyper Protect Virtual Server instance. Use the internal IP address to connect to this Virtual Server from other Virtual Servers created by the same IBM Cloud account in the same region.





 Example input
See [Discovery pricing plans](https://cloud.ibm.com/docs/discovery?topic=discovery-discovery-pricing-plans) for document limits.
* You can not use CSS selectors to extract fields (see [Using CSS selectors to extract fields](https://cloud.ibm.com/docs/discovery?topic=discovery-configserviceusing-css)) when using document segmentation.
* Documents are segmented each time the specified HTML tag is detected. Consequently, segmentation could lead to malformed HTML because the documents could be split before closing tags and after opening tags.
* HTML metadata; the title; plus any custom metadata, is extracted and included in the index with each segment. Every segment of a document includes identical metadata.
* Document segmentation is not supported when the Element Classification (elements) enrichment is specified.
* Re-ingesting a segmented document has additional considerations, see [Updating a segmented document](https://cloud.ibm.com/docs/discovery?topic=discovery-configserviceupdate-seg).





 Performing segmentation 

If your source documents are supported by Smart Document Understanding (see [Supported document types](https://cloud.ibm.com/docs/discovery?topic=discovery-sdudoctypes)), use document splitting instead of document segmentation. For details, see [document splitting](https://cloud.ibm.com/docs/discovery?topic=discovery-sdusplitting).

Segmentation is setup via the API in the conversions section.

{
"configuration_id": "a23c467d-1212-4b3a-5555-93e788a3622a",
"name": "Example configuration",
"conversions": {
"segment": {
"enabled": true,
"selector_tags": ["h1", "h2", "h3", "h4", "h5", "h6"]
}
}
}

enabled = true turns on document segmentation.
* Entity: The node is used when any value or synonym for the entity is recognized in the user input. Use the syntax, @entity_name. For example, @city checks whether any of the city names that are defined for the @city entity were detected in the user input. If so, the node or response is processed.

Consider creating a peer node to handle the case where none of the entity's values or synonyms are recognized.

For more information about entities, see [Defining entities](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-entities).
* Entity value: The node is used if the entity value is detected in the user input. Use the syntax, @entity_name:value and specify a defined value for the entity, not a synonym. For example: @city:Boston checks whether the specific city name, Boston, was detected in the user input.

If you check for the presence of the entity, without specifying a particular value for it, in a peer node, be sure to position this node (which checks for a particular entity value) before the peer node that checks only for the presence of the entity. Otherwise, this node will never be evaluated.

If the entity is a pattern entity with capture groups, then you can check for a certain group value match. For example, you can use the syntax: @us_phone.groups[1] == '617' See [Storing and recognizing pattern entity groups in input](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-dialog-tipsdialog-tips-get-pattern-groups) for more information.
* Intent: The simplest condition is a single intent. The node is used if, after your assistant's natural language processing evaluates the user's input, it determines that the purpose of the user's input maps to the pre-defined intent. Use the syntax, intent_name. For example, weather checks if the user input is asking for a weather forecast. If so, the node with the weather intent condition is processed.
* total_segments - The total number of segments the document was split into.



* extracted_metadata section which contains:



* sha1 - Corresponds to the original document.
* title - The title field is extracted from the HTML title field. (2019-03-25 or later API version string.)
* filename - Corresponds to the original document.
* file_type- Corresponds to the original document.



* text field
* html field









 Updating a segmented document 

If a segmented document is updated and needs to be ingested again, it can be replaced by using the [Update document](https://cloud.ibm.com/apidocs/discoveryupdate-a-document) method.

When updating a segmented document, you must upload the document by using the POST method of the /environments/{environment_id}/collections/{collection_id}/documents/{document_id} API, specifying the contents of the parent_id field of one of the current segments as the {document_id} path variable.

When updating, all segments are overwritten, unless the updated version of the document has fewer total sections than the original. Those older segments remain in the index and might be individually deleted, using the API. See the [API Reference](https://cloud.ibm.com/apidocs/discoverydelete-a-document) for details. You can identify how many segments were created by querying the notices. Each segment is given a document_id field that is comprised of the {parent_id}, followed by an underscore, followed by the segment number.

If any of the segments of the document that you intend to update are ranked for relevancy training, you must first delete all the segments of that document and then ingest the updated document as a new document. This results in a new document_id for each segment and any trained segments must be retrained. The trained index becomes inaccurate if you don't delete the old content first.

Alternately, consider creating a new document that contains only the new content and ingest it separately.
* deterministic
* dictionary
* dictionaries
* document
* element
* else
* empty_fileset
* entries
* exact
* export
* external
* external_name
* extract
* fetch
* file
* first
* flags
* folding
* from
* function
* group
* having
* import
* in
* include
* infinity
* inline_match
* input
* into
* insensitive
* java
* language
* left
* lemma_match
* like
* limit
* mapping
* matching_regex
* minus
* module
* name
* never
* not
* null
* on
* only
* order
* output
* part_of_speech
* parts_of_speech
* parameter
* pattern
* point
* points
* priority
* regex
* regexes
* retain
* required
* return
* right
* rows
* select
* separation
* set
* specific
* split
* table
* tagger
* then
* token
* Token
* tokens
* unicode
* union
* up
* using
* values
* view
* views
* when
* where
* with



The following reserved words are the names of built-in scalar types:



* Text
* Span
* Integer
* Float
* String
* Boolean
* ScalarList



The following other reserved names cannot be used as identifiers:



* Dictionary
* Regex
* Consolidate
* Block
* BlockTok
* Sentence
* Tokenize
* RegexTok
* PosTag







 Constants 

Constants are fixed values that can be one of these data types: String, Integer, Float, or Boolean.
<-- </ul> --><-- </section "id="section-dialog-runtime-disambig-choose-nodes" "> --><-- <section "id="section-dialog-runtime-disambig-disable" "> --> Hiding nodes from disambiguation altogether You can prevent every node in a dialog or an individual dialog node from being included in the disambiguation list.<-- <ul> --> * To disable disambiguation entirely:
<-- <ul> -->

* From the Skills menu, click Options.
* On the Disambiguation page, set the switch to Off.

<-- </ul> -->

* To prevent a single dialog node from being included in the disambiguation list:
<-- <ul> -->

* From the Skills menu, click Dialog. Click the node to open it in the edit view.

<-- </ul> -->

!]
<-- <ul> -->

* Click Settings.

<-- </ul> -->

!]
<-- <ul> -->

* Set the Show node name switch to Off.
* !]!Shows the external node name field.Shows the external node name field_.] ! If you added a node summary description to the external node name field instead of the name field, remove it.

<-- </ul> -->

The external node name field serves two purposes. It provides information about the node to customers when it is included in a disambiguation list. It also describes the node in a chat summary that is shared with service desk agents when a conversation is transferred to a person. The external node name field is only visible in skills that are part of a paid plan instance. If the external node name field contains text, its text is used, whether or not there is text in the name field.
"entities": {
"mentions": true
}
}
}
* Example response:

{
"usage": {
"text_units": 1,
"text_characters": 114,
"features": 1
},
"language": "en",
"entities": [
{
"type": "Company",
"text": "Intel",
"relevance": 0.33,
"mentions":
{
"text": "Intel",
"location":
0,
5
]
},
{
"text": "Intel",
"location":
73,
78
]
}
],
"disambiguation": {
"subtype":
"OperatingSystemDeveloper",
"ProcessorManufacturer",
"SoftwareDeveloper"
],
"name": "Intel",
"dbpedia_resource": "http://dbpedia.org/resource/Intel"
},
"count": 2
},
{
"type": "Company",
"text": "AMD",
"relevance": 0.33,
"mentions":
{
"text": "AMD",
"location":
96,
99
]
}
],
"disambiguation": {
"subtype":
"ProcessorManufacturer"
],
"name": "Advanced Micro Devices",
"dbpedia_resource": "http://dbpedia.org/resource/Advanced_Micro_Devices"
},
"count": 1
}
]
}
Show more







 17 November 2017 

Korean language support improved
[The rule editor that shows the "Create a rule" panel.](images/rule_step4.png "The rule editor that shows what the "Create a rule" panel looks like after you select text from your document and click the plus sign in the "Rules" panel. The following graphical elements are shown: the "Rule name" field with the term "AGE" entered, the "Create a rule" panel, and the "Class" panel. In the "Create a rule" panel, cells with dotted lines are shown across the top. One cell is shown for each token of the text that was selected. In these cells, you annotate the classes of the tokens. Across the bottom of the "Create a rule" panel, cells with solid lines are shown. Each cell contains a token of the selected text, "50-year-old". The tokens are "50", "-" (hyphen), "year", "-", and "old". Each solid-line cell has two icons next to it that you can use to adjust the condition of the word or annotation.")
10. Define the conditions by which each token participates in the pattern.

In the bottom layer of cells, click the first token to review its conditions. If you want to indicate that any token can be used in the current position in the pattern, then click Open Properties, and select Allow any token. Click Close Properties. If a token is a regex, such as AGE_REGEX in the example, Allow any token is not available.

The maximum number of group cells that can participate in a pattern is 15 when each cell has a repeat setting of 1 or less. Group cells include single tokens, annotations, or tokens where any token is allowed. The maximum number of total tokens allowed in a pattern is 20. Take the repeat setting of each cell into account as you define the pattern. For example, you can define a pattern that includes 15 tokens when each cell has a repeat setting of 1 or less.