vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Used to calculate time drift. 
 tags string optional Tags that are used to dynamically group hosts. 
 url string required URL represents the entry point to the service. Each region has a different URL. To get the endpoint for a location, see [Ingestion endpoints](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-endpointsendpoints_ingestion). For example, https://logs.us-south.logging.cloud.ibm.com/logs/ingest. 







 Example 

The following sample sends 2 log lines with different priorities (level) to an instance of the IBM Log Analysis service:



1. Save the following content to a file named send-logs.py.

!/usr/bin/env python3
import os
import logging

from logdna import LogDNAHandler

apiKey = '<INGESTION-KEY>'

log = logging.getLogger('logdna')
log.setLevel(logging.INFO)

options = {
'hostname': 'MyHostName',
'ip': '10.0.1.1',
'mac': 'C0:FF:EE:C0:FF:EE',
'env': 'Dallas',
'level': 'Info',
'index_meta': True,
'url': 'https://logs.us-south.logging.cloud.ibm.com/logs/ingest'
}

test = LogDNAHandler(apiKey, options)

log.addHandler(test)

log.warning("Sample message 1", {'app': 'sample-app'})
log.info("Sample message 2", {'app': 'sample-app'})
Show more
2. From the command line, run the following command:

python3 send-logs.py







 Limits when you send logs 

Consider the following limits when you send logs to an IBM Log Analysis instance:
"location": {
"end": 75,
"begin": 69
},
"text": "Armonk"
}
],
"text": "Armonk",
"type": "Location"
}
]
}
Show more







 Entity limits 

The Entities enrichment can identify up to 50 entities, each with one or many mentions, per document.







 Keywords 

Returns important keywords in the content.

For example, the following screen capture shows highlighted terms from the US Declaration of Independence that are recognized by the Keywords enrichment.

Zoom

![Shows the keywords that are recognized in the document text](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/nlu-rich-keywords.png)

Figure 5. Terms recognized by the Keywords enrichment

From the JSON view of the document, you can see the underlying JSON structure of the Declaration keyword mention.

Zoom

![Shows the JSON view of keywords that are identified in the document](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/nlu-json-keywords.png)

Figure 6. JSON representation of Keywords enrichment mentions



 Example 



 Input 

"Watson Discovery is an award-winning AI search technology."





 Response 

In the JSON output:



* text = The keyword text
* mentions = The entity mentions and locations



{
"keywords": [
{
"mentions":
{
"location": {
"end": 157,
"begin": 141
},
"text": "Watson Discovery"
}
],
},
{
"tags":
{
"name": "project",
"value": "moonshot",
"operator": "stringEquals"
},
{
"name": "pipeline",
"value": "test",
"operator": "stringEquals"
}
]
}
],
"href": "https://iam.cloud.ibm.com/v1/policies/12345678-abcd-1a2b-a1b2-1234567890ab",
"created_at": "2018-08-30T14:09:09.907Z",
"created_by_id": "USER_ID",
"last_modified_at": "2018-08-30T14:09:09.907Z",
"last_modified_by_id": "USER_ID",
"state": "active"
}

{
"id": "12345678-abcd-1a2b-a1b2-1234567890ab",
"type": "access",
"description": "Viewer role access for all instances of SERVICE_NAME in the account.",
"subjects": [
{
"attributes":
{
"name": "iam_id",
"value": "IBMid-123453user"
}
]
}
],
"roles": [
{
"roles_id": "crn:v1:bluemix:public:iam::::role:Viewer"
}
],
"resources": [
{
"attributes":
{
"name": "accountId",
"value": "ACCOUNT_ID",
"operator": "stringEquals"
},
{
"name": "serviceName",
Parsing contracts 

The Element Classification enrichment is deprecated and will no longer be available, effective 10 July 2020.

Element Classification returns parsed contracts with an analysis of each identified element.

The following sections describe how the returned JSON provides the analysis.



 Types 

The Element Classification enrichment is deprecated and will no longer be available, effective 10 July 2020.

The types array includes a number of objects, each of which contains nature and party keys whose values identify a couplet for the element.

The following tables list the possible values of the nature and party keys.



 nature Description 

 Definition This element adds clarity for a term, relationship, or similar. No action is required to fulfill the element, nor is any party that is affected. 
 Disclaimer The party in the element is not obligated to fulfill the terms that are specified by the element but is not prohibited from doing so. 
 Exclusion The party in the element does not fulfill the terms that are specified by the element. 
 Obligation The party in the element is required to fulfill the terms that are specified by the element. 
 Right The party in the element is guaranteed to receive the terms that are specified by the element. 



Each nature key is paired with a party key, which contains either the name or the role of the party or parties that apply to the nature (examples include, but are not limited to, Buyer, IBM, or All Parties). For the Definition nature, the party is always None.





 Parties 

The Element Classification enrichment is deprecated and will no longer be available, effective 10 July 2020.

The parties array specifies the participants that are listed in the contract. Each party object is associated with other objects that provide details about the party, including:



* role: The party's role. Values are listed in the table that follows this list.
* importance: The importance of the party. Possible values are Primary for a primary party and Unknown for a non-primary party.
* addresses: An array that identifies addresses.



* text: An address.
* location: The location of the address as defined by its begin and end indexes.



* contacts: An array that defines the names and roles of contacts that are identified in the input document.



* name: The name of a contact.
"usage": "attribute",
"regularAggregate": "countDistinct",
"taxonomyFamily": "cStreetAddress"
}, {
"name": "AddressLine2",
"datatype": "NVARCHAR(60)",
"nullable": true,
"label": "Address Line 2",
"usage": "attribute",
"regularAggregate": "countDistinct",
"taxonomyFamily": "cStreetAddress"
}, {
"name": "City",
"datatype": "NVARCHAR(30)",
"nullable": false,
"label": "City",
"usage": "identifier",
"regularAggregate": "countDistinct",
"taxonomyFamily": "cCity"
}, {
"name": "StateProvinceID",
"datatype": "INTEGER",
"nullable": false,
"label": "State Province Id",
"usage": "identifier",
"regularAggregate": "countDistinct"
}, {
"name": "PostalCode",
"datatype": "NVARCHAR(15)",
"nullable": false,
"label": "Postal Code",
"usage": "identifier",
"regularAggregate": "countDistinct",
"taxonomyFamily": "cPostalCode"
}, {
"name": "rowguid",
"datatype": "CHAR(36)",
"nullable": false,
"label": "Rowguid",
"usage": "identifier",
"regularAggregate": "countDistinct"
}, {
"name": "ModifiedDate",
"datatype": "TIMESTAMP",
"description": "serviceId-test",
"unique_instance_crns": []
}
}
4. To attach the tag to your service ID, call the [Global Search and Tagging - Tagging API](https://cloud.ibm.com/apidocs/taggingdetach-tag) with the following command.

Replace {SERVICEID_CRN} with the value that you extracted in the previous step.

Replace (serviceidCRN) with the value that you extracted in the previous step.

Replace serviceidCrn with the value that you extracted in the previous step.

Replace serviceid_crn with the value that you extracted in the previous step.

Replace &serviceidCRN with the value that you extracted in the previous step.



* Curl
* Java
* Python
* Go
* Node



curl -X POST --header "Authorization: Bearer {iam_token}" --header "Accept: application/json" --header "Content-Type: application/json" -d '{ "resources": [{ "resource_id": "{SERVICEID_CRN}" }], "tag_names": ["tag_test_1", "tag_test_2"] }' "{base_url}/v3/tags/attach?tag_type=user"

Resource resourceModel = new Resource.Builder().resourceId(serviceidCRN).build();
AttachTagOptions attachTagOptions = new AttachTagOptions.Builder()
.addResources(resourceModel)
.addTagNames("tag_test_1")
.addTagNames("tag_test_2")
.build();

Response<TagResults> response = service.attachTag(attachTagOptions).execute();
TagResults tagResults = response.getResult();
System.out.println(tagResults.toString());

resource_model = {'resource_id': serviceid_crn}
"attach_user_tags": [
"hourly-backups"
],
"copy_user_tags": false,
"created_at": "2023-05-09T15:16:37Z",
"cron_spec": "0 /2 * * ",
"deletion_trigger": {
"delete_after": 5
},
"href": "https://us-south.iaas.cloud.ibm.com/v1/backup_policies/8758bd18-344b-486a-b606-5b8cb8cdd044/plans/6e251cfe-6f7b-4638-a6ba-00e9c327b178",
"id": "6e251cfe-6f7b-4638-a6ba-00e9c327b178",
"lifecycle_state": "stable",
"name": "my-hourly-plan-2",
"remote_region_policies": {
"delete_over_count": 5,
"encryption_key": "crn:v1:bluemix:public:kms:us-south:a/dffc98a0f1f0f95f6613b3b752286b87:e4a29d1a-2ef0-42a6-8fd2-350deb1c647e:key:5437653b-c4b1-447f-9646-b2a2a4cd617" ,
"region": [
{
"name": "us-east"
},
{
"href": "https://us-east.iaas.cloud.ibm.com/v1/regions/us-east/zones/us-east-2"
}
],
},
"resource_type": "backup_policy_plan"
}
Show more
* ABNF - [list-numbers.abnf](https://watson-developer-cloud.github.io/doc-tutorial-downloads/speech-to-text/grammars/list-numbers.abnf)
* XML - [list-numbers.xml](https://watson-developer-cloud.github.io/doc-tutorial-downloads/speech-to-text/grammars/list-numbers.xml)



* The following grammar defines a list of valid names from which the user can choose, perhaps to select an individual from a telephone directory:



* ABNF - [list-names.abnf](https://watson-developer-cloud.github.io/doc-tutorial-downloads/speech-to-text/grammars/list-names.abnf)
* XML - Not available









 Vehicle identification number grammars 

Vehicle identification numbers (VINs) use a fixed alphanumeric format, as do credit card numbers, telephone numbers, and US social security numbers. A great advantage of grammars over classical n-grams is that grammars are very effective for specifying such formats.

The VIN format is well standardized and has a fixed number of characters. Classic n-grams perform poorly for these types of tasks. Unlike grammars, they cannot ensure that the required number of characters is provided.

The following grammars recognize Honda VIN codes. They are more complex than the previous examples but demonstrate the power of grammars nicely.



* ABNF - [vins.abnf](https://watson-developer-cloud.github.io/doc-tutorial-downloads/speech-to-text/grammars/vins.abnf)
* XML - [vins.xml](https://watson-developer-cloud.github.io/doc-tutorial-downloads/speech-to-text/grammars/vins.xml)



For more information about the VIN format, see [Vehicle identification number](https://wikipedia.org/wiki/Vehicle_identification_number).





 Grammars with optional elements
ibmcloud cr image-tag us.icr.io/birds/bluebird:peck us.icr.io/animals/dog:bark







 ibmcloud cr image-untag 

Remove a [tag](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_overviewoverview_elements_tag), or tags, from each specified image in IBM Cloud Container Registry.

To remove a specific tag from an image and leave the underlying image and any other tags in place, use the ibmcloud cr image-untag command. If you want to delete the underlying image, and all its tags, use the [ibmcloud cr image-rm](https://cloud.ibm.com/docs/cli?topic=cli-containerregclibx_cr_image_rm) command instead.

ibmcloud cr image-untag IMAGE [IMAGE...]



 Prerequisites 

To find out about the required permissions, see [Access roles for using Container Registry](https://cloud.ibm.com/docs/Registry?topic=Registry-iamaccess_roles_using).





 Command options 

IMAGE
: The name of the image for which you want to remove the tag. You can delete the tag from multiple images at the same time by listing each image in the command with a space between each name. IMAGE must be in the format repository:tag, for example, us.icr.io/namespace/image:latest.

To find the names of your images, run ibmcloud cr image-list. Combine the content of the Repository column (repository) and Tag column (tag) separated by a colon (:) to create the image name in the format repository:tag. If a tag is not specified in the image name, the command fails.





 Example 

Remove the tag 1 from the image us.icr.io/birds/bluebird:1.

ibmcloud cr image-untag us.icr.io/birds/bluebird:1







 ibmcloud cr info 

Displays the name and the account of the registry that you are logged in to.

ibmcloud cr info



 Prerequisites 

None
"name": "categories" "label": "Category" "multiple_selections_allowed": true 
 "name": "natures" "label": "Nature" "multiple_selections_allowed": false 
 "name": "contract_terms" "label": "Contract Term" "multiple_selections_allowed": false 
 "name": "contract_payment_terms" "label": "Contract Payment Term" "multiple_selections_allowed": false 
 "name": "contract_types" "label": "Contract Type" "multiple_selections_allowed": false 
 "name": "contract_currencies" "label": "Contract Currency" "multiple_selections_allowed": false 
 "name": "invoice_buyers" "label": "Invoice Buyer" "multiple_selections_allowed": false 
 "name": "invoice_suppliers" "label": "Invoice Supplier" "multiple_selections_allowed": false 
 "name": "invoice_currencies" "label": "Invoice Currency" "multiple_selections_allowed": false 
 "name": "po_payment_terms" "label": "Purchase Order Payment Term" "multiple_selections_allowed": false 
 "name": "po_buyers" "label": "Purchase Order Buyer" "multiple_selections_allowed": false 
 "name": "po_suppliers" "label": "Purchase Order Supplier" "multiple_selections_allowed": false 
 "name": "po_currencies" "label": "Purchase Order Currency" "multiple_selections_allowed": false
Content-Type: multipart/mixed; boundary=_Part_1_701508.1145579811786
Accept-Language:en-us
Content-ID: <data>

--_Part_1_701508.1145579811786
Content-Disposition: form-data; name="COMMANDTORUN"
Content-Type: application/xml; charset=UTF-8
Content-Transfer-Encoding: 8bit
Accept-Language:en-us

<AppParam><id>COMMANDTORUN</id><value>sleep 99</value><type></type></AppParam>
--bqJky99mlBWa-ZuqjC53mG6EzbmlxB--
--_Part_1_701508.1145579811786--
' http://localhost:8080/platform/webservice/pacclient/submitapp
Show more
* text: A contract term, which is listed as a string.
* text_normalized: The normalized text, if applicable.
* interpretation: The details of the normalized text, if applicable.



* value: A string listing the value that was found in the normalized text.
* numeric_value: An integer or float expressing the numeric value of the value key.
* unit**: A string listing the unit of the value that was found in the normalized text.



* provenance_ids: An array that contains zero or more keys. Each key is a hashed value that you can send to IBM to provide feedback or receive support.
* location: The location of the contract term as defined by its begin and end indexes.



* payment_terms: An array that identifies the document's payment duration or durations.



* confidence_level: The confidence level of the identification of the payment term. Possible values include High, Medium, and Low.
* text: A payment term, which is listed as a string.
* text_normalized: The normalized text, if applicable.
* interpretation: The details of the normalized text, if applicable.



* value: A string listing the value that was found in the normalized text.
* numeric_value: An integer or double expressing the numeric value of the value key.
* unit**: A string listing the unit of the value that was found in the normalized text.



* provenance_ids: An array that contains zero or more keys. Each key is a hashed value that you can send to IBM to provide feedback or receive support.
* location: The location of the contract term as defined by its begin and end indexes.



* contract_currencies: An array that identifiea the document's contract currency or currencies.



* confidence_level: The confidence level of the identification of the contract currency. Possible values include High, Medium, and Low.
* text: A contract currency, which is listed as a string.
* text_normalized: The normalized text, if applicable.
{
"context": {
"my_text": "This is a text."
}
}

This syntax:

{
"context": {
"my_text": "<? $my_text.append(' More text.') ?>"
}
}

Results in this output:

{
"context": {
"my_text": "This is a text. More text."
}
}





 String.contains(String) 

This method returns true if the string contains the input substring.

Input: Yes, I'd like to go.

This syntax:

{
"conditions": "input.text.contains('Yes')"
}

Results: The condition is true.





 String.endsWith(String) 

This method returns true if the string ends with the input substring.

For this input:

"What is your name?".

This syntax:

{
"conditions": "input.text.endsWith('?')"
}

Results: The condition is true.





 String.extract(String regexp, Integer groupIndex) 

This method returns a string extracted by specified group index of the input regular expression.

For this input:

"Hello 123456".

This syntax:

{
"context": {
"number_extract": "<? input.text.extract('[\d]+',0) ?>"
}
}

Important: To process \d as the regular expression, you have to escape both the backslashes by adding another \: \\d

Result:

{
"context": {
"number_extract": "123456"
}
}





 String.find(String regexp) 

This method returns true if any segment of the string matches the input regular expression.
One option to provide a glossary or corpus of aligned text segments for training the Language Translator service is to create a valid UTF-8 encoded document that conforms to the [Translation Memory Exchange (TMX) version 1.4 specification](http://www.ttt.org/oscarStandards/tmx/). TMX is an XML specification that is designed for machine-translation tools. Submit files to the service either with the file extension .tmx or a content-type of application/x-tmx+xml. The following example is a TMX-formatted glossary file with two translation units (<tu> elements):

<?xml version="1.0" encoding="UTF-8"?>
<tmx version="1.4">
<header creationtool="" creationtoolversion=""
segtype="sentence" o-tmf="" adminlang="EN"
srclang="en" datatype="rtf" o-encoding="UTF-8" />
<body>
<tu>
<tuv xml:lang="en">
<seg>international business machines</seg>
</tuv>
<tuv xml:lang="fr">
<seg>International Business Machines</seg>
</tuv>
</tu>
<tu>
<tuv xml:lang="en">
<seg>patent</seg>
</tuv>
<tuv xml:lang="fr">
<seg>brevet</seg>
</tuv>
</tu>
</body>
</tmx>
Show more

Each term and translation pair must be enclosed in <tu> tags. The xml:lang attribute in the <tuv> tag identifies the language in which a term is expressed. The <seg> tag contains the term or the translation.

<tu>
<tuv xml:lang="en">
This SpEL expression shows the entities that were detected in the user input as an array. The array includes the entity name, location of the entity mention within the user input string, the entity mention string, and the level of confidence that your assistant has that the term is a mention of the entity type specified.
* To see details for all of the artifacts at once, including other properties, such as the value of a given context variable at the time of the call, you can inspect the entire API response. See Viewing API call details]]!
Service returns an array of intents, including Customer_Care_Cancel_Account and eCommerce_Cancel_Product_Order.Service returns an array of intents_ including Customer_Care_Cancel_Account and eCommerce_Cancel_Product_Order_.] ! .

<-- </ul> -->

4. Temporarily remove the description you added to the external node name field for at least one of the nodes that you anticipate will be listed as a disambiguation option.5. Enter the test utterance into the Try it out pane again.
If you added the <? intents ?> expression to the response, then the text returned includes a list of the intents that your assistant recognized in the test utterance, and includes the confidence score for each one.

!]<-- </ol> -->After you finish testing, remove any SpEL expressions that you appended to node responses, or add back any original responses that you replaced with expressions, and repopulate any external node name fields from which you removed text.Again, keep in mind that the options included in the list and the order of the options might change from one test run to the next. This behavior is intended; the options are randomized on purpose.Previous topic:Gathering information with slots]] Next topic:Improving your conversation]] ! ! ! ! !
3. Click Finish. Give Discovery a few minutes to start creating documents. You can use the Manage collections page within the project to see the progress.

After the digestion process is completed, a summary page is displayed in Discovery, which is hosted in a separate web browser tab.
4. Wait for the collection to be fully ingested, then click Back to Watson Assistant.







 Configure the search 



1. On the Watson Assistant search skill page, ensure the Discovery project you want to use is selected, then click Next.
2. In the Configure result content section, review the Discovery fields and examples that are used in the search results shown to your customers. You can accept the defaults, or customize them as you want.

The appropriate collection fields to extract data from vary depending on your collection's data source and how the data source was enriched. After you choose a data collection type, the collection field values are prepopulated with source fields that are considered most likely to contain useful information given the collection's data source type. However, you know your data better than anyone. You can change the source fields to ones that contain the best information to meet your needs.

To learn more about the structure of the documents in your collection, including the names of fields that contain information you might want to extract, open the collection in Discovery, and then use the Identity fields and Manage fields tabs.

Each search result can consist of the following sections:



* Title: Search result title. Use the title, name, or similar type of field from the collection as the search result title.

You must select something for the title or no search result response is displayed in the Facebook and Slack integrations.
* Body: Search result description. Use an abstract, summary, or highlight field from the collection as the search result body.

You must select something for the body or no search result response is displayed in the Facebook and Slack integrations.
* URL: This field can be populated with any footer content that you want to include at the end of the search result.

For example, you might want to include a hypertext link to the original data object in its native data source.