vania-janet/multiturn-rag-retrieval-data huggingface dataset:
LOGDNA_TAGS Define tags to group hosts automatically into dynamic groups. production,serviceA 







 Linux: configuration parameters for logging agent V1 



Table 7. Configuration options for the logging agent V1

 Parameter Description 

 tags Define tags to group hosts automatically into dynamic groups. 
 logdir Define custom paths that you want the agent to monitor. <br>Separate multiple paths by using commas. You can use glob patterns. You can configure specific files. Enter glob patterns by using double quotation marks. 
 exclude Define the files that you do not want the logging agent to monitor. Note: These files can be located in any of the paths that are defined through the logdir parameter. <br>Separate multiple files by using commas. You can use glob patterns. You can configure specific files. 
 exclude_regex Define regex patterns to filter out any lines that match the pattern. Do not include leading and trailing /. 
 hostname Define the hostname. This value overrides the operating system hostname. 
 autoupdate Set to 1 to update the agent automatically when the public repo agent definition is updated. Set to 0 to disable this feature. 







 Configure tags to group data 

You can configure tags at the agent level so that all lines that are sent by this agent can be grouped automatically into a group when you filter data in a view.



* You can define multiple tags.
* You separate tags by using commas.
* The maximum number of characters that you can set to define multiple tags is 80 characters.





Table 8. Adding tags

 Platform How to install and configure 

 Kubernetes cluster [Adding tags to logs from a Kubernetes cluster](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-adding_tagsadding_tags_kube) 
 Linux Ubuntu or Debian [Adding tags to logs from Linux Ubuntu or Debian](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-adding_tagsadding_tags_linux)
Global Tagging 



 Resources Data sources 

 <br><br> * [ibm_resource_tag](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/resource_tag)<br><br><br> <br><br> * [ibm_resource_tag](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/data-sources/resource_tag)<br><br><br> 







 Hyper Protect Crypto Services 



 Resources Data sources
Editing engine description and tags 

To edit the description and tags of an engine, use one of the following methods:



*  Editing engine description and tags in list view





1.  Click the name of engine. Engine information window opens.
2.  In Details tab, click Edit.
3.  In the Description field, enter the description of the engine or edit the existing description.
4.  In the Tags field, select the tags from the list or start typing to define a new tag.
5.  Click Save.





*  Editing engine description and tags in topology view





1.  Click the engine. Engine information window opens.
2.  In Details tab, click Edit.
3.  In the Description field, enter the description of the engine or edit the existing description.
4.  In the Tags field, select the tags from the list or start typing to define a new tag.
5.  Click Save.
<span id="comments-content-2">All snow must go!</span>
</span>
</body></html>

After the preceding HTML is ingested and enhanced, Discovery returns the following JSON:

{
"extracted_metadata": { ... },
"html": "...",
"text": "...",
"extracted_fields": {
"authors": [ "Jane Rain", "Joe Snow" ],
"authors_str": "Jane RainnnJoe Snow",
"chapters": [ "The Rain in Spainnnfalls mainly on the plain.", "How I Learned to Stop Worryingnnand love my snowblower." ],
"comments": [! "Rain gives me pain.", "All snow must go" ]
}
}

After deciding which HTML elements you want to extract, you can then further modify the configuration file to specify the enrichments you want to apply to them.



 Common selectors 

Some common CSS selectors include the following:



* tag — Matches the tag name
* .class — Matches the value of class
* id — Matches the value of id
* [attribute] — Matches any tag with the specified attribute, regardless of value
* [attribute=value] or [attribute="value"] — Matches the specified attribute and value











 Splitting documents with document segmentation 

If your source documents are supported by Smart Document Understanding (see [Supported document types](https://cloud.ibm.com/docs/discovery?topic=discovery-sdudoctypes)), use document splitting instead of document segmentation. For details, see [document splitting](https://cloud.ibm.com/docs/discovery?topic=discovery-sdusplitting).

You can split your HTML documents into segments based on HTML heading tags. Once split, each segment is a separate document that is enriched and indexed separately. Since queries return these segments as separate documents, document segmentation can be used to:
Parses elements (sentences, lists, tables) in governing documents to classify important types and categories For more information, see [Element classification](https://cloud.ibm.com/docs/discovery?topic=discovery-element-classification).

[Smart Document Understanding](https://cloud.ibm.com/docs/discovery?topic=discovery-sdu) is unavailable if this enrichment is used.



 Enrichment pricing 

Enrichment pricing information is available on [IBM Cloud](https://cloud.ibm.com/catalog/services/discovery).





 Enrichment language support 

For information about enrichment language support, see [Discovery language support](https://cloud.ibm.com/docs/discovery?topic=discovery-language-support).







 Understanding the difference between Entities, Concepts, and Keywords 

At first glance, Entity extraction, Concept tagging and Keyword extraction appear to be similar enrichments. We'll use the text from the enrichment examples to show the differences between them.

"text": "The stockholders were pleased that Acme Corporation plans to build a new factory in Atlanta, Georgia."

Since the Entity Extraction enrichment extracts persons, places, and organizations in the input text, Entity extraction returns the following entity types:

"type": "City"
"text": "Atlanta"

"type": "Company"
"text": "Acme"

"type": "StateOrCounty"
"text": "Georgia"

Since the Concept tagging enrichment understands how concepts relate, it can identify concepts that are not directly referenced in the text. For example, if an article mentions CERN and the Higgs boson, it identifies Large Hadron Collider as a concept even if that term is not mentioned explicitly. Since our example document text is only one sentence, there are no related concepts, so Concept tagging returns the following concepts:

"text": "Acme Corporation"
"text": "factory"

Since the Keyword extraction enrichment identifies content typically used when indexing data, generating tag clouds, or searching, Keyword extraction returns the following keywords:

"text": "Acme Corporation"
"text": "new factory"
Re-upload the documents in your collection. After uploading is complete, you are redirected to the Overview screen.





Default field labels

 Field Definition 

 answer In a Q/A pair (often in an FAQ), the answer to the question. 
 author Name of author (or authors). 
 footer Use this tag to denote meta-information about the document (such as the page number or references), that appear at the bottom of the 
 header Use this tag to denote meta-information about the document that appears at the top of the page. 
 question In a Q/A pair (often in an FAQ), the question. 
 subtitle The secondary title of the document being annotated. 
 table_of_contents Use this tag on listings in the document table of contents. 
 text Use this tag for standard copy text, including paragraphs, definitions, or any set of words that is not a title, part of a table, answer, author, subtitle, header, or a footer. 
 title The main title of the document being annotated. 
 table Use this tag to annotate tables in your document. 
 image Use this tag to annotate images and diagrams in your document. 







 Splitting documents 

The Manage fields tab contains the option to Improve query results by splitting your documents. This option allows you to split your documents into segments based on a field name. After the document splits, each segment is a separate document that is enriched, indexed, and returned as a separate query result.

Documents are split based on a single field name, for example: title, author, question.

Considerations:



* The number of segments per document is limited to 250. Any document content remaining after 249 segments are stored within segment 250.
* Each segment counts towards the document limit of your plan. Discovery indexes segments, until the plan limit is reached. For document limits, see [Discovery pricing plans](https://cloud.ibm.com/docs/discovery?topic=discovery-discovery-pricing-plans).
* PDF and Word metadata, as well as any custom metadata, is extracted and included in the index with each segment. Every segment of a document includes identical metadata.
curl -s -H'Content-type:application/json' -H'Accept:application/json' -H"Authorization:Bearer $IAM_TOKEN" -X POST 'https://api.global-search-tagging.cloud.ibm.com/v3/resources/search' -d'{"query":"crn:"crn:v1:bluemix:public:codeengine:us-south:a/aabbccddeeff00112233445566778899:aabbccdd-1234-4321-aaaa-001122334455::"", "fields":["crn","name","tags", "access_tags", "service_tags"]}'

If the resource with the given CRN exists and you can read it, then you get the following example output, which includes all of the tag types attached to the resource:

{
"items": [
{
"name": "my-codeengine-instance",
"crn": "crn:v1:bluemix:public:codeengine:us-south:a/aabbccddeeff00112233445566778899:aabbccdd-1234-4321-aaaa-001122334455::",
"tags": "one", "two"],
"access_tags": "atag:one", "another-tag:two"],
"service_tags": "service-tag:one", "service-tag:two"]
}
],
"limit": 10,
"search_cursor": "..."
}

You can also use the [Get tags](https://cloud.ibm.com/apidocs/tagginglist-tags) API with the attached_to query parameter to obtain the same result. However, the Get tags API is strictly rate limited and doesn't return all of the tag types in a single call as the /v3/resources/search does which is the reason it's not the suggested method.
If this value is set, anything that matches one of the XPaths are removed from the output.





 keep_content 

"xpaths" : array - An array of XPaths that identify content that is converted. If this value is set, anything that matches one of the XPaths are included in the output. The inclusions specified by this parameter are processed after any processing specified by exclude_content.





 exclude_tag_attributes 

"exclude_tag_attributes" : array - An array of HTML attribute names that are removed by the conversion regardless of which HTML tag they are present in.

You receive an error message if you specify both exclude_tag_attributes and keep_tag_attributes in the same configuration - only one can be specified per configuration. If present, keep_tag_attributes must be completely removed from the configuration; it cannot be present as an empty array.





 keep_tag_attributes 

"keep_tag_attributes" : array - An array of HTML attribute names that are retained by the conversion.

You receive an error message if you specify both keep_tag_attributes and exclude_tag_attributes in the same configuration - only one can be specified per configuration. If present, exclude_tag_attributes must be completely removed from the configuration; it cannot be present as an empty array.





 extracted_fields 

This object defines any content from the HTML source that is to be extracted into a separate JSON field as part of the conversion. The content is identified by using CSS selectors.

Each field that you want to create is defined by an object as follows:

"{field_name}": {
"css_selector": "{CSS_selector_expression_1}",
"type": "{field_type}"
}



* "{field_name}" - The name of the field to be created.



Field names defined in your configuration must meet the restrictions defined in [Field Name Requirements](https://cloud.ibm.com/docs/discovery?topic=discovery-configreffield_reqs).



* "css_selector" : stringrequired - a CSS selector expression that defines the area of content to be stored in a field.
* <HOSTS> represents the list of hosts for which data is included in the view. Multiple hosts are separated by commas, for example, hosts=logging-agent-trkq9,logging-agent-trkq7.
* <APPS> represents the list of apps for which data is included in the view. Multiple apps are separated by commas, for example, apps=autoscaler,catalog-operator.
* <LEVELS> represents the list of levels for which data is included in the view. Multiple levels are separated by commas, for example, levels=warn,debug.
* <TAGS> represents the list of tags for which data is included in the view. Multiple tags are separated by commas, for example, tags=agent-v2,k8s.
* <TIMEFRAME> represents the timeframe that you apply to the data that is displayed through the view. For example, look at the following samples:

When you specify a timeframe of July 12, the value is t=July%2012.

When you specify a timeframe of July 12, 2020, the value is t=July%2012%2C%202020.

When you specify a timeframe of July 12, 2020 to July 15,2020, the value is t=July%2012%2C%202020%20to%20July%2015%2C2020.





You might need to log in to IBM Cloud to view the data if your token has expired or you are not logged in.
* Optional: The LANGUAGE the content is in.



* Flag: --content-language LANGUAGE



* Optional: SIZE of the body in bytes. This parameter is useful when the size of the body cannot be determined automatically. (default: 0)



* Flag: --content-length SIZE



* Optional: The base64-encoded 128-bit MD5 digest of the data.



* Flag: --content-md5 MD5



* Optional: A standard MIME type describing the format of the object data.



* Flag: --content-type MIME



* Optional: A MAP of metadata to store.



* Flag: --metadata MAP JSON Syntax: The --metadata flag takes the file:// prefix that is used to load the JSON structure from the specified file.





{
"file_name": "file_20xxxxxxxxxxxx45.zip",
"label": "texas",
"state": "Texas",
"Date_to": "2019-11-09T16:00:00.000Z",
"Sha256sum": "9e39dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx8ce6b68ede3a47",
"Timestamp": "Thu, 17 Oct 2019 09:22:13 GMT"
}



* Optional: The REGION where the bucket is present. If this flag is not provided, the program uses the default option that is specified in config.



* Flag: --region REGION



* Optional: Output FORMAT can be only json or text.



* Flag: --output FORMAT









 Upload objects by using S3Manager 



* Action: Upload objects to COS concurrently.
* labels: These are the lists of category labels, in the order of their hierarchy. For example, if you want to add labels with a hierarchy where B is a child of A, the list of labels here would be:

"labels": ["A", "B"]
* key_phrases: These are lists of phrases that can uniquely identify the corresponding labels, for example:

"key_phrases": ["films", "action movies"]



2. Up to 5 levels of hierarchy are accepted in labels. Following is an example training data format:

[
{
"labels":
"level1"
],
"key_phrases":
"key phrase",
"key phrase 2"
]
},
{
"labels":
"level1",
"level2"
],
"key_phrases":
"key phrase 3",
"key phrase 4"
]
}
]
Show more



If you do use a label hierarchy, you must define key_phrases for each label level.







 Training a custom categories model 

When your training data is ready, use the Create categories model method to create your custom model. Make sure to substitute your credentials for {apikey} and {url}, and use the path to your training data file in the training_data parameter.

curl -X POST -u "apikey:{apikey}" -H "Content-Type: multipart/form-data" -F "name=MyCategoriesModel" -F "language=en" -F "model_version=1.0.1" -F "training_data=@categories_data.json;type=application/json" "{url}/v1/models/categories?version=2021-02-16"

Use the model_id in the response to check the status of your model.





 Checking the status of categories models 

The following sample request for the Get categories model method checks the status for the model with ID 714a50d1-36c7-4a57-a790-99f13cc9301c.
.tagType("access")
.build();

Response<TagResults> response = service.attachTag(attachTagOptions).execute();
TagResults tagResults = response.getResult();
System.out.println(tagResults.toString());

resource_model = {'resource_id': resource_crn}

tag_results = global_tagging_service.attach_tag(
resources=[resource_model],
tag_names=['project:myproject'],
tag_type='access').get_result()

print(json.dumps(tag_results, indent=2))

resourceModel := &globaltaggingv1.Resource{
ResourceID: &resourceCRN,
}

attachTagOptions := globalTaggingService.NewAttachTagOptions(
[]globaltaggingv1.Resource{resourceModel},
)
attachTagOptions.SetTagNames([]string{"project:myproject"})
attachTagOptions.SetTagType("access")

tagResults, response, err := globalTaggingService.AttachTag(attachTagOptions)
if err != nil {
panic(err)
}
b, _ := json.MarshalIndent(tagResults, "", " ")
fmt.Println(string(b))

const resourceModel = {
resource_id: resourceCrn,
};

const params = {
resources: [resourceModel],
tagNames: ["project:myproject"],
tagType: 'access',
};

globalTaggingService.attachTag(params)
.then(res => {
console.log(JSON.stringify(res.result, null, 2));
})
.catch(err => {
What access is required to attach or detach tags? 



* To manage user tags on a resource, you must have at least the editor role for IAM-enabled resources.
* To manage user tags on a service ID, you must have at least the editor role on the IAM Identity service.
* To manage access management tags, you must have the administrator role on the targeted resource.
* To manage access management tags on service IDs, you must have the administrator role on the IAM Identity service.



For more information, see [Granting users access to tag resources](https://cloud.ibm.com/docs/account?topic=account-access).





 Can users in my account view my tags? 

Yes, tags are visible throughout your account. If your account users have permission to view a resource, they can also view all tags that are attached to those resources. For more information, see [Granting users access to tag resources](https://cloud.ibm.com/docs/account?topic=account-access).





 Can I delete my tags? 

Before you can delete a tag, you must detach it from all resources. The tag might be attached to a resource that you don't have permission to view. The same tag can be attached to several resources by different users in the same billing account. Users don't have the same visibility on all resources on the account. Contact the account owner who can resolve the problem by detaching the tag from the blocking resource.

If you still can't delete it, the tag might be attached to a reclaimed resource. You can use the IBM Cloud CLI to manage the reclamation process of specific resources. For more information, see [Using resource reclamations](https://cloud.ibm.com/docs/account?topic=account-resource-reclamation).

When you delete an access management tag from the account, any associated IAM policies are also deleted with it.





 How do I view my dashboard role? 

You can view the role that you are assigned on a dashboard by going to Manage > Account > Dashboards in the IBM Cloud console. All users besides the dashboard owner have the viewer role on the dashboard.





 Can I edit the default dashboard?
.addTagNames("tag_test_2")
.build();

Response<TagResults> response = service.attachTag(attachTagOptions).execute();
TagResults tagResults = response.getResult();
System.out.println(tagResults.toString());

resource_model = {'resource_id': serviceid_crn}

tag_results = global_tagging_service.attach_tag(
resources=[resource_model],
tag_names=['tag_test_1', 'tag_test_2'],
tag_type='user').get_result()

print(json.dumps(tag_results, indent=2))

resourceModel := &globaltaggingv1.Resource{
ResourceID: &serviceidCRN,
}

attachTagOptions := globalTaggingService.NewAttachTagOptions(
[]globaltaggingv1.Resource{resourceModel},
)
attachTagOptions.SetTagNames([]string{"tag_test_1", "tag_test_2"})
attachTagOptions.SetTagType("user")

tagResults, response, err := globalTaggingService.AttachTag(attachTagOptions)
if err != nil {
panic(err)
}
b, _ := json.MarshalIndent(tagResults, "", " ")
fmt.Println(string(b))

const resourceModel = {
resource_id: serviceidCrn,
};

const params = {
resources: [resourceModel],
tagNames: ["tag_test_1", "tag_test_2"],
tagType: 'user',
};

globalTaggingService.attachTag(params)
.then(res => {
* TAG_DATA are comma-separated tags that are formatted as TAG_NAME:TAG_VALUE. You can associate one or more tags to your monitoring agent. For example: role:serviceX,location:us-south. Later on, you can use these tags to identify metrics from the environment where the agent is running.
* Set sysdig_capture_enabled to false to disable the capture feature. By default is set to true. For more information, see [Working with captures](https://cloud.ibm.com/docs/monitoring?topic=monitoring-capturescaptures).



6. Verify that the monitoring agent is created successfully and its status. Run the following command:

kubectl get pods -n ibm-observe

The deployment is successful when you see one or more sysdig-agent pods. The number of sysdig-agent pods equals the number of worker nodes in your cluster. All pods must be in a Running state.







 Step 3: Launch the monitoring UI 

To launch the monitoring UI through the IBM Cloud console, complete the following steps.



1. [Log in to your IBM Cloud account](https://cloud.ibm.com/login).

After you log in with your user ID and password, the IBM Cloud Dashboard opens.
2. From the menu ![Menu icon](https://cloud.ibm.com/docs-content/v1/content/icons/icon_hamburger.svg), select Observability.
3. Select Monitoring. The list of instances that are available on IBM Cloud is displayed.
4. Find your instance and click Open dashboard. The web UI opens.



It may take some time before you see the cluster entry while the information is initally collected and processed by the monitoring agent.

You only can monitor one instance per browser. You could have multiple tabs for the same instance.





 Step 4: Monitor your cluster 

In the Advisor tab, you can monitor and troubleshoot the health, risk, and capacity of hosts and Kubernetes clusters.

Zoom

!
A snippet is a response that you author and that is returned when the associated document is returned for the specified natural language query.

{
"curations": [
{
"curation_id": "c1175536f509405bc68a9f76235fa7bbb6f9af2f",
"natural_language_query": "What is a project",
"curated_results":
{
"collection_id": "47477591-b520-6039-0000-017ea213e837",
"document_id": "web_crawl_123a2a56-8c26-5acb-9544-c4702ac899a4",
"snippet": "A project is a convenient way to collect and manage the resources in your application. You can assign a project type and connect your data to the project by creating a collection."
}
]
}
]
}
Show more

If passages.per_document is true, the text snippet that you specify is returned as the top passage in the passage_text field instead of the original passage that is chosen by search. Only one text snippet can be specified per document. If passages.max_per_document is greater than 1, the snippet is returned first, followed by the passages that are chosen by search. Query filters are applied to curation results.