vania-janet/multiturn-rag-retrieval-data huggingface dataset:
For example, to get the list of actions for a user, you can run the following query:

SELECT _source.eventTime, _source.action, _source.o_target.name
FROM cos://eu-gb/sql-results/jobid=3aa9e732-ba88-4ffe-b9fc-b8a265876467 STORED AS PARQUET
WHERE _source.o_initiator.name = "xxx@ibm.com"
ORDER BY _source.eventTime
INTO cos://eu-gb/sql-results STORED AS CSV

For example, to get the event time, the action, the criticality of the action, and the outcome, you can run the following query:

SELECT _source.eventTime AS EVENTTIME, _source.action AS ACTION, _source.severity AS SEVERITY, _source.outcome AS OUTCOME FROM PARQUET_FILE STORED AS PARQUET
INTO RESULTS_BUCKET STORED AS CSV





 Step 3.7. Run a query to get a subset of the event fields ordered by the event time 

To see information about each event, run the following query:

SELECT FIELDS FROM PARQUET_FILE STORED AS PARQUET
ORDER BY _source.eventTime
INTO RESULTS_BUCKET STORED AS CSV

Where



* FIELDS is the list of fields that you want to get information on for the different records. For example, you can enter _source.eventTime AS EVENTTIME, _source.action AS ACTION, _source.severity AS SEVERITY, _source.outcome AS OUTCOME
* PARQUET_FILE is the Result location URL that you get when you transform the archive file from JSON to PARQUET
* RESULTS_BUCKET is the SQL URL of the custom COS bucket that you plan to use to upload the query results



For example, to get the event time, the action, the criticality of the action, and the outcome, you can run the following query:

SELECT _source.eventTime AS EVENTTIME, _source.action AS ACTION, _source.severity AS SEVERITY, _source.outcome AS OUTCOME FROM PARQUET_FILE STORED AS PARQUET ORDER BY _source.eventTime
INTO RESULTS_BUCKET STORED AS CSV





 Step 3.8.
Use built-in Watson NLP to find common terms 

Take advantage of award-winning Watson Natural Language Processing (NLP) capabilities by adding prebuilt enrichments to your documents.

With Watson NLP, you can identify and tag meaningful information in your collections so you can understand what it all means and make more informed decisions.

The following Watson NLP enrichments are available:



* [Entities](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-nlunlu-entities): Recognizes proper nouns such as people, cities, and organizations that are mentioned in the content.
* [Keywords](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-nlunlu-keywords): Recognizes significant terms in your content.
* [Part of Speech](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-nlunlu-pos): Identifies the parts of speech (nouns and verbs, for example) in the content.
* [Sentiment](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-nlu-sentiment): Understands the overall sentiment of the content.



The following other pretrained enrichments are available with Discovery:



* [Contracts](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-contracts-schema)
* [Document structure](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-sdu-pretrained)
* [Table understanding](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-understanding_tables)





 Watson NLP enrichments 

For example, the following screen capture shows a transcript of the US Declaration of Independence that was added to a Discovery collection where the Entities and Keywords enrichments are enabled. The mentions that are recognized by the enrichments are highlighted in the document text.

Zoom

![Shows an excerpt of the US Declaration of Independence with several terms highlighted.]
Click [this link ![External link icon](https://cloud.ibm.com/docs-content/v1/content/icons/launch-glyph.svg)](https://www.youtube.com/watch?v=1VoS-xczBow&feature=youtu.be) to watch a less than 2 minute video that illustrates how to export a model and use it in IBM Watson Explorer.





 Procedure 



1. [Export a machine learning model](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-publish-mlexporting-a-machine-learning-model).
2. From the IBM Watson Explorer application, import the model.

You can then map the model to a machine learning model in Watson Explorer Content Analytics. After you perform the mapping step, when you crawl documents, the model finds instances of the entities and relations that your model understands. To learn how to import and configure the model in IBM Watson Explorer, see the technical document that describes the integration: [http://www.ibm.com/support/docview.wss?uid=swg27048147 ![External link icon](https://cloud.ibm.com/docs-content/v1/content/icons/launch-glyph.svg)](http://www.ibm.com/support/docview.wss?uid=swg27048147).





 Related tasks 

[Exporting analyzed documents from Watson Explorer Content Analytics](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-preannotationwks_uimawexca)
Start building your query in the query pane. Use the Document Explorer to explore the schema and documentation. The Document Explorer shows the different available options for datasets, dimensions, operations, and functions. The words "zone" and "domain" are synomyous in the Document Explorer.

Paste the following test snippet into the query pane and observe the response in the response pane, adjusting the datetime to meet your needs.

query {
viewer {
zones(filter: {zoneTag: "<your domain ID>"}) {
httpRequests1hGroups(limit: 5, filter: { datetime_gt: "2020-08-30T04:00:00Z", datetime_lt: "2020-08-31T06:00:00Z"}) {
sum {
countryMap {
bytes
clientCountryName
}
}
dimensions {
date
datetime
}
}
firewallEventsAdaptiveGroups(limit: 10, filter: { datetime_gt: "2020-08-30T04:00:00Z", datetime_lt: "2020-08-31T06:00:00Z"}) {
count
dimensions {
clientCountryName
clientAsn
datetimeHour
}
}
}
}
}
Show more







 Querying basics 

GraphQL structures data as a graph. To get the data you need, you can explore the edges of the graph. This is an example query format:

viewer {
zones(filter:...) {
requests(filter:...) {
date, time, bytes,...
}
}
}

The initial node of the user running the query is viewer. A viewer is able to access one or more domains (zones). Each zone contains different datasets, such as firewall events for a zone.

Nodes that represent aggregated data include the groups suffix, for example, firewallEventsAdaptiveGroups.
See [Copying a dialog node](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-tasksdialog-tasks-copy-node) for more information.

Capture groups in pattern entities
: You can identify groups in the regular expression pattern that you define for an entity. Identifying groups is useful if you want to be able to refer to a subsection of the pattern later. For example, your entity might have a regex pattern that captures US phone numbers. If you identify the area code segment of the number pattern as a group, then you can subsequently refer to that group to access just the area code segment of a phone number. See [Defining entities](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-creating-task) for more information.





 6 December 2017 

IBM Cloud® Functions integration (Beta)
: Call IBM Cloud® Functions (formerly IBM OpenWhisk) actions directly from a dialog node. This feature enables you to, for example, call an action to retrieve weather information from within a dialog node, and then condition on the returned information in the dialog response. Currently, you can call an action from a Cloud Functions instance that is hosted in the US South region from Watson Assistant instances that are hosted in the US South region. See [Making programmatic calls from a dialog node](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-actions-client) for more details.





 5 December 2017 

Redesigned UI for Intents and Entities
: The Intents and Entities tabs have been redesigned to provide an easier, more efficient workflow when creating and editing entities and intents. See [Defining intents](https://cloud.ibm.com/docs/assistant?topic=assistant-intents-create-task) and [Defining entities](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-creating-task) for information about working with these tabs.





 30 November 2017 

Eastern Arabic numeral support
: Eastern Arabic numerals are now supported in Arabic system entities.





 29 November 2017
Select the Menu icon ![Menu icon](https://cloud.ibm.com/docs-content/v1/content/f02b9c9adcbb7328d95a45e105cec371d50f15eb/vpc//images/menu_icon.png), then click VPC Infrastructure > Flow Logs. The Flow logs for VPC dashboard appears.

Zoom

![Flow log collector dashboard](https://cloud.ibm.com/docs-content/v1/content/f02b9c9adcbb7328d95a45e105cec371d50f15eb/vpc//images/list-view-01.png)

Figure 1. Flow log collector dashboard
3. Click Create flow log collector to go to the flow logs provisioning page.
4. Enter values for the following fields:



* Name - Type a unique name for your flow log collector.
* Resource group - Select a resource group for your flow log collector. You can use the default group for this flow log, or choose from the list (if defined). For more information, see [Best practices for organizing resources in a resource group](https://cloud.ibm.com/docs/account?topic=account-account_setup).
* Tags - Optionally, add tags to organize, track usage costs, or manage access to your resources.
* Access management tags - Optionally, add access management tags to resources to help organize access control relationships. The only supported format for access management tags is key:value. For more information, see [Controlling access to resources by using tags](https://cloud.ibm.com/docs/account?topic=account-access-tags-tutorial).



5. From the Attach the flow log connector to menu, choose a target type for the flow log. Depending on your selection, additional fields might be required.



* Virtual private cloud - Select a VPC. All network traffic within the selected VPC is logged.
* Subnet - Select a VPC and a subnet within the selected VPC.
* [Microsoft SharePoint OnPrem](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesconnectsp_op)
* [Web Crawl](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesconnectwebcrawl)
* [IBM Cloud Object Storage](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesconnectcos)



You can connect to a data source using the Discovery tooling or the API. The Discovery tooling provides a simplified method of connection that requires less understanding of the source systems, and the API provides a more granular and highly configurable interface, which requires a greater understanding of the source that you are connecting to. Consult the following process overview to see which sections of this document to read next:



1. Read the [General Source Requirements](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesgen_req).
2. Read the requirements for your data source. For the available data sources, see the list above.
3. Read the instructions to connect to Discovery by using one of the following methods:



* [Using the tooling](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcessource_tooling)
* [Using the API](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcessource_api)





To learn about integrating Discovery with App Connect, see [Adding content](https://cloud.ibm.com/docs/discovery?topic=discovery-addcontent).

If you select an on-premises data source, you must first install and configure IBM® Secure Gateway for IBM Cloud®. For more information about installing IBM® Secure Gateway for IBM Cloud®, see [Installing IBM Secure Gateway for on-premises data](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesgateway).



 Box 

You'll need to create a new Box custom application to connect to IBM Watson™ Discovery.
You can use the pipelinectl interface to access your repos and artifacts by using the list_repos, load_repo, list_artifacts, and load_artifact commands. For more information about the commands, see the [pipelinectl](https://cloud.ibm.com/docs/devsecops?topic=devsecops-devsecops-pipelinectl) documentation.





 Collecting compliance data on build 

When the pipeline runs successfully, you can collect information about the build.

Evidence is collected on all checks, scans, tests, and artifact signing into the evidence locker. The pipeline log files are also saved to the locker, along with the pipeline data itself, containing the Tekton definitions. Compliance data on peer reviews is also collected in this step. The pipeline uses pipelinectl to search for repos with pull requests that were merged since the last build. It also checks their review status, saves it as an artifact, and creates evidence based on the result.

The final script is an evaluator, which marks the pipeline status green or red, based on the evidence statuses. If they contain any failures, the continuous integration run is marked as red.
Analyzing documents based on their structure 

Create a model that understands the content of a document based on the document's format and structure.

First, decide whether you want to use a pretrained model or define your own.

Pretrained model
:   Applies a noncustomizable model that extracts text and identifies tables, lists, and sections.

Instead of training the model yourself, you can apply an existing model that is trained to identify tables, lists, and sections in various types of documents.

If capturing information from tables is critical to your use case, consider using a pretrained model.

For more information, see [Apply a pretrained SDU model](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-sdu-pretrained).

User-trained model
:   Opens the Smart Document Understanding tool that you can use to pick certain types of text to store in fields other than the text field.

When you label a section of a document as a custom field, later you can apply enrichments to the field or split your documents on each occurrence of the field. You can search or filter by the field, or omit the field from the index.

For more information, see [Define a user-trained SDU model](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-configuring-fields).

Text extraction only
:   Indexes any text that is recognized in the source documents in the text field. This option is used by default.
Understand tables 

Apply the Table Understanding enrichment to get detailed information about tables and table-related data within documents.

The following tasks generate an HTML field with table information and apply the Table Understanding enrichment to it for your collection automatically:



* If you use the Smart Document Understanding tool to define a user-trained or pretrained SDU model, the Table Understanding enrichment is applied to the html field that is generated for the collection.
* If you create a Document Retrieval for Contracts project type, a pretrained SDU model is applied to your collection automatically. As a result, the Table Understanding enrichment is applied to the html field that is generated for the collection.

For more information, see [Smart Document Understanding](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-configuring-fieldsidentify-fields).





 Before you begin 

The documents in your collection must contain a field with HTML representations of your tables. This information often is stored in the html field. If your collection consists of CSV or JSON files, it might have a field other than the html field that contains table information in HTML format.





 Applying the table understanding enrichment 

You can apply the enrichment only to a field that contains an HTML representation of the table.

To apply the enrichment, complete the following steps:



1. From the navigation pane, open the Manage collections page, and then click a collection to open it.
2. Click the Enrichments tab.
3. Find the Table Understanding enrichment.
4. Select the html field from the field list.

Choose the field that contains HTML representations of the tables.



After the enrichment is applied, you can get valid results when you submit queries that require Discovery to find information that is stored in tables.

A developer can query tables by using the API. For more information, see [Query parameters](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-query-parameterstable_retrieval).
If you created snapshots in a different region, expand the list and select the region.
3. As your list of snapshots grows, use the filter to indicate the number of snapshots to display per page. Use the page navigation arrows to move forward and back through the list.



Table 1 describes the information for all snapshots in the list of snapshots.



Table 1. List of all snapshots

 Field Description 

 Name The name that you provided when you created the snapshot. Click the name of the snapshot to see its [details](https://cloud.ibm.com/docs/vpc?topic=vpc-snapshots-vpc-view&interface=clisnapshots-vpc-view-snapshot-ui). 
 Status Status of the snapshot, depending on whether it's usable (active status), unusable, or being created. For more information, see [Snapshot statuses](https://cloud.ibm.com/docs/vpc?topic=vpc-snapshots-vpc-managesnapshots-vpc-status). 
 Encryption Shows [IBM-managed encryption](https://cloud.ibm.com/docs/vpc?topic=vpc-vpc-encryption-about&interface=uivpc-provider-managed-encryption) or [customer-managed encryption](https://cloud.ibm.com/docs/vpc?topic=vpc-vpc-encryption-about&interface=uivpc-customer-managed-encryption). The encryption is inherited from the source volume. 
 Size Size of the snapshot in GBs. The size is inherited from the source volume. 
 Source volume Shows the boot or data volume from which the snapshot was created. Click the name of the volume to see its [details](https://cloud.ibm.com/docs/vpc?topic=vpc-viewing-block-storage). 
 Fast restore status Status of restore as enabled, pending, or disabled. 
 Requested Date and time when the snapshot was requested.
Getting started with watsonx.data 

IBM® watsonx.data is a data management solution for collecting, storing, querying, and analyzing all your enterprise data with a single unified data platform. It provides a flexible and reliable platform that is optimized to work on open data formats. This tutorial is a short introduction to using a watsonx.data deployment.



 Before you begin 

You need to have an [IBM Cloud account](https://cloud.ibm.com/registration).





 Step 1: Provision an instance 



1. Go to the [IBM Cloud catalog](https://cloud.ibm.com/catalog) page.
2. Find the watsonx.data tile and click it. You are redirected to the provisioning page.
3. Select the cloud platform (IBM Cloud or Amazon Web Services) you want to deploy watsonx.data.
4. Select a location from the list of available locations for watsonx.data service.
5. Enter the service name. The service name can be any string. This service name is used in the web console to identify the new deployment.
6. Select a resource group. If you are organizing your services into resource groups, specify the resource group.
7. Enter a tag name.
8. Select the type of network endpoints that is used for accessing the service.

a. Public endpoint only - Public endpoints provide a connection to your deployment on the public network (single endpoint).

b. Private endpoint only - Private endpoints route traffic through the IBM Cloud Private network (single endpoint).

c. Both public and private endpoints - Public endpoints provide a connection to your deployment on the public network. Private endpoints route traffic through the IBM Cloud Private network. (Two separate endpoints).
9. Click Create.

After you click Create, the system displays a message to say that the instance is being provisioned, which returns you to the Resource list. From the Resource list, under Databases category, you see that the status for your instance is, Provision in progress.
10.
In order to operate the nodes from the console, you need to import all of the generated identities into the console wallet by using the following process:



1. Navigate to the console wallet and click Add identity.
2. Click the Upload JSON tab and browse to the generated identities in the tutorial folder and click Add identity.
3. Repeat these steps for each identity file in the following table.





Table 1. File names of organization admin identities

 Ordering Org Org1 Org2 

 Organization Admin Ordering Org Admin.json Org1 Admin.json Org1 Admin.json 
 Organization CA Admin Ordering Org CA Admin.json Org1 CA Admin.json Org2 CA Admin.json 



When you run the build_network.sh script with the destroy option, all of the nodes and associated metadata are removed from your cluster, but any identities that you manually imported into the console wallet are not removed. Before you run the build_network.sh script again with the build option, you need to manually remove each identity, by opening it in the wallet and then clicking Remove identity.

Using the wallet identities from the preceding table, you can now follow the steps in the following table to associate identities for each node type.

CAs

Peer nodes

Ordering service



Table 2. CA identities

 Associate an identity with CAs 

 <br><br> * Navigate to the Nodes tab.<br> * Open the Ordering Org CA and click Associate identity.<br> * Click the Existing CA tab and select the Ordering Org CA Admin.<br> * If you created the endorsing organizations for the peers, repeat these steps for the Org1 CA and Org2 CA nodes, associating the Org1 CA Admin and Org2 CA Admin identities respectively.<br><br><br> 









 Summary 

Congratulations. You have used the Ansible playbooks to configure IBM Blockchain Platform components on your cluster. You know how to run a playbook individually and have some sample shell scripts that show how to run the playbooks together to build a reproducible network and tear it back down.
[Facet grouping dialog](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/cm-facet-group.png)

Figure 2. Facet grouping dialog
4. Click Apply.
5. The facets that you grouped are now available from a folder with the group name.

Zoom

![Facet folder](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/cm-facet-folder.png)

Figure 3. Facet folder from the facet list
For more information about this endpoint URL, including a list of values, see [Endpoint URL](https://cloud.ibm.com/apidocs/toolchainendpoint-url). 
 {resource_group_id} The ID of the resource group in which the toolchain resides. To find the IDs of the available resource groups, run ibmcloud resource groups. 
 {token} A valid IAM bearer token. 
 {toolchain_id} The ID of your toolchain. 



For more information about listing toolchains with the API and getting a toolchain with the API, see [Get a list of toolchains](https://cloud.ibm.com/apidocs/toolchainlist-toolchains) and [Get a toolchain](https://cloud.ibm.com/apidocs/toolchainget-toolchain-by-id) in the IBM Cloud API Docs.





 Viewing a toolchain with Terraform 



1. Locate the Terraform file (for example, main.tf) that contains the resource block for the existing toolchain.
2. Add an output block to the Terraform file, if it does not already contain a block.

The resource in the following example describes an existing toolchain. The output block instructs Terraform to output the attributes of the toolchain resource.

data "ibm_resource_group" "group" {
name = "default"
}

resource "ibm_cd_toolchain" "cd_toolchain" {
name = "my toolchain"
resource_group_id = data.ibm_resource_group.group.id
}

output "cd_toolchain_attributes" {
value = ibm_cd_toolchain.cd_toolchain
}

For more information about ibm_cd_toolchain, see the argument reference details in the [Terraform Registry Documentation](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/cd_toolchain).
3. Initialize the Terraform CLI.