vania-janet/multiturn-rag-retrieval-data huggingface dataset:
A powerful feature of Discovery is that you can add your own custom enrichments, such as dictionaries, patterns, and machine learning models. When you create custom enrichments, they are listed on this page also. You can manage where they are used from here.

For more information about custom enrichments, see [Adding domain-specific resources](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain).
6. You are going to apply another enrichment to the collection. Find the Keywords enrichment in the list, and then click Select fields.

The Keywords enrichment recognizes significant commonly-known terms in your content.
7. Scroll through the list of fields until you find the text field, and select it.

Zoom

![Shows the fields where you can apply the Keyword enrichment .](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/gs-keyword-find.png)

Figure 7. Fields to which you can apply the Keywords enrichment
8. Click Apply changes and reprocess.

While your documents are being reprocessed to look for and tag keywords, you can continue to explore the tools available for managing a collection.
9. Click Identify fields.

Most content from a document is indexed in the text field automatically. You might want to index certain types of content in different fields or split up large documents so that the text field contains fewer passages per document. To do so, you can teach Discovery to recognize important fields in your documents by applying a Smart Document Understanding model to your collection.

Smart Document Understanding (SDU) is a technology that learns about the content of a document based on the document's structure. You can apply a prebuilt SDU model or create a custom SDU model.

Zoom

![Shows the SDU model options.]
Click Notes, and then click Next.
4. Name the collection.
5. If the language of the documents in HCL Notes is not English, select the appropriate language.

For a list of supported languages, see [Language support](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-language-support).
6. Optional: Change the synchronization schedule.

For more information, see [Crawl schedule options](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collectionscrawlschedule).
7. In the Enter your credentials section, add values to the following fields:

Host name
: The hostname of the HCL Notes server.

User name
: The username to use to crawl the HCL Notes server.

Password
: The password that is associated with the user.
8. In the Crawl type, choose what you want to crawl from the following options:



* If you want to crawl a specific HCL Notes database, choose Database, and then add the file name of the database to the Database file name field.
* If you want to crawl multiple databases, choose Directory. Specify the directory in which the databases that you want to crawl are stored in the Directory name field.



9. Optional: In the Security section, specify whether you want to enable document-level security.



* If you want to enable document-level security, set the Enable Document Level Security switch to On.

When set to On, your users can crawl the same content that they have access to in a HCL Notes database or directory.
* To use the Domino LDAP directory, set the Use remote LDAP directory switch to On. Provide details about the Domino LDAP directory. You collected this information when you performed the prerequisite step.

LDAP server URL
: The LDAP server URL to connect to. For example, ldap://<ldap_server>:<port>.

LDAP binding username
: The username to use to bind to the directory service.

LDAP binding user password
For more information, see [Reusing SDU models](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-configuring-fieldsimport).









 Creating the model 

To apply a user-trained Smart Document Understanding model to your collection, complete the following steps:



1. Open the Manage collections page from the navigation panel.
2. If your project has more than one collection, select the collection with documents that you want to annotate.
3. Open the Identify fields page.
4. Choose User-trained models.

The Text extraction only option is used by default. With this model, any text that is recognized in the source documents is indexed in the text field.
5. Click Submit, and then click Apply changes and reprocess.



A subset of documents is available for you to annotate. A set of 20 - 50 documents is displayed in a list. The number of documents that are available differs based on several factors, including the overall number of documents in your collection and how many of them are supported file types.





 Labeling video 

The following video shows you how to select a label, and then apply it to a representation of the text in your document.

In the video, the user clicks the title field label, and then clicks the text block that represents the Table of Contents page title to label the text as a title. Next, the user clicks the table_of_contents field label and selects the table of contents text block to label it. Then, the user clicks the footer field label and clicks the text block that represents the page footer. After the text is labeled, the user clicks the Submit page button.

Your browser does not support the video tag.





 Labeling the documents 

Before you begin, get a feel for the structure of the document you plan to annotate. Are there subtitled sections that you want Discovery to return per answer? If so, identify all subtitles. Later you can split the document into discrete subdocuments, each starting with a subtitle.
View network details 

After you filter for your server by using the appropriate tags, select your device. In the Network details section of your selected device, you can access your network information.

You must use this network information when you establish network connectivity, order more servers, or order more network capacity.





 View storage details 

Complete the following steps to identify and view your File Storage details.



1. In the IBM Cloud console, go to Menu icon ![Menu icon](https://cloud.ibm.com/docs-content/v1/content/e9cda99c64ea1ee68cf0eda257d6a1de69ddbc4e/icons/icon_hamburger.svg) > Classic Infrastructure > File Storage.
2. Your File Storage has the Citrix DaaS and Resource location name tags in the Notes field.









 Step 2. (Optional) Provision virtual server instance for management 

Optionally, you can provision a virtual server instance with a Windows operating system to manage your Citrix DaaS for IBM Cloud. You can use a jump server to start a Remote Desktop session to access the Active Directory server and Citrix Cloud Connectors. The jump server is also needed to start the XenCenter management interface. You can order a Windows virtual server instance through the [IBM Cloud catalog](https://cloud.ibm.com/gen1/infrastructure/provision/vs?cm_sp=Cloud-Product-_-OnPageNav-IBMCloudPlatform_IBMVirtualMachines-_-VSI_Prod_Midpage).

You must provision the virtual server instance with the same [network details](https://cloud.ibm.com/docs/citrix-daas?topic=citrix-daas-post-provisioning-cvadview-network-details) as the other service components that are attached to the resource location.





 Step 3. Establish network connectivity for Active Directory Topology 



 IBM Cloud 

If you chose IBM Cloud Active Directory topology and are using the Active Directory in IBM Cloud as your only domain controller, no further network configuration is required. The Cloud Connectors are joined to the Active Directory server domain and are configured to communicate with Citrix Cloud.
This documentation is for the classic Watson Assistant experience. To see the documentation for the new Watson Assistant, please go [here](https://cloud.ibm.com/docs/watson-assistant).



 Adding a node with slots to a dialog 

In this tutorial, you will add slots to a dialog node to collect multiple pieces of information from a user within a single node. The node you create will collect the information that is needed to make a restaurant reservation.



 Learning objectives 

By the time you finish the tutorial, you will understand how to:



* Define the intents and entities that are needed by your dialog
* Add slots to a dialog node
* Test the node with slots





 Duration 

This tutorial will take approximately 30 minutes to complete.





 Prerequisite 

Before you begin, complete the [Getting Started tutorial](https://cloud.ibm.com/docs/assistant?topic=assistant-getting-started). You will use the Watson Assistant tutorial skill that you created, and add nodes to the simple dialog that you built as part of the getting started exercise.

You can also start with a new dialog skill if you want. Just be sure to create the skill before you begin this tutorial.







 Step 1: Add intents and examples 

Add an intent on the Intents tab. An intent is the purpose or goal that is expressed in user input. You will add a #reservation intent that recognizes user input that indicates that the user wants to make a restaurant reservation.



1. From the Intents page of the tutorial skill, click Add intent.
2. Add the following intent name, and then click Create intent:

reservation

The #reservation intent is added. A number sign () is prepended to the intent name to label it as an intent. This naming convention helps you and others recognize the intent as an intent. It has no example user utterances associated with it yet.
3. In the Add user examples field, type the following utterance, and then click Add example:

i'd like to make a reservation
4. Add these additional examples to help Watson recognize the reservation intent.
[Shows the search step with End the action checkbox selected.](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/tut-convo-action-10.png)

Figure 15. Search step settings
11. Save your changes, and then click the X to close the step.

Zoom

![Shows the action close icon.](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/tut-convo-action-11.png)

Figure 16. Close action



Congratulations! You successfully created an action that recognizes questions about FRED research papers and returns a search response.

Zoom

![Shows the action that was created.](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/tut-convo-action-12.png)

Figure 17. Created action

In a later step, we will connect the search response in this action to a search extension that is configured for the assistant.





 Step 3: Create a Conversational Search project 

Now that the assistant can recognize questions about a subject, let's give it access to data from which it can retrieve accurate answers.

In Discovery, create a Conversational Search project type. This project type is optimized for retrieving answers during dialog-driven interactions. For example, unlike other project types, it does not apply prebuilt enrichments that aren't needed.



1. Open a new web browser page.

Keep the Watson Assistant page open in a separate tab so you can switch between the two applications.
2.
* To view toolchains in the targeted resource group, run the following command:



ibmcloud dev toolchains



* To view the details for a specific toolchain, run the following command:



ibmcloud dev toolchain-get [TOOLCHAIN-NAME]



* To view the toolchain details in a browser, run the following command:



ibmcloud dev toolchain-open [TOOLCHAIN-NAME]





 Getting a toolchain with the API 



1. [Obtain an IAM bearer token](https://cloud.ibm.com/apidocs/toolchainauthentication). Alternatively, if you are using an SDK, [obtain an IAM API key](https://cloud.ibm.com/iam/apikeys) and set the client options by using environment variables.

export CD_TOOLCHAIN_AUTH_TYPE=iam && export CD_TOOLCHAIN_APIKEY={iam_api_key} && export CD_TOOLCHAIN_URL={base_url}
2. [Look up the ID of the resource group](https://cloud.ibm.com/apidocs/resource-controller/resource-managerlist-resource-groups) in which your toolchain resides.
3. Choose the region in which your toolchain resides.
4. Get a list of toolchains in the resource group.



* Curl
* Java
* Node
* Python
* Go



curl -X GET "{base_url}/toolchains?resource_group_id={resource_group_id}" -H 'Authorization: Bearer {token}' -H 'Accept: application/json'

import com.ibm.cloud.continuous_delivery.cd_toolchain.v2.CdToolchain;
import com.ibm.cloud.continuous_delivery.cd_toolchain.v2.model.;
...
CdToolchain toolchainService = CdToolchain.newInstance();
[NodeSync](https://docs.datastax.com/en/dse/6.7/dse-admin/datastax_enterprise/config/aboutNodesync.html) is a continuous repair service that runs automatically in the background to validate that your data is in sync on all replicas and reduces the need for manual repairs. For operational simplicity and performance, Databases for DataStax enables the NodeSync service on all key spaces and tables to handle node repairs. Traditional repairs are unnecessary, as the automation ensures that NodeSync handles the repairs for you.

While the automation enables NodeSync, you can also manually create tables with NodeSync enabled so that the service can repair the tables' data when necessary:

CREATE TABLE myTable (...) WITH nodesync = { 'enabled': 'true'};

For more information, see [enabling the NodeSync service](https://docs.datastax.com/en/dse/6.7/dse-admin/datastax_enterprise/config/enablingNodesync.html).

Nodetool is unsupported. Manual repairs that are issued against any table that is enabled with the NodeSync service [are ignored](https://docs.datastax.com/en/opscenter/6.5/opsc/online_help/services/opscNodeSyncService.htmlNodeSyncServiceversusRepairService). See error:

WARNING: A manual nodetool repair or a repair operation from the OpsCenter node administration menu fails to run if a NodeSync-enabled table is targeted.





 Recommendations 



 Benchmark before production 



* Do not use cqlsh and COPY for benchmarking, as COPY does not mimic typical client behavior. Instead, [nosqlbench](https://github.com/nosqlbench/nosqlbench) can be used for benchmarking.







 Data migrations 



* DSBULK is recommended for data migration.
[winston](https://www.npmjs.com/package/winston) A multi-transport async logging library for Node.js. "CHILL WINSTON! ... I put it in the logs. 
 [ws](https://www.npmjs.com/package/ws) ws is a simple to use, blazing fast, and thoroughly tested WebSocket client and server implementation. 
 [xlsx](https://www.npmjs.com/package/xlsx) Parser and writer for various spreadsheet formats. 
 [xml2js](https://www.npmjs.com/package/xml2js) Simple XML to JavaScript object converter. It supports bidirectional conversion. 
 [xmlhttprequest](https://www.npmjs.com/package/xmlhttprequest) node-XMLHttpRequest is a wrapper for the built-in HTTP client to emulate the browser XMLHttpRequest object. 
 [yauzl](https://www.npmjs.com/package/yauzl) Yet another extract library for node. For zipping. 
 [yazl](https://www.npmjs.com/package/yauzl) Yet another extract library for node. For zipping. 



For more information about Node.js 12 packages, see [(Details on GitHub)](https://github.com/ibm-functions/runtime-nodejs/blob/master/nodejs12/package.json).

For more information about Node.js 10 packages, see [(Details on GitHub)](https://github.com/ibm-functions/runtime-nodejs/blob/master/nodejs10/package.json).







 Python runtimes 

By default, all Python actions are executed in a Python version 3.9 environment.



Table 4. Python versions

 Kind Python version Description Change log
(https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/node1-simple.png)

A single node with one condition and response can handle simple user requests. But, more often than not, users have more sophisticated questions or want help with more complex tasks. You can add child nodes that ask the user to provide any additional information that your assistant needs.

![Shows that the first node in the dialog asks which type of cupcake the user wants, gluten-free or regular, and has two child nodes that provide a different response depending on the user's answer.](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/node1-children.png)





 Dialog flow 

The dialog that you create is processed by your assistant from the first node in the tree to the last.

![Arrow points down next to 3 nodes to show that dialog flows from the first node to the last](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/node-flow-down.png)

As it travels down the tree, if your assistant finds a condition that is met, it triggers that node. It then moves along the triggered node to check the user input against any child node conditions. As it checks the child nodes it moves again from the first child node to the last.

Your assistant continues to work its way through the dialog tree from first to last node, along each triggered node, then from first to last child node, and along each triggered child node until it reaches the last node in the branch it is following.

!
There are three types of nodes: CA, peer, and ordering node.





 Ordering node 

The node that collects transactions from network members, orders the transactions and bundles them into blocks. Also known as orderer. These blocks are then distributed to peers, which then verify the blocks and add them to the ledgers on each channel. Ordering nodes contain the cryptographic identity material that is tied to each member and authenticate the identity of clients and peers to access the network. The overall function that an ordering node or collection of nodes provides is known as the ordering service.





 Organization 

See [Member](https://cloud.ibm.com/docs/blockchain?topic=blockchain-glossaryglossary-member).





 Out of band 

An expression used to refer to sharing network artifacts outside of the console UI, for example by email or some other file transfer mechanism. After submitting a smart contract proposal, the originator can share the smart contract package and package id with other channel members in an out of band operation by emailing this information to them.





 Peer 

A blockchain network resource that provides the services to execute and validate transactions, and maintain ledgers. The peer runs smart contract and is the holder of transaction history and the current state of assets on ledgers. They are owned and managed by organizations and are joined to channels.





 Propose 

Referring to a step in the Hyperledger Fabric v2.x smart contract lifecycle, a smart contract definition is proposed to a channel for approval by the organizations that are specified in the channel lifecycle endorsement policy. A smart contract proposal contains a smart contract definition name, version, package, endorsement policy, and optionally a private data collection.





 Quorum 

In a Raft ordering service, a quorum represents the number of nodes that must be available for transactions to be processed. This number is a majority of the total number of nodes in the consenter set of the channel. In other words, if you have one node, you need that node available to have a quorum, because the majority of one is one.
The [Peer channel-based event services](https://hyperledger-fabric.readthedocs.io/en/release-2.2/peer_event_services.html) provide replayable events to ensure the integrity of downstream data stores. For an example of how you can use an event listener to write data to an external database, see the [Off chain data sample](https://github.com/hyperledger/fabric-samples/tree/release-1.4/off_chain_data) in the Fabric Samples.





 Clock synchronization 

Synchronizing your application with a Network Time Protocol (NTP) server is important because it ensures that all time-based activities occur synchronously everywhere on the network. NTP is widely used to synchronize a computer to Internet time servers or other sources. The IBM Blockchain Platform runs in IBM Cloud and uses an internal NTP server servertime.service.softlayer.com.

The NTP server that you need to synchronize your application with depends on where your application is running:



* If your application is running in IBM Cloud Classic infrastructure, set your NTP server to servertime.service.softlayer.com.
* If your application is running in IBM Cloud VPC infrastructure, set your NTP server to time.adn.networklayer.com.
* If your application is not running in IBM Cloud, set your NTP server to time-a.nist.gov or time-b.nist.gov.







 Additional Resources 

[IBM Developer](https://developer.ibm.com/technologies/blockchain/)
For tutorials, code patterns, and videos that help developers get started and learn best practices for developing blockchain applications.

[Blockchain Design Patterns](https://developer.ibm.com/technologies/blockchain/articles/getting-started-with-blockchain-design-patterns)
For application developers who want to learn about common patterns for interacting with blockchain networks.
An NSX Edge Node is a transport node that runs the local control plane daemons and forwarding engines implementing the NSX-T data plane. It runs an instance of the NSX-T virtual switch called the NSX Virtual Distributed Switch, or N-VDS. The Edge Nodes are service appliances dedicated to running centralized network services that cannot be distributed to the hypervisors. An NSX Edge can belong to one overlay transport zone and multiple VLAN transport zones. In this example, the NSX edge belongs to one VLAN transport zone which represents the physical connectivity to provide the uplink access to IBM Cloud VPC.

Follow the recommended order of procedures to add IBM Cloud Bare Metal Servers for Virtual Private Cloud with VMware ESXi as host transport nodes to NSX-T.



1. Create host transport nodes. When configuring the host TEP IPs, configure each host individually by using the [previously provisioned TEP IPs](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-vpc-bm-vmware-nsx-t-hostsvpc-bm-vmware-nsx-t-vlannics) of the related IBM Cloud Bare Metal Servers for Virtual Private Cloud VLAN interface for TEP. For more information, see [Prepare Standalone Hosts as Transport Nodes or Prepare ESXi Cluster Hosts as Transport Nodes](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/3.1/installation/GUID-D7CA778B-6554-4A23-879D-4BC336E01031.htmlGUID-D7CA778B-6554-4A23-879D-4BC336E01031).



When you deploy an NSX-T edge, you can think of it as an empty container. The NSX-T edge does not do anything until you create logical routers, it provides the compute backing for Tier 0 and Tier 1 logical routers.