vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Creating a machine learning model 

This tutorial helps you understand the process for building a machine learning model that you can deploy and use with other Watson services.



 Learning objectives 

After you complete the lessons in this tutorial, you will know how to perform the following tasks:



* Create document sets
* Pre-annotate documents
* Create tasks for human annotators
* Analyze inter-annotator agreement and adjudicate conflicts in annotated documents
* Create machine learning models



This tutorial takes approximately 60 minutes to finish. If you explore other concepts that are related to this tutorial, it might take longer to complete.





 Before you begin 



* You're using a supported browser. See [Browser requirements](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-system-requirements).
* You successfully completed [Getting started with Knowledge Studio](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-wks_tutintro), which covers creating a workspace, creating a type system, and adding a dictionary.
* You must have at least one user ID in either the Admin or Project Manager role.

If possible, use multiple user IDs for the machine learning model tasks in this tutorial (one Admin or Project Manager user ID, and at least two Human Annotator user IDs). Using multiple user IDs provides the most realistic simulation of an actual IBM Watson® Knowledge Studio workspace, where a project manager must coordinate and adjudicate annotation that is performed by multiple human annotators. However, if you have access to only a single user ID, you can still simulate most parts of the process.

For more information about user roles, see [User roles in Knowledge Studio](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-roles).







 Results 

After completing this tutorial, you will have a custom machine learning model that you can use with other Watson services.





 Lesson 1: Adding documents for annotation
Go to the [Discovery](https://cloud.ibm.com/catalog/services/watson-discovery) page in the IBM Cloud catalog and create a Plus plan service instance.



If you decide not to continue using the Plus plan and don't want to pay for it, delete the Plus plan service instance before the 30-day trial period ends.





 Create the search skill 



1. From the assistant where you want to add the skill, click Add search skill.

You can only add a search skill if you are a user with a paid plan.
2. Take one of the following actions:



* To create a new search skill, stay on the Create skill tab.
* If you have created a search skill already, the Add existing skill tab is displayed, and you can click to add an existing skill.



3. Specify the details for the new skill:



* Name: A name no more than 64 characters in length. A name is required.
* Description: An optional description no more than 128 characters in length.



Next, you will connect to a Discovery service instance.
4. Click Continue.







 Connect to an existing Watson Discovery instance 



1. Choose the Discovery instance that you want to connect to.

If you see a warning that some of your Discovery service instances do not have credentials set, it means that you can access at least one instance that you never opened from the IBM Cloud dashboard directly yourself. You must access a service instance for credentials to be created for it, and credentials must exist before Watson Assistant can establish a connection to the Discovery service instance on your behalf. If you think a Discovery service instance is missing from the list, open the instance from the IBM Cloud® dashboard directly to generate credentials for it.
2. Indicate the collection to use, by doing one of the following things:



* Choose an existing project.

You can click the Open Discovery icon to review the configuration of a project before you decide which one to use.

Go to [Configure the search](https://cloud.ibm.com/docs/services/assistant?topic=assistant-skill-search-addskill-search-add-configure) and follow the remaining steps.
A snapshot is taken of the current skill and saved as a new version. You remain in the development version of the skill. Any changes you make continue to be applied to the development version, not the version you saved. To access the version you saved, go to the Versions page.





 Deploying a skill version 



1. From the skill menu, click Versions.

v1.3: Click the Skills tab, and then click Versions.
2. Click the ![Click to view actions](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/kebab-react.png) icon from the version you want to deploy, and then choose Assign to assistant.

A list of assistants to which you can link this version is displayed. The list is limited to those assistants that don't have any skills associated with them, or that are associated with a different version of this skill.
3. Click the checkbox for one or more of the assistants, and then click Assign.







 Skill version limits 



Limit details

 Versions per skill 

 50 







 Swapping skills 

If you find that an earlier version of the skill did a better job of recognizing and addressing customer needs than a later version, you can swap the skill that is linked to the assistant to use the earlier version instead.

Follow the same steps you use to [deploy a skill version](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-versionsversions-deploy) to change the version that is linked to an assistant.





 Accessing a saved version 

The only way to view a saved version is to overwrite the in-progress development version of the skill with the saved version. (But not before you have saved any work you have done in the current development version.)

You cannot edit a saved version. To achieve the same goal, you can use a saved version as the basis for a new version in which you incorporate any changes you want to make.
You are now ready to create one or more simulated devices.
7. Click Create Simulated Device to create a device and start sending data.
Tip: To create more than one device, click 1 x and select the number of devices to create.
8. To create more simulations, click New Simulation and repeat the configuration steps for each device that you want to apply custom settings to.
The message in the lower-right corner of the screen shows the number of active simulations.
9. On the Devices page, browse to one of the simulated devices and select the Recent Events tab to verify that the simulated events are working.



You can [export your simulated event configurations](https://cloud.ibm.com/docs/services/IoT?topic=IoT-sim_device_datareuse) for reuse or sharing.





 Event type functions 

You can use a number of special functions to create dynamic data for your event types.



 Function Usage Example 

 random(lower, upper) Generates a random number in the specified range. {"random": random(0, 100)} 
 $counter Displays the number of simulated devices of the current type. {"total": $counter} 
 increment(start, increment) Specifies the starting value and the increment by which it increases. {"myNumber": increment(10, 1)} <br>In this example, myNumber starts at 10 and increases by 1 each time a payload is sent. 







 CSV event payload file 

If you want to create controlled event data for certain scenarios, you can set up the data flow in a CSV input file.



 Requirements 

The CSV file must meet the following requirements:



* Each value is a valid [JSON primitive ![External link icon](https://cloud.ibm.com/docs-content/v1/content/dc2670e9228a1dbe562d3d2c7ee63f7a48ba27c3/icons/launch-glyph.svg)](https://json.org), except as noted:



* string:
Example: "Hello world"
* number:
Creating a machine learning model 

This tutorial helps you understand the process for building a machine learning model that you can deploy and use with other Watson services.



 Learning objectives 

After you complete the lessons in this tutorial, you will know how to perform the following tasks:



* Create document sets
* Pre-annotate documents
* Create tasks for human annotators
* Analyze inter-annotator agreement and adjudicate conflicts in annotated documents
* Create machine learning models



This tutorial takes approximately 60 minutes to finish. If you explore other concepts that are related to this tutorial, it might take longer to complete.





 Before you begin 



* You're using a supported browser. See [Browser requirements](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-system-requirements).
* You successfully completed [Getting started with Knowledge Studio](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-wks_tutintro), which covers creating a workspace, creating a type system, and adding a dictionary.
* You must have at least one user ID in either the Admin or Project Manager role.

> Note: If possible, use multiple user IDs for the machine learning model tasks in this tutorial (one Admin or Project Manager user ID, and at least two Human Annotator user IDs). Using multiple user IDs provides the most realistic simulation of an actual IBM Watson™ Knowledge Studio workspace, where a project manager must coordinate and adjudicate annotation that is performed by multiple human annotators. However, if you have access to only a single user ID, you can still simulate most parts of the process.

For information about user roles, see [User roles in Knowledge Studio](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-roles).







 Results 

After completing this tutorial, you will have a custom machine learning model that you can use with other Watson services.





 Lesson 1: Adding documents for annotation
You are now ready to create one or more simulated devices.
7. Click Create Simulated Device to create a device and start sending data.
Tip: To create more than one device, click 1 x and select the number of devices to create.
8. To create more simulations, click New Simulation and repeat the configuration steps for each device that you want to apply custom settings to.
The message in the lower-right corner of the screen shows the number of active simulations.
9. On the Devices page, browse to one of the simulated devices and select the Recent Events tab to verify that the simulated events are working.



You can [export your simulated event configurations](https://cloud.ibm.com/docs/IoT?topic=IoT-sim_device_datareuse) for reuse or sharing.





 Event type functions 

You can use a number of special functions to create dynamic data for your event types.



 Function Usage Example 

 random(lower, upper) Generates a random number in the specified range. {"random": random(0, 100)} 
 $counter Displays the number of simulated devices of the current type. {"total": $counter} 
 increment(start, increment) Specifies the starting value and the increment by which it increases. {"myNumber": increment(10, 1)} <br>In this example, myNumber starts at 10 and increases by 1 each time a payload is sent. 







 CSV event payload file 

If you want to create controlled event data for certain scenarios, you can set up the data flow in a CSV input file.



 Requirements 

The CSV file must meet the following requirements:



* Each value is a valid [JSON primitive ![External link icon](https://cloud.ibm.com/docs-content/v1/content/dc2670e9228a1dbe562d3d2c7ee63f7a48ba27c3/icons/launch-glyph.svg)](https://json.org), except as noted:



* string:
Example: "Hello world"
* number:
[open and close list of options](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/kebab.png) icon, and then choose Delete. Confirm the deletion.







 Downloading a skill 

You can download a dialog or actions skill in JSON format. You might want to download a skill if you want to use the same skill in a different instance of the Watson Assistant service. You can download a dialog skill from one instance and upload it to another instance as a new skill, for example.

You cannot download a search skill.

To download a skill, complete the following steps:



1. Find the skill tile on the Skills page or on the configuration page of an assistant that uses the skill.
2. Click the ![open and close list of options](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/kebab.png) icon, and then choose Download.
3. Specify a name for the JSON file and where to save it, and then click Save.



For dialog skills only:



* You can download a dialog skill by using the API also. Include the export=true parameter with the request. See the [API reference](https://cloud.ibm.com/apidocs/assistant/assistant-v1getworkspace) for more details.
* For information about how to download a specific dialog skill version, see [Downloading a skill version](https://cloud.ibm.com/docs/assistant?topic=assistant-versionsversions-export).







 Overwriting a skill 

To overwrite or replace an existing skill, upload the new version of the skill as a JSON file into the existing skill.

You can overwrite a dialog or actions skill. You cannot overwrite a search skill.
Before you create a vault for your Hyper Protect Crypto Services instance, keep in mind of the following considerations:



* Vaults can hold KMS keys and keystores, but not EP11 keys and keystores.

Vaults can contain both KMS keys and keystores. There is no limit on how many keys can exist within a vault. Vaults don't apply to Enterprise PKCS #11 (EP11) keys and keystores.
* A key or a keystore only can belong to one vault at a time.

A managed key or a target keystore can belong to only one vault. You need to assign a vault to a managed key or a target keystore upon creation.
* During master key rotation, you are not able to create a vault. However, you can create the vault again after the master key rotation process is complete.



For more information about granting access, see [Granting access to vaults](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-grant-access-vaults).



 Creating vaults with the IBM Cloud console 

To create a vault by using the console, complete the following steps through the Vaults page. Optionally, you can create a vault when you [create a managed key](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-create-managed-keys) or [add a keystore](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-create-internal-keystores).



1. [Log in to the Hyper Protect Crypto Services instance](https://cloud.ibm.com/login).
2. Click Vaults from the navigation to view all the available vaults.
3. To create a vault, click Create vault.
4. Enter a name in Vault name. Optionally, you can add an extended description to your vault in the Description section.

The vault name must be of 1 to 100 characters in length. The characters can be letters (case-sensitive), digits (0-9), or symbols (#@!$%\’_-).
* [Prerequisites](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regionsdeploy-setup-cis-prereq)
* [Create projects in different regions](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regionsdeploy-project-regions)
* [Deploy your apps in multiple regions](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regionsdeploy-app-regions)
* [Generate a certificate for your custom domain](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regionscustom-domain-cert)
* [Create a TLS secret](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regionscreate-tls-secret)
* [Configure the custom domain mappings](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regionsconfig-app-domain)
* [Configure a health check](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regionsconfig-health-check)
* [Configure the Cloud Internet Services (CIS) load-balancer](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regions=config-load-balancer)
* [Verify that your app is available](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regionsverify-app-domain)
* [Cleaning up your tutorial](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regionsclean-up)
Use with Logs Reader to provide access to Analytics. 
 Writer Create and edit within a resource. 
 Manager Manage everything in a resource. 
 Logs Reader Use Logs Reader in combination with the Reader or Writer role to provide access to Analytics. 
 Version Maker Create or delete versions of an assistant. Doesn't provide publish access. 



This table explains the minimum service roles that are required for common tasks in an assistant.



Minimum service role details

 Task Resource Minimum service role required 

 Assistant 
 Create assistant Service instance Writer 
 View assistant settings Assistant Writer 
 View assistant ID Assistant Writer 
 Update assistant settings Assistant Writer 
 Enable or disable dialog Assistant Writer 
 Delete assistant Service instance Writer 
 Actions 
 Create action Action skill Writer 
 Update action Action skill Writer 
 Delete action Action skill Writer 
 Download actions JSON file Action skill Reader 
 Upload actions JSON file Action skill Writer 
 Copy action Action skill (in destination assistant) Writer 
 Publish 
 Publish version Environment Writer 
 Create version without publishing Assistant Writer or Version Maker 
 Delete unpublished version Assistant Writer or Version Maker 
 Download version Assistant Reader 
 Environments 
 Create environment (Enterprise plan only) Assistant Writer 
 Update environment settings Environment Writer 
 Delete environment (Enterprise plan only) Service instance Writer 
 Integrations 
 Add integration Service instance Writer 
 Update integration Service instance Writer 
 Delete integration Service instance Writer 
 Dialog 
 Create intent Dialog skill Writer 
 Update intent Dialog skill Writer 
 Delete intent Dialog skill Writer 
 Import intents Dialog skill Writer 
 Export intents Dialog skill Reader 
 Create entity Dialog skill Writer 
 Update entity Dialog skill Writer 
 Delete entity Dialog skill Writer 
 Download intents and entities Dialog skill Reader 







 Example of limiting access to one assistant 

This example explains how to follow the steps in [Adding users from the Manage menu](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-access-controlaccess-control-add-users) and set specific resources that limit a user to building and publishing actions in one assistant. For each user, you need to add three access policies that identify the service instance, assistant ID, and skill ID.
To select multiple cells, click one, press the Shift key, and click additional cells.

Assign a class to the selected cell or cells. If the class you want to assign does not exist, you can add it. Type the class name into the Assign class field and press Enter.

You cannot add more than 10 classes for the rule.

![The "Create a rule" section showing the "Assign class" window that opens when you click a token. The window shows a field where you can enter the name of a new class, or select an existing class from the list.](https://cloud.ibm.com/docs-content/v1/content/148d3bd95f946aa1bb53ea1540475f522e6b61c9/watson-knowledge-studio-data/images/rule_step6.png)
13. Name the rule.

The rule name cannot be longer than 64 characters.
14. Click Save in the Rules panel to save the rule.
[IBM Developer<br><br>Visit IBM Developer for technical articles, code patterns, tutorials, and more.<br><br>![112-Developer-tools.svg](https://cloud.ibm.com/media/docs/images/homepage/112-Developer-tools.svg)](https://developer.ibm.com/?q=IBM%20cloud%20container%20registry)[IBM Training<br><br>Build your skills with IBM Cloud training.<br><br>![83-Elements-of-the-cloud.svg](https://cloud.ibm.com/media/docs/images/homepage/83-Elements-of-the-cloud.svg)](https://www.ibm.com/training/search?query=container%20registry)[Community<br><br>Join the IBM Cloud community and collaborate with others.<br><br>![35-Connectivity.svg](https://cloud.ibm.com/media/docs/images/homepage/35-Connectivity.svg)](https://community.ibm.com/community/user/publiccloud/home)
curl -u "apikey:{apikey}" "https://{service-hostname}/assistant/api/v1/workspaces/{workspace_id}/dialog_nodes?version=2018-09-20&include_count=true"

where {service-hostname} is the appropriate URL for your instance. For more details, see [Service endpoint](https://cloud.ibm.com/apidocs/assistant/assistant-v1service-endpoint).

In the response, the total attribute in the pagination object contains the number of dialog nodes.

See [Troubleshooting skill import issues](https://cloud.ibm.com/docs/assistant?topic=assistant-skill-dialog-addskill-dialog-add-import-errors) for information about how to edit skills that you want to continue using.







 27 November 2018 

A new service plan, the Plus plan, is available
: The new plan offers premium-level features at a lower price point. Unlike previous plans, the Plus plan is a user-based billing plan. It measures usage by the number of unique users that interact with your assistant over a given time period. To get the most from the plan, if you build your own client application, design your app such that it defines a unique ID for each user, and passes the user ID with each /message API call. For the built-in integrations, the session ID is used to identify user interactions with the assistant. See [User-based plans](https://cloud.ibm.com/docs/assistant?topic=assistant-services-informationservices-information-user-based-plans) for more information.



Plus plan limits

 Artifact Limit 

 Assistants 100 
 Contextual entities 20 
 Contextual entity annotations 2,000 
 Dialog nodes 100,000 
 Entities 1,000 
 Entity synonyms 100,000 
 Entity values 100,000 
 Intents 2,000 
 Intent user examples 25,000 
 Integrations 100 
 Logs 30 days 
 Skills 50 



User-based Premium plan
: The Premium plan now bases its billing on the number of active unique users.
To select multiple cells, click one, press the Shift key, and click additional cells.

Assign a class to the selected cell or cells. If the class you want to assign does not exist, you can add it. Type the class name into the Assign class field and press Enter.

You cannot add more than 10 classes for the rule.

![The "Create a rule" section showing the "Assign class" window that opens when you click a token. The window shows a field where you can enter the name of a new class, or select an existing class from the list.](https://cloud.ibm.com/docs-content/v1/content/50eddb8fd81f33092880335ff107a78ff5cd0f65/watson-knowledge-studio/images/rule_step6.png)
13. Name the rule.

The rule name cannot be longer than 64 characters.
14. Click Save in the Rules panel to save the rule.
Are you getting a "target_limit_reached" error when creating a target? 

When creating a target, the following message is returned: You have reached the limit of targets that you can define in your account. Remove unused targets, then try again..

  What’s happening 

Your request fails with an error message and a code of target_limit_reached.

  Why it’s happening 

You are trying to create a new target and you already have 16 targets configured in your account.

  How to fix it 

Only 16 targets are supported in an account across all regions. If you need to create a new target, delete an existing [COS target](https://cloud.ibm.com/docs/atracker?topic=atracker-target_v2_cos&interface=apitarget-delete-api-cos) or an existing [LogDNA target](https://cloud.ibm.com/docs/atracker?topic=atracker-target_v2_at&interface=apitarget-delete-api-at) that is no longer needed and retry the request.