vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Getting started with Natural Language Understanding 

This short tutorial introduces the Natural Language Understanding API with example requests and links to additional resources.

Watch the following video for a visual summary of getting started with the Natural Language Understanding service.



 Before you begin 



* Create an instance of the service:



1. Go to the [Natural Language Understanding](https://cloud.ibm.com/catalog/services/natural-language-understanding) page in the IBM Cloud catalog.
2. Sign up for a free IBM Cloud account or log in.
3. Click Create.



* Copy the credentials to authenticate to your service instance:



1. On the Manage page, click Show Credentials.
2. Copy the API Key and URL values.



* Make sure that you have the curl command.



* Test whether curl is installed. Run the following command on the command line. If the output lists the curl version with SSL support, you are set for the tutorial.

curl -V
* If necessary, install a version with SSL enabled from [curl.haxx.se](https://curl.haxx.se/). Add the location of the file to your PATH environment variables if you want to run curl from any command-line location.





This tutorial shows you how to use the Natural Language Understanding API from a command-line interface. To download client libraries for various programming languages, check out the [Watson SDKs](https://cloud.ibm.com/docs/natural-language-understanding?topic=watson-using-sdksusing-sdks).





 Step 1: Analyze a webpage 

Run the following command to analyze a webpage to get sentiment, concepts, categories, entities, and keywords.

curl -X POST -u "apikey:{apikey}" --header "Content-Type: application/json" --data '{
"url": "http://newsroom.ibm.com/Guerbet-and-IBM-Watson-Health-Announce-Strategic-Partnership-for-Artificial-Intelligence-in-Medical-Imaging-Liver",
"features": {
"sentiment": {},
How can I measure a deployment's workload? 

You'd like to upgrade your Databases for MongoDB deployment, but you want to check if it's being used.

  What’s happening 

You're ready to upgrade to a new major version, but you'd like to ensure that your deployment isn't under heavy workload.

  How to fix it 

Check your [platform metrics](https://cloud.ibm.com/docs/monitoring?topic=monitoring-platform_metrics_enabling) within IBM Cloud® Monitoring, specifically, [ibm_databases_for_mongodb_disk_read_latency_mean](https://cloud.ibm.com/docs/databases-for-mongodb?topic=databases-for-mongodb-monitoringibm_databases_for_mongodb_disk_read_latency_mean). Disk read latency mean is the time that it takes for MongoDB to read data from disk, measured in milliseconds. A low disk read latency indicates that MongoDB is able to read data from disk quickly. A high disk read latency indicates that MongoDB is taking a long time to read data from disk. For more information, see [Monitoring Integration](https://cloud.ibm.com/docs/databases-for-mongodb?topic=databases-for-mongodb-monitoring).
After identifying the root cause, indicate what the correct choice would have been.
6. Feed the annotated spreadsheet to the Analyze Watson Assistant Effectiveness notebook.



* Effectiveness: Performs a deeper analysis of your logs to help you understand the steps you can take to improve your assistant.



7. Use the Dialog Flow Analysis for Watson Assistant notebook to review your dialog. The notebook can help you pinpoint the dialog nodes where customers most frequently abandon the conversation.

For more information about how this notebook can help you analyze and assess abandonment, read this [Medium.com blog post](https://medium.com/ibm-watson/do-you-know-where-and-why-users-drop-off-the-conversation-6246e99baddc).



This process helps you to understand the steps you can take to improve your assistant.





 Using the notebooks with standard Python tools 

If you choose to use standard Python tools to run the notebooks, you can get the notebooks from GitHub.



* [Dialog Skill Analysis for Watson Assistant](https://github.com/watson-developer-cloud/assistant-dialog-skill-analysis)
* [Watson Assistant Recommendation notebooks (Measure and Analyze Effectiveness)](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook)
* [Watson Assistant Dialog Flow Analysis notebook](https://github.com/watson-developer-cloud/assistant-dialog-flow-analysis)



Again, the [Watson Assistant Continuous Improvement Best Practices Guide](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/raw/master/notebook/IBM%20Watson%20Assistant%20Continuous%20Improvement%20Best%20Practices.pdf) outlines which notebook to use at each stage of your improvement process.
For dictionary-based entities, you can use synonym recommendations to expand your entity definitions.
5. Test each function that you add to the assistant in the "Try it" pane, incrementally, as you go.
6. When you have a working assistant that can successfully handle key tasks, add an integration that deploys the assistant to a development environment. Test the deployed assistant and make refinements.
7. After you build an effective assistant, take a snapshot of the dialog skill and save it as a version.

Saving a version when you reach a development milestone gives you something you can go back to if subsequent changes you make to the skill decrease its effectiveness. See [Creating skill versions](https://cloud.ibm.com/docs/assistant?topic=assistant-versions).
8. Deploy the version of the assistant into a test environment, and test it.

If you use the preview, you can share the URL with others to get their help with testing.
9. Use metrics from the Analytics tab to find areas for improvement, and make adjustments.

If you need to test alternative approaches to addressing an issue, create a version for each solution, so you can deploy and test each one independently, and compare the results.
10. When you are happy with the performance of your assistant, deploy the best version of the assistant into a production environment.
11. Monitor the logs from conversations that users have with the deployed assistant.

You can view the logs for a version of a skill that is running in production from the Analytics tab of a development version of the skill. As you find misclassifications or other issues, you can correct them in the development version of the skill, and then deploy the improved version to production after testing. See [Improving across assistants](https://cloud.ibm.com/docs/assistant?topic=assistant-logslogs-deploy-id) for more details.



The process of analyzing the logs and improving the dialog skill is ongoing. Over time, you might want to expand the tasks that the assistant can handle for you. Customer needs also change.
As you find misclassifications or other issues, you can correct them in the development version of the skill, and then deploy the improved version to production after testing. See [Improving across assistants](https://cloud.ibm.com/docs/assistant?topic=assistant-logslogs-deploy-id) for more details.



The process of analyzing the logs and improving the dialog skill is ongoing. Over time, you might want to expand the tasks that the assistant can handle for you. Customer needs also change. As new needs arise, the metrics generated by your deployed assistants can help you identify and address them in subsequent iterations.
If a node that conditions on an entity type that is mentioned in the user input is higher in the tree than this node, its response is displayed instead.<-- </section "id="section-dialog-runtime-handle-none" "> --><-- <section "id="section-dialog-runtime-disambig-test" "> --> Testing disambiguation To test disambiguation, complete the following steps:<-- <ol> -->1. From the Try it out pane, enter a test utterance that you think is a good candidate for disambiguation, meaning two or more of your dialog nodes are configured to address utterances like it.2. If the response does not contain a list of dialog node options for you to choose from as expected, first check that you added summary information to the external node name field for each of the nodes.3. If disambiguation is still not triggered, it might be that the confidence scores for the nodes are not as close in value as you thought.
<-- <ul> -->

* To see the confidence scores of all the intents that are detected in the user input, temporarily add <? intents ?> to the end of the node response for a node that you know will be triggered.

This SpEL expression shows the intents that were detected in the user input as an array. The array includes the intent name and the level of confidence that your assistant has that the intent reflects the user's intended goal.
* To see which entities, if any, were detected in the user input, you can temporarily replace the current response with a single text response that contains the SpEL expression, <? entities ?>.

This SpEL expression shows the entities that were detected in the user input as an array. The array includes the entity name, location of the entity mention within the user input string, the entity mention string, and the level of confidence that your assistant has that the term is a mention of the entity type specified.
* To see details for all of the artifacts at once, including other properties, such as the value of a given context variable at the time of the call, you can inspect the entire API response.
5.2.7 Minimize the admission of containers with the NET_RAW capability Not Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared 
 5.2.8 Minimize the admission of containers with added capabilities Not Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared 
 5.2.9 Minimize the admission of containers with capabilities assigned Not Scored 2 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared 







 5.3 Network policies and CNI 



Section 5.3 Network policies and CNI benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.3.1 Ensure that the CNI in use supports Network Policies Not Scored 1 Pass IBM 
 5.3.2 Ensure that all Namespaces have Network Policies defined Scored 2 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared 







 5.4 Secrets management 



Section 5.4 Secrets management benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.4.1 Prefer using secrets as files over secrets as environment variables Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared 
 5.4.2 Consider external secret storage Not Scored 2 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared 







 5.5 Extensible admission control 



Section 5.5 Extensible admission control benchmark results
runtimeClassName: kata-tiny





 SLSA level 1 provenance 

Supply chain Levels for Software Artifacts (SLSA), also known as "salsa", is a security framework. SLSA is a checklist of standards and controls to prevent tampering, improve integrity, and secure packages and infrastructure in your projects, businesses, or enterprises. SLSA gets you from safe enough to being as resilient as possible, at any link in the chain.

To achieve SLSA level 1, the build process must be fully scripted and automated, and generate provenance. [Provenance](https://slsa.dev/provenance) is metadata about how an artifact was built, including the build process, top-level source, and dependencies. Knowing the provenance allows software consumers to make risk-based security decisions. Provenance at SLSA level 1 does not protect against tampering, but it offers a basic level of code source identification and can aid in vulnerability management.

To generate a provenance document in a Continuous Delivery Tekton pipeline run, you must provide statements by using the slsa-provenance-statement statements that are emitted in the pipeline step logs.

For example, in the steps that produce a build artifact, add the following statements:

::slsa-provenance-statement::{"subject":[{"name": "icr.io/johndoe/hello-containers-slsa-demo:1-master-287919f9-20220915220023", "digest": {"sha256": "6038ec21bd927a6bb578f58eed3b43d389d27671f1c7bb44ac2c4d7260c3176d"}}]}
* A broad interpretation of what constitutes valid input (but still only within the confines of what the specification leaves open to interpretation)
* A narrow interpretation of what constitutes valid output









 Best practices 

When a service team is responsible for developing an API definition and its canonical implementation (the service) in concert, Postel's robustness principle SHOULD NOT be (mis)applied. This section details best practices that contradict adherence to the robustness principle as it is colloquially understood.



 Specificity of validity 

The formal definition and documentation for a service API SHOULD be as explicit as possible about what constitutes valid input.

A client developer shouldn't have to guess at the boundaries of valid input. Armed with explicit knowledge of input validity, developers can write client applications which are more resistant to unexpected failure (and, in this way, are more robust). This specificity also allows quality and security testing to employ [boundary-value analysis](https://en.wikipedia.org/wiki/Boundary-value_analysis).

A service implementation SHOULD NOT be liberal in accepting input where the API definition is ambiguous about validity. Instead, the definition SHOULD be updated to remove such ambiguity.





 Input canonicalization 

The formal definition for a service API SHOULD NOT allow a broad range of semantically equivalent inputs without strong justification or mitigation of the inherent problems.

There are usability benefits to accepting various forms of input. It can be more convenient for a developer — and lower an API's barrier to entry — if a client application doesn't have to canonicalize input before sending it.

But the downsides of accepting noncanonical input can be significant:



* Canonicalization can be a complicated procedure with many corner cases and ambiguities that significantly increase the complexity of an API. The additional complexity of transforming noncanonical input makes the service harder to develop and maintain. It also expands the testing and attack surfaces.
* If canonicalization decisions are tied to specific libraries used by the service, it could be significantly harder to transition a service implementation to a new technology.
* If canonicalization is done lazily[1], the burden falls on clients to compute equivalency of values.
: You can now create Natural Language Understanding service instances in the IBM Cloud London location.
: Added support for Portuguese relations.
: Added support for Italian relations.
: Added a limit parameter for categories requests that controls the number of categories returned up to a maximum of 10.
: Improved accuracy for Japanese and German sentiment.





 6 December 2018 

Improved accuracy for Italian
: Improved accuracy for Italian categories, keywords, entities, and categories.







 November 2018 



 27 November 2018 

Improvements and bug fixes
: Improved quality of keywords results for English, French, Japanese, and Portuguese input.
: Keywords with different capitalizations now appear in results as the same keyword.
: The Get models method now returns additional fields that you can use to manage custom models across several deployments.



* version: user-provided version string from Knowledge Studio
* version_description: user-provided description of this version from Knowledge Studio (for example, what changed since the previous version)
* workspace_id: An ID provided by Knowledge Studio that remains constant over repeated deployments from the same Knowledge Studio workspace.
* created: A datetime string that indicates when the model was deployed to NLU.







 16 November 2018 

New algorithm, improvements, and bug fixes
: Released new algorithms for English keywords and sentiment to improve accuracy and performance.
: Improved keywords accuracy and performance for English, French, Japanese, and Portuguese input.
: Improved Italian sentiment accuracy and performance, including better accuracy for large text samples.
: Fixed a bug that caused URL-encoded text to appear in Spanish entity disambiguation results.
: Released a new Italian entities model with the latest entity type system. You can learn about the latest type system on the [Entity types and subtypes (Version 2)](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-entity-types-version-2) page. When your application is compatible with the new type system, change the version date parameter in your requests to 2018-11-16 to use the new model.





 9 November 2018 

Categories improvement
: Major improvement in accuracy for Categories in all supported languages.





 8 November 2018
Incorrect or misconfigured target configuration 

The IBM Cloud Metrics Routing service cannot connect to a target and cannot route metrics to a destination target.

  What’s happening 

Your dashboards display graphs with gaps due to missing metrics. You get an unauthorized RC 403 response when trying to access the target. A notification email is received indicating an incorrect target configuration.

  Why it’s happening 

You can get this error in any of the following scenarios:



*  The CRN of the Monitoring instance is not valid.
*  The Monitoring instance is deleted.
*  The service to service authorization that is used by the IBM Cloud Metrics Routing service to authenticate with the Monitoring instance is deleted or lack permissions.



  How to fix it 

To fix the configuration, complete the following tasks:



*  Check that the CRN of the Monitoring instance matches the CRN that is configured in the target. For more information, see [Targets](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-target-manage).
*  Check that you have an authorization definition in Manage > Access (IAM) > Manage access and users > Authorization for the IBM Cloud Metrics Routing service that allows you to authenticate with the Monitoring instance whose connection is failing.
*  Check the Monitoring instance is available in the IBM Cloud Resources dashboard.
* [Norwegian (Nyorsk)](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportnorwegian-nyorsk)
* [Polish](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportpolish)
* [Portuguese](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportportuguese)
* [Romanian](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportromanian)
* [Russian](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportrussian)
* [Slovak](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportslovak)
* [Spanish](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportspanish)
* [Swedish](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportswedish)
* [Turkish](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportturkish)



Language support might be different if you are a IBM Cloud® Dedicated customer. If you are using IBM Cloud® Dedicated, check with your IBM salesperson to confirm which languages are supported in your environment.



 Arabic 



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories X 
 Classifications X 
 Concepts 
 Emotion 
 Entities X* X 
 Keywords X 
 Metadata X 
 Relations X X 
 Semantic roles 
 Sentiment X 
 Syntax X
Alternatively, you can promote a document to ground truth by clicking Accept on the Documents page.







 Results 

After you resolve the annotation conflicts and promote the documents to ground truth, you can use them to train the machine learning model.







 Lesson 7: Creating a machine learning model 

In this lesson, you will learn how to create a machine learning model in Knowledge Studio.



 About this task 

When you create a machine learning model, you select the document sets that you want to use to train it. You also specify the percentage of documents that are to be used as training data, test data, and blind data. Only documents that became ground truth through approval or adjudication can be used to train the machine learning model.

For more information about the machine learning model, see [Training the machine learning model](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-train-ml) and [Analyzing machine learning model performance](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-evaluate-ml).





 Procedure 



1. Log in to Knowledge Studio as the administrator.
2. Click Machine Learning Model > Performance > Train and evaluate.
3. Select All, and then click Train & Evaluate.

> Note: Training might take more than ten minutes, or even hours, depending on the number of human annotations and the number of words in all the documents.
4. After the machine learning model is trained, you can export it from the Version page, or you can view detailed information about its performance by clicking the Detailed Statistics links that are located above each of the graphs on the Performance page.
5. To view the Training / Test / Blind Sets page, click the Train and evaluate button.
6. To see the documents that human annotators worked on, click View Ground Truth.
7. To see the annotations that the trained machine learning model created on that same set of documents, click View Decoding Results.
8. To view details about the precision, recall, and F1 scores for the machine learning model, click the Performance page.
9.