vania-janet/multiturn-rag-retrieval-data huggingface dataset:
This feature is currently available for English only. See [Adding contextual entities](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-create-annotation-based) for more information.

When you access the tool with an Internet Explorer web browser, you cannot label entity mentions in intent user examples nor edit user example text.

Entity recommendations
: Watson can now recommend synonyms for your entity values. The recommender finds related synonyms based on contextual similarity extracted from a vast body of existing information, including large sources of written text, and uses natural language processing techniques to identify words similar to the existing synonyms in your entity value. For more information see [Synonyms](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-create-dictionary-based).

New API version
: The current API version is now 2018-07-10. This version introduces the following changes:



* The content of the /message output object changed from being a text JSON object to being a generic array that supports multiple rich response types, including image, option, pause, and text.
* Support for contextual entities was added.
* You can no longer add user-defined properties in context.metadata. However, you can add them directly to context.



Overview page date filter
: Use the new date filters to choose the period for which data is displayed. These filters affect all data shown on the page: not just the number of conversations displayed in the graph, but also the statistics displayed along with the graph, and the lists of top intents and entities. See [Controls](https://cloud.ibm.com/docs/assistant?topic=assistant-logs-overviewlogs-overview-controls) for more information.

Pattern limit expanded
: When using the Patterns field to [define specific patterns for an entity value](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-patterns), the pattern (regular expression) is now limited to 512 characters.





 2 July 2018 

Jump-tos from conditional responses
: You can now configure a conditional response to jump directly to another node.
This can help you decide whether to add or exclude a suggested entry.
* Add this entry (+) adds the entry suggestion to your dictionary.
* Exclude this entry (-) removes the entry from the list of suggested entries and adds it to Excluded Terms.
* Add all adds the entire list of suggested entries to the dictionary (this does not add entries listed under Excluded Terms).
* Exclude all moves all current entry suggestions to Excluded Terms.



Related tasks:

[Pre-annotating documents with a dictionary](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-preannotationwks_preannot)

Related reference:

[Language support](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-language-support)
Making machine learning model improvements 

After you determine areas in which the model is having trouble, take steps to improve its performance.



 Creating model versions 

After you create a machine learning model, you can take a snapshot to keep a backup version of the current resources in case you want to restore the resources in a future iteration.



 About this task 

The F1 score provides an indication of the quality of the model. If the model performance results are good, you might want to store a version of the component before changing any of the resources. If changes that you make result in poorer quality, you can revert to a version that you stored. When you revert to an older version, all annotation tasks are archived because they are no longer valid.

You can have a maximum of 10 versions of a workspace. If you reach that limit, delete older versions or versions that you no longer need before creating a new version.

When you create a new version, the following resources are captured:



* Type system
* Corpus
* Ground truth
* Machine learning model
* Machine learning model evaluation results



The following resources are excluded:



* Annotation tasks, because they are temporal by design, used only for determining ground truth
* Dictionaries, because dictionaries can be large, and various types of dictionaries are managed in different ways







 Procedure 

To create and restore machine learning model versions:



1. Log in as a Knowledge Studio administrator or project manager, and select your workspace.
2. Select Machine Learning Model > Performance. Performance statistics about the current version, labeled version 1.0, are displayed.
3. To take a snapshot of the current version, click Machine Learning Model > Versions, and then click Take Snapshot. The resources in version 1.0 are frozen, and a new version, labeled 1.1, becomes the current version. For each new version that you create, the minor version number is incremented, for example, 1.0 becomes 1.1 and then becomes 1.2.
4. Revise the workspace resources as needed, re-train, and re-evaluate the model.
5.
For example, if you apply the Keywords prebuilt enrichment, you can create a facet based on the keywords that are found in your collection. As it processes your document, the Keywords enrichment recognizes any keyword mentions that occur and stores information about them in a set of fields that begin with the prefix enriched_{field_name}.keywords. To create a keyword facet, add a facet that takes its categories from the field where keyword mentions are stored, the enriched_{field_name}.keywords.mentions.text field. For more information about the Keywords enrichment, see [Applying prebuilt enrichments](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-nlunlu-keywords).

Similarly, you can use the fields that are generated by domain-specific enrichments as the source of the facet. In fact, as you create a Regular expression enrichment, you can define a facet by which to categorize any recognized expression mentions. For more information, see [Adding domain-specific resources](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domainregex).

To add a facet from existing fields, complete the following steps:



1. On the Improve and customize page, click Customize display, and then click Facets.
2. Click New facet > From existing fields in a collection.
3. Choose the field you'd like your facet to use, such as enriched_text.entities.type. Add the facet label and choose a filter control option.
4. Optional: You can adjust the maximum number of facet values that are displayed.

The maximum number is the sum of the number of facet values that are displayed for each facet type. By default, a total of 10 facet values are displayed.
5. As you test the facet, you can adjust the options.







 Creating a facet by creating a dictionary 

Add a facet to group a set of terms that have special meaning to your use case by creating a dictionary.
But if one reference to "encrypts" is a general reference, or if the two occurrences refer to two different acts of encryption, you would not coreference them. If two different verbs refer to the same occurrence of an action, you might want to coreference the mentions. For example, in the statement, "He encrypted the document, and after that processing he sent the file ... ", the mentions "encrypted" and "processing" could be coreferenced because they refer to the same instance of an action.

What's most important is consistency. Decide how you want annotate coreference and specify the rules, with examples, clearly in your annotation guidelines.





 Procedure 

To annotate mentions as coreferences:



1. Log in as a human annotator (or as an administrator or project manager who was assigned documents to annotate). Workspaces that contain tasks that are assigned to you are displayed.
2. Open a workspace, and then click Machine Learning Model > Annotations. Click the Annotation Tasks tab. The annotation tasks that are assigned to you are displayed.
3. Open the annotation task that you want to work on. The annotation sets that are assigned to you are displayed.
4. Click Annotate to open the annotation set you want to work on. The documents in the annotation set are displayed.
5. Open the document that you want to annotate. By default, the document opens in Mention mode, which is the mode you use to annotate entity mentions.
6. Click Coreferences.
7. Create a coreference chain:



1. Move through the document and click each mention that means the same thing and is labeled by the same entity type. For example, click each occurrence of IBM, International Business Machines, and IBM Corp., assuming all these mentions have the entity type ORGANIZATION.
2. Double-click the last mention that you want to add to the chain. A coreference chain is created in the side panel. The name of the chain matches the first mention that you selected.
For example, "Barack", "Michelle", "he", and "she" are all of the same entity type, PERSON, but "Barack" and "he" are one entity, and "Michelle" and "she" are another entity. In this example, you create two coreference chains.

When you create a coreference chain, you must select mentions that have been marked by the same entity type. In some cases, however, you might want to include mentions of different types in the same coreference chain. To do this, you must create multiple chains and then merge them. For example, think about how people progressively use shorthand to avoid repeating things in text. In a traffic incident report, the first reference to a vehicle might be "2004 Honda Accord Sedan". Later, the author might refer to the vehicle as "Accord", and then later refer to the vehicle as simply "vehicle". If the type system includes entries for vehicle manufacturer, model, and type, you could create multiple coreference chains per entity type, and then merge them to create a consolidated chain. The merged chain helps train the machine learning model to recognize that all of these mentions refer to the same thing.

Another way to combine mentions of different entity types is to create a chain with mentions of one entity type. You can then click a mention of another entity type, and then click the chain that you created to add the mention to that chain.

Depending on your annotation guidelines, you might want to create coreference chains for verbs as well as nouns if the verbs mention the same instance of an action. For example, if two mentions of the verb "encrypts" refer to the same occurrence of encryption, you might coreference the mentions. But if one reference to "encrypts" is a general reference, or if the two occurrences refer to two different acts of encryption, you would not coreference them. If two different verbs refer to the same occurrence of an action, you might want to coreference the mentions. For example, in the statement, "He encrypted the document, and after that processing he sent the file ... ", the mentions "encrypted" and "processing" could be coreferenced because they refer to the same instance of an action.
Language support 

When you create a collection, you specify the language of the collection. All of the documents that you add to a collection must be written in the same language.

Discovery is not optimized for multilingual search. Although you can add several collections, each one with documents in a separate language, into one project, the query results from the project will be unpredictable. The results might include irrelevant passages from a document in a language that is different from the language of the user's query.

The following table describes the product features that are supported in each language.



Feature support per language

 Language Supported features 

 Arabic (ar) Advanced rules models, Built-in entities, Classifier (Document and Text), Custom entities, Dictionary, Document sentiment, Keywords, Machine Learning, Optical character recognition v1, Parts of speech, Regular expressions, Smart Document Understanding, Stemmer, Table Understanding 
 Bosnian (bs) Classifier (Document and Text), Custom entities, Dictionary, Parts of speech, Regular expressions 
 Chinese, simplified (zh-CN) Advanced rules models, Built-in entities, Classifier (Document and Text), Custom entities, Dictionary, Document sentiment, Keywords, Machine Learning, Optical character recognition v1, Parts of speech, Phrase sentiment, Regular expressions, Smart Document Understanding, Table Understanding 
 Chinese, traditional (zh-TW) Advanced rules models, Classifier (Document and Text), Custom entities, Dictionary, Regular expressions, Machine Learning, Optical character recognition v1, Parts of speech, Phrase sentiment, Smart Document Understanding, Table Understanding 
 Croatian (hr) Classifier (Document and Text), Custom entities, Dictionary, Regular expressions, Parts of speech 
 Czech (cs) Classifier (Document and Text), Custom entities, Dictionary, Optical character recognition v1, Parts of speech, Phrase sentiment, Regular expressions, Smart Document Understanding, Stemmer, Table Understanding 
 Danish (da) Classifier (Document and Text), Custom entities, Dictionary, Optical character recognition v1, Parts of speech, Regular expressions, Smart Document Understanding, Stemmer, Table Understanding
See also [testing data](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-glossarygloss_T) and [training data](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-glossarygloss_T).







 C 



* concordance

Provides a way to ensure that the same mention is annotated with the same entity type throughout a document and across annotation sets. Concordance helps ensure consistency across multiple occurrences of a mention without requiring the human annotator to manually label each occurrence.
* confusion matrix

A table that provides a detailed numeric breakdown of annotated document sets. The table is used to compare the annotations that were added by a machine learning model to the annotations in the ground truth. The table reports the number of [false positives](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-glossarygloss_F), [false negatives](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-glossarygloss_F), [true positives](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-glossarygloss_T), and [true negatives](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-glossarygloss_T).
* coreference

A relationship between two words or phrases in which both refer to the same person or thing and one stands as a linguistic antecedent of the other. For example, there is a coreference between the two pronouns in the phrase "She taught herself" but not in the phrase "She taught her". A coreference links two equivalent entities in the same text.
* coreference chain

A list of entities that were annotated as coreferences. When you annotate mentions as coreferences, the system creates a coreference chain.
curl -X POST -u "apikey:{apikey}" --header "Content-Type: audio/flac" --data-binary @{path}audio-file.flac "{url}/v1/recognize?model=en-US_Telephony"





 Step 2: Identify the features that are available with next-generation models 

Next-generation models support slightly fewer features and parameters than previous-generation models. However, although they lack full parity, most features are available with both types of models. And where a feature is limited to a subset of languages, the limitations apply equally to both types of models.

For information about the features that are supported with the different model types, see



* [Supported features for previous-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-modelsmodels-features)
* [Supported features for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ngmodels-ng-features)
* [Parameter summary](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-summary)



The service continues to make new features available with next-generation models. All updates to feature support are documented in the release notes and in the documentation for the model types.

To migrate to next-generation models, you must remove features that aren't supported by next-generation models from your speech recognition requests. You can also consider using features such as low latency and character insertion bias that are available only with next-generation models.

For example, the following speech recognition request uses the profanity_filter, redaction, and word_alternatives_threshold parameters with the previous-generation en-US_NarrowbandModel:
Differences exist between the use of the sounds_like field for custom models that are based on next-generation and previous-generation models. For more information about using the sounds_like field with custom models that are based on next-generation models, see [Working with custom words for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWords-ngworkingWords-ng).

Important: Deprecated customization_id parameter removed from the documentation
: Important: On [9 October 2018](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-release-notesspeech-to-text-9october2018), the customization_id parameter of all speech recognition requests was deprecated and replaced by the language_customization_id parameter. The customization_id parameter has now been removed from the documentation for the speech recognition methods:



* /v1/recognize for WebSocket requests
* POST /v1/recognize for synchronous HTTP requests (including multipart requests)
* POST /v1/recognitions for asynchronous HTTP requests



Note: If you use the Watson SDKs, make sure that you have updated any application code to use the language_customization_id parameter instead of the customization_id parameter. The customization_id parameter will no longer be available from the equivalent methods of the SDKs as of their next major release. For more information about the speech recognition methods, see the [API & SDK reference](https://cloud.ibm.com/apidocs/speech-to-text/speech-to-textservice-endpoint).





 17 March 2022 

Grammar support for next-generation models is now generally available
: Grammar support is now generally available (GA) for next-general models that meet the following conditions:



* The models are generally available.
* The models support language model customization.



For more information, see the following topics:
* [Creating a custom acoustic model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-acoustic)
* [Using a custom acoustic model for speech recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-acousticUse)
* [Using custom acoustic and custom language models together](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-useBoth)
* [Working with audio resources](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-audioResources)
* [Managing custom acoustic models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-manageAcousticModels)
* [Managing audio resources](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-manageAudio)



* Grammars



* [Using grammars with custom language models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammars)
* [Understanding grammars](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarUnderstand)
* [Adding a grammar to a custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarAdd)
* [Using a grammar for speech recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarUse)
* [Managing grammars](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-manageGrammars)
Standard text responses were being saved as output.text instead of output.generic. For more information about the output object, see [Anatomy of a dialog call](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-runtime-context).





 13 September 2019 

Improved Entities and Intents page responsiveness
: The Entities and Intents pages were updated to use a new JavaScript library that increases the page responsiveness. As a result, the look of some graphical user interface elements, such as buttons, changed slightly, but the function did not.

Creating contextual entities got easier
: The process you use to annotate entity mentions from intent user examples was improved. You can now put the intent page into annotation mode to more easily select and label mentions. See [Adding contextual entities](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-create-annotation-based).





 6 September 2019 

Label character limit increase
: The limit to the number of characters allowed for a label that you define for an option response type changed from 64 characters to 2,048 characters.





 12 August 2019 

New dialog method
: The getMatch method was added. You can use this method to extract a specific occurrence of a regular expression pattern that recurs in user input. For more details, see the [dialog methods](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-methodsdialog-methods-strings-getMatch) topic.





 9 August 2019 

Introductory product tour
: For some first-time users, a new introductory product tour is shown that the user can choose to follow to perform the initial steps of creating an assistant.





 1 August 2019 

Webhook callouts are available
: Add webhooks to dialog nodes to make programmatic calls to an external application as part of the conversational flow. The new Webhook support simplifies the callout implementation process. (No more action JSON objects required.)
* [Adding variety](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-overviewdialog-overview-add-variety)







 Adding multiple lines 

If you want a single text response to include multiple lines separated by carriage returns, then follow these steps:



1. Add each line that you want to show to the user as a separate sentence into its own response variation field. For example:



Multiple line response

 Response variations 

 Hi. 
 How are you today? 



2. For the response variation setting, choose multiline.

If you are using a dialog skill that was created before support for rich response types was added to the product, then you might not see the multiline option. Add a second text response type to the current node response. This action changes how the response is represented in the underlying JSON. As a result, the multiline option becomes available. Choose the multiline variation type. Now, you can delete the second text response type that you added to the response.



When the response is shown to the user, both response variations are displayed, one on each line, like this:

Hi.
How are you today?





 Adding variety 

If your users return to your conversation service frequently, they might be bored to hear the same greetings and responses every time. You can add variations to your responses so that your conversation can respond to the same condition in different ways.

In this example, the answer that your assistant provides in response to questions about store locations differs from one interaction to the next:

![Shows a node that shows a user ask, Where are you located, and the dialog has three different responses defined.](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/variety.png)

You can choose to rotate through the response variations sequentially or in random order. By default, responses are rotated sequentially, as if they were chosen from an ordered list.

To change the sequence in which individual text responses are returned, complete the following steps: