vania-janet/multiturn-rag-retrieval-data huggingface dataset:
[IBM Watson® Assistant](https://cloud.ibm.com/docs/assistant?topic=assistant-getting-started) conversation [Location-based events](https://cloud.ibm.com/docs/assistant?topic=assistant-at-events) 
 [IBM Watson® Discovery](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-getting-started) discovery [Location-based events](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-at_events) 
 [IBM Watson® Knowledge Studio](https://cloud.ibm.com/docs/watson-knowledge-studio) knowledge-studio [Location-based events](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-activity-tracker-events) 
 [IBM Watson® Natural Language Understanding](https://cloud.ibm.com/docs/natural-language-understanding) natural-language-understanding [Location-based events](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-at_events) 
 [IBM Watson® Speech to Text](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-gettingStarted) speech-to-text [Location-based events](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-at-events) 
 [IBM Watson® Text to Speech](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-gettingStarted) text-to-speech [Location-based events](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-at-events) 







 Web and mobile services 

The following table lists web and mobile services that send auditing events:



Table 15. List of web and mobile events

 Service CRN service name Events
* [Train the custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreatetrainModel-language)
* [Working with corpora and custom words for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWords-ng)



Further improvements to updated next-generation language model customization
: Language model customization for English and Japanese next-generation models was recently improved. This service update includes further internal improvements. For more information about improved next-generation customization, see



* [27 February 2023 service update](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-release-notesspeech-to-text-27february2023) and
* [Improved language model customization for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-customizationcustomLanguage-intro-ng)







 13 March 2023 

Defect fix: Smart formatting for US English dates is now correct
: Defect fix: Smart formatting now correctly includes days of the week and dates when both are present in the spoken audio, for example, Tuesday February 28. Previously, in some cases the day of the week was omitted and the date was presented incorrectly. Note that smart formatting is beta functionality.

Defect fix: Update documentation for speech hesitation words for next-generation models
: Defect fix: Documentation for speech hesitation words for next-generation models has been updated. More details are provided about US English and Japanese hesitation words. Next-generation models include the actual hesitation words in transcription results, unlike previous-generation models, which include only hesitation markers. For more information, see [Speech hesitations and hesitation markers](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-basic-responseresponse-hesitation).





 27 February 2023 

New Japanese next-generation telephony model
* [Using upgraded custom models for speech recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-upgrade-use)







 7 October 2022 

New Swedish next-generation telephony model
: The service now offers a next-generation telephony model for Swedish: sv-SE_Telephony. The new model supports low latency and is generally available. It also supports language model customization and grammars. For more information about next-generation models and low latency, see



* [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* [Customization support for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-supportcustom-language-support-ng)
* [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency)



Updates to English next-generation telephony models
: The English next-generation telephony models have been updated for improved speech recognition:



* en-AU_Telephony
* en-GB_Telephony
* en-IN_Telephony
* en-US_Telephony



All of these models continue to support low latency. You do not need to upgrade custom models that are based on these models. For more information about all available next-generation models, see [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng).





 21 September 2022 

New Activity Tracker event for GDPR deletion of user information
: The service now returns an Activity Tracker event when you use the DELETE /v1/user_data method to delete all information about a user. The event is named speech-to-text.gdpr-user-data.delete.
(For more information about using the context in your dialog, see [How the dialog is processed](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-runtime).)

You can specify any variable name you want to use for a user-defined context variable. If the specified variable already exists, it is overwritten with the new value; if not, a new variable is added to the context.

The output from this request includes not only the usual output, but also the context, showing that the specified values have been added. If you are using the stateless message method, this context data must be stored locally and sent back to the Watson Assistant service as part of the next message. If you are using the stateful message method, this context is stored automatically and will persist for the life of the session.

{
"output": {
"generic": [!
{
"response_type": "text",
"text": "Welcome to the Watson Assistant example"
}
],
"intents": [
{
"intent": "hello",
"confidence": 1
}
],
"entities": []
},
"user_id": "my_user_id",
"context": {
"global": {
"system": {
"turn_count": 1,
"user_id": "my_user_id"
}
},
"skills": {
"main skill": {
"user_defined": {
"account_number": "123456"
}
}
}
}
}
Show more





 Restoring conversation state 

In some situations, you might want the ability to restore a conversation to a previous state.

You can use the export option on stateful message requests to specify that you want the context object in the response to include complete session state data. If you specify true for this option, the returned skill context includes an encoded state property that represents the current conversation state.
* Average messages per conversation - The total messages received during the selected time period divided by the total conversations during the selected time period.
* Total messages - The total number of messages received from active users over the selected time period.
* Active users - The number of unique users who have engaged with your assistant within the selected time period.
* Average conversations per user - The total conversations divided by the total number of unique users during the selected time period.

Statistics for Active users and Average conversations per user require a unique user_id parameter. See [Enabling user metrics](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-logs-resourceslogs-resources-user-id) for more information.







 Top Intents and Top Entities 

You can also view the intents and entities that were recognized most often during the specified time period.



* Top intents - Intents are shown in a simple list. In addition to seeing the number of times an intent was recognized, you can select an intent to open the User conversations page with the date range filtered to match the data you are viewing, and the intent filtered to match the selected intent.
* Top entities are also shown in a list. For each entity you can select from the Values column to see a list of the most common values that were identified for this entity during the time period. You can also select an entity to open the User conversations page with the date range filtered to match the data you are viewing, and the entity filtered to match the selected entity.



See [Improve your skill](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-logs) for tips on how to edit intents and entities based on discoveries you make by reviewing the intents and entities that your assistant recognizes.
Specifying a small threshold can potentially produce many matches. If you specify a threshold, you must also specify one or more keywords. Omit the parameter to return no matches.



The following limits apply to keyword spotting:



* You can spot a maximum of 1000 keywords with a single request.
* A single keyword can have a maximum length of 1024 characters. The maximum effective length for double-byte languages might be shorter.
* Most HTTP servers and proxies impose a limit of 8 KB on the parameters of a request. Spotting a very large number of keywords or many lengthy keywords can exceed this limit. If you need to match more keywords, consider using a [multipart HTTP request](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-httpHTTP-multi).



Keyword spotting is necessary to identify keywords in an audio stream. You cannot identify keywords by processing a final transcript because the transcript represents the service's best decoding results for the input audio. It does not include tokens with lower confidence scores that might represent a word of interest. So applying text processing tools to a transcript on the client side might not identify keywords. A richer representation of decoding results is needed, and that representation is available only at the server.



 Keyword spotting results 

The service returns the results in a keywords_result field that is an element of the results array. The keywords_result field is a dictionary, or associative array, of enumerable properties. Each property is identified by a specified keyword and includes an array of objects. The service returns one element of the array for each match that it finds for the keyword. The object for each match includes the following fields:



* normalized_text is the specified keyword that is normalized to the spoken phrase that matched in the input audio.
* start_time is the start time in seconds of the match.
* end_time is the end time in seconds of the match.
* confidence is the service's confidence that the match represents the specified keyword. The confidence must be at least as great as the specified threshold to be included in the results.
However, you can also check for an intent; just type the intent name into the field. You can use AND and OR operators here to define more complex conditions.

The Check for value is first used as a condition, but then becomes the value of the context variable that you name in the Save as field. It specifies both what to check for and what to save. If you want to change how the value is saved, then add the expression that reformats the value to the Check for field.

For example, if the entity is a pattern entity, such as @email, then after adding the entity name, append .literal to it. Adding .literal indicates that you want to capture the exact text that was entered by the user and was identified as an email address based on its pattern.

In some cases, you might want to use an expression to capture the value, but not apply the expression to what is saved. In such cases, you can use one value in the Check for field to capture the value, and then open the JSON editor to change the value of the context variable, so it saves something else.

Any edit you make to a slot's context variable value in the JSON editor is not reflected in the Check for field after you exit the JSON editor. And if you click the Check for field to give the field focus at any time after you use the JSON editor to edit the value, then the change you made is lost.

Avoid checking for context variable values in the Check for field. Because the value you check for is also the value that is saved, using a context variable in the condition can lead to unexpected behavior.
* Save as: Provide a name for the context variable in which to store the value of interest from the user's response to the slot prompt.

Do not reuse a context variable that is used elsewhere in the dialog. If the context variable has a value already, then the slot's prompt is not displayed. It is only when the context variable for the slot is null that the prompt for the slot is displayed.
* Prompt: Write a statement that elicits the piece of the information you need from the user. After displaying this prompt, the conversation pauses and your assistant waits for the user to respond.
* [Public and private network endpoints](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-public-private-endpoints)
* [Virtual Private Endpoints](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-virtual-private-endpoints)



* Service background



* [The science behind the service](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-science)
* [Research references](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-references)





* Reference



* [API & SDK reference](https://cloud.ibm.com/apidocs/speech-to-text)
* [High availability and disaster recovery](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-ha-dr)
* [Activity Tracker events](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-at-events)
* [Watson SDKs](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-using-sdks)
* [Watson GitHub repos](https://github.com/watson-developer-cloud/)



* Help



* [Known limitations](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-known-limitations)
* [Usage FAQs](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-faq-usage)
* [Pricing FAQs](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-faq-pricing)
* Developer community
For Not found responses (that are displayed when the user does not provide a valid value), you can choose one of these actions to perform:



* Wait for user input (Default): Pauses the conversation and your assistant waits for the user to respond. In the simplest case, the text you specify here can more explicitly state the type of information you need the user to provide. If you use this action with a conditional response, be sure to word the conditional response such that you clearly state what was wrong with the user's answer and what you expect them to provide instead.
* Prompt again: After displaying the Not found response, your assistant repeats the slot prompt again and waits for the user to respond. If you use this action with a conditional response, the response can merely explain what was wrong about the answer the user provided. It does not need to reiterate the type of information you want the user to provide because the slot prompt typically explains that.

If you choose this option, consider adding at least one variation of the Not found response so that the user does not see the exact same text more than once. Take the opportunity to use different wording to explain to the user what information you need them to provide and in what format.
* Skip this slot: Instructs your assistant to stop trying to fill the current slot, and instead, move on to the prompt for the next empty slot. This option is useful in a slot where you want to both make the slot optional and to display a prompt that asks the user for information. For example, you might have a @seating entity that captures restaurant seating preferences, such as outside, near the fireplace, private, and so on. You can add a slot that prompts the user with, Do you have any seating preferences? and checks for @seating.values. If a valid response is provided, it saves the preference information to $seating_preferences. However, by choosing this action as the Not found response next step, you instruct your assistant to stop trying to fill this slot if the user does not provide a valid value for it.
* Skip to response: If, when the condition you define is met, you no longer need to fill any of the remaining slots in this node, choose this action to skip the remaining slots and go directly to the node-level response next.
Analyzing dialog and actions 

The Analyze page provides a summary of the interactions between users and your assistant. If the dialog feature is enabled, the Analyze page remains the same, but some slight differences in functionality exist.



  Overview tab 

When you view the Overview page, you can see action completion information in the Action completion diagram if a dialog node triggers an action. The Action completion diagram is empty if you are using only a dialog in your assistant. The three cards that display information about the most frequent actions, least frequent actions, and least completed actions are not available if your assistant uses only a dialog.

For more information about the Analyze page and how to use analytics with actions, see [Use analytics to review your entire assistant at a glance](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-analytics-overview).





  Action completion tab 

The Action completion page of Watson Assistant provides an overview of how all your assistant's actions are doing. If the dialog feature is enabled, the Action completion tab is relevant only if a dialog node triggers an action. If your assistant uses only a dialog, then this tab will be empty.

For more information about understanding action completion with actions, see [Understand your most and least successful actions](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-analytics-action-completion).





  Conversations tab 

The Conversations page of Watson Assistant provides a history of conversations between users and a deployed assistant. You can use this history to improve how your assistants understand and respond to user requests.

From the Conversations page, an intent that directly calls an action is displayed in the Topics column. For example, you might set up an intent called buy_takeout in the dialog, and that intent calls the order pizza action. This conversation topic is listed as buy_takeout > order pizza in the Topics column.

You might also see Dialog called action listed in the Requests column next to a conversation. In this case, customer input triggered an intent. Then, the customer engaged with the assistant before an action was eventually called.

For more information about analyzing conversations with actions, see [Review customer conversations](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-analytics-conversations).
* For more information about the next-generation models to which you can migrate from each of the deprecated models, see [Previous-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models)
* For more information about the next-generation models, see [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* For more information about migrating from previous-generation to next-generation models, see [Migrating to next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-migrate).



Note: When the previous-generation en-US_BroadbandModel is removed from service on 15 September, the next-generation en-US_Multimedia model will become the default model for speech recognition requests.

Next-generation models now support audio-parsing parameters
: All next-generation models now support the following audio-parsing parameters as generally available features:



* end_of_phrase_silence_time specifies the duration of the pause interval at which the service splits a transcript into multiple final results. For more information, see [End of phrase silence time](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-parsingsilence-time).
* split_transcript_at_phrase_end directs the service to split the transcript into multiple final results based on semantic features of the input. For more information, see [Split transcript at phrase end](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-parsingsplit-transcript).



Defect fix: Correct speaker labels documentation
: Defect fix: Documentation of speaker labels included the following erroneous statement in multiple places: For next-generation models, speaker labels are not supported for use with interim results or low latency. Speaker labels are supported for use with interim results and low latency for next-generation models.
Currently, you can only directly reference synonym entities that you define (pattern values are ignored). You cannot use [system entities](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-system-entities).

If you choose to reference an entity as an intent example (for example, @PhoneModelName)anywhere in your training data it cancels the value of using a direct reference (for example, Galaxy S8) in an intent example anywhere else. All intents will then use the entity-as-an-intent-example approach. You cannot apply this approach for a specific intent only.

In practice, this means that if you have previously trained most of your intents based on direct references (Galaxy S8), and you now use entity references (@PhoneModelName) for just one intent, the change impacts your previous training. If you do choose to use @Entity references, you must replace all previous direct references with @Entity references.

Defining one example intent with an @Entity that has 10 values that are defined for it does not equate to specifying that example intent 10 times. The Watson Assistant service does not give that much weight to that one example intent syntax.







 Testing your intents 

After you have finished creating new intents, you can test the system to see if it recognizes your intents as you expect.



1. Click Try it.

![Ask Watson](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/ask_watson.png)
2. In the Try it out pane, enter a question or other text string and press Enter to see which intent is recognized. If the wrong intent is recognized, you can improve your model by adding this text as an example to the correct intent.

If you have recently made changes in your skill, you might see a message that indicates that the system is still retraining. If you see this message, wait until training completes before testing:

!