vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Keyword spotting and word alternatives 

The IBM Watson® Speech to Text service can identify user-specified keywords in its transcription results. It can also suggest alternative words that are acoustically similar to the words of a transcript. In both cases, the keywords and word alternatives must meet a user-specified level of confidence.



 Keyword spotting 

The keywords and keywords_threshold parameters are supported only with previous-generation models, not with next-generation models.

The keyword spotting feature detects specified strings in a transcript. The service can spot the same keyword multiple times and report each occurrence. The service spots keywords only in the final results, not in interim results. By default, the service does no keyword spotting.

To use keyword spotting, you must specify both of the following parameters:



* Use the keywords parameter to specify an array of strings to be spotted. The service spots no keywords if you omit the parameter or specify an empty array. A keyword string can include more than one token. For example, the keyword Speech to Text has three tokens. Keyword matching is case-insensitive, so Speech to Text is effectively equivalent to speech to text.

For US English, the service normalizes each keyword to match spoken versus written strings. For example, it normalizes numbers to match how they are spoken as opposed to written. For other languages, keywords must be specified as they are spoken.
* Use the keywords_threshold parameter to specify a probability between 0.0 and 1.0 for a keyword match. The threshold indicates the lower bound for the level of confidence the service must have for a word to match the keyword. A keyword is spotted in the transcript only if its confidence is greater than or equal to the specified threshold.

Specifying a small threshold can potentially produce many matches. If you specify a threshold, you must also specify one or more keywords. Omit the parameter to return no matches.



The following limits apply to keyword spotting:



* You can spot a maximum of 1000 keywords with a single request.
* A single keyword can have a maximum length of 1024 characters. The maximum effective length for double-byte languages might be shorter.
[Checkmark](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker//images/checkmark-icon.svg) [Location-based events](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-at_events) 
 [IBM Watson® Speech to Text](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-gettingStarted) speech-to-text ![Checkmark](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker//images/checkmark-icon.svg) [Location-based events](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-at-events) 
 [IBM Watson® Text to Speech](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-gettingStarted) text-to-speech ![Checkmark](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker//images/checkmark-icon.svg) [Location-based events](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-at-events) 







 Web and mobile services 

The following table lists web and mobile services that send auditing events:



Table 17. List of web and mobile events

 Service CRN service name Activity Tracker hosted event search offering Events
* [Using languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-service-featuresfeatures-languages) and [Using audio formats](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-service-featuresfeatures-audio) describe the basic underpinnings of the service's capabilities. You must choose a language and model that are suitable for your audio, and you must understand the characteristics of your audio to make that choice and to pass your audio to the service.
* [Recognizing speech with the service](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-service-featuresfeatures-recognition) provides links to simple examples of speech recognition requests and responses. There are also links to detailed presentations of each of the service's interfaces. Learn more about and experiment with the interfaces to determine which is best suited to your application needs.
* [Using speech recognition parameters](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-service-featuresfeatures-parameters) introduces the many parameters that you can use to tailor speech recognition requests and transcription responses to your needs. The service's WebSocket and HTTP interfaces support an impressive array of capabilities, most of which are common to all supported interfaces. Use the links to find the parameters that are right for you.
* [Customizing the service](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-service-featuresfeatures-customization) describes the more advanced topics of language model and acoustic model customization, which can help you gain the most from the service's capabilities. The section also presents grammars, which you can use with language models to limit possible responses to precise strings and phrases.
* [Pricing](https://www.ibm.com/cloud/watson-speech-to-text/pricing)
* [Blogs](https://developer.ibm.com/blogs/)
* [Webinars](https://www.ibm.com/watson/webinars/)





* How to



* Using languages and models



* [Previous-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models)
* [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* [Migrating to next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-migrate)
* [Using a model for speech recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-use)
* [Listing information about models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-list)



* Using audio formats



* [Audio terminology and characteristics](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-audio-terminology)
* [Supported audio formats](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-audio-formats)



* Recognizing speech with the service



* [Making a speech recognition request](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-basic-request)
* [Understanding speech recognition results](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-basic-response)
However, the conversation cannot digress away from a node under the following circumstances:



* If any of the child nodes of the current node contain the anything_else or true condition



These conditions are special in that they always evaluate to true. Because of their known behavior, they are often used in dialogs to force a parent node to evaluate a specific child node in succession. To prevent breaking existing dialog flow logic, digression are not allowed in this case. Before you can enable digressions away from such a node, you must change the child node's condition to something else.



* If the node is configured to jump to another node or skip user input after it is processed



The final step section of a node specifies what should happen after the node is processed. When the dialog is configured to jump directly to another node, it is often to ensure that a specific sequence is followed. And when the node is configured to skip user input, it is equivalent to forcing the dialog to process the first child node after the current node in succession. To prevent breaking existing dialog flow logic, digressions are not allowed in either of these cases. Before you can enable digressions away from this node, you must change what is specified in the final step section.







 Customizing digressions 

You do not define the start and end of a digression. The user is entirely in control of the digression flow at run time. You only specify how each node should or should not participate in a user-led digression. For each node, you configure whether:



* a digression can start from and leave the node
* a digression that starts elsewhere can target and enter the node
* a digression that starts elsewhere and enters the node must return to the interrupted dialog flow after the current dialog flow is completed



To change the digression behavior for an individual node, complete the following steps:



1. Click the node to open its edit view.
2. Click Customize, and then click the Digressions tab.

The configuration options differ depending on whether the node you are editing is a root node, a child node, a node with children, or a node with slots.
Currently, you can only directly reference synonym entities that you define (pattern values are ignored). You cannot use [system entities](https://cloud.ibm.com/docs/assistant?topic=assistant-system-entities).

If you choose to reference an entity as an intent example (for example, @PhoneModelName)anywhere in your training data it cancels the value of using a direct reference (for example, Galaxy S8) in an intent example anywhere else. All intents will then use the entity-as-an-intent-example approach. You cannot apply this approach for a specific intent only.

In practice, this means that if you have previously trained most of your intents based on direct references (Galaxy S8), and you now use entity references (@PhoneModelName) for just one intent, the change impacts your previous training. If you do choose to use @Entity references, you must replace all previous direct references with @Entity references.

Defining one example intent with an @Entity that has 10 values that are defined for it does not equate to specifying that example intent 10 times. The Watson Assistant service does not give that much weight to that one example intent syntax.







 Testing your intents 

After you have finished creating new intents, you can test the system to see if it recognizes your intents as you expect.



1. Click Try it.

![Ask Watson](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/ask_watson.png)
2. In the "Try it out" pane, enter a question or other text string and press Enter to see which intent is recognized. If the wrong intent is recognized, you can improve your model by adding this text as an example to the correct intent.

If you have recently made changes in your skill, you might see a message that indicates that the system is still retraining. If you see this message, wait until training completes before testing:

!
If you have [selected a data source](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-logslogs-deploy-id) other than the skill, you might not see an intent or entity from your data source logs as an option in the filters, unless those intents and entities are also in the skill.
* Refresh data: Select Refresh data to refresh the data that is used in the page metrics.

The statistics show traffic from customers who interact with your assistant; they do not include interactions from the Try it out pane.







 Scorecards 

The scorecards give you a quick view of your metrics. Scroll to see full interactive graphs later in the page.

![Shows the scorecards that are displayed at the start of the Analytics page](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/scorecard.png)



* Total conversations: The total number of conversations between active users and your assistant that occur during the selected time period.

A single conversation consists of messages that an active user sends to your assistant, and the messages your assistant sends to the user to initiate the conversation or respond.

If your assistant starts by saying Hi, how can I help you?, and then the user closes the browser without responding, that message is included in the total conversation count.

The total conversations metric is not used for billing purposes. An exchange with a user is not considered a billable conversation until the user submits a message.
* Avg. msg. per conversation: The total messages received during the selected time period divided by the total conversations during the selected time period, as shown in the corresponding graph.
* Max. conversations: The maximum number of conversations for a single data point within the selected time period.
* Weak understanding: The number of individual messages with weak understanding. These messages are not classified by an intent, and do not contain any known entities. Reviewing unrecognized messages can help you to identify potential dialog problems.







 Graphs and statistics 

Detailed graphs provide additional information.
To delete any query data that is associated with a specific customer, you must send a separate delete request directly to the Discovery service instance that is linked your the assistant. See the Discovery [information security](https://cloud.ibm.com/docs/discovery/information-security?topic=discovery-information-security) topic for details.





 Querying user data 

Use the v1 /logs method filter parameter to search an application log for specific user data. For example, to search for data specific to a customer_id that matches my_best_customer, the query might be:

curl -X GET -u "apikey:3Df... ...Y7Pc9" "{url}/v2/assistants/{assistant_id}/logs?version=2020-04-01&filter=customer_id::my_best_customer"

where {url} is the appropriate URL for your instance. For more details, see [Service endpoint](https://cloud.ibm.com/apidocs/assistant/assistant-v2service-endpoint).

See the [Filter query reference](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-filter-reference) for additional details.





 Deleting data 

To delete any message log data associated with a specific user that your assistant might have stored, use the DELETE /user_data v1 API method. Specify the customer ID of the user by passing a customer_id parameter with the request.

Only data that was added by using the POST /message API endpoint with an associated customer ID can be deleted using this delete method. Data that was added by other methods cannot be deleted based on customer ID. For example, entities and intents that were added from customer conversations, cannot be deleted in this way. Personal Data is not supported for those methods.

IMPORTANT: Specifying a customer_id will delete all messages with that customer_id that were received before the delete request, across your entire Watson Assistant instance, not just within one skill.
When the node is processed, your assistant retrieves a passage from an external data source and returns it as the response to a particular question. This type of search occurs only when the individual dialog node is processed. For more information, see [Adding a search skill response type](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-overviewdialog-overview-add-search-skill).
* Actions skill: In the And then field of the step where you want the search to be triggered, choose Search for the answer.



* From the dialog skill's Anything else node: If the assistant has a dialog skill and a search skill, any user input triggers the dialog skill first. The dialog addresses user input that it has a high confidence it can answer correctly. Any queries that would normally trigger the anything_else node in the dialog tree are sent to the search skill instead.

For example, instead of showing a standard message, such as I don't know how to help you with that. the assistant can say, Maybe this information can help:. The assistant passes the user input as the query to your search skill, and returns the search results as the response.

You can prevent the search from being triggered from the anything_else node by following the steps in [Disabling search](https://cloud.ibm.com/docs/assistant?topic=assistant-skill-search-addsearch-skill-add-disable).
* When only a search skill is used: If only a search skill is linked to an assistant, and no conversational skill is configured, then a search query is sent to the Discovery service when any user input is received from one of the assistant's integration channels.







 Test the search skill 

After you configure the search, you can send test queries to see the search results that get returned from Discovery by using the Preview pane of the search skill.

To test the full experience that customers will have when they ask questions that are either answered by the dialog or trigger a search, use the Preview button for your assistant.

You cannot test the full end-to-end user experience from the dialog "Try it out" pane.
* [Create a custom acoustic model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-acousticcreateModel-acoustic)
* [API & SDK reference](https://cloud.ibm.com/apidocs/speech-to-text)







 20 October 2022 

Updates to English next-generation telephony models
: The English next-generation telephony models have been updated for improved speech recognition:



* en-AU_Telephony
* en-GB_Telephony
* en-IN_Telephony
* en-US_Telephony



All of these models continue to support low latency. You do not need to upgrade custom models that are based on these models. For more information about all available next-generation models, see [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng).

Defect fix: Update Japanese next-generation multimedia model (upgrade required)
: Defect fix: The Japanese next-generation multimedia model, ja-JP_Multimedia, was updated to address an internal inconsistency that could cause an error during speech recognition with low latency. You need to upgrade any custom models that are based on the ja-JP_Multimedia model. For more information about upgrading custom models, see



* [Upgrading custom models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-upgrade)
* [Using upgraded custom models for speech recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-upgrade-use)







 7 October 2022 

New Swedish next-generation telephony model
: The service now offers a next-generation telephony model for Swedish: sv-SE_Telephony. The new model supports low latency and is generally available. It also supports language model customization and grammars. For more information about next-generation models and low latency, see
* [Working with corpora and custom words for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWords-ng)



Additional topics describe managing custom language models, corpora, and custom words. These operations are the same for custom models based on previous- and next-generation models.





 16 August 2021 

New beta Indian English, Indian Hindi, Japanese, and Korean next-generation models
: The service now supports the following new next-generation language models. All of the new models are beta functionality.



* Indian English: en-IN_Telephony. The model supports low latency.
* Indian Hindi: hi-IN_Telephony. The model supports low latency.
* Japanese: ja-JP_Multimedia. The model does not support low latency.
* Korean: ko-KR_Multimedia and ko-KR_Telephony. The models do not support low latency.



For more information about the next-generation models and low latency, see [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng) and [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency).





 16 July 2021 

New beta French next-generation model
: The French next-generation language model fr-FR_Multimedia is now available. The new model does not support low latency. The model is beta functionality.

Updates to beta US English next-generation model for improved speech recognition
: The next-generation US English en-US_Telephony model has been updated for improved speech recognition. The updated model continues to be beta functionality.

Defect fix: Update documentation for hesitation markers
: Defect fix: The documentation failed to state that next-generation models do not produce hesitation markers. The documentation has been updated to note that only previous-generation models produce hesitation markers. Next-generation models include the actual hesitations in transcription results.
* The source document contains a split verb that you want to annotate. How do you annotate noncontiguous text as a single entity type? You can annotate each entity mention, and identify them as being related to each other by using a relation mention.

[EVENT_ANSWER] [EVENT_ANSWER]

All of the phones were ringing, but he knew he should [pick] the red phone [up] first.

^----splitType-----^



* Avoid overlapping mentions, which are two different entity type labels that are applied to a single phrase in a document. For example, given the sentence, She donated her father's journals to the JFK Library., you would overlap mentions if you annotate JFK=PERSON and JFK Library=LOCATION for the single phrase JFK Library. The use of the term is more about the library than the person in this sentence, so only the latter annotation should be applied.

Decoding such structures requires multiple parallel invocations of a machine learning model because mention detection only looks for a single label or no label on each word token.
* Determine how the team will handle lists and plurals in running text. For example, the KLUE type system has PERSON and PEOPLE entity types that distinguish the singular from the plural. You can choose to annotate the list Barack, Michelle, Malia, and Sasha Obama, in one of the following ways:



* Annotate each item in the list as a singular entity mention (Barack, Michelle, Malia, and Sasha Obama are each a PERSON mention)
* Annotate the whole phrase as one plural entity mention (Barack, Michelle, Malia, and Sasha Obama is a single PEOPLE mention).



No one approach is necessarily better than the other. Just be sure that your team chooses one of them and applies it consistently to any lists that occur in the documents.
* A coreference is used when mentions refer to the same real-world entity. Relations are used between distinct entities. So, no two mentions should be connected by both coreference and a relation.







 Annotation with the ground truth editor 

When a human annotator annotates a document, the document is opened in the ground truth editor.
To deploy a Conversational Search project, connect this project to an assistant that is built with Watson Assistant. The general steps to follow include:



1. Create an assistant.

You can use a Watson Assistant Trial plan for testing purposes.
2. Add a search skill to your assistant, and then connect it to this project.
3. Deploy your assistant.

For more information about building a Watson Assistant search skill, see the appropriate documentation for your deployment:



IBM Cloud

From the new experience, see [Adding a search integration](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-search-add).
IBM Cloud

From the classic experience, see [Embedding existing help content](https://cloud.ibm.com/docs/assistant?topic=assistant-skill-search-add).
IBM Cloud Pak for Data

[Creating a search skill](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-skill-search-add).