vania-janet/multiturn-rag-retrieval-data huggingface dataset:
For more information, see [Creating a custom acoustic model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-acoustic).
* You can also use custom acoustic and custom language models together. If transcriptions or related corpora are available for your audio, you can use that data to create a complementary custom language model to further improve the quality of speech recognition based on your custom acoustic model. Grammars are supported only for previous-generation models. For more information, see [Using custom acoustic and custom language models together](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-useBoth).







 How many words can I add to a custom language model? 

You can add a maximum of 90 thousand out-of-vocabulary (OOV) words to a custom language model from all sources. You can add a maximum of 10 million total words to a custom language model from all sources. But many factors contribute to the amount of data that you need for an effective custom language model. Although it is not possible to provide the exact number of words that you need to add for any custom model or application, even adding a few words to a custom model can improve speech recognition. For more information about limits on the number of words that you can add and for other factors that affect the amount of data that you need, see [How much data do I need?](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWordswordsResourceAmount).





 How does custom model upgrading work? 

When a new version of a previous-generation base model is released to improve the quality of speech recognition, you must upgrade any custom language and custom acoustic models that are based on that model to take advantage of the updates. When you upgrade a custom model, you do not need to upgrade its resources individually. The service upgrades the resources automatically. Custom model upgrading applies only to previous-generation models.
* When you have a custom language model that is based on corpora or words that are relevant to the contents of your audio files.

If you don't have transcripts of your audio files, you can train with a custom language model that includes OOV words from the same domain as your audio data. Training with a custom language model that includes OOV words that are used in the audio can improve speech recognition.



For example, suppose you are creating a custom acoustic model that is based on call-center audio for specific products. Optimally, you can train the custom acoustic model with a custom language model that contains transcripts of related calls. If transcripts are unavailable, you can still train with a custom language model that includes just names of specific products that are handled by the call center.



 Guidelines for using lightly supervised training 

Follow these guidelines when you have transcripts of the audio files from the custom acoustic model:



* Create a dedicated custom language model whose sole purpose is to help train this specific custom acoustic model. The dedicated helper model can contain multiple transcripts for multiple audio files of the same custom acoustic model. It can also include custom words that are relevant to the audio files, though transcripts are more effective in training the custom acoustic model.
* Use the dedicated custom language model solely for training the custom acoustic model. Once the custom acoustic model is trained, the custom language model needs to be used only if you update the audio files in the acoustic model.
* Add transcripts of your audio files to the custom language model as corpora. It is not strictly necessary for a corpus to include only one sentence per line. But as with all corpora, the service makes noticeably better use of a corpus that includes each sentence on its own line. In general, shorter utterances that occupy separate lines of the corpus are most effective.
* It is neither necessary nor possible to associate a specific corpora with a specific audio file. A custom language model can contain multiple corpora, just as a custom acoustic model can include multiple audio files. All contents of a custom language model help improve the internal transcript of the audio that the service generates during the training process.
* A transcript does not need to be a verbatim reflection of all sentences and words from the audio.
For billing purposes, whitespace characters are not counted. For more information, see [Data limits](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-service-featuresfeatures-data-limits).
* How does customization work?

The customization interface of the Text to Speech service creates a dictionary of words and their translations for a specific language. This dictionary is referred to as a custom model. For more information, see [Understanding customization](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-customIntro).
* How do I create a custom model?

Review the guidelines for working with the customization interface before you begin. Then, see the steps and examples for creating, querying, updating, and deleting custom models in [Creating and managing custom models](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-customModels). Also review [Creating and managing custom entries](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-customWords) for examples and guidance about adding relevant training data.
* Can I create a custom voice?

IBM Cloud

As a premium customer, you can work with IBM to train a new custom voice for your specific use case and target market. Creating a custom voice is different from customizing one of the service's existing voices. A custom voice is a unique new voice that is based on audio training data that the customer provides. IBM can train a custom voice with as little as one hour of training data.

To request a custom voice or for more information, complete and submit this [IBM Request Form](https://form.asana.com/?k=CBuXK1uwlAf5ek6FwB6mcg&d=8612789739828).
* How do I use the Tune by Example feature?

Tune by Example lets you control exactly how specified text is spoken by the service.
They are not supported with the synchronous HTTP interface. For more information, see [Processing metrics](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metricsprocessing-metrics).





 17 May 2019 

New processing metrics and audio metrics features for speech recognition
: The service now offers two types of optional metrics with speech recognition requests:



* [Processing metrics](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metricsprocessing-metrics) provide detailed timing information about the service's analysis of the input audio. The service returns the metrics at specified intervals and with transcription events, such as interim and final results. Use the metrics to gauge the service's progress in transcribing the audio.
* [Audio metrics](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metricsaudio-metrics) provide detailed information about the signal characteristics of the input audio. The results provide aggregated metrics for the entire input audio at the conclusion of speech processing. Use the metrics to determine the characteristics and quality of the audio.



You can request both types of metrics with any speech recognition request. By default, the service returns no metrics for a request.

Updates to Japanese broadband model for improved speech recognition
: The Japanese broadband model (ja-JP_BroadbandModel) has been updated for improved speech recognition. By default, the service automatically uses the updated model for all speech recognition requests. If you have custom language or custom acoustic models that are based on the model, you must upgrade your existing custom models to take advantage of the updates by using the following methods:



* POST /v1/customizations/{customization_id}/upgrade_model
* POST /v1/acoustic_customizations/{customization_id}/upgrade_model



For more information, see [Upgrading custom models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-upgrade).
Tutorial: Understanding digressions 

In this tutorial, you will see firsthand how digressions work.



 Learning objectives 

By the time you finish the tutorial, you will understand how:



* digressions are designed to work
* digression settings impact the flow of the dialog
* to test digression settings for a dialog





 Duration 

This tutorial will take approximately 20 minutes to complete.





 Prerequisite 

If you have not provisioned an instance of the Watson Assistant service, complete the steps in [Provisioning an instance of the service](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-install-142install-142-install-service).







 Step 1: Import the Digressions showcase dialog skill 

First you will need to import the Digression showcase dialog skill into your Watson Assistant instance.



1. Download the [digression-showcase.json](https://github.com/watson-developer-cloud/community/raw/master/watson-assistant/digression-showcase.json) file.
2. In your Watson Assistant instance, click the ![Import](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/workspace_import.png) icon.
3. Click Choose a file, and then select the digression-showcase.json file that you downloaded earlier.
4. Click Import to finish importing the dialog skill.







 Step 2: Temporarily digressing away from dialog 

Digressions allow for the user to break away from a dialog branch in order to temporarily change the topic before returning to the original dialog flow. In this step, you will start to book a restaurant reservation, then digress away to ask for the restaurant's hours. After providing the opening hours information, your assistant will return back to the restaurant booking dialog flow.



1.
Objects written to a protected bucket cannot be deleted until the protection period has expired and all legal holds on the object are removed. The bucket's default retention value is given to an object unless an object specific value is provided when the object is created. Objects in protected buckets that are no longer under retention (retention period has expired and the object does not have any legal holds), when overwritten, will again come under retention. The new retention period can be provided as part of the object overwrite request or the default retention time of the bucket will be given to the object.

The minimum and maximum supported values for the retention period settings MinimumRetention, DefaultRetention, and MaximumRetention are a minimum of 0 days and a maximum of 365243 days (1000 years).

public static void addProtectionConfigurationToBucket(String bucketName) {
System.out.printf("Adding protection to bucket: %sn", bucketName);

BucketProtectionConfiguration newConfig = new BucketProtectionConfiguration()
.withStatus(BucketProtectionStatus.Retention)
.withMinimumRetentionInDays(10)
.withDefaultRetentionInDays(100)
.withMaximumRetentionInDays(1000);

cos.setBucketProtection(bucketName, newConfig);

System.out.printf("Protection added to bucket %sn", bucketName);
}

public static void addProtectionConfigurationToBucketWithRequest(String bucketName) {
System.out.printf("Adding protection to bucket: %sn", bucketName);

BucketProtectionConfiguration newConfig = new BucketProtectionConfiguration()
.withStatus(BucketProtectionStatus.Retention)
.withMinimumRetentionInDays(10)
.withDefaultRetentionInDays(100)
.withMaximumRetentionInDays(1000);
If the node that conditions on @sys-date was placed higher in the dialog tree than the nodes that condition on the Customer_Care_Cancel_Account and eCommerce_Cancel_Product_Order intents, disambiguation would never be triggered when a user enters, i must cancel it today. That's because your assistant would consider the date mention (today) to be more important than the intent references due to the placement of the corresponding nodes in the tree.

<-- <ul> -->

* The order of nodes impacts which nodes are included in the disambiguation options list

<-- </ul> -->

Sometimes a node is not listed as a disambiguation option as expected. This can happen if a condition value is also referenced by a node that is not eligible for inclusion in the disambiguation list for some reason. For example, an entity mention might trigger a node that is situated earlier in the dialog tree but is not enabled for disambiguation. If the same entity is the only condition for a node that is enabled for disambiguation, but is situated lower in the tree, then it might not be added as a disambiguation option because your assistant never reaches it. If it matched against the earlier node and was omitted, your assistant might not process the later node.
<-- </ul> --><-- </section "id="section-dialog-runtime-disambig-choose-nodes" "> --><-- <section "id="section-dialog-runtime-disambig-disable" "> --> Hiding nodes from disambiguation altogether You can prevent every node in a dialog or an individual dialog node from being included in the disambiguation list.<-- <ul> --> * To disable disambiguation entirely:
<-- <ul> -->

* From the Skills menu, click Options.
* On the Disambiguation page, set the switch to Off.

<-- </ul> -->

* To prevent a single dialog node from being included in the disambiguation list:
Speaker labels 

The speaker labels feature is beta functionality that is available for Czech, English (Australian, Indian, UK (narrowband/telephony), and US), German, Japanese, Korean, and Spanish only.

With speaker labels, the IBM Watson® Speech to Text service identifies which individuals spoke which words in a multi-participant exchange. You can use the feature to create a person-by-person transcript of an audio stream. For example, you can use it to develop analytics for a call-center or meeting transcript, or to animate an exchange with a conversational robot or avatar. For best performance, use audio that is at least a minute long. (Labeling who spoke what and when is sometimes referred to as speaker diarization.)

Speaker labels are optimized for two-speaker scenarios. They work best for telephone conversations that involve two people in an extended exchange. They can handle up to six speakers, but more than two speakers can result in variable performance. Two-person exchanges are typically conducted over narrowband (telephony) media, but you can use speaker labels with supported narrowband and broadband (multimedia) models.

To use the feature, you set the speaker_labels parameter to true for a recognition request; the parameter is false by default. The service identifies speakers by individual words of the audio. It relies on a word's start and end time to identify its speaker. (If you are recognizing multichannel audio, you can instead send each channel for transcription separately. For more information about this alternative approach, see [Speaker labels for multichannel audio](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-speaker-labelsspeaker-labels-multichannel).)

Setting the speaker_labels parameter to true forces the timestamps parameter to be true, regardless of whether you disable timestamps with the request. For more information, see [Word timestamps](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metadataword-timestamps).



 Speaker labels example 

An example is the best way to show how speaker labels work. The following example requests speaker labels with a speech recognition request:
Interim results and low latency 

With the WebSocket interface, the IBM Watson® Speech to Text service supports interim results, which are intermediate transcription hypotheses that arrive before final results. For the WebSocket and HTTP interfaces, most next-generation models also offer low latency to return results even more quickly than they already do, though transcription accuracy might be reduced.

When you use next-generation models that support low latency with the WebSocket interface, you must enable both interim results and low latency to get interim results. Only next-generation models that support low latency can return interim results. Note that previous- and next-generation models return slightly different results in certain cases.



 Interim results 

The interim results feature is available only with the WebSocket interface.

Interim results are intermediate transcription hypotheses that are likely to change before the service returns its final results. The service returns interim results as soon as it generates them. Interim results are useful for interactive applications and real-time transcription, and for long audio streams, which can take a while to transcribe.

Interim results evolve as the service's processing of an utterance progresses. They arrive more often and more quickly than final results. You can use them to enable your application to respond more quickly or to gauge the progress of transcription. Once its processing of an utterance is complete, the service sends final results that represent its best transcription of the audio for that utterance.



* Interim results are identified in a transcript with the field "final": false. The service can update interim results with more accurate transcriptions as it processes further audio. The service delivers one or more interim results for each final result.
* Final results are identified with the field "final": true. The service makes no further updates to final results.



How you request interim results depends on the type of model that you are using:



* For a previous-generation model, set the interim_results parameter to true in the JSON start message. Interim results are available for all previous-generation models.
* For a next-generation model, set the interim_results and low_latency parameters to true in the JSON start message.
* [Working with corpora and custom words for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWords-ng)



Additional topics describe managing custom language models, corpora, and custom words.





 29 July 2021 (Version 4.0.0) 

Version 4.0.0 is available
: IBM Watson® Speech to Text for IBM Cloud Pak® for Data version 4.0.0 is now available. Installation and administration of the service include many changes. This version supports IBM Cloud Pak for Data version 4.x and Red Hat OpenShift version 4.6. For more information about installing and managing the service, see [Installing IBM Watson Speech to Text for IBM Cloud Pak for Data](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-speech-install-data).

New next-generation language models
: The service now supports a growing number of next-generation language models. The next-generation multimedia and telephony models improve upon the speech recognition capabilities of the service's previous generation of broadband and narrowband models. The new models leverage deep neural networks and bidirectional analysis to achieve both higher throughput and greater transcription accuracy.

At this time, the next-generation language models and the low_latency parameter are beta functionality. The next-generation models support a limited number of languages and speech recognition features. The supported languages, models, and features will increase with future releases.

Many of the next-generation models also support a new low_latency parameter that lets you request faster results at the possible expense of reduced transcription quality. When low latency is enabled, the service curtails its analysis of the audio, which can reduce the accuracy of the transcription. This trade-off might be acceptable if your application requires lower response time more than it does the highest possible accuracy.

The low_latency parameter impacts your use of the interim_results parameter with the WebSocket interface. Interim results are available only for those next-generation models that support low latency, and only if both the interim_results and low_latency parameters are set to true.
* Each character counts as one character, regardless of whether the character is a single-byte character or a multibyte character.
* For metering, one Natural Language Understanding Item continues to be one feature (also known as an enrichment) per one text unit. One text unit is 10,000 characters or less.
* For detailed pricing information, see [Natural Language Understanding](https://cloud.ibm.com/catalog/natural-language-understanding) in the IBM Cloud® Catalog.



In addition to adding the limit_text_characters parameter, the following changes were made to text size limits and truncation:



* All text greater than 50,000 characters is truncated before processing. Previously, truncation was based on kilobytes, where one kilobyte equaled 1024 bytes.
* An informational message is returned in the response when text beyond 50,000 characters is truncated.
* Text limit size for custom models has increased from 10,000 characters to 50,000 characters.
* Usage information is added to the response for clarity around number of Natural Language Understanding Items used for each request. For example,



{
"usage": {
"text_units": 5,
"text_characters": 41186,
"features": 1
},
...
}





 8 May 2017 

Emotion tone score updates
: Updated the emotion tone score model. The training data set was expanded and feature engineering was altered and as a result, the model has higher precision on the IBM benchmark data set.





 March 2017 



 27 March 2017 

Model improvements
: The entity and sentiment models have been improved, which means that when you call those features, you will get better results.
: Relations now support Knowledge Studio custom models in French, German, Italian, and Portuguese.





 15 March 2017 

Emotion tone score updated
: We released updates to the emotion tone score model. The training data set was expanded and as a result, the model has higher precision on the IBM benchmark data set.







 27 February 2017 

Service GA
: The Natural Language Understanding service is now GA.
Language model customization is supported for both previous- and next-generation models. For more information, see [Creating a custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreate) and [Adding a grammar to a custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarAdd).
* If you are transcribing audio with unique characteristics (such as speaker accents, telephone conversations, or background noise), you can create a custom acoustic model to adapt a base model for your environment and speakers. Acoustic model customization is supported only for previous-generation models. For more information, see [Creating a custom acoustic model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-acoustic).
* You can also use custom acoustic and custom language models together. If transcriptions or related corpora are available for your audio, you can use that data to create a complementary custom language model to further improve the quality of speech recognition based on your custom acoustic model. Grammars are supported only for previous-generation models. For more information, see [Using custom acoustic and custom language models together](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-useBoth).



* How many words can I add to a custom language model?

You can add a maximum of 90 thousand out-of-vocabulary (OOV) words to a custom language model from all sources. You can add a maximum of 10 million total words to a custom language model from all sources. But many factors contribute to the amount of data that you need for an effective custom language model. Although it is not possible to provide the exact number of words that you need to add for any custom model or application, even adding a few words to a custom model can improve speech recognition. For more information about limits on the number of words that you can add and for other factors that affect the amount of data that you need, see [How much data do I need?]