vania-janet/multiturn-rag-retrieval-data huggingface dataset:
}
Show more

The response is just an example. The service can return one or more text messages with timing information for the input. It can also return a separate text message for each word of the input. Moreover, the messages can be interspersed with responses that contain binary chunks of audio. But the text message that contains the timing information for a word always arrives before the audio chunk that contains that word.



 Timings for plain text 

The service's synthesis process involves a text normalization step that spells out numbers, dates, times, monetary amounts, acronyms, and abbreviations. The results correspond to how such strings are spoken. For example, the string $200 is spoken as three words: two, hundred, and dollars. Because word timing information is used to synchronize the audio with the input text, the service returns timing information that corresponds to the non-normalized spelling of the input.

For example, consider the following input text:

The coldest recorded temperature is -89.2 degrees Celsius in Antarctica on July 21, 1983!

The service returns audio timings for the following strings:

The, coldest, recorded, temperature, is, -89.2, degrees, Celsius, in, Antarctica, on, July, 21,, 1983!

Although "-89.2" is spoken in the audio as five separate words (minus, eighty, nine, point, two), the text message provides timing information for the string as a single unit with the start time of minus and the end time of two.

As in the previous example, non-normalized strings can also contain punctuation. The service includes the punctuation that precedes or follows a word in the text message that it returns with the timings. For instance, the strings "21, and 1983!" include punctuation that the service returns in its text message. Although the punctuation results in silence, the audio timing for the word does not include that silence.

For example, consider input text that contains the following conditional statement:

If it is sunny, I will go to the beach.

The service returns timing information for all strings of the input, including "sunny, and beach., both of which end in punctuation that produces silence.
In general, shorter utterances that occupy separate lines of the corpus are most effective.
* It is neither necessary nor possible to associate a specific corpora with a specific audio file. A custom language model can contain multiple corpora, just as a custom acoustic model can include multiple audio files. All contents of a custom language model help improve the internal transcript of the audio that the service generates during the training process.
* A transcript does not need to be a verbatim reflection of all sentences and words from the audio. It can include only sentences and words of the audio that are relevant to the domain. For both language model and acoustic model customization, the training data needs to reflect the actual use-case of the speech that you want to recognize. If only 20 percent of the data (transcript or audio) is specific to the domain of the use-case, use just that 20 percent of the data for training.

However, if you are using acoustic model customization to achieve better results for accented speech (for example, for speech by non-native speakers), keep as much audio as possible, even if it's not relevant to the domain. The same is true if you are using acoustic model customization to improve speech recognition accuracy under difficult acoustic conditions, such as a noisy background.
* A verbatim transcript does not need to contain the disfluencies, verbal tics, and filler statements that are common to human speech. You can remove these elements from the transcript, the audio, or both. (You can also remove audio in which people are talking over each other, since that does not contribute to the training.)

For example, suppose a verbatim transcript includes the following sentence:

So that's, uhm, you know, that, as I say, is the is the predominant form today.

You can edit these peculiarities from the transcript, the audio, or both to create the following sentence:

So that as I say is the predominant form today.







 Performing lightly supervised training 

To train a custom acoustic model with a custom language model, you use the optional custom_language_model_id query parameter of the POST /v1/acoustic_customizations/{customization_id}/train method.
The service uses all of the words from a corpus to learn the context in which sequences of characters can occur, which is why corpora are a more effective means of improving recognition accuracy.

Adding a large number of corpora and words can increase the latency of speech recognition, but the exact effect is difficult to quantify or predict. As with the amount of data that is needed to produce an effective custom model, the performance impact of a large amount of data depends on many factors. Test your custom model with different amounts of data to determine the performance of your models.



 Guidelines for adding words to custom models based on improved next-generation models 

For custom models that are based on next-generation language models that use improved customization, limit the number of custom words that you add directly to the model with the following methods:



* POST /v1/customizations/{customization_id}/words
* PUT /v1/customizations/{customization_id}/words/{word_name}



Using these methods to add custom words to a custom model can significantly increase the training time of the model. Training time increases linearly with the total number of words that you add. However, the training time increases only when the model is first trained on the new custom words. The time required for subsequent training requests, without new custom words, returns to normal.

Training a custom model with words added only via corpora with the following method is typically quick:



* POST /v1/customizations/{customization_id}/corpora/{corpus_name}



For more information about improved customization for next-generation models, see [Improved language model customization for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-customizationcustomLanguage-intro-ng).





 Guidelines for adding words to Japanese models based on improved next-generation models 

Do not add custom words for known words, words that are commonly recognized and have a general correspondence between the word and how it is pronounced. Use custom words to create a mapping between how less common words are spelled and how they are pronounced.
There are 5 levels of sensitivity:



* high
* medium_high
* medium
* medium_low
* low



The default (auto) is medium_high if this option is not set.
* More predictable: The new disambiguation feature is more stable and predictable. The choices shown may sometimes vary slightly to enable learning and analytics, but the order and depth of disambiguation is largely stable.



These new features may affect various metrics, such as disambiguation rate and click rates, as well as influence conversation-level key performance indicators such as containment.

If the new disambiguation algorithm works differently than expected for your assistant, you can adjust it using the sensitivity parameter in the update workspace API. For more information, see [Update workspace](https://cloud.ibm.com/apidocs/assistant/assistant-v1updateworkspace).





 9 September 2021 

Actions skill improvements
: Actions skills now include these new features:



* Change conversation topic: In general, an action is designed to lead a customer through a particular process without any interruptions. In real life, however, conversations almost never follow such a simple flow. In the middle of a conversation, customers might get distracted, ask questions about related issues, misunderstand something, or just change their minds about what they want to do. The Change conversation topic feature enables your assistant to handle these digressions, dynamically responding to the user by changing the conversation topic as needed. For more information, see [Changing the topic of the conversation](https://cloud.ibm.com/docs/assistant?topic=assistant-actionsactions-change-topic).
* Fallback action: The built-in action, Fallback, provides a way to automatically connect customers to a human agent if they need more help. This action helps you to handle errors in the conversation, and is triggered by these conditions:



* Step validation failed: The customer repeatedly gave answers that were not valid for the expected customer response type.
* Agent requested: The customer directly asked to be connected to a human agent.
* No action matches: The customer repeatedly made requests or asked questions that the assistant did not understand.
Alter your table as described in [Updating retention time interval for tables](https://cloud.ibm.com/docs/netezza?topic=netezza-create-tablesupdating_retention_db).



When you set retention time interval to 0, you cannot run time travel queries and you do not have access to historical rows for that table anymore. You can reclaim some or all of the now inaccessible historical rows in the table with GROOM TABLE.





 Altering nontemporal tables to temporal with the web console 

To alter a nontemporal table to temporal, set retention time interval to a nonzero value.



1. Log in to the web console as described in [Getting started with the web console](https://cloud.ibm.com/docs/netezza?topic=netezza-getstarted-console).
2. Alter your table as described in [Updating retention time interval for tables](https://cloud.ibm.com/docs/netezza?topic=netezza-create-tablesupdating_retention_db).



If you first disabled your temporal table and then converted the same table to a temporal table, you do not have access to the prior historical rows for that table. Historical data is collected when rows are deleted or updated after the table is converted to temporal.

Similarly to when you create a table, a row that is inserted into the table receives a virtual insert timestamp that is equal to the commit time of the inserting transaction. A row that is deleted from the table receives a virtual delete timestamp that is equal to the commit time of the deleting (or truncating) transaction. The table’s retention lower bound and retention start time are equal to or just before the commit time of this altering transaction.

Unlike when you create a table, existing visible rows in the table are treated as if they were inserted by this altering transaction. The existing visible rows receive virtual insert timestamps that are equal to the retention start time. With these timestamps, the rows are potentially visible to time travel queries.





 Altering temporal schemas to nontemporal with the web console
After a conversation ends, use the v2 [Delete session](https://cloud.ibm.com/apidocs/assistant/assistant-v2deletesession) method to delete the session.



service .deleteSession({ assistant_id: assistantId, session_id: sessionId, })

 {: codeblock python}
service.delete_session(
assistant_id = assistant_id,
session_id = session_id
)

 {: codeblock java}

DeleteSessionOptions deleteSessionOptions = new DeleteSessionOptions.Builder(assistantId, sessionId .build(); service.deleteSession(deleteSessionOptions).execute();

If you do not explicitly delete the session, it will be automatically deleted after the configured timeout interval. (The timeout duration depends on your plan; for more information, see [Session limits](/docs/assistant?topic=assistant-assistant-settingsassistant-settings-session-limits).)

To see examples of the v2 APIs in the context of a simple client application, see [Building a client application](/docs/assistant?topic=assistant-api-client).

 Handle the v2 response format

Your application might need to be updated to handle the v2 runtime response format, depending on which parts of the response your application needs to access:

- Output for all response types (such as text and option) are still returned in the output.generic object. Application code for handling these responses should work without modification.

- Detected intents and entities are now returned as part of the output object, rather than at the root of the response JSON.

- The conversation context is now organized into two objects:

- The global context contains system-level context data shared by all skills used by the assistant.

- The skill context contains any user-defined context variables used by your dialog skill.

However, keep in mind that state data, including conversation context, is now maintained by the assistant, so your application might not need to access the context at all (see [Let the assistant maintain state](api-migration-state)).
* For more information about the next-generation models, see [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* For more information about migrating from previous-generation to next-generation models, see [Migrating to next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-migrate).



Note: When the previous-generation en-US_BroadbandModel is removed from service, the next-generation en-US_Multimedia model will become the default model for speech recognition requests.





 15 August 2022 

New French Canadian next-generation multimedia model
: The service now offers a next-generation multimedia model for French Canadian: fr-CA_Multimedia. The new model supports low latency and is generally available. It also supports language model customization and grammars. For more information about next-generation models and low latency, see



* [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* [Customization support for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-supportcustom-language-support-ng)
* [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency)



Updates to English next-generation telephony models
: The English next-generation telephony models have been updated for improved speech recognition:



* en-AU_Telephony
* en-GB_Telephony
* en-IN_Telephony
* en-US_Telephony



All of these models continue to support low latency. You do not need to upgrade custom models that are based on these models.
[Enter a topic name](https://cloud.ibm.com/docs-content/v1/content/bcd8b79ca2ed73bfcaefb20051f12cb6400286ab/atracker/images/es-topic-1.png)

Enter a topic name
6. Enter the number of partitions and click Next.

One or more partitions make up a topic. A partition is an ordered list of messages. Partitions are distributed across the brokers in order to increase the scalability of your topic. You can also use them to distribute messages across the members of a consumer group.

Zoom

![Enter the number of partitions](https://cloud.ibm.com/docs-content/v1/content/bcd8b79ca2ed73bfcaefb20051f12cb6400286ab/atracker/images/es-topic-2.png)

Enter the number of partitions
7. Select a Message retention and click Create Topic.

Message retention defines how long messages are retained before they are deleted. If your messages are not read by a consumer within this time, they will be missed.







 Step 2: Create credentials to authenticate with Event Streams 

You need the following information to connect Activity Tracker Event Routing to the Event Streams instance:



* Endpoint URLs to call the APIs
* Credentials for authentication



Complete the following steps to create service credentials that Activity Tracker Event Routing needs to communicate with the Event Streams instance:



1. In the IBM Cloud, click the Menu icon ![Menu icon](https://cloud.ibm.com/docs-content/v1/content/bcd8b79ca2ed73bfcaefb20051f12cb6400286ab/atracker/icons/icon_hamburger.svg) > Resource list.
2. Look for the Event Streams instance that you plan to use, and select it.
3. In the Event Streams console, click Service credentials.
4. Select New credential.
5.
When you initiate a streaming session, the service enforces inactivity and session timeouts from which your application must recover gracefully. If a timeout lapses during a streaming session, the service closes the connection.







 Interim results and low latency 



* [Interim results](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interiminterim-results) are intermediate hypotheses that the service returns as transcription progresses. They are available only with the WebSocket interface. The service returns final results when a transcript is complete. With the HTTP interfaces, the service always transcribes the entire audio stream before sending any results.
* [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency), when used with certain next-generation models, directs the service to produce final results even more quickly than the models usually do. Low latency is available with the WebSocket and HTTP interfaces. Although low latency further enhances the already improved response times of the models, it might reduce transcription accuracy. When you use the next-generation models with the WebSocket interface, low latency is required to obtain interim results.







 Speech activity detection 



* [Speech detector sensitivity](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-detectiondetection-parameters-sensitivity) adjusts the sensitivity of the service's detection of speech activity. Use the parameter to suppress word insertions from music, coughing, and other non-speech events that can adversely affect the quality of speech recognition.
* [Background audio suppression](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-detectiondetection-parameters-suppression) suppresses background audio based on its volume to prevent it from being transcribed as speech. Use the parameter to suppress side conversations or background noise from speech recognition.







 Speech audio parsing
While the service is processing new words, the status remains pending. Use a loop to check the status every 10 seconds until it becomes ready to indicate that the operation is complete. For more information about possible status values, see [Monitoring the train model request](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreatemonitorTraining-language).





 Modifying words in a custom model 

You can also use the POST /v1/customizations/{customization_id}/words and PUT /v1/customizations/{customization_id}/words/{word_name} methods to modify or augment a word in a custom model. You might need to use the methods to correct a typographical error or other mistake that was made when a word was added to the model. You might also need to add sounds-like definitions for an existing word.

You use the methods to modify the definition of an existing word exactly as you do to add a word. The new data that you provide for the word overwrites the word's existing definition. For custom models that are based on previous-generation models, you can also modify words that were added from corpora.







 Train the custom language model 

Once you populate a custom language model with new words (by adding corpora, by adding words directly, or by adding grammars), you must train the model on the new data. Training prepares the custom model to use the data in speech recognition. The model does not use words that you add via any means until you train it on the data.

You use the POST /v1/customizations/{customization_id}/train method to train a custom model. You pass the method the customization ID of the model that you want to train, as in the following example:

IBM Cloud

curl -X POST -u "apikey:{apikey}" "{url}/v1/customizations/{customization_id}/train"

IBM Cloud Pak for Data

curl -X POST --header "Authorization: Bearer {token}" "{url}/v1/customizations/{customization_id}/train"
The new retention period can be provided as part of the object overwrite request or the default retention time of the bucket will be given to the object.

def extend_retention_period_on_object(bucket_name, object_name, additional_seconds):
print("Extend the retention period on {0} in bucket {1} by {2} seconds.n".format(object_name, bucket_name, additional_seconds))

cos.extend_object_retention(
Bucket=bucket_ame,
Key=object_name,
AdditionalRetentionPeriod=additional_seconds
)

print("New retention period on {0} is {1}n".format(object_name, additional_seconds))





 List legal holds on a protected object 

This operation returns:



* Object creation date
* Object retention period in seconds
* Calculated retention expiration date based on the period and creation date
* List of legal holds
* Legal hold identifier
* Timestamp when legal hold was applied



If there are no legal holds on the object, an empty LegalHoldSet is returned. If there is no retention period that is specified on the object, a 404 error is returned.

def list_legal_holds_on_object(bucket_name, object_name):
print("List all legal holds on object {0} in bucket {1}n".format(object_name, bucket_name));

response = cos.list_legal_holds(
Bucket=bucket_name,
Key=object_name
)

print("Legal holds on bucket {0}: {1}n".format(bucket_name, response))





 Create a hosted static website 

This operation requires permissions, as only the bucket owner is typically permitted to configure a bucket to host a static website. The parameters determine the default suffix for visitors to the site as well as an optional error document.

def putBucketWebsiteConfiguration(bucket_name):
website_defaults = {
'ErrorDocument': {'Key': 'error.html'},
console.log(data);
}
.catch((e) => {
console.log(ERROR: ${e.code} - ${e.message}n);
});
}





 Upload an object to a bucket with retention policy 

This enhancement of the PUT operation adds three new request headers: two for specifying the retention period in different ways, and one for adding a single legal hold to the new object. New errors are defined for illegal values for the new headers, and if an object is under retention any overwrites will fail.

Objects in buckets with retention policy that are no longer under retention (retention period has expired and the object doesn't have any legal holds), when overwritten, will again come under retention. The new retention period can be provided as part of the object overwrite request or the default retention time of the bucket will be given to the object.

A Content-MD5 header is required.

These headers apply to POST object and multipart upload requests as well. If uploading an object in multiple parts, each part requires a Content-MD5 header.



 Value Type Description 

 Retention-Period Non-negative integer (seconds) Retention period to store on the object in seconds. The object can be neither overwritten nor deleted until the amount of time that is specified in the retention period has elapsed. If this field and Retention-Expiration-Date are specified a 400 error is returned. If neither is specified the bucket's DefaultRetention period will be used. Zero (0) is a legal value assuming the bucket's minimum retention period is also 0. 
 Retention-expiration-date Date (ISO 8601 Format) Date on which it is legal to delete or modify the object. You can only specify this or the Retention-Period header. If both are specified a 400 error will be returned. If neither is specified the bucket's DefaultRetention period will be used.