vania-janet/multiturn-rag-retrieval-data huggingface dataset:
This change is being phased into different locations from August to September 2022. For information about the schedule for the limit reduction and what it means for existing custom acoustic models that contain more than 50 hours of audio, see [Maximum hours of audio](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-audioResourcesaudioMaximum).





 3 August 2022 

Defect fix: Update speech hesitations and hesitation markers documentation
: Defect fix: Documentation for speech hesitations and hesitation markers has been updated. Previous-generation models include hesitation markers in place of speech hesitations in transcription results for most languages; smart formatting removes hesitation markers from US English final transcripts. Next-generation models include the actual speech hesitations in transcription results; smart formatting has no effect on their inclusion in final transcription results.

For more information, see:



* [Speech hesitations and hesitation markers](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-basic-responseresponse-hesitation)
* [What results does smart formatting affect?](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-formattingsmart-formatting-effects)







 1 June 2022 

Updates to multiple next-generation telephony models
: The following next-generation telephony models have been updated for improved speech recognition:



* en-AU_Telephony
* en-GB_Telephony
* en-IN_Telephony
* en-US_Telephony
* ko-KR_Telephony



You do not need to upgrade custom models that are based on these models. For more information about all available next-generation models, see [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng).





 25 May 2022 

New beta character_insertion_bias parameter for next-generation models
By default, the service previously stopped transcription at the first half-second of non-speech (typically silence) if the parameter was omitted or set to false.

Existing applications that set the parameter to true will see no change in behavior. Applications that set the parameter to false or that relied on the default behavior will likely see a change. If a request specifies the parameter, the service now responds by returning a warning message for the unknown parameter:

"warnings": [
"Unknown arguments: continuous."
]

The request succeeds despite the warning, and an existing session or WebSocket connection is not affected.

IBM removed the parameter to respond to overwhelming feedback from the developer community that specifying continuous=false added little value and could reduce overall transcription accuracy.

Sending audio required to avoid session timeout
: It is no longer possible to avoid a session timeout without sending audio:



* When you use the WebSocket interface, the client can no longer keep a connection alive by sending a JSON text message with the action parameter set to no-op. Sending a no-op message does not generate an error, but it has no effect.
* When you use sessions with the HTTP interface, the client can no longer extend the session by sending a GET /v1/sessions/{session_id}/recognize request. The method still returns the status of an active session, but it does not keep the session active.



You can now do the following to keep a session alive:



* Set the inactivity_timeout parameter to -1 to avoid the 30-second inactivity timeout.
* Send any audio data, including just silence, to the service to avoid the 30-second session timeout. You are charged for the duration of any data that you send to the service, including the silence that you send to extend a session.



For more information, see [Timeouts](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-inputtimeouts). Ideally, you would establish a session just before you obtain audio for transcription and maintain the session by sending audio at a rate that is close to real time. Also, make sure your application recovers gracefully from closed sessions or connections.
Defect fix: The next-generation Brazilian Portuguese multimedia model is now available
: Defect fix: The next-generation Brazilian Portuguese multimedia model is now available for Speech to Text for IBM Cloud Pak for Data. Previously, the model was unavailable.

Adding words directly to custom models that are based on next-generation models increases the training time
: Adding custom words directly to a custom model that is based on a next-generation model causes training of a model to take a few minutes longer than it otherwise would. If you are training a model with custom words that you added by using the POST /v1/customizations/{customization_id}/words or PUT /v1/customizations/{customization_id}/words/{word_name} method, allow for some minutes of extra training time for the model. For more information, see



* [Add words to the custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreateaddWords)
* [Monitoring the train model request](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreatemonitorTraining-language)



Additional information about working with service instances
: The documentation now includes information about creating a service instance with the command-line interface (cpl-cli) and about managing service instances. For more information, see the following topics of [Watson Speech services on IBM Cloud Pak for Data](https://www.ibm.com/docs/en/cloud-paks/cp-data/4.6.x?topic=services-watson-speech):



* Creating a Watson Speech services instance under Post-installation setup
* Managing your Watson Speech services instances under Administering



Security vulnerability addressed
: The following security vulnerability has been fixed:



* [Security Bulletin: IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data is vulnerable to denial of service in Pypa Setuptools (CVE-2022-40897)](https://www.ibm.com/support/pages/node/6958142)







 30 January 2023 (Version 4.6.2)
For more information, see [Tune by Example and word pronunciation](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-tbe-introtbe-intro-how-pronunciation).
* You cannot control the expressiveness (for example, happiness or sadness) of a phrase. These qualities are different from intonation and cannot be adequately captured by the feature.
* You cannot control the loudness of a phrase. The service ignores the loudness of the spoken prompt.









 Rules for creating speaker models 

The following rules apply to the name and enrollment audio that you provide when creating a speaker model.



 Rules for speaker names 

For the name of a speaker model:



* Include a maximum of 49 characters.
* Include only alphanumeric characters and _ (underscores).
* Do not include XML sensitive characters (double quotes, single quotes, ampersands, angle brackets, and slashes).
* Do not use the name of an existing speaker model that is already defined for the service instance. The name of a speaker model must be unique for its service instance. To re-create a speaker model for an existing speaker name, you must first delete the existing model that has that name.







 Rules for speaker audio 

For the enrollment audio of a speaker model:



* The audio must be in WAV format and must have a minimum sampling rate of 16 kHz. The service accepts audio with higher sampling rates, which it transcodes to 16 kHz before processing it.
* The length of the enrollment audio is limited to 1 minute. Speaking one or two paragraphs of text that include five to ten sentences is recommended.
* Say the enrollment audio as you would normally speak. This allows the service to determine your normal speaking voice and apply that information to prompts that are associated with the speaker model.
"container": {
"duration": 556,
"name": "audio2",
"details": {
"type": "archive",
"compression": "zip"
},
"status": "ok"
},
. . .
}



Use a loop to check the status of the audio resource every few seconds until it becomes ok. For more information about other fields that are returned by the method, see [Listing audio resources for a custom acoustic model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-manageAudiolistAudio).







 Train the custom acoustic model 

Once you populate a custom acoustic model with audio resources, you must train the model on the new data. Training prepares a custom model for use in speech recognition. A model cannot be used for recognition requests until you train it on the new data. Also, updates to a custom model in the form of new or changed audio resources are not reflected by the model until you train it with the changes.

You use the POST /v1/acoustic_customizations/{customization_id}/train method to train a custom model. You pass the method the customization ID of the model that you want to train, as in the following example.

IBM Cloud

curl -X POST -u "apikey:{apikey}" "{url}/v1/acoustic_customizations/{customization_id}/train"

IBM Cloud Pak for Data

curl -X POST --header "Authorization: Bearer {token}" "{url}/v1/acoustic_customizations/{customization_id}/train"

The method is asynchronous. Training time depends on the cumulative amount of audio data that the custom acoustic model contains and the current load on the service. When you train or retrain a model, the service uses all of the model's audio data (not just new data) in the training. So training time is commensurate with the total amount of audio that the model contains.
: The new model, which is being offered as a beta feature in English-language dialog and actions skills, is faster and more accurate. It combines traditional machine learning, transfer learning, and deep learning techniques in a cohesive model that is highly responsive at run time. For more information, see [Improved intent recognition](https://cloud.ibm.com/docs/assistant?topic=assistant-intent-detection).





 3 November 2020 

Suggestions are now generally available
: The Suggestions feature that is available for the web chat integration is generally available and is enabled by default when you create a new web chat integration. For more information, see [Showing more suggestions](https://cloud.ibm.com/docs/assistant?topic=assistant-deploy-web-chatdeploy-web-chat-alternate).

New languages supported by the dialog analysis notebook
: The Dialog skill analysis notebook was updated with language support for French, German, Spanish, Czech, Italian, and Portuguese. For more information, see [Analysis notebooks](https://cloud.ibm.com/docs/assistant?topic=assistant-logs-resourceslogs-resources-jupyter-logs).

Visit the learning center!
: Click the Learning center link that is displayed in the header of the skill pages to find helpful product tours. The tours guide you through the steps to follow to complete a range of tasks, from adding your first intent to a dialog skill to enhancing the conversation in an actions skill. The Additional resources page has links to relevant documentation topics and how-to videos. You can search the resource link titles to find what you're looking for quickly.





 29 October 2020 

System entity support changes
: For English, Brazilian Portuguese, Czech, Dutch, French, German, Italian, and Spanish dialog skills only the new system entities API version is supported. For backward compatibility, both the interpretation and metadata attributes are included with the recognized entity object. The new system entity version is enabled automatically for dialog skills in the Arabic, Chinese, Korean, and Japanese languages. You can choose to use the legacy version of the system entities API by switching to it from the Options>System Entities page.
For more information, see [Installing Watson Speech to Text](https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=text-installing-watson-speech) and [Using the default model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-usemodels-use-default).



Updates to English and French next-generation multimedia models to support low latency
: The following multimedia models have been updated to support low latency:



* Australian English: en-AU_Multimedia
* UK English: en-GB_Multimedia
* US English: en-US_Multimedia
* French: fr-FR_Multimedia



You do not need to upgrade custom language models that are built on these base models. For more information about the next-generation models and low latency, see



* [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency)



New Castilian Spanish next-generation multimedia model
: The service now offers a next-generation multimedia model for Castilian Spanish: es-ES_Multimedia. The new model supports low latency and is generally available. It also supports language model customization and grammars.

For more information about all available next-generation models and their customization support, see



* [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* [Customization support for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-supportcustom-language-support-ng)



Beta next-generation en-WW_Medical_Telephony model now supports smart formatting
default_retention_rule = {'DefaultRetention': {'Mode': 'COMPLIANCE', 'Years': 1}}
object_lock_config = {'ObjectLockEnabled': 'Enabled', 'Rule': default_retention_rule}
cos_cli.put_object_lock_configuration(Bucket=bucket_name, ObjectLockConfiguration=object_lock_config)
 Reading the objectlock configuration set on the bucket.
response = cos_cli.get_object_lock_configuration(Bucket=bucket_name)
print("Objectlock Configuration for {0} =>".format(bucket_name))
print(response.ObjectLockConfiguration)

def upload_object(bucket_name,object_name,object_content):
cos_cli.put_object(
Bucket=bucket_name,
Key=object_name,
Body=object_content
)
print("Object: {0} uploaded!".format(object_name))

def objectlock_retention(bucket_name,object_name):
 Put objectlock retenion on the object uploaded to the bucket.
date = datetime.now()+timedelta(seconds=5)
retention_rule = {'Mode': 'COMPLIANCE', 'RetainUntilDate': date}
cos_cli.put_object_retention(Bucket=bucket_name, Key=object_name, Retention=retention_rule)

 Get objectlock retention of the above object.
response = cos_cli.get_object_retention(Bucket=bucket_name, Key=object_name)
print("Objectlock Retention for {0}=>".format(object_name))
print(response.Retention)

def objectlock_legalhold(bucket_name,object_name):
 Setting the objectlock legalhold status to ON.
cos_cli.put_object_legal_hold(Bucket=bucket_name, Key=object_name, LegalHold={'Status': 'ON'})
* For more information about the parameters that you can use with next-generation models, including their language support and whether the parameters are GA or beta, see the [Parameter summary](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-summary).
* For more information about next-generation models' support for customization, see [Customization support for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-supportcustom-language-support-ng).



Next-generation models support all speech recognition parameters and headers except for the following:



* acoustic_customization_id (Next-generation models do not support acoustic model customization.)
* keywords and keywords_threshold
* processing_metrics and processing_metrics_interval
* word_alternatives_threshold



Next-generation models also support the following parameters, which are not available with previous-generation models:



* character_insertion_bias, which is supported by all next-generation models. For more information, see [Character insertion bias](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-parsinginsertion-bias).
* low_latency, which is supported by most next-generation models. For more information, see [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency).



Next-generation models also differ from previous-generation models with respect to the following additional features:



* Next-generation models do not produce hesitation markers. They instead include the actual hesitations in transcription results. For more information, see [Speech hesitations and hesitation markers](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-basic-responseresponse-hesitation).
* Next-generation models support automatic capitalization only for German models. Previous-generation models support automatic customization only for US English models.
Processing and audio metrics 

The IBM Watson® Speech to Text service can return two types of optional metrics for a speech recognition request:



* [Processing metrics](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metricsprocessing-metrics) provide periodic information about the service's processing of the input audio. Use the metrics to gauge the service's progress in transcribing the audio.
* [Audio metrics](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metricsaudio-metrics) provide information about the signal characteristics of the input audio. Use the metrics to determine the characteristics and quality of the audio.



By default, the service returns no processing or audio metrics for a request.



 Processing metrics 

The processing_metrics and processing_metrics_interval parameters are supported only with previous-generation models, not with next-generation models. Also, processing metrics are available only with the WebSocket and asynchronous HTTP interfaces. They are not available with the synchronous HTTP interface.

Processing metrics provide detailed timing information about the service's analysis of the input audio. The service returns the metrics at specified intervals and with transcription events, such as interim and final results.

The metrics include statistics about how much audio the service has received, how much audio the service has transferred to the speech recognition engine, and how long the service has been processing the audio. If you request speaker labels, the information also shows how much audio the service has processed to determine speaker labels.

Processing metrics can help you gauge the progress of a recognition request. They can also help you distinguish the absence of results due to



* Lack of audio.
* Lack of speech in submitted audio.
* Engine stalls at the server and network stalls between the client and the server. To differentiate between engine and network stalls, results are periodic rather than event-based.



The metrics can also help you estimate response jitter by examining the periodic arrival times. Metrics are generated at a constant interval, so any difference in arrival times is caused by jitter.
To help you identify areas that require investigation, scores that fall below the value that you specified for the inter-annotator agreement threshold are highlighted in red. In early stages of your annotation project, you might find that relation scores are often worse than mention scores because a perfect relation annotation requires that the mentions that define the relationship be in agreement first.

The score in the All column is a Fleiss Kappa score. It represents how consistently the same annotation was applied by multiple human annotators across all overlapping documents in the task. The value, which can range up to 1 and even be negative, can help you identify weaknesses in the annotation guidelines or particular human annotators. The following guidelines (Landis and Koch, 1977) provide a starting point for assessing overall performance.

Table 1. Inter-annotator guidelines



 Score Agreement level 

 < 0 Poor 
 .01 -.20 Slight 
 .21 -.40 Fair 
 .41 - .60 Moderate 
 .61 - .80 Substantial 
 .81 - 1.0 Perfect 



The score in the other columns is an F1 measure. It represents the level of annotation consistency between a pair of human annotators. The value can range from 0 to 1, where perfect agreement is indicated by the score 1. What constitutes an acceptable level of agreement depends on your domain data and type system. But to provide an example, here are the F1 thresholds that project managers expect to be met or exceeded in projects that are based on the KLUE type system:



* Mentions with entity types: 0.85
* Relations: 0.8
* Coreference chains: 0.9



Interpreting the scores depends on the complexity of your type system, the amount and complexity of the content annotated, how experienced the human annotators are, and other factors. For example, if the annotation task is focused on labeling parts of speech, you might expect to see a high score because of how well-defined the parts of speech are. But a task that requires deeper analysis of the text, where human interpretation is required, might show lower scores until you take steps to clarify the causes of the ambiguity.
To help you identify areas that require investigation, scores that fall below the value that you specified for the inter-annotator agreement threshold are highlighted in red. In early stages of your annotation project, you might find that relation scores are often worse than mention scores because a perfect relation annotation requires that the mentions that define the relationship be in agreement first.

The score in the All column is a Fleiss Kappa score. It represents how consistently the same annotation was applied by multiple human annotators across all overlapping documents in the task. The value, which can range up to 1 and even be negative, can help you identify weaknesses in the annotation guidelines or particular human annotators. The following guidelines (Landis and Koch, 1977) provide a starting point for assessing overall performance.

Table 1: Inter-annotator Guidelines



 Score Agreement level 

 <0 Poor 
 .01 - .20 Slight 
 .21 - .40 Fair 
 .41 - .60 Moderate 
 .61 - .80 Substantial 
 .81 - 1.0 Perfect 



The score in the other columns is an F1 measure. It represents the level of annotation consistency between a pair of human annotators. The value can range from 0 to 1, where perfect agreement is indicated by the score 1. What constitutes an acceptable level of agreement depends on your domain data and type system. But to provide an example, here are the F1 thresholds that project managers expect to be met or exceeded in projects that are based on the KLUE type system:



* Mentions with entity types: 0.85
* Relations: 0.8
* Coreference chains: 0.9



Interpreting the scores depends on the complexity of your type system, the amount and complexity of the content annotated, how experienced the human annotators are, and other factors. For example, if the annotation task is focused on labeling parts of speech, you might expect to see a high score because of how well-defined the parts of speech are. But a task that requires deeper analysis of the text, where human interpretation is required, might show lower scores until you take steps to clarify the causes of the ambiguity.