vania-janet/multiturn-rag-retrieval-data huggingface dataset:
: The service now offers a next-generation multimedia model for Netherlands Dutch: nl-NL_Multimedia. The new model supports low latency and is generally available. It also supports language model customization and grammars. For more information about next-generation models and low latency, see



* [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* [Customization support for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-supportcustom-language-support-ng)
* [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency)



New Swedish next-generation telephony model
: The service now offers a next-generation telephony model for Swedish: sv-SE_Telephony. The new model supports low latency and is generally available. It also supports language model customization and grammars. For more information about next-generation models and low latency, see



* [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* [Customization support for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-supportcustom-language-support-ng)
* [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency)



Updates to English next-generation telephony models
: The English next-generation telephony models have been updated for improved speech recognition:



* en-AU_Telephony
* en-GB_Telephony
* en-IN_Telephony
* en-US_Telephony



All of these models continue to support low latency.
Currently, the methods apply only to the new US English base models. But the same information will apply to upgrades of other base models as they become available.

Language model customization now available for UK English
: The service now supports language model customization for the UK English models, en-GB_BroadbandModel and en-GB_NarrowbandModel. Although the service handles UK and US English corpora and custom words in a generally similar fashion, some important differences exist:



* For more information about how the service parses corpora for UK English, see [Parsing of Dutch, English, French, German, Italian, Portuguese, and Spanish](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWordscorpusLanguages).
* For more information about creating sounds-like pronunciations for custom words in UK English, see [Guidelines for English](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWordswordLanguages-english). Specifically, for UK English, you cannot use periods or dashes in sounds-like pronunciations.







 2 October 2017 

New beta acoustic model customization interface for US English, Japanese, and Spanish
: The customization interface now offers acoustic model customization. You can create custom acoustic models that adapt the service's base models to your environment and speakers. You populate and train a custom acoustic model on audio that more closely matches the acoustic signature of the audio that you want to transcribe. You then use the custom acoustic model with recognition requests to increase the accuracy of speech recognition.

Custom acoustic models complement custom language models. You can train a custom acoustic model with a custom language model, and you can use both types of model during speech recognition. Acoustic model customization is a beta interface that is available only for US English, Japanese, and Spanish.



* For more information about the languages that are supported by the customization interface and the level of support that is available for each language, see [Language support for customization](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-support).
Updates to beta Castilian Spanish next-generation model for improved speech recognition
: The beta next-generation Castilian Spanish es-ES_Telephony model now supports the low_latency parameter. For more information, see [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency).

New beta support for speaker labels with next-generation models
: The speaker_labels parameter is now supported as beta functionality for use with the following next-generation models:



* Australian English en-AU_Telephony model
* UK English en-GB_Telephony model
* US English en-US_Multimedia and en-US_Telephony models
* German de-DE_Telephony model
* Castilian Spanish es-ES_Telephony model



With the next generation models, the speaker_labels parameter is not supported for use with the interim_results or low_latency parameters at this time. For more information, see [Speaker labels](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-speaker-labels).

New HTTP error code for use of word_confidence with next-generation models
: The word_confidence parameter is not supported for use with next-generation models. The service now returns the following 400 error code if you use the word_confidence parameter with a next-generation model for speech recognition:

{
"error": "word_confidence is not a supported feature for model {model}",
"code": 400,
"code_description": "Bad Request"
}





 12 April 2021 

New beta next-generation language models and low_latency parameter
: The service now supports a growing number of next-generation language models. The next-generation multimedia and telephony models improve upon the speech recognition capabilities of the service's previous generation of broadband and narrowband models. The new models leverage deep neural networks and bidirectional analysis to achieve both higher throughput and greater transcription accuracy. At this time, the next-generation models support only a limited number of languages and speech recognition features.
German (de) GA (except Covid-19) GA 
 Italian (it) GA (except Covid-19) GA 
 Japanese (ja) GA (except Covid-19) GA 
 Korean (ko) NA GA 
 Portuguese (Brazilian) (pt-br) GA GA 
 Spanish (es) GA GA 
 Universal (xx) NA NA 







 User input processing support details 



Table 4. User input processing support details

 Language [Dictionary-based entity support](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-create-dictionary-based) [Fuzzy matching (Misspelling)](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-fuzzy-matching) [Fuzzy matching (Stemming, Partial match)](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-fuzzy-matching) [Autocorrection](https://cloud.ibm.com/docs/assistant?topic=assistant-autocorrection) 

 English (en) GA GA GA GA 
 Arabic (ar) GA GA NA NA 
 Chinese (Simplified) (zh-cn) GA NA NA NA 
 Chinese (Traditional) (zh-tw) GA NA NA NA 
 Czech (cs) GA GA GA NA 
 Dutch (nl) GA GA NA NA 
 French (fr) GA GA GA Beta 
 German (de) GA GA GA NA 
 Italian (it) GA GA NA NA 
 Japanese (ja) GA GA NA NA 
 Korean (ko) GA GA NA NA 
 Portuguese (Brazilian) (pt-br) GA GA NA NA 
 Spanish (es) GA GA NA NA 
 Universal (xx) GA GA NA NA 







 Entity feature support details 



Table 5. Entity feature support details

 Language [Contextual entities](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-create-annotation-based) [Synonym recommendations](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-create-dictionary-based) 

 English (en) GA GA 
 Arabic (ar) NA NA
Map<String, Object> userDefinedContext = new HashMap<>();
userDefinedContext.put("account_number","123456");
MessageContextSkill mainSkillContext = new MessageContextSkill.Builder()
.userDefined(userDefinedContext)
.build();
Map<String, MessageContextSkill> skillsContext = new HashMap<>();
skillsContext.put("main skill", mainSkillContext);

MessageContext context = new MessageContext.Builder()
.global(globalContext)
.skills(skillsContext)
.build();

MessageOptions options = new MessageOptions.Builder()
.assistantId("{assistant_id}")
.sessionId("{session_id}")
.input(input)
.context(context)
.build();

MessageResponse response = service.message(options).execute().getResult();

System.out.println(response);

Show more

response=service.message(
assistant_id='{assistant_id}',
session_id='{session_id}',
input={
'message_type': 'text',
'text': 'Hello',
'options': {
'return_context': True
}
},
context={
'global': {
'system': {
'user_id': 'my_user_id'
}
},
'skills': {
'main skill': {
'user_defined': {
'account_number': '123456'
}
}
}
}
).get_result()

print(json.dumps(response, indent=2))

service
.message({
assistant_id: '{assistant_id}',
* For more information about creating sounds-like pronunciations for custom words in German, see [Guidelines for Dutch, French, German, Italian, Portuguese, and Spanish](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWordswordLanguages-esES-frFR).



Language model customization now available for Brazilian Portuguese
: The existing Brazilian Portuguese models, pt-BR_BroadbandModel and pt-BR_NarrowbandModel, now support language model customization (generally available). The models were not updated to enable this support, so no upgrade of existing custom acoustic models is required.



* For information about how the service parses corpora for Brazilian Portuguese, see [Parsing of Dutch, English, French, German, Italian, Portuguese, and Spanish](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWordscorpusLanguages).
* For more information about creating sounds-like pronunciations for custom words in Brazilian Portuguese, see [Guidelines for Dutch, French, German, Italian, Portuguese, and Spanish](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWordswordLanguages-esES-frFR).



Updates to US English and Japanese models for improved speech recognition
: New versions of the US English and Japanese broadband and narrowband models are available:



* US English broadband model (en-US_BroadbandModel)
* US English narrowband model (en-US_NarrowbandModel)
* Japanese broadband model (ja-JP_BroadbandModel)
* Japanese narrowband model (ja-JP_NarrowbandModel)



The new models offer improved speech recognition. By default, the service automatically uses the updated models for all recognition requests. If you have custom language or custom acoustic models that are based on these models, you must upgrade your existing custom models to take advantage of the updates by using the following methods:



* POST /v1/customizations/{customization_id}/upgrade_model
* POST /v1/acoustic_customizations/{customization_id}/upgrade_model
* Voice transformation SSML. This was an IBM extension to SSML that was supported only for the en-US_AllisonVoice, en-US_LisaVoice, and en-US_MichaelVoice voices.
* The volume attribute of the <prosody> element. This attribute was available for all standard voices.



Including these SSML elements with a synthesis request for a neural voice generates an HTTP 400 response code because the request fails SSML validation. For more information about SSML validation, see [SSML validation](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-ssmlssml-errors).

New support for Cross-Origin Resource Sharing
: Cross-Origin Resource Sharing (CORS) support is now available for all voices from the Google Chrome™ and Apple® Safari browsers. It is not available from the Mozilla Firefox™ browser for voices in the following languages: Arabic, Australian English, Chinese, Netherlands Dutch, and Korean. For more information, see [Leveraging CORS support](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-service-featuresfeatures-cors).





 10 September 2020 

Defect fix: Improve Japanese voice
: Defect fix: For the ja-JP_EmiV3Voice voice, the service now correctly parses SSML input text that includes a prosody rate specification. Previously, the following use of the <prosody> element worked properly:

<speak>成功する/繁栄する</speak>

But the following use of the rate attribute with the <prosody> element caused the service to read and speak the embedded SSML notation:

<speak>
<prosody rate="fast">成功する/繁栄する</prosody>
</speak>

The service now correctly parses and applies the rate attribute of the <prosody> element for Japanese input.





 4 September 2020 

Customization interface is now generally available
: The customization interface is now generally available (GA). Customization is no longer beta functionality.
es-MX_BroadbandModel <br>Deprecated Beta Beta Beta 
 Spanish <br>(Peruvian) es-PE_NarrowbandModel <br>Deprecated Beta Beta Beta 
 es-PE_BroadbandModel <br>Deprecated Beta Beta Beta 







 Customization support for next-generation models 

Table 2 lists the next-generation models that are supported for language model customization and grammars. Next-generation language models do not support acoustic model customization. To learn which models are supported for speech recognition for IBM Cloud®, IBM Cloud Pak® for Data, or both, see [Supported next-generation language models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ngmodels-ng-supported).

In the Language model customization column, an (improved) date indicates when a model was migrated to the new language model customization technology. Make sure to check for a date for the version of the service that you use, IBM Cloud or IBM Cloud Pak for Data. Models that do not include an (improved) date are not yet migrated to the new technology. For more information about the improved technology, see [Improved language model customization for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-customizationcustomLanguage-intro-ng).



Table 2. Next-generation language support for customization

 Language (dialect) Models Language model customization (improved) Grammars 

 Arabic <br>(Modern Standard) ar-MS_Telephony GA GA 
 Chinese <br>(Mandarin) zh-CN_Telephony GA GA 
 Czech cs-CZ_Telephony GA GA 
 Dutch <br>(Belgian) nl-BE_Telephony GA GA 
 Dutch <br>(Netherlands) nl-NL_Telephony GA GA 
 nl-NL_Multimedia GA GA 
 English <br>(Australian) en-AU_Telephony GA <br><br>IBM Cloud<br><br> 27 February 2023 <br><br>IBM Cloud Pak for Data<br><br> 1 May 2023 GA
* The voices do support all other SSML elements. However, they support only the International Phonetic Alphabet (IPA), not IBM Symbolic Phonetic Representation (SPR), with the <phoneme> element.



For more information about these and all available voices, see [Languages and voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices).





 12 December 2019 

Full support for IBM Cloud IAM
: The Text to Speech service now supports the full implementation of IBM Cloud Identity and Access Management (IAM). API keys for Watson services are no longer limited to a single service instance. You can create access policies and API keys that apply to more than one service, and you can grant access between services. For more information about IAM, see [Authenticating to Watson services](https://cloud.ibm.com/docs/watson?topic=watson-iam).

To support this change, the API service endpoints use a different domain and include the service instance ID. The pattern is api.{location}.text-to-speech.watson.cloud.ibm.com/instances/{instance_id}.



* Example HTTP URL for an instance hosted in the Dallas location:

https://api.us-south.text-to-speech.watson.cloud.ibm.com/instances/6bbda3b3-d572-45e1-8c54-22d6ed9e52c2
* Example WebSocket URL for an instance hosted in the Dallas location:

wss://api.us-south.text-to-speech.watson.cloud.ibm.com/instances/6bbda3b3-d572-45e1-8c54-22d6ed9e52c2



For more information about the URLs, see the [API & SDK reference](https://cloud.ibm.com/apidocs/text-to-speech/text-to-speechservice-endpoint).

These URLs do not constitute a breaking change. The new URLs work for both your existing service instances and for new instances.
[Croatian](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelscroatian) hr [Norwegian Bokmål](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsnorwegian-bokmal) nb 
 [Czech](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsczech) cs [Polish](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelspolish) pl 
 [Danish](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsdanish) da [Portuguese](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsportuguese) pt 
 [Dutch](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsdutch) nl [Punjabi (Indian)](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelspunjabi-india) pa 
 [English](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsenglish) en [Punjabi (Pakistani)](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelspunjabi-pakistan) pa-PK 
 [Estonian](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsestonian) et [Romanian](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsromanian) ro
* [Understanding phonetic symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-symbols)
* [Dutch (Netherlands) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-nlSymbols-new)
* [English (Australian) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-auSymbols-new)
* [English (United Kingdom) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-gbSymbols)
* [English (United States) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-usSymbols)
* [French (Canadian) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-caSymbols)
* [French (France) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-frSymbols)
* [German symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-deSymbols)
* [Italian symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-itSymbols)
* [Japanese symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-jaSymbols)
* [Korean symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-koSymbols-new)
* [Portuguese (Brazilian) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-ptSymbols)
Language Notation Language-specific rules 

 French and <br>Canadian French SPR All syllable stress symbols are honored. But syllable stress must immediately precede the vowel of the syllable. Syllable stress for French is much stricter than for other languages. An error occurs if you place the stress symbol in an invalid location. 
 French and <br>Canadian French IPA All syllable stress symbols are ignored. 
 Italian SPR and IPA You can specify only 1 (primary stress). An error occurs if you specify secondary or no stress. 
 Japanese SPR and IPA You can specify only 1 (primary stress) and 0 (no stress). An error occurs if you specify secondary stress. 
 Spanish SPR and IPA You can specify only 1 (primary stress). An error occurs if you specify secondary or no stress. 









 Phonetic symbols for supported languages 

Table 3 lists the languages that the service supports and provides links to topics that describe their SPR symbols, IPA symbols, and IPA Unicode values. The topics provide examples of each symbol in words from the language. Because of dialectal differences, the examples might not always match your pronunciation.

The Availability column indicates whether each voice is available for

IBM Cloud

,

IBM Cloud Pak for Data

, or both (All versions). For more information about the supported voices, see [Languages and voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices).



Table 3. Phonetic symbols for supported languages

 Language Availability 

 [Dutch (Netherlands) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-nlSymbols-new) All versions 
 [English (Australian) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-auSymbols-new) All versions 
 [English (United Kingdom) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-gbSymbols) All versions
Smart formatting is beta functionality. For more information, see [Smart formatting](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-formattingsmart-formatting).



Text to Speech



* How do I access my service credentials?

How you access your service credentials depends on whether you are using Text to Speech with IBM Cloud® or IBM Cloud Pak® for Data. For more information about obtaining your credentials for both versions, see [Before you begin](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-gettingStartedgetting-started-before-you-begin) in the getting started tutorial.

Once you have your service credentials, see the following topics for information about authenticating to the service:



* [Authenticating to IBM Cloud](https://cloud.ibm.com/docs/watson?topic=watson-iamgs-credential-cloud)
* [Authenticating to IBM Cloud Pak for Data](https://cloud.ibm.com/docs/watson?topic=watson-iamgs-credential-cpd)



* What languages does the service support?

The Text to Speech service supports male and female voices in various spoken languages:



* The service offers expressive neural voices for English (Australian and United States).
* The services offers enhanced neural voices for Dutch Netherlands, English (United Kingdom and United States), French (Canadian and France), German, Italian, Japanese, Korean, Portuguese (Brazilian), and Spanish (Castilian, Latin American, and North American).



Some languages and voices are available only for IBM Cloud®, not for IBM Cloud Pak® for Data. For more information about the available voices for all languages, see [Languages and voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices).
* How does the service synthesize audio?

The Text to Speech service offers voices that rely on neural technology to synthesize text to speech. The topic of synthesizing text to speech is inherently complex. For more information, see