vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories X 
 Classifications X 
 Concepts X 
 Emotion 
 Entities X X 
 Keywords X 
 Metadata X 
 Relations X X 
 Semantic roles 
 Sentiment X 
 Syntax X 







 Japanese 



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories X 
 Classifications X 
 Concepts X 
 Emotion 
 Entities X X 
 Keywords X 
 Metadata X 
 Relations X X 
 Semantic roles X 
 Sentiment X 
 Syntax X 







 Korean 



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories X 
 Classifications X 
 Concepts X 
 Emotion 
 Entities X X 
 Keywords X 
 Metadata X 
 Relations X X 
 Semantic roles X 
 Sentiment X 
 Syntax X 







 Norwegian 

Please note that Natural Language Understanding considers Norwegian (standard language code no) as equivalent to Norwegian-Bokmal (standard language code nb), and uses the same Norwegian-Bokmal model for both language codes.



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories 
 Classifications 
 Concepts 
 Emotion 
 Entities X* 
 Keywords X 
 Metadata 
 Relations 
 Semantic roles 
 Sentiment X 
 Syntax X 



* Relevance ranking is not supported.





 Norwegian (Bokmal) 

Please note that Natural Language Understanding considers Norwegian (standard language code no) as equivalent to Norwegian-Bokmal (standard language code nb), and uses the same Norwegian-Bokmal model for both language codes.



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories 
 Classifications
Portuguese (Brazilian) (pt-br) GA GA GA 
 Spanish (es) GA GA GA 
 Universal (xx) GA GA GA 



The Watson Assistant service supports multiple languages as noted, but the user interface itself (such as descriptions and labels) is in English. All supported languages can be input and trained through the English interface.

GB18030 compliance: GB18030 is a Chinese standard that specifies an extended code page for use in the Chinese market. This code page standard is important for the software industry because the China National Information Technology Standardization Technical Committee mandates that any software application that is released for the Chinese market after September 1, 2001, be enabled for GB18030. The Watson Assistant service supports this encoding, and is certified GB18030-compliant







 Changing an assistant language 

After an assistant is created, its language cannot be modified.





 Working with accented characters 

In a conversational setting, users might or might not use accents with the Watson Assistant service. As such, both accented and nonaccented versions of words might be treated the same for intent detection and entity recognition.

However, for some languages like Spanish, some accents can alter the meaning of the entity. Thus, for entity detection, although the original entity might implicitly have an accent, your assistant can also match the nonaccented version of the same entity, but with a slightly lower confidence score.

For example, for the word "barrió", which has an accent and corresponds to the past tense of the verb "barrer" (to sweep), your assistant can also match the word "barrio" (neighborhood), but with a slightly lower confidence.

The system provides the highest confidence scores in entities with exact matches. For example, barrio isn't detected if barrió is in the training set; and barrió isn't detected if barrio is in the training set.

You are expected to train the system with the proper characters and accents. For example, if you are expecting barrió as a response, put barrió into the training set.

Although not an accent mark, the same applies to words that use the Spanish letter ñ versus the letter n, such as "uña" versus "una". In this case, the letter ñ is not simply an n with an accent; it is a unique, Spanish-specific letter.
: The POST /v1/customizations/{custom_id}/words and PUT /v1/customizations/{customization_id}/words/{word} methods now return HTTP response code 400 with the error message Part of speech is supported for ja-JP language only if you attempt to specify a part_of_speech for a language other than Japanese.

Changes to response body for adding words method
: The POST /v1/customizations/{custom_id}/words method now returns an empty response body ({}).





 1 December 2016 

New Latin American Spanish voice: es-LA_SofiaVoice
: The service includes a new voice, es-LA_SofiaVoice, which is the Latin American equivalent of the es-US_SofiaVoice voice. The most significant difference between the two voices concerns how they interpret a $ (dollar sign): The Latin American version uses the term pesos; the North American version uses the term dolares. Other minor differences might also exist between the two voices.

New US English voices: en-US_LisaVoice and en-US_MichaelVoice
: In addition to the en-US_AllisonVoice, two more voices are now transformable with SSML voice transformation: en-US_LisaVoice and en-US_MichaelVoice.

Changes to customization for Japanese
: When you use the customization interface with Japanese, the service now matches the longest word from the word/translation pairs that are defined for a custom model. For example, consider the following two entries for a custom model:

{
"words": [
{"word":"ＮＹ", "translation":"ニューヨーク", "part_of_speech":"Mesi"},
{"word":"ＮＹＣ", "translation":"ニューヨークシティ", "part_of_speech":"Mesi"}
]
}

If the service finds the string ＮＹＣ in the input text, it matches that word because it is a longer match than ＮＹ. Previously, the service would have matched the string ＮＹ.
Relations X X 
 Semantic roles 
 Sentiment X 
 Syntax X 







 Romanian 



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories 
 Classifications 
 Concepts 
 Emotion 
 Entities X* 
 Keywords X 
 Metadata 
 Relations 
 Semantic roles 
 Sentiment X 
 Syntax X 



* Relevance ranking is not supported.





 Russian 



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories 
 Classifications 
 Concepts 
 Emotion 
 Entities X* 
 Keywords X 
 Metadata X 
 Relations 
 Semantic roles 
 Sentiment X 
 Syntax X 



* Relevance ranking is not supported.





 Slovak 



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories 
 Classifications 
 Concepts 
 Emotion 
 Entities X* 
 Keywords X 
 Metadata 
 Relations 
 Semantic roles 
 Sentiment X 
 Syntax X 



* Relevance ranking is not supported.





 Spanish 



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories X 
 Classifications X 
 Concepts X 
 Emotion 
 Entities X X 
 Keywords X 
 Metadata X 
 Relations X X 
 Semantic roles X 
 Sentiment X 
 Syntax X 







 Swedish 



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories 
 Classifications 
 Concepts 
 Emotion 
 Entities X* 
 Keywords X 
 Metadata X 
 Relations 
 Semantic roles 
 Sentiment X 
 Syntax X 



* Relevance ranking is not supported.





 Turkish
Supported languages for translation 

IBM is announcing the deprecation of the IBM Watson® Language Translator service for IBM Cloud® in all regions. As of 10 June 2023, the Language Translator tile will be removed from the IBM Cloud Platform for new customers; only existing customers will be able to access the product. As of 10 June 2024, the service will reach its End of Support date. As of 10 December 2024, the service will be withdrawn entirely and will no longer be available to any customers.

IBM Watson® Language Translator supports a wide array of translatable languages. It also supports customization of many translation models.



 Listing supported languages for translation 

You can use the [List supported languages](https://cloud.ibm.com/apidocs/language-translatorlistlanguages) method to retrieve the list of supported languages for translation. The following example calls the method:

curl -X GET --user "apikey:{apikey}" "{url}/v3/languages?version=2018-05-01"

The method returns a complete list of all supported languages, sorted by language code (for example, af, ar). In addition to basic information about each language, the response indicates whether the language is supported_as_source for translation and supported_as_target for translation. It also lists whether the language is identifiable.

"languages": [
{
"language": "af",
"language_name": "Afrikaans",
"native_language_name": "Afrikaans",
"country_code": "ZA",
"words_separated": true,
"direction": "left_to_right",
"supported_as_source": false,
"supported_as_target": false,
"identifiable": true
},
{
"language": "ar",
"language_name": "Arabic",
"native_language_name": "العربية",
"country_code": "AR",
"words_separated": true,
"direction": "right_to_left",
* [Languages and voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices)
* [Migrating from neural voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices-migrate)







 22 March 2023 

New beta Netherlands Dutch enhanced neural voice
: The service now supports a new enhanced neural female voice for Netherlands Dutch: nl-NL_MerelV3Voice. It supports the use of both standard International Phonetic Alphabet (IPA) and IBM Symbolic Phonetic Representation (SPR) phonetic symbols.

The new voice is beta functionality pending completion of support for SSML. At its initial release, the voice does not support use of the following SSML-related functionality:



* The <prosody> element with any speech synthesis request
* The rate_percentage and pitch_percentage parameters with any speech synthesis request
* The <mark> element with a WebSocket speech synthesis request
* The timings parameter of the JSON text message with a WebSocket speech synthesis request



For more information about the new voice, its support for IPA and SPR symbols, and migrating to the new voice from the deprecated Netherlands Dutch neural voices, see



* [Enhanced neural voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voiceslanguage-voices-enhanced-neural)
* [Dutch (Netherlands) symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-nlSymbols-new)
* [Migrating from neural voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices-migrate)



Defect fix: Update Korean phonetic symbols in documentation
: Defect fix: In the documentation for Korean SPR symbols, two-character symbols for consonants are now enclosed in single quotes, making them a single symbol. Previously, they were shown as two separate symbols, without enclosing quotes.
* Punjabi (Indian) to English (pa-en)
* Punjabi (Pakistani) to English (pa-PK-en)



The Kannada language is new; it is not identifiable. All of the languages except for Kannada already existed and are identifiable. For more information, see [Supported languages for translation](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-models).

Improved customizable translation models
: Updated the existing customizable translation models for the following languages for improved translation:



* Bengali to English (bn-en)
* Malayalam to English (ml-en)
* Tamil to English (ta-en)
* Telugu to English (te-en)



For more information, see [Supported languages for translation](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-models).





 23 October 2020 

New customizable translation models
: Added improved versions of the following customizable translation models for Korean:



* English to Korean (en-ko)
* Korean to English (ko-en)



The models are improved to preserve markup in the translation if present in the source.





 16 October 2020 

New customizable translation models
: Added the following customizable translation models for Basque:



* Basque to Spanish (eu-es)
* Spanish to Basque (es-eu)



For more information, see [List of supported languages](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelslist-languages-supported).





 25 September 2020 

New translation models
: Added the following translation models for Montenegrin:



* Montenegrin to English (cnr-en)
* English to Montenegrin (en-cnr)



Montenegrin cannot be identified by the service's language identification feature. (Bosnian, Croatian, and Montenegrin translation support is based on the Latin alphabet. Of the three very similar languages, only Croatian is identifiable.) The cnr-en and en-cnr translation models are not customizable at this time.
Specifically, the service does not support the features described in the following sections of the specification:



* [Section 1.4 Semantic Interpretation](https://www.w3.org/TR/speech-grammar/S1.4). IBM is working to support this feature in a future release of the service.
* [Section 1.5 Embedded Grammars](https://www.w3.org/TR/speech-grammar/S1.5). IBM is working to support this feature in a future release of the service.
* [Section 2.2.2 External Reference by URI](https://www.w3.org/TR/speech-grammar/S2.2.2). The service supports only local references, as described in [Section 2.2.1 Local References](https://www.w3.org/TR/speech-grammar/S2.2.1). In other words, a grammar must be self-contained.
* [Section 2.2.3 Special Rules](https://www.w3.org/TR/speech-grammar/S2.2.3).
* [Section 2.2.4 Referencing N-gram Documents (Informative)](https://www.w3.org/TR/speech-grammar/S2.2.4).
* [Section 2.7 Language](https://www.w3.org/TR/speech-grammar/S2.7). The service does not support language switching. The service supports only one global language per grammar.



Words in the grammar must be in UTF-8 encoding (ASCII is a subset of UTF-8). Using any other encoding can lead to issues when compiling the grammar or unexpected results in decoding. The service ignores an encoding specified in the header of the grammar.
* Colombian Spanish (es-CO_BroadbandModel and es-CO_NarrowbandModel)
* Mexican Spanish (es-MX_BroadbandModel and es-MX_NarrowbandModel)
* Peruvian Spanish (es-PE_BroadbandModel and es-PE_NarrowbandModel)



The Castilian Spanish models are not new. They are generally available (GA) for speech recognition and language model customization, and beta for acoustic model customization.

The other five dialects are new and are beta for all uses. Because they are beta, these additional dialects might not be ready for production use and are subject to change. They are initial offerings that are expected to improve in quality with time and usage.

For more information, see the following sections:



* [Supported previous-generation language models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-modelsmodels-supported)
* [Language support for customization](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-support)







 24 June 2019 

Updates to Brazilian Portuguese and US English models for improved speech recognition
: The following narrowband models have been updated for improved speech recognition:



* Brazilian Portuguese narrowband model (pt-BR_NarrowbandModel)
* US English narrowband model (en-US_NarrowbandModel)



By default, the service automatically uses the updated models for all speech recognition requests. If you have custom language or custom acoustic models that are based on the models, you must upgrade your existing custom models to take advantage of the updates by using the following methods:



* POST /v1/customizations/{customization_id}/upgrade_model
* POST /v1/acoustic_customizations/{customization_id}/upgrade_model



For more information, see [Upgrading custom models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-upgrade).

New support for concurrent requests to update different custom acoustic models
[Japanese](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsjapanese) ja [Urdu](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsurdu) ur 
 [Kannada](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelskannada) kn [Vietnamese](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelsvietnamese) vi 
 [Korean](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelskorean) ko [Welsh](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelswelsh) cy 



Notes:



1. Basque and Catalan are supported only for translation to and from Spanish.
2. The translation models for Montenegrin are not customizable at this time.
3. Serbian translation support is based on the Cyrillic alphabet. (Bosnian, Croatian, and Montenegrin translation support is based on the Latin alphabet.)







 Listing supported translation models for customization 

You can use the [List models](https://cloud.ibm.com/apidocs/language-translatorlistmodels) method to view the available translation models and any custom models that you created. You can filter results by language with the source and target parameters. The following example lists models that can translate from English to Spanish.

curl -X GET --user "apikey:{apikey}" "{url}/v3/models?source=en&target=es&version=2018-05-01"

The method returns information about the en-es translation model that you can use to translate from English to Spanish. The model is customizable.

"models": [
{
"model_id": "en-es",
"source": "en",
"target": "es",
* [French](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportfrench)
* [German](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportgerman)
* [Hebrew](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supporthebrew)
* [Hindi](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supporthindi)
* [Italian](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportitalian)
* [Japanese](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportjapanese)
* [Korean](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportkorean)
* [Norwegian](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportnorwegian)
* [Norwegian (Bokmal)](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportnorwegian-bokmal)
* [Norwegian (Nyorsk)](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportnorwegian-nyorsk)
* [Polish](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportpolish)
* [Portuguese](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-language-supportportuguese)
And the links between nodes are represented in a way that makes it easier to understand the relationships between the nodes.





 21 June 2017 

Arabic support
: Language support for Arabic is now generally available. For details, see [Configuring bidirectional languages](https://cloud.ibm.com/docs/assistant?topic=assistant-language-supportlanguage-support-configure-bidirectional).

Language updates
: The Watson Assistant service algorithms have been updated to improve overall language support. See the [Supported languages](https://cloud.ibm.com/docs/assistant?topic=assistant-language-support) topic for details.





 16 June 2017 

Recommendations (Beta - Premium users only)
: The Improve panel also includes a Recommendations page that recommends ways to improve your system by analyzing the conversations that users have with your chatbot, and taking into account your system's current training data and response certainty.





 14 June 2017 

Fuzzy matching for additional languages (Beta)
: Fuzzy matching for entities is now available for additional languages, as noted in the [Supported languages](https://cloud.ibm.com/docs/assistant?topic=assistant-language-support) topic. You can turn on fuzzy matching per entity to improve the ability of your assistant to recognize terms in user input with syntax that is similar to the entity, without requiring an exact match. The feature is able to map user input to the appropriate corresponding entity despite the presence of misspellings or slight syntactical differences. For example, if you define giraffe as a synonym for an animal entity, and the user input contains the terms giraffes or girafe, the fuzzy match is able to map the term to the animal entity correctly. See [Fuzzy matching](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-fuzzy-matching) for details.





 13 June 2017 

User conversations
: The Improve panel now includes a User conversations page, which provides a list of user interactions with your chatbot that can be filtered by keyword, intent, entity, or number of days. You can open individual conversations to correct intents, or to add entity values or synonyms.
: The Australian English models, en-AU_BroadbandModel and en-AU_NarrowbandModel, are now generally available (GA). They were previously beta. They also now support language model and acoustic model customization.



* For more information about supported languages and models, see [Previous-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models).
* For more information about language support for customization, see [Language support for customization](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-support).



Updates to Brazilian Portuguese models for improved speech recognition
: The Brazilian Portuguese models, pt-BR_BroadbandModel and pt-BR_NarrowbandModel, have been updated for improved speech recognition. By default, the service automatically uses the updated models for all speech recognition requests. If you have custom language or custom acoustic models that are based on the models, you must upgrade your existing custom models to take advantage of the updates by using the following methods:



* POST /v1/customizations/{customization_id}/upgrade_model
* POST /v1/acoustic_customizations/{customization_id}/upgrade_model



For more information, see [Upgrading custom models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-upgrade).

The split_transcript_at_phrase_end parameter now generally available for all languages
: The speech recognition parameter split_transcript_at_phrase_end is now generally available (GA) for all languages. Previously, it was generally available only for US and UK English. For more information, see [Split transcript at phrase end](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-parsingsplit-transcript).





 7 October 2020 

Updates to Japanese broadband model for improved speech recognition
: The ja-JP_BroadbandModel model has been updated for improved speech recognition.