vania-janet/multiturn-rag-retrieval-data huggingface dataset:
function main(params) {
console.log(JSON.stringify(params))
if ((params.payload.context.skills["main skill"].user_defined.language !== 'en') && (params.payload.context.skills["main skill"].user_defined.language !== 'none')) {
const options = { method: 'POST',
url: 'https://api.us-south.language-translator.watson.cloud.ibm.com/instances/572b37be-09f4-4704-b693-3bc63869nnnn/v3/translate?version=2018-05-01',
auth: {
'username': 'apikey',
'password': 'nnn'
},
body: {
text: [
params.payload.output.generic0].text
],
target: params.payload.context.skills["main skill"].user_defined.language
},
json: true
};
return rp(options)
.then(res => {
params.payload.context.skills["main skill"].user_defined["original_output"] = params.payload.output.generic[0].text;
params.payload.output.generic[0].text = res.translations[0].translation;
return {
body : params
}
})
}
return {
body : params
}
};
Show more





 Removing the webhook 

If you decide you do not want to process message responses, complete the following steps:



1. From the assistant overview page, click the !
* Canadian French to English (fr-CA-en) and English to Canadian French (en-fr-CA). Canadian French cannot be identified by the service's language identification feature.
* English to Welsh (en-cy). (The cy-en model was already available.)
* Serbian to English (sr-en) and English to Serbian (en-sr). Serbian translation support is based on the Cyrillic alphabet. (Bosnian and Croatian translation support is based on the Latin alphabet.)



For more information, see [List of supported languages](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelslist-languages-supported).





 31 July 2020 

New method for listing supported languages
: Added the [List supported languages](https://cloud.ibm.com/apidocs/language-translatorlistlanguages) (GET /v3/languages) method to retrieve the list of supported languages for translation. The method returns a complete list of all supported languages, sorted by language code (for example, af, ar). In addition to basic information about each language, the response indicates whether the language is supported_as_source for translation and supported_as_target for translation. It also lists whether the language is identifiable.

For more information, see [Listing supported languages for translation](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-modelslist-languages).





 14 July 2020 

New supported file types for customization
: Added support for new file types for customization: CSV, TSV, XLSX (Microsoft Excel), XLIFF, and JSON. For more information, see [Supported document formats for training data](https://cloud.ibm.com/docs/language-translator?topic=language-translator-customizingsupported-document-formats-for-training-data).

New translation models
: Added Welsh-to-English (cy-en) support in translation, and added support for Welsh (cy) in language detection.
"gender": "female",
"url": "{url}/v1/voices/es-LA_SofiaV3Voice",
"supported_features": {
"voice_transformation": false,
"custom_pronunciation": true
},
"description": "Sofia: Latin American Spanish (español latinoamericano) female voice."
},
{
"name": "pt-BR_IsabelaV3Voice",
"language": "pt-BR",
"customizable": true,
"gender": "female",
"url": "{url}/v1/voices/pt-BR_IsabelaV3Voice",
"supported_features": {
"voice_transformation": false,
"custom_pronunciation": true
},
"description": "Isabela: Brazilian Portuguese (português brasileiro) female voice."
},
. . .
]
}
Show more

The fields of the voice objects provide the following information:



* name is an identifier for the voice (for example, en-US_LisaV3Voice). Specify this value for the voice parameter of the /v1/synthesize method.
* language specifies the language and region of the voice (for example, en-US).
* gender identifies the voice as male or female.
* url identifies the URL for the voice.
* description provides a brief description of the voice.
* customizable is a boolean value that indicates whether the voice can be customized with the service's customization interface. (This field, which provides the same information as the custom_pronunciation field, is maintained for backward compatibility.)
* supported_features describes the additional service features that are supported by the voice:



* voice_transformation is a boolean value that indicates whether the voice supported the SSML <voice-transformation> element. The feature was available only for obsolete standard US English voices. You cannot use the <voice-transformation> element with any supported voice.
* Try the live [demo](https://www.ibm.com/demos/live/watson-language-translator).
* View the [API & SDK reference](https://cloud.ibm.com/apidocs/language-translator).
* View language support information:



* [Supported languages for translation](https://cloud.ibm.com/docs/language-translator?topic=language-translator-translation-models)
* [Identifiable languages](https://cloud.ibm.com/docs/language-translator?topic=language-translator-identifiable-languages)
* [Adding skills to your assistant](https://cloud.ibm.com/docs/assistant?topic=assistant-skill-add)
* [Deploying your assistant](https://cloud.ibm.com/docs/assistant?topic=assistant-deploy-integration-add)







 Browser support 

The Watson Assistant application (where you create assistants and skills) requires the same level of browser software as is required by IBM Cloud. For more information, see IBM Cloud [Prerequisites](https://cloud.ibm.com/docs/overview?topic=overview-prereqs-platformbrowsers-platform).

For information about the web browsers that are supported by the web chat integration, see [Browser Support](https://cloud.ibm.com/docs/assistant?topic=assistant-web-chat-basicsweb-chat-basics-browsers).





 Language support 

Language support by feature is detailed in the [Supported languages](https://cloud.ibm.com/docs/assistant?topic=assistant-language-support) topic.





 Terms and notices 

See [IBM Cloud Terms and Notices](https://cloud.ibm.com/docs/overview/terms-of-use?topic=overview-terms) for information about the terms of service.

US Health Insurance Portability and Accountability Act (HIPAA) support is available with Enterprise with Data Isolation plans that are hosted in the Washington, DC location created on or after 1 April 2019. For more information, see [Enabling HIPAA support for your account](https://cloud.ibm.com/docs/account?topic=account-enabling-hipaa).

To learn more about service terms and data security, read the following information:



* [Service terms](https://www-03.ibm.com/software/sla/sladb.nsf/sla/saas?OpenDocument) (Search for the Watson Assistant offering)
For more information, see [Before you begin](https://cloud.ibm.com/docs/language-translator?topic=language-translator-document-translator-tutorialtranslate-prerequisites) in the topic Translating documents.

New support for additional subtitle file formats
: The service now supports additional subtitle formats:



* Apple® iTunes® Timed Text (.itt files)
* Distribution Format Exchange Profile (.dxfp and .xml files)
* Source Code Control (.scc files)
* Synchronized Accessible Media Interchange (.sami and .smi files)
* SubStation Alpha (.ssa files)
* Time Text Markup Language (.ttml files)



For more information about all supported file formats, see [Supported file formats](https://cloud.ibm.com/docs/language-translator?topic=language-translator-document-translator-tutorialsupported-file-formats).

Identifying file formats for translation
: The service accepts two forms of identification for most file formats. You can identify the format of a file that you send for translation in one of two ways:



* By specifying the appropriate file extension for the format.
* By specifying the content type (MIME type) of the format as the type of the file parameter.



The documentation lists the valid file extensions and content types for each supported format. In most cases, specifying the correct file extension is preferred because it can eliminate ambiguity and is simpler. For subtitles, the documentation makes clear where either the file extension or the content type is needed. For more information about all file formats, their file extensions and content types, and how and when to specify the file extension or content type, see [Supported file formats](https://cloud.ibm.com/docs/language-translator?topic=language-translator-document-translator-tutorialsupported-file-formats).

Documentation updates for bidirectional translation
: The documentation now states that the service correctly translates from and to bidirectional languages that are written left-to-right and right-to-left (for example, Arabic, Hebrew, and Urdu).
Language support for customization 

The IBM Watson® Speech to Text service supports language model customization, acoustic model customization, and grammars at different levels for different languages and types of models. The tables in the following sections document the service's level of support for all languages and models:



* GA indicates that the feature is generally available for production use for that model.
* Beta indicates that the feature is available as a beta offering for that model.
* Not supported means that the feature is not available for that model.



Where applicable, customization features are also identified as specific to IBM Cloud® or IBM Cloud Pak® for Data. Unless otherwise indicated, a feature is available for both versions of the service.



 Customization support for previous-generation models 

Effective 31 July 2023, all previous-generation models will be removed from the service and the documentation. Most previous-generation models were deprecated on 15 March 2022. You must migrate to the equivalent next-generation model by 31 July 2023. For more information, see [Migrating to next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-migrate).

Table 1 lists the previous-generation models that are supported for language model customization, grammars, and acoustic model customization. To learn which models are supported for speech recognition for IBM Cloud®, IBM Cloud Pak® for Data, or both, see [Supported previous-generation language models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-modelsmodels-supported).



Table 1. Previous-generation language support for customization

 Language (dialect) Models Language model customization Grammars Acoustic model customization 

 Arabic <br>(Modern Standard) ar-MS_BroadbandModel <br>Deprecated Not supported Not supported GA 
 Chinese <br>(Mandarin) zh-CN_NarrowbandModel <br>Deprecated Not supported Not supported GA 
 zh-CN_BroadbandModel <br>Deprecated Not supported Not supported GA 
 Dutch <br>(Netherlands) nl-NL_NarrowbandModel <br>Deprecated GA GA GA
Language support 

IBM Watson™ Discovery offers two levels of language support, Basic and Full.



 Language                        Support Level 

 English (en)                    Full          
 Arabic (ar)                     Basic         
 Chinese, simplified (zh-CN)     Basic         
 Dutch (nl)                      Basic         
 French (fr)                     Full          
 German (de)                     Full          
 Italian (it)                    Full          
 Japanese (ja)                   Full          
 Korean (ko)                     Full          
 Portuguese, Brazilian (pt-br)   Full          
 Spanish (es)                    Full          





  Basic support 



*  Discovery support includes:



*  Document conversion
*  Orchestration
*  Discovery Tooling functionality
*  Front-end API (excluding Front-end Query API)
*  Language-optimized index
*  Relevancy training
*  Query expansion
*  Stopwords
*  Documentation in 9 languages: English, French, German, Japanese, Korean, Brazilian Portuguese, Spanish, Chinese Simplified, and Chinese Traditional. Scroll to the bottom of the IBM Cloud documentation to select the language.



*  Enrichments



*  Integration with IBM Watson™ Knowledge Studio custom models
*  Language detection





To create a collection in Dutch or Chinese, you must use the API. For more information, see the [API reference](https://cloud.ibm.com/apidocs/discoverycreatecollection).





  Full support 

Full support includes basic support, plus:



*  Passage retrieval
*  Translated IBM Watson™ Knowledge Studio interface
*  All Natural Language Understanding (NLU) enrichments, plus metadata extraction. NLU enrichments are:



*  Entity Extraction
*  Keyword Extraction
*  Category Classification
*  Concept Tagging
*  Semantic Role Extraction
*  Sentiment Analysis
*  Emotion Analysis (English only)









  English-only support 

The following features are currently supported in English only:



*  Element Classification enrichment
*  Document deduplication (beta)



The Element Classification enrichment is deprecated and will no longer be available, effective 10 July 2020.





  Japanese-only support 

The following feature is currently supported in Japanese only:



*  Custom tokenization dictionaries
See the [Supported languages](https://cloud.ibm.com/docs/assistant?topic=assistant-language-support) topic for details.





 16 June 2017 

Recommendations (Beta - Premium users only)
: The Improve panel also includes a Recommendations page that recommends ways to improve your system by analyzing the conversations that users have with your chatbot, and taking into account your system's current training data and response certainty.





 14 June 2017 

Fuzzy matching for additional languages (Beta)
: Fuzzy matching for entities is now available for additional languages, as noted in the [Supported languages](https://cloud.ibm.com/docs/assistant?topic=assistant-language-support) topic. You can turn on fuzzy matching per entity to improve the ability of your assistant to recognize terms in user input with syntax that is similar to the entity, without requiring an exact match. The feature is able to map user input to the appropriate corresponding entity despite the presence of misspellings or slight syntactical differences. For example, if you define giraffe as a synonym for an animal entity, and the user input contains the terms giraffes or girafe, the fuzzy match is able to map the term to the animal entity correctly. See [Fuzzy matching](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-fuzzy-matching) for details.





 13 June 2017 

User conversations
: The Improve panel now includes a User conversations page, which provides a list of user interactions with your chatbot that can be filtered by keyword, intent, entity, or number of days. You can open individual conversations to correct intents, or to add entity values or synonyms.

Regex change
: The regular expressions that are supported by SpEL functions like find, matches, extract, replaceFirst, replaceAll and split have changed. A group of regular expression constructs are no longer allowed, including look-ahead, look-behind, possessive repetition and backreference constructs. This change was necessary to avoid a security exposure in the original regular expression library.





 12 June 2017 

Updates
: This release includes the following updates:
IBM Watson® Assistant for IBM Cloud Pak® for Data version 1.4 is available
: Watson Assistant for IBM Cloud Pak® for Data version 1.4 is compatible with IBM Cloud Pak® for Data version 2.5.

Czech language not automatically enabled
: The Czech language is not enabled automatically anymore.

Assistants and Skills navigation menu update
: The main menu options of Assistants and Skills have moved from being displayed at the top of the page to being shown as icons in a new navigation pane.



* ![Assistants menu icon](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/nav-ass-icon.png) Assistants
* ![Skills menu icon](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/nav-skills-icon.png) Skills



Skills secondary navigation menu update
: The tabbed pages for the tools that you use to develop a dialog skill were moved to a secondary navigation bar that is displayed when you open the skill.

![Skills secondary navigation menu](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/secondary-nav.png)

Rich response types support
: Rich response types are now supported in a dialog node with slots. You can display a list of options for a user to choose from as the prompt for a slot, for example. For more information, see [Gathering information with slots](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-dialog-slots).

Improved Entities, Dialog, and Intents page responsiveness
Syntax X 







 Swedish 



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories 
 Classifications 
 Concepts 
 Emotion 
 Entities X* 
 Keywords X 
 Metadata X 
 Relations 
 Semantic roles 
 Sentiment X 
 Syntax X 



* Relevance ranking is not supported.





 Turkish 



 Feature Standard Support [Custom model](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-customizing) support 

 Categories 
 Classifications 
 Concepts 
 Emotion 
 Entities X* 
 Keywords X 
 Metadata 
 Relations 
 Semantic roles 
 Sentiment X 
 Syntax X 



* Relevance ranking is not supported.
* Spanish (Castilian, Latin American, and North American dialects)



For different languages, the service offers female voices, male voices, or both. For more information about the supported languages and voices, the types of voices that the service provides for each language, and their status for both versions of the service, see [Languages and voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices).





 Audio support 

The service produces audio in many popular formats:



* A-law
* Basic audio
* Free Lossless Audio Codec (FLAC)
* Linear 16-bit Pulse-Code Modulation (PCM)
* MP3 (or MPEG)
* Mu-law (or u-law)
* Ogg or Web Media (WebM) audio with the Opus or Vorbis codec
* Waveform Audio File Format (WAV)



Different formats support different sampling rates and other characteristics. For more information, see [Using audio formats](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-audio-formats).





 Beta features 

IBM occasionally releases features and language support that are classified as beta. Such features are provided so that you can evaluate their functionality. They might be unstable and are subject to change or removal with short notice. They are not intended for use in a production environment.

Beta features might not provide the same level of performance or compatibility as generally available features. Generally available features are ready for use in a production environment.





 Pricing 

IBM Cloud

The service offers multiple pricing plans to suit your usage and application needs. For more information about the pricing plans or to purchase a plan, see the Text to Speech service in the [IBM Cloud® Catalog](https://cloud.ibm.com/catalog/text-to-speech).
: The language detection model is faster and detects more languages overall. For the complete list of languages, see [Detectable languages](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-detectable-languages).



* The following languages are new additions to the list:



* Belarusian (be)
* Bihari (bh)
* Dhivehi (dv)
* Galician (gl)
* Ganda (lg)
* Inuktitut (iu)
* Javanese (jv)
* Kannada (kn)
* Khmer (km)
* Kinyarwanda (rw)
* Laothian (lo)
* Malayalam (ml)
* Marathi (mr)
* Oriya (or)
* Punjabi (pa)
* Scots Gaelic (gd)
* Sinhalese (si)
* Tamil (ta)
* Telugu (te)
* Yiddish (yi)



* The following languages are no longer detectable:



* Breton (br)
* Chamorro (ch)
* Esperanto (eo)
* Faroese (fo)
* Hausa (ha)
* Ndebele (nr)
* Ojibwa (oj)









 November 2017 





 28 November 2017 

Entity mentions
: Get the locations of entity mentions in your entities requests by adding the option "mentions": true.



* Example POST /analyze request body:

{
"text": "Intel planned to announce Monday a laptop-computer chip that combines an Intel processor and an AMD graphics unit.",
"features": {
"entities": {
"mentions": true
}
}
}
* Example response:

{
"usage": {
"text_units": 1,
"text_characters": 114,
"features": 1
},
"language": "en",
"entities": [
{
"type": "Company",
function main(params) {
console.log(JSON.stringify(params))
if ((params.payload.context.skills["actions skill"].user_defined.language !== 'en') && (params.payload.context.skills["actions skill"].user_defined.language !== 'none')) {
const options = { method: 'POST',
url: 'https://api.us-south.language-translator.watson.cloud.ibm.com/instances/572b37be-09f4-4704-b693-3bc63869nnnn/v3/translate?version=2018-05-01',
auth: {
'username': 'apikey',
'password': 'nnn'
},
body: {
text: [
params.payload.output.generic0].text
],
target: params.payload.context.skills["actions skill"].user_defined.language
},
json: true
};
return rp(options)
.then(res => {
params.payload.context.skills["actions skill"].user_defined["original_output"] = params.payload.output.generic[0].text;
params.payload.output.generic[0].text = res.translations[0].translation;
return {
body : params
}
})
}
return {
body : params
}
};
Show more





 Removing the webhook 

If you decide you do not want to process message responses with a webhook, complete the following steps:



1. From the assistant overview page, click the !
Table 5. Entity feature support details

 Language [Contextual entities](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-create-annotation-based) [Synonym recommendations](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-create-dictionary-based) 

 English (en) GA GA 
 Arabic (ar) NA NA 
 Chinese (Simplified) (zh-cn) NA NA 
 Chinese (Traditional) (zh-tw) NA NA 
 Czech (cs) NA NA 
 Dutch (nl) NA NA 
 French (fr) Beta GA 
 German (de) NA NA 
 Italian (it) NA NA 
 Japanese (ja) NA GA 
 Korean (ko) NA NA 
 Portuguese (Brazilian) (pt-br) NA NA 
 Spanish (es) NA GA 
 Universal (xx) NA NA 







 System entity feature support details 



Table 6. System entity feature support details

 Language [System entities](https://cloud.ibm.com/docs/assistant?topic=assistant-system-entities) [Legacy system entities](https://cloud.ibm.com/docs/assistant?topic=assistant-legacy-system-entities) 

 English (en) GA Deprecated 
 Arabic (ar) GA Deprecated 
 Chinese (Simplified) (zh-cn) GA Deprecated 
 Chinese (Traditional) (zh-tw) GA Deprecated 
 Czech (cs) GA Deprecated 
 Dutch (nl) GA Deprecated 
 French (fr) GA Deprecated 
 German (de) GA Deprecated 
 Italian (it) GA Deprecated 
 Japanese (ja) GA Deprecated 
 Korean (ko) GA Deprecated 
 Portuguese (Brazilian) (pt-br) GA Deprecated 
 Spanish (es) GA Deprecated 
 Universal (xx) GA NA 



The Watson Assistant service supports multiple languages as noted, but the tool interface itself (descriptions, labels, etc.) is in English. All supported languages can be input and trained through the English interface.