vania-janet/multiturn-rag-retrieval-data huggingface dataset:
"name": "confirm-abnf",
"status": "analyzed"
},
{
"out_of_vocabulary_words": 8,
"name": "list-abnf",
"status": "analyzed"
}
]
}
Show more





 List a specific grammar example 

The following example shows information about the specified grammar, list-abnf:

IBM Cloud

curl -X GET -u "apikey:{apikey}" "{url}/v1/customizations/{customization_id}/grammars/list-abnf"

IBM Cloud Pak for Data

curl -X GET --header "Authorization: Bearer {token}" "{url}/v1/customizations/{customization_id}/grammars/list-abnf"

The grammar contains eight OOV words and is fully analyzed:

{
"out_of_vocabulary_words": 8,
"name": "list-abnf",
"status": "analyzed"
}







 Deleting a grammar from a custom language model 

Use the DELETE /v1/customizations/{customization_id}/grammars/{grammar_name} method to remove an existing grammar from a custom language model. When it deletes the grammar, the service removes any OOV words that are associated with the grammar from the custom model's words resource unless



* The word was also added by another grammar or by a corpus.
* The word was modified in some way with the POST /v1/customizations/{customization_id}/words or PUT /v1/customizations/{customization_id}/words/{word_name} method.



Removing a grammar does not affect the custom model until you train the model on its updated data by using the POST /v1/customizations/{customization_id}/train method. If you successfully trained the model on the grammar, the grammar continues to be available for speech recognition until you retrain the model.



 Delete a grammar example
By default, the pronunciation is returned for the language of the default voice. For more information, see [Using the default voice](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices-usespecify-voice-default).
* The optional format parameter lets you specify the phonetic format of the pronunciation, either ipa or ibm. By default, the pronunciation is returned in IPA format.
* The optional customization_id parameter lets you specify a custom model for which the pronunciation is to be returned. If the word is not defined in the specified custom model, the service returns the default pronunciation for the model's language. Omit the parameter to see the translation for the specified voice with no customization. Do not specify both a voice and a custom model.



This method is useful because it allows you to query a word from any language and, because it does not require a customization ID, places no restrictions on the words that you can see. It can prove especially helpful when composing a phonetic translation for a new word; you can use it to obtain the pronunciation for an existing word and then use that as the basis of your new translation, which is far more convenient than creating a translation from scratch.

The following example obtains the pronunciation for the word IEEE in the default IPA format:

IBM Cloud

curl -X GET -u "apikey:{apikey}" "{url}/v1/pronunciation?text=IEEE"

IBM Cloud Pak for Data

curl -X GET --header "Authorization: Bearer {token}" "{url}/v1/pronunciation?text=IEEE"

The response shows the IPA symbols for the pronunciation:

{
"pronunciation": ".ˈaɪ .ˈi .ˈi .ˈi"
}

The following example enters a sounds-like translation for the word IEEE and obtains the phonetic equivalent in IBM SPR format. Obtaining the phonetic pronunciation for a sounds-like translation is an especially interesting approach to composing a phonetic translation. The spaces of the word are URL-encoded in the example.

IBM Cloud
If both are used, a compiler error is returned.



* case (exact | insensitive)

Specifies the type of case-folding that the dictionary performs when it determines whether a specific region of the document matches.



* exact

Specifies an exact case-sensitive match.
* insensitive

Specifies a match that is not case-sensitive match. This option is the default value.



* '<entry 1>', '<entry 2>', ... , '<entry n>'

Specifies the strings that you want to include in the inline dictionary. Entries in an inline dictionary can consist of one or more tokens.







 Usage notes 



* The from file and from table formats are recommended, especially when you anticipate modifying the entries, or when you have many entries. By using these formats, you can modify the contents of the dictionary without modifying the code.
* When the create dictionary statement is processed by the modular AQL compiler, references to dictionary file locations that are specified in the create dictionary ... from file syntax must be relative to the root of the module in which this create dictionary statement is issued.
* You can specify comments in a dictionary file by preceding the comment with the character . Comments can start anywhere in a line.
* If you want to specify comments that span multiple lines, you must precede each line with the comment character. If the comment character is part of a dictionary entry, you must escape it using the backslash character (\), as in #. If the backslash character is part of the dictionary entry, it must be escaped with itself, as in \.
* For external dictionaries, when modules are being loaded, you must specify a list of URIs to external dictionaries, as required by the modules that are being loaded.
* Lemmatization of dictionaries: The primary difference between the existing dictionary matching semantics and the lemmatized semantics is that the match is performed against the lemmatized form of the document, instead of the original form of the document.

Dictionary entries that belong to a dictionary with lemma match enabled have these prerequisites:



* A dictionary entry can contain one or more tokens, where each entry token is a lemma.
$(card-type) context['card-type'] 
 $card_type:VISA context['card_type'] == 'VISA' 
 $card_type:(MASTER CARD) context['card_type'] == 'MASTER CARD' 



You can include special characters, such as hyphens or periods, in context variable names. However, doing so can lead to problems when the SpEL expression is evaluated. The hyphen could be interpreted as a minus sign, for example. To avoid such problems, reference the variable by using either the full expression syntax or the shorthand syntax $(variable-name) and do not use the following special characters in the name:



* Parentheses ()
* More than one apostrophe ''
* Quotation marks "







 Shorthand syntax for entities 

The following table shows examples of the shorthand syntax that you can use when referring to entities.



 Shorthand syntax Full syntax in SpEL 

 @year entities['year']?.value 
 @year == 2016 entities['year']?.value == 2016 
 @year != 2016 entities['year']?.value != 2016 
 @city == 'Boston' entities['city']?.value == 'Boston' 
 @city:Boston entities['city']?.contains('Boston') 
 @city:(New York) entities['city']?.contains('New York') 



In SpEL, the question mark (?) prevents a null pointer exception from being triggered when an entity object is null.

If the entity value that you want to check for contains a ) character, you cannot use the : operator for comparison. For example, if you want to check whether the city entity is Dublin (Ohio), you must use @city == 'Dublin (Ohio)' instead of @city:(Dublin (Ohio)).





 Shorthand syntax for intents 

The following table shows examples of the shorthand syntax that you can use when referring to intents.



Intents shorthand syntax

 Shorthand syntax Full syntax in SpEL 

 `#help` `intent == 'help'` 
 `!
* Each pattern (regular expression) is limited to 128 characters.
* Importing or exporting via a CSV file does not currently support patterns.
* The REST API does not support direct access to patterns, but you can retrieve or modify patterns using the /values endpoint.



Fuzzy matching filtered by dictionary (English only)
: An improved version of fuzzy matching for entities is now available, for English. This improvement prevents the capturing of some common, valid English words as fuzzy matches for a given entity. For example, fuzzy matching will not match the entity value like to hike or bike, which are valid English words, but will continue to match examples such as lkie or oike.





 27 September 2017 

Condition builder updates
: The control that is displayed to help you define a condition in a dialog node has been updated. Enhancements include support for listing available context variable names after you enter the $ to begin adding a context variable.





 31 August 2017 

Improve section rollback
: The median conversation time metric, and corresponding filters, are being temporarily removed from the Overview page of the Improve section. This removal will prevent the calculation of certain metrics from causing the median conversation time metric, and the conversations over time graph, to display inaccurate information. IBM regrets removing functionality from the tool, but is committed to ensuring that we are communicating accurate information to users.

Dialog node names
: You can now assign any name to a dialog node; it does not need to be unique. And you can subsequently change the node name without impacting how the node is referenced internally. The name you specify is saved as a title attribute of the node in the workspace JSON file and the system uses a unique ID that is stored in the name attribute to reference the node.





 23 August 2017 

Updates to Korean, Japanese, and Italian
: Language support has been enhanced for Korean, Japanese, and Italian. Note that the Watson Assistant service learning models may have been updated as part of this enhancement, and when you retrain your model any changes will be applied.





 10 August 2017 

Accent normalization
: In a conversational setting, users may or may not use accents while interacting with the Watson Assistant service.
* zh-CN_WangWeiVoice
* zh-CN_ZhangJingVoice



The following additional changes have also been made:



* The Arabic voice is now named ar-MS_OmarVoice. The former name, ar-AR_OmarVoice, is deprecated. It will continue to function for at least one year but might be removed at a future date. You are encouraged to migrate to the new name at your earliest convenience.
* The Arabic language now supports the use of IPA symbols and Unicode values with the SSML <phoneme> element. To create a custom model for Arabic, you must use the language identifier ar-MS. The identifier ar-AR is not supported for customization.
* The IPA symbols for the Arabic language are new. The previously documented symbols have been completely replaced.
* The IPA symbols for the Netherlands Dutch language have been changed as follows:



* Netherlands Dutch no longer supports the following IPA symbols: tʲ (0074+02B2), ɲ (0272), ʦ (02A6), and ʔ (0294).
* Netherlands Dutch now supports the following IPA symbol: ɣ (0263).





Deprecated standard voices
: The following standard concatenative voices are now deprecated:



* de-DE_BirgitVoice
* de-DE_DieterVoice
* en-GB_KateVoice
* en-US_AllisonVoice
* en-US_LisaVoice
* en-US_MichaelVoice
* es-ES_EnriqueVoice
* es-ES_LauraVoice
* es-LA_SofiaVoice
* es-US_SofiaVoice
* fr-FR_ReneeVoice
* it-IT_FrancescaVoice
* ja-JP_EmiVoice
* pt-BR_IsabelaVoice



All of the standard voices that have been deprecated have equivalent neural counterparts, so no voice is being taken away.
Defect fix: Commas are now respected when using phoneme tags
: Defect fix: When using phoneme tags in SSML, commas (pauses) are now working as expected. Previously, when commas were preceding or following text with phoneme tags, the pause was ignored.





 6 April 2023 

Updates to beta Netherlands Dutch enhanced neural voice
: Defect fix: The beta Netherlands Dutch nl-NL_MerelV3Voice was updated for internal fixes and improvements. The limitations described for the initial release of the voice in the [31 March 2023 service update](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-release-notestext-to-speech-31march2023) still apply.





 31 March 2023 

Important: End of service for all neural voices
: Important: All neural voices have reached their end of service date and have been removed from the service and the documentation. No enhanced neural or expressive neural voices are effected. For complete list of all obsolete neural voices, see the [1 March 2023 service updates](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-release-notestext-to-speech-1march2023). An attempt to use an obsolete voice returns the HTTP response code 404 with the message Model '{voice}' not found.

New enhanced neural and expressive neural voices are available for the Australian English, Korean, and Netherlands Dutch languages. You must migrate to one of the new voices for Australian English, Korean, or Netherlands Dutch to continue using the obsolete languages.

For more information, see



* [Languages and voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices)
* [Migrating from neural voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices-migrate)







 22 March 2023 

New beta Netherlands Dutch enhanced neural voice
: The service now supports a new enhanced neural female voice for Netherlands Dutch: nl-NL_MerelV3Voice.
For more information, see [Accessing intents](https://cloud.ibm.com/docs/assistant?topic=assistant-expression-languageexpression-language-intent).





 23 March 2022 

Fuzzy matching updates
: Previously, an update was made so that interactions between the stemming and misspelling fuzzy matching features were not allowed. This change applied to the following languages: English, French, German, and Czech. This was updated so that this change applies only to the English language. For more information, see [How fuzzy matching works](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-fuzzy-matching).





 14 March 2022 

Closed entity matching with accent-normalized values in French
: Closed entities exact matches in French are completed using accent-normalized values or synonyms. For example, if you define a closed entity with a value or synonym with accent marks (for example, garçon or déjà), then variants without accent marks are also recognized (garcon or deja). Likewise, if a closed entity value or synonym is defined without accent marks, then user inputs with accent marks are also recognized. For more information about defining entities, see [Defining information to look for in customer input](https://cloud.ibm.com/docs/assistant?topic=assistant-entities).

Pattern entities do not prevent spelling autocorrection
: Pattern entities that match all characters and words that are usually used to count input words do not prevent spelling autocorrection. For example, if a customer defines the ^..{0,19}$ pattern entity that matches the first 20 characters of an input, then the entity match does not affect spelling autocorrection. In this example, an input of cancl transaction is autocorrected to cancel transaction.

This change applies to the following languages: English and French. For more information, see [Correcting user input](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-runtime-spell-checkdialog-runtime-spell-check-rules).





 1 March 2022 

Enhanced irrelevance detection update
Portuguese $100 and 100$ become cem dólares €100 and 100€ become cem euros 100% becomes cem por cento 
 Spanish $100 and 100$ become cien dólares €100 and 100€ become cien euros 100% becomes cien por ciento 



* Processes non-alphanumeric, punctuation, and special characters depending on their context. For example, the service removes a $ (dollar sign) or € (euro symbol) unless it is followed by a number. Processing is context-dependent and consistent across the supported languages.
* Ignores phrases that are enclosed in ( ) (parentheses), < > (angle brackets), [ ] (square brackets), or { } (curly braces).







 Parsing of Japanese 

The following information applies to Japanese:



* Converts all characters to full-width characters.
* Converts numbers to their equivalent words, for example, 500 becomes 五百, and 0.15 becomes 〇・一五.
* Does not convert tokens that include symbols to equivalent strings, for example, 100% becomes 百％.
* Does not automatically remove punctuation. IBM highly recommends that you remove punctuation if your application is transcription-based as opposed to dictation-based.







 Parsing of Korean 

The following information applies to Korean:



* Converts numbers to their equivalent words, for example, 10 becomes 십.
* Removes the following punctuation and special characters: - ( ) : . , ' ". However, not all punctuation and special characters that are removed for other languages are removed for Korean, for example:



* Removes a period (.) symbol only when it occurs at the end of a line of input.
* Does not remove a tilde () symbol.
* Does not remove or otherwise process Unicode wide-character symbols, for example, … (triple dot or ellipsis).



In general, IBM recommends that you remove punctuation, special characters, and Unicode wide-characters before you process a corpus file.
* Does not remove or ignore phrases that are enclosed in ( ) (parentheses), < > (angle brackets), [ ] (square brackets), or { } (curly braces).
Use the API to [list enrichments](https://cloud.ibm.com/apidocs/discovery-datalistenrichments) from the project where the custom enrichment is in use. The list that is returned includes enrichment ID information.

For prebuilt enrichments, the unique IDs do not change. The following table lists the IDs that are associated with each prebuilt enrichment type and identifies the collection languages for which the enrichment is supported. An enrichment cannot be applied to a collection unless the collection language is supported by the enrichment.



Prebuilt enrichment IDs

 Name Enrichment ID Supported languages 

 Contracts 701db916-fc83-57ab-0000-000000000014 en 
 Entities 701db916-fc83-57ab-0000-00000000001e ar, de, en, es, fr, it, ja, ko, nl, pt, zh-CN 
 Keywords 701db916-fc83-57ab-0000-000000000018 ar, de, en, es, fr, it, ja, ko, nl, pt, zh-CN 
 Part of Speech 701db916-fc83-57ab-0000-000000000002 All supported languages 
 Sentiment of Document 701db916-fc83-57ab-0000-000000000016 ar, de, en, es, fr, it, ja, ko, nl, pt, zh-CN 
 Table Understanding 701db916-fc83-57ab-0000-000000000012 All supported languages 



For more information about all of the supported languages, see [Language support](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-language-support).





 Apply enrichments by using the API 

To apply an enrichment by using the API, complete the following steps:



1. Determine the base URL for the endpoint and the token or API key for your deployment.
The service converts these keyword strings to symbols only in appropriate positions of a transcript. In the following example, the speaker says the word period at the end of the sentence:



* the warranty period is short period becomes the warranty period is short.



The service correctly differentiates between the noun that appears earlier in the sentence and the concluding punctuation.







 Japanese 



* Phone numbers must be 10 or 11 digits and begin with valid prefixes for telephone numbers in Japan. For example, valid prefixes include 03 and 090.
* English words are converted to ASCII (hankaku) characters. For example, ＩＢＭ is converted to IBM.
* Ambiguous terms might not be converted if sufficient context is unavailable. For example, it is unclear whether 一時 and 十分 refer to times.
* Punctuation is handled in the same way with or without smart formatting. For example, based on probability calculations, one of カンマ or , is selected.
* Strings that describe yen values are not replaced with the yen currency symbol.
* Internet email and web addresses in any form are not converted.
* The Japanese narrowband model (ja-JP_NarrowbandModel) includes some multigram word units for digits and decimal fractions. The service returns these multigram units regardless of whether you enable smart formatting. The following examples show the units that the service returns. The numbers in parentheses show the equivalent Arabic numeric expression for each unit.



* Digits:〇一 (01), ..., 〇九 (09), 一〇 (10), ..., 九〇 (90)
* Decimal fractions:〇・ (0.), 一・ (1.), ..., 十・ (10.)



The smart formatting feature understands and returns the multigram units that the model generates. If you apply your own post-processing to transcription results, you need to handle these units appropriately.









 Smart formatting results 

The following table shows examples of final transcripts both with and without smart formatting. The transcripts are based on US English audio.



Table 2. Smart formatting example transcripts

 Information Without smart formatting With smart formatting
Querying a single word from a custom model 

To query the translation of a single word from a custom model, use the GET /v1/customizations/{customization_id}/words/{word} method. You specify the word in the URL of the method. The translation is returned as it is defined in the custom model (sounds-like or phonetic).

The following example queries a custom model for the translation of the word IEEE:

IBM Cloud

curl -X GET -u "apikey:{apikey}" "{url}/v1/customizations/{customization_id}/words/IEEE"

IBM Cloud Pak for Data

curl -X GET --header "Authorization: Bearer {token}" "{url}/v1/customizations/{customization_id}/words/IEEE"

If the word has the sounds-like translation in the model, the example returns the following JSON output:

{
"translation": "I triple E"
}





 Querying all words from a custom model 

To see the translations for all of the words defined in a custom model, use the GET /v1/customizations/{customization_id}/words method. The following example uses the method to list the entries from a custom model that contains sounds-like translations for three words:

IBM Cloud

curl -X GET -u "apikey:{apikey}" "{url}/v1/customizations/{customization_id}/words"

IBM Cloud Pak for Data

curl -X GET --header "Authorization: Bearer {token}" "{url}/v1/customizations/{customization_id}/words"

The method returns a JSON array with the following data. For Japanese custom models, the output includes the part of speech for individual words.

{
"words": [
{
"word": "IEEE",
"translation": "I triple E"
},
{
"word": "NCAA",
"translation": "N C double A"
},
{
"word": "iPhone",