vania-janet/multiturn-rag-retrieval-data huggingface dataset:
For more information, see [Listing custom words from a custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-manageWordslistWords).

For custom models that are based on next-generation models, the service ignores the word_type_to_add parameter. The words resource contains only custom words that the user adds or modifies directly, so the parameter is unnecessary.
* The customization_weight parameter specifies the relative weight that is given to words from the custom model as opposed to words from the base vocabulary when the custom model is used for speech recognition. You can also specify a customization weight with any recognition request that uses the custom model. For more information, see [Using customization weight](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageUseweight).
* The strict parameter indicates whether training is to proceed if the custom model contains a mix of valid and invalid resources (corpora, words, and grammars). By default, training fails if the model contains one or more invalid resources. Set the parameter to false to allow training to proceed as long as the model contains at least one valid resource. The service excludes invalid resources from the training. For more information, see [Training failures for custom language models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreatefailedTraining-language).





 Monitoring the train model request 

The service returns a 200 response code if it initiates the training process successfully. The service cannot accept subsequent training requests, or requests to add new corpora, words, or grammars, until the existing request completes.

Adding custom words directly to a custom model that is based on a next-generation model, as described in [Add words to the custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreateaddWords), causes training of a model to take a few minutes longer than it otherwise would.
For more information, see [High availability for IBM Cloud services](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-high-availabilityibm-cloud-services).<br><br><br> <br><br> * Implement a system of high availability for all components in your IaaS / data center. For example, deploy Satellite hosts across multiple availability zones. For more information, see [High Availability, Disaster Recovery, and Disconnected Usage](https://cloud.ibm.com/docs/satellite?topic=satellite-h).<br><br><br> 
 [Use endpoint detection and remediation (EDR) tooling to detect malicious code](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-best-practicesbest-practices-endpoint-detection-remediation) <br><br> * Run regular, automated scans of the components in your environment to detect vulnerabilities so that they can be mitigated.<br><br><br> <br><br> * Run regular, automated scans of the components in your IaaS / data center environment to detect vulnerabilities so that they can be mitigated.<br><br><br> 
 [Regularly scan for open ports / protocols](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-best-practicesbest-practices-network-threat-detection) <br><br> * Regularly scan to make sure the list of open ports / protocols reflects the minimum number of ports / protocols that are needed by your workloads.<br><br><br> <br><br> * Regularly scan to make sure the list of open ports / protocols reflects the minimum number of ports / protocols that are needed within your IaaS / data center environment.<br><br><br>
2022/04/13 13:05:46 ----- Terraform Commands -----
2022/04/13 13:05:46 Could not execute job: Error : Drift cannot be executed since state file doesn't exist. Please run terraform apply to generate state file.









 Drift detection using the CLI 

You can initiate detecting drift from the create workspaces command. The drift detection initiates a job to detect drift for the workspace and its specific resources. The drift detection job is in progress or completed with the appropriate status such as failure or success. Instead, to know the details of the drift job, you need to check the drift job log for the drift status. Use the following commands to view the detect drift.



 Creating and viewing the detect drift logs using the CLI 

You can follow these steps to detect the drift in Schematics Workspaces through CLI.



1. [Create the Schematics Workspaces](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-new).
2. [Get your workspace ID](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-get).
3. Run the [ibmcloud schematics plan](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-plan).
4. Fetch the [ibmcloud schematics job logs](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-logs-job).
5. Run the [ibmcloud schematics apply](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-apply).
6.
A relationship between two words or phrases in which both refer to the same person or thing and one stands as a linguistic antecedent of the other. For example, there is a coreference between the two pronouns in the phrase "She taught herself" but not in the phrase "She taught her". A coreference links two equivalent entities in the same text.
* coreference chain

A list of entities that were annotated as coreferences. When you annotate mentions as coreferences, the system creates a coreference chain. The chain provides a way for you to view all of the mentions in context and verify that all of the occurrences belong together under the same entity type.
* corpus

A collection of source documents that were added to a workspace and used to train a machine learning model.
* curate

To select, collect, preserve, and maintain content relevant to a specific topic. Curation establishes, maintains, and adds value to data; it transforms data into trusted information and knowledge.







 D 



* dictionary

A collection of words that can be used to pre-annotate documents. A new annotation is created for each word in the document text that matches a term in the dictionary. A machine learning model can be configured with one or more independent dictionaries, which are typically domain-specific, such a dictionary for pharmaceuticals and a dictionary for wealth management. See also [lemma](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-glossarygloss_L) and [surface form](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-glossarygloss_S).
* dictionary pre-annotator

A component that identifies mentions in text that match a specific set of words. By using domain-specific terminology to pre-annotate text, dictionary pre-annotators can accelerate a human annotator's ability to prepare a set of ground truth documents.
* document set

A collection of documents. Documents that are imported together become a document set.
* The second example adds the same words from a file named words.json:

{
"words": [
{"word": "HHonors", "sounds_like": "hilton honors", "H. honors"], "display_as": "HHonors"},
{"word": "IEEE", "sounds_like": "I. triple E."]}
]
}

The following request adds the words from the file:

IBM Cloud

curl -X POST -u "apikey:{apikey}" --header "Content-Type: application/json" --data-binary @words.json "{url}/v1/customizations/{customization_id}/words"

IBM Cloud Pak for Data

curl -X POST --header "Authorization: Bearer {token}" --header "Content-Type: application/json" --data-binary @words.json "{url}/v1/customizations/{customization_id}/words"



The POST method is asynchronous. It can take on the order of minutes to complete. The time that it takes to complete depends on the number of words that you add and the current load on the service. For more information about checking the status of the operation, see [Monitoring the add words request](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreatemonitorWords).





 Add a word with the PUT method 

The PUT /v1/customizations/{customization_id}/words/{word_name} method adds individual words. You pass a JSON object that provides information about the word as the body of the request.

The following example adds the word NCAA to the model with the specified ID. The required Content-Type header again indicates that JSON data is being passed to the method.

IBM Cloud
Add a corpus to the custom language model 

Once you create your custom language model, the next step is to add domain-specific data to the model. The recommended means of populating a custom model is to add one or more corpora. A corpus is a plain text file that ideally contains sample sentences from your domain.



* For custom models that are based on previous-generation models, the service parses a corpus file and extracts any words that are not in its base vocabulary. Such words are referred to out-of-vocabulary (OOV) words. For more information about using corpora with custom models that are based on previous-generation models, see [Working with corpora for previous-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWordsworkingCorpora).
* For custom models that are based on next-generation models, the service parses and extracts character sequences from a corpus file. The characters help the service learn and predict character sequences from audio. For more information about using corpora with custom models that are based on next-generation models, see [Working with corpora for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWords-ngworkingCorpora-ng).



By providing sentences that include domain-specific words, corpora allow the service to learn the words and character sequences in context. You can also augment and modify a model's words individually. Training a model only on individual words as opposed to words added from corpora is more time-consuming and can produce less effective results.

You use the POST /v1/customizations/{customization_id}/corpora/{corpus_name} method to add a corpus to a custom model:

customization_id (required string)
: Specify the customization ID of the custom model to which the corpus is to be added.

corpus_name (required string)
: Specify a name for the corpus. Use a localized name that matches the language of the custom model and reflects the contents of the corpus.



* Include a maximum of 128 characters in the name.
These operations can cause a short blip in connectivity, so you might also see a short connection interruption.

You want to design your applications to handle a temporary loss in connectivity to your deployment or to IBM Cloud.

The RabbitMQ Documentation has an overview of what sorts of things you can do to help make your applications robust and stable in its [Production Checklist](https://www.rabbitmq.com/production-checklist.htmlapps), under the Applications section. It includes some general connection management and recovery advice.

RabbitMQ and RabbitMQ drivers support various features to help you design a resilient application.



* [Consumer Acknowledgments and Publisher Confirms](https://www.rabbitmq.com/confirms.html) - to ensure that messages are sent and received, and to catch and recover from situations where they are not.
* [Heartbeats and TCP Keepalives](https://www.rabbitmq.com/heartbeats.html) - using heartbeat and keepalive parameters when your application connects can detect and prevent zombie connections.
* [Consumer Cancel Notifications](https://www.rabbitmq.com/consumer-cancel.html) - detect and handle instances where a consumer stops consuming from a queue.



Messages for RabbitMQ also comes with the [Shovel plug-in](https://www.rabbitmq.com/shovel.html). Shovels can handle connecting, reading and writing messages, and connection failure and republishing.

Several minutes of database unavailability or connection interruption are not expected. Open a [support ticket](https://cloud.ibm.com/unifiedsupport/cases/add) with details if you have time periods longer than a minute with no connectivity so we can investigate.





 Messages for RabbitMQ Connection Limits 

At provision, Messages for RabbitMQ has four different endpoints and sets the maximum number of connections per node and for each endpoint as follows:



* HTTPS: 1024 x 3 nodes = 3072
* AMQPS: 20000 x 3 nodes = 60000
* MQTTS: 20000 x 3 nodes = 60000
* STOMPS: 20000 x 3 nodes = 60000
Japanese custom words for custom models that are based on improved next-generation models are handled differently
: For Japanese custom models that are based on next-generation models, custom words are handled differently from other languages. For Japanese, you can add a custom word or sounds-like that does not exceed 25 characters in length. If your custom word or sounds-like exceeds that limit, the service adds the word to the custom model as if it were added by a corpus. The word does not appear as a custom word for the model.

For more information, see [Guidelines for adding words to Japanese models based on improved next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWords-ngwordsResourceAmount-japanese-ng).





 12 April 2023 

Defect fix: The WebSocket interface now times out as expected when using next-generation models
: Defect fix: When used for speech recognition with next-generation models, the WebSocket interface now times out as expected after long periods of silence. Previously, when used for speech recognition of short audio files, the WebSocket session could fail to time out. When the session failed to time out, the service did not return a final hypothesis to the waiting client application, and the client instead timed while waiting for the results.





 6 April 2023 

Defect fix: Limits to allow completion of training for next-generation Japanese custom models
: Defect fix: Successful training of a next-generation Japanese custom language model requires that custom words and sounds-likes added to the model each contain no more than 25 characters. For the most effective training, it is recommended that custom words and sounds-likes contain no more than 20 characters. Training of Japanese custom models with longer custom words and sounds-likes fails to complete after multiple hours of training.

If you need to add the equivalent of a long word or sounds-like to a next-generation Japanese custom model, take these steps:



1. Add a shorter word or sounds-like that captures the essence of the longer word or sounds-like to the custom model.
2. Add one or more sentences that use the longer word or sounds-like to a corpus.
3.
Adding a grammar to a custom language model 

Before you can use a grammar for speech recognition, you must first use the customization interface to add the grammar to a custom language model. The steps to add a grammar to a custom language model parallel those used to add corpora or custom words:



1. [Create a custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarAddcreateModel-grammar). You can create a new custom model or use an existing model.
2. [Add a grammar to the custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarAddaddGrammar). The service validates the grammar to ensure its correctness.
3. [Validate the words from the grammar](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarAddvalidateGrammar). You verify the correctness of the sounds-like pronunciations for any out-of-vocabulary (OOV) words that are recognized by the grammar.
4. [Train the custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarAddtrainGrammar). The service prepares the custom model and grammar for use in speech recognition.
5. You can now use the custom model and grammar in speech recognition requests. For more information, see [Using a grammar for speech recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarUse).



These steps are iterative. You can add grammars, as well as corpora and custom words, to a custom language model as often as needed. You must train the custom model on any new data resources (grammars, corpora, or custom words) that you add. When you use it for speech recognition, a custom model reflects the data on which it was last trained.
Working with corpora and custom words for next-generation models 

This information is specific to custom models that are based on next-generation models. For information about corpora and custom words for custom models that are based on previous-generation models, see [Working with corpora and custom words for previous-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWords).

You populate a custom language model with words by adding corpora to the model or by adding custom words directly to the model. You use the same methods and operations for both previous- and next-generation models. For more information about adding corpora and custom words to a model, see [Working with corpora for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWords-ngworkingCorpora-ng) and [Working with custom words for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWords-ngworkingWords-ng).

Although language model customization is similar in usage and intent for previous- and next-generation models, there are differences between the two types of models at the implementation level. To understand how language model customization works for next-generation models, and how you can make the best use of customization, you need a high-level understanding of the differences.



* When you create and use a custom language model that is based on a previous-generation model, the service relies on words from the custom model to create transcripts that contain domain-specific terms. In combination with words from its base vocabulary, the service uses these words from the custom model to predict and transcribe speech from audio. You provide the information for a custom language model in the form of corpora, custom words, and grammars. The service stores this information in the words resource for the custom model.
* When you create a custom language model that is based on a next-generation model, the services relies on sequences of characters from the custom model to create transcripts that reflect domain-specific terms.
ibmcloud cr digests --format '{{if eq .Digest "sha256:1d71e323557502cc78ee6c237331a09b0c33ba59c14e5f683da3b1c6218779cc"}}{{.Repository}}@{{.Digest}}{{end}}'
icr.io/myrepo2/image2@sha256:1d71e323557502cc78ee6c237331a09b0c33ba59c14e5f683da3b1c6218779cc
2. If the previous command returns an image, you can copy it to the same repository as the manifest list.

ibmcloud cr image-tag icr.io/mynamespace/myrepo2@sha256:1d71e323557502cc78ee6c237331a09b0c33ba59c14e5f683da3b1c6218779cc icr.io/mynamespace/myrepo:ppc64le



* If the missing image was deleted in the last 30 days, you can restore it from the trash.



1. Detect if the image exists in trash by running the following command.

ibmcloud cr trash-list --restrict mynamespace

You receive the following response:

Listing the contents of the trash...

Digest Days until expiry Tags
icr.io/mynamespace/myrepo@sha256:1d71e323557502cc78ee6c237331a09b0c33ba59c14e5f683da3b1c6218779cc 30 ppc64le

OK
2. If the image does exist in trash, you can restore it by running the following command.
For example, the French word garçon contains a character that is not found in the English language. You can specify a sounds-like of gaarson, replacing ç with s, to tell the service how English speakers would pronounce the word.



The following sections provide language-specific guidelines for specifying a sounds-like pronunciation. Speech recognition uses statistical algorithms to analyze audio, so adding a word does not guarantee that the service transcodes it with complete accuracy. When you add a word, consider how it might be pronounced. Use the sounds_like field to provide various pronunciations that reflect how a word can be spoken.



 Guidelines for English 

Guidelines for Australian, United Kingdom, and United States English:



* Use English alphabetic characters: a-z and A-Z.
* Use real or made-up words that are pronounceable in English for words that are difficult to pronounce, for example, shuchesnie for the word Sczcesny.
* Substitute equivalent English letters for non-English letters, for example, s for ç or ny for ñ.
* Substitute non-accented letters for accented letters, for example, a for à or e for è.
* You can include multiple words that are separated by spaces. The service enforces a maximum of 40 total characters, not including leading or trailing spaces.



Guidelines for Australian and United States English only:



* To pronounce a single letter, use the letter followed by a period. If the period is followed by another character, be sure to use a space between the period and the next character. For example, use N. C. A. A., notN.C.A.A.
* Use the spelling of numbers, for example, seventy-five for 75.



Guidelines for United Kingdom English only:



* You cannot use periods or dashes in sounds-like pronunciations for UK English.
* To pronounce a single letter, use the letter followed by a space. For example, use N C A A, notN. C. A. A., N.C.A.A., or NCAA.
* Use the spelling of numbers without dashes, for example, seventy five for 75.
* The service supports a synthesis feature that lets you control how alphanumeric strings are spelled out for German voices. For more information, see [Specifying how strings are spelled out](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-synthesis-paramsparams-spell-out-mode).
* The service supports the use of the SSML <mark> element with the WebSocket interface to obtain timing information for words of the resulting audio. The WebSocket interface also allows you to request information for all strings of the input text. For more information, see



* [Specifying an SSML mark](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-timingtiming-mark)
* [Requesting word timings for all words](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-timingtiming-request)



* The service's customization interface supports the use of the SSML <phoneme> element to specify the phonetic spelling that it uses to pronounce a word. The phonetic spelling represents the sounds of a word, how those sounds are divided into syllables, and which syllables receive stress.



* For information about the customization interface, see [Understanding customization](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-customIntro).
* For information about the valid symbols that you can use in an International Phonetic Alphabet (IPA) or the IBM Symbolic Phonetic Representation (SPR) specification for any supported language, see [Understanding phonetic symbols](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-symbols).









 SSML validation 

The service validates all SSML elements that you submit in any content, either as input text for synthesis or as the definition of a word's translation for customization. The service cannot determine ahead of time whether text submitted for synthesis contains SSML elements. Therefore, it performs the same validation for all input text, regardless of whether it contains SSML.