vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Note: Enterprise: Set to any value between 100 KiB and 2 TiB. Standard: Set to any value between 100 KiB and 512 MiB.

segment.index.bytes
: The size of the index that maps offsets to file positions.

Note: Enterprise: Set to any value between 100 KiB and 1 TiB. Standard: Set to any value between 100 KiB and 100 MiB.

segment.ms
: The period of time after which Kafka will force the log to roll even if the segment file isn't full.

Note: Set to any value between 5 minutes and 30 days.

See the following example of default value settings.

Details for topic testit
Topic name Internal? Partition count Replication factor
testit false 1 3

Partition details for topic testit
Partition ID Leader Replicas In-sync
0 1 [1 5 0] [1 5 0]

Configuration parameters for topic testit
Name Value
cleanup.policy delete
min.insync.replicas 2
segment.bytes 536870912
retention.ms 86400000
segment.ms 604800000
retention.bytes 1073741824
segment.index.bytes 10485760
* How long does Event Streams set the log retention window for the consumer offsets topic?

Event Streams retains consumer offsets for 7 days. This corresponds to the Kafka configuration offsets.retention.minutes.

Offset retention is system-wide so you cannot set it at an individual topic level. All consumer groups get only 7 days of stored offsets even if using a topic with a log retention that has been increased to the maximum of 30 days.

The internal Kafka __consumer_offsets topic is visible to you as read-only on the Enterpise plan. You are strongly recommended not to attempt to manage the topic in any way. You cannot access the __consumer_offsets topic in any way on the Standard plan.
* How can I clean up a consumer group with no consumers?

After consumers have left, a group continues to exist only if it has offsets. Consumer offsets are deleted after 7 days of inactivity.
Watson How many people will be dining? 
 You 6 



This time Watson responds with, OK. I am making you a reservation for 6 on Friday, December 29 at 5:00 PM.



You have successfully improved the format that the dialog uses when it references context variable values in its responses. The dialog now uses Friday, December 29 instead of the more technical, 2017-12-29. And it uses 5:00 PM instead of 17:00:00. To learn about other SpEL methods you can use with date and time values, see [Methods to process values](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-methodsdialog-methods-date-time).





 Step 2: Ask for everything at once 

Now that you have tested the dialog more than once, you might have noticed that it can be annoying to have to answer one slot prompt at a time. To prevent users from having to provide one piece of information at a time, you can ask for every piece of information that you need up front. Doing so gives the user a chance to provide all or some of the information in a single input.

The node with slots is designed to find and save any and all slot values that the user provides while the current node is being processed. You can help users to take advantage of the design by letting them know what values to specify.

In this step, you will learn how to prompt for everything at once.



1. From the main node with slots, click Customize.
2. Select the Prompt for everything checkbox to enable the intial prompt, and then click Apply.



![Shows the customize dialog where you select the Prompt for everything checkbox.](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/slots-prompt-for-everything.png)



1. Back in the node edit view, scroll down to the newly-added If no slots are pre-filled, ask this first field.
[Open conversation panel that shows the original text for a term to which spelling correction logic was applied](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/open_convo_spellchecked.jpg)





 Improving across assistants 

Creating a dialog skill is an iterative process. While you develop your skill, you use the Try it out pane to verify that your assistant recognizes the correct intents and entities in test inputs, and to make corrections as needed.

From the User conversations page, you can analyze actual interactions between the assistant you used to deploy the skill and your users. Based on those interactions, you can make corrections to improve the accuracy with which intents and entities are recognized by your dialog skill. It is difficult to know exactly how your users will ask questions, or what random messages they might submit, so it is important to frequently analyze real conversations to improve your dialog skills.

For a Watson Assistant instance that includes multiple assistants, there might be times when it is useful to use message data from the dialog skill of one assistant to improve the dialog skill used by another assistant within that same instance.

As an example, say you have a Watson Assistant instance named HelpDesk. You might have both a Production assistant and a Development assistant in your HelpDesk instance. When working in the dialog skill for the Development assistant, you can use logs from the Production assistant messages to improve the Development assistant's dialog skill.

Any edits you then make within the dialog skill for the Development assistant will only affect the Development assistant's dialog skill, even though you’re using data from messages sent to the Production assistant.

Similarly, if you create multiple versions of a skill, you might want to use message data from one version to improve the training data of another version.

You cannot access log data from assistants that were created in other service instances.



 Picking a data source 

The term data source refers to the logs compiled from conversations between customers and the assistant or custom application by which a dialog skill was deployed.
![Create a topic part 3.](https://cloud.ibm.com/docs-content/v1/content/220cdcbf6f4421bd7fee83fd1028048dba3f16c8/monitoring/images/streaming-topic-2.png)

Enter number of partitions
8. Select a Message retention and click Create Topic.

Message retention defines how long messages are retained before they are deleted. If your messages are not read by a consumer within this time, they will be missed.







 Step 2. Create credentials to authenticate the monitoring instance with Event Streams 

You need the following information to connect the Monitoring instance to the Event Streams instance:



* Endpoint URLs to call the APIs
* Credentials for authentication



Complete the following steps to create service credentials that the Monitoring instance needs to communicate with the Event Streams instance:



1. In the IBM Cloud, click the Menu icon ![Menu icon](https://cloud.ibm.com/docs-content/v1/content/220cdcbf6f4421bd7fee83fd1028048dba3f16c8/icons/icon_hamburger.svg) > Resource list.
2. Look for the Event Streams instance that you plan to use, and select it.
3. In the Event Streams console, click Service credentials.
4. Select New credential.
5. Enter a name and select the writer role.

Zoom

![Create a credential](https://cloud.ibm.com/docs-content/v1/content/220cdcbf6f4421bd7fee83fd1028048dba3f16c8/monitoring/images/streaming-credentials.png)

Create a credential
6. Click Add.



To restrict access to 1 topic, complete the following steps:



1. From the menu bar, click Manage > Access (IAM), and select Service IDs.

Zoom

!
The service returns the following response with information about the prompt, including its initial status:

{
"prompt": "Thank you and good-bye!",
"prompt_id": "goodbye",
"status": "processing",
"speaker_id": "823068b2-ed4e-11ea-b6e0-7b6456aa95cc"
}

Adding a prompt is an asynchronous operation. For more information about checking the status of a new prompt, see [Monitoring the add prompt request](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-tbe-createtbe-create-add-prompt-monitor).



 Monitoring the add prompt request 

The service returns a 201 response code if it successfully initiates the request to add the prompt. Although adding a prompt accepts less audio than speaker enrollment, the service must align the audio with the provided text. The time that it takes to process a prompt depends on the length of the audio for the prompt. The processing time generally matches the length of the audio (for example, it takes 20 seconds to process a 20-second prompt).

For shorter prompts, you can wait for a reasonable amount of time and then check the status of the prompt with the GET /v1/customizations/{customization_id}/prompts/{prompt_id} method. For longer prompts, consider using that method to poll the service every few seconds to determine when the prompt is ready. The following example checks the status of the goodbye prompt:

IBM Cloud

curl -X GET -u "apikey:{apikey}" "{url}/v1/customizations/82f4809a-bf63-89a6-52ca-22731fe467ba/prompts/goodbye"

IBM Cloud Pak for Data

curl -X GET --header "Authorization: Bearer {token}" "{url}/v1/customizations/82f4809a-bf63-89a6-52ca-22731fe467ba/prompts/goodbye"
Training of Japanese custom models with longer custom words and sounds-likes fails to complete after multiple hours of training.

If you need to add the equivalent of a long word or sounds-like to a next-generation Japanese custom model, take these steps:



1. Add a shorter word or sounds-like that captures the essence of the longer word or sounds-like to the custom model.
2. Add one or more sentences that use the longer word or sounds-like to a corpus.
3. Consider adding sentences to the corpus that provide more context for the word or sounds-like. Greater context gives the service more information with which to recognize the word and apply the correct sounds-like.
4. Add the corpus to the custom model.
5. Retrain the custom model on the combination of the shorter word or sounds-like and the corpus that contains the longer string.



The limits and steps just described allow next-generation Japanese custom models to complete training. Keep in mind that adding large numbers of new custom words to a custom language model increases the training time of the model. But the increased training time occurs only when the custom model is initially trained on the new words. Once the custom model has been trained on the new words, training time returns to normal.

For more information, see



* [Add a corpus to the custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreateaddCorpus)
* [Add words to the custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreateaddWords)
* [Train the custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreatetrainModel-language)
* [Working with corpora and custom words for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWords-ng)



Further improvements to updated next-generation language model customization
The default value is 0.2 for most next-generation models; it is 0.1 for models that are based on new language model customization technology. For more information, see [Customization support for next-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-supportcustom-language-support-ng). 
 WebSocket Parameter of JSON start message 
 Synchronous HTTP Query parameter of POST /v1/recognize method 
 Asynchronous HTTP Query parameter of POST /v1/recognitions method 







 end_of_phrase_silence_time 

An optional double between 0.0 and 120.0 that indicates the pause interval at which the service splits a transcript into multiple final results if it encounters silence. By default, the service uses a pause interval of 0.8 seconds for all languages other than Chinese, for which it uses an interval of 0.6 seconds. For more information, see [End of phrase silence time](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-parsingsilence-time).



Table 9. The end_of_phrase_silence_time parameter

 Availability and usage Description 

 Previous-generation models Generally available for all languages. 
 Next-generation models Generally available for all languages. 
 WebSocket Parameter of JSON start message 
 Synchronous HTTP Query parameter of POST /v1/recognize method 
 Asynchronous HTTP Query parameter of POST /v1/recognitions method 







 grammar_name 

An optional string that identifies a grammar that is to be used for speech recognition. The service recognizes only strings that are defined by the grammar. You must specify both the name of the grammar and the customization ID of the custom language model for which the grammar is defined. For more information, see [Using a grammar for speech recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarUse).



Table 10. The grammar_name parameter

 Availability and usage Description
conversation.synonym.delete deletes a synonym that is associated with an entity value. 
 conversation.synonym.update edits a synonym that is associated with an entity value. 
 conversation.userdata.delete deletes data that was created by a specified customer. 
 conversation.value.create creates an entity value. 
 conversation.value.delete deletes an entity value. 
 conversation.value.update edits an entity value. 
 conversation.workspace.create creates a workspace. 
 conversation.workspace.delete deletes a workspace. 
 conversation.workspace.update makes changes to a workspace.
For more information, see the [Text to Speech documentation](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-elementsbreak_element).







 Fields 



 Name Type Description Required? 

 response_type string pause Y 
 time int How long to pause, in milliseconds. Y 
 typing boolean Whether to send the "user is typing" event during the pause. Ignored if the channel does not support this event. N 







 Example 

This examples sends the "user is typing" event while pausing for 5 seconds.

{
"output": {
"generic":[
{
"response_type": "pause",
"time": 5000,
"typing": true
}
]
}
}







 speech_to_text 

Sends a command to the Speech to Text service instance used by the phone integration. These commands can dynamically change the configuration or behavior of the service during a conversation.



 Integration channel support 



 Phone 

 ![Yes](https://cloud.ibm.com/docs-content/v1/content/ca9232b3702768f24bb897f48f10778fcda28718/watson-assistant/images/checkmark-icon.svg) 







 Fields 



 Name Type Description Required? 

 response_type string speech_to_text Y 
 command_info object Information specifying the command to send to the Speech to Text. Y 
 command_info.type string The command to send (currently only the configure command is supported). Y 
 command_info.parameters object See [Applying advanced settings to the Speech to Text service](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-phone-actionsphone-actions-speech-advanced) N 



The command_info.type field can specify any of the following supported commands:



* configure: Dynamically updates the Speech to Text configuration. Configuration changes can be applied only to the next conversation turn, or for the rest of the session.
[Open conversation panel that shows the original text for a term to which spelling correction logic was applied](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/open_convo_spellchecked.jpg)





 Improving across assistants 

Creating a dialog skill is an iterative process. While you develop your skill, you use the Try it out pane to verify that your assistant recognizes the correct intents and entities in test inputs, and to make corrections as needed.

From the User conversations page, you can analyze actual interactions between the assistant you used to deploy the skill and your users. Based on those interactions, you can make corrections to improve the accuracy with which intents and entities are recognized by your dialog skill. It is difficult to know exactly how your users will ask questions, or what random messages they might submit, so it is important to frequently analyze real conversations to improve your dialog skills.

For a Watson Assistant instance that includes multiple assistants, there might be times when it is useful to use message data from the dialog skill of one assistant to improve the dialog skill used by another assistant within that same instance.

As an example, say you have a Watson Assistant instance named HelpDesk. You might have both a Production assistant and a Development assistant in your HelpDesk instance. When working in the dialog skill for the Development assistant, you can use logs from the Production assistant messages to improve the Development assistant's dialog skill.

Any edits you then make within the dialog skill for the Development assistant will only affect the Development assistant's dialog skill, even though you’re using data from messages sent to the Production assistant.

Similarly, if you create multiple versions of a skill, you might want to use message data from one version to improve the training data of another version.

You cannot access log data from assistants that were created in other service instances.



 Picking a data source 

The term data source refers to the logs compiled from conversations between customers and the assistant or custom application by which a dialog skill was deployed.
Watson How many people will be dining? 
 You 6 



This time Watson responds with, OK. I am making you a reservation for 6 on Friday, December 29 at 5:00 PM.



You have successfully improved the format that the dialog uses when it references context variable values in its responses. The dialog now uses Friday, December 29 instead of the more technical, 2017-12-29. And it uses 5:00 PM instead of 17:00:00. To learn about other SpEL methods you can use with date and time values, see [Methods to process values](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-dialog-methodsdialog-methods-date-time).





 Step 2: Ask for everything at once 

Now that you have tested the dialog more than once, you might have noticed that it can be annoying to have to answer one slot prompt at a time. To prevent users from having to provide one piece of information at a time, you can ask for every piece of information that you need up front. Doing so gives the user a chance to provide all or some of the information in a single input.

The node with slots is designed to find and save any and all slot values that the user provides while the current node is being processed. You can help users to take advantage of the design by letting them know what values to specify.

In this step, you will learn how to prompt for everything at once.



1. From the main node with slots, click Customize.
2. Select the Prompt for everything checkbox to enable the intial prompt, and then click Apply.



![Shows the customize dialog where you select the Prompt for everything checkbox.](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/slots-prompt-for-everything.png)



1. Back in the node edit view, scroll down to the newly-added If no slots are pre-filled, ask this first field.
For more information about using the WebSocket interface to obtain interim results with previous- and next-generation models, see the following topics:



* [Interim results](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interiminterim-results)
* [Requesting interim results and low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interiminterim-low-latency)
* [How the service sends recognition results](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-websocketsws-results)







 The result_index field 

The result_index field provides an identifier for the results that is unique for that request. If you request interim results, the service sends multiple results fields for evolving hypotheses of the input audio. The indexes for interim results for the same audio always have the same value, as do the final results for the same audio.

The same index can also be used for multiple final results of a single request. Regardless of whether you request interim results, the service can return multiple final results with the same index if your audio includes pauses or extended periods of silence. For more information, see [Pauses and silence](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-basic-responseresponse-pauses-silence).

Once you receive final results for any audio, the service sends no further results with that index for the remainder of the request. The index for any further results is incremented by one.

If your audio produces multiple final results, concatenate the transcript elements of the final results to assemble the complete transcription of the audio. Assemble the results in the order in which you receive them. When you assemble a complete final transcript, you can ignore interim results for which the final field is false.





 Additional response content 

Many speech recognition parameters impact the contents of the service's response. Some parameters can cause the service to return multiple transcription results:



* end_of_phrase_silence_time
This figure includes OOV words from all sources (corpora, grammars, and individual custom words that you add directly). You can add a maximum of 10 million total words to a custom model from all sources. For more information, see [How much data do I need?](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWordswordsResourceAmount).





 3 April 2019 

New limits on maximum amount of audio for custom acoustic models
: Custom acoustic models now accept a maximum of 200 hours of audio. The previous maximum limit was 100 hours of audio.





 21 March 2019 

Visibility of service credentials now restricted by role
: Users can now see only service credential information that is associated with the role that has been assigned to their IBM Cloud account. For example, if you are assigned a reader role, any writer or higher levels of service credentials are no longer visible.

This change does not affect API access for users or applications with existing service credentials. The change affects only the viewing of credentials within IBM Cloud.





 15 March 2019 

New support for A-law audio format
: The service now supports audio in the A-law (audio/alaw) format. For more information, see [audio/alaw format](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-audio-formatsaudio-formats-alaw).





 11 March 2019 

Change to passing value of 0 for max_alternatives parameter
: For the max_alternatives parameter, the service again accepts a value of 0. If you specify 0. the service automatically uses the default value, 1. A change made for the March 4 service update caused a value of 0 to return an error. (The service returns an error if you specify a negative value.)

Change to passing value of 0 for word_alternatives_threshold parameter
: For the word_alternatives_threshold parameter, the service again accepts a value of 0 . A change made for the March 4 service update caused a value of 0 to return an error. (The service returns an error if you specify a negative value.)