vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* For more information about the low_latency parameter, see [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency).
* For more information about the interaction between the low_latency and interim_results parameters for next-generation models, see [Requesting interim results and low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interiminterim-low-latency).



Arabic language broadband model renamed
: The Arabic language broadband model is now named ar-MS_BroadbandModel. The former name, ar-AR_BroadbandModel, is deprecated. It will continue to function for at least one year but might be removed at a future date. You are encouraged to migrate to the new name at your earliest convenience.

Unified Speech to Text documentation
: The documentation for IBM Watson Speech to Text for IBM Cloud Pak for Data is now combined with the documentation for managed instances of the Speech to Text service that are hosted on IBM Cloud. This is true of both the guide and reference documentation for the two forms of the service. Links to the formerly separate version of the IBM Cloud Pak for Data documentation for the service redirect to the unified documentation.

For more information about identifying information that pertains to only one version of the product, see [About Speech to Text](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-aboutabout-version).

Defect fix: Improve documentation
: Defect fix: The documentation has been updated to correct the following information:



* The documentation failed to state that next-generation models do not produce hesitation markers. The documentation has been updated to note that only previous-generation models produce hesitation markers. Next-generation models include the actual hesitations in transcription results. For more information, see [Speech hesitations and hesitation markers](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-basic-responseresponse-hesitation).
The statistics can cover a longer time period than the period for which logs of conversations are retained.

![Time period control](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/oview-time.png)

You can choose whether to view data for a single day, a week, a month, or a quarter. In each case, the data points on the graph adjust to an appropriate measurement period. For example, when viewing a graph for a day, the data is presented in hourly values, but when viewing a graph for a week, the data is shown by day. A week always runs from Sunday through Saturday.

You can create custom time periods also, such as a week that runs from Thursday to the following Wednesday, or a month that begins on any date other than the first.

The time shown for each conversation is localized to reflect the time zone of your browser. However, API log calls are always shown in UTC time. As a result, if you choose a single day view, for example, the time shown in the visualization might differ from the timestamp specified in the log for the same conversation.

![Time period control](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/oview-time2.png)
* Intents and Entities filters - Use either of these drop-down filters to show data for a specific intent or entity in your skill.

The intent and entities filters are populated by the intents and entities in the skill, and not what is in the data source. If you have [selected a data source](https://cloud.ibm.com/docs/assistant?topic=assistant-logslogs-deploy-id) other than the skill, you might not see an intent or entity from your data source logs as an option in the filters, unless those intents and entities are also in the skill.
* [Requesting interim results and low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interiminterim-low-latency)
* [How the service sends recognition results](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-websocketsws-results)





Silence of 30 seconds in streamed audio can result in an inactivity timeout. For more information, see [Timeouts](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-inputtimeouts).





 Speech hesitations and hesitation markers 

Speech often includes hesitations or verbal pauses, which are also referred to as disfluencies. Hesitations occur when the user inserts fillers such as "uhm", "uh", "hmm", and related non-lexical utterances while speaking. The service handles hesitations differently for previous- and next-generation models.



 Hesitations for previous-generation models 

For previous-generation models, the service includes hesitation markers in transcription results for most languages. Different languages can use different hesitation markers or not indicate hesitations at all:



* For US English, hesitation markers are indicated by the token %HESITATION. Words that generate hesitation markers are aah, ah, hm, hmm, huh, huh-uh, hum, ohh, ugh, uh, uh-huh, uh-oh, uh-uh, um, and um-hum.
* For Japanese, hesitation markers typically begin with D_.
* For Spanish, the service does not produce hesitation markers.



The following example shows the token %HESITATION for a US English transcript:

{
"result_index": 0,
"results": [
{
"alternatives":
{
"confidence": 0.99,
"transcript": ". . . that %HESITATION that's a . . ."
}
],
"final": true
}
]
}
Using the administration Kafka Java client API 

If you use a Kafka client version 0.11 or later, or Kafka Streams version 0.10.2.0 or later, you can use APIs to create and delete topics. We put some restrictions on the settings that are allowed when you create topics. See the following settings that you can modify.

cleanup.policy
:   Set to delete (default), compact or delete,compact.

retention.ms
:   The default retention period is 24 hours. The minimum is 1 hour and the maximum is 30 days. Specify this value as multiples of hours.

In the Enterprise plan, you can set retention to any value.

retention.bytes
:   The maximum size a partition (which consists of log segments) can grow to before we discard old log segments to free up space.

Enterprise plan only. Set to any value larger than 10 MB.

segment.bytes
:   The segment file size for the log.

Enterprise plan only. Set to any value larger than 100 kB.

segment.index.bytes
:   The size of the index that maps offsets to file positions.

Enterprise plan only. Set to any value between 100 kB and 1 GB.

segment.ms
:   The period after which Kafka forces the log to roll even if the segment file isn't full.

Enterprise plan only. Set to any value between 5 minutes and 30 days.
FAQs 

Answers to common questions about the IBM® Event Streams service.



 How do I use Kafka APIs to create and delete topics? 

If you're using a Kafka client at 0.11 or later, or Kafka Streams at 0.10.2.0 or later, you can use APIs to create and delete topics. We've put some restrictions on the settings allowed when you create topics. Currently, you can modify the following settings only:

cleanup.policy
: Set to delete (default), compact or delete,compact

retention.ms
: The default retention period is 24 hours. The minimum is 1 hour and the maximum is 30 days. Specify this value as multiples of hours.

Note: In the Enterprise plan, you can set this to any value.

retention.bytes
: The maximum size a partition (which consists of log segments) can grow to before we discard old log segments to free up space.

Note: Enterprise: Set to any value between 100 KiB and 2 TiB. Standard: Set to any value between 100 KiB and 1 GiB.

segment.bytes
: The segment file size for the log.

Note: Enterprise: Set to any value between 100 KiB and 2 TiB. Standard: Set to any value between 100 KiB and 512 MiB.

segment.index.bytes
: The size of the index that maps offsets to file positions.

Note: Enterprise: Set to any value between 100 KiB and 1 TiB. Standard: Set to any value between 100 KiB and 100 MiB.

segment.ms
: The period of time after which Kafka will force the log to roll even if the segment file isn't full.

Note: Set to any value between 5 minutes and 30 days.

See the following example of default value settings.

Details for topic testit
Topic name Internal? Partition count Replication factor
testit false 1 3

Partition details for topic testit
Partition ID Leader Replicas In-sync
0 1 [1 5 0] [1 5 0]

Configuration parameters for topic testit
Name Value
cleanup.policy delete
curl -X POST --header "Authorization: Bearer {token}" "{url}/v1/acoustic_customizations/{customization_id}/train"

The method is asynchronous. Training time depends on the cumulative amount of audio data that the custom acoustic model contains and the current load on the service. When you train or retrain a model, the service uses all of the model's audio data (not just new data) in the training. So training time is commensurate with the total amount of audio that the model contains.

A general guideline is that training a custom acoustic model takes approximately as long as the length of its cumulative audio data. For example, it takes approximately 2 hours to train a model that contains a total of 2 hours of audio. For more information about checking the status of a training operation, see [Monitoring the train model request](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-acousticmonitorTraining-acoustic).

The method includes the following optional query parameters:



* The custom_language_model_id parameter specifies a separately created custom language model that is to be used during training. You can train with a custom language model that contains transcriptions of your audio files or that contains corpora or OOV words that are relevant to the contents of the audio files. For training to succeed, the custom language model must be fully trained and available, and the custom acoustic and custom language models must be based on the same version of the same base model. For more information, see [Training a custom acoustic model with a custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-useBothuseBothTrain).
* The strict parameter indicates whether training is to proceed if the custom model contains a mix of valid and invalid audio resources. By default, training fails if the model contains one or more invalid resources. Set the parameter to false to allow training to proceed as long as the model contains at least one valid resource. The service excludes invalid resources from the training.
But it can take longer depending on the size and complexity of the grammar and the current load on the service. For information about checking the status of a training operation, see [Monitoring the training request](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarAddmonitorTraining-grammar).



 Monitoring the training request 

The service returns a 200 response code if the training process is successfully initiated. The service cannot accept subsequent training requests, or requests to add new grammars, corpora, or words to the custom model, until the current training request completes.

To determine the status of a training request, use the GET /v1/customizations/{customization_id} method to poll the model's status. The method accepts the customization ID of the model and returns information like the following about the model:

{
"customization_id": "74f4807e-b5ff-4866-824e-6bba1a84fe96",
"created": "2018-06-01T18:42:25.324Z",
"language": "en-US",
"dialect": "en-US",
"owner": "297cfd08-330a-22ba-93ce-1a73f454dd98",
"name": "Example model",
"description": "Example custom language model",
"base_model_name": "en-US_BroadbandModel",
"status": "training",
"progress": 0
}

The status field has the value training while the model is being trained. Use a loop to check the status of the model every 10 seconds until it becomes available. For more information about monitoring a training request and other possible status values, see [Monitoring the train model request](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreatemonitorTraining-language).
The retention start timestamp is the larger of the following values:



* The beginning of the retention time period (the current date/time minus the retention interval).
* The retention lower bound.



A table’s retention start timestamp comes into play in the following operations:



* Time travel queries (SELECT and sub-SELECT)

If you attempt to run queries for historical rows that were deleted before the retention start timestamp, an error is returned.

If you want to query historical data as far back as possible, you can use the RETENTION_START_TIMESTAMP keyword in time travel queries. If you do this, you can avoid having to try to compute the right timestamp on your own. By extension, you eliminate the risk of running into an error if the value turns out to be too old (older than the retention start timestamp).
* GROOM TABLE Historical rows that were deleted before the retention start timestamp are no longer needed for time travel queries and can be reclaimed.







 Row timestamps and validity 

The insert timestamp of a current or historical row is the date/time that the transaction inserting the row committed. It is not the time when a particular INSERT, UPDATE, or MERGE statement that inserted the row was run.

If the inserting transaction for a row committed before the retention start timestamp, the row is treated as having been inserted at the retention start timestamp. This generally applies only to existing rows at the time of altering a nontemporal table to a temporal table.

An inserted row whose transaction has not yet committed does not have an insert timestamp. Such a row will never be visible to a time travel query.

In a time travel query, you can select the insert timestamp by using the _SYS_START virtual column of a temporal table.

The delete timestamp of a historical row is the date/time that the transaction deleting the row committed. It is not the time when a particular DELETE, UPDATE, MERGE, or TRUNCATE statement that deleted the row was run.

If a temporal table is truncated, the existing table rows are available to time travel queries and are treated as having been deleted as of the time the truncating transaction committed.
* [Low latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interimlow-latency)



Updates to English next-generation telephony models
: The English next-generation telephony models have been updated for improved speech recognition:



* en-AU_Telephony
* en-GB_Telephony
* en-IN_Telephony
* en-US_Telephony



All of these models continue to support low latency. You do not need to upgrade custom models that are based on these models. For more information about all available next-generation models, see [Next-generation languages and models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng).

The max_alternatives parameter is now available for use with next-generation models
: The max_alternatives parameter is now available for use with all next-generation models. The parameter is generally available for all next-generation models. For more information, see [Maximum alternatives](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metadatamax-alternatives).

Defect fix: Allow use of both max_alternatives and end_of_phrase_silence_time parameters with next-generation models
: Defect fix: When you use both the max_alternatives and end_of_phrase_silence_time parameters in the same request with next-generation models, the service now returns multiple alternative transcripts while also respecting the indicated pause interval. Previously, use of the two parameters in a single request generated a failure. (Use of the max_alternatives parameter with next-generation models was previously available as an experimental feature to a limited number of customers.)

Defect fix: Update to Japanese next-generation multimedia model (upgrade required)
: Defect fix: The Japanese next-generation multimedia model, ja-JP_Multimedia, was updated to address an internal inconsistency that could cause an error during speech recognition with low latency. You need to upgrade any custom models that are based on the ja-JP_Multimedia model.
The retention period of an object can only be extended. It cannot be decreased from the currently configured value.

The retention expansion value is set in one of three ways:



* additional time from the current value (Additional-Retention-Period or similar method)
* new extension period in seconds (Extend-Retention-From-Current-Time or similar method)
* new retention expiry date of the object (New-Retention-Expiration-Date or similar method)



The current retention period stored in the object metadata is either increased by the given additional time or replaced with the new value, depending on the parameter that is set in the extendRetention request. In all cases, the extend retention parameter is checked against the current retention period and the extended parameter is only accepted if the updated retention period is greater than the current retention period.

Objects in protected buckets that are no longer under retention (retention period has expired and the object does not have any legal holds), when overwritten, will again come under retention. The new retention period can be provided as part of the object overwrite request or the default retention time of the bucket will be given to the object.

public static void extendRetentionPeriodOnObject(String bucketName, String objectName, Long additionalSeconds) {
System.out.printf("Extend the retention period on %s in bucket %s by %s seconds.n", objectName, bucketName, additionalSeconds);

ExtendObjectRetentionRequest req = new ExtendObjectRetentionRequest(
bucketName,
objectName)
.withAdditionalRetentionPeriod(additionalSeconds);

cos.extendObjectRetention(req);

System.out.printf("New retention period on %s is %sn", objectName, additionalSeconds);
}





 List legal holds on a protected object 

This operation returns:



* Object creation date
* Object retention period in seconds
* Calculated retention expiration date based on the period and creation date
* List of legal holds
* Legal hold identifier
* Timestamp when legal hold was applied
The internal Kafka __consumer_offsets topic is visible to you as read-only on the Enterpise plan. You are strongly recommended not to attempt to manage the topic in any way. You cannot access the __consumer_offsets topic in any way on the Standard plan.
* How can I clean up a consumer group with no consumers?

After consumers have left, a group continues to exist only if it has offsets. Consumer offsets are deleted after 7 days of inactivity. Consequently, a consumer group is deleted when the last committed offset for that group expires.

If you want to explicitly delete a group at a time you choose, you can use the [deleteConsumerGroups() API](http://kafka.apache.org/23/javadoc/org/apache/kafka/clients/admin/AdminClient.htmldeleteConsumerGroups-java.util.Collection).
* How long are messages retained?

By default, messages are retained in Kafka for up to 24 hours and each partition is capped at 1 GB. If the 1 GB cap is reached, the oldest messages are discarded to stay within the limit.

You can change the time limit for message retention when you create a topic using either the user interface or the administration API. The time limit is a minimum of an hour and a maximum of 30 days.

For information about restrictions on the settings allowed when you create topics using a Kafka client or Kafka Streams, see [How do I use Kafka APIs to create and delete topics?](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-faqstopic_admin)
* What is Event Streams's availability behavior?

If you write Event Streams apps, use this information to understand what normal Event Streams availability behavior is and what your apps are expected to handle.



 APIs 

As part of the regular operation of Event Streams, the nodes of the Kafka clusters are occasionally restarted. In some cases, your apps will be aware as the cluster reassigns resources.
System.out.printf("Minimum Retention (Days): %sn", config.getMinimumRetentionInDays());
System.out.printf("Default Retention (Days): %sn", config.getDefaultRetentionInDays());
System.out.printf("Maximum Retention (Days): %sn", config.getMaximumRetentionInDays());
}
}





 Upload a protected object 

Objects in protected buckets that are no longer under retention (retention period has expired and the object does not have any legal holds), when overwritten, will again come under retention. The new retention period can be provided as part of the object overwrite request or the default retention time of the bucket will be given to the object.



 Value Type Description 

 Retention-Period Non-negative integer (seconds) Retention period to store on the object in seconds. The object can be neither overwritten nor deleted until the amount of time specified in the retention period has elapsed. If this field and Retention-Expiration-Date are specified a 400 error is returned. If neither is specified the bucket's DefaultRetention period will be used. Zero (0) is a legal value assuming the bucket's minimum retention period is also 0. 
 Retention-expiration-date Date (ISO 8601 Format) Date on which it will be legal to delete or modify the object. You can only specify this or the Retention-Period header. If both are specified a 400 error will be returned. If neither is specified the bucket's DefaultRetention period will be used. 
 Retention-legal-hold-id string A single legal hold to apply to the object. A legal hold is a Y character long string. The object cannot be overwritten or deleted until all legal holds associated with the object are removed. 



public static void putObjectAddLegalHold(String bucketName, String objectName, String fileText, String legalHoldId) {
You can specify that backups be kept 1 - 30 days (default). The retention period can't be shorter than the backup frequency or it returns an error.

You can also specify the number of backups to retain, up to 750 per volume, after which the oldest backups are deleted.
* Are there limitations on how many backups I can take?

Yes. You can create 10 backup policies per account and up to 750 backups of a volume. For other limitations of this release, see [Limitations in this release](https://cloud.ibm.com/docs/vpc?topic=vpc-backup-service-about&interface=uibackup-service-limitations).
* How do I create a new volume from a backup?

Restoring from a backup snapshot creates a volume with data from the snapshot. You can restore data from a backup by using the UI, the CLI, or the API. You can restore boot and data volumes during instance creation, when you modify an existing instance, or when you provision a stand-alone volume. When you restore data from a backup snapshot, the data is pulled from an Object Storage bucket. For best performance, you can enable backup snapshots for fast restore. By using the fast restore feature, you can restore a volume that is fully provisioned when the volume is created. When you use fast restore, the data is pulled from a cached backup snapshot in another zone of your VPC. For more information, see [About restoring from a backup snapshot](https://cloud.ibm.com/docs/vpc?topic=vpc-baas-vpc-restore).
* Am I charged for usage?

Yes. Cost for backups is calculated based on GB capacity that is stored per month, unless the duration is less than one month. The backup exists on the account until it reaches its retention period, or when you delete it manually, or when you reach the end of a billing cycle, whichever comes first.