vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Creating a library set for other packages or file download 

If you want to customize your instance by adding libraries to a library set that you fetch from sources other than through the conda or pip repositories, you should use script based customization.

If you are only using a set of JAR files or PY files, you should use the "application_details" > "conf" > "spark.submit.pyFiles" or "application_details" > "jars" arguments in your Spark application submission payload and only create a library set if this method doesn't work for you. For more information, see [Submitting a Spark application](https://cloud.ibm.com/docs/AnalyticsEngine?topic=AnalyticsEngine-spark-app-rest-apispark-submit-app)

With script based customization, you create a Python script using the module naming convention expected by IBM Analytics Engine. Also, you need to implement a Python function that acts as the executable entry point to your script. In this script, you can add your own logic for downloading the libraries and placing them in a predesignated directory, which is /home/spark/shared/user-libs/<libraryset_name>/custom/<subdir_if_applicable>, so that they become part of the library set and get stored for later consumption in your application.



 Creating a library set using script based customization 

Perform these steps to create a library set using script based customization:



1. Create a Python file named customization_script.py. Analytics Engine's customization component looks for a Python module with this name.
2. In your customization_script.py, implement a function called customize(<install_path>, <params>), where <install_path> is the predesignated location where the libraries are downloaded to, namely /home/spark/shared/user-libs/<libraryset_name>/custom/<subdir_if_applicable>. You can't change this path. <params> is a list of the parameters required by the customization script.
3.
Regions and endpoints 

You can use the IBM Cloud® App ID APIs for DevOps automation, customization, and management of your service instances. To understand availability of the service, you can review the following connectivity options.



  Available regions 

App ID is available in the following regions:

Zoom

![Visual representation of the availability of the service. The image is a map with pin points in the locations in which the service is available. If you are unable to view this image, see the table in the service endpoints section for a complete list.](https://cloud.ibm.com/docs-content/v1/content/27ecaa7a29890634603881a8e64789974a29916b/appid/images/regions.svg)

Figure 1. App ID availability

This image is an artistic representation and does not reflect actual political or geographic boundaries.





  Service endpoints 

If you're managing your service instances of App ID programmatically, see the following table to determine which API endpoints to use when you connect to the APIs.



Table 1. Lists public endpoints for interacting with App ID APIs

 Region      Endpoint   

 Dallas      us-south   
 Frankfurt   eu-de      
 London      eu-gb      
 Osaka       jp-osa     
 Sao Paulo   br-sao     
 Sydney      au-syd     
 Tokyo       jp-tok     
 Toronto     ca-tor     
 Washington  us-east    







  Formatting your API call 

Calls to the management API endpoint take the following structure:

https://<regionEndpoint>.appid.cloud.ibm.com/management



Table 2. Formatting for the available APIs

 API            Endpoint                                                   

 Authorization  https://<region>.appid.cloud.ibm.com/oauth/v4        
 Management     https://<region>.appid.cloud.ibm.com/management/v4   
 Profiles       https://<region>.appid.cloud.ibm.com/api/v1          







  Need help? 

If you run in to trouble while working with the APIs, try watching this tutorial video.
The relationship of speaker models to both custom models and custom prompts is one-to-many. The same speaker model can be associated with multiple prompts that are defined in different custom models. Moreover, you do not specify a speaker model with a synthesis request.



 Tune by Example and word pronunciation 

The service’s default pronunciation and tokenization rules might cause an unusual word that occurs in a prompt to be pronounced in an unsatisfactory way. However, Tune by Example is not intended to change a word's pronunciation. For example, the service pronounces the word "catastrophic" as one expects to hear it. You can use Tune by Example to change what syllables of the word are emphasized (for instance, "cataSTROPHic") or the tempo of the word (for instance "cat-a-strophic"). But you cannot use Tune by Example to change the word's default pronunciation.

To change how a word is pronounced, you use the service's customization interface to define an alternative pronunciation for the word. You add a custom word with the altered pronunciation to the same custom model as the prompt. The service then applies the pronunciation of the custom word when it occurs in a prompt. For more information, see [Understanding customization](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-customIntro).

If you add or modify a custom word that is part of an existing prompt, the service continues to honor the pronunciation of the word that was in place when the prompt was created. This occurs because the service creates a prompt at a point in time, and it employs the pronunciation rules that are in effect at that time. You need to re-create a prompt to take advantage of a new or updated custom word.

When you re-create a prompt, the service replaces the existing prompt. The new prompt reflects any changes to the text or the audio of the prompt along with any custom word pronunciations defined since the prompt was first created. You can always pass the same text and audio that you used to create the original prompt if you are rerecording the prompt after adding a custom word.







 Custom prompts
1. Select the cluster for which you want to create a logging logging configuration.
2. On the cluster Overview page, in the Logging section, click Disconnect.







 Detaching a logging agent by using ob commands 

This option is only valid when you deploy the logging agent by using ob commands.

Complete the following steps:



1. Set the cluster context.

Open a terminal to log in to IBM Cloud.

ibmcloud login -a cloud.ibm.com

Select the account where you provisioned the IBM Log Analysis instance.

List the clusters to find out in which region and resource group the cluster is available.

ibmcloud ks clusters

Set the resource group and region.

ibmcloud target -g RESOURCE_GROUP -r REGION

Where

RESOURCE_GROUP is the name of the resource group where the cluster is available, for example, default.

REGION is the region where the cluster is available, for example, us-south.

Set the cluster where you want to configure logging as the context for this session.

ibmcloud oc cluster config --cluster <cluster_name_or_ID>
2. Detach the logging agent by using the ob CLI. Run the following command:

ibmcloud ob logging config delete --cluster <cluster_name_or_ID> --instance <LogDNA_instance_name_or_ID> [--force]

Where



* <cluster_name_or_ID> is the name or the ID of the cluster.
* <LogDNA_instance_name_or_ID> is the name or the ID of the logging instance where you want to forward the cluster logs for analysis.
* [--force] is used to force the command to run with no user prompts.
This version supports IBM Cloud Pak for Data versions 3.5 and 3.0.1, and Red Hat OpenShift versions 4.5 and 3.11.

New voices
: The service now offers three new voices:



* UK English: en-GB_CharlotteV3Voice and en-GB_JamesV3Voice
* French: fr-FR_NicolasV3Voice



The service also offers an improved version of the existing UK voice, en-KateV3Voice. For more information about all supported languages and voices, see [Languages and voices](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-voices).

Defect fix: Fix <prosody> element for Japanese
: Defect fix: For the ja-JP_EmiV3Voice voice, the service now correctly parses SSML input text that includes a prosody rate specification. Previously, the following use of the <prosody> element worked properly:

<speak>成功する/繁栄する</speak>

But the following use of the rate attribute with the <prosody> element caused the service to read and speak the embedded SSML notation:

<speak>
<prosody rate="fast">成功する/繁栄する</prosody>
</speak>

The service now correctly parses and applies the rate attribute of the <prosody> element for Japanese input.





 4 September 2020 (Version 1.1.4) 

Customization interface is generally available
: The customization interface is now generally available. Customization is no longer beta functionality. You can use the customization interface to specify how the service pronounces unusual words that occur in your input text by creating language-specific custom dictionaries. For more information, see [Understanding customization](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-customIntro).





 15 July 2020 (Version 1.1.4) 

Red Hat OpenShift version 4.3 is going out of service
You can set a customization weight during both training and speech recognition. For more information, see [Using customization weight](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageUseweight).

Updates to Japanese broadband model for improved speech recognition
: The ja-JP_BroadbandModel language model was upgraded to capture improvements in the base model. The upgrade does not affect existing custom models that are based on the model.

New endianness parameter for audio/l16 audio format
: The service now includes a parameter to specify the endianness of audio that is submitted in audio/l16 (Linear 16-bit Pulse-Code Modulation (PCM)) format. In addition to specifying rate and channels parameters with the format, you can now also specify big-endian or little-endian with the endianness parameter. For more information, see [audio/l16 format](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-audio-formatsaudio-formats-l16).





 14 July 2017 

New support for MP3 (MPEG) audio format
: The service now supports the transcription of audio in the MP3 or Motion Picture Experts Group (MPEG) format. For more information, see [audio/mp3 and audio/mpeg formats](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-audio-formatsaudio-formats-mp3).

Beta language model customization now available for Spanish
: The language model customization interface now supports Spanish as beta functionality. You can create a custom model based on either of the base Spanish language models: es-ES_BroadbandModel or es-ES_NarrowbandModel; for more information, see [Creating a custom language model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreate). Pricing for recognition requests that use Spanish custom language models is the same as for requests that use US English and Japanese models.

New dialect field for method that creates a custom language model
During a failover, a VM moves to a secondary site. When the preferred site is recovered, the VM automatically returns back to the preferred site with the compute policies restored.

You can select the compute policy that you prefer for each VM and VMware Solutions attempts to keep the VM on the preferred data center if it is up and running.



1. From the tenant portal, click Virtual Machines from the left panel.
2. Click Hardware to expand the list and click Compute.
3. Locate Placement Policy and click EDIT.
4. Select the compute policy and click Save.







 Changing the hardware properties of a virtual machine 

You can change the hardware properties of a VM, number of vCPUs, memory, hard disk allocation, and network configuration.





 Changing the Guest OS Customization properties of a virtual machine 

Guest OS customization is optional for all platforms. It is required for VMs that must join a Windows domain when the VMs are being powered on.

When you use an IBM template to create the VM, use the Guest OS Customization pane to acquire or set the unique password for the OS instance. Ensure that the option Enable guest customization is selected and then use one of the Password Reset options to establish the initial administrator credential.

For more information, see [Change the guest OS customization properties of a virtual machine](https://docs.vmware.com/en/VMware-Cloud-Director/10.0/com.vmware.vcloud.tenantportal.doc/GUID-2B7A04E8-7479-4EE9-99B0-1A46751BE46F.html).





 Changing the advanced properties of a virtual machine 

In the Advanced settings, you can configure the resource allocation settings (shares, reservation, and limit) to determine the amount of virtual CPU (vCPU), memory, and storage resources provided for a VM.
Environment properties and resources 

The following properties and resources are available by default in IBM Cloud® Continuous Delivery Delivery Pipeline environments.



 Environment properties 



 General-purpose properties 

The following table lists and describes each of the general-purpose environment properties that are available by default in pipeline environments.



Table 1. General-purpose environment properties

 Environment Property Description 

 ARCHIVE_DIR The directory to archive or to download archives into. 
 BUILD_ID The unique ID for the current job execution. 
 BUILD_DISPLAY_NAME The BUILD_ID value, prefixed with "#". 
 BUILD_NUMBER The incremental stage ID that is shown in the pipeline UI. 
 GIT_BRANCH The Git branch that the job uses as input. This property is only available in jobs that use a Git repository as input. 
 GIT_COMMIT The Git commit that the job uses as input. This property is only available in jobs that use a Git repository as input. 
 GIT_PREVIOUS_COMMIT The Git commit value of the job's last successful run. This property is only available in jobs that use a Git repository as input. 
 GIT_URL The Git repository URL that the job uses as input. This property is only available in jobs that use a Git repository as input. 
 IDS_JOB_ID The unique ID of the job's configuration. 
 IDS_JOB_NAME The name of the job's configuration. 
 IDS_OUTPUT_PROPS Comma-separated names of your stage environment properties. 
 IDS_PROJECT_NAME The name of the project, for example, Owner - Project Name. 
 IDS_STAGE_NAME The name of the current stage. 
 IDS_URL The URL for the current pipeline. 
 IDS_VERSION The number for the build that is being deployed or the SCM identifier. This property is available only in deploy jobs. 
 JOB_NAME The unique job ID in the context of the current pipeline. 
 PIPELINE_ARTIFACT_URL The URL that you can use to download the artifacts of the current Build job after the job completes. You must use a valid Bearer token to access the artifacts.
Use the [ibmcloud ce project update](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-project-update) command and specify the --binding-resource-group option to configure a Code Engine project for service binding access for all service instances in a resource group. This command tells your Code Engine project which resource group that it can bind to. You can update your project to bind services to a different resource group only with the CLI.

The project update command works within the project that is selected as the current context. Before you use the project update command, confirm that you are in the desired project. Use the [ibmcloud ce project current](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-project-current) command to display details of the project that is currently targeted. If needed, use the [ibmcloud ce project select](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-project-select) command to select your project as the current context.



* To configure service binding access for all service instances in the Default resource group,

ibmcloud ce project update --binding-resource-group Default
* To configure service binding access for all service instances in a resource group, by specifying the ID of the resource group,

ibmcloud ce project update --binding-resource-group-id abcdabcdabcdabcdabcdabcdabcdabcd

To get a list of your resource groups, including the resource group IDs, run ibmcloud resource groups.
* To configure service binding access for all service instances in all resource groups:

ibmcloud ce project update --binding-resource-group ""



When you run the project update command, a service ID is created for the project, and is used to configure the current project for service bindings. If you do not have permission to create this service ID, then you receive an error that your project is not ready to work with service bindings.
Show a global load balancer in a zone.

ibmcloud dns glb DNS_DOMAIN_ID GLB_ID [-i, --instance INSTANCE] [--output FORMAT]



 Command options 

DNS_DOMAIN_ID
: The ID of DNS zone. Required.

GLB_ID
: The ID of global load balancer. Required.

-i, --instance
: Instance name or ID. If not set, the context instance specified by dns instance-target INSTANCE is used.

--output
: Specify output format. Currently, json is the only supported format.





 Examples 

Show global load balancer 699d98642c564d2e855e9661899b7252 in zone demo.com:31984fea73a15b45779fa0df4ef62f9b.

ibmcloud dns glb demo.com:31984fea73a15b45779fa0df4ef62f9b 699d98642c564d2e855e9661899b7252 -i "dns-demo"







 ibmcloud dns glb-delete 

Delete a global load balancer in a DNS domain.

ibmcloud dns glb-delete DNS_DOMAIN_ID GLB_ID [-i, --instance INSTANCE] [-f,--force]



 Command options 

DNS_DOMAIN_ID
: The ID of DNS zone. Required.

GLB_ID
: The ID of global load balancer. Required.

-i, --instance
: Instance name or ID. If not set, the context instance specified by dns instance-target INSTANCE is used.

-f, --force
: Delete load balancer without prompting for confirmation.





 Examples
"conditions": "input.text.startsWith('What')"
}

Results: The condition is true.





 String.substring(Integer beginIndex, Integer endIndex) 

This method gets a substring with the character at beginIndex and the last character set to index before endIndex. The endIndex character is not included.

For this Dialog runtime context:

{
"context": {
"my_text": "This is a text."
}
}

This syntax:

{
"context": {
"my_text": "<? $my_text.substring(5, $my_text.length()) ?>"
}
}

Results in this output:

{
"context": {
"my_text": "is a text."
}
}





 String.toJson() 

This method parses a string that contains JSON data and returns a JSON object or array, as in this example:

${json_var}.toJson()

If the context variable ${json_var} contains the following string:

"{ "firstname": "John", "lastname": "Doe" }"

the toJson() method returns the following object:

{
"firstname": "John",
"lastname": "Doe"
}





 String.toLowerCase() 

This method returns the original String converted to lowercase letters.

For this input:

"This is A DOG!"

This syntax:

{
"context": {
"input_lower_case": "<? input.text.toLowerCase() ?>"
}
}

Results in this output:

{
"context": {
"input_lower_case": "this is a dog!"
}
}





 String.toUpperCase() 

This method returns the original String converted to uppercase letters.
* UI
* CLI
* API







 Viewing details of custom resolver forwarding rules 

You view details of a single custom resolver forwarding rule and view a list of all custom resolver forwarding rules in IBM Cloud® DNS Services by using the UI, CLI, or API.



 Viewing custom resolver forwarding rules using the UI 

View the details of a custom resolver forwarding rule by navigating to the Forwarding rules tab in the custom resolver details page.

The Forwarding rules section lists all custom resolver rules associate with the custom resolver.





 Viewing custom resolver forwarding rules using the CLI 

To get the details of a custom resolver forwarding rule using the CLI, run the following command:

ibmcloud dns custom-resolver-forwarding-rule RESOLVER_ID RULE_ID [-i, --instance INSTANCE] [--output FORMAT]

Where:



* RESOLVER_ID is the ID of custom resolver.
* RULE_ID is the ID of custom resolver forwarding rule.
* -i, --instance is the instance name or ID. If this is not set, the context instance specified by dns instance-target INSTANCE is used instead.
* --output specifies output format. Currently, JSON is the only supported format.





 Listing all custom resolvers forwarding rule 

To list all custom resolver forwarding rules using the CLI, run the following command:

ibmcloud dns custom-resolver-forwarding-rules RESOLVER_ID [-i, --instance INSTANCE] [--output FORMAT]

Where:



* RESOLVER_ID is the ID of custom resolver.
* -i, --instance is the instance name or ID. If this is not set, the context instance specified by dns instance-target INSTANCE is used instead.
* --output specifies output format. Currently, JSON is the only supported format.









 Viewing custom resolver forwarding rules using the API 

To view the details of a custom resolver forwarding rule using the API, follow these steps:



1. Set up your API environment with the correct variables.
2.
Support for speaker_labels parameter for German and Korean
: The service now supports speaker labels (the speaker_labels parameter) for German and Korean language models. Speaker labels identify which individuals spoke which words in a multi-participant exchange. For more information, see [Speaker labels](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-speaker-labels).

Improved speech recognition for Japanese narrowband model
: The Japanese narrowband model (ja-JP_NarrowbandModel) now includes some multigram word units for digits and decimal fractions. The service returns these multigram units regardless of whether you enable smart formatting. The smart formatting feature understands and returns the multigram units that the model generates. If you apply your own post-processing to transcription results, you need to handle these units appropriately. For more information, see [Japanese](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-formattingsmart-formatting-japanese) in the smart formatting documentation.

Simplified backup and restore
: The service now offers greatly improved backup and restore procedures. Utilities are now available to back up data from your datastores, so you no longer need to re-create all of your data in the event of a disaster. For more information, see [Backing up and restoring your data](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-speech-backup).





 1 April 2020 (Version 1.1.3) 

Acoustic model customization is now generally available
: Acoustic model customization is now generally available (GA) for all supported languages. For more information about support for individual language models, see [Language support for customization](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-support).





 28 February 2020 (Version 1.1.3) 

Version 1.1.3 is available
: Speech to Text for IBM Cloud Pak for Data version 1.1.3 is now available.

New end_of_phrase_silence_time parameter