vania-janet/multiturn-rag-retrieval-data huggingface dataset:
This is done by setting update_strategy to merge_once and specifying the the ID of the custom language model in the configuration parameters.

{
"output": {
"generic": [
{
"response_type": "speech_to_text",
"command_info": {
"type": "configure",
"parameters": {
"update_strategy": "merge_once",
"narrowband_recognize": {
"language_customization_id": "ao45vohgFuxyOQRgztu-02I10ut7aJcM-AdInT-VWgj3V"
}
}
}
}
]
}
}

Show more









 Applying advanced settings to the Text to Speech service 

Use the text_to_speech response type to send configuration commands to the Text to Speech service instance used by the phone integration. By sending a text_to_speech response from a dialog node or action step, you can dynamically change the Text to Speech configuration during a conversation.

By default, any Text to Speech configuration changes you make persist for the remainder of the conversation, or until you update them again. You can change this behavior by specifying the update_strategy property of the parameters object.

The format of the speech_to_text response type is as follows:

{
"output": {
"generic": [
{
"response_type": "text_to_speech",
"command_info": {
"type": "<command type>",
"parameters": {
"parameter name": "parameter value",
"parameter name": "parameter value"
}
}
}
]
}
}
Show more



Each command type along with its related parameters are described in the following sections.



 command_info.type : configure 

Dynamically reconfigures the Text to Speech service by applying a set of configuration parameters, which can be based on the dialog or action flow.
[Overflow menu](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/kebab.png) icon, and then choose Settings.
2. Click Webhooks > Pre-message webhook.
3. Set the Pre-message webhook switch to Enabled.
4. Decide whether to return an error if the webhook call fails.

When enabled, everything stops until the preprocessing step is completed successfully.



* If you have a critical preprocessing step that must be taken before you want to allow the message to be processed by the assistant, enable this setting.

Take steps to test the process that you are calling on a regular basis so you will know if it's down, and can change this setting to prevent all of your message calls from failing.
* When this setting is disabled, the assistant ignores any errors it encounters and continues to process the incoming message without taking the preprocessing step. If the preprocessing step is helpful but not critical, consider keeping this setting disabled.



5. In the URL field, add the URL for the external application to which you want to send HTTP POST request callouts.

For example, you might write a Cloud Functions web action that checks whether a message is in a language other than English, and if so, send it to the Language Translator service to covert it to English. Specify the URL for your web action.

https://us-south.functions.cloud.ibm.com/api/v1/web/my_org_dev/default/translateToEnglish.json

You must specify a URL that uses the SSL protocol, so specify a URL that begins with https.

You cannot use a webhook to call a Cloud Functions action that uses token-based Identity and Access Management (IAM) authentication. However, you can make a call to a Cloud Functions web action or a secured web action.
6.
On the node you want to copy, click the More(images/search_dialog.png)More icon][!!] icon, and then select Duplicate.
2. Consider renaming the copied nodes or editing their conditions to make them distinct.







 Moving a dialog node 

Each node that you create can be moved elsewhere in the dialog tree.

You might want to move a previously created node to another area of the flow to change the conversation. You can move nodes to become siblings or peers in another branch.



1. On the node you want to move, click the More![More icon](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/kebab.png) icon, and then select Move.
2. Select a target node that is located in the tree near where you want to move this node. Choose whether to place this node before or after the target node, or to make it a child of the target node.







 Organizing the dialog with folders 

You can group dialog nodes together by adding them to a folder. There are lots of reasons to group nodes, including:



* To keep nodes that address a similar subject together to make them easier to find. For example, you might group nodes that address questions about user accounts in a User account folder and nodes that handle payment-related queries in a Payment folder.
* To group together a set of nodes that you want the dialog to process only if a certain condition is met. Use a condition, such as $isPlatinumMember, for example, to group together nodes that offer extra services that should only be processed if the current user is entitled to receive the extra services.
* To hide nodes from the runtime while you work on them. You can add the nodes to a folder with a false condition to prevent them from being processed.



These characteristics of the folder impact how the nodes in a folder are processed:



* Condition: If no condition is specified, then your assistant processes the nodes within the folder directly.
For more information, see [Migrating to the new experience](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-migrate-overview).





 28 March 2022 

New service desk support reference implementation
: You can use the reference implementation details to integrate the web chat with the Kustomer service desk. For more information, see [Adding service desk support](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-deploy-web-chatdeploy-web-chat-haa).





 18 March 2022 

Custom extensions
: If you need to integrate your assistant with an external service that has a REST API, you can now build a custom extension by importing an OpenAPI document. Your assistant can then send requests to the external service and receive response data it can use in the conversation. For example, you might use an extension to interact with a ticketing or customer relationship management (CRM) system, or to retrieve real-time data such as mortgage rates or weather conditions.

For more information about custom extensions, see [Building a custom extension](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-build-custom-extension) and [Calling a custom extension](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-call-extension).

Confirmation customer response type
: The confirmation customer response type is now available. Use this response type when a customer's response must be either Yes or No. For more information, see [Confirmation](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-collect-infocustomer-response-type-confirmation).

Search integration highlights text in browser
: Search results in Watson Assistant include a link. Now, when a customer clicks the link, search results are highlighted in their browser so it’s easier for them to see the relevant content. This feature is supported on Chromium browsers, including Google Chrome and Microsoft Edge.





 24 February 2022 

Regex customer response type
: The regex customer response type is now available.
* The HTTP GET /v1/synthesize method accepts a maximum of 8 KB of input, which includes the input text and the URL and headers.
* The HTTP POST /v1/synthesize method accepts a maximum of 8 KB for the URL and headers, and a maximum of 5 KB for the input text that is sent in the body of the request.
* The WebSocket /v1/synthesize method accepts a maximum of 5 KB of input text.



These limits include all characters of the input, including whitespace.

IBM Cloud

For billing purposes, whitespace characters are not counted. However, all other characters are counted, including those that are part of SSML elements.







 Using speech synthesis features 

The service supports additional features that you can use to tailor the text that you send and the audio that you receive.



 SSML 

You can pass the service plain text or text that is annotated with the Speech Synthesis Markup Language (SSML). SSML is an XML-based markup language that provides annotations of text for speech synthesis applications such as the Text to Speech service.



* For more information about specifying input text, see [Specifying input text](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-usingHTTPinput).
* For more information about using SSML, see [Understanding SSML](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-ssml).







 Modifying the speaking rate 

To modify the global rate of speech synthesis for a request, you can use the rate_percentage query parameter. The speaking rate is the speed at which the service speaks the text that it synthesizes into speech. A higher rate causes the text to be spoken more quickly; a lower rate causes the text to be spoken more slowly. The parameter changes the per-voice default rate for an entire request. For more information, see [Modifying the speaking rate](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-synthesis-paramsparams-rate-percentage).
Connecting to a live agent 

Your assistant can do a lot, but there might be some situations when your customers need help from a human. If your assistant is integrated with one of the supported service desk systems, you can build in logic that will transfer the conversation to a live agent when necessary. This is referred to as an escalation.

To use this feature, your assistant must interact with customers using a web chat or phone integration. For more information, see [Deploying your assistant](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-deploy-assistant).

You must add an integration to a service desk system before setting up live agent transfers to your assistant. The supported service desks depend upon how customers connect to your assistant (web chat or phone). For more information about integrating with a service desk, see [Basic web chat configuration](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-web-chat-overview) and [Phone integration configuration](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-deploy-phone-config).

Your agents can work with one of these supported service desk tools:



* [Genesys](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-deploy-phone-genesys)
* [NICE CXone](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-deploy-phone-nicecxone)
* [Salesforce](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-deploy-salesforce)
* [Twilio Flex](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-deploy-phone-flex)
* [Zendesk](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-deploy-zendesk)



There are two basic scenarios when your assistant might need to transfer a conversation to a live agent:
For more information, see [New system entities](https://cloud.ibm.com/docs/assistant?topic=assistant-system-entities).





 6 March 2020 

Transfer a web chat conversation to a human agent
: Delight your customers with 360-degree support by integrating your web chat with a third-party service desk solution. When a customer asks to speak to a person, you can connect them to an agent through a service desk solution, such as Zendesk or Salesforce. Service desk support is a beta feature. For more information, see [Adding support for transfers](https://cloud.ibm.com/docs/assistant?topic=assistant-deploy-web-chatdeploy-web-chat-haa).





 2 March 2020 

Known issue accessing logs
: If you cannot access user logs from the Analytics page, ask the owner of the service instance for the skill to change your service level access to make you a Manager of the instance. For more information about access control, see [Managing access to resources](https://cloud.ibm.com/docs/assistant?topic=assistant-access-control).





 1 March 2020 deprecation notice 

March 2020 deprecation notice
: To help us continue to improve and expand the capabilities of the assistants you build with Watson Assistant, we are deprecating some of the older technologies. Support for the older technologies will end in June 2020. Take action now to test and adopt the new technologies, so your skills and assistants will be ready when the old technologies stop being supported.

The following technologies are being deprecated:



* Legacy version of numeric system entities

We released a whole new infrastructure for our numeric system entities across all languages except Chinese, Korean, Japanese and Arabic. The updated @sys-number, @sys-date, @sys-time, @sys-currency, and @sys-percentage entities provide superior number recognition with higher precision. For more information about the new system entities, see [System entity details](https://cloud.ibm.com/docs/assistant?topic=assistant-system-entities).
![An example of the desktop launcher](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/desktop-launcher.png)

The expanded launcher remains in its expanded state even if the customer reloads the page or navigates to a different page. It stays in its expanded state until the customer either opens it by clicking on either of the two primary buttons, or closes it, at which point it returns to its initial small state for the rest of the session.
* For mobile browsers, the launcher shows only a single primary button:

![An example of the mobile launcher](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/mobile-launcher.png)

The customer can close the launcher by scrolling on the page, swiping right on the expanded launcher, or waiting 10 seconds, at which point the expanded launcher shrinks back to its initial small state automatically. If the user reloads the page or navigates to a different page while the laucher is expanded, it stays in its expanded state, and the 10-second timer restarts.



The color of the launcher is specified by the Accent color field on the Style tab of the web chat settings. To change the color, specify a new color using a standard hexadecimal RGB value.

You can customize the greeting message displayed by the launcher on the Launcher tab of the web chat settings. The settings include separate greeting messages for the desktop and mobile versions of the launcher.

The language of the default text shown within the launcher depends on the locale configured for the web chat. For more information, see [Languages](https://web-chat.global.assistant.watson.cloud.ibm.com/docs.html?to=api-instance-methodslanguages). If you customize the greeting text, the text you provide is used regardless of the locale settings.
Create an intent that can recognize when a customer asks to speak to someone. After defining the intent, you can add a root-level dialog node that conditions on the intent. As the dialog node response, add a Connect to human agent response type. At run time, if the user asks to speak to someone, this node is triggered and a transfer is initiated on the user's behalf.
* When the conversation broaches a topic that is sensitive in nature, you can start a transfer.

For example, an insurance company might want questions about bereavement benefits always to be handled by a person. Or, if a customer wants to close an account, you might want to transfer the conversation to a respresentative who is authorized to offer incentives to keep the customer's business.
* When the assistant repeatedly fails to understand a customer's request. Instead of asking the customer to retry or rephrase the question over and over, your assistant can offer to transfer the customer to a human agent.



To design a dialog that can transfer the conversation, complete the following steps:



1. Add an intent to your skill that can recognize a user's request to speak to a human.

You can create your own intent or add the prebuilt intent named General_Connect_to_Agent that is provided with the General content catalog.
2. Add a root node to your dialog that conditions on the intent you created in the previous step. Choose Connect to human agent as the response type.

For more information, see [Adding a Connect to human agent response type](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-dialog-overviewdialog-overview-add-connect-to-human-agent).
3. Add meaningful names to the dialog nodes in your dialog.

When a conversation is transferred to an agent, the name of the most-recently processed dialog node is sent with it. To ensure that a useful summary is provided to service desk agents when a conversation is transferred to them, add short dialog node names that reflect the node's purpose. For example, Find a store.

Some older skills specify the purpose of the node in the external node name field instead.
You can also use that extension to [connect directly to your network from Visual Studio Code](https://cloud.ibm.com/docs/blockchain?topic=blockchain-develop-vscodedevelop-vscode-connecting-ibp) and explore the inline tutorials.
* For a quick tutorial on developing smart contracts, see [Develop a smart contract with the IBM Blockchain Platform VS Code extension](https://developer.ibm.com/tutorials/ibm-blockchain-platform-vscode-smart-contract/).
* For a more in-depth end-to-end tutorial about using an application to interact with smart contracts, see [Hyperledger Fabric Commercial Paper tutorial](https://hyperledger-fabric.readthedocs.io/en/release-1.4/tutorial/commercial_paper.html).
* To learn about how to incorporate access control mechanisms into your smart contract, see [Chaincode for Developers](https://hyperledger-fabric.readthedocs.io/en/release-1.4/chaincode4ade.htmlchaincode-access-control).



Because Fabric v2.x peers do not have a "shim" (the external dependencies that allowed smart contracts to run on earlier versions of Fabric), you will have to vendor the shim and then repackage any smart contracts written in Golang (Go) that you installed on peers deployed using the 1.4.x Fabric image. Without this vendoring and repackaging, the smart contract will not run on a peer using a Fabric 2.x image. You will not need to do this on smart contracts that are written in Java or Node.js, nor for smart contracts written and packaged using the 2.0 package. See [Vendoring smart contracts](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-smart-contracts-v14ibp-console-smart-contracts-write-package-vendor) for details.

When you are ready to install and instantiate your smart contract to the IBM Blockchain platform, the smart contract must be packaged into .cds format.
For more information see [Securing the web chat](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-web-chat-security).
2. Encrypt sensitive information that you pass to the web chat.

When you enable security in Zendesk, you must provide the name and email address of the current user with each request. Configure the web chat to pass this information in the payload.

Specify the information by using the following syntax. Use the exact names (name and email) for the two name and value pairs.

{
user_payload : {
name: '{customerName}',
email: '{customerEmail}'
}
}

For more information, see [Passing sensitive data](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-web-chat-securityweb-chat-security-encrypt).

Zendesk also expects iat and external_id name and value pairs. However, there's no need for you to provide this information. IBM automatically provides a JWT that contains these values.

For example:

const userPayload = {
"name" : "Cade Jones",
"email" : "cade@example.com",
}

// Sample NodeJS code on your server.
const jwt = require('jsonwebtoken');
const RSA = require('node-rsa');

const rsaKey = new RSA(process.env.PUBLIC_IBM_RSA_KEY);

/
* Returns a signed JWT. Optionally, adds an encrypted user_payload in stringified JSON.
/
function mockLogin(userID, userPayload) {
const payload = {
sub: userID, // Required
iss: 'www.ibm.com', // Required
acr: 'loa1' // Required
Enabling visitor authentication also enables support for cross-domain traffic and cross-browser identification. For more information, see the [Zendesk documentation](https://support.zendesk.com/hc/en-us/articles/360022185314-Enabling-authenticated-visitors-in-the-Chat-widget).

Before you can secure the Zendesk connection, complete the following required tasks:



1. Secure the web chat. For more information see [Securing the web chat](https://cloud.ibm.com/docs/assistant?topic=assistant-web-chat-security).
2. Encrypt sensitive information that you pass to the web chat.

When you enable security in Zendesk, you must provide the name and email address of the current user with each request. Configure the web chat to pass this information in the payload.

Specify the information by using the following syntax. Use the exact names (name and email) for the two name and value pairs.

{
user_payload : {
name: '{customerName}',
email: '{customerEmail}'
}
}

For more information, see [Passing sensitive data](https://cloud.ibm.com/docs/assistant?topic=assistant-web-chat-securityweb-chat-security-encrypt).

Zendesk also expects iat and external_id name and value pairs. However, there's no need for you to provide this information. IBM automatically provides a JWT that contains these values.

For example:

const userPayload = {
"name" : "Cade Jones",
"email" : "cade@example.com",
}

// Sample NodeJS code on your server.
const jwt = require('jsonwebtoken');
const RSA = require('node-rsa');

const rsaKey = new RSA(process.env.PUBLIC_IBM_RSA_KEY);

/
* Returns a signed JWT.
Data security 

The Speech to Text service provides the following security features to help you protect your user data.



 Authenticating to IBM Cloud 

IBM Cloud

You authenticate to the service by using IBM Cloud Identity and Access Management (IAM). You can pass either an API key or a bearer token in an Authorization header. For more information, see [Authenticating to IBM Cloud](https://cloud.ibm.com/docs/watson?topic=watson-iamgs-credential-cloud).





 Authenticating to IBM Cloud Pak for Data 

IBM Cloud Pak for Data

You authenticate to the service by passing an access token with each request. You pass a bearer token in an Authorization header to authenticate. Several methods exist to generate the token, including by using an API key or by username. For more information, see [Authenticating to IBM Cloud Pak for Data](https://cloud.ibm.com/docs/watson?topic=watson-iamgs-credential-cpd).

Speech to Text for IBM Cloud Pak for Data is a multi-tenant cloud solution. Your credentials provide access to your data only, and your data is isolated from other users.





 Basic data security 

The service provides security for all user data both in motion and at rest:



* Transport Layer Security (TLS) 1.2 is used to secure data in transit.
* Advanced Encryption Standard (AES)-256 with Secure Hash Algorithm (SHA)-256 is used to secure data at rest.



For more information about data security for cloud applications, see [Security architecture for cloud applications](https://www.ibm.com/cloud/architecture/architectures/securityArchitecture/security-for-data).





 Information security 

The service supports the European Union General Data Protection Regulation (GDPR) to manage user data. You can pass the X-Watson-Metadata header with a request to associate a customer ID with data that the request passes to the service. If necessary, you can then delete the data by using the DELETE /v1/user_data method.