vania-janet/multiturn-rag-retrieval-data huggingface dataset:
While a human might be able to understand that Ena is a name of a person based on the text of the email, AQL rules that are written to identify names of persons with high precision from general text might be too conservative and not able to draw the same conclusion with high confidence, based on the fact that Ena is a capitalized word.

One way to boost the coverage of the rules is to use words from the From, To, and CC fields of the email as more evidence.

If the email is addressed to "Ena Smith," and the application makes this information available to the extractor, then the extractor developer can write more AQL rules to boost the coverage of the extractor that is based on the domain knowledge that emails are usually addressed to people.

For example, you can write AQL rules to identify person tokens from the email metadata fields. You then use that information as strong clues when you decide whether a capitalized token in the email text is a person name. In general, the email metadata is not part of the actual email message, but the application can make this metadata available to the extractor by using an external view.

For each email that must be processed, the application can pass the email text as document text (to populate the view Document) at run time. It can also pass the extra metadata by using an appropriately defined external view.

The following statement defines an external view that is named EmailMetadata. The external view has a schema that contains three fields of type Text. At run time, the view EmailMetadata is automatically populated from an external type named EmailMetadataSrc. You can then reference the EmailMetadata view in your AQL rules, similarly to how you would reference any other view.

create external view EmailMetadata
(fromAddress Text, toAddress Text, ccAddress Text)
external_name 'EmailMetadataSrc';





 Documenting the create external view statement with AQL Doc 

The AQL Doc comment for a create external view statement contains the following information:



* General description about the view.
* @field for every column name in the view.



/
* The external view named EmailMetadata, containing three fields
* of type Text.
* Conditional: If you want a slot to be enabled only under certain conditions, then you can add a condition to it. For example, if slot 1 asks for a meeting start time, slot 2 captures the meeting duration, and slot 3 captures the end time, then you might want to enable slot 3 (and ask for the meeting end time) only if a value for slot 2 is not provided. To make a slot conditional, edit the slot, and then from the More![More icon](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/kabob.png) menu, select Enable condition. Define the condition that must be met for the slot to be enabled.

You can condition on the value of a context variable from an earlier slot because the order in which the slots are listed is the order in which they are evaluated. However, only condition on a slot context variable that you can be confident will contain a value when this slot is evaluated. The earlier slot must be a required slot, for example.



6. Keep users on track. You can optionally define slot handlers that provide responses to questions users might ask during the interaction that are tangential to the purpose of the node.

For example, the user might ask about the tomato sauce recipe or where you get your ingredients. To handle such off-topic questions, click the Manage handlers link and add a condition and response for each anticipated question.

![Shows a user ask about the sauce recipe. The response is, I'll take it to my grave'.](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/sauce.png)

After responding to the off-topic question, the prompt associated with the current empty slot is displayed.

This condition is triggered if the user provides input that matches the slot handler conditions at any time during the dialog node flow up until the node-level response is displayed.
For applications in each region, one or more load balancers must be set up with routing rules that direct traffic to the shield connection with the lowest latency.





 Horizontal Pod Autoscaling (HPA) Configuration 

Data Security Broker advises using horizontal pod scaling for each cluster to increase service availability within the cluster. Data Security Broker Shield's deployment cluster's HorizontalAutoscaler (HPA) feature can be used to configure horizontal scaling. Based on the proper resource utilization, this HPA can be configured. The HPA configuration described in the following YAML file supports scaling for increased cluster availability. Data Security Broker recommends boosting the maxReplica setting to a value appropriate for the anticipated peak load in order to scale and handle the large peak loads. It is recommended to allocate the same number of CPUs to the database server's CPUs as you would do for the maximum number of replicas.

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: Data Security Broker-pg-shield-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: Data Security Broker-pg-shield
minReplicas: 1
maxReplicas: 2
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 75
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 75
Show more





 Deploying Data Security Broker Manager for High Availability 

Currently, Data Security Broker Manager supports a high availability deployment that is active-passive. To configure and manage Data Security Broker Shields across all regions for a specific application, administrators can only use a single instance of Data Security Broker Manager.

The metadata kept by the Data Security Broker Manager instance must regularly be backed up from the primary instance and restored to the standby instance in order to guarantee that the metadata for the primary and standby Data Security Broker Managers remain consistent.
6. Click Create.

A classifier enrichment is created based on the training data that you provided.
7. Choose the collection and field where you want to apply the text classifier enrichment, and then click Apply.



The following example shows how an enrichment that is created with the sample CSV file as its training data might classify text in a document. In the output, the classifier enrichment applies the facility_temperature label to the document text. The label is stored in the enriched_{field_name} array, within the classes array.

{
"enriched_text": [
{
"classes":
{
"confidence": 0.999692440032959,
"label": "facility_temperature"
}
]
}
],
"text": [
"I think more attendees would stay awake in the sessions if the rooms were colder."
]
}



 Classifier types 

The classifier that you add from the Discovery user interface is a text classifier. A text classifier can classify documents based on words and phrases that are extracted from the body text with their part of speech information taken into account.

You can create another classifier type, a document classifier, only from the deployed Content Mining application. A document classifier can classify documents based on words and phrases that are extracted from the body text fields with information from their part of speech and the other enrichments that are applied to the body text taken into account. The information from the other non-body fields are also used.

You can apply a document classifier to a collection in a project type other than a Content Mining project. To do so, you must create the classifier in the deployed Content Mining application and export it. You can then import the classifier and apply it to your collection as an enrichment. For more information, see [Creating and applying a document classifier](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-cm-doc-classifier).

The text classifier uses Part of Speech information regardless of whether the Part of Speech enrichment is applied to the project.
(https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/gs-ass-dialog-add-about-restaurant.png)
4. Start to type about_restaurant into the If assistant recognizes field of this node. Then select the about_restaurant option.
5. Add the following text as the response.



To copy the text, click the copy icon that is associated with the text block ![Indicates you can copy the code block.](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/cloud-copy.png):

 {: codeblock}
Truck Stop Gourmand is the brain child of Gloria and Fred Smith. What started out as a food truck in 2004 has expanded into a thriving restaurant. We now have one brick and mortar restaurant in downtown Portland. The bigger kitchen brought with it new chefs, but each one is faithful to the philosophy that made the Smith food truck so popular to begin with: deliver fresh, local produce in inventive and delicious ways. Join us for lunch or dinner seven days a week. Or order a cake from our bakery.




1. Let's add an image to the response also.

Click Add response type. Select Image from the drop-down list. In the Image source field, add https://www.ibmlearningcenter.com/wp-content/uploads/2018/02/IBM-Learning-Center-Food4.jpg.
2. Move the image response type up, so it is displayed in the response before the text is displayed. Click the Move up arrow to reorder the two response types.

![Shows the image response type listed before the text response type.]
7. Review your order, and read the service agreement. If you agree with the terms, check the box.
8. Click Place Order.







 Creating the initial replica from the SLCLI 

Replications work based on a snapshot schedule. You must first have snapshot space and a snapshot schedule for the source volume before you can replicate. Then, you can use the following command to order a replica volume.

 slcli file replica-order --help
Usage: slcli file replica-order [OPTIONS] VOLUME_ID

Options:
-s, --snapshot-schedule [INTERVAL|HOURLY|DAILY|WEEKLY]
Snapshot schedule to use for replication,
(INTERVAL | HOURLY | DAILY | WEEKLY)
[required]
-l, --location TEXT Short name of the data center for the
replicant (e.g.: dal09) [required]
--tier [0.25|2|4|10] Endurance Storage Tier (IOPS per GB) of the
primary volume for which a replicant is
ordered [optional]
-h, --help Show this message and exit.





 Viewing the replica volumes in the Volume List in the UI 

You can view your replication volumes on the File Storage for Classic page under Storage > File Storage for Classic. The volume name shows the primary volume's name followed by REP. The Type is Endurance or Performance – Replica.





 Viewing the replica volumes from the SLCLI 

List existing replicant volumes for a file volume with the following command.

 slcli file replica-partners --help
Usage: slcli file replica-partners [OPTIONS] VOLUME_ID

Options:
--sortby TEXT Column to sort by
--columns TEXT Columns to display. Options: ID, Username, Account ID,
Capacity (GB), Hardware ID, Guest ID, Host ID
-h, --help Show this message and exit.





 Viewing the replication history in the UI 

To view the Replication history, click Manage on the main menu bar. Select Account, and scroll to the Audit Log.
7. Review your order, and read the service agreement. If you agree with the terms, check the box.
8. Click Place Order.







 Creating the initial replica from the SLCLI 

Replications work based on a snapshot schedule. You must first have snapshot space and a snapshot schedule for the source volume before you can replicate. Then, you can use the following command to order a replica volume.

 slcli block replica-order --help
Usage: slcli block replica-order [OPTIONS] VOLUME_ID

Options:
-s, --snapshot-schedule [INTERVAL|HOURLY|DAILY|WEEKLY]
Snapshot schedule to use for replication,
(INTERVAL | HOURLY | DAILY | WEEKLY)
[required]
-l, --location TEXT Short name of the data center for the
replicant (e.g.: dal09) [required]
--tier [0.25|2|4|10] Endurance Storage Tier (IOPS per GB) of the
primary volume for which a replicant is
ordered [optional]
-h, --help Show this message and exit.





 Viewing the replica volumes in the Volume List in the UI 

You can view your replication volumes on the Block Storage for Classic page under Storage > Block Storage for Classic. The Volume Name shows the primary volume's name followed by REP. The Type is Endurance or Performance – Replica.





 Viewing the replica volumes from the SLCLI 

List existing replicant volumes for a file volume with the following command.

 slcli block replica-partners --help
Usage: slcli block replica-partners [OPTIONS] VOLUME_ID

Options:
--sortby TEXT Column to sort by
--columns TEXT Columns to display. Options: ID, Username, Account ID,
Capacity (GB), Hardware ID, Guest ID, Host ID
-h, --help Show this message and exit.





 Viewing the replication history in the UI 

To view the Replication history, click Manage on the main menu bar. Select Account, and scroll to the Audit Log.
options list A list of key/value pairs specifying the options from which the user can choose. Y 
 options[].label string The user-facing label for the option. Y 
 options[].value object An object defining the response that will be sent to the Watson Assistant service if the user selects the option. Y 
 options[].value.input object An object that includes the message input corresponding to the option, including input text and any other field that is a valid part of a Watson Assistant message. For more information about the structure of message input, see the [API Reference](https://cloud.ibm.com/apidocs/assistant/assistant-v2?curl=message). N 







 Example 

This example displays two options:



* Option 1 (labeled Buy something) sends a simple string message (Place order), which is sent to the workspace as the input text.
* Option 2 (labeled Exit) sends a complex message that includes both input text and an array of intents.



{
"output": {
"generic":[
{
"response_type": "option",
"title": "Choose from the following options:",
"preference": "button",
"options":
{
"label": "Buy something",
"value": {
"input": {
"text": "Place order"
}
}
},
{
"label": "Exit",
"value": {
"input": {
"text": "Exit"
},
"intents":
{
"intent": "exit_app",
"confidence": 1.0
}
]
}
}
]
}
]
}
}
Show more







 pause 

Pauses before sending the next message to the channel, and optionally sends a "user is typing" event (for channels that support it).



 Fields 



 Name Type Description Required? 

 response_type string pause Y
The backend should be able to uniquely track this ID in logs and other records. Send unique ID for each send notification. Same ID can be sent in case of failure of send notification.

source+id is logged in IBM Cloud Logging service. Using these combination IBM customers are able to trace the event movement from one system to another and will aid in debugging and tracing.



 Example 

id: qwer-1234-1qsd-po94







 source (URI-reference) 

This is the identifier of the event producer. A way to uniquely identify the source of the event. For IBM Cloud services this is the crn of the service instance producing the events. For API sources this can be something the event producer backend can uniquely identify itself with.



 Example 

source: com.mybank.customerbanking.accountmanagement







 specversion (string) 

This is the version of Cloud Events specification that Event Notifications currently supports. This value must be set to "1.0".



 Example 

specversion:1.0







 type (string) 

This describes the type of event. It is of the form <event-type-name>:<sub-type>. This type is defined by the producer.

The event type name must be prefixed with the reverse DNS names so the event type is uniquely identified. The same event type can be produced by two different sources. It is highly recommended to use hyphen - as a separator instead of _.



 Example 1 

type:com.acmebank.password:expiring-in-15-days
Type: com.acmebank.password
Sub type: expiring-in-15-days





 Example 2 

type:com.acmebank.password-changed
Type: com.acmebank.password-changed
Sub type: N/A









 CE optional attributes 

Following attributes are optional but highly recommended to take full advantage of Event Notifications.



 time (timestamp) 

UTC timestamp when the event occurred. Must be in the RFC 3339 format.



 Example
* [Istio 1.10.2](https://istio.io/latest/news/releases/1.10.x/announcing-1.10.2/).



: The API version of the IstioOperator (IOP) custom resource definition is updated from v1beta1 to v1. Although this update does not impact existing custom IOP resources, you can optionally back up your IOP resources before upgrading to Istio version 1.10.
: The IstioOperator deployment is now created by the Istio add-on instead of the Operator Lifecycle Manager (OLM). If you choose to uninstall the add-on, steps are included to uninstall the [deployment resources for the operator](https://cloud.ibm.com/docs/containers?topic=containers-istiouninstall_resources).
: In multizone clusters, the zone labels for default Istio ingress gateways are now sorted in reverse order to improve resiliency. This change does not affect any ingress gateways that are already enabled and assigned a zone label in the managed-istio-custom ConfigMap.
: Resolves the following CVEs



* [CVE-2021-3520](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-3520).
* [CVE-2018-16869](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-16869).
* [CVE-2021-3580](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-3580).









 Version 1.9 (unsupported) 

Version 1.9 of the managed Istio add-on is unsupported. (: deprecated)



 Change log for 1.9.8, released 14 September 2021 

Review the changes that are in version 1.9.8 of the managed Istio add-on.

Previous version
: 1.9.7
Understanding Tune by Example 

The Tune by Example feature is beta functionality that is supported only for US English custom models and voices.

The Tune by Example feature lets you control exactly how specified text is spoken by the service. The feature lets you dictate the intonation, stress, tempo, cadence, rhythm, and pauses of the synthesized text. These aspects of speech are collectively referred to as prosody.

You create a custom prompt by providing a sample recording that speaks the text as you want to hear it, and the service duplicates the qualities of the recorded speech with its voices. The spoken prompt can emphasize different syllables or words, introduce pauses, and generally make the synthesized audio sound more natural and appropriate for the context in which the prompt is used.

The feature provides a simpler mechanism than standard SSML for modifying how speech is synthesized. For example, using attributes of the SSML <prosody> element can be difficult. Tune by Example eliminates the need for such SSML by letting you record text as you want it to be spoken by the service rather than requiring you to emulate the intended prosody with SSML elements and attributes.

You can further enhance the quality of a prompt by creating an optional speaker model that contains information about a speaker's voice. You create a speaker model by providing an audio sample of a user's voice. The service extracts information from the sample audio to train itself on the voice, which can help it produce higher-quality prompts for that speaker.

For an overview of the Tune by Example feature, including a demo with example scripts, see the blog [Tune by Example: How to Tune Watson Text to Speech for Better Intonations](https://medium.com/ibm-data-ai/tune-by-example-how-to-tune-watson-text-to-speech-for-better-intonations-bcee8404d927).



 Status and support 

The following status and support information applies to Tune by Example:



* Tune by Example is beta functionality that is available only for US English. You can add prompts only to custom models whose language is en-US. You can use a custom prompt only with with US English voices.
(fromAddress Text, toAddress Text, ccAddress Text)
external_name 'EmailMetadataSrc';





 Documenting the create external view statement with AQL Doc 

The AQL Doc comment for a create external view statement contains the following information:



* General description about the view.
* @field for every column name in the view.



/
* The external view named EmailMetadata, containing three fields
* of type Text. At run time, the view EmailMetadata is
* automatically populated from an external type named
* EmailMetadataSrc.
* @field from the fromAddress field of the email
* @field to the toAddress field of the email
* @field cc the ccAddress field of the email
/

create external view EmailMetadata(fromAddress Text, toAddress Text, ccAddress Text)
external_name 'EmailMetadataSrc';







 File formats for external artifacts 

Three types of external artifacts are supported: external views, external dictionaries, and external tables.



 External dictionary 

The format for the file that contains entries for an external dictionary is defined here:



* Carriage-return-delimited text file.
* One dictionary entry per line.
* Recommended file extension is .dict, but other file extensions can be supported.
* Entries in the dictionary can consist of multiple tokens.
* Comments can be specified by preceding the content of the comment with the character, .
* Comments can start anywhere in a line.
* Multi-line comments must contain the character at the start of each line.
* Dictionary entries can contain comment characters, if each comment character is escaped with a backslash character. For example, #.







 External table 

The supported file format for the content of an external table is a .csv file with header.

The following example shows a create external table statement, and the .csv file that specifies the content of this external table.

create external table Company2Location
(name Text, location Text)
allow_empty false;

The first line of the .csv file contains the header.
This page will allow you to see the logs being generated by the selected client (shown in the upper right of the page). The displayed logs can be filtered by the check boxes below the logs.





 Access Control List 

This page will allow you to manipulate the Access Control List for the selected client (shown in the upper right of the page). Rules can be individually added to the allow/deny tables or a file can be uploaded at the bottom of the page.





 Connection Info 

This page will show the current connection information for the selected client (shown in the upper right of the page). Information such as gateway description, number of current connections, and reverse destination listeners can be seen here.

Return to [Getting Started - Adding a Client](https://cloud.ibm.com/docs/services/SecureGateway?topic=SecureGateway-add-client).







 Remote Client Termination 

If a client has been provided an ID, then it can be remotely terminated via the SG UI or through the SG API. If you terminate a client that is running as a service, the client will restart and obtain a new client ID; however, if the service has multiple clients connected, the terminated client will not restart until all of the remaining clients have been terminated.





 Client behind proxy 

The Secure Gateway client uses outbound port 443 and port 9000 to connect to npm registry and the IBM Cloud® environment as mentioned in the [network requirement](https://cloud.ibm.com/docs/services/SecureGateway?topic=SecureGateway-client-requirementsnetwork-requirements). If the Secure Gateway client use the proxy for the outbound request, you can use following option to define the proxy configuration.



 Port 443 for gateway authentication connection and port 9000 for WSS connection