vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* [Security Bulletin: A Google Guava vulnerability affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2018-10237)](https://www.ibm.com/support/pages/node/6575477)
* [Security Bulletin: Vulnerabilities in Apache Tomcat affect IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2022-23181)](https://www.ibm.com/support/pages/node/6575481)
* [Security Bulletin: A Cyrus SASL vulnerability affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2022-24407)](https://www.ibm.com/support/pages/node/6575483)
* [Security Bulletin: A vulnerability with GNU wget affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2016-4971)](https://www.ibm.com/support/pages/node/6575485)
* [Security Bulletin: A vulnerability with GNU Wget affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2018-0494)](https://www.ibm.com/support/pages/node/6575487)
* [Security Bulletin: A vulnerability in 'GNU Wget' affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2018-20483)](https://www.ibm.com/support/pages/node/6575489)
* [Security Bulletin: A vulnerability in ISC BIND affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2018-5741)](https://www.ibm.com/support/pages/node/6575493)
* [Security Bulletin: A vulnerability in Python affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2019-20916)](https://www.ibm.com/support/pages/node/6575495)
Sensitive data, for example API keys or passwords, are not included in generated events.

The metadata properties that are sent to Event Notifications vary depending on the event type and subtype. For example, if an com.ibm.cloud.compliance.posture:posture-scan-completed event takes place, Security and Compliance Center sends a notification payload to Event Notifications that is similar to the following example.

{
"action": "Validation completed",
"summary": "my-test-scan",
"scope": "my-test-scope",
"profile": "CIS IBM Foundations Benchmark 1.0.0",
"start_time": "2021-11-24 13:46:21.672",
"end_time": "2021-11-24 13:46:21.672"
}

Review the following table for more information about event notification properties.



Table 2. Properties in an event notification payload

 Property Description 

 action The [type of event](https://cloud.ibm.com/docs/security-compliance?topic=security-compliance-event-notificationsevent-notifications-list) that triggered a notification. 
 summary The name of the validation scan. 
 scope The name of the scope that is associated with the scan. 
 profile The name of the profile that is associated with the scan. 
 start_time The date and time the scan started. 
 end_time The date and time the scan completed.
* [Security Bulletin: IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data is vulnerable to arbitrary code execution in SQlite (CVE-2020-35527)](https://www.ibm.com/support/pages/node/6981851)
* [Security Bulletin: IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data is vulnerable to a security restrictions bypass in GNU Libtasn1 (CVE-2021-46848)](https://www.ibm.com/support/pages/node/6981853)
* [Security Bulletin: IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data is vulnerable to arbitrary code execution in Git (CVE-2022-23521)](https://www.ibm.com/support/pages/node/6981857)
* [Security Bulletin: IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data is vulnerable to arbitrary code execution in GnuPG Libksba (CVE-2022-3515)](https://www.ibm.com/support/pages/node/6981855)
* [Security Bulletin: IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data is vulnerable to an arbitrary code execution in libexpat (CVE-2022-40674)](https://www.ibm.com/support/pages/node/6981859)
* [Security Bulletin: IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data is vulnerable to arbitrary code execution in Git (CVE-2022-41903)](https://www.ibm.com/support/pages/node/6981861)







 23 February 2023 (Version 4.6.3) 

Version 4.6.3 is now available
: Speech to Text for IBM Cloud Pak for Data version 4.6.3 is now available. This version supports IBM Cloud Pak for Data version 4.6.x and Red Hat OpenShift version 4.10. Red Hat OpenShift version 4.8 is no longer supported.
Associating message data with a user for deletion 

There might come a time when you want to completely remove a set of your user's data from a Watson Assistant instance. When the delete feature is used, then the Overview metrics will no longer reflect those deleted messages; for example, they will have fewer Total Conversations.



 Before you begin 

To delete messages for one or more individuals, you first need to associate a message with a unique Customer ID for each individual. To specify the Customer ID for any message sent using the /message API, include the X-Watson-Metadata: customer_id property in your header. You can pass multiple Customer ID entries with semicolon separated field=value pairs, using customer_id, as in the following example:

curl -X POST -u "apikey:3Df... ...Y7Pc9" --header "Content-Type: application/json" "X-Watson-Metadata: customer_id={first-customer-ID};customer_id={second-customer-ID}" --data "{"input":{"text":"hello"}}" "{url}/v2/assistants/{assistant_id}/sessions/{session_id}/message?version=2019-02-28"

where {url} is the appropriate URL for your instance. For more details, see [Service endpoint](https://cloud.ibm.com/apidocs/assistant-data-v2service-endpoint) }.

The customer_id string cannot include the semicolon (;) or equal sign (=) characters. You are responsible for ensuring that each Customer ID parameter is unique across your customers.

To delete messages using customer_id values, see the [Information security](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-information-securityinformation-security-gdpr-wa) topic.







 Jupyter notebooks 

IBM created Jupyter notebooks that you can use to analyze the behavior or your assistant. A Jupyter notebook is a web-based environment for interactive computing.
For more information, see [Configuring backup support](https://cloud.ibm.com/docs/assistant?topic=assistant-deploy-phonedeploy-phone-transfer-service).
* Call failure message: Add the message to say to a caller before you transfer them to a human agent.



If, after you transfer the caller to a human agent, the connection to the human agent fails for any reason, you can configure what to do.



* Transfer failure message: Add the message to stream to callers if the transfer to a human agent fails. The message can be up to 1,024 characters in length.
* Disconnect call on transfer failure: Choose whether to disconnect the call after sharing the failure message. This option is enabled by default. If disabled, when a call transfer fails, your assistant can disconnect or process a different dialog node.







 Secure the phone connection 

You can add security to the phone connection by selecting one or both of the following configuration options:



* Force secure trunking: Select this option to use Secure Real-Time Transfer Protocol (SRTP) to secure the audio that is transmitted over the phone. For more information about RTP, see [Call routing details](https://cloud.ibm.com/docs/assistant?topic=assistant-deploy-phonedeploy-phone-route).
* Enable SIP authentication: Select this option if you want to require SIP digest authentication.

When SIP authentication is required, all inbound traffic (meaning requests from the SIP provider to your assistant) is authenticated using SIP digest authentication, and must be sent using Transport Layer Security (TLS). If this option is selected, the SIP digest user name and password must be configured, and the SIP trunk being used to connect to Assistant must be configured to use only TLS.

If you use Twilio as your SIP trunk provider, you cannot enable SIP authentication for outbound SIP trunks to Watson Assistant.







 Apply advanced SIP trunk configuration settings 



* SIP INVITE headers to extract: List headers that you want to use in your dialog.

The SIP request often sends INVITE headers with information about the request that is used by the SIP network.
Since the queue manager has the capability of encrypting and decrypting the messages, the recipient has no guarantee that the message that it receives is authentic. For example, in the cases of B2B, the topology is that of a hub and spoke where the clearinghouse serves as the hub and each spoke is a different business partner. In such scenarios for assurance of a tamper-proof and authentic message, an end to end protection will be the desired solution.

Application AMS provides end to end Message-level security by cryptographically binding the sender’s identity, as represented by the distinguished name in an X.509 certificate, to the message. If desired, the message can also be encrypted for specific recipients who are also identified by X.509 certificates. The ability to bind sender and recipient identities to a message and make it tamper-proof facilitates the creation of policies as to who can send or receive messages, and the enforcement of those policies is on a per-message basis.

This tutorial uses self-signed certificates for sender and receiver. It is highly recommended to use CA certificates for production use cases. Self-signed certificates are used only for test and demo purposes.







 Before you begin 



1. An existing MQ on Cloud queue manager or [create a queue manager](https://cloud.ibm.com/docs/services/mqcloud?topic=mqcloud-mqoc_create_qm) if you don't have one.
2. The connection details for your queue manager, see Appendix 1
3. An Admin username nad apikey, see Appendix 2
4. An IBM MQ Client, this tutorial uses runmqsc, to install see Appendix 3







 Setup Sender and Receiver of Message 

This tutorial uses two users alice and bob for the AMS setup and to demonstrate the end to end message security- alice as a sender of the message and bob as a recipient of the message. In the real world, senders and receivers of messages run on different systems and enforcing the message protection among those senders and receivers of the message using AMS ensures messages are tamper-proof and authentic.
Only data that was added by using the POST /message API endpoint with an associated customer ID can be deleted using this delete method. Data that was added by other methods cannot be deleted based on customer ID. For example, entities and intents that were added from customer conversations, cannot be deleted in this way. Personal Data is not supported for those methods.

IMPORTANT: Specifying a customer_id will delete all messages with that customer_id that were received before the delete request, across your entire Watson Assistant instance, not just within one skill.

As an example, to delete any message data associated with a user that has the customer ID abc from your Watson Assistant instance, send the following cURL command:

curl -X DELETE -u "apikey:3Df... ...Y7Pc9" "{url}/v2/user_data?customer_id=abc&version=2020-04-01"

where {url} is the appropriate URL for your instance. For more details, see [Service endpoint](https://cloud.ibm.com/apidocs/assistant/assistant-v2service-endpoint).

An empty JSON object {} is returned.

For more information, see the [API reference](https://cloud.ibm.com/apidocs/assistant/assistant-v2deleteuserdata).

Note: Delete requests are processed in batches and may take up to 24 hours to complete.







 Web chat usage data 

The Watson Assistant web chat sends limited usage data to the [Amplitude service](https://amplitude.com/). When the web chat widget is being interacted with by a user, we track the features that are being used, and events such as how many times the widget is opened and how many users start conversations. This information does not include Assistant training data or the content of any chat interactions. The information being sent to Amplitude is not Content as defined in the Cloud Service Agreement (CSA); it is Account Usage Information as described in Section 9.d of the CSA and is handled accordingly as described in the [IBM Privacy Statement](https://www.ibm.com/privacy).
* A potential security risk exists even though a basic firewall is installed and preconfigured.
* If you want to use a public interface, be sure to choose a sufficiently high value for Public Bandwidth when you order your server. This value determines the total amount of data that can be transferred through the interface during a one-month period. You either need to know the amount of data that is transferred at least by the order of magnitude or switch to the second option.



Access to the resources on the IBM Cloud Classic Infrastructure is available with greater security:



* Using the SSL Virtual Private Network (VPN), default available through the IBM Cloud Classic Infrastructure console
* Using the IPSec Virtual Private Network (VPN)
* Using a Gateway Appliance with Firewall, Network Address Translation (NAT)
* Direct Link



These connectivity options can provide you with a higher bandwidth, which help you in transferring larger amounts of data into an IBM Cloud data center.

It is advised that your networking department contact [IBM Cloud Support](https://cloud.ibm.com/docs/get-support?topic=get-support-using-avatargetting-support) after you determine the layout of your landscape and the connectivity that is required on the SAP application layer.







 Storage 

Within Classic Infrastructure, three types of storage are available:



* Local Disk Storage (SSD)
* Network Block Storage (LUNs only) that uses IBM Cloud Block Storage (Classic)
* Network File Storage (file system installed) that uses IBM Cloud File Storage (Classic)



Local disk storage is provided with your Bare Metal Servers and available in your choice of RAID configuration (by using a dedicated RAID Controller). It is ideal for storage-intensive applications with high I/O needs, such as an OS, database, and application software. This storage is the perfect companion for SAP HANA workloads.

Bare Metals with local disk storage use disks physically inserted into the drive bays; it is not using a SmartNIC to present network block storage as if it were local disk storage.

In addition to the local storage, you might require network storage (to expand beyond the Local SSD Disk total capacity, or for redundancy and backups).
If you set it to high, check the logs on CIS, and fine-tune the OWASP rule set to work for your application.

Keep in mind that you can only toggle OWASP rules on or off, unlike rules in the CIS rule sets, which can be set to Disable, Simulate, Challenge, or Block.





 Understanding the OWASP package 

The OWASP ModSecurity Core Rule Set assigns a score to each request based on how many OWASP rules trigger. Some OWASP rules have a higher sensitivity score than others. After OWASP evaluates a request, CIS compares the final score to the Sensitivity configured for the domain. If the score exceeds the Sensitivity, the request is handled based on the Action that is configured within Package:

The sensitivity score required to trigger the WAF for a specific Sensitivity is as follows:



Table 2. Sensitivity and triggers

 Sensitivity score Trigger threshold 

 Low 60 and higher 
 Medium 40 and higher 
 High 25 and higher 



The OWASP ModSecurity Core rule set takes the following actions:



Table 3. OWASP ModSecurity Core Rule Set

 Action Definition 

 Block The request is discarded. 
 Challenge The visitor receives a CAPTCHA challenge page. 
 Simulate The request is allowed through but is logged in the Events log. 



For Ajax requests, the following scores are applied instead:



Table 4. Ajax request sensitivity and triggers

 Sensitivity score Trigger threshold 

 Low 120 and higher 
 Medium 80 and higher 
 High 65 and higher 



Review the (Security) Events log to see the final score as well as the individual triggered rules.





 Managing OWASP packages 

The OWASP ModSecurity Core Rule Set contains several rules from the OWASP project. CIS does not write or curate OWASP rules. Click on a ruleset name under Group to reveal the rule descriptions. Unlike the CIS Managed Ruleset, specific OWASP rules are either turned On or Off.

To manage OWASP thresholds, set the Sensitivity to Low, Medium, or High in the Package: OWASP ModSecurity Core Rule Set section. Setting the Sensitivity to Off disables the entire OWASP package, including all its rules.
For any other connections into your location that your applications require, you can use Satellite Link to create layer 4 communications by setting up an endpoint for each destination resource in your location. All connections through your endpoints are always under your control, including completely disabling endpoints.





 How do I make my data secure in transit? 

Link endpoints between your location and IBM Cloud are secured through two levels of encryption: high-security encryption from the location’s connector to IBM Cloud that is provided by IBM , and an optional additional encryption layer between the source and destination resources.

All data that is transported over Satellite Link is encrypted using TLS 1.3 standards. This level of encryption is managed by IBM.

When you create an endpoint, you can optionally provide another level of encryption by specifying [data encryption protocols](https://cloud.ibm.com/docs/satellite?topic=satellite-link-location-cloudlink-protocols) for the endpoint connection between the client source and destination resource. For example, even if the traffic is not encrypted on the source side, you can specify TLS encryption for the connection that goes over the internet. You can provide your own signed certificates to ensure both internal security and operational auditability without exposing any data contents. IBM only transports the encrypted connection, and your resources must be configured for the data encryption protocols that you specify.







 Encryption protocols 

All communication over Satellite Link is encrypted by IBM. When you create an endpoint, you can optionally specify an additional data encryption protocol for the endpoint connection between the client source and destination resource. For example, even if the traffic is not encrypted on the source side, you can specify your own additional TLS encryption for the connection that goes over the internet. Note that your resources must be configured for the data encryption protocols that you specify.

Review the following information about how Satellite Link handles each type of connection protocol.

If you use the Satellite console to create an endpoint, the destination protocol is inherited from the source protocol that you select. To specify a destination protocol, use the CLI to create an endpoint and include the --dest-protocol option in the ibmcloud sat endpoint create command.



 TCP and TLS
{
"result_index": 0,
"results": [
{
"alternatives":
{
"confidence": 0.96,
"transcript": "several tornadoes touch down as a line of severe thunderstorms swept through Colorado on Sunday "
}
],
"final": true
}
]
}







 Making a multipart HTTP speech recognition request 

The asynchronous HTTP interface, WebSocket interface, and Watson SDKs do not support multipart speech recognition.

The POST /v1/recognize method also supports multipart requests for speech recognition. You pass all audio data as multipart form data. You specify some parameters as request headers and query parameters, but you pass JSON metadata as form data to control most aspects of the transcription.

Multipart speech recognition is intended for the following use cases:



* To pass multiple audio files with a single speech recognition request.
* With browsers for which JavaScript is disabled. Multipart requests based on form data do not require the use of JavaScript.
* When the parameters of a recognition request are greater than 8 KB, which is the limit imposed by most HTTP servers and proxies. For example, spotting a very large number of keywords can increase the size of a request beyond this limit. Multipart requests use form data to avoid this constraint.



The following sections describe the parameters that you use for multipart requests and show an example request.



 Parameters for multipart requests 

You specify a number of parameters as form data, request headers, or query parameters. For more information about request headers and query parameters, see the [Parameter summary](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-summary).



 Form data 

You specify the following parameters of multipart speech recognition requests as form data:

metadata (required object)
: A JSON object that provides the transcription parameters for the request. The object must be the first part of the form data. The information describes the audio in the subsequent parts of the form data.
For more information, see [audio/basic format](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-audio-formatsaudio-formats-basic).

Speech recognition methods now return warnings for invalid parameters
: The various recognition methods can return a warnings response that includes messages about invalid query parameters or JSON fields that are included with a request. The format of the warnings changed. For example, "warnings": "Unknown arguments: [u'{invalid_arg_1}', u'{invalid_arg_2}']." is now "warnings": "Unknown arguments: {invalid_arg_1}, {invalid_arg_2}."

Empty body required for HTTP POST methods that pass no data
: For HTTP POST requests that do not otherwise pass data to the service, you must include an empty request body of the form {}. With the curl command, you use the --data option to pass the empty data.





 10 March 2016 

New maximum limits on audio transmitted for speech recognition
: Both forms of data transmission (one-shot delivery and streaming) now impose a size limit of 100 MB on the audio data, as does the WebSocket interface. Formerly, the one-shot approach had a maximum limit of 4 MB of data. For more information, see [Audio transmission](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-inputtransmission) (for all interfaces) and [Send audio and receive recognition results](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-websocketsws-audio) (for the WebSocket interface). The WebSocket section also discusses the maximum frame or message size of 4 MB enforced by the WebSocket interface.

HTTP and WebSocket interfaces can now return warnings
: The JSON response for a recognition request can now include an array of warning messages for invalid query parameters or JSON fields that are included with a request. Each element of the array is a string that describes the nature of the warning followed by an array of invalid argument strings.
* [Security Bulletin: A vulnerability in ISC BIND affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2018-5741)](https://www.ibm.com/support/pages/node/6575493)
* [Security Bulletin: A vulnerability in Python affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2019-20916)](https://www.ibm.com/support/pages/node/6575495)
* [Security Bulletin: A vulnerability with ISC BIND affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2021-25214)](https://www.ibm.com/support/pages/node/6575497)
* [Security Bulletin: A vulnerability in ISC BIND affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2021-25215)](https://www.ibm.com/support/pages/node/6575499)
* [Security Bulletin: A vulnerability in ISC BIND affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2021-25216)](https://www.ibm.com/support/pages/node/6575503)
* [Security Bulletin: A vulnerability in ISC BIND affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2021-25219)](https://www.ibm.com/support/pages/node/6575505)
* [Security Bulletin: A vulnerability in PostgreSQL JDBC Driver (PgJDBC) affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2022-21724)](https://www.ibm.com/support/pages/node/6575507)
* [Security Bulletin: A vulnerability in GNU Tar affects IBM Watson Speech Services Cartridge for IBM Cloud Pak for Data (CVE-2019-9923)](https://www.ibm.com/support/pages/node/6575509)