vania-janet/multiturn-rag-retrieval-data huggingface dataset:
To delete messages using customer_id values, see the [Information security](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-information-securityinformation-security-gdpr-wa) topic.







 Jupyter notebooks 

IBM created Jupyter notebooks that you can use to analyze the behavior or your assistant. A Jupyter notebook is a web-based environment for interactive computing. You can run small pieces of code that process your data, and you can immediately view the results of your computation.

You can use the notebooks with English-language skills only.



 Analysis notebooks 

There is a set of analysis notebooks that you can use with standard Python tools and a set that is designed for optimal use with IBM Watson® Studio.

Watson Studio is a product that provides an environment in which you can pick and choose the tools you need to analyze and visualize data, to cleanse and shape data, to ingest streaming data, or to create, train, and deploy machine learning models. See the [product documentation](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/welcome-main.html) for more details.

The [Watson Assistant Continuous Improvement Best Practices Guide](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/raw/master/notebook/IBM%20Watson%20Assistant%20Continuous%20Improvement%20Best%20Practices.pdf) describes how to get the most out of these notebooks.





 Using the notebooks with Watson Studio 

The following notebooks are available:



* [Dialog skill analysis for Watson Assistant](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/4d77701840fcb2f21587e39fdb887049)
* [Measure Watson Assistant Performance](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/133dfc4cd1480bbe4eaa78d3f635e568).
if events[0]==(datatypes.Event_Log{}){
return allEvents
}
allEvents=append(allEvents, events...)
eventsSize=eventsSize+pagination
}
return allEvents
}

/
Request URL:
GET https://api.softlayer.com/rest/v3.1/SoftLayer_Event_Log/getAllObjects.json?
objectFilter= filter &
objectMask= mask &
resultLimit=0,50

It returns an array of Event_Log encountered.
/
func getAllObjects(
limit int,
offset int,
filter string,
mask string,
) (resp []datatypes.Event_Log) {
var service = services.GetEventLogService(sess)
result, err := service.
Limit(limit).
Offset(offset).
Filter(filter).
Mask(mask).
GetAllObjects()
if err != nil {
fmt.Printf("n Unable to get Events:n - %sn", err)
return
}

return result
}

/
Shows the Event Logs Names.
/
func getEventNames(){
var service = services.GetEventLogService(sess)
result, err := service.GetAllEventObjectNames()
if err != nil {
fmt.Printf("n Unable to get Events:n - %sn", err)
return
}
printAsJsonFormat(result)
}

/
Shows the Event Logs User Types.
/
func getEventUserTypes() {
var service = services.GetEventLogService(sess)
result, err := service.GetAllUserTypes()
if err != nil {
Logs that are filtered out (excluded) are not archived and are not available for search. You do not pay for log lines that are filtered out.

There are different ways in which you can filter out logs sent to Log Analysis:



* You can [configure the agent to drop logs](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-exclude_logs_from_agent) before sending them to the Log Analysis service.
* If you send logs to the Log Analysis service, you can define [exclusion rules](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-exclusion_rules) to drop logs before they are stored for search.



* You can drop the log lines entirely and not see them at all through the UI.
* You can view the log lines in the UI, but you cannot search on them. However, you can define views and alerts based on the data from these logs.



* You can also configure [usage quotas](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-control_usage_quotas) and define conditional usage quota exclusion rules.





Monitoring



* Where can I find the list of Cloud services that generate metrics?

You can find information about the services that generate metrics in the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/monitoring?topic=monitoring-cloud_services).
* Are you seeing errors when there are no problems?

Are you observing monitoring agent connection errors or receiving uptime alerts reporting an host is down when there are no problems?

IBM Cloud Monitoring has identified an issue with a subset of agent versions:



* monitoring agent 10.3.0
* monitoring agent 10.3.1
* monitoring agent 10.4.0
* monitoring agent 10.4.1



Where connectivity between your infrastructure and Monitoring's hosted service may fail.

You must upgrade all monitoring agents to 10.5. [Learn more](https://cloud.ibm.com/docs/monitoring?topic=monitoring-troubleshoottroubleshoot-entry-3).
Filtering actions 

IBM Cloud

You can locate specific actions by filtering the list.

You can filter by:



Filters

 Filter     Description                                                                                                                                                                                                                                                                                                                                                                                                                                                  

 Name       All or part of the action name                                                                                                                                                                                                                                                                                                                                                                                                                               
 Subaction  You can filter by subaction by:<br><br><br><br> *  Choosing a subaction from the dropdown list. The list shows other actions that trigger, or call, that subaction in Go to a subaction at the end of a step.<br> *  Clicking the Show all subactions checkbox. The list shows actions that are triggered, or called, by other actions in Go to a subaction at the end of a step.<br><br><br>             
 Extension  Choose a custom extension to see all the actions that use it. Or, choose Any to see all the actions that use at least one custom extension.                                                                                                                                                                                                                                                                                                                  
 Variable   Choose a variable to see all the actions that use it.                                                                                                                                                                                                                                                                                                                                                                                                        



To filter by all or part of a name, make an entry in the Filter by name field. The list is filtered as you type.

![Filter by name](https://cloud.ibm.com/docs-content/v1/content/ca9232b3702768f24bb897f48f10778fcda28718/watson-assistant/images/filter-action-name.png)

To filter by action, extension, or variable:



1.  Click the Filters icon ![Filters](https://cloud.ibm.com/docs-content/v1/content/icons/filter.svg) to see the filter fields.
2.  Use the Subaction, Extension, or Variable dropdown lists, and then click Apply.
3.  Or, click the Show all subactions checkbox, and then click Apply.
In addition to this, on MQ Explorer, you can browse a queue to view messages In the runmqsc command line, you can also get messages from a queue





 Before you begin 



* An existing queue manager (for instructions, follow the [creating a queue manager](https://cloud.ibm.com/docs/services/mqcloud?topic=mqcloud-mqoc_create_qm) guide).
* You have permission to access queue managers within your IBM MQ service instance (for instructions, follow the [configuring administrator access for a queue manager](https://cloud.ibm.com/docs/services/mqcloud?topic=mqcloud-tutorial-configure-admin-access) guide).
* Depending on your method of administration, either:





* An existing installation of IBM MQ Explorer. Download and installation instructions for Windows and Linux can be obtained from [here](http://www-01.ibm.com/support/docview.wss?uid=swg24021041) and for Mac OS from [here](https://github.com/ibm-messaging/mq-container/tree/master/incubating/mq-explorer). or
* An existing installation of IBM MQ Client on your own machine.





* Download the client from [here](http://www-01.ibm.com/support/docview.wss?uid=swg240421761).



* Clicking the HTTP link next to the latest available version of the CD Clients will take you to Fix Central. From there you can search for and select the appropriate Redist (redistributable) client bundle for your operating system platform. This will include the sample applications and runmqsc.
* Once downloaded, unpack the bundle into a location of your choosing.
* Make a note of the full path to the bin directory, the location of which will depend upon where you chose to unpack the bundle. This path will be referenced as <PATH_TO_BIN_DIR> for the rest of this task.
To modify an existing prefix filter, click the Actions menu ![Actions menu](https://cloud.ibm.com/docs-content/v1/content/6a28e4ae2660b609a18e90e1239ed772f80373d6/transit-gateway//images/overflow.png) and select Prefix filtering.
3. If the connection does not have existing prefix filters, click the Actions menu ![Actions menu](https://cloud.ibm.com/docs-content/v1/content/6a28e4ae2660b609a18e90e1239ed772f80373d6/transit-gateway//images/overflow.png) and select Prefix filtering.
4. Adjust the default filter as needed. Your choices are as follows:



* Permit prefixes (default) indicates that all prefixes in this connection are accessible to all other connections in this transit gateway. In this case, "all" means up to the established quota and limits.
* Deny prefixes indicates that no prefixes from this connection are accessible to any other connection on this transit gateway.



5. Click Create prefix filter, then configure the filter with the following options:



* Select an action.
* Enter the network prefix along with the subnet mask.
* Optionally, enter values if the network should be greater than or equal to a desired subnet mask.
* Adjust the order of the filter in the routing table as needed, then click Save.









 Deleting prefix filters 

To delete a prefix filter for an existing connection in the UI, follow these steps:



1. From the details page of the transit gateway, identify the connection where you want to delete prefix filters.

The prefix filtering icon ![Prefix filter icon](https://cloud.ibm.com/docs-content/v1/content/6a28e4ae2660b609a18e90e1239ed772f80373d6/transit-gateway/images/prefix-filter-icon.png) shows if a connection already has prefix filters.
2.
"message": [],
"performed_by": "user@us.ibm.com",
"performed_at": "2019-08-21T14:07:25.814722573Z",
"templates": [
{
"template_id": "a1234b-2124",
"template_type": "terraform_v0.12",
"start_time": "2019-08-21T14:07:26.120106199Z",
"end_time": "2019-08-21T14:07:30.088495607Z",
"status": "COMPLETED",
"message": "{"messagekey":"M2001_ActivitySuccessful","parms":{},"requestid":"1234b1234b123-1234n13","timestamp":"2019-08-21T14:07:30.088498532Z"}",
"log_url": "https://schematics.cloud.ibm.com/v1/workspaces/myworkspace-123456/runtime_data/a1234b-2124/log_store/actions/11223334b11224345"
}
]
}
INFO:root:Completed Retrieving the Schematic logs
If you try to set a percentage where an exclusion rule exists, you will be warned that the existing rule will be replaced.
3. Select the source of the logs you want to exclude.
4. Select the apps you want to exclude.
5. Specify a [query](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-views) to select the log lines you want to exclude.
6. Select if your want to discard all matching log lines until the next billing cycle and if you want those log lines preserved for live tail and alerting.

All configured alert recipients will be alerted when the percentage threshold of the monthly quota is reached. This behavior cannot be changed.
7. Click Apply to save the exclusion rule.

Check the query in a custom view by entering the search criteria in the search bar of the Everything view, and validating that the data that is displayed is the data that you want excluded.







 Enable and disabling exclusion rules 

Toggle the selection next to the exclusion rules to enable or disable the rule.

Zoom

![Example switch to enable or disable exclusion rule](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/usage_quota_rule_switch.png)

Figure 1. Example switch showing exclusion rule is enabled





 Editing exclusion rules 

To edit an exclusion rule click Edit next to the exclusion rule that you want to change.







 Configuring usage quota alerts 

By default the account owner is configured to receive alert emails. You can configure your systems to send email alerts, Slack notifications, or both.

To configure notifications when a quota threshold is exceeded, do the following:



1. Under Alert recipients select the desired recipient email addresses. There must be at least one email address configured if Slack notifcations are not configured.
2. If desired, select one or more Slack recipients to receive notifications.
* To remove specific log events.



Reducing logs helps reduce logging costs. However, you should keep all the logs that you need for operations, troubleshooting and compliance.

Complete the following steps to configure filtering in an ESXi host:



1. Launch the vSphere Client.
2. Select the ESXi host that you want to modify. Then, select Configure > System > Advanced System Settings.
3. Click Edit and search for entries that have the prefix Syslog.global.logFilters.
4. Enable Syslog.global.logFiltersEnable to apply log filters.
5. Enter your filtering conditions in Syslog.global.logFilters.

You can specify 1 or more conditions. You separate conditions by using a double vertical bar ||.

Use the format numLogs | ident | logRegexp.

numLogs defines the number of log entries that are logged and match the expression in logRegexp. Once the number is reached, log lines are dropped. When you set this value to 0, all the log lines that match the expression defined in logRegexp are dropped.

indent defines the system components where the filter is applied. You can define filters for components listed in the /etc/vmsyslog.conf.d directory of your ESXi host.

logRegexp defines the log expression that is applied to filter logs. It is a case-sensitive phrase with Python regular expression syntax.

For example, see the following condition on log entries in the vnkernel component: 2 | vmkernel | bad CDB .* scsi_op=0x9e || 0 | vmkernel | 0x1a. H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x2[04] 0x0
6. Restart the ESXi host to reload the syslog daemon and activate the log filters.









 Step 3. Verify logs in Log Analysis 

Complete the following steps:



1.
From the details page of the transit gateway, identify the connection where you want to delete prefix filters.

The prefix filtering icon ![Prefix filter icon](https://cloud.ibm.com/docs-content/v1/content/6a28e4ae2660b609a18e90e1239ed772f80373d6/transit-gateway/images/prefix-filter-icon.png) shows if a connection already has prefix filters.
2. Click the connection's Actions menu ![Actions menu icon](https://cloud.ibm.com/docs-content/v1/content/6a28e4ae2660b609a18e90e1239ed772f80373d6/transit-gateway//images/overflow.png) and select Prefix filtering.
3. Click the Actions menu ![Actions menu icon](https://cloud.ibm.com/docs-content/v1/content/6a28e4ae2660b609a18e90e1239ed772f80373d6/transit-gateway//images/overflow.png) next to the prefix filter that you want to delete. Then, click Delete.
4. Click Delete prefix filter to confirm deletion.









 Working with prefix filters from the CLI 

You can add prefix filters when you add a new connection using the CLI. You can also delete them.



 Adding prefix filters from the CLI 

To add prefix filters from the CLI, follow these steps:

ibmcloud tg prefix-filter-create GATEWAY_ID CONNECTION_ID --prefix PREFIX --action ACTION [--le LE] [--ge GE] [--before BEFORE] [--output json]

Where:



* GATEWAY_ID: ID of the gateway that the prefix filter is being applied to.
* CONNECTION_ID: ID of the connection that the prefix filter is being applied to.
* --prefix: Network prefix that the filter will be applied to.
* --action: Action to take on the specified prefix (permit | deny).
You can track recipients who opt into the Event Notifications dashboard.

By adding email addresses, you represent on behalf of yourself and your company that you inform the individuals, to whom the added emails pertain, of their addition to this recipient list and purpose thereof, and have the required consents to do so.

After a subscription is created, the list of users in the Invited tab automatically receives initial message with information that they are invited to subscribe to the list, which is the "Opt-in" message.

Opt-in message contains user or account who invited the recipient, the name of the subscription, the purpose of the notifications, the frequency of expected notifications, a way to accept or reject the invitation, and expiration time for the invitation.

The Event Notifications are only sent to the opted-in recipients.

You can either resend the invitation or remove the recipient from the Invited list. In the Invited tab, click and select the three vertical dots (overflow menu) and select Resend invitation for the recipient email address, to whom you need to resend the invitation. For deleting a user from the invited list, in the Invited tab, click the three vertical dots (overflow menu) and select Delete for the recipient email address, to whom you need to remove from the Invited list. For adding back a recipient after opted-out or not responded within the stipulated time that is mentioned in the invite email, you need to send a mail to the Reply to email address mentioned in the initial invite mail.





 Email charges 

Every message that is sent by the IBM Cloud email service counts as one outbound digital message and is charged. For example, if a subscription has 100 email addresses and the subscription topic received 1000 events during the month is as follows:

Total Outbound Digital messages = (number of email addresses) x (topic events)

In the example, the total outbound digital messages are as follows:

100 x 1000 = 100000 outbound digital messages

You are charged for messages that are successfully sent by the IBM Cloud email service regardless of whether the remote email server successfully delivered the message to the recipient. For example, bounced emails still count as outbound digital messages. So vet your email list carefully to prevent unnecessary charges.
You must choose the source and app, and leave blank the query field.

You can enter a query to define the exclusion rule, or to refine the exclusion rule when you specify a source, an app, or both.
6. Select **Preserve these lines for live-tail and alerting ** to show through the live tail the log lines that are excluded. Notice that you can still use these log lines to set up an alert.
7. Click Save.
8. After you configure an exclusion rule, verify that the exclusion rule behaves as you expect.

Check the query in a custom view by entering the search criteria in the search bar of the Everything view, and validating that the data that is displayed is the data that you want excluded.





 Sample 1: Exclude syslog data for a worker while keeping entries that report errors only 

You will configure the rule so that you are able to see all log data through views and be able to define alerts on all the data.

Complete the following steps to define the exclusion rule:

Prereq: You must have a cluster configured to forward logs to a logging instance. [Learn more](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-config_agent_kube_cluster).



1. [Launch the IBM Log Analysis web UI](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-view_logsview_logs_step2).
2. Click the Settings icon ![Configuration icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/admin.png). Then select Usage > Exclusion Rules.
3. Select Add Rule. The Create Rule section opens.
4. Enter a name for the rule in the section What is this rule for?. For example, enter Worker X no syslog data.
5.
-c, --cluster CLUSTER
: Required: The name or ID of the cluster that you want to update a logging filter for.

--id FILTER_ID
: The ID of the log filter to update.

--type LOG_TYPE
: The type of logs that you want to apply the filter to. Currently all, container, and host are supported.

-lc, --logging-config CONFIG
: Optional: The logging configuration ID. If not provided, the filter is applied to all the cluster logging configurations that are passed to the filter. You can view log configurations that match the filter by using the --show-matching-configs option with the command. To specify multiple IDs, use multiple options, such as -lc id1 -lc id2.

-n, --namespace KUBERNETES_NAMESPACE
: Optional: The Kubernetes namespace from which you want to filter logs.

--container CONTAINER_NAME
: Optional: The name of the container from which you want to filter out logs. This option applies only when you are using log type container.

--level LOGGING_LEVEL
: Optional: Filters out logs that are at the specified level and less. Acceptable values in their canonical order are fatal, error, warn/warning, info, debug, and trace. As an example, if you filtered logs at the info level, debug, and trace are also filtered. Note: You can use this option only when log messages are in JSON format and contain a level field. Example output {"log": "hello", "level": "info"}

--message MESSAGE
: Optional: Filters out any logs that contain a specified message anywhere in the log. Example: The messages "Hello", "!", and "Hello, World!", would apply to the log "Hello, World!".

--regex-message MESSAGE
: Optional: Filters out any logs that contain a specified message that is written as a regular expression anywhere in the log.