vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Managing presets through the UI 

To reuse an alert configuration, a service administrator can configure an alert preset (alert template).



 Creating a preset 

Complete the following steps to configure a preset:



1. In the web UI, select the Settings icon ![Settings icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/alerts/images/admin.png).
2. Select ALERTS.
3. Select Add Preset.
4. Specify a name for the preset.
5. Select the type of alert (Slack, Email, Webhook, or PagerDuty). If you selected the wrong alert type and you want to change it, click Delete Alert Channel.
6. Configure the type of alert.



1. Select if you want the alert to be sent when the condition exists (Presence) or does not exist (Absence).
2. Indicate the logging criteria when an alert should be sent. For example, when 100 lines matching in the view are logged in an hour. A graph will help you determine the number of log lines matching your specified criteria.
3. Select if the alert should be sent at the end of the selected period or immediately when the number of lines are logged.
4. Optionally you can specify a Custom schedule with alerting limited to a specified timezone, days of the week, or timeframe. To configure a Custom schedule:



1. Select on for Custom schedule.
2. Select the Timezone for the log entries.
3. Select the days of the week when alerts should be generated.
4. Optionally specify a time range for the selected days. A graph will help you determine the number of log entries for the timezone and time range.



5.
* --webhooks value is the webhook IDs for dispatching an alert notification. For example, --webhooks webhookID1,webhookID2.
* --domains value is the domain IDs for the alert policy. For example, --domains domainID1,domainID2.
* --services value specifies which services the alert monitors (for advanced security alerts). Valid services: country-access-rules, waf, firewall-rules, ratelimit, securitylevel, ip-access-rules, browser-integrity-check, ua-rules,lockdowns, iprange-access-rules, asn-access-rules, managed-firewall, hotlink, ssl-validation. (Enterprise plan only.)
* --enabled value determines whether the alert policy is enabled.
* -i, --instance value is the instance name or ID.
* --output value specifies the output format; only JSON is supported.



Security alerts and advanced security alerts use the same command. When you create an advanced security events alert command in the CLI, specify the services for the alert. If you do not specify the services for the alert, the mean detection time changes from 5 minutes to 2 hours.





 Updating a DDoS Attack Alert Policy by using the CLI 

To update a DDoS attack alert policy by using the CLI, run the following command:

ibmcloud cis alert-policy ddos-attack-l7-alert-update POLICY_ID [--name NAME] [--emails EMAILS] [--webhooks WEBHOOKS] [--enabled (true | false)] [--description DESCRIPTION] [-i, --instance INSTANCE] [--output FORMAT]

Where:



* POLICY_ID is the ID of alert policy.
* --name value is the name of the alert policy.
* --description value is the description for the alert policy.
* --emails value is the email addresses for dispatching an alert notification. For example, --emails test1@cn.ibm.com,test2@cn.ibm.com.
PagerDuty: Specify the Service. If required, you will be prompted to connect to PagerDuty.

Webhook: Specify the Method & URL, and the Headers and Body of the JSON used to interact with your webhook. You can use Validate JSON to make sure your JSON is correct before creating the alert.
7. You can click Test to test that your alert configuration is correct.

![Test option](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/alerts/images/alert_test.png)
8. Click Save Alert.
9. If you want to create an additional alert channel, click + and follow the prior steps to create additional channels. You can create different channel types, for example, an email and a Slack channel. Or, you can create multiple channels of the same type with different alert criteria.



Alerts will be generated based on your configuration.
{"notificationChannel":{"id":39209,"version":1,"customerId":34292,"enabled":true,"sendTestNotification":true,"createdOn":1678967870764,"modifiedOn":1678967870764,"name":"thursTest","options":{"notifyOnOk":true,"emailRecipients":["email@email.com"],"notifyOnResolve":true},"type":"EMAIL"}}% 

You have now created a notification channel for your alerts.

Make a note of the id field that is returned by the API call.





 Step 3: Create the alert 

Now that you have a notification channel, create your [alert rule](https://cloud.ibm.com/docs/monitoring?topic=monitoring-alert_api). Your alert rule describes the metric query to be monitored, the threshold value, and the action to take when the threshold is crossed. In this case, you're monitoring your ibm_service_instance_name to ensure that max utilization doesn't exceed 90%. If that happens, an alert is triggered and you're notified.

This alert is triggered at 90% disk utilization. However, 50-70% disk utilization is preferred.

To retrieve the name of the database instance you want to set up the alert for, list all your database instances with a command like:

ibmcloud cdb ls

Make sure to select a database in the same region as the monitoring instance.

You see output like the following:

Retrieving instances for all database types in all resource groups in all locations under IBM as …
OK
Name Location State
Databases for PostgreSQL-76 us-south inactive
testelastic eu-gb active
Databases for MySQL-9j us-south active

Now, use the name of your database to create the alert by using a command like:
* alert_type is the type of the alert (one of dos_attack_l7, load_balancing_pool_enablement_alert, clickhouse_alert_fw_anomaly, clickhouse_alert_fw_ent_anomaly, dedicated_ssl_certificate_event_type, universal_ssl_event_type, or load_balancing_health_alert).
* mechanisms are at least one of email, webhooks.
* description (optional) is the description of the alert.









 Pool toggle alert command 

curl -X POST https://api.cis.cloud.ibm.com/v1/:crn/alerting/policies -H 'content-type: application/json' -H 'x-auth-user-token: Bearer xxxxxx' -d '{"name":"Example Policy","enabled":true,"alert_type":"load_balancing_pool_enablement_alert","mechanisms":{"email":[{"id":"cistestemail@ibm.com"}],"webhooks":[]},
“filters”: {
“enabled”: [
“false”,
“true”
],
“pool_id”: [
“6e67c08e3bae7eb398101d08def8a68a”,
“df2d9d70fcb194ea60d2e58397cb35a6”
]
},
"conditions": {
}}'

Where:



* -d is the array of attributes that are required to create the alert.



* name is the name of the alert.
* enabled is the state of the alert (one of true, false).
For example, the URL field can be https://y545tf67898h.svc01.eu-gb.eventstreams.cloud.ibm.com/topics/my-sample-topic/records?apikey=27836238765487558695

When an alert is triggered, the notification is sent as a POST in JSON format to your webhook endpoint.
6. For other notification types, configure the appropriate fields for the notification.
7. Optionally, and for integrations that allow a test, enable the Test notification condition to receive a test notification. If you do not receive a test notification in 10 minutes, review your channel configuration.



6. Click Save.



Currently additionalHeaders cannot be created on the original create. However, you can patch the notification channel. For example, you can use cURL to modify a notification channel:



1. [Get the notification channel ID](https://cloud.ibm.com/docs/monitoring?topic=monitoring-notifications_apinotifications_api_get_all).
2. Run the following command:

curl -X PUT ’https://us-east.monitoring.cloud.ibm.com/api/notificationChannels/<CHANNEL_ID>' 
--header ‘Content-Type: application/json’
--header ‘Authorization: Bearer <token>’ -d @/tmp/notification.json







 Next. Setup alerts 

Next setup alerts in Monitoring by using this notification channel.
Example 

This examples displays a greeting message to the user.

{
"output": {
"generic":[
{
"response_type": "text",
"values":
{ "text": "Hello." },
{ "text": "Hi there." }
],
"selection_policy": "random"
}
]
}
}
GUID=$(ibmcloud resource service-instance <NAME> --output json | jq -r '.[].guid')
3. Run the cURL API query.

curl -X <METHOD> <MONITORING_ENDPOINT>/<API_URL> -H "Authorization: $AUTH_TOKEN" -H "IBMInstanceID: $GUID" -H "content-type: application/json"



For example, to create an alert where the alert definition and notification channel is defined through a JSON file, you can run the following command:

curl -X POST 
https://us-south.monitoring.cloud.ibm.com/api/alerts 
-H 'Authorization: Bearer eyJraW...' 
-H 'IBMInstanceID: fc8ceb8a-...' 
-H 'Content-Type: application/json' 
-d @alert.json
Sending an SMS notification 

You can send SMS alerts to notify on log data hosted in a Log Analysis instance.

To send an SMS alert, you can choose 1 of the following options:



* Integrate with the IBM Cloud Monitoring service to send SMS alerts.

Use this option when you need alerts on log data alongside system health metrics.
* Integrate with PagerDuty to send SMS alerts.

Use this option when you require call times and escalation management processes.





 Send an SMS notification by using the IBM Cloud Event Notifications service 

When the Log Analysis service sends a notification to the IBM Cloud Monitoring service, an alert is triggered in the IBM Cloud Monitoring instance. In the Events view section, you can then configure an event for that type of alert to send an SMS through the IBM Cloud Event Notifications service.

Complete the following steps to get an SMS notification when an alert in the Log Analysis service is triggered:



 Step 1. Configure an alert in IBM Log Analysis 

In the Log Analysis web UI, [configure a Monitoring alert](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-monitoring).

When an alert is triggered, a notification is sent to the Monitoring instance that you have configured. Then, you manage alerts through the Events view section in the Monitoring UI.





 Step 2. Configure the Monitoring instance 

Complete the following steps:



1. [Define an IBM Cloud Event Notifications notification channel in your Monitoring instance](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-monitoringmonitoring-config).
2. Define an alert from the event that triggers the IBM Log Analysis alert.

In the Events section of the Monitoring UI, configure an event that send an SMS notification based on the alert that youd defined previously:



1. Select an alert. Then select Create Alert from Event.
2. Enter a name, a description, a group, and the severity level. The group default is set unless you specify a custom one.
3. Define the condition that triggers the SMS notification.
Creating alerts through the UI 

You can create alerts graphically through the UI.

Complete the following steps to create an alert:



 Prereqs 

You must grant permissions to users in your account to be able to launch the web UI and create alerts.

You must be an administrator of the IBM Log Analysis service, an administrator of the IBM Log Analysis instance, or have account IAM permissions to grant other users policies.

The following table lists the minimum policies that a user must have to be able to launch the IBM Log Analysis Web UI, and view logs:



Table 1. IAM policies

 Service Role Permission granted 

 IBM Log Analysis Platform role: Viewer Allows the user to view the list of service instances in the Observability Logging dashboard. 
 IBM Log Analysis Service role: Manager or Standard-member Allows the user to launch the Web UI and create alerts. 



For more information on how to configure these policies for a user, see [Granting permissions to a user to view logs](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-work_iamuser_logdna).





 Step 1. Define the rule 

You must define the rule that specifies the scope of the data and save the rule as a view.

Complete the following steps to create a rule:



1. [Launch the IBM Log Analysis UI](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-launch).
2. Filter log data by using pre-defined filters.

You can filter logs by log source, application, and log level.



* A source can be a host, a computer, a virtual machine, or a Heroku app.
* An application represents a log file, a program, or a container.
* Examples of log levels are: INFO, DEBUG, ERROR



Complete the following steps to filter logs:



1. Click the Views icon !
ibmcloud cis alert-policy firewall-events-alert-update POLICY_ID --name NAME] --emails EMAILS] --webhooks WEBHOOKS] --enabled (true | false)] --domains DOMAINS] --services SERVICES] --description DESCRIPTION] -i, --instance INSTANCE] --output FORMAT] ! ! ! ! ! !
<-- <section "id="section-opt-update-firewall-events-alert" "> --> Command options POLICY_ID: The ID of alert policy. Required.--name: The name of the alert policy.--description: The description for the alert policy.--emails: The email addresses for dispatching an alert notification. For example: --emails test1@cn.ibm.com,test2@cn.ibm.com--webhooks: The webhook ID that for dispatching an alert notification. For example: --webhook webhookID1,webhookID2--enabled: Whether or not the alert policy is enabled.--domains: The domain IDs that for the alert policy. For example: --domains domainID1,domainID2--services: Specify which services the alert should monitor. Valid values: "country-access-rules", "waf", "firewall-rules", "ratelimit", "securitylevel", "ip-access-rules", "browser-integrity-check", "ua-rules", "lockdowns", "iprange-access-rules", "asn-access-rules", "Managed-firewall" (Enterprise plan only). Note: The 'SERVICES' is only used for advanced waf alert. If the alert policy you wanted to update is created without services specified, create a new one with sevices specified instead of updating.-i, --instance: Instance name or ID.
ibmcloud cis alert-policy firewall-events-alert-create --name NAME (--emails EMAILS | --webhooks WEBHOOKS) --enabled (true | false) --domains DOMAINS --services SERVICES] --description DESCRIPTION] -i, --instance INSTANCE] --output FORMAT] ! ! ! ! ! !
<-- <section "id="section-opt-firewall-events-alert" "> --> Command options --name: The name of the alert policy.--description: The description for the alert policy.--emails: The email addresses for dispatching an alert notification. For example: --emails test1@cn.ibm.com,test2@cn.ibm.com--webhooks: The webhook ID that for dispatching an alert notification. For example: --webhook webhookID1,webhookID2--enabled: Whether or not the alert policy is enabled.--domains: The domain IDs that for the alert policy. For example: --domains domainID1,domainID2--services: Specify which services the alert should monitor. Valid values: "country-access-rules", "waf", "firewall-rules", "ratelimit", "securitylevel", "ip-access-rules", "browser-integrity-check", "ua-rules", "lockdowns", "iprange-access-rules", "asn-access-rules", "Managed-firewall" (Enterprise plan only).-i, --instance: Instance name or ID. If not set, the context instance specified by ibmcloud cis instance-set INSTANCE is used.--output: Specify output format, only JSON is supported.<-- </section "id="section-opt-firewall-events-alert" "> --><-- <section "id="section-firewall-events-alert-examples" "> --> Examples Create a firewall-events alert for instance cis-demo.
You can use this number as a threshold for sending alerts when the number of available pods is fewer than your requested number of replicas for your app. If you requested 3 replicas of your app, you can set an alert for when the number of available pods stays fewer than the requested replica count for a certain time duration, for example, a 30 minute duration. In this example, when the pod availability remains fewer than the 3 requested replicas, the alert is triggered.
6. In the Pod Availability tile, create an alert by clicking the Options menu and then Create alert. Set the alert parameters based on the threshold that you observed and your requested app replicas.
7. Open the Notification section of the alert menu and create or select an alert notification channel.
8. Save your alert.
9. Repeat these steps for all the apps that are deployed in your cluster.
10. Test the alert that you created by editing the threshold that you configured to manually trigger the alert. For example, if you set a pod availability alert for when the number of available pods is less than 3 for 5 minutes, edit your alert threshold to be less than 4 pods for 5 minutes and select the at least once option.
11. Verify that the alert is triggered after 5 minutes. Once you have verified the alert, reset the values to the values you configured earlier.







 Troubleshooting persistent storage when an IBM Cloud Monitoring alert is triggered 

When an alert is triggered, review the alert details in IBM Cloud Monitoring and review troubleshooting guides for persistent storage, apps, worker nodes, and clusters to find the root cause of the alert. The alerts that you set up might not be related to a storage volume issue, but to issues that occurred within your app, on the worker node, or in the cluster.



* [Troubleshooting persistent storage volumes](https://cloud.ibm.com/docs/containers?topic=containers-debug_storage_file).
* [Troubleshooting worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reload).
Valid values are: slack, email, webhook, pagerduty, pagerduty-auto-resolve, and sysdig. 
 immediate Specified if the alert is sent immediately after the line criteria is met ("immediate": true) or after the line and time criteria is met ("immediate": false). 
 triggerlimit The number of matching log lines to trigger the alert. 
 triggerinterval The amount of time to look for the number of matching log lines. <br>Only following triggerinterval values are valid:<br><br><br><br> * 30 - 30 seconds<br> * 1m - 1 minute<br> * 5m - 5 minutes<br> * 30m - 30 minutes<br> * 1h - 1 hour<br> * 6h - 6 hours<br> * 12h - 12 hours<br> * 24h - 24 hours<br><br><br> 
 operator Specifies whether the alert is triggered when the condition exists (presence) or when it does not exist (absence). 
 blackout (Optional) Specifies the time period when the alert would be active. 



The following is an example blackout definition specifying that the alert is active Monday through Friday from 8:00 AM to 5:00 PM in (GMT-05:00) America/New York.

blackout": {
"enabled": true, A
"times": {
"enabled": true,
"periods": [

"17:00",
"08:00"
]
]
},
"days": [
"0",
"6"
],
"timezone": "America/New_York"
}
Show more

The following values are only valid for the specified preset type:

Email

PagerDuty

Slack



Channel configuration values for email presets

 Value Description 

 emails The email addresses where the alert will be sent. 
 timezone (Optional) The preferred timezone. 



Sysdig (IBM Cloud Monitoring)

Webhook



Channel configuration values for Sysdig (IBM Cloud Monitoring) presets

 Value Description
[Create an Event Notifications subscription](https://cloud.ibm.com/docs/event-notifications?topic=event-notifications-en-create-en-subscription) between an existing topic and the IBM Cloud email service to forward your alerts to various recipients by email.

An email from IBM Cloud that contains information about a Security and Compliance Center event resembles the following example:

Subject: Validation completed: my-test-validation-scan
Body: The following validation scan was successfully completed in the Security and Compliance Center: my-test-validation-scan. You can view the scan results from the Security and Compliance Center in the console. For more information, check out the docs.

To receive detailed information about an event notification in your email, select the Add notification payload option when you create an Event Notifications subscription. Your email displays the [notification payload details](https://cloud.ibm.com/docs/security-compliance?topic=security-compliance-event-notifications&interface=uievent-notifications-payload) that are associated with the event.





 Webhooks 

You can configure a webhook destination so that the properties of an incoming notification can be used programmatically by an app or service. For more information, check out the [Event Notifications documentation](https://cloud.ibm.com/docs/event-notifications?topic=event-notifications-en-destinations-webhook).







 Notification payload details 

Successful events that are generated by a Security and Compliance Center component contain various fields that help you to identify the source and details of an event.

Event notifications from Security and Compliance Center contain only metadata properties, such as names or identifiers of resources. Sensitive data, for example API keys or passwords, are not included in generated events.

The metadata properties that are sent to Event Notifications vary depending on the event type and subtype. For example, if an com.ibm.cloud.compliance.posture:posture-scan-completed event takes place, Security and Compliance Center sends a notification payload to Event Notifications that is similar to the following example.

{
"action": "Validation completed",
"summary": "my-test-scan",
An alert is created whenever you reach 90% of disk size. You receive the alert to the same email that you created in the notification channel. You can also use the following command to check current active alerts:

ibmcloud monitoring alert list --name <monitoring instance name>

You now receive an alert whenever your Databases for Elasticsearch instance disk utilization exceeds 90%, so you can act before the disk is too full.





 Next Steps 

To modify your alert or find out more about Monitoring, see [Getting started withIBM Cloud Monitoring](https://cloud.ibm.com/docs/monitoring?topic=monitoring-getting-started).



 Scaling Resources 

If you receive an alert that your disk utilization exceeds 90%, scale your disk so that you do not exceed 50-70% usage. Manually manage your service's resources or autoscale.



Table 1. Scaling Resources

 Service Managing Resources Autoscaling 

 Databases for DataStax [Scaling Disk, RAM, and CPU](https://cloud.ibm.com/docs/databases-for-cassandra?topic=databases-for-cassandra-resources-scaling&interface=ui) [Autoscaling](https://cloud.ibm.com/docs/databases-for-cassandra?topic=databases-for-cassandra-autoscaling&interface=ui) 
 Databases for EnterpriseDB [Scaling Disk, RAM, and CPU](https://cloud.ibm.com/docs/databases-for-enterprisedb?topic=databases-for-enterprisedb-resources-scaling&interface=ui) [Autoscaling](https://cloud.ibm.com/docs/databases-for-enterprisedb?topic=databases-for-enterprisedb-autoscaling&interface=ui) 
 Databases for etcd [Scaling Disk, RAM, and CPU](https://cloud.ibm.com/docs/databases-for-etcd?topic=databases-for-etcd-resources-scaling&interface=ui) [Autoscaling](https://cloud.ibm.com/docs/databases-for-etcd?topic=databases-for-etcd-autoscaling&interface=ui)