vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Audit logging of IBM Cloud events 

When you work in a cloud environment, such as the IBM Cloud, you must plan the cloud strategy for auditing and monitoring workloads and data in accordance with your internal policies and with industry and country-based compliance requirements. It is critical to capture audit events generated by the components of your service. With proper audit logs you can investigate abnormal activity and to comply with regulatory audit requirements. By actively monitoring these events, you can check into potential security problems and take corrective action shortly after they've occurred.

[Activity Tracker Event Routing](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-getting-started-routing&interface=cli) is used to collect [auditable events from IBM Cloud services](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-cloud_services) that are generated for variety of [event types](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-event_types). These events allow you to monitor the activity of your IBM Cloud account so that you can investigate abnormal activity and critical actions.

Activity Tracker Event Routing provides for either event routing or hosted event search. However, only the event routing features of Activity Tracker Event Routing have been Financial Services Validated. In regions where it's available, you must configure Activity Tracker Event Routing event routing to send events to Object Storage. All Object Storage buckets must be encrypted with KYOK using keys managed by Hyper Protect Crypto Services. See [Encryption at rest](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-encryption-at-rest) for more information.

Activity Tracker Event Routing event routing is only available in some regions (see [Locations for Activity Tracker Event Routing event routing](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-regionsregions-atracker) for more details). For regions where it's not available, you must use Activity Tracker Event Routing hosted event search until Activity Tracker Event Routing event routing is available.
For a user to monitor data within the context of a team, you must grant the user a policy for the IBM Cloud Monitoring service. The policy specifies the team and the service permissions for the user so the user can work with the data in scope for that team.

You can grant any of the following IAM service roles:



* Writer: A writer role allows a user to monitor data through dashboards, alerts, and notifications, and to manage resources such as dashboards, alerts, and notifications that are in scope for the team.
* Reader: A reader role allows a user to monitor data through dashboards, alerts, and notifications that are in scope for the team.
* Manager: The manager role is an instance level role that grants administrative permissions. If you grant this role in a policy for a team, you are granting admin permissions over the instance to the users that belong to that team.



The following table shows the user roles that you can grant a user to work with the Monitoring service:



Table 1. List of user roles

 User role IAM service role 

 ROLE_USER reader 
 ROLE_ADVANCED_USER writer 
 ROLE_ADMIN manager 



The following table shows the team roles that you can grant users to work within the context of a team in a Monitoring instance:



Table 2. List of team roles

 Team role IAM service role Team 

 ROLE_TEAM_READ reader Custom team 
 ROLE_TEAM_EDIT writer Custom team 
 ROLE_TEAM_ADMIN manager Custom team 
 ROLE_TEAM_MONITOR_MANAGER manager Monitor operations 
 ROLE_TEAM_SECURE_MANAGER manager Secure operations 



You can define in the Monitoring UI more teams to define different levels of access to data per team and set of users.

To grant a user access to 1 or more teams, an administrator must grant the user a policy for each team that the user needs access to. By using individual policies for each team, administrators can define different service access and permissions levels to work with data in the monitoring instance.

For example, a user that needs to work in a team requires the following policies:



* A policy with a platform role viewer to allow the user to see monitoring instances in the IBM Cloud.
* A policy to grant the user access to 1 team.
* When testing, you might notice that the order of the options in the disambiguation list changes from one test run to the next. Don't worry; this new behavior is intended. As part of work being done to help the assistant learn automatically from user choices, the order of the options in the disambiguation list is being randomized on purpose. Changing the order helps to avoid bias that can be introduced by a percentage of people who always pick the first option without first reviewing their choices.









 12 November 2019 

Slot prompt JSON editor
: You can now use the context or JSON editors for the slot response field where you define the question that your assistant asks to get information it needs from the customer. For more information about slots, see [Gathering information with slots](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-slots).

New South Korea location
: You can now create Watson Assistant instances in the Seoul location. As with other locations, the IBM Cloud Seoul location uses token-based Identity and Access Management (IAM) authentication.

Technology preview
: A technology preview experience was released. A select set of new users are being presented with a new user interface that takes a different approach to building an assistant.





 7 November 2019 

Irrelevance detection has been added
: When enabled, a supplemental model is used to help identify utterances that are irrelevant and should not be answered by the dialog skill. This new model is especially beneficial for skills that have not been trained on what subjects to ignore. This feature is available for English skills only. For more information, see [Irrelevance detection](https://cloud.ibm.com/docs/assistant?topic=assistant-irrelevance-detection).

Time zone support for now() method
: You can now specify the time zone for the date and time that is returned by the now() method. See [Now()](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-methodsdialog-methods-dates-now).





 24 October 2019 

Testing improvement
The events that are collected comply with the Cloud Auditing Data Federation (CADF) standard.

In COS, you can track management and data events.



* By default, these events are not enabled.
* Management events report actions that change the state of resource configurations, bucket properties, and object properties.
* Data events report actions on buckets, objects, and multi-part objects such as creation, deletion, and access.
* You must configure each bucket to enable management events, or management and data events. Notice that you cannot enable data events only for a bucket. For more information, see [Activity Tracker events](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-at-events).



Enable collection of COS management and data events on the bucket that you use to archive data from an auditing instance. Use these events to monitor activity in your COS bucket.

In IBM Cloud® Activity Tracker, you can define views, dashboard, and screens to monitor COS management and data events. You can also configure alerts on views to notify you when a specific condition occurs. On a view, you can configure an email alert, a Slack alert, a PagerDuty alert, or any combination of these. For more information, see [Creating custom views](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-views) and [Managing alerts](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-alerts).

For example, you can define a view that reports when an object is archived in a bucket. On that view, you can configure an email alert to notify you when an object has been archived. You might have compliance requirements that require you to control who accesses data that is archived. You can define a view that reports access to a bucket, and define an alert to notify you when that happens.

Define views, dashboard, screens, and alerts in IBM Cloud® Activity Tracker to investigate for abnormal activity and critical actions in your COS bucket, and to comply with regulatory audit requirements.





 Archived file
Service Description Integration instruction 

 [IBM Cloud Monitoring](https://cloud.ibm.com/docs/monitoring) IBM Cloud Monitoring is a cloud-native, and container-intelligence management system. You can use it to gain operational visibility into the performance and health of your Hyper Protect Crypto Services instance. [Getting started tutorial](https://cloud.ibm.com/docs/monitoring?topic=monitoring-getting-started) 
 [IBM Cloud Schematics](https://cloud.ibm.com/docs/schematics) IBM Cloud Schematics provides powerful tools to automate your cloud infrastructure provisioning and management process, the configuration and operation of your cloud resources, and the deployment of your application workloads. All data, user inputs, and the data that is generated at runtime during execution of automation code, are stored in IBM Cloud Object Storage. This data is encrypted by default by using encryption keys from Schematics. If you need to control the encryption keys, you can integrate with your Hyper Protect Crypto Services instance to use your own root keys. [Enabling customer-managed keys for Schematics](https://cloud.ibm.com/docs/schematics?topic=schematics-secure-datausing-byok) 
 [Event Streams](https://cloud.ibm.com/docs/EventStreams) The Event Streams service is a high-throughput message bus that is built with Apache Kafka. You can use it for event ingestion into IBM Cloud and event stream distribution between your services and applications. By default, message payload data in Event Streams is encrypted at rest by using a randomly generated key. If you need to control the encryption keys, you can integrate with your Hyper Protect Crypto Services instance to use your own root keys. [Enabling a customer-managed key for Event Streams](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-managing_encryptionenabling_encryption) 







 Security service integrations 

You can integrate Hyper Protect Crypto Services with the following security-related services. By default, the data that you store in these services is encrypted at rest by using an IBM-managed key.
To filter the list of services that is displayed, select the Logging and Monitoring category.
4. Click the IBM Cloud Monitoring tile.
5. Select the location.
6. Select a service plan. By default, the Lite plan is selected.

To provision an instance that only includes the Monitor component, select the plan Graduated Tier.

To provision an instance that include the Monitor and the Secure (Workload Protection) components, select the plan Graduated Tier - Sysdig Secure + Monitor.

For more information about Workload Protection, see the [IBM Cloud Security and Compliance Center Workload Protection documentation.](https://cloud.ibm.com/docs/workload-protection)

For more information about the service plans, see [Service plans](https://cloud.ibm.com/docs/monitoring?topic=monitoring-pricing_planspricing_plans).
7. Enter a service name.
8. Select a resource group. By default, the Default resource group is set.
9. Click Create to provision an instance.



The service UI opens.

To provision an instance through the CLI, see [Provisioning a Monitoring instance through the IBM Cloud CLI](https://cloud.ibm.com/docs/monitoring?topic=monitoring-provisionprovision_cli).





 Step 3. Collect metrics 

Next, configure your hosts to send metrics.

You can collect metrics from a number of platforms, orchestrators, and a wide range of applications such as Prometheus, JMX, StatsD, Kubernetes, and other application stacks, that are available in the IBM Cloud®, outside the IBM Cloud, or on-prem. You can also add more metrics by creating custom metrics and adding integrations. Learn more:



* [Collecting default metrics by using the Monitoring agent](https://cloud.ibm.com/docs/monitoring?topic=monitoring-about-collect-metricsabout-collect-metrics-2)
Auditing events for context-based restrictions 

As a security officer, auditor, or manager, you can use the IBM Cloud Activity Tracker to track how users and applications interact with the Context-based restrictions rules and network zones in IBM Cloud.

The IBM Cloud Activity Tracker service records user-initiated activities that change the state of these resources in IBM Cloud.

To get started monitoring your user's actions, see [IBM Cloud Activity Tracker](https://cloud.ibm.com/docs/services/activity-tracker?topic=activity-tracker-getting-startedgetting-started). An initiator can be a user, a service, or an application.



 Network zone events 

The following table lists the actions that generate nerwork zone events:



Table 1. Actions generating network zone events

 Action Description 

 context-based-restrictions.zone.create An event is generated when an initiator creates a CBR zone. 
 context-based-restrictions.zone.list An event is generated when an initiator lists CBR zones. 
 context-based-restrictions.zone.read An event is generated when an initiator looks at information that is related with a CBR zone. 
 context-based-restrictions.zone.update An event is generated when an initiator modifies a CBR zone. Users can identify system initiated updates (vs. user initiated updates) by the initiator name "IBM". 
 context-based-restrictions.zone.delete An event is generated when an initiator deletes a CBR zone. 







 Context-based restrictions rules events 

The following table lists the actions that generate context-based restricitons rule events:



Table 2. Actions generating context-based restrictions rule events

 Action Description 

 context-based-restrictions.policy.create An event is generated when an initiator creates a CBR rule. 
 context-based-restrictions.policy.list An event is generated when an initiator lists CBR rules. 
 context-based-restrictions.policy.read An event is generated when an initiator looks at information that is related with a CBR rule. 
 context-based-restrictions.policy.update An event is generated when an initiator modifies a CBR rule.
Your consumer has no knowledge of messages that were sent before it subscribed, therefore don't expect that all messages are consumed from a topic.
* earliest: Your consumer consumes all messages from the beginning because it is aware of all messages that were sent.



If a consumer fails after processing a message but before committing its offset, the committed offset information does not reflect the processing of the message. This means that the message is processed again by the next consumer in that group to be assigned the partition.

When committed offsets are saved in Kafka and the consumers are restarted, consumers resume from the point they last stopped at. When a committed offset exists, the auto.offset.reset property is not used.





 Committing offsets automatically 

The easiest way to commit offsets is to have the Kafka consumer do it automatically. It is simple but it does give less control than committing manually. By default, a consumer automatically commits offsets every 5 seconds. This default commit happens every 5 seconds, regardless of the progress the consumer is making toward processing the messages. In addition, when the consumer calls poll(), this also causes the latest offset returned from the previous call to poll() to be committed (because it was probably processed).

If the committed offset overtakes the processing of the messages and a consumer failure exists, it's possible that some messages are not processed. This is because processing restarts at the committed offset, which is later than the last message to be processed before the failure. For this reason, if reliability is more important than simplicity, it's usually best to commit offsets manually.





 Committing offsets manually 

If enable.auto.commit is set to false, the consumer commits its offsets manually. It can do this either synchronously or asynchronously. A common pattern is to commit the offset of the latest processed message based on a periodic timer. This pattern means that every message is processed at least once, but the committed offset never overtakes the progress of messages that are actively being processed. The frequency of the periodic timer controls the number of messages that can be reprocessed following a consumer failure. Messages are retrieved again from the last saved committed offset when the application restarts or when the group rebalances.
Are you seeing errors when there are no problems? 

Are you observing monitoring agent connection errors or receiving uptime alerts reporting an host is down when there are no problems?

IBM Cloud Monitoring has identified an issue with a subset of agent versions:



* monitoring agent 10.3.0
* monitoring agent 10.3.1
* monitoring agent 10.4.0
* monitoring agent 10.4.1



Where connectivity between your infrastructure and the IBM Cloud Monitoring hosted service may fail. For more information, see [Release Notes](https://docs.sysdig.com/en/sysdig-agent-release-notes.html).

If you are experiencing connectivity issues, complete the following steps:



1. Check the version of the monitoring agent.

You can view the dashboard template monitoring agent Health & Status that is available in Host Infrastructure to check the version of your monitoring agents.
2. [Upgrade your monitoring agent](https://cloud.ibm.com/docs/monitoring?topic=monitoring-upgrade_agent) to version 10.5 as soon as possible to ensure a reliable flow of metrics and data into our systems.







 Are you getting access denied when running API calls? 

When the authorization method that is allowed in a IBM Cloud Monitoring instance is set to IAM_ONLY, you can get the following response {"errors":[{"reason":"Not enough privileges to complete the action","message":"Access is denied"}]}.

First, [check the instance authorization methods that are allowed](https://cloud.ibm.com/docs/monitoring?topic=monitoring-iam_instance_authiam_instance_auth_step1).

Then, review [Headers for IAM Tokens](https://cloud.ibm.com/docs/monitoring?topic=monitoring-mon-curlmon-curl-headers-iam) and [Token headers](https://cloud.ibm.com/docs/monitoring?topic=monitoring-mon-curlmon-curl-headers-sysdig) before you retry the request.





 Are your dashboards, alerts, and scripts no longer returning what you intended?
When it synthesizes these interjections, it gives them the natural emphasis that a human would use in normal conversation. For some of the interjections, you can use SSML to enable or disable their emphasis. For more information, see [Emphasizing interjections](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-synthesis-expressiveemphasizing-interjections).

New word emphais with expressive neural voices
: The expressive voices use a conversational style that naturally applies the correct intonation from context. But you can indicate that one or more words are to be given more or less emphasis. The change in stress can be indicated by an increase or decrease in pitch, timing, volume, or other acoustic attributes. For more information, see [Emphasizing words](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-synthesis-expressiveemphasizing-words).





 21 September 2022 

New Activity Tracker event for GDPR deletion of user information
: The service now returns an Activity Tracker event when you use the DELETE /v1/user_data method to delete all information about a user. The event is named text-to-speech.gdpr-user-data.delete. For more information, see [Activity Tracker events](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-at-events).

Defect fix: Custom word translations now accept commas in all cases
: Defect fix: Word translations added to custom models now accept commas in all cases. Previously, a comma in a translation could occasionally cause the translation to fail to generate valid audio when used for speech syntheses. This problem was identified in US English custom models.

Defect fix: French synthesis of dates is now consistent
: Defect fix: French synthesis no longer includes the article "le" before dates of the form "the ordinal of month." Previously, the article was included only for the first day of the month for French (for example, "the first of September," "le premier septembre").
Detection is the process of monitoring the events occurring in your network and analyzing them for signs of possible threats to your security policies. Prevention refers to the process stopping the detected incidents. 
 [On-box logging and reporting CSB statistics in J-web](https://public.dhe.ibm.com/cloud/bluemix/network/vsrx/on-box-logging-reporting-11320.pdf) This guide describes how to enable on-box security event reporting using the set security log report CLI command. 
 [Juniper searchable IPS signature database](https://threatlabs.juniper.net/signatures/search//list/ips?page_number=1&page_size=20) Juniper's searchable database of IPS and application signatures to help protect your environment. These signature pages describe the vulnerabilities covered, their CVE numbers, CVSS scores, and Juniper's recommendation for deployment.
If this is nonzero and rebalancing high, it indicates a consumer group issue.





 Topic bytes in per second 

The number of bytes produced per second to a topic.



Table 25. Topic bytes in per second metric metadata

 Metadata Description 

 Metric Name ibm_eventstreams_instance_topic_bytes_in_per_second 
 Metric Type gauge 
 Value Type byte 
 Segment By Service instance, IBM Event Streams Kafka topic, Service instance name 



This is for information to help you monitor trends in your usage, particularly if any topics are producing unusual throughput, which is more or less than expected.





 Topic bytes out per second 

The number of bytes consumed per second from a topic.



Table 26. Topic bytes out per second metric metadata

 Metadata Description 

 Metric Name ibm_eventstreams_instance_topic_bytes_out_per_second 
 Metric Type gauge 
 Value Type byte 
 Segment By Service instance, IBM Event Streams Kafka topic, Service instance name 



This is for information to help you monitor trends in your usage, particularly if any topics are consuming unusually more or less throughput than expected.





 IAM ID bytes in per second 

The number of bytes in per second from an IAM ID.



Table 27. The number of bytes in per second per IAM ID

 Metadata Description 

 Metric Name ibm_eventstreams_iam_id_bytes_in_per_second 
 Metric Type gauge 
 Value Type byte 
 Segment By Service instance, Service instance name, Iam Id 



This is for information to help you monitor trends in your usage, particularly if any IAM IDs are producing unusually more throughput than expected. It allows to see any differences in the amount of data that is sent to the service from different users (IAM IDs) and if required, guides the setting of any quotas needed.





 IAM ID bytes out per second 

The number of bytes out per second from an IAM ID.



Table 28. The number of bytes out per second per IAM ID

 Metadata Description 

 Metric Name ibm_eventstreams_iam_id_bytes_out_per_second 
 Metric Type gauge 
 Value Type byte