vania-janet/multiturn-rag-retrieval-data huggingface dataset:
If you need to collect more than 3000 time series for all your Windows systems, you need more than one Linux agent.
2. Update the /opt/draios/etc/dragent.yml to [enable remote scraping](https://docs.sysdig.com/en/docs/installation/prometheus-remote-write/).

prometheus:
enabled: true
interval: 30
log_errors: true
max_metrics: 3000
max_metrics_per_process: 3000
max_tags_per_metric: 20
remote_services:
- <HOSTNAME>:
always: true
conf:
url: "http://<WINDOWS_IP>:9182/metrics"
tags:
region: us-east
instance: <HOSTNAME>
job: <JOBNAME>
Show more

Where

<WINDOWS_IP> is the IP of the Windows system

<HOSTNAME> is the name of the Windows system

<JOBNAME> is a custom attribute that you can set to identify the role of the node that you are scraping, and you can also use to scope the data

See the following sample configuration that you can set to enable scraping for a Windows system with hostname my-windows-hostname in your environment:

prometheus:
enabled: true
interval: 30
log_errors: true
max_metrics: 3000
max_metrics_per_process: 3000
max_tags_per_metric: 20
remote_services:
- my-windows-hostname:
always: true
conf:
url: "http://10.245.0.5:9182/metrics"
tags:
region: us-east
instance: my-windows-hostname
job: my-job-name
3. Configure the monitoring agent to reduce the number of metrics that are collected by the Windows windows_exporter.

You can configure the metrics_filter section to remove metrics. For example, you can remove collector metrics.
It automatically collects metrics. You view metrics via the web-based user interface. You can monitor Bare metals in IBM Cloud, on-prem, and in other clouds.

Monitoring



* 1 hour
* 2023-07-05



[Monitoring a Kubernetes cluster](https://cloud.ibm.com/docs/monitoring?topic=monitoring-kubernetes_cluster)Monitoring a Kubernetes cluster

Use this tutorial to learn how to configure an IBM Cloud® Kubernetes Service cluster to forward metrics to the IBM Cloud® Monitoring service. You can monitor clusters in IBM Cloud, on-prem, and in other clouds.

Monitoring



* 1 hour
* 2023-07-05



[Monitoring an Ubuntu Linux VPC server instance](https://cloud.ibm.com/docs/monitoring?topic=monitoring-ubuntu)Monitoring an Ubuntu Linux VPC server instance

Use this tutorial to learn how to configure an Ubuntu host to forward metrics to the IBM Cloud Monitoring service in the IBM Cloud.

Monitoring



* 1 hour
* 2023-07-05



[Monitoring a Windows environment](https://cloud.ibm.com/docs/monitoring?topic=monitoring-windows)Monitoring a Windows environment

The standard monitoring agent cannot be installed on a Windows platform. In order to monitor a Windows system with IBM Cloud Monitoring, you can leverage the Prometheus Windows Exporter to perform the collection of the metrics on the system.

Monitoring



* 1 hour
* 2023-04-28



[Collecting availability metrics by using the Prometheus Blackbox exporter](https://cloud.ibm.com/docs/monitoring?topic=monitoring-blackbox)Collecting availability metrics by using the Prometheus Blackbox exporter

The Prometheus Blackbox exporter allows blackbox probing of endpoints over HTTP, HTTPS, DNS, TCP and ICMP. The monitoring agent can be used in conjunction with the Blackbox exporter to collect availability metrics.
* Alarms - VMware vSphere® includes an events and alarms subsystem, which tracks events that occur in the vSphere environment and makes this information available in vCenter. This section describes this subsystem and how to enable and use the alarms in your enterprise.
* Proactive daily checks - These checks enable system administrators to keep the environment healthy. When carried out daily, it prevents many common issues that are related to capacity and performance from impacting your workloads.
* Troubleshooting - Even when you carry out proactive daily checks, issues occur that impact your workloads. Therefore, you need to fix the underlying issue as quickly as possible. These troubleshooting guides and some common troubleshooting scenarios assist system administrators with identifying and fixing these issues quickly.
* Compliance - The compliance guide provides some insights on maintaining compliance of the environment against a regulatory compliance regime or industry best practice. The focus of this guide is on the VMware hardening guide, which is a number of documented lists of best practice for a VMware environment.



Many of the previous tasks are automated in Operations Management on IBM Cloud, and for those tasks that are not, these tools make the manual processes easier for your systems administrators. It is imperative that you have the core components of the VMware environment monitored, in Operations Management on IBM Cloud, this is achieved as follows:





 Operations management on IBM Cloud 

You can have enterprise tools in place that you can use to monitor and manage your vCenter Server instance. Table 1 describes the core components of the vCenter Server environment, why they need to be monitored and how they are monitored by using Operations Management on IBM Cloud. For more information, see the reference architecture documentation.



Table 1. vCenter Server environment core components

 Component Why Monitored by 

 vCenter vCenter is the infrastructure management component that manages the vSphere hosts and manages virtualized constructs such as clusters. vSAN™ is monitored through vCenter. vSphere networking such as distributed switches and port groups are monitored through vCenter. VMware Aria® Operations™ Manager and the VMware SDDC Health Management Pack.
Metric Name ibm_is_resource_quota_limit 
 Metric Type gauge 
 Value Type none 
 Segment By IBM IS Generation, Name of resource type for quota, Associated Resource 









 Attributes for segmentation 



 Global attributes 

The following attributes are available for segmenting all previously mentioned metrics.



Table 4: Global attributes

 Attribute name Attribute Attribute description 

 Cloud Type ibm_ctype The cloud type is a value of public, dedicated or local 
 Location ibm_location The location of the monitored resource - this may be a region, data center or global 
 Resource ibm_resource The resource being measured by the service - typically a indentifying name or GUID 
 Resource Type ibm_resource_type The type of the resource being measured by the service 
 Resource group ibm_resource_group_name The resource group where the service instance was created 
 Scope ibm_scope The scope is the account, organization or space GUID associated with this metric 
 Service name ibm_service_name Name of the service generating this metric 







 Additional attributes 

The following attributes are available for segmenting one or more attributes as described in the Global attributes reference. See the individual metrics for segmentation options.



Table 5: Additional attributes

 Attribute name Attribute Attribute description 

 Associated Resource ibm_secondary_resource_id The ID of a linked or associated resource 
 IBM IS Generation ibm_is_generation IBM IS Generation 
 Name of resource type for quota ibm_resource_quota_name The name of the resource type being monitored for quota usage/limit 









 Next steps 



* [Quotas and service limits](https://cloud.ibm.com/docs/vpc?topic=vpc-quotas)
LOGDNA_INCLUSION_RULES Custom rules that you can define to configure what log files to monitor. <br>These files can be located in any of the paths that are defined through the logdir parameter. <br>You can use glob patterns. For more information, see [Glober rules](https://github.com/CJP10/globber) .json,.test 
 LOGDNA_LINE_EXCLUSION_REGEX Regex custom rules that you can define to configure what log files to exclude from monitoring. For more information, see [regex syntax](https://docs.rs/regex/1.2.1/regex/syntax) <br>These files can be located in any of the paths that are defined through the logdir parameter. 
 LOGDNA_EXCLUSION_RULES Custom rules that you can define to configure what log files to exclude from being monitored. <br>You can use glob patterns. For more information, see [Glober rules](https://github.com/CJP10/globber) 
 LOGDNA_EXCLUSION_REGEX_RULES Regex custom rules that you can define to configure what log files to exclude from being monitored. /var/log/containers/,/var/log/pods/ 
 LOGDNA_USE_SSL Boolean that defines whether TLS 1.2 should be used when the agent sends logs to the logging instance. <br>The default value is set to true. true Valid values are true and false. 
 LOGDNA_USE_COMPRESSION Boolean that defines whether compression is enabled when the agent sends logs to the logging instance. <br>The default value is set to true. true true 
 LOGDNA_GZIP_LEVEL Compression level for gzip. <br>Valid values are: 1, 2, 3, 4, 5, 6, 7, 8, 9 <br>When you set this variable to 1, you are configuring the agent to use the fastest compression speed but at a lower ratio.
LOGDNA_EXCLUSION_RULES Custom rules that you can define to configure what log files to exclude from being monitored. <br>You can use glob patterns. For more information, see [Glober rules](https://github.com/CJP10/globber) 
 LOGDNA_EXCLUSION_REGEX_RULES Regex custom rules that you can define to configure what log files to exclude from being monitored. /var/log/containers/,/var/log/pods/ 
 LOGDNA_USE_SSL Boolean that defines whether TLS 1.2 should be used when the agent sends logs to the logging instance. <br>The default value is set to true. true true 
 LOGDNA_USE_COMPRESSION Boolean that defines whether compression is enabled when the agent sends logs to the logging instance. <br>The default value is set to true. true true 
 LOGDNA_GZIP_LEVEL Compression level for gzip. <br>Valid values are: 1, 2, 3, 4, 5, 6, 7, 8, 9 <br>When you set this variable to 1, you are configuring the agent to use the fastest compression speed but at a lower ratio. When you set this variable to 9, you are configuring the agent to use the highest compression ratio but at a lower speed. 2 6 
 LOGDNA_TAGS Define tags to group hosts automatically into dynamic groups. production,serviceA 







 Standard Kubernetes clusters: environment variables for logging agent V1 



Table 6. Tags that are available for the logging agent V2

 Environment variable Description Default value Sample value 

 DEFAULT_CONF_FILE Default configuration file. /etc/logdna/config.yaml 
 LOGDNA_PLATFORM Log source type. k8s 
 LOGDNA_INGESTION_KEY Reference to the logging ingestion key. secretKeyRef 
 LDLOGHOST IBM Log Analysis ingestion endpoint. logs.us-south.logging.cloud.ibm.com 
 LDAPIHOST IBM Log Analysis API ingestion endpoint.
LOGDNA_INGESTION_KEY Reference to the logging ingestion key. secretKeyRef 
 LOGDNA_HOST IBM Log Analysis ingestion endpoint. logs.us-south.logging.cloud.ibm.com 
 LOGDNA_API_HOST IBM Log Analysis API ingestion endpoint. api.us-south.logging.cloud.ibm.com 
 LOGDNA_ENDPOINT Ingestion log path. /logs/agent/ 
 LOGDNA_HOSTNAME Log Source name. MyCluster 
 LOGDNA_LOG_DIRS Defines custom paths that you want the agent to monitor. <br>Separate multiple paths by using commas. <br>You can use glob patterns. Use double quotation marks to add a globe pattern. /var/log/ /output/,/mylogs/myapplogs/ 
 LOGDNA_INCLUSION_RULES Custom rules that you can define to configure what log files to monitor. <br>These files can be located in any of the paths that are defined through the logdir parameter. <br>You can use glob patterns. For more information, see [Glober rules](https://github.com/CJP10/globber) .json,.test 
 LOGDNA_INCLUSION_REGEX_RULES Regex custom rules that you can define to configure what log files to monitor. For more information, see [regex syntax](https://docs.rs/regex/1.2.1/regex/syntax) <br>These files can be located in any of the paths that are defined through the logdir parameter. 
 LOGDNA_EXCLUSION_RULES Custom rules that you can define to configure what log files to exclude from being monitored. <br>You can use glob patterns. For more information, see [Glober rules](https://github.com/CJP10/globber) 
 LOGDNA_EXCLUSION_REGEX_RULES Regex custom rules that you can define to configure what log files to exclude from being monitored.
For more information, see [regex syntax](https://docs.rs/regex/1.2.1/regex/syntax) <br>These files can be located in any of the paths that are defined through the logdir parameter. 
 LOGDNA_EXCLUDE Custom rules that you can define to configure what log files to exclude from being monitored. <br>These files can be located in any of the paths that are defined through the logdir parameter. <br>Separate multiple files by using commas. You can use glob patterns. You can configure specific files. 
 LOGDNA_EXCLUDE_REGEX Regex custom rules that you can define to configure what log files to exclude from being monitored. /var/log/containers/,/var/log/pods/ 
 LDLOGSSL Boolean that defines whether TLS 1.2 should be used when the agent sends logs to the logging instance. <br>The default value is set to true. true true 
 COMPRESS Boolean that defines whether compression is enabled when the agent sends logs to the logging instance. <br>The default value is set to true. true true 
 GZIP_COMPRESS_LEVEL Compression level for gzip. <br>Valid values are: 1, 2, 3, 4, 5, 6, 7, 8, 9 <br>When you set this variable to 1, you are configuring the agent to use the fastest compression speed but at a lower ratio. When you set this variable to 9, you are configuring the agent to use the highest compression ratio but at a lower speed. 2 6 
 LOGDNA_TAGS Define tags to group hosts automatically into dynamic groups. production,serviceA 







 Linux: configuration parameters for logging agent V1 



Table 7. Configuration options for the logging agent V1

 Parameter Description 

 tags Define tags to group hosts automatically into dynamic groups. 
 logdir Define custom paths that you want the agent to monitor. <br>Separate multiple paths by using commas. You can use glob patterns. You can configure specific files.
Changing Service Endpoints through the UI 

In the Overview tab of your deployment's dashboard, go to the Endpoints section. Toggle which types of connections are available to your deployment.

Changing the type of endpoints available on your deployment does not cause any downtime from a database perspective. However, if you disable an endpoint that is being used by you or your applications, those connections are dropped.





 Changing Service Endpoints through the CLI 

Use the [ibmcloud resource service-instance-update](https://cloud.ibm.com/docs/cli?topic=cli-ibmcloud_commands_resourceibmcloud_resource_service_instance_update) command in the CLI, specifying the endpoint with the --service-endpoints flag.

ibmcloud resource service-instance-update <service-name> --service-endpoints <endpoint-type>

Changing the type of endpoints available on your deployment does not cause any downtime from a database perspective. However, if you disable an endpoint that is being used by you or your applications, those connections are dropped.





 Changing Service Endpoints through the API 

Use the [Resource Controller API](https://cloud.ibm.com/apidocs/resource-controller), with a PATCH request to the [/resource_instances/{id}](https://cloud.ibm.com/apidocs/resource-controllerupdate-a-resource-instance) endpoint.

Changing the type of endpoints available on your deployment does not cause any downtime from a database perspective. However, if you disable an endpoint that is being used by you or your applications, those connections are dropped.





 Credentials for Private Endpoints 

Use either public or private connection strings with any set of credentials that you make on your deployment. By default, the connection strings for a set of credentials are filled with strings for connecting over a public endpoint. If you are using private endpoints, specify connection strings that contain the private endpoint be generated instead.
ESA-5 - Subcontractor Risk Management 



  Control requirements 

ESA-5 - 0
:   The organization must have oversight and controls on activities they outsource to subcontractors that contribute to the delivery of products or services contracted with the customer. The oversight and controls of these subcontractors must be commensurate with the level of risk associated with the products and services that the organization provides to the customer. If the organization is dependent on a subcontractor to meet Recovery Time Objectives (RTO) and Recovery Point Objectives (RPO) contracted with the customer, the organization must be able to demonstrate that it monitors and oversees subcontractor recovery capabilities to ensure they can also recover within those RTO/RPO timeframes. Organization monitoring and oversight of subcontractors must include annual: - Review of subcontractor business continuity plans to ensure recovery strategies of product and services support the contracted RTO/RPO timeframes with the customer - Testing of subcontractor business continuity plans to ensure recovery occurs within the contracted RTO/RPO timeframes with the customer - Review of subcontractor testing results and validation that any findings generated are remediated and monitored to closure, with critical findings being addressed within 45 days of issue identification - Review of subcontractor response framework to ensure adequate capabilities are in place for rapid assembly and response in the event of a disruption





  Implementation guidance 

See the resources that follow to learn more about how to implement this control.



*  [IBM Cloud account setup](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-account-setup)







  IBM Cloud for Financial Services profile 

The rules related to this control that follow are part of the IBM Cloud for Financial Services v1.2.0 profile in [IBM Cloud® Security and Compliance Center](https://cloud.ibm.com/docs/security-compliance?topic=security-compliance-getting-started).



*  Check whether provisioned services are IAM enabled
*  Check whether the Financial Services Validated setting is enabled in account settings
Cloud Type ibm_ctype The cloud type is a value of public, dedicated, or local. 
 Location ibm_location The location of the monitored resource - this might be a region, data center, or global. 
 Resource ibm_resource The resource being measured by the service - typically an identifying name or GUID. 
 Resource Type ibm_resource_type The type of the resource being measured by the service. 
 Scope ibm_scope The scope is the account, organization, or space GUID associated with this metric. 







 More Attributes 

The following attributes are available for segmenting one or more attributes as described in the reference. See the individual metrics for segmentation options.



Table 15. Additional Attributes metadata

 Attribute Attribute Name Attribute Description 

 Service instance ibm_service_instance The service instance segment identifies the instance that the metric is associated with. 
 Service instance name ibm_service_instance_name The service instance name provides the user-provided name of the service instance, which isn't necessarily a unique value depending on the name that is provided by the user. 
 Resource group ibm_resource_group_name The resource group where the service instance was created.
Unique 2048-bit ssh keys are issued to each IBM user in order to connect to the IBM Cloud VPN. Phone based authentication via PIN is also required each time an IBMer connects. These factoring mechanisms are maintained, managed and issued by IBM Cloud Security and the IBM TRIRIGA Application Suite Managed Service Environment Operations Manager.
* IBM's internal network prevents employees from accessing malicious websites using Symantec Bluecoat and ProofPoint Targeted Attack Protection (TAP)
* IBM TRIRIGA Application Suite Managed Service personal access credentials are role based and managed using an IBM internal access management system.
* Access is based by job duties (least privilege principal) in accordance with IBM IT Security Policy. The IBM Cloud Delivery Services security team performs the following processes to ensure only those individuals who require access to systems have it, and to ensure the right privileges are in place:



* Every quarter, a separation of duties review is performed by the IBM TRIRIGA Application Suite Managed Service management team to ensure no one individual has a conflict of roles without adequate safeguards beings in place
* Every quarter, a review of user access is performed to ensure existing users and privileges are still required
* A defined process is in place to ensure individuals who leave the IBM Cloud Delivery Services team, even if to other areas within IBM, have their UserID and privileges revoked



* The IBM TRIRIGA Application Suite Managed Service team performs proactive management and deployment of patches, updates and fixes to the Application, Middleware, Database and O/S layers via a planned Maintenance & Outage Calendar
* IBM Cloud Delivery Services employs a defense in depth strategy (DiD) for boundary protection that includes firewalls and encrypted communications for remote connectivity to access the environment. All communications that cross this boundary are controlled and monitored.
* The IBM Cloud network ports are protected with firewalls, which serve as intrusion detection (IDS) and intrusion prevention (IPS) agents. Details regarding on IBM Cloud intrusion detection management is IBM Confidential information and covered under IBM Cloud SOC2 compliance (see compliance section below).
* All IBM TRIRIGA SaaS and TRIRIGA Application Suite Managed Service environments are configured for Anti-Malware (Anti-Virus) protection and Endpoint Detection and Response (EDR) technology with associated telemetry.



Status and alerts are monitored continuously.
Global Attributes

 Attribute Attribute Name Attribute Description 

 Cloud Type ibm_ctype The cloud type is a value of public, dedicated, or local. 
 Location ibm_location The location of the monitored resource - this can be a region, data center, or global. 
 Resource ibm_resource The resource being measured by the service - typically an identifying name or GUID. 
 Resource Type ibm_resource_type The type of the resource being measured by the service. 
 Scope ibm_scope The scope is the account, organization, or space GUID associated with this metric. 







 More Attributes 

The following attributes are available for segmenting one or more attributes as described in the reference. See the individual metrics for segmentation options.



Table 18. Additional Attributes

 Attribute Attribute Name Attribute Description 

 Service instance ibm_service_instance The service instance segment identifies the instance that the metric is associated with. 
 Service instance name ibm_service_instance_name The service instance name provides the user-provided name of the service instance, which isn't necessarily a unique value depending on the name that is provided by the user. 
 Resource group ibm_resource_group_name The resource group where the service instance was created.
No operational tools are deployed to the gateway cluster. The gateway cluster is monitored and managed by using the tools that are deployed on the management cluster. All components of the gateway cluster are configured to deliver their log files to VMware Aria Operations for Logs, inclusive of the virtual gateway syslog facilities. If the physical FortiGate is deployed instead of the gateway cluster, its syslog facility must send logs to VMware Aria Operations for Logs.





 Workload cluster 

No operational tools are deployed to the workload cluster. The workload cluster is monitored and managed by using the tools that are deployed on the management cluster. All platform components that are deployed in the workload cluster are configured to deliver their log files to VMware Aria Operations for Logs.





 Related links 



* [IBM Cloud compliance programs](https://www.ibm.com/cloud/compliance)
* [VMware Aria Operations and VMware Aria Operations for Logs](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-vrops_overview)
* [Caveonix RiskForesight](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-caveonix_considerations)