vania-janet/multiturn-rag-retrieval-data huggingface dataset:
US-EAST https://ingest.private.us-east.monitoring.cloud.ibm.com/prometheus/remote/write 
 AU-SYD https://ingest.private.au-syd.monitoring.cloud.ibm.com/prometheus/remote/write 
 CA TOR https://ingest.private.ca-tor.monitoring.cloud.ibm.com/prometheus/remote/write 
 BR SAO https://ingest.private.br-sao.monitoring.cloud.ibm.com/prometheus/remote/write 







 Limitations 

The IBM Cloud Monitoring Prometheus Remote Write feature has the following limitations:



* Metrics that are sent to an instance can be accessed in Explore, but they are not compatible with the scope tree.
* This feature is not supported with teams.
* Only labels that are collected at source can be used to filter metrics.
* The metadata of a metric is not sent via remote write.
* Metrics that have a name that ends with _total, _sum, or _count are stored as a counters, otherwise they are managed as gauge.
* You can set units in Dashboards manually.
* You cannot mix metrics with different sampling in a dashboard, for example, 10s and 1min.

Consider configuring the scrape interval to be 10s so that you can combine Prometheus Remote Write metrics with Monitoring agent metrics.







 Collecting metrics from a Kubernetes cluster by using Prometheus remote write 

You can use the Prometheus remote write feature to send metrics directly from a Kubernetes cluster to an IBM Cloud Monitoring instance, in addition to metrics that are collected from the Monitoring Kubernetes agent. The Kubernetes cluster can run in the IBM Cloud, outside the IBM Cloud, or on-prem.



 Before you begin 

Complete the following steps:



1. Check that you have a Kubernetes cluster running and the kubectl CLI installed in your local environment.
2. Check that you have permissions to manage the cluster. You need administrator permissions to add secrets to the cluster and manage the Prometheus deployment.
3.
Error when configuring a target or route 

You are trying to configure an IBM Cloud® Metrics Routing target or route and receive an error.

  What’s happening 

When using the CLI or API you receive a message that the metadata_primary_region in settings is empty. API calls return a 409 error code.

  Why it’s happening 

You must have a primary metadata region set before you can configure IBM Cloud Metrics Routing targets or routes.

  How to fix it 

Configure your primary metadata region by using the [ibmcloud metrics-router setting update command](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-metrics-router-clisettings-update-cli) or the [modify settings API method.](https://cloud.ibm.com/apidocs/metrics-router/metrics-router-v3update-settings)
Deploying the agent on a Linux host with no public access 

After you provision an instance of the IBM Cloud Monitoring service in the IBM Cloud, you can deploy the Monitoring agent on your Linux hosts to collect data and metrics automatically. You can configure which metrics to monitor in each environment.

You can associate one or more tags to each monitoring agent. Tags are comma-separated values that are formatted as TAG_NAME:TAG_VALUE. When you monitor your environment, you can use these tags to identify metrics that are available from an agent. For example, you can include information about the service name and location with all of the metrics that are collected by this agent.



 Prereqs 

Check the topic [Tuning Sysdig Agent](https://docs.sysdig.com/en/docs/installation/sysdig-agent/troubleshooting-agent-installation/tuning-sysdig-agent/)





 Deploying the agent 

Follow these steps if your bare metal or VM is running on the IBM Cloud private network and does not have access to the public sites.

Complete the following steps to configure a monitoring agent on Linux to collect and forward metrics to an instance of the IBM Cloud Monitoring service:



1. [Obtain the access key](https://cloud.ibm.com/docs/monitoring?topic=monitoring-access_keyaccess_key_ibm_cloud_ui).
2. Obtain the private ingestion URL. For more information, see [collector endpoints](https://cloud.ibm.com/docs/monitoring?topic=monitoring-endpointsendpoints_ingestion).
3. Check that you can reach the repo http://mirrors.service.networklayer.com/sysdig/.

Whether you have a Bare metal or a Classic VSI, by default you get access to the repo.
* For more information, see [Advisor](https://docs.sysdig.com/en/docs/sysdig-monitor/advisor/).



In the Advisor section, choose to monitor by host. Check out the predefined dashboards that you can use to monitor the health of your resources.

When you choose to monitor by host, you can choose any of the following dashboards:



* Host Resource Usage
* File System Usage & Performance
* Memory Usage
* Network
* Sysdig Agent Health & Status







 Step 4: [Optional] Configure the Prometheus IPMI Exporter to monitor sensor metrics 

In addition to the set of metrics that are automatically collected by the monitoring agent, you might want to collect other metrics such as sensor metrics. You can use the Prometheus IPMI Exporter to perform the collection of Intelligent Platform Management Interface (IPMI) device sensor metrics from the bare metal server.



* The Prometheus IPMI Exporter exporter supports local IPMI devices and remote devices that can be accessed by using Remote Management Control Protocol (RMCP).
* When you use RMCP to access remote devices, you can use an IPMI exporter to monitor multiple IPMI devices. You identify each device by passing the target hostname as a parameter.
* The IPMI exporter relies on tools from the FreeIPMI suite.



You can collect the following metrics when you configure the IPMI exporter in a bare metal server:



* IPMI admin metrics

The metric ipmi_up {collector="<NAME>"} reports 1 when data from a remote host is collected successfully. It reports 0 for collection of data in a local host.

The metric ipmi_scrape_duration_seconds reports the amount of time that it takes the collector to retrieve the data.
* IPMI System event log (SEL) metrics

The metric ipmi_sel_entries_count reports the number of entries in the system event log.

The metric ipmi_sel_free_space_bytes reports the number of free bytes for new ystem event log entries.
* IPMI sensor data
You can then create a model that recognizes the rule patterns, and deploy it for use in other services.

See [Creating a rule-based model](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-rule-annotator) for more details.







 Analyze text with advanced rules 

The advanced rules feature is Beta. The feature is in a trial stage of development and is not intended for use in production environments.

The visual advanced rules editor allows you to create text extractors with deeper customization potential than what is available in the entities and relations rules editor. A number of sample extractors, such as Finance Actions extractors and Parts of Speech, are provided. You can edit and combine them to create your own advanced rules model. You can analyze documents directly in the editor, or you can export your model to use with other services such as [Natural Language Understanding](https://cloud.ibm.com/docs/natural-language-understanding).

To get started, see [Creating an advanced rules model](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-create-advanced-rules-model).





 Identify custom categories 

The custom categories feature is experimental. The feature is subject to change or be discontinued with short notice. It is not intended to use custom categories in production environments.

With a categories workspace, you can define a hierarchy of custom content categories and provide relevant key phrases that the service uses to categorize text content. You can deploy a custom categories model to use instead of the [standard categories](https://cloud.ibm.com/docs/natural-language-understanding?topic=natural-language-understanding-categories-hierarchy) offered by Natural Language Understanding and Discovery.





 Watson services integration 

Share domain artifacts and models between IBM Watson® Knowledge Studio and other Watson services.

Use Knowledge Studio to perform the following tasks:



* Bootstrap annotation by using the Natural Language Understanding service to automatically find and annotate entities in your documents.
Customizing Linux monitoring agents 

By default, the monitoring agent collects data from a range of platforms and applications. You can edit the agent config file to change its default behavior, and include or exclude data. You can customize the monitoring agent configuration file to include custom metrics such as JMX, StatsD, and Prometheus. You also can filter out metrics and containers.

The monitoring agent uses two configuration files that specify what data to collect automatically:



Table 1. monitoring agent configuration files

 File Description Location 

 dragent.default.yaml Main configuration file <br>IMPORTANT: Do not edit this file. /opt/draios/etc/dragent.default.yaml 
 dragent.yaml Configuration file that you edit to customize the monitoring agent. /opt/draios/etc/dragent.yaml 



You can edit the dragent.yaml file to customize the data that is collected. Changes take effect immediately after you save the file. The agent detects the change and automatically restarts. You can find the frequency and the files that a monitoring agent checks in the dragent.default.yaml file.

For example, a dragent.default.yaml includes the following information:

check_frequency_s: 5
files:
- /opt/draios/etc/dragent.yaml
- /opt/draios/etc/kubernetes/config/dragent.yaml
- /opt/draios/etc/kubernetes/secrets/access-key



 Editing the dragent yaml file 

The yaml file is loacated in /opt/draios/etc/.

Complete the following steps to edit the file and apply the changes:



1. Access the dragent.yaml directly. The file is located in /opt/draios/etc/dragent.yaml.
2. Edit the file. Use valid YAML syntax.
3. Restart the agent. Run the following command:

service dragent restart







 Blocking ports
Understanding routing precedence 

You can configure exactly where IBM Cloud® Metrics Routing routes platform metrics in your account. Route rule inclusion filters provides elevated control over how your metrics are routed. You can also configure default targets by using IBM Cloud® Metrics Routing settings. How the configuration is processed determines the final destination where metrics are sent; each data point is processed individually.



1.  If a route rule's inclusion filters match the data point, the data point is sent to the configured targets. Route rules are processed sequentially, the first match is used. If there are multiple routes, all route rules will be processed to find a match.

If a matched route rule uses the drop action, the data point will be dropped and no destinations will receive it.
2.  If the data point does not match any route rules, the [default targets setting](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-target-default) is used to route the data point to the default targets.
3.  If the data point does not match any route rules, and no default target is defined, the metric is routed to the IBM Cloud Monitoring instance defined in the region as the receiver of platform metrics. [A IBM Cloud Monitoring instance enabled to receive platform metrics must exist in the region to receive the metrics.](https://cloud.ibm.com/docs/monitoring?topic=monitoring-platform_metrics_enabling)
Smart scaling 

This feature is available only if you are on Netezza Performance Server on AWS.

With Netezza Performance Server smart scaling, you can gain a deeper understanding of your system resources, especially if you are a database administrator or an operational decision maker.

By using machine learning, smart scaling learns from historical resource data and produces predictions that anticipate future system utilization. You can now prepare and scale accordingly for varying workloads ahead of time. With this tool, you can ease system exhaustion, reduce wasted resources, and lower costs.

For example, as a database administrator you want to to get a deeper understanding of your system utilization over time. You are interested in both historical trends in resource data and future predictions for the next quarter. By understanding utilization trends for the next quarter, you can make an informed scaling schedule to match anticipated peaks and dips in workloads. During the next quarter, you will see performance improvement and reduction in resource waste and costs.



  Smart scaling with the web console 

For more information, see [Smart scaling](https://cloud.ibm.com/docs/netezza?topic=netezza-smartscaling).
Understanding your responsibilities when using IBM Cloud Metrics Routing 

Learn about the management responsibilities and terms and conditions that you have when you use IBM Cloud® Metrics Routing. For a high-level view of the service types in IBM Cloud and the breakdown of responsibilities between the customer and IBM for each type, see [Shared responsibilities for IBM Cloud offerings](https://cloud.ibm.com/docs/overview?topic=overview-shared-responsibilities).

Review the following sections for the specific responsibilities for you and for IBM when you use IBM Cloud Metrics Routing. For the overall terms of use, see [IBM Cloud Terms and Notices](https://cloud.ibm.com/docs/overview/terms-of-use?topic=overview-terms).



 Incident and operations management 



Table 1. Responsibilities for incident and operations

 Task IBM Responsibilities Your Responsibilities 

 Incident and operations management Maintain service instances and infrastructure workloads. Maintain incident and operations management of your data. 
 Monitor incidents Provide notifications for planned maintenance, security bulletins, or unplanned outages. Set preferences to [receive emails about platform notifications](https://cloud.ibm.com/docs/overview?topic=overview-uiemail-prefsl). <br>Monitor the [IBM Cloud announcements page](https://cloud.ibm.com/status?selected=announcement) for general announcements. 
 Maintain IBM Cloud high availability SLA for IBM Cloud Metrics Routing Provide IBM Cloud Metrics Routing functionality across availability zones in a Multi-Zone Region (MZR). <br>Provide replication, fail-over features, and infrastructure maintenance and updates. Keep your IBM Cloud Metrics Routing configuration in a version control system so that you can reconfigure a region if needed. <br>Comply with [Operational responsibilities when using IBM Cloud Monitoring](https://cloud.ibm.com/docs/monitoring?topic=monitoring-shared-responsibilities). 
 Monitor metrics for IBM Cloud Metrics Routing [Participating Cloud services](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-cloud-services-mr) publish relevant data to their subscribing clients.
Working with the Kubernetes agent 

After you provision an instance of the IBM Cloud® Monitoring service in the IBM Cloud, you can deploy the Monitoring agent on your cluster to collect data and metrics automatically. You can configure which metrics to monitor in each environment.

You can associate one or more tags to each monitoring agent. Tags are comma-separated values that are formatted as TAG_NAME:TAG_VALUE. When you monitor your environment, you can use these tags to identify metrics that are available from an agent. For example, you can include information about the service name and location with all of the metrics that are collected by this agent.



 Prereqs 



* Check the topic [Tuning Sysdig Agent](https://docs.sysdig.com/en/docs/installation/sysdig-agent/troubleshooting-agent-installation/tuning-sysdig-agent/)
* [Get information about Kubernetes monitoring agent images](https://cloud.ibm.com/docs/monitoring?topic=monitoring-monitoring_agent_image).
* [Install the IBM Cloud CLI and plug-ins](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_installcs_cli_install_steps):



* IBM Cloud CLI (ibmcloud)
* IBM Cloud Kubernetes Service plug-in (ibmcloud ks)
* IBM Cloud Container Registry plug-in (ibmcloud cr)
* IBM Cloud Kubernetes Service observability plug-in (ibmcloud ob)



* [Install the Kubernetes CLI (kubectl)](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_installkubectl)

Make sure that the kubectl version is compatible with your cluster version. If the kubectl version is not compatible, you can get an error such as kubectl create clusterrolebinding failed!. You can use kubectl version --short to check versions of your cluster and your kubectl client.
Setting up service endpoints for IBM Cloud Metrics Routing 

By default, private and public endpoints are enabled. For more information about supported endpoints, see [Endpoints](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-endpoints).





 Disabling public service endpoints 

You can disable public endpoints in your account for IBM Cloud Metrics Routing:



* If public and private endpoints are enabled, you can disable a public endpoint by using private or public endpoints.
* If public endpoints are disabled, you can enable public endpoints only by using the private endpoint.



For more information, see [Enforcing private endpoints to configure IBM Cloud Metrics Routing resources](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-endpoints-enforce-private).





 Disabling private service endpoints 

You cannot disable private endpoints.
To find the URL of an application by using the [Code Engine console](https://cloud.ibm.com/codeengine/overview), go to the Application overview page. If the application is in Ready state, you can open it in a web page by clicking Open application URL. Save the URL.





 Step 2: Running the Iter8 Docker container 

Iter8 is provided as a Docker container that includes all the dependencies that are needed to run Iter8. The following command starts the Docker container.

docker run --name ind --privileged -d iter8/ind:0.7.4





 Step 3: Initializing the Iter8 Docker container 

When you initialize this Docker container, the command starts a local Kubernetes cluster within the Docker container and then installs Iter8 inside of the container.

docker exec ind ./iter8.sh

Look for output that says All systems go....





 Step 4: Validating service-level objectives (SLOs) 

Verify that your Code Engine application satisfies latency and error-based service level objectives (SLOs) that you determined for your application. The following command generates requests for your Code Engine application, constructs its latency and error profile, and verifies that your application satisfies the specified SLOs.

Replace <URL-OF-YOUR-APPLICATION> with the URL obtained in [Step 1](https://cloud.ibm.com/docs/codeengine?topic=codeengine-slovalidationtutgeturl-slovalidationtut). You can also set custom limits for the three metrics that are used to evaluate your application. In the following example, you are verifying that the mean latency of your application is under 200.0 ms, the error rate is under 1% (you can make this 0.0), and the 95th percentile tail latency is under 500.0 ms.

docker exec ind helm install 
--set URL=<URL-OF-YOUR-APPLICATION> 
--set LimitMeanLatency=200.0
About IBM Cloud Metrics Routing in IBM Cloud 

IBM Cloud Metrics Routing is a platform service that you can use to configure the routing of platform metrics generated in your IBM Cloud account.

You can use IBM Cloud Metrics Routing, a platform service, to manage metrics at the account-level by configuring targets and routes that define where data points are routed. IBM Cloud Metrics Routing can only route metrics that are generated in [supported regions](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-regionsregions-metrics-router) by enabled services. Other regions, where IBM Cloud Metrics Routing is not available, continue to manage metrics by using [Monitoring](https://cloud.ibm.com/docs/monitoring?topic=monitoring-getting-started). For more information about IBM Cloud Metrics Routing, see [About IBM Cloud Metrics Routing](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-about).

IBM Cloud Metrics Routing is an IBM Cloud platform service that enables the routing of platform metrics to appropriate backend storage and analysis systems based on compliance needs.

Use IBM Cloud Metrics Routing to combine platform metrics from many regions into one monitoring instance or to segment platform metrics into multiple monitoring instances within a region as dictated by your operational requirements.

Zoom

![A diagram that shows a sample IBM Cloud Metrics Routing architecture.](https://cloud.ibm.com/docs-content/v1/content/dad8f28785f0dc036aaae4b777787e73db9ff0d4/metrics-router//images/metrics-routing-ov.png)

Figure 1. IBM Cloud Metrics Routing sample architecture



 Configuring the account 

To configure IBM Cloud Metrics Routing in your account, you must define where metrics are routed and stored. You must configure 1 or more targets, and 1 or more routes. You must also configure the account settings.
The following table outlines the pre-defined dashboards that you can use to monitor DNS Services metrics:



Table 6. Pre-defined dashboard

 Dashboard name Description 

 DNS Services - Metrics Summary The default summary dashboard that opens when you launch IBM Cloud Monitoring web UI from your service instance UI. 
 DNS Services - DP Metrics The default data path dashboard that opens when you launch IBM Cloud Monitoring web UI from your service instance UI. 
 DNS Services - Custom Resolver (CR) Metrics The default custom resolver dashboard that opens when you launch IBM Cloud Monitoring web UI from your service instance UI. 



The Default dashboard cannot be changed.
* CLI
* API







 Configuring account settings 

You can configure your account settings for IBM Cloud® Metrics Routing by using the IBM Cloud Metrics Routing CLI, the IBM Cloud Metrics Routing REST API, and Terraform scripts. Set these settings to define where and how metrics are collected, routed, and managed in your account.

When you configure or modify the IBM Cloud Metrics Routing account settings, consider the following information:



* Before you disable public endpoints by setting --private-api-endpoint-only TRUE, make sure your account has access to the private endpoint. You can do this by running the command ibmcloud account show. If VRF Enabled is true and Service Endpoint Enabled is true then you have access to the private endpoint. If you do not have access to the private endpoint, you will be unable to re-enable the public endpoint since private endpoint access is required to re-enable the public endpoint.



For more information, see [About account settings](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-settings-about&interface=api).



 IAM permissions 

Ensure you have the [correct IAM permissions to configure IBM Cloud Metrics Routing settings.](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-iam)





 CLI prerequisites 

Before you use the the CLI to manage IBM Cloud Metrics Routing account settings, [Install the IBM Cloud Metrics Routing CLI](https://cloud.ibm.com/docs/metrics_router?topic=metrics_router-metrics_router-cli-config).

Check that you have IAM permissions to read, update, or both the IBM Cloud Metrics Routing account settings.





 Getting account settings using the CLI 

Use this command to get the settings for the IBM Cloud Metrics Routing account configurations.

ibmcloud metrics-router setting get [--output FORMAT]



 Command options 

--output FORMAT