vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Analyzing event data trends by using the API 

In IBM® Log Analysis, you can query your logging instance and identify usage trends over a period of time by using the Usage API. You can get aggregated usage data information for applications, hosts and tags during a specific period of time and within the last 6 months.



 API method 

To query usage data, you can use the following methods:



* For all apps during a time period, you can list the aggregated usage.

curl "<ENDPOINT>/v1/usage/apps/<SERVICE_NAME>?from=<FROM_DATE>&to=<TO_DATE>&limit=<LIMIT_NUMBER>" -u <SERVICE_KEY>:
* For all tags during a time period, you can list the aggregated usage.

curl "<ENDPOINT>/v1/usage/tags/<SERVICE_NAME>?from=<FROM_DATE>&to=<TO_DATE>&limit=<LIMIT_NUMBER>" -u <SERVICE_KEY>:
* For all hosts during a time period, you can list the aggregated usage.

curl "<ENDPOINT>/v1/usage/hosts/<SERVICE_NAME>?from=<FROM_DATE>&to=<TO_DATE>&limit=<LIMIT_NUMBER>" -u <SERVICE_KEY>:
* For 1 app during a time period, you can get the aggregated usage.

curl "<ENDPOINT>/v1/usage/apps/<SERVICE_NAME>/<APP_NAME>?from=<FROM_DATE>&to=<TO_DATE>&limit=<LIMIT_NUMBER>" -u <SERVICE_KEY>:
* For 1 tag during a time period, you can get the aggregated usage.

curl "<ENDPOINT>/v1/usage/tags/<SERVICE_NAME>/<TAG_NAME>?from=<FROM_DATE>&to=<TO_DATE>&limit=<LIMIT_NUMBER>" -u <SERVICE_KEY>:
* For 1 host during a time period, you can get the aggregated usage.
See [Locations for resource deployment](https://cloud.ibm.com/docs/overview?topic=overview-locations) for more information.
* Are the two free months of transition period resources in addition to my existing server time?

Yes. You can [contact an appropriate support representative](https://www.ibm.com/cloud/data-centers?contactmodule=undefined) to help you through the process of acquiring your transition period servers.
* How do I determine my current hardware configuration?

You can find your system configuration details by selecting your device from your [list of resources](https://cloud.ibm.com/resources) in the IBM Cloud console.
* How do I determine the utilization of my current hardware?

In general, you need to understand which specific resources within the system are required regarding things like the processor, memory, disk, and network. Having this information can help you better size your new system. For example, a system where memory capacity is frequently overcommitted is likely to benefit from larger memory sizes in the target system that you migrate to.

Most operating systems provide tools that you can use to understand the utilization of your system, for example, vmstat and iostat on Linux or Windows System Performance Monitor. Performance monitoring and tuning is something that you might invest significant time and effort in.

For more information, contact the [Client Success team](https://www.ibm.com/cloud/data-centers/?focusArea=WCP%20-%20Pooled%20CSM&contactmodule).
* How do I compare old and new processors?

To compare the specifications of old and new Intel processors, go to [Intel processors product specifications page](https://ark.intel.com/content/www/us/en/ark.html@Processors). Newer processors tend to have more processor cores and typically run at slower clock speeds than older variants.

If you have a workload that is processor-bound, meaning that the performance is limited by the speed of the processor, choose a processor that has fewer cores and a higher clock speed.
Unless your profile contains additional controls, you can skip the Parameters tab by clicking Next.
6. Toggle Enable scan to On to ensure that the scan runs.
7. Select the frequency at which you want to evaluate your resource. Options include, Every day, Every 7 days, and Every 30 days.
8. Optionally, you can enable notifications.
9. Click Next. Review your selections and click Create.







 Next steps 

When the scan completes, your results become available in the Security and Compliance Center dashboard. Be sure to check back in a few hours to see what your results returned.
8088 <br><br> * <br><br><br> 9080 7080 <br><br> * <br><br><br> Pool_8088:9080/7080 



The displayed metrics contain a timestamp in UNIX epoch time and the metric value for the time interval ending at that timestamp. You can specify different scopes, as well as the time interval over which to report the metrics.

The supported listener protocols for IBM Cloud Application Load Balancer include:



* HTTP
* HTTPS
* TCP



Specifying a listener port filters the metric by that listener. For example, if you don't specify a port, and the metric is Throughput, then IBM Cloud Monitoring reports the total throughput for all listener protocols. However, if the protocol is HTTP on port 80, then IBM Cloud Monitoring reports the throughput for HTTP traffic only.

You can also specify the time interval over which to report your metrics. Time intervals that are supported in the dashboard are:



* 10 seconds
* 1 minute
* 10 minutes
* 1 hour
* 6 hours
* 2 weeks
* Custom



The number of data points you can report is roughly the same for each time interval. For example, if the interval is 1 hour, then each data point represents 5 minutes of data. If the interval is 2 weeks, then each data point represents 24 hours of data.







 Enabling metrics monitoring 

To receive monitoring metrics, you must set up IBM Cloud Monitoring with a monitoring instance.

To do so, follow these steps:



1. Navigate to the [metrics monitoring portal](https://cloud.ibm.com/observe/monitoring), then click Create a monitoring instance.
2. Select a region for your monitoring instance.

If you do not have an existing load balancer, see [Creating an IBM Cloud Application Load Balancer for VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-load-balancer) to provision one.

The region should match the location of your existing load balancer.
3. Choose your pricing plan.
You can configure one monitoring instance in each region to collect platform metrics for IBM Cloud Container Registry. The following tables list the locations where metrics can be collected if you enable collection of Container Registry service metrics in that region.



Table 1. The automatic collection of Container Registry service metrics in Americas locations

 Locations in Americas Platform metrics available 

 Dallas (us-south) Yes 
 Sao Paulo (br-sao) Yes 
 Toronto (ca-tor) Yes 





Table 2. The automatic collection of Container Registry service metrics in Asia Pacific locations

 Locations in Asia Pacific Platform metrics available 

 Osaka (jp-osa) Yes 
 Sydney (au-syd) Yes 
 Tokyo (jp-tok) Yes 





Table 3. The automatic collection of Container Registry service metrics in Europe locations

 Locations in Europe Platform metrics available 

 Frankfurt (eu-de) Yes 
 London (eu-gb) Yes 





Table 4. The automatic collection of Container Registry service metrics for Global

 Location for Global Platform metrics available 

 Global Yes 



For more information about where to see Container Registry metrics, see [Where to look for metrics](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_monitorregistry_monitor_ui).

For more information about the locations where IBM Cloud services are enabled to send metrics to IBM Cloud Monitoring, see [IBM Cloud services by location](https://cloud.ibm.com/docs/monitoring?topic=monitoring-cloud_services_locations).





 Where to look for metrics 



 Monitoring metrics 

The [region](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_overviewregistry_regions) in which a Container Registry or a Vulnerability Advisor metric is available corresponds to the region of the Container Registry that generated the metric.

The following table shows the location of Monitoring metrics.



Table 5. Location of Monitoring metrics

 Region for your account's registry Domain name of your registry Location of Monitoring metrics 

 ap-north jp.icr.io Tokyo (jp-tok) 
 ap-south au.icr.io Sydney (au-syd)
Routes are global under an account and are evaluated in all regions where IBM Cloud Metrics Routing is deployed. For more information, see [Understanding how routes work in your account](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-routes&interface=cliroute_behaviour).

In this step, you will configure a route to redirect metrics to the target destination that you configured in the previous step.

Run the following command to create the route:

ibmcloud metrics-router route create --name ROUTE_NAME --rules RULES

Where

--name ROUTE_NAME
: Defines the name to be given to the route.

--rules RULES
: Defines a JSON formatted rule definition enclosed in single quotes. [Learn more](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-route_rules_definitions&interface=cli).

After you configure a route, it might take up to 1 hour for the configuration to be enabled.

For example, to create a route to send metrics generated in us-east to the target that you created in the previous step, run the following command.

ibmcloud metrics-router route create --name "my-route" --rules '[{"action": "send", "targets":{"id":"TARGETID"}], "inclusion_filters":{"operand": "location","operator": "is","values": "us-east"}]}]''

Where TARGETID is the ID of the target that you created in the previous step.





 Step 4. Verify collection of metrics 

After the target and the route is configured, you must verify that metrics are available.

[Launch the IBM Cloud Monitoring UI](https://cloud.ibm.com/docs/monitoring?topic=monitoring-launch) for the Monitoring instance that you configured as your target, and explore the metrics.
* 6 hours
* 2 weeks
* Custom









 Enabling metrics monitoring 

To receive monitoring metrics, you must set up your IBM Cloud Monitoring instance.

To receive monitoring metrics, use the following steps:



1. Navigate to the [metrics monitoring portal](https://cloud.ibm.com/observe/monitoring) and click Create a monitoring instance.
2. Select a region for your IBM Cloud Monitoring instance.

If you do not have an existing VPN server, see [Creating a VPN server](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-a-vpc-using-the-ibm-cloud-consolevpn-ui) to provision one.

The region needs to match the location of your existing VPN server.
3. Choose your pricing plan. Pricing plan details are explained in the selection window. Select the plan that best meets your requirements.
4. Provide a unique service name for your instance. The name can be any name that you want and has no impact on functionality.

Do not give multiple IBM Cloud Monitoring instances the same name.
5. Optionally, select a resource group. A resource group organizes account resources in customizable groupings. Any account resource that is managed by using IBM Cloud Identity and Access Management (IAM) access control belongs to a resource group within your account.

If you do not have any pre-configured resource groups, or have no reason to share this resource selectively, use the default selection.

If your account has multiple resource groups, you can choose which group has access to this IBM Cloud Monitoring instance. By using this selective access, metrics can be available to some resource groups and not to others.
6. Check the Enable Platform Metrics checkbox. You must select this option to receive metrics from your VPN server.
7. Click Create. You are taken back to the monitoring metrics home page.



Within a few minutes, your new IBM Cloud Monitoring instance displays with several configurations. You might have to refresh your browser to see it.



 Working with the IBM Cloud Monitoring dashboard
* startDate - Provide the start date timestamp (UTC+0) for query.
* endDate - Provide the end date timestamp (UTC+0) for query.
* frequency - Provide the data frequency for data format, the following options are available to configure frequency:



* aggregate - Aggregated metrics data from startDate to endDatedefault.
* day - Daily metrics data from startDate to endDate.





* Return - A collection of objects of type SoftLayer_Container_Network_CdnMarketplace_Metrics.



--------------------





 getCustomerUsageMetrics 

Returns the total number of predetermined statistics for direct display (no graph) for a customer's account, over a given period of time.



* Parameters:



* vendorName - Provide the name of the CDN provider for this customer. Currently, only akamai is supported.
* startDate - Provide the start date timestamp (UTC+0) for query.
* endDate - Provide the end date timestamp (UTC+0) for query.
* frequency - Provide the data frequency for data format, the following options are available to configure frequency:



* aggregate - aggregated metrics data from startDate to endDatedefault.
* day - daily metrics data from startDate to endDate.





* Return - a collection of objects of type SoftLayer_Container_Network_CdnMarketplace_Metrics



--------------------





 getMappingIntegratedMetrics 

Returns the integrated metrics data for the given mapping, which includes the total number of predetermined statistics, detailed hits and bandwidth, hits by type, and the bandwidth by region. The value of frequency is day by default. For more details, see [View the examples](https://cloud.ibm.com/docs/CDN?topic=CDN-code-examples-using-the-cdn-apiexample-code-for-getting-integrated-metrics).



* Parameters:
* Use the Metrics Router to allow customers to configure which Sysdig instance their platform metrics flows to. To learn more about Metrics Router, see [IBM Cloud Metrics Routing](https://cloud.ibm.com/docs/metrics-router).



To monitor platform metrics, check that the Monitoring instance is provisioned in the same region where the IBM Cloud instance is provisioned.





 Viewing metrics 

To monitor DNS Services metrics, you must launch the IBM Cloud Monitoring web UI instance that is enabled for platform metrics in the region where your IBM Cloud instance is available.

You can use different options to launch the IBM Cloud Monitoring web UI and monitor metrics that are described in the following section.



 Launching IBM Cloud Monitoring web UI from the Observability page 

Complete the following steps to launch the IBM Cloud Monitoring web UI from the Observability page:



1. [Launch the IBM Cloud Monitoring web UI](https://cloud.ibm.com/docs/monitoring?topic=monitoring-launch).
2. Click DASHBOARDS.
3. In the Default Dashboards section, expand IBM.
4. Choose the DNS Services Dashboard from the list.

You can also reach your deployment's IBM Cloud Monitoring Dashboard from IBM Cloud Monitoring in the sidebar, under IBM.

Next, change the scope or make a copy of the Default dashboard to monitor an DNS Services instance.









 DNS Services metrics dictionary 

DNS Services collects two levels of platform metrics from two locations within the data path; the data path and custom resolvers. By default, you'll receive only the summary of metrics which gives a minimum amount of information about DNS query requests, DNS query results, and custom resolver location health.

Open a Support case to enable the total metrics set.

Platform metrics are priced per usage.

Enabling the total metrics set increases the amount of metrics you receive, and can incur a greater cost depending on your usage.



 Data path summary metrics 

The data path summary metrics provide the request count and response count with the corresponding response codes.



Table 2. Data path metrics summary

 Metric name Labels
Actual values might, for example, be 0.27, 0.52, and so on, depending on when the service receives and processes audio.
* Values for transcription events are not related to the processing interval. The service returns event-driven results as they occur.



* periodic indicates whether the metrics apply to a periodic interval or to a transcription event:



* true means that the response was triggered by a processing interval. The information contains processing metrics only.
* false means that the response was triggered by a transcription event. The information contains processing metrics plus transcription results.



Use this field to identify why the service generated the response and to filter different results if necessary.
* processed_audio includes a ProcessedAudio object that provides detailed timing information about the service's processing of the input audio.



The ProcessedAudio object includes the following fields. All of the fields refer to seconds of audio as of this response. Only the wall_clock_since_first_byte_received field refers to elapsed real-time.



* received is the seconds of audio that the service has received.
* seen_by_engine is the seconds of audio that the service has passed to its speech processing engine.
* transcription is the seconds of audio that the service has processed for speech recognition.
* speaker_labels is the seconds of audio that the service has processed for speaker labels. The response includes this field only if you request speaker labels.



The speech processing engine analyzes the input audio multiple times. The processed_audio object shows values for audio that the engine has processed and will not read again. Processed audio has no effect on future recognition hypotheses.

The metrics indicate the progress and complexity of the engine's processing:



* seen_by_engine is audio that the service has read and passed to the engine at least once.
* received - seen_by_engine is audio that has been buffered at the service but has not yet been seen or processed by the engine.
* The relationship between the times is received >= seen_by_engine >= transcription >= speaker_labels.



The following relationships can also be helpful in understanding the results:
The OR operator is defined using a comma (,) or OR keyword surrounded by whitespace. The AND operator is defined using a semicolon (;) or AND keyword surrounded by whitespace. Comparison options are: ==, =, >, <, >=, <=. An example value for filters is: event==connect AND coloName=SFO.--sort: The sort order for the result set. Sort fields must be included in metrics or dimensions. An example value for sort is: +count,-bytesIngress.--since: Start of time interval to query, defaults to until - 6 hours. This should be an absolute timestamp that conforms to RFC 3339.--until: End of time interval to query, defaults to current time. This should be an absolute timestamp that conforms to RFC 3339.--bytime: Analytics data for range applications grouped by time interval.--time-delta: Used to select time series resolution. Valid values: year, quarter, month, week, day, hour, dekaminute, minute. Only valid when --bytime is given.-i, --instance: Instance name or ID. If not set, the context instance specified by ibmcloud cis instance-set INSTANCE is used.--output: Specify output format, only JSON is supported.<-- </section "id="section-get-analytics-range-app-options" "> --><-- <section "id="section-get-analytics-range-app-examples" "> --> Examples Get analytics data for range applications in domain 31984fea73a15b45779fa0df4ef62f9b. ibmcloud cis range-analytics 31984fea73a15b45779fa0df4ef62f9b --metrics "count,bytesIngress" --dimensions "event,appID" --since "2020-05-22T02:20:00Z"
--until "2020-05-23T02:20:00Z" -i "cis-demo"
![Team platform metrics option](https://cloud.ibm.com/docs-content/v1/content/220cdcbf6f4421bd7fee83fd1028048dba3f16c8/monitoring/platform/images/sysdig-platform-10.png)

Team platform metrics

Enabling platform metrics, grants access to all platform metrics. However, you can reduce the scope by configuring 1 or more segments. Notice that the order of the segments is applied from the beginning of the list to the end.

Zoom

![Team platform metrics segments](https://cloud.ibm.com/docs-content/v1/content/220cdcbf6f4421bd7fee83fd1028048dba3f16c8/monitoring/platform/images/sysdig-platform-11.png)

Team platform metrics segments



 Limiting access to platform metrics by resource group 

Complete the following steps to limit the data to metrics collected for services that are created in that resource group:



1. [Launch the monitoring UI](https://cloud.ibm.com/docs/monitoring?topic=monitoring-launch).
2. Navigate to the Settings section.

Zoom

![Settings section](https://cloud.ibm.com/docs-content/v1/content/220cdcbf6f4421bd7fee83fd1028048dba3f16c8/monitoring/platform/images/sysdig-platform-12.png)

Settings
3. Select Teams. Then, select a team where you want to restrict access to the platform metrics.
4. In the Visibility section, select Platform metrics.
5. Select the attribute ibm_resource_group_id to segment data by resource group.

Zoom

!
"k0": "timestamp",
"v1": "ibm_is_vpn_gateway_status"
},
"time": {
"from": 1589877054000000,
"to": 1589877114000000,
"sampling": 60000000
},
"group": {
"by":
{
"metric": "k0",
"value" : 60000000
}
],
"aggregations": {
"v1": "avg"
},
"groupAggregations": {
"v1": "avg"
}
}
}
]
}'







 Working with IBM Cloud Monitoring by using VPN for VPC UI 

You can view individual VPN gateway metrics and launch the IBM Cloud Monitoring dashboard from the VPN UI.



1. Navigate to the [VPN UI](https://cloud.ibm.com/vpc-ext/network/vpngateways).
2. Click a VPN gateway name whose IBM Cloud Monitoring metrics that you want to view and go to its Overview page.

The Monitoring preview panel displays the sum of every metric over the last hour.

You can click Launch monitoring to launch the VPN gateway's default IBM Cloud Monitoring dashboard.
3. Open the VPN gateway's Monitoring page. The page displays a VPN gateway's throughput and packets in a time range.

Similarly, you can also launch the VPN gateway's default IBM Cloud Monitoring dashboard by clicking Launch monitoring.
Note that manual verification of your storage metrics is a temporary workaround and not a permanent monitoring solution for viewing metrics. After completing the following steps, if you are able to manually view the metrics in the CLI and not the dashboard, this indicates that there is a console issue for which you must [open a support issue](https://cloud.ibm.com/docs/containers?topic=containers-get-helphelp-support).

After you complete the following steps, make sure to remove the resources you created while debugging.



1. Create and deploy a custom clusterRole configuration. In this example, the clusterRole is named test-metrics-reader.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: test-metrics-reader
rules:
- nonResourceURLs:
- "/metrics"
verbs:
- get
- apiGroups:
- ""
resources:
- nodes/metrics
verbs:
- get
Show more

kubectl apply -f <file_name>
2. Create a service account. In this example, the service account is named test-sa.

kubectl create sa test-sa
3. Add a clusterRoleBinding to the clusterRole.

kubectl create clusterrolebinding test-metrics-reader --clusterrole test-metrics-reader --serviceaccount=default:test-sa
4. List your nodes and note the name and IP of the node for which you want to gather metrics.

kubectl get nodes

Example output.

NAME STATUS ROLES AGE VERSION
10.111.1.11 Ready <none> 1d v1.26+IKS
5. Create a yaml file to deploy a pod onto the node. Make sure to specify the service account you created and the node IP address.

apiVersion: v1
kind: Pod
metadata:
name: testpod
spec: