Chapter 3. Customizing Red Hat OpenStack Services on OpenShift Observability Use observability with Red Hat OpenStack Services on OpenShift (RHOSO) to get insight into the metrics, logs, and alerts from your deployment. The observability architecture in RHOSO is composed of services within OpenShift, as well as services on your Compute nodes that expose metrics, logs, and alerts.You can use the OpenShift observability ecosystem for insight into the RHOSO environment. Additionally, you have access to the logging infrastructure for collecting, storing, and searching through logs. RHOSO services such as ceilometer and sg-core make metrics from your compute nodes and associated virtual infrastructure available to the OpenShift Observability framework. 3.1. Configuring Red Hat OpenStack Services on OpenShift Observability The Telemetry service (ceilometer, prometheus) is enabled by default in a Red Hat OpenStack Services on OpenShift (RHOSO) deployment. You can configure observability by editing the openstack_control_plane.yaml CR file. Prerequisites Optional: If you enable logging, the Cluster Logging Operator is installed from OperatorHub . A LokiStack instance must be running. For more information, see the Logging Quick start and Storing logs with LokiStack . A ClusterLogForwarder instance must be configured with a Syslog receiver. For more information, see Configuring log forwarding and the Syslog receiver configuration example . To configure a dashboard for the logs, see Visualization for logging . Note You do not need these Operators to expose and query OpenStack metrics in Prometheus format. If you do not disable ceilometer, then a Prometheus metrics exporter is created and exposed from inside the cluster at the following URL: http://ceilometer-internal.openstack.svc:3000/metrics Procedure Open the OpenStackControlPlane CR definition file, openstack_control_plane.yaml , on your workstation. Update the telemetry section based on the needs of your environment: 1 Use the scrapeInterval field to control the amount of time that passes before new metrics are gathered. Changing this parameter can affect performance. 2 Use the retention field to adjust the length of time telemetry metrics are stored. This field affects the amount of storage required. 3 Use the pvcStorageRequest field to change the amount of storage to be allocated for the Prometheus time series database. 4 Use the ipaddr field to set the IP address that rsyslog sends messages to. Ensure that the IP address is reachable from the Compute node. The default IP address is the vIP for internalapi , which is 172.17.0.80. Ensure that ipaddr and loadBalancerIPs have the same IP address, so that the client and server can communicate. Update the control plane: Wait until RHOCP creates the resources related to the OpenStackControlPlane CR. Run the following command to check the status: The OpenStackControlPlane resources are created when the status is "Setup complete". Tip Append the -w option to the end of the get command to track deployment progress. Enable the logging service by editing the OpenStackDataplaneNodeSet CR. In the services list, add logging after telemetry : Optional: Enable the telemetry-power-monitoring service by editing the OpenStackDataplaneNodeSet CR. In the services list, add telemetry-power-monitoring after telemetry : Important This feature is available in this release as a Technology Preview , and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details . For more information about deploying data plane services, see Deploying the data plane in the Deploying Red Hat OpenStack Services on OpenShift guide. Optional: Confirm that the control plane is deployed by reviewing the pods in the openstack namespace for each of your cells: The control plane is deployed when all the pods are either completed or running. Verification Access the remote shell for the OpenStackClient pod from your workstation: Confirm that you can query prometheus and that the scrape endpoints are active: Example output: Note Each entry in the value field should be "1" when there are active workloads scheduled on the cluster, except for the prometheus container. The prometheus container reports a value of "0" due to TLS, which is enabled by default. You can find the openstack-telemetry-operator dashboards by clicking Observe and then Dashboards in the RHOCP console. For more information about RHOCP dashboards, see Reviewing monitoring dashboards as a cluster administrator in the RHOCP Monitoring Guide. Optional: Verify that you deployed the telemetry-power-monitoring service. Check for ceilometer_agent_ipmi and kepler containers in the dataplane nodes: Important This feature is available in this release as a Technology Preview , and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details . Additional resource Installing Logging

Commands:
telemetry: enabled: true template: metricStorage: enabled: true dashboardsEnabled: true monitoringStack: alertingEnabled: true scrapeInterval: 30s 1 storage: strategy: persistent retention: 24h 2 persistent: pvcStorageRequest: 20G 3 autoscaling: enabled: false aodh: databaseAccount: aodh databaseInstance: openstack secret: osp-secret heatInstance: heat ceilometer: enabled: true secret: osp-secret logging: enabled: false ipaddr: 172.17.0.80 4 annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80
oc apply -f openstack_control_plane.yaml -n openstack
oc get openstackcontrolplane -n openstack NAME STATUS MESSAGE openstack-control-plane Unknown Setup started
- - telemetry - logging -
- - telemetry - telemetry-power-monitoring -
oc get pods -n openstack
oc rsh -n openstack openstackclient
openstack metric query up --disable-rbac -c container -c instance -c value
+-----------------+------------------------+-------+ | container | instance | value | +-----------------+------------------------+-------+ | alertmanager | 10.217.1.112:9093 | 1 | | prometheus | 10.217.1.63:9090 | 0 | | proxy-httpd | 10.217.1.52:3000 | 1 | | | 192.168.122.100:9100 | 1 | | | 192.168.122.101:9100 | 1 | +-----------------+------------------------+-------+
podman ps | grep -i -e ceilometer_agent_ipmi -e kepler