Chapter 4. Monitoring Camel Spring Boot integrations This chapter explains how to monitor integrations on Red Hat build of Camel Spring Boot at runtime. You can use the Prometheus Operator that is already deployed as part of OpenShift Monitoring to monitor your own applications. For more information about deploying the Camel Spring Boot applications on OpenShift Container Platform, see Apache Camel on OCP Best practices . For information about the HawtIO Diagnostic Console, see the HawtIO Diagnostic Console documentation . 4.1. Enabling user workload monitoring in OpenShift You can enable the monitoring for user-defined projects by setting the enableUserWorkload: true field in the cluster monitoring ConfigMap object. Important In OpenShift Container Platform 4.13 you must remove any custom Prometheus instances before enabling monitoring for user-defined projects. Prerequisites You must have access to the cluster as a user with the cluster-admin cluster role access to enable monitoring for user-defined projects in OpenShift Container Platform. Cluster administrators can then optionally grant users permission to configure the components that are responsible for monitoring user-defined projects. You have cluster admin access to the OpenShift cluster. You have installed the OpenShift CLI ( oc ). Note Every time you save configuration changes to the user-workload-monitoring-config ConfigMap object, the pods in the openshift-user-workload-monitoring project are redeployed. It can sometimes take a while for these components to redeploy. You can create and configure the ConfigMap object before you first enable monitoring for user-defined projects, to prevent having to redeploy the pods often. Procedure Login to OpenShift with administrator permissions. Edit the cluster-monitoring-config ConfigMap object. Add enableUserWorkload: true in the data/config.yaml section. When it is set to true, the enableUserWorkload parameter enables monitoring for user-defined projects in a cluster. Save the file to apply the changes. The monitoring for the user-defined projects is then enabled automatically. Note When the changes are saved to the cluster-monitoring-config ConfigMap object, the pods and other resources in the openshift-monitoring project might be redeployed. The running monitoring processes in that project might also be restarted. Verify that the prometheus-operator , prometheus-user-workload and thanos-ruler-user-workload pods are running in the openshift-user-workload-monitoring project. 4.2. Monitoring a Camel Spring Boot application After you enable the monitoring for your project, you can deploy and monitor the Camel Spring Boot application. This section uses the monitoring-micrometrics-grafana-prometheus example listed in the Camel Spring Boot Examples . Procedure Add the openshift-maven-plugin to the pom.xml file of the monitoring-micrometrics-grafana-prometheus example. In the pom.xml , add an openshift profile to allow deployment to openshift through the openshift-maven-plugin. Add the openshift-maven-plugin to the pom.xml file of the monitoring-micrometrics-grafana-prometheus example. In the pom.xml , add an openshift profile to allow deployment to openshift through the openshift-maven-plugin . <profiles> <profile> <id>openshift</id> <build> <plugins> <plugin> <groupId>org.eclipse.jkube</groupId> <artifactId>openshift-maven-plugin</artifactId> <version>1.13.1</version> <executions> <execution> <goals> <goal>resource</goal> <goal>build</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </profile> </profiles> Add the Prometheus support. In order to add the Prometheus support to your Camel application, expose the Prometheus statistics on an actuator endpoint. Edit your src/main/resources/application.properties file. Add a management.endpoints.web.exposure.include entry if it doesn't exist. Add prometheus, metrics, and health to the management.endpoints.web.exposure.include entry: Add the following to the <dependencies/> section of your pom.xml to add some starter support to your application. <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context</artifactId> <version>6.1.8</version> </dependency> <dependency> <groupId>io.micrometer</groupId> <artifactId>micrometer-registry-prometheus</artifactId> <version>1.13.6</version> </dependency> <dependency> <groupId>org.jolokia</groupId> <artifactId>jolokia-core</artifactId> <version>2.1.0</version> </dependency> <dependency> <groupId>io.prometheus.jmx</groupId> <artifactId>collector</artifactId> <version>1.0.1</version> </dependency> Create the file config/prometheus_exporter_config.yml : startDelaySecs: 5 ssl: false blacklistObjectNames: ["java.lang:*"] rules: # Context level - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>ExchangesCompleted' name: org.apache.camel.ExchangesCompleted help: Exchanges Completed type: COUNTER labels: context: USD1 type: context - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>ExchangesFailed' name: org.apache.camel.ExchangesFailed help: Exchanges Failed type: COUNTER labels: context: USD1 type: context - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>ExchangesInflight' name: org.apache.camel.ExchangesInflight help: Exchanges Inflight type: GAUGE labels: context: USD1 type: context - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>ExchangesTotal' name: org.apache.camel.ExchangesTotal help: Exchanges Total type: COUNTER labels: context: USD1 type: context - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>FailuresHandled' name: org.apache.camel.FailuresHandled help: Failures Handled labels: context: USD1 type: context type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>ExternalRedeliveries' name: org.apache.camel.ExternalRedeliveries help: External Redeliveries labels: context: USD1 type: context type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>MaxProcessingTime' name: org.apache.camel.MaxProcessingTime help: Maximum Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>MeanProcessingTime' name: org.apache.camel.MeanProcessingTime help: Mean Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>MinProcessingTime' name: org.apache.camel.MinProcessingTime help: Minimum Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>LastProcessingTime' name: org.apache.camel.LastProcessingTime help: Last Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>DeltaProcessingTime' name: org.apache.camel.DeltaProcessingTime help: Delta Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>Redeliveries' name: org.apache.camel.Redeliveries help: Redeliveries labels: context: USD1 type: context type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>TotalProcessingTime' name: org.apache.camel.TotalProcessingTime help: Total Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=consumers, name=([^,]+)><>InflightExchanges' name: org.apache.camel.InflightExchanges help: Inflight Exchanges labels: context: USD1 type: context type: GAUGE # Route level - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>ExchangesCompleted' name: org.apache.camel.ExchangesCompleted help: Exchanges Completed type: COUNTER labels: context: USD1 route: USD2 type: routes - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>ExchangesFailed' name: org.apache.camel.ExchangesFailed help: Exchanges Failed type: COUNTER labels: context: USD1 route: USD2 type: routes - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>ExchangesInflight' name: org.apache.camel.ExchangesInflight help: Exchanges Inflight type: GAUGE labels: context: USD1 route: USD2 type: routes - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>ExchangesTotal' name: org.apache.camel.ExchangesTotal help: Exchanges Total type: COUNTER labels: context: USD1 route: USD2 type: routes - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>FailuresHandled' name: org.apache.camel.FailuresHandled help: Failures Handled labels: context: USD1 route: USD2 type: routes type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>ExternalRedeliveries' name: org.apache.camel.ExternalRedeliveries help: External Redeliveries labels: context: USD1 route: USD2 type: routes type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>MaxProcessingTime' name: org.apache.camel.MaxProcessingTime help: Maximum Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>MeanProcessingTime' name: org.apache.camel.MeanProcessingTime help: Mean Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>MinProcessingTime' name: org.apache.camel.MinProcessingTime help: Minimum Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>LastProcessingTime' name: org.apache.camel.LastProcessingTime help: Last Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>DeltaProcessingTime' name: org.apache.camel.DeltaProcessingTime help: Delta Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>Redeliveries' name: org.apache.camel.Redeliveries help: Redeliveries labels: context: USD1 route: USD2 type: routes type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>TotalProcessingTime' name: org.apache.camel.TotalProcessingTime help: Total Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>InflightExchanges' name: org.apache.camel.InflightExchanges help: Inflight Exchanges labels: context: USD1 route: USD2 type: routes type: GAUGE # Processor level - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>ExchangesCompleted' name: org.apache.camel.ExchangesCompleted help: Exchanges Completed type: COUNTER labels: context: USD1 processor: USD2 type: processors - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>ExchangesFailed' name: org.apache.camel.ExchangesFailed help: Exchanges Failed type: COUNTER labels: context: USD1 processor: USD2 type: processors - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>ExchangesInflight' name: org.apache.camel.ExchangesInflight help: Exchanges Inflight type: GAUGE labels: context: USD1 processor: USD2 type: processors - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>ExchangesTotal' name: org.apache.camel.ExchangesTotal help: Exchanges Total type: COUNTER labels: context: USD1 processor: USD2 type: processors - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>FailuresHandled' name: org.apache.camel.FailuresHandled help: Failures Handled labels: context: USD1 processor: USD2 type: processors type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>ExternalRedeliveries' name: org.apache.camel.ExternalRedeliveries help: External Redeliveries labels: context: USD1 processor: USD2 type: processors type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>MaxProcessingTime' name: org.apache.camel.MaxProcessingTime help: Maximum Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>MeanProcessingTime' name: org.apache.camel.MeanProcessingTime help: Mean Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>MinProcessingTime' name: org.apache.camel.MinProcessingTime help: Minimum Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>LastProcessingTime' name: org.apache.camel.LastProcessingTime help: Last Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>DeltaProcessingTime' name: org.apache.camel.DeltaProcessingTime help: Delta Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>Redeliveries' name: org.apache.camel.Redeliveries help: Redeliveries labels: context: USD1 processor: USD2 type: processors type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>TotalProcessingTime' name: org.apache.camel.TotalProcessingTime help: Total Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>InflightExchanges' name: org.apache.camel.InflightExchanges help: Inflight Exchanges labels: context: USD1 processor: USD2 type: processors type: COUNTER # Consumers - pattern: 'org.apache.camel<context=([^,]+), type=consumers, name=([^,]+)><>InflightExchanges' name: org.apache.camel.InflightExchanges help: Inflight Exchanges labels: context: USD1 consumer: USD2 type: consumers type: GAUGE # Services - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>MaxDuration' name: org.apache.camel.MaxDuration help: Maximum Duration labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>MeanDuration' name: org.apache.camel.MeanDuration help: Mean Duration labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>MinDuration' name: org.apache.camel.MinDuration help: Minimum Duration labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>TotalDuration' name: org.apache.camel.TotalDuration help: Total Duration labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>ThreadsBlocked' name: org.apache.camel.ThreadsBlocked help: Threads Blocked labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>ThreadsInterrupted' name: org.apache.camel.ThreadsInterrupted help: Threads Interrupted labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>NumLogicalRuntimeFaults' name: org.apache.cxf.NumLogicalRuntimeFaults help: Number of logical runtime faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>NumLogicalRuntimeFaults' name: org.apache.cxf.NumLogicalRuntimeFaults help: Number of logical runtime faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>AvgResponseTime' name: org.apache.cxf.AvgResponseTime help: Average Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>AvgResponseTime' name: org.apache.cxf.AvgResponseTime help: Average Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>NumInvocations' name: org.apache.cxf.NumInvocations help: Number of invocations type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>NumInvocations' name: org.apache.cxf.NumInvocations help: Number of invocations type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>MaxResponseTime' name: org.apache.cxf.MaxResponseTime help: Maximum Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>MaxResponseTime' name: org.apache.cxf.MaxResponseTime help: Maximum Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>MinResponseTime' name: org.apache.cxf.MinResponseTime help: Minimum Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>MinResponseTime' name: org.apache.cxf.MinResponseTime help: Minimum Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>TotalHandlingTime' name: org.apache.cxf.TotalHandlingTime help: Total Handling Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>TotalHandlingTime' name: org.apache.cxf.TotalHandlingTime help: Total Handling Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>NumRuntimeFaults' name: org.apache.cxf.NumRuntimeFaults help: Number of runtime faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>NumRuntimeFaults' name: org.apache.cxf.NumRuntimeFaults help: Number of runtime faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>NumUnCheckedApplicationFaults' name: org.apache.cxf.NumUnCheckedApplicationFaults help: Number of unchecked application faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>NumUnCheckedApplicationFaults' name: org.apache.cxf.NumUnCheckedApplicationFaults help: Number of unchecked application faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>NumCheckedApplicationFaults' name: org.apache.cxf.NumCheckedApplicationFaults help: Number of checked application faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>NumCheckedApplicationFaults' name: org.apache.cxf.NumCheckedApplicationFaults help: Number of checked application faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 Add the following to the Application.java of your Camel application. import java.io.InputStream; import io.micrometer.core.instrument.Clock; import org.apache.camel.CamelContext; import org.apache.camel.spring.boot.CamelContextConfiguration; import org.springframework.context.annotation.Bean; import org.apache.camel.component.micrometer.MicrometerConstants; import org.apache.camel.component.micrometer.eventnotifier.MicrometerExchangeEventNotifier; import org.apache.camel.component.micrometer.eventnotifier.MicrometerRouteEventNotifier; import org.apache.camel.component.micrometer.messagehistory.MicrometerMessageHistoryFactory; import org.apache.camel.component.micrometer.routepolicy.MicrometerRoutePolicyFactory; The updated Application.java is shown below. @SpringBootApplication public class SampleCamelApplication { @Bean(name = {MicrometerConstants.METRICS_REGISTRY_NAME, "prometheusMeterRegistry"}) public PrometheusMeterRegistry prometheusMeterRegistry( PrometheusConfig prometheusConfig, CollectorRegistry collectorRegistry, Clock clock) throws MalformedObjectNameException, IOException { InputStream resource = new ClassPathResource("config/prometheus_exporter_config.yml").getInputStream(); new JmxCollector(resource).register(collectorRegistry); new BuildInfoCollector().register(collectorRegistry); return new PrometheusMeterRegistry(prometheusConfig, collectorRegistry, clock); } @Bean public CamelContextConfiguration camelContextConfiguration(@Autowired PrometheusMeterRegistry registry) { return new CamelContextConfiguration() { @Override public void beforeApplicationStart(CamelContext camelContext) { MicrometerRoutePolicyFactory micrometerRoutePolicyFactory = new MicrometerRoutePolicyFactory(); micrometerRoutePolicyFactory.setMeterRegistry(registry); camelContext.addRoutePolicyFactory(micrometerRoutePolicyFactory); MicrometerMessageHistoryFactory micrometerMessageHistoryFactory = new MicrometerMessageHistoryFactory(); micrometerMessageHistoryFactory.setMeterRegistry(registry); camelContext.setMessageHistoryFactory(micrometerMessageHistoryFactory); MicrometerExchangeEventNotifier micrometerExchangeEventNotifier = new MicrometerExchangeEventNotifier(); micrometerExchangeEventNotifier.setMeterRegistry(registry); camelContext.getManagementStrategy().addEventNotifier(micrometerExchangeEventNotifier); MicrometerRouteEventNotifier micrometerRouteEventNotifier = new MicrometerRouteEventNotifier(); micrometerRouteEventNotifier.setMeterRegistry(registry); camelContext.getManagementStrategy().addEventNotifier(micrometerRouteEventNotifier); } @Override public void afterApplicationStart(CamelContext camelContext) { } }; } Deploy the application to OpenShift. Verify if your application is deployed. Add the Service Monitor for this application so that Openshift's prometheus instance can start scraping from the / actuator/prometheus endpoint. Create the following YAML manifest for a Service monitor. In this example, the file is named as servicemonitor.yaml . Add a Service Monitor for this application. Verify that the service monitor was successfully deployed. Verify that you can see the service monitor in the list of scrape targets. In the Administrator view, navigate to Observe Targets. You can find csb-demo-monitor within the list of scrape targets. Wait about ten minutes after deploying the servicemonitor. Then navigate to the Observe Metrics in the Developer view. Select Custom query in the drop-down menu and type camel to view the Camel metrics that are exposed through the /actuator/prometheus endpoint. Note Red Hat does not offer support for installing and configuring Prometheus and Grafana on non-OCP environments.

Commands:
login --user system:admin --token=my-token --server=https://my-cluster.example.com:6443
oc -n openshift-monitoring edit configmap cluster-monitoring-config
apiVersion: v1 kind: ConfigMap metadata: name: cluster-monitoring-config namespace: openshift-monitoring data: config.yaml: | enableUserWorkload: true
oc -n openshift-user-workload-monitoring get pod Example output NAME READY STATUS RESTARTS AGE prometheus-operator-6f7b748d5b-t7nbg 2/2 Running 0 3h prometheus-user-workload-0 4/4 Running 1 3h prometheus-user-workload-1 4/4 Running 1 3h thanos-ruler-user-workload-0 3/3 Running 0 3h thanos-ruler-user-workload-1 3/3 Running 0 3h
<profiles> <profile> <id>openshift</id> <build> <plugins> <plugin> <groupId>org.eclipse.jkube</groupId> <artifactId>openshift-maven-plugin</artifactId> <version>1.13.1</version> <executions> <execution> <goals> <goal>resource</goal> <goal>build</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </profile> </profiles>
expose actuator endpoint via HTTP management.endpoints.web.exposure.include=mappings,metrics,health,shutdown,jolokia,prometheus
<dependency> <groupId>org.springframework</groupId> <artifactId>spring-context</artifactId> <version>6.1.8</version> </dependency> <dependency> <groupId>io.micrometer</groupId> <artifactId>micrometer-registry-prometheus</artifactId> <version>1.13.6</version> </dependency> <dependency> <groupId>org.jolokia</groupId> <artifactId>jolokia-core</artifactId> <version>2.1.0</version> </dependency> <dependency> <groupId>io.prometheus.jmx</groupId> <artifactId>collector</artifactId> <version>1.0.1</version> </dependency>
startDelaySecs: 5 ssl: false blacklistObjectNames: ["java.lang:*"] rules: # Context level - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>ExchangesCompleted' name: org.apache.camel.ExchangesCompleted help: Exchanges Completed type: COUNTER labels: context: USD1 type: context - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>ExchangesFailed' name: org.apache.camel.ExchangesFailed help: Exchanges Failed type: COUNTER labels: context: USD1 type: context - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>ExchangesInflight' name: org.apache.camel.ExchangesInflight help: Exchanges Inflight type: GAUGE labels: context: USD1 type: context - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>ExchangesTotal' name: org.apache.camel.ExchangesTotal help: Exchanges Total type: COUNTER labels: context: USD1 type: context - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>FailuresHandled' name: org.apache.camel.FailuresHandled help: Failures Handled labels: context: USD1 type: context type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>ExternalRedeliveries' name: org.apache.camel.ExternalRedeliveries help: External Redeliveries labels: context: USD1 type: context type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>MaxProcessingTime' name: org.apache.camel.MaxProcessingTime help: Maximum Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>MeanProcessingTime' name: org.apache.camel.MeanProcessingTime help: Mean Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>MinProcessingTime' name: org.apache.camel.MinProcessingTime help: Minimum Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>LastProcessingTime' name: org.apache.camel.LastProcessingTime help: Last Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>DeltaProcessingTime' name: org.apache.camel.DeltaProcessingTime help: Delta Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>Redeliveries' name: org.apache.camel.Redeliveries help: Redeliveries labels: context: USD1 type: context type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=context, name=([^,]+)><>TotalProcessingTime' name: org.apache.camel.TotalProcessingTime help: Total Processing Time labels: context: USD1 type: context type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=consumers, name=([^,]+)><>InflightExchanges' name: org.apache.camel.InflightExchanges help: Inflight Exchanges labels: context: USD1 type: context type: GAUGE # Route level - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>ExchangesCompleted' name: org.apache.camel.ExchangesCompleted help: Exchanges Completed type: COUNTER labels: context: USD1 route: USD2 type: routes - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>ExchangesFailed' name: org.apache.camel.ExchangesFailed help: Exchanges Failed type: COUNTER labels: context: USD1 route: USD2 type: routes - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>ExchangesInflight' name: org.apache.camel.ExchangesInflight help: Exchanges Inflight type: GAUGE labels: context: USD1 route: USD2 type: routes - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>ExchangesTotal' name: org.apache.camel.ExchangesTotal help: Exchanges Total type: COUNTER labels: context: USD1 route: USD2 type: routes - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>FailuresHandled' name: org.apache.camel.FailuresHandled help: Failures Handled labels: context: USD1 route: USD2 type: routes type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>ExternalRedeliveries' name: org.apache.camel.ExternalRedeliveries help: External Redeliveries labels: context: USD1 route: USD2 type: routes type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>MaxProcessingTime' name: org.apache.camel.MaxProcessingTime help: Maximum Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>MeanProcessingTime' name: org.apache.camel.MeanProcessingTime help: Mean Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>MinProcessingTime' name: org.apache.camel.MinProcessingTime help: Minimum Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>LastProcessingTime' name: org.apache.camel.LastProcessingTime help: Last Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>DeltaProcessingTime' name: org.apache.camel.DeltaProcessingTime help: Delta Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>Redeliveries' name: org.apache.camel.Redeliveries help: Redeliveries labels: context: USD1 route: USD2 type: routes type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>TotalProcessingTime' name: org.apache.camel.TotalProcessingTime help: Total Processing Time labels: context: USD1 route: USD2 type: routes type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=routes, name=([^,]+)><>InflightExchanges' name: org.apache.camel.InflightExchanges help: Inflight Exchanges labels: context: USD1 route: USD2 type: routes type: GAUGE # Processor level - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>ExchangesCompleted' name: org.apache.camel.ExchangesCompleted help: Exchanges Completed type: COUNTER labels: context: USD1 processor: USD2 type: processors - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>ExchangesFailed' name: org.apache.camel.ExchangesFailed help: Exchanges Failed type: COUNTER labels: context: USD1 processor: USD2 type: processors - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>ExchangesInflight' name: org.apache.camel.ExchangesInflight help: Exchanges Inflight type: GAUGE labels: context: USD1 processor: USD2 type: processors - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>ExchangesTotal' name: org.apache.camel.ExchangesTotal help: Exchanges Total type: COUNTER labels: context: USD1 processor: USD2 type: processors - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>FailuresHandled' name: org.apache.camel.FailuresHandled help: Failures Handled labels: context: USD1 processor: USD2 type: processors type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>ExternalRedeliveries' name: org.apache.camel.ExternalRedeliveries help: External Redeliveries labels: context: USD1 processor: USD2 type: processors type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>MaxProcessingTime' name: org.apache.camel.MaxProcessingTime help: Maximum Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>MeanProcessingTime' name: org.apache.camel.MeanProcessingTime help: Mean Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>MinProcessingTime' name: org.apache.camel.MinProcessingTime help: Minimum Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>LastProcessingTime' name: org.apache.camel.LastProcessingTime help: Last Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>DeltaProcessingTime' name: org.apache.camel.DeltaProcessingTime help: Delta Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>Redeliveries' name: org.apache.camel.Redeliveries help: Redeliveries labels: context: USD1 processor: USD2 type: processors type: COUNTER - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>TotalProcessingTime' name: org.apache.camel.TotalProcessingTime help: Total Processing Time labels: context: USD1 processor: USD2 type: processors type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=processors, name=([^,]+)><>InflightExchanges' name: org.apache.camel.InflightExchanges help: Inflight Exchanges labels: context: USD1 processor: USD2 type: processors type: COUNTER # Consumers - pattern: 'org.apache.camel<context=([^,]+), type=consumers, name=([^,]+)><>InflightExchanges' name: org.apache.camel.InflightExchanges help: Inflight Exchanges labels: context: USD1 consumer: USD2 type: consumers type: GAUGE # Services - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>MaxDuration' name: org.apache.camel.MaxDuration help: Maximum Duration labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>MeanDuration' name: org.apache.camel.MeanDuration help: Mean Duration labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>MinDuration' name: org.apache.camel.MinDuration help: Minimum Duration labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>TotalDuration' name: org.apache.camel.TotalDuration help: Total Duration labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>ThreadsBlocked' name: org.apache.camel.ThreadsBlocked help: Threads Blocked labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.camel<context=([^,]+), type=services, name=([^,]+)><>ThreadsInterrupted' name: org.apache.camel.ThreadsInterrupted help: Threads Interrupted labels: context: USD1 service: USD2 type: services type: GAUGE - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>NumLogicalRuntimeFaults' name: org.apache.cxf.NumLogicalRuntimeFaults help: Number of logical runtime faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>NumLogicalRuntimeFaults' name: org.apache.cxf.NumLogicalRuntimeFaults help: Number of logical runtime faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>AvgResponseTime' name: org.apache.cxf.AvgResponseTime help: Average Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>AvgResponseTime' name: org.apache.cxf.AvgResponseTime help: Average Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>NumInvocations' name: org.apache.cxf.NumInvocations help: Number of invocations type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>NumInvocations' name: org.apache.cxf.NumInvocations help: Number of invocations type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>MaxResponseTime' name: org.apache.cxf.MaxResponseTime help: Maximum Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>MaxResponseTime' name: org.apache.cxf.MaxResponseTime help: Maximum Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>MinResponseTime' name: org.apache.cxf.MinResponseTime help: Minimum Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>MinResponseTime' name: org.apache.cxf.MinResponseTime help: Minimum Response Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>TotalHandlingTime' name: org.apache.cxf.TotalHandlingTime help: Total Handling Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>TotalHandlingTime' name: org.apache.cxf.TotalHandlingTime help: Total Handling Time type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>NumRuntimeFaults' name: org.apache.cxf.NumRuntimeFaults help: Number of runtime faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>NumRuntimeFaults' name: org.apache.cxf.NumRuntimeFaults help: Number of runtime faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>NumUnCheckedApplicationFaults' name: org.apache.cxf.NumUnCheckedApplicationFaults help: Number of unchecked application faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>NumUnCheckedApplicationFaults' name: org.apache.cxf.NumUnCheckedApplicationFaults help: Number of unchecked application faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+), operation=([^,]+)><>NumCheckedApplicationFaults' name: org.apache.cxf.NumCheckedApplicationFaults help: Number of checked application faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4 operation: USD5 - pattern: 'org.apache.cxf<bus.id=([^,]+), type=([^,]+), service=([^,]+), port=([^,]+)><>NumCheckedApplicationFaults' name: org.apache.cxf.NumCheckedApplicationFaults help: Number of checked application faults type: GAUGE labels: bus.id: USD1 type: USD2 service: USD3 port: USD4
import java.io.InputStream; import io.micrometer.core.instrument.Clock; import org.apache.camel.CamelContext; import org.apache.camel.spring.boot.CamelContextConfiguration; import org.springframework.context.annotation.Bean; import org.apache.camel.component.micrometer.MicrometerConstants; import org.apache.camel.component.micrometer.eventnotifier.MicrometerExchangeEventNotifier; import org.apache.camel.component.micrometer.eventnotifier.MicrometerRouteEventNotifier; import org.apache.camel.component.micrometer.messagehistory.MicrometerMessageHistoryFactory; import org.apache.camel.component.micrometer.routepolicy.MicrometerRoutePolicyFactory;
@SpringBootApplication public class SampleCamelApplication { @Bean(name = {MicrometerConstants.METRICS_REGISTRY_NAME, "prometheusMeterRegistry"}) public PrometheusMeterRegistry prometheusMeterRegistry( PrometheusConfig prometheusConfig, CollectorRegistry collectorRegistry, Clock clock) throws MalformedObjectNameException, IOException { InputStream resource = new ClassPathResource("config/prometheus_exporter_config.yml").getInputStream(); new JmxCollector(resource).register(collectorRegistry); new BuildInfoCollector().register(collectorRegistry); return new PrometheusMeterRegistry(prometheusConfig, collectorRegistry, clock); } @Bean public CamelContextConfiguration camelContextConfiguration(@Autowired PrometheusMeterRegistry registry) { return new CamelContextConfiguration() { @Override public void beforeApplicationStart(CamelContext camelContext) { MicrometerRoutePolicyFactory micrometerRoutePolicyFactory = new MicrometerRoutePolicyFactory(); micrometerRoutePolicyFactory.setMeterRegistry(registry); camelContext.addRoutePolicyFactory(micrometerRoutePolicyFactory); MicrometerMessageHistoryFactory micrometerMessageHistoryFactory = new MicrometerMessageHistoryFactory(); micrometerMessageHistoryFactory.setMeterRegistry(registry); camelContext.setMessageHistoryFactory(micrometerMessageHistoryFactory); MicrometerExchangeEventNotifier micrometerExchangeEventNotifier = new MicrometerExchangeEventNotifier(); micrometerExchangeEventNotifier.setMeterRegistry(registry); camelContext.getManagementStrategy().addEventNotifier(micrometerExchangeEventNotifier); MicrometerRouteEventNotifier micrometerRouteEventNotifier = new MicrometerRouteEventNotifier(); micrometerRouteEventNotifier.setMeterRegistry(registry); camelContext.getManagementStrategy().addEventNotifier(micrometerRouteEventNotifier); } @Override public void afterApplicationStart(CamelContext camelContext) { } }; }
mvn -Popenshift oc:deploy
get pods -n myapp NAME READY STATUS RESTARTS AGE camel-example-spring-boot-xml-2-deploy 0/1 Completed 0 13m camel-example-spring-boot-xml-2-x78rk 1/1 Running 0 13m camel-example-spring-boot-xml-s2i-2-build 0/1 Completed 0 14m
apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: labels: k8s-app: csb-demo-monitor name: csb-demo-monitor spec: endpoints: - interval: 30s port: http scheme: http path: /actuator/prometheus selector: matchLabels: app: camel-example-spring-boot-xml
apply -f servicemonitor.yml servicemonitor.monitoring.coreos.com/csb-demo-monitor "myapp" created
get servicemonitor NAME AGE csb-demo-monitor 9m17s