Chapter 3. Monitoring and logging with Azure Kubernetes Services (AKS) in Red Hat Developer Hub Monitoring and logging are integral aspects of managing and maintaining Azure Kubernetes Services (AKS) in Red Hat Developer Hub. With features like Managed Prometheus Monitoring and Azure Monitor integration, administrators can efficiently monitor resource utilization, diagnose issues, and ensure the reliability of their containerized workloads. 3.1. Enabling Azure Monitor metrics To enable managed Prometheus monitoring, use the -enable-azure-monitor-metrics option within either the az aks create or az aks update command, depending on whether you're creating a new cluster or updating an existing one, such as: az aks create/update --resource-group <your-ResourceGroup> --name <your-Cluster> --enable-azure-monitor-metrics The command installs the metrics add-on, which gathers Prometheus metrics . Using the command, you can enable monitoring of Azure resources through both native Azure Monitor metrics. You can also view the results in the portal under Monitoring Insights . For more information, see Monitor Azure resources with Azure Monitor . Furthermore, metrics from both the Managed Prometheus service and Azure Monitor can be accessed through Azure Managed Grafana service. For more information, see Link a Grafana workspace section. By default, Prometheus uses the minimum ingesting profile, which optimizes ingestion volume and sets default configurations for scrape frequency, targets, and metrics collected. The default settings can be customized through custom configuration. Azure offers various methods, including using different ConfigMaps, to provide scrape configuration and other metric add-on settings. For more information about default configuration, see Default Prometheus metrics configuration in Azure Monitor and Customize scraping of Prometheus metrics in Azure Monitor managed service for Prometheus documentation. 3.2. Configuring annotations for monitoring You can configure the annotations for monitoring Red Hat Developer Hub specific metrics in both Helm deployment and Operator-backed deployment. Helm deployment To annotate the backstage pod for monitoring, update your values.yaml file as follows: upstream: backstage: # --- TRUNCATED --- podAnnotations: prometheus.io/scrape: 'true' prometheus.io/path: '/metrics' prometheus.io/port: '9464' prometheus.io/scheme: 'http' Operator-backed deployment Procedure As an administrator of the operator, edit the default configuration to add Prometheus annotations as follows: # Update OPERATOR_NS accordingly OPERATOR_NS=rhdh-operator kubectl edit configmap backstage-default-config -n "USD{OPERATOR_NS}" Find the deployment.yaml key in the ConfigMap and add the annotations to the spec.template.metadata.annotations field as follows: deployment.yaml: |- apiVersion: apps/v1 kind: Deployment # --- truncated --- spec: template: # --- truncated --- metadata: labels: rhdh.redhat.com/app: # placeholder for 'backstage-<cr-name>' # --- truncated --- annotations: prometheus.io/scrape: 'true' prometheus.io/path: '/metrics' prometheus.io/port: '9464' prometheus.io/scheme: 'http' # --- truncated --- Save your changes. Verification To verify if the scraping works, navigate to the corresponding Azure Monitor Workspace and view the metrics under Monitoring Metrics . 3.3. Viewing logs with Azure Kubernetes Services (AKS) You can access live data logs generated by Kubernetes objects and collect log data in Container Insights within AKS. Prerequisites You have deployed Developer Hub on AKS. For more information, see Installing Red Hat Developer Hub on Microsoft Azure Kubernetes Service . Procedure View live logs from your Developer Hub instance Navigate to the Azure Portal. Search for the resource group <your-ResourceGroup> and locate your AKS cluster <your-Cluster> . Select Kubernetes resources Workloads from the menu. Select the <your-rhdh-cr>-developer-hub (in case of Helm Chart installation) or <your-rhdh-cr>-backstage (in case of Operator-backed installation) deployment. Click Live Logs in the left menu. Select the pod. Note There must be only single pod. Live log data is collected and displayed. View real-time log data from the Container Engine Navigate to the Azure Portal. Search for the resource group <your-ResourceGroup> and locate your AKS cluster <your-Cluster> . Select Monitoring Insights from the menu. Go to the Containers tab. Find the backend-backstage container and click it to view real-time log data as it's generated by the Container Engine.

Commands:
az aks create/update --resource-group <your-ResourceGroup> --name <your-Cluster> --enable-azure-monitor-metrics
upstream: backstage: # --- TRUNCATED --- podAnnotations: prometheus.io/scrape: 'true' prometheus.io/path: '/metrics' prometheus.io/port: '9464' prometheus.io/scheme: 'http'
Update OPERATOR_NS accordingly OPERATOR_NS=rhdh-operator edit configmap backstage-default-config -n "USD{OPERATOR_NS}"
deployment.yaml: |- apiVersion: apps/v1 kind: Deployment # --- truncated --- spec: template: # --- truncated --- metadata: labels: rhdh.redhat.com/app: # placeholder for 'backstage-<cr-name>' # --- truncated --- annotations: prometheus.io/scrape: 'true' prometheus.io/path: '/metrics' prometheus.io/port: '9464' prometheus.io/scheme: 'http' # --- truncated ---