Chapter 1. Web console Learn how to access and use components of the Red Hat Advanced Cluster Management for Kubernetes console from the following documentation: Console overview Search in the console Accessing your console Enabling virtual machine actions (Technology Preview) 1.1. Console overview Learn more about console components that you can use to view, manage, or customize your console. See the following image of the Navigation from the Red Hat Advanced Cluster Management for Kubernetes console, which is described in more detail later in each section. See that the navigation represents major production function. 1.1.1. Console components Home Infrastructure Applications Governance Credentials 1.1.2. Home From the Red Hat Advanced Cluster Management for Kubernetes Home page in the All clusters view, you can access more information and you can search across the product. Click Welcome for more introductory information about each product function. 1.1.2.1. Overview Click Overview to see summary information, or to access clickable Cluster percentage values for policy violations, and more. From the Overview page, you can view the following information: Cluster and node counts across all clusters and for each provider Cluster status Cluster compliance Pod status Cluster add-ons You can also access all APIs from the integrated console. From the local-cluster view, go to Home > API Explorer to explore API groups. You can also use the Fleet view switch from the Overview page header to filter the page data by using cluster labels, and display metrics. The following information is displayed from the Fleet view switch: Number of clusters Application types Number of enabled policies on your cluster Cluster version Total number of nodes on your cluster Number of worker cores The following information from Red Hat Insights is displayed: Cluster recommendations Number of risk predictions Cluster health which includes the status and violations A view of your resources based on your custom query. If observability is enabled, alert and failing operator metrics from across your fleet are also displayed. To learn about Search, see Search in the console . 1.1.2.2. Command line tools From the Home page, you can access Command Line Interface (CLI) downloads by using the following steps: Click the ? icon in the toolbar of the console. Click Command Line Tools from the drop-down menu. Find the Advanced Cluster Management header to find a list of tools that are available for Red Hat Advanced Cluster Management, which is specified with the operating system and architecture. Select the appropriate binary file to download and use on your local system. 1.1.3. Infrastructure From Clusters , you can create new clusters or import existing clusters. From Automation , you can create an Ansible template. For more information about managing clusters, see The multicluster engine operator cluster lifecycle overview . Additionally, see specific information on these cluster types at Configuring Ansible Automation Platform tasks to run on managed clusters . 1.1.4. Applications Create an application and edit a .yaml file. Access an overview or more advanced information about each application. For more information about application resources, see Managing applications . 1.1.5. Governance Create and edit a .yaml file to create a policy. Use the Governance dashboard to manage policies and policy controllers. For more information, see Governance . 1.1.6. Credentials The credential stores the access information for a cloud provider. Each provider account requires its own credential, as does each domain on a single provider. Review your credentials or add a credential. See Managing credentials overview for more specific information about providers and credentials. 1.2. Search in the console For Red Hat Advanced Cluster Management for Kubernetes, search provides visibility into your Kubernetes resources across all of your clusters. Search also indexes the Kubernetes resources and the relationships to other resources. Search components Search customization and configurations Search operations and data types 1.2.1. Search components The search architecture is composed of the following components: Table 1.1. Search component table Component name Metrics Metric type Description search-collector Watches the Kubernetes resources, collects the resource metadata, computes relationships for resources across all of your managed clusters, and sends the collected data to the search-indexer . The search-collector on your managed cluster runs as a pod named, klusterlet-addon-search . search-indexer Receives resource metadata from the collectors and writes to PostgreSQL database. The search-indexer also watches resources in the hub cluster to keep track of active managed clusters. search_indexer_request_duration Histogram Time (seconds) the search indexer takes to process a request (from managed cluster). search_indexer_request_size Histogram Total changes (add, update, delete) in the search indexer request (from managed cluster). search_indexer_request_count Counter Total requests received by the search indexer (from managed clusters). search_indexer_requests_in_flight Gauge Total requests the search indexer is processing at a given time. search-api Provides access to all cluster data in the search-indexer through GraphQL and enforces role-based access control (RBAC). search_api_requests Histogram Histogram of HTTP requests duration in seconds. search_dbquery_duration_seconds Histogram Latency of database requests in seconds. search_api_db_connection_failed_total Counter The total number of database connection attempts that failed. search-postgres Stores collected data from all managed clusters in an instance of the PostgreSQL database. Search is configured by default on the hub cluster. When you provision or manually import a managed cluster, the klusterlet-addon-search is enabled. If you want to disable search on your managed cluster, see Modifying the klusterlet add-ons settings of your cluster for more information. 1.2.2. Search customization and configurations You can modify the default values in the search-v2-operator custom resource. To view details of the custom resource, run the following command: oc get search search-v2-operator -o yaml The search operator watches the search-v2-operator custom resource, reconciles the changes and updates active pods. View the following descriptions of the configurations: PostgreSQL database storage: When you install Red Hat Advanced Cluster Management, the PostgreSQL database is configured to save the PostgreSQL data in an empty directory ( emptyDir ) volume. If the empty directory size is limited, you can save the PostgreSQL data on a Persistent Volume Claim (PVC) to improve search performance. You can select a storageclass from your Red Hat Advanced Cluster Management hub cluster to back up your search data. For example, if you select the gp2 storageclass your configuration might resemble the following example: apiVersion: search.open-cluster-management.io/v1alpha1 kind: Search metadata: name: search-v2-operator namespace: open-cluster-management labels: cluster.open-cluster-management.io/backup: "" spec: dbStorage: size: 10Gi storageClassName: gp2 This configuration creates a PVC named gp2-search and is mounted to the search-postgres pod. By default, the storage size is 10Gi . You can modify the storage size. For example, 20Gi might be sufficient for about 200 managed clusters. Optimize cost by tuning the pod memory or CPU requirements, replica count, and update log levels for any of the four search pods ( indexer , database , queryapi , or collector pod). Update the deployment section of the search-v2-operator custom resource. There are four deployments managed by the search-v2-operator , which can be updated individually. Your search-v2-operator custom resource might resemble the following file: apiVersion: search.open-cluster-management.io/v1alpha1 kind: Search metadata: name: search-v2-operator namespace: open-cluster-management spec: deployments: collector: resources: 1 limits: cpu: 500m memory: 128Mi requests: cpu: 250m memory: 64Mi indexer: replicaCount: 3 database: 2 envVar: - name: POSTGRESQL_EFFECTIVE_CACHE_SIZE value: 1024MB - name: POSTGRESQL_SHARED_BUFFERS value: 512MB - name: WORK_MEM value: 128MB queryapi: arguments: 3 - -v=3 1 You can apply resources to an indexer , database , queryapi , or collector pod. 2 You can add multiple environment variables in the envVar section to specify a value for each variable that you name. 3 You can control the log level verbosity for any of the four pods by adding the - -v=3 argument. See the following example where memory resources are applied to the indexer pod: indexer: resources: limits: memory: 5Gi requests: memory: 1Gi You can define the node placement for search pods. You can update the Placement resource of search pods by using the nodeSelector parameter, or the tolerations parameter. View the following example configuration: spec: dbStorage: size: 10Gi deployments: collector: {} database: {} indexer: {} queryapi: {} nodeSelector: node-role.kubernetes.io/infra: "" tolerations: - effect: NoSchedule key: node-role.kubernetes.io/infra operator: Exists Specify your search query by selecting the Advanced search drop-down button to filter the Column , Operator , and Value options or add a search constraint. 1.2.3. Search operations and data types Specify your search query by using search operations as conditions. Characters such as >, >=, <, <=, != are supported. See the following search operation table: Table 1.2. Search operation table Default operation Data type Description = string, number This is the default operation. ! or != string, number This represents the NOT operation, which means to exclude from the search results. <, ⇐, >, >= number > date Dates matching the last hour, day, week, month, and year. * string Partial string match. 1.2.4. Additional resources For instruction about how to manage search, see Managing search . For more topics about the Red Hat Advanced Cluster Management for Kubernetes console, see Web console . 1.2.5. Managing search Use search to query resource data from your clusters. Required access: Cluster administrator Continue reading the following topics: Creating search configurable collection Customizing the search console Querying in the console Updating klusterlet-addon-search deployments on managed clusters 1.2.5.1. Creating search configurable collection To define which Kubernetes resources get collected from the cluster, create the search-collector-config config map. Complete the following steps: Run the following command to create the search-collector-config config map: oc apply -f <your-search-collector-config>.yaml List the resources in the allow ( data.AllowedResources ) and deny list ( data.DeniedResources ) sections within the config map. Your config map might resemble the following YAML file: apiVersion: v1 kind: ConfigMap metadata: name: search-collector-config namespace: <namespace where search-collector add-on is deployed> data: AllowedResources: |- 1 - apiGroups: - "*" resources: - services - pods - apiGroups: - admission.k8s.io - authentication.k8s.io resources: - "*" DeniedResources: |- 2 - apiGroups: - "*" resources: - secrets - apiGroups: - admission.k8s.io resources: - policies - iampolicies - certificatepolicies 1 The config map example displays services and pods to be collected from all apiGroups , while allowing all resources to be collected from the admission.k8s.io and authentication.k8s.io apiGroups . 2 The config map example also prevents the central collection of secrets from all apiGroups while preventing the collection of policies , iampolicies , and certificatepolicies from the apiGroup admission.k8s.io . Note: If you do not provide a config map, all resources are collected by default. If you only provide AllowedResources , all resources not listed in AllowedResources are automatically excluded. Resources listed in AllowedResources and DeniedResources at the same time are also excluded. 1.2.5.2. Customizing the search console Customize your search results and limits. Complete the following tasks to perform the customization: Customize the search result limit from the OpenShift Container Platform console. Update the console-mce-config in the multicluster-engine namespace. These settings apply to all users and might affect performance. View the following performance parameter descriptions: SAVED_SEARCH_LIMIT - The maximum amount of saved searches for each user. By default, there is a limit of ten saved searches for each user. The default value is 10 . To update the limit, add the following key value to the console-config config map: SAVED_SEARCH_LIMIT: x . SEARCH_RESULT_LIMIT - The maximum amount of search results displayed in the console. Default value is 1000 . To remove this limit set to -1 . SEARCH_AUTOCOMPLETE_LIMIT - The maximum number of suggestions retrieved for the search bar typeahead. Default value is 10,000 . To remove this limit set to -1 . Run the following patch command from the OpenShift Container Platform console to change the search result to 100 items: oc patch configmap console-mce-config -n multicluster-engine --type merge -p '{"data":{"SEARCH_RESULT_LIMIT":"100"}}' To add, edit, or remove suggested searches, create a config map named console-search-config and configure the suggestedSearches section. Suggested searches that are listed are also displayed from the console. It is required to have an id, name, and searchText for each search object. View the following config map example: kind: ConfigMap apiVersion: v1 metadata: name: console-search-config namespace: <acm-namespace> 1 data: suggestedSearches: |- [ { "id": "search.suggested.workloads.name", "name": "Workloads", "description": "Show workloads running on your fleet", "searchText": "kind:DaemonSet,Deployment,Job,StatefulSet,ReplicaSet" }, { "id": "search.suggested.unhealthy.name", "name": "Unhealthy pods", "description": "Show pods with unhealthy status", "searchText": "kind:Pod status:Pending,Error,Failed,Terminating,ImagePullBackOff,CrashLoopBackOff,RunContainerError,ContainerCreating" }, { "id": "search.suggested.createdLastHour.name", "name": "Created last hour", "description": "Show resources created within the last hour", "searchText": "created:hour" }, { "id": "search.suggested.virtualmachines.name", "name": "Virtual Machines", "description": "Show virtual machine resources", "searchText": "kind:VirtualMachine" } ] 1 Add the namespace where search is enabled. 1.2.5.3. Querying in the console You can type any text value in the Search box and results include anything with that value from any property, such as a name or namespace. Queries that contain an empty space are not supported. For more specific search results, include the property selector in your search. You can combine related values for the property for a more precise scope of your search. For example, search for cluster:dev red to receive results that match the string "red" in the dev cluster. Complete the following steps to make queries with search: Click Search in the navigation menu. Type a word in the Search box , then Search finds your resources that contain that value. As you search for resources, you receive other resources that are related to your original search result, which help you visualize how the resources interact with other resources in the system. Search returns and lists each cluster with the resource that you search. For resources in the hub cluster, the cluster name is displayed as local-cluster . Your search results are grouped by kind , and each resource kind is grouped in a table. Your search options depend on your cluster objects. You can refine your results with specific labels. Search is case-sensitive when you query labels. See the following examples that you can select for filtering: name , namespace , status , and other resource fields. Auto-complete provides suggestions to refine your search. See the following example: Search for a single field, such as kind:pod to find all pod resources. Search for multiple fields, such as kind:pod namespace:default to find the pods in the default namespace. Notes: When you search for more than one property selector with multiple values, the search returns either of the values that were queried. View the following examples: When you search for kind:Pod name:a , any pod named a is returned. When you search for kind:Pod name:a,b , any pod named a or b are returned. Search for kind:pod status:!Running to find all pod resources where the status is not Running . Search for kind:pod restarts:>1 to find all pods that restarted at least twice. If you want to save your search, click the Save search icon. To download your search results, select the Export as CSV button. 1.2.5.4. Updating klusterlet-addon-search deployments on managed clusters To collect the Kubernetes objects from the managed clusters, the klusterlet-addon-search pod is run on all the managed clusters where search is enabled. This deployment is run in the open-cluster-management-agent-addon namespace. A managed cluster with a high number of resources might require more memory for the klusterlet-addon-search deployment to function. Resource requirements for the klusterlet-addon-search pod in a managed cluster can be specified in the ManagedClusterAddon custom resource in your Red Hat Advanced Cluster Management hub cluster. There is a namespace for each managed cluster with the managed cluster name. Complete the following steps: Edit the ManagedClusterAddon custom resource from the namespace matching the managed cluster name. Run the following command to update the resource requirement in xyz managed cluster: oc edit managedclusteraddon search-collector -n xyz Append the resource requirements as annotations. View the following example: apiVersion: addon.open-cluster-management.io/v1alpha1 kind: ManagedClusterAddOn metadata: annotations: addon.open-cluster-management.io/search_memory_limit: 2048Mi addon.open-cluster-management.io/search_memory_request: 512Mi The annotation overrides the resource requirements on the managed clusters and automatically restarts the pod with new resource requirements. Note: You can discover all resources defined in your managed cluster by using the API Explorer in the console. Alternatively, you can discover all resources by running the following command: oc api-resources 1.2.5.5. Additional resources See multicluster global hub for more details. See Observing environments introduction . 1.3. Accessing your console The Red Hat Advanced Cluster Management for Kubernetes web console is integrated with the Red Hat OpenShift Container Platform web console as a console plug-in. You can access Red Hat Advanced Cluster Management within the OpenShift Container Platform console from the cluster switcher by selecting All Clusters . The cluster switcher is a drop-down menu that initially displays local-cluster . Select local-cluster when you want to use OpenShift Container Platform console features on the cluster where you installed Red Hat Advanced Cluster Management. Select All Clusters when you want to use Red Hat Advanced Cluster Management features to manage your fleet of clusters. If the cluster switcher is not present, the required console plug-ins might not be enabled. For new installations, the console plug-ins are enabled by default. If you upgraded from a version of Red Hat Advanced Cluster Management and want to enable the plug-ins, or if you want to disable the plug-ins, complete the following steps: To disable the plug-in, be sure you are in the Administrator perspective in the OpenShift Container Platform console. Find Administration in the navigation and click Cluster Settings , then click the Configuration tab. From the list of Configuration resources , click the Console resource with the operator.openshift.io API group, which contains cluster-wide configuration for the web console. Select the Console plug-ins tab. Both the acm and mce plug-ins are listed. Modify plug-in status from the table. In a few moments, you are prompted to refresh the console. Note: To enable and disable the console, see MultiClusterHub advanced for information. To learn more about the Red Hat Advanced Cluster Management for Kubernetes console, see Console overview . 1.4. Enabling virtual machine actions (Technology Preview) To view VirtualMachine resources across all the clusters that Red Hat Advanced Cluster Management for Kubernetes manages, use the Search feature to list and filter the VirtualMachine resources created with the Red Hat OpenShift Virtualization. You can also enable the following actions from the Red Hat Advanced Cluster Management console on your VirtualMachine resources: Start Stop Restart Pause Unpause Required access: Cluster administrator 1.4.1. Prerequisites Confirm that the ManagedServiceAccount add-on is enabled. See ManagedServiceAccount add-on . 1.4.2. Enabling virtual machine actions for Red Hat Advanced Cluster Management You can enable the virtual machine actions for Red Hat Advanced Cluster Management by updating the console config map. Complete the following steps: To update the Red Hat Advanced Cluster Management console config map for enabling virtual machine actions, run the following command: oc patch configmap console-mce-config -n multicluster-engine -p '{"data": {"VIRTUAL_MACHINE_ACTIONS": "enabled"}}' To configure Red Hat Advanced Cluster Management to process the actions, create and configure a ManagedServiceAccount resource for each managed cluster. Save the following YAML file: apiVersion: authentication.open-cluster-management.io/v1beta1 kind: ManagedServiceAccount metadata: name: vm-actor labels: app: search spec: rotation: {} --- apiVersion: rbac.open-cluster-management.io/v1alpha1 kind: ClusterPermission metadata: name: vm-actions labels: app: search spec: clusterRole: rules: - apiGroups: - subresources.kubevirt.io resources: - virtualmachines/start - virtualmachines/stop - virtualmachines/restart - virtualmachineinstances/pause - virtualmachineinstances/unpause verbs: - update clusterRoleBinding: subject: kind: ServiceAccount name: vm-actor namespace: open-cluster-management-agent-addon Note: You must repeat this step for each new managed cluster. Apply the ManagedServiceAccount resource to your hub cluster by running the following command: oc apply -n <MANAGED_CLUSTER> -f /path/to/file The virtual machine actions are enabled for Red Hat Advanced Cluster Management. 1.4.3. Disabling virtual machine actions To disable virtual machine actions for Red Hat Advanced Cluster Management, run the following command: oc patch configmap console-mce-config -n multicluster-engine -p '{"data": {"VIRTUAL_MACHINE_ACTIONS": "disabled"}}' The virtual machine actions are disabled for Red Hat Advanced Cluster Management. 1.4.4. Deleting ManagedServiceAccounts and ClusterPermissions resources To delete ManagedServiceAccounts and ClusterPermissions resources that use virtual machine actions, complete the following steps: To delete the resources, run the following command: oc delete managedserviceaccount,clusterpermission -A -l app=search You might receive the following output: managedserviceaccount.authentication.open-cluster-management.io "vm-actor" deleted managedserviceaccount.authentication.open-cluster-management.io "vm-actor" deleted clusterpermission.rbac.open-cluster-management.io "vm-actions" deleted clusterpermission.rbac.open-cluster-management.io "vm-actions" deleted To confirm that the clean up is complete, run the following command: oc get managedserviceaccount,clusterpermission -A -l app=search When the resources are deleted successfully, you receive the following message: "No resources found" The ManagedServiceAccounts and ClusterPermissions resources are deleted.

Commands:
get search search-v2-operator -o yaml
apiVersion: search.open-cluster-management.io/v1alpha1 kind: Search metadata: name: search-v2-operator namespace: open-cluster-management labels: cluster.open-cluster-management.io/backup: "" spec: dbStorage: size: 10Gi storageClassName: gp2
apiVersion: search.open-cluster-management.io/v1alpha1 kind: Search metadata: name: search-v2-operator namespace: open-cluster-management spec: deployments: collector: resources: 1 limits: cpu: 500m memory: 128Mi requests: cpu: 250m memory: 64Mi indexer: replicaCount: 3 database: 2 envVar: - name: POSTGRESQL_EFFECTIVE_CACHE_SIZE value: 1024MB - name: POSTGRESQL_SHARED_BUFFERS value: 512MB - name: WORK_MEM value: 128MB queryapi: arguments: 3 - -v=3
indexer: resources: limits: memory: 5Gi requests: memory: 1Gi
spec: dbStorage: size: 10Gi deployments: collector: {} database: {} indexer: {} queryapi: {} nodeSelector: node-role.kubernetes.io/infra: "" tolerations: - effect: NoSchedule key: node-role.kubernetes.io/infra operator: Exists
apply -f <your-search-collector-config>.yaml
apiVersion: v1 kind: ConfigMap metadata: name: search-collector-config namespace: <namespace where search-collector add-on is deployed> data: AllowedResources: |- 1 - apiGroups: - "*" resources: - services - pods - apiGroups: - admission.k8s.io - authentication.k8s.io resources: - "*" DeniedResources: |- 2 - apiGroups: - "*" resources: - secrets - apiGroups: - admission.k8s.io resources: - policies - iampolicies - certificatepolicies
patch configmap console-mce-config -n multicluster-engine --type merge -p '{"data":{"SEARCH_RESULT_LIMIT":"100"}}'
kind: ConfigMap apiVersion: v1 metadata: name: console-search-config namespace: <acm-namespace> 1 data: suggestedSearches: |- [ { "id": "search.suggested.workloads.name", "name": "Workloads", "description": "Show workloads running on your fleet", "searchText": "kind:DaemonSet,Deployment,Job,StatefulSet,ReplicaSet" }, { "id": "search.suggested.unhealthy.name", "name": "Unhealthy pods", "description": "Show pods with unhealthy status", "searchText": "kind:Pod status:Pending,Error,Failed,Terminating,ImagePullBackOff,CrashLoopBackOff,RunContainerError,ContainerCreating" }, { "id": "search.suggested.createdLastHour.name", "name": "Created last hour", "description": "Show resources created within the last hour", "searchText": "created:hour" }, { "id": "search.suggested.virtualmachines.name", "name": "Virtual Machines", "description": "Show virtual machine resources", "searchText": "kind:VirtualMachine" } ]
edit managedclusteraddon search-collector -n xyz
apiVersion: addon.open-cluster-management.io/v1alpha1 kind: ManagedClusterAddOn metadata: annotations: addon.open-cluster-management.io/search_memory_limit: 2048Mi addon.open-cluster-management.io/search_memory_request: 512Mi
patch configmap console-mce-config -n multicluster-engine -p '{"data": {"VIRTUAL_MACHINE_ACTIONS": "enabled"}}'
apiVersion: authentication.open-cluster-management.io/v1beta1 kind: ManagedServiceAccount metadata: name: vm-actor labels: app: search spec: rotation: {} --- apiVersion: rbac.open-cluster-management.io/v1alpha1 kind: ClusterPermission metadata: name: vm-actions labels: app: search spec: clusterRole: rules: - apiGroups: - subresources.kubevirt.io resources: - virtualmachines/start - virtualmachines/stop - virtualmachines/restart - virtualmachineinstances/pause - virtualmachineinstances/unpause verbs: - update clusterRoleBinding: subject: kind: ServiceAccount name: vm-actor namespace: open-cluster-management-agent-addon
apply -n <MANAGED_CLUSTER> -f /path/to/file
patch configmap console-mce-config -n multicluster-engine -p '{"data": {"VIRTUAL_MACHINE_ACTIONS": "disabled"}}'
delete managedserviceaccount,clusterpermission -A -l app=search
managedserviceaccount.authentication.open-cluster-management.io "vm-actor" deleted managedserviceaccount.authentication.open-cluster-management.io "vm-actor" deleted clusterpermission.rbac.open-cluster-management.io "vm-actions" deleted clusterpermission.rbac.open-cluster-management.io "vm-actions" deleted
get managedserviceaccount,clusterpermission -A -l app=search
"No resources found"