vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Worker node flavors for Montreal.

 Name Memory Network speed Cores Type OS Primary storage Secondary storage 

 b3c.16x64 64GB 1000Mbps 16 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 b3c.16x64.300gb 64GB 1000Mbps 16 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 300GB SSD 
 b3c.32x128 128GB 1000Mbps 32 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 b3c.4x16 16GB 1000Mbps 4 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 b3c.56x242 242GB 1000Mbps 56 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 b3c.8x32 32GB 1000Mbps 8 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 c3c.16x16 16GB 1000Mbps 16 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 c3c.16x32 32GB 1000Mbps 16 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 c3c.32x32 32GB 1000Mbps 32 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 c3c.32x64 64GB 1000Mbps 32 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 m3c.4x32 32GB 1000Mbps 4 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 mb4c.20x192 192GB 10000Mbps 20 Physical REDHAT_7_64 (default), REDHAT_8_64 2000GB HDD 960GB SSD
VMware Aria Operations Manager design 

The VMware Aria® Operations™ Manager (formerly known as VMware vRealize® Operations Manager™) Analytics Cluster contains the nodes that analyze and store data from the monitored components. In this deployment, four nodes are deployed and two VMware NSX® Load Balancers. This size allows monitoring of up to 30,000 VMs and 9,000,000 metrics to be collected.

The 4-node analytics cluster consists of the following components:



* Primary node – The primary node is the initial node in a VMware Aria Operations cluster. In a large environment, this node manages all the other nodes.
* Primary node replica – This node enables high availability of the primary node.
* Data nodes – The data node enables scale out of VMware Aria Operations in larger environments, two are deployed in this design.



Additionally, the design uses Remote Collector nodes, which act as a proxy or relay server to collect data only and forward collected data to the primary and data nodes. Data nodes and Remote Collectors can be added to scale up depending on environment size. The placement of VMware Aria Operations components onto VLANs or VXLANs is shown in the following diagram.

Zoom

![Operations Manager network diagram](https://cloud.ibm.com/docs-content/v1/content/5bdb37c0149145723c5437e8176651928565e722/vmwaresolutions/images/opsmgmt-vropsnw.svg)

Figure 1. Operations Manager networking



* Primary node, primary node replica, and data nodes are deployed on the tools subnet by using IBM Cloud® Portable IP addresses. This deployment facilitate communication to all components that are addressed out of the IBM Cloud RFC1918 address space. These components include vSphere hosts, vCenter, Platform Services Controller (PSC), NSX Manager, and NSX Controllers™. An NSX Load Balancer is used along with a VIP for HA.
* Customer workloads use IP addressing from the BYOIP address space so this design uses Remote Collectors that are hosted in a VXLAN.
To view a list of VPC worker node flavors, see [VPC flavors](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-flavors&interface=ui)
5. Verify that the creation of the cluster was requested. It can take a few minutes for the worker node machines to be ordered, and for the cluster to be set up and provisioned in your account.

ibmcloud oc cluster ls

When the provisioning of your Red Hat OpenShift master is completed, the state of your cluster changes to normal. After the Red Hat OpenShift master is ready, your worker nodes are set up.

NAME ID State Created Workers Zone Version Resource Group Name Provider
mycluster aaf97a8843a29941b49a598f516da72101 normal 20170201162433 3 Dallas 4.11.42_1544_openshift Default vpc-gen2

Is your cluster not in a normal state? Check out the [Debugging clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_clusters) guide for help. For example, if your cluster is provisioned in an account that is protected by a firewall gateway appliance, you must [configure your firewall settings to allow outgoing traffic to the appropriate ports and IP addresses](https://cloud.ibm.com/docs/openshift?topic=openshift-firewallfirewall_outbound).
6. Check the status of the worker nodes.

ibmcloud oc worker ls --cluster <cluster_name_or_ID>

When the worker nodes are ready, the worker node State changes to normal and the Status changes to Ready. When the node Status changes to Ready, you can access the cluster. Note that even if the cluster is ready, some parts of the cluster that are used by other services, such as Ingress secrets or registry image pull secrets, might still be in process.

ID Public IP Private IP Flavor State Status Zone Version
: 4:P:master1:master:worker:master
* hana_prd_site 	: SiteA
* hana_prd_srmode 	: syncmem
* hana_prd_sync_state 	: PRIM
* hana_prd_version 	: 2.00.070.00.1679989823
* hana_prd_vhost 	: cl-prd-1
* lpa_prd_lpt 	: 1682692638
* master-SAPHana_PRD_00 	: 150
* Node: cl-prd-2 (2):
* hana_prd_clone_state 	: DEMOTED
* hana_prd_op_mode 	: logreplay
* hana_prd_remoteHost 	: cl-prd-1
* hana_prd_roles 	: 4:S:master1:master:worker:master
* hana_prd_site 	: SiteB
* hana_prd_srmode 	: syncmem
* hana_prd_sync_state 	: SOK
* hana_prd_version 	: 2.00.070.00.1679989823
* hana_prd_vhost 	: cl-prd-2
* lpa_prd_lpt 	: 30
* master-SAPHana_PRD_00 	: 100

Migration Summary:

Tickets:

PCSD Status:
cl-prd-1: Online
cl-prd-2: Online

Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled

Run the following command to check the defined constraints.

Sample output:

 pcs constraint --full
Location Constraints:
Resource: group_dev_non_prod
Disabled on:
Node: cl-prd-1 (score:-INFINITY) (resource-discovery=never) (id:loc-dev-avoid-cl-prd-1)
If you want to permit egress from your allowlist-protected services to your cluster, you must add your worker nodes' private IP addresses or your cluster's VPC subnet CIDRs in your service's allowlist. Note that because worker nodes in VPC clusters have only private IP addresses, connections into the VPC cluster worker nodes can only originate from systems that are connected to your IBM Cloud private network.

Before you begin



1. [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster)
2. Install the infrastructure-service CLI plug-in. The prefix for running VPC infrastructure commands is ibmcloud is.

ibmcloud plugin install infrastructure-service





 Allowing ingress from a cluster to another service 

To permit ingress from your cluster to another service, modify that service's allowlist or your on-premises allowlist.



1. Get the Worker Zones and VPCs that your cluster is created in.

ibmcloud ks cluster get -c <cluster>

Example output

...
Worker Zones: us-south-1, us-south-2, us-south-3
Ingress Subdomain: vpc-prod.us-south.containers.appdomain.cloud
Ingress Secret: vpc-prod
Creator: -
Public Service Endpoint URL: https://c2.us-south.containers.cloud.ibm.com:20267
Private Service Endpoint URL: https://c2.private.us-south.containers.cloud.ibm.com:20267
Pull Secrets: enabled in the default namespace
VPCs: ff537d43-a5a4-4b65-9627-17eddfa5237b
...
2.
ibmcloud oc zone add vpc-gen2 --zone <zone> --cluster <cluster_name_or_ID> --worker-pool <pool_name> --subnet-id <VPC_SUBNET_ID>





 Deploying an app with the Red Hat OpenShift service catalog 

From the Red Hat OpenShift console, you can deploy one of the built-in service catalog apps and expose the app with a route.



1. From the cluster details page, click OpenShift web console.
2. From the side navigation menu in the Administrator perspective, click Home > Projects and then click Create Project. Enter a name for your project, and click Create. Now that your project is created by the administrator, you can switch to the Developer perspective to create an app.
3. From the side navigation menu, select Developer from the perspective switcher.
4. Click +Add. From the Add pane menu bar, make sure that the Project is the one that you just created.
5. Click From Catalog. The Developer Catalog opens in the pane.
6. From the navigation menu in the pane, click Languages > JavaScript.
7. Click Node.js, and then click Create Application. Note that you might need to click Clear All Filters to display the Node.js option. After you select Node.js, the Create Source-to-Image Application pane opens.
8. In the Git section, click Try Sample.
9. Scroll to confirm that Deployment and Create a route to the application are selected, and then click Create.
10. Wait a few minutes for the pods to deploy. To check the status of the pods, from the Topology pane, click your nodejs app and review its sidebar. You must see that the nodejs build is complete, and that the nodejs pod is in a Running state to continue.
11. When the deployment is complete, click the route location URL, which has a format similar to the following.
The user makes a request to port 30001 of the NLB containing the port range. This request is directed to the VPC NLB service that directs the request to the Nodeport service in the cluster that is also listening on port 30001, which in this case is for Deployment 2. The Nodeport service then directs the request to the target port of the selected pods of Deployment 2.

Zoom

![VPC NLB that uses port range.](https://cloud.ibm.com/docs-content/v1/content/4d497aff9e27b92fbbc2ee4e343a9ec4549cb1d7/containers//images/nlb-port-range.svg)

Figure 3. VPC NLB with port range

Create an NLB that uses a port range by using the following example. The selector and backend pods must be associated with the port range load balancer service so that health checks return success and data is delivered to the ports in the port range. To use port range, you must create additional NodePort services that have port values in the range that is defined by the load balancer service.



1. Save the following example LoadBalancer configuration as a file called loadbalancer.yaml.

apiVersion: v1
kind: Service
metadata:
annotations:
service.kubernetes.io/ibm-load-balancer-cloud-provider-enable-features: nlb
service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-port-range: 30000-30010
name: nlb-port-range
spec:
externalTrafficPolicy: Cluster
ports:
- port: 30000 Must match min from the port range
protocol: TCP
nodePort: 30011 Can be port in range or not
targetPort: 8080
selector:
app: echo-server Must be valid for health checks to work
type: LoadBalancer
Show more
2. Create the service.

kubectl apply -f loadbalancer.yaml
3.
The public network interface for worker nodes is protected by [predefined Calico network policy settings](https://cloud.ibm.com/docs/openshift?topic=openshift-network_policiesdefault_policy) that are configured on every worker node during cluster creation. By default, all outbound network traffic is allowed for all worker nodes. Inbound network traffic is blocked except for a few ports. These ports are opened so that IBM can monitor network traffic and automatically install security updates for the Kubernetes master, and so that connections can be established to NodePort, LoadBalancer, and Ingress services.

Because the default Calico network policies allow inbound public traffic to these services, you can create Calico policies to instead block all public traffic to the services. For example, a NodePort service opens a port on a worker node over both the private and public IP address of the worker node. An NLB service with a portable private IP address opens a public NodePort on every worker node. You must create a [Calico preDNAT network policy](https://cloud.ibm.com/docs/openshift?topic=openshift-network_policiesblock_ingress) to block public NodePorts.

Check out the following methods for private app networking:



Characteristics of network deployment patterns for a public and a private VLAN setup

 Name Load-balancing method Use case Implementation 

 Route HTTP(S) load balancing that exposes the app with a subdomain and uses custom routing rules Implement custom routing rules and SSL termination for multiple apps. Choose this method to remain Red Hat OpenShift-native; for example, you can use the Red Hat OpenShift web console to create and manage routes. <br><br>1. [Create a ClusterIP service](https://kubernetes.io/docs/concepts/services-networking/service/defining-a-service) to assign an internal IP address to your app.<br>2.
Classic flavors 

Review the classic worker node flavors by metro.



 ams 



Table 1. Worker node flavors for Amsterdam.

 Name Memory Network speed Cores Type OS Primary storage Secondary storage 

 b3c.16x64 64GB 1000Mbps 16 Virtual UBUNTU_18_64, UBUNTU_20_64 (default) 25GB SSD 100GB SSD 
 b3c.16x64.300gb 64GB 1000Mbps 16 Virtual UBUNTU_18_64, UBUNTU_20_64 (default) 25GB SSD 300GB SSD 
 b3c.32x128 128GB 1000Mbps 32 Virtual UBUNTU_18_64, UBUNTU_20_64 (default) 25GB SSD 100GB SSD 
 b3c.4x16 16GB 1000Mbps 4 Virtual UBUNTU_18_64, UBUNTU_20_64 (default) 25GB SSD 100GB SSD 
 b3c.56x242 242GB 1000Mbps 56 Virtual UBUNTU_18_64, UBUNTU_20_64 (default) 25GB SSD 100GB SSD 
 b3c.8x32 32GB 1000Mbps 8 Virtual UBUNTU_18_64, UBUNTU_20_64 (default) 25GB SSD 100GB SSD 
 c3c.16x16 16GB 1000Mbps 16 Virtual UBUNTU_18_64, UBUNTU_20_64 (default) 25GB SSD 100GB SSD 
 c3c.16x32 32GB 1000Mbps 16 Virtual UBUNTU_18_64, UBUNTU_20_64 (default) 25GB SSD 100GB SSD 
 c3c.32x32 32GB 1000Mbps 32 Virtual UBUNTU_18_64, UBUNTU_20_64 (default) 25GB SSD 100GB SSD 
 c3c.32x64 64GB 1000Mbps 32 Virtual UBUNTU_18_64, UBUNTU_20_64 (default) 25GB SSD 100GB SSD
The worker node can be fixed by using the ibmcloud ks worker reload[command](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reload) or the ibmcloud ks worker update[command](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_update). 
 Worker node runtime components (kubelet, kube-proxy, docker) N/A N/A Removed dependencies of runtime components on the primary disk. This enhancement prevents worker nodes from failing when the primary disk is filled up. 
 Root password expiration N/A N/A Root passwords for the worker nodes expire after 90 days for compliance reasons. If your automation tooling needs to log in to the worker node as root or relies on cron jobs that run as root, you can disable the password expiration by logging into the worker node and running chage -M -1 root. Note: If you have security compliance requirements that prevent running as root or removing password expiration, don't disable the expiration. Instead, you can [update](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_update) or [reload](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reload) your worker nodes at least every 90 days. 
 systemd N/A N/A Periodically clean transient mount units to prevent them from becoming unbounded. This action addresses [Kubernetes issue 57345](https://github.com/kubernetes/kubernetes/issues/57345) 
 Docker N/A N/A Disabled the default Docker bridge so that the 172.17.0.0/16 IP range is now used for private routes.
The worker node can be fixed by using the ibmcloud ks worker reload[command](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reload) or the ibmcloud ks worker update[command](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_update). 
 Worker node runtime components (kubelet, kube-proxy, docker) N/A N/A Removed dependencies of runtime components on the primary disk. This enhancement prevents worker nodes from failing when the primary disk is filled up. 
 Root password expiration N/A N/A Root passwords for the worker nodes expire after 90 days for compliance reasons. If your automation tooling needs to log in to the worker node as root or relies on cron jobs that run as root, you can disable the password expiration by logging into the worker node and running chage -M -1 root. Note: If you have security compliance requirements that prevent running as root or removing password expiration, don't disable the expiration. Instead, you can [update](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_update) or [reload](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reload) your worker nodes at least every 90 days. 
 systemd N/A N/A Periodically clean transient mount units to prevent them from becoming unbounded. This action addresses [Kubernetes issue 57345](https://github.com/kubernetes/kubernetes/issues/57345). 
 Docker N/A N/A Disabled the default Docker bridge so that the 172.17.0.0/16 IP range is now used for private routes.
Classic flavors 

Review the classic worker node flavors by metro.



 ams 



Table 1. Worker node flavors for Amsterdam.

 Name Memory Network speed Cores Type OS Primary storage Secondary storage 

 b3c.16x64 64GB 1000Mbps 16 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 b3c.16x64.300gb 64GB 1000Mbps 16 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 300GB SSD 
 b3c.32x128 128GB 1000Mbps 32 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 b3c.4x16 16GB 1000Mbps 4 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 b3c.56x242 242GB 1000Mbps 56 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 b3c.8x32 32GB 1000Mbps 8 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 c3c.16x16 16GB 1000Mbps 16 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 c3c.16x32 32GB 1000Mbps 16 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 c3c.32x32 32GB 1000Mbps 32 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 c3c.32x64 64GB 1000Mbps 32 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD 
 m3c.4x32 32GB 1000Mbps 4 Virtual REDHAT_7_64 (default), REDHAT_8_64 25GB SSD 100GB SSD
By default, all outbound network traffic is allowed for all worker nodes. Inbound network traffic is blocked except for a few ports. These ports allow IBM to monitor network traffic and automatically install security updates for the Kubernetes master, and so that connections can be established to NodePort, LoadBalancer, and Ingress services.

Because the default Calico network policies allow inbound public traffic to these services, you can create Calico policies to instead block all public traffic to the services. For example, a NodePort service opens a port on a worker node over both the private and public IP address of the worker node. An NLB service with a portable private IP address opens a public NodePort on every worker node. You must create a [Calico preDNAT network policy](https://cloud.ibm.com/docs/containers?topic=containers-network_policiesblock_ingress) to block public NodePorts.

Review the load balancing deployment patterns for private networking.

NodePort
: Load-balancing method: Port on a worker node that exposes the app on the worker's private IP address
: Use case: Test private access to one app or provide access for only a short amount of time.
: Implementation: [Create a NodePort service](https://cloud.ibm.com/docs/containers?topic=containers-nodeport). A NodePort service opens a port on a worker node over both the private and public IP address of the worker node. You must use a [Calico preDNAT](https://cloud.ibm.com/docs/containers?topic=containers-network_policiesblock_ingress) network policy to block traffic to the public NodePorts.

NLB v1.0
: Load-balancing method: Basic load balancing that exposes the app with a private IP address.
: Use case: Quickly expose one app to a private network with a private IP address.
: Implementation: 1. [Create a private NLB service](https://cloud.ibm.com/docs/containers?topic=containers-loadbalancer). An NLB with a portable private IP address still has a public node port open on every worker node. 2.
The flavor determines the amount of virtual CPU, memory, and disk space that is set up in each worker node and made available to your apps. Worker nodes in classic clusters can be created as virtual machines on shared or dedicated infrastructure, or as bare metal machines that are dedicated to you. For more information, see [Planning your worker node setup](https://cloud.ibm.com/docs/containers?topic=containers-planning_worker_nodes). After you create your cluster, you can add different flavors by [adding a worker pool](https://cloud.ibm.com/docs/containers?topic=containers-add_workersadd_pool).

Before you create a bare metal machine, be sure that you want to provision one. Bare metal machines are billed monthly. If you order a bare metal machine by mistake, you are charged for the entire month, even if you cancel the machine immediately.

ibmcloud ks flavors --zone <zone>
6. Check if you have existing VLANs in the zones that you want to include in your cluster, and note the ID of the VLAN. If you don't have a public or private VLAN in one of the zones that you want to use in your cluster, IBM Cloud Kubernetes Service automatically creates these VLANs for you when you create the cluster.

ibmcloud ks vlan ls --zone <zone>

Example output

ID Name Number Type Router
1519999 vlan 1355 private bcr02a.dal10
1519898 vlan 1357 private bcr02a.dal10
1518787 vlan 1252 public fcr02a.dal10
1518888 vlan 1254 public fcr02a.dal10



* To create a cluster in which you can run internet-facing workloads, check to see whether a public and private VLAN exist. If a public and private VLAN already exist, note the matching routers. Private VLAN routers always begin with bcr (back-end router) and public VLAN routers always begin with fcr (front-end router).
If an IBM Cloud service does not support private cloud service endpoints, your worker nodes must be connected to a subnet that has a public gateway attached to it. The pods on those worker nodes can securely communicate with the services over the public network through the subnet's public gateway. Note that a public gateway is not required on your subnets to allow inbound network traffic from the internet to LoadBalancer services or ALBs.

Within one VPC, you can create only one public gateway per zone, but that public gateway can be attached to multiple subnets within the zone. For more information about public gateways, see the [Networking for VPC documentation](https://cloud.ibm.com/docs/vpc?topic=vpc-about-networking-for-vpcpublic-gateway-for-external-connectivity).





 Virtual private endpoints (VPE) 

Worker nodes can communicate with the Kubernetes master through the cluster's [virtual private endpoint (VPE)](https://cloud.ibm.com/docs/vpc?topic=vpc-about-vpe).

A VPE is a virtual IP address that is bound to an endpoint gateway. One VPE gateway resource is created per cluster in your VPC. One IP address from one subnet in each zone where your cluster has worker nodes is automatically used for the VPE gateway, and the worker nodes in this zone use this IP address to communicate with the Kubernetes master. To view the VPE gateway details for your cluster, open the [Virtual private endpoint gateways for VPC dashboard](https://cloud.ibm.com/vpc-ext/network/endpointGateways) and look for the VPE gateway in the format iks-<cluster_ID>.

Note that your worker nodes automatically use the VPE that is created by default in your VPC.