vania-janet/multiturn-rag-retrieval-data huggingface dataset:
When you have more than 14 worker nodes, another subnet is ordered, so the subnets that you need to allow can change. To reduce the frequency of change, create worker pools with worker node flavors of higher CPU and memory resources so that you don't need to add worker nodes as often.



1. List the worker nodes in your cluster.

ibmcloud ks worker ls --cluster <cluster_name_or_ID>
2. From the output of the previous step, note all the unique network IDs (first three octets) of the Public IP for the worker nodes in your cluster. If you want to allow traffic from a private-only cluster, note the Private IP instead. In the following output, the unique network IDs are 169.xx.178 and 169.xx.210.

ID Public IP Private IP Machine Type State Status Zone Version
kube-dal10-crb2f60e9735254ac8b20b9c1e38b649a5-w31 169.xx.178.101 10.xxx.xx.xxx b3c.4x16.encrypted normal Ready dal10 1.26
kube-dal10-crb2f60e9735254ac8b20b9c1e38b649a5-w34 169.xx.178.102 10.xxx.xx.xxx b3c.4x16.encrypted normal Ready dal10 1.26
kube-dal12-crb2f60e9735254ac8b20b9c1e38b649a5-w32 169.xx.210.101 10.xxx.xx.xxx b3c.4x16.encrypted normal Ready dal12 1.26
kube-dal12-crb2f60e9735254ac8b20b9c1e38b649a5-w33 169.xx.210.102 10.xxx.xx.xxx b3c.4x16.encrypted normal Ready dal12 1.26
3. List the VLAN subnets for each unique network ID.
: Worker nodes that run Kubernetes version 1.14 or later set [process ID (PID) reservations and limits that vary by flavor](https://cloud.ibm.com/docs/openshift?topic=openshift-planning_worker_nodesresource_limit_node), to help prevent malicious or runaway apps from consuming all available PIDs.

Version change log
: Worker node patch updates are available.
: Red Hat OpenShift [3.11.154_1533_openshift](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_changelog_311311154_1533_worker).





 22 November 2019 

Bring your own DNS for load balancers
: Added steps for bringing your own custom domain for [NLBs](https://cloud.ibm.com/docs/openshift?topic=openshift-loadbalancer_hostnameloadbalancer_hostname_dns) in classic clusters and [VPC load balancers](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-lbaasvpc_lb_dns) in VPC clusters.

Gateway appliance firewalls
: Updated the [required IP addresses and ports](https://cloud.ibm.com/docs/openshift?topic=openshift-firewallvyatta_firewall) that you must open in a public gateway device firewall.

Ingress ALB subdomain format
: [Changes are made to the Ingress subdomain](https://cloud.ibm.com/docs/containers?topic=containers-managed-ingress-aboutmanaged-ingress-subdomain). New clusters are assigned an Ingress subdomain in the format cluster_name.globally_unique_account_HASH-0000.region.containers.appdomain.cloud and an Ingress secret in the format cluster_name.globally_unique_account_HASH-0000.
If you limited your ODF deployment to a subset of worker nodes by specifying node names during installation, you must update the ocscluster CRD to include the new name. If you did not limit your configuration to only certain worker nodes, you do not need to update the ocscluster CRD.

oc edit ocscluster

apiVersion: ocs.ibm.io/v1
kind: OcsCluster
metadata:
name: ocscluster-auto
spec:
. . .
osdSize: 250Gi
osdStorageClassName: ibmc-vpc-block-metro-10iops-tier
workerNodes:
- NODE-NAME Example 10.248.128.42
- NODE-NAME
- NODE-NAME
2. Wait for the OpenShift Data Foundation pods to deploy to the new worker. Verify the new persistent volumes are created and that all pods are in a Running state.

oc get pv
oc get ocscluster
oc get pods -n openshift-storage
3. Verify that all other required OpenShift Data Foundation pods are in Running state.

oc get pod -n openshift-storage | grep mon

Example output:

rook-ceph-mon-a-cd575c89b-b6k66 2/2 Running
0 38m
rook-ceph-mon-b-6776bc469b-tzzt8 2/2 Running
0 38m
rook-ceph-mon-d-5ff5d488b5-7v8xh 2/2 Running
0 4m8s
4. Verify that new OSD pods are running on the replacement node:

oc get pods -o wide -n openshift-storage| egrep -i <new_node_name> | egrep osd
5. Identify the crashcollector pod deployment.
Then, the old ALB pod is removed.
2. Verify that the worker pool and worker nodes have the dedicated=edge label.



* To check the worker pool

ibmcloud oc worker-pool get --cluster <cluster_name_or_ID> --worker-pool <worker_pool_name_or_ID>
* To check individual worker nodes, review the Labels field of the output of the following command.

oc describe node <worker_node_private_IP>



3. Retrieve all existing NLBs in the cluster.

oc get services --all-namespaces | grep LoadBalancer

In the output, note the Namespace and Name of each load balancer service. For example, the following output shows one public NLB in the default namespace.

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default webserver-lb LoadBalancer 172.21.190.18 169.46.17.2 80:30597/TCP 11m
4. Using the output from the previous step, run the following command for each NLB. This command redeploys the NLB to an edge worker node.

oc get service -n <namespace> <service_name> -o yaml | oc apply -f -

Example output

service "webserver-lb" configured
5. To verify that networking workloads are restricted to edge nodes, confirm that NLB are scheduled onto the edge nodes and are not scheduled onto non-edge nodes.



* NLB pods



1. Confirm that the NLB pods are deployed to edge nodes. Search for the external IP address of the load balancer service that is listed in the output of step 3. Replace the periods (.) with hyphens (-). Example for the webserver-lb NLB that has an external IP address of 169.46.17.2:

oc describe nodes -l dedicated=edge | grep "169-46-17-2"

Example output
helm install ibm-object-storage-plugin ./ibm-object-storage-plugin --set dcname="${DC_NAME}" --set provider="${CLUSTER_PROVIDER}" --set workerOS="${WORKER_OS}" --region="${REGION} --set platform="${PLATFORM}" --set license=true [--set bucketAccessPolicy=false]

DC_NAME
: The cluster data center. To retrieve the data center, run oc get cm cluster-info -n kube-system -o jsonpath="{.data.cluster-config.json}{'n'}". Store the data center value in an environment variable by running SET DC_NAME=<datacenter>. Optional: Set the environment variable in Windows PowerShell by running $env:DC_NAME="<datacenter>".

CLUSTER_PROVIDER
: The infrastructure provider. To retrieve this value, run oc get nodes -o jsonpath="{.items[].metadata.labels.ibm-cloud.kubernetes.io/iaas-provider}{'n'}". If the output from the previous step contains softlayer, then set the CLUSTER_PROVIDER to "IBMC". If the output contains gc, ng, or g2, then set the CLUSTER_PROVIDER to "IBMC-VPC". Store the infrastructure provider in an environment variable. For example: SET CLUSTER_PROVIDER="IBMC-VPC".

WORKER_OS and PLATFORM
: The operating system of the worker nodes. To retrieve these values, run oc get nodes -o jsonpath="{.items[].metadata.labels.ibm-cloud.kubernetes.io/os}{'n'}". Store the operating system of the worker nodes in an environment variable. For Red Hat OpenShift on IBM Cloud clusters, run SET WORKER_OS="redhat" and SET PLATFORM="openshift".

REGION
: The region of the worker nodes.
RHEL 7, which is currently the default operating system for cluster versions 4.10 and earlier, is not supported in version 4.11. If you upgrade a cluster master from version 4.10 to 4.11, you must migrate your worker nodes from RHEL 7 to RHEL 8. You cannot upgrade RHEL 7 worker nodes directly to RHEL 8. Instead, after you have upgraded to 4.11, you must provision a new RHEL 8 worker pool in your 4.11 cluster and then remove the RHEL 7 worker pool.

For more information about creating worker pools and adding worker nodes, see [Adding worker nodes in classic clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersclassic_pools) or [Adding worker nodes in VPC clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersvpc_pools).



1. [Upgrade your cluster master](https://cloud.ibm.com/docs/openshift?topic=openshift-updateupdate) from version 4.10 to 4.11.
2. In your 4.11 cluster, create a new worker pool to contain your RHEL 8 worker nodes. Make sure that the number of nodes specified with the --size-per-zone option matches the number of RHEL 7 worker nodes that are to be replaced. By default, any worker nodes added to your new worker pool run RHEL 8.

For classic clusters. See the [CLI reference](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_pool_create) for command details.

ibmcloud oc worker-pool create classic --name <worker_pool_name> --cluster <cluster_name_or_ID> --flavor <flavor> --size-per-zone <number_of_workers_per_zone>

For VPC clusters.
IBM Cloud File Storage for Classic plug-in and monitor 408 410 Updated universal base image (UBI) to version 8.6-751 to resolve CVEs. 
 IBM Cloud RBAC Operator 8c8c82b 8c96932 Updated Go to version 1.18.1 
 Key Management Service provider v2.5.4 v2.5.5 Updated Go to version 1.17.10 and updated the golang dependencies. 
 Kubernetes add-on resizer 1.8.13 1.8.14 See the [Kubernetes add-on resizer release notes](https://github.com/kubernetes/autoscaler/releases/tag/addon-resizer-1.8.14). 
 Kubernetes Dashboard v2.3.1 v2.3.1 The default Kubernetes Dashboard settings found in the kubernetes-dashboard-settings config map in the kube-system namespace have been updated to set resourceAutoRefreshTimeInterval to 60. This default change is only applied to new clusters. The previous default value was 5. If your cluster has Kubernetes Dashboard performance problems, see the steps for [changing the auto refresh time interval](https://cloud.ibm.com/docs/containers?topic=containers-ts-kube-dashboord-oom). 
 Load balancer and load balancer monitor for IBM Cloud Provider 1916 1998 Updated Go to version 1.17.10 and updated dependencies. 







 Change log for worker node fix pack 1.20.15_1581, released 23 May 2022 

The following table shows the changes that are in the worker node fix pack 1.20.15_1581. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.20.15_1580

 Component Previous Current Description
Change log for worker node fix pack 4.9.59_1588_openshift, released 10 May 2023 

The following table shows the changes that are in the worker node fix pack 4.9.59_1588_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.9.59_1587_openshift

 Component Previous Current Description 

 RHEL 7 Packages N/A N/A Package updates for [CVE-2022-43750](https://nvd.nist.gov/vuln/detail/CVE-2022-43750). 
 RHEL 8 Packages N/A N/A N/A 
 Red Hat OpenShift on IBM Cloud N/A N/A N/A 
 Haproxy N/A N/A N/A 







 Change log for master fix pack 4.9.59_1587_openshift, released 27 April 2023 

The following table shows the changes that are in the master fix pack 4.9.59_1587_openshift. Master patch updates are applied automatically.



Changes since version 4.9.57_1583_openshift

 Component Previous Current Description 

 Cluster health image v1.3.17 v1.3.19 Updated Go to version 1.19.8 and updated dependencies. 
 IBM Calico extension 1366-amd64 1390-amd64 Eliminate IP syscall. 
 IBM Cloud Block Storage driver and plug-in v2.4.0 v2.4.5 Updated universal base image (UBI) to resolve CVEs. Updated Go to version 1.19.8 and updated dependencies. 
 IBM Cloud Controller Manager v1.22.17-5 v1.22.17-8 Update Go dependencies. 
 IBM Cloud File Storage for Classic plug-in and monitor 429 431 Updated Go to version 1.19.8 and updated dependencies. Update UBI base image. 
 Key Management Service provider v2.6.4 v2.6.5 Updated Go to version 1.19.7 and updated dependencies.
Change log for worker node fix pack 4.6.61_1602_openshift, released 10 October 2022 

The following table shows the changes that are in the worker node fix pack 4.6.61_1602_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.6.61_1601_openshift

 Component Previous Current Description 

 RHEL 7 Packages N/A N/A N/A 
 RHEL 8 Packages N/A N/A N/A 
 Red Hat OpenShift on IBM Cloud N/A N/A N/A 







 Change log for master fix pack 4.6.61_1600_openshift, released 26 September 2022 

The following table shows the changes that are in the master fix pack 4.6.61_1600_openshift. Master patch updates are applied automatically.



Changes since version 4.6.601598openshift

 Component Previous Current Description 

 Cluster health image v1.3.10 v1.3.11 Updated Go dependencies and to Go version 1.18.6. 
 IBM Calico extension 1006 1096 Updated image for [CVE-2022-2526](https://nvd.nist.gov/vuln/detail/CVE-2022-2526). 
 IBM Cloud Block Storage driver and plug-in v2.2.9 v2.3.1 Updated to Go version 1.18.6. Updated universal base image (UBI) to version 8.6-941 to resolve CVEs. 
 IBM Cloud File Storage for Classic plug-in and monitor 414 416 Updated to Go version 1.18.6. Updated universal base image (UBI) to version 8.6-941 to resolve CVEs. 
 Key Management Service Provider v2.5.8 v2.5.9 Updated Go dependencies and to Go version 1.18.6.
Change log for worker node fix pack 3.11.570_1616_openshift, released 20 December 2021 

The following table shows the changes that are in the worker node fix pack patch update 3.11.570_1616_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 3.11.542_1612_openshift

 Component Previous Current Description 

 Red Hat OpenShift node 3.11.542 3.11.570 See [change logs](https://docs.openshift.com/container-platform/3.11/release_notes/ocp_3_11_release_notes.htmlocp-3-11-570) 







 Change log for master fix pack 3.11.542_1614_openshift, released 7 December 2021 

The following table shows the changes that are in the master fix pack patch update 3.11.542_1614_openshift. Master patch updates are applied automatically.



Changes since version 3.11.542_1611_openshift

 Component Previous Current Description 

 Cluster health image v1.1.27 v1.1.29 Updated universal base image (UBI) to the latest 8.4 version to resolve CVEs. Updated to use Go version 1.16.10. 
 Key Management Service provider v1.0.19 v1.0.21 Updated universal base image (UBI) to the latest 8.4 version to resolve CVEs. Updated to use Go version 1.16.10. 
 Load balancer and load balancer monitor for IBM Cloud Provider 1589 1660 Updated Alpine base image to the latest 3.14 version to resolve CVEs. Updated to use Go version 1.16.10. 
 IBM Cloud File Storage for Classic plug-in and monitor 401 402 Updated universal base image (UBI) to the 8.5-204 version to resolve CVEs. Updated to use Go version 1.16.10. 







 Change log for worker node fix pack 3.11.542_1615_openshift, released 6 December 2021
Change log for worker node fix pack 4.13.4_1525_openshift, released 03 July 2023 

The following table shows the changes that are in the worker node fix pack 4.13.4_1525_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.13.3_1523_openshift

 Component Previous Current Description 

 Red Hat OpenShift on IBM Cloud. 4.13.3 4.13.4 see [change log](https://docs.openshift.com/container-platform/4.13/release_notes/ocp-4-13-release-notes.htmlocp-4-13-4). 







 Change log for master fix pack 4.13.0_1524_openshift, released 27 June 2023 

The following table shows the changes that are in the master fix pack 4.13.0_1524_openshift. Master patch updates are applied automatically.



Changes since version 4.13.0_1522_openshift

 Component Previous Current Description 

 Cluster health image v1.3.20 v1.3.21 Updated Go dependencies and to Go version 1.19.10. 
 IBM Cloud Controller Manager v1.26.5-5 v1.26.6-1 Updated to support the Kubernetes 1.26.6 release. Updated Go dependencies and to Go version 1.19.10. Updated calicoctl and vpcctl. 
 IBM Cloud File Storage for Classic plug-in and monitor 431 433 Updated Go to version 1.20.4. Updated UBI base image. 
 Load balancer and load balancer monitor for IBM Cloud Provider 2486 2584 Updated Go dependencies and to Go version 1.19.9. Updated base image. 
 OpenVPN client 2.5.8-r0-IKS-674-amd64 2.6.4-r0-IKS-34-amd64 Upgrade openvpn to version 2.6.4-r0.
Kubernetes configuration N/A N/A The Kubernetes API server audit policy configuration is updated to include auditing the scheduling.k8s.io API group and the tokenreviews resource. 







 Change log for worker node fix pack 1.15.12_1546, released 20 July 2020 

The following table shows the changes that are in the worker node fix pack 1.15.12_1546. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.15.12_1544

 Component Previous Current Description 

 HAProxy 2.0.15-afe432 1.8.25-384f42 See the [HAProxy change log](https://www.haproxy.org/download/1.8/src/CHANGELOG). Fixes a connection leak that happens when HAProxy is under high load.
Updated image for [DLA-2424-1](https://www.debian.org/lts/security/2020/dla-2424). 
 IBM Cloud File Storage for Classic plug-in and monitor 378 379 Updated to use the universal base image (UBI) and to use Go version 1.15.2. 
 Key Management Service provider v1.0.3 v1.0.4 Updated image for [DLA-2424-1](https://www.debian.org/lts/security/2020/dla-2424). 
 Kubernetes v1.17.13 v1.17.14 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.17.14). 







 Change log for worker node fix pack 1.17.13_1544, released 9 November 2020 

The following table shows the changes that are in the worker node fix pack 1.17.13_1544. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.17.13_1543

 Component Previous Current Description
* For Kubernetes on Classic infrastructure see reference documentation [Creating classic cluster](https://cloud.ibm.com/docs/containers?topic=containers-cluster-create-classic&interface=ui).
* Choose a resource group.
* Uncheck all zones except one.
* Scale down to 1 Worker nodes per zone.
* Choose the smallest Worker Pool flavor.
* Enter a Cluster name.
* Click Create.







 Step 2: Clone a sample application 

In this section, you will clone a GitHub repo with a simple Helm-based [NodeJS](https://nodejs.dev) sample application with a landing page and two endpoints to get started. You can always extend the sample application based on your requirement.



1. On a terminal, run the below command to clone the [GitHub repository](https://github.com/IBM-Cloud/kubernetes-node-app/) to your machine:

git clone https://github.com/IBM-Cloud/kubernetes-node-app
2. Change to the application directory:

cd kubernetes-node-app



This sample application code contains all the necessary configuration files for local development and deployment to Kubernetes.





 Step 3: Deploy application to cluster using helm chart 



 Deploy the application with Helm 3 

The container image for the application as already been built and pushed to a public Container Registry. In this section you will deploy the sample application using [Helm](https://helm.sh/). Helm helps you manage Kubernetes applications through Helm Charts, which helps define, install, and upgrade even the most complex Kubernetes application.

Note: If you want to build and push the application to your own container registry you can use the Docker CLI to do so. The Dockerfile is provided in the repository and images can be pushed to the Container Registry or any other container registry.



1. Define an environment variable named MYAPP and set the name of the application by replacing the placeholder with your initials:

export MYAPP=<your-initials>kubenodeapp
2. Identify your cluster:
Follow the [Red Hat Marketplace instructions](https://marketplace.redhat.com/en-us/workspace/clusters/add/register) to create a namespace, operator, and global pull secret for the Red Hat Marketplace.
2. Verify that the global pull secret for the cluster is updated with the registry.marketplace.redhat.com secret.

oc get secret pull-secret -n openshift-config --output="jsonpath={.data..dockerconfigjson}" | base64 --decode | grep "marketplace" -A4
3. To pick up the global configuration changes, reload all the worker nodes in your cluster.



1. Note the ID of the worker nodes in your cluster.

ibmcloud oc worker ls -c <cluster_name_or_ID>
2. Reload each classic worker node, or replace each VPC worker node. You can reload or replace multiple worker nodes by including multiple -w options, but make sure to leave enough worker nodes running at the same time for your apps to avoid an outage because they are reloaded concurrently.

Classic: Reload worker nodes.

ibmcloud oc worker reload -c <cluster_name_or_ID> -w <workerID_1> -w <workerID_2>

VPC: Replace worker nodes.

ibmcloud oc worker replace -c <cluster_name_or_ID> -w <workerID_1> -w <workerID_2>



4. After your worker nodes are reloaded, verify that your cluster is listed in your [Red Hat Marketplace workspace](https://marketplace.redhat.com/en-us/workspace/clusters).



Now, you can install and manage software from Red Hat Marketplace in your cluster. For more information, see the [Red Hat Marketplace documentation](https://marketplace.redhat.com/en-us/documentation/).
: If you want to upgrade a 4.9 cluster to 4.10, make sure to migrate your worker nodes from RHEL 7 to RHEL 8 before you upgrade.
: For more information about updating to 4.9, see the [Version information and update actions](https://cloud.ibm.com/docs/openshift?topic=openshift-cs_versions_49).

Version 4.8 clusters
: RHEL 7 is supported until end of support for these cluster versions. For more information about cluster version support, see the [Version information](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_versions).
: If you want to upgrade a version 4.8 cluster to 4.9, make sure to migrate your worker nodes to RHEL 8 after you upgrade.





 Creating RHEL 8 worker pools in the command line 



1. You can use the following example commands to create a new worker pool with RHEL 8 worker nodes for your cluster type. Note that you must include the --operating-system option and specify REDHAT_8_64. Make sure that the number of nodes specified with the --size-per-zone option matches the number of RHEL 7 worker nodes that are to be replaced. If you have a Cloud Pak entitlement, make sure to include the --entitlement cloud_pak option.

Classic: Example command to create a RHEL 8 worker pool. For more information about the worker pool create classic command, see the [CLI reference](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_worker_pool_create). For more information about creating worker pools and adding worker nodes, see [Adding worker nodes in classic clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersclassic_pools).