vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Cluster autoscaler add-on versions 1.0.7_988 and 1.0.8_987.
: For more information, see [the change log](https://cloud.ibm.com/docs/containers?topic=containers-ca_changelog).





 28 March 2023 

Master fix packs 1.23.17_1569, 1.24.12_1559, 1.25.8_1536, and 1.26.3_1528.
: For more information, see the change logs: [1.23](https://cloud.ibm.com/docs/containers?topic=containers-changelog_123), [1.24](https://cloud.ibm.com/docs/containers?topic=containers-changelog_124), [1.25](https://cloud.ibm.com/docs/containers?topic=containers-changelog_125), and [1.26](https://cloud.ibm.com/docs/containers?topic=containers-changelog_126).

Worker node fix packs 1.23.17_1570, 1.24.12_1560, 1.25.8_1537, and 1.26.3_1529.
: For more information, see the change logs: [1.23](https://cloud.ibm.com/docs/containers?topic=containers-changelog_123), [1.24](https://cloud.ibm.com/docs/containers?topic=containers-changelog_124), [1.25](https://cloud.ibm.com/docs/containers?topic=containers-changelog_125), and [1.26](https://cloud.ibm.com/docs/containers?topic=containers-changelog_126).





 24 March 2023 

Ingress ALB version 1.4.0_3896_iks, 1.5.1_3897_iks, and 1.6.4_3898_iks
What default components can I update separately from the cluster?
: You can optionally disable automatic updates for the following components:



* [Fluentd for logging](https://cloud.ibm.com/docs/containers?topic=containers-updatelogging-up)
* [Ingress application load balancer (ALB)](https://cloud.ibm.com/docs/containers?topic=containers-updatealb)



Are there components that I can't update separately from the cluster?
: Yes. Your cluster is deployed with the following managed components and associated resources that can't be changed, except to scale pods or edit configmaps for certain performance benefits. If you try to change one of these deployment components, their original settings are restored on a regular interval when they are updated with the cluster master. However, note that resources that you create that are associated with these components, such as Calico network policies that you create to be implemented by the Calico deployment components, are not updated.



* calico components
* coredns components
* ibm-cloud-provider-ip
* ibm-file-plugin
* ibm-keepalived-watcher
* ibm-master-proxy
* ibm-storage-watcher
* kubernetes-dashboard components
* metrics-server
* olm-operator and catalog components (1.16 and later)
* vpn
* In gateway-enabled classic clusters, ibm-gateway-controller



Can I install other plug-ins or add-ons than the default components?
: Yes. IBM Cloud Kubernetes Service provides other plugin-ins and add-ons that you can choose from to add capabilities to your cluster.
Red Hat OpenShift 4.3.33 4.3.35 See the [Red Hat OpenShift release notes](https://docs.openshift.com/container-platform/4.3/release_notes/ocp-4-3-release-notes.htmlocp-4-3-35). 
 RHEL 7 packages N/A N/A Updated worker node image with package updates. 







 Change log for worker node fix pack 4.3.33_1537_openshift, released 31 August 2020 

The following table shows the changes that are in the worker node fix pack update 4.3.33_1537_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.3.31_1534_openshift

 Component Previous Current Description 

 Master proxy 1.8.25-384f42 1.8.26-561f1a See the [HAProxy change logs](https://www.haproxy.org/download/1.8/src/CHANGELOG). 
 Red Hat OpenShift 4.3.31 4.3.33 See the [Red Hat OpenShift release notes](https://docs.openshift.com/container-platform/4.3/release_notes/ocp-4-3-release-notes.htmlocp-4-3-33). 
 RHEL 7 packages 3.10.0-1127.18.2.el7 3.10.0-1127.19.1.el7 Updated worker node image with kernel and package updates. 







 Change log for master fix pack 4.3.31_1536_openshift, released 21 August 2020 

The following table shows the changes that are in the master fix pack patch update 4.3.31_1536_openshift. Master patch updates are applied automatically.



Changes since version 4.3.31_1534_openshift

 Component Previous Current Description 

 Cluster master operations N/A N/A Fixed a problem that might cause cluster master operations to fail for clusters with a private cloud service endpoint enabled.
After you run this command, use the [ibmcloud ks cluster master private-service-endpoint allowlist command](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_master_pse_allowlist_add) to add subnets to the allowlist. Only authorized requests to your cluster master that originate from subnets in the allowlist are permitted through the cluster's private cloud service endpoint. If the public cloud service endpoint is enabled for your cluster, authorized requests are still permitted through the public cloud service endpoint.

ibmcloud ks cluster master private-service-endpoint allowlist enable --cluster CLUSTER [-f] [-q]

Minimum required permissions
: Editor platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

-f
: Optional: Force the command to run without user prompts.

-q
: Optional: Do not show the message of the day or update reminders.



 Example cluster master private-service-endpoint allowlist enable command 

ibmcloud ks cluster master private-service-endpoint allowlist enable --cluster mycluster







 ibmcloud ks cluster master private-service-endpoint allowlist get 

Virtual Private Cloud

Classic infrastructure

List all subnets in the allowlist for a cluster's private cloud service endpoint.

This list includes subnets that you manually added by using the ibmcloud ks cluster master private-service-endpoint allowlist add command and subnets that are automatically added and managed by IBM, such as worker node subnets.

ibmcloud ks cluster master private-service-endpoint allowlist get --cluster CLUSTER [-q]

Minimum required permissions
: Viewer platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

-q
: Optional: Do not show the message of the day or update reminders.



 Example cluster master private-service-endpoint allowlist get command
Updating Portworx in your cluster 

Beginning with version 2.12 Portworx uses an operator-based deployment model instead of the Helm based model use in version 2.11 and earlier. If you are updating from Portworx 2.11 to version 2.12, follow the migration steps in the [Portworx documentation](https://docs.portworx.com/operations/operate-kubernetes/migrate-daemonset/).



 VPC: Updating worker nodes with Portworx volumes 

Virtual Private Cloud

When you update a worker node in a VPC cluster, the worker node is removed from your cluster and replaced with a new worker node. If Portworx volumes are attached to the worker node that you replaced, you must attach the volumes to the new worker node.

Update only one worker node at a time. When the worker node update is complete, attach your Block Storage for VPC and restart the Portworx pod.



1. [Enter maintenance mode on the worker nodes that you want to update](https://docs.portworx.com/portworx-install-with-kubernetes/operate-and-maintain-on-kubernetes/troubleshooting/enter-maintenance-mode/).
2. [Update your VPC worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-updatevpc_worker_node).
3. [Attach raw Block Storage for VPC to your worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-utilitiesvpc_api_attach).
4. Verify that your storage is attached by running the following command.

kubectl get pv -o yaml | grep "attachstatus"
5. Restart the Portworx pod on the new worker node.

kubectl delete pod <portworx_pod> -n <portworx_namespace>
6. Exit maintenance mode

pxctl service maintenance --exit
7.
Check the add-ons and plug-ins that are installed in your cluster for any impact that might be caused by updating the cluster version.



* Checking add-ons



1. List the add-ons in the cluster.

ibmcloud oc cluster addon ls --cluster CLUSTER
2. Check the supported Red Hat OpenShift version for each add-on that is installed.

ibmcloud oc addon-versions
3. If the add-on must be updated to run in the Red Hat OpenShift version that you want to update your cluster to, [update the add-on](https://cloud.ibm.com/docs/openshift?topic=openshift-managed-addonsupdating-managed-add-ons).



* Checking plug-ins



1. In the [Helm catalog](https://cloud.ibm.com/kubernetes/helm), find the plug-ins that you installed in your cluster.
2. From the side menu, expand the SOURCES & TAR FILE section.
3. Download and open the source code.
4. Check the README.md or RELEASENOTES.md files for supported versions.
5. If the plug-in must be updated to run in the Red Hat OpenShift version that you want to update your cluster to, update the plug-in by following the plug-in instructions.





4. Update your API server and associated master components by using the [IBM Cloud console](https://cloud.ibm.com/login) or running the CLI ibmcloud oc cluster master update[command](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_cluster_update).
5. Wait a few minutes, then confirm that the update is complete. Review the API server version on the IBM Cloud clusters dashboard or run ibmcloud oc cluster ls.
6.
Component Location Previous Current Description 

 Cluster health image Master N/A N/A Cluster health status now includes links to IBM Cloud documentation. 
 IBM Cloud Provider Master v1.16.7-170 v1.16.8-192 Updated to support the Kubernetes 1.16.8 release and to use Go version 1.13.8. Fixed VPC load balancer error message to use the current IBM Cloud Kubernetes Service plug-in CLI syntax. 
 Kubernetes Both v1.16.7 v1.16.8 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.16.8). 
 Load balancer and load balancer monitor for IBM Cloud Provider Master 169 177 Version 2.0 network load balancers (NLB) were updated to fix problems with long-lived network connections to endpoints that failed readiness probes. 
 Operator Lifecycle Manager Catalog Master v1.5.9 v1.5.11 See the [Operator Lifecycle Manager Catalog release notes](https://github.com/operator-framework/operator-registry/releases/tag/v1.5.11).
Example output when the enablement is in progress

NAME: <cluster_name>
ID: <cluster_ID>
...
Master Status: Key management service feature enablement in progress.

Example output when the master is ready

NAME: <cluster_name>
ID: <cluster_ID>
...
Master Status: Ready (1 min ago)
...
Key Management Service: enabled

After the KMS provider is enabled in the cluster, all cluster secrets are automatically encrypted.
6. Optional: [Verify that your secrets are encrypted](https://cloud.ibm.com/docs/containers?topic=containers-encryptionverify_kms).



Do not delete root keys in your KMS instance, even if you rotate to use a new key. If you delete a root key that a cluster uses, the cluster becomes unusable, loses all its data, and can't be recovered. When you rotate a root key, you can't reuse a previous root key for the same cluster. Similarly, if you disable a root key, operations that rely on reading secrets fail. Unlike deleting a root key, however, you can reenable a disabled key to make your cluster usable again.





 Enabling KMS encryption for the cluster through the console 

You can enable a KMS provider, update the KMS provider instance, or update the root key through the IBM Cloud console.



1. Complete the [prerequisite steps](https://cloud.ibm.com/docs/containers?topic=containers-encryptionkms_prereqs) to create a KMS instance and root key.
2. From the [Kubernetes clusters console](https://cloud.ibm.com/kubernetes/clusters), select the cluster that you want to enable encryption for.
3. From the Overview tab, in the Summary > Key management service section, click Enable. If you already enabled the KMS provider, click Update.
4. Select the Key management service instance and Root key that you want to use for the encryption.
Change log for master fix pack 1.16.14_1544, released 18 August 2020 

The following table shows the changes that are in the master fix pack patch update 1.16.14_1544. Master patch updates are applied automatically.



Changes since version 1.16.13_1542

 Component Previous Current Description 

 Cluster health image v1.1.8 v1.1.9 Updated to use Go version 1.13.13. 
 GPU device plug-in and installer 8c24345 bacb9e1 Updated image for [CVE-2020-14039](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-14039) and [CVE-2020-15586](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-15586). 
 IBM Cloud Provider v1.16.13-289 v1.16.14-311 Updated to support the Kubernetes 1.16.14 release and to use Go version 1.13.15. 
 IBM Cloud File Storage for Classic plug-in and monitor 376 377 Fixed a bug that prevents persistent volume claim (PVC) creation failures from being retried. 
 Key Management Service provider v1.0.0 v1.0.1 Updated image for [CVE-2020-15586](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-15586). 
 Kubernetes v1.16.13 v1.16.14 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.16.14). 
 Kubernetes add-on resizer 1.8.5 1.8.11 See the [Kubernetes add-on resizer release notes](https://github.com/kubernetes/autoscaler/releases/tag/addon-resizer-1.8.11).
Component Location Previous Current Description 

 Cluster health image Master N/A N/A Cluster health status now includes links to IBM Cloud documentation. 
 IBM Cloud Provider Master v1.15.10-252 v1.15.11-274 Updated to support the Kubernetes 1.15.11 release and to use Go version 1.12.17. Fixed VPC load balancer error message to use the current IBM Cloud Kubernetes Service plug-in CLI syntax. 
 Kubernetes Both v1.15.10 v1.15.11 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.15.11).
[Create a configmap](https://cloud.ibm.com/docs/containers?topic=containers-updateworker-up-configmap) to control the maximum number of worker nodes that can be unavailable at a time in your cluster. When you update your worker nodes, the ConfigMap helps prevent downtime for your apps as the apps are rescheduled orderly onto available worker nodes.
4. Update all the worker nodes in your cluster to remove the public cloud service endpoint configuration.

ibmcloud ks worker update --cluster <cluster_name_or_ID> --worker <worker1,worker2>

By issuing the update command, the worker nodes are reloaded to pick up the service endpoint configuration. If no worker update is available, you must reload the worker nodes manually with the ibmcloud ks worker reload[command](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reload). If you reload, be sure to cordon, drain, and manage the order to control the maximum number of worker nodes that are unavailable at a time.







 Steps to disable the public cloud service endpoint 

To disable the public cloud service endpoint, you must first enable the private cloud service endpoint so that your worker nodes can communicate with the Kubernetes master.



1. [Enable the private cloud service endpoint](https://cloud.ibm.com/docs/containers?topic=containers-cs_network_clusterset-up-private-se).
2. Disable the public cloud service endpoint.

ibmcloud ks cluster master public-service-endpoint disable --cluster <cluster_name_or_ID>
3. Refresh the Kubernetes master API server to remove the public cloud service endpoint by following the CLI prompt or by manually running the following command. It might take several minutes for the master to refresh.

ibmcloud ks cluster master refresh --cluster <cluster_name_or_ID>
4.
ibmcloud ks cluster master private-service-endpoint disable --cluster my_cluster







 ibmcloud ks cluster master private-service-endpoint enable 

Classic infrastructure

Enable the [private cloud service endpoint](https://cloud.ibm.com/docs/containers?topic=containers-plan_basicsworkeruser-master) to make your cluster master privately accessible.

To run this command:



1. Enable [VRF](https://cloud.ibm.com/docs/account?topic=account-vrf-service-endpointvrf) in your IBM Cloud infrastructure account. To check whether a VRF is already enabled, use the ibmcloud account show command.
2. [Enable your IBM Cloud account to use service endpoints](https://cloud.ibm.com/docs/account?topic=account-vrf-service-endpointservice-endpoint).
3. Run ibmcloud ks cluster master private-service-endpoint enable --cluster <cluster_name>.
4. Follow the prompt in the CLI to refresh the Kubernetes master API server.
5. [Reload all the worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reload) in your cluster to pick up the private endpoint configuration.



ibmcloud ks cluster master private-service-endpoint enable --cluster CLUSTER [-q]

Minimum required permissions
: Administrator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

-q
: Optional: Do not show the message of the day or update reminders.

-y
: Optional: Refresh the cluster master and reload worker nodes with no user prompts.



 Example cluster master private-service-endpoint enable command 

ibmcloud ks cluster master private-service-endpoint enable --cluster my_cluster







 ibmcloud ks cluster master public-service-endpoint disable 

Virtual Private Cloud
Review [Security Bulletin: IBM Cloud Kubernetes Service is affected by an endpoint resource security design flaw in Kubernetes (CVE-2021-25740)](https://www.ibm.com/support/pages/node/6574821)before updating.



 Update before master 

The following table shows the actions that you must take before you update the Kubernetes master.



Changes to make before you update the master to Kubernetes 1.22

 Type Description 

 Unsupported: Beta versions of PriorityClass API Migrate manifests and API clients to use the scheduling.k8s.io/v1 API version, available since Kubernetes version 1.14. For more information, see [Deprecated API Migration Guide - v1.22](https://kubernetes.io/docs/reference/using-api/deprecation-guide/v1-22). 
 Unsupported: Beta versions of ClusterRole, ClusterRoleBinding, Role, and RoleBinding APIs Migrate manifests and API clients to use the rbac.authorization.k8s.io/v1 API version, available since Kubernetes version 1.8. For more information, see [Deprecated API Migration Guide - v1.22](https://kubernetes.io/docs/reference/using-api/deprecation-guide/v1-22). 
 Unsupported: Beta versions of ValidatingWebhookConfiguration and MutatingWebhookConfiguration APIs Migrate manifests and API clients to use the admissionregistration.k8s.io/v1 API version, available since Kubernetes version 1.16. For more information, see [Kubernetes API and Feature Removals In 1.22: Here’s What You Need To Know](https://kubernetes.io/blog/2021/07/14/upcoming-changes-in-kubernetes-1-22/).
: Updated the [CLI reference page](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cli) to reflect multiple changes for the [release of version 0.3.28](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_changelog) of the IBM Cloud Kubernetes Service CLI plug-in.

Cluster add-ons change log
: Updated the [ALB nginx-ingress image to build 457](https://cloud.ibm.com/docs/containers?topic=containers-cluster-add-ons-changelog).

Cluster and worker states
: Updated the [Logging and monitoring page](https://cloud.ibm.com/docs/containers?topic=containers-health-monitorstates) to add reference tables about cluster and worker node states.

Cluster planning and creation
: You can now find information about cluster planning, creation, and removal and network planning in the following pages:



* [Planning your cluster network setup](https://cloud.ibm.com/docs/containers?topic=containers-plan_clusters)
* [Planning your cluster for high availability](https://cloud.ibm.com/docs/containers?topic=containers-ha_clusters)
* [Planning your worker node setup](https://cloud.ibm.com/docs/containers?topic=containers-planning_worker_nodes)
* [Creating clusters](https://cloud.ibm.com/docs/containers?topic=containers-clusters)
* [Adding worker nodes and zones to clusters](https://cloud.ibm.com/docs/containers?topic=containers-add_workers)
* [Removing clusters](https://cloud.ibm.com/docs/containers?topic=containers-remove)
* [Changing service endpoints or VLAN connections](https://cloud.ibm.com/docs/containers?topic=containers-cs_network_cluster)



Cluster version updates
ibmcloud ks cluster master private-service-endpoint disable --cluster my_cluster







 ibmcloud ks cluster master private-service-endpoint enable 

Classic infrastructure

Enable the [private cloud service endpoint](https://cloud.ibm.com/docs/containers?topic=containers-plan_basicsworkeruser-master) to make your cluster master privately accessible.

To run this command:



1. Enable [VRF](https://cloud.ibm.com/docs/account?topic=account-vrf-service-endpointvrf) in your IBM Cloud infrastructure account. To check whether a VRF is already enabled, use the ibmcloud account show command.
2. [Enable your IBM Cloud account to use service endpoints](https://cloud.ibm.com/docs/account?topic=account-vrf-service-endpointservice-endpoint).
3. Run ibmcloud ks cluster master private-service-endpoint enable --cluster <cluster_name>.
4. Follow the prompt in the CLI to refresh the Kubernetes master API server.
5. [Reload all the worker nodes](https://cloud.ibm.com/docs/cli?topic=cli-kubernetes-service-clics_worker_reload) in your cluster to pick up the private endpoint configuration.



ibmcloud ks cluster master private-service-endpoint enable --cluster CLUSTER [-q]

Minimum required permissions
: Administrator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

-q
: Optional: Do not show the message of the day or update reminders.

-y
: Optional: Refresh the cluster master and reload worker nodes with no user prompts.



 Example cluster master private-service-endpoint enable command 

ibmcloud ks cluster master private-service-endpoint enable --cluster my_cluster







 ibmcloud ks cluster master public-service-endpoint disable 

Virtual Private Cloud