vania-janet/multiturn-rag-retrieval-data huggingface dataset:
CAs are less critical for daily transaction processing. After all the users have been registered and enrolled with the CA, it is no longer required until the next time those services are required.



 Peer considerations 

HA for peers means always having redundant peers, that is at least two peers available for each organization on the same channel to process requests from client applications. Multiple peers can be deployed to a single worker node, or spread across worker nodes, zones (if you are using IBM Cloud), or even regions. Whenever you deploy multiple peers and join them to the same channel, the peers act as HA pairs because the channel and the data are automatically synchronized across all peers in the channel. By design, a blockchain network is meant to have multiple organizations that transact on the same channels. Therefore, the common deployment model is that for any given channel, there are redundant peers for each organization spread across several organization account clusters that are all synchronizing data between each other. Each organization can have a peer in their own cluster in any region.

For even more robust HA coverage, you can stand up multiple clusters in multiple regions and deploy peers in all of them. However, if high performance is desired, care must be taken when distributing peers to ensure the latency and bandwidth between them is sufficient to achieve your performance targets.

Anchor peers on a channel facilitate cross-organization communication that is required for private data,

gossip, and service discovery to work. If only one anchor peer exists on a channel, and that peer becomes unavailable, the organizations are no longer connected and the cross-organization gossip is no longer possible. Therefore, when you create redundant peers for an organization, be sure to add redundant [anchor peers on the channel](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-governibp-console-govern-channels-anchor-peers) as well.

Finally, your peer redundancy strategy needs to take into account your smart contract endorsement policies to ensure that you always have enough peers available to satisfy the

endorsement policyrequirements. For example, if an endorsement policy requires a specific number of endorsements, your peer HA strategy needs to ensure that there are always that number of peers available.
However, to prevent other workloads from running on edge worker nodes and consuming worker node resources, you must use [Kubernetes taints](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/).

Before you begin



* Ensure you that have the [Manager IBM Cloud IAM service access role for all namespaces](https://cloud.ibm.com/docs/openshift?topic=openshift-userschecking-perms).
* [Access your Red Hat OpenShift cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-access_cluster).



Complete the following steps to prevent other workloads from running on edge worker nodes.

If you want to allow the Sysdig agent to run on your edge nodes, see [Deploying the Sysdig agent to edge worker nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-edgesysdig-edge).



1. Apply a taint to the worker nodes with the dedicated=edge label. The taint prevents pods from running on the worker node and removes pods that don't have the dedicated=edge label from the worker node. The pods that are removed are redeployed to other worker nodes with capacity.

To apply a taint to all existing and future worker nodes in a worker pool:

ibmcloud oc worker-pool taint set -c <cluster_name_or_ID> --worker-pool <worker_pool_name_or_ID> --taint dedicated=edge:NoExecute

To apply a taint to individual worker nodes:

oc adm taint node -l dedicated=edge dedicated=edge:NoExecute

Now, only pods with the dedicated=edge toleration are deployed to your edge worker nodes.
2. Verify that your edge nodes are tainted.

oc describe nodes -l dedicated=edge | egrep "Taints|Hostname"

Example output

Taints: dedicated=edge:NoExecute
Additionally, a Kubernetes daemon set is created. This daemon set automatically configures routes on all untainted worker nodes in the cluster to each of the remote subnets. The strongSwan VPN pod uses the routes on worker nodes to forward requests to the remote subnet in the on-premises network.

Routes are not configured on tainted nodes unless you specify the taint in the tolerations setting in the value.yaml file. By tainting worker nodes, you can prevent any VPN routes from being configured on those workers. Then, you can specify the taint in the tolerations setting for only the VPN deployment that you do want to permit on the tainted workers. In this way, the strongSwan VPN pods for one tenant's Helm chart deployment only use the routes on that tenant's worker nodes to forward traffic over the VPN connection to the remote subnet.

Before you use this solution, review the following considerations and limitations.



* By default, Kubernetes places app pods onto any untainted worker nodes that are available. To make sure that this solution works correctly, each tenant must first ensure that they deploy their app pods only to workers that are tainted for the correct tenant. Additionally, each tainted worker node must also have a toleration to allow the app pods to be placed on the node. For more information about taints and tolerations, see the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/).
* Cluster resources might not be optimally utilized because neither tenant can place app pods on the shared non-tainted nodes.



The following steps for limiting strongSwan VPN traffic by worker node use this example scenario: Say that you have a multi-tenant IBM Cloud Kubernetes Service cluster with six worker nodes. The cluster supports tenant A and tenant B. You taint the worker nodes in the following ways.



* Two worker nodes are tainted so that only tenant A pods are scheduled on the workers.
* Two worker nodes are tainted so that only tenant B pods are scheduled on the workers.
* Attempts to perform unauthorized functions or access data that the user is not authorized to access,
* Modification of security rules,
* Application and related systems start-ups and shut-downs



* For privileged accounts:



* Attempts to execute privilege elevation
* System errors relevant to security events, including but not limited to SQL errors that indicate an SQL injection, fuzzing, multiple failed logins, failed configuration change, failed/disabled anti-virus software, service failures
* Web access events in extended log format





The events that are collected should comply with the Cloud Auditing Data Federation (CADF) standard. And, audit events must contain at a minimum:



* Type of event
* When the event occurred
* Where the event occurred
* Source of the event
* Outcome (success or failure) of the event
* Identity of any user/subject/device associated with the event
* Port and protocol, where available
* Session duration, identification, and modification where available
* Authentication method







 Storing events in a SIEM 

A SIEM system provides the ability to gather security data from information system components and presents that data as actionable information through a single interface. Events of the type in the previous section should be stored in a SIEM. You need to install your own software solution within your VPC for SIEM.





 Related IBM Cloud for Financial Services controls 

The following [IBM Cloud Framework for Financial Services controls](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-aboutframework-control-requirements) are most related to this guidance. However, in addition to following the guidance here, do your own due diligence to ensure you meet the requirements.



Table 1. Related controls in IBM Cloud Framework for Financial Services

 Family Control
Combine the identifier mask with the CIDR value to get the full subnet address. In the previous output, the subnet addresses are:



* 169.xx.210.xxx/28
* 169.xx.178.xxx/28





* Individual worker node IP addresses: If you have a small number of worker nodes that run only one app and don't need to scale, or if you want to add only one worker node, list all the worker nodes in your cluster and note the Public IP addresses. If your worker nodes are connected to a private network only and you want to connect to IBM Cloud services by using the private cloud service endpoint, note the Private IP addresses instead. Only these worker nodes are added. If you delete the worker nodes or add worker nodes to the cluster, you must update your allowlist accordingly.

ibmcloud ks worker ls --cluster <cluster_name_or_ID>



3. Add the subnet CIDR or IP addresses to your service's allowlist for outbound traffic or your on-premises allowlist for inbound traffic.
4. Repeat these steps for each cluster that you want to allow traffic to or from.







 Updating IAM allowlists for Kubernetes Service IP addresses 

By default, all IP addresses can be used to log in to the IBM Cloud console and perform actions to manage your cluster, such as creating, updating, deleting or viewing credentials. In the IBM Cloud Identity and Access Management (IAM) console, you can [create an allowlist by specifying which IP addresses have access](https://cloud.ibm.com/docs/account?topic=account-ips), and all other IP addresses are restricted.

In your allowlist, you must also add the CIDRs of the IBM Cloud Kubernetes Service control plane for the zones in the region where your cluster is located so that IBM Cloud Kubernetes Service can create or access components such as Ingress ALBs

Before you begin, the following steps require you to change the IAM allowlist for the user whose credentials are used for the cluster's region and resource group infrastructure permissions. If you are the credentials owner, you can change your own IAM allowlist settings.
Why does my worker node show a NetworkUnavailable error? 

Virtual Private Cloud

Classic infrastructure

Satellite

 What’s happening 

When you update your master or worker nodes, your worker nodes enter a NetworkUnavailable state.

 Why it’s happening 

Your worker nodes might enter a NetworkUnavailable state whenever the calico-node pod has been shut down. This might happen during a Calico patch update, but shouldn't impact your application availability.

When Calico is updated, the node.kubernetes.io/network-unavailable:NoSchedule taint is added to your worker node and the NetworkUnavailable condition becomes True. Both of these conditions are cleared when Calico restarts, which typically takes only a few seconds.

Sometimes, the restart might take longer. In nearly all cases, the restart is fast enough to avoid any worker node network issues. However, there are situations where a Calico restart is delayed and thus, there could be network interruptions. For these cases, the node network unavailable taint and condition are designed to keep new apps from being deployed to the new node until Calico and the node are fixed. Calico updates are rolled out in a very controlled manner so as to minimize overall application impact should there be a node problem.

 How to fix it 

Monitor the NetworkUnavailable state with IBM Cloud Monitoring

By using services to monitor applications such as IBM Cloud Monitoring, you can configure alerts for when a worker node goes into a NetworkUnavailable state, and count each time this happens. You can also configure thresholds and tune your alerts to allow for when worker nodes NetworkUnavailable state during routine Calico patches.

When you set up IBM Cloud Monitoring alerts, take the following scenarios into consideration.



* A NetworkUnavailable alert might become a problem when a calico-node pod fails to achieve a Running state, and its container restart count continues to increase.
* A worker node remains in NetworkUnavailable state for a long amount of time.



After a worker update or replace, sometimes the calico-node pod still does not start on Red Hat OpenShift VPC Cluster.
Kubernetes version 1.21 change log 

Kubernetes version 1.21 is unsupported as of 14 September 2022. Update your cluster to at least [version 1.22](https://cloud.ibm.com/docs/containers?topic=containers-cs_versions_121) as soon as possible.

View information about version changes for major, minor, and patch updates that are available for your IBM Cloud® Kubernetes Service clusters that run version 1.21. Changes include updates to Kubernetes and IBM Cloud Provider components.



 Overview 

Unless otherwise noted in the change logs, the IBM Cloud Kubernetes Service provider version enables Kubernetes APIs and features that are at beta. Kubernetes alpha features, which are subject to change, are disabled.

For more information about major, minor, and patch versions and preparation actions between minor versions, see [Kubernetes versions](https://cloud.ibm.com/docs/containers?topic=containers-cs_versions).

Check the [Security Bulletins on IBM Cloud Status](https://cloud.ibm.com/status?component=containers-kubernetes&selected=security) for security vulnerabilities that affect IBM Cloud Kubernetes Service. You can filter the results to view only Kubernetes Cluster security bulletins that are relevant to IBM Cloud Kubernetes Service. Change log entries that address other security vulnerabilities but don't also refer to an IBM security bulletin are for vulnerabilities that are not known to affect IBM Cloud Kubernetes Service in normal usage. If you run privileged containers, run commands on the workers, or execute untrusted code, then you might be at risk.

Some change logs are for worker node fix packs, and apply only to worker nodes. You must [apply these patches](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_update) to ensure security compliance for your worker nodes. These worker node fix packs can be at a higher version than the master because some build fix packs are specific to worker nodes.
ibmcloud ks zone add classic --zone ZONE --cluster CLUSTER --worker-pool WORKER-POOL --private-vlan PRIVATE-VLAN-ID --public-vlan PUBLIC-VLAN-ID
* VPC clusters:

ibmcloud ks zone add vpc-gen2 --zone ZONE --cluster CLUSTER --worker-pool WORKER-POOL --subnet-id VPC-SUBNET-ID





4. Wait for the worker nodes to be deployed. When the worker node state changes to Normal, the deployment is finished.

ibmcloud ks worker ls --cluster CLUSTER
5. Remove the old worker node. Note: If you are removing a flavor that is billed monthly (such as bare metal), you are charged for the entire the month.



1. Remove the worker pool with the old machine type. Removing a worker pool removes all worker nodes in the pool in all zones. This process might take a few minutes to complete.

ibmcloud ks worker-pool rm --worker-pool WORKER-POOL --cluster CLUSTER
2. Verify that the worker pool is removed.

ibmcloud ks worker-pool ls --cluster CLUSTER



6. Verify that the worker nodes are removed from your cluster.

ibmcloud ks worker ls --cluster CLUSTER
7. Repeat these steps to update other worker pools or stand-alone worker nodes to different flavors.







 Updating cluster components 

Your IBM Cloud Kubernetes Service cluster comes with components, such as Ingress, that are installed automatically when you provision the cluster. By default, these components are updated automatically by IBM. However, you can disable automatic updates for some components and manually update them separately from the master and worker nodes.

What default components can I update separately from the cluster?
: You can optionally disable automatic updates for the following components:



* [Fluentd for logging](https://cloud.ibm.com/docs/containers?topic=containers-updatelogging-up)
* [Ingress application load balancer (ALB)](https://cloud.ibm.com/docs/containers?topic=containers-updatealb)
You might receive an error message in your worker node status similar to the following example.

incorrect account for worker - The 'classic' infrastructure user credentials changed and no longer match the worker node instance infrastructure account.

 Why it’s happening 

The cluster might be provisioned in a classic IBM Cloud infrastructure account that is no longer linked to your Red Hat OpenShift on IBM Cloud account. The cluster is orphaned. Because the resources are in a different account, you don't have the infrastructure credentials to modify the resources.

Consider the following example scenario to understand how clusters might become orphaned.



1. You have an IBM Cloud Pay-As-You-Go account.
2. You create a cluster named Cluster1. The worker nodes and other infrastructure resources are provisioned into the infrastructure account that comes with your Pay-As-You-Go account.
3. Later, you find out that your team uses a legacy or shared classic IBM Cloud infrastructure account. You use the ibmcloud oc credential set command to change the IBM Cloud infrastructure credentials to use your team account.
4. You create another cluster named Cluster2. The worker nodes and other infrastructure resources are provisioned into the team infrastructure account.
5. You notice that Cluster1 needs a worker node update, a worker node reload, or you just want to clean it up by deleting it. However, because Cluster1 was provisioned into a different infrastructure account, you can't modify its infrastructure resources. Cluster1 is orphaned.
6. You follow the resolution steps in the following section, but don't set your infrastructure credentials back to your team account. You can delete Cluster1, but now Cluster2 is orphaned.
7. You change your infrastructure credentials back to the team account that created Cluster2. Now, you no longer have an orphaned cluster!



 How to fix it 

Follow the steps to review your infrastructure credentials and determine why you are seeing the credentials error.



1. Log in to the [Red Hat OpenShift clusters console](https://cloud.ibm.com/kubernetes/clusters?platformType=openshift).
2.
After you deploy your network, you can use the Fabric SDKs, the Peer CLI, or the console UI to create channels and deploy smart contracts. If you need to programmatically create a channel, you must provide the consortium name. For IBM® Blockchain Platform, the consortium name must be set to SampleConsortium.



The service credential that is used for API authentication must have the Manager role in IAM to be able to create components. See the table on [user roles](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-manage-consoleibp-console-manage-console-role-mapping) for more information.



 Creating a node within a specific zone 

If you are using a multizone cluster, you can use the APIs to deploy a blockchain component to a specific zone of IBM Cloud. This allows your network to maintain availability in the event of a zone failure. You can use the following steps to deploy a peer or ordering node to a specific zone.



1. Find the zones that your worker nodes are located. Navigate to the [Kubernetes service](https://cloud.ibm.com/kubernetes/clusters) or [OpenShift](https://cloud.ibm.com/kubernetes/clusters?platformType=openshift) cluster overview screen of your multizone cluster on IBM Cloud. From the cluster overview screen, select your cluster and click Worker Nodes to see a table of all the worker nodes in your cluster. You can find the zone where each worker node is located in the Zone column of the table.

You can also find the zones of your worker nodes by using the kubectl CLI. Navigate to the Access panel and follow the instructions under Gain access to your cluster to connect to your cluster by using the IBM Cloud and kubectl CLI tools. When you are connected, use the command kubectl get nodes --show-labels to get the full list of nodes and zones of your cluster. You will be to find the zone that each worker node is located after zone field under the LABELS column.





1.
The source IP address of client requests is preserved by default in version 2.0 NLBs.

When a client request to your app is sent to your cluster, a load balancer service pod receives the request. If no app pod exists on the same worker node as the load balancer service pod, the NLB forwards the request to a different worker node. The source IP address of the package is changed to the public IP address of the worker node where the load balancer service pod runs.

To preserve the original source IP address of the client request, you can [enable source IP](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/preserving-the-client-source-ip) for load balancer services. The TCP connection continues all the way to the app pods so that the app can see the actual source IP address of the initiator. Preserving the client’s IP is useful, for example, when app servers have to apply security and access-control policies.

After you enable the source IP, load balancer service pods must forward requests to app pods that are deployed to the same worker node only. Typically, load balancer service pods are also deployed to the worker nodes that the app pods are deployed to. However, some situations exist where the load balancer pods and app pods might not be scheduled onto the same worker node:



* You have edge nodes that are tainted so that only load balancer service pods can deploy to them. App pods are not permitted to deploy to those nodes.
* Your cluster is connected to multiple public or private VLANs, and your app pods might deploy to worker nodes that are connected only to one VLAN. Load balancer service pods might not deploy to those worker nodes because the NLB IP address is connected to a different VLAN than the worker nodes.



To force your app to deploy to specific worker nodes where load balancer service pods can also deploy to, you must add affinity rules and tolerations to your app deployment.



 Adding edge node affinity rules and tolerations
Review the requirements to [install Portworx](https://docs.portworx.com/install-portworx/prerequisites/).

For production environments, choose one of the [SDS worker node flavors](https://cloud.ibm.com/docs/openshift?topic=openshift-planning_worker_nodessds-table) for best performance.





 How can I make sure that my data is stored highly available? 

You need at least three worker nodes in your Portworx cluster so that Portworx can replicate your data across nodes. By replicating your data across worker nodes, Portworx can ensure that your stateful app can be rescheduled to a different worker node in case of a failure without losing data. For even higher availability, use a [multizone cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-ha_clustersmz-clusters) and replicate your volumes across worker nodes in 3 or more zones.





 What volume topology offers the best performance for my pods? 

One of the biggest challenges when you run stateful apps in a cluster is to make sure that your container can be rescheduled onto another host if the container or the entire host fails. In Docker, when a container must be rescheduled onto a different host, the volume does not move to the new host. Portworx can be configured to run hyper-converged to ensure that your compute resources and the storage are always placed onto the same worker node. When your app must be rescheduled, Portworx moves your app to a worker node where one of your volume replicas resides to ensure local-disk access speed and best performance for your stateful app. Running hyper-converged offers the best performance for your pods, but requires storage to be available on all worker nodes in your cluster.

You can also choose to use only a subset of worker nodes for your Portworx storage layer. For example, you might have a worker pool with SDS worker nodes that come with local raw block storage, and another worker pool with virtual worker nodes that don't come with local storage.