vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Then, label these worker nodes so that the local storage drivers are installed on only these worker nodes.



1. [Create a Satellite location](https://cloud.ibm.com/docs/satellite?topic=satellite-locations).
2. [Set up Satellite Config](https://cloud.ibm.com/docs/satellite?topic=satellite-setup-clusters-satconfig).
3. If you do not have any clusters in your location, [create a Red Hat OpenShift on IBM Cloud cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-satellite-clusters) or [attach existing Red Hat OpenShift on IBM Cloud clusters to your location](https://cloud.ibm.com/docs/satellite?topic=satellite-register-openshift-clusters). Ensure that the worker nodes in your cluster that you want to use in your storage configuration have at least one available local disk in addition to the disks required by Satellite. The extra disks must be unformatted.
4. [Get the device details of your worker nodes](https://cloud.ibm.com/docs/satellite?topic=satellite-storage-local-volume-filesat-storage-file-local-devices).
5. [Label the worker nodes](https://cloud.ibm.com/docs/satellite?topic=satellite-storage-local-volume-filesat-storage-file-local-labels) that have an available disk and that you want to use in your configuration. The local storage drivers are installed only on the labeled worker nodes.





 Getting the device details for your local file storage configuration 

When you create your file storage configuration, you must specify which devices that you want to use. The device paths that you retrieve in the following steps are specified as parameters when you create your configuration.



1. Log in to your cluster and get a list of available worker nodes. Make a note of the worker nodes that you want to use in your configuration.

oc get nodes
2. Log in to each worker node that you want to use for your local storage configuration.
Verify that you can access the public IP address the worker node through the node port. Note: Because worker nodes in VPC clusters don't have a public IP address, you can access an app through a NodePort only if you are connected to your private VPC network, such as through a VPN connection. Then, you can use the worker node's private IP address and NodePort: <worker_private_IP>:<NodePort>.

curl --connect-timeout 10 <worker_IP>:<NodePort>

The following example output confirms that the request to your app came through the private IP address 10.1.1.1 for the worker node and the 31024 node port. The webserver-855556f688-xd849 app pod received the curl request:

Hostname: webserver-855556f688-xd849
Pod Information:
-no pod information available-
Server values:
server_version=nginx: 1.13.3 - lua: 10008
Request Information:
client_address=1.1.1.1
method=GET
real path=/
query=
request_version=1.1
request_scheme=http
request_uri=http://10.1.1.1:8080/
Request Headers:
accept=/
host=10.1.1.1:31024
user-agent=curl/7.60.0
Request Body:
-no body in request-





At this point, your app is exposed from multiple IP addresses and ports. Most of these IPs are internal to the cluster and can be accessed only over the private network. Only the public node port and public NLB port are exposed to the public internet.

Next, you can start creating and applying Calico policies to block public traffic.





 Step 2: Block all incoming traffic to all node ports 

To secure the PR firm's cluster, you must block public access to both the NLB service and node ports that are exposing your app. Start by blocking access to node ports.
You can assign a worker pool a label when you [create the worker pool](https://cloud.ibm.com/docs/containers?topic=containers-add_workersadd_pool), or later by following these steps. After a worker pool is labeled, all existing and subsequent worker nodes get this label. You might use labels to deploy specific workloads only to worker nodes in the worker pool, such as [edge nodes for load balancer network traffic](https://cloud.ibm.com/docs/containers?topic=containers-edge).

Do not include personal information in your labels. Learn more about [securing your personal information](https://cloud.ibm.com/docs/containers?topic=containers-securitypi) when you work with Kubernetes resources.

Before you begin: [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster)



1. List the worker pools in your cluster.

ibmcloud ks worker-pool ls --cluster <cluster_name_or_ID>
2. List the existing custom labels on worker nodes in the worker pool that you want to label.

ibmcloud ks worker-pool get --cluster <cluster_name_or_ID> --worker-pool <pool>
3. Label the worker pool with a key=value label. When you set a worker pool label, all the existing custom labels are replaced. To keep any existing custom labels on the worker pool, include those labels with this option.

You can also rename an existing label by assigning the same key a new value. However, don't modify the worker pool or worker node labels that are provided by default because these labels are required for worker pools to function properly. Modify only custom labels that you previously added.

ibmcloud ks worker-pool label set --cluster <cluster_name_or_ID> --worker-pool <worker_pool_name_or_ID> --label <key=value>
Optional: To encrypt the local disk of each worker node in the worker pool, get the details of your key management service (KMS) provider. If you want to use a KMS instance from a different account, you need to do these steps under the account where the KMS instance resides.



1. Complete the steps in [VPC worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-encryptionworker-encryption-vpc) to create your KMS instance and to authorize your service in IAM.
2. List available KMS instances and note the ID.

ibmcloud ks kms instance ls
3. List the available root keys for the KMS instance that you want to use and note the ID.

ibmcloud ks kms crk ls --instance-id <KMS_instance_ID>



5. Create a worker pool. Include the --label option to automatically label worker nodes that are in the pool with the label key=value. Include the --vpc-id option if the worker pool is the first in the cluster. Optionally include the --kms-instance and --crk options with the values you previously retrieved and if the KMS instance resides in a different account, include the --kms-account-id option as well. To attach additional security groups to the workers in the worker pool, [specify the security group IDs with the --security-group option](https://cloud.ibm.com/docs/containers?topic=containers-vpc-security-groupvpc-sg-worker-pool). For more options, see the [CLI documentation](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clicli_worker_pool_create_vpc_gen2). Note that the new worker nodes run the same major.minor version as the cluster master, but the latest worker node patch of that major.minor version. If you want to create your worker pool on dedicated hosts, make sure to specify the --dedicated-host-pool option.
[Deploying OpenShift Data Foundation on Satellite clusters with Azure worker nodes](https://cloud.ibm.com/docs/satellite?topic=satellite-odf-tutorialodf-tutorial)



* [Objectives](https://cloud.ibm.com/docs/satellite?topic=satellite-odf-tutorialodf-tutorial-objectives)
* [Audience](https://cloud.ibm.com/docs/satellite?topic=satellite-odf-tutorialodf-tutorial-audience)
* [Deploy the Azure location template](https://cloud.ibm.com/docs/satellite?topic=satellite-odf-tutorialodf-tutorial-deploying-azure-location)
* [Create a Satellite cluster that uses your Azure hosts](https://cloud.ibm.com/docs/satellite?topic=satellite-odf-tutorialodf-tutorial-deploying-azure-cluster)
* [Deploy the Azure Disk CSI driver](https://cloud.ibm.com/docs/satellite?topic=satellite-odf-tutorialodf-tutorial-deploying-azure)
* [Deploy OpenShift Data Foundation](https://cloud.ibm.com/docs/satellite?topic=satellite-odf-tutorialodf-tutorial-deploying-odf)
* [Verify your deployment](https://cloud.ibm.com/docs/satellite?topic=satellite-odf-tutorialodf-tutorial-verify)
* [Next Steps](https://cloud.ibm.com/docs/satellite?topic=satellite-odf-tutorialodf-tutorial-next-steps)







 Attaching bare metal hosts to a location 

[Attaching bare metal hosts to a location](https://cloud.ibm.com/docs/satellite?topic=satellite-assign-bare-metalassign-bare-metal)



* [Prerequisites](https://cloud.ibm.com/docs/satellite?topic=satellite-assign-bare-metalassign-bare-metal-prereq)
Deploying OpenShift Data Foundation on Satellite clusters with Azure worker nodes 



 Objectives 

In this tutorial, you complete the following tasks to set up OpenShift Data Foundation.



* A Satellite Location that uses Azure hosts.
* A Satellite cluster that uses your Azure hosts as worker nodes.
* A Satellite storage configuration that deploys the Azure Disk CSI driver to your cluster.
* A Satellite storage configuration that deploys OpenShift Data Foundation to your cluster.







 Audience 

This tutorial is for location administrators who are using OpenShift Data Foundation and Satellite storage config for the first time.





 Step 1: Deploy the Azure location template 



1. [Verify you have the required permissions in your Azure account](https://cloud.ibm.com/docs/satellite?topic=satellite-azure&interface=uiinfra-creds-azure).
2. [Follow the steps to deploy an Azure location by using the Schematics template](https://cloud.ibm.com/docs/satellite?topic=satellite-azure&interface=uiazure-template). Make sure to deploy hosts across 3 zones and select a VM size that has at least 16CPUs and 64GB RAM.







 Step 2: Create a Satellite cluster that uses your Azure hosts 

After setting up your location, you have unassigned hosts that can be used to create a cluster.



1. [Follow the steps to deploy a cluster to your location](https://cloud.ibm.com/docs/openshift?topic=openshift-satellite-clusterssatcluster-create-console). Make sure to select the Enable admin access for Satellite config option.
2. Wait until your cluster finishes deploying then deploy the Azure Disk CSI driver.







 Step 3: Deploy the Azure Disk CSI driver 



1. [Follow the steps to create an assign an Azure Disk storage configuration](https://cloud.ibm.com/docs/satellite?topic=satellite-storage-azuredisk-csi-driver&interface=uiazuredisk-csi-driver-config-create-console) to your cluster.
2.
Built artifacts in the Inventory are [promoted](https://cloud.ibm.com/docs/devsecops?topic=devsecops-cd-devsecops-inventory) to deployment environments, like staging or pre-production.
4. Change management automation uses data from the inventory, the evidence locker, and the promotion PR to create the change request deployments. Change management automaton also leaves evidence of acceptance tests, for example. Successfully deployed and tested artifacts are further promoted to production environments.



Every deployment to every environment and region must file a change request on the Change Management System. Change management automation helps you to create these change requests based on all the evidence and information that is collected from the pipelines.

For more information, see [Automating change management](https://cloud.ibm.com/docs/devsecops?topic=devsecops-cd-devsecops-automate-changemgmt).





 Change management command order 

The sequence of change request steps is as follows:



 Create change request 

Everything that changes the baseline must be tracked by using a change request. The changes include, for example, updates to the existing code level, changes to the configuration, and updates of the worker nodes. Collecting peer review compliance data is based on the data that is accessible in the inventory, the evidence locker, and the incident issue repository.

Finally, this step creates the change request that is based on the Promotion PR fields, and attaches available compliance data. Deployment readiness is calculated by the collected compliance status, based on the available evidence.





 Request for approval 

If the created change request deployment state is not ready, this step requests approval for the change.





 Check for approval 

If every compliance check (for example, unit test, CRA tasks, branch protection, detect secrets) is successful, the change request is approved automatically, and the task runs successfully.

If a compliance check fails, the change request state is not approved.

You can approve the change request manually and add the change-request-id to the environment properties to use the already created change request in the next run.

Another solution is to use the emergency label in the promotion pull requests.
* stdout and stderr are automatically collected and forwarded from all containers. Log data includes application logs and worker logs.
* By default, the Log Analysis agent pod that runs on a worker collects logs from all namespaces on that node.



After the agent is configured logs from this cluster will be visible in the Log Analysis web UI covered in the next section. If after a period of time you cannot see logs, check the agent logs.

To check the logs that are generated by a Log Analysis agent, run the following command:

oc logs logdna-agent-<ID> -n ibm-observe

Where ID is the ID for a Log Analysis agent pod.

For example,

oc logs logdna-agent-mdgdz -n ibm-observe





 Launch the Log Analysis webUI 

Launch the web UI within the context of a Log Analysis instance, from the IBM Cloud UI.



1. Navigate to [Red Hat OpenShift on IBM Cloud clusters](https://cloud.ibm.com/kubernetes/clusters?platformType=openshift)
2. Click on your cluster and verify the Overview tab on the left is selected
3. Next to Logging, click the Launch button.



The Log Analysis UI should open in a new tab.





 Create a custom view 

In Log Analysis, you can configure custom views to monitor a subset of data. You can also attach an alert to a view to be notified of the presence or absence of log lines.

In the Log Analysis web UI notice the log entries are displayed with a predefined format. You can modify in the User Preferences section how the information in each log line is displayed. You can also filter logs and modify search settings, then bookmark the result as a view. You can attach and detach one or more alerts to a view. You can define a custom format for how your lines are shown in the view. You can expand a log line and see the data parsed.





 Simulate Load on the Application
When you update worker nodes in [VPC](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_updateportworx_vpc_up) clusters, you must take additional steps to re-attach your Portworx volumes. You can attach your storage volumes by using the API or CLI.



* [Attaching Block Storage for VPC volumes with the CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_storage_att_cr).
* [Attaching Block Storage for VPC volumes with the API](https://cloud.ibm.com/docs/openshift?topic=openshift-utilitiesvpc_api_attach).



2. To remove a Portworx volume, storage node, or the entire Portworx cluster, see [Portworx cleanup](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_removing).









 Creating a secret to store the KMS credentials 

Before you begin:[Set up encryption](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_encryption)



1. Encode the credentials that you retrieved in the previous section to base64 and note all the base64 encoded values. Repeat this command for each parameter to retrieve the base64 encoded value.

echo -n "<value>" | base64
2. Create a project in your cluster called portworx.

oc create ns portworx
3. Create a Kubernetes secret named px-ibm in the portworx project of your cluster to store your IBM Key Protect information.



1. Create a configuration file for your Kubernetes secret with the following content.

apiVersion: v1
kind: Secret
metadata:
name: px-ibm
namespace: portworx
type: Opaque
data:
f8b95e43-a408-4dc8-a489-ed649fc4cfec pgw-18a3ebb0-b539-11e9-9838-f3f4efa02374 available 169.XX.XXX.XX prod us-south-1
2ba9a280-fffa-4b0c-bdca-7970f09f9b8a pgw-73b62bc0-b53a-11e9-9838-f3f4efa02374 available 169.XX.XXX.XX prod us-south-2
057ddef6-631f-4b22-89eb-1e99982a54fa pgw-64c5cae0-0be2-11ea-8f26-e1565e79a36c available 52.XX.XXX.XXX prod us-south-3
4. Add the public gateway IP addresses to your service's allowlist or your on-premises allowlist for inbound traffic.
5. Repeat these steps for each cluster that you want to allow traffic to or from.







 Allowing egress to a cluster from another service 

To permit egress to your cluster from another service, modify that service's allowlist or your on-premises allowlist.



1. Get the worker node subnets or the worker node IP addresses.



* Worker node subnet CIDRs: If you anticipate changing the number of worker nodes in your cluster frequently, such as if you enable the [cluster autoscaler](https://cloud.ibm.com/docs/containers?topic=containers-cluster-scaling-classic-vpc), you might not want to update your allowlist for each new worker node. Instead, you can add the VPC subnets that the cluster uses. Keep in mind that the VPC subnet might be shared by worker nodes in other clusters.



1. Get the Worker Zones and VPCs that your cluster is created in.

ibmcloud ks cluster get -c <cluster>

Example output

...
For each zone, review the [available flavors for worker nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-planning_worker_nodesvm).

ibmcloud oc flavors --zone <ZONE> --provider vpc-gen2
4. Optional: To encrypt the local disk of each worker node in the worker pool, get the details of your key management service (KMS) provider. If you want to use a KMS instance from a different account, you need to do these steps under the account where the KMS instance resides.



1. Complete the steps in [VPC worker nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-encryptionworker-encryption-vpc) to create your KMS instance and to authorize your service in IAM.
2. List available KMS instances and note the ID.

ibmcloud oc kms instance ls
3. List the available root keys for the KMS instance that you want to use and note the ID.

ibmcloud oc kms crk ls --instance-id <KMS_instance_ID>



5. Create a worker pool. Include the --label option to automatically label worker nodes that are in the pool with the label key=value. Include the --vpc-id option if the worker pool is the first in the cluster. Optionally include the --kms-instance and --crk options with the values you previously retrieved and if the KMS instance resides in a different account, include the --kms-account-id option as well. To attach additional security groups to the workers in the worker pool, [specify the security group IDs with the --security-group option](https://cloud.ibm.com/docs/containers?topic=containers-vpc-security-groupvpc-sg-worker-pool).
NodeLocal DNS cache is generally available in clusters that run Kubernetes 1.18 or later, but still disabled by default.



 Enable NodeLocal DNS cache 

Enable NodeLocal DNS cache for one or more worker nodes in your Kubernetes cluster.

The following steps update DNS pods that run on particular worker nodes. You can also [label the worker pool](https://cloud.ibm.com/docs/containers?topic=containers-add_workersworker_pool_labels) so that future nodes inherit the label.

Before you begin, update any [DNS egress network policies](https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dns/nodelocaldnsnetwork-policy-and-dns-connectivity) that are impacted by this feature, such as policies that rely on pod or namespace selectors for DNS egress.

kubectl get networkpolicy --all-namespaces -o yaml



1. [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster)
2. If you [customized stub domains and upstream DNS servers for CoreDNS](https://cloud.ibm.com/docs/containers?topic=containers-cluster_dnsdns_customize), you must also [customize the NodeLocal DNS cache](https://cloud.ibm.com/docs/containers?topic=containers-cluster_dnsdns_nodelocal_customize) with these stub domains and upstream DNS servers.
3. List the nodes in your cluster. The NodeLocal DNS caching agent pods are part of a daemon set that run on each node.

kubectl get nodes
4. Add the ibm-cloud.kubernetes.io/node-local-dns-enabled=true label to the worker node.
f8b95e43-a408-4dc8-a489-ed649fc4cfec pgw-18a3ebb0-b539-11e9-9838-f3f4efa02374 available 169.XX.XXX.XX prod us-south-1
2ba9a280-fffa-4b0c-bdca-7970f09f9b8a pgw-73b62bc0-b53a-11e9-9838-f3f4efa02374 available 169.XX.XXX.XX prod us-south-2
057ddef6-631f-4b22-89eb-1e99982a54fa pgw-64c5cae0-0be2-11ea-8f26-e1565e79a36c available 52.XX.XXX.XXX prod us-south-3
4. Add the public gateway IP addresses to your service's allowlist or your on-premises allowlist for inbound traffic.
5. Repeat these steps for each cluster that you want to allow traffic to or from.







 Allowing egress to a cluster from another service 

To permit egress to your cluster from another service, modify that service's allowlist or your on-premises allowlist.



1. Get the worker node subnets or the worker node IP addresses.



* Worker node subnet CIDRs: If you anticipate changing the number of worker nodes in your cluster frequently, such as if you enable the [cluster autoscaler](https://cloud.ibm.com/docs/openshift?topic=openshift-cluster-scaling-classic-vpc), you might not want to update your allowlist for each new worker node. Instead, you can add the VPC subnets that the cluster uses. Keep in mind that the VPC subnet might be shared by worker nodes in other clusters.



1. Get the Worker Zones and VPCs that your cluster is created in.

ibmcloud oc cluster get -c <cluster>

Example output

...
Adding compute profile types for auto scaling 

Complete the following steps to add compute profile types when worker nodes are automatically added by the resource connector.

For users who only need to add multiple profiles, step 1 is sufficient and restart mbatchd is enough, The rest of steps are for submitting a job to a specific by tagging the vm with templateID as a string resource.



1. Add a new template section to the templates list. Review the values for all of the properties to ensure that they map to the correct VPC configuration. Update the file ibmcloudgen2_templates.json in $LSF_ENDIR/resource_connector/ibmcloudgen2/conf. A typical HPC cluster that is installed on IBM Cloud would have this file in /opt/ibm/lsf/conf/resource_connector/ibmcloudgen2/conf. See the following sample content for an example:

{
"templates": [
{
"templateId": "<TemplateID>",
"maxNumber": 200,
"attributes": {
"type": "String", "X86_64"],
"ncores": "Numeric", "1"],
"ncpus": "Numeric", "1"],
"mem": "Numeric", "1024"],
"icgen2host": "Boolean", "1"],
"templateID": "String", "<TemplateID>"]
},
"imageId": "<IMAGE_ID>",
"subnetId": "<SUBNET_ID>",
"vpcId": "<SUBNET_ID>",
"vmType": "cx2-2x4",
"securityGroupIds": "<SECURITY_GROUP_ID>"],
"sshkey_id": "<SSH_KEY_ID>",
"region": "us-south",
"zone": "us-south-1",