vania-janet/multiturn-rag-retrieval-data huggingface dataset:
4. Identify the Persistent Volume (PV) associated with the Persistent Volume Claim (PVC).

oc get pv -L kubernetes.io/hostname | grep localblock | grep Released

Example output

PV_NAME 1490Gi RWO Delete Released openshift-storage/ocs-deviceset-0-data-0-6c5pw localblock 2d22h compute-1
5. If there is a PV in Released state, delete it.

oc delete pv PERSISTENT-VOLUME







 Step 6: Clean up the resources from the old node 

Major

Minor



1. Navigate to the openshift-storage project.

oc project openshift-storage
2. Identify the Persistent Volume (PV) associated with the Persistent Volume Claim (PVC) from old node:

oc get pv -L kubernetes.io/hostname | grep localblock | grep Released

If there is a PV in Released state, delete it:

oc delete pv <persistent_volume>
3. Remove the failed OSD from the cluster. You can specify multiple failed OSDs if required:

oc process -n openshift-storage ocs-osd-removal -p FAILED_OSD_IDS=<failed_osd_id> [-p FORCE_OSD_REMOVAL=true] | oc create -f OSD-ID,OSD-ID, OSD-ID

The FORCE_OSD_REMOVAL value must be changed to true in clusters that have only three OSDs, or clusters with insufficient space to restore all three replicas of the data after the OSD is removed.
4. Verify that the OSD was removed successfully by checking the status of the ocs-osd-removal-job pod.

oc get pod -l job-name=ocs-osd-removal-job -n openshift-storage
5. Verify that the OSD removal is completed.

oc logs -l job-name=ocs-osd-removal-job -n openshift-storage --tail=-1 | egrep -i 'completed removal'
The second regex group (groupIndex 1) matches with the first occurrence of a number group. The third group (groupIndex 2) matches with the first occurrence of a text group after a number group.

{
"context": {
"number_extract": "<? input.text.extract('([\d]+)(\b [A-Za-z]+)',n) ?>"
}
}

When you specify the regex in JSON, you must provide two backslashes (\). If you specify this expression in a node response, you need one backslash only. For example:

<? input.text.extract('([d]+)(b [A-Za-z]+)',n) ?>

Input:

"Hello 123 this is 456".

Result:



* When n=0, the value is 123 this.
* When n=1, the value is 123.
* When n=2, the value is this.







 String.find(String regexp) 

This method returns true if any segment of the string matches the input regular expression. You can call this method against a JSONArray or JSONObject element, and it will convert the array or object to a string before making the comparison.

For this input:

"Hello 123456".

This syntax:

{
"conditions": "input.text.find('^[^d][d]{6}[^d]$')"
}

Result: The condition is true because the numeric portion of the input text matches the regular expression ^[^d][d]{6}[^d]$.





 String.getMatch(String regexp, Integer matchIndex) 

This method returns a string from the input that matches the occurrence of the regular expression pattern that you specify. This method returns an empty string if no match is found.

As matches are found, they are added to what you can think of as a matches array.
Keep in mind that if you create a flow log for a subnet or VPC, each network interface in that subnet or VPC is monitored.



Table 1. Flow log target scopes

 Target What data gets collected 

 VPC Collects data for all network interfaces on a specific VPC. 
 Subnet Collects data for all network interfaces on a specific subnet. 
 Instance Collects data for all network interfaces on a specific virtual server. 
 Interface Collects data for a specific network interface on a specific virtual server. 



During each collector upload interval, there are two flow logs (ingress and egress) per network interface that are written to the designated Object Storage bucket.



 Finest granularity wins 

Each flow log target can have a single flow log collector, which can lead to overlaps. Here are examples, with the finest granularity listed first:



* A virtual server instance interface might have a flow log collector.
* The virtual server instance that the interface is attached to might have a separate flow log collector.
* The subnet that the virtual server instance is attached to might have its own flow log collector.
* The VPC that the subnet is part of might have its own flow log collector.



If an overlap exists, the most targeted flow log collector takes precedence. This precedence is important because each flow log collector might log to a different Object Storage bucket, and where flow log data is stored, can change over the lifetime of a virtual server instance.







 Getting started 

To get started using flow log collectors, follow these steps:



1. Complete any [prerequisites](https://cloud.ibm.com/docs/vpc?topic=vpc-ordering-flow-log-collectorfl-before-you-begin) before ordering IBM Cloud Flow Logs.
2. Decide on the collection scope and [create one or more flow log collectors](https://cloud.ibm.com/docs/vpc?topic=vpc-ordering-flow-log-collector).
3. Review the flow logs that were generated. See [Viewing flow log objects](https://cloud.ibm.com/docs/vpc?topic=vpc-fl-analyze) for details.
"end": 1587772800,
"start": 1587686400
}





 Sample: Extract CPU data 

This example shows how to extract CPU data by host with start, end, and, sampling limit.

The following example of the metrics.json file shows how to configure the file to extract CPU data:

{
"start":"1555404790",
"end":"1555404850",
"sampling": 10,
"metrics": [
{
"id": "host.hostName"
},
{
"id": "cpu.used.percent",
"aggregations": {"time": "avg"}
}
],
"dataSourceType": "host"
}

The result for extracting data returns the following information:

{
"data": [
{
"d":
"j-rhel71.fyre.ibm.com",
3.621
],
"t": 1555404800
},
{
"d":
"j-rhel71.fyre.ibm.com",
3.597
],
"t": 1555404810
},
{
"d":
"j-rhel71.fyre.ibm.com",
3.993
],
"t": 1555404820
},
{
"d":
"j-rhel71.fyre.ibm.com",
3.145
],
"t": 1555404830
},
{
"d":
"j-rhel71.fyre.ibm.com",
4.09
],
"t": 1555404840
},
{
"d":
oc scale deployment rook-ceph-osd-2 --replicas=0 -n openshift-storage
oc scale deployment --selector=app=rook-ceph-crashcollector,node_name=NODE-NAME --replicas=0 -n openshift-storage







 Step 4: Cordon and drain the worker node 

Major

Minor



1. Cordon the node. Cordoning the node prevents any pods from being scheduled on this node.

oc adm cordon NODE_NAME

Example output

node/10.241.0.4 cordoned
2. Drain the node to remove all the pods. When you drain the worker node, the pods move to the other worker nodes ensuring there is no downtime. Draining also ensures that there is no disruption of the pod disruption budget.

oc adm drain NODE_NAME --force --delete-emptydir-data --ignore-daemonsets

Example output

evicting pod "managed-storage-validation-webhooks-7fd79bc9f7-pdpv6"
evicting pod "calico-kube-controllers-647dbbd685-fmrp9"
evicting pod "certified-operators-2v852"
evicting pod "csi-snapshot-controller-77fbf474df-47ddt"
evicting pod "calico-typha-8574d89b8c-7f2cc"
evicting pod "dns-operator-6d48cbff67-vrrsw"
evicting pod "router-default-6fc798b98b-9m6kh"
evicting pod "prometheus-adapter-5b77ffdd5f-hzqrp"
evicting pod "alertmanager-main-1"
evicting pod "prometheus-k8s-0"
evicting pod "network-check-source-66c7fbb86-2r78z"
3.
In the [Virtual server instances](https://cloud.ibm.com/power/servers) view, go to both virtual server instances of the cluster. Verify that all volumes that are in scope for ASCS and ERS are attached to both virtual server instances.

When you attach a new storage volume to a virtual server instance, make sure that you rescan the SCSI bus to detect the new volume. Afterward, update the multipath configuration of the virtual server instance.

On the nodes with new storage volume attachments, run the following command.

rescan-scsi-bus.sh && sleep 10 && multipathd reconfigure

Log in to both cluster nodes, and add the WWN to the environment variables of user root.

The matching WWN values can also be found with the pvs --allcommand.

On NODE1, run the following command.

export ASCS_pvid=3<WWN> WWN of shared storage volume for ASCS

On NODE2, run the following command.

export ERS_pvid=3<WWN> WWN of shared storage volume for ERS

Make sure that you set the environment variable by using the hexadecimal number with lowercase letters.





 Creating physical volumes 

On NODE1, run the following command.

pvcreate /dev/mapper/${ASCS_pvid}

Sample output:

 pvcreate /dev/mapper/${ASCS_pvid}
Physical volume "/dev/mapper/360050768108103357000000000002ddc" successfully created.

On NODE2, run the following command.

pvcreate /dev/mapper/${ERS_pvid}

Sample output:

 pvcreate /dev/mapper/${ERS_pvid}
Physical volume "/dev/mapper/360050768108103357000000000002e31" successfully created.





 Creating volume groups 

Create the volume group for the ASCS.

On NODE1, run the following command.
<RootFolder>E:\ProjectsAnalytics\datatest1</RootFolder>
<ConnectionNumber>9</ConnectionNumber>
<StartTime>1244709336093</StartTime>
<LastTime>1244709385843</LastTime>
<IdlePeriod>219</IdlePeriod>
</SpaceStatus>
<SpaceStatus>
<SpaceId>013</SpaceId>
<RootFolder>E:\ProjectsAnalytics\datatest2</RootFolder>
<ConnectionNumber>10</ConnectionNumber>
<StartTime>1244709336093</StartTime>
<LastTime>1244709385843</LastTime>
<IdlePeriod>219</IdlePeriod>
</SpaceStatus>
Show more







 Connecting to a Windows File System data source 

From your Discovery project, complete the following steps.

If you completed the prerequisite steps, return to the Windows File System data source collection that you started to create, and then skip to Step 4.



1. From the navigation pane, choose Manage collections.
2. Click New collection.
3. Click Windows File System, and then click Next.
4. Name the collection.
5. If the language of the documents that you want to crawl is not English, select the appropriate language.

For a list of supported languages, see [Language support](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-language-support).
6. Optional: Change the synchronization schedule.

For more information, see [Crawl schedule options](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collectionscrawlschedule).
7. In the Enter your credentials section, add values to the following fields.
2022/05/09 12:51:12 Terraform plan | on main.tf line 89, in data "ibm_is_ssh_key" "storage_ssh_key":
2022/05/09 12:51:12 Terraform plan | 89: data "ibm_is_ssh_key" "storage_ssh_key" { This block is trying to fetch the key_pair details for compute node, which is dependent on the code present on public repo
2022/05/09 12:51:12 Terraform plan 
2022/05/09 12:51:12 Terraform plan 
2022/05/09 12:51:12 �[ ! ! ! ! ! ! 1m�31mTerraform PLAN error: Terraform PLAN errorexit status 1�39m�0m
2022/05/09 12:51:12 �1m�31mCould not execute job: Error : Terraform PLAN errorexit status 1�39m�0m
<-- </section "id="section-failed-apply-plan" "> --><-- </section "id="section-retrieve-apply-plan-logs-schematics-workspace" "> --><-- <section "id="section-retrieve-apply-plan-logs-bootstrap-node" "> --> Retrieving apply plan logs from bootstrap node You can view both successful and failed logs from the bootstrap node. Complete the following steps to view the logs.<-- <ol> -->1. Log in to the bootstrap node by using the following command:
ssh -J ubuntu@<IP_address_bastion_host> vpcuser@<IP_address_bootstrap_node>
2. After you log in to the bootstrap node, run the following command to gain root user access:
sudo su
3. Change the directory to /var/log by running the following command:
cd /var/log
4. For a complete stack deployment log, run the following command:
cat stack_deployment.log
[More options](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/kebab.png) icon on the yes node, and then select Add node below.
31. Type true into the If assistant recognizes field of this node.

Do not add a response. Instead, you will redirect users to the branch that asks for the order number details that you created earlier.
32. In the And finally section, choose Jump to.

![Shows a true node that has no response, but has the jump to menu option selected.](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/gs-ass-true-jump-to.png)
33. Select the Ask for order number node's condition.

![Shows choosing the Ask for order number node condition as the jump to target.](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/gs-ass-true-jump-to-destination.png)
34. Click ![Close](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/close.png) to close the edit view.
35. Move the Number provided node before the Ask for order number node. Click the More!
rescan-scsi-bus.sh && sleep 10 && multipathd reconfigure

The WWN value of a volume can also be found with the pvs --all command.





 Preparing environment variables 

To simplify the setup, prepare the following environment variables for user ID root on both nodes. These environment variables are used in subsequent commands in the remainder of this document.

On both nodes, create a file with the environment variables. Then, adapt them to your configuration.

Adapt NFS_vh, NFS_ip, NFS_clientspec, and NFS_options to your environment. For NFS_pvid, use the WWN that you identified previously. In addition to the file system that is used for the NFS share, the example shows two more file systems that are used for an SAP system landscape with system ID ${SID} and the SAP transport directory. The sample sizes ${NFS_sz1}, ${NFS_sz2}, and ${NFS_sz3} are percentages of the ${NFS_vg} volume group size and need to be modified according to your requirements. The volume group names and mount point names are suggestions and need to be changed to match your own naming conventions.

Make sure that you set the NFS_pvid environment variable by using lowercase letters in the hexadecimal number.

 virtual hostnames
export NFS_vh=<virtual hostname> virtual hostname for NFS server
export NFS_ip=<IP address> virtual IP address for NFS server

 LVM storage for NFS file systems
export NFS_pvid=3<WWN> WWN of shareable storage volume used for NFS
export NFS_vg="nfssharevg" volume group name for NFS exported file systems

 NFS share file system
export NFS_lv1="nfssharelv" logical volume name export 1
export NFS_sz1="5%VG" logical volume size
export NFS_fs1="/nfsshare" file system mount point
To get your SSH login command, go to the Schematics > Workspaces page in the IBM Cloud console, and then select your workspace. You can find your SSH login command in the Apply plan log file in the Jobs section of your workspace. The following command is an example:

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@169.63.100.221
2. Install the IBM Cloud container-service plug-in by running the following command:

ibmcloud plugin install container-service
3. Connect to your IBM Cloud account with your API key by running the following command:

ibmcloud login --apikey <your-API-key> -r <region-name>
4. Get your cluster ID by running the following command:

ibmcloud oc cluster ls | grep "your-cluster-prefix-name" | {'print $2'}
5. Copy the cluster ID from the previous step into the following command to generate the config file:

ibmcloud oc cluster config -c <your-cluster-ID> --admin
6. Verify your connection by running the following command:

oc get nodes









 Step 6: Validate NFS server 

Complete the following steps to validate that the NFS storage is set up and exported correctly:



1. Log in to the storage node by using SSH. To get your SSH login command, go to the Schematics > Workspaces page in the IBM Cloud console, and then select your workspace. You can find your SSH login command in the Apply plan log file in the Jobs section of your workspace. The following command is an example:

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -J root@169.63.100.221 root@10.241.4.12
2. The following command and example output shows that the data volume, either /dev/vde or /dev/vdd is mounted to /data on the storage node:
Enter passphrase to use the private key '--admin-priv-key'>
OK

Starting to decrypt key '/Users/hexiaohu/.hpnet-cfg/hpnet-test1-c3-10-f7-a3.priv'....

Connecting to the server hpnet-test1 [52.117.54.147] ...

The connecting server's certificate is valid and already confirmed in the previous command
The certificate can be confirmed again by removing the HPVS instance from file /Users/hexiaohu/.hpnet-cfg/known_instances

Continue connecting to the server...

Deleting the node hpnet-test1 ...

OK
Node hpnet-test1 was deleted.
Show more
* The hpnet connect list command, to get view all configured VPN links on the VPN node:

ibmcloud hpnet connect list --instance "$HPVS_A"

Connecting to the server hpnet-test1 [52.117.54.147] ...

The connecting server's certificate is valid and already confirmed in previous command
The certificate can be confirmed again by removing the HPVS instance from file /Users/hexiaohu/.hpnet-cfg/known_instances

Continue connecting to the server...

Querying connections of hpnet-test1 ...

The connections:
+---------------+-------------+----------+-----------+--------------+
 INSTANCE_NAME TARGET_NAME TYPE STATE SYNCHRONIZED 
+---------------+-------------+----------+-----------+--------------+
 hpnet-test1 hpnet-test2 outgoing connected true
If needed, see the Red Hat Enterprise Linux documentation.



1. Select the required language.
2. Set the date and time.
3. Configure the network and hostname.
4. Select the installation destination.
5. Set the root password.
6. Create a user.







 Registering the Red Hat VM 

For this step, you require your Red Hat subscription details:



* Username
* Password
* Subscription Pool



After the bastion node is deployed, you are required to register and subscribe it with the Red Hat public repositories. From the jump-host or remote device, SSH to connect to the bastion node. Use the su command to get root privileges and run the following commands after you replace the username, password, and pool with your variables.

export rhel_subscription_username=<email address>
export rhel_subscription_password=<password>
sudo subscription-manager register --username=${rhel_subscription_username} --password=${rhel_subscription_password} --force
subscription-manager refresh
subscription-manager attach --pool=<pool>
subscription-manager repos --disable=""
subscription-manager repos --enable rhel-8-server-rpms
subscription-manager repos --enable rhel-8-server-extras-rpms
subscription-manager repos --enable rhel-server-rhscl-8-rpms





 Installing NGINX (HTTP Server) 

The deployment of the Red Hat OpenShift nodes uses Ignition, and this process requires an HTTP Server to be available to download the required configuration. This deployment uses an NGINX instance that runs on the bastion node. To install NGNIX, complete the following steps after you are connected to the bastion node and have root privileges:



1. Use a text editor such as vi to create the following file vi /etc/yum.repos.d/nginx.repo.
2. Type i to insert and paste the following information into the file:

[nginx]
* [Scale down OpenShift Data Foundation](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-storage-update-classicscale-down-odf-classic)
* [Cordon and drain the worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-storage-update-classiccordon-drain-worker-node-classic)
* [Update the worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-storage-update-classicupdate-worker-node-classic)
* [Clean up the resources from the old node](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-storage-update-classiccleanup-os-storage-classic)
* [Add the new storage nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-storage-update-classicadd-storage-node-classic)
* [Update the OpenShift Data Foundation add-on](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-storage-update-classicupdate-ocs-add-on-classic)
* [Update your cluster resource](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-storage-update-classicupdate-ocs-resource-yaml-classic)



[Deploying an app on OpenShift Data Foundation](https://cloud.ibm.com/docs/openshift?topic=openshift-odf-deploy-appodf-deploy-app)

[Managing your OpenShift Data Foundation deployment](https://cloud.ibm.com/docs/openshift?topic=openshift-ocs-manage-deploymentocs-manage-deployment)
In all cases, they pull an image from the container registry and get fed some environment variables with the credentials they require to access external services.





 The Truck Tracker system 

The Truck Tracker system is a set of simple Node.js scripts that use four main packages:



1. [@ibm-cloud/cloudant](https://github.com/IBM/cloudant-node-sdk) to connect to IBM Cloudant and read/write data.
2. [Redis](https://www.npmjs.com/package/redis) to connect to the Redis instance and read/write data.
3. [kafkajs](https://www.npmjs.com/package/kafkajs) to connect to the Event Streams instance.
4. [Express](https://expressjs.com/) to enable a simple web server that allows interaction with the data.



There are five main files:



1. server.js: This runs the web server and communicates with Redis. It is getting regular calls from the front end to the /data route. It connects to Redis, downloads an object with the ID and position of every truck and returns that to the front end.
2. index.html: This is the one and only page of the front end application. When it loads, it shows a map of the US from the awesome [Open Street Map](https://www.openstreetmap.org/copyright) service. Then it is on a five-second loop where it makes an HTTP request to the backend (server.js), retrieves all the trucks and their positions and loops through them to plot their latest position on the map. Each truck has a map marker whose lat/long gets updated with every cycle.
3. producer.js: This takes in a location file and a truck_id from the script invocation and then loops through all the data points in the location file. Each data point is turned into a message with the truck ID and lat/long and posted once a second to Event Streams.
A good place to start monitoring your IBM Blockchain Platform is the Dashboards tab where you can copy and tailor default dashboards according to your monitoring requirements. A dashboard shows groups of metrics that report on the health, performance, and state of your infrastructure, applications, and services for a single host or a group of hosts. Dashboards offer a specialized insight into network data, application data, topology, services, hosts, and containers. Use dashboards to monitor your infrastructure, applications, and services.

The following sets of instructions are based on the default dashboard named Overview by Container that provides a good set of widgets for getting started monitoring the resource allocation of your nodes. Each dashboard includes a set of preconfigured panels. We can copy the dashboard and then customize the panels for the node type that we want to monitor.



 Copy the Overview by Container dashboard 



1. Click the Dashboards tab in the IBM Cloud Monitoring web UI.
2. Under the My Dashboards twistie, click Overview by Container.
3. In the top right corner of the panel, click Options![action menu](https://cloud.ibm.com/docs-content/v1/content/c5288bed34c3820e3a5251d820ee91adcd2591a3/icons/overflow-menu.svg), then Copy Dashboard.
4. Give the Dashboard a name, Blockchain Operations - Peers and click Copy and Open.
5. Now we need to filter the data on the page to only include the peer containers. In the top right corner, click Edit Scope.
6. Click the Everywhere drop down list and specify the filter container.name contains peer.
7. (Optional) If your IBM Cloud account includes multiple Kubernetes clusters, you should also filter by cluster. Add an additional cluster filter, by selecting agent.tag.clusterin and then select your cluster name. Click Save.



Next, we will segment the data in the panels by peer containers.





 Configure panels to view CPU and Memory usage 

Now the data on the dashboard is filtered to only show data from the peers in your cluster.