vania-janet/multiturn-rag-retrieval-data huggingface dataset:
You can optionally instruct the job to mount a Persistent Volume Claim (PVC) on its pod to use for storing the backup data. To do so, specify the --pvc option when you run the script. The scripts use emptyDir of Kubernetes otherwise.

In most cases, you don't need to use a persistent volume. If you choose to use a persistent volume, the volume must be 3 times as large as the largest backup file in the data store. The size of the data store's backup file depends on usage. After you create a backup, you can [extract files from the archive file](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-backup-restorebackup-unpack) to check the file sizes.

Also, you must have 2 times as much disk space available on the local system as the size of the data store because the archive of the data is split and then recombined to prevent issues that might otherwise occur when you copy large files from the cluster node to the local system.





 Mapping multitenant clusters 

When you restore data that was backed up from a version earlier than 4.0.6 to any later release and the backed-up deployment had more than one instance of the service provisioned, an extra step is required. You must create a JSON file that maps the service instance IDs between the backed-up cluster and the cluster where the data is being restored.

This mapping step is not required if the instance IDs did not change between the back up and restore steps. For example, you can skip this step if you are restoring data to the same cluster where it was backed up from or if you are restoring data to a brand new cluster that has no Discovery instances.

To create a mapping, complete the following steps:



1. Extract the mapping template file from the backup archive file.

tar xf <backup_file_name> tmp/instance_mapping.json -O > <mapping_file_name>
2. Make a list of the names and instance IDs of the service instances that are provisioned to the cluster where the data is being restored.

The instance ID is part of the URL that is specified in the instance summary page.
* The enroll ID and secret of the application identity.
* The connection profile.
* The smart contract name.
* The name of the channel the smart contract was deployed on.





If you are the application developer, use the information that is provided by the network operator to complete following steps:



1. Generate a certificate and private key by using the enroll ID and secret of the application identity, along with CA endpoint information inside your connection profile.
2. Configure a connection by using the Fabric SDK gateway and enable [service discovery](https://cloud.ibm.com/docs/blockchain?topic=blockchain-glossaryglossary-sd).
3. Use the connection profile, channel name, smart contract name, and application keys to invoke the smart contract.



The connection profile that you downloaded from the IBM Blockchain Platform console can be used to connect to your network by using the Node.js (JavaScript and TypeScript), Java, and Go Fabric SDKs.

The application developer can use two programming models to interact with the network, either the High-Level Programming Model APIs or the Low-Level Fabric SDK APIs.



 High-Level Programming Model APIs 

Starting with Fabric v1.4, a simplified application and smart contract programming model, known as the fabric-network API, was introduced, and is now the recommended way to develop applications. The new model reduces the number of steps and amount of code that is required to submit transactions, which are written in Node.js, Java, or Go. The Fabric High-Level Programming Model APIs include the High-level Fabric Gateway SDKs for writing client applications and the High-Level Fabric contract APIs for writing smart contracts. This tutorial focuses on using the High-level Fabric Gateway SDKs.

IBM recommends the High-level Fabric Gateway SDKs which allow client applications to interact with IBM Blockchain Platform networks. These SDKs, available for Node, Java, and Go, allow a client application to invoke smart contracts for the purpose of submitting transactions and evaluating queries. It is recommended that administrative tasks, such as creating channels, deploying smart contracts, are done by using the console, APIs, or Ansible scripts.
To find the management and login node IPs, see the instructions for [Gathering IP addresses](https://cloud.ibm.com/docs/ibm-spectrum-lsf?topic=ibm-spectrum-lsf-accessing-lsf-guigathering-ip-addresses).
2. Open a second terminal and download and install Python 3.6 or newer. The following examples are for Linux and MAC systems. If you are using Windows, the installation is different. For more information, see [Python releases for Windows](https://www.python.org/downloads/windows/).

Run all of the following commands from this second terminal.



1. Install Python 3 (3.6 or newer) and pip3 by running the following commands:

yum install python3
yum install python3-pip
2. Install httplib2 by running the following command:

pip3 install httplib2
3. Install configparser by running the following command:

pip3 install configparser
4. Install urllib3 by running the following command:

pip3 install urllib3



3. Clone the following repository to your local device:

git clone https://github.com/IBMSpectrumComputing/lsf-integrations.git
cd Spectrum/LSF/Application/Center/pacclient/







 Connecting to LSF Application Center with http 

Open a new command-line terminal and run the following commands:

$ ./pacclient.py help
pacclient.py usage:

ping --- Check whether the web service is available
logon --- Log on to IBM Spectrum LSF Application Center
logout --- Log out from IBM Spectrum LSF Application Center
app --- List applications or parameters of an application
submit --- Submit a job
job --- Show information for one or more jobs
jobaction --- Perform a job action on a job
jobdata --- List all the files for a job
download --- Download job data for a job
The monitoring agent collects metrics from the pod where it is installed, and forwards the data to your instance.

In order to provide the full suite of system metrics, the monitoring agent needs to have a privileged status.

To configure your Red Hat OpenShift on IBM Cloud cluster to forward metrics to your IBM Cloud Monitoring instance, complete the following steps from the command-line.



 Set the cluster context and log in to the cluster 

Complete the following steps:



1. Open a terminal to log in to IBM Cloud.

ibmcloud login -a cloud.ibm.com

Select the account where you provisioned the IBM Cloud Monitoring instance.
2. List the clusters to find out in which region and resource group the cluster is available.

ibmcloud oc clusters
3. Set the resource group and region.

ibmcloud target -g RESOURCE_GROUP -r REGION

Where

RESOURCE_GROUP is the name of the resource group where the cluster is available, for example, default.

REGION is the region where the cluster is available, for example, us-south.
4. Set the cluster context in your session.

ibmcloud oc cluster config --cluster <cluster_name_or_ID>
5. Log in to the cluster. Choose a method to login to an OpenShift cluster. [Learn more about the methods to login](https://cloud.ibm.com/docs/openshift?topic=openshift-access_cluster).







 Install the IBM Cloud Monitoring agent in your cluster 



1. Run the following command for your public or private endpoint.

Private endpoints

curl -sL https://ibm.biz/install-sysdig-k8s-agent | bash -s -- -a <MONITORING_ACCESS_KEY> -c ingest.private.us-south.monitoring.cloud.ibm.com -ac 'sysdig_capture_enabled: false' --openshift

Public endpoints
From the edit view of the node with slots, click the Edit slot![Edit slot](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/edit-slot.png) icon for the @sys-number slot.
13. From the Options![More icon](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/kabob.png) menu in the Configure slot 3 header, select Enable conditional responses.
14. In the Found section, add a conditional response by clicking the ![Edit response](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/edit-slot.png) icon, and then add the following condition and response:



Slot 3 conditional response details

 Condition Response Action 

 `@sys-number == 0` Please specify a number that is larger than 0. Clear slot and prompt again 
 `(event.previous_value != null) && (event.previous_value != event.current_value)` Ok, updating the number of guests from ` ` to ` `. Move on 
 `true` Ok. The reservation is for $guests guests. Move on 









 Step 5: Add a confirmation slot 

You might want to design your dialog to call an external reservation system and actually book a reservation for the user in the system. Before your application takes this action, you probably want to confirm with the user that the dialog has understood the details of the reservation correctly. You can do so by adding a confirmation slot to the node.



1. The confirmation slot will expect a Yes or No answer from the user.
Step 1. Setup the Sysdig Helm repository 

Add the Workload Protection Helm repository to your Helm instance.

Complete the following steps:



1. Set the cluster context.

ibmcloud ks cluster config --cluster <CLUSTER_NAME>
2. Add the Helm repository.

helm repo add sysdig https://charts.sysdig.com

If you get the following error:

helm repo add sysdig https://charts.sysdig.com --debug
Error: context deadline exceeded
helm.go:84: [debug] context deadline exceeded

Run the following command and retry adding the Helm repository.

rm $HOME/Library/Preferences/helm/repositories.lock
3. Update the repos to retrieve the latest versions of all Helm charts.

helm repo update
4. List the Helm charts that are currently available for the Sysdig repo.

helm search repo sysdig
5. Verify the Helm chart sysdig/sysdig-deploy is listed.







 Step 2. Create the values yaml file 

Define a yaml file and include the values to deploy the Workload Protection agent and the Secure components that you plan to deploy. For example, name the file agent-values-monitor-secure.yaml.

The following yaml is a template that you can use to configure the Workload Protection agent and the Secure components. You can customize the file by removing or commenting with the sections that are not required for your agent deployment.

global:
clusterConfig:
name: CLUSTER_NAME
sysdig:
accessKey: SERVICE_ACCESS_KEY
kspm:
deploy: true
agent:
image:
registry: icr.io
slim:
enabled: true
image:
repository: ext/sysdig/agent-slim
kmoduleImage:
repository: ext/sysdig/agent-kmodule
collectorSettings:
collectorHost: INGESTION_ENDPOINT
nodeAnalyzer:
logging offer libraries for Node.JS, Python, Rails, Ruby, Go, iOS, Java, and PHP.



For any method that you adopt, you have the flexibility to choose the logging instance where you want to send data per log source. Decide how many instances you might need to collect data from all your log sources based on who can see the data and the type of data that is collected. Avoid sending data to a logging instance that has the platform logs flag enabled.

Whenever a logging agent is available for a type of log source, configure the agent to automatically collect and forward logs from the log source to the logging instance. The agent is the preferred log collection mechanism.

The logging agent authenticates by using the logging Ingestion Key and opens a secure web socket to the IBM Log Analysis ingestion servers; monitors all files with extension .log*, and extensionless files under /var/log/; and can be customized to exclude data that you do not want to collect or to include custom paths that you want to monitor, and more.

For example, you can configure a Kubernetes cluster and an OpenShift cluster with a logging agent. For more information, see [Configuring a logging agent for a standard Kubernetes cluster](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-config_agent_kube_cluster) and [Configuring a logging agent for an OpenShift Kubernetes cluster](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-config_agent_os_cluster).

To configure a logging agent on Linux Ubuntu or Debian, see [Configuring a logging agent on Linux Ubuntu or Debian](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-config_agent_linux).

To send data and attach custom metadata to each log record, you can use the REST API.





 Define the IAM strategy 

Use IBM Cloud® Identity and Access Management (IAM) to securely authenticate users and service IDs, and to control access to all cloud resources and data consistently in the IBM Cloud.
terraform output -json >../config.json







 Step 4: Run your app locally 



1. To connect to the database from your local machine, ensure that you are in your service folder, then install the node dependencies and run the service with the following commands:

npm install

npm run start

If successful, the output shows you are connected:

Connected!
Server is listening on port 8080
2. Open a browser and visit [http://localhost:8080](http://localhost:8080). You are greeted by a welcome page with a database logo that is displayed in your browser window.
3. To test the interface, enter a word and its definition. The data pair is added to the database and appears in a list at the bottom of the page.







 Step 5 (optional): Run the app from a Docker container 

The first step toward hosting your application from a service like [Code Engine](https://www.ibm.com/cloud/code-engine) is to containerize the app code inside a Docker container and run it from there.



1. Make sure you are logged in to your Docker account. In the service folder of your chosen database, enter the following command:

docker build -t database-hello-world:1.0 .
docker run -p 8080:8080 database-hello-world:1.0
2. Open a browser and visit [http://localhost:8080](http://localhost:8080) to see the same welcome page from the [Step 4](https://cloud.ibm.com/docs/databases-for-mysql?topic=databases-for-mysql-create-instance-tutorialstep-4-run-your-app-locally).



Congratulations, you've created an app with a front end that feeds data into your Cloud Databases deployment!
oc create serviceaccount logging-agent -n ibm-observe
3. Grant the serviceaccount access to the Privileged SCC so the service account has permissions to create priviledged logging pods. Run the following command:

oc adm policy add-scc-to-user privileged system:serviceaccount:PROJECT:SERVICEACCOUNT_NAME

Where

PROJECT is the namespace where the logging pods run. Set this value to ibm-observe.

SERVICEACCOUNT_NAME is the name of the service account that you use to deploy the logging agent. Set this value to logging-agent.

oc adm policy add-scc-to-user privileged system:serviceaccount:ibm-observe:logging-agent
4. Add a secret. The secret sets the ingestion key that the logging agent uses to send logs.

oc create secret generic logdna-agent-key --from-literal=logdna-agent-key=INGESTION_KEY -n PROJECT

Where

PROJECT is the namespace where the logging pods run. Set this value to ibm-observe.

INGESTION_KEY is the ingestion key for the logging instance where you plan to forward and collect the cluster logs. To get the ingestion key, see [Get the ingestion key through the IBM Log Analysis with logging UI](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-ingestion_key).







 Step 3. Enable virtual routing and forwarding (VRF) 

This step is required if you plan to use private endpoints.

You must enable virtual routing and forwarding (VRF) and connectivity to service endpoints for your account. [Learn more](https://cloud.ibm.com/docs/account?topic=account-vrf-service-endpoint)





 Step 4. Deploy the logging agent in the cluster 

Create a Kubernetes daemonset to deploy the logging agent on every worker node of your Kubernetes cluster.

The logging agent collects the following logs:



* STDOUT and STDERR
Locate the MSP tile for the organization MSP from step two and click the Export icon.
5. Open the downloaded MSP JSON file in a text editor.
6. Edit the admins element by appending the new base64-encoded certificate string that you generated in the previous section to the end of the list of comma-separated admin certificates. If the identity was enrolled from the console, ensure that the Root Certificate Authority was selected. Open the identity JSON file that you exported, and copy the string from the cert field.
7. Save your changes.
8. In the Organizations tab, open the MSP tile for the peer and click the Settings icon.
9. In the side panel, click Add file and select the updated MSP JSON file.
10. Notice that the Node OU checkbox is selected for you, which means that Node OU support will be enabled on the MSP and that you will not have to repeat this process again in another year when the certificates expire. For now though, you still need to complete all of the steps in this manual renewal process.

Zoom

![Node OU checkbox](https://cloud.ibm.com/docs-content/v1/content/c5288bed34c3820e3a5251d820ee91adcd2591a3/blockchain/images/nodeou-checkbox.png)

Figure 9. Update MSP with Node OU enabled
11. Click Update MSP definition. All Peer and ordering nodes in this console that include this MSP as their node admin are automatically updated with the new MSP and restarted.
12. Export this updated MSP, and in an out of band action, share the file with all of the members of the network who must import it into their console. It is important for them to import the updated MSP, so when they create a new channel, they are using the MSP definition with the latest admin certificate.



To verify that the update worked, open the MSP tile and view the updated expiration date.





 Step three: Associate new admin identity on peer or ordering service
* [Node.js and npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm)







 Step-by-step instructions 



 Step 1: Obtain an API key to deploy infrastructure to your account 

Follow the steps [in this document](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=uicreate_user_key) to create an API key and make a note of it for Step 2.





 Step 2: Clone the repo and cd into the Terraform directory 

git clone https://github.com/IBM-Cloud/truck-tracker.git
cd truck-tracker/terraform

Create a document called terraform.tfvars with the following fields:

ibmcloud_api_key = "<your_api_key_from_step_1>"
region = "eu-gb"
redis_password = "<make_up_a_password>"

The terraform.tfvars document contains variables that you may want to keep secret so it is ignored by the GitHub repository.





 Step 3: Create the infrastructure 

TL;DR — run the Terraform script:

terraform init
terraform apply --auto-approve

The Terraform folder contains a number of simple scripts:



* main.tf tells Terraform to use the IBM Cloud.
* variables.tf contains the variable definitions whose values will be populated from terraform.tfvars.
* cloudant.tf creates a standard Cloudant DB and some credentials that we will use later to access it.
* redis.tf creates the Redis instance and some credentials that we will use later to access it.
* eventstreams.tf creates the Event Streams instance and some credentials that we will use later to access it.
* registry.tf creates the Container Registry that will hold your container images.
* iam.tf creates the access key that is needed to interact with the Container Registry.
This tutorial walks you through the steps for defining the details of a Node-RED Operator in IBM Cloud® Partner Center. You can onboard an Operator bundle from a TGZ file or an Operator from a CSV file. By completing this tutorial, you review and sign the IBM® Digital Provider Agreement, and define your catalog entry and product page, pricing model, and support experience.

Selling on IBM Cloud



* 10 minutes
* 2021-03-31



[Define the product details of your service](https://cloud.ibm.com/docs/sell?topic=sell-svc-define)Define the product details of your service

This tutorial walks you through the steps for defining specific details about your service in IBM Cloud® Partner Center. By completing this tutorial, you review and sign the IBM® Digital Provider Agreement, customize your catalog entry and product page, and define support experience.

Selling on IBM Cloud



* 10 minutes
* 2023-02-01



[Defining the product details of a Terraform template](https://cloud.ibm.com/docs/sell?topic=sell-terraform-template-define)Defining the product details of a Terraform template

This tutorial walks you through the steps to define the details of a Terraform template in IBM Cloud® Partner Center. You can onboard a Terraform template by using a .tgz file. By completing this tutorial, you review and sign the IBM® Digital Provider Agreement, and define your catalog entry, pricing model, and customer support experience.

Selling on IBM Cloud



* 10 minutes
* 2022-03-31



[Defining the product details of a virtual server image](https://cloud.ibm.com/docs/sell?topic=sell-vsimage-define)Defining the product details of a virtual server image

This tutorial walks you through the steps for defining the details of a virtual server image with Terraform in IBM Cloud® Partner Center. By completing this tutorial, you review and sign the IBM® Digital Provider Agreement, and define your catalog entry and product page, pricing model, and support experience.

Selling on IBM Cloud



* 10 minutes
* 2022-03-31
Configuring a CA, peer, or ordering node to use the HSM 

Before attempting these steps you should have:



* Deployed an HSM for your cluster.
* Created a partition and PIN for the slot.
* Deployed the HSM client image or the HSM proxy for your organization.



When a node is configured with HSM, a temporary Kubernetes job is started to run this HSM "enrollment" process. Before configuring a node to use HSM, ensure that you have enough resources in your cluster to support this job that takes approximately 0.1CPU and 100Mi memory.

Then you are ready to deploy a new CA, peer, or ordering node that uses the HSM.

When you deploy a new node from the console, ensure that you select the advanced deployment option Hardware security module (HSM). This option is only available on paid clusters.

If you published an HSM client image and created the HSM configmap, the Use HSM client image toggle is visible. When it is on, you can enter the following values:



* HSM label - Enter the name of the HSM partition to be used for this node.
* HSM PIN - Enter the PIN for the HSM slot.



If you did not create the HSM configmap, an additional field is required:



* HSM proxy endpoint -Enter the URL for the endpoint.



Lastly, on the CA Summary panel, you can override the default HSM configuration, for example if you want to customize which crypto library implementation to use. Click Edit configuration JSON (Advanced) on the Summary panel to view the JSON. Scroll down to the BCCSP (Blockchain Crypto Service Provider) section where you can modify the crypto library settings.

When the node is deployed, a private key for the specified node enroll ID and secret is generated by the HSM and stored securely in the appliance.