vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* [Create a secret to store your IBM Cloud Object Storage service credentials](https://cloud.ibm.com/docs/containers?topic=containers-storage-cos-understandcreate_cos_secret).
* [Decide on the configuration for your IBM Cloud Object Storage](https://cloud.ibm.com/docs/containers?topic=containers-storage_cos_installconfigure_cos).



To deploy a stateful set that uses object storage:



1. Create a configuration file for your stateful set and the service that you use to expose the stateful set. The following examples show how to deploy NGINX as a stateful set with three replicas, each replica with a separate bucket or sharing the same bucket.

Example to create a stateful set with three replicas, with each replica using a separate bucket.

apiVersion: v1
kind: Service
metadata:
name: nginx-v01
namespace: default
labels:
app: nginx-v01 must match spec.template.metadata.labels and spec.selector.matchLabels in stateful set YAML
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx-v01 must match spec.template.metadata.labels and spec.selector.matchLabels in stateful set YAML
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web-v01
namespace: default
spec:
selector:
matchLabels:
app: nginx-v01 must match spec.template.metadata.labels in stateful set YAML and metadata.labels in service YAML
serviceName: "nginx-v01"
replicas: 3
template:
metadata:
labels:
app: nginx-v01 must match spec.selector.matchLabels in stateful set YAML and metadata.labels in service YAML
spec:
terminationGracePeriodSeconds: 10
containers:
- name: nginx
for (final Bucket bucket : bucketList) {
System.out.println(bucket.getName());
}
System.out.println();
}

// Retrieve the list of contents for a bucket
public static void listObjects(String bucketName, AmazonS3 cosClient)
{
System.out.println("Listing objects in bucket " + bucketName);
ObjectListing objectListing = cosClient.listObjects(new ListObjectsRequest().withBucketName(bucketName));
for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
System.out.println(" - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")");
}
System.out.println();
}

// Create file and upload to new bucket
public static void createTextFile(String bucketName, String itemName, String fileText) {
System.out.printf("Creating new item: %sn", itemName);

InputStream newStream = new ByteArrayInputStream(fileText.getBytes(Charset.forName("UTF-8")));

ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(fileText.length());

PutObjectRequest req = new PutObjectRequest(bucketName, itemName, newStream, metadata);
_cosClient.putObject(req);

System.out.printf("Item: %s created!n", itemName);
}

// Delete item
public static void deleteItem(String bucketName, String itemName) {
System.out.printf("Deleting item: %sn", itemName);
_cosClient.deleteObject(bucketName, itemName);
A list of valid codes for LocationConstraint can be referenced in [the Storage Classes guide](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-classesclasses-locationconstraint).





 Create a bucket CORS 

curl -X "PUT" "https://(endpoint)/(bucket-name)/?cors"
-H "Content-MD5: (md5-hash)"
-H "Authorization: bearer (token)"
-H "Content-Type: text/plain; charset=utf-8"
-d "<CORSConfiguration>
<CORSRule>
<AllowedOrigin>(url)</AllowedOrigin>
<AllowedMethod>(request-type)</AllowedMethod>
<AllowedHeader>(url)</AllowedHeader>
</CORSRule>
</CORSConfiguration>"



The Content-MD5 header needs to be the binary representation of a base64-encoded MD5 hash.

echo -n (XML block) | openssl dgst -md5 -binary | openssl enc -base64







 Get a bucket CORS 

curl "https://(endpoint)/(bucket-name)/?cors"
-H "Authorization: bearer (token)"







 Delete a bucket CORS 

curl -X "DELETE" "https://(endpoint)/(bucket-name)/?cors"
-H "Authorization: bearer (token)"







 List objects 

curl "https://(endpoint)/(bucket-name)"
-H "Authorization: bearer (token)"







 Get bucket headers 

curl --head "https://(endpoint)/(bucket-name)/"
-H "Authorization: bearer (token)"







 Get bucket metadata 

Note the use of the config API endpoint isn't the same as the endpoint for your bucket itself. Use of this command returns metadata for the specified bucket.
* You can either use curl commands or an API REST Client such as [Postman](https://cloud.ibm.com/docs/services/cloud-object-storage?topic=cloud-object-storage-postman) to access the [Key Protect API](https://cloud.ibm.com/docs/services/key-protect?topic=key-protect-set-up-apiset-up-api).



3. Retrieve the CRN of the root key you will use to enabled Key Protect on the your bucket. The CRN will look similar to below:



crn:v1:bluemix:public:kms:us-south:a/3d624cd74a0dea86ed8efe3101341742:90b6a1db-0fe1-4fe9-b91e-962c327df531:key:0bg3e33e-a866-50f2-b715-5cba2bc93234





 Creating a bucket with Key Protect enabled 

function createBucketKP(bucketName) {
console.log(Creating new encrypted bucket: ${bucketName});
return cos.createBucket({
Bucket: bucketName,
CreateBucketConfiguration: {
LocationConstraint: '<bucket-location>'
},
IBMSSEKPEncryptionAlgorithm: '<algorithm>',
IBMSSEKPCustomerRootKeyCrn: '<root-key-crn>'
}).promise()
.then((() => {
console.log(Bucket: ${bucketName} created!);
logDone();
}))
.catch(logError);
}
Show more

Key Values



* <bucket-location> - Region or location for your bucket (Key Protect is only available in certain regions.
For example, the ID crn:v1:bluemix:public:cloud-object-storage:global:a/81caa0254631ce5f9330ae427618f209:39d8d161-22c4-4b77-a856-f11db5130d7d:: can be abbreviated to 39d8d161-22c4-4b77-a856-f11db5130d7d.





 List buckets 

curl "https://(endpoint)/"
-H "Authorization: bearer (token)"
-H "ibm-service-instance-id: (resource-instance-id)"







 Add a bucket 

curl -X "PUT" "https://(endpoint)/(bucket-name)"
-H "Authorization: Bearer (token)"
-H "ibm-service-instance-id: (resource-instance-id)"







 Add a bucket (storage class) 

curl -X "PUT" "https://(endpoint)/(bucket-name)"
-H "Content-Type: text/plain; charset=utf-8"
-H "Authorization: Bearer (token)"
-H "ibm-service-instance-id: (resource-instance-id)"
-d "<CreateBucketConfiguration>
<LocationConstraint>(provisioning-code)</LocationConstraint>
</CreateBucketConfiguration>"



A list of valid codes for LocationConstraint can be referenced in [the Storage Classes guide](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-classesclasses-locationconstraint).





 Create a bucket CORS 

curl -X "PUT" "https://(endpoint)/(bucket-name)/?cors"
-H "Content-MD5: (md5-hash)"
-H "Authorization: bearer (token)"
-H "Content-Type: text/plain; charset=utf-8"
Unlike deployments, a stateful set ensures that your pod has a unique network identity that maintains its state across rescheduling. When you want to run workloads in the cloud, try to [design your app to be stateless](https://cloud.ibm.com/docs/containers?topic=containers-strategycloud_workloads) so that your service instances are independent from each other and can fail without a service interruption. However, some apps, such as databases, must be stateful. For those cases, consider to create a stateful set and use [file](https://cloud.ibm.com/docs/containers?topic=containers-file_storagefile_statefulset), [block](https://cloud.ibm.com/docs/containers?topic=containers-block_storageblock_statefulset), or [object](https://cloud.ibm.com/docs/containers?topic=containers-storage_cos_appscos_statefulset) storage as the persistent storage for your stateful set. You can also install [Portworx](https://cloud.ibm.com/docs/containers?topic=containers-storage_portworx_about) on your bare metal worker nodes and use Portworx as a highly available software-defined storage solution to manage persistent storage for your stateful set. 
 [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) Use a daemon set when you must run the same pod on every worker node in your cluster. Pods that are managed by a daemon set are automatically scheduled when a worker node is added to a cluster. Typical use cases include log collectors, such as logstash or prometheus, that collect logs from every worker node to provide insight into the health of a cluster or an app. 
 [Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) A job ensures that one or more pods run successfully to completion.
IBM Cloud Object Storage plug-in version 2.2.2 is available.
: For more information, see the [change log](https://cloud.ibm.com/docs/containers?topic=containers-cos_plugin_changelog).





 7 November 2022 

Worker node fix pack 1.22.15_1577, 1.23.13_1551, 1.24.7_1543, and 1.25.3_1521
: For more information, see the change logs: [1.22.15_1577](https://cloud.ibm.com/docs/containers?topic=containers-changelog_122), [1.23.13_1551](https://cloud.ibm.com/docs/containers?topic=containers-changelog_123), [1.24.7_1543](https://cloud.ibm.com/docs/containers?topic=containers-changelog_124), and [1.25.3_1521](https://cloud.ibm.com/docs/containers?topic=containers-changelog_125).





 3 November 2022 

Changes to the Portworx update process beginning with version 2.12.
: Beginning with version 2.12 Portworx uses an operator-based deployment model instead of the Helm based model use in version 2.11 and earlier. If you want to update from Portworx 2.11 to version 2.12, follow the migration steps in the [Portworx documentation](https://docs.portworx.com/operations/operate-kubernetes/migrate-daemonset/).

Cluster autoscaler add-on version 1.0.7_883.
: For more information, see [Cluster autoscaler add-on change log](https://cloud.ibm.com/docs/containers?topic=containers-ca_changelog).







 October 2022 



 31 October 2022 

Ingress ALB change log updates
* [ibmcloud oc credential unset](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_credentials_unset)



* [infra-permissions commands](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliinfra-commands)



* [ibmcloud oc infra-permissions get](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliinfra_permissions_get)



* [kms commands](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliks_kms)



* [ibmcloud oc kms crk ls](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliks_kms_crk_ls)
* [ibmcloud oc kms enable](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliks_kms_enable)
* [ibmcloud oc kms instance ls](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliks_kms_instance_ls)



* [quota commands](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_quota)



* [ibmcloud oc quota ls](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_quota_ls)
-g RESOURCE_GROUP
: The name of the target resource group. Optional.

--unset-region
: Clear the targeted region.

--unset-resource-group
: Clear the targeted resource group.

If none of the options are specified, the current account and region are displayed.





 Examples 

Set the current account:

ibmcloud target -c MyAccountID

Switch to a new region:

ibmcloud target -r eu-gb

View the current account and region:

ibmcloud target







 ibmcloud update 

Update the CLI to the most recent version:

ibmcloud update [-f]



 Prerequisites 

None.





 Command options 

-f
: Force an update without confirmation. Root privilege is required.







 General classic infrastructure service commands 

Use classic infrastructure commands in the IBM Cloud CLI to configure and manage infrastructure services.

Run the ibmcloud sl command to see the list of available commands:

USAGE:
ibmcloud sl command [arguments...] [options...]

COMMANDS:
block Classic infrastructure Block Storage
call-api Call arbitrary API endpoints
file Classic infrastructure File Storage
dns Classic infrastructure Domain Name System
globalip Classic infrastructure Global IP addresses
hardware Classic infrastructure hardware servers
image Classic infrastructure Compute images
ipsec Classic infrastructure IPSEC VPN
order Classic infrastructure Orders
placement-group Classic infrastructure Placement Group
security Classic infrastructure SSH Keys and SSL Certificates
securitygroup Classic infrastructure network security groups
subnet Classic infrastructure Network subnets
ticket Classic infrastructure Manage Tickets
user Classic infrastructure Manage Users
vlan Classic infrastructure Network VLANs
vs Classic infrastructure Virtual Servers
help Print command usage message

To view help information about a command, run the following command:

ibmcloud sl [command] -h

For detailed information about each command, see the related reference topics in this documentation.

The ibmcloud sl init command is no longer available as of CLI version 0.14.
* [ibmcloud oc credential unset](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_credentials_unset)







 ibmcloud oc infra-permissions 

[View information about infrastructure permissions that allow you to access the IBM Cloud classic infrastructure portfolio through your IBM Cloud account](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliinfra-commands).



* [ibmcloud oc infra-permissions get](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliinfra_permissions_get)







 ibmcloud oc kms 

[View and configure Key Management Service integrations](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliks_kms).



* kms crk: List and configure the root keys for a Key Management Service instance.



* [ibmcloud oc kms crk ls](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliks_kms_crk_ls)



* [ibmcloud oc kms enable](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliks_kms_enable)
* kms instance: View and configure available Key Management Service instances.



* [ibmcloud oc kms instance ls](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cliks_kms_instance_ls)









 ibmcloud oc quota
: If your cluster is on a private VLAN only, a portable private IP address is used.

Example configuration file to create a private NLB 1.0 service that uses a specified IP address on private VLAN 2234945:

apiVersion: v1
kind: Service
metadata:
name: myloadbalancer
annotations:
service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: private
service.kubernetes.io/ibm-load-balancer-cloud-provider-vlan: "2234945"
spec:
type: LoadBalancer
selector:
app: nginx
ports:
- protocol: TCP
port: 8080
targetPort: 8080 Optional. By default, the targetPort is set to match the port value unless specified otherwise.
loadBalancerIP: 172.21.xxx.xxx
Show more
3. Optional: Make your NLB service available to only a limited range of IP addresses by specifying the IPs in the spec.loadBalancerSourceRanges field. loadBalancerSourceRanges is implemented by kube-proxy in your cluster via Iptables rules on worker nodes. For more information, see the [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/).
4. Create the service in your cluster.

kubectl apply -f myloadbalancer.yaml



3. Verify that the NLB service was created successfully. It might take a few minutes for the service to be created and for the app to be available.

kubectl describe service myloadbalancer

Example CLI output:

NAME: myloadbalancer
Namespace: default
Labels: <none>
Selector: app=liberty
Type: LoadBalancer
Location: dal10
IP: 172.21.xxx.xxx
LoadBalancer Ingress: 169.xx.xxx.xxx
Port: <unset> 8080/TCP
Redeploy your app and reference the debug-pvc. Alternatively, you can use the following sample pod.

apiVersion: v1
kind: Pod
metadata:
name: debug-pod
spec:
volumes:
- name: nginx
persistentVolumeClaim:
claimName: cos-debug
containers:
- name: nginx
image: nginx
volumeMounts:
- mountPath: /debug
name: nginx
4. Save the following DaemonSet configuration file as inspectnode-ds.yaml. After you deploy an app that references the PVC that you created earlier, deploy the DaemonSet when the Transport endpoint is not connected. error occurs to gather the logs.

apiVersion: apps/v1
kind: DaemonSet
metadata:
namespace: default
name: ibm-inspectnode
labels:
app: ibm-inspectnode
spec:
selector:
matchLabels:
app: ibm-inspectnode
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
app: ibm-inspectnode
spec:
tolerations:
- operator: "Exists"
hostNetwork: true
containers:
- name: inspectnode
image: alpine:latest
command: ["/bin/sh"]
args: ["-c", "while true; do msg=$(date); echo $msg; sleep 30; done"]
securityContext:
runAsUser: 0
volumeMounts:
- mountPath: /host/var/log
name: host-log
volumes:
- name: host-log
hostPath:
path: /var/log
Show more
5. Create the DaemonSet in your cluster when the Transport endpoint is not connected. error occurs.

oc create -f ./inspectnode-ds.yaml
6.
For details, see [supported Satellite APIs](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-apis-cos-satellite)

Syntax

HEAD https://{endpoint}/{bucket-name}/{object-name} path style
HEAD https://{bucket-name}.{endpoint}/{object-name} virtual host style

Example request

HEAD /apiary/soldier-bee HTTP/1.1
Authorization: Bearer {token}
Host: s3-api.sjc-us-geo.objectstorage.s3.us-south.cloud-object-storage.appdomain.cloud.net

Example request

HEAD /apiary/soldier-bee HTTP/1.1
Authorization: 'AWS4-HMAC-SHA256 Credential={access-key}/{date}/{region}/s3/aws4_request,SignedHeaders=host;x-amz-date;,Signature={signature}'
x-amz-date: {timestamp}
Host: s3.us.cloud-object-storage.appdomain.cloud

Example response

HTTP/1.1 200 OK
Date: Thu, 25 Aug 2016 18:32:44 GMT
X-Clv-Request-Id: da214d69-1999-4461-a130-81ba33c484a6
Accept-Ranges: bytes
Server: Cleversafe/3.9.0.121
X-Clv-S3-Version: 2.5
x-amz-request-id: da214d69-1999-4461-a130-81ba33c484a6
ETag: "37d4c94839ee181a2224d6242176c4b5"
Content-Type: text/plain; charset=UTF-8
Last-Modified: Thu, 25 Aug 2016 17:49:06 GMT
Content-Length: 11
Run the following command to edit the sriov-network-config-daemon DaemonSet in the openshift-sriov-network-operator namespace.

oc edit daemonsets.apps -n openshift-sriov-network-operator sriov-network-config-daemon
7. Change the value of CLUSTER_TYPE from openshift to kubernetes.

- name: NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: CLUSTER_TYPE
value: kubernetes
8. Save and close the DaemonSet.
9. You can now begin [configuring your SRIOV-enabled network device](https://docs.openshift.com/container-platform/4.6/networking/hardware_networks/configuring-sriov-device.html).



The default Calico network interface for pod-to-pod network is always present in all pod networking stacks, including pods that have an SR-IOV network attachment. This means, that in such pods, the Calico network interface is also present next to SR-IOV virtual interfaces. Also, the Calico interface provides the default gateway for IP routing.
Unlike deployments, a stateful set ensures that your pod has a unique network identity that maintains its state across rescheduling. When you want to run workloads in the cloud, try to [design your app to be stateless](https://cloud.ibm.com/docs/openshift?topic=openshift-strategycloud_workloads) so that your service instances are independent from each other and can fail without a service interruption. However, some apps, such as databases, must be stateful. For those cases, consider to create a stateful set and use [file](https://cloud.ibm.com/docs/openshift?topic=openshift-file_storagefile_statefulset), [block](https://cloud.ibm.com/docs/openshift?topic=openshift-block_storageblock_statefulset), or [object](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_cos_appscos_statefulset) storage as the persistent storage for your stateful set. You can also install [Portworx](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_about) on your bare metal worker nodes and use Portworx as a highly available software-defined storage solution to manage persistent storage for your stateful set. 
 [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) Use a daemon set when you must run the same pod on every worker node in your cluster. Pods that are managed by a daemon set are automatically scheduled when a worker node is added to a cluster. Typical use cases include log collectors, such as logstash or prometheus, that collect logs from every worker node to provide insight into the health of a cluster or an app. 
 [Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) A job ensures that one or more pods run successfully to completion.
line string Text of the log line. 
 app string Name of the application that generates the log line. 
 level string Set a value for the level. For example, sample values for this parameter are INFO, WARNING, ERROR. 
 meta This field is reserved for custom information that is associated with a log line. To add metadata to an API call, specify the meta field under the lines object. Metadata can be viewed inside that line's context. 



For example, the following sample shows the JSON for a log line that you want to ingest:

{
"lines": [
{
"timestamp": 2018-11-02T10:53:06+00:00,
"line":"This is my first log line.",
"app":"myapp",
"level": "INFO",
"meta": {
"customfield": {"nestedfield": "nestedvalue"}
}
}
]
}





 Example 

The following sample shows the cURL command to send 1 log line to an instance of the IBM Log Analysis service:

curl "https://logs.us-south.logging.cloud.ibm.com/logs/ingest?hostname=MYHOST&now=$(date +%s)000" -u xxxxxxxxxxxxxxxxxxxxxxx: --header "Content-Type: application/json; charset=UTF-8" -d "{"lines":[{"line":"This is a sample test log statement","timestamp":"2018-11-02T10:53:06+00:00","level":"INFO","app":"myapp"}]}"





 Limits when you send logs 

There are limits when sending logs. For more information see [Limits when sending logs.](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-ingest_limits)
You can subscribe to valuable product notifications at [My Notifications](https://www.ibm.com/support/pages/node/253211).
4. Cloud Optical Repository (COR) is a virtual image that can be deployed and used as a Network File Server (NFS) to perform various IBM i tasks that require media. This virtual optical image includes a collection of the media necessary for various IBM i tasks, for all supported IBM i releases. With the COR image deployed, a second Power Systems Virtual Server Instance can be deployed on the same VLAN that is set up as the client and pointed to the COR (target) NFS Server Instance. For more information on COR images, see [Cloud Optical Repository](https://cloud.ibm.com/media/docs/downloads/power-iaas/Cloud_Optical_Repository.pdf).
NodePort: <unset> 32040/TCP
Endpoints:
Session Affinity: None
External Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal EnsuringLoadBalancer 1m service-controller Ensuring load balancer
Normal EnsuredLoadBalancer 1m service-controller Ensured load balancer
3. Verify that the VPC load balancer is created successfully in your VPC. In the output, verify that the VPC load balancer has a Provision Status of active and an Operating Status of online.

The VPC load balancer is named in the format kube-<cluster_ID>-<kubernetes_lb_service_UID>. To see your cluster ID, run ibmcloud ks cluster get --cluster <cluster_name>. To see the Kubernetes LoadBalancer service UID, run kubectl get svc hw-lb-svc -o yaml and look for the metadata.uid field in the output.

ibmcloud is load-balancers

In the following example CLI output, the VPC load balancer that is named kube-bsaucubd07dhl66e4tgg-1f4f408ce6d2485499bcbdec0fa2d306 is created for the Kubernetes LoadBalancer service:

ID Name Family Subnets Is public Provision status Operating status Resource group
r006-d044af9b-92bf-4047-8f77-a7b86efcb923 kube-bsaucubd07dhl66e4tgg-1f4f408ce6d2485499bcbdec0fa2d306 Application mysubnet-us-south-3 true active online default
4. Send a request to your app by curling the hostname and port of the Kubernetes LoadBalancer service that is assigned by the VPC load balancer that you found in step 2.