vania-janet/multiturn-rag-retrieval-data huggingface dataset:
To create an Object Storage bucket, see the [IBM Cloud Object Storage](https://cloud.ibm.com/catalog/services/cloud-object-storage) ordering page.

The Object Storage bucket must be a single-region bucket in the same region as the target resource. Additionally, it is recommended that you secure the bucket through IAM access groups and audit logging.
3. Authorize resources of type Flow Logs for VPC to use the Object Storage instance created in Step 2.

To do so, use the following steps:



* In the IBM Cloud console, click Manage > Access (IAM), then select Authorizations from the navigation pane.
* Click Create and complete the following information:

For Source service:



* Select VPC Infrastructure Services.
* Select Resources based on selected attributes.
* For Resource type, select Flow Logs for VPC.
* For Source service instance, select All instances.



For Target service:



* Select Cloud Object Storage.
* Select Resources based on selected attributes.
* For Service instance, select string equals > All instances.



For Service access, select the Writer role to assign access to the source service that accesses the target service.
* Click Authorize.



For more information, see [Using authorizations to grant access between services](https://cloud.ibm.com/docs/account?topic=account-serviceauthcreate-auth).







 Creating a flow log collector in the UI 

To create a flow log collector by using the IBM Cloud console, follow these steps:



1. Go to the [IBM Cloud console](https://cloud.ibm.com/login) and log in to your account.
2. Select the Menu icon ![Menu icon](https://cloud.ibm.com/docs-content/v1/content/f02b9c9adcbb7328d95a45e105cec371d50f15eb/vpc//images/menu_icon.png), then click VPC Infrastructure > Flow Logs. The Flow logs for VPC dashboard appears.

Zoom

!
Direct endpoint provide better performance and do not incur charges. An example of an endpoint for US-South Cross region is: s3.direct.us.cloud-object-storage.appdomain.cloud







 Inviting users and granting permissions 

If you, as Object Storage administrator, do not want to create and share the service credentials, you can invite users to your Object Storage account and grant them permissions by using access policies on Object Storage buckets. The invited users access your account by using their IBM Cloud® API keys. See [Assigning access to user groups](https://cloud.ibm.com/docs/account?topic=account-access-getstartedgroup_access).



 Structure of the API key 

When you, as user of Object Storage, view an API key in IBM Cloud®, you will notice that it has the following structure:

{
"name": "MyAPIKey",
"description": "",
"createdAt": "2019-07...",
"apiKey": "lipsum999..."
}

If you need an API key to use with Object Storage, see [Creating an API key](https://cloud.ibm.com/docs/account?topic=account-userapikeycreate_user_key).







 Summary 

The following table shows you which access method to the Object Storage service to select for the action you want to carry out.



Object Storage access methods

 Action Type: Service ID Type: API key 

 Create credentials Service administrator Service users 
 Grant access permissions Service administrator Service administrator 
 Select credential type HMAC and IAM API keys IAM API key only 
 Control access for user roles and data buckets Yes Yes
ibm-object-storage-plugin change log 

Review the version history for the ibm-object-storage-plugin Satellite storage template.



 Version 2.2 



 Revision 11, released 04 May 2023 



* Resolves the following CVEs: [CVE-2023-24536](https://nvd.nist.gov/vuln/detail/CVE-2023-24536)[CVE-2023-24537](https://nvd.nist.gov/vuln/detail/CVE-2023-24537)[CVE-2023-24538](https://nvd.nist.gov/vuln/detail/CVE-2023-24538)[CVE-2023-0361](https://nvd.nist.gov/vuln/detail/CVE-2023-0361)
* Updates the UBI to version 8.7-1107.
* Updates Go to version 1.19.8.
* Enable PVC metrics export to node stats







 Revision 10, released 04 April 2023 



* Resolves the following CVEs: [CVE-2022-4304](https://nvd.nist.gov/vuln/detail/CVE-2022-4304)[CVE-2022-4450](https://nvd.nist.gov/vuln/detail/CVE-2022-4450)[CVE-2023-0215](https://nvd.nist.gov/vuln/detail/CVE-2023-0215)[CVE-2023-0286](https://nvd.nist.gov/vuln/detail/CVE-2023-0286)
* Updates the UBI to version 8.7-1085.1679482090.
* Updates Go to version 1.19.7.
* Fix for helm chart installation failing on regions with no Regional COS endpoint







 Revision 9, released 21 March 2023
* Replace <BUCKET> with the name of your IBM Cloud Object Storage bucket.
* Replace <COS_ENDPOINT> with your IBM Cloud Object Storage endpoint associated with the bucket.
* Replace <API_KEY> with the API key required to authenticate with IBM Cloud Object Storage.
* Replace <ID> With the CRN of the IBM Cloud Object Storage instance where the bucket is located.



The response will be similar to the following if the configuration is successfully updated:

{"integration":"ibm","bucket":"my-bucket","endpoint":"s3.private.us-south.cloud-object-storage.appdomain.cloud","apikey":"xxxxxxxxx>","resourceinstanceid":"crn:v1:bluemix:public:cloud-object-storage:global:a/yyyyyyy::"}

If archiving is not configured, the following will be returned when trying to do an update:

{"error":"Active archiving configuration does not exist for this account. Try creating one instead.","code":"ServerError","status":"error"}





 Check the status of a archiving configuration 

To check if archiving is enabled or disabled, you can use either the GET or PUT methods.

When streaming is not enabled, the following is returned when running the [GET method](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-archiving-manage-apiarchivingapi-get-conf):

{"error":"No active archiving configuration exists","code":"NotFound","status":"error"}

When archiving is not enabled, the following is returned when running the [PUT method](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-archiving-manage-apiarchivingapi-update-conf):

{"error":"Active archiving configuration does not exist for this account. Try creating one instead."
Your Object Storage instances are listed under Storage in [the resource list](https://cloud.ibm.com/resources).

The terms 'resource instance' and 'service instance' refer to the same concept, and can be used interchangeably.
2. You will need a bucket before you can store data in your new service instance. To Create a bucket, start by choosing a unique name. All buckets in all regions across the globe share a single namespace. Ensure that you have the [correct permissions](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-iam-bucket-permissions) to create a bucket.

When you name buckets or objects, be sure to avoid the use of Personally Identifiable Information (PII). PII is information that can identify any user (natural person) by name, location, or any other means.
3. First, choose the [level of resiliency](https://cloud.ibm.com/docs/cloud-object-storage/basics?topic=cloud-object-storage-endpoints) you want. Then, choose a location where you would like your data to be physically stored. Resiliency refers to the scope and scale of the geographic area across which your data is distributed. Cross Region resiliency spreads your data across several metropolitan areas, while Regional resiliency spreads data across a single metropolitan area. A Single Data Center distributes data across devices within a single site only.
4. Choose the [bucket's storage class](https://cloud.ibm.com/docs/cloud-object-storage/basics?topic=cloud-object-storage-classes) to accurately reflect how often you expect to read the stored data. This is important as it determines your billing details. Follow the Create link to create and access your new bucket.
5. Determine the advanced configurations, if any, suitable to your content. You can store data by transitioning from any of the storage tiers (Standard, Vault, Cold Vault and Flex) to long-term offline archive or use the online Cold Vault option.
: Version 2.1.17 of the IBM Cloud Object Storage plug-in [is available](https://cloud.ibm.com/docs/openshift?topic=openshift-cos_plugin_changelog).





 27 June 2022 

Block Storage for VPC
: You can now [set up the Block Storage for VPC add-on with Cloud Identity and Access Management trusted profiles](https://cloud.ibm.com/docs/openshift?topic=openshift-storage-block-vpc-trusted-profiles).





 24 June 2022 

Block Storage for VPC
: You can now [create snapshots of your PVCs](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-volume-snapshot) by using version 5.0.0-beta of the Block Storage for VPC add-on.

Block Storage for VPC add-on.
: [Version 4.4.6_1446](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc_bs_changelog) is available.





 22 June 2022 

New! Portworx Cloud Drive support
: You can now dynamically provision storage for Portworx during installation by using Portworx Cloud Drives. For more information, see [About Portworx](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_about).





 20 June 2022 

Worker node fix pack
Make sure the bucket exists in the target region.





 cos_scheme_uri_invalid 

Message: The Cloud Object Storage URI provided is not in the correct format.

This error occurs if the URI has an incorrect scheme. The URI should follow this format: cos://<region>/<bucket>/<sourceFile>





 cos_uri_not_supported 

Message: The Cloud Object Storage URI that was specified is not supported.

A Cloud Object Storage URI cannot specify a single data center, such as ams03. A Cloud Object Storage URI cannot have an image nested within a folder in the file path, for example: cos://us-south/myBucket/MyFolder/myImage.vhd.

The following example shows a Cloud Object Storage URI with a supported format: cos://us-south/myBucket/myImage.vhd.





 cos_image_not_encrypted 

Message: Encryption keys are specified in the image template, but the provided image is not encrypted.

Either remove the encryption keys, or specify an encrypted image in the image template and try again.





 cos_image_encrypted 

Message: No encryption key is specified in the image template, but the provided image is encrypted.

Either specify the encryption keys, or specify an unencrypted image in the image template and try again.





 cos_encrypted_image_not_supported 

Message: The encrypted image feature is not supported in the current configuration.

Verify that the image is not encrypted, or remove the encryption keys in the image template and try again.





 default_address_prefix_not_found 

Message: Default address prefix not found.

You might see this error message if the default address prefix is not found.





 duplicate_error 

Message: The input provided already exists.

The resource specified already exists. Try using a different name for the resource you want to create. For example, when creating a new VPC, you can list existing VPCs by running ibmcloud is vpcs, and choose a name that is not shown.
The actual space that you use in IBM Cloud Object Storage might be different and is billed based on the [pricing table](https://cloud.ibm.com/objectstorage/create). If you enabled quotas when you installed the plug-in , the quota for your bucket is equal to this size.

storageClassName
: Choose between the following options.
: If ibm.io/auto-create-bucket: "true": Enter the storage class that you want to use for your new bucket.
: If ibm.io/auto-create-bucket: "false": Enter the storage class that you used to create your existing bucket.
: If you manually created the bucket in your IBM Cloud Object Storage service instance or you can't remember the storage class that you used, find your service instance in the IBM Cloud dashboard and review the Class and Location of your existing bucket. Then, use the appropriate [storage class](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_cos_reference). The IBM Cloud Object Storage API endpoint that is set in your storage class is based on the region that your cluster is in. If you want to access a bucket that is located in a different region than the one where your cluster is in, you must create a custom storage class and use the appropriate API endpoint for your bucket.

ibm.io/secret-name
: Enter the name of the secret that holds the IBM Cloud Object Storage credentials that you created earlier. If you add your [IBM Cloud Object Storage credentials](https://cloud.ibm.com/docs/openshift?topic=openshift-storage-cos-understandservice_credentials) to the default storage classes, you must not list secrets in the PVC. If you want to integrate Key Protect encryption when creating new buckets from PVCs in your cluster, you must include the root key CRN when creating your [IBM Cloud Object Storage secret](https://cloud.ibm.com/docs/openshift?topic=openshift-storage-cos-understandcreate_cos_secret). Note that you cannot add Key Protect encryption to existing buckets.

secret-namespace
IBM Cloud Object Storage plug-in 

View information for updates to the IBM Cloud Object Storage plug-in in your IBM Cloud® Kubernetes Service clusters.

Refer to the following tables for a summary of changes for each version of the [IBM Cloud Object Storage plug-in](https://cloud.ibm.com/docs/containers?topic=containers-storage-cos-understand).



IBM Cloud Object Storage plug-in versions

 IBM Cloud Object Storage plug-in version Supported? Kubernetes version support Supported architecture 

 2.2.16 Yes Greater than or equal to 1.20 x86 
 2.2.15 Yes Greater than or equal to 1.20 x86 
 2.2.14 Yes Greater than or equal to 1.20 x86 
 2.2.13 Deprecated Greater than or equal to 1.20 x86 
 2.2.12 Deprecated Greater than or equal to 1.20 x86 





 Change log for version 2.2.16, released 3 July 2023 



* Adds support for auto creation and deletion of user defined bucket names.







 Change log for version 2.2.15, released 19 June 2023 



* Updates UBI image to 8.8-860.
* Updates Golang to 1.19.10.
In a browser tab, go to the [list of Object Storage instances](https://cloud.ibm.com/objectstorage). Click on the service name for the tutorial, e.g., secure-file-storage-cos.
2. In a second browser tab, visit the already used Activity Tracker dashboard with the activity logs.
3. Back in the tab with the Object Storage overview, in the list of Buckets, click on the storage bucket.
4. Use Upload to import a file into the bucket. Leave the setting as Standard transfer and use the Upload files (objects) area to select a file. Finish by clicking Upload.
5. Back in the browser tab with the activity logs, there should be CBR-related log records titled Context restriction matched while in report mode. This is due to the fact that the rules have been deployed in report mode. Expand some records to check the reported decision and isEnforced data. Depending on the configured IP range for the homezone, decision might be Permit or Deny. The value for isEnforced should be false because of the reporting mode.



For further testing, you might want to change the IP range of the homezone. It is the variable homezone_iprange in the Schematics settings.





 Remove resources 

To remove the resource, use the browser and navigate to the [IBM Cloud Schematics workspaces overview](https://cloud.ibm.com/schematics/workspaces). Select the workspace, then in the Actions menu, first select Destroy resources, thereafter use Delete workspace.





 Related content 



* Blog post [Towards Zero Trust with Context-Based Restrictions](https://www.ibm.com/cloud/blog/towards-zero-trust-with-context-based-restrictions)
* Blog post [Introducing Context-Based Restrictions](https://www.ibm.com/cloud/blog/announcements/introducing-context-based-restrictions)
* [What is Zero Trust?](https://www.ibm.com/topics/zero-trust)
Name:	host-0.direct.cloud-object-storage.appdomain.cloud
Address: 10.240.64.6

root@test-vpc-vsi: nslookup host-1.direct.cloud-object-storage.appdomain.cloud
Server:		127.0.0.53
Address:	127.0.0.5353
Non-authoritative answer:
Name:	host-1.direct.cloud-object-storage.appdomain.cloud
Address: 10.240.64.6

root@test-vpc-vsi: nslookup host-2.direct.cloud-object-storage.appdomain.cloud
Server:		127.0.0.53
Address:	127.0.0.5353
Non-authoritative answer:
Name:	host-2.direct.cloud-object-storage.appdomain.cloud
Address: 10.240.64.6 < ---- your VPE IP address
7. You can now use your instance in the VSI.







 VPE Discoverability 

Following the previous steps results in a service instance with private endpoints that is reachable with the Virtual Private Endpoints from your VPC network.

Object storage instances with private endpoints are reachable from any account within the private network and access to each instance requires authentication. To restrict this access to specific IP addresses, or ranges of IP addresses, configure [context-based restrictions](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-allowlisting).

For more information, see [Secure access to services by using service endpoints](https://cloud.ibm.com/docs/account?topic=account-service-endpoints-overview).





 More resources 



* [Planning for virtual private endpoint gateways](https://cloud.ibm.com/docs/vpc?topic=vpc-planning-considerations)
* [Creating an endpoint gateway](https://cloud.ibm.com/docs/vpc?topic=vpc-ordering-endpoint-gateway)
IBM Cloud Object Storage plug-in version 2.2.9 is available.
: For more information, see the [change log](https://cloud.ibm.com/docs/containers?topic=containers-cos_plugin_changelog).

Worker node fix packs 1.23.16_1562, 1.24.10_1553, 1.25.6_1531, and 1.26.1_1521.
: For more information, see the change logs: [1.23](https://cloud.ibm.com/docs/containers?topic=containers-changelog_123), [1.24](https://cloud.ibm.com/docs/containers?topic=containers-changelog_124), [1.25](https://cloud.ibm.com/docs/containers?topic=containers-changelog_125), and [1.26](https://cloud.ibm.com/docs/containers?topic=containers-changelog_126).





 10 February 2023 

New troubleshooting doc for how to recover after deleting a portable subnet in Classic clusters.
: For more information, see [I deleted a portable subnet and now my Classic cluster my Load Balancers are failing. How do I recover?](https://cloud.ibm.com/docs/containers?topic=containers-ts-network-subnet-recover).

Ingress ALB version 1.3.1_3754_iks and 1.4.0_3755_iks
: For more information, see the [change log](https://cloud.ibm.com/docs/containers?topic=containers-cluster-add-ons-changelog).





 9 February 2023 

Block Storage for VPC add-on version 5.1.
: For more information, see [the change log](https://cloud.ibm.com/docs/containers?topic=containers-vpc_bs_changelog).





 8 February 2023 

ALB OAuth Proxy add-on version 2.0.0_1420.
Monitoring archiving by using IBM Cloud Monitoring 

IBM Cloud Object Storage is integrated with the Monitoring service. Monitoring provides a default template that you can customize to monitor the bucket that you configure to store data for long term from an Activity Tracker instance.

Complete the following steps to monitor the Event Streams instance:



1.  Check that you have an instance of the Monitoring in the same region as your IBM Cloud Object Storage instance. This instance must be configured to collect platform metrics. For more information, see [Enabling platform metrics](https://cloud.ibm.com/docs/monitoring?topic=monitoring-platform_metrics_enabling).
2.  [Launch the Monitoring UI](https://cloud.ibm.com/docs/monitoring?topic=monitoring-launch).
3.  In the Dashboards section, go to Dashboard templates and select the template IBM > IBM COS Bucket.
4.  Create a copy of the template by clicking Create custom dashboard.

Select 2W resolution.

You can use the metric ibm_cos_bucket_used_bytes to see the used space in bytes.

You can use the metric ibm_cos_bucket_object_count to see how many objects are uploaded into the bucket.
5.  (Optional) Edit each panel to view data by bucket name and storage class.

![Edit panel.](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker/images/archive-monitor-1.png)

Then, customize the metric to see data per storage class.

![Customize metric.](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker/images/archive-monitor-2.png)

Check that the resolution is set to 10M.



For more information, see [Using Monitoring with IBM Cloud Object Storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-mm-cos-integration).
Login to your IBM Cloud account.
2. Navigate to Resource List in the menu.
3. Navigate to Storage in the Resource list.
4. Click the IBM Cloud Object Storage name that will display your IBM Cloud Object Storage console.
5. In the IBM Cloud Object Storage console, navigate to Buckets in the menu.
6. Click the required Bucket name to view the Bucket configuration details.
7. Navigate to Endpoints section and copy your public or private endpoints as required.
If you want to use HMAC authentication, you must have the following information ready:

Access key id
: The access_key_id that was generated when the IBM Cloud Object Storage instance was created. For example, 347aa3a4b34344f8bc7c7cccdf856e4c.

Secret access key
: The secret_access_key to use to sign requests. This key was generated when the IBM Cloud Object Storage instance was created. For example, gvurfb82712ad14W7a7915h763a6i87155d30a1234364f61.





 IAM authentication 

If you want to use IAM authentication, you must have the following information ready:

IAM API key
: For example, 0viPHOY7LbLNa9eLftrtHPpTjoGv6hbLD1QalRXikliJ.

Resource instance ID
: For example, cloud-object-storage:global:a/3ag0e9402tyfd5d29761c3e97696b71n:d6f74k03-6k4f-4a82-b165-697354o63903::.







 Connecting to the data source 

From your Discovery project, complete the following steps:



1. From the navigation pane, choose Manage collections.
2. Click New collection.
3. Click IBM Cloud Object Storage, and then click Next.
4. Choose a credential type, and then complete the fields with the information that you collected earlier.



* IAM
* HMAC



Click Next.
5. Name the collection.
6. If the language of the documents in storage is not English, select the appropriate language.

For a list of supported languages, see [Language support](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-language-support).
7. Optional: Change the synchronization schedule.
* storageClass is a [valid provisioning code](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-classesclasses-locationconstraint) that corresponds to the endpoint value. This is then used as the S3 API LocationConstraint variable.
* location should be set to the location portion of the storageClass. For us-south-standard, this would be us-south. This variable is used only for the calculation of [HMAC signatures](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-hmac-signature), but is required for any client, including this example that uses an IAM API key.



package com.cos;

import java.time.LocalDateTime;
import java.util.List;

import com.ibm.cloud.objectstorage.ClientConfiguration;
import com.ibm.cloud.objectstorage.auth.AWSCredentials;
import com.ibm.cloud.objectstorage.auth.AWSStaticCredentialsProvider;
import com.ibm.cloud.objectstorage.client.builder.AwsClientBuilder.EndpointConfiguration;
import com.ibm.cloud.objectstorage.services.s3.AmazonS3;
import com.ibm.cloud.objectstorage.services.s3.AmazonS3ClientBuilder;
import com.ibm.cloud.objectstorage.services.s3.model.Bucket;
import com.ibm.cloud.objectstorage.services.s3.model.ListObjectsRequest;
import com.ibm.cloud.objectstorage.services.s3.model.ObjectListing;
import com.ibm.cloud.objectstorage.services.s3.model.S3ObjectSummary;
import com.ibm.cloud.objectstorage.oauth.BasicIBMOAuthCredentials;

public class CosExample
{
public static void main(String[] args)
{