vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* How can I archive and restore objects in Object Storage?

Archived objects must be restored before you can access them. While restoring, specify the time limit the objects should remain available before being re-archived. For details, see [archive-restore data](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-archive).
* Can I enable Object Storage replication between two different regions for DR purposes?

Yes, it is possible to configure buckets for automated [replication of objects to a destination bucket](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-replication-overview).
* How can I track events in Object Storage?

The Object Storage Activity Tracker service records user-initiated activities that change the state of a service in Object Storage. For details, see [IBM Cloud Activity Tracker](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-at-events).
* How can I setup notifications when objects are updated or written to a bucket?

Use [Cloud Functions for object storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-functions) to setup an Event Source (notification).
* Does Object Storage have rate limits when writing to or reading from buckets?

Yes, Object Storage has rate limiting. For details, see [COS support](https://cloud.ibm.com/unifiedsupport/cases/form).
* Is Object Storage HIPAA compliant to host PHI data?

Yes, Object Storage is HIPAA compliant.
* Is there any option in Object Storage to enable accelerate data transfer?

Object Storage offers [Aspera](https://www.ibm.com/cloud/object-storage/aspera) service for high speed data transfer.
* How can I compare various attributes of an object in two different buckets?

Use [Rclone](https://rclone.org/commands/rclone_check). It enables you to compare various attributes.
Object Storage is ‘immediately consistent’ for data and ‘eventually consistent’ for usage accounting.
* Can Object Storage partition the data automatically using HDFS, so I can read the partitions in parallel, for example, with Spark?

Object Storage supports a ranged GET on the object, so an application can do a distributed striped-read-type operation. Doing the striping is managed by the application.
* Can I restore a bucket from a specific back-up file?

It is possible to overwrite an existing bucket. Restore options depend on the capabilities provided by the back-up tool you use; check with your back-up provider. As described in [Your responsibilities when using IBM Cloud Object Storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-responsibilities), you are responsible for ensuring data back-ups if necessary. IBM Cloud® Object Storage does not provide a back-up service.
* Can a web browser display the content of files stored in IBM Cloud Object Storage?

Web browsers can display web content in IBM Cloud Object Storage files, using the COS endpoint as the file location. To create a functioning website, however, you need to set up a web environment; for example, elements such as a CNAME record. IBM Cloud Object Storage does not support automatic static website hosting. For information, see [Static websites](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-iam-public-accesspublic-access-static-website) and this [tutorial](https://www.ibm.com/cloud/blog/static-websites-cloud-object-storage-cos).
* If I set an archive policy on an existing bucket, does the policy apply to existing files?

The policy applies to the new objects uploaded but does not affect existing objects on a bucket. For details, see [Add or manage an archive policy on a bucket](https://cloud.ibm.com/docs/cloud-object-storage/basics?topic=cloud-object-storage-archivearchive-add).
* Can I unzip a file after I upload it?
Bucket operations 

The modern capabilities of IBM Cloud® Object Storage are conveniently available via a RESTful API. Operations and methods concerning buckets (where objects are stored) are documented here.

For more information about the permissions and access, see [Bucket permissions](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-iam-bucket-permissions).



 A note regarding Access/Secret Key (HMAC) authentication 

When authenticating to your instance of IBM Cloud Object Storage [using HMAC credentials](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-uhc-hmac-credentials-main), you will need the information represented in Table 1 when [constructing an HMAC signature](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-hmac-signature).



Table 1. HMAC signature components

 Key Value Example 

 {access_key} Access key assigned to your Service Credential cf4965cebe074720a4929759f57e1214 
 {date} The formatted date of your request (yyyymmdd) 20180613 
 {region} The location code for your endpoint us-standard 
 {signature} The hash created using the secret key, location, and date ffe2b6e18f9dcc41f593f4dbb39882a6bb4d26a73a04326e62a8d344e07c1a3e 
 {timestamp} The formatted date and time of your request 20180614T001804Z 







 List buckets 

A GET request sent to the endpoint root returns a list of buckets that are associated with the specified service instance. For more information about endpoints, see [Endpoints and storage locations](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-endpointsendpoints).

Not all operations are supported in Satellite environments.
* [Smart Tier](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-billingsmart-tier-pricing-details)
* [Monitoring](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-mm-cos-integration)



Downloads that use Aspera high-speed transfer incur extra egress charges. For more information, see the [pricing page](https://www.ibm.com/cloud/object-storage).



 Cross Region 



 Region Aspera Key Protect Hyper Protect Crypto Services Archive Data Object Lock Immutable Object Storage Activity Tracker Functions Smart Tier Monitoring Replication One Rate 

 ap Yes Yes (in jp-tok) No No No No ap-tok No Yes ap-tok Yes No 
 eu Yes Yes (in eu-de) No No No No eu-de No Yes eu-de Yes No 
 us Yes Yes (in us-south) Yes (failover in us-east) No Yes Yes us-south No Yes us-south Yes No 







 Regional 



 Region Aspera Key Protect Hyper Protect Crypto Services Archive Data Object Lock Immutable Object Storage Activity Tracker Functions Smart Tier Monitoring Replication One Rate 

 au-syd Yes Yes Yes (see note) Yes Yes Yes au-syd Yes Yes au-syd Yes Yes 
 jp-tok Yes Yes Yes (see note) Yes Yes Yes ap-tok Yes Yes ap-tok Yes Yes 
 jp-osa No Yes No Yes Yes Yes ap-osa Yes Yes ap-osa Yes Yes 
 eu-gb Yes Yes Yes (see note) Yes Yes Yes eu-gb Yes Yes eu-gb Yes Yes 
 eu-de Yes Yes Yes (see note) Yes Yes Yes eu-de Yes Yes eu-de Yes Yes 
 us-south Yes Yes Yes (see note) Yes Yes Yes us-south Yes Yes us-south Yes Yes 
 us-east Yes Yes Yes (see note) Yes Yes Yes us-east Yes Yes us-east Yes Yes 
 ca-tor No Yes Yes (see note) Yes Yes Yes ca-tor Yes Yes ca-tor Yes Yes
About the IBM Cloud Object Storage S3 API 

The IBM Cloud® Object Storage API is a REST-based API for reading and writing objects.

It uses IBM Cloud® Identity and Access Management for authentication and authorization, and supports a subset of the S3 API for easy migration of applications to IBM Cloud.

This reference documentation is being continuously improved. If you have technical questions about using the API in your application, post them on [StackOverflow](https://stackoverflow.com/). Add both ibm-cloud-platform and object-storage tags and help improve this documentation thanks to your feedback.

As Cloud Identity and Access Management tokens are relatively easy to work with, curl is a good choice for basic testing and interaction with your storage. More information can be found in [the curl reference](https://cloud.ibm.com/docs/cloud-object-storage/cli?topic=cloud-object-storage-curl).

The following tables describe the complete set of operations of the IBM Cloud Object Storage API. For more information, see [the API reference page for buckets](https://cloud.ibm.com/docs/cloud-object-storage/api-reference?topic=cloud-object-storage-compatibility-api-bucket-operations) or [objects](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-object-operations).



 Bucket operations 

These operations create, delete, get information about, and control behavior of buckets.



 Bucket operation Note 

 GET Buckets Used to retrieve a list of all buckets that belong to an account. 
 DELETE Bucket Deletes an empty bucket. 
 DELETE Bucket CORS Deletes any CORS (cross-origin resource sharing) configuration set on a bucket. 
 GET Bucket Lists objects in a bucket. Limited to listing 1,000 objects at a time. 
 GET Bucket CORS Retrieves any CORS configuration set on a bucket. 
 HEAD Bucket Retrieves a bucket's headers. 
 GET Multipart Uploads Lists multipart uploads that aren't completed or canceled. 
 PUT Bucket Buckets have naming restrictions. Accounts are limited to 100 buckets.
For details, see [cloud-object-storage-billing](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-billing).





 What are the considerations for choosing the correct storage class in Object Storage? 

You can choose the correct storage class based on your requirement. For details, see [billing-storage-classes](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-billingbilling-storage-classes).





 How to invoke IBM Cloud Object Storage bucket operations using cURL? 

You have the most power by using the command line in most environments with IBM Cloud Object Storage and cURL. However using cURL assumes a certain amount of familiarity with the command line and Object Storage. For details, see [Using cURL](https://cloud.ibm.com/docs/cloud-object-storage/cli?topic=cloud-object-storage-curl).





 Is encryption applied to a bucket by default? 

Yes, by default, all objects stored in Object Storage are encrypted using randomly generated keys and an all-or-nothing-transform (AONT). You can get the encryption details using IBM Cloud UI/CLI. For details, see [Cloud Storage Encryption](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-encryption).





 How can I list all permissions of a bucket? 

The IAM feature creates a report at the instance level which may extend to their buckets. It does not specifically report at the bucket level. For details, see [Account Access Report](https://cloud.ibm.com/docs/account?topic=account-access-report).





 How can I monitor bucket changes in the public cloud without using the cloud functions? 

You must use cloud functions to get notifications for object changes.





 How can I monitor Object Storage resources? 

Use the Activity Tracker service to capture and record Object Storage activities and monitor the activity of your IBM Cloud account. Activity Tracker is used to track how users and applications interact with Object Storage.
[COS on Satellite Endpoints](https://cloud.ibm.com/docs-content/v1/content/63322f2f3a72050206eb05ba7f590fdcb741105c/cloud-object-storage/images/satellite_endpoints.png)

Figure 2. Object Storage for Satellite Endpoints

Keep in mind that requests made to Object Storage for Satellite infrastructure must originate within the satellite location as the service endpoint may not be accessible from the outside of that location.





 What features are currently supported? 



* [IBM Cloud IAM access policies](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-iam)
* [Object Expiration](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-expiry)
* [Object Versioning](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-versioning)
* [Object Tagging](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-object-tagging)
* [Static Web hosting](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-static-website-options)
* [Key Protect managed encryption](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-kp)



Any Key Protect instances must be in IBM Cloud and must be located in the same IBM Cloud region from where the Satellite location is managed.

Activity Tracker will log service instance creation and deletion, but not any actions specific to object storage, such as listing buckets or reading/writing data.

Other features that are currently not supported (such as Activity Tracking, Metrics Monitoring, [Compliance](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-compliance), Security and Compliance Center) will be added in the future.
"objectstorage",
"softlayer",
"net"
],
"path":
"{{bucket}}kp"
]
}
},
"response": ]
},
{
"name": "Delete bucket (key protect)",
"event":
{
"listen": "test",
"script": {
"id": "5cf3d531-13f1-4cf6-a4fc-2f8d2d8e48af",
"type": "text/javascript",
"exec":
"pm.test("Request was successful", function() {",
" pm.response.to.be.success; ",
"});"
]
}
}
],
"request": {
"auth": {
"type": "bearer",
"bearer":
{
"key": "token",
"value": "{{iamtoken}}",
"type": "string"
}
]
},
"method": "DELETE",
"header":
{
"key": "ibm-service-instance-id",
"value": "{{serviceid}}"
}
],
"body": {
"mode": "raw",
"raw": ""
},
"url": {
"raw": "https://{{endpoint-region}}cloud-object-storage.appdomain.cloud/{{bucket}}kp",
"protocol": "https",
"host":
"{{endpoint-region}}",
"objectstorage",
"softlayer",
"net"
],
"path":
"{{bucket}}kp"
]
}
Use Object Storage [Direct Link Connection](https://cloud.ibm.com/docs/direct-link?topic=direct-link-using-ibm-cloud-direct-link-to-connect-to-ibm-cloud-object-storage) to create a global direct link.
* How does frequency of data access impact the pricing of Object Storage?

Storage cost for Object Storage is determined by the total volume of data stored, the amount of public outbound bandwidth used, and the total number of operational requests processed by the system. For details, see [cloud-object-storage-billing](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-billing).
* What are the considerations for choosing the correct storage class in Object Storage?

You can choose the correct storage class based on your requirement. For details, see [billing-storage-classes](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-billingbilling-storage-classes).
* How to invoke IBM Cloud Object Storage bucket operations using cURL?

You have the most power by using the command line in most environments with IBM Cloud Object Storage and cURL. However using cURL assumes a certain amount of familiarity with the command line and Object Storage. For details, see [Using cURL](https://cloud.ibm.com/docs/cloud-object-storage/cli?topic=cloud-object-storage-curl).
* Is encryption applied to a bucket by default?

Yes, by default, all objects stored in Object Storage are encrypted using randomly generated keys and an all-or-nothing-transform (AONT). You can get the encryption details using IBM Cloud UI/CLI. For details, see [Cloud Storage Encryption](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-encryption).
* How can I list all permissions of a bucket?

The IAM feature creates a report at the instance level which may extend to their buckets. It does not specifically report at the bucket level.
* [IBM Cloud Identity and Access Management - IAM](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-iam)
* [Activity Tracker](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-at)
* [Metrics Monitoring](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-mm-cos-integration)
* [Object Expiry](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-expiry)
* [Cloud Object Storage Firewall](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-setting-a-firewall)
* [Content Delivery Network - CDN](https://cloud.ibm.com/docs/cis?topic=cis-resolve-override-cos)
* [Archive](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-archive)
* [Key Protect](https://cloud.ibm.com/docs/key-protect?topic=key-protect-integrate-cos)
* [IBM Cloud Functions](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_obstorage)







 Set up a compute resource to run the migration tool 



1. Choose a Linux™/macOS™/BSD™ machine or an IBM Cloud Infrastructure Bare Metal or Virtual Server with the best proximity to your data. Selecting a data center in the same region as the destination bucket is generally the best choice (e.g. if moving data from mel01 to au-syd, use a VM or Bare Metal in au-syd). The recommended Server configuration is: 32 GB RAM, 2-4 core processor, and private network speed of 1000 Mbps.
2.
: Updates to a bucket's metadata using the [Resource Configuration API](https://cloud.ibm.com/apidocs/cos/cos-configuration) (such as [adding or modifying a firewall](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-setting-a-firewall)) will now show the details of the change in the requestData fields shown in [Activity Tracker](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-at).





 15 December 2020 

New feature!
: Objects can now be efficiently [tagged](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-object-tagging) with custom key-value pairs.





 5 November 2020 

New location!
: Buckets can now be created in a Regional configuration in Osaka, Japan. More information can be found in the [Select Regions and Endpoints](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-endpoints).





 27 October 2020 

New feature!
: Buckets can now be configured to [serve static websites](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-static-website-options).





 12 October 2020 

Encryption updates
: Lifecycle actions on [Key Protect](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-kpkp-lifecycle) and [Hyper Protect Crypto Services](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-hpcs-lifecycle) encryption keys can now generate bucket events in Activity Tracker.
: Buckets can now be encrypted using Hyper Protect Crypto Services in the US East region.





 18 August 2020 

New feature!
-d supplies the new document body, and the URL not only contains the database and ID of the document, but critically the rev - the revision of the document we intend to mutate.

If we forget and omit the rev parameter, we get an error response.

HTTP response codes show whether a request succeeds or not. Responses in the 200 range are successful. Responses in the 400 range are user errors (for example, invalid parameters), and responses in the 500 range are server-side errors. Additionally, you can see the full HTTP request and response by supplying the -v command-line option to curl/acurl.

Also, updates to documents happen in their entirety or not at all. No API construct to modify part of a document exists. A whole document must be supplied to overwrite a previous revision.

Finally, to delete a document, we use the DELETE method, so -X DELETE. We direct the request to the URL that includes the database name and document to be deleted, and critically, we also supply the rev - the revision of the document to delete.

If we omit the revision token, an error is returned, and the request fails.

To summarize, understanding the HTTP API helps you grasp the relationship between your code and the IBM Cloudant service.

The URLs are hierarchical: service/database/document or service/database/endpoint.

The HTTP methods act as verbs defining the action to be done.

All actions can be triggered by using simple HTTP API calls, from the command line or from your code and so can be easily scripted.

That's the end of this part. The next part is called The Bulk API.







 The Bulk API video 

Learn how to use two API calls to perform all the basic IBM Cloudant operations while also acting on more than one document per API call.



* The Bulk API video script

Welcome to the Introduction to IBM Cloudant course, an 17-part video series that gives you an overview of the IBM Cloudant database-as-a-service.

This video is part 8 - The Bulk API.

In the previous part, we saw how documents could be easily added, updated, and deleted singly by using the IBM Cloudant HTTP API.
Choose the [bucket's storage class](https://cloud.ibm.com/docs/cloud-object-storage/basics?topic=cloud-object-storage-classes) to accurately reflect how often you expect to read the stored data. This is important as it determines your billing details. Follow the Create link to create and access your new bucket.
5. Determine the advanced configurations, if any, suitable to your content. You can store data by transitioning from any of the storage tiers (Standard, Vault, Cold Vault and Flex) to long-term offline archive or use the online Cold Vault option. See the example in Figure 1 for options in creating an archive policy.



Zoom

![Create an archive policy](https://cloud.ibm.com/docs-content/v1/content/63322f2f3a72050206eb05ba7f590fdcb741105c/cloud-object-storage/images/bucket-create-ui-archive-rule.jpg)

Figure 1. Create an archive policy

Buckets are a way to organize your data, but they're not the sole way. Object names (often referred to as object keys) can use one or more forward slashes for a directory-like organizational system. You then use the portion of the object name before a delimiter to form an object prefix, which is used to list related objects in a single bucket through the Object Storage API.





 Add some objects to your buckets 

Now go ahead and go to one of your buckets by selecting it from the list. Click Add Objects. New objects overwrite existing objects with the same names within the same bucket. When you use the console to upload objects the object name always matches the file name. There doesn't need to be any relationship between the file name and the object key if you're using the API to write data. Go ahead and add a handful of files to this bucket.
Can I have multiple revisions of objects in my bucket?
: Yes! Object Storage version support allows you to have more than one version of an object in your bucket. For more information, see [Versioning objects](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-versioning).

Can I use replication with my IBM Cloud Object Storage trigger?
: Yes! With replication, you can define rules for automatically copying bucket objects from a source bucket to one or more destination buckets. For more information, see [Replicating objects](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-replicating-objects).



 Prerequisites for working with the IBM Cloud Object Storage trigger 

Before you begin

You must [create an IBM Cloud Object Storage service instance](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-gs-devgs-dev-provision) and [create a regional bucket](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-getting-started-cloud-object-storagegs-create-buckets) in one of the supported regions. Note that your bucket must be in the same region as your Cloud Functions namespace.

To use the IBM Cloud Object Storage trigger, the following conditions must be met.



* Only [IAM-enabled Cloud Functions namespaces](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-namespaces) are supported.
* Your IBM Cloud Object Storage bucket must be a regional bucket and must be in the same region as your Cloud Functions namespace. Cross-region and single-site buckets are not supported.
* You must [assign the Notifications Manager](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_obstoragepkg_obstorage_auth) role to your Cloud Functions namespace for your IBM Cloud Object Storage.







 1.