vania-janet/multiturn-rag-retrieval-data huggingface dataset:
On that view, you can configure an email alert to notify you when an object has been archived. You might have compliance requirements that require you to control who accesses data that is archived. You can define a view that reports access to a bucket, and define an alert to notify you when that happens.

Define views, dashboard, screens, and alerts in IBM Cloud® Activity Tracker to investigate for abnormal activity and critical actions in your COS bucket, and to comply with regulatory audit requirements.





 Archived file 

Although you define policies to manage your COS objects (IBM Cloud Activity Tracker archived files) at the bucket level, there might be situations where you need to hold on to specific files for longer for compliance and auditing purposes. In COS, you can set different attributes per object that allow you to meet this requirements.



* You can define a legal hold flag to an archive file. Legal holds can be applied to objects during initial uploads or after an object is written. [Learn more](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-immutableimmutable-terminology-hold).
* You can define an indefinite retention flag to store the object indefinitely until a new retention period is applied. [Learn more](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-immutableimmutable-terminology-indefinite).



Add a legal hold flag or an indefinite retention flag to individual archived files if you need to keep a file longer than the default retention period specified at the bucket level.







 Define the strategy to query archived data 

You can download data locally, and then use your own tools to query the data. When you download data for analysis, you are responsible for ensuring that your users comply with the regulations and compliance requirements that may be required for your organization. For example, if you must comply with GDPR, you need to control that users are following the download guidelines per GDPR rules.

You can use the Data Engine service to query IBM Cloud Activity Tracker archived files that are stored in a COS bucket in your account. You can run queries from the IBM Cloud UI, or programmatically.
Delete marker: An 'invisible' object that allows for accessing versions of the deleted object.

Version ID: A Unicode, UTF-8 encoded, URL-safe, opaque string that indicates a unique version of an object and associated metadata, and is used to target requests to that particular version. Version IDs are a maximum of 1,024 bytes long.

'null': A special version ID assigned to objects that existed when versioning was enabled on a bucket.





 Consistency and data integrity 

While IBM COS provides strong consistency for all data IO operations, bucket configuration is eventually consistent. After enabling versioning for the first time on a bucket, it may take a few moments for the configuration to propagate across the system. Although versioning may appear to be enabled, it is recommended to wait 15 minutes after enabling versioning to make any requests that are expected to create versions or delete markers.





 IAM actions 

There are new IAM actions associated with versioning.



Table 1. IAM actions associated with versioning

 IAM Action Role 

 cloud-object-storage.bucket.put_versioning Manager, Writer 
 cloud-object-storage.bucket.get_versioning Manager, Writer, Reader 
 cloud-object-storage.object.get_version Manager, Writer, Reader, Content Reader, Object Reader 
 cloud-object-storage.object.head_version Manager, Writer, Reader, Content Reader, Object Reader 
 cloud-object-storage.bucket.delete_version Manager, Writer 
 cloud-object-storage.object.get_versions Manager, Writer, Reader, Content Reader, Object Reader 
 cloud-object-storage.object.copy_get_version Manager, Writer, Reader 
 cloud-object-storage.object.copy_part_get_version Manager, Writer, Reader 
 cloud-object-storage.object.restore_version Manager, Writer 
 cloud-object-storage.object.put_tagging_version Manager, Writer, Object Writer 
 cloud-object-storage.object.get_tagging_version Manager, Writer, Reader 
 cloud-object-storage.object.delete_tagging_version Manager, Writer 







 Activity Tracker events 

Versioning will generate new events.
4000-7999 Gi 300-48000 IOPS 
 8000-9999 Gi 500-48000 IOPS 
 10000-12000 Gi 1000-48000 IOPS 







 Reclaim policy 

Choose if you want to keep your data after the cluster or the persistent volume claim (PVC) is deleted.



* If you want to keep your data, then choose a retain storage class. When you delete the PVC, only the PVC is deleted. The PV, the physical storage device in your IBM Cloud infrastructure account, and your data still exist. To reclaim the storage and use it in your cluster again, you must remove the PV and follow the steps for [using existing File Storage for Classic](https://cloud.ibm.com/docs/containers?topic=containers-file_storageexisting_file).
* If you want the PV, the data, and your physical File Storage for Classic device to be deleted when you delete the PVC, choose a storage class without retain.







 Billing type 

Choose hourly or monthly. Review the [pricing](https://cloud.ibm.com/cloud-storage/file/order) for more information.

By default, all File Storage for Classic devices are provisioned with an hourly billing type.

If you choose a monthly billing type, when you remove the persistent storage, you still pay the monthly charge for it, even if you used it only for a short amount of time.







 Adding File Storage for Classic to apps 

Create a persistent volume claim (PVC) to [dynamically provision](https://cloud.ibm.com/docs/containers?topic=containers-kube_conceptsdynamic_provisioning) File Storage for Classic for your cluster. Dynamic provisioning automatically creates the matching persistent volume (PV) and orders the physical storage device in your IBM Cloud infrastructure account.

Before you begin:



* If you have a firewall, [allow egress access](https://cloud.ibm.com/docs/containers?topic=containers-firewallvyatta_firewall) for the IBM Cloud infrastructure IP ranges of the zones that your clusters are in so that you can create PVCs.
[Create an archive policy](https://cloud.ibm.com/docs-content/v1/content/63322f2f3a72050206eb05ba7f590fdcb741105c/cloud-object-storage/images/bucket-create-ui-archive-rule.png)

Figure 1. Create an archive policy

To immediately archive new objects uploaded to a bucket, enter 0 days on the archive policy.

Archive is available in certain regions only. See [Integrated Services](https://cloud.ibm.com/docs/cloud-object-storage/basics?topic=cloud-object-storage-service-availability) for more details.





 Restore an archived object 

In order to access an archived object, you must restore it to the original storage tier. When restoring an object, you can specify the number of days you want the object to be available. At the end of the specified period, the restored copy is deleted.

The restoration process for "Accelerated Archive" takes up to 2 hours, while the restoration process for Archive takes up to 12 hours.

The archived object sub-states are:



* Archived: An object in the archived state has been moved from its online storage tier (Standard, Vault, Cold Vault and Flex) to the offline archive tier based on the archive policy on the bucket.
* Restoring: An object in the restoring state is in the process of generating a copy from the archived state to its original online storage tier.
* Restored: An object in the restored state is a copy of the archived object that was restored to its original online storage tier for a specified amount of time. At the end of the period, the copy of the object is deleted, while maintaining the archived object.





 Restoring an object using the AWS CLI 

The following examples uses environment variables for clarity. These must be set to the desired values, for example $ENDPOINT would be set to https://s3.us.cloud-object-storage.appdomain.cloud, or https://s3.eu-de.private.cloud-object-storage.appdomain.cloud, or any other required value.



1.
For storage that was [statically provisioned](https://cloud.ibm.com/docs/containers?topic=containers-kube_conceptsstatic_provisioning), VPC Block Storage, or storage that you provisioned with a storage class that sets reclaimPolicy: Retain, the PVC and the PV are removed when you delete the cluster, but your storage instance and your data remain. You are still charged for your storage instance. Also, if you deleted your cluster in an unhealthy state, the storage might still exist even if you chose to remove it.

How do I delete the storage when I want to keep my cluster?
: When you dynamically provisioned the storage with a storage class that sets reclaimPolicy: Delete, you can remove the PVC to start the deletion process of your persistent storage. Your PVC, PV, and storage instance are automatically removed.
: For storage that was [statically provisioned](https://cloud.ibm.com/docs/containers?topic=containers-kube_conceptsstatic_provisioning), VPC Block Storage, or storage that you provisioned with a storage class that sets reclaimPolicy: Retain, you must manually remove the PVC, PV, and the storage instance to avoid further charges.

How does the billing stop after I delete my storage?
: Depending on what storage components you delete and when, the billing cycle might not stop immediately. If you delete the PVC and PV, but not the storage instance in your IBM Cloud account, that instance still exists and you are charged for it.

If you delete the PVC, PV, and the storage instance, the billing cycle stops depending on the billingType that you chose when you provisioned your storage and how you chose to delete the storage.



* When you manually cancel the persistent storage instance from the IBM Cloud console or the ibmcloud sl CLI, billing stops as follows:



* Hourly storage: Billing stops immediately. After your storage is canceled, you might still see your storage instance in the console for up to 72 hours.
* Monthly storage: You can choose between immediate cancellation or cancellation on the anniversary date.
See [api-head-bucket](https://cloud.ibm.com/docs/cloud-object-storage/api-reference?topic=cloud-object-storage-compatibility-api-bucket-operationscompatibility-api-head-bucket).
* Use SDK : See [headbucket property](https://ibm.github.io/ibm-cos-sdk-js/AWS/S3.htmlheadBucket-property).



* Can I migrate a bucket from one COS instance to another?

Yes, You can achieve the same by creating a bucket in the target Object Storage instance and perform a sync. For details see [cloud-object-storage-region-copy](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-region-copy).
* My COS service is locked. How do I reactivate the COS service?

Exceeding the data limit for the Lite account is one of the reasons why your account is locked or deactived. The [COS support](https://cloud.ibm.com/unifiedsupport/cases/form) team can help to unlock your account.
* After deleting a Object Storage instance, is it possible to reuse the same bucket names that were part of the deleted COS instance?

When an empty bucket is deleted, the name of the bucket is held in reserve by the system for 10 minutes after the delete operation. After 10 minutes the name is released for re-use.
* How can I archive and restore objects in Object Storage?

Archived objects must be restored before you can access them. While restoring, specify the time limit the objects should remain available before being re-archived. For details, see [archive-restore data](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-archive).
* Can I enable Object Storage replication between two different regions for DR purposes?
getExpirationInDays() Returns the time in days from an object's creation to its expiration. 
 setExpirationInDays(int expirationInDays) Sets the time, in days, between when an object is uploaded to the bucket and when it expires. 



--------------------





 Get an object's headers 

A HEAD given a path to an object retrieves that object's headers. This operation does not make use of operation specific query parameters or payload elements.

Syntax

HEAD https://{endpoint}/{bucket-name}/{object-name} path style
HEAD https://{bucket-name}.{endpoint}/{object-name} virtual host style

Response headers for archived objects



 Header Type Description 

 x-amz-restore string Included if the object has been restored or if a restoration is in progress. If the object has been restored, the expiry date for the temporary copy is also returned. 
 x-amz-storage-class string Returns GLACIER or ACCELERATED if archived or temporarily restored. 
 x-ibm-archive-transition-time date Returns the date and time when the object is scheduled to transition to the archive tier. 
 x-ibm-transition string Included if the object has transition metadata and returns the tier and original time of transition. 
 x-ibm-restored-copy-storage-class string Included if an object is in the RestoreInProgress or Restored states and returns the storage class of the bucket. 



Sample request

HEAD /images/backup HTTP/1.1
Authorization: {authorization-string}
x-amz-date: 20160825T183244Z
Host: s3.us.cloud-object-storage.appdomain.cloud

Sample response

HTTP/1.1 200 OK
Date: Wed, 7 Feb 2018 19:51:00 GMT
X-Clv-Request-Id: da214d69-1999-4461-a130-81ba33c484a6
Accept-Ranges: bytes
Server: 3.x
X-Clv-Request-Id: 06d67542-6a3f-4616-be25-fc4dbdf242ad
Accept-Ranges: bytes
Server: Cleversafe/3.9.1.114
X-Clv-S3-Version: 2.5





 Temporarily restore an archived object 

A POST request that is issued to an object with query parameter restore to request temporary restoration of an archived object. A Content-MD5 header is required as an integrity check for the payload.

An archived object must be restored before downloading or modifying the object. The lifetime of the object must be specified after which the temporary copy of the object will be deleted.

For buckets with a lifecycle policy transition storage class of GLACIER, there can be a delay of up to 12 hours before the restored copy is available for access. If the transition storage class was set to ACCELERATED, there can be a delay of up to two (2) hours before the restored object is available. A HEAD request can check whether the restored copy is available.

To permanently restore the object, it must be copied to a bucket that doesn't have an active lifecycle configuration.

Not all operations are supported in Satellite environments. For details, see [supported Satellite APIs](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-apis-cos-satellite)

Syntax

POST https://{endpoint}/{bucket-name}/{object-name}?restore path style
POST https://{bucket-name}.{endpoint}/{object-name}?restore virtual host style

Payload Elements

The body of the request must contain an XML block with the following schema:



 Element Type Children Ancestor Constraint 

 RestoreRequest Container Days, GlacierJobParameters None None 
 Days Integer None RestoreRequest Specified the lifetime of the temporarily restored object. The minimum number of days that a restored copy of the object can exist is 1. After the restore period has elapsed, temporary copy of the object will be removed. 
 GlacierJobParameters String Tier RestoreRequest None
Near-term data retention (TB) 7 33 329 
 Long-term data retention (TB) 18 88 877 



From a data perspective, data is used as follows:



* Scan data is used in compliance management.
* Log data is used in forensic management.
* Policy and flow data is used in risk management. Flow data is available from NSX Manager only.



Data storage has three tiers:



* Replicated
* Near term
* Long term



The following table provides a summary of the deployments.



Table 4. Summary

 Deployment model All-in-one Partially distributed Fully distributed 

 Number of assets 100 500 5,000 
 Online data generated in 30 days (TB) 4 22 219 
 Nearline data retention (90 days) (TB) 7 33 329 
 Offline data retention (8 months) (TB) 18 88 877 
 Total data storage retention (1 year) (TB) 28 142 1,425 
 Base VMs 1 1 20 
 Scaling VMs 0 3 28 
 Total VMs 1 4 48 





 Notes 

When you delete the Caveonix RiskForesight service, the IBM Cloud for VMware Solutions automation deletes only the single all-in-one Caveonix VM that was deployed and the dedicated private subnet that was ordered for it. Therefore,



* If you scaled out the Caveonix VM into multiple VMs, those additional VMs are not removed.
* If you used the IP addresses of the dedicated private subnet on additional VMs, those VMs must be assigned new IP addresses to continue to function.
* If you delete vCenter Server instance A with the Caveonix RiskForesight service installed, and you used the IP addresses of the dedicated private subnet that is ordered for the service in vCenter Server instance B, the dedicated private subnet is canceled upon deletion of vCenter Server instance A.
* [VMware® VRA](https://cloud.ibm.com/docs/Backup?topic=Backup-VRAVRA)



The plug-ins that are listed here are only compatible with Windows® servers, except for the Oracle or VMware® plug-ins. Each agent is available as an add-on to your backup service at no cost.
* How frequently can we back up the data?

Within Cloud Backup Portal, backups can be made manually, or can be scheduled as a single instance, or to be recurring. Recurring backups can be made daily, weekly, monthly or on a custom schedule and can be updated or canceled at any time.

Highly frequent backups that run several times daily or hourly can cause backup jobs to become corrupted. This corruption occurs because backup vault does not get enough time to run required background maintenance tasks. Backup Jobs take precedence over maintenance tasks. So when backups are done with high frequency, the vault continues to run the backup jobs and cause the number of safe-sets to grow.
* How do the retention schemes work?

IBM Cloud Backup for Classic allows for data-retention depending on how long you want to roll back to. Daily retention schemes hold data for seven days, while weekly schemes hold data for one month and monthly schemes hold data for one year. At the end of each period, the oldest data set gets rotated out, and the first "delta backup" that was made becomes the oldest available restore point.

You can modify default retention schemes and can create custom retention schemes. It's best to use the default retention schemes as a starting point. When you create a new retention scheme or modify an existing retention, make sure that the Archiving option is not selected. Archiving is not supported.
* What is Delta Technology?

The first backup is a "seed" (a complete, full backup), the next and subsequent ones are "deltas" (that is, changes only), but they are equivalent to, and still considered a "full backup". That is, you're able to restore all or any files from it.
For details, see [COS Region Copy](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-region-copy).
* Is there a way to verify an object’s integrity during an upload to Object Storage?

Object Storage supports object integrity and ensures that the payload is not altered during transit.
* How can I set a notification when usage in a Object Storage instance is near a certain billing amount?

You can use a "soft" bucket quota feature by integrating with Metrics Monitoring and configuring for notifications. For details on establishing a hard quota that prevents usage beyond a set bucket size, see [Using Bucket Quota](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-quota).
* Why am I unable to delete a Object Storage instance?

It isn't possible to delete an instance if the API key or Service ID being used is locked. You'll need to navigate in the console to Manage > Access (IAM) and unlock the API Key or Service ID. The error provided may seem ambiguous but is intended to increase security:

> An error occurred during an attempt to complete the operation. Try fixing the issue or try the operation again later. Description: 400

This is intentionally vague to prevent any useful information from being conveyed to a possible attacker. For more information on locking API keys or Service IDs, [see the IAM documentation](https://cloud.ibm.com/docs/account?topic=account-serviceids&interface=uilock_serviceid).
* How do I download the Root CA certificate for Object Storage?

Object Storage root CA certificates can be downloaded from [https://www.digicert.com/kb/digicert-root-certificates.htm](https://www.digicert.com/kb/digicert-root-certificates.htm). Please download PEM or DER/CRT format from "DigiCert TLS RSA SHA256 2020 CA1" that is located under "Other intermediate certificates."
[Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/checkmark-icon.svg) 
 Long-term retention policy NO NO NO NO 



[7]: Standard is used for active workloads. No charge incurs for data that is retrieved (other than the cost of the operational request itself and public outbound bandwidth).

[8]: Vault is used for cool workloads where data isn't accessed frequently, but a retrieval charge applies for reading data. The service includes a threshold for object size and storage period consistent with the intended use of this service for cooler, less-active data.

[9]: Cold Vault is used for cold workloads where data is primarily archived (accessed every 90 days or less) - a larger retrieval charge applies for reading data. The service includes a threshold for object size and storage period consistent with the intended use of this service: storing cold, inactive data.

[10]: Flex is used for dynamic workloads where access patterns are more difficult to predict. Depending on usage, if the cost of cooler storage that is combined with retrieval charges exceeds a cap value, then the storage charge increases and no any retrieval charges apply. If the data isn't accessed frequently, Flex storage can be more cost effective than Standard storage.