vania-janet/multiturn-rag-retrieval-data huggingface dataset:
The Service Provider assigns Assets that are collected from vCenter Server to Tenant Organizations and Sub-Organizations of the Service Provider. An Asset can be assigned only to one Organization.
* Remote Access - Provides end machine credentials to enable scans for vulnerability and compliance monitoring and to collect system events logs. The Service Provider can enable remote access for their own assets only. Tenants have control over remote access from their Assets.
* Applications and Subapplications - A logical way to group assets. Example application - SAP. Example subapplications - SAP Front End, SAP Middle Tier, and SAP Back End.
* Locations - Assets are uniquely grouped by Location, Cloud Provider, and Asset Repository.
* Environments - A way to group Assets and Applications. Each Environment is assigned a risk factor 1 - 10. This factor is applied in the risk score calculation. Service Provider defines the Environments
* Tasks - They are used in RiskForesight to complete the following actions:



* Periodically synchronize the asset repository with RiskForesight.
* Perform SCAP vulnerability and compliance scans.
* Collect network flows and other information that uses NSX scans.
* Collect information about the software that runs on monitored assets.
* Collect syslog and system events for assets.



* Task Types - The following tasks are supported:



* VMware vCenter Scan - Collects Assets from vCenter.
* VMware VCD Scan - Collects Assets from VCD.
* VMware NSX Scan - Collects network information and network flow from NSX Manager.
* SCAP Scan - This scan type is used for scanning workloads for compliance risk and cyberrisk. This scan is based on the SCAP (Security Content Automation Protocol). Future releases are planned to support custom content in the OVAL format.
* Software Scan - This scan collects the software inventory that is installed on the target workload under management. This inventory is then available for search through the search option on the user interface.
* LogExtract - This scan supports log collection from Microsoft® Windows and Linux® workloads. The results are used for forensics purposes and ingested through machine learning to produce useful information.
...
dns {
instance_crn = local.network_context.dns.crn
zone_id = local.network_context.dns.zone_id
}
}
3. Provide a DNS CNAME record shared.widgets.example.com to identify the load balancer so the applications continue to work without source code changes:

 shared.widgets.example.com
resource ibm_dns_resource_record "shared_lb" {
count = var.shared_lb ? 1 : 0 shared load balancer?
instance_id = local.network_context.dns.guid
zone_id = local.network_context.dns.zone_id
type = "CNAME"
name = "shared"
rdata = ibm_is_lb.shared_lb[0].hostname
ttl = 3600
}

The same count = var.shared_lb ? 1 : 0 is a used. The load balancer hostname will be something like b7911a41-us-south.widgets.example.com. The CNAME record will be used by the applications.
4. Edit the terraform.tfvars file and uncomment shared_lb = true. Then apply the changes:

terraform apply
5. Execute the curl .../remote command from the previous application1 section (ignore the output just generated for the shared microservice). Notice that the remote_ip is 10.0.1.4, the load balancer, and the remote_info is 10.0.0.4, the instance. Curl a few more times and notice the remote_ip for the load balancer may change.

$ curl 169.48.152.220:3000/remote

{
"remote_url": "http://shared.widgets.example.com:3000/info",
"remote_ip": "10.0.1.4",
"remote_info": {
"req_url": "/info",
Permanent retention can only be enabled at a IBM Cloud Object Storage bucket level with retention policy enabled and users are able to select the permanent retention period option during object uploads. Once enabled, this process can't be reversed and objects uploaded that use a permanent retention period cannot be deleted. It's the responsibility of the users to validate at their end if there's a legitimate need to permanently store objects by using Object Storage buckets with a retention policy.

When using Immutable Object Storage, you are responsible for ensuring that your IBM Cloud Account is kept in good standing per IBM Cloud policies and guidelines for as long as the data is subject to a retention policy. Refer to IBM Cloud Service terms for more information.







 Immutable Object Storage and considerations for various regulations 

When using immutable Object Storage, it is the client's responsibility to check for and ensure whether any of the feature capabilities that are discussed can be used to satisfy and comply with the key rules around electronic records storage and retention that is generally governed by:



* [Securities and Exchange Commission (SEC) Rule 17a-4(f)](https://www.ecfr.gov/cgi-bin/text-idx?SID=b6b7a79d18d000a733725e88d333ddb5&mc=true&node=pt17.4.240&rgn=div5se17.4.240_117a_64),
* [Financial Industry Regulatory Authority (FINRA) Rule 4511(c)](https://www.finra.org/rules-guidance/rulebooks/finra-rules/4511), and
* [Commodity Futures Trading Commission (CFTC) Rule 1.31(c)-(d)](https://www.ecfr.gov/cgi-bin/text-idx?SID=2404f765a6f79e0b7fcf05b6844046cb&mc=true&node=se17.1.1_131&rgn=div8)



To assist clients in making informed decisions, IBM engaged Cohasset Associates Inc. to conduct an independent assessment of IBM’s Immutable Object Storage.
Optionally, set a default retention period.
7. Click on Save







 Adding a Retain Until Date or Legal Hold to an object 



1. Navigate to the bucket with the target object
2. Toggle Display Versions
3. Go to the details of the target version
4. Add a retention period and/or toggle on a legal hold.









 Using Object Lock for business continuity and disaster recovery 

Object Lock can be used to provide continuity of service in the event of a ransomware attack, as protected data is unable to be modified or destroyed.





 Consistency and data integrity 

While IBM Cloud Object Storage provides strong consistency for all data IO operations, bucket configuration is eventually consistent. After enabling, modifying, or deleting a default retention period on a bucket it may take a few moments for the configuration to propagate across the system. Operations on objects, such as adding a legal hold, are immediately consistent.





 Usage and accounting 

Locked objects (and their versions) contribute usage just like any other data and you will be responsible for the [usage costs](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-billing) for as long as object remains locked with a retention period.





 Interactions 

Object Lock can be used in combination with several object storage features as per your use case requirements.



 Versioning 

[Enabling versioning](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-versioning) is a prerequisite for enabling Object Lock. If a bucket is created using the x-amz-bucket-object-lock-enabled header, versioning will automatically be enabled.

Deleting a versioned object creates a delete marker. The object may appear to be deleted, but if the object is protected it is not possible to delete the protected version. Delete markers themselves are not protected.





 Replication 

Object Lock cannot be used on the source bucket for replication, only on the destination. Objects will be assigned the default retention period.





 Key Management Systems 

Protected objects will be encrypted using the root key of the bucket.
Policies are enforced until the end of a retention period, and can not be altered until the retention period has expired. While IBM Cloud Object Storage makes use of the S3 API for most operations, the APIs used for configuring retention policies is not the same as the S3 API, although some terminology may be shared. Read this documentation carefully to prevent any users in your organization from creating objects that can not be deleted, even by IBM Cloud administrators.

Not all operations are supported in Satellite environments. For details, see [supported Satellite APIs](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-apis-cos-satellite)

echo -n (XML block) | openssl dgst -md5 -binary | openssl enc -base64

Syntax

PUT https://{endpoint}/{bucket-name}?protection= path style
PUT https://{bucket-name}.{endpoint}?protection= virtual host style

The body of the request must contain an XML block with the following schema:



 Element Type Children Ancestor Constraint 

 ProtectionConfiguration Container Status, MinimumRetention, MaximumRetention, DefaultRetention <br><br> * <br><br><br> <br><br> * <br><br><br> 
 Status String <br><br> * <br><br><br> ProtectionConfiguration Valid status string 
 MinimumRetention Container Days ProtectionConfiguration <br><br> * <br><br><br> 
 MaximumRetention Container Days ProtectionConfiguration <br><br> * <br><br><br> 
 DefaultRetention Container Days ProtectionConfiguration <br><br> * <br><br><br> 
 Days Integer <br><br> * <br><br><br> MinimumRetention, MaximumRetention, DefaultRetention Valid retention integer 



Example request

PUT /example-bucket?protection= HTTP/1.1
Authorization: {authorization-string}
However, legal holds are more flexible and don't have a defined temporal component. Instead they simply remain in effect until removed. Legal holds can be freely placed and removed by any user who has the cloud-object-storage.object.put_object_lock_legal_hold and cloud-object-storage.object.put_object_lock_legal_hold_version actions.

Legal holds have the additional benefit of acting as method for applying indefinite retention on an object.

Legal holds and retention periods operate independently. Legal holds have no impact on retention periods, and vice-versa.

Imagine an object with both a legal hold and a retention period. When the retention period ends, the object version remains protected until the legal hold is removed. If you remove a legal hold while an object version is subject to a retention period it remains protected until the retention period is complete.

Objects locked and stored with a retention period cannot be deleted until retention period expires and any associated legal hold is removed.

Locking objects with a Governance Mode is not supported.







 Getting started with Object Lock 

In order to get started, there are some some prerequisites:



* You'll need the Writer or Manager platform role on a bucket, or a custom role with the appropriate actions (such as cloud-object-storage.bucket.put_object_lock_configuration) assigned.
* Object Versioning must be enabled
* You will need to use Standard pricing plan, see [pricing](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-billing) for details.
* You will need to pick a region where Object Lock is supported, refer to [Integrated Services](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-service-availability) for details.
* A maximum default retention period of 100 years (or 36500 days) is supported.
* When using the console, it is also possible to set a Retain Until Date in months, in addition to days or years.



The retention period for an object cannot be decreased.
If the Object Storage service instance is deleted, all bucket names in that instance will be held in reserve by the system for 7 days. After 7 days the names will be released for re-use.

Not all operations are supported in Satellite environments. For details, see [supported Satellite APIs](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-apis-cos-satellite)

Syntax

DELETE https://{endpoint}/{bucket-name} path style
DELETE https://{bucket-name}.{endpoint} virtual host style



 Configure Object Lock on an existing bucket 

Not all operations are supported in Satellite environments. For details, see [supported Satellite APIs](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-apis-cos-satellite)

A PUT request addressed to an empty bucket with the ?object-lock query parameter will set a new object lock configuration on a bucket.

Syntax

PUT https://{endpoint}/{bucket-name}?object-lock path style
PUT https://{bucket-name}.{endpoint}?object-lock virtual host style

The Object Lock configuration is provided as XML in the body of the request. New requests will overwrite any existing replication rules that are present on the bucket.

An Object Lock configuration must include one rule.



 Header Type Description 

 Content-MD5 String Required: The base64 encoded 128-bit MD5 hash of the payload, which is used as an integrity check to ensure that the payload wasn't altered in transit. 



The body of the request must contain an XML block with the following schema:



 Element Type Children Ancestor Constraint 

 ObjectLockConfiguration Container ObjectLockEnabled, Rule None Limit 1. 
 ObjectLockEnabled String None ObjectLockConfiguration The only valid value is ENABLED. 
 Rule Container DefaultRetention ObjectLockConfiguration Limit 1 
 DefaultRetention Container Days, Mode, Years Rule Limit 1.
[Learn more](https://cloud.ibm.com/docs/devsecops?topic=devsecops-cd-devsecops-cos-bucket-evidencecd-devsecops-cos-bucket-resiliency).





 Naming your bucket 

Cloud Object Storage object names consist of the following components:

{NAMESPACE} / {PIPELINE_RUN_ID} / {TYPE} / {FILE_NAME} _ {HASH}



 Examples: 

ci/48decaa9-9042-498f-b58d-3577e0ac0158/evidences/build-vulnerability-advisor.json_362c06afa88b3f304878f0d0979e834f

ci/48decaa9-9042-498f-b58d-3577e0ac0158/artifacts/app-image-va-report.json_b3f30487f0d0979e834f362c06afaaa8

The name component {NAMESPACE} / {PIPELINE_RUN_ID} / {TYPE} is useful as a prefix when you're looking for collected data from a certain pipeline run.





 Retention policy 

You can set Cloud Object Storage buckets to enforce a retention policy or period for uploaded objects, otherwise known as [Immutable Object Storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-immutable). Immutable Object Storage preserves electronic records and maintains data integrity. Retention policies ensure that data is stored in a Write-One-Read-Many (WORM), non-erasable, and non-rewritable manner. You cannot change or delete objects in protected buckets within the retention period, or delete protected buckets with objects themselves until the retention period is over. The policy is enforced until the end of a retention period and the removal of any legal holds.

It is recommended that teams set a retention policy for the buckets that are used as their evidence locker that stores every object for a minimum of 365 days.







 Audit event log
Your storage instances remain active until the next anniversary date and you can continue to use them until this date, such as to give your team time to make backups of your data.



* When you dynamically provisioned the storage with a storage class that sets reclaimPolicy: Delete and you choose to remove the PVC, the PV and the storage instance are immediately removed. For hourly billed storage, billing stops immediately. For monthly billed storage, you are still charged for the remainder of the month. After your storage is removed and billing stops, you might still see your storage instance in the console or the CLI for up to 72 hours.



What do I need to be aware of before I delete persistent storage?
: When you clean up persistent storage, you delete all the data that is stored in it. If you need a copy of the data, make a backup for [file storage](https://cloud.ibm.com/docs/containers?topic=containers-file_storagefile_backup_restore) or [block storage](https://cloud.ibm.com/docs/containers?topic=containers-block_storageblock_backup_restore).

I deleted my storage instance. Why can I still see my instance?
: After you remove persistent storage, it can take up to 72 hours for the removal to be fully processed and for the storage to disappear from your IBM Cloud console or CLI.

I deleted my cluster. How do I remove the remaining storage volumes?
: See the steps in [Why am I still seeing charges for block storage devices after deleting my cluster](https://cloud.ibm.com/docs/containers?topic=containers-ts_storage_clean_volume).





 Cleaning up persistent storage 

Remove the PVC, PV, and the storage instance from your IBM Cloud account to avoid further charges for your persistent storage.

Before you begin:



* Make sure that you backed up any data that you want to keep.
* [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster)
New requests will overwrite any existing replication rules that are present on the object, provided the RetainUntilDate is farther in the future than the current value.



 Header Type Description 

 Content-MD5 String Required: The base64 encoded 128-bit MD5 hash of the payload, which is used as an integrity check to ensure that the payload wasn't altered in transit. 



Optionally, you can specify the version for which to apply the RetainUntilDate.





 Optional query parameters 



 Parameter Required? Type Description 

 versionID Optional string Version ID. 



The body of the request must contain an XML block with the following schema:



 Element Type Children Ancestor Constraint 

 Retention Container Mode, RetainUntilDate None Required Limit 1. 
 Mode String None Retention Required Only COMPLIANCE is supported at this time (case-sensitive). 
 RetainUntilDate String None Retention Required The date after which an object is eligible for deletion in ISO8601 Date-Time Format. 



This example will retain any new objects for at least until March 12, 2023.

curl -X "PUT" "https://$BUCKET.s3.$REGION.cloud-object-storage.appdomain.cloud/?retention" 
-H 'Authorization: Bearer $TOKEN' 
-H 'Content-MD5: fT0hYstki6zUvEh7abhcTA==' 
-H 'Content-Type: text/plain; charset=utf-8' 
-d $'<Retention>
<Mode>COMPLIANCE</Mode>
<RetainUntilDate>2023-03-12T23:01:00.000Z</RetainUntilDate>
</Retention>'

A successful request returns a 200 response.

In the event that the RetainUntilDate values is not beyond any existing value, the operation will fail with a 403 Access Denied.





 Add or remove a legal hold for an object 

The Object Lock configuration is provided as XML in the body of the request. New requests will overwrite any existing replication rules that are present on the object, provided the RetainUntilDate is farther in the future than the current value.



 Header Type Description
If both are specified a 400 error will be returned. If neither is specified the bucket's DefaultRetention period will be used. 
 Retention-legal-hold-id string A single legal hold to apply to the object. A legal hold is a Y character long string. The object cannot be overwritten or deleted until all legal holds associated with the object are removed. 



function putObjectAddLegalHold(bucketName, objectName, legalHoldId) {
console.log(Add legal hold ${legalHoldId} to ${objectName} in bucket ${bucketName} with a putObject operation.);
return cos.putObject({
Bucket: bucketName,
Key: objectName,
Body: 'body',
RetentionLegalHoldId: legalHoldId
}).promise()
.then((data) => {
console.log(Legal hold ${legalHoldId} added to object ${objectName} in bucket ${bucketName});
})
.catch((e) => {
console.log(ERROR: ${e.code} - ${e.message}n);
});
}

function copyProtectedObject(sourceBucketName, sourceObjectName, destinationBucketName, newObjectName, ) {
console.log(Copy protected object ${sourceObjectName} from bucket ${sourceBucketName} to ${destinationBucketName}/${newObjectName}.);
return cos.copyObject({
Bucket: destinationBucketName,
Key: newObjectName,
CopySource: sourceBucketName + '/' + sourceObjectName,
RetentionDirective: 'Copy'
}).promise()
.then((data) => {
console.log(Protected object copied from ${sourceBucketName}/${sourceObjectName} to ${destinationBucketName}/${newObjectName});
})
.catch((e) => {
In both cases, you are billed until the end of the current billing cycle, and billing stops for the next billing cycle. After your storage is canceled, you might still see your storage instance in the console or the CLI for up to 72 hours.



* Immediate cancellation: Choose this option to immediately remove your storage. Neither you nor your users can use the storage anymore or recover the data.



* Anniversary date: Choose this option to cancel your storage on the next anniversary date. Your storage instances remain active until the next anniversary date and you can continue to use them until this date, such as to give your team time to make backups of your data.



* When you dynamically provisioned the storage with a storage class that sets reclaimPolicy: Delete and you choose to remove the PVC, the PV and the storage instance are immediately removed. For hourly billed storage, billing stops immediately. For monthly billed storage, you are still charged for the remainder of the month. After your storage is removed and billing stops, you might still see your storage instance in the console or the CLI for up to 72 hours.



What do I need to be aware of before I delete persistent storage?
: When you clean up persistent storage, you delete all the data that is stored in it. If you need a copy of the data, make a backup for [file storage](https://cloud.ibm.com/docs/containers?topic=containers-file_storagefile_backup_restore) or [block storage](https://cloud.ibm.com/docs/containers?topic=containers-block_storageblock_backup_restore).

I deleted my storage instance. Why can I still see my instance?
: After you remove persistent storage, it can take up to 72 hours for the removal to be fully processed and for the storage to disappear from your IBM Cloud console or CLI.

I deleted my cluster. How do I remove the remaining storage volumes?
: See the steps in [Why am I still seeing charges for block storage devices after deleting my cluster](https://cloud.ibm.com/docs/containers?topic=containers-ts_storage_clean_volume).





 Cleaning up persistent storage
In both cases, you are billed until the end of the current billing cycle, and billing stops for the next billing cycle. After your storage is canceled, you might still see your storage instance in the console or the CLI for up to 72 hours.



* Immediate cancellation: Choose this option to immediately remove your storage. Neither you nor your users can use the storage anymore or recover the data.



* Anniversary date: Choose this option to cancel your storage on the next anniversary date. Your storage instances remain active until the next anniversary date and you can continue to use them until this date, such as to give your team time to make backups of your data.



* When you dynamically provisioned the storage with a storage class that sets reclaimPolicy: Delete and you choose to remove the PVC, the PV and the storage instance are immediately removed. For hourly billed storage, billing stops immediately. For monthly billed storage, you are still charged for the remainder of the month. After your storage is removed and billing stops, you might still see your storage instance in the console or the CLI for up to 72 hours.



What do I need to be aware of before I delete persistent storage?
: When you clean up persistent storage, you delete all the data that is stored in it. If you need a copy of the data, make a backup for [file storage](https://cloud.ibm.com/docs/openshift?topic=openshift-file_storagefile_backup_restore) or [block storage](https://cloud.ibm.com/docs/openshift?topic=openshift-block_storageblock_backup_restore).

I deleted my storage instance. Why can I still see my instance?
: After you remove persistent storage, it can take up to 72 hours for the removal to be fully processed and for the storage to disappear from your IBM Cloud console or CLI.

I deleted my cluster. How do I remove the remaining storage volumes?
: See the steps in [Why am I still seeing charges for block storage devices after deleting my cluster](https://cloud.ibm.com/docs/containers?topic=containers-ts_storage_clean_volume).





 Cleaning up persistent storage