vania-janet/multiturn-rag-retrieval-data huggingface dataset:
To limit the amount of secondary storage that is consumed by your pod, use resource requests and limits for [ephemeral storage](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/local-ephemeral-storage). 
 Data access pattern Read and write operations of any frequency Read and write operations of any frequency 
 Access Via the container's local file system Via [Kubernetes hostPath](https://kubernetes.io/docs/concepts/storage/volumes/hostpath) for access to worker node primary storage. Via [Kubernetes emptyDir volume](https://kubernetes.io/docs/concepts/storage/volumes/emptydir) for access to worker node secondary storage. 
 Performance High High with lower latency when you use SSD 
 Resiliency Low Low 
 Availability Specific to the container Specific to the worker node 
 Scalability Difficult to extend as limited to the worker node's secondary disk capacity Difficult to extend as limited to the worker node's primary and secondary disk capacity 
 Durability Data is lost when the container crashes or is removed. Data in hostPath or emptyDir volumes is lost when the worker node is deleted, the worker node is reloaded or updated, the cluster is deleted, the IBM Cloud account reaches a suspended state. In addition, data in an emptyDir volume is removed when the assigned pod is permanently deleted from the worker node, the assigned pod is scheduled on another worker node. 
 Common use cases Local image cache or container logs Setting up a high-performance local cache, accessing files from the worker node file system, or running unit tests. 
 Non-ideal use cases Persistent data storage or sharing data between containers Persistent data storage 







 Persistent storage for single zone clusters 

If you have a single zone cluster, you can choose between the following options in Red Hat OpenShift on IBM Cloud that provide fast access to your data. For higher availability, use a storage option that is designed for geographically distributed data and, if possible for your requirements, create a multizone cluster.

The following image shows the options that you have in Red Hat OpenShift on IBM Cloud to permanently store your data in a single cluster.

Zoom

!
That trusted app can assume the identity of the trusted profile and perform tasks with the assigned privileges.

The benefit of utilizing a trusted profile based on a compute resource is that this solution avoids using an API key. Thus, there are no requirements and challenges on how to create, store and protect any shared API key, how to assign and manage privileges. The app which assumes the identity of a trusted profile simply fetches a special compute resource token, then turns it into a regular IAM access token for the trusted profile. Thereafter, the intended tasks can be performed with the token provided for authentication.

See the blog post [Developer Tricks: Simulate Cloud Security for Local App Development](https://www.ibm.com/cloud/blog/developer-tricks-simulate-cloud-security-for-local-app-development) for some background on the compute resource token. Learn how to locally develop and test apps utilizing that token.





 Service ID 

Another method to establish trust is by specifying a service ID. The service ID can be from the same or other account. Because service IDs are unique identities across all IBM Cloud accounts, no further attributes need to be configured. With that setup in place, a service ID from an account A can now request to assume the identity of a trusted profile in account B and perform (administrative) tasks.





 Cloud service instance 

Similar to a service ID, it is possible to specify the cloud resource name (CRN) of an IBM Cloud service instance, so that the instance is a trusted resource. That service instance can be located in the same or another account. Right now, its only supported scenario is for an [enterprise project to deploy an architecture](https://cloud.ibm.com/docs/secure-enterprise?topic=secure-enterprise-tp-project). Projects, as service instances, with deployable architectures can be managed centrally in one account. By establishing trust through the project's CRN, it can assume the identity of a trusted profile in another account in the same or another enterprise account hierarchy, then deploy a solution pattern with its resources.







 Step 1: Trusted profile with compute resource 

To put theory into praxis, you are going to authorize a containerized app to perform tasks in an IBM Cloud account.
Delete the document as described in [Deleting a document that has not been associated with an annotation task](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-documents-for-annotationdeletenotask).
4. After you delete the document, re-create the annotation task and associate the same annotation set, which now has one less document in it.









 Deleting a document that is associated with an annotation task and human annotation has begun 

If the document you want to delete is associated with an annotation task and human annotation has begun, complete these steps to delete the document.

Do not delete a task if human annotation is in progress or you will lose the work that is in progress.



 Procedure 



1. Tell the human annotators to ignore the unwanted document in the set.
2. After all annotation work is complete on the other documents, and the human annotators submit all the documents to add the set to the ground truth, [review and accept the submitted documents](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-build-groundtruthwks_haaccuracy).
3. [Resolve any annotation conflicts](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-build-groundtruthwks_haadjudicate).
4. When all the documents are part of the ground truth and the task is complete, delete the task as described in [Deleting a document that is associated with an annotation task and human annotation has not begun](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-documents-for-annotationdeletenoannop).
5.
If the tool integration is misconfigured, edit the tool integration to correct the error condition, and then save the configuration.
* Check that the CRN of the Secrets Manager service instance that is configured in the tool integration references the same instance as the CRN of the failing secret.







 When I use Terraform or the API to configure a tool integration, why does the configuration fail with the A problem was encountered while attempting to resolve secret reference... error? 

The tool integration is configured with a reference to a secret, but the toolchain is not authorized to retrieve the secret from the secret store where the secret resides.

 What’s happening 

When you use Terraform or the API to create or update a tool integration, the configuration fails with the A problem was encountered while attempting to resolve secret reference... error message.

 Why it’s happening 

Many tool integrations include configuration properties that are classified as secrets. When you set these properties in a Terraform resource or API call, you can set them as [secrets references](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-cd_data_securitycd_secrets_references). A secret reference is a specially formatted string that identifies the name and location of a secret in a secret store that is integrated into the toolchain. With IBM Cloud secret store services such as Key Protect and Secrets Manager, the error indicates that the toolchain can contact the secret store service, but is not authorized to retrieve secrets from the service. Typically, this issue occurs because IBM Cloud Identity and Access Management (IAM) does not have a service-to-service authorization policy that allows the toolchain to read secrets from the service.

 How to fix it 

The error message includes the secret reference string that cannot be resolved.

Complete the following tasks for secrets references by name:



* Examine the first segment of the {vault::...} secret reference. This segment is the name of the secret store tool integration in the toolchain. For example, the {vault::my-kms.my-key} secret reference identifies a secret store tool integration that is named my-kms.
The number of servers is set to two and cannot be changed. Both servers have the same configuration.





 Hostname prefix 

The hostname prefix applies to all clusters in the instance. It must meet the following requirements:



* Only lowercase alphabetic, numeric, and dash (-) characters are allowed.
* The hostname prefix must start with a lowercase alphabetic character.
* The hostname prefix must end with a lowercase alphabetic or numeric character.
* The maximum length of the hostname prefix is 10 characters.







 Configure hostnames individually 

You can customize the hostnames prefix individually by toggling the switch on.

The hostnames prefix must meet the following requirements:



* Only lowercase alphabetic, numeric, and dash (-) characters are allowed.
* No consecutive dash characters are allowed.
* The hostname prefix must start with a lowercase alphabetic character.
* The hostname prefix must end with a lowercase alphabetic or numeric character.
* The maximum length of the hostname prefix is 13 characters.







 Uplink speed 

The uplink speed of 10 Gb is selected by default.





 Networking type 

Select either Public and private network or Private network only for the gateway cluster.







 Summary 

Based on your selected configuration for the cluster, the estimated price is instantly generated and displayed in the Summary pane. Click Pricing details to generate a PDF document with the price summary for the VMware Solutions resources.

You can also add the provisioned resources to the IBM Cloud estimate tool, by clicking Add to estimate. The estimate tool is useful if you want to get an approximate price of the selected VMware Solutions resources together with other IBM Cloud resources that you might consider purchasing.





 Procedure to add clusters to vCenter Server instances 



1. From the VMware Solutions console, click Resources > vCenter Server from the left navigation pane.
2. In the vCenter Server table, click the instance that you want to add clusters to.

Ensure that the instance status is Available. Otherwise, you cannot add clusters to the instance.
3. Click the Infrastructure tab and click Create on the upper right of the Clusters table.
4.
Tokens are constantly added to this bucket based on the sustained rate (X-RateLimit-Purge-Paths-Limit-Per-Second). However, if the bucket is full, no more tokens can be added.

When a multiple file purge request is made, the remaining number of tokens is checked against the number of file paths. The paths bucket must contain enough tokens to satisfy all of the paths in the request. If there are enough tokens in the purge bucket, the tokens are removed from the bucket and the request is accepted. If there are not enough tokens, tokens are not removed, and the request is denied.

This [example](https://cloud.ibm.com/docs/CDN?topic=CDN-code-examples-using-the-cdn-apicreate-group-example) shows the rate limiting response headers returned by the purge group API.

The following table lists the default values for rate limit headers:



Table 1. Default values for rate limit headers

 Rate Header Value Description 

 X-RateLimit-Purge-Paths-Limit-Per-Second 20 The token number added into the bucket per second. 
 X-RateLimit-Purge-Paths-Limit-Burst 1000 The maximum paths can be purged in a burst. Capability of the bucket. 
 X-RateLimit-Purge-Paths-Remaining N/A Number of paths remaining before the rate limit is exceeded. 







 Is there a limit on the number of purge group entries? 

The number of purge group entries doesn't have any restrictions.





 What is the maximum size for purge group name? 

The maximum purge group name size is 999 characters.





 I received an error similar to "Could not allocate more resource to build new rule in Akamai: x behaviors are needed while only x can be used." What should I do? 

When you reach the maximum resource limit, the console shows this error message. To resolve this issue, you can:



* Delete unused resources (TTL, origin, and so on).
* Create a domain to store new resources.
This period doubles between each consecutive retry, and never goes beyond 5 minutes. The minimum value before the first retry is 0.25 seconds. The default value is 10 retries.

socket_options
: A list of options to pass to the connection sockets. The available options can be found in the [documentation for the Erlang function setopts of the inet module](https://www.erlang.org/doc/man/inet.htmlsetopts-2). Default value is [{keepalive, true},{nodelay, false}].

worker_batch_size
: Worker processes run batches of replication tasks, where the batch size is defined by this parameter. The size corresponds to the number of _changes feed rows. Larger values for the batch size might result in better performance. Smaller values mean that checkpointing is done more frequently. Default value is 500.

worker_processes
: The number of processes the replicator uses in each replication task to transfer documents from the source to the target database. Higher values might produce better throughput because of greater parallelism in network and disk IO activities, but this improvement comes at the cost of requiring more memory and potentially CPU time. Default value is 4.

See the following example that includes performance options in a replication document:

{
"source": "https://$ACCOUNT1:$PASSWORD1@example.com/example-database",
"target": "https://$ACCOUNT2:$PASSWORD2@example.org/example-database",
"connection_timeout": 60000,
"retries_per_request": 20,
"http_connections": 30
}





 The effect of large attachments 

Having large numbers of attachments on documents might cause an adverse effect on replication performance.

For more information about the effect of attachments on replication performance, see [Performance considerations](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-attachmentsperformance-considerations).



 Avoiding the /_replicate endpoint
Build fails when the memory limit is exceeded 

 What’s happening 

After you create and run a build, your build does not complete successfully and you receive a message that the memory limit is exceeded.

 Why it’s happening 

If you receive a message that the memory limit is exceeded, then your build size is too small.

When a build runs, it is running steps, which include code compilations or container image packaging. These steps require memory. Depending on whether you choose a small, medium, large, or xlarge size for your build, a maximum amount of memory is available to a build run. When the maximum memory is reached, the build run is stopped with an error message; for example,

Example error message

Summary: Failed
Reason: OOMKilled

 How to fix it 

Try using the following information to resolve your problem.

Whether you are running your build in the console or in the CLI, use the CLI for troubleshooting problems with your build.



1. Run the ibmcloud ce buildrun get --name BUILDRUN_NAME command to display the details of your build run.
2. Review the Reason in the command output.



To resolve this problem, use a larger size for the build.

A larger build size also means that more memory and CPU cores are assigned to the build runs. Increasing this size will probably speed up the build runs, but also increases their cost.

For more information about build sizes, see [Determining build sizes](https://cloud.ibm.com/docs/codeengine?topic=codeengine-plan-buildbuild-size).



1. Use the [ibmcloud ce build update](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-build-update) command to update the build configuration to use a larger size; for example,

ibmcloud ce build update --name <BUILD_NAME> --size <SIZE>
2.
The value field must contain escaped string for the variable store for the list, map, or complex variable. For more information, see [Providing values to Schematics for the declared variables](https://cloud.ibm.com/docs/schematics?topic=schematics-create-tf-configdeclare-variable).





 Can I update the Terraform version (TF_VERSION) using a JSON file? 

Currently, the workaround to updating the TF_VERSION is to pass the TF_VERSION while updating the variable store. Schematics will auto detect what is specified in the Terraform version block in the TF files. This is the default behavior.

For more information, see [setting and changing the version](https://cloud.ibm.com/docs/schematics?topic=schematics-migrating-terraform-versionterraform-version-upgrade1x).





 Can start with a new Terraform state file on each job run? 

No, you need to only create new workspace. For more information, see [Workspace job execution](https://cloud.ibm.com/docs/schematics?topic=schematics-job-downloadwks-job-execution).





 Can I import an existing Terraform state file? 

Yes, you can use --state flag option through the [ibmcloud schematics workspace new](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-new).





 What is the maximum variable size? 

The maximum length of characters that Schematics Workspace variables support is 1 MB.





 What is the maximum state file size of import? 

The terraform.tfstate file size must be less than 2 MB. When you create workspace from an existing Terraform state file, the terraform.tfstate file size must be less than 2 MB. Greater than 2 MB state file size is not supported in the Schematics. You will see an error message with 413 Request Entity Too Large error when creating a new Schematics workspace.
mend-org-token The identifier of the organization. Use the org token returned by the onboarding request in the Org Token text field. 
 mend-user-key A secret Product ServiceUser that is created after ServiceNow onboarding. 
 mend-product-name Use the product name that is returned by the onboarding request in the Product Name on Mend text field. 
 mend-product-token Use the product token that is returned by the onboarding request in the Product Token text field 
 opt-in-mend Set this parameter to any nonempty string value to include the Mend Unified Agent scan as part of the Compliance Checks pipeline stage. 







 Optional Mend Unified Agent parameters 



Table 2. Optional Mend Unified Agent parameters

 Parameter name Default value Description 

 mend-config-file-name mendUnifiedAgent.config The file name of the Mend Unified Agent configuration file that the scan looks for in the root of each scanned source code repository. 
 mend-print-scan-results false (property is not set) Enter any nonzero length string to print each Mend scan results JSON file to the stage log. 
 mend-print-scan-summaries false (property is not set) Enter any nonzero length string to print each Mend scan summary report to the stage log. 
 mend-jar-url https://unified-agent.s3.amazonaws.com/wss-unified-agent.jar The scanner JAR file that is downloaded for each pipeline run. 









 Evidence and attachments 

The created evidence is based on the values in table 3. The DevSecOps pipeline uploads evidence to the locker and includes the evidence in the evidence summary for Change Requests.



Table 3. Evidence fields and values

 field value 

 tool type mend 
 evidence type com.ibm.code_vulnerability_scan 
 asset type repo 
 attachments <scan report containing detected libraries and list of vulnerabilities in JSON> 
 attachments <scan summary report in text format> 







 Debugging and logging 



Table 4. Debug parameters

 Parameter name Default value Description 

 pipeline-debug 0 Debug flag 0 off 1 on
disk_size Size in bytes of the data as stored on the disk. Views indexes aren't included in the calculation. 
 doc_count A count of the documents in the specified database. 
 doc_del_count Number of deleted documents. 
 instance_start_time Always 0. 
 other JSON object that contains a data_size field. 
 purge_seq The number of purge operations on the database. 
 sizes A JSON object, containing file, external, and active sizes. active is the size in bytes of data that is stored internally (excluding old revisions). external is the size in bytes of decompressed user data. This value is the billable data size. The other/data_size field is an alias for the external field. file is the size in bytes of data that is stored on the disk. Indexes aren't included in the calculation. The disk_size field is an alias for the file field. This size includes data that is waiting for compaction. 
 update_seq An opaque string that describes the state of the database. Don't rely on this string for counting the number of updates. 
 partitioned_indexes A JSON object that appears only if the database is partitioned. count is the number of partitioned indexes. indexes list the type of partitioned indexes, and limit shows the maximum number of allowed partitioned indexes. 



See the following example (abbreviated) response that contains database details:

{
"update_seq": "982...uUQ",
"db_name": "db",
"sizes": {
"file": 46114703224,
"external": 193164408719,
"active": 34961621142
},
"purge_seq": 0,
"other": {
"data_size": 193164408719
},
"doc_del_count": 5564,
"doc_count": 9818541,
"disk_size": 46114703224,
Capacity 

IBM Cloudant lets you: view the current and target provisioned throughput capacity setting, set the target provisioned throughput capacity setting, and view the current consumption of provisioned throughput capacity used.

For more information, see [provisioned throughput capacity](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-ibm-cloud-publicprovisioned-throughput-capacity) about how IBM Cloudant allocates and uses capacity, and how to view and change the capacity in the UI.

The Capacity API requires either IBM Cloudant legacy auth admin role or IAM Manager role to access the API endpoints. The following curl examples omit the authentication aspect for simplicity. See the [authentication overview](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-connectingauthentication-overview) section for more details on using both types of authentication.



 View the current and target provisioned throughput capacity setting 

Use a GET to the /_api/v2/user/capacity/throughput endpoint to see what amount of provisioned throughput capacity is allocated to the IBM Cloudant instance and what is the target provisioned throughput capacity. When you change the target capacity, the current capacity asynchronously changes to meet the target capacity. The size of the capacity change and the amount of data that is stored in the IBM Cloudant instance determines the time that it takes before the current and target capacities match. The capacity change is complete when the current and target capacity is the same.

Method
: GET

Path
: /_api/v2/user/capacity/throughput

Response
: Both the current and target capacity setting. Each includes the number of capacity blocks and total reads/sec, writes/sec, and global queries/sec of throughput capacity.

See the following example request to retrieve the current and target capacity by using HTTP:

GET /_api/v2/user/capacity/throughput

See the following example request to retrieve the current and target capacity:



* Curl
* Java
* Node
* Python
* Go