vania-janet/multiturn-rag-retrieval-data huggingface dataset:
CPU 1.0 12.0 [Contact IBM support](https://cloud.ibm.com/docs/get-support?topic=get-support-open-case&interface=ui) 
 Ephemeral storage 400 M 48 G <br>(limited by memory) [Contact IBM support](https://cloud.ibm.com/docs/get-support?topic=get-support-open-case&interface=ui) 
 Memory 4 G 48 G [Contact IBM support](https://cloud.ibm.com/docs/get-support?topic=get-support-open-case&interface=ui) 
 Retries 3 5 [Contact IBM support](https://cloud.ibm.com/docs/get-support?topic=get-support-open-case&interface=ui) 
 Timeout 7200 seconds (2 hours) 86400 seconds (24 hours) [Contact IBM support](https://cloud.ibm.com/docs/get-support?topic=get-support-open-case&interface=ui) 



Array indices are comma-separated lists or hyphen-separated range of indices, which specifies the job instances to run; for example, 1,3,6,9 or 1-5,7-8,10.

Number of instances is the number of job instances to run in parallel.

For more information about supported CPU and memory combinations, see [Supported memory and CPU combinations](https://cloud.ibm.com/docs/codeengine?topic=codeengine-mem-cpu-combo).

Code Engine is limited to 100 jobs per project. After 100 job runs are started, be sure to clean up older job runs before you start new job runs.



 Job size limit 

Code Engine limits the size of jobs and job runs with a maximum of 10 KiB. When you create or update jobs and job runs with the console, CLI, or API, Code Engine checks the size of the job or job run. If the operation exceeds the limit, a size limit exceeded error is given. If you receive this error, try reducing the size of your job or job run in one of the following ways.
access_id is the host_id obtained by: slcli block access-list <volume_id>

Options:
--subnet-id INTEGER ID of the subnets to assign; e.g.: --subnet-id 1234
-h, --help Show this message and exit.



* How many volumes can be ordered?

By default, you can provision a combined total of 700 block and file storage. To increase your volume limit, contact your sales representative. For more information, see [Managing storage limits](https://cloud.ibm.com/docs/BlockStorage?topic=BlockStorage-managingstoragelimits).
* How many Block Storage for Classic volumes can be mounted to a host?

That depends on what the host operating system can handle, it’s not something that IBM Cloud® limits. Refer to your OS Documentation for limits on the number of volumes that can be mounted.
* Can I attach multiple LUNs with different OS settings?

No. A host cannot be authorized to access LUNs of differing OS types at the same time. A host can be authorized to access LUNs of a single OS type. If you attempt to authorize a host to access multiple LUNs with different OS types, the operation results in an error.
* Which Windows® version am I to choose for my Block Storage for Classic LUN?

When you create a LUN, you must specify the OS type. The OS type specifies the operating system of the host that's going to access the LUN. It also determines the layout of data on the LUN, the geometry that is used to access that data, and the minimum and maximum size of the LUN. The OS Type can't be modified after the LUN is created. The actual size of the LUN might vary slightly based on the OS type of the LUN. Choosing the correct type for your Windows® OS helps to prevent mis-aligned IO operations.

If the LUN is being presented as a raw block device to a guest, select the OS type of the guest's OS. If the LUN is being presented to the hypervisor to serve Virtual Hard Drive (VHD) files, choose Hyper-V.
2. Use the [ibmcloud ce jobrun list](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-jobrun-list) command to list all defined jobs and confirm that you are referencing a defined job.
3. The image that is defined in your job cannot be overwritten by specifying a different image value when you submit a job run that refers to your job. If you want to use another image for your job run, create a new job that refers to the image that you want to use, or you can submit a job run that does not refer to the job.
4. The maximum size for jobs and job runs is 10 KiB. When you create or update jobs and job runs with the console, CLI, or API, Code Engine checks the size of the job or job run. If the operation exceeds the limit, you receive a size limit exceeded error. If you receive this error, try reducing the size of your job or your job run in one of the following ways:



* If you are using commands and arguments, reduce the use of these options, make them shorter, or move them into the container image that is used by your job or job run.
* If you are using environment variables, use fewer environment variables or make them shorter.
* --ip-name: The user-defined name for this reserved IP. Names must be unique within the subnet that the reserved IP resides in. Names beginning with ibm- are reserved for provider-owned resources.
* --allow-ip-spoofing: Disables the source and destination checks on this interface. If false, source IP spoofing is not allowed on this interface. One of: false, true.
* --primary-network-interface: PRIMARY_NETWORK_INTERFACE_JSON|@PRIMARY_NETWORK_INTERFACE_JSON_FILE, primary network interface in JSON or JSON file. For the data schema, see the primary_network_interface property in the [API documentation](https://cloud.ibm.com/apidocs/vpccreate-instance).
* --network-interface: NETWORK_INTERFACE_JSON|@NETWORK_INTERFACE_JSON_FILE, network interface attachment in JSON or JSON file. For the data schema, see the network_interfaces property in the [API documentation](https://cloud.ibm.com/apidocs/vpccreate-instance).
* --metadata-service, --ms: Enable or disable the Instance Metadata Service. One of: true, false.
* --metadata-service-protocol, --msp: The communication protocol for the metadata service endpoint. Applies only when the metadata service is enabled. One of: http, https. (default: http).
* --metadata-service-response-hop-limit, --msrhl: The hop limit (IP time to live) for IP response packets from the metadata service.
* --host-failure-policy: The action to perform if the compute host experiences a failure. One of: restart, stop. (default: restart).
* --resource-group-id: ID of the resource group. This ID is mutually exclusive with --resource-group-name.
* --resource-group-name: Name of the resource group. This name is mutually exclusive with --resource-group-id.
You can choose to evaluate your entire enterprise, specific account groups, or single accounts. It is recommended that if you want to evaluate a single account that you do so from that account.

If you are working in an enterprise account, you must have additional permissions for the enterprise to manage and see results. For more information, see [Assigning access](https://cloud.ibm.com/docs/security-compliance?topic=security-compliance-assign-roles).

A scope defines which resources in your accounts are evaluated. It is defined when you create an attachment by selecting the parent account or resource group that you want to evaluate. Anything that exists within that account or group is evaluated. So, for example, if you create an attachment at the enterprise account level, then all of the account groups and accounts within them are included in your evaluation. If you don't want to evaluate an account, you can always exclude it from your scope when you create the attachment. When an account is excluded, any of its child accounts are also excluded. However, as new accounts are added to your enterprise, they are automatically evaluated according to the parent account's attachment.

Check out the following diagram to see how three attachments can co-exist within an enterprise.

Zoom

![The image shows how two attachments are applied across an enterprise. One rule moves down the hierarchy. Another rule is attached only to a specific account, so its properties are applied only to the resources that it contains.](https://cloud.ibm.com/docs-content/v1/content/a6478671e8f845a84102d9cf49bf28ee7f4f3227/security-compliance/images/access-model.svg)

Figure 1. Attachment hierarchy

Attachment A
: In Attachment A, the target scope is the full enterprise. As you can see, all account groups and accounts that exist within the enterprise are evaluated. That is, unless they have been purposefully excluded.

Attachment B
: The target scope of Attachment B is a specific account group within an enterprise.
You can choose to evaluate your entire enterprise, specific account groups, or single accounts. It is recommended that if you want to evaluate a single account that you do so from that account.

If you are working in an enterprise account, you must have additional permissions for the enterprise to manage and see results. For more information, see [Assigning access](https://cloud.ibm.com/docs/security-compliance?topic=security-compliance-assign-roles).

A scope defines which resources in your accounts are evaluated. It is defined when you create an attachment by selecting the parent account or resource group that you want to evaluate. Anything that exists within that account or group is evaluated. So, for example, if you create an attachment at the enterprise account level, then all of the account groups and accounts within them are included in your evaluation. If you don't want to evaluate an account, you can always exclude it from your scope when you create the attachment. When an account is excluded, any of its child accounts are also excluded. However, as new accounts are added to your enterprise, they are automatically evaluated according to the parent account's attachment.

Check out the following diagram to see how three attachments can co-exist within an enterprise.

Zoom

![The image shows how two attachments are applied across an enterprise. One rule moves down the hierarchy. Another rule is attached only to a specific account, so its properties are applied only to the resources that it contains.](https://cloud.ibm.com/docs-content/v1/content/d4595e5202a9a27767cf034e81b038cdf772e0d5/secure-enterprise/includes/security-compliance/includes/security-compliance/images/access-model.svg)

Figure 1. Attachment hierarchy

Attachment A
: In Attachment A, the target scope is the full enterprise. As you can see, all account groups and accounts that exist within the enterprise are evaluated. That is, unless they have been purposefully excluded.

Attachment B
: The target scope of Attachment B is a specific account group within an enterprise.
"year",
"_id",
"title"
]
}





 The $in operator 

The $in operator requires that the document field must exist in the list provided.

See the following example that uses the $in operator:

{
"selector": {
"year": {
"$in": [2010, 2015]
}
},
"fields": [
"year",
"_id",
"title"
],
"limit": 10
}





 The $nin operator 

The $nin operator requires that the document field must not exist in the list provided.

See the following example that uses the $nin operator:

{
"selector": {
"year": {
"$nin": [2010, 2015]
}
},
"fields": [
"year",
"_id",
"title"
],
"limit": 10
}





 The $size operator 

The $size operator matches the length of an array field in a document.

See the following example that uses the $size operator:

{
"selector": {
"genre": {
"$size": 4
}
},
"fields": [
"title",
"genre"
],
"limit": 25
}





 The $mod operator 

The $mod operator matches documents where the expression (field % Divisor == Remainder) is true, and only when the document field is an integer. The Divisor and Remainder must be integers. They can be positive or negative integers. A query where the Divisor or Remainder is a non-integer returns a [404 status](https://cloud.ibm.com/apidocs/cloudantlist-of-http-codes).
Optimizing TCP window size for virtual instances 

Use the following guidelines to tune the TCP window size on a virtual server instance for optimal performance on an IBM Cloud® hypervisor. IBM Cloud virtual hosts have a default TCP buffer size with the range 3 - 6 MB.

To get the best performance out of your virtual server instance, you need to set the TCP window size under the 3 - 6 MB range. The kernel sets the actual memory limit to twice the requested value, effectively doubling the maximum settings. Keep the maximum TCP window size at or under 2 MB. With auto-tuning enabled, the receiver buffer size (and TCP window size) dynamically updates for each connection. The per connection memory space defaults are set with the following element arrays:



* net.ipv4.tcp_rmem - This is the memory that is reserved for TCP receive buffers
* net.ipv4.tcp_wmem - This is the memory that is reserved for TCP send buffers



The values in these arrays are the minimum, initial, and maximum buffer sizes that you use to set the limits on auto-tuning and balance memory usage. To view your current memory window settings, run the command: sysctl -a | grep mem

 sysctl -a | grep mem
vm.overcommit_memory = 0
vm.nr_hugepages_mempolicy = 0
vm.lowmem_reserve_ratio = 256 256 32
vm.meminfo_legacy_layout = 1
vm.memory_failure_early_kill = 0
vm.memory_failure_recovery = 1
net.core.wmem_max = 124928
net.core.rmem_max = 124928
net.core.wmem_default = 124928
net.core.rmem_default = 124928
net.core.optmem_max = 20480
net.ipv4.igmp_max_memberships = 20
net.ipv4.tcp_mem = 365184 486912 730368
Make sure that the volume is in the same zone as the worker node for the attachment to succeed.

The instructions in this topic are available for VPC worker nodes only. If you want to attach raw, unformatted block storage to a classic worker node, you must install the [IBM Cloud Block Storage attacher plug-in](https://cloud.ibm.com/docs/openshift?topic=openshift-utilitiesblock_storage_attacher).

Before you begin:

[Access your Red Hat OpenShift cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-access_cluster).



1. List your storage volumes and note the ID of the volume that you want to attach.

ibmcloud is vols
2. List the worker nodes in your cluster and note the ID of the worker node where you want to attach your volume.

ibmcloud oc worker ls -c <cluster_name_or_ID>
3. Attach your Block Storage for Classic to your VPC worker node.

ibmcloud oc storage attachment create --cluster <cluster_name_or_ID> --volume <volume> --worker <worker_ID>





 Removing raw Block Storage for VPC from VPC worker nodes by using the CLI 

You can remove storage from your worker node by using the ibmcloud oc storage attachment rm command.

[Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster)



1. List your storage volumes and note the ID of the volume that you want to remove.

ibmcloud is vols
2. Get the details of your volume such as the worker-id where the volume is attached. The worker-id is listed as the Instance name in the Volume Attachment Instance Reference section of the command output.

ibmcloud is vol <volume-ID>

Example output

Volume Attachment Instance Reference Attachment type Instance ID Instance name Auto delete Attachment ID Attachment name
For more information, see [Access management tags for file shares](https://cloud.ibm.com/docs/vpc?topic=vpc-file-storage-vpc-aboutfs-about-mgt-tags).
* What is the dp2 profile?

 What is the dp2 profile? 

The dp2 profile is the latest file storage profile, offering greater capacity and performance for your file shares. With this profile, ou can specify the total IOPS for the file share within the range for a specific file share size. You can provision shares with IOPS performance from 100 IOPS to 96,000 IOPS, based on share size. For more information, see [dp2 file storage profile](https://cloud.ibm.com/docs/vpc?topic=vpc-file-storage-profilesdp2-profile).
* Can I migrate all my file share profiles to dp2?

 Can I migrate all my file share profiles to dp2? 

You can migrate file shares that were created by using either the IOPS tier profile or custom IOPS profile to the latest dp2 profile. By migrating to the [dp2 profile](https://cloud.ibm.com/docs/vpc?topic=vpc-file-storage-profilesdp2-profile), you can take advantage of the latest File Storage for VPC features. Currently, you can use the File Storage for VPC UI, CLI, or API to revise a single file share profile. For migrating multiple shares, you need to create your own script that would first list these shares and then go through the list of shares and update each individual share profile.
* Can I mount a file share to a specific virtual server instance?

 Can I mount a file share to a specific virtual server instance? 

Yes. You can specify an access control mode that either uses Security Groups to restrict mounting a file share to specific resources in the VPC or allows VPC-wide file share mounting. File share mount targets created before 13-June-2023 have a default of VPC-wide file share mounting. File shares created after that date can specify security group access control mode to restrict access to specific virtual server instances.
To improve performance, you can try adjusting the host settings or [enabling Jumbo frames](https://cloud.ibm.com/docs/FileStorage?topic=FileStorage-jumboframes).
* What happens when I use a smaller block size for measuring performance?

Maximum IOPS can be obtained even if you use smaller block sizes. However, the throughput is less in this case. For example, a volume with 6000 IOPS has the following throughput at various block sizes:



* 16 KB * 6000 IOPS == 93.75 MB/sec
* 8 KB * 6000 IOPS == 46.88 MB/sec
* 4 KB * 6000 IOPS == 23.44 MB/sec



* Is the allocated IOPS enforced by instance or by volume?

IOPS is enforced at the volume level. Said differently, two hosts connected to a volume with 6000 IOPS share that 6000 IOPS.
* Does the volume need to be pre-warmed to achieve expected throughput?

Pre-warming is not needed. You can observe the specified throughput immediately upon provisioning the volume.
* Can more throughput be achieved if a faster Ethernet connection is used?

Throughput limits are set at a per-volume level. That limit cannot be increased by using a faster Ethernet connection. However, with a slower Ethernet connection, your bandwidth can be a potential bottleneck.
* Do firewalls and security groups impact performance?

It's best to run storage traffic on a VLAN, which bypasses the firewall. Running storage traffic through software firewalls increases latency and adversely affects storage performance.
* How do I route File Storage for Classic traffic to its own VLAN interface and bypass a firewall?

To enact this good practice, complete the following steps.



1. Provision a VLAN in the same data center as the host and the File Storage for Classic device.
2. Provision a secondary private subnet to the new VLAN.
3. Trunk the new VLAN to the private interface of the host. This action momentarily disrupts the network traffic on the host while the VLAN is being trunked to the host.
4.
Run the following command to check whether the Elasticsearch disk is full:

oc logs -l tenant=wd,run=elastic,ibm-es-master=True -c elasticsearch --tail=100000|grep 'disk watermark'
2. If the log shows a message such as watermark exceeded on x-data-1, it means the disk on the node that is specified is full and you need to increase the disk size by using the following command:

oc patch pvc $(oc get pvc -l tenant=wd,run=elastic,ibm-es-data=True,ibm-es-master=False -o jsonpath='{.items[N].metadata.name}') -p '{"spec": {"resources": {"requests":{"storage": "60Gi"}}}}'

where N denotes the data node number that was reported from the log.

For example, if the log mentions data-1 in the node name, then the command to use is:

oc patch pvc $(oc get pvc -l tenant=wd,run=elastic,ibm-es-data=True,ibm-es-master=False -o jsonpath='{.items[1].metadata.name}') -p '{"spec": {"resources": {"requests":{"storage": "60Gi"}}}}'







 Setting the shard limit in Discovery for Cloud Pak for Data 

In Discovery version 2.2.0, there is a limit to the number of shards that can stay open on a cluster. In development instances, the limit is 1,000 open shards, and in production instances, the limit is two data nodes, which is equal to 2,000 open shards, or 1,000 open shards per data node. After you reach either limit, you cannot create any more projects and collections on your cluster, and if you try to create a new project and collection, you receive an error message.

This limit is due to the fact that, when you install Discovery version 2.2.0, Elasticsearch version 7.8.0 automatically runs on your clusters.
ibmcloud is volume-create VOLUME_NAME PROFILE_NAME ZONE_NAME [--capacity CAPACITY] [--iops IOPS] [--resource-group-id RESOURCE_GROUP_ID | --resource-group-name RESOURCE_GROUP_NAME] [--tags TAG_NAME1,TAG_NAME2,...] [--json]

See the following example.

$ ibmcloud is volume-create demovolume1 custom us-east-1 --capacity 500 --iops 3000 --tags env:test,env:prod
Creating volume demovolume1 in resource group Default under account VPC 01 as user rtuser1@mycompany.com...

ID r014-e45f9c8c-4655-4a3e-9d90-70c2d64d1746
Name demovolume1
CRN crn:v1:bluemix:public:is:us-east-1:a/a10d63fa66daffc9b9b5286ce1533080::volume:r014-e45f9c8c-4655-4a3e-9d90-70c2d64d1746
Status pending
Attachment state unattached
Capacity 500
IOPS 3000
Bandwidth(Mbps) 6291
Profile custom
Encryption key -
Encryption provider_managed
Resource group defaults
Created 2023-07-24T16:20:52+00:00
Zone us-east-1
Health State inapplicable
Volume Attachment Instance Reference -
Active false
Unattached capacity update supported false
Unattached iops update supported false
Busy false
Tags env:test,env:prod

Capacity, indicated in megabytes, can range 10 - 16,000 GBs. If not specified, the default capacity is 100 GBs. IOPS values can be 100 - 48,000 IOPS, depending on the profile and volume size. If not specified, the IOPS value defaults to the valid configuration per volume profile.
The IOPS range within a custom band is based on the volume size. If you later [increase the size of a volume](https://cloud.ibm.com/docs/vpc?topic=vpc-expanding-block-storage-volumes), you can increase the IOPS again.

To adjust a volume's IOPS, the volume must be in an available state and the instance must be running. Your user authorization is verified before IOPS is adjusted.

You can use the [UI](https://cloud.ibm.com/docs/vpc?topic=vpc-adjusting-volume-iopsadjust-vpc-iops-ui-block), [CLI](https://cloud.ibm.com/docs/vpc?topic=vpc-adjusting-volume-iopsadjust-vpc-iops-cli-block), or [API](https://cloud.ibm.com/docs/vpc?topic=vpc-adjusting-volume-iopsadjust-vpc-iops-api-block) to adjust IOPS. You can adjust the volume's IOPS multiple times up to its maximum limit or reduce IOPS to its lower limit.

You can monitor the progress of your volume's IOPS change from the UI or CLI. You can also use the [Activity Tracker](https://cloud.ibm.com/docs/vpc?topic=vpc-at-events) to verify that the IOPS were adjusted.





 Limitations 

The following limitations apply to this release.



 General limitations 



* Primary boot volume IOPS cannot be adjusted.
* To adjust IOPS, the volume must be attached to a virtual server instance that is powered on and running.
* To adjust IOPS, the volume must be in an available state.







 Volume and IOPS limitations 



* For a volume that was created by using an [IOPS tier profile](https://cloud.ibm.com/docs/vpc?topic=vpc-block-storage-profilestiers), select a different profile for the volume size to increase or decrease IOPS.