vania-janet/multiturn-rag-retrieval-data huggingface dataset:
If a limit is exceeded, you receive an exception and are not allowed to create any new resources beyond that limit.

If you have a specific use case that requires an extended limit, you can request an increase. For more information, see [Increasing account limits](https://cloud.ibm.com/docs/account?topic=account-account-limits).



Table 1. IAM account limits

 Resource Max 

 Access groups per account 500 
 Access groups per user 50 
 Access management tags per account 250 
 API Keys per identity 20 
 Custom roles per account 40 
 Dynamic rules per access group 5 
 Dynamic rules per trusted profile 20 
 Dynamic rules per Identity provider (IdP) 2000 
 IdPs per account 5 
 Policies per account <br><br>[1] 4020 
 Policies per subject within an account 1000 
 Policies with access management tags within an account 25 
 Service IDs per account 2000 
 Trusted profiles per account 2000 
 Users per trial account 100 
 Users per billable account 7500 



A maximum of 1,000 policies and service to service authorizations within one account is recommended to ensure optimal performance within your account. For more information about limiting the number of policies in your account, see the [Best practices for organizing resources and assigning access](https://cloud.ibm.com/docs/account?topic=account-account_setup).

If you want to check the number of policies in your account, see [Viewing the total number of policies per account](https://cloud.ibm.com/docs/account?topic=account-account-limits&interface=clitotal-number-policies-cli). To request an increase in the account limit, see [Requesting a policy and rule shared limit increase](https://cloud.ibm.com/docs/account?topic=account-account-limits&interface=clilimit-increase).



 Policy limitations based on attributes 

Access management tags are only available when you create an access policy that is scoped for all IAM-enabled services. In this case, when you enable the access based on tags, no other attributes can be added.
* If the original volume is the Endurance 0.25 tier, then the IOPS tier can’t be updated.
* Older storage types can't be upgraded. Ensure that the storage was ordered in an upgraded Data Center that allows for [Expandable storage](https://cloud.ibm.com/docs/FileStorage?topic=FileStorage-expandCapacity).
* The permissions that you have in the Cloud console can be a factor. For more information, see the topics within [User roles and permissions](https://cloud.ibm.com/docs/account?topic=account-userroles).



* Are File Storage for Classic volumes thin or thick provisioned?

All Block and File Storage for Classic services are thin-provisioned. This method is not modifiable.
* My billing ID changed, what does this mean?

You might notice that your Storage volumes are now billed as "Endurance Storage Service” or "Performance Storage Service" instead of "Enterprise Storage". You might also have new options in the console, such as the ability to adjust IOPS or increase capacity. IBM Cloud® strives to continuously improve storage capabilities. As hardware gets upgraded in the data centers, storage volumes that reside in those data centers are also upgraded to use all enhanced features. The price that you pay for your Storage volume does not change with this upgrade.
* How durable is File Storage for Classic?

When you store your data in File Storage for Classic, it's durable, highly available, and encrypted. The durability target for a single Availability zone is 99.999999999% (11 9's). For more information, see [Availability and Durability of File Storage for Classic](https://cloud.ibm.com/docs/FileStorage?topic=FileStorage-storageavailability).
* What's the average uptime for File Storage for Classic?

When you store your data in File Storage for Classic, it's durable, highly available, and encrypted. File Storage is built upon best-in-class, proven, enterprise-grade hardware and software to ensure high availability and uptime.
<prices SOAP-ENC:arrayType="ns1:SoftLayer_Product_Item_Price[3]" xsi:type="SOAP-ENC:Array">
<item xsi:type="ns1:SoftLayer_Product_Item_Price">
<id xsi:type="xsd:int">189433</id> 
</item>
<item xsi:type="ns1:SoftLayer_Product_Item_Price">
<id xsi:type="xsd:int">196033</id> 
</item>
<item xsi:type="ns1:SoftLayer_Product_Item_Price">
<id xsi:type="xsd:int">196093</id> 
</item>
</prices>
</orderData>
</ns1:placeOrder>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>



The Operating system must rescan the storage and reload the multipath device map to reflect the expanded volume size. Resizing of the partition and file system are also required. For more information about expanding the file system, see your OS Documentation. For example, [RHEL 8 - Modifying Logical Volume](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_logical_volumes/modifying-the-size-of-a-logical-volume_configuring-and-managing-logical-volumes) or [Microsoft® - Extend a basic volume](https://docs.microsoft.com/en-us/windows-server/storage/disk-management/extend-a-basic-volume).





 Resizing storage with Terraform 

You can increase your storage capacity by using the ibm_storage_block resource, and specifying a larger number in the capacity argument. The following example increases the capacity of an Endurance volume to 40 GB.

resource "ibm_storage_block" "test1" {
type = "Endurance"
To improve performance, you can try [adjusting the host queue depth settings](https://cloud.ibm.com/docs/BlockStorage?topic=BlockStorage-hostqueuesettings) or [enabling Jumbo frames](https://cloud.ibm.com/docs/BlockStorage?topic=FileStorage-jumboframes).





 What happens when a smaller block size is used to measure performance? 

Maximum IOPS can still be obtained when you use smaller block sizes. However, throughput becomes smaller. For example, a volume with 6000 IOPS would have the following throughput at various block sizes:



* 16 KB * 6000 IOPS == 93.75 MB/sec
* 8 KB * 6000 IOPS == 46.88 MB/sec
* 4 KB * 6000 IOPS == 23.44 MB/sec







 How can I tell how much of the Storage is being used? Why are the usage details of Block Storage for Classic not displayed in the UI? 

Block Storage for Classic is yours to format and manage the way that you want to. IBM Cloud® can't see the contents of the LUN, and so the UI can't provide information about the disk space usage. You can obtain more information about the volume, such as how much disk space is taken and how much is available, from your Compute host's operating system.

You can use the following commands.



* Linux®:

df -h

The command provides an output that shows how much space is available space and the percentage used.

$ df -hT /dev/sda1
Filesystem Type Size Used Avail Use% Mounted on
/dev/sda1 disk 6.0G 1.2G 4.9G 20% /
* Windows®: you have two options.

fsutil volume diskfree C:

dir C:

The last line of the output shows how much space is unused.

You can also view the free disk space in the File Explorer by clicking This PC.







 Does the volume need to be pre-warmed to achieve expected throughput? 

Pre-warming is not needed. You can observe specified throughput immediately upon provisioning the volume.
If you want to use a different file system, such as xfs or ext3, [create a customized storage class](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-blockvpc-customize-storage-class).



* 10 IOPS/GB: ibmc-vpc-block-10iops-tier or ibmc-vpc-block-retain-10iops-tier
* 5 IOPS/GB: ibmc-vpc-block-5iops-tier or ibmc-vpc-block-retain-5iops-tier
* 3 IOPS/GB: ibmc-vpc-block-general-purpose or ibmc-vpc-block-retain-general-purpose
* Custom: ibmc-vpc-block-custom or ibmc-vpc-block-retain-custom



3. Decide on your Block Storage for VPC configuration.



1. Choose a size for your storage. Make sure that the size is supported by the Block Storage for VPC profile that you chose.
2. Choose if you want to keep your data after the cluster or the persistent volume claim (PVC) is deleted.



* If you want to keep your data, then choose a retain storage class. When you delete the PVC, only the PVC is deleted. The persistent volume (PV), the physical storage device in your IBM Cloud account, and your data still exist. To reclaim the storage and use it in your cluster again, you must remove the PV and follow the steps for using existing Block Storage for VPC.
* If you want the PV, the data, and your physical Block Storage for VPC device to be deleted when you delete the PVC, choose a storage class without retain.





4. Create a configuration file to define your persistent volume claim and save the configuration as a YAML file.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: <pvc_name> Enter a name for your PVC.
spec:
accessModes:
- ReadWriteOnce ReadWriteOnce is the only supported access mode.
* If you want the PV, the data, and your physical Block Storage for VPC device to be deleted when you delete the PVC, choose a storage class without retain.





4. Create a configuration file to define your persistent volume claim and save the configuration as a YAML file.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: <pvc_name> Enter a name for your PVC.
spec:
accessModes:
- ReadWriteOnce ReadWriteOnce is the only supported access mode. You can mount the PVC to one pod on one worker node in the cluster at a time.
resources:
requests:
storage: 10Gi Enter the size. Make sure that the size is supported in the profile that you chose.
storageClassName: <storage_class> Enter the storage class name that you selected earlier.
5. Create the PVC in your cluster.

kubectl apply -f pvc.yaml
6. Verify that your PVC is created and bound to the PV. This process can take a few minutes.

kubectl describe pvc <pvc_name>

Example output

Name: mypvv
Namespace: default
StorageClass: ibmc-vpc-block-5iops-tier
Status: Bound
Volume:
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"name":"csi-block-pvc-good","namespace":"default"},"spec":{...
volume.beta.kubernetes.io/storage-provisioner: vpc.block.csi.ibm.io
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 10Gi
Access Modes:
VolumeMode: Filesystem
Events:
Type Reason Age From Message
Planning for storage 

Review the following sections for an overview of the available storage options for your cluster.

Before you can decide what type of storage is the correct solution for your IBM Cloud® Kubernetes Service clusters, you must understand the IBM Cloud infrastructure provider, your app requirements, the type of data that you want to store, and how often you want to access this data.

Decide whether your data must be permanently stored.
: Persistent storage: Data stored on persistent storage persists even when the container, the worker node, or the cluster is removed. Use persistent storage in the for stateful apps, core business data or data that must be available due to legal requirements, such as a defined retention period. Persistent storage is also a good option for auditing.
: Non-persistent storage: Your data can be removed when the container, the worker node, or the cluster is removed. Non-persistent storage is typically used for logging information, such as system logs or container logs, development testing, or when you want to access data from the host's file system.

If you must persist your data, analyze if your app requires a specific type of storage. When you use an existing app, the app might be designed to store data in one of the following ways.
: In a file system: The data can be stored as a file in a directory. For example, you could store this file on your local hard disk. Some apps require data to be stored in a specific file system, such as nfs or ext4 to optimize the data store and achieve performance goals.
: In a database: The data must be stored in a database that follows a specific schema. Some apps come with a database interface that you can use to store your data. For example, WordPress is optimized to store data in a MySQL database. In these cases, the type of storage is selected for you.

Determine the type of data that you want to store.
: Structured data: Data that you can store in a relational database where you have a table with columns and rows. Data in tables can be connected by using keys and is usually easy to access due to the pre-defined data model. Examples are phone numbers, account numbers, Social Security numbers, or postal codes.
There are two possible causes for high storage usage:



* The storage attached to your system is reaching maximum capacity.
* There is empty space on the disks that must be reclaimed.







 Diagnosing and resolving 

If the output from the previous SQL query indicates that storage usage on your instance is high, the following steps will help to diagnose and resolve this scenario:



1. Check if the instance has reclaimable space by running the following SQL query as an admin user:

db2 "SELECT SUBSTR(TBSP_NAME, 1,30) AS TBSP_NAME, SUM(TBSP_USED_PAGES * TBSP_PAGE_SIZE)/ 1073741824 AS TBSP_USED_SIZE_GB, SUM((TBSP_PENDING_FREE_PAGES+ tbsp_free_pages) TBSP_PAGE_SIZE)/ 1073741824 AS FREE_OR_PENDING_GB FROM TABLE(MON_GET_TABLESPACE('',-2)) where TBSP_NAME not like '%SYS%' and TBSP_NAME not like '%TEMP%' and RECLAIMABLE_SPACE_ENABLED = 1 and (TBSP_PENDING_FREE_PAGES+ tbsp_free_pages) >= 1 GROUP BY TBSP_NAME ORDER BY 3 DESC"
. If the data is very skewed across Db2 partitions, meaning that one or more Db2 partitions is almost full while other Db2 partitions still have significant capacity remaining, then see [Choosing a hash distribution key for a table in an MPP database](https://www.ibm.com/support/knowledgecenter/SS6NHC/com.ibm.swg.im.dashdb.doc/learn_how/choosing_dist_key_mpp.html){: external}.
2. If one or more table spaces have reclaimable space, complete the following steps to reclaim that space:

Do not reclaim space during an online backup. The reclaim operation can also impact the performance of ongoing workloads, so it is best to run the following steps during off-peak hours.

a. Verify that an online backup is not running:
Remove operating system ISCSI sessions and, if applicable, unmount the device.
2. Revoke access for the host from the storage device in the [IBM Cloud® console](https://cloud.ibm.com/login).
3. Remove automatic discovery, and if applicable, remove connect database entries from the operating system for ISCSI connections.







 How do endurance and performance storage differ? 

Endurance and Performance are provisioning options that you can select for storage devices. In short, Endurance IOPS tiers offer predefined performance levels whereas you can fine-tune those levels with the Performance tier. The same devices are used but delivered with different options. For more information, see [IBM Cloud Block Storage: Details](https://www.ibm.com/cloud/block-storage/details).





 I am unable to upgrade storage. What can affect the ability to upgrade or expand storage? 

The following situations can affect the ability to upgrade or expand storage:



* If the original volume is the Endurance 0.25 tier, then the IOPS tier can't be updated.
* Older storage types can't be upgraded. For more information, see [expanding Block Storage for Classic Capacity](https://cloud.ibm.com/docs/BlockStorage?topic=BlockStorage-expandingcapacity).
* The permissions that you have in the [IBM Cloud® console](https://cloud.ibm.com/login) can be a factor. For more information, see the topics within [User roles and permissions](https://cloud.ibm.com/docs/account?topic=account-userroles).







 Are ISCSI LUNs thin or thick provisioned? 

All File and Block Storage for Classic services are thin-provisioned. This method is not modifiable.





 My billing ID changed, what does this mean? 

You might notice that your Storage volumes are now billed as "Endurance Storage Service” or "Performance Storage Service" instead of "Enterprise Storage". You may also have new options in the console, such as the ability to adjust IOPS or increase capacity. IBM Cloud® strives to continuously improve storage capabilities.
: For storage that was [statically provisioned](https://cloud.ibm.com/docs/containers?topic=containers-kube_conceptsstatic_provisioning), VPC Block Storage, or storage that you provisioned with a storage class that sets reclaimPolicy: Retain, you must manually remove the PVC, PV, and the storage instance to avoid further charges.

How does the billing stop after I delete my storage?
: Depending on what storage components you delete and when, the billing cycle might not stop immediately. If you delete the PVC and PV, but not the storage instance in your IBM Cloud account, that instance still exists and you are charged for it.

If you delete the PVC, PV, and the storage instance, the billing cycle stops depending on the billingType that you chose when you provisioned your storage and how you chose to delete the storage.



* When you manually cancel the persistent storage instance from the IBM Cloud console or the ibmcloud sl CLI, billing stops as follows:



* Hourly storage: Billing stops immediately. After your storage is canceled, you might still see your storage instance in the console for up to 72 hours.
* Monthly storage: You can choose between immediate cancellation or cancellation on the anniversary date. In both cases, you are billed until the end of the current billing cycle, and billing stops for the next billing cycle. After your storage is canceled, you might still see your storage instance in the console or the CLI for up to 72 hours.



* Immediate cancellation: Choose this option to immediately remove your storage. Neither you nor your users can use the storage anymore or recover the data.



* Anniversary date: Choose this option to cancel your storage on the next anniversary date. Your storage instances remain active until the next anniversary date and you can continue to use them until this date, such as to give your team time to make backups of your data.



* When you dynamically provisioned the storage with a storage class that sets reclaimPolicy: Delete and you choose to remove the PVC, the PV and the storage instance are immediately removed. For hourly billed storage, billing stops immediately. For monthly billed storage, you are still charged for the remainder of the month.
Definitions of the term data lake vary, but in the context of this tutorial, a data lake is an approach to storing data in its native format for organizational use. To that end, you will create a data lake for your organization using Object Storage. By combining Object Storage and Data Engine, data analysts can query data where it lies using Structured Query Language (SQL). You'll also leverage the Data Engine (previously SQL Query) service in a Jupyter Notebook to conduct a simple analysis. When you're done, allow non-technical users to discover their own insights.

![solution tutorial icon](https://cloud.ibm.com/media/docs/images/homepage/solution-tutorial.svg) [Build a database-driven Slackbot](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-slack-chatbot-database-watson)Build a database-driven Slackbot Solution tutorial

In this tutorial, you are going to build a Slackbot which allows to search and create entries in a backend Db2 on Cloud database. The Slackbot is backed by the IBM Watson® Assistant service. You will integrate Slack and IBM Watson® Assistant using an Assistant integration. Db2 on Cloud is made available to Watson Assistant as custom extension.

![solution tutorial icon](https://cloud.ibm.com/media/docs/images/homepage/solution-tutorial.svg) [Build, deploy, test and monitor a predictive machine learning model](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-create-deploy-retrain-machine-learning-model)Build, deploy, test and monitor a predictive machine learning model Solution tutorial

This tutorial walks you through the process of building a predictive machine learning model, deploying the generated model as an API to be used in your applications and testing the model all of this happening in an integrated and unified self-service experience on IBM Cloud. You will then monitor the deployed model with IBM Watson OpenScale.

!
* 15 minutes
* 2023-01-31



[Build a data lake using object storage](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-smart-data-lake)Build a data lake using object storage

Definitions of the term data lake vary, but in the context of this tutorial, a data lake is an approach to storing data in its native format for organizational use. To that end, you will create a data lake for your organization using Object Storage. By combining Object Storage and Data Engine, data analysts can query data where it lies using Structured Query Language (SQL). You'll also leverage the Data Engine (previously SQL Query) service in a Jupyter Notebook to conduct a simple analysis. When you're done, allow non-technical users to discover their own insights.

Object Storage Data Engine



* 1 hour
* 2023-06-14



Blockchain[Setting up multiregion High Availability (HA) deployments for the ordering service](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-hadr-mr-os)Setting up multiregion High Availability (HA) deployments for the ordering service

In this tutorial, you learn how to set up a Raft ordering service with five ordering nodes that span multiple regions for maximum high availability.

IBM Blockchain Platform



* 2023-02-17



[Using certificates from an external Certificate Authority](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-tutorial-extca)Using certificates from an external Certificate Authority

In this tutorial, you learn how to use certificates that were generated by an external Certificate Authority (CA) with your IBM® Blockchain Platform network. After you gather the required certificates for a peer or ordering node, you build a Membership Service Provider (MSP) definition that is used by your blockchain components.

IBM Blockchain Platform



* 30 minutes
* 2023-02-17
Overview 

Data lakes are an essential tool for storing structured and unstructured data on the cloud. With IBM® Netezza® Performance Server for IBM Cloud Pak® for Data as a Service, you can use external tables to access parquet files that are stored outside of your database in data lakes (on AWS S3). Also, you can analyze this data by using the robust and massively parallel Netezza Performance Server execution engine.

External data sources use connection strings to specify how you can access an external system. Each connection string describes where your data is placed and how to authenticate to your data source. Each external data source has a definition (schema), but the actual data exists external to the Netezza Performance Server database.

You cannot backup (nzbackup) and restore (nzrestore) external data source objects.

Use cases for external data source include:



*  [Running queries against parquet data that is stored in a data lake](https://cloud.ibm.com/docs/netezza?topic=netezza-querying_singularity).
*  [Ingesting data into Netezza Performance Server](https://cloud.ibm.com/docs/netezza?topic=netezza-ingest_singularity).
*  [Querying both local and remote data](https://cloud.ibm.com/docs/netezza?topic=netezza-merging_singularity).