vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* The current version of the document in the database.
* A user context, which provides details about the user that supplied the updated document.



The function inspects the request to determine whether the update is allowed to proceed. If the update is acceptable, the function returns. If the update is not acceptable, a suitable error object is returned. In particular, if the user is not authorized to make the update, an unauthorized error object is returned, along with an explanatory error message. Similarly, the requested update might not be allowed for some reason, such as when some mandatory fields are absent from the new document. In that case, a forbidden error object is returned, again with an explanatory error message.

For tombstone removal, a suitable validate_doc_update function would work as follows:



1. If the update is to apply a change to an existing document (oldDoc) within the target database, the function allows the change by returning.



The reason is that the update affected a document that was copied to the target database during the replication, but then changed in the source database during the replication. It's possible that the change was a DELETE, resulting in a tombstone record in the target database. The tombstone record is removed by a subsequent replication process at some point in the future. 2. The target database does not have a copy of the current document, and the document to be updated has the _deleted property (indicating that it's a tombstone). Therefore, the updated document must be a tombstone and was encountered before, so the update to the target database must be rejected.

See an example JavaScript validate_doc_update function to reject deleted documents not already present in the target database:

function(newDoc, oldDoc, userCtx) {
// any update to an existing doc is OK
if(oldDoc) {
return;
}

// reject tombstones for docs we don’t know about
if(newDoc["_deleted"]) {
throw({forbidden : "Deleted document rejected"});
}

return; // Not strictly necessary, but clearer.
}
Once you have deployed your applications to a new IBM Cloud® hosting compute environment, there are several ways to move your user's routing from the IBM Cloud Foundry hosted application to your application running on your selected IBM compute service.



* You can redirect traffic from the IBM Cloud Foundry domain or host to the new location.
* If you are using a load balancer you can use it to send your traffic to your IBM Cloud Foundry hosted app. Then, you can slowly change the percentage of traffic from the IBM Cloud Foundry hosted app to your application running on your selected IBM compute service. By slowly migrating you can ensure that your application is operating properly on the new service and can that it can handle the workload.







 Step 5: Shutting down IBM Cloud Foundry applications 

After you have migrated your applications to the new IBM Cloud® compute service and your traffic is being handled 100% by the new service, you are ready to turn off your IBM Cloud Foundry application usage. Stop your IBM Cloud Foundry applications and delete them. After deleting your applications, your migration is complete.







 Migration assistance 

If you feel your organization needs help with your IBM Cloud Foundry migration, or wants to engage another organization to assist you, here are some internal and external IBM partners that have a great deal of skill and experience with cloud application migrations and cloud native development improvements and transformations. Contact them if needed to discuss your specific requirements.

These are for-pay migration assistance options by IBM teams and third-party partners that are available to you. IBM does not provide support for the listed third-party partners and is providing their information for your information and use as you see fit.

IBM Cloud® Expert Labs



* IBM Cloud Expert Labs is the IBM Cloud Platform Professional Services team providing deep technical expertise and a proven approach to modernize applications on IBM Cloud platform technologies. For your Cloud Foundry journey, we offer both Advisory Services to assess, recommend, and plan your Cloud Foundry modernization approach and Migration Services to assist with application remediation and migration. See [this document](https://cloud.ibm.com/media/docs/downloads/cloud-foundry-public/IBM%20Cloud%20Expert%20Labs%20-%20Migration%20Experts.pdf) for more information and contacts to get started.



HCL Technologies
For more information about what is included in a security patch, refer to the [version change log](https://cloud.ibm.com/docs/containers?topic=containers-cs_versionscs_versions_available).
* Does the service offer support for bare metal and GPU?

Certain VPC worker node flavors offer GPU support. For more information, see the [VPC flavors](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-flavors).

Yes, you can provision your worker node as a single-tenant physical bare metal server. Bare metal servers come with high-performance benefits for workloads such as data, GPU, and AI. Additionally, all the hardware resources are dedicated to your workloads, so you don't have to worry about "noisy neighbors".

For more information about available bare metal flavors and how bare metal is different from virtual machines, see [Physical machines (bare metal)](https://cloud.ibm.com/docs/openshift?topic=openshift-planning_worker_nodesbm).
* What is the smallest size cluster that I can make?

Your cluster must have at least 2 worker nodes to run default Kubernetes and OpenShift Container Platform components. You can't have a cluster with 0 worker nodes, and you can't power off or suspend billing for your worker nodes. Additionally, the type of cluster and the number of worker pools that you have can impact the size of your cluster.



* Single zone clusters: [Create a cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-clusters) with 2 worker nodes in the default worker pool.
* Multizone clusters: You must [create a cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-clusters) with 1 worker node per zone in the worker pool.
A MongoDB deployment cannot support both [public and private endpoints simultaneously](https://cloud.ibm.com/docs/databases-for-mongodb?topic=databases-for-mongodb-service-endpoints&interface=uiprovisioning-service-endpoints). This cannot be changed after provisioning.





 Provisioning with Service Endpoints through the CLI 

Service Endpoints are enabled through an optional parameter when you provision through the CLI. Provisioning is handled by the Resource Controller. Pass the service-endpoints parameter one of the options public, private, or public-and-private.

ibmcloud resource service-instance-create <service-name> --service-endpoints <endpoint-type>

Cloud Databases deployments except Databases for MongoDB allow for both public and private networking to be enabled at the same time.





 Provisioning with Service Endpoints through the API 

Service Endpoints are enabled through an optional parameter when you provision through the API. Provisioning is handled by the Resource Controller. Pass the service-endpoints parameter one of the options public, private, or public-and-private.

ibmcloud resource service-instance-create <service-name> --service-endpoints <endpoint-type>

Cloud Databases deployments except Databases for MongoDB allow for both public and private networking to be enabled at the same time.





 Changing Service Endpoints 

After you deploy, it is possible to change your public and private service endpoints configuration, except for Databases for MongoDB.





 Changing Service Endpoints through the UI 

In the Overview tab of your deployment's dashboard, go to the Endpoints section. Toggle which types of connections are available to your deployment.

Changing the type of endpoints available on your deployment does not cause any downtime from a database perspective. However, if you disable an endpoint that is being used by you or your applications, those connections are dropped.





 Changing Service Endpoints through the CLI
If the SDU model finds and processes a structure, such as a table, in the document, it stores a representation of the structure in a field named enriched_{field}, where {field} is the field where the structure was stored.

The following excerpt shows the JSON representation of a table from the enriched_html field of a document that was processed by the pretrained SDU model.

Zoom

![Shows a JSON snippet that contains an enriched_html field with a table object that contains sections such as section_title, row_headers, table_headers, location, and so on.](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/sdu-table-output.png)

Figure 1. JSON table representation

If you want to extract text from the processed structure, you can use the location field to find the index values that identify where the text string starts and ends.

For more information about the structure of indexed tables, see [Understanding tables](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-understanding_tables).





 Troubleshooting issues 

Follow these workarounds if you experience problems when working with the Smart Document Understanding tool.



 Insufficient resources to process document 

Error
: When you apply a pretrained model to your collection, document processing does not complete successfully and an Insufficient resources to process document message is displayed.

Cause
: The error is displayed because out of memory errors occur during the parse, structure identification, or assembly phases of the process that builds the machine learning model. Resources are insufficient when one or more of the documents in your collection are too large or have too many complex tables for the tool to handle.

Solution
: Review your collection for large documents or documents with many tables and break them up into more smaller documents before you apply the pretrained model to the collection. Exact limits differ based on the complexity of your documents. Generally, split documents that are over 400 pages long and avoid including more than 20 complex tables in a single document.
ibmcloud cbr zone-update <ZONE-ID> --addresses=1.2.3.4 --name=<NAME>

Updating requires the ZONE-ID, not the zone name. Use the following command to list your zones and retrieve the relevant ZONE-ID:

ibmcloud cbr zones

The zone-update command is an overwrite. Include all of the fields that are required as if you are creating the rule from scratch. If you omit any required fields, the rule overwrites those missing fields as empty, and the rule might fail because some of those fields are required, regardless of whether they are changing the rule.

Delete a zone by using a command like:

ibmcloud cbr zone-delete <ZONE-ID>







 Creating rules 

Rules restrict access to specific cloud resources based on resource attributes and contexts. A created rule can accept up to 2,000 IP/CIDR values for private endpoints and up to 2,000 IP/CIDR values for public endpoints. This limit is specfic to Cloud Databases. Other IBM Cloud® service limits may vary.

Cloud Databases does not support IPv6 addresses. If an IPv6 address is included, it will be ignored.

Full Closure of Access to Non-Allowlisted Endpoints: To provide a more robust security framework, we have implemented a significant change in access control for public and private endpoints. Going forward, access to both public and private endpoints that are not explicitly allowlisted will be fully closed. This restriction ensures only authorized access to your endpoints, minimizing the risk of unauthorized access.



 Creating rules in the UI 



1. Go to Manage > Context-based restrictions in the IBM Cloud® console.
2. Select Rules.
3. Click Create.
4. Select Specific APIs and then select Data plane. Any other selection results in an error.

Cloud Databases does not currently support Control plane as an option.
5. Click Next.
6. Scope the rule to All resources or Specific resources.
Introduction to SSL technology 

Secure Sockets Layer (SSL) is a technology that encrypts traffic between the client application and the server application. This encryption is accomplished by using a public key/private key system that uses an SSL certificate. The SSL certificate contains the server’s public key, dates for which the certificate is valid, a host name for which the certificate is valid, and a signature from the certificate authority that issued it.



 SSL terminology 

SSL certificates have unique terminology. You might encounter the following terms while working with SSL certificates.



 Bit size 

Encryption keys are measured by their size in bits. For example, 512 bit, 1024 bit, 2048 bit. Generally, a longer key is going to be safer but slower to use. Currently, the minimum size for the keys that are used in SSL certificates is 1024 bit, though the Extended Validation certificates require 2048 bit.





 Certificate chain 

SSL certificates are not generally used alone. In most implementations, you deal with a certificate chain. For example:

Root > intermediate1 > server cert.

> Intermediate2 > server2 cert

In this example, your server certificate is signed by the intermediate certificate, which is in turn signed by the root certificate. Chaining in this fashion can make SSL more secure because it means that the root certificate is not used (and exposed to risk) so often. If intermediate1 was compromised, then server cert might be in danger, but server2 cert would be fine because they are part of different chains.





 Certificate signing request 

The certificate signing request (CSR) is a document that you generate on the server that contains information that the certificate authority uses to create your actual certificate.





 Common name 

The common name (CN) is the host name for which the certificate is valid (for example, www.domain.com).

www.domain.com, smtp.domain.com, and mail.domain.com are three different host names and the same SSL certificate is not valid for all three of them (unless you are using a wildcard certificate but currently IBM Cloud® does not offer those).





 Private/Public key 

SSL uses a technique called public key cryptography.
IBM Cloud Databases for PostgreSQL IBM Cloud Metrics Routing uses IBM Cloud Databases for PostgreSQL for storing metadata. 
 Key Protect IBM® Key Protect for IBM Cloud® is used to store encrypted keys that are required to operate the service. This does not include Bring Your Own Key (BYOK). 







 Workload isolation 

Each regional deployment serves multiple tenants that are identified by the IBM Cloud account ID.



* There is 1 deployment per region that is responsible for running user workloads in the region.
* In a region, the deployment is highly available.
* The data that is collected is associated with the IBM Cloud account ID and not visible to the other users by virtue of this association.
* Data for all tenants is co-located in the same data stores and segmented by the tenant-specific IBM Cloud account ID to enforce access control policies.
* You can use Cloud Identity and Access Management (IAM) to control which users see, create, use, and manage resources.
Managing Users, Roles, and Privileges 

IBM Cloud® Databases for EnterpriseDB uses a system of roles to manage database permissions. Roles are used to give a single user or a group of users a set of privileges. You can determine roles, groups, and privileges for all roles across your deployment by using the psql command du.

Zoom

![Table Results from \du command](https://cloud.ibm.com/docs-content/v1/content/e2b18e1112dc55facb736744a935365dce13849a/databases-for-enterprisedb/images/user_management_du.png)

Figure 1. Table Results from \du command

When you provision a new deployment in IBM Cloud, you are automatically given an admin user to access and manage Databases for EnterpriseDB.



 The admin user 

When you provision a new deployment in IBM Cloud, you are automatically given an admin user to access and manage Databases for EnterpriseDB. Once you [set the admin password](https://cloud.ibm.com/docs/databases-for-enterprisedb?topic=databases-for-enterprisedb-admin-password), you can use it to connect to your Databases for EnterpriseDB deployment.

When admin creates a resource in a database, like a table, admin owns that object. Resources that are created by admin are not accessible by other users, unless you expressly grant permissions to them.

The biggest difference between the admin user and any other users you add to your deployment is the [pg_monitor](https://www.postgresql.org/docs/current/default-roles.html) and [pg_signal_backend](https://www.postgresql.org/docs/current/default-roles.html) roles. The pg_monitor role provides a set of permissions that makes the admin user appropriate for monitoring the database server. The pg_signal_backend role provides the admin user the ability to send signals to cancel queries and connections that are initiated by other users. It does not provide the ability to send signals to processes owned by superusers.
Connecting an external application 

Your applications and drivers use connection strings to make a connection to IBM Cloud® Databases for MongoDB. Each deployment has connection strings specifically for drivers and applications. Connection strings are displayed in the Endpoints panel of your deployment's Overview, and can also be retrieved from the [Cloud Databases CLI plug-in](https://cloud.ibm.com/docs/databases-cli-plugin?topic=databases-cli-plugin-cdb-referencedeployment-connections) and the [Cloud Databases API](https://cloud.ibm.com/apidocs/cloud-databases-apidiscover-connection-information-for-a-deployment-f-e81026).

The connection strings can be used by any of the users you create in your deployment. While you can use the admin user for all of your connections and applications, it might be better to create users specifically for your applications to connect with. For more information, see [Creating Users and Getting Connection Strings](https://cloud.ibm.com/docs/databases-for-mongodb?topic=databases-for-mongodb-connection-strings).

When connecting an external application, use only drivers that are supported by [MongoDB](https://www.mongodb.com/docs/drivers/) or [MongoDB's Featured Community-Supported Libraries](https://www.mongodb.com/docs/drivers/featured-community-supported-libraries). Cloud Databases does not support any drivers that are not supported by MongoDB.



 Using Connection Information 

All the information a driver needs to make a connection to your deployment is in the "MongoDB" section of your connection strings. The table contains a breakdown for reference.



Table 1. mongodb/URI connection information

 Field Name Index Description 

 Type Type of connection - for MongoDB, it is "URI" 
 Scheme Scheme for a URI - for MongoDB, it is "mongodb" 
 Path Path for a URI - for MongoDB, it is the database name.
[Key Protect tool integration form with required fields and example values](https://cloud.ibm.com/docs-content/v1/content/5eb88a2575e30e316dbb83829c7f4c8b1ddbc67c/devsecops/images/kp-int.png)

Figure 3. IBM Key Protect tool integration

If you stored the role id and secret id in Key Protect in advance, then you can select the Key Protect instance that contains those secrets in the tool card as shown in Figure 2. After that is done, then you can click the key icons on the role id and secret id fields in the HashiCorp Vault tool card, and use the picker to apply the secrets to those fields.

Similarly, any other secrets that are used in the toolchain have a key icon that is attached to the text field. You can use the same picker control to apply the HashiCorp Vault secrets to all the remaining instances.
Booleans 

A true or false value.



 Example of a boolean in JSON format 

{
"value": true
}







 Arrays 

A list of values enclosed in brackets. The values that are enclosed can be any valid JSON.



 Example of an array in JSON format 

[
"one",
2,
"three",
],
true,
{
"foo":
"bar"
}
]





 Example of an array in JSON format (linear) 

[ "one", 2, "three", ], true, { "foo": "bar" } ]







 Objects 

A set of key-value pairs, such as an associative array, or a hash. The key must be a string, but the value can be any of the supported JSON values.



 Example of a JSON object 

{
"servings" : 4,
"subtitle" : "Easy to make in advance, and then cook when ready",
"cooktime" : 60,
"title" : "Chicken Coriander"
}
"reason": "The _sum function requires that map values be numbers, arrays of numbers, or objects. Objects can't be mixed with other data structures. Objects can be arbitrarily nested, if the values for all fields are themselves numbers, arrays of numbers, or objects.",
"caused_by":
{
"a": 1
},
{
"a": 2
},
{
"a": 3
},
{
"a": 4
}
]
}
}
]
}
Show more





 Map and reduce function restrictions 

Map and reduce function restrictions are described here.



 Referential transparency 

The map function must be referentially transparent. Referential transparency means that an expression can be replaced with the same value without changing the result, in this case, a document, and a key-value pair. Because of referential transparency, IBM Cloudant views can be updated incrementally and reindex only the delta since the last update.





 Commutative and associative properties 

In addition to referential transparency, the reduce function must also have commutative and associative properties for the input. These properties make it possible for the MapReduce function to reduce its own output and produce the same response, for example:

f(Key, Values) == f(Key, [ f(Key, Values) ] )

As a result, IBM Cloudant can store intermediate results to the inner nodes of the B-tree indexes. These restrictions also make it possible for indexes to spread across machines and reduce at query time.





 Document partitioning 

Due to sharding, IBM Cloudant offers no guarantees that the output of any two specific map functions passes to the same instance of a reduce call. You must not rely on any ordering. The reduce function that you use must consider all the values that are passed to it and return the correct answer irrespective of ordering. IBM Cloudant is also guaranteed to call your reduce function with rereduce=true at query time even if it didn't need to do so when it built the index.