vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Each collection can contain documents that you upload or that you crawl from a single data source, such as a website, Microsoft SharePoint, and more.

Things to consider when you adapt your application to use projects:



* Although the concept of an environment does not exist in v2, data is still organized into collections. In v2, collections are grouped into projects. In most cases, you want to migrate a single v1 collection to a single v2 collection.

If you want to keep relevancy training information that is applied to a v1 collection, add the collection documents to a single collection in your v2 project.
* Decide how many collections you want to add to each v2 project. All project types, except Content Mining projects, can contain up to 5 collections. Choose the right type of project for your data.

To optimize search results, different enrichments and configuration options are applied automatically to collections that are added to different project types. For more information, see the following topics:



* [Project descriptions](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-projectsproject-descriptions)
* [Default project settings](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-project-defaults)
* [Default query settings](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-query-defaults)



* The Discovery v2 API changed to account for projects and collections, among other enhancements. Some API calls changed to support actions at the project level instead of the collection level, such as submitting a query and running relevancy training. Many other API methods changed and some are not available in v2. For a detailed comparison of the v1 and v2 API methods, see [API version comparison](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-migrate-to-v2-api).







 Picking a service plan 

Choose among the Plus, Enterprise, and Premium managed plans or opt for an on-premises installation by purchasing the Discovery Cartridge for IBM Cloud Pak for Data.
Define your SAP system landscape 

Your business and functional requirements determine the SAP solutions powered by the SAP HANA Database Server or SAP NetWeaver Application Server, and therefore determine how your applications are run the infrastructure available.

Your requirements have an influence on how you size your server. You have a wide selection of SAP NetWeaver-based applications (which may use SAP HANA) to choose from, including SAP S/4HANA, SAP ERP Central Component (ECC), SAP BW/4HANA, SAP BW and many more solutions for differnt business operations.

For complete list of solutions, see [SAP NetWeaver installation guidance](https://help.sap.com/viewer/product/SAP_NETWEAVER/ALL/en-US). For information about supported operating systems and database platforms, see [SAP Product Availability Matrix (PAM)](https://support.sap.com/en/release-upgrade-maintenance.htmlsection_1969201630). Search for Product Availability Matrix. An [SAP S-User ID](https://cloud.ibm.com/docs/sap?topic=sap-necessary-credentials) is required.



 Determining your SAP applications 

An SAP landscape is a group of two or more SAP Systems that usually include development, quality and test, and production.

One SAP System consists of one or more SAP instances, which are a group of processes that are started and stopped at the same time. These SAP instances are grouped to form a specified SAP system for a defined use for a region or business unit. Then, the instances are grouped in a landscape as development, test, or production SAP Systems with one or multiple tracks (such as "project" and "business"). This landscape design is up to each business, dependent on business requirements.

Landscapes have several possible configurations, such as server (CPU, RAM) size and storage size, for all SAP solutions in the market. These solutions include SAP NetWeaver-based products. SAP NetWeaver-based products range from older solutions, such as SAP ECC and SAP BW (that use "AnyDB" vendors that are approved by SAP), to the new range of SAP solutions, such as SAP S/4HANA and SAP BW/4HANA (that use SAP HANA database).
* [Replications](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-configuring-ibm-cloudant-for-cross-region-disaster-recoveryreplications)
* [Indexes](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-configuring-ibm-cloudant-for-cross-region-disaster-recoveryindexes)



If you implement rerouting for requests or failover based on a health test, you might want to incorporate corresponding checks to avoid premature rerouting back to a service instance that is still recovering.



 Replications 



* Are any replications in an error state?
* Do any replications need restarting?
* How many pending changes are still waiting for replication into the database?



For more information, see how to [retrieve replication scheduler documents](https://cloud.ibm.com/apidocs/cloudantgetschedulerdocs) and monitor replication status.

If a database is being changed continuously, the replication status is unlikely to be zero. You must decide what status threshold is acceptable, or what represents an error state.





 Indexes 



* Are the indexes sufficiently up to date? Verify that indexes are updated by using the [active tasks](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-active-tasksactive-tasks) endpoint.
* Test the level of "index readiness" by sending a query to the index, and deciding whether it returns within an acceptable time.
Organizations take advantage of available resources such as the Common Weakness Enumeration (CWE) or Common Vulnerabilities and Exposures (CVE) databases in remediating flaws discovered in organizational information systems. By incorporating flaw remediation into ongoing configuration management processes, required/anticipated remediation actions can be tracked and verified. Flaw remediation actions that can be tracked and verified include, for example, determining whether organizations follow US-CERT guidance and Information Assurance Vulnerability Alerts. Organization-defined time periods for updating security-relevant software and firmware may vary based on a variety of factors including, for example, the security category of the information system or the criticality of the update (i.e., severity of the vulnerability related to the discovered flaw). Some types of flaw remediation may require more testing than other types. Organizations determine the degree and type of testing needed for the specific type of flaw remediation activity under consideration and also the types of changes that are to be configuration-managed. In some situations, organizations may determine that the testing of software and/or firmware updates is not necessary or practical, for example, when implementing simple anti-virus signature updates. Organizations may also consider in testing decisions, whether security-relevant software or firmware updates are obtained from authorized sources with appropriate digital signatures.
About VLANs 

VLANs are central in directing traffic to your resources. You might never need to interact directly with any VLANs because they are managed automatically; they are assigned as needed and removed when not.

A VLAN is a network concept. VLANs allow you to create broadcast domains at the [OSI Model](https://en.wikipedia.org/wiki/OSI_model) layer 2 level, the data link layer. VLANs provide one method of packet identification, and they allow multiple workloads to coexist on the same physical equipment. For more information about VLANs, see [this article](https://en.wikipedia.org/wiki/Virtual_LAN).



 Types of VLANs 

IBM Cloud® refers to the two distinct types of VLANs as Automatic and Premium. They each play a different role, and depending on your needs, understanding their differences is important.



 Automatic VLANs 

Automatic VLANs are managed by IBM Cloud automatically. They're assigned and removed as needed to fulfill the needs of other products that you order. You typically have one automatic VLAN per router. Automatic VLANs are associated to servers ordered without a specific VLAN selected. It's not possible to order or cancel automatic VLANs because they exist on your account only when our systems determine that they are required. They're removed when our systems determine that they are no longer needed.





 Premium VLANs 

Premium VLANs are acquired by ordering a VLAN. See [Ordering VLANs](https://cloud.ibm.com/docs/vlans?topic=vlans-ordering-premium-vlans) for instructions. You can order as many premium VLANs as you like, subject to capacity limitations. If capacity is unavailable, seek VLANs in another pod or data center.

An important distinction of Premium VLANs is that they're not selected automatically to fulfill a server order. Premium VLANs must be explicitly selected while you are ordering servers to have servers reside on them.
For more information, see [Region and data center locations for resource deployment](https://cloud.ibm.com/docs/overview?topic=overview-locations). list(string) null 
 ssh_key_name Your IBM Cloud SSH key name such as "smc-ssh-key" that is created in a specific region in IBM Cloud. Mark it as sensitive to hide the SSH key in the IBM Cloud console. string n/a 
 api_key Your API key value. Mark it as sensitive to hide the API key in the IBM Cloud console. string n/a 
 sym_license_confirmation Confirm your use of IBM Symphony Multi Cluster licenses. When you select 'true', you agreed to one of two conditions. 1. You are using the software in production and confirm you have sufficient licenses to cover your use under the International Program License Agreement (IPLA). 2. You are evaluating the software and agree to abide by the International License Agreement for Evaluation of Programs (ILAE). NOTE: Failure to comply with licenses for production use of software is a violation of IBM International Program License Agreement. For more information, see[IBM International Program License Agreement](https://www.ibm.com/software/passportadvantage/programlicense.html). string n/a 
 smc_zone IBM Cloud zone name within the selected region where the Symphony Multi Cluster resources are deployed. SMC supports combination of single smc_zone as ["us-east-1"] or double smc_zone as ["au-syd-3","eu-de-1"] or triple smc_zone as ["us-east-1","ca-tor-3","jp-tok-3"]. list(string) n/a 
 remote_allowed_ips Comma-separated list of IP addresses that can access the Symphony Multi Cluster instance through an SSH interface. For security purposes, provide the public IP addresses assigned to the devices that are authorized to establish SSH connections (for example, "169.45.117.34").
Standardizing application infrastructure where possible, and then hosting many applications on the same infrastructure, is essential to reduce operations costs. This set up allows the infrastructure to be better utilized and shares the operations cost across many applications. This also makes it practical to use infrastructure as code to deploy many copies of the infrastructure, amortizing the development and maintenance cost of that infrastructure as code across potentially thousands of applications.

As shown in the volumetric analysis, from 1-100 applications should be hosted per account on this shared infrastructure with an average of perhaps 10-20 applications per account. With hundreds of accounts (some that are designated for development and test and some that are designated for production workloads), this model can support many thousands of applications and their development.

A good starting point for developing the standardized application hosting infrastructure is IBM Cloud's VPC landing zone deployable architecture. This IBM Cloud for Financial Services compliant solution sets up everything that is needed to host VM or Container based applications in a fully secure and compliant fashion. Also, the deployable architecture is supported and maintained by IBM. As new compliance requirements arrive or vulnerabilities are discovered, the solution is updated to maintain security and compliance.





 Separate management from workload 

In the IBM Cloud Framework for Financial Services [reference architecture](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-vpc-architecture-about), management workloads are separated from application workloads to support the different access models, data privacy requirements, network access requirements, and so on. Because these principles apply more broadly than just in the Financial Services industry, they have been applied in this general recommendation.



 Related controls in IBM Cloud Framework for Financial Services 

The following [IBM Cloud Framework for Financial Services controls](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-aboutframework-control-requirements) are most related to this guidance. However, in addition to following the guidance here, do your own due diligence to ensure you meet the requirements.



Table 3. IBM Cloud Framework for Financial Services controls

 Family Control
After you understand how many CAs, peers, and ordering nodes are required, you can examine the default resource allocations table for your [nodes](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-adv-deploymentibp-console-adv-deployment-allocate-resources) to get an approximate estimate of the CPUs (VPCs) required for your network. If you are purchasing IBM Blockchain Platform on IBM Cloud, you can estimate your cost through this method, but you also have the ability to scale dynamically if more sources are needed.
* How does pricing work on the IBM Blockchain Platform for IBM Cloud?

IBM Blockchain Platform for IBM Cloud is priced based on the VPCs that you allocate to your blockchain nodes on the IBM Kubernetes Service. For more information, see [Pricing for IBM Blockchain Platform for IBM Cloud](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-saas-pricingibp-saas-pricing).
* What are the limitations of the free IBM Blockchain Platform using the IBM Cloud Kubernetes Service free cluster?



* Preview the IBM Blockchain Platform at no charge for 30 days when you link your IBM Blockchain Platform service instance to an IBM Cloud Kubernetes free cluster.
* Performance will be limited by throughput, storage, and functionality. Read more about the [limitations of free clusters](https://cloud.ibm.com/docs/containers?topic=containers-cs_ovcluster_types).
* IBM Cloud will delete your Kubernetes cluster after 30 days.
* Only one blockchain console can be connected to a free cluster at a time.
* You cannot migrate any nodes or data from a free cluster to a paid cluster.



The following capabilities are only available on a paid cluster:



* Customizing resource allocation for a node during or after deployment.
* Using a Hardware Security Module (HSM) to secure the private key for a node.
* Configuring a Certificate Authority (CA) for high availability by using a PostgreSQL database and replica sets.
AC-3 - Access Enforcement 



  Control requirements 

AC-3 - 0
:   The information system enforces approved authorizations for logical access to information and system resources in accordance with applicable access control policies.





  Additional IBM Cloud for Financial Services specifications 



*  Customer consent must be obtained prior to releasing any data outside its intended use.
*  Authorized individuals include Federal Regulators in the event of insolvency.







  Implementation guidance 

See the resources that follow to learn more about how to implement this control.



*  [Access management in IBM Cloud](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-account-access-management)







  IBM Cloud for Financial Services profile 

The rules related to this control that follow are part of the IBM Cloud for Financial Services v1.2.0 profile in [IBM Cloud® Security and Compliance Center](https://cloud.ibm.com/docs/security-compliance?topic=security-compliance-getting-started).



*  Check whether permissions for service ID creation are limited and configured in IAM settings for the account owner
*  Check whether permissions for API key creation are limited and configured in IAM settings for the account owner
*  Check whether permissions for API key creation are limited and configured in IAM settings for the account owner
*  Check whether user list visibility restrictions are configured in IAM settings for the account owner
*  Check whether IAM roles are used to create IAM policies for IBM resources
*  Check whether IAM users are attached to at least one access group
*  Check whether Cloud Object Storage public access is disabled in IAM settings (not applicable to ACLs managed using S3 APIs)







  NIST supplemental guidance 

Access control policies (e.g., identity-based policies, role-based policies, control matrices, cryptography) control access between active entities or subjects (i.e., users or processes acting on behalf of users) and passive entities or objects (e.g., devices, files, records, domains) in information systems. In addition to enforcing authorized access at the information system level and recognizing that information systems can host many applications and services in support of organizational missions and business operations, access enforcement mechanisms can also be employed at the application and service level to provide increased information security.
See the [3 September 2019 release notes](https://cloud.ibm.com/docs/discovery?topic=discovery-release-notesdiscovery-3september2019) for more information.





 30 November 2017 

New beta Knowledge Graph : Released the beta version of IBM Watson™ Discovery Knowledge Graph, which provides new end-points for querying entities and relations across documents. This includes context-based searches and relevance ranking. : Update: As of 30 September 2019, the IBM Watson™ Discovery Knowledge Graph Beta APIs are no longer accessible. See the [3 September 2019 release notes](https://cloud.ibm.com/docs/discovery?topic=discovery-release-notesdiscovery-3september2019) for more information.

New Korean and Spanish language support for Discovery News : Released [IBM Watson™ Discovery News](https://cloud.ibm.com/docs/discovery?topic=discovery-watson-discovery-news) in two additional languages: Korean (collection_id: news-ko) and Spanish (collection_id: news-es). IBM Watson™ Discovery News Korean and Spanish are available for use via the API-only; for information about querying a collection via the API, see [API Reference](https://cloud.ibm.com/apidocs/discoveryquery-your-collection). : Update: [Resolved](https://cloud.ibm.com/docs/discovery?topic=discovery-release-notes26jan18). IBM Watson™ Discovery News English now has the collection_id of news-en. Formerly, the collection_id was news. If you use the former collection_id, it continues to work; however, you might want to switch to the new collection_id for new projects.

Updated confidence score calculations : Query results return a score value, which indicates the relative relevancy between query results. Starting 30 November 2017, the way that score is calculated changed. It is recommended that you only use the score value to rank documents in a single search, not across searches or sessions. If you trained a collection, a score value is returned in the results of a natural language query.
For more information about forest and domain models, see [Designing the logical structure](https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/designing-the-logical-structure).



 Single forest, single domain model 

The single forest, single domain model is the easiest to administer and maintain. The design consists of a forest that contains a single domain. As it is the only domain, it is the forest root domain, and it contains all of the user and group accounts in the forest and the resource objects. For some enterprises, this model is all that is required. However, more complex enterprises can't use this model.





 Single forest, multiple domains model 

The single forest, multiple domains model is used by larger enterprises where the forest includes many users that are either distributed across different geographic locations or different business units. If each location has its own domain, then replication traffic between locations is reduced. Ease of management is often the use case for domains that are used for different business units. This domain model consists of a forest root domain and one or more regional domains. Often, the forest root domain is restricted to just the Enterprise Admins and Schema Admins groups. These groups are used to manage forest-level operations such as the addition and removal of domains and the implementation of changes to the schema.

Creating a multiple domain model design involves identifying what domain is the forest root domain and determining the number of extra domains that are required to meet your replication requirements. These domains can be parent or child domains, depending on how the domain structure needs to reflect the organizational structure. Typically, it's better to have as few domains as possible for easy management. To reduce complexity, the enterprise needs to balance complexity with the separation of resources into different domains and subdomains, which has security benefits.





 Multiple forest models 

If the enterprise includes groups that require data isolation or service isolation from other groups in the enterprise, then the multiple forest model can be used to create a separate forest for these groups. Domains do not provide data isolation or service isolation. The forest service administrators have full access to all resources in the forest.
* UI
* CLI







 Creating an instance group for auto scaling 

With Auto Scale for VPC, you can create an instance group to scale according to your requirements. Based on the target utilization metrics that you define, the instance group can dynamically add or remove instances to achieve your specified instance availability.

If you are going to use a custom image in a private catalog, you must first create a service-to-service policy to globalcatalog-collection.instance.retrieve before you can create the instance group. For more information, see [Using a custom image in a private catalog with an instance group](https://cloud.ibm.com/docs/vpc?topic=vpc-private-catalog-image-instance-group&interface=ui).



 Auto scale for VPC 

With Auto Scale for VPC, you can improve performance and costs by dynamically creating virtual server instances to meet the demands of your environment. You set scaling policies that define the average utilization that you want to achieve for metrics like CPU, memory, and network usage. The policies that you define determine when virtual server instances are added or removed from your instance group.

As an example, imagine that the fictitious company, Acme Web Retailer, sets up an instance group for auto scaling. They define that they always want to maintain a minimum of three instances and a maximum of seven instances. They create a dynamic scaling policy for CPU usage with their specified average utilization for instances at 70%. They set an aggregation window of 10 minutes, so the instance group manager monitors each instance for 10 minutes before it calculates the average utilization. If adjustments are needed to meet the target utilization across instances, the instance group manager provisions or reclaims more instances as needed.

Auto scale uses the following computation to determine how many instances are running at any time:

Σ(Current average utilization of each instance)/target utilization = membership count

If Acme Web Retailer has four virtual server instances that are running when the aggregation window elapses, the formula looks like this: VSI1 + VSI2 + VSI3 + VSI4 / 70% = membership count. CPU utilization for the four running instances is 80%, 70%, 65%, and 85%, so the following computation takes place: