vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Cluster and management component <instance>-<cluster>-<datacenter>-<component> mcv1-mgmt-dal10-private <br>DAL10-DVS 









 Instance sizing considerations 

The witness and management clusters can be deployed either by using NFS storage, with a minimum of two nodes required, or by using vSAN storage, with a minimum of four nodes required. Conversion of the witness or management clusters from NFS to vSAN is not supported or offered.

Your choice of effective RAID level determines the minimum number of hosts that must be deployed to Sites A and B.



* FTT levels are the default and are not monitored or managed by automation for Day 2 operations
* Minimum host count is determined by: Site A (workload) + Site B (workload) + Site A (management) + Site B (management) + witness



The following table provides the Redundant Array of Independent Disks (RAID) and Failures to Tolerate (FTT) levels.



Table 2. RAID and FTM levels

 RAID level FTT Erasure Coding Minimum host count 

 RAID 1 PFTT=1 SFTT=1 No 3+3+2+2+2 
 RAID 5 PFTT=1 SFTT=1 Yes 4+4+2+2+2 
 RAID 6 PFTT=1 SFTT=2 Yes 6+6+2+2+2 







 Day 2 operations 

Multizone instance automation provides provisioning, installation, and configuration of hardware and VMware components only. Automation is not available for Day 2 operations other than scale-up and scale-down.



 Witness site 

The witness site can be scaled up and down with a maximum of 51 hosts.





 Management and resource clusters 

Scale-up and scale-down are done in tandem across the two sites with a single user interface interaction.

The management cluster can be scaled up and down with a maximum of 51 hosts. The stretched cluster can be scaled up and down with a maximum of 32 hosts.

For example, you can add capacity by adding pairs of hosts, one in Site A and one in Site B to maintain uniformity in the configuration across the two sites. The status reflects that capacity is being added or removed to both sites simultaneously.
PE-5 - Access Control for Output Devices 



  Control requirements 

PE-5 - 0
:   The organization controls physical access to information system output devices to prevent unauthorized individuals from obtaining the output.





  NIST supplemental guidance 

Controlling physical access to output devices includes, for example, placing output devices in locked rooms or other secured areas and allowing access to authorized individuals only, and placing output devices in locations that can be monitored by organizational personnel. Monitors, printers, copiers, scanners, facsimile machines, and audio devices are examples of information system output devices.
Cloud type values are public, dedicated, or local. 
 Location ibm_location Location of the monitored resource. This value can be a region, data center, or global. 
 Resource ibm_resource Resource being measured by the service. Typically, this value is an identifying name or GUID. 
 Resource type ibm_resource_type Type of resource measured by the service. 
 Scope ibm_scope Scope of the account, organization, or space GUID that is associated with this metric. 
 Service name ibm_service_name Name of the service generating this metric. 







 Additional attributes 

The following attributes are available for segmenting one or more attributes as described in the preceding reference. See the individual metrics for segmentation options.



Table 3: Additional attributes

 Attribute Attribute Name Attribute Description 

 IBM IS Generation 2 ibm_is_generation IBM IS Generation ( 2 for Gen. 2) 
 Flow log collector VPS attachment ibm_flow_log_collector_instance The virtual server instance ID where the flow log collector is attached. 









 Enabling metrics monitoring 

To receive monitoring metrics, you must set up your IBM Cloud Monitoring instance.

To receive monitoring metrics, use the following steps:



1. Navigate to the [metrics monitoring portal](https://cloud.ibm.com/observe/monitoring) and click Create a monitoring instance.
2. Select a region for your monitoring instance.

The region needs to match the location of your existing VPN gateway.
3. Choose your pricing plan.

Pricing plan details are explained in the selection window. Select the plan that best meets your requirements.
4. Provide a unique service name for your instance. The name can be any name that you want. The name has no impact on functionality.

Do not give multiple monitoring instances the same name.
5. Optionally, select a resource group. A resource group organizes account resources in customizable groupings. Any account resource that is managed by using IBM Cloud Identity and Access Management (IAM) access control belongs to a resource group within your account.

If you do not have any pre-configured resource groups, or have no reason to share this resource selectively, use the default selection.
Using the COVID-19 kit 

The COVID-19 kit is a special collection available in US instances of IBM Watson™ Discovery. This pre-built collection includes more than 10 data sources you can use to fuel a dynamic chatbot built with IBM Watson™ Assistant and Discovery to answer your customers' questions about COVID-19.

FAQ extraction is a beta feature that was used to create question and answer pairs for the kit. The FAQ extraction beta feature is now deprecated and will stop being supported in v1 Discovery service instances on 1 March 2022. As a consequence, the COVID-19 kit data collection will stop being supported also.

The COVID-19 kit extracts answers from trusted sources and automatically updates your chatbot as the content from those sources are updated. It uses FAQ extraction machine learning models developed by IBM Research labs to extract question/answer pairs. The kit is pre-configured with web crawl seeds from trusted sources such as the CDC, Harvard, and the United States Department of Labor. An expanded stopwords list and query expansions are also included in this collection to improve search results. It is designed to be augmented with your own data and customized to your needs.

For more information about this kit, and how you can create a chatbot and connect it to a data source with the IBM Watson™ Assistant search skill using IBM Watson™ Assistant and Discovery, see [COVID-19 — Are Your Virtual Assistant’s Answers Up-To-Date?](https://medium.com/ibm-watson/covid-19-are-your-virtual-assistants-answers-up-to-date-c9e1ba70eb65654b).

To learn how to create a search skill in Watson Assistant, see [Creating a search skill](https://cloud.ibm.com/docs/services/assistant?topic=assistant-skill-search-add). (This feature is available only to Watson Assistant Plus or Premium plan users.)

See the following to learn more about working with Discovery:



* To learn how to create a service instance, see [Getting started with the Discovery tooling](https://cloud.ibm.com/docs/discovery?topic=discovery-getting-started).
Caveonix RiskForesight licenses that are ordered separately from the Resources page are not per-host. Therefore, it is recommended that you do not order new Caveonix RiskForesight licenses and apply the key to your Caveonix RiskForesight service on Regulated Workloads or Security and Compliance Readiness Bundle instances. Ordering in this way will:



* Provide less coverage than when the license was initially deployed
* Result in you being charged twice for Caveonix RiskForesight licenses because your initial license cannot be canceled. Therefore, you would pay for both per-host and per-VM licensing.



You can still order Caveonix RiskForesight licenses for other types of instances, for example, for VMware vServer Center instances.







 Considerations when you delete Caveonix RiskForesight 

Review the following considerations before you delete the service:



* Deleting Caveonix RiskForesight automatically deletes the initial Caveonix RiskForesight license originally that is associated with the service. However, you need to manually delete any other unwanted licenses from the Caveonix RiskForesight licenses table on the Licenses page in the IBM Cloud® for VMware Solutions console.
* When you delete the service, the IBM Cloud for VMware Solutions automation deletes only the single all-in-one Caveonix virtual machine (VM) that was deployed and the dedicated private subnet that was ordered for it. Therefore,



* If you scaled out the Caveonix VM into multiple VMs, those additional VMs are not deleted.
* If you used the IP addresses of the dedicated private subnet on more VMs, those VMs must be assigned new IP addresses to continue to function.
* If you delete the vCenter Server instance A with Caveonix RiskForesight installed, and you used the IP addresses of the dedicated private subnet that was ordered for the service in the vCenter Server instance B, the dedicated private subnet is canceled upon deletion of the vCenter Server instance A.



* If you installed the Caveonix RiskForesight service before VMware Solutions v4.0, and you then delete that service, you must manually remove the DNS entries.
AU-5 - Response to Audit Processing Failures 



  Control requirements 

The information system:

AU-5 (a)
:   Alerts [Assignment: organization-defined personnel or roles] in the event of an audit processing failure; and

AU-5 (b)
:   Takes the following additional actions: [IBM Assignment: organization-defined actions to be taken (overwrite oldest record)].





  Additional IBM Cloud for Financial Services specifications 



*  The status of all security audit log sources must be monitored for integrity and functionality as designed, at least hourly, and following policy changes, patch updates, and changes.







  Implementation guidance 

See the resources that follow to learn more about how to implement this control.



*  [Audit logging of application provider events and SIEM](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-logging-audit-provider)
*  [Audit logging of IBM Cloud events](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-logging-audit)
*  [IBM Cloud account setup](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-shared-account-setup)







  NIST supplemental guidance 

Audit processing failures include, for example, software/hardware errors, failures in the audit capturing mechanisms, and audit storage capacity being reached or exceeded. Organizations may choose to define additional actions for different audit processing failures (e.g., by type, by location, by severity, or a combination of such factors). This control applies to each audit data storage repository (i.e., distinct information system component where audit records are stored), the total audit storage capacity of organizations (i.e., all audit data storage repositories combined), or both.
Because the ordering service is located in the United States, a hash of the data that is created in Germany still leaves the country.

Creating a separate channel can prevent transaction details from being shared with other organizations in the consortium. If the organizations in Germany used a shared channel, Org A and Org B would be able to see the number of transaction hashes that are committed to the channel ledger by the private data collection. This might provide those organizations visibility into the fact that Orgs C and D are transacting and the volume of transactions that are generated between them. Consider though that forming a new channel requires additional management expense to create and update it. Forming a new channel also makes it harder for Org C and Org D to share data with Org A and Org B.





 Option three: A channel with all components in one country 

Org C and Org D can also create a channel with all of the infrastructure within the same country. This requires that the peers joined to the channel, the applications, and the ordering service all reside within the same region. In this scenario, none of the data that is stored on the channel ledger will leave the region and be stored outside the country.

Zoom

![Creating a separate channel](https://cloud.ibm.com/docs-content/v1/content/c5288bed34c3820e3a5251d820ee91adcd2591a3/blockchain/images/data_res_separate_channel.svg)

Figure 5. Org C and Org D create a new channel using an ordering service located in Germany. All data is stored in the country on Ledger Y.

In Figure 5, Org C and Org D have created a new channel for data that must not leave Germany. This requires the creation of a new ordering service that is located in Germany to ensure that the orderer's copy of the channel ledger is stored inside the country. Since in this case the ordering service, the OrgC-peer, and the OrgD-peer are located in Germany, Org C and Org D can now keep the data public on the channel if they choose, or they could still decide to use private data collections to prevent the full transaction data from being stored on the ordering service.
Logging with Linux VPC instances 

You can use the IBM® Log Analysis service to monitor and manage logs from a Linux VPC server instance in a centralized logging system on the IBM Cloud. You can collect and monitor system and application logs.

By default, the logging agent for Linux VPC instances monitors log files in the /var/log directory. For example, the Ubuntu system log (/var/log/syslog) is monitored by default.

On the IBM Cloud, you can configure a Linux VPC server to forward logs to an IBM Log Analysis instance by completing the following steps:



1. Provision a VPC running Ubuntu Linux for example.
2. Provision an instance of the IBM Log Analysis service.
3. Configure the logging agent in the Ubuntu server.
4. Optionally, add additional directories to be monitored by the agent.



Zoom

![Component overview on the IBM Cloud](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/Log-Analysis-03-Ubuntu.svg)

Log Analysis component overview on the IBM Cloud for Ubuntu Linux

For more information, see [Logging with Linux VPC server instances](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-ubuntu).





 Logging with Syslog 

You can send logs to an IBM Log Analysis instance via Syslog. TCP and TCP+TLS are both supported.

To configure syslog, you must enable a port to send logs via syslog to your logging instance. If you are using (a) the classic syslog protocol, (b) a custom port in syslog-ng, or (c) a custom port in rsyslog, there is no authentication available and anyone with knowledge of the endpoint can submit logs to your instance. Depending on your environment, this may present a significant security risk.
There are several use cases for the snapshot feature. For example, an administrator plans to upgrade the middleware on his system but would like to be able to revert to its original state before proceeding with an upgrade. If the middleware fails, the administrator can restore the source disk to its previous state. To accomplish this, the administrator would perform the following steps:



1. Initiate the snapshot API with the source disks where the middleware information resides.
2. Upgrade the middleware.
3. If the upgrade fails, restore the source disks by using the snapshot that is created in the previous step.
4. If the upgrade succeeds, delete the snapshot that is created in the first step.



You can initiate multiple snapshot operations. However, these concurrent snapshot operations occur on a different set of disks.



 Metering and pricing of snapshot 

Each snapshot that you create is monitored every hour and charged depending on the disk space that is requested for the snapshot. The space that is utilized by a snapshot is charged at 30% of the base rate. For example, if you have M disks on a VM that add up to 600 GB of space, and those M disks are used as source disks for snapshots, the following charges are applicable:



* If you create one snapshot, you are charged for the disk space that is used by the base 600 GB of M disks plus 30% of 600 GB of disk space. That is, 600 GB (space of M disks) + 180 GB (30% of 600 GB) = 780 GB of disk space.
* If you create one more snapshot by using same disks, you will be charged for disk space that is used by M disks (600 GB) + (30% of 600 GB) + (30% of 600 GB) = 960 GB of disk space.
* The pricing that is charged for the snapshot is based on the storage tier that contains the source volumes: Tier 1 or Tier 3.







 Best practices 



* Before you take a snapshot, ensure that all of the data is flushed to the disk. If you take a snapshot on a running virtual machine (VM) and did not flush the file system, you might lose some content that is residing in memory.
ibmcloud cf cups my-dynatrace-collector -p '{"server":"DynatraceCollectorIPaddress","profile":"Monitoring","options" : {"dynatrace-parameterr-1": "value", "dynatrace-parameter-2": "value"}}'

See the [Agent Settings section of Agent Configuration](https://community.dynatrace.com/community/display/DOCDT65/Set+up+Agents) at the Dynatrace community website for more information about available options. For example, by using the exclude option, you can exclude classes from being monitored by Dynatrace. See [Dynatrace Agent Framework](https://github.com/cloudfoundry/ibm-websphere-liberty-buildpack/blob/master/docs/framework-dynatrace_one_agent.md) for more details about configuring the user-provided service.
3. After you push your app to IBM Cloud, bind the user-provided service that you created to the app. For example, use the following command:

ibmcloud cf bs myApp my-dynatrace-collector

Note: You must restage your app after binding the service.







 Optional configuration 

You may choose to acquire and host the Dynatrace agent .jar file yourself. In that case the following additional configuration steps are needed.



1. Acquire and host the Dynatrace agent .jar file so that the Liberty buildpack can download it.
2. Configure your Liberty app so that it can download the Dynatrace agent.





 Hosting the Dynatrace agent 

The Dynatrace agent must be hosted on a web server, and the Liberty buildpack must be able to download the agent .jar file from that server. The server must be configured with an index.yml file that specifies details about the agent .jar file. Complete the steps that follow to set up the Dynatrace agent:



1. Download the Dynatrace agent .jar file.
git clone https://github.com/IBM-Cloud/kube-samples.git
2. Change directories to the public policy directory for the region that your cluster is in. Example command for a cluster in US South:

cd <filepath>/IBM-Cloud/kube-samples/calico-policies/public-network-isolation/us-south
3. Review each policy for any changes you might need to make. For example, if you specified a custom subnet when you created your cluster that provides the private IP addresses for your pods, you must specify that CIDR instead of the 172.30.0.0/16 CIDR in the allow-ibm-ports-public.yaml policy. Also review these policies for any connections you might not want to allow. For example, if a certain port listed in the policy is described as only being needed for Istio, and you are not deploying Istio in your cluster, you can remove that port.
4. Apply the public or private policies that you want to use.

calicoctl apply -f allow-egress-pods-public.yaml
calicoctl apply -f allow-ibm-ports-public.yaml
calicoctl apply -f allow-public-service-endpoint.yaml
calicoctl apply -f deny-all-outbound-public.yaml
calicoctl apply -f allow-k8s-master-to-dashboard.yaml
5. Optional: To allow your worker nodes to access other IBM Cloud services over the public network, apply the allow-public-services.yaml and allow-public-services-pods.yaml policies. The policy allows access to the IP addresses for IBM Cloud Container Registry, and if the services are available in the region, IBM Log Analysis and IBM Cloud Monitoring. To access other IBM Cloud services, you must manually add the subnets for those services to this policy.

calicoctl apply -f allow-public-services.yaml
calicoctl apply -f allow-public-services-pods.yaml
6. Verify that the policies are applied.
Because the Dutch and Italian models are beta, their support for language model customization is also beta.

Japanese narrowband model now includes some multigram word units
: The Japanese narrowband model (ja-JP_NarrowbandModel) now includes some multigram word units for digits and decimal fractions. The service returns these multigram units regardless of whether you enable smart formatting. The smart formatting feature understands and returns the multigram units that the model generates. If you apply your own post-processing to transcription results, you need to handle these units appropriately. For more information, see [Japanese](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-formattingsmart-formatting-japanese) in the smart formatting documentation.

New speech activity detection and background audio suppression parameters for speech recognition
: The service now offers two new optional parameters for controlling the level of speech activity detection. The parameters can help ensure that only relevant audio is processed for speech recognition.



* The speech_detector_sensitivity parameter adjusts the sensitivity of speech activity detection. You can use the parameter to suppress word insertions from music, coughing, and other non-speech events.
* The background_audio_suppression parameter suppresses background audio based on its volume to prevent it from being transcribed or otherwise interfering with speech recognition. You can use the parameter to suppress side conversations or background noise.



You can use the parameters individually or together. They are available for all interfaces and for most language models. For more information about the parameters, their allowable values, and their effect on the quality and latency of speech recognition, see [Speech activity detection](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-detection).

Activity Tracker now supported for customization interfaces
: The service now supports the use of Activity Tracker events for all customization operations. IBM Cloud Activity Tracker records user-initiated activities that change the state of a service in IBM Cloud. You can use this service to investigate abnormal activity and critical actions and to comply with regulatory audit requirements. In addition, you can be alerted about actions as they happen.
"relevancy": 1.1170819184294765,
"total_matching_documents": 4017,
"estimated_matching_results": 1011
},
{
"key": "FL",
"matching_results": 511,
"relevancy": 0.828014956418841,
"total_matching_documents": 2199,
"estimated_matching_results": 553
}
]
}
],
"results": []
Show more





 Combining aggregation types 

There are different types of aggregations that you can use to analyze or group the query results. And you can combine more than one aggregation in a request to do more targeted analysis.

The following example shows a request that is composed of two term operators. The first term aggregation groups the input documents by US STATE values and selects 3 groups. The second term aggregation applies to each of those 3 groups and groups them further by the value of CITY. Only 2 of those CITY subgroups are returned per STATE group.

The relevancy parameter is being excluded to make the results easier to read.

{
"query":"brake",
"aggregation": "term(field:STATE,count:3).term(field:CITY,count:2)"
}

The response contains city information from each state.

{
"matching_results": 9064,
"retrieval_details": {
"document_retrieval_strategy": "untrained"
},
"aggregations": [
{
"type": "term",
"field": "STATE",
"count": 3,
"results":
{
"key": "CA",
"matching_results": 1210,
"aggregations":
{
"type": "term",
"field": "CITY",
"count": 2,
"results":
{
"key": "LOS ANGELES",