vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Entity subtypes are for stratifying an inventory of entity types into a hierarchy. For example, if you care to distinguish 40 things, but they logically group into 10 sets of 4 (VITALSIGN_BLOODPRESSURE, VITALSIGN_HEARTRATE, MEDICATION_DOSAGE, MEDICATION_FREQUENCY), you could structure them into types and subtypes, which yields more compact menus, but adds a level of depth and nuisance to the labeling process. For the purposes of information extraction, a combination of types and subtypes is interchangeable with lots and lots of types with no subtypes. By contrast, the word or phrase to which you apply a role is an example of one entity type, but it plays the role of another entity type, and one that is not a subtype of the other.





 How are roles treated? 

The Knowledge Studio machine learning model defines classifier labels for each mention of an entity that it finds in the source documents by concatenating the entity type and role values. When you provide role values, you create more precise labels. Each group of instances of one label type that are found in the training data forms a more uniform set of mentions. The challenge is that by choosing to be more precise, you must also accede to providing more training data to ensure that the model performs well. But, the more training data you provide, the better the model becomes. So, if you don't mind doing additional annotation, using roles gets you better results.

As an example, let's assume that the following sentences occur in a source document, and we want to capture the Acme (where Acme is a well-known truck manufacturer), sedan, and truck as similar entities because they all refer to a physical vehicle:



* a) The Acme crashed into the concrete barrier.
* b) The sedan crashed into the concrete barrier.
* c) The Acme A-160 truck crashed into the concrete barrier.



You can handle the type system design in two ways:



1. Define two entity types: MANUFACTURER and VEHICLE. For the MANUFACTURER entity, define a VEHICLE role that can be used in addition to the default MANUFACTURER role.

Using this type system, the sentences are annotated as follows:
What is a Power Systems Virtual Server? 

IBM® Power® Systems Virtual Server is a Power Systems offering. The Power Systems Virtual Servers are located in the IBM data centers, distinct from the IBM Cloud servers with separate networks and direct-attached storage. The environment is in its own pod and the internal networks are fenced but offer connectivity options to meet customer requirements. This infrastructure design enables Power Systems Virtual Server to maintain key enterprise software certification and support as the Power Systems Virtual Server architecture is identical to certified on-premises infrastructure. The virtual servers, also known as logical partitions (LPAR), run on IBM Power Systems hardware with the PowerVM hypervisor.

With the Power Systems Virtual Server, you can quickly create and deploy one or more virtual servers (that are running either the AIX, IBM i, or Linux operating systems). After you provision the Power Systems Virtual Server, you get access to infrastructure and physical computing resources without the need to manage or operate them. However, you must manage the operating system and the software applications and data. The following graphic represents a responsibility assignment (RACI) matrix for Power Systems Virtual Servers:

Zoom

![Power Systems Virtual Server responsibility assignment matrix](https://cloud.ibm.com/docs-content/v1/content/1d36273917990fbf48078805e201d2e85e2fcb27/power-iaas/images/RACI_matrix.png)

Figure 1. Power Systems Virtual Server responsibility assignment matrix



 Key features 

The following are some of the key features for the Power Systems Virtual Server.



 Straightforward billing 

Power Systems Virtual Server uses a monthly billing rate that includes the licenses for the AIX and IBM i operating systems. The monthly billing rate is pro-rated by the hour based on the resources that are deployed to the Power Systems Virtual Server instance for the month. When you create the Power Systems Virtual Server instance, you can see the total cost for your configuration based on the options that you specify. You can quickly identify what configuration options provide you with the best value for your business needs.
* [Action required now](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_notices_listsregistry_notices_lists_action)



[Container Registry private IP addresses changed on 5 July 2022](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_notices_iam_private_networkregistry_notices_iam_private_network)



* [What you need to know about this change](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_notices_iam_private_networkregistry_notices_iam_private_network_know)
* [How you benefit from this change](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_notices_iam_private_networkregistry_notices_iam_pivate_network_benefit)
* [Understanding if you are impacted by this change](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_notices_iam_private_networkregistry_notices_iam_pivate_network_impact)
* [What actions you need to take](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_notices_iam_private_networkregistry_notices_iam_pivate_network_action)



[IAM access policies are required from 5 July 2022](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_notices_iam_policyregistry_notices_iam_policy)



* [What are the changes?](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_notices_iam_policynotices_iam_policy_change)
* [Check whether these changes affect you](https://cloud.ibm.com/docs/Registry?topic=Registry-registry_notices_iam_policynotices_iam_policy_affect)
Creating collections 

A collection is a set of documents that you add to a project so that you can analyze, enrich, and extract useful information from it.

You can add data to your project in the following ways:



* Upload locally accessible files by using the product user interface. This method is the best way to get started and test your use case.
* Set up a scheduled crawl of documents that are stored on an external data source.

The product user interface offers several built-in data source connectors for you to choose from. The options differ depending on your deployment type. For more information, see [Supported data sources](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collectionscollections-include-data-sources-reuse).
* Connect to an external data source for which no built-in support is available:

IBM Cloud

: Use IBM App Connect to set up a scheduled crawl of documents that are stored on other external data sources.

IBM Cloud Pak for Data

: Build a connector to crawl documents that are stored on other external data sources.
* To automate the process of adding data to your project, use the Discovery APIs to create a collection and upload documents to it.



When you add documents to Discovery, the original documents are crawled and information from the documents is stored in an index so that it can be enriched and analyzed or retrieved later. Not all rich content from the original document is retained. For example, images from .ppt or .doc files are not stored. For more information, see [How your data source is processed](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-index-overview).



 Choosing what to add to a collection 

There a few things to consider as you decide how to break up your source content into collections.



* Getting content from different data sources

If you store similar content in more than one type of data source (a website and Salesforce, for example), you can create one project with two separate collections. Each collection adds documents from a single data source. When they are built together into a single project, a user can search across both sources at the same time.
If the partition location is not specified, it is inferred from the location of the table and the value(s) of the partitioning column(s). ADD PARTITION does not validate the specified or inferred location.

-- alter the table partitions by adding a partition
ALTER TABLE customers_partitioned ADD IF NOT EXISTS PARTITION ( COUNTRY = 'Spain') LOCATION cos://us-geo/sql/customers_partitioned.csv/COUNTRY=Spain
-- alter the table partitions by dropping a partition
ALTER TABLE customers_partitioned DROP IF EXISTS PARTITION ( COUNTRY = 'Nowhere')

The SET LOCATION option can be used to change the location of an existing partition.

-- modify the location of an existing partition
ALTER TABLE customers_partitioned PARTITION (country = 'Spain') SET LOCATION cos://eu-de/sql/customers_partitioned.csv/COUNTRY=Spain

Use the EXISTS option to avoid getting errors during ADD or DROP.







 Alter table columns 



 alterTableColumns 

Use alter table to add new columns to the schema of a catalog table.

Adding columns to the schema has no effect on the objects in Object Storage for the table. If some or all of these objects do not contain a column, values for the corresponding rows are treated as NULL. You can add new objects to the table location (for nonpartitioned tables) or new partitions (for partitioned tables) that contain the new columns to evolve the table schema.

-- create a partitioned sample table in PARQUET format
CREATE TABLE customers_addcol USING PARQUET LOCATION cos://us-geo/sql/customers_partitioned.parquet
-- add a new column that does not yet occur in existing partitions. new partitions can add data for that column
ALTER TABLE customers_addcol ADD COLUMNS (priority INTEGER)

Do not use the ADD COLUMNS option with CSV tables. The CSV data format identifies columns by order (not by name), so any schema change leads to a schema mismatch with existing data.

Alternatively, you can perform schema changes by dropping and re-creating catalog tables. It does not affect the stored data in Object Storage.
GROOM TABLES processes and reorganizes table records in each data slice in a series of steps. You can still run operations such as SELECT, UPDATE, DELETE, and INSERT while data grooming is happening.
SELECT operations run in parallel with the groom operations.
INSERT, UPDATE, and DELETE operations run serially between the groom steps.
For cluster based tables (CBTs), GRROM TABLE takes longer. Tthe INSERT, UPDATE, and DELETE operations might be pending longer until the current step completes.

You can use the Groom Table functionality to reclaim disk space from deleted or outdated rows. You can also use this option to reorganize tables based on the clustered base table organizing keys or to migrate data from tables that have multiple stored versions.



1. Go to Databases.
2. Select your database and schema.
3. Ensure that you are in the DB Objects > Tables tab.
4. Select the table that you want to groom.
5. Go to the Grooms tab.
6. Click Groom table.





* To run the operation, the user must have object privileges. The user does not have to be an administrator.
* You cannot groom tables inside a transaction block (begin or commit pair) or with a stored procedure.
* When you specify organizing keys for an existing table to make it a CBT, the new organization might impact the compression size of the table. The new organization can create sequences of records that improve the overall compression benefit, or it can create sequences that do not compress so well. After a groom operation, your table size might change.





 Modes 

You can choose different modes when you groom tables.



Table 1. The table lists modes-related values and their definitions.

 Value Description 

 RECORDS READY Reclaims and reorganizes records in the tables that were not groomed, and in those that were already groomed but got marked for regrooming. This is the default setting for clustered base tables (CBT). 
 RECORDS ALL Reclaims and reorganizes all records in a table. This is the default for a non-CBT.
About data ingestion 

Data ingestion is the process of importing and loading data into IBM® watsonx.data. You can use the Create table option from the Data manager page to load local or external sources of data files to create tables.

When you ingest a data file into the watsonx.data, the table schema is generated and inferred when a query is run. Data ingestion in watsonx.data supports CSV and Parquet formats. The files to be ingested must be of the same format type and same schema. watsonx.data auto-discovers the schema based on the source file being ingested.



 Loading or ingesting data through CLI 

An ingestion job in watsonx.data can be run with the ibm-lh tool. The tool must be pulled from the ibm-lh-client and installed in the local system to run the ingestion job through the CLI. For more details and instructions to install ibm-lh-client package and use the ibm-lh tool for ingestion, see [Installing ibm-lh-client](https://www.ibm.com/docs/en/watsonxdata/1.0.x?topic=package-installing-lh-client) and [Setting up the ibm-lh command-line utility](https://www.ibm.com/docs/en/watsonxdata/1.0.x?topic=package-setting-up-lh-cli-utility).

The ibm-lh tool supports the following features:



* Auto-discovery of schema based on the source file or target table.
* Advanced table configuration options for the CSV files:



* Delimiter
* Header
* File encoding
* Line delimiter
* Escape characters



* Ingestion of a single, multiple file(s), or a single folder (no sub folders) of S3 and local Parquet file(s).
* Ingestion of a single, multiple file(s), or a single folder (no sub folders) of S3 and local CSV file(s).
This error can be seen using [Activity Tracker](https://cloud.ibm.com/docs/databases-for-mysql?topic=databases-for-mysql-activity-tracker) or [Log Analysis](https://cloud.ibm.com/docs/databases-for-mysql?topic=databases-for-mysql-logging).



 Resolving corrupted_table error 

To resolve the corrupted_table error, query and optimize the Xtrabackup table using a command like:

SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLES WHERE TOTAL_ROW_VERSIONS > 0;

If the results are a list of tables, run OPTIMIZE TABLE on the list before taking a backup.

For more infortmation, see [Error Message: Found tables with row versions due to INSTANT ADD/DROP columns](https://docs.percona.com/percona-xtrabackup/8.0/em/instant.html).



* Resolving corrupted_table error

 Resolving corrupted_table error 

To resolve the corrupted_table error, query and optimize the Xtrabackup table using a command like:

SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLES WHERE TOTAL_ROW_VERSIONS > 0;

If the results are a list of tables, run OPTIMIZE TABLE on the list before taking a backup.

For more infortmation, see [Error Message: Found tables with row versions due to INSTANT ADD/DROP columns](https://docs.percona.com/percona-xtrabackup/8.0/em/instant.html).



Db2 Warehouse on Cloud



* How do I sign up for Db2 Warehouse on Cloud?

You can provision an instance of Db2 Warehouse on Cloud directly through the IBM Cloud® catalog. You can [create a free IBM Cloud account](https://cloud.ibm.com/registration?target=%2Fcatalog%2Fservices%2Fdb2-warehouse) and get an IBM Cloud credit of $200 that you can use towards Db2 Warehouse on Cloud.
The schema for this command is: curl -option1 -option2 http://example.com/url

For example:



* curl -svo /dev/null http://www.example.com
* curl -svo /dev/null -A “USER_AGENT_STRING” http://www.example.com
* curl -svo /dev/null -H “host: www.example.com” http://ORIGIN_IP
* curl -svo /dev/null -H https://www.example.com --resolve www.example.com:443:ORIGIN_IP







 Using mtr and traceroute 

mtr and traceroute are Unix command line tools that let you measure performance or latency along a specific network path to a specified host or destination server.

For example:



* mtr -rwc 20 example.com -T -4
* mtr -rwc 20 8.8.8.8 -T -6
* traceroute example.com -T -4
* traceroute 8.8.8.8 -T -6





Table 1. Command options

 Option Definition 

 -c Sets the number of pings sent 
 -T Forces a TCP traceroute (normally ICMP) 
 -4 Forces the use of IPv4 
 -6 Forces the use of IPv6 







 Generating an HAR file 

An HAR file is a recording of HTTP requests from a web browser. Browsers, such as Chrome, have a Developer Tools section that can help you get set up to make HAR files. For more information on how to create a HAR file, see [How do I generate a HAR file?](https://cloud.ibm.com/docs/cis?topic=cis-generate-har-files) in the troubleshooting section.
Allow an app to connect to a cluster by assigning read access to cluster resource type. 
 Writer Writers have permissions beyond the reader role, including editing Event Streams resources. Allow an app to produce to topics by assigning write access to topic resource and topic name types. 
 Manager Managers have permissions beyond the writer role to complete privileged actions. In addition, you can create and edit Event Streams resources. Allow full access to all resources by assigning manage access to the Event Streams instance. 







 How do I assign access? 

Cloud Identity and Access Management (IAM) policies are attached to the resources to be controlled. Each policy defines the level of access that a particular user must have and to which resource or set of resources. A policy consists of the following information:



* The type of service the policy applies to. For example, Event Streams. You can scope a policy to include all service types.
* The instance of the service to be secured. You can scope a policy to include all instances of a service type.
* The type of resource to be secured. The valid values are cluster, topic, group, schema, or txnid. Specifying a type is optional. If you do not specify a type, the policy then applies to all resources in the service instance. If you want to specify more than one type of resource, you must create one policy per resource.
* The resource to be secured. Specify for resources of type topic, group, schema, and txnid. If you do not specify the resource, the policy then applies to all resources of the type specified in the service instance.
* The role that is assigned to the user. For example, Reader, Writer, or Manager.







 What are the default security settings? 

By default, when Event Streams is provisioned, the user who provisioned it is granted the manager role to all the instance's resources. Additionally, any user who has a manager role for either 'All' services or 'All' Event Streams service instances' in the same account also has full access.

You can then apply more policies to extend access to other users. You can either scope a policy to apply to Event Streams as a whole or to individual resources within Event Streams.
Functions and variable operators are not supported in this release.

A future release intends to implement functions and operators.
* How are blueprint module dependencies and execution order determined?

Dependencies between blueprint modules are created using the value references between module inputs and outputs. Similar to Terraform, a DAG (Directed Acyclic Graph) is generated from the dependencies and relationships to determine execution order. Dependencies are created using '$module' references.
* How do I edit and validate blueprint templates?

Blueprint templates can be edited in any editor or IDE. Follow the instructions on how to use and configure VSCode to [edit templates and input files](https://cloud.ibm.com/docs/schematics?topic=schematics-edit-blueprints). The [Red Hat YAML VSCode extension](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) provides a framework for editing blueprint YAML files, using a [blueprint schema](https://github.com/Cloud-Schematics/vscode-blueprint-schema) defined using [JSON-Schema](https://json-schema.org) .
* Why do blueprints get the error 'Length for variable variable name greater than the given length'?

The default size of values that can be passed to Blueprints as inputs is set to 1KB. If the expected size is greater than 1KB, the [max_length inputs meta-data](https://cloud.ibm.com/docs/schematics?topic=schematics-bp-template-schema-yamlbp-inputs-max-len) can be defined to set the length that Blueprints should accept.

A value larger than 1KB or the specified length will result in the error Length for variable <variable name> greater than the given length
* Why do blueprint operations require a blueprint ID?

In Schematics Blueprints, the displayed name is not a unique identifier. Only the blueprint ID that is generated at create time is unique.
"city": "Happenstance",
"state": "CT",
"postal_code": "06058"
}
}

A graph fragment variant of the same schema might define objects such as:

{
"name": "Stanley Q. Dallas",
"address": {
"postal_code": "06058"
}
}





 Resource collection schemas 

The schema for a [resource collection](https://cloud.ibm.com/docs/api-handbook?topic=api-handbook-collections-overview) returned by a [list operation](https://cloud.ibm.com/docs/api-handbook?topic=api-handbook-operationsstandard-operations) SHOULD be the name of the canonical schema for the resource type, suffixed with Collection.

For example, a GET /goat/{goat_id}/chores operation would return a resource collection represented with the GoatChoreCollection schema.

Each resource in a collection SHOULD be represented with its canonical schema. If performance considerations make it infeasible to return the canonical representation of a schema, a lighter-weight schema variant suffixed with Summary MAY be used. The summary representation of a resource SHOULD include all the critical information that would be expected in a user interface or command-line interface listing such resources.

For example, a GreebleCollection would contain an array of resources represented with the Greeble schema or, if infeasible, a GreebleSummary schema.

A summary schema MUST be a [graph fragment](https://cloud.ibm.com/docs/api-handbook?topic=api-handbook-schemasgraph-fragment-pattern) variant of the canonical schema for the resource type.





 Resource creation and replacement schemas 

The request schema used in an operation to create a new resource with a POST (or PUT) request or to replace a resource

[1]with a PUT request SHOULD be the name of the canonical schema for the resource type, suffixed with Prototype.
Revoke access Y N N N 
 Create catalog Y N N N 
 Modify files into the bucket Y Y N N 
 Browse (bucket browser in UI) Y Y Y N 
 View existance Y Y Y Y 







 Database 



Table 4. Roles and privileges for a database

 Privileges Administrator Writer Reader User without a role 

 Unregister Y N N N 
 Update database connection properties (credentials) Y N N N 
 Grant access Y N N N 
 Revoke access Y N N N 
 Create catalog Y N N N 
 Modify database objects Y Y N N 
 Browse remote data (when connected to an engine) Y Y Y N 
 View existance Y Y Y Y 







 Schema 



Table 4. Roles and privileges for a schema

 Privileges Catalog Administrator or Schema creator Others 

 Grant access Y N 
 Revoke access Y N 
 Column access Y N 
 Create table Y N 







 Table 



Table 4. Roles and privileges for a table

 Privileges Catalog Administrator or Schema Administrator or Table creator Others 

 Create Y N 
 Drop Y N 
 Alter Y N 
 Column access Y N 
 Select Y N 
 Insert Y N 
 Update Y N 
 Delete Y N 



Where

Y indicates that members of this role have this permission. N indicates that members of this role do not have this permission.