vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Databases 

You can access database information by using the web console.



 Information about databases 



Table 1. The table lists database-related values and their definitions.

 Value Description 

 Name Specifies the name of the database. 
 Default schema Specifies the deafult schema. 
 Owner Specifies the owner of the database. 
 User tables Specifies the number of tables that are created in the database. 
 Character set Specifies the character set for the database. 
 Created on Specifies when the database was created. 







 Creating databases 



1. Go to Databases.
2. Click Create database.
3. Type a name for the database.
If the name contains special characters, enclose it in double quotation marks. The dot character (".") is not supported.
4. Optional: Specify the retention time interval (in days) for the database.
You can select between 1 day and up to 99 days.
For more information on retention time interval and time travel, see [Netezza Performance Server time travel](https://cloud.ibm.com/docs/netezza?topic=netezza-enablingdisabling_tt).
5. Click Create.







 Assigning owners to databases 



1. Go to Databases.
2. Select the database for which you want to assing an owner.
3. From the overflow menu, click Assign owner.
4. Select an owner for the database.
5. Click Assign.







 Renaming databases 



1. Go to Databases.
2. Select the database that you want to rename.
3. From the overflow menu, click Rename.
4. Type a new name for the database.
If the name contains special characters, enclose it in double quotation marks. The dot character (".") is not supported.
5. Click Rename.







 Updating retention time interval (time travel) for databases 



1. Go to Databases.
2. Select the database for which you want to update the retention time interval.
3.
if (returnSchema.size () != arg1Schema.size () + arg2Schema.size ()) { throw new TableUDFException (
"Schema sizes don't match (%d + %d != %d)", arg1Schema.size (), arg2Schema.size (), returnSchema.size ()); }

// Then check types
for (int i = 0; i < arg1Schema.size (); i++) {
if (false == (arg1Schema.getFieldTypeByIx (i).equals (returnSchema.getFieldTypeByIx (i)))) { throw new TableUDFException (
"Field type %d of output doesn't match corresponding field of first input arg (%s != %s)", i,
returnSchema.getFieldTypeByIx (i), arg1Schema.getFieldTypeByIx (i)); }
}

for (int i = 0; i < arg2Schema.size (); i++) {
if (false == (arg2Schema.getFieldTypeByIx (i).equals (returnSchema.getFieldTypeByIx (i + arg1Schema.size ())))) { throw new TableUDFException (
"Field type %d of output doesn't match corresponding field of first input arg (%s != %s)", i
+ arg1Schema.size (), returnSchema.getFieldTypeByIx (i + arg1Schema.size ()), arg2Schema.getFieldTypeByIx (i)); }
}
}

}
Show more







 Declaring user-defined functions 

You can make the user-defined scalar functions and machine learning models from PMML files available to AQL by using the create function statement.



 Syntax 

The general syntax of the create function statement is as follows:

create function <function-name>(<input-schema-definition>)
4. Click Create host group.
5. Enter a name for your host group and select the Schematics Workspaces that provisioned the target hosts that you want to add to your host group.
6. Optional: Click Add query to add another condition to your query and limit the number of target hosts that are added to your host group. You can choose to identify your hosts by using the resource name or a tag. For more information, see [supported resource queries](https://cloud.ibm.com/docs/schematics?topic=schematics-inventories-setupsupported-queries).
7. Repeat step 4-6 to add more host groups.
8. From the table list, select the host groups that you want to include in your resource inventory.
9. Click Create inventory.
10. Follow the [steps](https://cloud.ibm.com/docs/schematics?topic=schematics-action-workingcreate-action) to create the Schematics Actions and use the resource inventory that you created.





 Supported resource queries 



Resource query

 Supported query Description 

 Workspace name Select all the IBM Cloud resources from a specific workspace. 
 Workspace name AND resource name Select a specific resource from a specific workspace by using the resource name. To select multiple resources from the same workspace, you can add multiple queries of this type. 
 Workspace name AND tag Select a specific resource from a specific workspace by using tags. Tags are added to the resource when you create the workspace and provision your resource. To select multiple resources with different tags from the same workspace, you can add multiple queries of this type. 







 Limitations 

Review the following limitations of dynamic inventories in Schematics:



* You can choose among the [supported queries](https://cloud.ibm.com/docs/schematics?topic=schematics-inventories-setupsupported-queries) to select the target virtual server instances to include in your resource inventory.
* Schematics retrieves the IP address of a target Virtual Servers for VPCs and adds the IP address to the resource inventory.
Known issues (Limitations) 

The following limitations and known issues, apply to IBM® watsonx.data.



 Issue: Unable to view created schema 

When a user with the User role and the Create access (the user only has the Create access) is added to an external database, they cannot see the schemas that they created. Though the user can create schemas, they cannot view them. Following is the system response:

presto:default> show schemas;
Schema
--------
(0 rows)

Workaround: Provide select privilege for the schema the user created.





 Issue: Access denied message occurs when querying an external database 

When a user with the User role and Create access (the user only has Create access), is added to an external database, they cannot run the select query from the table they have created. Though the user can connect to the Presto engine and create tables and schemas, they cannot query from the table. The system displays an Access Denied message.

Query 20230608_132213_00042_wpmk2 failed: Access Denied: Cannot select from columns [id] in table or view tab_appiduser_01

Workaround: Provide select privilege for the table the user created.





 Issue: Schema created under different catalog 

Schemas are available across Iceberg and Hive catalogs. When a schema is created under Iceberg catalog, it is listed under Hive catalog and vice versa.





 Issue: Presto does not support deletion of Iceberg tables 





 Issue: DROP SCHEMA in Db2 

In Db2, the schema can be dropped only if it is empty. Initiating DROP SCHEMA statement against a non-empty schema may result in Db2 SQL Error SQLCODE=-478 and SQLSTATE=42893.





 Issue: CREATE VIEW statement that is partially supported by Db2 

Db2 connector partially supports CREATE VIEW statement. The Presto supported SQL syntax does not include creating views with custom column names (different than the table column names).





 Issue: CREATE VIEW statement that is partially supported by Netezza 

Netezza connector partially supports CREATE VIEW statement.
* [Creating a workspace using the CLI](https://cloud.ibm.com/docs/schematics?topic=schematics-sch-create-wkscreate-wks-cli)
* [Creating a workspace using the API](https://cloud.ibm.com/docs/schematics?topic=schematics-sch-create-wkscreate-wks-api)
* [Creating a workspace using a Terraform template](https://cloud.ibm.com/docs/schematics?topic=schematics-sch-create-wkscreate-wks-terraform)
* [Next steps](https://cloud.ibm.com/docs/schematics?topic=schematics-sch-create-wkssch-create-wks-nextsteps)



[Creating Terraform templates](https://cloud.ibm.com/docs/schematics?topic=schematics-create-tf-configcreate-tf-config)



* [Configuring the provider block](https://cloud.ibm.com/docs/schematics?topic=schematics-create-tf-configconfigure-provider)
* [Adding IBM Cloud resources to the resource block](https://cloud.ibm.com/docs/schematics?topic=schematics-create-tf-configconfigure-resources)



* [Referencing resources in other resource blocks](https://cloud.ibm.com/docs/schematics?topic=schematics-create-tf-configreference-resource-info)



* [Managing resources in other account](https://cloud.ibm.com/docs/schematics?topic=schematics-create-tf-configmanage-resource-account)
* [Using variable blocks to customize resources](https://cloud.ibm.com/docs/schematics?topic=schematics-create-tf-configconfigure-variables)
CREATE TABLE flight_data AS SELECT * FROM EXTERNAL '/home/user/Downloads/customer_data.csv' (flight_id bigint, passenger_id int, last_name varchar(225), first_name varchar(225), seat_number int) USING (RemoteSource 'nzsql' DELIM ',' SkipRows 1);

For more examples on loading data with external tables, see [External tables](https://www.ibm.com/docs/en/netezza?topic=dl-external-tables-2).

Your data is loaded into Netezza Performance Server. You can start running queries now.







 Loading data from Amazon S3 

You can load data to Netezza Performance Server from a data file on Amazon S3 that you previously uploaded there.



1. Prepare the data that you want to load.



1. Ensure that the data is on Amazon S3.
2. Note the file format.



2. On Netezza Performance Server, create a table by using the [CREATE TABLE AS](https://www.ibm.com/docs/en/netezza?topic=npsscr-create-table-as-2) command.

The CREATE TABLE AS command creates a table on Netezza Performance Server and fills it with the data from S3.

As a part of this command, your data file is turned into [a transient external table](https://www.ibm.com/docs/en/netezza?topic=et-transient-external-tables-2). In other words, your data file is temporarily treated as a database table that you can query for loading to a Netezza Performance Server table. When you are finished, the transient external table is automatically deleted.

CREATE TABLE <table> AS SELECT * FROM EXTERNAL <file> USING (
REMOTESOURCE <source_type>
DELIM <delimiter_type>
UNIQUEID <unique_ID>
ACCESSKEYID <access_key_ID>
SECRETACCESSKEY <secret_access_key>
DEFAULTREGION <default_region>
When you register your own bucket, ensure to provide the correct details for bucket configuration. Quick start wizard does not validate the bucket configuration details and you cannot modify them later.

Ensure that the data bucket contains data.
2. Click Next. This displays the Catalogs configuration page.
3. In the Catalogs configuration page, select Apache Iceberg.
4. Click Next. This displays the Engine configuration page.
5. In the Engine configuration page, select the engine type and size.

Depending on the workload that you have, you can select the type and size of the Presto engine.
6. Click Next. This displays the Summary page.
7. In the Summary page, review the configurations before you finish setting up your data infrastructure.

When the setup is complete, the watsonx.data home page is displayed. You are all set to use the watsonx.data or you can configure it further.
8. Click Finish and go. This displays the Infrastructure manager page.



You can add more engines, catalogs, buckets, and databases in the Infrastructure manager page if you want to. For more information about creating engines, catalogs, buckets, databases, see Configuring watsonx.data components in the How To section.

--------------------





 Step 4: Querying data 

In this section of the tutorial, you learn how to navigate to the Query workspace, and create SQL queries to query your data from the bucket.

To run the SQL queries, do the following steps:



1. From the navigation menu, select SQL. The Query workspace page opens.
2. Select the Presto engine from the Engine drop-down.
3. Select the catalog. In this scenario, consider the Apache Iceberg catalog, default schema, order_detail table, to run the query.
4. Click the overflow menu and select the required query.



* For a catalog and schema, you can run the Generate Path query.
Go to Resource list>Databases.
3. Click your watsonx.data instance link. The service instance page opens.
4. Click the Open web console button to start the web console.
5. Log in to the console with your IBM ID and password.







 Step 3: Adding a Presto engine 

After you log in to the watsonx.data web console, you are on the home page. The home page provides information about the logged in user, the login timestamp, and region where the watsonx.data is created. In this part of the tutorial, you learn how to navigate to the Infrastructure manager page where you can design your watsonx.data by provisioning engines, catalogs, and databases.

This scenario assumes that it is not the first log in attempt to watsonx.data web console.

To create a Presto engine, complete the following steps:



2. From the navigation menu, select Infrastructure manager.
3. To provision an engine, click Add component and select Create engine.
4. In the Create engine window, provide the following details to a sign up new compute to work with your data.



Table 1. Provision engine

 Field Description 

 Name Enter your compute engine name. 
 Type Select Presto v0.279. 
 Size Select the required size based on whether your workload is compute intense or storage intense. 
 Description (optional) Describe the intended purpose of engine. 
 Tags (optional) Enter the tag name or select from drop-down list. 
 Associate catalogs If required, you can associate a catalog. 



5. Click Create. The engine is created.







 Step 4: Adding database 

Next, you learn the steps to setup the two specific databases, Db2 and Netezza, which serves as the source of data for information retrieval.



1. From the navigation menu, select Infrastructure manager.
2. To add a database, click Add component and select Add database.
3. In the Add database form, select IBM Db2 from the database type drop-down menu to create Db2 database.

Select IBM Netezza from the database type drop-down menu to create Netezza database.
* Install the [Schematics command-line](https://cloud.ibm.com/docs/schematics?topic=schematics-setup-cliinstall-schematics-plugin) plug-in, or [update the command line plug-in](https://cloud.ibm.com/docs/schematics?topic=schematics-setup-clischematics-cli-update) to access the Schematics blueprints commands.
* Check that you have the right [permissions](https://cloud.ibm.com/docs/schematics?topic=schematics-accessblueprint-permissions) to create blueprints.





 Step 1: Select a blueprint template 

This tutorial uses a [sample blueprint](https://github.com/Cloud-Schematics/blueprint-basic-example) to create two blueprint modules referencing a resource group and creating an IBM Cloud® Object Storage instance and bucket. No costs are incurred deploying this example.





 Step 2: Create the blueprint configuration 

Create your blueprint environment by using CLI command [ibmcloud schematics blueprint config create](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-blueprint-create). Use the following parameters to create a blueprint.

For Schematics blueprints, the [Schematics plug-in](https://cloud.ibm.com/docs/schematics?topic=schematics-setup-cliinstall-schematics-plugin) version must be greater than the 1.12.3.



* The Name of the blueprint: Blueprint_basic
* The Schematics management resource group: <default-resource-group-name>
* The blueprint URL: https://github.com/Cloud-Schematics/blueprint-basic-example
* The blueprint file: basic-blueprint.yaml
* An Input file URL: https://github.com/Cloud-Schematics/blueprint-basic-example
For example, a view that is exported does not qualify as a view that is output. In addition to these views, the unique Document view represents the input document that is annotated by this extractor.



 Document view 

At the AQL module level, Document view is a special view that represents the current document that is being annotated by that module. When two or more modules are combined to form an extractor, the duplicate-free union of Document schemas of all modules is the required Document view. Use the require document with columns statement to specify the schema of the Document view. If this statement is absent from a module, the default schema that is assumed for the Document view is (text Text, label Text):



* text

The textual content of the current document.
* label

The label of the current document that is being annotated.



The keyword Document is reserved as an identifier for the Document view, which is automatically populated during execution. Therefore, you cannot define another view or table with the same name. However, you can use the word Document as an identifier for attribute names and aliases.







 AQL statements 

By using AQL statements, you can create and then use modules, views, tables, dictionaries, and functions.

The following statements are supported in AQL:



* Statements for creating modules and declaring the interaction between them

The module statement

The export statement

The import statement
* Statements for creating AQL objects: views, dictionaries, tables, or functions

The create dictionary and create external dictionary statements

The create table statement

The create view statement

The create external view statement

The detag statement

The extract statement

The select statement

The require document with columns statement

The set default dictionary language statement

The create function statement



The syntax specifications in the AQL statements contain brackets ([ ]). These specifications indicate that the brackets and the constructs that they hold are optional when the corresponding syntax is used in writing a statement. In addition, they stand as place holders to define how to specify additional instances of a construct or argument.





 The module statement
Refer to the section on [Creating Blueprints](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-blueprint-create) for examples of command syntax and output.

Before you begin:



* Install or update the [Schematics plug-in](https://cloud.ibm.com/docs/schematics?topic=schematics-setup-cliinstall-schematics-plugin) version that is greater than the 1.12.5.
* Select the IBM Cloud region that you wish to use to manage your Schematics. Set the region by running [ibmcloud target -r <region>](https://cloud.ibm.com/docs/cli?topic=cli-ibmcloud_cliibmcloud_target) command. For more information, see [FAQ](https://cloud.ibm.com/docs/schematics?topic=schematics-blueprints-faqfaqs-bp-target-region).
* Check that you have the [IAM permissions](https://cloud.ibm.com/docs/schematics?topic=schematics-accessblueprint-permissions) to create blueprints.



Syntax to create blueprint config using CLI parameters:
Add the following properties to the db2.properties file.

connector.name=db2

connection-url=jdbc:db2://<hostname>:<port>/<database name>

connection-user=<user name>

connection-password=<password>

allow-drop-table=<true/false>



If you want to create multiple catalogs for Db2, name the properties files with respective catalog names.





 Limitations 



1. In Db2, the schema can be dropped only if it is empty. Initiating DROP SCHEMA statement against a nonempty schema is expected to result in Db2 SQL Error SQLCODE=-478 and SQLSTATE=42893.
2. Db2 connector partially supports CREATE VIEW statement. The Presto supported SQL syntax does not include creating views with custom column names (different than the table column names).
{
"name": "<INVENTORY_NAME>",
"description": "<DESCRIPTION",
"location": "<GEOGRAPHY>",
"resource_group": "<RESOURCE_GROUP>",
"resource_queries": [
"<RESOURCE_QUERY_ID>"
]
}

Example

{
"name": "myinventory",
"description": "This is the resource inventory for production",
"location": "us-east",
"resource_group": "default",
"resource_queries": [
"default.RESOURCEQUERY.string.df3d8a47"
]

}

ibmcloud schematics inventory create --file inventory.json





 Using the interactive mode 

Instead of entering your inventory details by using the command options or a payload file, you can also use the interactive mode for the command. This mode prompts you to enter the required values to create an inventory in Schematics.



1. Enter the command to create the inventory without any command options.

ibmcloud schematics inventory create
2. Enter a name for your inventory and press the return key.
3. Enter the resource group where you want to create the inventory and press the return key.
4. Enter the location where you want to create the inventory, such as us-south, us-east, eu-de, or eu-gb. Then, press the return key.
5. Review the details of the inventory that was created.









 ibmcloud schematics inventory delete 

Delete the resource inventory definition by using the inventory ID from the IBM Cloud Schematics service. Note you can delete the location and region, resource group from where your inventory was created. Also, make sure your IP addresses are in the [allowlist](https://cloud.ibm.com/docs/schematics?topic=schematics-allowed-ipaddresses).

Syntax

ibmcloud schematics inventory delete --id ACTION_ID [--force]

Command options
The table lists modes-related values and their definitions.

 Value Description 

 RECORDS READY Reclaims and reorganizes records in the tables that were not groomed, and in those that were already groomed but got marked for regrooming. This is the default setting for clustered base tables (CBT). 
 RECORDS ALL Reclaims and reorganizes all records in a table. This is the default for a non-CBT. 
 PAGES ALL Identifies and marks the data pages in the table with no visible record as Empty to free up disk extents. 
 PAGES START Identifies and marks the leading data pages in the table with no visible record as Empty. Stops when it finds a non-empty data page. 
 VERSIONS Migrates records from previous table versions. Dropped columns do not appear and added columns show default values. 









 Assigning owners to tables 



1. Go to Databases.
2. Select the database and schema in which the table that you want to update is.
3. Select the table for which you want to assign an owner.
4. From the overflow menu, click Assign owner.
5. Select an owner for the table.
6. Click Assign.







 Renaming tables 



1. Go to Databases.
2. Select the database and schema in which the table that you want to rename is.
3. From the overflow menu, click Rename.
4. Type a new name for the table.
If the name contains special characters, enclose it in double quotation marks. The dot character (".") is not supported.
5. Click Rename.







 Updating retention time interval (time travel) for tables 



1. Go to Databases.
2. Select the database and schema in which the table that you want to update is.
3. Select the table.
4. From the overflow menu, click Update interval.
5. Type a retention time interval.
You can select between 1 day and up to 99 days, or zero to alter a temporal database to nontemporal.
* [Select a blueprint template](https://cloud.ibm.com/docs/schematics?topic=schematics-get-started-blueprintsget-started-blueprints-select)
* [Creating a blueprint in Schematics](https://cloud.ibm.com/docs/schematics?topic=schematics-get-started-blueprintsget-started-blueprints-create)
* [Applying the blueprint to create cloud resources](https://cloud.ibm.com/docs/schematics?topic=schematics-get-started-blueprintsget-started-blueprints-install)
* [Next steps](https://cloud.ibm.com/docs/schematics?topic=schematics-get-started-blueprintsget-started-blueprints-nextsteps)









 About 

[Learning about Schematics](https://cloud.ibm.com/docs/schematics?topic=schematics-learn-about-schematicslearn-about-schematics)



* [IaC automation as-a-service](https://cloud.ibm.com/docs/schematics?topic=schematics-learn-about-schematicssc-IaCaas)
* [Schematics IaC offerings](https://cloud.ibm.com/docs/schematics?topic=schematics-learn-about-schematicssc-offerings)



* [Schematics Blueprints](https://cloud.ibm.com/docs/schematics?topic=schematics-learn-about-schematicssc-blueprints)
* [Schematics Workspaces](https://cloud.ibm.com/docs/schematics?topic=schematics-learn-about-schematicssch-workspaces)
* [Schematics Actions](https://cloud.ibm.com/docs/schematics?topic=schematics-learn-about-schematicssc-actions)