vania-janet/multiturn-rag-retrieval-data huggingface dataset:
right_table.col1 AS r_col1,
right_table.col2 AS r_col2
FROM
VALUES (0, 10), (1, 11), (2, 12), (3,13), (4, 14), (5, 14) AS left_table
RIGHT OUTER JOIN
VALUES (0, 10), (2, 12), (4, 14), (6, 16) AS right_table
ON left_table.col1 = right_table.col1

The result of the example query is shown in the following table.



Table 20. Query result for example: right outer join query.

 L_COL1 L_COL2 R_COL1 R_COL2 

 0 10 0 10 
 2 12 2 12 
 4 14 4 14 
 null null 6 16 



A FULL OUTER or FULL join includes the rows from both tables that were missing from the inner join.

-- full outer join query
SELECT
left_table.col1 AS l_col1,
left_table.col2 AS l_col2,
right_table.col1 AS r_col1,
right_table.col2 AS r_col2
FROM
VALUES (0, 10), (1, 11), (2, 12), (3, 13), (4, 14), (5, 14) AS left_table
FULL OUTER JOIN
VALUES (0, 10), (2, 12), (4, 14), (6, 16) AS right_table
ON left_table.col1 = right_table.col1

The result of the example query is shown in the following table.



Table 21. Query result for example: full outer join query.

 L_COL1 L_COL2 R_COL1 R_COL2 

 1 11 null null 
 null null 6 16 
 3 13 null null 
 5 14 null null 
 4 14 4 14 
 2 12 2 12 
 0 10 0 10 







 Cross join
The result of the example query is shown in the following table.



Table 4. Query result for the example set

 COL1 

 1 
 2 
 3 
 1 
 2 
 3 



-- intersecting set eliminating duplicate rows
SELECT * FROM VALUES 1, 2 ,3 INTERSECT VALUES 2, 2, 3, 3, 3

-- intersecting set eliminating duplicate rows
SELECT * FROM VALUES 1, 2 ,3 INTERSECT DISTINCT VALUES 2, 2, 3, 3, 3

The result of the example queries is shown in the following table.



Table 5. Query result for the example intersecting

 COL1 

 3 
 2 



-- Difference quantity eliminating duplicate rows
SELECT * FROM VALUES 1, 2 ,3 EXCEPT VALUES 2, 2, 3, 3, 3

-- difference quantity eliminating duplicate rows
SELECT * FROM VALUES 1, 2 ,3 EXCEPT DISTINCT VALUES 2, 2, 3, 3, 3

-- difference quantity eliminating duplicate rows
SELECT * FROM VALUES 1, 2 ,3 MINUS VALUES 2, 2, 3, 3, 3

-- difference quantity eliminating duplicate rows
SELECT * FROM VALUES 1, 2 ,3 MINUS DISTINCT VALUES 2, 2, 3, 3, 3

The result of the example queries is shown in the following table.



Table 6. Query result for the example difference

 COL1 

 1 







 More topics - fullselect 

For more information about the clauses that are used in a fullselect, see the following topics:



* [expression](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referenceexpression)
* [fullselect](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencefullselect)
* [namedWindows](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencenamedWindows)
* [simpleselect](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencesimpleselect)
If the specified data type is not supported, you receive an error.



 castExpression 

In case an expression cannot be cast to the data type specified in the cast expression, the expression result is null.





 More topics - cast expression 

For more information about the clauses that are used by a cast expression, see the following topics:



* [dataType](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencedataType)
* [expression](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referenceexpression)
* [identifier](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referenceidentifier)







 Related references - cast expression 

A cast expression is referenced by the following clause:



* [primaryExpression](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referenceprimaryExpression)









 Case expressions 

A case expression allows an expression to be selected based on the evaluation of one or more conditions.

The syntax of a case expression is described by the following syntax diagrams.



 caseExpression 

The upper path in the syntax diagram represents a searched when clause, which means that the WHEN keyword follows directly after the CASE keyword. The lower path is a simple when clause, which means that an expression follows the CASE keyword.

In general, the value of the case expression is the value of the result expression, following the first (leftmost) case that evaluates to true. If no case evaluates to true and the ELSE keyword is present, the result is the value of the ELSE case result expression. If no case evaluates to true and the ELSE keyword is not present, the result is NULL. When a case evaluates to unknown (because of NULLs), the case is not true, and hence is treated the same way as a case that evaluates to false.

When you use the simple when clause, the value of the expression before the first WHEN keyword is tested for equality with the value of the expression that follows the WHEN keyword.
Views 



  Creating views 



1.  Go to Databases
2.  Select the database in which you want to create a view.
3.  Select the schema in which you want to create a view.
4.  Go to the DB Objects > Views tab.
5.  Click Create view.
6.  Type a name for the view.
The name can be up to 128 characters long and it must begin with a letter or underscore and cannot contain embedded spaces. The name must be unique.
If the name contains special characters, enclose it in double quotation marks.
7.  Specify whether the view is materialized.
8.  Specify the query.
Enter the SQL query that provides the columns and rows of this view.
If the database object name contains special characters, enclose it in double quotation marks.
9.  Click Create.
database="ibmclouddb")
except:
print("Unable to connect to database")

cur = conn.cursor()
cur.execute("SELECT datname FROM pg_database")
rows = cur.fetchall()

print("List of databases:")
for row in rows:
print(" ",row[0])
Show more

The following example uses the information from your connection string and the Node driver [node-postgres](https://node-postgres.com/) to connect to your database.

const pg = require("pg");
const fs = require("fs");

let connectionString = "postgres://<username>:<password>@<host>:<port>/<database>";
let caCert = fs.readFileSync('/path/to/cert');

// set up a client with your PostgreSQL connection string and TLS options
let client = new pg.Client({
connectionString: connectionString,
ssl: {
ca: caCert,
rejectUnauthorized: true
}
});

client.connect(function(err) {
if (err) {
console.log(err);
process.exit(1);
} else {
// query for the names of the databases
client.query(
"SELECT datname FROM pg_database;",
function(err, result) {
if (err) {
console.log(err);
}
// return the names of the databases
console.log(result.rows);
client.end();
}
);
}
});
Show more

To use the node-postgres driver, remove the sslmode parameter from the deployment's connection string. If not, this parameter overrides the ssl: {...
* RESULTS_BUCKET is the SQL URL of the custom COS bucket that you plan to use to upload the query results
* Use [cleancols](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-reference) to avoid transformation problems into PARQUET format when the name of the columns include special characters or blanks.



For example, the following query is used to transform an archive file:

SELECT * FROM cleancols(cos://ams03/at-eu-de/999999d8f1f.2019-06-03.62.json.gz STORED AS JSON)
INTO cos://eu-de/results-at STORED AS PARQUET
2. Click Run.

You can see the query result in the Result area of the UI.

You can see the target COS URL in the Result location area of the UI. This URL points to the object storage bucket used by Data Engine to store the results.

With the lite plan, you can run only 1 query. You can run up to 5 queries simultaneously with a paid plan.
3. Copy the Result location URL. You need it to run other queries to analyze the data that is included in that archive file.



After you run the transformation query, three objects are written as a result set in your COS results bucket:



1. jobid=<job_id>
2. jobid=<job_id>/_SUCCESS
3. jobid=<job_id>/<part-number>



Only one object contains the result set (jobid=<job_id>/<part-number>), and the other two are empty and don't contain any data.

It is important not to delete any of the files if you want to use the result set.

Each result is stored with an own job ID prefix that allows you to use the result directly in a query.
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
import java.util.logging.;

public class PGConnect {

private final static Logger LOGGER = Logger.getLogger(Logger.GLOBAL_LOGGER_NAME);

private Connection connect() {

final String url = "jdbc:postgresql://host:port/ibmclouddb";

Properties props = new Properties();
props.setProperty("user","admin");
props.setProperty("password","mypassword123");
props.setProperty("ssl","true");
props.setProperty("sslmode","verify-full");
props.setProperty("sslrootcert", "/path/to/cert");

Connection conn = null;
while (conn == null) {
try {
conn = DriverManager.getConnection(url, props);
System.out.println("Connected to PG");
} catch (SQLException e) {
System.out.printf("%sn", e);
LOGGER.info("Not connected, retying ...");
}
}

return conn;

}

public static void main(String[] args) {

PGConnect icd = new PGConnect();

try {
Connection connection = icd.connect();
Statement stmt = connection.createStatement();

ResultSet rs = stmt.executeQuery("SELECT * from pg_database");
while (rs.next()) {
System.out.println("DB Name: " + rs.getString(1));
}
Running SQL queries 

SQL is a standardized language for defining and manipulating data in a relational database. You can use the Query workspace interface in IBM® watsonx.data to run SQL queries and scripts against your data.

The Query workspace has the following components:



*  Data objects: To view the engines, catalogs, schemas, tables, and columns.
*  Engine: To select an engine and view the associated catalogs.
*  Saved queries: To view the saved queries.
*  Worksheet: To write SQL queries.



The Query workspace page provides basic options to undo, redo, cut, copy, paste, save, clear, and delete.

Format selection option is enabled only when an SQL statement in a query worksheet is selected. The Format worksheet option formats all the content in the worksheet. Comment selection is used to explain sections of SQL statements.

The Delete option is enabled only after an SQL query is saved.

To run the SQL queries, do the following steps:



1.  Log in to the watsonx.data console.
2.  From the navigation menu, select SQL. The Query workspace page opens.
3.  Select an engine from the Engine drop-down.
4.  Select the catalog, schema, table, or column in which you want to run the query.
5.  Click the overflow menu and select the required query.



*  For a catalog and schema, you can run the Generate Path query.
*  For a table, you can run the Generate path, Generate SELECT, Generate ALTER, and Generate DROP query.
*  For a column, you can run the Generate path, Generate SELECT, and Generate DROP query.



6.  Click the Save icon to save the query. A Save query confirmation dialog appears.
7.  Click Save.
8.  Click the Run button to run the query.
9.  Select Result set or Details tab to view the results.
10. Click Saved queries to view the saved queries.
11. Click [Explain](https://cloud.ibm.com/docs/watsonxdata?topic=watsonxdata-explain_sql_query) to view the logical or distributed plan of execution for a specified SQL query.
To provision an instance, see [Create your Data Engine service instance](https://cloud.ibm.com/docs/sql-query?topic=sql-query-getting-started).

To see the list of supported regions, see [Availability zones](https://cloud.ibm.com/docs/sql-query?topic=sql-query-overviewavailability).





 Step 2. Granting user permissions to run a query 

To run queries with the SQL query service, a user needs a platform role and a service role. For more information about roles, see [Required user roles](https://cloud.ibm.com/docs/sql-query?topic=sql-query-overviewuser-roles).

The following tables show the roles and actions that you can assign a user to run SQL queries:



Table 1. Platform roles

 Platform actions Administrator Editor Operator Viewer 

 View details of the Data Engine service instance ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/checkmark-icon.svg) !
The EXISTS predicate tests for the existence of certain rows. The query can specify any number of columns, and the following applies:



* The result is true only if the number of rows that is specified by the fullselect is not zero.
* The result is false only if the number of rows that is specified is zero.
* The result cannot be unknown.







 Related references - Boolean expression 

A Boolean expression is referenced by the following clauses:



* [booleanExpression](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencebooleanExpression)
* [expression](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referenceexpression)
* [relation](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencerelation)
* [simpleselect](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencesimpleselect)









 Value expressions 



 valueExpression 

A value expression is one of the following:



* A primary expression.
* The result of applying an unary operator to a value expression.
* The result of performing an arithmetic operation with two value expressions.
* The result of performing a comparison operation between two value expressions.







 More topics - value expression 

For more information about the clauses that are used by a value expression, see the following topics:



* [arithmeticOperator](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencearithmeticOperator)
* [comparisonOperator](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencecomparisonOperator)
* [primaryExpression](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referenceprimaryExpression)
> SELECT trim(LEADING 'SL' FROM 'SSparkSQLS')
: Result value

parkSQLS
: Example of an SQL function usage fragment

> SELECT trim(TRAILING 'SL' FROM 'SSparkSQLS')
: Result value

SSparkSQ





 trunc 

trunc(date, fmt)
: Returns date with the time portion of the day truncated to the unit specified by the format model fmt. fmt is one of ["year", "yyyy", "yy", "mon", "month", "mm"].
: Example of an SQL function usage fragment

> SELECT trunc('2009-02-12', 'MM')
: Result value

2009-02-01
: Example of an SQL function usage fragment

> SELECT trunc('2015-10-27', 'YEAR')
: Result value

2015-01-01





 ucase 

ucase(str)
: Returns str with all characters that are changed to uppercase.
: Example of an SQL function usage fragment

> SELECT ucase('SparkSql')
: Result value

SPARKSQL





 unbase64 

unbase64(str)
: Converts the argument from a base 64 string str to a binary.
: Example of an SQL function usage fragment

> SELECT unbase64('U3BhcmsgU1FM')
: Result value

Spark SQL





 unhex 

unhex(expr)
: Converts hexadecimal expr to binary.
: Example of an SQL function usage fragment

> SELECT decode(unhex('537061726B2053514C'), 'UTF-8')
: Result value

Spark SQL





 unix_timestamp 

unix_timestamp([expr[, pattern]])
: Returns the UNIX timestamp of current or specified time.
: Example of an SQL function usage fragment

> SELECT unix_timestamp()
: Result value
* AQL currently does not support advanced SQL features such as correlated subqueries and recursive queries.
* AQL has a new statement type, extract, which is not present in SQL.
* AQL does not allow keywords (reserved words) as view, column, or function names.
* AQL allows, but does not require, regular expressions to be expressed in Perl syntax. Regular expressions begin with a slash (/) and end with a slash (/), as in Perl syntax. AQL also allows regular expressions that start with a single quotation mark (') and end with a single quotation mark ('). For example, you can use /regex/ instead of 'regex' as the regular expression in AQL.
* Identifiers: Identifiers are used to define the names of AQL objects, including names of modules, views, tables, dictionaries, functions, attributes, and function parameters.
* Reserved words: Reserved words are words that have a fixed meaning within the context of the AQL structure and cannot be redefined. Keywords are reserved words that have special meanings within the language syntax.
* Constants: Constants are fixed values that can be one of these data types: String, Integer, Float, or Boolean.
* Comments" Use comments to augment AQL code with basic descriptions to help others understand the code and to generate self-describing compiled modules.
* Expressions: An AQL expression is a combination of one or more scalar values and functions that evaluates to a single scalar value.







 Identifiers 

Identifiers are used to define the names of AQL objects, including names of modules, views, tables, dictionaries, functions, attributes, and function parameters.

Two types of case-sensitive AQL identifiers exist:



* Simple identifier

A simple identifier must start with a lowercase (a-z) or uppercase (A-Z) letter or the underscore character (_). Subsequent characters can be lowercase or uppercase letters, the underscore character, or digits (0-9). A simple identifier must be different from any AQL key word.
* [sortItem](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencesortItem)
* [tableValuedFunction](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencetableValuedFunction)
* [valuesClause](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencevaluesClause)
* [whenClause](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencewhenClause)
* [windowSpec](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencewindowSpec)









 Boolean expressions 

The syntax of a Boolean expression is defined by the following syntax diagrams.



 booleanExpression 

A Boolean expression is one of the following:



* A negation of a Boolean expression by using Boolean operator NOT.
* A conjunction of Boolean expressions by using Boolean operator AND.
* A disjunction of Boolean expressions by using Boolean operator OR.
* An EXIST predicate.
* A value expression optionally followed by a predicate.



Refer to [booleanOperator](https://cloud.ibm.com/docs/sql-query?topic=sql-query-sql-referencebooleanOperator) for details about Boolean operators NOT, AND, and OR.

The EXISTS predicate tests for the existence of certain rows. The query can specify any number of columns, and the following applies:



* The result is true only if the number of rows that is specified by the fullselect is not zero.
* The result is false only if the number of rows that is specified is zero.
* The result cannot be unknown.







 Related references - Boolean expression 

A Boolean expression is referenced by the following clauses:
(8, 'C03'),
(9,'D01') AS emp
WHERE emp.col2 IN ('D01','D02')

The result of the example query is shown in the following table.



Table 40. Query result for the example all

 EMP_ID EMP_DEPT 

 0 D01 
 4 D01 
 5 D02 
 7 D01 
 9 D01 



-- all employees that are managing a department
SELECT
emp.col1 AS emp_id,
emp.col2 AS emp_dept
FROM VALUES
(0, 'D01'),
(2, 'C01'),
(3, 'C02'),
(4, 'D01'),
(5, 'D02'),
(6, 'C01'),
(7, 'D01'),
(8, 'C03'),
(9,'D01') AS emp
WHERE (emp.col1,emp.col2) IN (
SELECT
mgr.col1,
mgr.col2
FROM VALUES
(2, 'C01'),
(4, 'D01'),
(5, 'D02') AS mgr
)
Show more

The result of the example query is shown in the following table.



Table 41. Query result for the example all

 EMP_ID EMP_DEPT 

 2 C01 
 4 D01 
 5 D02 







 LIKE examples 

-- all employees that work in a department that starts with letter C
SELECT
emp.col1 AS emp_id,
emp.col2 AS emp_dept
FROM VALUES
(0, 'D01'),
(2, 'C01'),
(3, 'C02'),
(4, 'D01'),
(5, 'D02'),
As you add field mappings, a preview of the search result is displayed with information from the corresponding fields of your data collection. This preview shows you what gets included in the search result response that is returned to users.

To get help with configuring the search, see [Troubleshooting](https://cloud.ibm.com/docs/assistant?topic=assistant-skill-search-addskill-search-add-troubleshoot).
3. Use the Message, No results found and Connectivity issue tabs to customize different messages to share with users based on the successfulness of the search.



Search result messages

 Tab Scenario Example message 

 Message Search results are returned I found this information that might be helpful: 
 No results found No search results are found I searched my knowledge base for information that might address your query, but did not find anything useful to share. 
 Connectivity issue I was unable to complete the search for some reason I might have information that could help address your query, but am unable to search my knowledge base at the moment. 



4. Choose whether to enable Emphasize the answer.

This option is available only if your Discovery instance uses the v2 Discovery API.

When you enable this feature, the sentence that is determined by Discovery to be the exact answer to the customer's question is highlighted in the block of text that is displayed to the customer as the search result.
5. In the Adjust result quantity section, specify the number of results to return.

The top three results are returned automatically. You can choose to show fewer or more (up to 10) results in the response.

By default, customers can choose to see more results. If you don't want to give customers this choice, clear the Include link for customers to view up to 10 results checkbox.
6. In the Set result selectivity section, decide whether to be more selective with the answers that are returned. By increasing result selectivity, Search returns fewer but more accurate results. In most cases, Search is accurate enough that the default setting (off) is sufficient.
7. Click Preview to open the Preview pane for testing. Enter a test message to see the results that are returned when your configuration choices are applied to the search.