vania-janet/multiturn-rag-retrieval-data huggingface dataset:
What day would you like to come in? 
 You Nevermind 
 Watson Ok, we'll stop there. No reservation will be made. I look forward to helping you with your next reservation. Have a good day. 









 Step 8: Apply a valid value if the user fails to provide one after several attempts 

In some cases, a user might not understand what you are asking for. They might respond again and again with the wrong types of values. To plan for this possibility, you can add a counter to the slot, and after 3 failed attempts by the user to provide a valid value, you can apply a value to the slot on the user's behalf and move on.

For the $time information, you will define a follow-up statement that is displayed when the user does not provide a valid time.



1. Create a context variable that can keep track of how many times the user provides a value that does not match the value type that the slot expects. You want the context variable to be initialized and set to 0 before the node with slots is processed, so you will add it to the parent reservation node.
2. Click to edit the reservation node. Open the JSON editor associated with the node response, by clicking the Options![More icon](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/kebab.png) icon in the response section, and choosing Open JSON editor. Add a context variable called counter to the existing "context" block, after the confirmation variable. Set the counter variable equal to 0.

{
"context": {
"date": null,
"time": null,
"guests": null,
"confirmation": null,
"counter": 0
},
"output": {}
}
3. From the tree view, expand the reservation node, and then click to edit the node with slots.
4. Click the Edit slot!
[Operation](https://cloud.ibm.com/docs-content/v1/content/78bb71851d95d7580503eb9ba10cf3ae31490ade/apiconnect/tutorials/images/23-operation.png)
4. In the right panel under the examples, scroll to the Try this operation section. Enter the parameters and select Call operation.
![Try this operation](https://cloud.ibm.com/docs-content/v1/content/78bb71851d95d7580503eb9ba10cf3ae31490ade/apiconnect/tutorials/images/24-try-this-operation.png)
5. Scroll down to view the request and response of the operation call. A returned response of 200 OK and the message body are displayed, indicating that the operation call was successful.
![Operation response](https://cloud.ibm.com/docs-content/v1/content/78bb71851d95d7580503eb9ba10cf3ae31490ade/apiconnect/tutorials/images/25-operation-response.png)



--------------------





 Subscribe to APIs 

This section shows how a portal user would subscribe to APIs in the developer portal.



1. Select Create an account. ![API Products](https://cloud.ibm.com/docs-content/v1/content/78bb71851d95d7580503eb9ba10cf3ae31490ade/apiconnect/tutorials/images/31-create-account.png)
2. Complete the required fields and select Create new account at the bottom of the page. Note: Use a different email address than the one that you used to create your developer portal in the previous tutorial. !
{"_id":"4058553","_rev":"1-56100f7e7742028facfcc50ab6b07a04","name":"Decatur","latitude":34.60593,"longitude":-86.98334,"country":"US","population":55683,"timezone":"America/Chicago"},
{"_id":"4059102","_rev":"1-612ae37d982dc71eeecf332c1e1c16aa","name":"Dothan","latitude":31.22323,"longitude":-85.39049,"country":"US","population":65496,"timezone":"America/Chicago"}
] ,
"bookmark": "g1AAAAA-eJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqzmxiYWhoaGIGkOWDSyBJZAO9qD40",
}
Show more
See the documentation about query parameters]] ! ! ! for further details.<-- </section "id="section-how-cloudant-search-works" "> --><-- <section "id="section-mapreduce-views-accept-bookmarks" "> --> Do MapReduce views accept bookmarks? No. MapReduce views don't accept a bookmark. Instead, use one of the following tricks to page through results:<-- <ul> --> * The \u0000 trick]The _u0000 trick] . * Fetch one document too many]] . * Skip and limit]iterate through a result set] ! ! ! ! ! ! ! ! ! ! ! !
* <where clause>

Specifies one or more predicates. A join occurs when any predicate in a where clause involves fields from more than one view that belongs to the from list. This predicate must be a conjunction of a set of built-in predicate functions or other user-defined functions that return the data type Boolean:

function1() and function2()
and ... and functionn()

The where clause is optional and can be omitted from a select statement if no predicates exist to apply.







 Examples 

Example 1: Filter out joined tuples by using a predicate in the WHERE clause

This example shows a where clause that finds only phrases that consist of valid given names that are followed within 0-1 characters by valid surnames.

-- a view containing words that are valid given names
create view FirstName as
extract dictionary 'first.dict'
on D.text as firstname
from Document D;

-- a view containing words that are valid surnames
create view LastName as
extract dictionary 'last.dict'
on D.text as lastname
from Document D;

-- a view containing phrases consisting of valid given names
-- followed within 0-1 characters by valid surnames.
create view FullName as
select 
from
FirstName FN,
LastName LN
where
Follows (FN.firstname, LN.lastname, 0, 1);
Show more

The following names are contained in the external dictionary first.dict:

Dictionary for given names
Aaron
Candra
Freeman
Mathew
Matthew
Zoraida

The following names are contained in the external dictionary last.dict:

Dictionary for surnames
Anthony
Lehman
Radcliff







 The consolidate on clause 

The optional consolidate on clause specifies how overlapping spans are resolved across tuples that are output by a select or extract statement. Tuples with non-overlapping spans are not affected when this clause is used.



 Syntax 

The following code is an example of the general structure of this clause:

consolidate on <target>
* RESULTS_BUCKET is the SQL URL of the custom COS bucket that you plan to use to upload the query results



For example, to get the event time, the action, the criticality of the action, and the outcome, you can run the following query:

SELECT _source.eventTime AS EVENTTIME, _source.action AS ACTION, _source.severity AS SEVERITY, _source.outcome AS OUTCOME FROM PARQUET_FILE STORED AS PARQUET ORDER BY _source.eventTime
INTO RESULTS_BUCKET STORED AS CSV





 Step 3.8. Run a query to list all events for a specific action 

To see all the events for a specific action, run the following query:

SELECT FIELDS
FROM PARQUET_FILE STORED AS PARQUET
WHERE _source.action = ACTION
ORDER BY _source.eventTime
INTO RESULTS_BUCKET STORED AS CSV

Where



* FIELDS is the list of fields that you want to get information on for the different records. For example, you can enter _source.eventTime AS EVENTTIME, _source.action AS ACTION, _source.severity AS SEVERITY, _source.outcome AS OUTCOME
* PARQUET_FILE is the Result location URL that you get when you transform the archive file from JSON to PARQUET
* RESULTS_BUCKET is the SQL URL of the custom COS bucket that you plan to use to upload the query results
* ACTION* is the action value that is set in the action field of an event



For example, to get all the events for the action iam-identity.serviceid-apikey.login, you can run the following query:

SELECT _source.eventTime AS EVENTTIME, _source.action AS ACTION, _source.severity AS severity, _source.outcome AS OUTCOME
FROM cos://eu-de/results-marisa/jobid=f178778e-7707-46a9-982d-1e89261b63a5 STORED AS PARQUET
WHERE _source.action = "iam-identity.serviceid-apikey.login"
ORDER BY _source.eventTime
INTO cos://eu-de/results-marisa STORED AS CSV







 Reference.
Sending multiple queries to a database 

Now, the following instructions describe how to send multiple queries to a database by using _all_docs and _view endpoints.



 Sending multiple queries to a database by using _all_docs 

To send multiple queries to a specific database, send a POST request to https://$ACCOUNT.cloudant.com/$DATABASE/_all_docs/queries.

See the following example that uses HTTP to send multiple queries to a database:

POST /$DATABASE/_all_docs/queries HTTP/1.1

See the following example to multi-query the list of all documents in a database:



* Curl
* Java
* Python
* Go
* Node



curl -H "Authorization: Bearer $API_BEARER_TOKEN" -X POST "$SERVICE_URL/products/_all_docs/queries" -H "Content-Type: application/json" --data '{
"queries": [
{
"keys":
"small-appliances:1000042",
"small-appliances:1000043"
]
},
{
"limit": 3,
"skip": 2
}
]
}'

import com.ibm.cloud.cloudant.v1.Cloudant;
import com.ibm.cloud.cloudant.v1.model.AllDocsQuery;
import com.ibm.cloud.cloudant.v1.model.AllDocsQueriesResult;
import com.ibm.cloud.cloudant.v1.model.PostAllDocsQueriesOptions;

import java.util.Arrays;

Cloudant service = Cloudant.newInstance();

AllDocsQuery query1 = new AllDocsQuery.Builder()
.keys(Arrays.asList("small-appliances:1000042",
"small-appliances:1000043"))
.build();

AllDocsQuery query2 = new AllDocsQuery.Builder()
.limit(3)
.skip(2)
From mysql1 to mysql2: nc -w5 -z -v < ipaddress of mysql1/mysql2 > 3306.







 Step 7: Install Software on Virtual Server Instances 



 Install MySQL software on DB nodes: 



1. Log in to DB nodes through Bastion server.
2. Make sure that the name resolution works from each of the nodes as update /etc/hosts on each mysql node with each other's address.
3. Install mysql server:

apt-get install mysql-server mysql-client

This command installs the sql server in the default location.
4. Move the sql files to the data volume that you created earlier.



1. Stop the mysql service.

service mysql stop
2. Check the new volume on the mysql servers with lsblk. For instance, if it is named as vdb:

vdb     254:16   0   100G  0 disk
3. Partition this volume with parted or fdisk. For example, with fdisk, you can choose n to create a new partition, set first sector as 2048, and size as +100G, then save the changes with w. Verify that the partitions are created with lsblk or fdisk.
4. Create a Linux® file system on the vdb1 partition.

mkfs.xfs /dev/vdb1
5. Identify the UUID for secondary volume (vdb1) partition.

blkid | grep /dev/vdb1

/dev/vdb1: UUID="a65690bf-0c5e-467b-bb27-7c1558f1ed65" TYPE="xfs" PARTUUID="c4022fcb-01"
6. Enter this UUID command in fstab to make the mounts persistent after it restarts.
What day would you like to come in? 
 You Nevermind 
 Watson Ok, we'll stop there. No reservation will be made. I look forward to helping you with your next reservation. Have a good day. 









 Step 8: Apply a valid value if the user fails to provide one after several attempts 

In some cases, a user might not understand what you are asking for. They might respond again and again with the wrong types of values. To plan for this possibility, you can add a counter to the slot, and after 3 failed attempts by the user to provide a valid value, you can apply a value to the slot on the user's behalf and move on.

For the $time information, you will define a follow-up statement that is displayed when the user does not provide a valid time.



1. Create a context variable that can keep track of how many times the user provides a value that does not match the value type that the slot expects. You want the context variable to be initialized and set to 0 before the node with slots is processed, so you will add it to the parent reservation node.
2. Click to edit the reservation node. Open the JSON editor associated with the node response, by clicking the Options![More icon](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/kabob.png) icon in the response section, and choosing Open JSON editor. Add a context variable called counter to the end of the existing "context" block, after the confirmation variable. Set the counter variable equal to 0.

{
"context": {
"date": null,
"time": null,
"guests": null,
"confirmation": null,
"counter": 0
},
"output": {}
}
3. From the tree view, expand the reservation node, and then click to edit the node with slots.
4. Click the Edit slot!
It is used with [ETL-type statements](https://cloud.ibm.com/docs/sql-query?topic=sql-query-jdbcetl-type-statements), where the INTO options are provided as part of the statement.









 Driver functions 

This driver is designed as a facade to JVM applications for easy access to Data Engine service. The driver uses the REST API of the service to run queries, stores the results in Cloud Object Storage, and makes these results accessible through JDBC interfaces.

The JDBC driver supports Java 17. To use of the JDBC driver in a Java 17 JVM, you must pass the following option to the JVM:

--add-opens java.base/java.lang=ALL-UNNAMED

The driver does not implement full JDBC-compliant functions, but only the parts of the API that are most commonly used by clients. The following functions are explicitly tested:



* Retrieving query results with the following primitive SQL types: STRING or VARCHAR, INTEGER, LONG, FLOAT, DOUBLE, DECIMAL, DATE, TIME, TIMESTAMP. TIMESTAMP columns are handled as Coordinated Universal Time (UTC).
* Inspecting result columns and types through the JDBC ResultSetMetaData interface.
* Running DDL statements CREATE or DROP TABLE to manage objects in the [Data Engine catalog](https://cloud.ibm.com/docs/sql-query?topic=sql-query-hivemetastore).
* Accessing a list of tables and table columns through the JDBC DatabaseMetaData interface. The schema information is retrieved from the Data Engine catalog.



The SQL syntax itself is the full syntax that is supported by Data Engine except for the INTO clause. An INTO clause is implicitly added by the driver based on the Cloud Object Storage location that is given with the targetcosurl connection property.

Results in the targetcosurl location are never deleted by the driver. Use [expiration rules](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-expiry) to clean up your results on the IBM Cloud automatically.





 Basic limitations
Ensure that you have the necessary privileges as described in [Privileges for creating external tables](https://www.ibm.com/docs/en/netezza?topic=et-create-external-table-command-2).

create EXTERNAL table 'TABLE NAME' on 'DATA SOURCE'
using ( 
  DATAOBJECT ('DATA OBJECT') FORMAT 'PARQUET' 
);

The DATAOBJECT argument must reference a single file in the parquet format. If you want to query from multiple parquet files, you must create additional external tables.

Example:

create EXTERNAL table YELLOW_TAXI_JANUARY_2022 on EXAMPLEDATALAKE 
using ( 
   DATAOBJECT ('/yellow_tripdata_2022-01.parquet') FORMAT 'PARQUET' 
);





 3. Query your data. 

You can query external parquet format tables like you would any other Netezza Performance Server table without having to load the data into the database.

The parquet column names are case sensitive. You must use double quotation marks ("") when you are querying specific columns.



* To identify the total number of passengers that travelled by taxis in New York in January 2022, run:

select
sum("passenger_count") 
from YELLOW_TAXI_JANUARY_2022;

Output:

SUM
-----
3324167
(1 row)
* To identify the vendor that had the most passengers between 1:00 AM and 6:00 AM, run:

select
"VendorID",
sum("passenger_count") as "passengers"
from
YELLOW_TAXI_JANUARY_2022
where
"tpep_pickup_datetime"::time > '1:00am'
and "tpep_pickup_datetime"::time < '6:00am'
group by
"VendorID"
order by
"passengers" desc;

Output:

VendorID| passengers
--------|----------
2 | 122251
Make sure that you review pricing information for [classic VSIs](https://cloud.ibm.com/gen1/infrastructure/provision/vs) and [PostgreSQL](https://cloud.ibm.com/databases/databases-for-postgresql/create).



 Before you begin 

Before you can use this template, you must complete the following tasks.



* Make sure that you have the permissions to [create classic virtual servers](https://cloud.ibm.com/docs/virtual-servers?topic=virtual-servers-managing-device-access).
* [Create a classic API key](https://cloud.ibm.com/docs/account?topic=account-classic_keyscreate-classic-infrastructure-key) and retrieve your classic infrastructure username. This username and API key are used to verify that you have sufficient permissions to create classic infrastructure.
* Make sure that you have the permissions to create an [IBM Cloud Databases for PostgreSQL instance](https://cloud.ibm.com/docs/databases-for-postgresql?topic=cloud-databases-iam).







 Setting up and configuring a classic VSI to run PostgreSQL with Schematics 

Use one of the IBM provided software templates to set up and configure a classic VSI so that you can store data in an instance of IBM Cloud Databases for PostgreSQL.



1. Open the [VSI database software template](https://cloud.ibm.com/catalog/content/VSI-database) from the IBM Cloud catalog. Click Log in, in case you haven't log in to your IBM Cloud account.

Observe IBM Cloud is specified in Select your deployment target and Terraform Version 1.0.7 or Terraform Version 2.0.0 is displayed in Select a delivery method.
2. In the Configure your workspace section, enter Name for your Schematics Workspace, select your Resource group, and the Location where you want to create the workspace.
3. Check Override default Terraform version to configure the template to support your Terraform version.
Depending on the filter, the subsequent drop-down lists reflect the appropriate options.





 Option Description 

 length Filter according to whether the column content is shorter, longer, or equal to a specified number of characters or tokens. 
 text Filter according to whether the text matches or contains a regular expression or dictionary from the canvas, or you can enter a new regular expression. You can also set case sensitivity. 
 range Filter on the basis of specified criteria within a range of the specified column. For example, you might use starts with to narrow the results within matches so that you only get military rank matches that start with Chief. 





5. Once you complete your filter options, execute the extractor again to see how the results have changed.



Your Military Ranks extractor might produce a match for text Chief Warrant Officer John Doe, but you do not want to include results that have the word except preceding the match.



1. Create a dictionary with the term except and any other terms that you might want to use to exclude results.
2. Select Military Ranks from the canvas and open the Output tab under Extractor Properties.
3. Click New Filter.
4. Select Exclude and choose a column.
5. Select range and occurs after.
6. Select your dictionary that includes except.
7. Select the column and between0 to 2tokens.



This filter excludes any matches that have the word except within 0-2 tokens before a match.

![Sample filter](https://cloud.ibm.com/docs-content/v1/content/148d3bd95f946aa1bb53ea1540475f522e6b61c9/watson-knowledge-studio-data/images/samplefilter.png)









 Extracting in languages other than English 

To extract text from non-English documents, you can sometimes use or modify the provided extractors or define new extractors based on linguistic patterns. Also, if target terms are based on a pattern, you can design a sequence pattern.

You can use the workspace to build extractors to be used with any language, such as Spanish and French, that is based on tokens defined by white space and punctuation.
For more information about Java syntax, see [http://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html](http://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html).



* To select text that includes any ordinal number of one or more digits followed by th, st, nd, or rd (21st, 2nd, 3rd, and so forth), specify:

d+(st|nd|rd|th)
* To select text that includes US Social Security numbers formatted as 999-99-9999, specify:

d{3}-d{2}-d{4}
* To select all text that includes a Canadian postal code formatted as A9A-9A9, A9A 9A9 or A9A9A9 (for example, K1G 3K9, V5g-4X3, and x2H3m5), specify:

[a-zA-Z][a-zA-Z](-|)[0-9][0-9]
* To select all text that includes a United Kingdom postal code formatted as A9 9AA, A99 9AA, AA99 9AA, A9A 9AA, or a9d AA9A 9AA (for example, M1 1AE, B33 8TH, DN55 1PT, W1A 0AX, CR2 6XH, and SW1A 2AA), specify:

[A-Z]{1,2}[1-9]?[A-Z]?s[0-9]{2,}|GIR 0AA
* To select all text that includes number plates for private and commercial vehicles in India formatted as AA-99-AA-9999, AA-99-AAA-9999 and, for states where the 0 is optional, AA-9-AA-9999 with separators that may be hyphens, spaces or not present (for example, TN-86-AF-1199, WB 06 F 5971, and DL4CAF4943), specify:
WITH ( format = 'PARQUET')
AS
SELECT 
FROM <TABLE_NAME>;

CREATE TABLE IF NOT EXISTS
hive.default.branch
WITH ( format = 'PARQUET')
AS
SELECT 
FROM hive.gosales.branch;
2. To change the table format to Iceberg, run Create table as command:

CREATE TABLE IF NOT EXISTS
<TABLE_NAME>
WITH ( format = 'PARQUET')
AS
SELECT 
FROM <TABLE_NAME>;

CREATE TABLE IF NOT EXISTS
iceberg-beta.default.branch
WITH ( format = 'PARQUET')
AS
SELECT 
FROM hive.gosales.branch;

You can also include any additional SQL into the SELECT clause for any transformations or conversion business logic or sort the data for optimized access. You can also add column partitions for additional performance improvements.

Statistics are automatically generated as part of the ingest of the new table.
Property name:



16. If required, add another property name. If you do not require any more properties press Enter to finish adding the model.



By default, a LoopBack; project comes configured with an in-memory data source, suitable for development and testing. However, at some point you'll want to connect your models to a real data source such as database server. Complete the following steps to add a data source:



1. Enter the following command:



apic create --type datasource

The tool prompts you for the name of the data source.

? Enter the data-source name:



2. Enter a name for the data source. You can use any alphanumeric character, dashes, and underscores in a data source name. The tool prompts you to select the connector to use for the data source:



? Select the connector for myds: (Use arrow keys)
> In-memory db (supported by StrongLoop)
Email (supported by StrongLoop)
MySQL (supported by StrongLoop)
PostgreSQL (supported by StrongLoop)
Oracle (supported by StrongLoop)
Microsoft SQL (supported by StrongLoop)
MongoDB (supported by StrongLoop)
(Move up and down to reveal more choices)



3. Use the arrow keys to select the connector that you want to use, then press Enter. The tool then adds the data source to the project.
4. Enter your host, port, user, password, and database credentials. The tool prompts you to install the LoopBack connector.



Install loopback-connector-<connector>?



5. Type Yes. The tool installs the connector.



Note: If you selected the Oracle connector, you must set an environment variable in IBM Cloud for the Oracle app to start. To do this, complete the following steps:



1. In the IBM Cloud UI, select Compute.
2. In CF Applications, select the application that you want to work with.
3. Click Runtime.
4.