vania-janet/multiturn-rag-retrieval-data huggingface dataset:
The create external dictionary statement allows you to specify the dictionary content when the extractor is instantiated, instead of in source AQL code, and you do not have to recompile the module. Therefore, external dictionaries are powerful constructs that allow the AQL developer to expose customization points in a compiled module.

Dictionaries can be created from three sources:



* Dictionary files
* Inline dictionary declarations
* Tables that are created with the create table statement and the create external table statement.





 Syntax 

The internal create dictionary statement has three syntactical forms, from file, from table, and an inline format.



* Internal dictionary

From file:

create dictionary <dictionary name>

from file '<file name>'
[with language as '<language code(s)>']
[and case (exact | insensitive)]
[and lemma_match];

From table:

create dictionary <dictionary name>

from table <table name>
with entries from <column name>
[and language as '<language code(s)>']
[and case (exact | insensitive)]
[and lemma_match];

Inline format

create dictionary <dictionary name>
[with language as '<language code(s)>']
[and case (exact | insensitive)]
[and lemma_match]
as
(
'<entry 1>', '<entry 2>', ... , '<entry n>'
)
;
* External dictionary

create external dictionary <dictionary-name>
required [true|false]
[with language as '<language codes>']
[and case (exact | insensitive )]
[and lemma_match];







 Description 



* <dictionary name>

Specifies a name for the new internal or external dictionary. Can be a simple identifier or double-quoted identifier.
* '<file name>'

Specifies the name of the file that contains dictionary entries. Dictionary files are carriage-return-delimited text files with one dictionary entry per line.
1. From the IBM Cloud menu, select Schematics.
2. Click Create workspace.
3. On the Specify template page:



* Enter the [URL](https://github.com/IBM-Cloud/sap-s4hana-sz-ha/tree/main/schematics) for the github repository that contains the Schematics code for this offering.
* Select the Terraform version.
* Click Next.



4. On the Workspace details page:



* Enter a name for the workspace.
* Select a Resource group.
* Select a Location for your workspace. The workspace location does not have to match the resource location.
* Select Next.



5. Select Create to create your workspace.
6. On the workspace Settings page, in the Input variables section, review the default input variables and provide values that match your solution.

For a more detailed description of each of the parameters, check the GitHub repo readme file, chapter “Input parameter file”. Also, make sure to mark the parameters that contain sensitive information like passwords, API, and ssh private keys as "sensitive". These parameters are marked as “sensitive” in the readme file, under “Input parameter file”.

Save each parameter that you modify.
7. On the workspace Settings page, click Generate plan. Wait for the plan to complete.
8. Click View log to review the log files of your Terraform execution plan.
9. Apply your Terraform template by clicking Apply plan.
10. Review the log file to ensure that no errors occurred during the provisioning, modification, or deletion process.







 Next steps 

If you need to rename your resources after they are created, modify the input.auto.tfvars file to change the names and run terraform plan and terraform apply again. Do not use the IBM Cloud Dashboard and user interface to modify your VPC after it is created. The Terraform scripts create a complete solution and selectively modifying resources with the user interface might cause unexpected results.
create dictionary EnglishFrenchNames
from file 'en_fr_names.dict';

/
* Dictionary of Italian names. Language clause in the dictionary
* definition will override the module level dictionary matching setting.
/

create dictionary ItalianNames
with language as 'it'
as
(
'firstEntry','secondEntry'
);

/
* View to extract pattern: Phone, followed by one to three tokens,
* followed by Email. Language clause at the atom level will override
* the module level dictionary setting.
/

create view PhoneEmailPattern as
extract pattern <'Phone'[ with language as 'en']>
<Token> {1,3} 'Email'
as match
from Document D;

output view PhoneEmailPattern;
Show more







 The require document with columns statement 

By using the require document with columns statement, you can define the schema of the special view Document at compile time. This schema definition specifies the list of required fields and their types to be found in each tuple of the view Document.



 Syntax 

require document with columns <columnName> <columnType>
[and <columnName> <columnType>];





 Description 



* <columnName>

Specifies the name of the column to be used in the schema. The <columnName> is an attribute identifier, which is a simple identifier or a double-quoted identifier.
* <columnType>

Specifies the type of column to be used in the schema of the Document view. The <columnType> can be one of the following data types: Integer, Float, Boolean, or Text.
* [ and <columnName> <columnType> ]

Specifies additional column names and types to the schema of the Document view. They follow the same rules as <columnName> and <columnType>.



In earlier versions of AQL, the schema of the special view Document was predefined to consist of either a single field (text Text) or two fields (text text, label Text).
For more information about how to complete the tasks for creating a REST API with IBM Integration Bus, see [Developing integration solutions by using REST APIs](https://www.ibm.com/support/knowledgecenter/SSMKHH_10.0.0/com.ibm.etools.mft.doc/bi12016_.htm) in the IBM Knowledge Center.
2. Open the Create a REST API wizard by selecting File > New > REST API.
3. Enter a name for the REST API. The name that you specify is used as the name of the project in the IBM Integration Toolkit.
4. Select one of the following methods of creating the REST API, and complete the procedure:

From scratch by using the REST API Editor provided with the IBM Integration Toolkit:



1. Select Create a REST API and define resources and operations yourself.
2. Click Finish. The REST API Editor for the new REST API opens automatically, which you must use to define resources, models, and operations.



From an existing OpenAPI 2.0 document:



1. Select Import resources and operations defined in a OpenAPI document and click Next.
2. Select the path to the OpenAPI document that describes the resources and operations that you want in the REST API. You can import the OpenAPI document from the file system or from an existing project in the workspace. The file is validated for use in a REST API. If any errors are found while the selected OpenAPI document is validated, those validation errors are displayed at the beginning of the wizard. You cannot proceed if validation errors are found in the selected OpenAPI document.
3. Select Finish to create the API.



5. Implement the subflows for your REST API, which are the REST operations that you want to include.



2. Add the REST API project to a new or existing broker archive (BAR) file of the IBM Integration Bus.
3. Deploy the BAR file to IBM Integration Bus on Cloud.



1. Log in to IBM Integration Bus on Cloud by using your IBMid, if you are not already logged in.
Refer to the sections [understanding blueprint templates and configurations](https://cloud.ibm.com/docs/schematics?topic=schematics-blueprint-templates) and [blueprint template reference](https://cloud.ibm.com/docs/schematics?topic=schematics-bp-template-schema-yaml) for guidance to the blueprint syntax when creating or modifying a template.

Change in blueprint environments is explicitly managed through version control. Template and input documents are all sourced from a version control system such as GitHub, GitLab or IBM Catalog.





 Steps to define a blueprint 

The steps to create a blueprint template and define the versioned inputs are illustrated in the diagram.

Zoom

![Blueprint template and inputs](https://cloud.ibm.com/docs-content/v1/content/ba89e173287b28d1453d2a327d8a1f74ae2f1662/schematics//images/new/sc-bp-define.svg)

Blueprint template and inputs



1. Defining a cloud environment starts with a high-level reference architecture for the environment. Using the reference architecture as a guide, select the automation modules to implement the infrastructure layers of the architecture. Publicly available modules can be combined with private modules to create a custom architecture. Modules can be sourced from the public repositories containing IBM and third-party authored modules, along with any custom developed modules from private repositories. Refer to the section [using Terraform modules with blueprint templates](https://cloud.ibm.com/docs/schematics?topic=schematics-blueprint-terraform) for details of how to work with Terraform root and child modules.



* IBM authored modules can be found in the [Terraform IBM Modules](https://github.com/terraform-ibm-modules) GitHub repository.



2. Create and edit a blueprint template to implement the desired reference architecture from the selected modules.
See [Setting up the CLI](https://cloud.ibm.com/docs/schematics?topic=schematics-setup-cli) for more information.
5. Create your [Terraform template](https://cloud.ibm.com/docs/schematics?topic=schematics-create-tf-config).



1. Review the list of [supported images](https://cloud.ibm.com/docs/vpc?topic=vpc-about-images).
2. Create an instance of [IBM Cloud Object Storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-getting-started-cloud-object-storage) and add your image to a bucket.





To share software with other accounts, your software must be approved in Partner Center. For more information, see [Getting set up to sell software](https://cloud.ibm.com/docs/sell?topic=sell-sw-getting-started).





 Step 1: Create a private catalog 

Complete the following steps to add your software by using the CLI. You can use this task in a CI/CD process.

Create a private catalog. Private catalogs provide a way for you to manage access to products for users in your account. For more information, see the [cli documentation](https://cloud.ibm.com/docs/account?topic=cli-manage-catalogs-plugincreate-catalog) for creating a private catalog.

ibmcloud catalog create --name CATALOG [--catalog-description "DESCRIPTION"]





 Step 2: Add the Terraform template to your catalog 

Add software to your private catalog. For more information, see the [cli documentation](https://cloud.ibm.com/docs/cli?topic=cli-manage-catalogs-plugincreate-offering) for adding software to your private catalog.

ibmcloud catalog offering create --catalog "Name of catalog" --zipurl https://software.url.com.tgz
"spark.hadoop.javax.jdo.option.ConnectionURL": "jdbc:postgresql://<CHANGEME>.databases.appdomain.CHANGEME/ibmclouddb?sslmode=verify-ca&sslrootcert=/home/spark/shared/user-libs/certificate_library_set/custom/postgres.cert&socketTimeout=30",
"ae.spark.librarysets":"certificate_library_set"
5. Set up the Hive metastore schema in the Databases for PostgreSQL instance because there are no tables in the public schema of Databases for PostgreSQL database when you create the instance. This step executes the Hive schema related DDL so that metastore data can be stored in them. After running the following Spark application called postgres-create-schema.py, you will see the Hive metadata tables created against the "public" schema of the instance.

from pyspark.sql import SparkSession
import time
def init_spark():
  spark = SparkSession.builder.appName("postgres-create-schema").getOrCreate()
  sc = spark.sparkContext
  return spark,sc
def create_schema(spark,sc):
  tablesDF=spark.sql("SHOW TABLES")
  tablesDF.show()
  time.sleep(30)
def main():
  spark,sc = init_spark()
  create_schema(spark,sc)
if __name__ == '__main__':
  main()
6. Now run the following script called postgres-parquet-table-create.py to create a Parquet table with metadata from IBM Cloud Object Storage in the Databases for PostgreSQL database.

from pyspark.sql import SparkSession
import time
def init_spark():
  spark = SparkSession.builder.appName("postgres-create-parquet-table-test").getOrCreate()
migration-portal-setup
4. Click Create & Assess.
5. Fix any issue report by the EMP and make sure you are getting 100% coverage successfully







 Migrate your exported converted schema into your EnterpriseDB formation 

Export and deploy the converted schema from migration portal to your provisioned Databases for EnterpriseDB formation.



1. Select the Migrate to... button to begin the migration process.

Zoom

![Begin migration](https://cloud.ibm.com/docs-content/v1/content/e2b18e1112dc55facb736744a935365dce13849a/databases-for-enterprisedb/images/emp-migrate-to.png)

Figure 3. Begin migration
2. Choose the EDB Postgres Advanced Server on Cloud option to migrate your schemas on EDB Postgres Advanced Server on Cloud, and click "next".

Zoom

![Select Cloud destination](https://cloud.ibm.com/docs-content/v1/content/e2b18e1112dc55facb736744a935365dce13849a/databases-for-enterprisedb/images/emp-select-cloud.png)

Figure 4. Select Cloud destination
3. Select the schemas and click "next".

Zoom

![Select schemas](https://cloud.ibm.com/docs-content/v1/content/e2b18e1112dc55facb736744a935365dce13849a/databases-for-enterprisedb/images/emp-select-schemas.png)

Figure 5. Select schemas
4. Select the schemas and click "next".

Zoom

!
* Make sure that you have the required permissions to create an instance of [IBM® Key Protect](https://cloud.ibm.com/docs/key-protect?topic=key-protect-manage-access) and [IBM Cloud Object Storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-iam).







 Step 1: Creating your Schematics Workspace 

Use the IBM-provided Terraform template to create and configure your Schematics Workspace.



1. Review the [IBM-provided template](https://github.com/IBM-Cloud/terraform-provider-ibm/tree/master/examples/ibm-cluster/vpc-gen2-cluster) to create an IBM Cloud Kubernetes Service cluster on VPC infrastructure.



* main.tf: This file includes the Terraform code that you run in Schematics. Your Terraform code includes data sources and resources to create a VPC with subnets in two different zones, create an IBM Cloud Kubernetes Service cluster, and configure the cluster to bind an IBM Cloud Object Storage service instance.
* output.tf: This file includes the content that you want to return after Schematics applied your Terraform template. In this case, you get the file path on your local machine where the cluster configuration and certificates are stored. You use these files to access your cluster later.
* variables.tf: This file includes all the variables that you need to specify to run your Terraform template. You can use the default values that are provided, or override them when you create the Schematics Workspace.
* versions.tf: This file includes the Terraform version that this template requires.



2. Create a JSON file where you store the configuration of your Schematics Workspace.







 Step 2: Creating your IBM Cloud Schematics Workspace 



1. Specify your Schematics Workspaces setting by copying the following workspace JSON file and saving it as cluster_payload.json on your local machine.
To add an advanced rule model, complete the following steps:



1. Create the model and export the ZIP file that contains the model resources.

For more information about how to export the model, see the instructions for your model source:



* [Knowledge Studio for IBM Cloud Pak® for Data](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-create-advanced-rules-model)
* [Knowledge Studio for IBM Cloud](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-create-advanced-rules-model)
* [Open source NLP Editor](https://github.com/CODAIT/nlp-editor)



2. From the Teach domain concepts section of the Improvement tools panel, choose Advanced rules model.
3. Click Upload.
4. Specify a name for the model, and then choose the language that was used to define the model.
5. Specify a name for the result field, which is the field in the index where the output of this enrichment will be stored.
6. Click Upload to browse for the ZIP file that you exported earlier.
7. Click Create.
8. Choose the collection and field where you want to apply the enrichments from the model, and then click Apply.







 Output format for advanced rules 

Knowledge Studio uses the Annotation Query Language (AQL) to define the rules in an advanced rules model. Each model is defined by one or more views. Each view is a relational data structure that contains multiple data records. Each record is composed of values in columns that are defined by the view’s schema. To facilitate representing these models, which are custom and therefore have various schemas, a uniform JSON output schema is used.



* Each JSON object represents an Annotation Query Language (AQL) view.
* The name-and-value pairs in the JSON objects represent the names and values of the attributes in the view.
Make sure that the JSON file follows the structure as outlined in this command. To successfully create a workspace, ensure the CLI region is set to the region where you want to create or update workspaces and actions. The workspace name, location, resource group and git repo are all defined in the JSON input file. For more information about Schematics locations where your workspace data is stored, see [Where is my information stored?](https://cloud.ibm.com/docs/schematics?topic=schematics-secure-datapi-location).

Syntax

ibmcloud schematics workspace new --file FILE_NAME --state STATE_FILE_PATH [--agent-id AGENT_ID] [--github-token GITHUB_TOKEN] [--output OUTPUT] [--json]

Command options



Schematics Workspaces create flags

 Flag Required / Optional Description 

 --file or -f Optional The relative path to a JSON file on your local machine that is used to configure your workspace. For more information about the sample JSON file with the details, see [JSON file create template](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referencejson-file-create-template). 
 --state Optional The relative path to an existing Terraform state file on your local machine. To create the Terraform state file: <br>1. Show the content of an existing Terraform state file by using the [ibmcloud schematics state pull](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referencestate-pull) command. <br>2. Copy the content of the state file from your command-line output in to a file on your local machine that is named terraform.tfstate. <br>3. Use the relative path to the file in the --state command parameter. Note The Schematics Workspace supports the terraform.tfstate file size of less than 25 MB.
Log in to your [GitHub](https://github.com/) account.
3. Open the Terraform template to [create an IBM Cloud Object Storage](https://github.com/IBM-Cloud/terraform-provider-ibm/tree/master/examples/ibm-cos-bucket).
4. From the right side of the GitHub page, click Fork icon to create your own fork of the shared repository.

You need to copy the URL of the Terraform template of the GitHub or the GitLab Repository URL to create your Schematics Workspace.







 Step 2: Creating your IBM Cloud Schematics Workspace 

Complete these steps to create the IBM Cloud Schematics and the Terraform template URL.



1. From the [Schematics Workspaces dashboard](https://cloud.ibm.com/schematics/workspaces), click Create workspace.
2. In Specify template section. Enter GitHub, GitLab, or Bitbucket Repository URL as

https://github.com/IBM-Cloud/terraform-provider-ibm/blob/master/examples/ibm-cos-bucket
3. For the private repository, provide your GitHub personal access token. See the steps to fetch the [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token).
4. Select terraform_v01.0 from the Terraform version drop-down.
5. Click Next.
6. In the workspace details section, enter your Workspace name, Tags, Resource group, Location, and Description.

Ensure you provide the right resource group, and the location details where you want to create the workspace.
7. Click Next and then click Create to create Schematics Workspaces successfully.







 Step 3: Configuring variables 

Click ...
4. You can discover assets from connections to Db2 on Cloud, which gives you the ability to add all tables from the connection as data assets to a project. Select Discover data assets and choose a project.
5. Click Create. The connection appears on the Assets page.



Watch this video to see how to create a connection and add connected data to a project.





 To create a new connection in a legacy project: 



1. From your project's Assets page, click the Find and add data icon.
2. On the Connections pane, click Create Connection.
3. Enter a name and description and choose a Service Category:

Data Service = IBM Cloud data service

When you choose Data Service, your existing IBM Cloud data services appear in the Service Instance list.
4. Choose the service or database server from the list.
5. Enter the connection information:

When you choose Data Service, your existing IBM Cloud data services appear in the Service Instance list.
6. Click Create. The connection is available to all legacy projects.
7. On the Connections pane, select the connection and click Apply.



To add an existing connection to a legacy project:



1. From your project's Assets page, click the Find and add data icon.
2. On the Connections pane, select the connection and click Apply.









 SPSS Statistics 

These instructions explain how to create a connection from IBM® SPSS® Statistics to a Db2 on Cloud database.



 Prerequisites 

Before attempting to connect to your Db2 on Cloud database, verify that you have the [prerequisites](https://cloud.ibm.com/docs/Db2onCloud/connecting?topic=Db2onCloud-connect_ovprereqs).





 Procedure 



1. In SPSS Statistics, click File > Open Database > New Query.
2. In the Database Wizard, click Add ODBC data source.
3. Use the ODBC Data Source Administrator window to add an ODBC data source name (DSN) for the Db2 on Cloud database:

a.