vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Before you get started, be sure that you have the required level of access to create and manage libraries. To manage libraries, you need the [Editor platform role or higher](https://cloud.ibm.com/docs/security-compliance?topic=security-compliance-access-management).





 Creating a library 

You can create a custom library by using the Security and Compliance Center UI.



1. In the IBM Cloud console, click the Menu icon ![Menu icon](https://cloud.ibm.com/docs-content/v1/content/a6478671e8f845a84102d9cf49bf28ee7f4f3227/icons/icon_hamburger.svg)> Security and Compliance to access the Security and Compliance Center.
2. In the navigation, click Controls > Control libraries.
3. Click Create. A side panel opens.
4. Provide a name, description, and version for your library.
5. Click Create. Your library opens in a new page but is empty.
6. To start adding controls, click Create.



1. Provide the details for your control by including a Name, Description, and Category.
2. Click Add to start adding existing specifications to your control.
3. Provide a Description for your specification.
4. Select a Component.
5. Select the existing assessments that can help to evaluate your specification, then click Create.

If you don't see an assessment that is specific to your use case, you can create a rule. When the rule is created, it shows in the assessment section.
6. When you are finished adding specifications to your control, click Create.
7. Repeat these steps until you add all your controls.









 Next steps 

After your library is added to the service, you can use it to create a profile. From your library details, click Actions > Create profile to start evaluating your resources.
Review the command that you want to create, update, list, delete and to work with your IBM Cloud Schematics inventory.



 ibmcloud schematics inventory create 

Create a resource inventory in IBM Cloud Schematics that you can reference in a Schematics action. A resource inventory includes all the target hosts where you want to run an Ansible playbook. You can create an inventory by using a payload file or the interactive mode.

Syntax

ibmcloud schematics inventory create --name INVENTORY_NAME [--description DESCRIPTION] [--location GEOGRAPHY] [--resource-group RESOURCE_GROUP] [--inventories-ini INVENTORY_INI_FILE] [--resource-query RESOURCE_QUERY_ID] [--file FILE_NAME ] [--output OUTPUT] [--no-prompt]

You need to pass either --inventories-ini file path or --resource-query ID for the inventory to use the target host details.

Command options



Schematics inventory create flags

 Flag Required / Optional Description 

 --name or -n Required The unique name of a resource inventory. 
 --description or -d Optional The short description of an inventory. 
 --location or -l Optional The location where you want to store your resource inventory, such as us-south, us-east, eu-de, or eu-gb. For more information about data storage in Schematics, see [Where is my data stored?](https://cloud.ibm.com/docs/schematics?topic=schematics-secure-datapi-location). 
 resource-group or -r Optional The name of the resource group where you want to create the action. 
 --inventories-ini or -y Optional The file path to the resource inventory file where you specified all target hosts. The resource inventory file must be provided in INI format. For more information about how to create a static resource inventory file, see [Creating static inventory files](https://cloud.ibm.com/docs/schematics?topic=schematics-inventories-setupstatic-inv). 
 --resource-query Optional Enter the ID of a resource query that you created.
ibmcloud schematics inventory create --name INVENTORY_NAME [--description DESCRIPTION] [--location GEOGRAPHY] [--resource-group RESOURCE_GROUP] [--inventories-ini INVENTORY_INI_FILE] [--resource-query RESOURCE_QUERY_ID] [--file FILE_NAME ] [--output OUTPUT] [--no-prompt]

You need to pass either --inventories-ini file path or --resource-query ID for the inventory to use the target host details.

Command options



Schematics inventory create flags

 Flag Required / Optional Description 

 --name or -n Required The unique name of a resource inventory. 
 --description or -d Optional The short description of an inventory. 
 --location or -l Optional The location where you want to store your resource inventory, such as us-south, us-east, eu-de, or eu-gb. For more information about data storage in Schematics, see [Where is my data stored?](https://cloud.ibm.com/docs/schematics?topic=schematics-secure-datapi-location). 
 resource-group or -r Optional The name of the resource group where you want to create the action. 
 --inventories-ini or -y Optional The file path to the resource inventory file where you specified all target hosts. The resource inventory file must be provided in INI format. For more information about how to create a static resource inventory file, see [Creating static inventory files](https://cloud.ibm.com/docs/schematics?topic=schematics-inventories-setupstatic-inv). 
 --resource-query Optional Enter the ID of a resource query that you created. A resource query helps to dynamically build your resource inventory by using the IBM Cloud resources that you created with a Schematics Workspace. To create a resource query, see the ibmcloud schematics resource-query create[command](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-create-rq).
Focus on creating an inventory of entity types and relation types that cover the information that is needed by the application in which the type system will be used. Do not cover things that are not needed. Do not split types or make distinctions that are not needed by the application. For example, if the source document contains the sentence Murder on the Orient Express made headlines, then how you define entity types to capture the key information in the sentence differs depending on the type of application that will use the model that you build with the type system.



* For a literary application, you might create types that capture this information:

[NOVEL] [CRITICAL_RECEPTION]

[Murder on the Orient Express] [made headlines]
* For a public safety application, you might create these types:

[EVENT_CRIME] [LOCATION]

[Murder] on the [Orient Express] made headlines



The structure is often driven by characteristics of the documents from which information will be extracted, but should always be tempered by what information the application will actually use from those documents. For example, you might be creating an application that gains insights from medical records. To build the type system, you start looking at patient records to see what kinds of information must be captured. The patient records might all contain a field that describes what the patient ate for lunch. However, if the application will not use that information, do not add it to the type system. And in making the decision to make this omission, recognize that this means that sections of your patient record documents will be left unannotated. That is alright and even expected.

Mentions annotate a span of text; they do not replace the text. A type system is not an ontology for a domain. Expect to have general entity types such as MEDICATION_NAME instead of having an entity type for each medicine type. The document text will continue to contain the specific medicine name. It will just be enhanced with an annotation that identifies its type, which makes the information easier to find and extract programmatically.

As you get started, define 10 to 40 entity types and relation types. Stay to the lower end of the range if the human annotators who will be working on the workspace are not highly specialized in the field. Do not define more than 50.
Log in to the web console as described in [Getting started with the web console](https://cloud.ibm.com/docs/netezza?topic=netezza-getstarted-console).
2. Create a temporal schema as described in [Creating schemas](https://cloud.ibm.com/docs/netezza?topic=netezza-create-schemas).







 Creating temporal databases with the command-line 

To create a temporal database, set retention time interval to a nonzero value.



1. Log in to the web console as described in [Getting started with the web console](https://cloud.ibm.com/docs/netezza?topic=netezza-getstarted-console).
2. Create a temporal database as described in [Creating databases](https://cloud.ibm.com/docs/netezza?topic=netezza-create-db).









 Altering time travel objects with the command-line 



 Altering temporal tables to nontemporal with the command-line 

To alter a temporal table to nontemporal, set DATA_VERSION_RETENTION_TIME to 0.

For detailed syntax and the necessary privileges, see [the ALTER TABLE command](https://www.ibm.com/docs/en/netezza?topic=npsscr-alter-table-2).

ALTER TABLE <table> DATA_VERSION_RETENTION_TIME 0;

Example:

ALTER TABLE PRODUCT DATA_VERSION_RETENTION_TIME 0;

When you set DATA_VERSION_RETENTION_TIME to 0, you cannot run time travel queries and you do not have access to historical rows for that table anymore. You can reclaim some or all of the now inaccessible historical rows in the table with GROOM TABLE.





 Altering nontemporal tables to temporal with the command-line 

To alter a nontemporal table to temporal, set DATA_VERSION_RETENTION_TIME] to a nonzero value.
1. Enter an Action name and an Action description for your action. Note that the name can be up to 128 characters long and can include alphanumeric characters, spaces, dashes, and underscores.
2. Optional, enter the Tags that you want to add to your action. Tags help in quick search operation of your action.
3. Select the Resource group where you want to create the action.
4. Select the Location where you want to create the action. The location determines where your action runs and action data are stored. You can choose a geography, such as North America, or a location, such as Frankfurt or London. If you select a geography, Schematics decides on a location within this geography based on availability. Be sure that you can store your action data in this location as you cannot change the location after the action is created. For more information, see [where is the information stored?](https://cloud.ibm.com/docs/schematics?topic=schematics-secure-datapi-location) Note that the location of your action is independent from the location of your IBM Cloud resource where you want to run your Ansible playbook.
5. Click Create. Your action is created with a Normal state, and you are directed to the Details section.



3. In the Ansible playbook section, click Edit icon to import your Ansible playbook.



1. Enter the GitHub or GitLab repository URL where your Ansible playbook is stored. The URL can point to the master branch, any other branch, or a subdirectory. If your repository stores multiple playbooks, you must select the playbook that you want to run. A Schematics action can point to one playbook at a time. To run multiple playbooks, you must create a separate action for each playbook.



* Example for master branch - https://github.com/myorg/myrepo
* Example for other branches - https://github.com/myorg/myrepo/tree/mybranch
For more information about variable store, see [Variable store parameter](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-update). 



2. Create the workspace by using the JSON file from command-line interface.

ibmcloud schematics workspace new --file <fully qualified path of cluster_payload.JSON file>

For more information about workspace creation, see [command-line commands and syntax](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-reference).

Sample example output

Creation Time Mon Feb 15 19:18:55
Description
Frozen false
ID mytest1_cluster-62183a6b-fbed-43
Name mytest1_cluster
Status DRAFT

Template Variables for: examples-d3d10ae5-76ef-47
Name Value
worker_pool_name workerpool
service_instance_name myservice
flavor cx2.2x4
cluster_name cluster
region us-south
worker_count 1
resource_group Default

OK
Show more

You can also view the new workspace mytest1_cluster in IBM Cloud dashboard.
3. Verify that your workspace is created by using list command.

ibmcloud schematics workspace list

Sample example output

Name ID Description Status Frozen
mytest1_cluster mytest1_cluster-62183a6b-fbed-43 ACTIVE False

OK







 Step 3: Planning and applying the Terraform template 

Create a Schematics execution plan. The execution plan shows the IBM Cloud resources that must be added, modified, or removed to achieve the state that is described in your Terraform template.

Your workspace must be in an Active state to perform a Schematics plan action.
Creating a database 

Your service instance of Planning Analytics with Watson does not come with a pre-created database. To create a database, complete the following steps:



1.  On the Planning Analytics Workspace home page, click Administration.
2.  Click the Databases tile.
3.  Click Add.
4.  A new dialog will appear. Specify the appropriate database configuration.



*  Each Database requires its own storage space.
*  Replicas enable High Availability. A value of 1 represents no High Availability and a value of 2 or more enables High Availability.
*  Overall RAM used is represented by the amount of Selected Memory (GB) multiplied by the number of Replicas. For example, 8 GB with 2 replicas means 16 GB in total will be in use.



5.  Once you have set up the configuration, click Create.
Select a blueprint template 

Use one of the [sample blueprints](https://github.com/Cloud-Schematics/blueprint-complex-inputs) to create a scalable blueprint cloud environment, created from multiple blueprint modules.

This sample blueprint is a simple scenario using two modules to demonstrate linking and resource data flows between the modules. No cloud resources are created by this example.





 Creating a blueprint in Schematics 

Create your blueprint by using the [ibmcloud schematics blueprint create](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-blueprint-create) command.

For Schematics Blueprints, the [Schematics plug-in](https://cloud.ibm.com/docs/schematics?topic=schematics-setup-cliinstall-schematics-plugin) version must be greater than the 1.12.5 version.



 Syntax 

The name of default resource group for the account is required for the blueprint creation. The blueprint config will be created in this resource group.

ibmcloud resource groups --default

Retrieving default resource group under account 12345678901234567890123456789012 as anon@anon.com...
OK
Name ID Default Group State
Default aac37f57b20142dba1a435c70aeb12df true ACTIVE

In the blueprint create command, replace DEFAULT-RESOURCE-GROUP-NAME with the name of the default resource group.
* Entity type names cannot be longer than 64 characters.
* Relation type names cannot be longer than 128 characters.



By convention, entity type names are specified in uppercase characters (ORGANIZATION) and relation type names are specified in camel case (employedBy). But, this convention is optional.





 Procedure 



1. Log in as a Knowledge Studio administrator or project manager and open the Assets> Entity Types page.
2. Choose one of the following methods to create a type system:



* Upload an existing type system:



1. On the Entity Types tab, click Upload.

If you previously downloaded a type system from a Knowledge Studio workspace, the type system is downloaded in JSON format. You can upload this type system to jump-start the creation of a workspace. For details, see [Uploading resources from another workspace](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-exportimport).

> Note: Regardless of the origin of the type system, the entries in it must meet the naming rules listed earlier.
2. Add, edit, and delete entity types and relation types the same way that you do when you create a type system from scratch.



* Create a type system from scratch:



1. On the Entity Types tab, click Add Entity Type.
2. Specify a descriptive entity type name, keeping in mind that the entity type is a label for annotating important spans of text (mentions) in your domain content. Keep the name short and representative, so human annotators can remember it easily.

You can also optionally define roles and subtypes for the entity type:



* A role helps qualify the entity type in the context in which the mention occurs. For example, the mention Software Engineer might be labeled with the entity type PEOPLE and, in this context, by the role OCCUPATION. See [When to define roles](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-typesystemwks_typesystem_roles) for more information.
create-if-not-exist Create target table if not existed ibm-lh data-copy --create-if-not-exist 



2. Configuration file option

The Configuration file contains a global ingest configuration section and multiple individual ingest configuration sections to perform the ingestion job. The specifications of the individual ingestion sections override the specifications of the global ingestion section.

a. Global ingest config section



Table 2. Global ingest config options and variables

 Parameter Description Declaration 

 target-tables Data migration target table. One target table can only be specified. <catalog>.<schema>.<table1> target-tables:<table_name> 
 ingestion-engine-endpoint Specifies connection parameters of the ingestion engine. Endpoint of ingestion engine. hostname=<hostname>,port=<port> ingestion-engine:hostname=<hostname>, port=<port> 
 create-if-not-exist Create target table if not existed `create-if-not-exist:<true 



b. Individual ingest config section

There can be multiple individual ingest sections in a configuration file option. Each individual ingest config sections will be ingested separately.



Table 3. Individual ingest config options and variables

 Parameter Description Declaration 

 source-files Data file(s) or folder(s) for data migration. File name ending with / is considered a folder source-files:<SOURCE_DATA_FILE> 
 source-s3-creds S3 credentials of data source source-s3-creds:<SOURCE_S3_CREDS> 
 staging-location Location where CSV files are converted to Parquet are staged. Even parquet files in some circumstances are staged like, Parquet file is local, Parquet files in local folder, a huge Parquet file(local or S3) all of which may not fit in memory, source file is S3 parquet file, and file from S3 parquet folder which may contain special column types such as date or decimal.
* SQL Server: mssql-jdbc-7.2.2.jre8.jar
* PostgreSQL: postgresql-42.2.6.jar



2. Compress the JAR files into a single compressed file.

If you have a JDBC driver that has only one JAR file, skip this step.
3. Make a note of where the driver is stored. You must specify the directory where you store this JAR or compressed file in the next procedure so that Discovery can upload it.







 Connecting to a database data source 

Before you begin, if you plan to apply enrichments to your data, create the collection in a Content Mining project type. If you are using a different project type and plan to apply enrichments, stop here. For more information, see [Applying enrichments to content from a database](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-connector-database-cp4dconnector-database-cp4d-enrich-db).

From your Discovery project, complete the following steps:



1. From the navigation pane, choose Manage collections.
2. Click New collection.
3. Click Database, and then click Next.
4. Name the collection.
5. If the language of the documents in the database is not English, select the appropriate language.

For a list of supported languages, see [Language support](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-language-support).
6. Optional: Change the synchronization schedule.

For more information, see [Crawl schedule options](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collectionscrawlschedule).
7. Complete the following fields in the Enter your credentials section:

Database URL
: The URL of the database server.

The following table shows example database URLs:



Example database URLs

 Database Syntax Example
Notice that your project is also selected for context, so all subsequent application-related commands are within the scope of this new sample project.





 Step 3: Creating a directory and source code 



1. Create a directory on your local workstation that is named myapp and navigate into it. In this directory, save all the files that are required to build the image and to run your app.

mkdir myapp && cd myapp
2. Create a file called server.js and copy the following source code into it.

const http = require('http');

http.createServer(function (request, response) {
response.writeHead(200, {'Content-Type': 'text/plain'});
response.end( "Hello worldn" );
}).listen(8080);

This example uses Node.js. You can substitute code from any [supported runtime](https://cloud.ibm.com/docs/codeengine?topic=codeengine-plan-buildbuild-buildpack-strat).







 Step 4: Deploying your application 

Push your code to Code Engine by using the application create command. You must provide a name for your application and the location of the source code. The following example creates an application called myapp that uses the buildpack strategy and provides the location for the source code in the current directory (.).

ibmcloud ce app create --name myapp --build-source . --strategy buildpacks

Example output

Creating application 'myapp'...
Packaging files to upload from source path '.'...
Submitting build run 'myapp-run-220999-210706331'...
Creating image 'private.us.icr.io/ce--6ef04-khxrbwa0lci/app-myapp:220418-0207-askql'...
Waiting for build run to complete...
Build run status: 'Running'
Build run completed successfully.
From the Event subscriptions page, click Create to create your subscription.
4. From the Create an event subscription page, complete the following steps.



1. For Event type, select the Cloud Object Storage tile. Click Next.
2. For General, provide a name for the Object Storage subscription, for example, mycos. You can optionally provide event attributes. Note that if the Object Storage event consumer is an application, event attributes are available as HTTP headers. If the event consumer is a job, event attributes are available as environment variables. Click Next to proceed.
3. For Bucket event details, select or type the name of an existing Object Storage bucket. Specify the types of changes for your object and optionally provide an object name prefix or suffix to filter objects in the bucket that to trigger events for the subscription. Click Next to proceed.
4. For Event consumer, specify the application to receive events. Notice that you can choose from a list of defined applications and jobs. For this example, use the myapp application that references the icr.io/codeengine/cos-listen image. If your app does not exist, you can provide the name of your application and [create your application](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-appdeploy-app-console) after you create the Object Storage subscription. For applications only, you can optionally specify a path. By default, events are routed to the root URL of the destination application. You can send events to a different destination within the app by specifying a path. For example, if your subscription path specifies /events, the events are sent to https://<base application URL>/events. Click Next to proceed.
5. For Summary, review the settings for your Object Storage event subscription and make changes if needed. When ready, click Create to create the Object Storage subscription.



5.