vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* UI
* CLI
* API







 Generating a plan 

After you've created your workspace, you need to generate a plan to validate all of the configuration properties.



 Generating a plan using the UI 



1. In the IBM Cloud® console, after the workspace is created, you can review the properties and the variables that are associated with that workspace by using the Settings tab. You can review all the values, make changes if applicable, and click Generate plan.
2. When you click Generate plan, a new log is generated that can be viewed in the Jobs tab by clicking Jobs.
3. Review the log file for any errors, fix the properties, and regenerate the plan by clicking Generate plan again.







 Next steps 

After you've successfully generated a plan, you can begin [Applying a plan](https://cloud.ibm.com/docs/hpc-slurm?topic=hpc-slurm-applying-plan&interface=ui) to deploy your IBM Cloud resources to build a Slurm cluster instance.





 Generating a plan using the CLI 

Run the following command to generate the plan for your workspace.

ibmcloud schematics plan --id <WORKSPACE_ID>

You can view the log file to look for errors or confirm that the action was completed successfully. You might need to run this command multiple times to track the outcome of the command until it's completed.

ibmcloud schematics logs --id <WORKSPACE_ID>





 Next steps 

After you've successfully generated a plan, you can begin [Applying a plan](https://cloud.ibm.com/docs/hpc-slurm?topic=hpc-slurm-applying-plan&interface=cli) to deploy your IBM Cloud resources to build a Slurm cluster instance.





 Generating a plan using the API 



1. To generate a plan by using IBM Cloud Schematics Python APIs, create a Python file and provide a name of your choice, for example, schematics_generate_plan.py.
2.
schematics-workspace-prefix Prefix used for workspace creation when no workspace is specified for the deployment action. 
 schematics-workspace-resource-group Defaults to the resource group of the toolchain. Resource group to be used for the schematics workspace creation. 
 schematics-workspace-region Defaults to the region of the toolchain. Region to be used for the schematics workspace creation. 
 schematics-workspace-netrc Computed from known repositories. Value for the netrc configuration of the schematics workspace to be created. For more information, see [Supporting to download modules from private remote host](https://cloud.ibm.com/docs/schematics?topic=schematics-download-modules-pvt-git). 
 schematics-workspace-terraform-version terraform_v1.1 Terraform version used for the schematics workspace to be created. See [Overview of Schematics images and packaged Terraform providers](https://cloud.ibm.com/docs/schematics?topic=schematics-version-constraintsschematics-image-ov). 



To configure the scripts in the IaC CD deployment process, define the parameter that is scoped for a given inventory entry. To specify Schematics as deployment tool related environment properties for a given scope (inventory entry), prefix the property with the inventory entry name, for example: <inventory_entry>_. This prefix applies for the all the schematics-related environment entries.

Example:

hello-iac-sample_schematics-workspace-name : workspace-for-deployment-of-hello-iac-sample



 Configuration to use Terraform CLI as deployment tool 



Table 9. Configuration to use Terraform CLI as deployment tool

 Property Description 

 tf-backend-s3-bucket Bucket name that stores the state. 
 tf-backend-s3-key Name to use to persist the state. 
 tf-backend-s3-region Region of the Cloud Object Storage instance.
Go to the [SAZ GitHub repository](https://github.ibm.com/workload-eng-services/ha-saz).
2. Select Settings from your account icon menu.
3. Select Developer settings.
4. Select Personal Access Tokens.
5. Select Generate new token.
6. Copy the new token and save it in a text file.



After you have your access token, use Schematics to create your infrastructure.



1. From the IBM Cloud menu, select [Schematics](https://cloud.ibm.com/schematics/overview).
2. Click Workspaces.
3. Click Create workspace.
4. On the Specify template page:



* Enter the URL of high availability SAZ repo, [https://github.ibm.com/workload-eng-services/ha-saz](https://github.ibm.com/workload-eng-services/ha-saz).
* Enter your Personal Access Token.
* Select the Terraform version that is listed in the readme file.
* Click Next.



5. On the Workspace details page:



* Enter a name for the workspace.
* Select a Resource group.
* Select a Location for your workspace. The workspace location does not have to match the resource location.
* Select Next.



6. Select Create to create your workspace. The files from the GitHub repo are loaded in to Schematics. It can take several minutes for the files to load.
7. On the workspace Settings page, in the Input variables section, review the default input variables and provide values that match your solution. Some parameters have default values. Make sure to mark as “sensitive” the parameters that contain sensitive information like API keys. The values that you must specify include:



 Component type Parameter values needed 

 User-specific information <br><br> * API key<br> * SSH key name.
You can create zones and rules by using a RESTful [API](https://cloud.ibm.com/apidocs/context-based-restrictionsintroduction) or with [context-based restrictions UI](https://cloud.ibm.com/context-based-restrictions/overview). After you create or update a zone or a rule, it might take a few minutes for the change to take effect.

You need to allowlist Schematics in the CBR rules for other services if you require Schematics to connect to the service to perform post-configuration of service resources, for instance IBM Cloud® PostgreSQL Database. If you configure a CBR rule for a service, for instance IBM Cloud Kubernetes Service, Schematics cannot access a cluster in your account unless you include Schematics in the CBR allowlist.

CBR rules do not apply to resource and service provisioning requests, only to the post-configuration of provisioned services.

Currently, CBR is supported by Schematics public endpoints in both US and EU regions. CBR support with Schematics private endpoints is limited to EU region only.

If a CBR rule is blocking access to Schematics, the workspace page cannot list any workspaces. Other operations like workspace create, plan, and apply requests that are blocked by CBR will be failed as access denied errors.





 Understanding network zones 

By creating network zones, you can define an allowlist of network locations where Schematics access requests originate, to determine when a rule can be applied. The list of network locations can be specified by using IP addresses, such as individual addresses, ranges or subnets, and Virtual Private Cloud (VPC) IDs, or IBM Cloud® services.

After you create a network zone, you can add it to a rule.



 Creating network zones by using the CBR API 

The CBR API supports defining [network zones](https://cloud.ibm.com/apidocs/context-based-restrictionsintroduction).

Use GET /v1/zones to list the zones. By using POST /v1/zones, you can create a new zone with the appropriate information.
schematic_obj = HPCCEnvironmentValues(api_key)

    workspace_response = schematic_obj.schematics_service.get_workspace(w_id="<workspace id>").get_result()
    schematic_obj.update_variables(w_id="<workspace id>", 
        t_id=workspace_response["template_data"]["id"], 
        variablestore=config_data["template_data"]["variablestore"]
    )

logging.info("Schematic Variable Update Completed")
Show more





 Example Python request for schematics_env_class.py file 

import json
import logging

from ibm_cloud_sdk_core.authenticators import IAMAuthenticator
from ibm_schematics import SchematicsV1

class HPCCEnvironmentValues:

    def __init__(self, api_key):
        """
        create schematic services
        """
        self.authenticator = IAMAuthenticator(api_key)
        self.schematics_service = SchematicsV1(authenticator=self.authenticator)

        self.schematics_service.set_service_url('https://us.schematics.cloud.ibm.com')
        logging.info("Schematic Services created")

    def update_variables(self, w_id, t_id, variablestore):

        print("Function Schematic Update Variables Started")
        variables_update_result = self.schematics_service.replace_workspace_inputs(w_id=w_id, t_id=t_id, variablestore=variablestore
    ).get_result()

        return variables_update_result   
Show more





 config.json template file
* UI
* CLI
* API







 Generating a plan 

After you've created your workspace, you need to generate a plan to validate all of the configuration properties.



 Generating a plan using the UI 



1. In the IBM Cloud® console, after the workspace is created, you can review the properties and the variables that are associated with that workspace by using the Settings tab. You can review all the values, make changes if applicable, and click Generate plan.
2. When you click Generate plan, a new log is generated that can be viewed in the Jobs tab by clicking Jobs.
3. Review the log file for any errors, fix the properties, and regenerate the plan by clicking Generate plan again.







 Next steps 

After you've successfully generated a plan, you can begin [Applying a plan](https://cloud.ibm.com/docs/ibm-spectrum-lsf?topic=ibm-spectrum-lsf-applying-plan&interface=ui) to deploy your IBM Cloud resources to build a Spectrum LSF cluster instance.





 Generating a plan using the CLI 

Run the following command to generate the plan for your workspace.

ibmcloud schematics plan --id <WORKSPACE_ID>

You can view the log file to look for errors or confirm that the action was completed successfully. You might need to run this command multiple times to track the outcome of the command until it's completed.

ibmcloud schematics logs --id <WORKSPACE_ID>





 Next steps 

After you've successfully generated a plan, you can begin [Applying a plan](https://cloud.ibm.com/docs/ibm-spectrum-lsf?topic=ibm-spectrum-lsf-applying-plan&interface=cli) to deploy your IBM Cloud resources to build a Spectrum LSF cluster instance.





 Generating a plan using the API 



1. To generate a plan by using IBM Cloud Schematics Python APIs, create a Python file and provide a name of your choice, for example, schematics_generate_plan.py.
2.
Remote database where the table is present:

Hostname : <remote_host_name>
DB2 Version: v11.5
Service/Port Number: <port>
User: <remote_user>
Password: <remote_password>
Database Name: BLUDB
Schema Name: TESTDB
Table Name: TEST1





 Create a wrapper 

Create a wrapper by running the following command from the console of the target database instance:

create wrapper drdawrapper library 'libdb2drda.so' options(DB2_FENCED 'Y')

DB20000I The SQL command completed successfully.





 Create the server for a remote database 

Create a server to host the remote database by running the following command from the console of the target database instance:

create server fed_server type DB2/UDB version 11 wrapper drdawrapper authorization "<remote_user>" PASSWORD "<remote_password>" options(host '<remote_host_name>', port '<port>', dbname 'bludb', security 'SSL', password 'y')

DB20000I The SQL command completed successfully.





 Create the user mapping 

Create the user mapping by running the following command from the console of the target database instance:

create user mapping for user server fed_server OPTIONS (remote_authid '<remote_user>', remote_password '<remote_password>')

DB20000I The SQL command completed successfully.





 Create a nickname 

Create a nickname for the remote database and test the nickname by running the following commands from the console of the target database instance:

create nickname rmttest1 FOR fed_server.testdb.test1

DB20000I The SQL command completed successfully.

select * from rmttest1

C1 C2
----------- -----------
13 32

1 record(s) selected.
* Make sure that you have the [permissions](https://cloud.ibm.com/docs/schematics?topic=schematics-access) to create a workspace.



Ensure the location and the url endpoint are pointing to the same region when you create or update the Schematics Workspaces and actions. For more information about location and endpoint, see [Where is your information stored](https://cloud.ibm.com/docs/schematics?topic=schematics-secure-datapi-location)?





 Deleting the workspace through UI 



1. Log in to [IBM Cloud](https://cloud.ibm.com/).
2. Access Schematics > Workspaces > [Delete workspace](https://cloud.ibm.com/schematics/workspaces).



* In workspace List section:



* Click required Workspace to delete. If you do not see the required workspace in the list, check your navigation page.
* Click Next.



* In Workspace list section. Click workspace name and click delete icon. Enter a name of your workspace name for confirmation before delete. Enter the name while creation and click delete.
* Click Delete. Your workspace can be deleted with a Draft, Inactive, Active state.







 Verifying workspace delete 



1. Click your workspace that is listed in the [IBM Cloud console](https://cloud.ibm.com/schematics/workspaces) to view the results of the destroyed operation.
2. Click Workspaces tab to see the workspace list.
3. Type Workspace name in the search box to get confirmation about your workspace is deleted.









 Deleting the workspace through CLI 



1. Delete workspace configuration.
You can use the default values that are provided, or override them when you create the Schematics Workspace.
* versions.tf: This file includes the Terraform version that this template requires.



2. Create a JSON file where you store the configuration of your Schematics Workspace.







 Step 2: Creating your IBM Cloud Schematics Workspace 



1. Specify your Schematics Workspaces setting by copying the following workspace JSON file and saving it as cluster_payload.json on your local machine. For more information about the payload parameters, see [Schematics Workspaces new](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-new) command.

Example of the cluster_payload.json:

{
"name": "mytest1_cluster",
"type": [
"terraform_v1.0"
],
"description": "",
"template_repo": {
"url":"https://github.com/IBM-Cloud/terraform-provider-ibm/tree/master/examples/ibm-cluster/vpc-gen2-cluster"
},
"template_data": [
{
"folder": ".",
"type": "terraform_v1.0",
"variablestore":
{
"name": "worker_pool_name",
"value": "workerpool",
"type": "string"
},
{
"name": "service_instance_name",
"value": "myservice",
"type": "string"
},
{
"name": "flavor",
"value": "cx2.2x4",
"type": "string"
},
{
"name": "cluster_name",
"value": "cluster",
* UI
* CLI
* API







 Generating a plan 

After you've created your workspace, you need to generate a plan to validate all of the configuration properties.



 Generating a plan using the UI 



1. In the IBM Cloud® console, after the workspace is created, you can review the properties and the variables that are associated with that workspace by using the Settings tab. You can review all the values, make changes if applicable, and click Generate plan.
2. When you click Generate plan, a new log is generated that can be viewed in the Jobs tab by clicking Jobs.
3. Review the log file for any errors, fix the properties, and regenerate the plan by clicking Generate plan again.







 Next steps 

After you've successfully generated a plan, you can begin [Applying a plan](https://cloud.ibm.com/docs/hpc-spectrum-scale?topic=hpc-spectrum-scale-applying-plan&interface=ui) to deploy your IBM Cloud resources to build a Spectrum Scale cluster instance.





 Generating a plan using the CLI 

Run the following command to generate the plan for your workspace.

ibmcloud schematics plan --id <WORKSPACE_ID>

You can view the log file to look for errors or confirm that the action was completed successfully. You might need to run this command multiple times to track the outcome of the command until it's completed.

ibmcloud schematics logs --id <WORKSPACE_ID>





 Next steps 

After you've successfully generated a plan, you can begin [Applying a plan](https://cloud.ibm.com/docs/hpc-spectrum-scale?topic=hpc-spectrum-scale-applying-plan&interface=cli) to deploy your IBM Cloud resources to build a Spectrum Scale cluster instance.





 Generating a plan using the API 



1. To generate a plan by using IBM Cloud Schematics Python APIs, create a Python file and provide a name of your choice, for example, schematics_generate_plan.py.
2.
For a complete list of an agent create options, see [ibmcloud schematics agent create](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-reference&interface=uischematics-agent-create) command.

To deploy a Schematics agent, the IBM Cloud CLI [Schematics plug-in](https://cloud.ibm.com/docs/schematics?topic=schematics-setup-cliinstall-schematics-plugin) version must be 1.12.9 or higher.

Before starting the deployment the cluster, COS instance and bucket must have been created.

Select the IBM Cloud region where you wish to define and manage your agent from. Set the CLI region by running [ibmcloud target -r <region>](https://cloud.ibm.com/docs/cli?topic=cli-ibmcloud_cliibmcloud_target) command. This must be the same region as the location specified on the agent create command.

Example agent create syntax. The text between <> should be replaced with your values:

ibmcloud schematics agent create --name <agent-testing-prod-cli-mar-27-5> --location <eu-de> --agent-location <Frankfurt MZR> --version <1.0.0> --infra-type <ibm_kubernetes> --cluster-id <cb1c2dus01uf9mc0hkbg> --cluster-resource-group <job-runner> --cos-instance-name <COSForAgentLogging> --cos-bucket <agentlogs> --cos-location <eu-de> --resource-group <Default>

Output

Creating agent...
OK

ID agent-testing-prod-cli-mar-27-5.deA.dc97
Name agent-testing-prod-cli-mar-27-5
Status ACTIVE
Version 1.0.0
Location eu-de
Agent Location Frankfurt MZR
* Click Continue.



You created a trusted profile for a compute resource (Cluster running agent micro-services in schematics-job-runner namespace in your default service account.
3. Assign access to the trusted profile.



* Click the Access policy.
* Select Schematics service.
* Click Next.
* Select All resources.
* Click Next.
* Select Operator role.
* Click Add. The trusted profile is provided Operator access in Schematics services to allow Agents to fetch jobs to process.
* Select All Identity and Access enabled services.



* Click Next.
* Select Specific Resources option. Enter the name of the resource group where your Agents are registered.
* Click Next and select Viewer role.
* Click Next and select Reader role.
* Click Add and select Assign in the right navigation bar. The trusted profile provides Reader, and Viewer access for the resource group that allow Agents to read an Agent registration detail.



* Click Create.



4. View trusted profile ID.



* From your Trusted profiles page, click Details tab.
* Record the Profile ID, for example, Profile-1bd5eala-000-4a6666-00011 to override the input variable profile_id of Agents service.







 Next Step 

Refer to [Deploying the Agent services settings](https://cloud.ibm.com/docs/schematics?topic=schematics-agents-setupagents-setup-svc) to override the profile_id input variable.
Ingest a single CSV/Parquet file from S3 location by using config file 

To ingest a single Parquet file from a S3 location, update the following configuration file:

[global-ingest-config]
target-tables:<TARGET_TABLES>
ingestion-engine:hostname=<hostname>,port=<port>
create-if-not-exist:<true>
[ingest-config1]
source-files:<SOURCE_DATA_FILE>
source-s3-creds:AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ASSESS_KEY>,AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>,AWS_REGION=<YOUR_REGION>, BUCKET_NAME=<YOUR_BUCKET>, ENDPOINT_URL=<YOUR_ENDPOINT_URL>
staging-location:<STAGING_LOCATION>
staging-s3-creds:AWS_SECRET_ACCESS_KEY=<YOUR_TARGET_SECRET_ASSESS_KEY>,AWS_ACCESS_KEY_ID=<YOUR_TARGET_ACCESS_KEY_ID>,AWS_REGION=<YOUR_TARGET_REGION>, BUCKET_NAME=<YOUR_TARGET_BUCKET>, ENDPOINT_URL=<YOUR_TARGET_ENDPOINT_URL>

For example:

[global-ingest-config]
target-tables:iceberg_cat.ice_schema.customer1_tab
ingestion-engine:hostname=localhost,port=8080
create-if-not-exist:true

[ingest-config1]
source-files:s3://cust-bucket/warehouse/a_source_file.parquet
source-s3-creds:AWS_ACCESS_KEY_ID=xxxxxx,AWS_SECRET_ACCESS_KEY=yyyyyy,AWS_REGION=us-east-1,BUCKET_NAME=cust-bucket
staging-location:s3://cust-bucket/warehouse/staging/
Working with Schematics Actions 

Schematics Actions delivers Ansible-as-a-Service capabilities for you to automate configuration and management of your IBM Cloud environment, and deploy complex multitiered apps to your cloud infrastructure.

Review the section [Schematics Actions](https://cloud.ibm.com/docs/schematics?topic=schematics-sc-actions) for the background to Actions and Red Hat Ansible.



 Creating and running a Schematics action 

Create a Schematics action and specify the Ansible playbook that you want to run against your IBM Cloud resources.



 Prerequisites 



* Create an Ansible playbook and store the playbook in a GitHub or GitLab repository. Or you can try to use one of the [IBM-provided Ansible playbooks](https://github.com/Cloud-Schematics?q=topic%3Aansible-playbook).
* Make sure that you have the [right permissions](https://cloud.ibm.com/docs/schematics?topic=schematics-access) to create the Schematics action.



Ensure the location and the url endpoint are pointing to the same region when you create or update the Schematics Workspaces and Actions. For more information about location and endpoint, see [Where is the information stored?](https://cloud.ibm.com/docs/schematics?topic=schematics-secure-datapi-location)





 To create an action 



1. From the [Schematics Actions dashboard](https://cloud.ibm.com/schematics/actions), click Create action.
2. Configure your action.



1. Enter an Action name and an Action description for your action. Note that the name can be up to 128 characters long and can include alphanumeric characters, spaces, dashes, and underscores.
2. Optional, enter the Tags that you want to add to your action. Tags help in quick search operation of your action.
3. Select the Resource group where you want to create the action.
4.
* Workspace name as schematics-agent-service.
* Tags as my-tags. Optional: Enter tags for your workspace. You can use the tags later to find your workspace faster.
* Resource group as default or other resource group for this workspace.
* Location as North America or other [region](https://cloud.ibm.com/docs/schematics?topic=schematics-multi-region-deployment) for this workspace. Decide where you want to create your workspace? The location determines where your Schematics jobs run?, and where your workspace data is stored? You can choose between a location, such as North America, or a metro city, such as Frankfurt or London. If you select a location, Schematics determines the location based on availability. If you select a metro city, your workspace is created in this location. For more information about where your data is stored, see [Where is your information stored](https://cloud.ibm.com/docs/schematics?topic=schematics-secure-datapi-location)? The location that you choose is independent from the region or regions where you want to provision your IBM Cloud resources. The console does not support all available locations. To create the workspace in a different location, use the [CLI](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-new) or [API](https://cloud.ibm.com/apidocs/schematics/schematicscreate-a-workspace) instead.
* Optional, enter a descriptive name for your workspace.
* Click Next.



* Click Update. Your workspace is Updated with a Draft state and the workspace Settings page opens.







 Verifying workspace update 



1. Click your workspace that is listed in the [IBM Cloud console](https://cloud.ibm.com/schematics/workspaces) to view the results of the workspace details.
2. Click Jobs tab to see the workspace logs.
3.