vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Make sure to update the required parameters, such as api_key, ssh_key_name, ibm_customer_number, and zone in the config.json file.
6. Run the Python script by using python3 to update the variables in the Schematics workspace in the IBM Cloud.



Note: The following parameters might not be required in config.json as the Schematics update variables API uses the workspace ID w_id to update the variables against that workspace.

{
  "name": "Schematic Dev Workspace",
  "type": [
    "terraform_v0.14"
  ],
  "location": "us-south",
  "description": "Schematic Dev Workspace",
  "tags": [],
  "template_repo": {
    "url": "<GitHub repo URL>",
    "githubtoken": "<github-token>"
  }



 Example Python request for schematics_variables_update.py file 

The following Python example request is for the example file, schematics_variables_update.py.

import logging, os, json

logging.basicConfig()
logging.root.setLevel(logging.NOTSET)
logging.basicConfig(level=logging.NOTSET)

from schematics_env_class import HPCCEnvironmentValues

logging.info("Schematic Variable Update Started")

if __name__ == '__main__':

    json_files = os.path.join(os.path.abspath("."), "config.json")

    with open(json_files, "r") as file:
        config_data = json.load(file)

    api_key = config_data["template_data"]["variablestore"]["value"]

    schematic_obj = HPCCEnvironmentValues(api_key)
About IBM COS SDKs 

IBM Cloud® Object Storage provides SDKs for Java, Python, NodeJS, and Go featuring capabilities to make the most of IBM Cloud Object Storage.

These SDKs are based on the official AWS S3 API SDKs, but are modified to use IBM Cloud features like IAM, Key Protect, Immutable Object Storage, and others.



Table 1. Features supported per SDK

 Feature Java Python NodeJS GO CLI Terraform 

 [IAM API key support](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-sdk-aboutsdk-about-iam) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) 
 [Managed multipart uploads](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-sdk-aboutsdk-about-tmup) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) !
watsonx.data docs 

Welcome to the watsonx.data product documentation.

  Developer tools 

[API & SDK reference](https://cloud.ibm.com/apidocs/watsonxdata)

  Recommended content 

[Getting started A brief tutorial to introduce using watsonx.data.](https://cloud.ibm.com/docs/watsonxdata?topic=watsonxdata-getting-started)[FAQs Answers to frequently asked questions about watsonx.data.](https://cloud.ibm.com/docs/watsonxdata?topic=watsonxdata-faqs)

  Learn more 

[IBM Developer<br><br>Visit IBM Developer for technical articles, code patterns, tutorials, and more.<br><br>![112-Developer-tools.svg](https://cloud.ibm.com/media/docs/images/homepage/112-Developer-tools.svg)](https://developer.ibm.com/depmodels/cloud/)[Architecture Center<br><br>Discover the architecture references available for this product.<br><br>![190-Application-Platform.svg](https://cloud.ibm.com/media/docs/images/homepage/190-Application-Platform.svg)](https://www.ibm.com/cloud/architecture)[IBM Training<br><br>Build your skills with IBM Cloud training.<br><br>![83-Elements-of-the-cloud.svg](https://cloud.ibm.com/media/docs/images/homepage/83-Elements-of-the-cloud.svg)](https://www.ibm.com/training/cloud)[Community<br><br>Join the IBM Cloud community and collaborate with others.<br><br>![35-Connectivity.svg](https://cloud.ibm.com/media/docs/images/homepage/35-Connectivity.svg)](https://community.ibm.com/community/user/publiccloud/home)
refresh_token='<refresh-token>'
).get_result()

print(json.dumps(workspace_activity_plan_result, indent=2))

logging.info("Completed Generating Schematic Plan")





 Example Python response 

INFO:root:Started Generating Schematic Plan
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): iam.cloud.ibm.com:443
DEBUG:urllib3.connectionpool:https://iam.cloud.ibm.com:443 "POST /identity/token HTTP/1.1" 200 985
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): us.schematics.cloud.ibm.com:443
DEBUG:urllib3.connectionpool:https://us.schematics.cloud.ibm.com:443 "POST /v1/workspaces/us-south.workspace.Schematic-Sunil-Test-Workspace.5a4cbf11/plan HTTP/1.1" 202 49
{
  "activityid": "8c2ce5f031d23a60d26690d3cb398bb1"
}
INFO:root:Completed Generating Schematic Plan







 Next steps 

After you've successfully generated a plan, you can begin [Applying a plan](https://cloud.ibm.com/docs/hpc-openshift?topic=hpc-openshift-applying-plan&interface=api) to deploy your IBM Cloud resources to build a Red Hat OpenShift cluster instance.
Status: Ready

Resources:
CPU: 0.25
Memory: 500M
Max Execution Time: 60 seconds

Build Information:
Build Run Name: myfun-local-run-230123-111011111
Build Type: local
Build Strategy: codebundle-nodejs-18
Timeout: 600
Source: main.js

Build Run Summary: Succeeded
Build Run Status: Succeeded
Build Run Reason: All Steps have completed executing
Run 'ibmcloud ce buildrun get -n myfun-local-run-230123-111011111' for details.

Function Code:
Runtime: nodejs-18 (managed)
Bundle Secret: ce-auto-icr-us-south
Code Bundle: cr://icr.io/ce--abcde-glxo4kabcde/function-myfun-local:230123-1650-yrj86
Main: main()







 Including dependencies for your Function 

You can create Functions in many different programming languages. When your Function code grows complex, you can add code modules as dependencies for your Function. Each language has its own modules to use with your Function code. For example, Node.js dependencies are usually existing npm modules, whereas Python uses Python packages. These dependencies must be declared and created in a file with your source code



 Including modules for a Node.js Function 

Create a function that includes a dependency for a specific Python module by creating a package.json file. In this case, both the source code and package file are located in the same folder.



1. Create your source code by writing your code into a main.js file. For example, copy the following code example into a file called main.js.

function main(args) {
const oneLinerJoke = require('one-liner-joke');
let getRandomJoke = oneLinerJoke.getRandomJoke();

return {
headers: { 'Content-Type': 'text/plain;charset=utf-8' },
* [Example: Calling an external API with JavaScript](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js_api)
* [Packaging JavaScript code with the webpack module](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js_pkg)
* [Packaging JavaScript code as NPM files](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js_npm)
* [NPM libraries with native dependencies](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js_npm_dependencies)
* [Using ES6 module in your action](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_es6-module)



* [How do I package my Python app for deployment in Cloud Functions](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prephow_to_package_python)



* [Structuring Python code](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_struct)
* [Packaging multiple Python files into an archive](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_multiple_zip)
* [Packaging Python code with a local virtual environment in a compressed file](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_local_virt_zip)
* [Packaging Python code with a Docker virtual environment in a compressed file](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_docker_virt_zip)
Retrieving action logs with Schematics API 



1. To retrieve the logs for an action, such as generate a plan, apply a plan, and destroy resources using IBM Cloud® Schematics Python APIs, create a Python file and provide a name of your choice, for example, schematics_logs_api.py.
2. Copy and paste the [Retrieve Schematics workspace action details using Python API](https://cloud.ibm.com/docs/hpc-slurm?topic=hpc-slurm-retrieve-action-logs&interface=apiexample-request-retrieve-action-logs) example request into your Python file.
3. Change the following parameters as part of the request:



* Replace the authenticator = IAMAuthenticator('<ibm-api-key>') variable with your IBM Cloud API key.
* Change the API endpoint to the endpoint mentioned in this link according to the location where you want your Schematics workspace to reside, for example, schematics_service.set_service_url('https://us.schematics.cloud.ibm.com').



4. Inside the schematics_service.get_workspace_activity function, provide the following parameters:



* Provide the workspace ID that you generated in the [Creating a workspace](https://cloud.ibm.com/docs/hpc-slurm?topic=hpc-slurm-creating-workspace&interface=apiexample-request-create) task, for example, us-south.workspace.Terraform-Schematics-Python-Workspace.b3bbc9f5.
* Provide an activity ID that was generated as a successful response for the specific action for which you want to retrieve logs. You received an activity ID when you successfully completed the action in the Schematics workspace.



5. Run the Python script by using python3 <python-file-name> to retrieve the logs for an action for your Schematics workspace in the IBM Cloud.
6.
You can create a compressed file that includes your Python files and deploy the compressed file when you create your action.

Example command

ibmcloud fn action create <action_name> <compressed_python_files.zip> --kind python:3.11

For more information, see [Packaging multiple Python files into a compressed file](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_pkg).





 Packaging Python code with a local virtual environment in a compressed file 

When to use this method

If your app requires dependencies that are not included with the base Cloud Functions [Python runtime](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-runtimesopenwhisk_ref_python_environments), you can install those dependencies into a virtualenv folder and then compress into a compressed file to deploy in Cloud Functions. Your compressed file must be smaller than the maximum codeSize as described in the [Action Limits](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-limitslimits_actions).

Example command

ibmcloud fn action create <action_name> <compressed_python_virtualenv.zip> --kind python:3.11

For more information, see [Packaging Python code with a local virtual environment in a compressed file](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_local_virtenv).





 Packaging Python code with a Docker virtual environment in a compressed file 

When to use this method

If your app requires dependencies that are not included with the base Cloud Functions [Python runtime](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-runtimesopenwhisk_ref_python_environments), you can install those dependencies into a virtualenv folder by using the Python environment inside the Cloud Functions Python runtime image. You can then compress the folder into a compressed file to deploy in Cloud Functions.
(https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/tut-sdu-drag-file.png)

Figure 4. File upload dialog
5. Click Finish.

You add only one file. In a real scenario, you might upload multiple files with information about the same topic. By adding more files, you can expand the breadth of the information that your search application can leverage.



The service uploads the document. As it uploads the document, Discovery crawls the data and indexes key information. Because you created a Document Retrieval project type, Discovery makes a note of the Parts of speech and Entities information that it finds and recognizes as it crawls the document.





 Step 3: Review the document 

Analyzing and indexing the document can take a few minutes. While the processing is under way, review the source document to get a feel for its content. It is a good idea to understand the structure of your own documents before you use the Smart Document Understanding tool to annotate them.

Smart Document Understanding (SDU) uses visual imaging technologies to understand the structure of a document by analyzing the format and positioning of the text. You label sections of the document, such as subtitles or tables, to teach Discovery to recognize the sections. You can also label sections that you want the search function to ignore. For example, you might not want to search page footers or the table of contents information. After you teach the SDU tool to recognize footers, for example, you can exclude the footer field from the index.



1. Monitor the progress of collection processing by opening the Activity tab.

Click Manage collections from the navigation panel.

Zoom

![Shows the Manage collections menu option from the navigation panel.]
Uploading or downloading all actions 

You can upload or download all your actions as a JSON file.



 Downloading 

To back up all your actions, download a JSON file and store it.



1. On the Actions page, click Global settings![Gear icon](https://cloud.ibm.com/docs-content/v1/content/icons/settings.svg).
2. On the Upload/Download tab, click the Download button.







 Uploading 

To reinstate a backup copy of actions that you exported from another service instance or environment, import the JSON file of the actions you exported.

If the Watson Assistant service changes between the time you export the actions and import it, due to functional updates that are regularly applied to instances in cloud-hosted continuous delivery environments, your imported actions might function differently than before.



1. On the Actions page, click Global settings![Gear icon](https://cloud.ibm.com/docs-content/v1/content/icons/settings.svg).
2. On the Upload/Download tab, drag and drop a JSON file onto the tab or click to select a file from your local system, then click Upload.



The imported JSON file must use UTF-8 encoding, without byte order mark (BOM) encoding. The JSON file cannot contain tabs, newlines, or carriage returns.





 Uploading intents as actions 

If you created intents in the classic Watson Assistant experience, you can migrate your intents to actions in the new Watson Assistant experience. When you upload intents, each intent is created as a new action. All phrases corresponding to an intent are created as example phrases for the new action. This can provide a helpful starting point when you are ready to start building actions in the new experience.



1. Download the intents that you want to migrate to actions from the classic Watson Assistant experience. For more information, see [Downloading intents](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-migrate-intents-entitiesmigrate-intents-download).
About IBM COS SDKs 

IBM Cloud® Object Storage provides SDKs for Java, Python, NodeJS, and Go featuring capabilities to make the most of IBM Cloud Object Storage.

These SDKs are based on the official AWS S3 API SDKs, but are modified to use IBM Cloud features like IAM, Key Protect, Immutable Object Storage, and others.



Table 1. Features supported per SDK

 Feature Java Python NodeJS GO CLI Terraform 

 [IAM API key support](https://cloud.ibm.com/docs/cloud-object-storage/basics?topic=cloud-object-storage-sdk-aboutsdk-about-iam) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) 
 [Managed multipart uploads](https://cloud.ibm.com/docs/cloud-object-storage/basics?topic=cloud-object-storage-sdk-aboutsdk-about-tmup) ![Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/icons/checkmark-icon.svg) !
Download, install, and start Cyberduck.
2. When the application opens, you can create a connection to Object Storage. Click Open Connection to configure the connection.
3. A pop-up window opens. From the menu, select the option, Amazon S3. Enter your information into the following fields:



* Server: enter the appropriate endpoint for your data at IBM Cloud Object Storage



Ensure that the endpoint region matches the intended bucket. For more information about endpoints, see [Endpoints and storage locations](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-endpointsendpoints).



* Access Key ID generated by selecting the appropriate HMAC option when creating a [Service Credential](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-service-credentials);
* Secret Access Key also from the HMAC option.
* Add to Keychain: Save the configuration to the your personal keychain (optional).
* Ignore the other options like the Anonymous Login checkbox, and SSH Private Key.



4. Cyberduck takes you to the root of the account where buckets can be created.



* Right-click within the main pane and select New Folder.



Cyberduck supports many transfer protocols where Folder is the more common name for a container construct.



* Enter the bucket name and then click Create.



5. After the bucket is created, double-click the bucket to view it. Within the bucket you can perform various functions such as:



* Upload files to the bucket
* List bucket contents
* Download objects from the bucket
* Synchronize local files to a bucket
* Synchronize objects to another bucket
* Create an archive of a bucket



6. Right-click within the bucket and select Synchronize. A pop-up window opens where you can browse to the folder that you want to synchronize to the bucket. Select the folder and click Choose.
7.
If your app requires dependencies that are not included with the base Cloud Functions [Python runtime](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-runtimesopenwhisk_ref_python_environments), you can install those dependencies into a virtualenv folder by using the Python environment inside the Cloud Functions Python runtime image. You can then compress the folder into a compressed file to deploy in Cloud Functions. Your compressed file must be smaller than the maximum codeSize as described in the [Action Limits](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-limitslimits_actions).

Example command

You can use the compressed file to create an action. Replace <file_path> with the file path to your compressed file.

ibmcloud fn action create <action_name> <compressed_python_virtualenv.zip> --kind python:3.11

For more information, see [Packaging Python code with a Docker virtual environment in a compressed file](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_virtenv).





 Packaging large Python dependencies in a custom Docker image 

When to use this method

Your app requires dependencies that are not included with the base Cloud Functions [Python runtime](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-runtimesopenwhisk_ref_python_environments). You can specify a base Cloud Functions image in your Dockerfile and also specify the dependencies to install when you build your Docker image. You can then specify your custom Docker image when you deploy your app in Cloud Functions. Note that only public Docker images are supported.

Example command

ibmcloud fn action create <action_name> <app_code.py> --docker <dockerhub_username>/<repo_name>:<tag_name>
Asynchronous method calls 

The SDK MUST support asynchronous invocation of any method that performs a network operation. The SDK SHOULD support synchronous or pseudo-synchronous (e.g. async/await) invocation of methods that perform network operation in languages where this method style is idiomatic.





 Instrumentation 

The SDK SHOULD be instrumented so that relevant usage data can be collected by the service. In particular, the SDK should supply the SDK name and version in the User-Agent header of each API request, along with supporting information such as language version, OS name and version, etc. whenever such information is readily available.





 Parameter validation 

The SDK MUST perform basic validation of method parameters and return errors/exceptions before invoking the underlying API. For example, the SDK should ensure all required parameters have been supplied before invoking the API.

However, validation of specific enum values SHOULD NOT be performed in the SDK, since this would preclude the SDK from being used with a later version of the service that supports new enum values for a parameter.





 Default values 

The SDK SHOULD NOT supply default values for any parameter not supplied by the caller, even if a default value is clearly specified in the API documentation. Rather, the assignment of the default should be delegated to the service, so that the service can alter the default value when appropriate without impacting the SDK.





 Serialization and deserialization 

The SDK MUST silently ignore additional properties in response payloads, to allow the SDK to work with newer versions of the service.

The SDK SHOULD silently ignore additional properties in dictionary- or map-type parameters. This is to allow values that may have been returned from a newer version of the service to be supplied on subsequent method invocations.





 Retry 

The SDK MAY provide an automated retry mechanism provided that the user can specify:



* The maximum number of retry attempts, including zero.
* The minimum and maximum time interval between retry attempts.
* The HTTP status codes or error classes that should be retried.



The retry mechanism should respect a "retry-after" response header if one is returned from the service.





 Logging
To minimize these dependencies on your local environment, use the [Packaging Python code with a Docker virtual environment in a compressed file](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_virtenv) approach. This approach creates the compressed action file, but also leverages the Python environment inside the Cloud Functions Python runtime image itself so that both the generated action compressed file and the later execution environment fully match.

Before you begin



* The following steps assume that you are running the commands on a Linux-based distribution on a processor with AMD64-based architecture.
* [Review the packages that are included with the Python runtime](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-runtimesopenwhisk_ref_python_environments) to see whether a dependency of your app is already included with the runtime. If your dependency is not included, you must package it with your app.
* Make sure that the locally installed Python version to create the compressed action file (for example, Python 3.11.x) matches the Cloud Functions kind that is chosen to later create the action (--kind python:3.11).
* Install the virtualenv Python package.

pip install virtualenv



To package your app:



1. Create a directory that you can use to create your virtual environment. In this example, a jokes directory is created on the desktop. After you create the jokes directory, cd to it.

cd desktop; mkdir jokes; cd jokes
2. From the jokes directory, create a virtual environment named virtualenv.

The virtual environment must be named virtualenv.

virtualenv virtualenv

Example output

created virtual environment CPython3.9.10.final.0-64 in 398ms
creator CPython3Posix(dest=/action/Projects/python/virtualenv, clear=False, no_vcs_ignore=False, global=False)