vania-janet/multiturn-rag-retrieval-data huggingface dataset:
ok, res = sdclient.get_dashboards()

 Loop through all fetched dashboards
for dashboard in res[1]:
 Delete dashboard if it matches the pattern (one or many)
if DASHBOARD_NAME in dashboard['name']:
print("Deleting " + dashboard['name'])
res = sdclient.delete_dashboard(dashboard)
Show more





 Delete a dashboard from a team 

The following code shows the structure of a Python script to delete a dashboard from a specific team.

!/usr/bin/env python3

import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))

from sdcclient import SdMonitorClient

 Parse arguments.
def usage():
print('usage: %s <endpoint-url> <sysdig_token>' % sys.argv[0])
print('endpoint-url: The endpoint URL that should point to IBM Cloud')
print('sysdig_token: Sysdig token for the team')
sys.exit(1)

if len(sys.argv) != 3:
usage()

URL = sys.argv[1]

 Set to the Sysdig token of the team
SYSDIG_TOKEN = sys.argv[2]

sdclient = SdMonitorClient(token=SYSDIG_TOKEN,sdc_url=URL)

 Show the list of dashboards
ok, res = sdclient.get_dashboards()

 Loop through all fetched dashboards
for dashboard in res[1]:
 Delete dashboard if it matches the pattern (one or many)
if DASHBOARD_NAME in dashboard['name']:
print("Deleting " + dashboard['name'])
Alternatively, on your app's Overview page, on the Continuous delivery card, click View toolchain. Then, click Overview.
2. Click Add tool.
3. In the Tool Integrations section, click GitHub.
4. Click the GitHub server that you want to use.
5. If you have a GitHub or GitHub Enterprise repo and want to use it, for the repository type, click Existing and type the URL.
6. If you want to use a new GitHub or GitHub Enterprise repo, type a name for the repo, type the URL for the repo that you are cloning or forking, and select the repository type:

a. To create an empty repo, click New.

b. To create a copy of a GitHub or GitHub Enterprise repo, click Clone.

c. To fork a GitHub or GitHub Enterprise repo so that you can contribute changes through pull requests, click Fork.
7. If you are a GitHub.com user with an upgraded account, or you selected a GitHub Enterprise server and you want to make a new private repo on the server, select the Make this repository private checkbox.
8. If you want to use GitHub's Issues for issue tracking, select the Enable GitHub Issues checkbox.
9. If you want to track the deployment of code changes by creating tags and comments on commits, and labels and comments on issues that are referenced by the commits, select the Track deployment of code changes checkbox. For more information, see [Track where your code is deployed with toolchains](https://www.ibm.com/cloud/blog/announcements/track-code-deployed-toolchains/).
10. Click Create Integration.
11. From your toolchain's Overview page, on the Repositories card, click the GitHub or GitHub Enterprise repo that you want to work with.
See [Uploading resources from another workspace](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-exportimport) for information about how to download these artifacts so that they can be imported into a new workspace.







 Recreating workspaces 

NOTE: In this step, the only setting that must match the setting of the original (downloaded) workspace is the language. The rest of the settings can differ from the settings of the original workspace.

Recreate each workspace by copying the following information from the previous instance to the new one:



* Workspace name
* Workspace description
* Language of documents (this setting must match the setting in the original workspace)
* If you previously used a dictionary-based tokenizer in the workspace, and have a valid need to continue using it, you must specify that you want to use the dictionary-based tokenizer instead of the default tokenizer at the time that you create the workspace. For more information about the options, see [Tokenizers](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-create-projectwks_tokenizer).

To use a dictionary-based tokenizer, expand the Advanced Options section of the Create Workspace window and change the tokenizer setting.







 Restoring workspace data 

After recreating the workspaces, upload the previously downloaded artifacts:



1. Upload the type system from the previously created type system backup. For details, see [Uploading resources from another workspace](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-exportimport).

Note: You must upload the type system before you can upload any other artifacts that you are moving from the backed up workspace.
2. Upload the dictionaries from the previously created dictionary backup.
Enabling versioning using the IBM COS SDK for Python can be done using either the [high-level resource](https://ibm.github.io/ibm-cos-sdk-python/reference/services/s3.htmlservice-resource) or [low-level client](https://ibm.github.io/ibm-cos-sdk-python/reference/services/s3.htmlclient) syntax.

Using a resource:

!/usr/bin/env python3

import ibm_boto3
from ibm_botocore.config import Config
from ibm_botocore.exceptions import ClientError

Define constants
API_KEY = os.environ.get('IBMCLOUD_API_KEY')
SERVICE_INSTANCE = os.environ.get('SERVICE_INSTANCE_ID')
ENDPOINT = os.environ.get('ENDPOINT')

BUCKET = "my-versioning-bucket" The bucket that will enable versioning.

Create resource client with configuration info pulled from environment variables.
cos = ibm_boto3.resource("s3",
ibm_api_key_id=API_KEY,
ibm_service_instance_id=SERVICE_INSTANCE,
config=Config(signature_version="oauth"),
endpoint_url=ENDPOINT
)

versioning = cos.BucketVersioning(BUCKET)

versioning.enable()

Show more

Versioning for the bucket can then be suspended using versioning.suspend()

Using that same cos resource, all versions of objects could be listed using the following:

versions = s3.Bucket(BUCKET).object_versions.filter(Prefix=key)

for version in versions:
obj = version.get()
print(obj.get('VersionId'), obj.get('ContentLength'), obj.get('LastModified'))

Using a client:

!/usr/bin/env python3
import BMSPush

// with this

import ENPushDestination
2. Replace SDK Initialization code

// Replace this

let push = BMSClient.sharedInstance

push.initialize(bluemixRegion: "<IBM cloud region>")

push.initializeWithAppGUID(
appGUID: "<IBM Cloud Push Instance GUID>",
clientSecret:"<IBM Cloud Push Instance ClientSecret>"
)

// with this

let instanceGUID = "<IBM-Cloud-en-instance_guid>>";
let destinationID = "<IBM-Cloud-en-instance-destination-id>";
let apiKey = "<IBM-Cloud-en-instance-apikey>";

let push = ENPush.sharedInstance

push.setCloudRegion(region: "<IBM cloud region>")

push.initialize(instanceGUID, destinationID, apiKey)
Show more

There are extra fields like destinationID and apikey in new initialize() method. For more information, on getting the apikey for client SDK see [Managing service access](https://cloud.ibm.com/docs/event-notifications?topic=event-notifications-service-access-management).
3. Change the device registration step



* If you are registering without userID

// Replace this

push.registerWithDeviceToken(deviceToken: "<apns-device-token>") {
(response, statusCode, error) -> Void in

print(response)

}

// with this

push.registerWithDeviceToken(deviceToken: "<apns-device-token>") {
response, statusCode, error in

print(response?.id ?? "")
}
Show more
* If you are using user ID for registration

// Replace this
The following example shows the content of the requirements.txt file, where web.py==0.37 indicates the version of the web.py library that will be downloaded is 0.37, and wsgiref==0.1.2 indicates the version of the web server gateway interface that is required by the web.py library is 0.1.2.

web.py==0.37
wsgiref==0.1.2

For more information about how to configure the requirements.txt file, see [Requirements files](https://pip.pypa.io/en/stable/user_guide/?highlight=requirements.txtrequirements-files).
2. Add a Procfile file to the root directory of your Python app. The Procfile file must contain the start command for your Python app. In the following command, <yourappname> is the name of your Python app, and PORT is the port number that your Python app must use to receive requests from users of the app. $PORT is optional. If you don't specify PORT in the start command, the port number under the VCAP_APP_PORT environment variable that is inside the app is used.

web: python <yourappname>.py $PORT



You can now import the third-party Python library into IBM Cloud.
URL = sys.argv[1]
APIKEY = sys.argv[2]
GUID = sys.argv[3]

 Name for the dashboard to create
dashboard_name = "My Overview by Process"
 Existing dashboard to copy
default_dashboard_name = "Overview by Process"
 New filter to apply
scope_filter = 'host.hostName = "virtualserver02"'

 Instantiate the client
ibm_headers = IbmAuthHelper.get_headers(URL, APIKEY, GUID)
sdclient = SdMonitorClient(sdc_url=URL, custom_headers=ibm_headers)

 Copy a dashboard
ok, res = sdclient.create_dashboard_from_view(dashboard_name, default_dashboard_name, scope_filter, shared=False, public=True)

 Check the result

if ok:
print('Dashboard created successfully')
else:
print(res)
sys.exit(1)
Show more





 Copy a dashboard in a team 

The following code shows the structure of a Python script to copy a dashboard to a team.

!/usr/bin/env python3

import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))

from sdcclient import SdMonitorClient

 Parse arguments.
def usage():
print('usage: %s <endpoint-url> <sysdig_token>' % sys.argv[0])
print('endpoint-url: The endpoint URL that should point to IBM Cloud')
print('sysdig_token: Sysdig token for the team')
sys.exit(1)

if len(sys.argv) != 3:
usage()

URL = sys.argv[1]
Uploading resources from another workspace 

To accelerate the creation of a model, you can upload resources like documents (with or without ground truth annotations), a type system, and dictionaries that you downloaded from another workspace.

The ability to separately download and upload different resources gives you flexibility when designing and creating a model. For example, you might create one workspace to design the type system and perform human annotation, and then create a separate workspace, perhaps in a separate instance of Knowledge Studio with different users, to train the machine learning model. Being able to upload the resources, including the ground truth created by human annotators, makes this scenario possible.

You cannot download and upload a machine learning model. Instead, you can download all of the artifacts that were used to create the model from one workspace and upload them into a new workspace. From the new workspace, you can run training again to recreate the model. The new model should produce similar results to the original model because they were both trained with the same set of artifacts.

The files that you download are operating system-independent. The Knowledge Studio instances where you download and upload files do not have to run the same version of Linux.



 Type systems 

To download a type system, open the Assets> Entity Types page and click Download Types. The system creates a file named types-ID.json and prompts you to download the file to your local system. To use this type system in a new workspace, open the Entity Types page and upload the JSON file that you downloaded.





 Dictionaries 

To download all dictionaries, select the Assets > Dictionaries page, click the Menu icon next to the Create Dictionary button, and select Download Dictionaries. For each dictionary that you download, the system creates a file named dictionary name_timestamp.csv, combines the files into a ZIP file named workspace name_dictionary_timestamp.zip, and prompts you to download the file to your local system.

You can download an individual dictionary by opening the dictionary and clicking Download. Preview-only dictionaries that you uploaded as a dictionary CSV file cannot be downloaded.
* [IBM COS SDK for Javascript (Node.js)](https://github.com/IBM/ibm-cos-sdk-js)
* [IBM COS SDK for Go](https://github.com/IBM/ibm-cos-sdk-go)
* [IBM COS Plug-in for IBM Cloud CLI](https://github.com/IBM/ibmcloud-cos-cli)
* [S3FS-FUSE](https://github.com/s3fs-fuse/s3fs-fuse)



While there is no dedicated API for a multipart download, it is possible to use a Range header in a GET request to read only a specific part of an object, and many ranged reads can be issued in parallel, just like when uploading parts. After all parts have been downloaded, they can be concatenated and the complete object can be checked for integrity. As mentioned previously, use of SDKs or other tooling is recommended to avoid the complexities of manually managing these transfers.

Workflows that need to store large numbers of very small objects may be better served by aggregating the small files into a larger data structure (such as [Parquet](https://parquet.apache.org/)) that can allow for access using [IBM Cloud® Data Engine](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-sql-query).

For objects greater than 200mb in size, especially in less stable networks or over very long distances where packet loss is a concern, Aspera High-Speed Transfer can deliver excellent performance. Aspera transfers can also upload nested directory structures efficiently within a single request.





 Throttling batch deletes 

The S3 API provides a mechanism for deleting up to 1,000 objects with a single batch delete request. It is recommended to throttle these requests client-side in order to minimize the chances of derogatory performance within the COS System.
template_repo_request_model['url'] = 'https://github.com/IBM-Cloud/hpc-cluster-openshift'

logging.info("Started Creating Schematic Workspace")

workspace_response = schematics_service.create_workspace(
description="HPC Cluster schematic workspace using API",
name="Sample Schematic API workspace",
template_data=[template_source_data_request_model],
template_repo=template_repo_request_model,
type=['terraform_v1.0'],
location="us-south",
resource_group="Default",
tags=[""],
).get_result()

print(json.dumps(workspace_response, indent = 2))





 Example Python response 

The following Python response is a generic example for reference.

INFO:root:Started Creating Schematic Workspace
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): iam.cloud.ibm.com:443
DEBUG:urllib3.connectionpool:https://iam.cloud.ibm.com:443 "POST /identity/token HTTP/1.1" 200 1086
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): us.schematics.cloud.ibm.com:443
DEBUG:urllib3.connectionpool:https://us.schematics.cloud.ibm.com:443 "POST /v1/workspaces HTTP/1.1" 201 1974
INFO:root:Completed Creating Schematic Workspace
{
"id": "us-south.workspace.Sample-Schematic-API-workspace.341cedd5",
"name": "Sample Schematic API workspace",
* Change the API endpoint to the endpoint mentioned in [API endpoints](https://cloud.ibm.com/apidocs/schematics?code=pythonapi-endpoints) according to the location that you want your Schematics workspace to reside, for example, schematics_service.set_service_url('https://us.schematics.cloud.ibm.com').



4. Inside the schematics_service.apply_workspace_command function, provide the following parameters:



* Provide the workspace ID that you generated in the [Creating a workspace](https://cloud.ibm.com/docs/hpc-openshift?topic=hpc-openshift-creating-workspace&interface=api) task, for example, us-south.workspace.Terraform-Schematics-Python-Workspace.b3bbc9f5.
* Export your IBM Cloud API key by using the following command:



export IBMCLOUD_API_KEY =”<ibm-cloud-api-key>”



* Run the following curl command to create a refresh token:



curl -X POST "https://iam.cloud.ibm.com/identity/token" -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=urn:ibm:params:oauth:grant-type:apikey&apikey=$IBMCLOUD_API_KEY" -u bx:bx
5. Run the Python script by using python3 <python-file-name> to apply a plan in the IBM Cloud.
6. You get an activity ID in response if the parameters passed as part of the request are valid. You should see the plan being applied in the Schematics workspace that you created in the IBM Cloud console. If you don’t get a successful response, the error response contains the errors that you need to resolve. Resolve those errors and run the script until you are able to get a valid response and apply a plan.
7.
For best practices and examples of how to use the SDKs see the Fabric [Asset Transfer Sample](https://github.com/hyperledger/fabric-samples/tree/master/asset-transfer-basic)

For information about migrating your applications created using the v1.4 SDK to the 2.x SDK, check out [Migrating client applications from v1.4 to v2.0](https://hyperledger.github.io/fabric-sdk-node/release-2.2/tutorial-migration.html).

Client applications can leverage the capabilities of the Go SDK in the high-level programming model in the [gateway package](https://pkg.go.dev/github.com/hyperledger/fabric-sdk-go/pkg/gateway)

If you want to take advantage of the High-Level Fabric contract-APIs, you can also use this tutorial to complete the following actions on an IBM Blockchain Platform network:



* [Generate certificates for your application](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-appibp-console-app-enroll) by using the SDK.
* [Invoke a smart contract from the SDK](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-appibp-console-app-invoke).
* Learn about application development by deploying the [commercial paper tutorial](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-appibp-console-app-commercial-paper) to the nodes managed from your console.







 Low-Level Fabric SDK APIs**
Creating a library set for Python package install 

A library set is a collection of libraries that you can create and reference in Spark applications that consume the libraries. The library set is stored in the instance home storage associated with the instance at the time the instance is created.

Currently, you can only install Python packages through conda or pip install.

Analytics Engine bundles a Spark application called customize_instance_app.py that you run to create a library set with your custom packages and can be consumed by your Spark applications.

Prerequites: To create a library set, you must have the permissions to submit a Spark application. See [User permissions](https://cloud.ibm.com/docs/AnalyticsEngine?topic=AnalyticsEngine-grant-permissions-serverless).

To create a library set:



1. Prepare a JSON file like the following:

{
"library_set": {
"action": "add",
"name": "my_library_set",
"libraries": {
"conda": {
"python": {
"packages": ["numpy"]
}
}
}
}
}

The description of the JSON attributes are as follows:



* "library_set": The top level JSON object that defines the library set.
* "action": Specifies the action to be taken for the library set. To create a library set, we use "add". Currently, "add" is the only option supported.
* "name": Specifies the name with which the library set is identified. The created library set is stored in a file with this name in the IBM Cloud Object Storage instance that you specified as instance home. Important: If you create more than one library set, you must use unique names for each set. Otherwise they will overwrite one another.
* "libraries": Defines a set of libraries. You can specify one or more libraries in this section. This element has one child JSON object per library package manager. Currently, only the "conda" and "pip" package managers are supported.
Retrieving action logs with Schematics API 



1. To retrieve the logs for an action, such as generate a plan, apply a plan, and destroy resources by using IBM Cloud® Schematics Python APIs, create a Python file and provide a name of your choice, for example, schematics_logs_api.py.
2. Copy and paste the [Retrieve Schematics workspace action details using Python API](https://cloud.ibm.com/docs/hpc-openshift?topic=hpc-openshift-retrieve-action-logs&interface=apiexample-request-retrieve-action-logs) example request into your Python file.
3. Change the following parameters as part of the request:



* Replace the authenticator = IAMAuthenticator('<ibm-api-key>') variable with your IBM Cloud API key.
* Change the API endpoint to the endpoint mentioned in this link according to the location where you want your Schematics workspace to reside, for example, schematics_service.set_service_url('https://us.schematics.cloud.ibm.com').



4. Inside the schematics_service.get_workspace_activity function, provide the following parameters:



* Provide the workspace ID that you generated in the [Creating a workspace](https://cloud.ibm.com/docs/hpc-openshift?topic=hpc-openshift-creating-workspace&interface=api) task, for example, us-south.workspace.Terraform-Schematics-Python-Workspace.b3bbc9f5.
* Provide an activity ID that was generated as a successful response for the specific action for which you want to retrieve logs. You received an activity ID when you successfully completed the action in the Schematics workspace.



5. Run the Python script by using python3 <python-file-name> to retrieve the logs for an action for your Schematics workspace in the IBM Cloud.
6.
Copy and paste the [Create a Schematics workspace using Python API](https://cloud.ibm.com/docs/hpc-openshift?topic=hpc-openshift-creating-workspace&interface=apiexample-request-create) example request to your Python file.
3. Change the following parameters as part of the request:



* Replace your IBM Cloud API key to the authenticator = IAMAuthenticator('<ibm-api-key>') variable.
* Change the API endpoint to the endpoint mentioned in [API endpoints](https://cloud.ibm.com/apidocs/schematics?code=pythonapi-endpoints) according to the location that you want your Schematics workspace to reside, for example, schematics_service.set_service_url('https://us.schematics.cloud.ibm.com').
* Provide environment values that you want to have in the form of a key and value pair, such as [{ 'Environment': 'hpc-dev-cluster' }]
* Change the folder location variable template_source_data_request_model['folder'] to the folder name inside your GitHub repository where your Terraform files reside. If the Terraform files exist in the base location of your repository, then set this variable to template_source_data_request_model['folder'] = ' '; otherwise, mention the folder name in the template_source_data_request_model['folder'] variable.
* Change the template_source_data_request_model['type'] variable to the Terraform version that you are using to create IBM Cloud resources such as terraform_v0.14. Ensure that your Terraform templates are compatible with the version that you are mentioning in this field.
* Provide your GitHub or GitLab Repository HTTPS URL where your Terraform files reside in the template_repo_request_model['url'] variable.
Edit your lithops config file and add the following keys:

lithops:
storage_backend: ibm_cos

ibm:
iam_api_key: <IAM_API_KEY>

ibm_cos:
endpoint : <REGION_ENDPOINT>
private_endpoint : <PRIVATE_REGION_ENDPOINT>









 Deploy your first Code Engine job by using Lithops 

Run the following hello world example,

import lithops

iterdata = [1,2,3,4,5]

def my_map_function(data):
return data + 1

if __name__ == '__main__':
pw = lithops.function_executor(type = "serverless", backend='code_engine')
futures = pw.map(my_map_function, iterdata)
print (pw.get_result())
pw.clean()





 Next steps for Lithops 

You can find more examples and use cases, as well as ask questions, on the [Lithops project page](https://github.com/lithops-cloud/lithops).