vania-janet/multiturn-rag-retrieval-data huggingface dataset:
The provided sample application show you how to reference a .py word count application file and a data file in a job payload.

To learn how to quickly get started using pre-bundled sample application files:



1. Generate an IAM token if you haven’t already done so. See [Retrieving IAM access tokens](https://cloud.ibm.com/docs/AnalyticsEngine?topic=AnalyticsEngine-retrieve-iam-token-serverless).
2. Export the token into a variable:

export token=<token generated>
3. Prepare the payload JSON file. For example, submit-spark-quick-start-app.json:

{
"application_details": {
"application": "/opt/ibm/spark/examples/src/main/python/wordcount.py",
"arguments": ["/opt/ibm/spark/examples/src/main/resources/people.txt"]
}
}
4. Submit the Spark application:

curl -X POST https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance_id>/spark_applications --header "Authorization: Bearer $token" -H "content-type: application/json" -d @submit-spark-quick-start-app.json







 Referencing files from an Object Storage bucket 

To reference your Spark application file from an Object Storage bucket, you need to create a bucket, add the file to the bucket and then reference the file from your payload JSON file.

The endpoint to your IBM Cloud Object Storage instance in the payload JSON file should be the private endpoint. Direct endpoints provide better performance than public endpoints and do not incur charges for any outgoing or incoming bandwidth.

To submit a Spark application:



1. Create a bucket for your application file. See [Bucket operations](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-compatibility-api-bucket-operations) for details on creating buckets.
2.
* [Packaging Python code with a local virtual environment in a compressed file](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_local_virt_zip)
* [Packaging Python code with a Docker virtual environment in a compressed file](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_docker_virt_zip)
* [Packaging large Python dependencies in a custom Docker image](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-preppackaging_large_python_docker)
* [Packaging your app within a custom Docker image](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-preppackaging_docker_skeleton)



* [Packaging Python code](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python)



* [Packaging multiple Python files into a compressed file](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_pkg)
* [Packaging Python code with a local virtual environment in a compressed file](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_local_virtenv)
* [Packaging Python code with a Docker virtual environment in an archive](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_virtenv)
* [Packaging large Python dependencies in a custom Docker image](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_python_docker)



* [Preparing apps in Docker images](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_docker)
Mobile SDK 

IBM Cloud® Functions provides a mobile SDK for iOS and watchOS devices that enables mobile apps to fire remote triggers and invoke remote actions. A version for Android is not available, so Android developers can use the OpenWhisk REST API directly. The mobile SDK is written in Swift 4 and supports iOS 11 and later releases. You can build the mobile SDK by using Xcode 9.

The mobile SDK is not supported for IAM-based namespaces. Use a Cloud Foundry-based namespace instead.



 Add the SDK to your app 

You can install the mobile SDK by using CocoaPods, Carthage, or from the source directory.



 Install mobile SDK with CocoaPods 

The IBM Cloud® Functions SDK for mobile is available for public distribution through CocoaPods. Assuming CocoaPods is installed, put the following lines into a file called Podfile inside the starter app project directory.

install! 'cocoapods', :deterministic_uuids => false
use_frameworks!

target 'MyApp' do
pod 'OpenWhisk', :git => 'https://github.com/apache/incubator-openwhisk-client-swift.git', :tag => '0.3.0'
end

target 'MyApp WatchKit Extension' do
pod 'OpenWhisk', :git => 'https://github.com/apache/incubator-openwhisk-client-swift.git', :tag => '0.3.0'
end

From the command line, type pod install. This command installs the SDK for an iOS app with a watchOS extension. Use the workspace file CocoaPods creates for your app to open the project in Xcode.

After installation, open your project workspace. You might get the following warning when building: Use Legacy Swift Language Version” (SWIFT_VERSION) is required to be configured correctly for targets which use Swift. Use the [Edit > Convert > To Current Swift Syntax…] menu to choose a Swift version or use the Build Settings editor to configure the build setting directly.
* [Install mobile SDK with Carthage](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_mobile_sdkinstall-mobile-sdk-carthage)
* [Install mobile SDK from source code](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_mobile_sdkinstall-mobile-sdk-source-code)



* [Install the starter app example for mobile SDK](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_mobile_sdkinstall-mobile-sdk-starter-app)
* [Getting started with mobile SDK](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_mobile_sdkget-started-mobile-sdk)
* [Invoke a mobile SDK action](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_mobile_sdkinvoke-mobile-sdk-action)
* [Fire a mobile SDK trigger](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_mobile_sdkfire-mobile-sdk-trigger)
* [Use mobile SDK actions that return a result](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_mobile_sdkmobile-sdk-actions-results)
* [Configuring the mobile SDK](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_mobile_sdkconfigure-mobile-sdk)



* [Support for qualified names with mobile SDK](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_mobile_sdkqualified-names-mobile-sdk)
sys.exit(1)

if len(sys.argv) != 3:
usage()

URL = sys.argv[1]

 Set to the Sysdig token of the team
SYSDIG_TOKEN = sys.argv[2]

sdclient = SdMonitorClient(token=SYSDIG_TOKEN,sdc_url=URL)

 Show the list of dashboards
ok, res = sdclient.get_dashboards()

if not ok:
print(res)
sys.exit(1)

for db in res['dashboards']:
print("%s [Team ID: %s] Name: %s, Charts: %d" % ( db['id'], db['teamId'], db['name'], len(db['widgets'] if 'widgets' in db else [])))
Show more







 Create a dashboard 

You can use Python to create dashboards.





 Create a dashboard in the default team 

The following code shows the structure of a Python script to create a new dashboard for the default team.

The following code shows the structure of a Python script:

!/usr/bin/env python3

import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import IbmAuthHelper, SdMonitorClient

 Parse arguments.
def usage():
print('usage: %s <endpoint-url> <apikey> <instance-guid>' % sys.argv[0])
print('endpoint-url: The endpoint URL that should point to IBM Cloud')
print('apikey: IBM Cloud IAM apikey that will be used to retrieve an access token')
print('instance-guid: GUID of an IBM Cloud Monitoring with monitoring instance')
Getting started with Spark use case 

This topic provides the procedure to run Spark use cases for watsonx.data by using Python samples. All the samples are written by using Spark Python APIs.



 Prerequisites 



* Provision an IBM® watsonx.data instance.
* Configure an IBM Analytics Engine instance.
* Cloud Object Storage bucket connection details.







 About the sample use case 

The sample file demonstrates the following functionalities:



* Accessing tables from watsonx.data

The Create a database in Lakehouse catalog section from the [sample python file](https://cloud.ibm.com/docs/watsonxdata?topic=watsonxdata-run_samp_filepython_file) creates a database demodb in the configured watsonx.data instance with a catalog named lakehouse. demodb is configured to store all the data and metadata under the Cloud Object Storage(COS) bucket lakehouse-bucket. It also creates an iceberg table testTable and accesses it.
* Ingesting data to watsonx.data

The Ingest parquet data into a lakehouse table section from the [sample python file](https://cloud.ibm.com/docs/watsonxdata?topic=watsonxdata-run_samp_filepython_file) allows you to ingest data in parquet and CSV format from a source Cloud Object Storage bucket source-bucket into a watsonx.data table. Sample data in parquet format is inserted from source COS bucket source-bucket into the watsonx.data table yellow_taxi_2022. It also shows ingesting data in CSV format from COS bucket source-bucket into the table zipcode in the database demodb.

You can download the sample formats from:



* [Sample parquet file](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page).
sdclient = SdMonitorClient(sdc_url=URL, custom_headers=ibm_headers)

 Copy a dashboard
ok, res = sdclient.create_dashboard_from_dashboard(dashboard_name, existing_dashboard_name, scope_filter, shared=False, public=True)

 Check the result

if ok:
print('Dashboard created successfully')
else:
print(res)
sys.exit(1)
Show more





 Copy a pre-defined dashboard in the default team 

The following code shows the structure of a Python script to copy a pre-defined dashboard to the default team.

!/usr/bin/env python3

import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import IbmAuthHelper, SdMonitorClient

 Parse arguments.
def usage():
print('usage: %s <endpoint-url> <apikey> <instance-guid>' % sys.argv[0])
print('endpoint-url: The endpoint URL that should point to IBM Cloud')
print('apikey: IBM Cloud IAM apikey that will be used to retrieve an access token')
print('instance-guid: GUID of an IBM Cloud Monitoring with monitoring instance')
sys.exit(1)

if len(sys.argv) != 4:
usage()

URL = sys.argv[1]
APIKEY = sys.argv[2]
GUID = sys.argv[3]

 Name for the dashboard to create
dashboard_name = "My Overview by Process"
 Existing dashboard to copy
default_dashboard_name = "Overview by Process"
 New filter to apply
scope_filter = 'host.hostName = "virtualserver02"'

 Instantiate the client
[External link icon](https://cloud.ibm.com/docs-content/v1/content/adc643f3bedeea02b638bc1c34732852dcb40a45/icons/launch-glyph.svg)](https://pip.pypa.io/en/latest/user_guide/requirements-files) in your file in your test directory that contains the pip modules and versions to install.

touch requirements.txt
3. Use vim to edit the requirements.txt file. Enter the names of the pip modules and versions you want to install. In this example, pyjokes is the module that is used.

vim requirements.txt

Example requirements.txt

pyjokes
4. Press ESC, then :wq to save and close your requirements.txt file.

To keep the virtualenv to a minimum size, add only the modules that are not part of the selected runtime environment to the requirements.txt. For more information about the packages that are included in Python runtimes, see the Python [runtime reference](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-runtimesopenwhisk_ref_python_environments).



* For python:3.11, use the Docker image ibmfunctions/action-python-v3.11.
* For python:3.9, use the Docker image ibmfunctions/action-python-v3.9.
* For python:3.7, use the Docker image ibmfunctions/action-python-v3.7.
* For python:3.6, use the Docker image ibmfunctions/action-python-v3.6.



Example

docker pull ibmfunctions/action-python-v3.9:1.0.0

Example output

Using default tag: latest
latest: Pulling from ibmfunctions/action-python-v3.9:1.0.0
5. Create a virtual environment and install the additional Python packages.
Distribution 



  Open-source 

The SDK MUST be open-source and hosted on a public GitHub repository in the [IBM organization](https://github.com/IBM). The repository name SHOULD follow the pattern of <service>-<language>-sdk (for example, platform-services-node-sdk). The SDK SHOULD be discoverable through simple GitHub and Google searches. The SDK SHOULD use the Apache 2.0 license.





  Package management 

The SDK SHOULD be distributed using the most popular package management systems for the language. See [package management](https://cloud.ibm.com/docs/sdk-handbook?topic=sdk-handbook-devtoolsdevtools-pkg-mgmt) for the recommended package managers by language and their respective requirements.





  Package dependencies 

The SDK SHOULD use the common dependency management system for declaring its dependencies, including "lock" files wherever supported to specify checksummed versions of dependencies used in an SDK release. Dependency versions should be specified with a fixed major version to avoid incompatible changes that might occur in major version updates.





  Semantic versioning 

The SDK MUST use [semantic versioning](https://semver.org/) for SDK releases (this requirement is enforced by some package management systems).
For more information, see [Track where your code is deployed with toolchains](https://www.ibm.com/cloud/blog/announcements/track-code-deployed-toolchains/).
10. Click Create Integration.
11. From your toolchain's Overview page, on the Repositories card, click the GitHub or GitHub Enterprise repo that you want to work with. Depending on the repo that you selected, either the GitHub website or your company's GitHub Enterprise repo opens, where you can view the contents of the repo.
12. If you enabled GitHub Issues, click GitHub Issues to open it. You can use this instance of GitHub Issues for your entire toolchain, even if the toolchain contains multiple GitHub or GitHub Enterprise repos.

If you don't have admin privileges for the repo that you are linking to, your integration is limited because you can't use a webhook. Webhooks are required to automatically run a pipeline when a commit is pushed to the repo. Without a webhook, you must start your pipelines manually.





 Configuring GitHub by using the API 

The GitHub tool integration supports the following configuration parameters that you can use with the [Toolchain HTTP API and SDKs](https://cloud.ibm.com/apidocs/toolchain) when you [create](https://cloud.ibm.com/apidocs/toolchaincreate-tool), [read](https://cloud.ibm.com/apidocs/toolchainget-tool-by-id), and [update](https://cloud.ibm.com/apidocs/toolchainupdate-tool) tool integrations.

You must specify the tool_type_id property in the request body with the githubconsolidated value.



Table 1. GitHub tool integration parameters

 Parameter Usage Type Terraform argument Description 

 api_root_url optional, updatable String api_root_url The URL of the API root for the GitHub server.
* Click Create to display subscription wizard.
* Complete the following subscription details:



* Subscription name: name of the subscription.
* Subscription description: add an optional description.



* Under the Subscribe to a topic section, select a topic from the drop-down list and select a destination from the destination drop-down list.
* Destination type: select type under Destination and click Add.







 Step 7: Set up Event Notifications Safari web SDK 

The Safari web SDK enables Safari websites to receive push notifications. Complete the following steps to install Event Notifications Firefox web SDK, initialize the SDK, and register for notifications for your website.



* To include the SDK in your project, add the ENPushSDK.js, ENPushServiceWorker.js, and manifest_Website.json files to your project root folder.
* Edit the manifest_Website.json file.

{
"name": "YOUR_WEBSITE_NAME"
}
* Change the manifest_Website.json file name to manifest.json.
* Include the manifest.json in the <head> tag of your html file.

<link rel="manifest" href="manifest.json">
* Include IBM Cloud web push SDK to the script.

<script src="ENPushSDK.js" async></script>
* Complete the following steps to enable the website to initialize the SDK

var enPush = new ENPush()

function callback(response) {
alert(response.response)
}

var initParams = {
"instanceGUID": "<instance_guid>",
"apikey": "<instance_apikey>",
"region": "<region>",
"deviceId": "<YOUR_DEVICE_ID>",
"safariDestinationId": "<safari_destination_id>",
"websitePushIdSafari": "<Safari Web Push Id Identifier>"
}
{'id': 'cpu.used.percent', 'aggregations': {'time': 'avg', 'group': 'avg'}}
]
ok, res = sdclient.add_dashboard_panel(
dashboard_name, PANEL_NAME, panel_type, metrics)

 Check the result
if ok:
print('Panel added successfully')
dashboard_name = res['dashboard']
else:
print(res)
sys.exit(1)
Show more





 Create a dashboard in a team 

The following code shows the structure of a Python script to create a dashboard for a specific team.

!/usr/bin/env python3

import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))

from sdcclient import SdMonitorClient

 Parse arguments.
def usage():
print('usage: %s <endpoint-url> <sysdig_token>' % sys.argv[0])
print('endpoint-url: The endpoint URL that should point to IBM Cloud')
print('sysdig_token: Sysdig token for the team')
sys.exit(1)

if len(sys.argv) != 3:
usage()

URL = sys.argv[1]

 Set to the Sysdig token of the team
SYSDIG_TOKEN = sys.argv[2]

DASHBOARD_NAME = 'My New Dashboard from Python'
PANEL_NAME = 'CPU Over Time'

sdclient = SdMonitorClient(token=SYSDIG_TOKEN,sdc_url=URL)

 Create an empty dashboard
ok, res = sdclient.create_dashboard(DASHBOARD_NAME)

 Check the result
dashboard_name = None
if ok:
If you're downloading or uploading large files or large bodies of data, you can use the Requestdownload or upload methods. To monitor the progress of the download or upload, create a custom ProgressListener and pass it to the download or upload methods.

For complete usage examples, see the BMSCore GitHub [README](https://github.com/ibm-bluemix-mobile-services/bms-clientsdk-android-core).





 iOS 



1. Make sure that you have [imported the Client SDK and initialized it](https://cloud.ibm.com/docs/cli/sdk?topic=cli-sdk_BMSClientinit-BMSClient-ios) in your iOS app.
2. Create a network request.

Swift 3.0

let customResourceURL = "<your resource URL>"
let request = Request(url: customResourceURL, method: HttpMethod.GET)

let callBack:BMSCompletionHandler = {(response: Response?, error: Error?) in
if error == nil {
print ("Response: (response?.responseText), no error")
} else {
print ("Error: (error)")
}
}
request.send(completionHandler: callBack)

Swift 2.2

let customResourceURL = "<your resource URL>"
let request = Request(url: customResourceURL, method: HttpMethod.GET)

let callBack:BMSCompletionHandler = {(response: Response?, error: NSError?) in
if error == nil {
print ("Response: (response?.responseText), no error")
} else {
print ("Error: (error)")
}
}
request.send(completionHandler: callBack)



The Request class is a simple way to make an HTTP request and get the response after the request is completed.
The created library set is stored in a file with this name in the IBM Cloud Object Storage instance that you specified as instance home. Important: If you create more than one library set, you must use unique names for each set. Otherwise they will overwrite one another.
* "libraries": Defines a set of libraries. You can specify one or more libraries in this section. This element has one child JSON object per library package manager. Currently, only the "conda" and "pip" package managers are supported. Use "pip" or "conda" to install Python packages.
* "conda": Library package manager.
* "python": The library language. Currently, only Python is supported.
* "packages": List of packages to install. To install a specific version of a package, pass the version using this format: package_name==version.



2. Get the [IAM token](https://cloud.ibm.com/docs/AnalyticsEngine?topic=AnalyticsEngine-retrieve-iam-token-serverless).
3. Pass the JSON file as "arguments" in the following REST API call. Make sure that you escape the quotes as required, while passing to the REST API call.

curl -X POST https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance_id>/spark_applications --header "Authorization: Bearer <IAM token>" -H "content-type: application/json" -d @createLibraryset.json

Example for createLibraryset.json:

{
"application_details": {
"application": "/opt/ibm/customization-scripts/customize_instance_app.py",
"arguments": ["{"library_set":{"action":"add","name":"my_library_set","libraries":{"conda":{"python":{"packages":"numpy"]}}}}}"]
}
}
* [Reading an object with the CLI](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_obstoragepkg_obstorage_read_cli)
* [Reference for Object Storage and Cloud Functions](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_obstoragepkg_obstorage_actions)





[Slack](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_slackpkg_slack)



* [Posting a message to a Slack channel](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_slackpkg_slack_post)
* [Using the Slack token-based API](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_slackpkg_slack_api)



[Utilities](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_utilspkg_utils)

[WebSocket](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_websocketpkg_websocket)



* [Send a message to a WebSocket](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_websocketsend-websocket)



[Creating custom event provider feeds](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-feeds_customfeeds_custom)



* [Feed architecture](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-feeds_customfeeds_arch)
To download all dictionaries, select the Assets > Dictionaries page, click the Menu icon next to the Create Dictionary button, and select Download Dictionaries. For each dictionary that you download, the system creates a file named dictionary name_timestamp.csv, combines the files into a ZIP file named workspace name_dictionary_timestamp.zip, and prompts you to download the file to your local system.

You can download an individual dictionary by opening the dictionary and clicking Download. Preview-only dictionaries that you uploaded as a dictionary CSV file cannot be downloaded.

Before you upload a downloaded dictionary into a new workspace, you must download the type system from the old workspace and upload it into the new workspace. The type system and dictionaries must originate from the same Knowledge Studio workspace, and the type system must exist in the new workspace before you upload the dictionaries.

To upload dictionaries, open the Dictionaries tab and either add a CSV file that you downloaded or upload the ZIP file.





 Documents 

To download documents that you added to the corpus, open the Assets> Documents > Document Sets tab and click Download Document Sets. The system creates a file named corpus-ID.zip and prompts you to download the file to your local system. The ZIP file contains all documents in the corpus. Annotations that were added to annotation sets are included in the ZIP file, but only after the annotation sets have been approved and promoted to ground truth.

> Restriction: Any documents that were pre-annotated will be obscured into a non-readable format when they are downloaded. Even annotations in those documents that were added by human annotation are unreadable.

Before you upload documents that include ground truth into a new workspace, you must download the type system from the old workspace and upload it into the new workspace. The type system and documents must originate from the same Knowledge Studio workspace, and the type system must exist in the new workspace before you upload the ground truth annotations.

To upload documents, open the Document Sets tab in the new workspace, click Upload Document Sets and select the corpus ZIP file that you downloaded.