vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Retrieving action logs with Schematics API 



1. To retrieve the logs for an action, such as generate a plan, apply a plan, and destroy resources using IBM Cloud® Schematics Python APIs, create a Python file and provide a name of your choice, for example, schematics_logs_api.py.
2. Copy and paste the [Retrieve Schematics workspace action details using Python API](https://cloud.ibm.com/docs/hpc-spectrum-scale?topic=hpc-spectrum-scale-retrieve-action-logs&interface=apiexample-request-retrieve-action-logs) example request into your Python file.
3. Change the following parameters as part of the request:



* Replace the authenticator = IAMAuthenticator('<ibm-api-key>') variable with your IBM Cloud API key.
* Change the API endpoint to the endpoint mentioned in this link according to the location where you want your Schematics workspace to reside, for example, schematics_service.set_service_url('https://us.schematics.cloud.ibm.com').



4. Inside the schematics_service.get_workspace_activity function, provide the following parameters:



* Provide the workspace ID that you generated in the [Creating a workspace](https://cloud.ibm.com/docs/ibm-spectrum-lsf?topic=ibm-spectrum-lsf-creating-workspace) task, for example, us-south.workspace.Terraform-Schematics-Python-Workspace.b3bbc9f5.
* Provide an activity ID that was generated as a successful response for the specific action for which you want to retrieve logs. You received an activity ID when you successfully completed the action in the Schematics workspace.



5. Run the Python script by using python3 <python-file-name> to retrieve the logs for an action for your Schematics workspace in the IBM Cloud.
6.
This file contains dictionary terms in CSV format, suitable for uploading into a Knowledge Studio dictionary.
2. Click Assets > Dictionaries.
3. Click Create Dictionary to add a dictionary.

Do not click Upload dictionary, which is used to upload a dictionary that you want to use as-is. For the tutorial, you will create a new editable dictionary and then upload terms into it.
4. In the Name field, type Test dictionary and click Save to create the (empty) dictionary.

The new dictionary is created and automatically opened for editing.
5. In the dictionary pane, click Upload.
6. Upload the dictionary-items-organization.csv file from your computer. The terms in the file are uploaded into the dictionary.
7. Click Add Entry to create a new term. An editable row is added at the top of the table.
8. In the Surface Forms column, type IBM and International Business Machines Corporation on separate lines. (When you begin to type a new surface form, a space is added below for an additional surface form.) Leave the radio button next to IBM selected, which indicates that IBM is the lemma.
9. In the Part of Speech column, select Noun.
10. Click Save.

![A screen capture of the Dictionaries page. The "IBM" dictionary entry is selected and its surface forms and part of speech are highlighted.](images/wks_tutdict4.png "A screen capture of the Dictionaries page. The "IBM" dictionary entry is selected and its surface forms and part of speech are highlighted.")







 What to do next 

After you create a dictionary, you can use it to speed up human annotation tasks by pre-annotating the documents.







 Next steps 

Learn how to [create a machine learning model](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-wks_tutml_intro).





 Tutorial summary
Updating variables with Schematics API 



1. To update variables by using the IBM Cloud® Schematics Python APIs, create two Python files, and provide a name of your choice for those files, for example, schematics_variables_update.py and schematics_env_class.py.
2. Copy and paste the [schematics_variables_update.py](https://cloud.ibm.com/docs/hpc-openshift?topic=hpc-openshift-update-variables&interface=uiexample-request-update-variables) and [schematics_env_class.py](https://cloud.ibm.com/docs/hpc-openshift?topic=hpc-openshift-update-variables&interface=uiexample-request-update-variables-file) Python example code requests to the respective Python files.
3. Copy and paste the [config.json template file](https://cloud.ibm.com/docs/hpc-openshift?topic=hpc-openshift-update-variables&interface=apitemplate-file) to a JSON file, for example config.json.
4. Change the following parameters as part of the request:



* Provide the workspace ID w_id generated in both the following functions: schematic_obj.get_workspace(w_id="<w_id>) and schematic_obj.update_variables(w_id="<wi_id>").



5. Make sure to update the required parameters, such as api_key, region, ssh_key_name, cluster_prefix, and worker_nodes_per_zone in the config.json file.
6. Run the Python script by using python3 to update the variables in the Schematics workspace in the IBM Cloud.



The following parameters might not be required in config.json as the Schematics update variables API uses the workspace ID w_id to update the variables against that workspace.

{
* Pull request pipeline: Runs when a developer merges changes from their development branch to the master branch, or to any other branch in the repo. The pull request pipeline runs the Unit Test and Static Scans on the Application Source Code.
* Continuous integration pipeline: Runs when you merge a change into the master branch of the Application Source Code repo. The continuous integration pipeline runs the Unit Test, Code Coverage, and Static Scans on the Application Source Code, CIS check, and Bill Of Materials (BOM) check. The continuous delivery pipeline also generates the binary build artifacts and uploads them to the IBM Cloud® Kubernetes Service, as configured in the toolchain. And the continuous integration pipeline generates the metadata of the build artifacts and stores it in the Inventory repo.
* Continuous deployment pipeline: Deploys a Code Engine component image (that the continuous integration pipeline builds and deploys) to a different Code Engine component to be hosted in a different project.







 Run the pull request and continuous integration pipelines 

To start the pull request pipeline, create a merge request in your app repo:



1. On the Toolchain's Overview page, on the Repositories card, click the secure-app-toolchain-code-engine-demo app repo.
2. From the master repo, create a branch.
3. Update some code in the sample node app or readme file and save these changes.
4. Submit the merge request.
5. On the Toolchain's Overview page, on the Delivery pipelines card, click the pr-pipeline pipeline to open the pull request pipeline dashboard. The corresponding merge request in your app repo remains in the pending state until all of the stages of the pull request pipeline successfully complete.
6. After the pull request pipeline run succeeds, you can select it to explore the completed steps.



Zoom

![Pull request pipeline success](https://cloud.ibm.com/docs-content/v1/content/562806c0b18119f4d21cb66f5ae6d051634b4a16/ContinuousDelivery/images/code_engine_explore_pr_pipeline_success.png)
In the COS UI, switch to object view by clicking on the icon next to Upload, You should see that there are a couple of metadata objects to track, such as the latest offset that has been consumed and landed. But, in addition, you can find the Parquet files with the actual payload data.
5. Return to the Data Engine UI and in the Details tab click on Query the result and then click Run to execute a Batch job. You should see the query in the panel pointing to the Object Storage file (under FROM) with the log message(s) you sent above. Wait for the job to change to Completed.
6. Click on the Results tab to see the log messages in a tabular format.

The query saves the result to a CSV file under a different bucket with name sql-<SQL_QUERY_GUID>. Check the INTO part of the query.







 Increasing message load 

For later analysis purposes increase the message volume sent to Event Streams. The provided script simulates a flow of messages to Event Streams based on traffic to the webserver. To demonstrate the scalability of Event Streams, you will increase the throughput of log messages.



1. Download and unzip the Jul 01 to Jul 31, ASCII format, 20.7 MB gzip compressed log file from NASA:

curl ftp://ita.ee.lbl.gov/traces/NASA_access_log_Jul95.gz -o NASA_access_log_Jul95.gz
gunzip NASA_access_log_Jul95.gz
2. Turn the access logs into JSON format by running:

awk -F " " '{ print "{"host":"" $1 "","time_stamp":"" $4 " " $5 "","request":" $6 " " $7 " " $8 ","responseCode":"" $9 "","bytes":"" $10 ""}" }' NASA_access_log_Jul95 > NASA_logs.json
3. Create a shell script to only send few log lines per second.
},
IndexDocument: {
Suffix: "index.html"
}
}
};
function putBucketWebsiteConfiguration(websiteParams) {
return cos.putBucketWebsite({websiteParams}).promise()
.then((data) => {
console.log(Website configured for ${bucketName});
})
.catch((e) => {
console.log(ERROR: ${e.code} - ${e.message}n);
});
}
Show more







 Next Steps 

More detail on individual methods and classes can be found in [the SDK's API documentation](https://ibm.github.io/ibm-cos-sdk-js/). Check out the source code on [GitHub](https://github.com/IBM/ibm-cos-sdk-js).
To avoid loss of this information, [export your models](https://cloud.ibm.com/docs/discovery?topic=discovery-sduimport), back them up, and store them locally. SDU models have the file extension of .sdumodel.







 Restoring your data to a new Watson Discovery instance 

Consider using your backups to restore to a new Discovery instance in a different Data Center, also known as a region or location. For example, these regions or locations include Dallas, Washington DC, and London).

To begin restoration, first start by reviewing your list of collections and associated data sources, as well as your file backups.



* Create your [environment](https://cloud.ibm.com/apidocs/discoverycreate-an-environment) and [collections](https://cloud.ibm.com/apidocs/discoverycreate-a-collection) using the saved configuration information. Ensure that the appropriate configuration is defined appropriately, and that the language for the collection is set properly. Failure to do so delays getting your system back up and running.
* If you have any custom configurations set up in Discovery, recreate them. See the [API reference](https://cloud.ibm.com/apidocs/discoveryadd-configuration) for details.
* Add back tokenization dictionaries or stopwords into the collections. See [Creating custom tokenization dictionaries](https://cloud.ibm.com/docs/discovery?topic=discovery-query-conceptstokenization) and [Defining stopwords](https://cloud.ibm.com/docs/discovery?topic=discovery-query-conceptsstopwords).
* If you use custom query expansion, [add your query expansions](https://cloud.ibm.com/apidocs/discoverycreate-or-update-expansion-list) for each collection, as well.
* If you use any custom entity models from IBM Watson™ Knowledge Studio for enrichment, [reimport that model](https://cloud.ibm.com/docs/discovery?topic=discovery-configservicecustom-entity-model) into your Discovery instance.
Create a function that includes a dependency for a specific Python module by creating a requirements.txt file. In this case, both the source code and requirements file are located in the same folder.



1. Create your Function by saving your code into a __main__.py file

import pyjokes
def main(params):
return {
"headers": { 'Content-Type': 'text/plain;charset=utf-8' },
"body": pyjokes.get_joke()
}
2. Create a requirements.txt containing your required dependencies for your Function

pyjokes==0.6.0
3. Create your files as a Function in Code Engine. In this case, you are in the directory where the local files exist so you can use . as the build source.

ibmcloud ce fn create -n pyjokes -runtime python-3.11 --build-source .
4. Invoke your Function by pasting the provided URL into a web browser. Your browser displays a random joke!



For more information about the fn create command and its options, see [Create a Function](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-function-create).







 Next steps 

Now that your function is created and deployed from local source code, you can update the function to meet your needs by using the [ibmcloud ce function update](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-function-update) command. If you want to update your source to use with your function, you must provide the --build-source option on the function update command.

When your function is deployed from local source or from [repository source code](https://cloud.ibm.com/docs/codeengine?topic=codeengine-app-source-code) from the CLI, the resulting build run is not based on a build configuration. Build runs that complete are ultimately automatically deleted.
* [jq](https://stedolan.github.io/jq/) - a lightweight and flexible command-line JSON processor
* [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) - an open source-distributed version control system



To avoid the installation of these tools, you can also use the [IBM Cloud](https://www.ibm.com/cloud/cloud-shell) from the IBM Cloud console.





 IBM Solutions Tutorials 



 Moving a VM-based app to Kubernetes 

For a detailed tutorial on moving a VM-based app to a Kubernetes cluster by using Kubernetes Service, check out [Moving a VM-based app to Kubernetes ](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-vm-to-containers-and-kubernetes). This tutorial walks you through storing app pod data in an Cloud Databases service, allowing Cloud Databases to easily configure and manage your database-as-a-service (DBaaS) with built-in backups and scaling.





 Modern web application with MEAN stack 

Using [Databases for MongoDB](https://cloud.ibm.com/docs/databases-for-mongodb), [create a Modern web application with MEAN stack](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-mean-stack). You learn how to run a MEAN starter locally, create and use a managed database-as-a-service (DBasS), deploy the app to IBM Cloud, and scale the database resources.





 Scale workloads in shared and dedicated VPC environments
* [Implementing feeds with polling](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-feeds_customfeeds_polling)
* [Implementing feeds by using connections](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-feeds_customfeeds_connections)



[IBM Cloud Monitoring](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-alerts-notifyalerts-notify)



 Watson packages 

[Watson Assistant](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_watson_assistantpkg_watson_assistant)



* [Creating a Watson Assistant service instance](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_watson_assistantservice_instance_conversation)
* [Installing the Watson Assistant package](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_watson_assistantinstall_conversation)



* [Installing from the Cloud Functions CLI](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_watson_assistantconversation_cli)
* [Installing from the Cloud Functions console](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_watson_assistantconversation_ui)



* [Using the Watson Assistant package](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_watson_assistantusage_conversation)



[Natural Language Classifier](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-pkg_natlang_classifierpkg_natlang_classifier)
print('usage: %s <endpoint-url> <sysdig_token>' % sys.argv[0])
print('endpoint-url: The endpoint URL that should point to IBM Cloud')
print('sysdig_token: Sysdig token for the team')
sys.exit(1)

if len(sys.argv) != 3:
usage()

URL = sys.argv[1]

 Set to the Sysdig token of the team
SYSDIG_TOKEN = sys.argv[2]

 Name for the dashboard to create
dashboard_name = "My new CPU dashboard"
 Existing dashboard to copy
existing_dashboard_name = "My Existing Dashboard"
 New filter to apply
scope_filter = 'host.hostName = "virtualserver02"'

sdclient = SdMonitorClient(token=SYSDIG_TOKEN,sdc_url=URL)

 Copy a dashboard
ok, res = sdclient.create_dashboard_from_dashboard(dashboard_name, existing_dashboard_name, scope_filter, shared=False, public=True)

 Check the result

if ok:
print('Dashboard created successfully')
else:
print(res)
sys.exit(1)
Show more







 Delete a dashboard 

You can use Python to delete a dashboard.

You must know the ID of a dashboard to delete it.





 Delete a dashboard from the default team 

The following code shows the structure of a Python script to delete a dashboard from the default team.

!/usr/bin/env python3

import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import IbmAuthHelper, SdMonitorClient

 Parse arguments.
def usage():
See [Using community buildpacks](https://cloud.ibm.com/docs/cloud-foundry-public?topic=cloud-foundry-public-using_buildpacks) for details.



Optional: If you change your app, you must upload those changes by entering the cf push command again. The ibmcloud cf command line interface uses your previous options and your responses to the prompts to update any running instances of your app with the new bits of code.



 Notes: 



* When you use the ibmcloud cf push command, the ibmcloud cf command line interface copies all of the files and directories from your current directory to IBM Cloud. Ensure that you have only the required files in your app directory.
* Ensure that your organization has enough memory for all instances of your app. To view the memory quota for your org, use ibmcloud cf org <org_name>.
* For more information about ibmcloud cf push, see [Pushing an app](https://docs.cloudfoundry.org/devguide/deploy-apps/deploy-app.html).











 Migrating your data and using services 

After you upload your app to IBM Cloud, select the service that your app is connected to from the IBM Cloud catalog, create a service instance, bind the instance to your app, and then restart your app.

The VCAP_SERVICES environment variable of your app is a JSON object that contains information about how to interact with a service instance in IBM Cloud. The information includes the service instance name, credentials, and the connection URL to the service instance.

To run your code in IBM Cloud, you must add the code logic for parsing the VCAP_SERVICES variable to obtain information about service connection. Modify your app to get the dynamically assigned host and port of the service instance through the environment variables. The following example shows how to get the credentials of a Postgre SQL service instance in a Ruby app:

services = JSON.parse(ENV['VCAP_SERVICES'], :symbolize_names => true)
url = services.values.map do srvs
srvs.map do srv
if srv[:credentials] = /^postgres/
In this section, you are going to set up the needed services and deploy the backend app. All of this can be accomplished from the command line interface (CLI) in a terminal.



1. Clone the [GitHub repository](https://github.com/IBM-Cloud/slack-chatbot-database-watson) and navigate into the cloned directory:

git clone https://github.com/IBM-Cloud/slack-chatbot-database-watson
cd slack-chatbot-database-watson
2. If you are not logged in, use ibmcloud login or ibmcloud login --sso to log in interactively. Make sure to target the region and the resource group to work with. It is used to create the services and actions. You can list your available resource groups using ibmcloud resource groups.

ibmcloud target -r <region> -g <resource group>

Use ibmcloud target -g default to switch to the default resource group.
3. Create a [Db2 on Cloud](https://cloud.ibm.com/docs/Db2onCloud) instance and name it eventDB. Adapt the region us-south to your region, e.g., eu-de.

ibmcloud resource service-instance-create eventDB dashdb-for-transactions free us-south

You can also use another than the free (lite) plan. The free plan is not available in all locations. Wait for the service to be provisioned. See the output from the above command for instructions. You can also run the next command to check the status:

ibmcloud resource service-instance eventDB
4. To access the database service from the Code Engine app later on, it needs the proper authorization. Thus, you create service credentials and label them slackbotkey:

ibmcloud resource service-key-create slackbotkey Manager --instance-name eventDB --output json > slackbotkey.json

The output is stored in the file slackbotkey.json and needed at a later step. Run this command to extract the database connection URL:
,"logging.googleapis.com/labels":{},"logging.googleapis.com/sourceLocation":{"file":"github.com/tektoncd/pipeline/vendor/knative.dev/pkg/logging/config.go","line":"115","function":"github.com/tektoncd/pipeline/vendor/knative.dev/pkg/logging.newLoggerFromConfig"}}
{"severity":"INFO","timestamp":"2021-02-23T14:12:05.655937558Z","caller":"logging/config.go:116","message":"Logging level set to: info","logging.googleapis.com/labels":{},"logging.googleapis.com/sourceLocation":{"file":"github.com/tektoncd/pipeline/vendor/knative.dev/pkg/logging/config.go","line":"116","function":"github.com/tektoncd/pipeline/vendor/knative.dev/pkg/logging.newLoggerFromConfig"}}







 ibmcloud ce buildrun submit 

Submit a build run.

ibmcloud ce buildrun submit (--build BUILD_NAME [--name NAME]) | (--name NAME [--commit COMMIT] [--context-dir CONTEXT_DIR] [--dockerfile DOCKERFILE] [--git-repo-secret GIT_REPO_SECRET] [--registry-secret REGISTRY_SECRET] [--size SIZE] [--strategy STRATEGY]) [--image IMAGE] [--no-wait] [--output OUTPUT] [--quiet] [--service-account SERVICE_ACCOUNT] [--source SOURCE] [--timeout TIMEOUT] [--wait] [--wait-timeout WAIT_TIMEOUT]



 Command Options 

--build, -b, --bd
[get-training-data](https://cloud.ibm.com/apidocs/discoveryget-training-data) Action username, password, iam_access_token, iam_apikey, iam_url, headers, headers[X-Watson-Learning-Opt-Out], url, environment_id, collection_id, query_id Get details about a query. 
 [get-training-example](https://cloud.ibm.com/apidocs/discoveryget-training-example) Action username, password, iam_access_token, iam_apikey, iam_url, headers, headers[X-Watson-Learning-Opt-Out], url, environment_id, collection_id, query_id, example_id Get details for a training data example. 
 [list-training-data](https://cloud.ibm.com/apidocs/discoverylist-training-data) Action username, password, iam_access_token, iam_apikey, iam_url, headers, headers[X-Watson-Learning-Opt-Out], url, environment_id, collection_id List training data. 
 [list-training-examples](https://cloud.ibm.com/apidocs/discoverylist-training-examples) Action username, password, iam_access_token, iam_apikey, iam_url, headers, headers[X-Watson-Learning-Opt-Out], url, environment_id, collection_id, query_id List examples for a training data query. 
 [update-training-example](https://cloud.ibm.com/apidocs/discoveryupdate-training-example) Action username, password, iam_access_token, iam_apikey, iam_url, headers, headers[X-Watson-Learning-Opt-Out], url, environment_id, collection_id, query_id, example_id, cross_reference, relevance Change a label or cross reference for an example.