vania-janet/multiturn-rag-retrieval-data huggingface dataset:
You will only see information about transfers that are completed in the current session.

Preferences: You can set the default for uploads and downloads to High speed.

Downloads that use Aspera high-speed transfer incur egress charges. For more information, see the [pricing page](https://cloud.ibm.com/objectstorage/createpricing).

Advanced Preferences: You can set bandwidth for uploads and downloads.







 Using the Aspera Transfer SDK 



1. [Download the Aspera Transfer SDK from the IBM API Hub.](https://developer.ibm.com/apis/catalog/aspera--aspera-transfer-sdk/Introduction) The SDK is a collection of binaries (command line utilities and a daemon to listen for transfer requests), configuration files, and language specific connectors.
2. Install the grpc dependencies from the appropriate package manager (pip, maven, gem, etc).
3. Launch the daemon and import the relevant programming language connector files to your project.
4. Instantiate an Aspera client by passing it the local port used by the daemon.
5. Create a transfer_spec containing all the information needed for the transfer:



1. icos information:



1. API key
2. Service instance ID
3. Target endpoint
4. Bucket name
5. Transfer direction
6. Remote host (you find this by sending a GET request to a bucket with a ?faspConnectionInfo query parameter)
7. Assets for transfer (basically a set of file paths)





6. Pass the transfer specification and configuration info to a transfer request.



The following is an example using Python:

import random
import string

import grpc
import json
import os.path

from urllib3.connectionpool import xrange

import transfer_pb2 as transfer_manager
import transfer_pb2_grpc as transfer_manager_grpc

def run():
 create a connection to the transfer manager daemon
FAQs for DevOps Insights 

Get answers to frequently asked questions about using DevOps Insights.



 Can DevOps Insights be installed and run in an on-premises environment? 

DevOps Insights isn't available for an on-premises environment. It's only available in IBM Public Cloud.





 My Jenkins or Travis CI is running in an on-premises environment or in another public cloud. Can I still publish build records, test records, deployment records to DevOps Insights, and use quality gates? 

Yes, it doesn't matter from where your pipeline tool is running.





 I am deploying my applications to on-premises environments or other public clouds. Can I publish build records, test records, deployment records to DevOps Insights, and use quality gates for these applications? 

Yes, it doesn't matter where your applications are deployed.





 Why can't I see my app on the Quality Dashboard page? 

For an application to show up in the Quality Dashboard page, it must have a build record for the selected branch, and at least one test record for that build. For more information about build and test records, see [Integrating your Continuous Delivery](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-setting-values-cli).

You can find published build records on the Build Frequency page. For more information, see [Viewing the build frequency](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-publish-build-clibuild-frequency-cli).





 How do I delete DevOps Insights? 

Deleting your tool integration deletes all data that is associated with that toolchain. For more information about how to delete your DevOps Insights instance, see [Deleting a DevOps Insights tool integration](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-deleting_data).





 Can I delete build, test, and deploy data from DevOps Insights? 

You can delete data sets from for toolchain, environment, application, and for a branch.
* I am deploying my applications to on-premises environments or other public clouds. Can I publish build records, test records, deployment records to DevOps Insights, and use quality gates for these applications?

Yes, it doesn't matter where your applications are deployed.
* Why can't I see my app on the Quality Dashboard page?

For an application to show up in the Quality Dashboard page, it must have a build record for the selected branch, and at least one test record for that build. For more information about build and test records, see [Integrating your Continuous Delivery](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-setting-values-cli).

You can find published build records on the Build Frequency page. For more information, see [Viewing the build frequency](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-publish-build-clibuild-frequency-cli).
* How do I delete DevOps Insights?

Deleting your tool integration deletes all data that is associated with that toolchain. For more information about how to delete your DevOps Insights instance, see [Deleting a DevOps Insights tool integration](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-deleting_data).
* Can I delete build, test, and deploy data from DevOps Insights?

You can delete data sets from for toolchain, environment, application, and for a branch. For more information about how to delete a specific data set, see [Deleting DevOps Insights data sets](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-deleting_data).
* Can I manage DevOps Insights by using Terraform or APIs?

You can use Terraform or APIs to add DevOps Insights to a toolchain or to remove it from a toolchain.
Frequently asked questions (FAQs) 

Frequently asked questions about IBM Log Analysis.



 Where can I find the list of Cloud services that send logs? 

You can find the list of Cloud services that generate logs in the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-cloud_services).





 Where can I find details about the logs that a Cloud service generates? 

You can access more information about the logs that each Cloud service generates from the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-cloud_services). For each Cloud service, you can link to the logging topic specific to a Cloud service where you can get more information.





 Where can I find the logs that a Cloud service generates? 

First, you must [check whether you have enabled Platform logs in the location where your Cloud service is available](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-config_svc_logs).

Then, check the [Cloud services by location](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-cloud_services_locations) documentation to find out the location where the logs are available for analysis.





 How do I configure archiving for my instance? 

To configure archiving see [Archiving events to IBM Cloud Object Storage](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-archiving).





 How do I access the logging web UI? I cannot see the logging instance in the Observability page. 

To lauch the logging web UI, complete the following steps:



1. Check that you have IAM permissions to view the logging instances through the Logging dashboard. You must have viewer platform role permissions for the IBM Log Analysis service. [Learn more about managing IAM policies and access groups](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-work_iam).
2.
* What are the different ways to manage auditing events?

Activity Tracker offers 2 different ways to manage auditing events in an IBM Cloud account. You can use Activity Tracker hosted event search, an IAM enabled service, to manage auditing events through instances that you provision in each IBM Cloud region where you operate. Alternatively, you can use Activity Tracker Event Routing, a platform service, to manage auditing events at the account-level by configuring targets and routes that define where auditing data is routed.

Activity Tracker hosted event search routes location-based auditing events to an Activity Tracker instance in the region where they are generated and routes global auditing events to the Activity Tracker instance that is provisioned in Frankfurt.

Activity Tracker Event Routing routes events based on the location that is specified in the logSourceCRN field included in the event. You can define a target, the resource where events are routed to, in any Activity Tracker Event Routing supported region. However, the target resource can be located in any region where that type of target is supported, in the same account or in a different account. You can define rules to determine where auditing events are to be routed by configuring 1 or more routes in the account. You can define rules for managing global events and location-based events that are generated in regions where Activity Tracker Event Routing is supported.
* Where can I find the list of Cloud services that generate Activity Tracker events?

You can find information about the Cloud services that generate audit events and send those to Activity Tracker in the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-cloud_services).
* Where can I find the actions that a service generates?

You can find links to the list of events that each Cloud service generates in the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-cloud_services).
* Where can I find the events that a Cloud service generates?
You can find information about the Cloud services that generate audit events and send those to Activity Tracker in the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-cloud_services).





 Where can I find the actions that a service generates? 

You can find links to the list of events that each Cloud service generates in the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-cloud_services).





 Where can I find the events that a Cloud service generates? 

In Activity Tracker Event Routing, you can differentiate events by scope as [global events](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-event_typesevent_types_global) or [location-based events](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-event_typesevent_types_location), and by operational impact as management or data events.

First, you need to check if you need to configure your service, upgrade your plan, or both to be able to collect Activity Tracker events.



* Management events are collected automatically for most Cloud services except Watson services that require a paid plan.

If you are looking for Watson Activity Tracker events, check your plan and make sure you have a service plan that includes them.
* Data events are collected automatically for most services except the following ones:

AppID requires a paid plan and you opting in.

Cloud Object Storage requires that you enable them by bucket.

Cloudant Database requires that you enable them per service instance.



Then, you need to determine the location of the events based on scope.

For Activity Tracker hosted event search, global events are available through the Activity Tracker instance in Frankfurt. Therefore, to view global events, you must provision an instance of the IBM Cloud Activity Tracker service in Frankfurt.

For Activity Tracker Event Routing, global events are collected in the region that you configure. Therefore, to find those events, you must find the route that is configured in 1 region of your account to collect global events.
FAQs 

Frequently asked questions about Activity Tracker.



 What are the different ways to manage auditing events? 

Activity Tracker offers 2 different ways to manage auditing events in an IBM Cloud account. You can use Activity Tracker hosted event search, an IAM enabled service, to manage auditing events through instances that you provision in each IBM Cloud region where you operate. Alternatively, you can use Activity Tracker Event Routing, a platform service, to manage auditing events at the account-level by configuring targets and routes that define where auditing data is routed.

Activity Tracker hosted event search routes location-based auditing events to an Activity Tracker instance in the region where they are generated and routes global auditing events to the Activity Tracker instance that is provisioned in Frankfurt.

Activity Tracker Event Routing routes events based on the location that is specified in the logSourceCRN field included in the event. You can define a target, the resource where events are routed to, in any Activity Tracker Event Routing supported region. However, the target resource can be located in any region where that type of target is supported, in the same account or in a different account. You can define rules to determine where auditing events are to be routed by configuring 1 or more routes in the account. You can define rules for managing global events and location-based events that are generated in regions where Activity Tracker Event Routing is supported.





 Where can I find the list of Cloud services that generate Activity Tracker events? 

You can find information about the Cloud services that generate audit events and send those to Activity Tracker in the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-cloud_services).





 Where can I find the actions that a service generates? 

You can find links to the list of events that each Cloud service generates in the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-cloud_services).





 Where can I find the events that a Cloud service generates?
Migration strategies for IBM Power Systems Virtual Servers 

Learn how to migrate your data and workloads to a IBM® Power Systems™ Virtual Server.



 IBM Cloud Object Storage (COS) 

COS can be used as an intermediary location to store files from your on-premises environment. You can retrieve and send your files to the Power Systems Virtual Server environment from this location. You must create COS buckets to transfer data over the public internet and or privately secured links. For more information, see [IBM Cloud Object Storage: FAQ](https://www.ibm.com/cloud/object-storage/faq).

To copy data from COS to your AIX virtual machine (VM), you must install the [Amazon Web Services (AWS) CLI](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-aws-cli) by using either the Yellowdog Updater, Modified (yum) or Pip installs Python pip (pip) package managers. If you use the yum package manager, you can install the AWS CLI with the yum install aws-cli command. For the pip package manager, you can use pip install awscli to install the AWS CLI. After the installation, you can use the universal S3 commands that are supported by AWS to copy objects.

You can also find a script that ships (as a sample command) with AIX 7.2 TL3, or later, that simplifies the installation of yum, pip, and the AWS CLI. You can find the script at: /usr/samples/nim/cloud_setup.



* [Uses of AIX toolbox for open source software](https://www.ibm.com/support/pages/node/882892)
* [Python for AIX](https://www.ibm.com/support/pages/node/883796)
You can find information about the Cloud services that generate audit events and send those to Activity Tracker in the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-cloud_services).
* Where can I find the actions that a service generates?

You can find links to the list of events that each Cloud service generates in the following documentation topic: [Cloud services](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-cloud_services).
* Where can I find the events that a Cloud service generates?

In Activity Tracker Event Routing, you can differentiate events by scope as [global events](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-event_typesevent_types_global) or [location-based events](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-event_typesevent_types_location), and by operational impact as management or data events.

First, you need to check if you need to configure your service, upgrade your plan, or both to be able to collect Activity Tracker events.



* Management events are collected automatically for most Cloud services except Watson services that require a paid plan.

If you are looking for Watson Activity Tracker events, check your plan and make sure you have a service plan that includes them.
* Data events are collected automatically for most services except the following ones:

AppID requires a paid plan and you opting in.

Cloud Object Storage requires that you enable them by bucket.

Cloudant Database requires that you enable them per service instance.



Then, you need to determine the location of the events based on scope.

For Activity Tracker hosted event search, global events are available through the Activity Tracker instance in Frankfurt. Therefore, to view global events, you must provision an instance of the IBM Cloud Activity Tracker service in Frankfurt.

For Activity Tracker Event Routing, global events are collected in the region that you configure. Therefore, to find those events, you must find the route that is configured in 1 region of your account to collect global events.
Discovery v2 home page
* Can I integrate Watson Discovery with Watson Assistant?

You can integrate Discovery and Watson Assistant to make information that is stored in external data sources available to a virtual assistant. Create a Conversational Search project in Discovery, and then add the data sources that you want to make available to it. Next, create a search integration in Watson Assistant, and connect it to your Discovery project and collection.
* Can I increase the collection limit for a project?

If you want to add more than 5 collections to your project and you have a Premium plan, you can request an increase to the collection limit by opening a support request. For more information, see [Getting help](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-get-help).
* Can I find related documents after I add them to a collection

If you want to retain information about the relationship of two or more documents to one another, you can do so. For example, if 3 documents are uploaded from the same folder and their placement in the folder is significant to their meaning, you might want to retain the parent folder information.

When you upload a document, no such information about its relationships to other documents is stored by default. To add the information, you can use the API to add the documents. When you add documents by using the API, you can specify metadata values. You might want to specify a metadata value, such as "foldername": "company_a", for each document.

Alternatively, you can copy the document body of each document into a JSON file, where each document is an object in a single array. When the JSON file is ingested, each item in the array is added as a separate document with a seprarate document ID. Each document shares the same parent ID, which identifies the relationship between them.

You can quickly find documents that share the same parent ID or other common metadata value from the Manage data page. Customize the view to show the field, such as extracted_metadata.parent_document_id or extracted_metadata.foldername, that the documents share in common.
* Can I customize Discovery to understand my data

Yes.
else:
print("'{0}' successfully deleted.n".format(databaseName))
2. Review the basic error handling that was included to illustrate how problems can be caught and addressed.







 Step 7: Closing the connection to the service instance 



1. Disconnect the Python client application from the service instance.
2. Run the disconnect command.

client.disconnect()







 Execute the complete Python script 

This script is the complete Python script for steps 2, 3, and 4. When you run the script, it connects to your service instance, creates the database, stores a small set of data in the database, and creates JSON documents.



1. In the code example in the next step, replace the values for serviceUsername, servicePassword, and serviceURL with the values from your service credentials.

For more information about where to find your credentials, see [Locating your credentials](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-locating-your-service-credentialslocating-your-service-credentials).
2. Copy the following script into a text editor and name it demo.py.

!/usr/bin/env python

 Connect to service instance by running import statements.
from cloudant.client import Cloudant
from cloudant.error import CloudantException
from cloudant.result import Result, ResultByKey

 Add credentials to authenticate to the service instance.
serviceUsername = "apikey-v2-58B528DF5397465BB6673E1B79482A8C"
servicePassword = "680b037145f9dc8ef9e6a6d8b480783cbc1d1c12e71a0f4ced6b1eee30a243cd"
serviceURL = "serviceURL = "https://0c869093-c3ee-4a3f-bcec-00f01c8df8d8-bluemix.cloudantnosqldb.appdomain.cloud""
In a region, you can manage auditing events in IBM Cloud Activity Tracker Event Routing or in IBM Cloud Activity Tracker. Both options are not allowed in parallel in a region. If a route is not defined in a region, by default, IBM Cloud Activity Tracker is the service that you can use to monitor auditing events.
* Can I import archived data into the UI?

Archived data cannot be imported to be searched or used in the IBM Cloud Activity Tracker UI.

Use the [IBM Cloud Data Engine service](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-sqlquery) to query archive data.
* Can I control the ingested events so I can control my service cost?

You can control the volume of events that you ingest. Controlling the events helps you control your Activity Tracker service cost. Events that are filtered out (excluded) are not archived and are not available for search. You do not pay for events that are filtered out.

There are different ways in which you can filter out events sent to Activity Tracker:



* For some Cloud services you can [configure the collection of events.](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-events-opt-in)
* You can define [exclusion rules](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-exclusion_rules) for events before they are stored for search.



* You can drop the events entirely and not see them at all through the UI
* You can view the events in the UI, but you cannot search on them. However, you can define views and alerts based on the data from these logs.



* You can also configure [usage quotas](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-control_usage_quotas) and define conditional usage quota exclusion rules.





Log Analysis



* Where can I find the list of Cloud services that send logs?