vania-janet/multiturn-rag-retrieval-data huggingface dataset:
For more information about using toolchains and tool integrations with the API, see [Creating a toolchain with the API](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-toolchains_getting_started&interface=apicreating_a_toolchain_api), [Deleting a toolchain with the API](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-toolchains-using&interface=apideleting_a_toolchain_api), [Working with tool integrations](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-integrations&interface=api), and the [CD Toolchain API docs](https://cloud.ibm.com/apidocs/toolchain).





 Can I use the console, APIs, or CLI to modify resources that are managed by Terraform? 

When you use Terraform to manage resources such as Continuous Delivery service instances, toolchains, and Tekton pipelines, avoid changing the resources by using the console, APIs, or CLI, or by any other method outside of Terraform's control.

If you circumvent Terraform by directly changing resources, you might cause [resource drift](https://developer.hashicorp.com/terraform/tutorials/state/resource-drift), a situation in which the states of your actual resources on IBM Cloud deviate from the definition of the resources in Terraform. The next time that you apply the Terraform configuration, Terraform attempts to update your resources to align them with the Terraform configuration. This action might lead to unintended consequences, such as reverting changes or deleting and then re-creating resources.

For more information about resource drift, see [Manage Resource Drift](https://developer.hashicorp.com/terraform/tutorials/state/resource-drift).
Watson Discovery supports a wide selection of data sources you can crawl, like webpages, Cloud Object Storage, Microsoft SharePoint, and more. You can even upload your own data from any data source.

After connecting and processing your data, you can apply enrichments to bring your data to life. Some commonly used enrichments are entities, contracts, and table understanding. Entities enrichment can be used to recognize people, organizations, and more. Contracts enrichment can be used to decompose contracts to fields, clauses, and relationships. The table understanding enrichment can be used to identify tables and return them as an answer to a query.

You can also create custom enrichments, such as a dictionary, so Discovery can understand your industry-specific terminology and support intelligent queries.

Now, you know the basics.

To get started, take our step-by-step product tour to get familiar with the user interface and sample project.







 Using Discovery 

Discovery can be deployed as a managed cloud service or can be installed on premises. This documentation describes how to use the product regardless of how it is deployed. Information that applies exclusively to one deployment type is denoted by the appropriate icon:



IBM Cloud Pak for Data

for installed instances, such as IBM Watson® Discovery Cartridge for IBM Cloud®.
IBM Cloud

for managed instances, such as Discovery Plus, Enterprise, and Premium plan instances that are hosted by IBM Cloud or instances that are provisioned with [IBM Cloud Pak for Data as a Service](https://dataplatform.cloud.ibm.com/docs/content/wsj/landings/watsondisc.html).



Click the Help icon ![Help icon](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/help.svg) from the header of any page in the product user interface to open the Discovery documentation.





 Browser support 

IBM Cloud Pak for Data



* The minimum required browser software for the product user interface includes the following browsers:

Google Chrome
: Latest version -1 for your operating system

Mozilla Firefox
The IBM Cloud reusable Terraform modules in the [terraform-ibm-modules](https://github.com/terraform-ibm-modules) GitHub repo and the Terraform registry adhere to these guidelines to support composition.

The use of blueprint templates to compose and deploy large-scale environments from modules is discussed in more detail in these sections:



* [Understanding blueprint templates and configuration](https://cloud.ibm.com/docs/schematics?topic=schematics-blueprint-templates)
* [Using Terraform modules with blueprint templates](https://cloud.ibm.com/docs/schematics?topic=schematics-blueprint-terraform)
* [Infrastructure as code modular architectures](https://cloud.ibm.com/docs/schematics?topic=schematics-iac-bp-modularity)
See [Deploying your application with commands and arguments](https://cloud.ibm.com/docs/codeengine?topic=codeengine-application-workloadsdeploy-app-cmd-args).
* Want to define environment variables for your job? Find out how with [Working with environment variables](https://cloud.ibm.com/docs/codeengine?topic=codeengine-envvar).
* Does your job take advantage of configmaps or secrets? Take a look at [Working with secrets](https://cloud.ibm.com/docs/codeengine?topic=codeengine-secret) or [Working with configmaps](https://cloud.ibm.com/docs/codeengine?topic=codeengine-configmap).
* Do you want to deploy your app in a toolchain? See [Integrating Code Engine workloads with Continuous Delivery](https://cloud.ibm.com/docs/codeengine?topic=codeengine-toolchain-ce).
* Do you want to use Terraform to deploy your app? Find out more with [Setting up Terraform for Code Engine](https://cloud.ibm.com/docs/codeengine?topic=codeengine-terraform-setup-ce).



Code Engine supports Lithops for running jobs. See [running jobs with Lithops framework](https://cloud.ibm.com/docs/codeengine?topic=codeengine-lithops).

Ready to create and run your job?

You can run your job directly or create a job definition and run your job based on that configuration.



* To run a job directly, see [Run a job with the CLI without first creating a job configuration](https://cloud.ibm.com/docs/codeengine?topic=codeengine-run-jobrun-job-cli-withoutjobconfig).
* [Step three: Get started with using the playbooks](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansible-buildansible-build-get-start)
* [Step four: Run a playbook](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansible-buildansible-build-run-playbook)
* [Step five: Run a script](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansible-buildansible-build-run-script)
* [Hints and Tips](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansible-buildansible-build-hints)
* [Next steps](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansible-buildansible-build-next-steps)



* [Generated identities](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansible-buildansible-build-import-identities)



* [Summary](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansible-buildansible-build-summary)







 IBM Blockchain Platform console 

[Migrating to IBM Support for Hyperledger Fabric](https://cloud.ibm.com/docs/blockchain?topic=blockchain-migrating-to-hlf-supportmigrating-to-hlf-support)



* [What is changing?](https://cloud.ibm.com/docs/blockchain?topic=blockchain-migrating-to-hlf-supportibp-migration-what)
* [Why is it changing?](https://cloud.ibm.com/docs/blockchain?topic=blockchain-migrating-to-hlf-supportibp-migration-why)
* [How do I transition my license?]
For more information about using Terraform with Tekton pipelines, see [Working with Tekton pipelines](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-tekton-pipelines), and the [ibm_cd_tekton_pipeline resources documentation](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/cd_tekton_pipeline).

You cannot use Terraform to trigger Tekton pipeline runs, or to manage Tekton pipeline runs and logs. You can perform these tasks with the Tekton pipeline APIs, or by using the console.

You cannot use Terraform to manage Classic pipelines. You can manage Classic pipelines only by using the console.
* Can I manage delivery pipelines by using an API?

You can use HTTP APIs or selected programming language SDKs to provision, update, and de-provision Tekton pipelines, definitions, properties, triggers, and trigger properties. You can also use the APIs and SDKs to trigger Tekton pipeline runs, and to manage Tekton pipeline runs and logs. For more information about using Tekton pipelines with the API, see [Working with Tekton pipelines](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-tekton-pipelines), and the [CD Tekton Pipeline API docs](https://cloud.ibm.com/apidocs/tekton-pipeline).

You cannot use APIs to manage Classic pipelines. You can manage Classic pipelines only by using the console.
* What is the difference between ibm_cd_toolchain_tool_pipeline and ibm_cd_tekton_pipeline in Terraform?

An ibm_cd_tekton_pipeline resource represents a Tekton Pipeline. An ibm_cd_toolchain_tool_pipeline resource represents the tool integration that binds a Tekton Pipeline into a toolchain. In the architecture of the Continuous Delivery toolchain platform, tool integrations are distinct entities from the tools that they represent.
To learn about how to use contextual entites to identify names and locations, see the [Detecting Names And Locations With Watson Assistant](https://medium.com/ibm-watson/detecting-names-and-locations-with-watson-assistant-e3e1fa2a8427) blog post on Medium.







 Creating entities 



1. Open your dialog skill and then click the Entities tab.
2. Click Create entity.

You can also click System entities to select from a list of common entities, provided by IBM, that can be applied to any use case. See [Enabling system entities](https://cloud.ibm.com/docs/assistant?topic=assistant-entitiesentities-enable-system-entities) for more detail.
3. In the Entity name field, type a descriptive name for the entity.

The entity name can contain letters (in Unicode), numbers, underscores, and hyphens. For example:



* @location
* @menu_item
* @product



Do not include spaces in the name. The name cannot be longer than 64 characters. Do not begin the name with the string sys- because it is reserved for system entities.

The at sign @ is prepended to the entity name automatically to identify the term as an entity. You do not need to add it.
4. Click Create entity.

![Screen capture of creating an entity](https://cloud.ibm.com/docs-content/v1/content/417db917d18067d9dcc4cf2612564a9a87c3ddc8/assistant/images/create-entity.png)
5. For this entity, choose whether you want your assistant to use a dictionary-based or annotation-based approach to find mentions of it, and then follow the appropriate procedure.

For each entity that you create, choose one entity type to use only. As soon as you add an annotation for an entity, the contextual model is initialized and becomes the primary approach for analyzing user input to find mentions of that entity.
For more information about the GitLab Terraform provider, see the [GitLab Provider documentation](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs).





 Can I manage GitHub, GitLab, and Git Repos and Issue Tracking by using an API? 

You can use HTTP APIs or selected programming language SDKs to add GitHub, GitLab, and Git Repos and Issue Tracking tool integrations to a toolchain, to update those tool integrations, or to remove those tool integrations from a toolchain. For more information about working with the GitHub, GitLab, and Git Repos and Issue Tracking tool integrations, see [Working with tool integrations](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-integrations) and [Creating toolchains with Git](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-toolchains_git).

You might be able to use APIs to work directly with some repos. For more information about the GitHub API, see [REST API](https://docs.github.com/en/rest). For more information about the GitLab API, see [REST API](https://docs.gitlab.com/ee/api).





 What is the difference between the initialization block and the parameters block in Git tool integration Terraform resources? 

In a Git tool integration Terraform resource, the initialization block consists of arguments that control how the tool integration prepares and binds itself to a specific target repo. If you change any of the arguments in the initialization block, Terraform deletes the tool integration from the toolchain and creates a replacement tool integration. All of the arguments in the initialization block are annotated with the Terraform behavior Forces new resource.

By contrast, arguments in the parameters block influence how the tool integration works after it is initialized. If an argument is not annotated with Forces new resource and you change the argument, Terraform applies the change to the existing tool integration.
* Review information about the script's use of the MinIO client. The client is used for the for MinIO commands.



* The scripts download the client from the [MinIO website](https://dl.min.io/) if the client isn't installed.
* If you want the script to use your installed version, verify that you can run the client by issuing the command mc on the command line.









 all-backup-command script 

The all-backup-command script.sh script backs up or restores the MongoDB, PostgreSQL, Minio databases, and PVC.

Unless you need to back up a single database, use the all-backup-command script, which deactivates and reactivates Knowledge Studio.

The script backs up or restores the data in the following order:



1. MongoDB
2. PostgreSQL
3. MinIO
4. PVC



all-backup-restore.sh backup | restore RELEASE_NAME BACKUP_DIR -n NAMESPACE

Use either the backup or restore command.

backup
: Backs up each database to a subdirectory of the `BACKUP_DIR` directory.

restore
: Restores the data from each database in the `BACKUP_DIR` directory.



 Arguments and options 

RELEASE_NAME
: The release name that was specified when the Knowledge Studio Helm chart was installed in your cluster. Required.
: For version 1.1.2, the value is `wks`.

BACKUP_DIR
: The base directory of each database where backups are stored to or restored from. Each database is stored in a subdirectory of the backup directory (`mongodb`, `postgresql`, `minio`, or `pvc`). Required.

-n NAMESPACE
: Namespace for the pods.
: The default value is `zen`.





 Output 

The script returns the following output:

[SUCCESS] MongoDB,PostgreSQL,Minio,PVC

and indicates either the backup or restore command.
* [What's new in the new lifecycle](https://cloud.ibm.com/docs/blockchain?topic=blockchain-write-powerful-smart-contractswrite-powerful-smart-contracts-new-vs-old)
* [Packages and definitions](https://cloud.ibm.com/docs/blockchain?topic=blockchain-write-powerful-smart-contractswrite-powerful-smart-contracts-packages-definitions)



* [Taking advantage of the ability for each organization to use a separate package](https://cloud.ibm.com/docs/blockchain?topic=blockchain-write-powerful-smart-contractswrite-powerful-smart-contracts-packages-definitions-separate)



* [Updating smart contracts](https://cloud.ibm.com/docs/blockchain?topic=blockchain-write-powerful-smart-contractswrite-powerful-smart-contracts-updating)
* [Putting it all together](https://cloud.ibm.com/docs/blockchain?topic=blockchain-write-powerful-smart-contractswrite-powerful-smart-contracts-summary)
* [Smart contract development tooling](https://cloud.ibm.com/docs/blockchain?topic=blockchain-write-powerful-smart-contractswrite-powerful-smart-contracts-tooling)
* [Installing a package and proposing a definition](https://cloud.ibm.com/docs/blockchain?topic=blockchain-write-powerful-smart-contractswrite-powerful-smart-contracts-installing-proposing)



[Best practices for application development](https://cloud.ibm.com/docs/blockchain?topic=blockchain-best-practices-appbest-practices-app)





 Advanced tutorials 

[Creating an intermediate Certificate Authority (CA)](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-icaibp-ica)



* [Why would I want to use an intermediate CA with my IBM Blockchain Platform network?]
Prerequisites 

Before using the playbook, you need to complete the following steps:



* [Link an instance of the IBM Blockchain Platform](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-deploy-iks-icibp-v2-deploy-iks-create-service-instance) to your Kubernetes cluster on IBM Cloud.
* Review the topic on [Getting started with Ansible playbooks on the IBM Blockchain Platform](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansible).
* Install [Docker](https://docs.docker.com/get-docker/).







 Step one: Gather console connection information 

After you deploy an IBM Blockchain Platform service instance, you need to gather the connection information for the console.

When your cluster is running in IBM Cloud, you need to create service credentials for the IBM Blockchain Platform service instance. The Ansible playbooks use the IBM Blockchain Platform REST APIs to interact with the console. The REST APIs use the API key (api_key) and API endpoint (api_endpoint) to authenticate with the server and retrieve a Cloud Identity and Access Management (IAM) access token. The values of the api_endpoint and api_key for the console can be found in the generated service credentials.

Follow instructions on [retrieving service credentials](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-apisibp-v2-apis-retrieve-service-credentials) in IBM Cloud to gather these values now that are used with the playbooks later in this topic.





 Step two: Clone the Ansible collection 

The Ansible collection source code is available in a GitHub repository. You need to clone the Ansible collection to your local system so that you can run the playbooks and customize them according to your use case. From the command line on your local system run the command:
For more information about using Tekton pipelines with the API, see [Working with Tekton pipelines](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-tekton-pipelines), and the [CD Tekton Pipeline API docs](https://cloud.ibm.com/apidocs/tekton-pipeline).

You cannot use APIs to manage Classic pipelines. You can manage Classic pipelines only by using the console.





 What is the difference between ibm_cd_toolchain_tool_pipeline and ibm_cd_tekton_pipeline in Terraform? 

An ibm_cd_tekton_pipeline resource represents a Tekton Pipeline. An ibm_cd_toolchain_tool_pipeline resource represents the tool integration that binds a Tekton Pipeline into a toolchain. In the architecture of the Continuous Delivery toolchain platform, tool integrations are distinct entities from the tools that they represent. A tool integration is owned by a toolchain, refers to a tool, and manages how the tool contributes its capabilities to the toolchain.

To construct a toolchain that includes a Tekton pipeline, you must declare three resources:



* An [ibm_cd_toolchain resource](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/cd_toolchain) that is the toolchain.
* An [ibm_cd_toolchain_tool_pipeline resource](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/cd_toolchain_tool_pipeline) that is the tool integration that binds the Tekton pipeline into the toolchain.
* An [ibm_cd_tekton_pipeline resource](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/cd_tekton_pipeline) that is the Tekton pipeline.



The following example shows a toolchain that is constructed by declaring these resources.

data "ibm_resource_group" "group" {
name = "default"
* [Part Three: Register the client with the HSM server](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-hsm-gemaltoibp-hsm-gemalto-part-three)
* [Part Four: Build a Docker image](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-hsm-gemaltoibp-hsm-gemalto-part-four)
* [What's next](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-hsm-gemaltoibp-hsm-gemalto-next-steps)









 Ansible Playbooks 

[Getting started with Ansible playbooks on the IBM Blockchain Platform](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansibleansible)



* [What is Ansible](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansibleansible-whatis)
* [How does the IBM Blockchain Platform Ansible collection work?](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansibleansible-how)
* [Why would I want to use an Ansible playbook?](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansibleansible-why)
* [Considerations and Limitations](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansibleansible-limits)
* [Getting started](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansibleansible-getting-started)
* [Next steps](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ansibleansible-next-steps)
By using a template, you can quickly add toolchains in the future. If you’re creating only a few toolchains and a standard template exists that is similar to what you need, use that template and customize your toolchains after they are created.
* I created a toolchain template that specified a particular tag and then created toolchains from that template. If I edit my template to change the tag are my toolchains updated with that tag?

Because the template doesn't link to the toolchains that were created from it, toolchains that were created from the original template are not updated with the new tag.
* Why does the Toolchains page show that a Continuous Delivery service is required?

Toolchains in a resource group must be accompanied by an instance of the Continuous Delivery service in the same resource group. The error message indicates that the resource group does not contain an instance of the Continuous Delivery service. To resolve this error, create an instance of the Continuous Delivery service in the resource group. For more information about this requirement, see [Plan limitations and usage](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-limitations_usage).
* How do I find my toolchain ID?

You can find your toolchain ID in the URL of your selected toolchain tool. For more information, see [Identifying your toolchain ID](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-aggregating-multiple-sources).
* Can I manage toolchains by using Terraform?

You can use Terraform to create, read, update, and delete toolchains and tool integrations.
The from and size properties are not supported in the request body although they are supported in the open source Livy REST interface.

The response body for listing all submitted Spark batch jobs:



Table 5. Response body for listing all submitted batch jobs

 Name Description Type 

 from The start index of the Spark batch jobs that are retrieved int 
 total The total number of batch jobs that are retireved int 
 sessions The details for each batch job in a session list 



For example:

curl -i -X GET https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/43f79a18-768c-44c9-b9c2-b19ec78771bf/livy/batches -H 'content-type: application/json' -H "Authorization: Bearer $TOKEN"

Sample response:

{
"from": 0,
"sessions": [{
"id": 13,
"appId": "app-20201203115111-0000",
"appInfo": {
"sparkUiUrl": null
},
"state": "success"
},
{
"id": 14,
"appId": "app-20201213175030-0000",
"appInfo": {
"sparkUiUrl": null
},
"state": "success"
}],
"total": 2
}
Show more

The SparkUiUrl property in response will have a non-null value when the UI is available for serverless Spark instance.





 Delete a job 

To delete a submitted batch job enter:

curl -H 'Authorization: Bearer <TOKEN>' -H 'Content-Type: application/json' -X DELETE https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance-id>/livy/batches/<batch-id>

For example: