vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Quick start watsonx.data console 

When you log in to the IBM® watsonx.data web console for the first time, you are presented with the quick start wizard. In this tutorial, you learn how to use the quick start wizard to configure the core components and get started with watsonx.data in few minutes.

The wizard guides you through the initial configuration process for the infrastructure components of watsonx.data.



 Step 1: Configure a bucket 

Your watsonx.data needs an object storage bucket to store your raw data files. You can provision a new IBM-managed bucket or register your own bucket. You can add more buckets and register them later.

In the Bucket configuration page, complete the following steps:



1. Select one of the following options and provide details.



* Provision new IBM-managed bucket
* Register my own



Table 1. Add bucket

 Field Description 

 Type Select from Amazon S3 or IBM Cloud Object Storage. 
 Name Enter your bucket name. 
 Endpoint Enter the endpoint URL. 
 Access key Enter your access key. 
 Secret key Enter your secret key. 





2. Click Next.



The IBM-managed bucket is a default 10 GB bucket. It is meant for an exploratory purpose and cannot be used to store production or sensitive data. The watsonx.data instance administrators can disable this bucket for compliance reasons.

When you register your own bucket, ensure to provide the correct details for bucket configuration. Quick start wizard does not validate the bucket configuration details and you cannot modify them later.





 Step 2: Configure a catalog 

Your watsonx.data needs metadata catalogs to manage your table schemas.

In the Catalogs configuration page, complete the following steps:



1. Select the table format for managing your data. Apache Hive and Apache Iceberg catalogs are selected by default.
2. Click Next.







 Step 3: Configure an engine 

Your watsonx.data needs a query engine to work with your data.

In the Engine configuration page, complete the following steps:



1. Select the engine to run and process the data that you attached.
The Discovery v1 document ID can be maintained in Discovery v2. For more information about how to retain the document ID, see [Retaining document IDs](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-migrate-to-v2migrate-to-v2-keep-docid).









 Transferring relevancy training 

Relevancy training that was done in Discovery v1 can be transferred to Discovery v2. Transferring the training works best with a Discovery v2 project that has one collection that contains the same documents from the Discovery v1 collection.

Even if collections were added or documents changed, the relevancy training can be transferred. However, you must update the training to account for the changes.

To transfer relevancy training, complete the following steps:



1. Load the documents in Discovery v2.
2. Programmatically download the queries that were used for relevancy training in Discovery v1. For more information, see [List training data](https://cloud.ibm.com/apidocs/discoverylisttrainingdata).
3. Programmatically re-create the relevancy training data in Discovery v2. Add each training query separately by using the Create a query method. For more information, see [Create a training query](https://cloud.ibm.com/apidocs/discovery-datacreatetrainingquery).

Be sure to specify the v2 collection ID. You must also specify the document ID also.

If you did not [retain the document IDs](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-migrate-to-v2migrate-to-v2-keep-docid) between the v1 and v2 collections, then you must find the v2 document ID that corresponds to the v1 document ID that is referenced in the downloaded query example.







 Transferring models 

You can reuse some of the models that you created in v1 with your v2 project.

Smart Document Understading (SDU) models
: You can import an SDU model that was built with Discovery v1 into Discovery v2.
By default, the service automatically attempts to complete the field with the word's spelling. But the service cannot generate a pronunciation for all words. After adding or modifying words, you must validate the words resource to ensure that each word's definition is complete and valid. For more information, see [Validating a words resource for previous-generation models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-corporaWordsvalidateModel).

You can provide as many as five alternative pronunciations for a word that is difficult to pronounce or that can be pronounced in different ways. Consider using the field to



* Provide different pronunciations for acronyms. For example, the acronym NCAA can be pronounced as it is spelled or colloquially as N. C. double A. The following example adds both of these sounds-like pronunciations for the word NCAA:

IBM Cloud

curl -X PUT -u "apikey:{apikey}" --header "Content-Type: application/json" --data "{"sounds_like": ["N. C. A. A.", "N. C. double A."]}" "{url}/v1/customizations/{customization_id}/words/NCAA"

IBM Cloud Pak for Data

curl -X PUT --header "Authorization: Bearer {token}" --header "Content-Type: application/json" --data "{"sounds_like": ["N. C. A. A.", "N. C. double A."]}" "{url}/v1/customizations/{customization_id}/words/NCAA"
* Handle foreign words. For example, the French word garçon contains a character that is not found in the English language. You can specify a sounds-like of gaarson, replacing ç with s, to tell the service how English speakers would pronounce the word.



The following sections provide language-specific guidelines for specifying a sounds-like pronunciation. Speech recognition uses statistical algorithms to analyze audio, so adding a word does not guarantee that the service transcodes it with complete accuracy.
In addition, consider the following information:



* You must have the manager role to configure archiving in the IBM Cloud Activity Tracker instance. This role includes the logdna.dashboard.manage IAM action role that allows a user to perform admin tasks such as configure archiving.
* When you configure archiving, the Activity Tracker instance and the IBM Cloud Object Storage (COS) instance must be provisioned in the same account.
* The credential that Activity Tracker uses to write data into a COS bucket must have writer role.







 Monitor archiving 

To monitor archiving, you can use the following services:



* IBM Cloud Monitoring service:

IBM Cloud Object Storage is integrated with the Monitoring service. Monitoring provides a default template that you can customize to monitor the bucket that you configure to store data for long term.

For more information, see [Monitoring archiving by using IBM Cloud Monitoring](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-archiving-monitor).
* IBM Cloud Activity Tracker:

Archiving generates Activity Tracker events with the action logdnaat.archiving.send to notify of failures sending data to Event Streams. There are different reasons for failure such as invalid credentials and topic deleted.

For more information, see [Configuring an alert to monitor archiving](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-archiving-at-monitor).







 Viewing archived events by using the SQL Query service 

Data Engine provides a serverless, no-ETL solution to easily query data stored in COS. [Learn more](https://cloud.ibm.com/docs/sql-query?topic=sql-query-overview).

You can use this service to analyze data from archived files in COS.

Once you have SQL Query running on IBM Cloud, you can immediately start querying your data using the SQL Query user interface, programmatically by using either the REST API or the Python ibmcloudsql library, or write a serverless function by using Cloud Functions.

When you query events:



* You must provision an instance of the Data Engine service.
Show more

Replace the following values:



* organization_name: Specify any name to be used to identify this MSP definition in the console.
* organization_id: Specify an ID that is used to represent this MSP internally in the console.
* root_certs: Paste in an array that contains one or more root certificates from the external CA in base64 format. You must provide either a CA root certificate or an intermediate CA certificate. You can also provide both. This is certificates 1 and 3 in the MSP certificates table.
* admins: Paste in the signing certificate of the organization admin identity in base64 format. This is certificate 5 in the MSP certificates table.
* tls_root_certs: Paste in an array that contains one or more root certificates from the TLS CA in base64 format. You must provide either an external TLS CA root certificate or an external intermediate TLS CA certificate, you can also provide both. This is certificate 2 in the MSP certificates table.
* ou_root_cert: Specify the trusted CA root certificate for each role. Typically this value would be the same as the root_certs. This is certificate 1 in the MSP certificates table.
* host_url: Specify the URL of the blockchain console where this MSP collects signatures. This is the URL from your browser minus /nodes. For example, from the Nodes tab, the browser URL is similar to https://d0ddb7-optools.uss02.blockchain.cloud.ibm.com/nodes. Therefore the host_url would be https://d0ddb7-optools.uss02.blockchain.cloud.ibm.com.
* fabric_node_OUs: Fabric-specific Organization Units (OUs) that enable identity classification. NodeOUs contain information for how to distinguish clients, peers, and orderers based on their OU. If the check is enforced, by setting Enabled to true, the MSP considers an identity valid only if it is an identity of type client, a peer, an admin, or an orderer.
6. We will now import the OpenAPI weather definition. In the "Import OpenAPI (Swagger)" dialog box that opens, enter the following URL: https://raw.githubusercontent.com/IBM-Bluemix-Docs/apiconnect/master/tutorials/weather-provider-api_1.yaml. Leave the other options with their default values and click Import.
![import-2.png](https://cloud.ibm.com/docs-content/v1/content/78bb71851d95d7580503eb9ba10cf3ae31490ade/apiconnect/tutorials/images/import-2.png)
7. After you import the OpenAPI spec, you are taken to the Design view of the API. Here you can view various sections of the OpenAPI definition. You can also view the OpenAPI under the Source tab.
8. Your API is saved.







 Test your API proxy 



 Test with the API Manager test tool 



1. In the Assemble tab, click the icon for more actions, then select Generate a default product.
![generate-default-product-3.png](https://cloud.ibm.com/docs-content/v1/content/78bb71851d95d7580503eb9ba10cf3ae31490ade/apiconnect/tutorials/images/generate-default-product-3.png)
2. Accept the default options in the New Product dialog box, and click Create Product. The Weather Provider API product is created and published to the Sandbox catalog. A message indicating successful product generation is displayed.
![generate-default-product-2.png](https://cloud.ibm.com/docs-content/v1/content/78bb71851d95d7580503eb9ba10cf3ae31490ade/apiconnect/tutorials/images/generate-default-product-2.png)
--output or -o Optional Returns the command-line output in JSON format. Currently only JSON file format is supported. 
 --json or -j Deprecated Prints the output in the JSON format. 



Example

ibmcloud schematics plan --id myworkspace-a1aa1a1a-a11a-11 --json







 Resource query commands 

Dynamically build resource inventories by using resource queries. Resource queries help you to retrieve your target hosts from existing IBM Cloud Schematics Workspaces. For more information about resource queries and conditions, see [Creating resource inventories for Schematics Actions](https://cloud.ibm.com/docs/schematics?topic=schematics-inventories-setup).



 ibmcloud schematics resource query create 

Create a resource query in IBM Cloud Schematics that you can use to build your resource inventory. You can create a resource query by using a payload file or the command's interactive mode. You can create resource conditions by using [resource queries](https://cloud.ibm.com/docs/schematics?topic=schematics-inventories-setupsupported-queries).

Syntax

ibmcloud schematics resource-query create --name RESOURCE_QUERY_NAME [--type RESOURCE_QUERY_TYPE] [--query-file QUERY_FILE_PATH] [--file FILE_NAME ] [--output OUTPUT] [--no-prompt]

Command options



Schematics resource query create flags

 Flag Required / Optional Description 

 --name or -n Required The unique name for a resource query. 
 --type or -t Optional The type of resource that you want to retrieve. Supported values are vsi. 
 --query-file or -f Optional The path to the JSON file where you specified the details of your resource query. To find a list of supported queries, see [Supported resource queries](https://cloud.ibm.com/docs/schematics?topic=schematics-inventories-setupsupported-queries).
"root_certs": [
"<root_certs>"
],
"tls_root_certs": [
"<tls_root_certs>"
],
"fabric_node_ous": {
"enable": true,
"admin_ou_identifier": {
"certificate": "<ou_root_cert>",
"organizational_unit_identifier": "admin"
},
"client_ou_identifier": {
"certificate": "<ou_root_cert>",
"organizational_unit_identifier": "client"
},
"orderer_ou_identifier": {
"certificate": "<ou_root_cert>",
"organizational_unit_identifier": "orderer"
},
"peer_ou_identifier": {
"certificate": "<ou_root_cert>",
"organizational_unit_identifier": "peer"
}
},
"host_url": "<url>",
}
Show more



* organization_name: Specify any name to be used to identify this MSP definition in the console.
* organization_id: Specify an ID that is used to represent this MSP internally in the console.
* root_certs: Paste in an array that contains one or more root certificates from the external CA in base64 format. You must provide either a CA root certificate or an intermediate CA certificate. You can also provide both.
* admins: Paste in the signing certificate of the organization admin in base64 format.
* tls_root_certs: Paste in an array that contains one or more root certificates from the TLS CA in base64 format. You must provide either an external TLS CA root certificate or an external intermediate TLS CA certificate, you can also provide both.
* ou_root_cert: Specify the trusted CA root certificate for each role.
Creating and managing custom entries 

Once a custom model exists, the next step is to add custom entries in the form of word/translation pairs to define how specified words are to be pronounced during synthesis. The definitions override the service's default regular pronunciation rules. You can add and query translations for one or more words at a time, and you can delete individual words that you no longer need. Once you are familiar with the customization interface, manipulating multiple words at once can be more convenient than working on a word-by-word basis. You must use credentials for the instance of the service that owns a custom model to use any method that requires its customization ID.

For more information about the rules and limits that apply to custom entries, see [Rules for creating custom entries](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-rules).



 Adding a single word to a custom model 

To add a single word/translation pair to a custom model, use the PUT /v1/customizations/{customization_id}/words/{word} method. You specify the word to be added in the URL of the method. You provide the translation for the word as a JSON object with a single translation attribute. Adding a new translation for a word that already exists in a model overwrites the word's existing translation.

You can provide a translation by using the sounds-like or the phonetic method (or a combination of the two). For phonetic translations, you can use either the International Phonetic Alphabet (IPA) or the IBM Symbolic Phonetic Representation (SPR) format. The following examples use different approaches to add equivalent translations for the word IEEE to a custom model. The required Content-Type header identifies the type of the input as application-json.



* Sounds-like: For this example, the sounds-like method is the simplest approach:

IBM Cloud

curl -X PUT -u "apikey:{apikey}" --header "Content-Type: application/json" --data "{"translation":"I triple E"}" "{url}/v1/customizations/{customization_id}/words/IEEE"

IBM Cloud Pak for Data
3. Check if the service that you want to deploy requires a location parameter to be set in addition to the region parameter. For example, if you use the [ibm_database](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/database) Terraform resource, you must set both the region parameter in the provider block and the location parameter in the ibm_database resource definition.
4. Follow the [steps](https://cloud.ibm.com/docs/schematics?topic=schematics-manage-lifecycledeploy-resources) to deploy your IBM Cloud resources.







 Deploying services across regions 

You can add multiple multiple provider configurations to the provider block to specify the regions where you want to deploy your IBM Cloud resources. For more information, see [Multiple Provider Instances](https://developer.hashicorp.com/terraform/language/providers/configurationalias-multiple-provider-configurations).



1. In the provider block of your Terraform configuration file or the provider.tf file, create multiple provider blocks with the same provider name. The provider configuration without an alias is considered the default provider configuration and is used for every resource where you do not specify a specific provider configuration. If you add more provider configurations, you must include an alias so that you can reference this provider from your resource definition in the Terraform configuration file. In the following example, the default provider configuration deploys resources in us-south while the provider configuration with the alias east deploys all resources in us-east.

provider "ibm" {
region = "us-south"
}

provider "ibm" {
alias = "east"
region = "us-east"
}
2. In your resource definition of your Terraform configuration file, specify the provider configuration that you want to use. If you do not specify a provider, the default provider configuration is used.

resource "ibm_container_cluster" "cluster" {
provider = ibm.east
...
}

resource ibm_is_vpc "vpc" {
name = "myvpc"
}
3.
* admins: Paste in the signing certificate of the organization admin in base64 format.
* tls_root_certs: Paste in an array that contains one or more root certificates from the TLS CA in base64 format. You must provide either an external TLS CA root certificate or an external intermediate TLS CA certificate, you can also provide both.
* ou_root_cert: Specify the trusted CA root certificate for each role. Typically this value would be the same as the root_certs.
* host_url: Specify the URL of the blockchain console where this MSP will collect signatures.
* fabric_node_OUs: Fabric-specific OUs that enable identity classification. NodeOUs contain information for how to distinguish clients, peers, and orderers based on their OU. If the check is enforced, by setting Enabled to true, the MSP considers an identity valid only if it is an identity of type client, a peer, an admin, or an orderer. An identity should have only one of these special OUs, which are assigned to an identity when it is registered with the CA. See this topic for an example of [how to specify the fabric_node_OU in an MSP](https://hyperledger-fabric.readthedocs.io/en/latest/discovery-cli.htmlconfiguration-query) in the Fabric Service Discovery documentation. Or learn more about using [Node OUs](https://hyperledger-fabric.readthedocs.io/en/release-2.2/membership/membership.htmlnode-ou-roles-and-msps) in Fabric.



The following additional fields are also available in your MSP definition but are not required:



* intermediate_certs: (if an intermediate CA was used) Paste in an array that contains one or more certificates from the external intermediate CA in base64 format. You must provide either a CA root certificate or an intermediate CA certificate, you can also provide both.
* tls_intermediate_certs: (if an intermediate TLS CA was used) Paste in an array that contains one or more certificates from the intermediate TLS CA in base64 format.
'https://api.<region>.hs-crypto.cloud.ibm.com:<port>/api/v2/keys?state=<state_integers>' 
-H 'accept: application/vnd.ibm.collection+json' 
-H 'authorization: Bearer <IAM_token>' 
-H 'bluemix-instance: <instance_ID>'

Replace the state variable in your request according to the following table.



Table 4. Describes the state variable

 Variable Description 

 state The states of the keys to be retrieved. States are integers and correspond to the Pre-active = 0, Active = 1, Suspended = 2, Deactivated = 3, and Destroyed = 5 values. For example, if you want to only list keys in the active state in your service instance, use ../keys?state=1. You can also pair state withoffset with limit to page through your available resources. 



For usage notes, check out the following examples for setting your state query parameter.



Table 5. Provides usage notes for the stage query parameter

 URL Description 

 .../keys Lists all of your available resources, up to the first 200 keys. 
 .../keys?state=5 Lists keys in the deleted state. 
 .../keys?state=2,3 Lists keys in the suspended and deactivated state. 







 Retrieving keys by Extractable value 

By specifying the extractable parameter at query time, you can retrieve keys whose material can leave the service.

For example, you might have both standard and root keys in your Hyper Protect Crypto Services instance, but you only want to retrieve keys with extractable key material when you make a GET /keys request.

The extractable query parameter takes in a boolean.

You can use the following example request to retrieve a different set of keys.
root@aix-7200-03-03: /
:>lsvg
old_rootvg
rootvg
root@aix-7200-03-03: /
:>lspv
hdisk0 00f6db0a6c7aece5 old_rootvg
hdisk1 00c25ab0062fa576 rootvg active
hdisk2 00c25ab0056a002a none
root@aix-7200-03-03: /
:>
Show more
2. Upon the successful removal of the volume group definition, remove the disk definition by using the rmdev command.

root@aix-7200-03-03: /
:>lspv
hdisk0 00f6db0a6c7aece5 old_rootvg
hdisk1 00c25ab0062fa576 rootvg active
hdisk2 00c25ab0056a002a None
root@aix-7200-03-03: /
:>rmdev - dl hdisk2
hdisk2 deleted
root@aix-7200-03-03: /
:>lspv
hdisk0 00f6db0a6c7aece5 old_rootvg
hdisk1 00c25ab0062fa576 rootvg active
root@aix-7200-03-03: /
:>
3. You can now detach the image volume (disk) containing the source mksysb from the helper VM. To detach the disk from AIX-7200-03-03, select Manage existing volumes and click a volume.
4. After you successfully detach the disk from AIX-7200-03-03, you can attach the saved image volume to other VM instances.