vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* Error: The utility does not complete successfully and the following message is written to the log: preBackupViaConfigHookRule on backupconfig/watson-discovery in namespace cpd (status=error)
* Cause: Discovery cannot be backed up and restored by using the OADP backup and restore utility. When the Discovery service is present, and an administrator attempts to backup an entire IBM Cloud Pak for Data instance, Discovery prevents the utility from completing successfully.
* Solution: Apply a patch that stops Discovery from preventing the utility from completing successfully.



1. Download the wd-aux-br-patch.zip file from the [Watson Developer Cloud Github](https://github.com/watson-developer-cloud/doc-tutorial-downloads/blob/master/discovery-data/latest/wd-aux-br-patch.zip) repository.
2. Extract the wd-aux-br-patch.yaml file from the ZIP file.
3. Run the following command in a terminal that is logged in to the cluster:

oc create -f wd-aux-br-patch.yaml





This issue was fixed with the 4.0.8 release. (You still cannot back up the Discovery service by using the OADP utility, but the OADP utility can back up other services when Discovery is installed.)







 4.0.0, 13 July 2021 



* Machine learning model enrichments that you apply by using the Analyze API can fail.



* Error: [WKSML_MODEL_NAME]: Enrichment of a document failed
* Cause: There is a known issue in Watson Knowledge Studio that can cause a timeout in enrichment processing.
* Solution: When you use the Analyze API to apply a Watson Knowledge Studio model enrichment to a collection, keep the size of the input document under 50 KB.







 2.2.1 issues that were fixed in subsequent releases 



* [Fixed in version 4] If you add an IBM Watson® Knowledge Studio machine learning enrichment to a collection, the ingestion process might run very slowly but will eventually complete. If ingestion processes slowly, you might see the following error message in Warnings and errors:
Go to the [IBM Cloud resources](https://cloud.ibm.com/resources) page. Log in with your IBMid.
2. Select Services and click on your Knowledge Studio service instance to open the Knowledge Studio service dashboard.
3. Click Launch tool from the Manage page.



After you launch Knowledge Studio, you can copy the URL and use it to launch the application directly.

Example URL:

https://gateway.watsonplatform.net/knowledge-studio/tools/app/zefz11/enf01h9le5c63p50/ui//

If you want to [add users to Knowledge Studio](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-teamteam-add) but don't want to grant them an IBM Cloud role that allows access to your service instance, share the application URL to provide access to the application.





 Lesson 1: Assigning user roles 

In this lesson, you will learn about the different roles that you can assign to users in Knowledge Studio.



 About this task 

The creation of a machine learning model requires input from subject matter experts, project managers, and users who can understand and interpret statistical models. Administrators assign roles to each user, such that they have appropriate authority for their tasks. For more information about user roles, see [Assembling a team](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-team).





 Procedure 



1. Log in to Knowledge Studio with your administrator ID. If you have an existing Knowledge Studio instance, you can find it on the IBM Cloud [resources page](https://cloud.ibm.com/resources) under Services.
2. Click the Settings icon to open the Service Details page. The page lists all the user IDs that are registered as Knowledge Studio users. Each user ID has one of the following roles (in decreasing order of included permissions):



* Admin
* Project Manager
* Human Annotator
Click on the Launch in IBM Cloud Pak for Data button.
4. Create a project by clicking on New project under the Projects tile and then in the subsequent page click Create an empty project.
5. Provide iris_project as the project name and Leave the Restrict who can be a collaborator checkbox unchecked as there's no confidential data.
6. Under Storage, choose an existing Object Storage service.
7. Click Create. Your new project opens and you can start adding resources to it.







 Import data 

As mentioned earlier, you will be using the Iris data set. The Iris dataset was used in R.A. Fisher's classic 1936 paper, The Use of Multiple Measurements in Taxonomic Problems, and can also be found on the [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/datasets/iris). This small dataset is often used for testing out machine learning algorithms and visualizations. The aim is to classify Iris flowers among three species (Setosa, Versicolor or Virginica) from measurements of length and width of sepals and petals. The iris data set contains 3 classes of 50 instances each, where each class refers to a type of iris plant.

Zoom

![Iris Example](https://cloud.ibm.com/docs-content/v1/content/f84e2238288b7f0355715a4c355609ce7ac48fb0/solution-tutorials/images/solution22-build-machine-learning-model/iris_machinelearning.png)

Iris Example

Download[iris_initial.csv](https://github.com/IBM-Cloud/ml-iris-classification/raw/master/data/iris_initial.csv) which consists of 40 instances of each species. Make sure the downloaded file is named iris_initial.csv.



1. Under Data in this project, click Drop data files here or browse for files to upload.
2. Upload the downloaded iris_initial.csv.
3.
"clazz":null,
"and so on"
}
},
{
"id":"b9d6caa2-90ac-47ff-91f6-2149b8ffcf20",
"label":"FURNITURE_DINING",
"sireProp":{
"mentionType":null,
"subtypes":null,
"roles":["b9d6caa2-90ac-47ff-91f6-2149b8ffcf20","93ba1f27-173f-4714-b31e-77bdd8cb9932"],
"clazz":null,
"and so on"
}
},
Show more
4. Assign a unique ID to each new root-level entity type.
5. Export the corpus for your machine learning model from Knowledge Studio as a compressed file.

Follow the appropriate steps for exporting based on your Knowledge Studio deployment type:



IBM Cloud

[Uploading resources from another workspace](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-exportimportwks_exportimport_expimp_doc)
IBM Cloud Pak for Data

[Uploading resources from another workspace](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-exportimportwks_exportimport_expimp_doc)



6. In the downloaded corpus, for all mentions with a subtype defined, update the type information for the mention to specify the new root-level entity type.

For example, the original type system might include the PATIO subtype mention:

{
"id" : "Blogs_shopper.com_dc5cf4764d91f87575b17ac8a5268462.en-M92",
"source" : "IMPORT",
"properties" : {
"SIRE_ENTITY_CLASS" : "SPC",
Collecting the metadata 

VMware® Update Manager (VUM) downloads metadata about the upgrades, patches, or extensions via a predefined automatic process that you can modify. At regular configurable intervals, VUM contacts VMware, or third-party sources, to gather the latest metadata about available upgrades, patches, or extensions. However, the default settings from VMware are acceptable for use in the VMware vCenter Server® instance you can change them as needed for your enterprise requirements.

VUM displays system-managed baselines that are generated by vSAN™. The system-managed baselines automatically update their content periodically, which requires VUM to have constant access to the internet. The vSAN system baselines are typically refreshed every 24 hours.

You can use the system-managed baselines to upgrade your vSAN clusters to recommended critical patches, drivers, updates, or latest supported ESXi™ host version for vSAN.

For most enterprises, the VMware default settings for VUM are considered to be suitable. The following information provides how to change these settings if your enterprise would like to use different settings.



 Download schedule 

Updates are virtual appliance upgrades, host patches, and extensions and by default VUM downloads updates daily. To change the download schedule, access the vSphere® Web Client. Go to Home > Update Manager > Manage > Settings, and select Download Schedule, and then click Edit.





 Notification check schedule 

Notifications are information about patch recalls, new fixes, and alerts and by default VUM downloads notifications on an hourly basis. To change the notification check schedule, access the vSphere Web Client. Go to Home > Update Manager > Manage > Settings, and select Notification Check Schedule, and then click Edit.





 Virtual machine settings 

To change the virtual machine (VM) Settings, access the vSphere Web Client. Go to Home > Update Manager > Manage > Settings, and VM Settings, and then click Edit.





 Host/cluster settings 

To change the Host/cluster Settings, access the vSphere Web Client. Go to Home > Update Manager > Manage > Settings, and Host/Cluster Settings, and then click Edit.





 Related links
You must define a mapping between the UIMA types and Knowledge Studio entity types. You must also create a .zip file that contains all the files that are required to upload the analyzed data into a Knowledge Studio workspace.

Related information:



* [Exporting crawled or analyzed documents](https://www.ibm.com/support/knowledgecenter/SS8NLW_11.0.0/com.ibm.discovery.es.ad.doc/iiysatexportca.htm)
* [Output paths for exported documents](https://www.ibm.com/support/knowledgecenter/SS8NLW_11.0.0/com.ibm.discovery.es.ad.doc/iiysaexportoutput.htm)









 Exporting an analyzed collection from Content Analytics Studio 

You can export a collection of analyzed documents from Watson Explorer Content Analytics Studio, and upload the analyzed documents as XMI files into a Knowledge Studio project.



 Procedure 

To get analyzed documents from a Content Analytics Studio collection, follow these steps:



1. Launch Content Analytics Studio and open the Studio project.
2. Right-click a folder that contains documents that you want to use for training a machine learning model and select Analyze Collection.
3. Select a UIMA pipeline configuration file.
4. Go to the Collection Analysis view and click the Save icon in the Collection Analysis view. Specify the folder where the saved results are to be written and specify the file name.
5. Open the folder that you specified. The file extension of the saved file is .annotations.
6. Copy the .annotations file to your local file system and rename the file extension from .annotations to .zip.
7. Extract all files from the .zip file. The extracted contents include XMI files (.xmi), the UIMA TypeSystem descriptor file (TypeSystem.xml), and other files.







 What to do next 

You must define a mapping between the UIMA types and Knowledge Studio entity types. You must also create a .zip file that contains all of the files that are required to upload the analyzed data into a Knowledge Studio workspace.
Virtual machine settings 

To change the virtual machine (VM) Settings, access the vSphere Web Client. Go to Home > Update Manager > Manage > Settings, and VM Settings, and then click Edit.





 Host/cluster settings 

To change the Host/cluster Settings, access the vSphere Web Client. Go to Home > Update Manager > Manage > Settings, and Host/Cluster Settings, and then click Edit.





 Related links 



* [VMware HCX solution architecture](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-hcx-archi-introhcx-archi-intro)
* [VMware Solutions on IBM Cloud Digital Technical Engagement](https://www.ibm.com/demos/collection/IBM-Cloud-for-VMware-Solutions/) (demonstrations)
export SCORING_ENDPOINT='<SCORING_ENDPOINT_FROM_ABOVE_STEP>'

IBM Cloud Shell is a cloud-based shell workspace that you can access through your browser. It's preconfigured with the full IBM Cloud CLI and tons of plug-ins and tools that you can use to manage apps, resources, and infrastructure.
4. To use the Watson Machine Learning REST API, you need to obtain an IBM Cloud Identity and Access Management (IAM) token. Run the below command

ibmcloud iam oauth-tokens --output JSON | jq -r .iam_token
5. Copy the complete IAM token along with Bearer from the above response and export it as an IAM_TOKEN to be used in the subsequent API requests

export IAM_TOKEN='<IAM_TOKEN>'
6. Run the below cURL code in the cloud shell to see the prediction results.

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header "Authorization: $IAM_TOKEN" -d '{"input_data": [{"fields": "sepal_length", "sepal_width", "petal_length","petal_width"],"values": 5.1,3.5,1.4,0.2], 3.2,1.2,5.2,1.7]]}]}' $SCORING_ENDPOINT

If you observe, the code is from the cURL tab of the deployment your created above. Thereafter, replacing the [$ARRAY_OF_INPUT_FIELDS] placeholder with ["sepal_length", "sepal_width", "petal_length","petal_width"], [$ARRAY_OF_VALUES_TO_BE_SCORED] placeholder with [5.1,3.5,1.4,0.2] and [$ANOTHER_ARRAY_OF_VALUES_TO_BE_SCORED] placeholder with [3.2,1.2,5.2,1.7] respectively.







 Step 6: Monitor your deployed model with IBM Watson OpenScale
Update the proxy settings on the client machine’s internet browser (optional) 

As a final, optional step to configuring forward proxy traffic redirection with your Citrix Netscaler VPX, you can update your proxy settings using your client machine's internet browser.

To do so, perform the following steps:



1.  Go to Internet Options in your browser settings and configure it to use a proxy server for outgoing requests.
2.  Use the IP address of your cache redirection virtual server that was defined in previous steps as your proxy.



These proxy settings might not be necessary if the Citrix Netscaler VPX appliance is in the direct layer 3 path between client machines and the internet.
You can integrate a Knowledge Studio custom model into the [Entity Extraction](https://cloud.ibm.com/docs/discovery?topic=discovery-configserviceentity-extraction) or [Relation Extraction](https://cloud.ibm.com/docs/discovery?topic=discovery-configservicerelation-extraction) enrichments with the Discovery tooling.

You cannot apply multiple custom models to the same field using the Discovery tooling. It is possible to apply more than one custom model to identical fields using the API. See [Integrating your custom model with the API](https://cloud.ibm.com/docs/discovery?topic=discovery-integrating-with-wksintegrate-customAPI).



1. Get the Model ID of your Knowledge Studio model.
2. In the Discovery tooling, click the Manage Data icon to open the Manage data screen, then create or open a collection.



If you choose an existing collection, it must be empty. If not, create a new collection and reingest the documents after you have followed these steps.



1. Click Configure data. The Enrich fields tab opens.
2. Click Add enrichments and select either the Entity Extraction or Relation Extraction enrichment.
3. Enter the Model ID in the Custom Model ID box of the selected enrichment. The custom IBM Watson™ Knowledge Studio model overrides the default for that enrichment.
4. Click Apply, then Done.



When documents are uploaded to a data collection, they are converted and enriched using the configuration file chosen for that collection. If you switch an existing collection to a new configuration file after documents are uploaded, those uploaded documents remain converted by the original configuration file. Any documents uploaded after switching the configuration file use the new configuration file. If you want the entire collection to use the new configuration, you must create a new collection, choose that new configuration file, and re-upload all the documents.





 Next Steps 

Use Discovery with your new configuration to ingest private data. Documents you ingest with the updated configuration are automatically enriched with the data from your custom model.
* If you don't specify a user, the Connections commands return information for the admin user, by default.
* If you don't specify an endpoint-type, the connection string returns the public endpoint by default.
* If your deployment has only a private endpoint, specify --endpoint-type private or the commands return an error. The user and endpoint type is not enforced. You can use any user on your deployment with either endpoint (if both exist on your deployment).









 Getting Connection Strings in the API 

To retrieve users' connection strings from the [Cloud Databases API](https://cloud.ibm.com/apidocs/cloud-databases-api/cloud-databases-api-v5introduction), use the [Connections endpoint](https://cloud.ibm.com/apidocs/cloud-databases-api/cloud-databases-api-v5getconnection). To create the connection strings, ensure that the path includes the specific user and endpoint type (public or private) that should be used. The user and endpoint type are not restricted or enforced. You have the flexibility to utilize any user available in your deployment, along with either endpoint (if both are present in your deployment).

The API command looks like:

curl -X GET -H "Authorization: Bearer <>" 'https://api.{region}.databases.cloud.ibm.com/v5/ibm/deployments/{id}/users/{userid}/connections/{endpoint_type}'

Remember to replace {region}, {id}, {userid}, and {endpoint_type} with the appropriate values.





 Additional Users and Connection Strings 

Access to your Databases for MongoDB deployment is not limited to the admin user.
User data can be used on cloud-init and non-cloud-init images. 







 Attached storage disks 

If you need extra storage, you can attach storage disks to your instance. The type of storage disks that are available to you depend on the profile that you select.





 Network interface 



Table 3. Network interface options

 Field Details 

 Uplink port speeds You can select the uplink speed for your instance, up to 1 Gbps. These virtual uplinks are backed by redundant physical uplinks to the IBM public and dedicated networks. The public and dedicated speed is always the same at the time of order, with the option to upgrade or downgrade a link if needed. If you select the 100 Mbps rate-limited option, the maximum instance throughput is limited only by the physical bandwidth available to the virtual server host. If you select the 1 Gbps non rate-limited option, you can achieve higher network performance through extra configuration. For more information, see [Configuring virtual server settings for improved network performance](https://cloud.ibm.com/docs/virtual-servers?topic=virtual-servers-configuring-network-performance). 
 Private and public security group You can use security groups to enact a set of IP filter rules that define how to handle incoming and outgoing traffic to both the public and private interfaces your instance. For more information, see [About IBM Security Groups](https://cloud.ibm.com/docs/security-groups?topic=security-groups-about-ibm-security-groups). 
 Private and public VLAN Your virtual server instance is placed on an automatically assigned VLAN by default. You can choose a different VLAN if you already have one in your selected data center. For more information, see [About VLANs](https://cloud.ibm.com/docs/vlans?topic=vlans-about-vlans). 
 Private and public subnet Selecting a subnet is optional and to be used only when you require your device to use an IP address from the subnet. If you select a subnet, verify that you have enough IP addresses to fulfill the request.
* cos_endpoint includes information about the target. For more information on how to get the bucket details, see Getting the bucket configuration details]bucket instance CRN] .
PRIVATE_COS_ENDPOINT indicates the IBM Cloud Activity Tracker Event Routing endpoint to look for this bucket. Use the private endpoint.

COS_CRN indicates the CRN]] of the COS instance where you provisioned the bucket.

BUCKET_NAME indicates the name of the bucket.

API_KEY contains the API key that has permissions to upload objects into the bucket. This value is ignore if service_to_service_enabled is true.

SERVICE_TO_SERVICE indicates if service-to-service authorization]service_-to_-service authorization] ! ! ! ! ! ! has been enabled for the bucket. Specify true if service-to-service authorization is enabled and false if service-to-service authorization is not enable. By default, service-to-service authorization is false.
<-- </ul> -->For example, you can use the following cURL request to create a target in Dallas: curl -X POST https://private.us-south.atracker.cloud.ibm.com/api/v2/targets -H "Authorization: $ACCESS_TOKEN" -H "content-type: application/json" -d '{
"name": "My COS target",
"target_type": "cloud_object_storage",
"cos_endpoint": {
"endpoint": "s3.private.us-south.cloud-object-storage.appdomain.cloud",
"target_crn": "crn:v1:bluemix:public:cloud-object-storage:global:a/<account-id>:<instance-id>::",
"bucket": "my-activity-tracking-bucket",
"api_key": "xxxxxxxxxxxxxxxxxx",
"service_to_service_enabled": false
}
}'
* (DBaaS only) The DB endpoint access - either 'private', 'public' or 'public-and-private' (for mysql databases), default is 'private'
* (DBaaS only) Whether to enable auto-scaling in DBaaS - true or false, default is false



* DBaaS has optional variables in the database variables file, that you can modify for your solution:



 Parameter Description 

 service The type of Cloud Databases that you want to create. Example: databases-for-mysql, databases-for-mongodb 
 plan The name of the service plan that you choose for your instance. All databases use standard. enterprise is supported only for cassandra (databases-for-cassandra) and mongodb(databases-for-mongodb) 
 version The version of the database to be provisioned. If omitted, the database is created with the most recent major and minor version. (For MySQL, 5.7 is the version that is supported) 
 members_memory_allocation_mb The amount of memory in megabytes for the database, allocated per-member. Member group memory must be >= 1024 and <= 114688 in increments of 128. If not specified, the default setting of the database service is used, which can vary by database type. 
 members_disk_allocation_mb The amount of disk space for the database, allocated per member. Member group disk must be >= 20480 and <= 4194304 in increments of 1024.If not specified, the default setting of the database service is used, which can vary by database type. 
 members_cpu_allocation_count Enables and allocates dedicated CPU per-member to your deployment. Member group cpu must be >= 3 and <= 28 in increments of 1. 
 users A list of users that you want to create on the database. Multiple blocks are allowed. 
 Nested scheme for users: 
 name - The user ID to add to the database instance. The user ID must be in the range 5 - 32 characters. 
 password - The password for the user ID. The password must be in the range 10 - 32 characters