vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* You successfully completed [Getting started with Knowledge Studio](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-wks_tutintro), which covers creating a workspace, creating a type system, and adding a dictionary.
* You must have at least one user ID in either the Admin or Project Manager role. For information about user roles, see [User roles in Knowledge Studio](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-roles).







 Results 

After you create the rule-based model, you can use it in one of the following ways to find text patterns in documents:



* [Pre-annotate your documents](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-preannotationwks_preannotrule) before you create a machine learning model.
* [Deploy or export the model](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-wks_rule_publish) to other Watson services or products.







 Lesson 1: Adding a dictionary of months 

In this lesson, you will learn how to add a dictionary to a workspace in Knowledge Studio. The dictionary contains terms related to the months of the year.



 About this task 

In a later lesson, you will define a class based on this dictionary. When you create that class, all terms in this dictionary that are found in documents will be automatically annotated as a mention of the associated class type. For more information about dictionaries, see [Adding dictionaries to a workspace](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-dictionarieswks_projdictionaries).





 Procedure 



1. Download the [dictionary-items-month.csv!
See [Establishing a type system](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-typesystem) <br> <br>- Source documents: Create a corpus by uploading sample documents that are representative of your domain content into the workspace. See [Adding documents for annotation](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-documents-for-annotation). Partition the corpus into document sets, specify the percentage of documents that are shared among all document sets, and assign the document sets to human annotators. <br> <br>- Dictionaries: Upload or create dictionaries for annotating text. You can choose to manually add dictionary entries or upload entries from a file, and then edit the entries. See [Creating dictionaries](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-dictionaries). 
 Optional: Pre-annotate documents Pre-annotate documents according to the terms in the workspace dictionaries or based on rules that you define. See [Bootstrapping annotation](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-preannotation). 
 Annotate documents <br><br>1. Admins and project managers can annotate documents directly, without needing to create annotation tasks or go through the adjudication process. See [Annotating document sets directly](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-annotating-document-sets-directly).<br>2. The project manager generally assigns annotation tasks to human annotators, configures the inter-annotator agreement threshold, and provides annotation guidelines for the human annotators to follow.
Creating a rule-based model 

This tutorial helps you understand how to create a rule-based model that you can use to find text patterns that you define in documents.

You will build a model that can find text in documents that matches the pattern month day, year. For example, the model would find the date reference May 1, 2010. Before you define the rule pattern itself, you will create artifacts that will help you build the pattern, including a dictionary class that recognizes month mentions and a regular expression class that recognizes year mentions in text.



 Learning objectives 

After you complete this tutorial, you will know how to perform the following tasks:



* Create classes
* Add documents for defining rules
* Associate dictionaries with classes
* Define regular expressions to capture sequences of characters
* Define rules



This tutorial should take approximately 30 minutes to finish. If you explore other concepts related to this tutorial, it could take longer to complete.





 Before you begin 



* You're using a supported browser. For information, see [Browser requirements](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-system-requirements).
* You successfully completed [Getting started with Knowledge Studio for IBM Cloud Pak for Data](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-wks_tutintro), which covers creating a workspace, creating a type system, and adding a dictionary.
* You must have at least one user ID in either the Admin or Project Manager role. For information about user roles, see [User roles in Knowledge Studio](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-roles).







 Results 

After you create the rule-based model, you can use it in one of the following ways to find text patterns in documents:
[Launch the Knowledge Studio application](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-wks_tutintrolaunching-the-knowledge-studio-application).
2. If you already have other workspaces, click Create workspace.
3. Click Create advanced rules workspace. Enter a name for your workspace, then click Create. The advanced rules workspace loads after a few seconds.



2. Create a new project (optional)

![New project button](https://cloud.ibm.com/docs-content/v1/content/50eddb8fd81f33092880335ff107a78ff5cd0f65/watson-knowledge-studio/images/create-advanced-rules-project.png)



1. In the Projects tab of your workspace, click the New button.
2. Enter a name for your project and click Create.



For more information, see [Managing projects and extractors](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-managing-projects-and-extractors)
3. Add documents to your project

![Add Documents button](https://cloud.ibm.com/docs-content/v1/content/50eddb8fd81f33092880335ff107a78ff5cd0f65/watson-knowledge-studio/images/advanced-rules-add-documents.png)



1. Click the Add Documents button.
2. Select the file type and language of your file.
3. Click Choose File and select a file to upload.
4. Click Add to add your file to the project.



4. Create an extractor



1. Follow the tutorials from the Get Help menu to create and run extractors.
Refer to the section [using Terraform modules with blueprint templates](https://cloud.ibm.com/docs/schematics?topic=schematics-blueprint-terraform) for details of how to work with Terraform root and child modules.



* IBM authored modules can be found in the [Terraform IBM Modules](https://github.com/terraform-ibm-modules) GitHub repository.



2. Create and edit a blueprint template to implement the desired reference architecture from the selected modules. Review the sections [Understanding blueprint templates and configuration](https://cloud.ibm.com/docs/schematics?topic=schematics-blueprint-templates) and [blueprint template YAML schema](https://cloud.ibm.com/docs/schematics?topic=schematics-bp-template-schema-yaml) for guidance on the syntax and structure of blueprint templates.



* Create and name a new blueprint template YAML file in your favorite editor. [VSCode with the YAML language extension](https://cloud.ibm.com/docs/schematics?topic=schematics-edit-blueprints) is recommended, as it provides blueprint template syntax validation and auto complete.
* Working blueprint examples and template YAML files can be found in the [Cloud-Schematics](https://github.com/orgs/Cloud-Schematics/repositories?q=blueprint) GitHub repository. Or work from the [sample template](https://github.com/Cloud-Schematics/blueprint-sample-template/).
* Review the module metadata and readme file information for the selected [modules](https://github.com/terraform-ibm-modules) to guide writing the template blueprint [module definitions](https://cloud.ibm.com/docs/schematics?topic=schematics-bp-template-schema-yamlbp-modules-schema).
The request schema used in an operation to update a resource with a [JSON merge patch](https://datatracker.ietf.org/doc/html/rfc7396) SHOULD be the name of the canonical schema for the resource type, suffixed with Patch. In accordance with the JSON merge patch format, this schema MUST be a [graph fragment](https://cloud.ibm.com/docs/api-handbook?topic=api-handbook-schemasgraph-fragment-pattern) of the canonical schema, and MUST NOT have any required properties.







 Descriptions 

Every schema and property SHOULD have a description. Schema descriptions SHOULD NOT include references to implementation details, such as describing a schema as a “JSON object”. Rather, use the generic “object” in schema descriptions.





 Nested object schemas 

All object schemas MUST be defined separately in the components / schemas section of the API document, and MUST NOT be nested within the definition of another schema.

[2]
![Serialization and deserialization diagram.](https://cloud.ibm.com/docs-content/v1/content/e0c3acba1296ff980e76d5db456d53adefaabba4/EventStreams/schema_registry3.svg)

Serializer and deserializer

Zoom

![Compatibility and versions diagram.](https://cloud.ibm.com/docs-content/v1/content/e0c3acba1296ff980e76d5db456d53adefaabba4/EventStreams/schema_registry4.svg)

Compatibility and versions





 Versions and compatibility 

Whenever you add a schema, and any subsequent versions of the same schema, Event Streams can validate the format automatically and reject the schema if any issues exist. You can evolve your schemas over time to accommodate changing requirements. Create a new version of an existing schema, and the Schema Registry ensures that the new version is compatible with the existing version, meaning that producers and consumers that use the existing version are not broken by the new version.

Schemas are compared to avoid creating duplicate schemas where the schemas differ only in a way that does not affect the semantics of the schema. In some cases, the ordering of the JSON properties within a schema can be crucial to how the schema is used for encoding and decoding data, but in other cases it might not be relevant.

For example, the name property of a record schema is not used as part of the encoding and decoding process so you can position it anywhere inside the record JSON object. All these variations are considered the same schema.

The fields property in the JSON of a record schema is a case where its ordering is important. The Avro specification requires that a record’s fields are encoded and decoded in the order they appear in the schema that is used for the encode and decode operation.

As an example, consider the following three schemas.



 Schema 1 

{
"type": "record",
"name": "book",
Use the cluster_id, and logdna_name that are recorded while provisioning the Agent infrastructure.
* Run [workspace new](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-new) command to create the Agent service workspace.

ibmcloud schematics workspace new --file <Provide the path where the JSON file resides>/create_agent_service_workspace.json
* Record your workspace ID, and template ID to [upload the tar files](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-upload) provided for installing the Agent infrastructure. Follow the Readme file to extract the .tar file from the [Cloud Schematics Git repository](https://github.com/Cloud-Schematics/schematics-agents/tree/main/tarfiles).
* Run [workspace upload](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-upload) to upload the tar file.

ibmcloud schematics workspace upload --id <provide yourworkspace ID> --file </provide-path-where-tar-fileresides>/schematics-agents/tarfiles/agent-service-templates.tar --template <provide your template_id>
* On successful creation of schematics-agent-svc-workspace workspace, review, and edit the following Agent service input variables in the [IBM Cloud console](https://cloud.ibm.com/workspaces) > Settings page. The Agent service workspace should have the same input values for Resource Group, Location, and Tags. Use the cluster_id, and logdna_name that are recorded while provisioning the Agent infrastructure.
Zoom

![blueprint deployment](https://cloud.ibm.com/docs-content/v1/content/ba89e173287b28d1453d2a327d8a1f74ae2f1662/schematics//images/new/sc-bp-deploy.svg)

blueprint deployment



1. Prepare the configuration for the blueprint. The configuration specifies the source of the blueprint template in Git, the input files to customize the environment instance and version information.



* Create and save the blueprint configuration in Schematics. Refer to the section [Creating a blueprint config](https://cloud.ibm.com/docs/schematics?topic=schematics-create-blueprint-config) for guidance on creating the config via the CLI or UI.
* This step retrieves the YAML template and input file from Git.



2. After the config is successful created and validated, the blueprint will be in an Incomplete state.
3. Schematics automatically initializes its dependencies. The set of blueprint automation modules defined by the template is initialized to manage the cloud resources of the environment. Each module shown in the UI or CLI represents an automation module in the blueprint template.



* The saved blueprint configuration can be reviewed before environment and resource deployment.
* This step is performed internally. In a future release the resulting plan results will be presented for review.



4. Deploy the blueprint with the blueprint apply command or UI apply operation. Refer to the section [Apply changed to a blueprint environment](https://cloud.ibm.com/docs/schematics?topic=schematics-apply-blueprint) for guidance applying the config via the CLI or UI. .



* Based on your blueprint configuration, Schematics creates an internal deployment plan and runs the IaC module code in dependency order to create the environment and resources. In a future release the deployment plan will be presented for review in a separate step before it can be applied.



5.
"begin" : 2063,
"end" : 2071,
"inCoref" : false
},
Show more
7. Add annotations for relationships that are missing based on the new flattened entity types.
8. Create a Knowledge Studio workspace, and then upload the converted type system.

Follow the appropriate steps for uploading a type system based on your Knowledge Studio deployment type:



IBM Cloud

[Adding a type system to the workspace](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-exportimportwks_exportimport_expimp_type)
IBM Cloud Pak for Data

[Adding a type system to the workspace](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-exportimportwks_exportimport_expimp_type)



9. Upload the annotated documents to the workspace. Retain the original file structure of the exported data. Ensure that the compressed file has the same root-level directory as the original exported file, for example.

Follow the appropriate steps for uploading documents based on your Knowledge Studio deployment type:



IBM Cloud

[Adding documents to a workspace](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-exportimportwks_exportimport_expimp_doc)
IBM Cloud Pak for Data

[Adding documents to a workspace](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-exportimportwks_exportimport_expimp_doc)



10. From Knowledge Studio, click Train to retrain the model.

For more information, see the appropriate topic for your deployment type:



IBM Cloud

[Training the machine learning model](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-train-ml)
IBM Cloud Pak for Data
With Schematics Workspaces, use Terraform to automate the [provisioning and configuration management](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-open-projects) of your IBM Cloud resources, and rapidly build, duplicate, and scale complex, multitiered cloud environments. For more information, see [Schematics Workspaces](https://cloud.ibm.com/docs/schematics?topic=schematics-sch-workspaces).





 Schematics Actions 

With Schematics Actions, use Ansible playbooks to perform complex day-2 operations on your cloud resources, cloud environment, and app workloads. Whether you want to deploy multitiered apps, start or stop virtual servers or clusters, rotate keys, backup and restore app data, perform security scans, manage database schemas, or manage users, simply specify the tasks that you want to run in your playbook, and let Schematics securely connect and complete the tasks.For more information about managing Schematics Actions and its features, see [Schematics Actions](https://cloud.ibm.com/docs/schematics?topic=schematics-sc-actions).





 Schematics Agents 

Agents extends and integrates Schematics workspaces that running in your private network to the IBM Cloud Schematics service. Agents provisions, configure, and operate your private or on-premises cloud cluster resources without any time, network, or software restrictions. For more information about architecture and features of Agents, see [Schematics Agents](https://cloud.ibm.com/docs/schematics?topic=schematics-agents-intro).







 Benefits of using Schematics 

You do not need to install the open source projects on your machine or learn their API and command-line. You need to simply point Schematics to your IaC code repository and let Schematics run the specified tasks.



Schematics benefits

 Benefits Description 

 The open source projects used by Schematics Terraform, Ansible, Helm provisioning engine, and execution platform are tested, maintained, and monitored by IBM.
Creating a workspace 

The first step in building a custom model is to create a workspace.



 About this task 

For each model that you want to build and use, you create a single workspace that contains the artifacts and resources needed to build the model. You then train the model to produce a custom model that can be deployed to an external service for use.

Before creating a workspace, answer these questions:



* What type of model do you want to create?



* Machine learning model: Uses statistical approach to finding entities and relationships in documents. This type of model can adapt as the amount of data grows.
* Rule-based model: Uses a declarative approach to finding entities in documents. This type of model is more predictable, and is easier to comprehend and maintain. However, it does not learn from new data. It can only find patterns it has been taught to look for.



You can also create one workspace that contains both one rule-based model and one machine learning model.
* What services will use the model?

See [Watson services integration](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-wks_overview_fullwks_watsoninteg) for information about the other Watson services that custom models can be used with.







 Procedure 

To create a workspace, complete the following steps:



1. Log in as a Knowledge Studio administrator, and click Create Workspace.

People with the project manager role can perform almost all tasks except creating a workspace. An administrator must create the workspace initially and assign project managers to it.
2. Give the workspace a name. Choose a short name that reflects your domain content or the purpose of the model. If you need to, you can change the workspace name later.
3. Identify the language of the documents in your workspace. The documents that you add to the workspace, and the dictionaries that you create or upload, must be in the language that you specify.
4.
"value": "true",
"type": "",
"description": ""
}
],
"has_githubtoken": false
}
],
"runtime_data": [
{
"id": "3e68db7f-0bea-4d",
"engine_name": "terraform",
"engine_version": "v0.14.11",
"state_store_url": "https://us.schematics.cloud.ibm.com/v1/workspaces/us-south.workspace.Sample-Schematic-API-workspace.341cedd5/runtime_data/3e68db7f-0bea-4d/state_store",
"log_store_url": "https://us.schematics.cloud.ibm.com/v1/workspaces/us-south.workspace.Sample-Schematic-API-workspace.341cedd5/runtime_data/3e68db7f-0bea-4d/log_store"
}
],
"shared_data": {
"resource_group_id": ""
},
"applied_shareddata_ids": null,
"updated_at": "0001-01-01T00:00:00Z",
"last_health_check_at": "0001-01-01T00:00:00Z",
"cart_id": ""
}
 INFO:root:Completed Creating Schematic Workspace







 Next steps 

After you've successfully created a workspace, you can begin [Generating a plan](https://cloud.ibm.com/docs/hpc-spectrum-symphony?topic=hpc-spectrum-symphony-generate-plan&interface=api) to validate all of the configuration properties.
[Importing Schematics templates into the IBM Cloud catalog](https://cloud.ibm.com/docs/schematics?topic=schematics-private-catalog)Importing Schematics templates into the IBM Cloud catalog

Understand how to Create your private catalog, manage your private catalog, assign access to the private catalog in IBM Cloud®. And import your Terraform templates as products to make them available to your users. With a private catalog, you can limit the services that you want your users to see and the service settings that they can adjust. This way, you have more control over the type of service that is provisioned in your account and that naming conventions for services and service components are followed in your organization.

Schematics



* 30 minutes
* 2022-12-02



[Creating an IBM Cloud Kubernetes Service cluster on VPC infrastructure](https://cloud.ibm.com/docs/schematics?topic=schematics-provisioning-terraform-template)Creating an IBM Cloud Kubernetes Service cluster on VPC infrastructure

Use one of the IBM® provided templates to create an IBM Cloud® Kubernetes Service cluster in a Virtual Private Cloud (VPC). Then, you bind the cluster to an IBM Cloud® Object Storage service instance.

Schematics Kubernetes service

+1

Key Protect



* 60 minutes
* 2023-05-26



[Getting Started with IBM Cloud for VMware Solutions Shared](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-vmware-solutions-shared-getting-started)Getting Started with IBM Cloud for VMware Solutions Shared

On IBM Cloud there are a number of deployment offerings for VMware that you can choose from, with each providing a different level of abstraction. VMware Cloud Director (VCD) is offered under the banner of IBM Cloud for VMware Solutions Shared. It is a multi-tenant service with elasticity and two subscription types:

VMware Solutions Schematics



* 2 hours
* 2023-05-05