vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* [Regular expressions](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-managing-projects-and-extractorscreating-a-regular-expression): syntax-based pattern matching





 Creating extractors with linguistic patterns 

Defining a new extractor using natural language processing (NLP) concepts such as tokenization and part-of-speech requires developing an extractor using the Annotation Query Language (AQL).

To see the languages that are supported for tokenization and part-of-speech analysis, see [Multilingual Support for Text Analytics](http://www.ibm.com/support/knowledgecenter/SSPT3X_4.0.0/com.ibm.swg.im.infosphere.biginsights.text.doc/doc/ana_txtan_lang-support.html). For more information how to write AQL and available functions, see [Annotation Query Language reference](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-annotation-query-language-reference)





 Creating a dictionary 

A dictionary is a list of terms that relate to a concept. Use a dictionary as a convenient substitute for individual literals when analyzing text for multiple terms that fit into the same category.

You can define a dictionary as a simple list of terms or as a mapping table that maps a value or term to a preferred term. Mapping tables are typically used to resolve synonyms, abbreviations, and encoded values.

Use of a dictionary can sometimes result in overlapping matches. For example, a dictionary used to extract information about people with a military rank such as Officer, Warrant Officer, or Chief Warrant Officer might result in three overlapping matches for every occurrence of Chief Warrant Officer. See [Eliminate duplicate and overlapping results](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-managing-projects-and-extractorseliminate-duplicate-and-overlapping-results) for information needed to address this issue.



 Defining a list
The number of administrator signatures that is required to remove an administrator after you leave imprint mode. The valid value is in the range 1 - 8. 
 admins Required. The list of administrators for the instance crypto units. You can set up to eight administrators and the number needs to be equal to or greater than the thresholds that you specify. The following values need to be set for each administrator:<br><br>name: <br>The name of the administrator. It needs to be no more than 30 characters in length.<br><br>key:<br><br><br><br> * If you are using signature key files on the local workstation that are created by the TKE CLI plug-in and are not using a third-party signing service, specify the absolute path and file name of the signature key file that is to be used.<br> * If you are using a signing service to provide signature keys, specify the name of the signature key depending on the signing service definition. The character string for the key name is appended to a URI that is sent to the signing service and must contain only unreserved characters as defined by section 2.3 of [RFC3986](https://datatracker.ietf.org/doc/html/rfc3986).<br><br><br><br>token:<br><br><br><br> * If you are using signature key files on the local workstation that are created by the TKE CLI plug-in and are not using a third-party signing service, specify the administrator password to access the corresponding signature key file.<br> * If you are using a signing service to provide signature keys, specify the token that authorizes use of the signature key depending on the signing service definition.<br><br><br><br>Note: The token parameter is optional. If you don't specify the token, you are prompted to enter the token value when you run Terraform commands. After the instance initialization, the value that you enter for the token parameter is stored in a .tfstate file.
You can now divide the corpus into multiple document sets and assign the document sets to human annotators.







 Lesson 2: Pre-annotating with a dictionary-based annotator 

In this lesson, you will learn how to use a dictionary-based annotator to pre-annotate documents in Knowledge Studio.



 About this task 

Pre-annotating documents is an optional step. However, it is a worthwhile step because it makes the job of human annotators easier later.

For more information about pre-annotation with dictionaries, see [Pre-annotating documents with a dictionary](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-preannotationwks_preannot).





 Procedure 



1. Within your workspace, click Assets > Dictionaries.

The Test dictionary dictionary opens. The [Adding a dictionary](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-wks_tutintrowks_tutless4) lesson of the Getting started with Knowledge Studio tutorial shows how to create this dictionary.
2. From the Entity type list, select the ORGANIZATION entity type to map it to the Test dictionary dictionary.

The [Creating a type system](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-wks_tutintrowks_tutless3) lesson of the Getting started with Knowledge Studio tutorial shows how to create the type system that contains the ORGANIZATION entity type.
3. On the Machine Learning Model > Pre-annotation > Dictionaries tab, click Apply This Pre-annotator.
4. Select the document set that you created in [Lesson 1](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-wks_tutml_introtut_lessml1).
5. Click Run.







 Results
* You can create mappings based on facets that are used in the Pattern Matcher annotator or Dictionary Lookup annotator in Watson Explorer Content Analytics. In text analysis rule files (.pat), the facet is represented as the category attribute. To define a mapping, use the following syntax:

com.ibm.takmi.nlp.annotation_type.ContiguousContext:category={FACET_PATH}[TAB]{WKS_ENTITY_TYPE}

The following example, which applies to the Pattern Matcher and Dictionary Lookup annotators, defines a mapping between the category $.mykeyword.product and the Knowledge Studio entity type PRODUCT:

com.ibm.takmi.nlp.annotation_type.ContiguousContext:category=$.mykeyword.product PRODUCT









 What to do next 

You must create a .zip file that contains all of the files that are required to upload the analyzed data into a Knowledge Studio workspace.

Related information:

[Pattern Matcher annotator](http://www.ibm.com/support/knowledgecenter/SS8NLW_11.0.0/com.ibm.discovery.es.ta.doc/iiystapattmatch.htm)

[Dictionary Lookup annotator](http://www.ibm.com/support/knowledgecenter/SS8NLW_11.0.0/com.ibm.discovery.es.ta.doc/iiystadictlookup.htm)

[Named Entity Recognition annotator](http://www.ibm.com/support/knowledgecenter/SS8NLW_11.0.0/com.ibm.discovery.es.ta.doc/iiystasystemt.htm)







 Uploading UIMA CAS XMI files into a workspace 

To use the pre-annotated documents that you downloaded to train a model, you must create a .zip file that contains all the files required to upload the XMI files, and then upload the .zip file into a Knowledge Studio workspace.



 Before you begin
This choice is best for fields with unique or specialized terminology because this pre-annotator does not analyze the context in which the term is used in the way a machine learning pre-annotator does; it instead relies on the term being distinct enough to have a decipherable meaning regardless of the context in which it is used. For example, it is easier to recognize asbestos as a mineral entity type than to determine the entity type of squash, which can refer to a vegetable, a sport, or a verb meaning to crush something.

Dictionary pre-annotators do not recognize entity subtypes. Human annotators can specify entity subtypes for each pre-annotated mention by working on an [annotation task](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-annotate-documentswks_hatask) with the pre-annotated document.
* Machine learning

Uses a machine learning model to automatically annotate documents. This option is only available if you have created a machine learning model with Knowledge Studio already. If you add a document set, you can run the machine learning annotator that you created previously to pre-annotate the new documents. If the new set of documents is similar to the documents that were used to train the machine learning annotator originally, then this is probably your best choice for pre-annotation.
* Rule

Uses a rule-based model to automatically annotate documents. This option is only available if you have created a rule-based model with Knowledge Studio already. If your documents contain common patterns of tokens from which you can derive meaning, this model might be a good choice. It can incorporate some of the function of the dictionary pre-annotator if you enable it, by identifying class types for dictionary terms that it finds in the documents also.



Alternatively, you can upload already-annotated documents, and use them to start training the machine learning model. You cannot run a pre-annotator on annotated documents that you upload or the existing annotations will be stripped from the documents and replaced with annotations produced by the pre-annotator only.





 Running multiple pre-annotators
After uploading documents from another workspace, do a quick review of the annotations to address any discrepancies.
* You must upload the type system from the original workspace into the current workspace before you upload ground truth annotations. For details, see [Uploading resources from another workspace](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-exportimport).



Notes about ZIP files of documents in UIMA CAS XMI format



* If you previously downloaded annotated documents that are in UIMA CAS XMI format, you can upload the ZIP file that contains the analyzed content. Specify that this is the type of content you want to upload before you click Upload. For details about how to create these files and requirements for uploading them, see [Uploading pre-annotated documents](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-preannotationwks_uima).



5. After the documents have been added, click the document names to preview them and verify that the content looks OK. For example, verify that text files are in UTF-8 format and that no diacritical marks or character normalization issues are visible in the documents, and check for poor sentence breaks. If problems exist, you might need to pre-process the files before you add them to the corpus. You want the documents to be as clean and well-formatted as possible before dictionary or human annotation begins.







 What to do next 

Before you start any human annotation tasks, divide the corpus into multiple document sets and assign the document sets to human annotators.

Admins and project managers can [annotate document sets directly](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-annotating-document-sets-directly) without creating annotation tasks.







 Deleting documents 

You can delete a document if you determine that it does not represent standard industry text that will benefit the model.
Pre-annotating documents is an optional step. However, it is a worthwhile step because it makes the job of human annotators easier later.

For more information about pre-annotation with dictionaries, see [Pre-annotating documents with a dictionary](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-preannotationwks_preannot).





 Procedure 



1. Within your workspace, click Assets > Dictionaries.

The Test dictionary dictionary opens. The [Adding a dictionary](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-wks_tutintrowks_tutless4) lesson of the Getting started with Knowledge Studio tutorial shows you how to create this dictionary.
2. From the Entity type list, select the ORGANIZATION entity type to map it to the Test dictionary dictionary.

The [Creating a type system](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-wks_tutintrowks_tutless3) lesson of the Getting started with Knowledge Studio tutorial shows how to create the type system that contains the ORGANIZATION entity type.
3. On the Machine Learning Model > Pre-annotation page, click Run Pre-annotators.
4. Select Dictionary, then click Next.
5. Select the document set that you created in [Lesson 1](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-wks_tutml_introtut_lessml1).
6. Click Run.







 Results 

The documents in the selected sets are pre-annotated by using the dictionary that you created. If you like, you can use the dictionary to pre-annotate document sets or annotation sets that you add later.







 Lesson 3: Creating an annotation task
This annotation was added by the dictionary pre-annotator that was applied in [Lesson 2](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-wks_tutml_introwks_tutless_ml3). This pre-annotation is correct, so it does not need to be modified.

![This screen capture shows an open document with an existing pre-annotation for "IBM".](images/wks_tut_preannotation.png "This screen capture shows an open document with an existing pre-annotation for "IBM".")
6. Annotate a mention:



1. Click the Entity tab.
2. In the document body, select the text Thomas Watson.
3. In the list of entity types, click PERSON. The entity type PERSON is applied to the selected mention.

![This screen capture shows the "PERSON" entity type applied to the mention, "Thomas Watson".](images/wks_tut_annotatemention3.png "This screen capture shows the "PERSON" entity type applied to the mention, "Thomas Watson".")



7. Annotate a relation:



1. Click the Relation tab.
2. Select the Thomas Watson and IBM mentions (in that order). To select a mention, click the entity type label above the text.
3. In the list of relation types, click founderOf. The two mentions are connected with a founderOf relationship.

![This screen capture shows two mentions connected by the relation type, "founderOf".](images/wks_tut_annotaterelation.png "This screen capture shows two mentions connected by the relation type, "founderOf".")



8. From the status menu, select Completed, and then click the Save button.
9. Click Open document list to return to the list of documents for this task and click Submit All Documents to submit the documents for approval.
Another example defines a mapping between UIMA types produced by custom annotator that was created in Watson Explorer Content Analytics Studio and Knowledge Studio entity types:

com.ibm.Person PERSON
com.ibm.Date DATE
* You can create mappings based on facets that are used in the Pattern Matcher annotator or Dictionary Lookup annotator in Watson Explorer Content Analytics. In text analysis rule files (.pat), the facet is represented as the category attribute. To define a mapping, use the following syntax:

com.ibm.takmi.nlp.annotation_type.ContiguousContext:category={FACET_PATH}[TAB]{WKS_ENTITY_TYPE}

The following example, which applies to the Pattern Matcher and Dictionary Lookup annotators, defines a mapping between the category $.mykeyword.product and the Knowledge Studio entity type PRODUCT:

com.ibm.takmi.nlp.annotation_type.ContiguousContext:category=$.mykeyword.product PRODUCT









 What to do next 

You must create a .zip file that contains all of the files that are required to upload the analyzed data into a Knowledge Studio workspace.

Related information:



* [Pattern Matcher annotator](https://www.ibm.com/support/knowledgecenter/SS8NLW_11.0.0/com.ibm.discovery.es.ta.doc/iiystapattmatch.htm)
* [Dictionary Lookup annotator](https://www.ibm.com/support/knowledgecenter/SS8NLW_11.0.0/com.ibm.discovery.es.ta.doc/iiystadictlookup.htm)
* [Named Entity Recognition annotator](https://www.ibm.com/support/knowledgecenter/SS8NLW_11.0.0/com.ibm.discovery.es.ta.doc/iiystasystemt.htm)









 Uploading UIMA CAS XMI files into a workspace
: Annotation to specify a service that accepts public or private requests. If you don't include this annotation, a public LoadBalancer is created.

service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-node-selector
: Annotation to specify a worker node label selector. To identify the worker nodes that receive traffic, you can select one of the supported label selector keys. Note that you can include only one label selector in the annotation, and that the selector must be specified in the "key=value" format. If this annotation is not specified, all worker nodes in your cluster are configured to receive traffic from the VPC ALB. If specified, this annotation takes precedence over the service.kubernetes.io/ibm-load-balancer-cloud-provider-zone annotation, and any dedicated: edge labels on worker nodes are ignored.
: The following keys are permitted:



* ibm-cloud.kubernetes.io/internal-ip
* ibm-cloud.kubernetes.io/machine-type
* ibm-cloud.kubernetes.io/os
* ibm-cloud.kubernetes.io/region
* ibm-cloud.kubernetes.io/subnet-id
* ibm-cloud.kubernetes.io/worker-pool-id
* ibm-cloud.kubernetes.io/worker-pool-name
* ibm-cloud.kubernetes.io/zone
* kubernetes.io/arch
* kubernetes.io/hostname
* kubernetes.io/os
* node.kubernetes.io/instance-type
* topology.kubernetes.io/region
* topology.kubernetes.io/zone



service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-subnets
: Annotation to specify one or more subnets that the VPC ALB service deploys to.
Some sensitive operations, such as committing a master key register, changing the signature thresholds, and adding or removing administrators after the crypto unit exits imprint mode, require a sufficient number of crypto unit administrators to enter their credentials. Quorum authentication assures that no single person can make a critical change on the crypto unit. Instead, a minimum number of crypto unit administrators (at least two) must cooperate to do these operations. Quorum authentication requires more than one crypto unit administrator to approve an operation, which enables an extra layer of protection on the crypto unit.





 Signature thresholds 

The signature thresholds of a crypto unit control how many administrative signatures are needed to run a command. In imprint mode, the signature thresholds are set to zero. To exit imprint mode, set the signature thresholds to a value greater than zero. When a crypto unit is zeroized, the signature thresholds are reset to zero.

There are two types of signature thresholds on a crypto unit. The main signature threshold controls how many signatures are needed to run most administrative commands. The revocation signature threshold controls how many signatures are needed to remove an administrator. Some commands need only one signature, regardless of how the signature threshold is set.

Setting the signature thresholds to a value greater than one enables quorum authentication from multiple administrators for sensitive operations. The maximum value that you can set the signature threshold and revocation signature threshold is eight, which is also the maximum number of administrators that can be added to a crypto unit.





 Master key 

The master key, also known as HSM master key, is used to encrypt the service instance for key storage. It is a symmetric 256-bit AES key. With the master key, you take the ownership of the cloud HSM and own the root of trust that encrypts the entire hierarchy of encryption keys, including root keys and standard keys in the key management keystore and Enterprise PKCS #11 (EP11) keys in the EP11 keystore. You need to configure the master key first before you can manage encryption keys. One service instance can have only one master key.
You might define conditions that determine the placement of the FullName class in relation to common prefixes, such as Dr. and common suffixes, such as Jr.. When the rule-based model is used, the FullName class is mapped to the PERSON entity type.

Another reason to avoid mapping intermediate classes to entities in your type system is that if you pre-annotate documents with the rule-based model, and then add them to your ground truth for training a machine learning model, you do not want to define rules in such a way that they will produce overlapping entity mentions. For example, if you were to map both the intermediate class FirstName and the complex class FullName to the PERSON entity, then an occurrence of John Doe, Jr. would result in an overlapping mention.





 Rule editor tools 

The rule editor provides some tools that help you to define rules.



* Dictionary

Add a dictionary and assign it a class name. Any words that are found that match entries in the dictionary are automatically annotated with the assigned class.
* Regular expression

A regular expression (regex) is a sequence of characters that define a search pattern. A basic example is [A-Z]* which finds capitalized words.

[A-Z] matches any capital alphabetical letter (A through Z) and [a-z]* matches any lower-case alphabetical letter (a through z) zero or more times. The asterisk (*) is the character that defines the repeating setting (zero or more times).

Consider using a free web-based regex utility to help you determine the right expression to use to capture the pattern you want to find.

For example, your documents might have several references that are similar to the following phrases:

35-year-old driver
16-year-old learner

The syntax n-year-old x is a pattern that typically represents a person. You can define a regular expression rule to find phrases that match the n-year-old x pattern, and annotate them as PERSON entity mentions.





 Regular expression best practices 

Although the regex tool that is included in the rule editor recognizes expressions that follow the java.util.regex.Pattern syntax, the entire syntax is not supported.
Bootstrapping annotation 

Simplify the job of the human annotator by pre-annotating the documents in a workspace. A pre-annotator is a Knowledge Studio dictionary, rule-based model, or machine learning model that you can run to find and annotate mentions automatically.

Pre-annotation makes the job of human annotators easier because it covers the straightforward annotations, and gets the job of annotating the documents underway.

The method that you use to pre-annotate documents in no way restricts the ways that you can use the resulting model. For example, just because you use the Natural Language Understanding service to pre-annotate documents does not mean you must deploy the final machine learning model that you build to the Natural Language Understanding service.



 Pre-annotation methods 

The following pre-annotators are available:



* Dictionary

Uses a dictionary of terms that you provide and associate with an entity type to find mentions of that entity type in the documents. This choice is best for fields with unique or specialized terminology because this pre-annotator does not analyze the context in which the term is used in the way a machine learning pre-annotator does; it instead relies on the term being distinct enough to have a decipherable meaning regardless of the context in which it is used. For example, it is easier to recognize asbestos as a mineral entity type than to determine the entity type of squash, which can refer to a vegetable, a sport, or a verb meaning to crush something.

Dictionary pre-annotators do not recognize entity subtypes. Human annotators can specify entity subtypes for each pre-annotated mention by working on an [annotation task](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-annotate-documentswks_hatask) with the pre-annotated document.
* Machine learning

Uses a machine learning model to automatically annotate documents. This option is only available to you if you have created a machine learning model with Knowledge Studio already.