vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Change log for worker node fix pack 1.24.6_1541, released 25 October 2022 

The following table shows the changes that are in the worker node fix pack 1.24.6_1541. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.24.6_1540

 Component Previous Current Description 

 Ubuntu 18.04 packages 4.15.0-193 4.15.0-194 Worker node kernel & package updates for [CVE-2018-16860](https://nvd.nist.gov/vuln/detail/CVE-2018-16860), [CVE-2019-12098](https://nvd.nist.gov/vuln/detail/CVE-2019-12098), [CVE-2020-16156](https://nvd.nist.gov/vuln/detail/CVE-2020-16156), [CVE-2021-3671](https://nvd.nist.gov/vuln/detail/CVE-2021-3671), [CVE-2021-43618](https://nvd.nist.gov/vuln/detail/CVE-2021-43618), [CVE-2022-3116](https://nvd.nist.gov/vuln/detail/CVE-2022-3116), [CVE-2022-3515](https://nvd.nist.gov/vuln/detail/CVE-2022-3515), [CVE-2022-39253](https://nvd.nist.gov/vuln/detail/CVE-2022-39253), [CVE-2022-39260](https://nvd.nist.gov/vuln/detail/CVE-2022-39260). 
 Kubernetes N/A N/A N/A 







 Change log for worker node fix pack 1.24.6_1540, released 10 October 2022
Kubernetes v1.24.13 v1.24.14 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.26.5). 
 Kubernetes NodeLocal DNS cache 1.22.18 1.22.21 See the [Kubernetes NodeLocal DNS cache release notes]([https://github.com/kubernetes/dns/releases/tag/1.22.21](https://github.com/kubernetes/dns/releases/tag/1.22.21) 
 Portieris admission controller v0.13.4 v0.13.5 See the [Portieris admission controller release notes](https://github.com/IBM/portieris/releases/tag/v0.13.5). 







 Change log for worker node fix pack 1.24.14_1570, released 23 May 2023 

The following table shows the changes that are in the worker node fix pack 1.24.14_1570. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.24.13_1567

 Component Previous Current Description 

 Ubuntu 18.04 packages 4.15.0-210-generic 4.15.0-211-generic Worker node kernel & package updates for [CVE-2021-3979](https://nvd.nist.gov/vuln/detail/CVE-2021-3979), [CVE-2023-1118](https://nvd.nist.gov/vuln/detail/CVE-2023-1118). 
 Ubuntu 20.04 packages 5.4.0-139-generic 5.4.0-148-generic Worker node kernel & package updates for [CVE-2023-2004](https://nvd.nist.gov/vuln/detail/CVE-2023-2004).
Operator Lifecycle Manager 0.16.1 0.16.1-IKS-3 Updated image for [CVE-2020-1971](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1971). 







 Change log for worker node fix pack 1.19.7_1532, released 18 January 2021 

The following table shows the changes that are in the worker node fix pack 1.19.7_1532. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.19.5_1530

 Component Previous Current Description 

 Kubernetes v1.19.5 v1.19.7 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.19.7). 
 Ubuntu 16.04 packages 4.4.0-197-generic 4.4.0-200-generic Updated worker node image with kernel and package updates for [CVE-2018-20482](https://nvd.nist.gov/vuln/detail/CVE-2018-20482), [CVE-2019-9923](https://nvd.nist.gov/vuln/detail/CVE-2019-9923), [CVE-2020-28374](https://nvd.nist.gov/vuln/detail/CVE-2020-28374), [CVE-2020-29361](https://nvd.nist.gov/vuln/detail/CVE-2020-29361) external}, and [CVE-2020-29362](https://nvd.nist.gov/vuln/detail/CVE-2020-29362) external}.
For all supported languages, the Multilingual tokenizer uses the part-of-speech tags that are listed in the following table.



 Tag Descriptions 

 ADJ adjective 
 ADP adposition 
 ADV adverb 
 AUX auxiliary 
 CCONJ coordinating conjunction 
 DET determiner 
 INTJ interjection 
 NOUN noun 
 NUM numeral 
 PART particle 
 PRON pronoun 
 PROPN proper noun 
 PUNCT punctuation 
 SCONJ subordinating conjunction 
 SYM symbol 
 VERB verb 
 X other 







 Examples 

Example 1: Using a part of speech tag directly in an extract statement

The view EnglishNoun extracts English nouns (singular or mass) or proper nouns (singular).

create view EnglishNoun
as extract parts_of_speech 'NOUN' and 'PROPN'
with language 'en' on D.text
as noun from Document D;







 Sequence patterns 

Use the pattern extraction specification to perform pattern matching across an input document and other spans extracted from the input document.



 Syntax 

The general syntax of a sequence pattern is to first specify the pattern to be matched in the text, and then to specify what is to be returned by the extractor. The final part of the sequence pattern specifies what is the input to the pattern; it might be a column from a previously defined view, or it might be the entire document text.

pattern <pattern specification> [return clause] [with inline_match on <viewname.colname>]





 Description 



* <pattern specification>

A <pattern specification> is composed of multiple Atoms. An individual Atom can be a column from an already-defined view, a fixed string, or a regular expression. You can specify your Atoms to be optional and repeating, and specify token gaps between Atoms.

The pattern specification is part of a larger AQL statement, which includes an extract clause.

Here is a simple example of how to create a view that contains three adjacent matches from earlier defined views. In this example, the entire combination is returned, which is what group 0 refers to:

create view Money as
extract pattern <C.match> <N.match> <Q.match>
return group 0 as match
"configuration_id": "a23c467d-1212-4b3a-5555-93e788a3622a",
"name": "Example configuration",
"conversions": {
"segment": {
"enabled": true,
"selector_tags": ["h1", "h2", "h3", "h4", "h5", "h6"]
}
}
}

enabled = true turns on document segmentation.

selector_tags is an array that specifies the heading tags documents can be segmented on.



 Example 

Configuration:

"conversions": {
"segment": {
"enabled": true,
"selector_tags": ["h1", "h2"]

Original HTML document:

<html>
<head>
<title>My Document</title>
</head>
<body>
first line
<div name="section 1">
<h1>heading 1</h1>
<div name="section 2">This is the text that falls under <b>heading 1</b> and should be therefore split into its own segment.
<h2>heading 2</h2>
line under heading 2
</div>
<h3>heading 3</h3>
line under heading 3
</div>
last line
</body>
</html>
Show more

The first document segment looks like this:

{
"id": "94c686c3-a790-4d8d-ba42-2bddf01b311c",
"result_metadata": {
"parent_id": "94c686c3-a790-4d8d-ba42-2bddf01b311c",
"segment": 1,
"total_segments": 3
},
* [and mapping from <mapping table name>]

Specifies the name of an AQL table that maps raw part-of-speech tags such as "NOUN" to combinations of high-level parts of speech and flags. While the optional mapping table can have variable names, a part-of-speech mapping table is required to have these column names:



* tag

The column that holds a Multilingual tokenizer part-of-speech tag.
* basetag

The column that holds the corresponding internal tag.
* flagstr

The column that holds a comma-delimited list of flags that are associated with the indicated part of speech.



The mapping table must be defined by using the create table statement in the same module as the extract part_of_speech statement that uses it. It cannot be an imported table, and it cannot be an external table.

create table POSMapping_EN(tag Text, basetag Text, flagstr Text)
as values
('CCONJ','CONJ','coordinating'),
('SCONJ','CONJ','subordinating');
* <input column>

Specifies the column of the input view from which to extract part-of-speech information.
* <output column>

Specifies the name of the column where the spans of the tokens with the indicated parts of speech are sent.
* <input view>

Specifies the input view from which to extract part-of-speech information.







 Usage notes 



* Part of speech extraction works only when is using the Multilingual tokenizer. If the system uses the Standard tokenizer, a part_of_speech extraction generates an error.







 Parts of speech tags for languages 

For all supported languages, the Multilingual tokenizer uses the part-of-speech tags that are listed in the following table.



 Tag Descriptions 

 ADJ adjective 
 ADP adposition 
 ADV adverb 
 AUX auxiliary 
 CCONJ coordinating conjunction 
 DET determiner 
 INTJ interjection 
 NOUN noun 
 NUM numeral 
 PART particle 
 PRON pronoun 
 PROPN proper noun 
 PUNCT punctuation
Each element has five important sections:



* location: The begin and end indexes indicating the location of the element in the input document.
* text: The text of the classified element.
* types: An array that includes zero or more label objects. Each label object includes a nature field that lists the effect of the element on the identified party (for example, Right or Exclusion) and a party field that identifies the party or parties affected by the element. See [Types](https://cloud.ibm.com/docs/discovery?topic=discovery-contract_parsingcontract_types) in [Understanding contract parsing](https://cloud.ibm.com/docs/discovery?topic=discovery-contract_parsing) for additional information.
* categories: An array that contains zero or more label objects. The value of each label object lists a functional category into which the identified element falls. See [Categories](https://cloud.ibm.com/docs/discovery?topic=discovery-contract_parsingcontract_categories) in [Understanding contract parsing](https://cloud.ibm.com/docs/discovery?topic=discovery-contract_parsing) for additional information.
* attributes: An array that lists zero or more objects that define attributes of the element. Currently supported attribute types include Currency, DateTime, Duration, Location, Number, Organization, Percentage, and Person. Each object in the attributes array also includes the identified element's text and location; location is defined by the begin and end indexes of the text in the input document. See [Attributes](https://cloud.ibm.com/docs/discovery?topic=discovery-contract_parsingattributes) in [Parsing contracts](https://cloud.ibm.com/docs/discovery?topic=discovery-contract_parsing) for additional information.



Additionally, each object in the types and categories arrays includes a provenance_ids array. The values listed in the provenance_ids array are hashed values that you can send to IBM to provide feedback or receive support about the part of the analysis associated with the element.
* element_locations: An array that specifies the begin and end values of the leading sentences in the section.



* paragraphs: An array containing one object per paragraph, in parallel with the section_titles and leading_sentences arrays. Each object lists the span (beginning and end location) of the corresponding paragraph.



* location: The location of the paragraph in the input document as defined by its begin and end indexes.





* parties: An array that defines the parties that are identified by the service.



* party: A string that provides the normalized form of the party's name.
* role: A string that identifies the role of the party.
* importance: A string that identifies the importance of the party. Possible values include Primary for a primary party and Unknown for a non-primary party.
* addresses: An array of objects that identify addresses.



* text: A string that contains the address.
* location: The location of the address as defined by its begin and end indexes.



* contacts: An array that defines the name and role of contacts that are identified in the input document.



* name: A string that lists the name of an identified contact.
* role: A string that lists the role of the identified contact.



* mentions: An array of objects that identify mentions of the party.



* text: A string that lists the name of the party.
* location: The location of the mention as defined by its begin and end indexes.









 *Notes on tables 

The Element Classification enrichment is deprecated and will no longer be available, effective 10 July 2020.



* Row and column index values per cell are zero-based and so begin with 0.
* Multiple values in arrays of row_header_ids and row_header_texts elements indicate a possible hierarchy of row headers.
* Multiple values in arrays of column_header_ids and column_header_texts elements indicate a possible hierarchy of column headers.







 **Note on unit values 

The value of unit is the [ISO-4217 currency code](https://www.iso.org/iso-4217-currency-codes.html) identified for the currency amount (for example, USD or EUR).
* text: The text contents of a related material from the input document, without HTML markup.
* location: The location of the related material in the input document as defined by its begin and end indexes.





* document_structure: An object that describes the structure of the input document.



* section_titles: An array that contains one object per section or subsection that is detected in the input document. Sections and subsections are not nested. Instead, they are flattened out and can be placed back in order by using the begin and end values of the element and the level value of the section.



* text: A string that lists the section title, if detected.
* location: The location of the title in the input document as defined by its begin and end indexes.
* level: An integer that indicates the level at which the section is located in the input document. For example, represents a top-level section, represents a subsection within the level section.
* element_locations: An array that specifies the begin and end values of the sentences in the section.



* leading_sentences: An array that contains one object per leading sentence of a list or subsection, in parallel with the section_titles and paragraph arrays. The object details the leading sentences in the matching section or subsection. As in the section_titles array, the objects are not nested; instead, they are flattened out and can be placed back in order by using the begin and end values of the element or any level markers in the input document.



* text: A string that lists the leading sentence, if detected.
* location: The location of the leading sentence in the input document as defined by its begin and end indexes.
* element_locations: An array that specifies the begin and end values of the leading sentences in the section.



* paragraphs: An array containing one object per paragraph, in parallel with the section_titles and leading_sentences arrays. Each object lists the span (beginning and end location) of the corresponding paragraph.



* location: The location of the paragraph in the input document as defined by its begin and end indexes.





* parties: An array that defines the parties that are identified by the service.
</tuv>
</tu>
</body>
</tmx>
Show more

Each term and translation pair must be enclosed in <tu> tags. The xml:lang attribute in the <tuv> tag identifies the language in which a term is expressed. The <seg> tag contains the term or the translation.

<tu>
<tuv xml:lang="en">
<seg>international business machines</seg>
</tuv>
<tuv xml:lang="fr">
<seg>International Business Machines</seg>
</tuv>
</tu>

The Language Translator service uses only the <tu>, <tuv>, and <seg> elements. All other elements in the example are required to make a valid TMX file, or are informational, but are not used by the service.



 Escaping XML control characters 

The following characters are reserved as control characters in XML. You must escape these reserved characters by replacing them with equivalent XML entities.



* & must be replaced with &.
* < must be replaced with <.
* > must be replaced with >.



For example, the <seg> element can contain an inline <subtag> element. You need to escape the characters in this inline element by specifying it as <subtag>.





 Using the example as a template 

To use the provided example as a template for your own glossary or corpus, complete the following steps:



1. Copy and paste the example into a text editor.
2. Change the srclang attribute of the <header> tag to the [language code](http://www-01.sil.org/iso639-3/codes.asp) of the source language that your glossary or parallel corpus uses.
3. Change the xml:lang attribute of the <tuv> tags to the correct [language code](http://www-01.sil.org/iso639-3/codes.asp) for each term or translation.
4.
"location": {
"begin": int,
"end": int
}
},
...
]
},
...
]
},
"parties": [
{
"party": string,
"role": string,
"importance": string,
"addresses":
{
"text": string,
"location": {
"begin": int,
"end": int
}
},
...
],
"contacts":
{
"name": string,
"role": string
},
...
],
"mentions":
{
"text": string,
"location": {
"begin": int,
"end": int
}
},
...
]
},
...
]
}
Show more





 Schema arrangement 

The contracts schema is arranged as follows.



* elements: An array of the document elements detected by the service.



* location: An object that identifies the location of the element. The object contains two index numbers, begin and end. The index numbers indicate the beginning and ending positions of the characters that constitute the element in HTML.
* text: The text of the element.
* types: An array that describes what the element is and whom it affects.



* label: An object that defines the type by using a pair of the following elements:



* nature: The type of action the sentence requires. Current values are Definition, Disclaimer, Exclusion, Obligation, and Right.
* party: A string that identifies the party to whom the sentence applies.



* provenance_ids: An array of one or more hashed values that you can send to IBM to provide feedback or receive support.



* categories: An array that lists the functional categories into which the element falls; in other words, the subject matter of the element.



* label: A string that lists the identified category.
Specifies that verification never occurs before the detag operation is attempted. The system attempts to detag the target text, even if the text does not contain any HTML or XML content.



* <element name>

Specifies the name of the HTML or XML element to be annotated. The optional annotate clause can direct the runtime component to remember information about the removed tags by creating one or more views.
* <auxiliary view name>

Specifies the name of the view that is created to hold the original tags and their attributes. Can be a simple identifier or a double-quoted identifier.
* <attribute name>

Name of an attribute of the HTML or XML element.
* <column name>

The name of the column in the <auxiliary view name> that is used to store the values of <attribute name>. Can be a simple identifier or a double-quoted identifier.







 Examples 

Example 1: Specifying the detag output view and an auxiliary view

In this example, the DetaggedDoc view is created to hold the detagged version of the original text in the text attribute of the view Document. In addition to creating a DetaggedDoc view, the annotate clause creates an auxiliary view called Anchor. This auxiliary view has two columns. One column, called match, contains the anchor text. The other column, called linkTarget, contains the actual target of the link as text. The spans in the match column are over the text value of the DetaggedDoc view.

detag Document.text as DetaggedDoc
annotate
'a' as Anchor
with attribute 'href' as linkTarget;

Example 2: Using the Remap function

The following example illustrates how the Remap function is used to map spans from the detagged text back to their equivalents in the original source.

-- Strip out tags from each document, provided that the document
-- is in HTML or XML format.
-- Remember the locations and content of all <A> and <META> tags
-- in the original source document.
detag Document.text as DetaggedDoc
detect content_type always
* 1 - Pronoun
* 2 - Verb
* 3 - Noun
* 4 - Adjective
* 5 - Adverb
* 6 - Adposition
* 7 - Interjection
* 8 - Conjunction
* 9 - Determiner
* 10 - Quantifier



In English, noun (3), verb (2), and adjective (4) are the most common parts of speech that are used for dictionary entries.

> Note: The part of speech does not automatically determine the type of a mention. Do not assume that all nouns equate to entity type mentions and all verbs equate to relation type mentions. For example, American is an adjective but might be best annotated as entity type GPE (geopolitical entity) or PERSON. Met is a verb, but might be best annotated as an EVENT_MEETING.

In other languages, such as German, which uses compound words, the accuracy of the part of speech information is even more important to help determine word boundaries.
* poscode (Chinese)

Specifies a code that identifies the part of speech. The part of speech value is important for text tokenization and pre-annotation in languages like Chinese (simplified and traditional) that do not use white space to denote word boundaries.



* 32 - Noun
* 31 - Noun (Family name)
* 35 - Noun (Organization)
* 34 - Noun (Other)
* 33 - Noun (Person Name)



* poscode (Japanese)

Specifies a code that identifies the part of speech. The part of speech value is important for text tokenization and pre-annotation in languages like Japanese that do not use white space to denote word boundaries.



* 19 - Noun
* 23 - Common Prefix
* 24 - Common Suffix
* 140 - Proper Noun (Last Name)
* 141 - Proper Noun (First Name)
* 146 - Proper Noun (Person Name)
* 142 - Proper Noun (Organization)
* 144 - Proper Noun (Place Name)
* 143 - Proper Noun (Region)
* 145 - Proper Noun (Other)



* poscode (Korean)

Specifies a code that identifies the part of speech.
* 1 - Pronoun
* 2 - Verb
* 3 - Noun
* 4 - Adjective
* 5 - Adverb
* 6 - Adposition
* 7 - Interjection
* 8 - Conjunction
* 9 - Determiner
* 10 - Quantifier



In English, noun (3), verb (2), and adjective (4) are the most common parts of speech that are used for dictionary entries.

The part of speech does not automatically determine the type of a mention. Do not assume that all nouns equate to entity type mentions and all verbs equate to relation type mentions. For example, American is an adjective but might be best annotated as entity type GPE (geopolitical entity) or PERSON. Met is a verb, but might be best annotated as an EVENT_MEETING.

In other languages, such as German, which uses compound words, the accuracy of the part of speech information is even more important to help determine word boundaries.
* poscode (Chinese)

Specifies a code that identifies the part of speech. The part of speech value is important for text tokenization and pre-annotation in languages like Chinese (simplified and traditional) that do not use white space to denote word boundaries.



* 32 - Noun
* 31 - Noun (Family name)
* 35 - Noun (Organization)
* 34 - Noun (Other)
* 33 - Noun (Person Name)



* poscode (Japanese)

Specifies a code that identifies the part of speech. The part of speech value is important for text tokenization and pre-annotation in languages like Japanese that do not use white space to denote word boundaries.



* 19 - Noun
* 23 - Common Prefix
* 24 - Common Suffix
* 140 - Proper Noun (Last Name)
* 141 - Proper Noun (First Name)
* 146 - Proper Noun (Person Name)
* 142 - Proper Noun (Organization)
* 144 - Proper Noun (Place Name)
* 143 - Proper Noun (Region)
* 145 - Proper Noun (Other)



* poscode (Korean)

Specifies a code that identifies the part of speech.