vania-janet/multiturn-rag-retrieval-data huggingface dataset:
You can get the overall status of the pipeline run from the environment variable PIPELINE_STATUS. The value can be "Succeeded" , "Failed" or "" (case sensitive). PIPELINE_STATUS comes from the underlying Tekton framework.
2. Because the evaluate step evaluates a pipeline run from the perspective of evidence, this evaluation might differ from the value set in PIPELINE_STATUS. To see the pipeline status based on evidence evaluation, use the environment variable finish_evaluate_exit_code.









 Stage output 

Some data is expected to be available by other stages on the workspace, so be sure to create them by using the [pipelinectl commands](https://cloud.ibm.com/docs/devsecops?topic=devsecops-pipelinectl). To make the required outputs available to the other stages and scripts in the pipeline, use pipelinectl's save methods, like save_repo, save_artifact, save_result, and set_env.



Table 4. Stage output

 Source stage Description Pipelinectl method Required 

 setup You can add more repos to the pipeline compliance checks and static code scan save_repo No 
 test To [attach test results to the compliance evidence as evidence artifacts](https://cloud.ibm.com/docs/devsecops?topic=devsecops-cd-devsecops-add-pipeline-steps), use the save_result command in this stage. save_result No 
 static-scan To [attach test results to the compliance evidence as evidence artifacts](https://cloud.ibm.com/docs/devsecops?topic=devsecops-cd-devsecops-add-pipeline-steps), use the save_result command in this stage. save_result No 
 dynamic-scan To [attach test results to the compliance evidence as evidence artifacts](https://cloud.ibm.com/docs/devsecops?topic=devsecops-cd-devsecops-add-pipeline-steps), use the save_result command in this stage. save_result No
If you query a trained collection using the Discovery Query Language, or the trained model is temporarily unavailable, the confidence number returned has a document_retrieval_strategy of untrained. The confidence score for a result with the document_retrieval_strategy of untrained is an unsupervised estimate of how relevant the document results are to the query; it is not interchangeable with the the score returned for trained collections. A trained collection can provide better answers to natural language queries than untrained collections.

Note that the confidence score is not the same as the score. It is recommended that the score not be used to set absolute thresholds, as it is a relative calculation. Instead, we recommend that applications always perform the same behavior for all results that do not include the confidence field. For example, an application might show all results without the confidence field or hide all results without the confidence field, but we recommend that you not use the value of score to show some and hide others. Because the way confidence is calculated varies between the different retrieval methods, consider the document_retrieval_strategy, when setting a threshold, and review the values before and after training.

For more information about querying, see [Getting started with querying](https://cloud.ibm.com/docs/discovery?topic=discovery-getting-started-with-querying).

For more on supervised training methods, see



* [Relevancy training](https://cloud.ibm.com/docs/discovery?topic=discovery-improving-result-relevance-with-the-tooling)
* [Continuous Relevancy Training](https://cloud.ibm.com/docs/discovery?topic=discovery-crt)
If a response does not meet the header criteria then the request is not counted towards the rate limiting rule. The header matching criteria includes following properties.

<-- <ul> -->

* name: The name of the response header to match.
* op: The operator when matching, eq means equals, ne means not equals. Valid values are eq and ne.
* value: The value of the header, which is exactly matched.

<-- </ul> -->

<-- </ul> -->

<-- </ul> -->

* threshold: The threshold that triggers the rate limit mitigations, combined with period. For example, threshold per period. Min value: 2, max value: 1000000.
* period: The time, in seconds, to count matching traffic. If the count exceeds threshold within this period the action is performed. Min value: 10, max value: 86400.
* action: The action performed when the threshold of matched traffic within the period defined is exceeded.

<-- <ul> -->

* mode: The type of action performed. Valid values are: simulate, ban, challenge, js_challenge.
* timeout: The time, in seconds, as an integer to perform the mitigation action. Timeout be the same or greater than the period. This field is valid only when mode is simulate or ban. Min value: 10, max value: 86400.
* response: Custom content-type and body to return. This overrides the custom error for the zone. This field is not required. Omission results in the default HTML error page. This field is valid only when mode is simulate or ban.

<-- <ul> -->

* content_type: The content-type of the body, which must be one of the following: text/plain, text/xml, application/json.
* body: The body to return. The content here must conform to the content_type. Max length is 10240.

<-- </ul> -->
Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 5.2.3 Minimize the admission of containers wanting to share the host IPC namespace. Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 5.2.4 Minimize the admission of containers wanting to share the host network namespace. Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 5.2.5 Minimize the admission of containers with allowPrivilegeEscalation. Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 5.2.6 Minimize the admission of root containers. Not Scored 2 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 5.2.7 Minimize the admission of containers with the NET_RAW capability. Not Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 5.2.8 Minimize the admission of containers with added capabilities. Not Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 5.2.9 Minimize the admission of containers with capabilities assigned. Not Scored 2 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 







 5.3 Network policies and CNI
1.2.24 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared 
 1.2.25 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared 
 1.2.26 Ensure that the --request-timeout argument is set as appropriate. Scored 1 Pass IBM 
 1.2.27 Ensure that the --service-account-lookup argument is set to true. Scored 1 Pass IBM 
 1.2.28 Ensure that the --service-account-key-file argument is set as appropriate. Scored 1 Pass IBM 
 1.2.29 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate. Scored 1 Pass IBM 
 1.2.30 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate. Scored 1 Pass IBM 
 1.2.31 Ensure that the --client-ca-file argument is set as appropriate. Scored 1 Pass IBM 
 1.2.32 Ensure that the --etcd-cafile argument is set as appropriate Scored 1 Pass IBM 
 1.2.33 Ensure that the --encryption-provider-config argument is set as appropriate Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared
"sentiment": {
"score": 0
},
"text": "Acme Corporation",
"relevance": 0.98389,
"type": "Company"
},
{
"count": 1,
"sentiment": {
"score": 0
},
"text": "Atlanta",
"relevance": 0.532754,
"type": "Location",
"disambiguation": {
"subtype":
"AdministrativeDivision",
"GovernmentalJurisdiction",
"OlympicHostCity",
"PlaceWithNeighborhoods",
"City"
],
"name": "Atlanta",
"dbpedia_resource": "http://dbpedia.org/resource/Atlanta"
}
},
{
"count": 1,
"sentiment": {
"score": 0
},
"text": "Georgia",
"relevance": 0.469643,
"type": "Location",
"disambiguation": {
"subtype":
"StateOrCounty"
]
}
}
]
}
}
Show more

In the preceding example, you could query the entity type by accessing enriched_text.entities.type

sentiment is calculated for entity types even if the sentiment enrichment is not selected. To learn more about sentiment scoring, see [Sentiment analysis](https://cloud.ibm.com/docs/discovery?topic=discovery-configservicesentiment-analysis).

The relevance score ranges from 0.0 to 1.0. The higher the score, the more relevant the entity. The disambiguation field contains the disambiguation information for the entity, which includes the entity subtype information and links to the resource(s), if applicable. The count is the number of times the entity is mentioned in the document.
Low recall scores can occur for many different reasons that depend on the domain, type system complexity, appropriateness of training documents, human annotator skills, and other factors.





 Resolving the problem 

Tune the performance of your machine learning model by performing one or more of following steps then retrain your model:



1. Identify commonly occurring types with low recall.
2. Identify commonly confused types. This information can be found by looking at the numbers that are off the diagonal in the confusion matrix.
3. Review errors where the machine learning model has high confidence.
4. Find patterns in the false positives in the confusion matrix.
5. If certain types have low recall scores, review the clarity of the annotation guidelines that apply to those types.



![How to resolve low recall scores.](https://cloud.ibm.com/docs-content/v1/content/50eddb8fd81f33092880335ff107a78ff5cd0f65/watson-knowledge-studio/images/wks_lowrecall.svg)

Figure 3. How to resolve low recall scores
1.2.24 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-412ibm-remediations-and-explanations-412) Shared 
 1.2.25 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-412ibm-remediations-and-explanations-412) Shared 
 1.2.26 Ensure that the --request-timeout argument is set as appropriate. Scored 1 Pass IBM 
 1.2.27 Ensure that the --service-account-lookup argument is set to true. Scored 1 Pass IBM 
 1.2.28 Ensure that the --service-account-key-file argument is set as appropriate. Scored 1 Pass IBM 
 1.2.29 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate. Scored 1 Pass IBM 
 1.2.30 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate. Scored 1 Pass IBM 
 1.2.31 Ensure that the --client-ca-file argument is set as appropriate. Scored 1 Pass IBM 
 1.2.32 Ensure that the --etcd-cafile argument is set as appropriate Scored 1 Pass IBM 
 1.2.33 Ensure that the --encryption-provider-config argument is set as appropriate Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-412ibm-remediations-and-explanations-412) Shared
Scored 1 Pass IBM 
 4.1.5 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive. Scored 1 Pass IBM 
 4.1.6 Ensure that the kubelet.conf file ownership is set to root:root. Scored 1 Pass IBM 
 4.1.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive. Scored 1 Pass IBM 
 4.1.8 Ensure that the client certificate authorities file ownership is set to root:root. Scored 1 Pass IBM 
 4.1.9 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive. Scored 1 Pass IBM 
 4.1.10 Ensure that the kubelet configuration file ownership is set to root:root. Scored 1 Pass IBM 







 4.2 Kubelet 



Section 4.2 Kubelet benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 4.2.1 Ensure that the --anonymous-auth option is set to false. Scored 1 Pass IBM 
 4.2.2 Ensure that the --authorization-mode option is not set to AlwaysAllow. Scored 1 Pass IBM 
 4.2.3 Ensure that the --client-ca-file option is set as appropriate. Scored 1 Pass IBM 
 4.2.4 Ensure that the --read-only-port option is set to 0. Scored 1 Pass IBM 
 4.2.5 Ensure that the --streaming-connection-idle-timeout option is not set to 0. Scored 1 Pass IBM 
 4.2.6 Ensure that the --protect-kernel-defaults option is set to true. Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) IBM 
 4.2.7 Ensure that the --make-iptables-util-chains option is set to true. Scored 1 Pass IBM 
 4.2.8 Ensure that the --hostname-override option is not set.
To explore how consistently a pair of human annotators annotated entities, relations, or coreferences in specific overlapping documents, select Document in the second menu and then select the pair of annotators that you want to evaluate.
6. After reviewing the scores, you can decide whether you want to approve or reject annotation sets that are in Submitted status. After an annotation set is submitted, a check box is displayed next to its name. Take one of these actions:



* If the inter-annotator agreement scores are acceptable for an annotation set, select the check box and click Accept. Documents that do not overlap with other annotation sets are promoted to ground truth. Documents that do overlap must first be reviewed through adjudication so that conflicts can be resolved.
* If the inter-annotator agreement scores are not acceptable for a annotation set, select the check box and click Reject. The annotation set needs to be revisited by the human annotator to improve the annotations.











 Adjudication 

If multiple human annotators work on the same document, you'll likely want to resolve inconsistencies between the annotations before promoting the annotations to ground truth. This conflict resolution process is known as adjudication.

When you approve annotation sets that are submitted by human annotators, documents that do not overlap between the annotation sets are promoted to ground truth. Before promoting documents that overlap, you should check the annotations for conflicts. When you find instances of disagreement, you must decide how to resolve the conflict, either by selecting the correct annotation from those applied by the human annotators or by selecting a different annotation to apply.

A document is available for adjudication when at least one of the following conditions is true:



* The project manager approves two or more annotation sets in a task at the same time, and the same document exists in at least two of the annotation sets (an overlapping document).
* The project manager approves another annotation set before documents in the previously approved annotation sets are adjudicated.
1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true. Scored 2 Pass IBM 
 1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1. Scored 1 Pass IBM 







 1.4 Scheduler 



Section 1.4 Scheduler benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 1.4.1 Ensure that the --profiling argument is set to false. Scored 1 Pass IBM 
 1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1. Scored 1 Pass IBM 









 2 Etcd node configuration 

Review the Etcd Node Configuration results of the version 1.5 CIS Kubernetes benchmark.



Section 2 Etcd Node Configuration benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate. Scored 1 Pass IBM 
 2.2 Ensure that the --client-cert-auth argument is set to true. Scored 1 Pass IBM 
 2.3 Ensure that the --auto-tls argument is not set to true. Scored 1 Pass IBM 
 2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate. Scored 1 Pass IBM 
 2.5 Ensure that the --peer-client-cert-auth argument is set to true. Scored 1 Pass IBM 
 2.6 Ensure that the --peer-auto-tls argument is not set to true. Scored 1 Pass IBM 
 2.7 Ensure that a unique Certificate Authority is used for etcd. Not Scored 2 Pass IBM 







 3 Control plane configuration 

Review the Control Plane Configuration results of the version 1.5 CIS Kubernetes benchmark.



 3.1 Authentication and authorization 



Section 3.1 Authentication and Authorization benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 3.1.1 Client certificate authentication should not be used for users. Not Scored 2 Pass Shared 







 3.2 Logging 



Section 3.2 Logging benchmark results
To learn more about sentiment scoring, see [Sentiment analysis](https://cloud.ibm.com/docs/discovery?topic=discovery-configservicesentiment-analysis).

The relevance score ranges from 0.0 to 1.0. The higher the score, the more relevant the entity. The disambiguation field contains the disambiguation information for the entity, which includes the entity subtype information and links to the resource(s), if applicable. The count is the number of times the entity is mentioned in the document.



 Using a custom entity model 

If you wish to create a custom enrichment model, you can do so in IBM Watson™ Knowledge Studio and import the model into Discovery by adding the ID in the Custom Model ID box of the Discovery tooling. For more information about integrating with IBM Watson™ Knowledge Studio, see [Integrating with IBM Watson™ Knowledge Studio](https://cloud.ibm.com/docs/discovery?topic=discovery-integrating-with-wks). The custom IBM Watson™ Knowledge Studio model overrides the default Entity Extraction enrichment.

Only one IBM Watson™ Knowledge Studio model can be assigned to an enrichment.







 Relation extraction 

Recognizes when two entities are related and identifies the type of relation. You can also create and add a [custom relation model](https://cloud.ibm.com/docs/discovery?topic=discovery-configservicecustom-relation-model) with IBM Watson™ Knowledge Studio.

Example portion of a document enriched with Relation Extraction:

{
"text": "The stockholders were pleased that Acme Corporation plans to build a new factory in Atlanta, Georgia.",
"enriched_text": {
"relations": [
{
"type": "locatedAt",
"sentence": "The stockholders were pleased that Acme Corporation plans to build a new factory in Atlanta, Georgia.",
"score": 0.989245,
"arguments":
{
"text": "Atlanta",
"location":
94,
101
],
"entities":
{
4.2.6 Ensure that the --protect-kernel-defaults argument is set to true. Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) IBM 
 4.2.7 Ensure that the --make-iptables-util-chains argument is set to true. Scored 1 Pass IBM 
 4.2.8 Ensure that the --hostname-override argument is not set. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) IBM 
 4.2.9 Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture. Not Scored 2 Pass IBM 
 4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate. Scored 1 Pass IBM 
 4.2.11 Ensure that the --rotate-certificates argument is not set to false. Scored 1 Pass IBM 
 4.2.12 Ensure that the RotateKubeletServerCertificate argument is set to true. Scored 1 Pass IBM 
 4.2.13 Ensure that the Kubelet makes use of only Strong Cryptographic Ciphers. Not Scored 1 Pass IBM 









 5 Kubernetes policies 

Review the Kubernetes Policies results of the version 1.5 CIS Kubernetes benchmark.



 5.1 RBAC and service accounts 



Section 5.1 RBAC and Service Accounts benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.1.1 Ensure that the cluster-admin role is only used where required. Not Scored 1 Pass Shared 
 5.1.2 Minimize access to secrets . Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) Shared
For example, you can now expect the same confidence score for the following two inputs: ik ben een kleine krijger? and ik ben een kleine krijger. In this example, the question mark (?) doesn't affect the confidence score.

Improved enhanced intent detection
: The exact match in enhanced intent detection now better handles small differences between training examples and runtime utterances when the differences do not change the meaning of a sentence.

For example, suppose in your training examples, covid-19 is in the covid intent and @doctortype_facilitytype around Palm Beach is in the find_provide_master intent. In this example, the @doctortype_facilitytype direct entity reference contains entity values, including hospital. At run time, covid19 is predicted as 100% confident for the covid intent, and hospital around palm beach is predicted as 100% confident for the find_provide_master intent.

This update applies to the following languages: English, French, Spanish, Italian, and the universal language model. For more information, see [Accessing intents](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-expression-languageexpression-language-intent).





 30 March 2022 

IBM Watson® Assistant Cartridge for IBM Cloud Pak® for Data 4.0.7 is available
: Watson Assistant for IBM Cloud Pak® for Data 4.0.7 is compatible with IBM Cloud Pak® for Data 4.0 on Red Hat OpenShift 4.8. This release of Watson Assistant for IBM Cloud Pak® for Data includes various fixes and features.

Enhanced irrelevance detection update
: We revised the enhanced irrelevance detection classification algorithm. Now, enhanced irrelevant detection uses any provided counterexamples in training. This change does not affect workspaces without counterexamples. This update applies to the following languages: English, French, Spanish, Italian, and the universal language model. For more information, see [Defining what's irrelevant](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-irrelevance-detection).

Fuzzy matching updates
4.2.7 Ensure that the --make-iptables-util-chains option is set to true. Scored 1 Pass IBM 
 4.2.8 Ensure that the --hostname-override option is not set. Not Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-125cis-benchmark-remediations-125) IBM 
 4.2.9 Ensure that the --event-qps option is set to 0 or a level that ensures appropriate event capture. Not Scored 2 Pass IBM 
 4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file options are set as appropriate. Scored 1 Pass IBM 
 4.2.11 Ensure that the --rotate-certificates option is not set to false. Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-125cis-benchmark-remediations-125) IBM 
 4.2.12 Ensure that the RotateKubeletServerCertificate option is set to true. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-125cis-benchmark-remediations-125) IBM 
 4.2.13 Ensure that the kubelet makes use of only Strong Cryptographic Ciphers. Not Scored 1 Pass IBM 









 5 Kubernetes policies 

Review the Kubernetes Policies results of the version 1.5 CIS Kubernetes benchmark.



 5.1 RBAC and service accounts 



Section 5.1 RBAC and Service Accounts benchmark results

 Section Recommendation Scored? Level Result Responsibility 

 5.1.1 Ensure that the cluster-admin role is only used where required. Not Scored 1 Pass Shared 
 5.1.2 Minimize access to secrets. Not Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-125cis-benchmark-remediations-125) Shared