vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Scored 1 Pass IBM 
 1.1.17 Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive. Scored 1 Pass IBM 
 1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root. Scored 1 Pass IBM 
 1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root. Scored 1 Pass IBM 
 1.1.20 Ensure that the Kubernetes PKI certificate file permissions are set to 644 or more restrictive. Scored 1 Pass IBM 
 1.1.21 Ensure that the Kubernetes PKI key file permissions are set to 600. Scored 1 Pass IBM 







 1.2 API server 



Section 1.2 API server benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 1.2.1 Ensure that the --anonymous-auth option is set to false. Not Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-121cis-benchmark-remediations-121) IBM 
 1.2.2 Ensure that the --basic-auth-file option is not set. Scored 1 Pass IBM 
 1.2.3 Ensure that the --token-auth-file parameter is not set. Scored 1 Pass IBM 
 1.2.4 Ensure that the --kubelet-https option is set to true. Scored 1 Pass IBM 
 1.2.5 Ensure that the --kubelet-client-certificate and --kubelet-client-key options are set as appropriate. Scored 1 Pass IBM 
 1.2.6 Ensure that the --kubelet-certificate-authority option is set as appropriate. Scored 1 Pass IBM 
 1.2.7 Ensure that the --authorization-mode option is not set to AlwaysAllow. Scored 1 Pass IBM 
 1.2.8 Ensure that the --authorization-mode option includes Node. Scored 1 Pass IBM 
 1.2.9 Ensure that the --authorization-mode option includes RBAC. Scored 1 Pass IBM
Scored 1 Pass IBM 
 1.2.9 Ensure that the --authorization-mode option includes RBAC. Scored 1 Pass IBM 
 1.2.10 Ensure that the admission control plug-in EventRateLimit is set. Not Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-125cis-benchmark-remediations-125) IBM 
 1.2.11 Ensure that the admission control plug-in AlwaysAdmit is not set. Scored 1 Pass IBM 
 1.2.12 Ensure that the admission control plug-in AlwaysPullImages is set. Not Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-125cis-benchmark-remediations-125) IBM 
 1.2.13 Ensure that the admission control plug-in SecurityContextDeny is set if PodSecurityPolicy is not used. Not Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-125cis-benchmark-remediations-125) IBM 
 1.2.14 Ensure that the admission control plug-in ServiceAccount is set. Scored 1 Pass IBM 
 1.2.15 Ensure that the admission control plug-in NamespaceLifecycle is set. Scored 1 Pass IBM 
 1.2.16 Ensure that the admission control plug-in PodSecurityPolicy is set. Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-125cis-benchmark-remediations-125) IBM 
 1.2.17 Ensure that the admission control plug-in NodeRestriction is set. Scored 1 Pass IBM 
 1.2.18 Ensure that the --insecure-bind-address option is not set. Scored 1 Pass IBM 
 1.2.19 Ensure that the --insecure-port option is set to 0. Scored 1 Pass IBM 
 1.2.20 Ensure that the --secure-port option is not set to 0.
4.2.8 Ensure that the --hostname-override argument is not set Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-412ibm-remediations-and-explanations-412) IBM 
 4.2.9 Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture Not Scored 2 Pass IBM 
 4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate Scored 1 Pass IBM 
 4.2.11 Ensure that the --rotate-certificates argument is not set to false Scored 1 Pass IBM 
 4.2.12 Ensure that the RotateKubeletServerCertificate argument is set to true Scored 1 Pass IBM 
 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers Not Scored 1 Pass 









 5 Kubernetes policies 

Review the Kubernetes Policies results of the version 1.5 CIS Kubernetes benchmark.



 5.1 RBAC and service accounts 



Section 5.1 RBAC and Service Accounts benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.1.1 Ensure that the cluster-admin role is only used where required Not Scored 1 Pass Shared 
 5.1.2 Minimize access to secrets Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-412ibm-remediations-and-explanations-412) Shared 
 5.1.3 Minimize wildcard use in Roles and ClusterRoles Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-412ibm-remediations-and-explanations-412) Shared 
 5.1.4 Minimize access to create pods Not Scored 1 Pass Shared 
 5.1.5 Ensure that default service accounts are not actively used.
4.2.6 Ensure that the --protect-kernel-defaults option is set to true. Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-48cis-benchmark-remediations-48) IBM 
 4.2.7 Ensure that the --make-iptables-util-chains option is set to true. Scored 1 Pass IBM 
 4.2.8 Ensure that the --hostname-override option is not set. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-48cis-benchmark-remediations-48) IBM 
 4.2.9 Ensure that the --event-qps option is set to 0 or a level that ensures appropriate event capture. Not Scored 2 Pass IBM 
 4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file options are set as appropriate. Scored 1 Pass IBM 
 4.2.11 Ensure that the --rotate-certificates option is not set to false. Scored 1 Pass IBM 
 4.2.12 Ensure that the RotateKubeletServerCertificate option is set to true. Scored 1 Pass IBM 
 4.2.13 Ensure that the Kubelet makes use of only Strong Cryptographic Ciphers. Not Scored 1 Pass IBM 









 5 Kubernetes policies 

Review the Kubernetes Policies results of the version 1.5 CIS Kubernetes benchmark.



 5.1 RBAC and service accounts 



Section 5.1 RBAC and Service Accounts benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.1.1 Ensure that the cluster-admin role is only used where required. Not Scored 1 Pass Shared 
 5.1.2 Minimize access to secrets. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-48cis-benchmark-remediations-48) Shared
A measurement of ground truth that shows the number of words that were annotated with a given entity type or relation type out of the total number of words, whether annotated or unannotated. This statistic is not available for coreferenced mentions. This value can help you to see how prevalent mentions of this type are compared to all of the other words in your domain documents.
* Percentage of documents that contain the type

A measurement of ground truth that shows how many documents contain a given entity type or relation type. This statistic is not available for coreferenced mentions. This value can help you to assess whether the documents in the set represent the domain sufficiently. If the percentage is low for key entity types, then you might want to add more documents with mentions of under-represented types.







 Procedure 

To view performance statistics for how well the model was trained:



1. Log in as a Knowledge Studio administrator or project manager, and select your workspace.
2. Select Machine Learning Model > Performance.
3. For the mentions, relations, or coreferences, select the Detailed Statistics link.
4. In the Summary view, specify whether you want to evaluate test data or training data, and then specify the type of annotations you want to see statistics for: entity types, relation types, or coreferenced mentions. As you scroll through the data, items that have low scores are flagged and highlighted to indicate that they require investigation and improvement. The triangle warning icon indicates that the F1 value is less than the fixed value, 0.5.

For example, the F1 score for some entity types might be high because the document was annotated through pre-annotation as well as by a human annotator. But the F1 score for other entity types might be low because differences in phrasing, and differences in how human annotators interpret the text or annotation guidelines, make it more difficult for the machine learning model to recognize the pattern and apply the correct annotation.
5. In the Confusion Matrix view for test data, specify the type of annotations that you want to see statistics for: entity types or relation types. For each entity type or relation type:
The severity categories are defined based on the scan results and can be one of the following: severity:critical, severity:high, severity:medium, severity:low, severity:informational.
* The has-exempt is a VA tool-specific label that is assigned to the issue if it is exempted based on the scan result. If the exempt status is not included in the scan result, you can exempt the issue manually by assigning the exempt label and adding a link to the source of the exempt issue ticket in a comment. For more information, see [Exemptions](https://cloud.ibm.com/docs/devsecops?topic=devsecops-issue-processingdue-date-issue-options-exempt). If an issue is marked exempt, rerun the CI pipeline. Otherwise, the pipeline does not produce new evidence.



You can also add a default label for issues with the incident-labels pipeline parameter. For more information, see [Labeling issues](https://cloud.ibm.com/docs/devsecops?topic=devsecops-issue-processinglabel-issues).





 Searching on Git Repos and Issue Tracking 

When you use Git Repos and Issue Tracking, you can search for issues in various ways in the UI. For example, you can filter by texts in the title, the body, or in comments. You can also filter by any label.

For more information, see the following documentation:



* How to [search on GitLab](https://docs.gitlab.com/ee/user/search/search-issues-and-merge-requests).
* How to [search on GitHub](https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests).







 Labeling issues 

Issues that are created by the continuous integration (CI) or continuous compliance (CC) pipelines can have default labels.

Issues that are created by any scanning tool contain a default severity label. The severity levels are informational, low, medium, high, or critical.
Not Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 
 5.2.2 Minimize the admission of containers wanting to share the host process ID namespace. Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 
 5.2.3 Minimize the admission of containers wanting to share the host IPC namespace. Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 
 5.2.4 Minimize the admission of containers wanting to share the host network namespace. Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 
 5.2.5 Minimize the admission of containers with allowPrivilegeEscalation. Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 
 5.2.6 Minimize the admission of root containers. Not Scored 2 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 
 5.2.7 Minimize the admission of containers with the NET_RAW capability. Not Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 
 5.2.8 Minimize the admission of containers with added capabilities.
1.2.7 Ensure that the --authorization-mode argument is not set to AlwaysAllow. Scored 1 Pass IBM 
 1.2.8 Ensure that the --authorization-mode argument includes Node. Scored 1 Pass IBM 
 1.2.9 Ensure that the --authorization-mode argument includes RBAC. Scored 1 Pass IBM 
 1.2.10 Ensure that the admission control plug-in EventRateLimit is set. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) IBM 
 1.2.11 Ensure that the admission control plug-in AlwaysAdmit is not set. Scored 1 Pass IBM 
 1.2.12 Ensure that the admission control plug-in AlwaysPullImages is set. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) IBM 
 1.2.13 Ensure that the admission control plug-in SecurityContextDeny is set if PodSecurityPolicy is not used. Not Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) IBM 
 1.2.14 Ensure that the admission control plug-in ServiceAccount is set. Scored 1 Pass IBM 
 1.2.15 Ensure that the admission control plug-in NamespaceLifecycle is set. Scored 1 Pass IBM 
 1.2.16 Ensure that the admission control plug-in PodSecurityPolicy is set. Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) IBM 
 1.2.17 Ensure that the admission control plug-in NodeRestriction is set. Scored 1 Pass IBM
"normalized_text": "Colorado",
"start_time": 4.94,
"confidence": 0.91,
"end_time": 5.62
}
],
"tornadoes":
{
"normalized_text": "tornadoes",
"start_time": 1.52,
"confidence": 1.0,
"end_time": 2.15
}
]
},
"alternatives":
{
"confidence": 0.96,
"transcript": "several tornadoes touch down as a line of
severe thunderstorms swept through Colorado on Sunday "
}
],
"final": true
}
]
}
Show more







 Word alternatives 

The word_alternatives_threshold parameter is supported only with previous-generation models, not with next-generation models.

The word alternatives feature (also known as Confusion Networks) reports hypotheses for acoustically similar alternatives for words of the input audio. For instance, the word Austin might be the best hypothesis for a word from the audio. But the word Boston is another possible hypothesis in the same time interval. Hypotheses share a common start and end time but have different spellings and usually different confidence scores.

By default, the service does not report word alternatives. To indicate that you want to receive alternative hypotheses, you use the word_alternatives_threshold parameter to specify a probability between 0.0 and 1.0. The threshold indicates the lower bound for the level of confidence the service must have in a hypothesis to return it as a word alternative. A hypothesis is returned only if its confidence is greater than or equal to the specified threshold.

You can think of word alternatives as the timeline for a transcript that is chopped into smaller intervals, or bins. Each bin can have one or more hypotheses with different spellings and confidence. The word_alternatives_threshold parameter controls the density of the results that the service returns. Specifying a small threshold can potentially produce many hypotheses.



 Word alternatives results
[This screen capture shows two mentions connected by the relation type, "founderOf".](images/wks_tut_annotaterelation.png "This screen capture shows two mentions connected by the relation type, "founderOf".")



8. From the status menu, select Completed, and then click the Save button.
9. Click Open document list to return to the list of documents for this task and click Submit All Documents to submit the documents for approval.

> Note: In a realistic situation, you would create many more annotations and complete all the documents in the set before submitting.
10. Close this annotation set, and then open the other annotation set in the Test task.

Depending on how you set up the annotation tasks and which users you assigned them to, you might need to log in to Knowledge Studio as the user who is assigned to the other annotation set in the annotation task.
11. Repeat the same annotations in the Technology - gmanews.tv document, except this time, use the employedBy relation instead of the founderOf relation.

Logging in as another user will help illustrate inter-annotator agreement in the next lesson. But if you have only one user, you can still complete the tutorial to get an understanding of how inter-annotator agreement works.
12. After you complete the annotations for the second annotation set, click Submit All Documents.









 Lesson 5: Analyzing inter-annotator agreement 

In this lesson, you will learn how to compare the work of multiple human annotators in Knowledge Studio.



 About this task 

To determine whether different human annotators are annotating overlapping documents consistently, review the inter-annotator agreement (IAA) scores.

Knowledge Studio calculates IAA scores by examining all overlapping documents in all document sets in the task, regardless of the status of the document sets. The IAA scores show how different human annotators annotated mentions, relations, and coreference chains. It is a good idea to check IAA scores periodically and verify that human annotators are consistent with each other.
Tune the performance of your machine learning model by performing one or more of following steps, and then retraining your model:



1. Identify commonly occurring types with low accuracy.

When analyzing relations, look at both the F1 score of the relation type itself and at the F1 score of each of the two entities that participates in the relationship.
2. Identify commonly confused types. This information can be found by looking at the numbers that are off the diagonal in the confusion matrix.
3. Review errors where the machine learning model has high confidence.
4. Find patterns in the false negatives and false positives in the confusion matrix.
5. If certain types occur infrequently in the training data, add training data that contains those types.

You can determine the frequency of occurrence by checking the percentage statistics (there are three: % of annotations, % of corpus density, and % of documents) for the type.
6. If certain types have low F1 scores, review the clarity of the annotation guidelines that apply to those types.
7. Add a dictionary for types that occur infrequently in the training data.



![How to resolve low F1 scores.](https://cloud.ibm.com/docs-content/v1/content/50eddb8fd81f33092880335ff107a78ff5cd0f65/watson-knowledge-studio/images/wks_lowf1.svg) Figure 1. How to resolve low F1 scores







 Analyzing low precision scores 

Tune the performance of your machine learning model to address low precision scores. At a high level, low precision indicates a need to improve annotation consistency.



 Symptoms 

A precision score reaches its best value at 1 and worst value at 0. A low precision score indicates that the machine learning model generated incorrect annotations.





 Causes 

Low precision scores can occur for many different reasons that depend on the domain, type system complexity, appropriateness of training documents, human annotator skills, and other factors.





 Resolving the problem 

Tune the performance of your machine learning model by performing one or more of following steps then retrain your model:
The progress field is 100.
* upgrading indicates that the model is being upgraded. The progress field is 0.
* failed indicates that training of the model failed. The progress field is 0. For more information, see [Training failures for custom language models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageCreatefailedTraining-language).



Use a loop to check the status every 10 seconds until it becomes available. For more information about checking the status of a custom model, see [Listing custom language models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-manageLanguageModelslistModels-language).





 Training failures for custom language models 

Training fails to start if the service is handling another request for the custom language model. For instance, a training request fails to start with a status code of 409 if the service is



* Processing a corpus or grammar to generate a list of OOV words or to extract character sequences
* Processing custom words to validate or auto-generate sounds-like pronunciations
* Handling another training request



Training also fails to start with a status code of 400 if the custom model



* Contains no new valid training data (corpora, words, or grammars) since it was created or last trained
* Contains one or more invalid corpora, words, or grammars (for example, a custom word has an invalid sounds-like pronunciation)



If the training request fails with a status code of 400, the service sets the custom model's status to failed. Take one of the following actions:



* Use methods of the customization interface to examine the model's resources and fix any errors that you find:



* For an invalid corpus, you can correct the corpus text file and use the allow_overwrite parameter of the POST /v1/customizations/{customization_id}/corpora/{corpus_name} method to add the corrected file to the model.
3.1.1 Client certificate authentication should not be used for users Not Scored 2 Pass Shared 







 3.2 Logging 



Section 3.2 Logging benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 3.2.1 Ensure that a minimal audit policy is created Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared 
 3.2.2 Ensure that the audit policy covers key security concerns Not Scored 2 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-411ibm-remediations-and-explanations-411) Shared 









 4 Worker node security configuration 

Review the Worker Node Security Configuration results of the version 1.5 CIS Kubernetes benchmark.



 4.1 Worker node configuration files 



Section 4.1 Worker Node Configuration benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 4.1.1 Ensure that the kubelet service file permissions are set to 644 or more restrictive Scored 1 Pass IBM 
 4.1.2 Ensure that the kubelet service file ownership is set to root:root Scored 1 Pass IBM 
 4.1.3 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive Scored 1 Pass IBM 
 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root Scored 1 Pass IBM 
 4.1.5 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive Scored 1 Pass IBM 
 4.1.6 Ensure that the kubelet.conf file ownership is set to root:root Scored 1 Pass IBM 
 4.1.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive Scored 1 Pass IBM 
 4.1.8 Ensure that the client certificate authorities file ownership is set to root:root Scored 1 Pass IBM 
 4.1.9 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive Scored 1 Pass IBM