vania-janet/multiturn-rag-retrieval-data huggingface dataset:
From the status menu, select Completed, and then click the Save button.
9. Click Open document list to return to the list of documents for this task and click Submit All Documents to submit the documents for approval.

In a realistic situation, you would create many more annotations and complete all the documents in the set before submitting.
10. Close this annotation set, and then open the other annotation set in the Test task.

Depending on how you set up the annotation tasks and which users you assigned them to, you might need to log in to Knowledge Studio as the user who is assigned to the other annotation set in the annotation task.
11. Repeat the same annotations in the Technology - gmanews.tv document, except this time, use the employedBy relation instead of the founderOf relation.

Logging in as another user will help illustrate inter-annotator agreement in the next lesson. But if you have only one user, you can still complete the tutorial to get an understanding of how inter-annotator agreement works.
12. After you complete the annotations for the second annotation set, click Submit All Documents.









 Lesson 5: Analyzing inter-annotator agreement 

In this lesson, you will learn how to compare the work of multiple human annotators in Knowledge Studio.



 About this task 

To determine whether different human annotators are annotating overlapping documents consistently, review the inter-annotator agreement (IAA) scores.

Knowledge Studio calculates IAA scores by examining all overlapping documents in all document sets in the task, regardless of the status of the document sets. The IAA scores show how different human annotators annotated mentions, relations, and coreference chains. It is a good idea to check IAA scores periodically and verify that human annotators are consistent with each other.

In this tutorial, the human annotators submitted all the document sets for approval. If the inter-annotator agreement scores are acceptable, you can approve the document sets. If you reject a document set, it is returned to the human annotator for improvement.
The default speaking rate for a voice is equivalent to passing a value of 0%.
* A number with no unit designation specifies the number of words per minute. The number is absolute; you cannot specify a relative increase or decrease in words per minute.



* For the expressive neural voices, the pitch and rate attributes support only percentage values.



* For the pitch attribute, do not use Hertz, semitones, or keywords.
* For the rate attribute, do not use words per minute or keywords.





For more information about the features supported by SSML version 1.1, refer to section [3.2.4 prosody Element](https://www.w3.org/TR/speech-synthesis/S3.2.4) of the SSML specification.





 The pitch attribute 

The pitch attribute modifies the baseline pitch, or tone, for the text within the element. Accepted values are



* A number followed by the Hz (Hertz) designation: The baseline pitch is transposed (up or down) to the specified value. For example, 150Hz.
* A relative change in percent: A number that causes a relative shift from the default baseline. The number is preceded by + (an increase) or - (a decrease) and followed by a % (percent sign). An unsigned number that is followed by a % is interpreted as a positive increase. For example, +10% or 10%. The default pitch for a voice is equivalent to passing a value of 0%.
* A relative change in semitones: A number that causes an absolute shift from the default baseline. The number is preceded by + (an increase) or - (a decrease) and followed by st (semitones). An unsigned number followed by st is interpreted as a positive increase. For example, +5st or 5st.
* A keyword: One of the following six keywords, which modify the pitch to the corresponding predefined values:



* default uses the service's default baseline pitch.
* x-low shifts the pitch baseline down by 12 semitones.
* low shifts the pitch baseline down by six semitones.
* medium produces the same behavior as default.
Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.1.1 Ensure that the cluster-admin role is only used where required. Not Scored 1 Pass Shared 
 5.1.2 Minimize access to secrets . Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) Shared 
 5.1.3 Minimize wildcard use in Roles and ClusterRoles. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) Shared 
 5.1.4 Minimize access to create pods. Not Scored 1 Pass Shared 
 5.1.5 Ensure that default service accounts are not actively used. Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) Shared 
 5.1.6 Ensure that Service Account Tokens are only mounted where necessary. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) Shared 







 5.2 Pod security policies 



Section 5.2 Pod Security Policies benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.2.1 Minimize the admission of privileged containers. Not Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) Shared 
 5.2.2 Minimize the admission of containers wanting to share the host process ID namespace. Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) Shared
1.2.19 Ensure that the --insecure-port option is set to 0. Scored 1 Pass IBM 
 1.2.20 Ensure that the --secure-port option is not set to 0. Scored 1 Pass IBM 
 1.2.21 Ensure that the --profiling option is set to false. Scored 1 Pass IBM 
 1.2.22 Ensure that the --audit-log-path option is set. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-124cis-benchmark-remediations-124) Shared 
 1.2.23 Ensure that the --audit-log-maxage option is set to 30 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-124cis-benchmark-remediations-124) Shared 
 1.2.24 Ensure that the --audit-log-maxbackup option is set to 10 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-124cis-benchmark-remediations-124) Shared 
 1.2.25 Ensure that the --audit-log-maxsize option is set to 100 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-124cis-benchmark-remediations-124) Shared 
 1.2.26 Ensure that the --request-timeout option is set as appropriate. Scored 1 Pass IBM 
 1.2.27 Ensure that the --service-account-lookup option is set to true. Scored 1 Pass IBM 
 1.2.28 Ensure that the --service-account-key-file option is set as appropriate. Scored 1 Pass IBM 
 1.2.29 Ensure that the --etcd-certfile and --etcd-keyfile options are set as appropriate. Scored 1 Pass IBM
1.2.19 Ensure that the --insecure-port option is set to 0. Scored 1 Pass IBM 
 1.2.20 Ensure that the --secure-port option is not set to 0. Scored 1 Pass IBM 
 1.2.21 Ensure that the --profiling option is set to false. Scored 1 Pass IBM 
 1.2.22 Ensure that the --audit-log-path option is set. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 1.2.23 Ensure that the --audit-log-maxage option is set to 30 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 1.2.24 Ensure that the --audit-log-maxbackup option is set to 10 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 1.2.25 Ensure that the --audit-log-maxsize option is set to 100 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-122cis-benchmark-remediations-122) Shared 
 1.2.26 Ensure that the --request-timeout option is set as appropriate. Scored 1 Pass IBM 
 1.2.27 Ensure that the --service-account-lookup option is set to true. Scored 1 Pass IBM 
 1.2.28 Ensure that the --service-account-key-file option is set as appropriate. Scored 1 Pass IBM 
 1.2.29 Ensure that the --etcd-certfile and --etcd-keyfile options are set as appropriate. Scored 1 Pass IBM
1.2.19 Ensure that the --insecure-port option is set to 0. Scored 1 Pass IBM 
 1.2.20 Ensure that the --secure-port option is not set to 0. Scored 1 Pass IBM 
 1.2.21 Ensure that the --profiling option is set to false. Scored 1 Pass IBM 
 1.2.22 Ensure that the --audit-log-path option is set. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-123cis-benchmark-remediations-123) Shared 
 1.2.23 Ensure that the --audit-log-maxage option is set to 30 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-123cis-benchmark-remediations-123) Shared 
 1.2.24 Ensure that the --audit-log-maxbackup option is set to 10 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-123cis-benchmark-remediations-123) Shared 
 1.2.25 Ensure that the --audit-log-maxsize option is set to 100 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-123cis-benchmark-remediations-123) Shared 
 1.2.26 Ensure that the --request-timeout option is set as appropriate. Scored 1 Pass IBM 
 1.2.27 Ensure that the --service-account-lookup option is set to true. Scored 1 Pass IBM 
 1.2.28 Ensure that the --service-account-key-file option is set as appropriate. Scored 1 Pass IBM 
 1.2.29 Ensure that the --etcd-certfile and --etcd-keyfile options are set as appropriate. Scored 1 Pass IBM
When decreasing the pitch, a value of -50 halves the baseline pitch; lower values might not yield appreciable or positive results. Similarly, when increasing the pitch, a value of 100 doubles the pitch; higher values might produce results that are not discernibly different from 100.

Table 2 provides examples of how the service changes the speaking pitch with different values for the pitch_percentage parameter. The examples use the en-US_AllisonV3Voice.



Table 2. Using the pitch_percentage parameter

 Specification of the pitch_percentage parameter Audio sample 

 pitch_percentage=0 <br>The service uses the default speaking pitch. You can also omit the parameter to get the default behavior. Your browser does not support the audio tag. 
 pitch_percentage=-20 <br>The service uses a speaking pitch that is 20 percent lower. Your browser does not support the audio tag. 
 pitch-percentage=20 <br>The service uses a speaking pitch that is 20 percent higher. Your browser does not support the audio tag. 





 Interaction with SSML <prosody> element 

The pitch_percentage parameter changes the pitch for an entire request. You can also use pitch attribute of the the SSML <prosody> element to modify the pitch of specific words of text. Although the pitch_percentage parameter accepts only a percentage value, the pitch attribute accepts values of different forms. For more information, see [The pitch attribute](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-elementsprosody-pitch).

If you use the pitch_percentage parameter and also use the pitch attribute of the <prosody> tag to modify the pitch for specific text of the same request, the effects are multiplicative. For example, suppose you specify the pitch_percentage parameter with a value of 10, which increases the pitch by 10 percent. If you also use the <prosody> element to specify an equivalent pitch of +10% for specific text of the request, that text is spoken with a pitch that is 121 percent of the default baseline pitch for the voice.





 Speaking pitch examples
Scored 2 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 







 5.4 Secrets management 



Section 5.4 Secrets management benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.4.1 Prefer using secrets as files over secrets as environment variables. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 
 5.4.2 Consider external secret storage Not Scored 2 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 







 5.5 Extensible admission control 



Section 5.5 Extensible admission control benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.5.1 Configure Image Provenance using ImagePolicyWebhook admission controller. Not Scored 2 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 







 5.6 General policies 



Section 5.6 General policies benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.6.1 Create administrative boundaries between resources using namespaces. Not Scored 1 Pass Shared 
 5.6.2 Ensure that the seccomp profile is set to docker/default in your pod definitions. Not Scored 2 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-410ibm-remediations-and-explanations-410) Shared 
 5.6.3 Apply security context to your pods and containers.
Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.1.1 Ensure that the cluster-admin role is only used where required. Not Scored 1 Pass Shared 
 5.1.2 Minimize access to secrets. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-47cis-benchmark-remediations-47) Shared 
 5.1.3 Minimize wildcard use in Roles and ClusterRoles. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-47cis-benchmark-remediations-47) Shared 
 5.1.4 Minimize access to create pods. Not Scored 1 Pass Shared 
 5.1.5 Ensure that default service accounts are not actively used. Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-47cis-benchmark-remediations-47) Shared 
 5.1.6 Ensure that Service Account Tokens are only mounted where necessary. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-47cis-benchmark-remediations-47) Shared 







 5.2 Pod Security Policies 



Section 5.2 Pod Security Policies benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.2.1 Minimize the admission of privileged containers. Not Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-47cis-benchmark-remediations-47) Shared 
 5.2.2 Minimize the admission of containers wanting to share the host process ID namespace. Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-47cis-benchmark-remediations-47) Shared 
 5.2.3 Minimize the admission of containers wanting to share the host IPC namespace.
Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.1.1 Ensure that the cluster-admin role is only used where required. Not Scored 1 Pass Shared 
 5.1.2 Minimize access to secrets. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-48cis-benchmark-remediations-48) Shared 
 5.1.3 Minimize wildcard use in Roles and ClusterRoles. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-48cis-benchmark-remediations-48) Shared 
 5.1.4 Minimize access to create pods. Not Scored 1 Pass Shared 
 5.1.5 Ensure that default service accounts are not actively used. Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-48cis-benchmark-remediations-48) Shared 
 5.1.6 Ensure that Service Account Tokens are only mounted where necessary. Not Scored 1 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-48cis-benchmark-remediations-48) Shared 







 5.2 Pod Security Policies 



Section 5.2 Pod Security Policies benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.2.1 Minimize the admission of privileged containers. Not Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-48cis-benchmark-remediations-48) Shared 
 5.2.2 Minimize the admission of containers wanting to share the host process ID namespace. Scored 1 [Pass](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-48cis-benchmark-remediations-48) Shared 
 5.2.3 Minimize the admission of containers wanting to share the host IPC namespace.
Progress Summary 
Experiment stage: Completed
Number of completed iterations: 1

 Winner Assessment 
> If the version being validated; i.e., the baseline version, satisfies the experiment objectives, it is the winner.
> Otherwise, there is no winner.
Winning version: my-app

 Objective Assessment 
> Identifies whether or not the experiment objectives are satisfied by the most recently observed metrics values for each version.
+--------------------------------------+--------+
 OBJECTIVE MY-APP 
+--------------------------------------+--------+
 iter8-system/mean-latency <= true 
 200.000 
+--------------------------------------+--------+
 iter8-system/error-rate <= true 
 0.010 
+--------------------------------------+--------+
 iter8-system/latency-95th-percentile true 
 <= 500.000 
+--------------------------------------+--------+

 Metrics Assessment 
> Most recently read values of experiment metrics for each version.
+--------------------------------------+--------+
 METRIC MY-APP
Not Scored 2 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-120cis-benchmark-remediations-120) Shared 
 5.2.7 Minimize the admission of containers with the NET_RAW capability. Not Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-120cis-benchmark-remediations-120) Shared 
 5.2.8 Minimize the admission of containers with added capabilities. Not Scored 1 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-120cis-benchmark-remediations-120) Shared 
 5.2.9 Minimize the admission of containers with capabilities assigned. Not Scored 2 [Pass](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-120cis-benchmark-remediations-120) Shared 







 5.3 Network policies and CNI 



Section 5.3 Network policies and CNI benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.3.1 Ensure that the CNI in use supports network policies. Not Scored 1 Pass IBM 
 5.3.2 Ensure that all namespaces have network policies defined. Scored 2 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-120cis-benchmark-remediations-120) Shared 







 5.4 Secrets management 



Section 5.4 Secrets management benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility 

 5.4.1 Prefer using secrets as files over secrets as environment variables. Not Scored 1 Pass Shared 
 5.4.2 Consider external secret storage. Not Scored 2 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-120cis-benchmark-remediations-120) Shared 







 5.5 Extensible admission control 



Section 5.5 Extensible admission control benchmark results

 Section Recommendation Scored/Not Scored Level Result Responsibility
1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate Scored 1 Pass IBM 
 1.3.2 Ensure that the --profiling argument is set to false Scored 1 Pass IBM 
 1.3.3 Ensure that the --use-service-account-credentials argument is set to true Scored 1 Pass IBM 
 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate Scored 1 Pass IBM 
 1.3.5 Ensure that the --root-ca-file argument is set as appropriate Scored 1 Pass IBM 
 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true Scored 2 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-126cis-benchmark-remediations-126) IBM 
 1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 Scored 1 Pass IBM 







 1.4 Scheduler 



Section 1.4 Scheduler benchmark results

 Section Recommendation Scored? Level Result Responsibility 

 1.4.1 Ensure that the --profiling argument is set to false Scored 1 Pass IBM 
 1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 Scored 1 Pass IBM 









 2 Etcd node configuration 

Review the Etcd Node Configuration results of the version 1.5 CIS Kubernetes benchmark.



Section 2 Etcd Node Configuration benchmark results

 Section Recommendation Scored? Level Result Responsibility 

 2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate Scored 1 Pass IBM 
 2.2 Ensure that the --client-cert-auth argument is set to true Scored 1 Pass IBM 
 2.3 Ensure that the --auto-tls argument is not set to true Scored 1 Pass IBM 
 2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate Scored 1 Pass IBM 
 2.5 Ensure that the --peer-client-cert-auth argument is set to true Scored 1 Pass IBM
1.2.19 Ensure that the --insecure-port option is set to 0. Scored 1 Pass IBM 
 1.2.20 Ensure that the --secure-port option is not set to 0. Scored 1 Pass IBM 
 1.2.21 Ensure that the --profiling option is set to false. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-119cis-benchmark-remediations-119) IBM 
 1.2.22 Ensure that the --audit-log-path option is set. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-119cis-benchmark-remediations-119) Shared 
 1.2.23 Ensure that the --audit-log-maxage option is set to 30 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-119cis-benchmark-remediations-119) Shared 
 1.2.24 Ensure that the --audit-log-maxbackup option is set to 10 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-119cis-benchmark-remediations-119) Shared 
 1.2.25 Ensure that the --audit-log-maxsize option is set to 100 or as appropriate. Scored 1 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-119cis-benchmark-remediations-119) Shared 
 1.2.26 Ensure that the --request-timeout option is set as appropriate. Scored 1 Pass IBM 
 1.2.27 Ensure that the --service-account-lookup option is set to true. Scored 1 Pass IBM 
 1.2.28 Ensure that the --service-account-key-file option is set as appropriate. Scored 1 Pass IBM