vania-janet/multiturn-rag-retrieval-data huggingface dataset:
For more information, see [Public and private network endpoints](https://cloud.ibm.com/docs/watson?topic=watson-public-private-endpoints).





Custom categories deprecation
: The experimental custom categories feature is deprecated and will be retired on 17 December 2019. Access to custom categories models will not be available after that date.







 November 2019 



 New features and changes 

New South Korea location
: You can now create Knowledge Studio instances in the Seoul location. As with other locations, the IBM Cloud Seoul location uses token-based Identity and Access Management (IAM) authentication.







 October 2019 



 New features and changes 

Advanced Rules workspaces (Beta) in Dallas and Frankfurt
: Advanced rules workspaces (Beta) are now available for service instances in Dallas and Frankfurt. Create advanced rules extractors and run them to extract text from documents that you upload to the workspace. To get started, see [Creating an advanced rules model](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-create-advanced-rules-model).

Dictionary suggestions
: As you add entries to a dictionary, Knowledge Studio will search the documents in your workspace to recommend additional entries that are likely to be in the same domain.







 September 2019 



 New features and changes 

Rule-based model versions increased
: Increased the maximum number of rule-based model versions per workspace from 5 to 10.

Updated visual style
: Updated the visual style of the Knowledge Studio application.







 July 2019 



 New features and changes 

Annotation tasks are no longer required to create ground truth
: Admins and Project Managers can annotate document sets directly from the Ground Truth tab on the Annotations page.
: Annotation tasks are still available. You can manage them from the Annotation Tasks tab on the Annotations page.
: Annotations applied directly to ground truth are not applied to related active annotation tasks. It is recommended that you annotate document sets directly only when they are not associated with any active annotation tasks.
: The Annotation Sets tab has been removed from the Documents page.
AU-2 (d) <br><br> * Check whether App ID runtime activity capture is enabled<br> * Check that Activity Tracker Event Routing is configured to collect global events generated by IBM Cloud services<br><br><br> 







 NIST supplemental guidance 

An event is any observable occurrence in an organizational information system. Organizations identify audit events as those events which are significant and relevant to the security of information systems and the environments in which those systems operate in order to meet specific and ongoing audit needs. Audit events can include, for example, password changes, failed logons, or failed accesses related to information systems, administrative privilege usage, PIV credential usage, or third-party credential usage. In determining the set of auditable events, organizations consider the auditing appropriate for each of the security controls to be implemented. To balance auditing requirements with other information system needs, this control also requires identifying that subset of auditable events that are audited at a given point in time. For example, organizations may determine that information systems must have the capability to log every file access both successful and unsuccessful, but not activate that capability except for specific circumstances due to the potential burden on system performance. Auditing requirements, including the need for auditable events, may be referenced in other security controls and control enhancements. Organizations also include auditable events that are required by applicable federal laws, Executive Orders, directives, policies, regulations, and standards. Audit records can be generated at various levels of abstraction, including at the packet level as information traverses the network. Selecting the appropriate level of abstraction is a critical aspect of an audit capability and can facilitate the identification of root causes to problems. Organizations consider in the definition of auditable events, the auditing necessary to cover related events such as the steps in distributed, transaction-based processes (e.g., processes that are distributed across multiple organizations) and actions that occur in service-oriented architectures.
The prosody target models help guide the search toward those units that meet the prosodic criteria that are predicted by the models.
* Search - Given the list of candidates that are returned by the acoustic model and the target prosody, this module carries out a Viterbi search. The search extracts a sequence of acoustic units that minimizes a cost function, which considers both concatenation and target costs. As a result, audible artifacts from joining two units are minimized, and the module tries to approximate the target prosody suggested by the prosody target models. This search also favors contiguous chunks in the synthesis corpus to further reduce such artifacts.
* Waveform generation - When the search returns the optimal sequence of units, the system uses time-domain Pitch Synchronous Overlap and Add (PSOLA) to generate the output waveform. PSOLA is a digital signal-processing technique that is used for speech processing. Specifically, it is used for speech synthesis. It can modify the pitch and duration of a speech signal and blend the units that are returned by the search in a seamless way.

For all of the linguistic features in the previous back-end processes, the service uses a text-processing front end to parse the text before it synthesizes the text into audio form. The front end sanitizes the text of any formatting artifacts such as HTML tags. It then uses a proprietary language that is driven by language-dependent linguistic rules to prepare the text and generate pronunciations. This module normalizes language-dependent features of the text such as dates, times, numbers, and currency. For example, it performs abbreviation expansion from a dictionary and numeric expansion from rules for ordinals and cardinals.

Some words have multiple permissible pronunciations, so the text-processing front-end first produces a single, canonical pronunciation at run time. This approach might not reflect the pronunciation that the speaker used when the audio corpus was recorded. So the service augments a candidate set of pronunciations with alternative forms that are inventoried in an alternate-baseform dictionary. It lets the search choose forms that reduce cost in terms of pitch, duration, and contiguity concerns and constraints. This algorithm facilitates selection of longer contiguous chunks from the data set, resulting in an optimal flow of speech in the synthesized result.
Activity Tracker events 

As a security officer, auditor, or manager, you can use the IBM Cloud® Activity Tracker service to track how users and applications interact with IBM Cloud® Virtual Private Cloud (VPC).

IBM Cloud Activity Tracker records user-initiated activities that change the state of a service in IBM Cloud. You can use this service to investigate abnormal activity and critical actions and to comply with regulatory audit requirements. In addition, you can be alerted about actions as they happen. The events that are collected comply with the Cloud Auditing Data Federation (CADF) standard. For more information, see the [getting started tutorial for IBM Cloud Activity Tracker](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-getting-started).



 Network resources 

The following tables list the actions that are related to network resources and the generation of events.



 ACL events 



Table 1. Actions that generate events for Network ACL

 Resource Action Description 

 network-acl is.network-acl.network-acl.create Network ACL was created 
 network-acl is.network-acl.network-acl.update Network ACL was updated 
 network-acl is.network-acl.network-acl.delete Network ACL was deleted 
 network-acl is.network-acl.network-acl.read One or more network ACL was retrieved 
 network-acl is.network-acl.rule.create Rule was added to Network ACL 
 network-acl is.network-acl.rule.update Network ACL Rule was updated 
 network-acl is.network-acl.rule.delete Rule was removed from Network ACL 
 network-acl is.network-acl.rule.read One or more network ACL rules was retrieved 







 Custom route events 

The following table lists the actions that are related to VPC routing tables and routes.



Table 2. Actions that generate events for VPC routing tables and routes

 Resource Action Description 

 routing table is.vpc.routing-table.create Routing table was created 
 routing table is.vpc.routing-table.update Routing table was updated
For more information about the deprecation of the concatenative voices, see the [2 December 2020 service update](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-release-notestext-to-speech-2december2020) in the release notes for IBM Cloud.

Concatenative synthesis relies on an inventory of acoustic units from a large synthesis corpus to produce output speech for arbitrary input text. It is based on the following pipeline of processes. These processes facilitate an efficient, real-time search over this inventory of units followed by a post-processing of the units.



* Acoustic model - This model consists of a decision tree that is responsible for generating candidate units for the search. For each of the phones in a sequence of phones to be synthesized, the model considers the phone in the context of the preceding and following two phones. It then produces a set of acoustic units that the search evaluates for fitness. This step effectively reduces the complexity of the search by restricting it to only those units that meet some contextual criteria and discarding all others.
* Prosody target models - The prosody target models for some voices are based on Deep Recurrent Neural Networks (RNNs). For other voices, the models rely on decision trees to determine the prosody. In both cases, the models are responsible for generating target values for prosodic aspects of the speech (such as duration and intonation) given a sequence of linguistic features that are extracted from the input text. This list includes attributes such as part of speech, lexical stress, word-level prominence, and positional features (for example, the position of the syllable or word in the sentence). The prosody target models help guide the search toward those units that meet the prosodic criteria that are predicted by the models.
* Search - Given the list of candidates that are returned by the acoustic model and the target prosody, this module carries out a Viterbi search. The search extracts a sequence of acoustic units that minimizes a cost function, which considers both concatenation and target costs. As a result, audible artifacts from joining two units are minimized, and the module tries to approximate the target prosody suggested by the prosody target models.
An import statement puts objects in the context of the current module, and not in the context of the current file. Therefore, a view that is imported by 1.aql inside module A is made visible to 2.aql inside the same module without the need for any additional import statements.
* All import statements must follow immediately after the module declaration, and must precede all other types of statements. Only those AQL objects that are explicitly exported from any module can be imported in another module. If this requirement is not observed, a compilation error results.
* A compilation error is introduced when an import view statement introduces a naming conflict with any create view statement or other import statements within the same module (not just within the current file). This restriction applies to the import of other objects in addition to views.
* The AQL compiler adheres to the naming conventions that are used in previous versions of AQL:



* A module cannot contain a view and a table with the same name.
* A dictionary with the same name as a table or view is allowed.
* A function with the same name as a table, view, or dictionary is allowed.



* A compilation error is introduced when different import statements within one or multiple AQL files in the module give the same name to different AQL objects.
* A compilation error is introduced when an AQL file inside a module tries to reference another exported view of a module without using the import view statement. The same applies to dictionaries, tables, or functions.
* When two AQL files inside a module import the same view X from another module under two different aliases, for example, A and B, then the two aliases are treated synonymously. This rule applies also to tables, dictionaries, and functions.







 Examples 

Example 1: Create views that you export to be imported into other modules.

This example creates two views, FirstName, and NotFirstName. The view FirstName collects information about the given names that are represented in the FirstNamesDictionary dictionary. The second view collects the names that are left when you exclude the given names. Two dictionaries are necessary to make the text extraction easier. One dictionary contains all of the given names that you want to search for.
Click Next to begin app registration.





 App registration 



1. Go to the [Microsoft Azure portal](https://portal.azure.com/), and log in with your admin credentials.
2. On the App registrations page, click New registration.
3. On the Register an application page, enter a name, select the multi-tenant option that applies to your app, and then click Register.
4. Copy the application ID from the Overview page of your app, and paste it into the App registration field of your Watson Assistant Microsoft Teams integration.
5. On the same Microsoft Azure Overview page, click the hyperlink Add a certificate or secret next to Client Credentials.
6. On the Certificates & secrets page for token creation, click New client secret. Enter a description and then select Recommended 180 days. Click Add.
7. Copy the string under Value and paste into Client secret value on the App registration page of your Watson Assistant Microsoft Teams integration. Note: You must generate a new value before the current one expires on day 180.
8. Click Next to create your bot.







 Create your bot 



1. Go to the [Microsoft Bot Framework developer portal](https://dev.botframework.com/bots/new), and log in with your admin credentials.
2. On the Tell us about your bot page, complete your bot profile.
3. Copy the generated endpoint from the Create your bot page of your Watson Assistant Microsoft Teams integration and paste into the Messaging endpoint field of the Configuration section.
4. Select Multi-Tenant as the app type.
5. Copy and paste your app ID, and then click Register.
6. On the Connect to channels page, click Configure Microsoft Teams channel in the Add a featured channel section.
7. On the Configure Microsoft Teams page, specify options in the Messaging, Calling, and Publish tabs that fit your bot needs, and then click Save.
8. In your Watson Assistant Microsoft Teams integration, click Next to create your Teams app.







 Create your Teams app 



1.
Accessing log fields 

If fields are not specified in the request, a limited set of default fields are returned. Find the full list of all available fields using the following request.

ibmcloud cis logpull DNS_DOMAIN_ID --available-fields

Fields are passed as a comma-separated list. For example, to have "ZoneID" and "RayID", use:

ibmcloud cis logpull DNS_DOMAIN_ID --start 2019-01-02T01:00:00+00:00 --end 2019-01-02T01:00:00+00:00 --fields ZoneId,RayID



 Available Fields 

The following tables describe the fields available by log category.



 HTTP requests 

This table contains the fields available for HTTP requests.



Table 1. HTTP events

 Field Value Type 

 BotScore Cloudflare Bot Score (available for Bot Management customers; please contact your account team to enable) int 
 BotScoreSrc Underlying detection engine or source on where a Bot Score is calculated. Possible values are Not ComputedHeuristicsMachine LearningBehavioral AnalysisVerified Bot string 
 CacheCacheStatus unknownmissexpiredupdatingstalehitignoredbypassrevalidated string 
 CacheResponseBytes Number of bytes returned by the cache int 
 CacheResponseStatus HTTP status code returned by the cache to the edge; all requests (including non-cacheable ones) go through the cache; also see CacheStatus field int 
 CacheTieredFill Tiered Cache was used to serve this request bool 
 ClientASN Client AS number int 
 ClientCountry Country of the client IP address string 
 ClientDeviceType Client device type string 
 ClientIP IP address of the client string 
 ClientIPClass unknowncleanbadHostsearchEnginewhitelistgreylistmonitoringServicesecurityScannernoRecordscanbackupServicemobilePlatformtor string 
 ClientRequestBytes Number of bytes in the client request int 
 ClientRequestHost Host requested by the client string 
 ClientRequestMethod HTTP method of client request string 
 ClientRequestPath URI path requested by the client string 
 ClientRequestProtocol HTTP protocol of client request string 
 ClientRequestReferer HTTP request referrer string
fetch(request, {
cf: { cacheKey: event.request.url },
})
)
})
Show more

Remember, edge functions operating on behalf of different zones cannot affect each other's cache. You can only override cache keys when making requests within your own zone (in the previous example event.request.url was the key stored), or requests to hosts that are not on CIS. When making a request to another CIS zone (for example, belonging to a different CIS customer), that zone fully controls how its own content is cached within CIS; you cannot override it.





 Override based on origin response code 

This feature is available only to enterprise customers.

// Force response to be cached for 86400 seconds for 200 status
// codes, 1 second for 404, and do not cache 500 errors.
fetch(request, {
cf: { cacheTtlByStatus: { "200-299": 86400, 404: 1, "500-599": 0 } },
})

This option is a version of the cacheTtl feature which chooses a TTL based on the response's status code and does not automatically set cacheEverything: true. If the response to this request has a status code that matches, CIS caches for the instructed time, and override cache directives sent by the origin.



 TTL interpretation 

The following TTL values are interpreted by CIS:



* Positive values: Indicate in seconds how long CIS should cache the asset for.
* 0: The asset is cached but expires immediately (revalidate from origin every time).
* -1 or any negative value: Instructs CIS not to cache at all.











 Cache API 

Cache using the CIS Cache API. This example can also cache POST requests.

const someOtherHostname = "my.herokuapp.com"

async function handleRequest(event) {
const request = event.request
const cacheUrl = new URL(request.url)

// Hostname for a different zone
cacheUrl.hostname = someOtherHostname
[IBM Cloud Network path from client application through CIS and BIG-IP to provider's Red Hat OpenShift on IBM Cloud application](https://cloud.ibm.com/docs-content/v1/content/9ac5c38b41e0aa1cb4ccc6cc7f547751ec805221/framework-financial-services/vpc/images/f5-bigip/cis-flow.svg)

Figure 2. Network path from client application to Red Hat OpenShift on IBM Cloud application passing through CIS and BIG-IP.

The network flow goes through the following steps:



1. Client applications connect to a CIS Range application through a public hostname that is exposed by CIS. This connection is treated as a TCP connection (not an HTTPS connection) by CIS and passes through CIS without TLS termination. The Range application provides DDoS protection.
2. A CIS global load balancer routes traffic to the WAF provided by BIG-IP instances. The global load balancer has a public hostname, but it is not advertised for use.
3. The first TLS termination occurs in BIG-IP running in IBM Cloud. BIG-IP treats the connection as an HTTPS connection. The application's public certificate is used by BIG-IP for TLS termination.
4. Network traffic is reencrypted between BIG-IP and the application. The Red Hat OpenShift on IBM Cloud ingress controller is configured for pass through so that it does not terminate the TLS connection. The final TLS termination is done by the application.



The following table summarizes the fully qualified domain names (FQDNs) used by the three layers:



 Host description Example FQDN DNS How Protected 

 Range application hostname (user application hostname) www.application.example.com Managed by CIS, no explicit record CIS Layer 3/4 
 GLB hostname <uuid>-example.com Managed by CIS, no explicit record, public network Obfuscated, BIG-IP Virtual Server accepts only the Range Application Host Header
Using the compliance operator 

With the compliance operator, you can check whether the worker nodes in your cluster meet the set of criteria in the profile that is tailored to Red Hat® OpenShift® on IBM Cloud®. For more information, see the [Red Hat OpenShift documentation](https://docs.openshift.com/container-platform/4.11/security/compliance_operator/compliance-operator-understanding.html).

Looking for the CIS Kubernetes benchmark? See [Running the worker node CIS Kubernetes benchmark](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmarkcis-worker-test).

To use the compliance operator, review the OpenShift Container Platform documentation, such as the following resources.



*  [Understanding the compliance operator](https://docs.openshift.com/container-platform/4.11/security/compliance_operator/compliance-operator-understanding.html) documentation set.
*  [Self-paced compliance operator workshop in GitHub](https://github.com/openshift/compliance-operator/tree/master/doc/tutorials).
*  How does the compliance operator work? [Part 1](https://cloud.redhat.com/blog/how-does-compliance-operator-work-for-openshift-part-1) and [Part 2](https://cloud.redhat.com/blog/how-does-compliance-operator-work-for-openshift-part-2) blogs.
HIPAA advisory 

Use of caching (CDN) for regulated data (for example, PHI, ITAR) is prohibited. All regulated data flows that use IBM Cloud® Internet Services must be set to not cache.

There are multiple ways to disable content caching by the CIS CDN.



*  The origin server can set no-cache in the Cache-Control header for the regulated content
*  Use the CIS page rules to disable caching for any content on a specified path, even if the origin does not send a no-cacheCache-Control header.
To learn how to set up your account for connecting to a private network, see [Enabling VRF and service endpoints](https://cloud.ibm.com/docs/account?topic=account-vrf-service-endpoint).







 Step 3. Create a CIS resource on the private network 

Test your private network connection by using the [CIS CLI plug-in](https://cloud.ibm.com/docs/cis-cli-plugin?topic=cis-cli-plugin-cis-cli).



1. Log into a CIS private endpoint instance.

ibmcloud login -a private.cloud.ibm.com









 Using the API 

Use the service endpoint's FQDN api.private.cis.cloud.ibm.com in the URL to access the service. For example:

curl https://api.private.cis.cloud.ibm.com/v1/:crn/zones -H 'content-type: application/json' -H 'accept: application/json' -H 'x-auth-user-token: Bearer xxxxxx'