vania-janet/multiturn-rag-retrieval-data huggingface dataset:
New Salesforce proxy support
: You can now connect to a Salesforce data source when using a proxy server. For more information, see [Salesforce](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-connector-salesforce-cp4d).

Improved custom connector improvements
: Support was added for Optical character recognition (OCR)
: Support was added for Document-level security

For more information about the custom connector, see [Building a Cloud Pak for Data custom connector](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-build-connector).

Change to Dynamic Faceted Search
: Support for Dynamic Faceted Search and its associated suggested_refinements API query parameter was removed.





 2.2.1 release, 26 February 2021 

New release now available
: IBM Watson™ Discovery for IBM Cloud Pak for Data version 2.2.1 is available.

Support for upgrade
: Discovery for Cloud Pak for Data supports an in-place upgrade from version 2.2.0 to 2.2.1 so that you do not need to manually uninstall an earlier version and then install the latest version of the service. For more information, see [Upgrading Discovery for Cloud Pak for Data](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-installupgrade-discovery).

New SDK download support
: You can now download the custom connector SDK package from your Discovery for Cloud Pak for Data cluster, instead of retrieving the images and the SDK package from the Docker registry. For more information, see [Downloading the custom-crawler-docs.zip file in Discovery 2.2.1 and later](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-connector-devdownload-ccs-zip).

Change to Invoices and Purchase orders
: Invoices and Purchase orders models can no longer be enabled in the tooling.
For more information, see [Getting started with Watson Discovery](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-getting-started).

New passage retrieval
: Will return the most relevant passages from your documents, plus you can specify the number of passages returned per document. See [Passages](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-query-parameterspassages).

New project-level querying and relevancy training
: Query multiple collections at once including relevance training.

Improved Web crawl connector
: Additional options now available for the Web crawl connector - For more information, see [Web crawl](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collection-typesconnectwebcrawl).

New Local File System connector
: Crawl Linux or other file systems. For more information, see [Local file system](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collection-typeslocalfilesystemconnect)

New dynamic Facets
: Automatically generate facets based on the understanding of your data. For more information, see [Facets](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-facets).

New Dictionary suggestions
: Dictionary terms are suggested based on your content. For more information, see [Dictionary](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain-dictionary).

New beta Curations
: Specify a particular result for a given query. For more information, see the [API reference](https://cloud.ibm.com/apidocs/discovery/discovery-datacreatecuration).





 2.0.1 release, 30 August 2019 

New release now available
: IBM Watson® Discovery for IBM Cloud Pak® for Data version 2.0.1 is available.
: Discovery for Cloud Pak for Data now works with IBM Cloud Pak® for Data 2.1.0.1.
All solution docs[Accelerate a dynamic website using Dynamic Content Acceleration](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-dynamic-content-cdn)Accelerate a dynamic website using Dynamic Content Acceleration Solution tutorial

Web applications are composed of static content like text, images, cascading style sheets, and JavaScript files. The tutorial Accelerate delivery of static files using a CDN shows how to host and serve static assets (images, videos, and documents) of a website from IBM Cloud Object Storage with IBM® Content Delivery Network (CDN).

![solution tutorial icon](https://cloud.ibm.com/media/docs/images/homepage/solution-tutorial.svg) [Accelerate delivery of static files using a CDN](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-static-files-cdn)Accelerate delivery of static files using a CDN Solution tutorial

This tutorial walks you through how to host and serve website assets (images, videos, documents) and user generated content in a IBM Cloud Object Storage, and how to use a IBM® Content Delivery Network (CDN) for fast and secure delivery to users around the world.

![solution tutorial icon](https://cloud.ibm.com/media/docs/images/homepage/solution-tutorial.svg) [Analyze logs and monitor application health ](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-application-log-analysis)Analyze logs and monitor application health Solution tutorial

This tutorial shows how the IBM Log Analysis service can be used to configure and access logs of a Kubernetes application that is deployed on IBM Cloud. You will deploy a Python application to a cluster provisioned on IBM Cloud Kubernetes Service, configure a logging agent, generate different levels of application logs and access worker logs, pod logs or network logs. Then, you will search, filter and visualize those logs through Log Analysis Web UI.

!
Use the Data Engine user interface (UI) to [develop your queries](https://cloud.ibm.com/docs/sql-query?topic=sql-query-running) and the [Data EngineREST API](https://cloud.ibm.com/docs/sql-query?topic=sql-query-overviewrestapi) to automate them.



 Input and output of queries 

Before you can use the Data Engine service to run SQL queries, the input data must be uploaded to one or more Cloud Object Storage instances. You must also have at least 'Writer' access to at least one Cloud Object Storage bucket, so that result objects (that is, the objects that contain output data) can be written there. For more information about Cloud Object Storage, including how to provision an instance, create buckets, and upload data, see the [Cloud Object Storage Getting Started Guide](https://cloud.ibm.com/docs/cloud-object-storage/getting-started.htmlgetting-started-console) and [Reading and writing to Cloud Object Storage](https://cloud.ibm.com/docs/sql-query/blob/draft/reading_cos.md). You can also [write to databases](https://cloud.ibm.com/docs/sql-query/blob/draft/writing_databases.md) and take advantage of [index management](https://cloud.ibm.com/docs/sql-query?topic=sql-query-index_management).





 Programmatic access 



 REST API 

You can use the [Data Engine service REST API](https://cloud.ibm.com/apidocs/sql-query/sql-query-v3) to run queries and retrieve information about their status. This is especially helpful when you write code that automatically queries data.

Note: The Cloud Resource Name (CRN) is a mandatory part of a Data Engine REST endpoint call. The CRN Copy button copies your CRN to clipboard and you can paste it into your API call.





 Python applications and Notebooks
For a list of new features and bug fixes, see [What's new and changed in Watson Discovery](https://www.ibm.com/docs/SSQNUZ_4.5.x/fixlist/discovery-fixlist.html)

Features that are not available in this release
: The following features are generally available from managed IBM Cloud deployments at the time of this release, but not from installed deployments:



* Answer finding
* Manage data page
* Advanced document view for search results







 4.0.9 release, 25 May 2022 

IBM Watson® Discovery Cartridge for IBM Cloud Pak® for Data 4.0.9 is available.

API usage information is now available from the user interface
: You can now get information about analyze API usage from the Data usage>API usage page in the product user interface. For more information about the analyze API, see [Analyze API](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-analyzeapi).

A new document status API is supported in IBM Cloud Pak® for Data instances
: Use the new document status API to programmatically get a list of the documents in a collection and to get details about a single document.



* The API is supported for collections that are created after 23 March 2022.

If you want to get status information about a collection that was created earlier, trigger a process that runs the conversion step of ingestion on the documents. For example, from the Activity page for the collection, click Recrawl.
* The API is not supported from the SDKs currently.



For more information about the new API, see the [API reference documentation](https://cloud.ibm.com/apidocs/discovery-datalistdocuments).

Security vulnerabilities were addressed
: The following security patches were applied:



* [Security Bulletin: IBM Watson Discovery for IBM Cloud Pak for Data affected by vulnerability in Node.js](https://www.ibm.com/support/pages/node/6584209)
Disclaimer 

ATTENTION: You must review the following information before you use any IBM® Blockchain plans.



 IBM support statement 

IBM's long history of leadership in innovation continues with the IBM Blockchain Platform offerings on IBM Cloud. Blockchain is a rapidly progressing technology that is projected to disrupt the financial industry, local and global supply chains, and logistical support in any number of business spaces. Through various early adoption programs, IBM customers and business partners are actively driving blockchain as an industrial solution. IBM Blockchain Platform for IBM Cloud is one such program. As with any new technology, IBM Blockchain Platform for IBM Cloud users need to be aware of the potential for rapid and fundamental change.

The architecture behind IBM Blockchain is the Linux Foundation's Hyperledger Fabric project. Each open-source community contribution improves Hyperledger Fabric but can present adoption challenges. IBM cautions against defining or exchanging financial assets directly on any Hyperledger Fabric blockchain network.

IBM does not support networks that use Hyperledger Composer in production, including the Composer CLI, JavaScript APIs, REST server, and Web Playground.





 Open-source statement 

IBM Blockchain offerings are built based on the Linux Foundation's Hyperledger Fabric stack. Hyperledger Project members, including IBM, continue to contribute to various subprojects under the Hyperledger umbrella. All contributions are reviewed, vetted, and tested by the community.





 Chaincode support statement 

The following coding practices are NOT supported on IBM Blockchain networks:



1. Using associative arrays with iteration (the order is randomized in Go).
2. Reading a list of items from a KVS table (the order isn't guaranteed).
3. Writing thread-unsafe chaincode (query and invoke might be called in parallel).
4. Substituting global memory or cache storage for ledger state variables in chaincode.
5. Accessing external services, such as databases, directly from chaincode.
6. Using libraries or global variables that could introduce non-determinism (such as using "random" or "time").
And you can create a MapReduce view with last_edit_date as the key:

function(doc) {
emit(doc.last_edit_date, null)
}

This view can be queried to return any documents that are modified on or after a supplied date and time:

/orders/_design/query/_view/by_last_edit?startkey="2022-01-13T00:00:00"

This technique produces a time-ordered set of results with no repeated values in a performant and repeatable fashion. The consumer of this data does not need to manage the data idempotently, making for a simpler development process.
* What is the IBM Cloudant changes feed good for now?

The IBM Cloudant changes feed is good for the following tasks:



* Powering IBM Cloudant replication, optionally with a selector to filter some changes.
* Clients consuming the changes feed in batches but dealing with each change idempotently while not being concerned with sort order and expecting to see some changes more than one time.



The IBM Cloudant changes feed is not good for the following components:



* A message queue. For more information, see [IBM Messages for RabbitMQ](https://www.ibm.com/cloud/messages-for-rabbitmq) for managing queues.
* A message broker. For more information, see [IBM Event Streams](https://www.ibm.com/cloud/event-streams) for handling scalable, time-ordered streams of events.
* A real-time publish and subscribe system. For more information, see [IBM Databases for Redis](https://www.ibm.com/uk-en/cloud/databases-for-redis) for handling publish and subscribe topics.
* A transaction log. Some databases store each change in a transaction log, but the distributed and eventually consistent nature of IBM Cloudant means that no definitive time-ordered transaction log exists.
* A querying mechanism.
: The latest enhancements to IBM Cloud catalog include support for filtering products by provider name. If you're looking for a specific provider's products or curious about how many products that a provider offers, you can use the Provider filter to narrow down your search. To explore the filter updates, see the [catalog](https://cloud.ibm.com/catalog).







 September 2021 



 27 September 2021 

Securing your resources with context-based restrictions
: You can now use context-based restrictions to configure and enforce access restrictions for IBM Cloud resources. The access restrictions are based on the network location where an access request is created. These restrictions work in tandem with traditional IAM access policies, which are based on identity, to provide an extra layer of protection. Since both IAM access and context-based restrictions must enforce access, context-based restrictions offer protection even in the face of compromised or mismanaged credentials.

You can create context-based restrictions by defining one or more network zones, which contain allowed network locations, and then associating the zones with the cloud resource through a context-based restrictions rule. Network zones can be defined in terms of IP address constructs, VPCs, and service references, which grant access to a resource from another cloud service.

For more information, see [What are content-based restrictions?](https://cloud.ibm.com/docs/account?topic=account-context-restrictions-whatis)





 24 September 2021 

Assigning access policies based on resource location
: For supporting services, like Container Registry, you can now scope access to more granular locations in your policies, such as geography, country, metro, satellite location.





 20 September 2021 

Checking the root cause of an incident
: You can now visit the IBM Cloud [Incident reports page](https://cloud.ibm.com/status/incident-reports) to check the health of an IBM Cloud service. You can find Customer Incident Reports (CIR) on the page, which provide Root Cause Analysis (RCA) for past major outages. You are able to view and download the incident reports from the page about any events that affect IBM Cloud availability. These reports are in PDF file formats and available for 5 years starting from the date when the event happened.
High-Availability 

IBM Cloud® Databases for etcd is a managed cloud database service that is fully integrated into the IBM Cloud environment. The database, storage, and supporting infrastructure all run in IBM Cloud.

Databases for etcd provides replication, fail-over, and high-availability features to protect your databases and data from infrastructure maintenance, upgrades, and failures. Deployments contain a cluster with three nodes. All three nodes store data and maintain cluster state with a quorum. The consensus algorithm monitors cluster health, triggers failovers if one member becomes unhealthy, and accepts a new or rejoining member. If one data member becomes unreachable, your cluster continues to operate normally. If there is permanent loss of a quorum, then the cluster is restarted and recovers to a previous state.



  Application-level High-Availability 

Applications that communicate over networks and cloud services are subject to transient connection failures. You want to design your applications to retry connections when errors are caused by a temporary loss in connectivity to your deployment or to IBM Cloud.

Because Databases for etcd is a managed service, regular updates and database maintenance occurs as part of normal operations. Maintenance can occasionally cause short intervals where your database is disabled.

Your applications must be designed to handle temporary interruptions to the database, implement error handling for failed database commands, and implement retry logic to recover from a temporary interruption.

Several minutes of database unavailability or connection interruptions are not expected. Open a [support ticket](https://cloud.ibm.com/unifiedsupport/cases/add) with details if you have time periods of no connectivity longer than a minute, so we can investigate.





  High availability, disaster recovery, and SLA resources 

Databases for etcd deployments conform to the IBM Cloud Databases [HA, DR, and SLA](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-ha-dr) information and terms.
Create a Secure Content Store 

Are you looking to store content securely (locally or globally) at an affordable cost, for things like cloud native apps, media storage, backup storage and archive data? IBM Secure Content Store powered by IBM Cloud® Object Storage provides unparalleled agility in supporting fast, highly consistent application deployment around the world to help customers securely expand their business into new regions, from business-critical data to video archive solutions. It also offers immutable storage, immutable backup, and archive data with industry-leading security and controls for regulatory/compliance requirements.



* Gain security and control over your data with encryption options, governance policies, access permissions, and context-based restrictions.
* Have immediate consistency across regions or locations for cloud-native apps, disaster recovery, storage backup, video content and delivery. etc.
* Leverage your own encryption keys (BYOK) with Key Protect.
* Monitor and retain your account & data activity with Activity Tracker and IBM Monitoring.
* APIs & SDKs, Static Web Hosting, High Speed Transfer, Tagging, Replication.





 Promotion for new customers! 

IBM Cloud is offering a $500 promotional credit to quickly get started with our Secure Content Store with Object Storage. The credit has a duration of 90 days against your metered consumption of Object Storage. See instructions below for how to apply your promo code. To qualify for this offer you must be a new paid user of Object Storage. There is a limit of one promotion code per customer account. The USD 500 credit is for use with this offer only and cannot to be applied to other offers. Offer is subject to availability.





 Overview 

This tutorial is for customers looking to set up a Secure Content Store using Object Storage, Activity Tracker, and Key Protect. In this tutorial, you are guided through the process of quickly getting started with these essential services to ensure the security and integrity of your content. Secure Content Store is comprised of the following services:



* Object Storage: a scalable and flexible storage solution that allows you to store and manage your data securely.
* Activity Tracker: a powerful tool that provides comprehensive visibility into the activities happening within your IBM Cloud environment and allows for ease of audit observability.
Getting Started 

With IBM Cloud Databases (ICD) enabled by IBM Cloud Satellite, you can deploy ICD instances into a Satellite location. (ICD) enabled by IBM Cloud Satellite currently supports the following ICD managed database services:



* [Databases for etcd](https://cloud.ibm.com/docs/databases-for-etcd)
* [Databases for PostgreSQL](https://cloud.ibm.com/docs/databases-for-postgresql)
* [Databases for Redis](https://cloud.ibm.com/docs/databases-for-redis)
* [Messages for RabbitMQ](https://cloud.ibm.com/docs/messages-for-rabbitmq)



After your Satellite location instance is deployed, ICD will install an ICD Satellite service cluster in your Satellite location into which your database instances will be deployed.

Your ICD service cluster can operate multiple database instances, even with different database types. For instance, you might operate a Databases for etcd and a Databases for PostgreSQL instance on the same service cluster.

The service cluster name is based on the first database instance, even if multiple instances exist. For example, if your service cluster has both a Databases for etcd and a Databases for PostgreSQL instance on the same service cluster, the name is based on your Databases for etcd instance.

ICD enabled by IBM Cloud Satellite supports Satellite locations on [Amazon Web Services (AWS)](https://cloud.ibm.com/docs/satellite?topic=satellite-aws) and [on-premises](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-satellite-on-prem).



 An Overview 

Zoom

!
See [Creating a collection using the tooling](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcessource_tooling) and [Available data sources](https://cloud.ibm.com/docs/discovery?topic=discovery-sourcesavailable) for details.
* To learn how to extract custom fields from your documents, see [Smart Document Understanding](https://cloud.ibm.com/docs/discovery?topic=discovery-sdu).
* You can automatically filter stopwords (words that add little value) out of queries. For details, see [Defining stopwords](https://cloud.ibm.com/docs/discovery?topic=discovery-query-conceptsstopwords).
* With query expansion, you can expand the scope of queries beyond exact matches. See [Query expansion](https://cloud.ibm.com/docs/discovery?topic=discovery-query-conceptsquery-expansion) for details.
* By default, Discovery adds cognitive metadata to your documents with these four enrichments - Entity Extraction, Sentiment Analysis, Category Classification, and Concept Tagging. To learn more about these and other enrichments, see [Adding enrichments](https://cloud.ibm.com/docs/services/discovery?topic=discovery-configserviceadding-enrichments).



See [IBM's response to Covid-19](https://www.ibm.com/impact/covid-19/?cm_sp=ThinkDigitalResources-_-CHQ-_-IBMCOVIDResponse) for additional resources from IBM.