vania-janet/multiturn-rag-retrieval-data huggingface dataset:
See the following example of a design document that includes a search index for the geographic data:

function(doc) {
if (doc.type && doc.type == 'city') {
index('city', doc.name, {'store': true});
index('lat', doc.lat, {'store': true});
index('lon', doc.lon, {'store': true});
}
}

See the following example that uses HTTP for a query that sorts cities in the northern hemisphere by their distance to New York:

GET /examples/_design/cities-designdoc/_search/cities?q=lat:[0+TO+90]&sort="<distance,lon,lat,-74.0059,40.7127,km>" HTTP/1.1
Host: $ACCOUNT.cloudant.com

See the following example that uses the command line for a query that sorts cities in the northern hemisphere by their distance to New York:



* Curl
* Java
* Node
* Python
* Go



curl "https://$ACCOUNT.cloudant.com/examples/_design/cities-designdoc/_search/cities?q=lat:[0+TO+90]&sort="<distance,lon,lat,-74.0059,40.7127,km>""

import com.ibm.cloud.cloudant.v1.Cloudant;
import com.ibm.cloud.cloudant.v1.model.PostSearchOptions;
import com.ibm.cloud.cloudant.v1.model.SearchResult;

import java.util.Arrays;

Cloudant service = Cloudant.newInstance();

PostSearchOptions searchOptions = new PostSearchOptions.Builder()
.db("examples")
.ddoc("cities-designdoc")
.index("cities")
.query("lat:\[0+TO+90\]")
Setup and analysis 

[Analyzing Data with IBM Cloud Data Engine](https://www.ibm.com/cloud/blog/analyzing-data-with-ibm-cloud-sql-query) shows you how to set up and start to use IBM Cloud® Data Engine from the UI. It also shows as an example how to use ibmcloudsql to query traffic accident data stored in IBM Cloud® Object Storage, and then visualize accidents in a Jupyter Notebook with PixieDust.
See also a brief overview of the underlying querying mechanism that you use to select the query mechanism that is best for each query your application needs to make.



 Global querying 

You can make global queries to the following index types:



* IBM Cloudant Query
* Views
* Search



When you make a global query, the database must perform a scatter-gather operation across all data in the database. This action means making requests of many individual database servers. The API coordination node receives the responses from all these servers and combines them to form a single response to the client. This response might involve buffering data and delaying the response to the client if, for example, data requires sorting.





 Partition querying 

You can make partition queries to the following index types:



* IBM Cloudant Query
* Views
* Search



When you make a partition query, the database can query just the data within a single partition. A partition's data resides in just one shard (with three replicas). The API coordination node can make a request directly to servers that host that data rather than needing to combine responses from many servers. The API coordination node is also free from buffering the response since it has no combination step to carry out. As a result, the data arrives at the client more quickly.

As the size of a database increases, the number of shards must also increase. The increase in shards directly increases the number of queries that the API coordination node needs to make to servers that host data when you use global queries. However, when you use partition queries, the number of shards has no effect on the number of servers the API coordination node needs to contact. As this number stays small, increasing data size has no effect on query latency, unlike global queries.







 Partitioned databases tutorials 

You can see two examples of using partitioned databases:



1. Read about [partitioned databases and Node.js](https://blog.cloudant.com/2019/05/24/Partitioned-Databases-with-Cloudant-Libraries.html) in this blog article that includes how to create a partitioned database, search, views, and a global index.
2.
(parameters: dbname, filter, iamApiKey, iamUrl, query_params)

This output shows that the IBM Cloudant package includes several actions and a feed.

The IBM Cloudant package also defines the parameters username, password, host, and port. These parameters must be specified for the actions and feeds to be meaningful.
3. Get a description of an action or feed to see the parameters that are required.

Example

ibmcloud fn action get --summary /whisk.system/cloudant/read

Example output

action /whisk.system/cloudant/read: Read document from database
(parameters: apihost, bluemixServiceName, dbname, id, params)

This output shows that the IBM Cloudant read action requires three parameters, including the database and document ID to retrieve.







 Binding parameters to pre-installed packages 

Although you can use the entities in a package directly, you might find yourself passing the same parameters to the action every time. You can simplify the process by binding to a package and specifying default parameters, which are inherited by the actions in the package.

For example, in the /whisk.system/cloudant package, you can set default username, password, and dbname values in a package binding and these values are automatically passed to any actions in the package.

In the following example, you bind to the /whisk.system/samples package.



1. Bind to the /whisk.system/samples package and set a default place parameter value.

ibmcloud fn package bind /whisk.system/samples valhallaSamples --param place Valhalla

Example output

ok: created binding valhallaSamples
2. Get a description of the package binding.

ibmcloud fn package get --summary valhallaSamples

Example output

package /<namespace_ID>/valhallaSamples
action /<namespace_ID>/valhallaSamples/greeting: Returns a friendly greeting
action /<namespace_ID>/valhallaSamples/wordCount: Count words in a string
Click [Log in to IBM Cloud](https://cloud.ibm.com/login) to sign in to the IBM Cloud.

After you log in with your user ID and password, the IBM Cloud console opens.
2. Select your account.
3. Click the Menu icon ![Menu icon](https://cloud.ibm.com/docs-content/v1/content/icons/icon_hamburger.svg) > Observability.
4. Select Activity Tracker.
5. For your instance, click Open Dashboard. The UI will be displayed.
6. Click ![Settings](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker//images/config.png) > Settings > Organization > Groups.
7. Click Add Group.
8. Enter a Group Name for your group.

![Add Group](https://cloud.ibm.com/docs-content/v1/content/6f3c07c34058f637b1c86044ca4f9b24214330a7/activity-tracker//images/addgroup.png)

Consider a naming convention similar to your Cloud Identity and Access Management access groups for ease of management.
9. Specify the Access Scope.

The access scope is defined as a search query. The query uses the following format.





Table 2. Example access scope search queries

 Example Query Behavior Example Matches 

 level:error Case-insensitive prefix match Error, error, errors 
 level:=error Case-sensitive prefix match error, errors 
 level:==error Case-insensitive exact term match error, Error 
 level:===error Case-sensitive exact term match error 
 level:[warning,error] Case-insensitive list of prefixes warning, Warning, Warnings, error, ERROR, errors
A $regex operator can be tagged onto the end of an already performant query. For example, if we already have a type=json index on firstname and surname, we can use the following example:

{
"selector": {
"firstname": "Charles",
"surname": "Dickens",
"title": {
"$regex": "^Oliv"
}
},
"limit": 10
}

IBM Cloudant uses the index to find documents by firstname and surname and winnows the result set that uses the regular expression on the title field.
IBM Cloudant audit logs can be used to understand:



* What and when databases and documents were accessed within an account, and by whom.
* What and when queries were run, and by whom.
* What a specific principal or user that is accessed, updated, or deleted, and when.
* What and when replication documents were created or deleted.



To request access to the audit logs for your account, contact IBM Cloudant support. Support provides a copy of the audit logs that are of interest to you.

When you contact support, be sure to include the following information:



* The IBM Cloudant account that the request relates to.
* The time frame for audit logs (must not be more than one month per support request).
* Any specific databases, documents, or principals of interest.
Or you can determine what the CRN is by using the ibmcloud resource service-instance SERVICE_INSTANCE_NAME command. 
 legacyCredentials An optional parameter that defaults to true and dictates whether the instance uses both legacy and IAM credentials or IAM credentials only. See the [IAM guide](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-managing-access-for-cloudant) for more details on choosing an authentication method. 



The following example command includes the extra parameters.

ibmcloud resource service-instance-create cloudant_on_ded_hardware_cli cloudantnosqldb standard us-south -p '{"environment_crn":"crn:v1:bluemix:public:cloudantnosqldb:us-south:a/b43434444bb7e2abb0841ca25d28ee4c:301a3118-7678-4d99-b1b7-4d45cf5f7b29::","legacyCredentials":false}'





 Step 5: Creating the credentials for your IBM Cloudant service 

Applications that require access to your IBM Cloudant service must have the necessary credentials.

Service credentials are valuable. If anyone or any application gains access to the credentials, they can effectively do whatever they want with the service instance. For example, they might create spurious data, or delete valuable information. Protect these credentials carefully. For more information about the fields included in the service credentials, see the [IAM guide](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-managing-access-for-cloudant).

The fields for the basic command format that is used in this exercise are described in the following table.



Table 5. Basic command format

 Field Description 

 NAME Arbitrary name that you give the service credentials. 
 ROLE_NAME This field currently allows the Manager role only. 
 SERVICE_INSTANCE_NAME The name that you give to your IBM Cloudant instance.
Pagination and bookmarks 

Bookmarks help release the next page of results from a result set. While with pagination, you iterate through a range of documents efficiently.

You can use the skip/limit pattern to [iterate through a result set](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-using-views), but it gets progressively slower the larger the value of skip.

[IBM Cloudant Query](https://cloud.ibm.com/apidocs/cloudantpostfind) and [IBM Cloudant Search](https://cloud.ibm.com/apidocs/cloudantgetsearchinfo) both use bookmarks as the key to unlock the next page of results from a result set. This practice is described in full in a later section that is called [Bookmarks](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-pagination-and-bookmarksbookmarks). It's less complicated to manage since no key manipulation is required to formulate the request for the next result set. You pass the bookmark that was received in the first response to the second request.

Now, you can see a better way to page through a large document set.



 Paging with _all_docs and views 

If you use the GET or POST$SERVICE_URL/$DATABASE/_all_docs endpoint to fetch documents in bulk, then you might see the limit and skip parameters. By using these parameters, you can define how many documents you would like, and the offset into the range you want to start from. Using the skip/limit pattern to iterate through results works, but it gets progressively slower the larger the value of skip.





 What is the _all_docs endpoint? 

The GET and POST$SERVICE_URL/$DATABASE/_all_docs are used to fetch data from an IBM Cloudant database's primary index, that is, the index that keeps each document's _id in order. The _all_docs endpoint takes a number of optional parameters that configure the range of data that is requested and whether to return each document's body or not.
"_id": "23966717-5A6F-E581-AF79-BB55D6BBB613",
"_rev": "1-96daf2e7c7c0c277d0a63c49b57919bc",
"doc_name": "Markdown Reference",
"body": "Lorem Ipsum",
"ts": 1422358827
}

Each data document includes a name, a body, and a timestamp. You create a [MapReduce view](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-creating-views-mapreduce) to sort your documents by timestamp.

You can sort your documents by timestamp by creating a Map function.

See an example map function that returns a document's timestamp field, if present:

function(doc) {
if (doc.ts) {
emit( doc.ts, null);
}
}

The function emits the document's timestamp so that you can use it as the key to the index. Since we're not interested in the value in the index, null is emitted. The effect is to provide a time-ordered index into the document set.

We're going to call this view by_ts and put it into a design document that is called fetch.

See an example design document that defines a view by using a map function:

{
"_id": "_design/fetch",
"views": {
"by_ts": {
"map": "function(doc) {
if (doc.ts) {
emit( doc.ts, null);
}
}"
}
},
"language": "javascript"
}

The result is that the map code is turned into a JSON-compatible string, and included in a design document.

Once the design document is saved, IBM Cloudant triggers server-side processes to build the fetch/by_ts view.
Curl comes preinstalled on most Macs and Unix-like operating systems. If it's not present on your computer, Google curl and follow the installation instructions.

Let's first use curl to fetch a web page - Google's home page.



1. In a command-line terminal, type curl https://www.google.com.

You get a pageful of HTML in reply.
If this method works, then curl is installed, and you can proceed with the next tasks. Now, we don't want to type the URL of our IBM Cloudant service every time, so let's save the IBM Cloudant URL in an environment variable-called URL.
2. Run the export URL command to create a variable that is called URL, which we can access later.

export URL=https://username:password@host
3. Create an alias.

alias acurl="curl -sgH 'Content-type: application/json'"

This alias is a shortcut that is called acurl that saves us further typing. This acurl command is an alias for curl but with the JSON content-type header and a couple of useful command-line switches.
4. Test the alias by fetching acurl $URL/. We get some JSON back from IBM Cloudant.

You completed your first IBM Cloudant API call. Now, our acurl alias is set up. We can start exploring the API. Let's start with the _all_dbs endpoint, which returns a list of databases.
5. Type acurl $URL/_all_dbs to see an array of databases.



A quick note here on formatting JSON on the command line. We can send the output of our acurl command to another tool, which formats the data nicely on the terminal. The following tools are available for your use:



* Jq available from the URL on the page, which is more than just a JSON formatter - it allows JSON to be parsed, queried, and manipulated too.
Data modeling 

The Data modeling document is the first best practice document in the series. It shows you the following best practices:



* What you need to know about your APIs.
* How to model your data.
* What size documents you must use.
* What to avoid.
* How to configure your databases.



For more information, see [Indexing and querying](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-indexing-and-querying) or [IBM Cloudant in practice](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-cloudant-in-practice).

The content in this document was originally written by Stefan Kruger as a [Best and worst practice](https://blog.cloudant.com/2019/11/21/Best-and-Worst-Practices.html) blog post on 21 November 2019.



 Understand the API that you are targeting 

You can use [Java™](https://github.com/IBM/cloudant-java-sdk), [Python](https://github.com/IBM/cloudant-python-sdk), [Go](https://github.com/IBM/cloudant-go-sdk), or [Node.js](https://github.com/IBM/cloudant-node-sdk) or some other use-case-specific language or platform. One of these languages most likely comes with convenient client-side libraries that integrate IBM Cloudant access nicely, following the conventions that you expect for your tools. These languages are great for programmer efficiency, but they also hide the API from view.

This abstraction is what you want, the whole reason for using a client library is to save yourself repeated, tedious boiler-plating. However, you must understand the underlying API is vital when you troubleshoot and report problems. When you report a suspected problem to IBM Cloudant, it helps us help you if you can provide a way for us to reproduce the problem.
[Conversation filters](https://cloud.ibm.com/docs-content/v1/content/ca9232b3702768f24bb897f48f10778fcda28718/watson-assistant/images/analytics-conversations-filters.png)

If you search for a specific session ID, enclose your search in quotation marks to ensure you receive an exact match on the full ID with its numbers and hyphen characters, for example: "9015ab9a-2e13-4627-ae33-4179b1125cb5".





 Exploring conversations in detail 

To explore individual conversations in detail, you can click on any of the utterances or conversation time stamps. A panel opens showing the full back and forth between your customer and the assistant, including step interactions. The panel also provides a summary of how many requests there were, how many were recognized, whether search was initiated, and the duration of the conversation.

![Conversation detail](https://cloud.ibm.com/docs-content/v1/content/ca9232b3702768f24bb897f48f10778fcda28718/watson-assistant/images/analytics-conversations-side.png)
Review customer conversations 

The Conversations page of Watson Assistant provides a history of conversations between users and a deployed assistant. You can use this history to improve how your assistants understand and respond to user requests.

Each timestamp represents a single conversation. The Actions column shows you how many actions, search queries, or unrecognized requests are included in that conversation. The Requests column includes the questions or requests the user entered that initiated an action, started a search query, or weren't recognized.

If you have activated dialog in your assistant, the Actions column is replaced by a Topics column.

![Conversations page](https://cloud.ibm.com/docs-content/v1/content/ca9232b3702768f24bb897f48f10778fcda28718/watson-assistant/images/analytics-conversations.png)



 Choosing the environment and time period 

To get started, choose the [environment](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-publish-overviewenvironments) and date range you want to analyze. All conversations reflect data based on the environment and the date range you select. When you change the environnment or the date range, the conversations on the page update to reflect the new date range. You can also use Refresh ensure the latest data is shown.

![Time period](https://cloud.ibm.com/docs-content/v1/content/ca9232b3702768f24bb897f48f10778fcda28718/watson-assistant/images/analytics-conversations-time-period.png)





 Filtering conversations 

You can locate specific conversations by filtering the list of conversations. This lets you explore specific areas where your assistant might need improvement or updates to properly handle what your customers are asking about.

You can filter by:



* Actions: Select specific actions. You can choose one or more actions to review.
* Keyword: Search by session ID or for specific key terms, phrases, or words in the conversations.
From the Conversations tab of the Analyze page, open the Actions filter and select Greet customer. For more information, see [Filtering conversations](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-analytics-conversationsanalytics-conversations-filtering).

Filter actions by name
: You can now find actions more easily. On the Actions page, you can filter actions by name. Click the search icon, then enter a search string. Your list of actions filters to match what you enter.





 12 August 2022 

Actions templates
: When creating actions, you can choose a template that relates to the problem you’re trying to solve. Templates help tailor your actions to include items specific to your business need. The examples in each template can also help you to learn how actions work. Actions templates include features such as intents, entities, condition-based responses, synonyms, response validations, and agent fallback. For more information, see [Building actions from a template](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-actions-templates).

Channel name variable
: The Channel name integration variable lets you add step conditions using these channels: web chat, phone, SMS, WhatsApp, Slack, or Facebook Messenger. For more information, see [Adding conditions to a step](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-step-conditions).





 11 August 2022 

Algorithm version options available in more languages
: Algorithm version options are now available in Arabic, Czech, and Dutch. This allows you to choose which Watson Assistant algorithm to apply to your future trainings. For more information, see [Algorithm version and training](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-algorithm-version).





 9 August 2022 

New API methods
: The v2 API now supports new Environments and Releases methods:



* Environments: Retrieve information about the environments associated with an assistant.
Improve your skill 

The Analytics page of Watson Assistant provides a history of conversations between users and a deployed assistant. You can use this history to improve how your assistants understand and respond to user requests.

The Analytics page was introduced with version 1.5.0.

To open a list of individual messages between customers and the assistant that uses this dialog skill, select User conversations in the navigation bar.

When you open the User conversations page, the default view lists inputs that were submitted to the assistant for the last day, with the newest results first. The top intent (#intent) and any recognized entity (@entity) values used in a message, and the message text are available. For intents that are not recognized, the value shown is Irrelevant. If an entity is not recognized, or has not been provided, the value shown is No entities found.

![Logs default page](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/logs_page1.png)

The User conversations page displays the total number of messages between customers and your assistant. A message is a single utterance that a user sends to the assistant. A conversation typically consists of multiple messages. Therefore, the number of results on the User conversations page is different from the number of conversations that are shown on the Overview page.



 Log limits 

Message logs are retained for 90 days.





 Filtering messages 

You can filter messages by Search user statements, Intents, Entities, and Last n days.

Search user statements - Type a word in the search bar. This searches the users' inputs, but not your assistant's replies.

Intents - Select the drop-down menu and type an intent in the input field, or choose from the populated list. You can select more than one intent, which filters the results using any of the selected intents, including Irrelevant.

!