vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Click the Open dashboard link, and expand IBM to view the IBM Schematics Summary Counts and IBM Schematics Summary Charts dashboard list.



* Use the IBM Schematics Summary Counts dashboard to monitor the counts of your workspace state, action, and its success and failure status.
* Use the IBM Schematics Summary Charts dashboard to monitor the charts of your workspace by state, by type and outcome, and the vulnerability count.









 Creating service instance 

You can create your services instance in the IBM Cloud console.

Complete these steps to create your services instance:



1. Log in to your [IBM Cloud](https://cloud.ibm.com) account by using your credentials.
2. Select Navigation menu > Observability > Monitoring.
3. Click Create by using your plan.
4. Select a region, for example, Dallas.
5. Create an IBM Cloud Monitoring instance by using the Lite plan.
6. Select the instance and accept the license.
7. Click create.
8. Select the created Sysdig instance from the Monitoring page.
9. Click Configure platform metrics, select the region and instance that you created to view the Platform metrics in the Region column.
10. Click View Sysdig icon to view your workspace and action that you created.

You can monitor the status of your workspaces state and action through the IBM Cloud dashboards. For more information about monitoring the status, see [Monitoring workspace](https://cloud.ibm.com/docs/schematics?topic=schematics-monitoring-instanceslaunch-dashboard). To create a custom dashboard, see [Creating custom dashboard](https://cloud.ibm.com/docs/schematics?topic=schematics-monitoring-instancescreate-dashboard).







 Creating custom dashboard 

You can create your custom dashboard in the IBM Cloud console.

Complete these steps to create your custom dashboard:



1. Log in to your [IBM Cloud](https://cloud.ibm.com) account by using your credentials.
2.
The first step when using Schematics is to create a workspace with the specific configuration parameters defined in the corresponding Terraform source code.

Use the following CLI command to create a workspace with your config.json file. Make sure that the config.json file exists in the directory where you run the command.

ibmcloud schematics workspace new -f hpc_workspace_config.json --github-token GITHUB_TOKEN

The --github-token parameter is optional and only needed if you are using a private GitHub repository. If you are using the [IBM Cloud public GitHub repository](https://github.com/IBM-Cloud/hpc-cluster-slurm), you do not need to provide it.



 Listing available workspaces 

You can list the workspaces in your account by using the following command:

ibmcloud schematics workspace list

Example response with workspace details:

Name         ID       Description           Status         Frozen
slurm-test   us-east.workspace.bcc-slurm-test.7cbc3f6b      Sample workspace      INACTIVE   False





 Retrieving workspace details 

You can retrieve the details of an existing workspace, including the values of all input variables, by running the following command:

ibmcloud schematics workspace get --id WORKSPACE_ID --output JSON





 Updating a workspace 

You can update the details for an existing workspace, such as the workspace name, variables, or source control URL by running the following command:

ibmcloud schematics workspace update --id WORKSPACE_ID --file FILE_NAME [--github-token GITHUB_TOKEN]

To provision or modify IBM Cloud resources, you can run the command ibmcloud schematics plan command. For more information, see the [IBM Cloud Schematics CLI](https://cloud.ibm.com/docs/hpc-slurm?topic=schematics-schematics-cli-referenceschematics-plan) reference.







 Next steps
* Read all folders stored in Box
* Write all folders stored in Box
* Manage Users



2. Enable the following advanced feature:



* Generate User Access Tokens



3. Share the target Box folders and files with the appropriate app users.

For more information, see [the Box documentation](https://developer.box.com/guides/getting-started/user-types/app-users/creation).







3. Create keys for authentication. Make sure that you complete the following tasks:



1. From the General settings page, click Add a Public Key.
2. Save the downloaded configuration file that is generated for the private key.
3. Click Save Changes.
4. From the Configuration page, copy the Client ID value.



4. Next, you must ask a Box administrator to authorize the app. In the Box admin console, complete the following tasks:



1. In the Apps > Custom Apps Manager page, click Add App.
2. Enter the client ID, and then click Next.

For more information, see [Custom App Approval](https://developer.box.com/guides/authorization/custom-app-approval/).



5. Box administrator step: From the Box admin console, check that the app information is accurate, and then click Authorize.







 Connecting to the Box data source 

From your Discovery project, complete the following steps:



1. From the navigation pane, choose Manage collections.
2. Click New collection.
3. Click Box, and then click Next.
4. Name the collection.
5. If the language of the documents in Box is not English, select the appropriate language.

For a list of supported languages, see [Language support](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-language-support).
6. Optional. Change the synchronization schedule.
[Add Documents button](https://cloud.ibm.com/docs-content/v1/content/148d3bd95f946aa1bb53ea1540475f522e6b61c9/watson-knowledge-studio-data/images/advanced-rules-add-documents.png)



1. Click the Add Documents button.
2. Select the file type and language of your file.
3. Click Choose File and select a file to upload.
4. Click Add to add your file to the project.



4. Create an extractor



1. Follow the tutorials from the Get Help menu for more information on how to create and run extractors. See [Managing projects and extractors > Creating an extractor](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-managing-projects-and-extractorscreating-an-extractor).



5. Run your extractors on documents



1. Right-click the extractor on the canvas and click Run Selected from the menu.
2. Alternatively, select the extractor and click Run Selected from the extractor toolbar.
3. To run multiple extractors, select them and click Run Selected in the toolbar. When the extractor has finished running, click the Results pane to view the output. The results are highlighted in the Documents pane as well.



For more information, see [Managing projects and extractors > Running an extractor](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-managing-projects-and-extractorsrunning-an-extractor).
6. Export your model



1. Click Save to save your model.
2. In the Extractors pane, right-click an extractor category, or right-click a single view under an extractor category. Click Export.
3.
Create the three directories required for the SAP installation, /sapmnt, /usr/sap, and /db2.



[root@e2e1270 ] mkdir /sapmnt
[root@e2e1270 ] mkdir /usr/sap
[root@e2e1270 ] mkdir /db2

Your IBM Cloud® Bare Metal Servers is now ready for external storage and the installation of your SAP applications and software.







 Step 2: Provisioning 192 GB and 32 GB servers for a three-tier environment 

A three-tier environment is a more complex scenario using a 192 GB server as the database server and a 32 GB server as the SAP NetWeaver application server.



 Ordering your SAP NetWeaver Application Server 

Follow the same steps in [Provisioning a 32 GB server for a single-host environment](https://cloud.ibm.com/docs/sap?topic=sap-quickstudy-bm-netweaver-rhelprovisioning_single_host) to order the SAP NetWeaver Application Server.





 Ordering your Database Server 

Use the following steps to order an SAP-certified server as your database server.



1. Log in to the [IBM Cloud® console)](https://cloud.ibm.com) with your unique credentials.
2. Click Create resource > Compute > Bare Metal Server.
3. Click Continue. If you can't click Continue, you don't have the correct permissions to create a server. Check with your system administrator about your permissions.
4. Leave 1 in the Quantity field.
5. Enter sdb192 in the Hostname field. Hostname is a permanent or temporary name for your servers. Click Information for formatting specifics.
6. Enter mycloud.com in the Domain field. Domain is the identification string that defines administrative control within the internet. Click Information for formatting specifics.
7. Billing defaults to Monthly. Currently, 1-year contract and 3-year contract are not available for SAP-certified servers.
8.
* UI
* CLI
* API







 Creating a workspace 

With IBM Cloud® Schematics workspaces, you can manage the Terraform-based templates to create and manage IBM Cloud resources. A workspace allows you to view and update configuration properties, view the list of resources that are provisioned through the workspace, and clean up resources if and when needed.



 Creating a workspace using the UI 



1. Log in to the [IBM Cloud catalog](https://cloud.ibm.com/catalog) by using your unique credentials.
2. In the Software section, select Compute and then select the IBM® Spectrum Scale tile.
3. In the Configure your workspace section:



* Specify the Name for your Schematics workspace
* Select a Resource group
* Select a Location
* Define any Tags that you want to associate with the resources provisioned through the offering. The tags can later be used to query the resources in the IBM Cloud console.



4. In the Set the deployment values section, specify the values for the required properties.
5. Expand the Parameters with default values section, and review it to determine whether you need to override any of the default values provided for the configuration properties.
6. Review and accept the IBM Spectrum Scale license terms and conditions in the order summary.
7. Click Install. The Schematics workspace is created with the name you specified. You can see the list of workspaces in View the existing installations. If the workspace creation is successful, the Apply Plan action is started to trigger the deployment of the respective IBM Cloud VPC resources in your IBM Cloud account.
8. You can also review the status of your deployment process by identifying the workspace name in the View the existing installations section. When you click a record in View the existing installations section, you are taken to the Schematics workspace view.







 Next steps
2. Click Overview to view the blueprint summary, that includes Modules status, Variables summary, Blueprint Details and Recent Job runs of your blueprint.



* Optional: From the Modules status card, Click View details to navigate to the module details page.
* Optional: From Variables summary card, Click View details to navigate to the variable summary page.



3. Click Modules tab to see the list of blueprint modules and their current status.



* Optional: Click Show details to view more details about the blueprint module definition.
* Optional: Click Name that will take you to the modules related Schematics Workspace page.



4. Click Resources tab to view the list of resources provisioned by the blueprint.
5. Click Variables tab to view the blueprint Inputs and Outputs variables and values. Optional: on this page you can edit the input variable values and save.
6. Click Jobs history tab view the job logs of the blueprint and module jobs.
7. Click Settings tab to view and edit the configuration settings of the blueprint.









 Applying a blueprint using the API 

Follow the [steps](https://cloud.ibm.com/docs/schematics?topic=schematics-setup-apics_api) to retrieve your IAM access token and authenticate with IBM Cloud Schematics by using the API. For more information, about Blueprint update, refer to, [Installing Blueprint](https://cloud.ibm.com/apidocs/schematics/schematicscreate-blueprint) by using API.

Blueprint installation API runs blueprint_create_init, and blueprint_installAPIs, jobs to perform the create and install the modules using Blueprint operations.

Create Blueprint Job Example:

POST /v2/jobs HTTP/1.1
Host: schematics.cloud.ibm.com
Content-Type: application/json
Authorization: Bearer <auth_token>
refresh_token: <refresh_token>

{
"command_name": "blueprint_create_init",
"command_object": "blueprint",
This practice saves IBM Cloudant from having to choose which index to use from the available ones, and it makes it easy for you to remember which index is which.

Let's create an index on our books database from the dashboard. Select the database, then choose the Design Documents tab and Query Indexes from the menu.

Any existing indexes are listed on the side: A special index must exist that represents the primary index, based on the document's _id.

Complete the index definition with the JSON:

Click Create Index when you're done.

Clicking the button sends a POST request to the _index endpoint (other API calls are available to update and delete existing indexes).

Indexes are built asynchronously by IBM Cloudant in the background. For large databases, it can take IBM Cloudant some time to construct the index for the first time. The index cannot use the database until that initial build is ready.

We can repeat our query for books in the 20th century. This time we specify the index name with the use_index field. The answer returns - this time powered by our index. You might not notice a speed improvement for a small database, but the benefit is definitely felt as your data size and query volume grows. Indexing helps your queries remain performant as your application scales.

When you tell IBM Cloudant to create a secondary index, it starts a background task that looks at all the documents in turn and creates a new data structure on disk: the index. The index is a balanced tree which pairs the keys (the attribute or attributes that you need indexed) with the document _id they came from.

The index can be used to efficiently lookup known keys and ranges of keys without having to rescan the entire database.

Another trick that you can employ at index time is the partial filter. You can optionally supply a partial filter in your index definition. This IBM Cloudant Query selector is executed at index time to decide which documents' data makes it to the index and which are ignored.

In this example, a selector is employed that allows only dates that fall on a weekend to make it to the index. Smaller indexes are faster and more efficient.
* IBM Cloudant guide to using [views](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-using-viewsusing-views)
* Performance implications of using [include_docs](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-using-viewsmulti-document-fetching)







 Never rely on the default behavior of IBM Cloudant Query’s no-indexing 

It’s tempting to rely on IBM Cloudant Query's ability to query without creating explicit indexes. This practice is costly in terms of performance, as every lookup is a full scan of the database rather than an indexed lookup. If your data is small, this full-scan lookup doesn’t matter, but as the data set grows, performance becomes a problem for you, and for the cluster as a whole. It is likely that we will limit this facility soon. The IBM Cloudant Dashboard provides a method for creating indexes in an easy way.

Creating indexes and crafting IBM Cloudant Queries that take advantage of them requires some flair. To identify which index is being used by a particular query, send a POST to the _explain endpoint for the database, with the query as data.

For more information, see [IBM Cloudant Query docs](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-query).





 In IBM Cloudant Search (or IBM Cloudant Query indexes of type text), limit the number of fields 

IBM Cloudant Search and IBM Cloudant Query indexes of type text (both of which are Apache Lucene under the hood) provide you with a way to index any number of fields into the index. Some examples exist where this type of indexing is abused either deliberately, or mostly by mistake. Plan your indexing to comprise only the fields required by your actual queries. Indexes take up space and can be costly to rebuild if the number of indexed fields are large.

We also have the issue of which fields that you store in an IBM Cloudant Search.
When you create a secondary index, you choose whether its purpose is for per-partition or global scope.

That's the end of this part. The next part is called IBM Cloudant Search.







 IBM Cloudant Search video 

Learn how to use IBM Cloudant Search, as well as Lucene query language and faceting.



* IBM Cloudant Search video script

Welcome to the Introduction to IBM Cloudant course, an 17-part video series that gives you an overview of the IBM Cloudant database-as-a-service.

This video is part 16 - IBM Cloudant Search.

We have another method of querying and indexing in IBM Cloudant called IBM Cloudant Search that we briefly explore in this part.

IBM Cloudant Search is built on another open source project, Apache Lucene, which powers the search capabilities of many products that include Elasticsearch.

It is primarily designed for free text search, where blocks of text are pre-processed before they are indexed: removing case, punctuation, common noise words, and trimming common language-specfic word endings, for example, farmer becomes farm, and farms becomes farm.

This text-processing is performed by a choice of analyzers at query time, searching. Before this time, it also allows some aggregation functionality that uses a technique that is called faceting.

An IBM Cloudant Search index is created by supplying JavaScript function. It is not unlike MapReduce, except this time, the emit function is replaced by an index function, which expects the name of the field, the data itself, and some options.

In this example, the document's name and title are indexed with default options. The category is nominated for faceting (the aggregation functionality) and the ISBN is stored in the index but not indexed for search itself. Sometimes it is more efficient to store some items in the index rather than doing include_docs=true at query time.

Lucene has its own query language that creates queries that match combinations of clauses with logic, fuzzy matching, ranges, and term boosting.

See the following examples:



* Find documents whose title matches gastby and whose author starts with fitz. Notice the asterisk wildcard.
* Find documents whose author is in the range austen to dickens.
Working with indexes 

IBM® Cloudant® for IBM Cloud® endpoints can be used to create, list, update, and delete indexes in a database, and to query data by using these indexes.



Table 1. Available methods and endpoints

 Methods Path Description 

 DELETE /$DATABASE/_index Delete an index. 
 GET /$DATABASE/_index List all IBM Cloudant Query indexes. 
 POST /$DATABASE/_find Find documents by using a global index. 
 POST /$DATABASE/_partition/$PARTITION_KEY/_find Find documents by using a partitioned index. 
 POST /$DATABASE/_index Create an index. 





 Creating a partial index 

IBM Cloudant Query supports partial indexes by using the partial_filter_selector field. For more information, see the [CouchDB documentation](https://docs.couchdb.org/en/stable/api/database/find.html?highlight=partialpartial-indexes) and the original example.

The partial_filter_selector field replaces the selector field, previously only valid in text indexes. The selector field is still compatible with an earlier version for text indexes only.

See the following example query:

{
"selector": {
"status": {
"$ne": "archived"
},
"type": "user"
}
}

Without a partial index, this query requires a full index scan to find all the documents of type:user that don't have a status of archived. This situation occurs because a normal index can be used to match contiguous rows, and the $ne operator can't guarantee that.

To improve response time, you can create an index that excludes documents with status: { $ne: archived } at index time by using the partial_filter_selector field that is shown in the following example:

POST /db/_index HTTP/1.1
Content-Type: application/json
Content-Length: 144
Host: localhost:5984

{
"index": {
"partial_filter_selector": {
"status": {
"$ne": "archived"
2. Copy the following sample JSON and replace the existing text in the new query window:

{
"selector": {
"lastname" : "Greene",
"firstname" : "Anna"
}
}
3. Click Run Query.

The query displays the results. You can see them from the Table view in the following screen capture:

Zoom

![Query results](https://cloud.ibm.com/docs-content/v1/content/522c6f62358b063f921283400a601c3f9bc66f08/Cloudant/images/dashboard_query1_results.png)

Figure 2. Query results



For more information, see the [IBM Cloudant Query](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-creating-an-ibm-cloudant-query) tutorial or the API reference on [IBM Cloudant Query](https://cloud.ibm.com/apidocs/cloudantgetindexesinformation).







 Step 4: Replicating a database 

When you replicate a database, it synchronizes the state of two databases: source and target. A replication copies all the changes that happened in the source database to the target database. When a document is deleted from the source database, the document is also deleted from the target database.

For more information, see [Replication](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-replication-apireplication-operation).



1. Click Replication.
2. Click New Replication.

The Job configuration page opens.

Additionally, you can create a replication from the databases page by clicking Replicate in the Actions column.
3. Enter the following information for your replication job. Use the following information in the Source section:



* Type - Select Remote database.
* Name - Enter the database URL: $SERVICE_URL/query-movies.
* Authentication - Leave as None.



Use the following information in the Target section:
If you use the sort by distance feature as described in [Geographical searches](https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-cloudant-searchgeographical-searches), then the first element is the distance from a point. The distance is measured by using either kilometers or miles.

The second element in the order array can be ignored. It is used for troubleshooting purposes only.





 Faceting 

IBM Cloudant Search also supports faceted searching, which enables the discovery of aggregate information about matches quickly and easily. You can match all documents by using the special ?q=: query syntax, and use the returned facets to refine your query. To indicate that a field must be indexed for faceted queries, set {"facet": true} in its options.

See the following example search query, specifying that faceted search is enabled:

function(doc) {
index("type", doc.type, {"facet": true});
index("price", doc.price, {"facet": true});
}

To use facets, all the documents in the index must include all the fields that have faceting enabled. If your documents don't include all the fields, you receive a bad_request error with the following reason, "The field_name does not exist." If each document does not contain all the fields for facets, create separate indexes for each field. If you don't create separate indexes for each field, you must include only documents that contain all the fields. Verify that the fields exist in each document by using a single if statement.

See the following example if statement to verify that the required fields exist in each document:

if (typeof doc.town == "string" && typeof doc.name == "string") {
index("town", doc.town, {facet: true});
index("name", doc.name, {facet: true});
}



 Counts 

The counts option is only available when you make global queries.