vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* [Deleting a context variable in JSON](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-runtime-contextdialog-runtime-context-delete-json)
* [Updating a context variable value in JSON](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-runtime-contextdialog-runtime-context-update-json)
* [Setting one context variable equal to another](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-runtime-contextdialog-runtime-context-var-equals-var)





 Deleting a context variable in JSON 

To delete a context variable, set the variable to null.

{
"context": {
"order_form": null
}
}

If you want to remove all trace of the context variable, you can use the JSONObject.remove(string) method to delete it from the context object. However, you must use a variable to perform the removal. Define the new variable in the message output so it will not be saved beyond the current call.

{
"output": {
"text" : {},
"deleted_variable" : "<? context.remove('order_form') ?>"
}
}

Alternatively you can delete the context variable in your application logic.





 Updating a context variable value in JSON 

In general, if a node sets the value of a context variable that is already set, then the previous value is overwritten by the new value.



 Updating a complex JSON object 

Previous values are overwritten for all JSON types except a JSON object. If the context variable is a complex type such as JSON object, a JSON merge procedure is used to update the variable. The merge operation adds any newly defined properties and overwrites any existing properties of the object.

In this example, a name context variable is defined as a complex object.

{
"context": {
"complex_object": {
When you provision a virtual server instance, you can specify customer-managed encryption for your boot volume and any data volumes that you want to add at that time. If you want, you can use a combination of provider-managed encryption and customer-managed encryption for the volumes that are associated with your instance.

Follow these steps to create an instance with a new block storage volume.



1. In [IBM Cloud console](https://console.cloud.ibm.com/vpc), navigate to menu icon ![menu icon](https://cloud.ibm.com/docs-content/v1/content/f02b9c9adcbb7328d95a45e105cec371d50f15eb/icons/icon_hamburger.svg) > VPC Infrastructure > Compute > Virtual server instances.
2. Click New instance and complete the required fields. (For information about these required fields, see Table 1 - Instance provisioning selections in [Creating virtual server instances](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-virtual-servers).)
3. In the Boot volume section, the default mode of encryption is Provider managed encryption. To specify customer-managed encryption, click the pencil icon in the boot volume row. On the Edit boot volume page, update the fields in the Encryption section. See Table 1 for more information. When your changes are complete, click Apply.
4. In the Attached block storage volume section, you can click New block storage volume to add a data volume and specify customer-managed encryption. On the New block storage volume page, update the fields in the Encryption section. See Table 1 for more information. When your changes are complete, click Attach.





Table 1. Values for specifying customer-managed encryption of volumes

 Field Value 

 Encryption Provider managed is the default encryption mode. To use customer-managed encryption, select a key management service from the drop-down list. The key management service instance must include the customer root key that you want to use for customer-managed encryption.
The services support Amazon Elastic Block Store, which you specify by setting the blockStorageClass property of the Speech services custom resource to gp2-csi or gp3-csi.

New storage classes are now supported
: Watson Speech services for IBM Cloud Pak for Data now support two additional storage classes:



* IBM Cloud Block Storage (ibmc-block-gold)
* NetApp Trident (ontap-nas)



You specify the storage class with the blockStorageClass property of the Speech services custom resource. For more information about all supported storage classes, see the following topics in [Watson Speech services on IBM Cloud Pak for Data](https://www.ibm.com/docs/en/cloud-paks/cp-data/4.6.x?topic=services-watson-speech):



* Before you begin in Installing Watson Speech services
* Specifying a storage class in Using the Watson Speech services custom resource



Known issue: Some Watson Speech services pods do not have annotations that are used for scheduling
: Known issue: Some Watson Speech services pods are missing the cloudpakInstanceId annotation. If you use the IBM Cloud Pak for Data scheduling service, any Watson Speech services pods without the cloudpakInstanceId annotation are



* Scheduled by the default Kubernetes scheduler rather than the scheduling service
* Not included in the quota enforcement



Monitoring of the PostgreSQL datastore is now available
: You can now enable monitoring of the PostgreSQL datastore to receive updates on its usage and status by the Watson Speech services. The events can be consumed by Prometheus monitoring software or whatever application you use for monitoring. By enabling monitoring for user-defined projects in addition to the default platform monitoring, you can monitor your own projects with the Red Hat® OpenShift® Container Platform monitoring stack. This capability includes an additional property, spec.global.datastores.postgressql.enablePodMonitor, in the Speech services custom resource.
Cloud Object Storage bucket 

[IBM Cloud Object Storage](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-about-cloud-object-storage) is used to store the evidence and artifacts that are generated by the DevSecOps pipelines. If you want to use this feature, you must have a Cloud Object Storage instance and a bucket that is created during your DevSecOps CI toolchain. [Read the recommendation](https://cloud.ibm.com/docs/devsecops?topic=devsecops-cd-devsecops-cos-config) for configuring a bucket that can act as a compliance evidence locker.

You can optionally set any kind of Cloud Object Storage bucket as a locker, even without a retention policy. The pipeline doesn't check or enforce settings at the moment. For help, see the [Cloud Object Storage documentation](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-getting-started-cloud-object-storage).

The following details of the bucket in your resource group and region are fetched and displayed. You must choose the one that is configured with the CI toolchain.



* Cloud Object Storage instance
* Bucket name
* Cloud Object Storage endpoint



You need to enter the Service API key to write to Cloud Object Storage instance.

The endpoint field is optional. It is recommended to select or provide the endpoint during the setup of the toolchain or during the pipeline run.

Zoom

![DevSecOps Cloud Object Storage configuration](https://cloud.ibm.com/docs-content/v1/content/5eb88a2575e30e316dbb83829c7f4c8b1ddbc67c/devsecops/images/devsecops-cc-toolchain-setup-COS.png)

Figure 10. DevSecOps Cloud Object Storage configuration





 DevOps Insights 

You can link to an existing IBM Cloud DevOps Insights toolchain instance from the CI toolchain.
Accessing the VMware Cloud Director Management console 

You can use the VMware Cloud Director Management console to manage the IBM Cloud® for VMware Solutions Shared virtual data centers. You can access the VMware Cloud Director console from the virtual data center details page.



 Before you begin 



* The first time that you access the VMware Cloud Director console for the virtual data center region, you must set the admin credentials to generate an initial, complex, and random password. After the first admin password is generated, the VMware Cloud Director console option is enabled on the virtual data center details page.
* You do not need to generate an admin password for each virtual data center. Reuse the same password for all virtual data centers in the same region.



IBM Cloud for VMware Solutions does not store the admin password. After a password is generated, you must capture it. If the password is lost or you get locked out of your account, you can generate a new password at any time.





 Procedure to access the VMware Cloud Director Management console 



1. From the Properties pane on the site details page, click Set Site Admin Password to generate a random password.
2. On the upper right of the site details page, click VMware Cloud Director console to access the console.
3. Use the admin username and password to log in to the VMware Cloud Director console.

After the admin is logged in to the VMware Cloud Director console, you can create extra users who have roles that allow them to access the VMware Cloud Director console.

All virtual data centers in the same region have the same admin password. Resetting the password on a virtual data center resets the admin password for accessing the VMware Cloud Director Management console for all virtual data centers in the same region.







 Related links 



* [Resizing virtual data centers](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-shared_resize)
You have the choice of either CouchDB or LevelDB when you configure your peer database. Because data is modeled differently in a Couch database than in a Level database, the peers in a channel must all use the same database type. See [LevelDB versus CouchDB](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-adv-deploymentibp-console-adv-deployment-level-couch) to decide what is best for your business needs.
* What types of off-chain databases are supported with the IBM Blockchain Platform?

As a best practice it is recommended that you do not query the entire blockchain ledger for the purpose of aggregation or reporting. If you want to build a dashboard or collect large amounts of data as part of your application, you can query an off chain database that replicates the data from your blockchain network. This allows you to understand the data on the blockchain without degrading the performance of your network or disrupting transactions.

You can use block or smart contract events from your application to write transaction data to an off-chain database or analytics engine. For each block received, the block listener application would iterate through the block transactions and build a data store by using the key/value writes from each valid transaction's read-write set. The [Peer channel-based event services](https://hyperledger-fabric.readthedocs.io/en/release-2.2/peer_event_services.htmlpeer-channel-based-event-services) provide replayable events to ensure the integrity of downstream data stores. For an example of how you can use an event listener to write data to an external database, see the [Off chain data sample](https://github.com/hyperledger/fabric-samples/tree/release-1.4/off_chain_data) in the Fabric samples.

Blockchain solutions can use any RDBMS or NoSQL DB such as IBM Cloudant for offchain data storage. Hyplerledger Fabric does not govern, interact with, or manage off-chain databases.
workers Number of worker threads. Increasing the number requires more job resources. 
 filter Provides control over which objects are imported from the Cloud Object Storage bucket 
 prefix Allows for objects from a specific date to be imported 
 documentTimestampWindow Allows for objects during a specific time range to be imported 
 cos Cloud Object Storage credentials 
 schema Transformation schema, which allows the importer to modify Watson Assistant fields to allow analytic data from pprd instance to be viewed in train. For more information, see [Generating a JSON ID mapping definition for the data governor importer service](https://cloud.ibm.com/docs/assistant-data?topic=assistant-data-cos-importercos-importer-json-mapping). 
 fromTenant Allows import on specific filter tenant data 



instance_id_consumer1, draft_or_live_environment_id_consumer1, and action_skill_id_consumer1 values can be found on the Assistant settings page in on the Cloud Object Storage consumer cluster.

![Assistant settings](https://cloud.ibm.com/docs-content/v1/content/a2b18c153837bce00a5b4de496e4b4ca64bbf56e/assistant-data/images/assistant-settings.png)

For example:



* instance_id_consumer1 can be extracted from the Service Instance URL. For example, 00000000-0000-0000-1679-443304652288.
* draft_or_live_environment_id_consumer1 values are the Draft environment or Live Environment IDs.
* action_skill_id_consumer1 value is the Action Skill ID.



Similarly, you can extract Cloud Object Storage importer instances, assistants, and workspaces target_id values from assistant settings for the Cloud Object Storage importer cluster.





 Generating a JSON ID mapping definition for the data governor importer service 

You can use the Analytics reports in your development environment to gain insight into your production chat logs.

You need to move your production chat logs data to your development environment, and then transform the production chat logs data to be used by the Analytics reports in your development assistants.
Then it is on a five-second loop where it makes an HTTP request to the backend (server.js), retrieves all the trucks and their positions and loops through them to plot their latest position on the map. Each truck has a map marker whose lat/long gets updated with every cycle.
3. producer.js: This takes in a location file and a truck_id from the script invocation and then loops through all the data points in the location file. Each data point is turned into a message with the truck ID and lat/long and posted once a second to Event Streams. For added fun, it picks a random starting point along the route and turns around and comes back when it reaches the end of the location file.
4. cloudantConsumer.js: This script creates a trucktracker database in Cloudant if it does not already exist and then polls Event Streams at regular intervals for more data. When data comes in, it gets written in bulk to Cloudant.
5. redisConsumer.js: This script polls Event Streams at regular intervals for data. When data comes in, it gets written into a Redis "hash" where the hash key is the truck ID and the value is the truck data (lat/long). This hash arrangement allows all the truck data to be pulled down by the server.js script in one go. The great thing is that if for some reason one of your consumers crashes, it will pick up where it left off without losing any data, because Event Streams knows the last data point it served to each consumer. Decoupling data-producing systems from data-consuming systems in this way is a good design practice that increases resilience and fault tolerance.







 Summary 

In this tutorial, we have combined a number of IBM services to model a realistic IoT pattern where Internet-enabled devices on trucks are producing data that gets consumed by various applications for different purposes.

Because of the managed nature of all these services, as an application developer, you can focus on solving your business problems and leave us to do the heavy lifting of managing the infrastructure for you, thereby saving potentially months of work on non-core activities like server provisioning, software deployment, patching, security and others.
For example, cos://mybucket/cislog?region=us-south&instance-id=c84e2a79-ce6d-3c79-a7e4-7e7ab3054cfe.
* --name specifies the Logpush job name.
* --enable is the flag to enable or disable the Logpush job. Valid values are true or false (default).
* --fields specifies the list of log fields to be included in log files. Use commas to separate multiple fields.

Use command ibmcloud cis logpull DNS_DOMAIN_ID --available-fields to get a comprehensive list of available log fields, or use all to include all available fields in the log files.
* --timestamps sets the format in which response timestamps are returned. Valid values are unix, unixnano, and rfc3339 (default).
* --dataset is the category of logs that you want to receive. Valid values are range_events and http_requests (default).

You cannot change this value after the job is created.
* -i or --instance is the instance name. If not set, the context instance specified by ibmcloud cis instance-set INSTANCE is used.



A domain can only have one Logpush job. Use the command line to interactively address the Object Storage bucket ownership challenge. When a challenge token is written to a file in the given Object Storage bucket, you must:



* Download the file from your Object Storage bucket and open it.
* Copy and paste the challenge token in the command prompt to address the ownership challenge.



A Logpush job is created successfully after CIS validates the ownership challenge. The Logpush job pushes request logs to your Object Storage bucket every 30 seconds or every 100,000 records, whichever comes first. More than one file might be pushed per 30-second period or per 100,000 records.

Logpush jobs created prior to September 2020 might continue pushing every 5 minutes. Any modification to one of these older jobs triggers an update to the push frequency.
You have the choice of either CouchDB or LevelDB when you configure your peer database. Because data is modeled differently in a Couch database than in a Level database, the peers in a channel must all use the same database type. See [LevelDB versus CouchDB](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-adv-deploymentibp-console-adv-deployment-level-couch) to decide what is best for your business needs.
* What types of off-chain databases are supported with the IBM Blockchain Platform?

As a best practice it is recommended that you do not query the entire blockchain ledger for the purpose of aggregation or reporting. If you want to build a dashboard or collect large amounts of data as part of your application, you can query an off chain database that replicates the data from your blockchain network. This allows you to understand the data on the blockchain without degrading the performance of your network or disrupting transactions.

You can use block or smart contract events from your application to write transaction data to an off-chain database or analytics engine. For each block received, the block listener application would iterate through the block transactions and build a data store by using the key/value writes from each valid transaction's read-write set. The [Peer channel-based event services](https://hyperledger-fabric.readthedocs.io/en/release-2.2/peer_event_services.htmlpeer-channel-based-event-services) provide replayable events to ensure the integrity of downstream data stores. For an example of how you can use an event listener to write data to an external database, see the [Off chain data sample](https://github.com/hyperledger/fabric-samples/tree/release-1.4/off_chain_data) in the Fabric samples.

Blockchain solutions can use any RDBMS or NoSQL DB such as IBM Cloudant for offchain data storage. Hyperledger Fabric does not govern, interact with, or manage off-chain databases. In most cases, the off-chain database is used for reference data and non-transactional data.
The arrival of 250 more docs makes the index incomplete again.](https://cloud.ibm.com/docs-content/v1/content/522c6f62358b063f921283400a601c3f9bc66f08/Cloudant/images/DesDocMan01.svg)

Figure 3. Index scheduled for update

When querying the view, you have the following choices.



* The default behavior is to ensure that the index is up to date, with the latest documents in the database, before it returns the answer. When you query the view, IBM Cloudant first indexes the 250 new documents, and then returns the answer.
* An alternative is adding the stale=ok parameter to the API call. This parameter means, return me the data that is already indexed. I don't care about the latest updates. In other words, when you query the view with stale=ok, IBM Cloudant returns the answer immediately, without any additional reindexing.
* A second alternative is to add the stale=update_after parameter to the API call. This parameter means, return me the data that is already indexed, and then reindex any new documents. In other words, when you query the view with stale=update_after, IBM Cloudant returns the answer immediately, and then schedules a background task to index the new data.



Adding stale=ok or stale=update_after can be a good way of getting answers more quickly from a view, but at the expense of freshness.

The default behavior distributes load evenly across nodes in the IBM Cloudant cluster. If you use the alternative stale=ok or stale=update_after options, these options might favor a subset of cluster nodes in order to return consistent results from across the eventually consistent set. The stale parameter isn't a perfect solution for all use cases. However, it can provide timely responses on fast-changing data sets if your application is happy to accept stale results. If your data's change rate is small, adding stale=ok or stale=update_after doesn't bring a performance benefit, and might unevenly distribute the load on larger clusters.
ibm_vpc_flowlogs_v1/account={account}/region={region}/vpc-id={vpc-id}/subnet-id={subnet-id}/endpoint-type=vnics/instance-id={vsi-id}/vnic-id={vnic-id}/record-type={all|ingress|egress}/year={xxxx}/month={yy}/day={zz}/hour={hh}/stream-id={stream-id}/{sequence-number}.gz

Where:



* {stream-id} is a date and time string in ISO 8601 format yyyymmddThhmmssZ (Coordinated Universal Time), defined as the time the first object in the directory was created.
* {sequence-number} is a running counter of objects within the stream, represented as an eight-character, zero-filled field (%08d).
* {all|ingress|egress} shows the type of traffic the flow includes.
* {xxxx} is the year in four-digit format.
* {yy} is the month in two-digit, zero-leading format.
* {zz} is the day of the month in two-digit, zero-leading format.
* {hh} is the hour in zero-leading format (for example, 00-24).



Any instances of URL-unsafe characters in the path are URL-encoded.

A flow log's Object Storage object contains a single valid JSON object. The Object Storage object is .gz compressed.

The object header fields that are specified in the following table are written to the Object Storage object metadata.



 Flow log fields 



Table 2. Flow log fields

 Field Type Description 

 start_time string When the first byte in a flow log was captured and seen in the data path (RFC 3339 Date and Time - Coordinated Universal Time). 
 end_time string When the last byte in a flow log was captured and seen in the data path (RFC 3339 Date and Time - Coordinated Universal Time).