vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* region - The [region alias](https://cloud.ibm.com/docs/sql-queryendpoints) of the bucket that holds your flow logs.



To optimize flow logs layout with IBM Cloud® Data Engine, follow these steps:



1. Convert all flow logs for a specific date to parquet, writing one parquet object per hour:

SELECT * FROM FLOW_FLAT
WHERE
day = day(DATE("2020-07-24")) AND
month = month(DATE("2020-07-24")) AND
year = year(DATE("2020-07-24"))
INTO cos://<region>/<bucket>/ibm_vpc_flowlogs_v1_parquet/ JOBPREFIX NONE STORED AS PARQUET PARTITIONED BY (year,month,day,hour)
2. Create a table for conveniently working with the optimized data:

CREATE TABLE FLOW_PARQUET
USING PARQUET
LOCATION cos://<region>/<bucket>/ibm_vpc_flowlogs_v1_parquet/

Replace the following place holders in the following steps:



* bucket - The bucket where you stored the optimized flow logs.
* region - The [region alias](https://cloud.ibm.com/docs/sql-queryendpoints) of the bucket that holds your flow logs.



3. Use the table FLOW_PARQUET instead of FLOW_FLAT.

For more information about how data layout influences query execution times, see [How to Layout Big Data in IBM Cloud Object Storage for Spark SQL](https://www.ibm.com/cloud/blog/big-data-layout).







 Example queries for flow logs with IBM Cloud® Data Engine 

The following example queries a maximum of 100 individual flows for the date specified:

SELECT * FROM FLOW_FLAT WHERE CAST(capture_start_time AS date) = DATE("2020-07-24") LIMIT 100

The following query lists all accepted TCP connection for the specified date, extracting the hour that the connection was made:

SELECT hour(capture_start_time) as hour,
Open your custom dashboard that was customized with Scenario 1. The custom dashboard is available in the dashboard navigation menu, Dashboards->My Dashboards.
2. Modify the Total number of Applications number panel and change the query to sum the number of application instances and the number of job runs.



1. Click the pencil icon to edit the panel.
2. From the Query tab, click PromQL to use PromQL queries.
3. From the Query tab, replace the contents of the query with sum(max(ibm_codeengine_application_actual_instances) + max(ibm_codeengine_jobruns)). Update Options to use number as the unit. Click Run query.
4. From the Panel tab, rename this panel to Total number of Application instances and Job Runs.
5. Click Save to save the changes for this panel.



3. Use the actions menu to delete the Total number of Job Runs number panel because we aggregated both the total number of applications and job runs into a single number panel in the previous step.
4. You can adjust the size and layout of the individual panels to meet your needs.

Whenever you change the layout of your dashboard, be sure to click Save Layout to save your changes. You must save the layout changes or cancel before you continue to customize to your dashboard.



After you complete scenario 2, your custom dashboard includes two panels: Total number of Application instances and Job Runs and Application and Job Run Instances.
--query-file or -f Optional The path to the JSON file containing queries. 
 --file or -f Optional Path to the JSON file containing the definition of an inventory. 
 --output or -o Optional Returns the command-line output in JSON format. Currently only JSON file format is supported. 
 --no-prompt Optional Set this flag to create the resource query without an interactive command-line session. 



Example

ibmcloud schematics resource-query update --id us-east.INVENTORY.inventory12312 --name inventoryname600 --description "Short description" --location us-east --resource-group Default --resource-query default.RESOURCEQUERY.string.12121







 Stop commands 

After invoking a Workspaces job, like a plan, an apply, or a destroy, you may want to stop the running job, or to stop the provisioning of resources. When stopping, or canceling a long running job, it is advisable to first check the job logs to determine whether the job is actually stuck and needs stopping, or if it is performing long running operations that are taking time to complete.

Schematics provides a number of options to allows users to (gracefully) stop, force-stop, or terminate the running job in order of immediacy and impact of the stop operation.

Review the commands to (gracefully) stop, force-stop or terminate jobs.



 ibmcloud schematics workspace job stop 

Stops a running action or a job in IBM Cloud Schematics Workspaces by sending and interrupt signal to Terraform to

Syntax

ibmcloud schematics workspace job stop --id WORKSPACE_ID --job-id JOB_ID [--stop] [--force-stop] [--terminate]

Command options



Schematics job stop flags

 Flag Required / Optional Description 

 --id or -i Required The workspace ID to update. 
 --job-id or --jid Required The job ID of the job. 
 --stop, Optional Removes the job from the pending queue. 
 --force-stop or --fs Optional Sends a kill signal to the Terraform execution in the engine, also attempts to immediately stop the execution.
As expected, the assistant now prompts you to select the account you want to withdraw money from.







 Duplicating a step 

You can duplicate a step so you don't have to re-create variable settings and customizations. Duplicating a step is helpful when you need to add a step similar to a previous step, but with minor modifications.

Complete the following steps to duplicate a step:



1. Click the Duplicate icon on the step that you want to duplicate.

Zoom

![Duplicate button on a step](https://cloud.ibm.com/docs-content/v1/content/ca9232b3702768f24bb897f48f10778fcda28718/watson-assistant/images/action-editor-duplicate-button.png)

Duplicate icon

A step appears immediately following the step that you duplicated. This step is identical to the duplicated step and displays a blue circle in the upper right to indicate that the step is a duplicate.

Zoom

![Duplicated step](https://cloud.ibm.com/docs-content/v1/content/ca9232b3702768f24bb897f48f10778fcda28718/watson-assistant/images/action-preview-duplicate-step.png)

Duplicated step
2. Edit the information in the new step as necessary.







 Adding conditional steps 

Suppose our bank charges a fee for withdrawals from checking accounts, and we need to confirm that the customer understands. Our action needs to have slightly different behavior depending on which kind of account the customer selects. We can handle this using step conditions.

When a step asks for information from the user, the user's response is stored as an action variable. By referring to the action variables stored by previous steps, you can construct step conditions based on your customer's previous responses.



1. Click New step.
2. In the Step 2 is taken field, select with conditions. The Conditions section expands.

Zoom

!
Step 5: Configure the default dashboard and alerts to analyze the Blackbox status of your server 

Complete the following steps:



1. [Launch the monitoring UI](https://cloud.ibm.com/docs/monitoring?topic=monitoring-launch).
2. Create a dashboard.
3. Add a panel for each of the following queries:





Table 1. Blackbox exporter sample queries

 Metric Query Options 

 Status code by Instance probe_http_status_code{instance=$instance} number (auto) 
 SSL Cert Expiry in Days probe_ssl_earliest_cert_expiry{instance=$instance}-time() time (auto) 
 DNS Lookup probe_dns_lookup_time_seconds{instance=$instance} time (auto) 
 Probe duration probe_duration_seconds{instance=$instance} time (auto) 



For example, you can create a dashboard that looks as follows:

Zoom

![Sample Blackbox dashboard](https://cloud.ibm.com/docs-content/v1/content/220cdcbf6f4421bd7fee83fd1028048dba3f16c8/monitoring/exporters/images/blackbox-ui-1.png)

Figure 1. Sample Blackbox dashboard
An action cannot be deleted while in use. To delete the action, remove it from the triggers first. The Uses column shows the number of triggers that are associated with this action. Delete cannot be undone.
* Associated triggers Add a trigger and associate it with an action.





 Working with triggers 

Triggers (routes) determine domain traffic routing to the actions. Triggers associate certain URL patterns, based on a domain on the account, with a predefined action. The URL must contain the domain, but it can contain wildcards either as a prefix to the domain, or at the end of the path. If no path is given on the pattern, a / is added implicitly. The URL pattern cannot contain infix wildcards or query parameters.

You must add a domain to add triggers. You can add triggers without having actions.



* Adding triggers Go to the Triggers tab and click Add trigger. Enter a URL pattern and select an action from the list of existing actions.



* For an action, you can also select Avoid Edge Functions. This allows the trigger's path to remain active, but avoid using any Edge function actions. For example, there is an action called my-function and a trigger with the path gamma.cistest-load.com/. If the path gamma.cistest-load.com/data should not use the action my-function, create another trigger with the path gamma.cistest-load.com/data and the option Avoid Edge Functions. This allows the path gamma.cistest-load.com/data to remain active without using the action my-function.



* Editing triggers Update a trigger using the menu option in the table row for a selected trigger. After updating select Save.
* Deleting triggers Delete a trigger using the menu option in the table row for a selected trigger. This action cannot be undone.
* To reuse existing public and private VLANs, if they are available, choose Select existing VLANs. Specify the VLANs and the subnets.
* If the consolidated or management cluster and the workload clusters are in the same location, you cannot use existing VLANs. Instead, choose Reuse VLANs from the consolidated cluster so that the workload cluster and the consolidated cluster use the same VLANs.





8. Choose the firewall appliance for your instance and follow the steps, depending on your selection:



* For Gateway cluster with Juniper® vSRX, Gateway cluster with FortiGate® Virtual Appliance, and Bring your own gateway appliance, specify the gateway [cluster name](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-cr-orderinginstance-edgecr-orderinginstance-edge-cluster-name), the CPU model, the RAM size, the uplink speed, and the networking type.
* For Gateway cluster with Juniper vSRX and Gateway cluster with FortiGate Virtual Appliance, you must also specify the corresponding service settings in a later step.
* For FortiGate Security Appliance, you must order the FortiGate Security Appliance 10 Gbps service from the [IBM Cloud catalog](https://cloud.ibm.com/catalog/infrastructure/fortigate-security-appliance-10gb). Confirm that you ordered the service and continue with the following steps.



9. Specify the Gateway cluster settings:



1. Accept or change the default name of the gateway cluster.
2. Select the CPU model and the RAM size.
3. Select the Uplink speed. The 25 Gb option is available only for specific pods and data center locations. For more information, see [Uplink speed](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-vc_orderinginstance-addl-clustersvc_orderinginstance-addl-clusters-gate-uplink).
4. Select the Networking type, either Public and private network or Private network only.



10.
In the Authentication step, specify the authentication and server information you want your assistant to use when it calls the service.



* In the Authentication type field, select the type of authentication to use (or No authentication if the API is not authenticated). The available authentication types are determined by the security schemes that are defined in the OpenAPI document.
* Specify the additional information required for the authentication type you selected (such as the username and password, API key, bearer token, or OAuth 2.0 details).

For more information about configuring OAuth 2.0 authentication, see [OAuth 2.0 authentication](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-add-custom-extensionadd-custom-extension-oauth).
* In the Servers field, select the server URL to use.

If the selected URL contains any variables, also specify the values to use. Depending on how each variable is defined in the OpenAPI document, you can either select from a list of valid values or type the value to use in the field.

The Generated URL message shows the full URL that the assistant uses, including the variable values.



Click Next.
5. In the Review extension step, review the operations that are supported by the extension.

The Review operations table shows the operations that the assistant is able to call from an action step. An operation is a request by using a particular HTTP method, such as GET or POST, on a particular resource.

![Review operations table](https://cloud.ibm.com/docs-content/v1/content/ca9232b3702768f24bb897f48f10778fcda28718/watson-assistant/images/extension-review-operations.png)

For each operation, a row in the table shows the following information:



* Operation: A description of the operation, which is derived from either the summary (if present) or description in the OpenAPI file.
* Method: The HTTP method used to send the API request for the operation.
* Resource: The path to the resource the operation acts upon.
[Checkmark icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/log-analysis/images/checkmark-icon.svg) 



Notice that users with the service role reader get an access error when they launch the SQL Query UI.

To manage access or assign new access for users by using IAM policies, you must be the account owner, administrator on all services in the account, or an administrator for the particular service or service instance.

Choose any of the following actions to manage IAM policies in the IBM Cloud:



* To grant permissions to a user, see [Assigning access](https://cloud.ibm.com/docs/account?topic=account-assign-access-resourcesassign-new-access).
* To revoke permissions, see [Removing access](https://cloud.ibm.com/docs/account?topic=account-assign-access-resourcesremoving-access-console)).
* To review a user's permissions, see [Reviewing your assigned access](https://cloud.ibm.com/docs/account?topic=account-assign-access-resourcesreview-your-access-console).







 Step 3. Running a query through the Data Engine UI 

In SQL, the term query is just another way of saying SELECT statement.

To run a query, complete the following steps:



 Step 3.1. Launch the Data Engine query UI 



1. [Log in to your IBM Cloud account](https://cloud.ibm.com/login).

After you log in with your user ID and password, the IBM Cloud dashboard opens.
2. Click the Menu icon ![Menu icon](https://cloud.ibm.com/docs-content/v1/content/f61dbc5d84bf0c89e477a05cab04c72f429e1b7a/icons/icon_hamburger.svg) > Resource list > Services.
"parameter": {
"kind": "workspace",
"tags": [
"env:dev",
"k8s"
],
"resource_groups": [
"test"
],
"locations": [
"us-south"
]
},
"target": "action"
}







 Next steps 

You can now use the agent to run Schematics Terraform or Ansible jobs. The agent will execute any jobs for workspaces or actions that match the defined selection policy parameters:



* resource group
* location
* tags



Note at this time, tags must be set at workspace or action create time. Any changes to tags performed via the Schematics UI will not be detected or considered during policy evaluation.

After execution the workspace or action job logs will contain a header indicating the agent the job was executed on.

2023/04/08 15:22:07 [ 1m----- New Workspace Action -----21m0m
2023/04/08 15:22:07 Request: activitId=e3fcfdfdb13b07a1c60176e4b95c41ba, account=, owner=steve_strutt@uk.ibm.com, requestID=0a8a3428-b461-4dd0-8104-e859d68d35f6, OrchestratorID=orchestrator-5c8585dc74-6z9s5, agentID=agent-test-da.deA.e055, agentName=agent-test-da, jobRunnerID=jobrunner-5d99b5cfb7-p5xcz
2023/04/08 15:22:07 Related Workspace: name=myworkspace, agentID=agent-test-da.deA.e055 sourcerelease=(not specified), sourceurl=https://github.com/stevestrutt/multitier-vpc-bastion-host, branch=(not specified), folder=.
Provisioning your resources in VPC 

After you edit your discovered resources, you can provision that workspace on IBM Cloud®. You can either provision your entire VPC or select the components that you want to provision, allowing you to migrate the rest at a later time.



1.  To select components for provisioning, enable the Select for Provisioning option and choose the components that you want to provision and click Continue.
2.  Click Provision to start the provisioning process.



Provisioning can take a significant amount of time depending on the size of your instances and images. There are several processes that run in the background during migration, for example, capturing the images, exporting them to IBM Cloud Object Storage, converting them to the required VPC format, and then importing them into the VPC infrastructure.

When the provisioning process is complete, you will see a success message.



  Next steps 

After your VPC environment is provisioned, see [Validating your resources in VPC](https://cloud.ibm.com/docs/wanclouds-vpc-plus?topic=wanclouds-vpc-plus-migrate-validate).
* [Getting logs for my function](https://cloud.ibm.com/docs/codeengine?topic=codeengine-troubleshoot-functionts-funcrtion-gettinglogs)
* [Keep your runtime and CLI versions up to date](https://cloud.ibm.com/docs/codeengine?topic=codeengine-troubleshoot-functionts-function-uptodate)
* [Additional topics](https://cloud.ibm.com/docs/codeengine?topic=codeengine-troubleshoot-functionts-function-topics)







 Troubleshooting images 

[Debugging images](https://cloud.ibm.com/docs/codeengine?topic=codeengine-troubleshoot-imagestroubleshoot-images)



* [Working with images](https://cloud.ibm.com/docs/codeengine?topic=codeengine-troubleshoot-imagests-image-workwith)
* [Getting logs for my images](https://cloud.ibm.com/docs/codeengine?topic=codeengine-troubleshoot-imagests-images-logs)
* [Getting system event information for my images](https://cloud.ibm.com/docs/codeengine?topic=codeengine-troubleshoot-imagests-images-systemevents)



[Why can't Code Engine pull an image?](https://cloud.ibm.com/docs/codeengine?topic=codeengine-image-cannot-pullimage-cannot-pull)



* [Next steps](https://cloud.ibm.com/docs/codeengine?topic=codeengine-image-cannot-pullimage-cannot-pull-next)







 Troubleshooting jobs 

[Debugging jobs](https://cloud.ibm.com/docs/codeengine?topic=codeengine-troubleshoot-jobtroubleshoot-job)
If the issue persists for a while, you can [open a support case](https://cloud.ibm.com/docs/satellite?topic=satellite-get-help).

To find more details about your issue, set up Log Analysis for Satellite location.



1. [Set up Log Analysis for Satellite location platform logs](https://cloud.ibm.com/docs/satellite?topic=satellite-healthsetup-la).
2. Search the platform logs for the error code for more details, such as failed API method due to a permissions error.
3. If the details indicate a permission error:



1. As the account administrator, log in to the IBM Cloud CLI and target the resource group and region that the location is in.

ibmcloud login -g <resource_group> -r <region>
2. Reset the API key that is used for permissions.

ibmcloud ks api-key reset









 R0009: Unable to recover 

Location message
: R0009: Satellite is unable to recover from issues.

Steps to resolve
: Satellite tried unsuccessfully to automatically resolve the issue. Review any further messages to troubleshoot further, such as adding more hosts to the location. If the issue persists for a while, you can [open a support case](https://cloud.ibm.com/docs/satellite?topic=satellite-get-help).





 R0010, R0030, R0031, R0032: Control plane needs hosts 

Location message
: R0010: Assign more hosts to the location control plane, or replace unhealthy hosts.
: R0030: A zone for the Satellite location control plane is reaching a critical capacity. If critical capacity is reached, you cannot add more clusters to location. Add more hosts to the control plane zone, or replace unhealthy hosts.
: R0031: A zone for the Satellite location control plane is reaching a warning capacity. Add more hosts to the control plane zone, or replace unhealthy hosts.
: R0032: Manually assign hosts to the control plane across all 3 zones.

Steps to resolve
* The host remediation process might take an extensive amount of time to complete.
* By design, only one host from a vSAN cluster can be in a maintenance mode at any time.
* VUM remediates hosts that are part of a vSAN cluster sequentially even if you set the option to remediate the hosts in parallel.
* Any VM on the host that uses a VM storage policy with a setting of 0 for Number of failures to tolerate, the host might experience unusual delays when it enters maintenance mode. The delay occurs because vSAN must migrate the VM data from one disk to another in the vSAN datastore cluster and this can take many hours. You can work around this by setting Number of failures to tolerate to 1 for the VM storage policy, which results in creating two copies of the VM files in the vSAN datastore.
* Any VM on the host that uses a VM storage policy with a setting of 1 for Number of failures to tolerate, then the VM becomes non-redundant when the host enters maintenance mode. If this is not acceptable, see [Virtual machine vSAN redundancy](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-vum-vsan-redundancy).



To remediate hosts and clusters, follow these steps:



1. Use the vSphere Web Client, select Home > Hosts and Clusters.
2. From the inventory object navigator, select a data center, a cluster, or a host, click the Update Manager tab and click Remediate. If you selected a container object, all hosts under the selected object are remediated. The Remediate wizard opens.
3. Select Patch Baselines or Extension Baselines depending on what type of update you want to perform on the host. On the Select baseline page of the Remediate wizard, select the baseline group and baselines to apply.
4. Select the target hosts that you want to remediate and click Next. If you chose to remediate a single host and not a container object, the host is selected by default.
5.
Cloud Pak for Data Unlimited 
 Premium 100 
 Enterprise 100 
 Plus (includes Trial) 20 



The Sample project is excluded from the total number of projects.





 Renaming a project 

You cannot rename the Sample Project.

To rename a project after you create it, complete the following steps:



1. Go to the My Projects page.
2. Find the project that you want to rename, click the Project actions icon ![Overflow menu](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/overflow-menu-vertical.png), and then choose Rename.
3. Edit the project name, and then click Apply.







 Deleting a project 

If you want to delete a project, but keep a collection from the project, share the collection with another project before you complete these steps. From another project (a type that allows multiple collections), open the Manage collections tab. Click New collection, and then click Reuse data from an existing collection. Select the collection that you want to keep, and then click Finish.

You cannot delete the Sample Project.

To delete a project, complete the following steps:



1. Go to the My Projects page.
2. Find the project that you want to delete, click the Project actions icon ![Overflow menu](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/overflow-menu-vertical.png), and then choose Delete.
3. Click Delete.