vania-janet/multiturn-rag-retrieval-data huggingface dataset:
You can filter the catalog by type to view a specific type of product, for example, only services, software, or deployable architectures. Enter keywords or set additional filters to further scope your view of the catalog. For example, if you want to deploy an analytics instance to Red Hat® OpenShift® on IBM Cloud®, you can select the Analytics category, and filter the results by selecting Red Hat OpenShift as the deployment target.

See the following table for the list of filters that you can use to search the catalog.

Category

Compliance

Pricing plan



Table 1. Options for filtering by category
Use the buttons before the table to change the context of the table. The column headers identify the options for fitering based on filter type.

 Option Description 

 AI / Machine Learning Products that enable systems to learn from data rather than through explicit programming 
 Analytics Products that facilitate the analysis of data, typically large sets of business data, by the use of mathematics, statistics, and other means 
 Blockchain Products that facilitate the process of recording transactions and tracking assets in a business network 
 Compute Infrastructure resources that serve as the basis for building apps in the cloud 
 Containers A standard unit of software that packages up code and all its dependencies so the app runs quickly and reliably from one computing environment to another 
 Converged infrastructure A way of structuring an information technology (IT) system by grouping multiple components into a single optimized computing package, components of which may include servers; data storage devices; networking equipment; and software for IT infrastructure management, automation and orchestration 
 Databases Products that provide some form of access to a database without the need for setting up physical hardware, installing software, or configuring for performance 
 Developer tools Products that support developing, testing, and debugging software 
 Enterprise applications Bundle of compatible products that deliver enterprise-grade app solutions for information sharing, automation, and agility 
 Integration Products that facilitate the connection of data, apps, APIs, and devices across an organization to be more efficient, productive, and agile 
 Internet of Things Products that support receiving and transferring data over wireless networks without human intervention 
 Logging and monitoring Products that support storing, searching, analyzing, and monitoring log data and events. And, products that support reviewing and managing the operational workflow and processes being logged
REMOTESOURCE 'S3'
DELIM '|'
UNIQUEID 'sample'
ACCESSKEYID 'xxxxx'
SECRETACCESSKEY 'xxxxx'
DEFAULTREGION 'ap-geo'
BUCKETURL 'my.sample.bucket'
ENDPOINT 's3.eu-west-1.cloud-object-storage.appdomain.cloud'
MULTIPARTSIZEMB '50'
);



Your data is loaded into Netezza Performance Server. You can start running queries now.
Host: s3.us.cloud-object-storage.appdomain.cloud
Authorization: Bearer {token}

Sample request (simple)

GET /apiary?list-type=2 HTTP/1.1
Authorization: 'AWS4-HMAC-SHA256 Credential={access-key}/{date}/{region}/s3/aws4_request,SignedHeaders=host;x-amz-date;,Signature={signature}'
x-amz-date: {timestamp}
Content-Type: text/plain
Host: s3.us.cloud-object-storage.appdomain.cloud

Example response (simple)

HTTP/1.1 200 OK
Date: Wed, 24 Aug 2016 17:36:24 GMT
X-Clv-Request-Id: 9f39ff2e-55d1-461b-a6f1-2d0b75138861
Accept-Ranges: bytes
Server: Cleversafe/3.13.3.57
X-Clv-S3-Version: 2.5
x-amz-request-id: 9f39ff2e-55d1-461b-a6f1-2d0b75138861
Content-Type: application/xml
Content-Length: 814

<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Name>apiary</Name>
<Prefix/>
<KeyCount>3</KeyCount>
<MaxKeys>1000</MaxKeys>
<Delimiter/>
<IsTruncated>false</IsTruncated>
<Contents>
<Key>drone-bee</Key>
<LastModified>2016-08-25T17:38:38.549Z</LastModified>
<ETag>"0cbc6611f5540bd0809a388dc95a615b"</ETag>
<Delimiter/>
<IsTruncated>false</IsTruncated>
<Contents>
<Key>drone-bee</Key>
<LastModified>2016-08-25T17:38:38.549Z</LastModified>
<ETag>"0cbc6611f5540bd0809a388dc95a615b"</ETag>
<Size>4</Size>
<StorageClass>STANDARD</StorageClass>
</Contents>
<Contents>
<Key>soldier-bee</Key>
<LastModified>2016-08-25T17:49:06.006Z</LastModified>
<ETag>"37d4c94839ee181a2224d6242176c4b5"</ETag>
<Size>11</Size>
<StorageClass>STANDARD</StorageClass>
</Contents>
<Contents>
<Key>worker-bee</Key>
<LastModified>2016-08-25T17:46:53.288Z</LastModified>
<ETag>"d34d8aada2996fc42e6948b926513907"</ETag>
<Size>467</Size>
<StorageClass>STANDARD</StorageClass>
</Contents>
</ListBucketResult>
Show more

Example request (max-keys parameter)

This request lists the objects inside the "apiary" bucket with a max key returned set to 1.

GET /apiary?list-type=2&max-keys=1 HTTP/1.1
Content-Type: text/plain
Host: s3.us.cloud-object-storage.appdomain.cloud
Authorization: Bearer {token}

Sample request (max-keys parameter)

GET /apiary?list-type=2&max-keys=1 HTTP/1.1
[Artifactory](https://jfrog.com/artifactory/) provides a number of different options for sourcing of Terraform providers and fully supports the [Terraform provider registry protocol](https://developer.hashicorp.com/terraform/internals/provider-registry-protocol). It supports, remote, local and virtual repositories which aggregate the first two types with a defined search order.

Local repositories are physical, user managed local repositories acting as a Terraform private registry where you can host custom developed providers and manually upload and save public providers to eliminate the need for public network access. Or limit the public providers made available to Terraform users.

Remote repositories can serve as a caching proxy for both private Terraform registries and the public Terraform registry. Implementing a remote repository still requires public internet access. Here public network access is via Artifactory and not Terraform. Typically many organizations have existing Artifactory installations, with network monitoring and network access rules in place to allow secure public access from Artifactory.

A virtual Terraform repository, combining a local repo with a remote proxy repo, allows for hosting of custom providers locally along with secure access to any additional public Terraform providers.



 Configuring a local Artifactory provider registry 

A local Artifactory registry can be used to host custom developed providers for use with agents in a users private network. Artifactory access is configured using the following workspace environment variables, to configure the Terraform CLI to refer to the the local repository and select providers by name and namespace from this registry.

TF_TOKEN_name.artifactory.user.com:<artifactory_local_registry_token>TF_NETWORK_MIRROR_PROVIDER_NAME:"user_namespace/provider_name"TF_NETWORK_MIRROR_URL=https://name.artifactory.user.com/artifactory/api/terraform/user-terraform-virtual/providers/

Refer to the Artifactory documentation and UI to source the values for the bearer token and URL of the local registry.

Schematics will generate a Terraform CLI configuration of the form below.

provider_installation {
network_mirror {
"catalog_offering": {
"offering": {
"crn": "'$offering_crn'"
}
}'
* Create an instance by using a private catalog image from a specific version of a catalog product offering.

curl -X POST "$vpc_api_endpoint/v1/instances?version=$api_version&generation=2" -H "Authorization:$iam_token" -d '{
"name": "my-instance",
"zone": {
"name": "us-south-3"
},
"vpc": {
"id": "'$vpc'"
},
"primary_network_interface": {
"subnet": {
"id": "'$subnet'"
}
},
"keys":[{"id": "'$key'"}],
"profile": {
"name": "'$profile_name'"
},
"catalog_offering": {
"version": {
"crn": "'$version_crn'"
}
}'







 Provision from an existing volume 

Reusing an existing, bootable volume is faster than creating a new volume from a snapshot or an image.

You can provision an instance with an existing volume by specifying the existing volume's id or crn subproperty as the value of the boot_volume_attachment property.

The existing, bootable volume must be an unattached bootable volume that has the same operating system and architecture as the instance profile. Use the [list volumes](https://cloud.ibm.com/apidocs/vpc/latestlist-volumes) filter and reference the attachment_state property and operating_system property to view a volume's eligibility.

For example, to see unattached volumes in us-south-1 with an x86 operating system.
To ingest all Parquet files in a folder from a S3 location, update the following configuration file:

[global-ingest-config]
target-tables:<TARGET_TABLES>
ingestion-engine:hostname=<hostname>,port=<port>
create-if-not-exist:<true>
[ingest-config1]
source-files:<SOURCE_DATA_FILE>
source-s3-creds:AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ASSESS_KEY>,AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>,AWS_REGION=<YOUR_REGION>, BUCKET_NAME=<YOUR_BUCKET>, ENDPOINT_URL=<YOUR_ENDPOINT_URL>

For example:

[global-ingest-config]
target-tables:iceberg_cat.ice_schema.customer1_tab
ingestion-engine:hostname=localhost,port=8080
create-if-not-exist:true

[ingest-config1]
source-files:s3://cust-bucket/warehouse/
source-s3-creds:AWS_ACCESS_KEY_ID=xxxxxx,AWS_SECRET_ACCESS_KEY=yyyyyy,AWS_REGION=us-east-1,BUCKET_NAME=cust-bucket

In general, this option does not require a staging location. However, there are a few exception scenarios when a staging location must be specified. When the staging location is not used, ensure that the hive catalog configured with Presto can be used with source-files location. Following are the exception cases where staging location is required:



* Any or all of the parquet files in the folder are huge.
* Any or all of the parquet files in the folder have special columns, such as date or decimal.







 Ingest a CSV/Parquet file or a folder of files from a local file system by using config file
X-Clv-Request-Id: 9f39ff2e-55d1-461b-a6f1-2d0b75138861
Accept-Ranges: bytes
Server: Cleversafe/3.9.0.115
X-Clv-S3-Version: 2.5
x-amz-request-id: 9f39ff2e-55d1-461b-a6f1-2d0b75138861
Content-Type: application/xml
Content-Length: 909

<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Name>apiary</Name>
<Prefix/>
<Marker/>
<MaxKeys>1000</MaxKeys>
<Delimiter/>
<IsTruncated>false</IsTruncated>
<Contents>
<Key>drone-bee</Key>
<LastModified>2016-08-25T17:38:38.549Z</LastModified>
<ETag>"0cbc6611f5540bd0809a388dc95a615b"</ETag>
<Size>4</Size>
<Owner>
<ID>{account-id}</ID>
<DisplayName>{account-id}</DisplayName>
</Owner>
<StorageClass>STANDARD</StorageClass>
</Contents>
<Contents>
<Key>soldier-bee</Key>
<LastModified>2016-08-25T17:49:06.006Z</LastModified>
<ETag>"37d4c94839ee181a2224d6242176c4b5"</ETag>
<Size>11</Size>
<Owner>
<ID>{account-id}</ID>
<DisplayName>{account-id}</DisplayName>
</Owner>
* --network-interface: NETWORK_INTERFACE_JSON|@NETWORK_INTERFACE_JSON_FILE, network interface attachment in JSON or JSON file. For the data schema, see the network_interfaces property in the [API documentation](https://cloud.ibm.com/apidocs/vpccreate-instance).
* --metadata-service, --ms: Enable or disable the Instance Metadata Service. One of: true, false.
* --metadata-service-protocol, --msp: The communication protocol for the metadata service endpoint. Applies only when the metadata service is enabled. One of: http, https. (default: http).
* --metadata-service-response-hop-limit, --msrhl: The hop limit (IP time to live) for IP response packets from the metadata service.
* --host-failure-policy: The action to perform if the compute host experiences a failure. One of: restart, stop.
* --resource-group-id: ID of the resource group. This ID is mutually exclusive with --resource-group-name.
* --resource-group-name: Name of the resource group. This name is mutually exclusive with --resource-group-id.
* --output: Specify output format, only JSON is supported. One of: JSON.
* -q, --quiet: Suppress verbose output.



--------------------







 ibmcloud is instance-template-update 

Update an instance template.

ibmcloud is instance-template-update TEMPLATE --name NEW_NAME [--output JSON] [-q, --quiet]



 Command examples 



* ibmcloud is instance-template-update 72251a2e-d6c5-42b4-97b0-b5f8e8d1f479 --name new-name-template
iam_apikey_description: Auto-generated for key crn:v1:bluemix:public:cloud-object-storage:global:a/1836f778c93b485e:9ee8478-b2019a4b4e20:resource-key:21a305a9bf1f4
iam_apikey_name: test-service-key-cos-bucket
iam_role_crn: crn:v1:bluemix:public:iam::::serviceRole:Writer
iam_serviceid_crn: crn:v1:bluemix:public:iam-identity::a/1836f77885e521c5ab2523aac93b485e::serviceid:ServiceId-702ca222-3615-464c-92d3-1849c03170cc
resource_instance_id: crn:v1:bluemix:public:cloud-object-storage:global:a/1836f73b485e:9ee135f9-4667-4794e20::
Show more







 Step 2: Create an Analytics Engine serverless instance 

Create an serverless Analytics Engine instance by using the CLI.



1. Create the Analytics Engine service instance:

Action
: Enter:

ibmcloud resource service-instance-create INSTANCE_NAME ibmanalyticsengine standard-serverless-spark us-south -p @provision.json

Parameter values:



* INSTANCE_NAME: Any name of your choice
* @provision.json: Structure the JSON file as shown in the following example. Use the access and secret key from the response the of Cloud Object Storage service key creation call.



Example of the provision.json file
: Sample JSON file:

{
"default_runtime": {
"spark_version": "3.1" },
"instance_home": {
"region": "us-south",
"access_token" : "exampleString",
"account_name" : "exampleString",
"api_key" : "exampleString",
"auth_type" : "exampleString",
"client_id" : "exampleString",
"client_secret" : "exampleString",
"collection" : "exampleString",
"credentials" : "exampleString",
"database" : "TPCDS",
"host" : "192.168.0.1",
"http_path" : "exampleString",
"jar_uris" : "exampleString",
"jdbc_driver" : "exampleString",
"jdbc_url" : "exampleString",
"password" : "password",
"port" : "50000",
"project_id" : "exampleString",
"properties" : "exampleString",
"refresh_token" : "exampleString",
"role" : "exampleString",
"sap_gateway_url" : "exampleString",
"server" : "exampleString",
"service_name" : "exampleString",
"sid" : "exampleString",
"ssl" : "false",
"ssl_certificate" : "exampleString",
"ssl_certificate_host" : "exampleString",
"ssl_certificate_validation" : "exampleString",
"username" : "db2inst1",
"warehouse" : "exampleString"
}
Show more





 VirtualizeTableSourceTableDefFile 

The following example shows the format of the VirtualizeTableSourceTableDefFile object.

[ {
"column_name" : "Column1",
"column_type" : "INTEGER"
} ]





 VirtualizeTableVirtualTableDefFile
The SAP HANA nameserver provides a Python-based API that is called at important points during the HANA System Replication takeover process. These API calls are used to run customer-specific operations ([Implementing a HA/DR Provider](https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/1367c8fdefaa4808a7485b09815ae0f3.html)).

In the cost-optimized scenario, the SAP HANA HA/DR provider hook is used to automatically reconfigure the SAP HANA instance during the takeover event.

The following section shows a sample set up of a hook script for a cost-optimized SAP HANA System Replication environment. When you implement automation of the cost-optimized SAP HANA System Replication HA environment in the cluster, the takeover hook script must be thoroughly tested. Run the tests manually. Shut down the nonproduction SAP HANA instance on the secondary node, perform a takeover, and verify that the hook script correctly reconfigures the primary HANA DB.



 Creating a database user in the SAP HANA production database 

Use the following steps to create a database user in the SAP HANA production database.



1. Create a database user in the SystemDB of the SAP HANA production system, or provide credentials of an existing user. The hook script uses this database user to connect to the production database and alter the configuration parameters.

Log in to the SystemDB of the primary instance by using the SAP HANA database interactive terminal hdbsql or SAP HANA Cockpit, and create a new user.

For example, connect to the database by using hdbsql in a terminal session.

sudo -i -u ${sid}adm -- hdbsql -i ${INSTNO} -d SYSTEMDB -u SYSTEM

Create a user.

CREATE USER HA_HOOK PASSWORD <Password> no force_first_password_change;

Grant the required privileges to the user.

Grant privilege INIFILE ADMIN to allow for changes of profile parameters.

GRANT INIFILE ADMIN TO HA_HOOK;

Verify the HA_HOOK user.
x-amz-request-id: 9f39ff2e-55d1-461b-a6f1-2d0b75138861
Content-Type: application/xml
Content-Length: 598

<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Name>apiary</Name>
<Prefix/>
<NextContinuationToken>1dPe45g5uuxjyASPegLq80sQsZKL5OB2by4Iz_7YGR5NjiOENBPZXqvKJN6_PgKGVzZYTlws7qqdWaMklzb8HX2iDxxl72ane3rUFQrvNMeIih49MZ4APUjrAuYI83KxSMmfKHGZyKallFkD5N6PwKg</NextContinuationToken>
<KeyCount>1</KeyCount>
<MaxKeys>1</MaxKeys>
<Delimiter/>
<IsTruncated>true</IsTruncated>
<Contents>
<Key>drone-bee</Key>
<LastModified>2016-08-25T17:38:38.549Z</LastModified>
<ETag>"0cbc6611f5540bd0809a388dc95a615b"</ETag>
<Size>4</Size>
<StorageClass>STANDARD</StorageClass>
</Contents>
</ListBucketResult>
Show more

Example request (continuation-token parameter)

This request lists the objects inside the "apiary" bucket with a continuation token specified.
: It enables delivery of container platforms that support DevOps and can be distributed anywhere. Same applications can be instantly moved to cloud only during peak times, optimizing costs.







 Distribution use cases for Satellite 

This use case is applicable to the distribution industry.



 Delivering and integrating store or warehouse digitization solutions for the distribution or retail sectors 

Use Satellite to deliver digitization solutions and integrate with store systems such as store computers and POS, warehouse systems. Leverage AI where applicable.

Pain points:
: It is difficult to deliver and integrate store digitization solution. It is also hard to collect and leverage edge data. Data transfer and privacy concerns hamper the use of AI.

Satellite enables:
: It enables easy delivery of store digitization solution as well as integration with POS data and store computers. IBM Edge Application Manager and Watson Machine Learning can be run on Satellite, allowing analytics at the source of data. Additionally, it allows use and management of container-based applications for retail stores and warehouses.







 Public sector or health use cases for Satellite 

This use case is applicable to the public sector or health industry.



 On-premises service platform 

For hospitals that cannot readily connect to or use cloud, Red Hat OpenShift and IBM Cloud services such as IBM Watson are deployed as managed services within the customer site. These services then become available to use within the hospital environment, while the platform is distributed, managed and controlled remotely.

Pain points:
: It is difficult to use cloud services as hospital medical and patient records cannot be taken offsite from a security standpoint. It is also a challenge to meet data processing needs that will emerge with increased use of wireless technology and local 5G in medical equipment.

Satellite enables:
: Cloud platform services that were previously only available on cloud can be deployed and made available for use within the hospital environment.







 Communications use cases for Satellite 

This use case is applicable to the communications industry.



 Multi-access edge computing (MEC) platform support service 

Configure edge cloud distributed nationwide and use a single control point to remotely run and manage the platform for containerized applications and data.

Pain points:
Content-MD5: xj/vf7lD7vbIe/bqHTaLvg==
Host: s3.us.cloud-object-storage.appdomain.cloud

Example response

HTTP/1.1 200 OK
Date: Wed, 30 Nov 2016 18:54:53 GMT
X-Clv-Request-Id: a6232735-c3b7-4c13-a7b2-cd40c4728d51
Accept-Ranges: bytes
Server: Cleversafe/3.9.0.137
X-Clv-S3-Version: 2.5
x-amz-request-id: a6232735-c3b7-4c13-a7b2-cd40c4728d51
Content-Type: application/xml
Content-Length: 207

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Deleted>
<Key>surplus-bee</Key>
</Deleted>
<Deleted>
<Key>unnecessary-bee</Key>
</Deleted>
</DeleteResult>

--------------------







 Add or extend retention on an object 

A PUT issued to an object with the proper parameters adds or extends retention period of the object.

Not all operations are supported in Satellite environments. For details, see [supported Satellite APIs](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-apis-cos-satellite)

Syntax

PUT https://{endpoint}/{bucket-name}/{object-name}?retention path style
PUT https://{bucket-name}.{endpoint}/{object-name}?retention virtual host style

Payload Elements
WHERE responseCode == 500
GROUP BY HOST
ORDER BY 2 DESC

-- Which requests showed a page not found error to the user?
SELECT DISTINCT REQUEST
FROM cos://<REGION>/<BUCKET-NAME>/log-stream-landing/topic=webserver/jobid=<JOBID> STORED AS PARQUET
WHERE responseCode == 404

-- What are the top 10 largest files?
SELECT DISTINCT REQUEST, BYTES
FROM cos://<REGION>/<BUCKET-NAME>/log-stream-landing/topic=webserver/jobid=<JOBID> STORED AS PARQUET
WHERE BYTES > 0
ORDER BY CAST(BYTES as Integer) DESC
LIMIT 10

-- What is the distribution of total traffic by hour?
SELECT SUBSTRING(TIME_STAMP, 13, 2), COUNT()
FROM cos://<REGION>/<BUCKET-NAME>/log-stream-landing/topic=webserver/jobid=<JOBID> STORED AS PARQUET
GROUP BY 1
ORDER BY 1 ASC

-- Why did the previous result return an empty hour?
-- Hint, find the malformed hostname.
SELECT HOST, REQUEST
FROM cos://<REGION>/<BUCKET-NAME>/log-stream-landing/topic=webserver/jobid=<JOBID> STORED AS PARQUET
WHERE SUBSTRING(TIME_STAMP, 13, 2) == ''







 Step 4: Investigating data using Analytics Engine serverless plan 

The data stream landed to cos can be also queried using Apache Spark that is part of the Analytics Engine service serverless plan. Programs will need to be loaded into a bucket and executed from the command line. The pyspark python environment will be used in this tutorial.

Open the analytics engine service:



1. Access the Analytics Engine service instance from the [Resource List](https://cloud.ibm.com/resources) under Analytics.
2. Click the Applications tab.



Try the following: