vania-janet/multiturn-rag-retrieval-data huggingface dataset:
: To set up an HTTP proxy, you must first [remove your hosts](https://cloud.ibm.com/docs/satellite?topic=satellite-host-remove) from your location. After you remove your hosts, see [Configuring your HTTP proxy](https://cloud.ibm.com/docs/satellite?topic=satellite-config-http-proxyhttp-proxy-config). Note that you must also update the hosts that make up your location control plane. See [Updating Satellite location control plane hosts](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-location).

New Red Hat CoreOS enabled locations
: Before attaching your hosts to your location, [configure your HTTP proxy](https://cloud.ibm.com/docs/satellite?topic=satellite-config-http-proxyhttp-proxy-config).





 What type of hosts can I use? 

You can use RHEL or Red Hat CoreOS hosts when you set up an HTTP proxy. You must edit each host that is attached to your location, including the hosts that make up the control plane.





 What else do I need to know about HTTP proxy? 

For your Satellite location and clusters to work with a proxy, the kubelet on the control plane infrastructure nodes that are deployed to a Satellite location must be able to communicate to the IBM Cloud control plane node API server. To enable this communication, you must meet one of the following requirements.



* Option 1: The current proxy can support long lived TCP connections (TCP tunneling).
* Option 2: You can create a secondary proxy on a VSI in same network as your Satellite hosts that supports long lived TCP connections.
* Option 3: You can open the firewall outbound to allow the TCP connections. For more information, see [Required outbound connectivity for hosts in all regions](https://cloud.ibm.com/docs/satellite?topic=satellite-reqs-host-network-outbound) and then find the specific outbound network requirements for your region.
Create a configuration file for your stateful set and the service that you use to expose the stateful set. The following examples show how to deploy NGINX as a stateful set with three replicas, each replica with a separate bucket or sharing the same bucket.

Example to create a stateful set with three replicas, with each replica using a separate bucket.

apiVersion: v1
kind: Service
metadata:
name: nginx-v01
namespace: default
labels:
app: nginx-v01 must match spec.template.metadata.labels and spec.selector.matchLabels in stateful set YAML
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx-v01 must match spec.template.metadata.labels and spec.selector.matchLabels in stateful set YAML
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web-v01
namespace: default
spec:
selector:
matchLabels:
app: nginx-v01 must match spec.template.metadata.labels in stateful set YAML and metadata.labels in service YAML
serviceName: "nginx-v01"
replicas: 3
template:
metadata:
labels:
app: nginx-v01 must match spec.selector.matchLabels in stateful set YAML and metadata.labels in service YAML
spec:
terminationGracePeriodSeconds: 10
containers:
- name: nginx
image: k8s.gcr.io/nginx-slim:0.8
ports:
- containerPort: 80
name: web
volumeMounts:
- name: mypvc
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: mypvc
annotations:
ibm.io/auto-create-bucket: "true"
Deploying your app with a project endpoint 

You can set the endpoint visibility for your app such that it is deployed with an internal (project) endpoint. Setting a project-only endpoint means that your app is not accessible from the public internet and network access is only possible from other Code Engine components that are running within the same Code Engine project. This endpoint is always enabled. Applications are still accessible through shared components and therefore need to be secured.

For example, if your solution consists of several applications within a project, you might set up your solution such that only one of those applications is visible from the internet so that it handles incoming traffic. This public-facing application can delegate work to other applications in your solution so that they do not need to be visible from the internet.

With the CLI, set the endpoint visibility for your app so that it is deployed with a project endpoint by using the --visibility=project option on the [app create](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-application-create) or [app update](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-application-update) command. You can obtain the available URLs for your app that reflect your endpoint definition by using the [app get](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-application-get) command.

From the console, set the visibility of endpoints for your app by using the Endpoints setting when you create your app. After your app is deployed, you can view and modify these system domain mapping settings on the Domain mappings tab on your application page.







 Options for deploying a Code Engine application 

Learn about the options that you can specify when you deploy your app. Note that options can vary between the console and the CLI.



 Memory and CPU 

When you deploy your app, you can specify the amount of memory and CPU that your app can consume. These amounts can vary, depending on if your app is compute-intensive, memory-intensive, or balanced.

By default, your application is assigned 4 G of memory and 1 vCPU.
Select a zone 2 location. For example, London 2.
4. Select the number of IP addresses.
5. Set Virtual Private Cloud.
6. Set the Public gateway to Detached.
7. Click Create subnet.



To confirm the creation of the subnet, click Subnets on the left pane and wait until the status changes to Available.







 Step 2: Create two security groups 

These security groups allow only specific inbound traffic to the server and application.

To specify which traffic to allow to the application, you deploy the following rules, which are added to virtual servers in the later steps.



1. Navigate to Security Groups and click Create.
2. Verify Regions is correct. For example, London.
3. Click Create to create a new security group.
4. Type vpc-region1-jumphost-sg for Name.
5. Create the following Inbound rules:



* Protocol: TCP
* Source Type: Any
* Port: Any



6. Create the Outbound rules as:



* Protocol: TCP
* Destination Type: Any
* Port: Any



7. Verify that the Region is correct. For example, London.





 Provision a bastion (jumphost) virtual server 



1. Go to Subnets.
2. Verify that the status is Available, then click vpc-region1-zone1-mgmt > Attached > Attached Instances and click Create to create a new Attached Instance.
3. Enter jumphost-vsi as your virtual server's unique name.
4. Select the VPC that you created (vpc-region1), the Resource group, and Location.
5. Set the Operating System image to Ubuntu Linux and pick any version of the image.
6. Select Compute with 2vCPUs and 8 GB RAM as your profile. To check other available profiles, click All profiles.
7.
Operator: core.StringPtr("and"),
Conditions: []iampolicymanagementv1.V2PolicyAttribute{
weeklyConditionAttribute, startConditionAttribute, endConditionAttribute},
}

options := iamPolicyManagementService.NewV2CreatePolicyOptions(
"access",
v2PolicyControl,
)
options.SetSubject(policySubject)
options.SetResource(policyResource)
options.SetRule(policyRule)
options.SetPattern(core.StringPtr("time-based-conditions:weekly"))

policy, response, err := iamPolicyManagementService.V2CreatePolicy(options)
if err != nil {
panic(err)
}
b, _ := json.MarshalIndent(policy, "", " ")
fmt.Println(string(b))

const policySubject = {
attributes: [
{
key: 'iam_id',
operator: 'stringEquals',
value: exampleUserId,
},
],
};
const policyResourceAccountAttribute = {
key: 'accountId',
value: exampleAccountId,
operator: 'stringEquals',
};
const policyResourceServiceAttribute = {
key: 'serviceName',
operator: 'stringEquals',
value: 'iam-groups',
};
const policyResource = {
attributes: [policyResourceAccountAttribute, policyResourceServiceAttribute]
};
const policyControl = {
grant: {
roles: [{
Setting up your API environment 

Before you can use the transit gateway API, you need to setup your environment.



 General prerequisites 

Set up your account to access the transit gateway. Make sure that your account is [upgraded to a paid account](https://cloud.ibm.com/docs/account?topic=account-accountfaqschangeacct).





 API prerequisites 

Before you can use the API to interact with a transit gateway, you must get an IAM token, store the endpoint as a variable, and verify that you have access to the IBM Cloud Transit Gateway API service.

The following examples use the transit.cloud.ibm.com global endpoint.



 Step 1: Store your API key as a variable 

Run the following command to store the API key for your account in an environment variable. If you don't have an API key, see [Creating an API key](https://cloud.ibm.com/docs/account?topic=account-userapikeycreate_user_key).

apikey="<YOUR_API_KEY>"





 Step 2: Get an IBM Identity and Access Management (IAM) token 

Run the following command to get and parse an IAM token by using the JSON processing utility [jq](https://stedolan.github.io/jq/). You can modify the command to use another parsing tool, or you can remove the last part of the command if you prefer to manually parse the token.

IAM_TOKEN=curl -k -X POST --header "Content-Type: application/x-www-form-urlencoded" --header "Accept: application/json" --data-urlencode "grant_type=urn:ibm:params:oauth:grant-type:apikey" --data-urlencode "apikey=$apikey" "https://iam.cloud.ibm.com/identity/token" |jq -r '(.token_type + " " + .access_token)'

To view the IAM token, run echo $IAM_TOKEN. The result should look like this:
SetAttributeValueResponse, err := cryptoClient.SetAttributeValue(context.Background(), SetAttributeValueRequest)
* JavaScript code snippet

const attributeTemplate = new util.AttributeMap(
new util.Attribute(ep11.CKA_SIGN, true)
);

client.SetAttributeValue({
Object: keys.PrivKey,
Attributes: attributeTemplate
}, (err, response) => {
callback(err, response);
});









 Generating random data 

You can generate high-quality random data, such as initialization values (IVs), PIN, and password, for use in cryptographic operations.



 GenerateRandom 

The GenerateRandom function generates random data. When you use this function, make sure not to set the length of the random data to be zero and the pointer that points to the random data location to be NULL.

Enterprise PKCS #11 over gRPC

Enterprise PKCS #11

PKCS #11





 Description Binds to EP11 m_GenerateRandom, which is an implementation of PKCS #11 C_GenerateRandom. 
 Parameters message GenerateRandomRequest { uint64 Len = 1; } message GenerateRandomResponse { bytes Rnd = 1; } 
 Return values Wraps EP11 error into message Grep11Error. 



Code snippets



* Golang code snippet

GenerateRandomRequest := &pb.GenerateRandomRequest {
Len: 1024,
}

GenerateRandomResponse, err := cryptoClient.GenerateRandom(context.Background(), GenerateRandomRequest)
* JavaScript code snippet

client.GenerateRandom({
Len: ep11.AES_BLOCK_SIZE
}, (err, response) => {
callback(err, response);
});









 Encrypting and decrypting data
For applications to be able to scale down cleanly, your application code must handle a SIGTERM signal. When Code Engine automatically scales down, a SIGTERM signal is sent to your application. If your application does not handle the SIGTERM signal, then your app instances stay in Terminating status for the time that is configured by the request timeout value (default value is 300 seconds). After the duration of the request timeout, Code Engine sends a SIGKILL signal to forcefully stop the app instances in Terminating status.



 Scaling boundaries 

You can configure scaling boundaries by setting a range of values for the minimum and maximum number of instances within which Code Engine autoscales the number of running instances of your application. Configure this range when you create or update applications.



* Minimum number of instances - The minimum number of instances of the app that are always running, even if no requests are processed. When set to 0 (default) Code Engine removes all instances when no traffic is reaching the application. To always keep a running instance of your application, set this value greater than 0. When you create or update an application with Code Engine from the console, set the minimum number of instances value in the Autoscaling section of the Runtime tab. With the CLI, specify the --min-scale option on the app create and app update commands.
* Maximum number of instances - The maximum number of instances that can be running for the app. Autoscaling occurs up to this limit. If you set this value to 0, the application scales as needed, and the app scaling is limited only by the instances per the resource quota for the project of your app. See [Limits and quotas for Code Engine](https://cloud.ibm.com/docs/codeengine?topic=codeengine-limits). When you create or update an application with Code Engine from the console, set the maximum number of instances value in the Autoscaling section of the Runtime tab. With the CLI, specify the --max-scale option on the app create and app update commands. The default value for this option is 10.
Setting up your API environment 

Before you can use the Direct Link API, you must set up your environment.



 General prerequisites 

Set up your account to access the direct link. Make sure that your account is [upgraded to a paid account](https://cloud.ibm.com/docs/account?topic=account-accountfaqschangeacct).





 API prerequisites 

Before you can use the API to interact with a direct link, you must get an IAM token, store the endpoint as a variable, and verify that you have access to the IBM Cloud Direct Link API service.

The following examples use the directlink.cloud.ibm.com global endpoint.



 Step 1: Store your API key as a variable 

Run the following command to store the API key for your account in an environment variable. If you don't have an API key, see [Creating an API key](https://cloud.ibm.com/docs/account?topic=account-userapikeycreate_user_key).

apikey="<YOUR_API_KEY>"





 Step 2: Get an IBM Identity and Access Management (IAM) token 

Run the following command to get and parse an IAM token by using the JSON processing utility [jq](https://stedolan.github.io/jq/). You can modify the command to use another parsing tool, or you can remove the last part of the command if you prefer to manually parse the token.

IAM_TOKEN=curl -k -X POST --header "Content-Type: application/x-www-form-urlencoded" --header "Accept: application/json" --data-urlencode "grant_type=urn:ibm:params:oauth:grant-type:apikey" --data-urlencode "apikey=$apikey" "https://iam.cloud.ibm.com/identity/token" |jq -r '(.token_type + " " + .access_token)'

To view the IAM token, run echo $IAM_TOKEN. The result should look like this:
Decide if you want to use the default Key Protect service instance or your own key management service provider.
7. Review the pricing plan.
8. Click Create to set up your service instance. The setup might take a few minutes to complete.



3. Create service credentials for your Databases for etcd service instance.



1. In the navigation on the service details page, click Service Credentials.
2. Click New credentials.
3. Enter a name for your service credentials and click Add.



4. Retrieve your service credentials and certificate.



1. From the navigation on the service details page, select Service credentials.
2. Find the credentials that you want to use, and from the Actions column in the service credentials table, click View credentials.
3. Find the grp.authentication section of your service credentials and note the username and password.

Example output for username and password:

"grpc": {
"authentication": {
"method": "direct",
"password": "123a4567ab89cde09876vaa543a2bc2a10a123456bcd123456f0a7895aab1de",
"username": "ibm_cloud_1abd2e3f_g12h_3bc4_1234_5a6bc7890ab"
}
4. Find the composed section of your service credentials and note the etcd --endpoints. You need this information when you install Portworx.

Example output for --endpoints:

--endpoints=https://1ab234c5-12a1-1234-a123-123abc45cde1.123456ab78cd9ab1234a456740ab123c.databases.appdomain.cloud:32059
5. Find the certificate section of your service credentials and note the certificate_base64.

Example output for certificate:

"certificate": {
"certificate_base64": "AB0cAB1CDEaABcCEFABCDEF1ACB3ABCD1ab2AB0cAB1CDEaABcCEFABCDEF1ACB3ABCD1ab2AB0cAB1CDEaABcCEFABCDEF1ACB3ABCD1ab2..."



5.
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(fileText.length());

PutObjectRequest req = new PutObjectRequest(
bucketName,
objectName,
newStream,
metadata
);
req.setRetentionLegalHoldId(legalHoldId);

cos.putObject(req);

System.out.printf("Legal hold %s added to object %s in bucket %sn", legalHoldId, objectName, bucketName);
}

public static void copyProtectedObject(String sourceBucketName, String sourceObjectName, String destinationBucketName, String newObjectName) {
System.out.printf("Copy protected object %s from bucket %s to %s/%s.n", sourceObjectName, sourceBucketName, destinationBucketName, newObjectName);

CopyObjectRequest req = new CopyObjectRequest(
sourceBucketName,
sourceObjectName,
destinationBucketName,
newObjectName
);
req.setRetentionDirective(RetentionDirective.COPY);

cos.copyObject(req);

System.out.printf("Protected object copied from %s/%s to %s/%sn", sourceObjectName, sourceBucketName, destinationBucketName, newObjectName);
}

Show more

def put_object_add_legal_hold(bucket_name, object_name, file_text, legal_hold_id):
print("Add legal hold {0} to {1} in bucket {2} with a putObject operation.n".format(legal_hold_id, object_name, bucket_name))

cos.put_object(
Bucket=bucket_name,
Key=object_name,
Body=file_text,
RetentionLegalHoldId=legal_hold_id)
: In the spec volume claim templates spec resources requests section, if you want to provision [performance storage](https://cloud.ibm.com/docs/containers?topic=containers-file_storagefile_predefined_storageclass), enter the number of IOPS. If you use an endurance storage class and specify a number of IOPS, the number of IOPS is ignored. Instead, the IOPS that is specified in your storage class is used.

storageClassName
: In the spec volume claim templates spec section, enter the storage class that you want to use. To list existing storage classes, run kubectl get storageclasses | grep file. If you don't specify a storage class, the PVC is created with the default storage class that is set in your cluster. Make sure that the default storage class uses the ibm.io/ibmc-file provisioner so that your stateful set is provisioned with File Storage for Classic.
4. Create your stateful set.

kubectl apply -f statefulset.yaml
5. Wait for your stateful set to be deployed.

kubectl describe statefulset <statefulset_name>



To see the current status of your PVCs, run kubectl get pvc. The name of your PVC is formatted as <volume_name>-<statefulset_name>-<replica_number>.





 Static provisioning: Using an existing PVC with your stateful set 

You can pre-provision your PVCs before creating your stateful set or use existing PVCs with your stateful set.

When you [dynamically provision your PVCs when creating the stateful set](https://cloud.ibm.com/docs/containers?topic=containers-file_storagefile_dynamic_statefulset), the name of the PVC is assigned based on the values that you used in the stateful set YAML file. In order for the stateful set to use existing PVCs, the name of your PVCs must match the name that would automatically be created when using dynamic provisioning.

Before you begin: [Log in to your account.
metadata.setContentLength(theBytes.size()); // set metadata for the length of the data stream
cos.putObject(
"sample", // the name of the bucket to which the object is being written
"serialized-object", // the name of the object being written
stream, // the name of the data stream writing the object
metadata // the metadata for the object being written
);
Show more

Alternatively, you can use a CipherInputStream to more easily encrypt the data stream without needing to overload the existing InputStream object.

public CipherInputStream encryptStream(InputStream inputStream) {
// Generate key
KeyGenerator kgen = KeyGenerator.getInstance("AES");
kgen.init(128);
SecretKey aesKey = kgen.generateKey();
// Encrypt cipher
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.ENCRYPT_MODE, aesKey);
CipherInputStream cis = new CipherInputStream(inputStream, cipher);
return cis;
}





 Download object to a file 

This example assumes that the bucket sample exists.

GetObjectRequest request = new // create a new request to get an object
GetObjectRequest( // request the new object by identifying
"sample", // the name of the bucket
"myFile" // the name of the object
);

s3Client.getObject( // write the contents of the object
request, // using the request that was just created
new File("retrieved.txt") // to write to a new file
);





 Download object by using a stream 

This example assumes that the bucket sample exists.

S3Object returned = cos.getObject( // request the object by identifying
"sample", // the name of the bucket
You need to configure the e0b interface to create a CIFS/SMB mount point. Use the following steps to configure the e0b interfave.



1. Go to Settings > Data Interfaces and expand the e0b interface. Select Enable Data Interface, and enter the IP address, Subnet Mask, and Gateway that you use to mount the CIFS/SMB share.
2. Leave the default MTU value of 1500 bytes. \n Although the default maximum transmission unit (MTU) is set to 1,500, you can change it to 9,000 if you use jumbo frames. Your ESXi host and physical infrastructure is required to support jumbo frames. By default, IBM Cloud already supports an MTU size of 9,000 bytes with no configuration changes needed.
3. Click Apply. The mount point is ready for configuration.
4. Select Storage > CIFS > Add CIFS Share and enter a unique name.
5. Click the Pin share drop-down menu and select Yes. Note: Veeam Backup and Replication backups can fail to an unpinned share.
6. Enter a unique path for the share in the Path field. It is preferable to use the share name as the path, that is, if the share name is cifs_share0, enter /cifs_share0 as the path.
7. Clear Allow Everyone Access if security is not an issue. It is preferable to allowlist the clients that use the CIFS/SMB share. Otherwise, leave Allow Everyone Access selected if security is an issue and click Add Share.
8. Click Add CIFS User to create accounts for authorized users and complete the username and Password fields.
9. Expand the new CIFS share and click Add a user or group to add the authorized user accounts.
10. Go to Global CIFS Settings and click the Listening Interface drop-down menu and select e0b and click Apply.



The AltaVault appliance is configured to allow communications between itself, IBM Cloud Object Storage, and the computer that is running Veeam Backup and Replication.