vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Using the Kafka Java client 

The Java™ Kafka API sample is an example producer and consumer that is written in Java, which directly uses the Kafka API. You can run this sample locally or in IBM Cloud®.

The sample code is in the [event-streams-samples GitHub project](https://github.com/ibm-messaging/event-streams-samples/tree/master/kafka-java-console-sample). Although the sample uses the Kafka API to send and receive messages, the sample uses the Event Streams Administration API to create the topic it sends messages to and receives messages from.

For more information about the setup and running of the sample, see the [README.md](https://github.com/ibm-messaging/event-streams-samples/tree/master/kafka-java-console-sample).

For a detailed walkthrough of how to run the sample, see [Getting started with Event Streams](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-getting-startedgetting_started_steps).



 How to use, download, and run the Liberty for Java sample 

The Liberty for Java sample implements a simple application that is deployed onto the Liberty runtime. The application uses the Kafka API for Event Streams to produce and consume messages. The application also serves up a web front end that you can use for administration.

You can find the sample code in the [event-streams-samples GitHub project](https://github.com/ibm-messaging/event-streams-samples/tree/master/kafka-java-liberty-sample).





 Using the sasl.jaas.config property 

If you use a Kafka client at 0.10.2.1 or later, you can use the sasl.jaas.config property for client configuration instead of a JAAS file. To connect to Event Streams, set sasl.jaas.config as in the following example:

sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required
Code Risk Analyzer plug-in for IBM Cloud 

The IBM Cloud® command-line interface (CLI) provides code risk analysis commands. You can use the IBM Cloud CLI to analyze your code for vulnerabilities and compliance with certain rules. Code Risk Analyzer is available in all IBM Cloud regions where toolchains are supported.

You can use the CLI to complete the following tasks:



* Generate a Bill of Materials (BOM) file that lists the dependencies and available license information of all third-party OS packages and application packages. You can also generate this output in CycloneDX-specific format.
* Discover vulnerabilities in packages that are listed in the BOM file. You can also view the generated report in CycloneDX-specific format, or use vulnerability auto-remediation for Node.js, Maven, or Gradle applications.
* Analyze a Terraform plan for compliance with certain rules.
* Analyze Kubernetes files for compliance with certain rules.





 Supported content 

Code Risk Analyzer supports the Java™, Node.js, Python, and Go languages. The following table lists and describes the content that Code Risk Analyzer supports.



Table 1. Supported content

 Content Description 

 Java The repo must use Maven or Gradle. For Maven, dependencies are computed by using the pom.xml file. For Gradle, dependencies are computed by using the Build.gradle file. Code Risk Analyzer can also perform auto-remediation for both Maven and Gradle. 
 Node.js Dependencies are computed by using the package-lock.json file. For Node.js, Code Risk analyzer can also perform auto-remediation. Make sure that the installed version of npm matches the project npm version. 
 Python Dependencies are computed by using the requirements.txt file. 
 Golang Supports go mod and go dep dependency management. For go mod, the go.sum file must be in the repo. For go dep, the Gopkg.lock file must be in the repo.
Getting instances of build run 'tutorial-build-run-851026-090000000'...
Getting logs for build run 'tutorial-build-run-851026-090000000'...
OK

tutorial-build-run-851026-090000000-7vlrw-pod-c9t6g/step-git-source-source-7tbwh:
{"level":"info","ts":1610378702.42616,"caller":"git/git.go:165","msg":"Successfully cloned https://github.com/IBM/CodeEngine @ 6b66f2bc3e1277c2e6475608a9c50335712116e0 (grafted, HEAD, origin/main) in path /workspace/source"}
{"level":"info","ts":1610378703.5453625,"caller":"git/git.go:203","msg":"Successfully initialized and updated submodules in path /workspace/source"}
[...]

Your container image is successfully built from source code by using the buildpacks build strategy!





 Step 4: Work with the container image 

Now that your container image is built and the image is pushed to the configured container registry location, you can deploy the image in Code Engine. For example, you can create a Code Engine app that uses the created image.

ibmcloud ce application create --name tutorial-app --image <your_docker_ID>/tutorial

Example output

Creating application 'tutorial-app'...
[...]





 Next steps for buildpacks 

For more information, see [Deploying applications](https://cloud.ibm.com/docs/codeengine?topic=codeengine-application-workloads).

Does your source code project use Dockerfile?
In addition to the sample code snippets in this topic, you can work with Node.js code samples from the [IBM Analytics Engine V3 API reference](https://cloud.ibm.com/apidocs/ibm-analytics-engine-v3?code=nodeintroduction).

The following code samples show how to:



* Authenticate to the IBM Analytics Engine service and create a service client:

const IbmAnalyticsEngineApiV3 = require('iaesdk/ibm-analytics-engine-api/v3');
const { IamAuthenticator } = require('iaesdk/auth');

const IAM_API_KEY = "{apikey}" // eg "W00YiRnLW4a3fTjMB-odB-2ySfTrFBIQQWanc--P3byk"
const IAE_ENDPOINT_URL = "{url}" // Current list available at https://cloud.ibm.com/apidocs/ibm-analytics-engineservice-endpoints
const API_AUTH_URL = "{api auth url}" // "https://iam.cloud.ibm.com/identity/token"
const DEFAULT_SERVICE_NAME = 'ibm_analytics_engine_api'

let options = {};

// Create an IAM authenticator.
options.authenticator = new IamAuthenticator({
apikey: IAM_API_KEY,
url: API_AUTH_URL
});

options.serviceUrl = IAE_ENDPOINT_URL;
options.serviceName = DEFAULT_SERVICE_NAME;

// Construct the service client.
const ibmAnalyticsEngineApiService = new IbmAnalyticsEngineApiV3.newInstance(options);
Show more

Key values:



* serviceUrl: public endpoint to the IBM Analytics Engine instance.
Building a container image with stand-alone build commands (CLI) 

When your code exists as source in a local file or in a Git repository, Code Engine provides options for you to build your code as a container image, per the [Open Container Initiative (OCI) standard](https://opencontainers.org/). With the Code Engine CLI, you can build a container image with a single command.

Code Engine offers the flexibility to handle the build process for you whenever you are [deploying your app from repository source code](https://cloud.ibm.com/docs/codeengine?topic=codeengine-app-source-code), [deploying your app from local source code with the CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-app-local-source-code), or you are [creating a job from repository source code](https://cloud.ibm.com/docs/codeengine?topic=codeengine-run-job-source-code) or [creating your job from local source code with the CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-job-local-source-code).

However, if you want more control over the build of your container image, Code Engine gives you the flexibility to build your container image by using one of the following options:



* Use a single CLI command, buildrun submit, to run one build run. The benefit of this option is that you can obtain your build with a single CLI command; however, the configuration for the build is not preserved.
* Define a build configuration from which you can run multiple build runs. See [Building a container image by using a build configuration](https://cloud.ibm.com/docs/codeengine?topic=codeengine-build-image).



After your build is complete and your image exists, you can deploy the container image as an application or create a job that references your image.

Let's learn about how to build your container image with a single command and not reference a build configuration.
[Toolchain name and region](https://cloud.ibm.com/docs-content/v1/content/562806c0b18119f4d21cb66f5ae6d051634b4a16/ContinuousDelivery/images/cd-only-satellite-toolchain-settings.png)

Figure 2. Toolchain name and region
2. To proceed to the next step to configure your toolchain, click Continue. To create the toolchain by using the current settings, click Create toolchain.







 Configure the deployment source repo 

The deployment source repo contains all of the deployment source code that is required to deploy the app.



1. In the Source repository step, the default options for the deployment source repo are displayed. To view all of the available options for the underlying Source Provider, click Switch to advanced configuration. By default, the toolchain uses the sample that clones the sample deployment as an IBM-hosted Git Repos and Issue Tracking repo.

Zoom

![Deployment source repo](https://cloud.ibm.com/docs-content/v1/content/562806c0b18119f4d21cb66f5ae6d051634b4a16/ContinuousDelivery/images/cd-only-satellite-deployment-source-repo.png)

Figure 3. Deployment source repo
2. Specify the name of the deployment source repo that you want to use. The region of the repo remains the same as the region of the toolchain.

The toolchain template provides an [IBM Cloud Satellite Config Sample application](https://github.com/IBM/satellite-config-example). To link an existing deployment source repo for the toolchain, click Switch to advanced configuration and specify the URL for the repo. The toolchain supports linking only to existing Git Repos and Issue Tracking repos.



By default, the deployment source repo template is cloned to your Git Repos and Issue Tracking org.
<DaysAfterInitiation>3</DaysAfterInitiation>
</AbortIncompleteMultipartUpload>
</Rule>
</LifecycleConfiguration>
Show more

Code sample for use with NodeJS COS SDK

Using the IBM Cloud® Object Storage SDKs only requires calling the appropriate functions with the correct parameters and proper configuration.

var aws = require('ibm-cos-sdk');
var ep = new aws.Endpoint('s3.us-south.cloud-object-storage.appdomain.cloud');
var config = {
endpoint: ep,
apiKeyId: 'ZRZDoNoUseOLL7bRO8SAMPLEHPUzUL_-fsampleyYE',
ibmAuthEndpoint: 'https://iam.cloud.ibm.com/identity/token',
serviceInstanceId: 'crn:v1:bluemix:public:cloud-object-storage:global:a/<CREDENTIAL_ID_AS_GENERATED>:<SERVICE_ID_AS_GENERATED>::',
};
var s3 = new aws.S3(config);

var params = {
Bucket: 'STRING_VALUE', /* required /
LifecycleConfiguration: {
Rules: [ /* required /
{
Status: 'Enabled', /* required /
ID: 'OPTIONAL_STRING_VALUE',
Filter: {}, /* required /
AbortIncompleteMultipartUpload: {
DaysAfterInitiation: 'NUMBER_VALUE'
}
},
]
}
};

s3.putBucketLifecycleConfiguration(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});
Show more

Code sample for use with Python COS SDK
For Node.js, Code Risk analyzer can also perform auto-remediation. Make sure that the installed version of npm matches the project npm version. 
 Python Dependencies are computed by using the requirements.txt file. 
 Golang Supports go mod and go dep dependency management. For go mod, the go.sum file must be in the repo. For go dep, the Gopkg.lock file must be in the repo. 
 Dockerfiles Files with the Dockerfile pattern in the repo are considered. For container images, the Debian, Red Hat Enterprise Linux®, Alpine, and Ubuntu Linux distros are supported. 
 Kubernetes Files that are suffixed with .yaml and .yml are considered. The kind value must be set to Pod, ReplicaSet, ReplicationController, Deployment, Daemonset, Statefulset, Job, CronJob, NetworkPolicy, or Ingress. 
 Calico Files that are suffixed with .yaml and .yml are considered. The kind value must be set to NetworkPolicy, GlobalNetworkPolicy, Profile, NetworkSet, GlobalNetworkSet, or HostEndpoint. 
 Terraform The Terraform plan file must be generated by using IBM Cloud as the Terraform provider. 







 Known issues with Code Risk Analyzer 

Code Risk Analyzer cannot discover vulnerabilities on application packages that do not use a versioning scheme, such as major.minor.patch. For example, pre-release versions or versions that contain build metadata are not supported.





 Prerequisites 



* Install the IBM Cloud CLI. See [Download IBM Cloud CLI](https://cloud.ibm.com/docs/cli?topic=cli-getting-started) for instructions.
* Install the Code Risk Analyzer CLI plug-in by running the following command:



ibmcloud plugin install cra



* Make sure that you can access a toolchain in one of the supported regions. The toolchain is not required to have any tools.
* If you are starting with source code that resides on a local workstation, you can choose to let Code Engine take care of building the image from your source and creating the job with a single CLI command. In this scenario, Code Engine uploads your image to IBM Cloud® Container Registry. To learn more, see [Creating your job from local source code with the CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-job-local-source-code). If you want more control over the build of your image, then you can choose to [build the image](https://cloud.ibm.com/docs/codeengine?topic=codeengine-build-image) with Code Engine before you create (or update) your job and run the job.



For example, you might choose to let Code Engine handle the build of your local source while you evolve the development of your source for the job. Then, after the image is matured, you can update the job to reference the specific image that you want. You can repeat this process as needed.

When you run your updated job, the latest version of your referenced container image is used for the job run, unless a tag is specified for the image. If a tag is specified for the image, then the tagged image is used for the job run.



Looking for more code examples? Check out the [Samples for IBM Cloud Code Engine GitHub repo](https://github.com/IBM/CodeEngine).
Use the ibmcloud ce jobrun submit command to run your job that references the built image. For a complete listing of options, see the [ibmcloud ce job create](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-job-create) and [ibmcloud ce jobrun submit](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-jobrun-submit) commands.

For information about required permissions for accessing image registries, see [Setting up authorities for image registries](https://cloud.ibm.com/docs/codeengine?topic=codeengine-add-registryauthorities-registry).



1. Use the job create command to create the myjob-repo job to reference an image that is built from the https://github.com/IBM/CodeEngine build source. This command automatically builds the image and uploads the image to an IBM Cloud® Container Registry namespace in your account and job runs for this job reference this built image. By specifying the --build-context-dir option, the build uses the source in the helloworld directory. This example command uses the default dockerfile strategy, and the default medium build size. Because the branch name of the repository is not specified with the --build-commit option, Code Engine automatically uses the default branch of the specified repository. By adding the --wait option, this specifies for the job create to wait for the image build to complete.

ibmcloud ce job create --name myjob-repo --build-source https://github.com/IBM/CodeEngine --build-context-dir helloworld --wait

Example output

Creating job 'myjob-repo'...
Submitting build run 'myjob-repo-run-220420-15590196'...
Creating image 'private.us.icr.io/ce--abcde-glxo4kabcde/job-myjob-repo'...
* [Creating a sequence from the console](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-sequencesactions_seq_console)
* [Creating a sequence from the CLI](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-sequencesactions_seq_cli)



[Preparing apps for actions](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep)



* [Preparing JavaScript apps](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js)



* [Structuring JavaScript code](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js_struct)
* [Structuring JavaScript code with synchronous behavior](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js_sync)
* [Structuring JavaScript code with asynchronous behavior](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js_async)
* [Structuring JavaScript code with synchronous and asynchronous behavior](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js_both)
* [Example: Calling an external API with JavaScript](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js_api)
* [Packaging JavaScript code with the webpack module](https://cloud.ibm.com/docs/openwhisk?topic=openwhisk-prepprep_js_pkg)
Creating Function workloads with repository source code 

You can create your function directly from source code that is located in a Git repository with the IBM Cloud® Code Engine CLI.

Use the function create command to both build a code bundle from your Git repository source, and create your function to reference this built code bundle.

Before you begin



* Set up your [Code Engine CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-install-cli) environment.
* [Create and work with a project](https://cloud.ibm.com/docs/codeengine?topic=codeengine-manage-project).



In this scenario, Code Engine builds a code bundle from your Git repository source, automatically uploads the code bundle to your container registry, and then creates your function to reference this built code bundle. You need to provide only a name for the function, the URL to the Git repository, and the runtime for the Function. In this case, Code Engine manages the namespace for you. However, if you want to use a different container registry, then you must specify the code bundle and a registry secret for that container registry. For a complete listing of options, see the [ibmcloud ce function create](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-function-create) command.

For information about required permissions for accessing image registries, see [Setting up authorities for image registries](https://cloud.ibm.com/docs/codeengine?topic=codeengine-add-registryauthorities-registry).

The following example function create command creates the myfun function, which references code that is located in https://github.com/IBM/CodeEngine. This command automatically builds the code bundle and uploads it to an IBM Cloud® Container Registry namespace in your account. The function references this built code bundle. By specifying the --build-context-dir option, the build uses the source in the helloworld-samples/function-nodejs directory.
* (SOURCE) The name of the source bucket and key name of the source object, which is separated by a slash (/). Must be URL-encoded.



* Flag: --copy-source SOURCE



* Optional: Copies the object if its entity tag (Etag) matches the specified tag (ETAG).



* Flag: --copy-source-if-match ETAG



* Optional: Copies the object if it has been modified since the specified time (TIMESTAMP).



* Flag: --copy-source-if-modified-since TIMESTAMP



* Optional: Copies the object if its entity tag (ETag) is different than the specified tag (ETAG).



* Flag: --copy-source-if-none-match ETAG



* Optional: Copies the object if it hasn't been modified since the specified time (TIMESTAMP).



* Flag: --copy-source-if-unmodified-since TIMESTAMP



* Optional: The range of bytes to copy from the source object. The range value must use the form bytes=first-last, where the first and last are the zero-based byte offsets to copy. For example, bytes=0-9 indicates that you want to copy the first ten bytes of the source. You can copy a range only if the source object is greater than 5 MB.



* Flag: --copy-source-range value



* Optional: The REGION where the bucket is present. If this flag is not provided, the program uses the default option that is specified in config.



* Flag: --region REGION



* Optional: Output FORMAT can be only json or text.



* Flag: --output FORMAT













 Next Steps 

As every procedure always goes exactly as planned, you might not have seen any of the [common header and error codes](https://cloud.ibm.com/docs/services/cloud-object-storage?topic=cloud-object-storage-compatibility-common). For more reference, check the [API reference](https://cloud.ibm.com/apidocs/cos/cos-compatibility).
[Apache Kafka client](http://kafka.apache.org/downloads) Java Latest 0.10.2 [Java console sample](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-kafka_java_using)<br><br>[Liberty sample](https://github.com/ibm-messaging/event-streams-samples/tree/master/kafka-java-liberty-sample) 
 Third-party clients: 
 [node-rdkafka](https://github.com/Blizzard/node-rdkafka) Node.js Latest 2.2.2 [Node.js sample](https://github.com/ibm-messaging/event-streams-samples/tree/master/kafka-nodejs-console-sample) 
 [confluent-kafka-python](https://github.com/confluentinc/confluent-kafka-python) Python Latest 0.11.0 [Kafka Python sample](https://github.com/ibm-messaging/event-streams-samples/tree/master/kafka-python-console-sample) 
 [confluent-kafka-go](https://github.com/confluentinc/confluent-kafka-go) Golang Latest 0.11.0 
 [librdkafka](https://github.com/edenhill/librdkafka) C or C++ Latest 0.11.0 







 Connecting your client to Event Streams 

For information about how to configure your Java client to connect to Event Streams, see [Configuring your client](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-kafka_usingkafka_api_client).







 Configuring your Kafka API client
: See [CLI version history](https://cloud.ibm.com/docs/codeengine?topic=codeengine-cli_versions).

Code Engine batch CRDs updated to v1beta1.
: See [Batch CRDs](https://cloud.ibm.com/docs/codeengine?topic=codeengine-apiapi-crd-batch).







 August 2020 

Review the release notes for August 2020.



 21 August 2020 

Review the release notes for 21 August 2020.

CLI version 0.3.1802 released.
: See [CLI version history](https://cloud.ibm.com/docs/codeengine?topic=codeengine-cli_versions).

Updated example
: Updated ibmcom/hello sample source in [Tutorial Deploying applications](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-app-tutorial).

Project quotas
: Added project quotas to [Limits and quotas](https://cloud.ibm.com/docs/codeengine?topic=codeengine-limitsproject_quotas) topic.





 17 August 2020 

Review the release notes for 17 August 2020.

CLI version 0.3.1712 released.
: See [CLI version history](https://cloud.ibm.com/docs/codeengine?topic=codeengine-cli_versions).

Ephemeral storage limit
: Added ephemeral storage limit to [apps](https://cloud.ibm.com/docs/codeengine?topic=codeengine-limitslimits_application) and [jobs](https://cloud.ibm.com/docs/codeengine?topic=codeengine-limitslimits_job).





 14 August 2020 

Review the release notes for 14 August 2020.

CLI version 0.3.1675 released.
: See [CLI version history](https://cloud.ibm.com/docs/codeengine?topic=codeengine-cli_versions).





 4 August 2020