vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Working with jobs and job runs 

Learn how to run jobs in IBM Cloud® Code Engine. A job runs one or more instances of your executable code. Unlike applications, which handle HTTP requests, jobs are designed to run one time and exit. When you create a job, you can specify workload configuration information that is used each time that the job is run.

Before you begin



* If you want to use the Code Engine console, go to [Code Engine overview](https://cloud.ibm.com/codeengine/overview).
* If you want to use the CLI, [set up your Code Engine CLI environment](https://cloud.ibm.com/docs/codeengine?topic=codeengine-install-cli).
* Plan and choose your approach for making your code run as a Code Engine job component.



Code Engine provides custom resource definition (CRD) methods. For more information, see [Code Engine API reference - Batch CRD methods](https://cloud.ibm.com/docs/codeengine?topic=codeengine-apiapi-crd-batch).



 How do I make my code run as a Code Engine job component? 

Whether your code exists as source in a local file or in a Git repository, or your code is a container image that exists in a public or private registry, Code Engine provides a streamlined way for you to run your code as a job.



* If you have a container image, per the [Open Container Initiative (OCI) standard](https://opencontainers.org/), then you need to provide only a reference to the image, which points to the location of your container registry when you create your job. You can create your job from images in a [public registry](https://cloud.ibm.com/docs/codeengine?topic=codeengine-create-job) or [private registry](https://cloud.ibm.com/docs/codeengine?topic=codeengine-create-job-private) and then access the referenced image from your job run.
If your registry is private, you must [set up access](https://cloud.ibm.com/docs/codeengine?topic=codeengine-add-registry) to it.
11. Provide registry information about where to store the image of your build output. Select an existing Registry access secret or create a new one. If you are building your image to a Container Registry instance that is in your account, you can select Code Engine managed secret and let Code Engine create and manage the secret for you.
12. Select a namespace, name, and a tag for your image. If you are building your image to an IBM Cloud Container Registry instance that is in your account, you can select an existing namespace or let Code Engine create and manage the namespace for you.
13. Click Done.
14. Modify any runtime settings or environment variables for your app. For more information about these options, see [Options for endpoint visibility of apps](https://cloud.ibm.com/docs/codeengine?topic=codeengine-application-workloadsoptionsdeploy) and [Options for deploying an app](https://cloud.ibm.com/docs/codeengine?topic=codeengine-application-workloadsoptionsdeploy).
15. Click Create.
16. After your build run is submitted, the built container image is sent to Container Registry and then your application pulls the image and deploys for you. After the application status changes to Ready, you can test the application. Click Test application and then click Send request in the Test application pane. To open the application in a web page, click Application URL.



Build runs that complete are ultimately automatically deleted. When your build run is based on build configuration, this build run is deleted after 3 hours if the build run is successful. If the build run is not successful, this build run is deleted after 48 hours.

Now that you have deployed your application, you can view information about application revisions and any running instances, and configuration details.





 Deploying your app from repository source code with the CLI 

You can deploy your application directly from repository source code with the CLI.
* UI
* CLI







 Using Blueprints to deploy large-scale cloud environments 

Use one of the IBM® provided [samples](https://github.com/orgs/Cloud-Schematics/repositories/?q=topic:blueprint) to deploy a blueprint linking multiple module environments.

IBM Cloud Schematics Blueprints supports deploying and managing large-scale application environments using solution architecture definitions composed from building blocks of open source IaC Code. Building on open source Infrastructure as Code (IaC) automation, Blueprints scales infrastructure deployments by linking multiple environments to create large-scale application architectures.



 Deploy blueprint through UI 



 Creating a blueprint in Schematics 



1. Log in to [IBM Cloud](https://cloud.ibm.com/).
2. Click on Schematics in the left hand navigator pane, then click Blueprints
3. On the Blueprints dashboard page, click the Create Blueprint button



* In Blueprint Details section:



* Name<Provide unique name for your blueprint>.
* Location as North America or other [region](https://cloud.ibm.com/docs/schematics?topic=schematics-multi-region-deployment) for this blueprint.
* Resource group as Default or other resource group for this blueprint. For more information, see [Creating a resource group](https://cloud.ibm.com/docs/account?topic=account-rgs). Ensure you have right access permission for the resource group.
* Tags as <Provide the tag name for your blueprint>.
* Description for the blueprint. Supports maximum character range from 0 - 2048.
* Click Next.



* In Blueprint URL section:



* Repository URL - <Provide your valid GitHub or GitLab repository URL that hosts your blueprint configuration file>. For example, https://github.com/Cloud-Schematics/blueprint-basic-example/blob/main/basic-blueprint.yaml.
You can deploy your app with an image in a [public registry](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-app) or [private registry](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-app-private).

If you created your app by using the app create command and you specified the --build-source option to build the container image from local or repository source, and you want to change your app to point to a different container image, you must first remove the association of the build from your app. For example, run ibmcloud ce application update -n APP_NAME --build-clear. After you remove the association of the build from your app, you can update the app to reference a different image.
* If you are starting with source code that resides in a Git repository, you can choose to let Code Engine take care of building the image from your source and deploying the app with a single operation. In this scenario, Code Engine uploads your image to IBM Cloud® Container Registry. To learn more, see [Deploying your app from repository source code](https://cloud.ibm.com/docs/codeengine?topic=codeengine-app-source-code). If you want more control over the build of your image, then you can choose to [build the image](https://cloud.ibm.com/docs/codeengine?topic=codeengine-build-image) with Code Engine before you deploy your app.
* If you are starting with source code that resides on a local workstation, you can choose to let Code Engine take care of building the image from your source and deploying the app with a single CLI command. In this scenario, Code Engine uploads your image to IBM Cloud® Container Registry. To learn more, see [Deploying your app from local source code with the CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-app-local-source-code).
https://www.terraform.io/docs/configuration/providers.htmlprovider-source
}
}
required_version = ">= 0.13"
}
Show more
5. Edit the versions.tf configuration file. Comment out source = "your-registry.example.com/organization/ibm" parameter and provide source value as source = "IBM-Cloud/ibm" as shown in the example. For more information, about the provider registry, see [IBM Cloud provider registry](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest).

Example versions.tf

terraform {
required_providers {
ibm = {
 TF-UPGRADE-TODO

 No source detected for this provider. You must add a source address
 in the following format:

source = "IBM-Cloud/ibm"

 For more information, see the provider source documentation:

 https://www.terraform.io/docs/configuration/providers.htmlprovider-source
}
}
required_version = ">= 0.13"
}
Show more



This completes your Terraform version upgrade.





 Version control 

The versions that are associated with the resources and data sources are:



* IBM Cloud Provider plug-in for Terraform. For more information, see [Provider version releases](https://github.com/IBM-Cloud/terraform-provider-ibm/releases).
* Terraform. For more information, see [Terraform version](https://releases.hashicorp.com/terraform/).



With the release of Terraform version 0.12, the syntax for configuration files changed. If you want to run your infrastructure code by using Terraform version 0.12, you must first refer to refer to [Upgrading the Terraform template version](https://cloud.ibm.com/docs/schematics?topic=schematics-migrating-terraform-version).
Afterward, you can test your knowledge and [!take a quiz](https://ibm.biz/BdfFxR)



Table 1. Code Engine Terms

 Term Description 

 Project A project is a grouping of Code Engine entities such as applications, jobs, and builds. Projects are used to manage resources and provide access to its entities. 
 Application An application, or app, runs your code to serve HTTP requests. An application has a URL for incoming requests. The number of running instances of an application are automatically scaled up or down (to zero) based on incoming workload. 
 Build A build, or image build, is a mechanism that you can use to create a container image from your source code. Code Engine supports building from a Dockerfile or Cloud Native Buildpacks. 
 Function A Function is a stateless code snippet that performs tasks in response to an HTTP request. 
 Job A job runs one or more instances of your executable code. Unlike applications, which include an HTTP Server to handle incoming requests, jobs are designed to run one time and exit. 



For more information about terms, see [Code Engine terminology](https://cloud.ibm.com/docs/codeengine?topic=codeengine-aboutterminology).

Not sure what to choose? See [Planning for Code Engine](https://cloud.ibm.com/docs/codeengine?topic=codeengine-plan-codeengine).





 Deploying your first Code Engine app 

Create your first Code Engine app by using the icr.io/codeengine/helloworld image.



1. Open the [Code Engine](https://cloud.ibm.com/codeengine/overview) console.
2. Select Start creating.
3. Select a project from the list of available projects. You can also [create a new one](https://cloud.ibm.com/docs/codeengine?topic=codeengine-manage-projectcreate-a-project). Note that you must have a selected project to deploy an app.
4. Select Application.
5.
Include this file in the How to nav group of your toc file



 IBM Blockchain for Education User Guide 

Each student will receive access to an instance of the IBM Blockchain Platform.

The URL will be in the format: https://bda-2021-lsu-<000-032>-ibp-console-console.lsu-dev-0b8a4e8de101bcbaf4eafc441eda83b3-0000.us-south.containers.appdomain.cloud

They will also receive access to a VM on the IBM Cloud for Education available through the URL: [https://app.education.cloud.ibm.com](https://app.education.cloud.ibm.com)



 Blockchain for maintaining Digital Assets 

Code repository: [https://github.com/IBM/Blockchain-for-maintaining-Digital-Assets](https://github.com/IBM/Blockchain-for-maintaining-Digital-Assets)

In this code pattern, we will be building a digital asset management application by creating and deploying a smart contract on a Hyperledger Fabric Network created on IBM Blockchain Platform. We will then interact with this application via a user interface created using VueJS.

Digital Asset Management Systems ensure that operations are only performed on a digital asset by individuals (or organizations) that have the right access rights and permissions for the asset. The digital asset is defined as the content (an image, a music file, a document, a video file, etc.) and its metadata. The metadata could be as simple as the name of the asset, the name of the owner of the asset and the date of creation of the asset, or it could be something more complex, such as extracted speech from a video (subtitles). In any Digital Asset Management system, there can be any number of users and these users can have the ability to perform various actions on the asset in the system based on the permissions they have.

When you have completed this code pattern, you will understand how to:



* Package a blockchain smart contract using the IBM Blockchain Platform Extension for VS Code.
Updating a job 

You can make changes to a defined job and run the updated job from the console or with the CLI. Changes to your job might include updating the code container image, code arguments or commands, runtime instance resources, or environment variables.

Each time your job runs, the latest version of your referenced container image is used for the job run, unless a tag is specified for the image. If a tag is specified for the image, then the tagged image is used for the job run.

Submitted batch jobs are run in parallel, if possible. If the number or size of the submitted jobs exceeds the configured quota limits, such as maximum number of running instances, then Code Engine queues the jobs and delays running them until enough jobs finish. For more information about quotas and limits for jobs, including memory and CPU, see [Limits and quotas for Code Engine](https://cloud.ibm.com/docs/codeengine?topic=codeengine-limits).

You can [update your job](https://cloud.ibm.com/docs/codeengine?topic=codeengine-update-job) and its referenced code in any of the following ways, independent of how you created or previously updated your job.



* If you have a container image, per the [Open Container Initiative (OCI) standard](https://opencontainers.org/), then you need to provide only a reference to the image, which points to the location of your container registry when you create (or update) your job. You can create (or update) your job from images in a [public registry](https://cloud.ibm.com/docs/codeengine?topic=codeengine-create-job) or [private registry](https://cloud.ibm.com/docs/codeengine?topic=codeengine-create-job-private) and then access the referenced image from your job run.

If you created your job by using the job create command and you specified the --build-source option to build the container image from local or repository source, and you want to change your job to point to a different container image, you must first remove the association of the build from your job.
For example, if the URL is https://cloud.ibm.com/devops/toolchains/aaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee?env_id=ibm:yp:us-south then the toolchain's ID is aaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee.

Be sure to include the ID only, not the full URL. In the existing toolchain, you can update doi-toolchain-id with the CI toolchain-id to link with the CI DevOps Insights instance.

You can also set a target environment for the DevOps Insights (DOI) interactions. This parameter is optional. If you provide this parameter, it is used instead of the target environment from the inventory.

Zoom

![DevSecOps DevOps Insights Toolchain](https://cloud.ibm.com/docs-content/v1/content/5eb88a2575e30e316dbb83829c7f4c8b1ddbc67c/devsecops/images/devsecops-cc-devops-insights.png)

Figure 11. DevSecOps DevOps Insights Toolchain





 SonarQube 

Configure SonarQube as the static code analysis tool for the toolchain. SonarQube provides an overview of the overall health and quality of your source code and highlights issues that are found in new code. The static code analyzers detect tricky bugs, such as null-pointer dereferences, logic errors, and resource leaks for multiple programming languages.

With Default Configuration, the pipeline provisions a new SonarQube instance in the Kubernetes cluster that is configured in the cc-static-scan step. A new instance is provisioned during each CC pipeline run.

If you want the toolchain to use an existing SonarQube Instance that you provisioned on another host, use the Custom Configuration option.

Zoom

!
You can create (or update) your job from images in a [public registry](https://cloud.ibm.com/docs/codeengine?topic=codeengine-create-job) or [private registry](https://cloud.ibm.com/docs/codeengine?topic=codeengine-create-job-private) and then access the referenced image from your job run.

If you created your job by using the job create command and you specified the --build-source option to build the container image from local or repository source, and you want to change your job to point to a different container image, you must first remove the association of the build from your job. For example, run ibmcloud ce job update -n JOB_NAME --build-clear. After you remove the association of the build from your job, you can update the job to reference a different image.
* If you are starting with source code that resides in a Git repository, you can choose to let Code Engine take care of building the image from your source and creating (or updating) the job with a single operation. In this scenario, Code Engine uploads your image to IBM Cloud® Container Registry. To learn more, see [Creating a job from repository source code](https://cloud.ibm.com/docs/codeengine?topic=codeengine-run-job-source-code). If you want more control over the build of your image, then you can choose to [build the image](https://cloud.ibm.com/docs/codeengine?topic=codeengine-build-image) with Code Engine before you create (or update) your job and run the job.
* If you are starting with source code that resides on a local workstation, you can choose to let Code Engine take care of building the image from your source and creating the job with a single CLI command. In this scenario, Code Engine uploads your image to IBM Cloud® Container Registry. To learn more, see [Creating your job from local source code with the CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-job-local-source-code).
Use the [ibmcloud ce buildrun submit](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-buildrun-submit) command to submit a new build run. For the buildrun submit command, you must specify the --build option to provide the name of your build configuration. You can optionally specify the --name option to provide the name for this build run. If you specify the --name option, make sure that you use a different build run name from the failed build run, or ensure that you delete the failed build run by using the [ibmcloud ce buildrun delete](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-buildrun-delete) command. For example,

ibmcloud ce buildrun submit --build <BUILD_NAME> --name <BUILDRUN_NAME>







 Resolution for a wrong protocol or missing secret during build 

The URL to a Git repository can be specified by using either the HTTPS or SSH protocol. GitHub and GitLab provide a way to toggle the URL format in the Git UI. The HTTPS protocol requires no authentication but can be used only if the repository is public. For private repositories, you must use the SSH protocol and provide a secret for the repository. Repositories in a GitHub Enterprise setup can be public but still require authentication, and those GitHub Enterprise repositories can also be used only by using SSH protocol.



 For public repositories 

If the failure happened for a public repository, then update the existing build to use the HTTPS URL of the Git repository, and run the build.



1. Use the [ibmcloud ce build update](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-build-update) command to update the build configuration to use the HTTPS URL of the Git repository; for example,

ibmcloud ce build update --name <BUILD_NAME> --source <GIT_REPO>
2.
After you obtain an IBM Cloud API key, use that key to generate a token that you can use to invoke the APIs provided by API Connect.



 Using the API key to generate a token 

Call the API Connect token service with your IBM Cloud API key.

Format the request as shown in the following example.

POST https://login.service.<region>.apiconnect.ibmcloud.com/apikey?apikey=<api_key>

The API responds with a JSON payload. Use the value of the jwt property as your token.





 Invoking APIs with the token 

When you invoke an API provided by API Connect, use the token as the authorization header; for example:

curl https://apimanager.us-south.apiconnect.cloud.ibm.com/v1/me -H ‘Authorization: Bearer ’







 API key best practices 

API keys are used as the method of authentication to your service, so they must be kept secure. The following best practices help you maintain a secure API key and reduce the chance of publicly exposing credentials that compromise your application.



* Use an API key with the role that is appropriate for the function that you are using. When you create an application, consider using an API key with the role Reader for all calls to GET API methods. The Reader role can't perform any destructive or additive functions to the service instance.
* Do not embed the API key directly in code. API keys that are embedded in code might be exposed to your users. Instead of embedding the API keys in your application code, consider storing them either in environment variables or in files outside of your source code control system.
* Do not store an API key in files inside your application's source code control system. If you store API keys in files, keep the files outside of your application's source code. This practice is important if you use a public source-code management system such as GitHub.
* Regenerate your API keys periodically. You can create new credentials at any time.



1. From the resource list, select a service instance.
2. In the navigation list, click Service credentials.
Now that your app is created and deployed from repository source code, you can update the app to meet your needs by using the [ibmcloud ce app update](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-application-update) command. For more information about updating apps, see [Updating your app](https://cloud.ibm.com/docs/codeengine?topic=codeengine-update-app). If you want to update your source to use with your app, you must provide the --build-source option on the application update command.

When your app is deployed from repository source code or from [local source](https://cloud.ibm.com/docs/codeengine?topic=codeengine-app-local-source-code) with the CLI, the resulting build run is not based on a build configuration. Build runs that complete are ultimately automatically deleted. Build runs that are not based on a build configuration are deleted after 1 hour if the build run is successful. If the build run is not successful, this build run is deleted after 24 hours. You can only display information about this build run with the CLI. You cannot view this build run in the console.





 Next steps 



* After your app deploys, [access your app](https://cloud.ibm.com/docs/codeengine?topic=codeengine-access-service) through a URL.
* You can create a [custom domain mapping](https://cloud.ibm.com/docs/codeengine?topic=codeengine-domain-mappings) and assign it to your app. For more information about deploying apps across multiple regions with a custom domain name, see [Configuring a highly available application](https://cloud.ibm.com/docs/codeengine?topic=codeengine-deploy-multiple-regions).
* Now that your app is deployed, consider making your apps event-driven.