vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Environment properties and secrets (defined on the pipeline UI) are accessible through pipelinectl:

 if you set my_config_value on the UI, it's accessible via get_env
my_config_value=$(get_env my_config_value)

You can set other variables by using set_env:

set_env my_config_value "some value"

 later on, even in another stage
my_config_value=$(get_env my_config_value)

These values are not persisted, they're scoped for a single pipeline run.



 Example 

You can create an Artifactory auth credential for your own build by adding the required parameters to the pipeline UI:

my-artifactory-email
my-artifactory-user
my-artifactory-password
my-artifactory-url

When these parameters are available in the pipeline, you can access them in the user scripts and create a Docker auth JSON file. For example:

kubectl create secret docker-registry regcred --o json --dry-run=client --docker-server="$(get_env my-artifactory-url)" --docker-username="$(get_env my-artifactory-user)" --docker-email="$(get_env my-artifactory-email)" --docker-password="$(get_env my-artifactory-password)" | jq -r '.data[".dockerconfigjson"]'

{"auths":{"us.icr.io":{"username":"user@ibm.com","password":"secret-password","email":"user@ibm.com","auth":"dXNlckBpYm0uY29tOnNlY3JldC1wYXNzd29yZA=="}}}







 Parameters for custom scripts 

In addition to the pipeline environment variables and secrets, these parameter values are provided for user script stages.
Secrets Manager CLI 

You can use the IBM Cloud® Secrets Manager command-line interface (CLI) to manage secrets in your Secrets Manager instance.

Current version: 2.0.1



 Prerequisites 



* Install the [IBM Cloud CLI](https://cloud.ibm.com/docs/cli?topic=cli-getting-started).
* Install the Secrets Manager CLI by running the following command:

ibmcloud plugin install secrets-manager

You're notified on the command line when updates to the IBM Cloud CLI and plug-ins are available. Be sure to keep your CLI up to date so that you can use the latest commands. You can view the current version of all installed plug-ins by running ibmcloud plugin list.
* Export an environment variable with your Secrets Manager service endpoint URL.

To target an instance for operations, export the following variable.

export SECRETS_MANAGER_URL=https://{instance_ID}.{region}.secrets-manager.appdomain.cloud

Replace {instance_ID} and {region} with the values that apply to your Secrets Manager service instance. To find the endpoint URL that is specific to your instance, you can copy it from the Endpoints page in the Secrets Manager UI. For more information, see [Viewing your endpoint URLs](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-endpointsview-endpoint-urls)

Need to override the URL? You can also specify the endpoint at the command-level by using the --service-url INSTANCE flag. For example, ibmcloud secrets-manager secrets --service-url https://{instance_ID}.{region}.secrets-manager.appdomain.cloud.







 Globals 



 Options 

--instance-id (string)
: The Secrets Manager Instance ID assigned by the service provider.

--region (string)
: The region where you provisioned your Secrets Manager Instance.
Managing TLS and non-TLS certificates and secrets 

Learn how you can use certificates and secrets in your cluster.

Consider using [Secrets Manager](https://cloud.ibm.com/docs/containers?topic=containers-secrets-mgr) to centrally manage and automatically update your secrets.



 Managing TLS certificates and secrets with Ingress 

Your Ingress TLS certificate is stored as a Kubernetes secret. To manage the TLS secrets in your cluster, you can use the ibmcloud ks ingress secret set of commands.

For example, you can import a certificate from Secrets Manager to a Kubernetes secret in your cluster by running the following command.

ibmcloud ks ingress secret create --cluster <cluster_name_or_ID> --cert-crn <crn> --name <secret_name> --namespace <namespace>

To import the certificate with the ibmcloud ks ingress secret create command, you must have a default [Secrets Manager](https://cloud.ibm.com/docs/containers?topic=containers-secrets-mgr) instance registered to your cluster. If you do not have a Secrets Manager instance and your secrets are instead written directly to your cluster, your secrets do not have the required CRN value and you must manually copy them with kubectl commands.

To view all Ingress secrets for TLS certificates in your cluster, run the following command.

ibmcloud ks ingress secret ls -c <cluster>



 Setting up TLS secrets for the IBM-provided Ingress subdomain 

IBM provides an Ingress subdomain and a default TLS certificate, stored as a Kubernetes secret in your cluster, that you can specify in your Ingress resource. IBM-provided TLS certificates are signed by LetsEncrypt and are fully managed by IBM.

The IBM-provided Ingress subdomain wildcard, .<cluster_name>.<globally_unique_account_HASH>-0000.<region>.containers.appdomain.cloud, is registered by default for your cluster. The IBM-provided TLS certificate is a wildcard certificate and can be used for the wildcard subdomain.
Managing TLS and non-TLS certificates and secrets 

Learn how you can use certificates and secrets in your cluster.

Consider using [Secrets Manager](https://cloud.ibm.com/docs/openshift?topic=openshift-secrets-mgr) to centrally manage and automatically update your secrets.



 Managing TLS certificates and secrets with Ingress 

Your Ingress TLS certificate is stored as a Kubernetes secret. To manage the TLS secrets in your cluster, you can use the ibmcloud oc ingress secret set of commands.

For example, you can import a certificate from Secrets Manager to a Kubernetes secret in your cluster by running the following command.

ibmcloud oc ingress secret create --cluster <cluster_name_or_ID> --cert-crn <crn> --name <secret_name> --namespace <namespace>

To import the certificate with the ibmcloud oc ingress secret create command, you must have a default [Secrets Manager](https://cloud.ibm.com/docs/openshift?topic=openshift-secrets-mgr) instance registered to your cluster. If you do not have a Secrets Manager instance and your secrets are instead written directly to your cluster, your secrets do not have the required CRN value and you must manually copy them with oc commands.

To view all Ingress secrets for TLS certificates in your cluster, run the following command.

ibmcloud oc ingress secret ls -c <cluster>



 Setting up TLS secrets for the IBM-provided Ingress subdomain 

IBM provides an Ingress subdomain and a default TLS certificate, stored as a Kubernetes secret in your cluster, that you can specify in your Ingress resource. IBM-provided TLS certificates are signed by LetsEncrypt and are fully managed by IBM.

The IBM-provided Ingress subdomain wildcard, .<cluster_name>.<globally_unique_account_HASH>-0000.<region>.containers.appdomain.cloud, is registered by default for your cluster. The IBM-provided TLS certificate is a wildcard certificate and can be used for the wildcard subdomain.
If you do not have a Secrets Manager instance and your secrets are instead written directly to your cluster, your secrets do not have the required CRN value and you must manually copy them with oc commands.

ibmcloud oc ingress secret create --name <secret_name> --cluster <cluster_name_or_ID> --cert-crn <certificate_crn> --namespace <namespace>
2. Repeat the previous step for each namespace where your apps exist.









 Managing non-TLS secrets 

To manage non-TLS secrets, you can use the ibmcloud oc ingress secret commands.

There are 4 types of non-TLS secrets:



* Arbitrary secrets hold one string value.
* IAM credentials hold an IAM API key.
* Username and password secrets hold a username and password as two separate values.
* Key values hold JSON values.



Learn how you can centrally manage your non-TLS secrets with [IBM Cloud Secrets Manager](https://cloud.ibm.com/docs/openshift?topic=openshift-secrets-mgr). With Secrets Manager, you can create managed Kubernetes secrets, update your secrets automatically, create secret groups that control who has access to the secrets in your cluster, and more.



 Creating a non-TLS secret in your cluster 

Create a non-TLS secret by specifying the --type Opaque option in the ibmcloud oc ingress secret create command. With the Opaque type, you can include multiple non-certificate CRN values. If the --type option is not specified, TLS is applied by default. For more information and additional command options, see the [CLI reference](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_ingress_secret_create).

The following example command creates a non-TLS secret with the Opaque type specified. Non-TLS secrets require at least one secret [field](https://cloud.ibm.com/docs/openshift?topic=openshift-secretsnon-tls-field-add).
To display the details of this secret,

ibmcloud ce secret get --name mysecret-tls

Example output

Getting secret 'mysecret-tls'...
OK

Name: mysecret-tls'
ID: abcdefgh-abcd-abcd-abcd-1a2b3c4d5e6f
Format: tls
Project Name: myproject
Project ID: 01234567-abcd-abcd-abcd-abcdabcd1111
Age: 66s
Created: 2023-03-07 16:14:46 +0000 UTC

Data:
---
tls.crt: REDACTED
tls.key: REDACTED





 Listing secrets with the CLI 

After secrets are created, use the [secret list](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-secret-list) command to list all secrets in your project. For example,

ibmcloud ce secret list

Example output

Listing secrets...
OK

Name Format Data Age
ce-auto-icr-private-us-south registry 4 333d
ce-auto-private-icr-us-south registry 4 335d
myregistry-seccmd registry 4 3h31m
mysecret-basicauth basic_auth 2 7m37s
mysecret-generic generic 1 7m7s
mysecret-genericfromfile generic 2 2m29s
mysecret-registry registry 4 111s
mysecret-ssh ssh_auth 2 42m
mysecret-tls tls 2 3h47m









 Updating secrets 

You can change key-value pairs for existing secrets.



 Updating secrets from the console 

You can update key-value pairs for your defined secrets from the console.



1. You can update key-value pairs for your defined secrets from the console in one of the following ways.
* UI
* CLI







 Creating a Secrets Manager service instance 

Get started with IBM Cloud® Secrets Manager by creating a service instance in IBM Cloud console, or with the IBM Cloud CLI.

Provisioning Secrets Manager in your IBM Cloud account can take 5 - 15 minutes to complete as the service creates a single tenant, dedicated instance.



 Creating a Secrets Manager instance in the UI 

To create an instance of Secrets Manager from the IBM Cloud console, complete the following steps.



1. In the console, go to the [Secrets Manager offering details page](https://cloud.ibm.com/catalog/services/secrets-manager).
2. In the Create tab, select the region that represents the geographic area where you want provision your instance.
3. Review and select a pricing plan.

You can create only one Trial instance of Secrets Manager per account. Before you can create a new Trial instance, you must delete the existing Trial instance and its reclamation.
4. Provide a name for your instance.
5. Select a resource group.
6. Optional: Add tags to help you to organize the instance in your account.
7. Determine an option for enabling customer-managed encryption for your instance.

You can enhance the security of your secrets at rest by integrating with a key management service. For more information about customer-managed encryption, check out [Protecting your sensitive data in Secrets Manager](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-mng-datadata-encryption).
8. Determine an option for connecting to Secrets Manager.

To further protect your connection to Secrets Manager, you can choose to provision an instance that uses a private service endpoint. For more information about setting up your account to support the private connectivity option, see [Enabling VRF and service endpoints](https://cloud.ibm.com/docs/account?topic=account-vrf-service-endpoint).
9. Click Create to create an instance of Secrets Manager in the account, region, and resource group that you selected.
Getting secret 'mysecret-tls'...
OK

Name: mysecret-tls'
ID: abcdefgh-abcd-abcd-abcd-1a2b3c4d5e6f
Format: tls
Project Name: myproject
Project ID: 01234567-abcd-abcd-abcd-abcdabcd1111
Age: 66s
Created: 2023-03-07 16:14:46 +0000 UTC

Data:
---
tls.crt: REDACTED
tls.key: REDACTED





 Listing secrets with the CLI 

After secrets are created, use the [secret list](https://cloud.ibm.com/docs/codeengine?topic=codeengine-clicli-secret-list) command to list all secrets in your project. For example,

ibmcloud ce secret list

Example output

Listing secrets...
OK

Name Format Data Age
ce-auto-icr-private-us-south registry 4 333d
ce-auto-private-icr-us-south registry 4 335d
myregistry-seccmd registry 4 3h31m
mysecret-basicauth basic_auth 2 7m37s
mysecret-generic generic 1 7m7s
mysecret-genericfromfile generic 2 2m29s
mysecret-registry registry 4 111s
mysecret-ssh ssh_auth 2 42m
mysecret-tls tls 2 3h47m









 Updating secrets 

You can change key-value pairs for existing secrets.



 Updating secrets from the console 

You can update key-value pairs for your defined secrets from the console.



1. You can update key-value pairs for your defined secrets from the console in one of the following ways.



* Go to the Secrets and configmaps page for your project and locate the secret that you want to update. Click the name of the secret that you want to update to open it.
* UI
* CLI
* API
* Terraform







 Importing SSL/TLS certificates 

You can use IBM Cloud® Secrets Manager to import SSL/TLS certificates that you can use for your apps or services.

An SSL/TLS certificate is a type of digital certificate that is used to establish communication privacy between a server and a client. Certificates are issued by

certificate authorities (CA)and contain information that is used to create trusted and secure connections between endpoints. After you add a certificate to your Secrets Manager instance, you can use it to secure network communications for your cloud or on-premises deployments. Your certificate is stored securely in your dedicated Secrets Manager service instance, where you can centrally manage its lifecycle.

In Secrets Manager, certificates that you import to the service are imported certificates (imported_cert). Certificates that you order through Secrets Manager from a third-party certificate authority are [public certificates](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-public-certificatesorder-public-certificates) (public_cert). Certificates that you create by using a private certificate authority are [private certificates](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-private-certificates) (private_cert).

To learn more about the types of secrets that you can manage in Secrets Manager, see [What is a secret?](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-what-is-secret).



 Before you begin 

Before you get started, be sure that you have the required level of access. To create or add secrets, you need the [Writer service role or higher](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-iam).

Before you import a certificate, be sure that you:



* Create an X.509 compliant certificate with a matching private key (optional).
* Convert your files into Privacy-enhanced electronic mail (PEM) format.
* Keep the private key unencrypted to ensure that it can be imported into Secrets Manager.
The following example applies an Arbitrary secret type that is stored in Secrets Manager to an IBM Cloud API key that is required by the [Delivery Pipeline tool integration](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-deliverypipeline). You can follow the same steps to apply secrets to any of the Continuous Delivery tool integrations that require secret values.



1. Click the key icon to retrieve secrets from secure stores such as Secrets Manager for the IBM Cloud API key.
2. In the Provider field, specify the provider and the name of the Secrets Manager tool integration that you use to manage your toolchain secrets. For example, to use the Secrets Manager tool integration, select Secrets Manager: ibm-secrets-manager-1. You can use other providers to manage your toolchain secrets, such as [HashiCorp Vault](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-hashicorpvault) and [Key Protect](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-keyprotect).
3. Select a secret group and a secret name and click OK to apply the stored secret to the field that is associated with it.

Zoom

![Secret Reference to a Vault](https://cloud.ibm.com/docs-content/v1/content/562806c0b18119f4d21cb66f5ae6d051634b4a16/ContinuousDelivery/images/secret-pill.png)

Figure 1. Secret reference to a vault

The name of the secret that you select appears in capsule form. You cannot edit the secret name inline, but you can click ![remove icon](https://cloud.ibm.com/docs-content/v1/content/562806c0b18119f4d21cb66f5ae6d051634b4a16/ContinuousDelivery/images/secret-pill-delete-16.png) to delete the name.
Regions and endpoints 

Review region and connectivity options for interacting with IBM Cloud® Secrets Manager.



 Supported regions 

You can create Secrets Manager resources in one of the supported IBM Cloud regions, which represents the geographic area where your Secrets Manager requests are handled and processed.



* Dallas (us-south)
* Frankfurt (eu-de)
* London (eu-gb)
* Osaka (jp-osa
* Sao Paulo (br-sao)
* Sydney (au-syd)
* Tokyo (jp-tok)
* Toronto (ca-tor)
* Washington DC (us-east)







 Service endpoints 

You can use the Secrets Manager APIs to manage your secrets programmatically. Secrets Manager offers two connectivity options for interacting with its service APIs.

Public endpoints
: By default, you can connect to resources in your account over the IBM Cloud public network. Your data is encrypted in transit by using the Transport Security Layer (TLS) 1.2 protocol.

Private endpoints
: To further secure your connection, you can also enable [virtual routing and forwarding (VRF) and service endpoints](https://cloud.ibm.com/docs/account?topic=account-vrf-service-endpoint) for your infrastructure account. When you enable VRF for your account, you can [connect to Secrets Manager by using a private IP](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-service-connection) that is accessible only through the IBM Cloud private network.



 Public endpoints 

If you are managing your Secrets Manager resources over a public network, see the following table to determine the API endpoints to use when you connect to the Secrets Manager API.



Table 1. Public endpoints for interacting with Secrets Manager by using the native Vault APIs

 Region Endpoint URL 

 Dallas https://{instance_ID}.us-south.secrets-manager.appdomain.cloud 
 Frankfurt https://{instance_ID}.eu-de.secrets-manager.appdomain.cloud 
 London https://{instance_ID}.eu-gb.secrets-manager.appdomain.cloud
: As of April 17, 2023, the IBM Cloud® Secrets Manager API v1 has been deprecated in favor of v2. If you're still actively working with the Secrets Manager API v1, please be sure to start your upgrade as soon as possible. On 31 October 2023, support for the Secrets Manager API v1 will be removed.





 3 March 2023 

Now available: Terraform support
: Secrets Manager now supports Terraform on IBM Cloud. For more information, see [Setting up Terraform for Secrets Manager](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-terraform-setup) and the [Terraform registry](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/data-sources/secrets_manager_secret).





 11 December 2022 

Support for context-based restrictions (CBR)
: Manage user and service access to your Secrets Manager resources by using context-based restrictions, based on defined criteria. For more information, see [Managing access with context-based restrictions](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-access-control-cbr).

As part of the support for CBR, the private service inner IPs were modified. The hostnames were not modified. If needed, you can use DNS lookup to find the exact IPs.





 14 November 2022 

Manually configure your own DNS provider
: If you work with a DNS provider that is not currently integrated with the service, you can manually configure it when you use the Secrets Manager UI to order new certificates. If you choose to manually configure a provider, you must complete a challenge to prove ownership over your domain. For more information, see [Ordering public certificates with your own DNS provider in the UI](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-public-certificatesorder-public-cert-manual-ui).

Create a leaf certificate with a certificate signing request
* UI
* API
* Terraform







 Organizing your secrets 

When you work with IBM Cloud® Secrets Manager, you can create groups to organize your secrets and control who on your team has access to them. Then, if you don't need them anymore, you can delete the groups.

Similar to the way that

resource groupshelp to ensure correct policy enforcement at the platform level, you can create secret groups at the instance level to organize secrets.

Zoom

![The image shows two examples of a secret group and how they're mapped to access groups. One where the reader role is assigned and one where the manager role is assigned. The content is explained fully in the surrounding text.](https://cloud.ibm.com/docs-content/v1/content/991f60ddbe3ab0a9aa69481a07a0a73b359fa329/secrets-manager//images/secret-group.svg)

Figure 1. Assigning access to secret groups

As shown in the previous image, users with Reader access to a secret group can see that the group exists and understand which secrets are assigned to it. Users with Writer access can view and edit the secret group and secrets themselves. By design, the default secret group inherits all of the same permissions that are set for the instance.

You can choose to group your secrets by phase of development, specific to the type of roles that people on your team have, or in any way that might help you. Each secret can be mapped to one group only and the mapping occurs at the time of secret creation.

To learn about the suggested guidelines for using secret groups, check out [Best practices for organizing secrets and assigning access](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-best-practices-organize-secrets).



 Before you begin 

Before you begin, be sure that you have the required level of access. To create and manage secret groups, you need the [Manager service role](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-iam).





 Creating secret groups