vania-janet/multiturn-rag-retrieval-data huggingface dataset:
For more information about adding and removing vault integrations, including prerequisites and by using hints, see [Managing IBM Cloud secrets](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-manage-secrets-ibm-cloud).

By using hints within a template, a toolchain is automatically populated with preconfigured secrets; you don't need to manually select secrets from vault integrations that are attached to the toolchain.

This tutorial uses the [IBM Secrets Manager](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-getting-started) as the secrets vault.

Zoom

![Code Engine secure app secrets options](https://cloud.ibm.com/docs-content/v1/content/efbf7b33e3aca029f69f9244bced9dd4d8acb848/apps/includes/ContinuousDelivery/includes/ContinuousDelivery/images/code_engine_setup_secrets.png)

Figure 3. Code Engine secure app secrets options

IBM Secrets Manager securely stores and applies secrets such as API keys, Image Signature, or HashiCorp credentials that are part of your toolchain.

Zoom

![Code Engine Secrets Manager options](https://cloud.ibm.com/docs-content/v1/content/efbf7b33e3aca029f69f9244bced9dd4d8acb848/apps/includes/ContinuousDelivery/includes/ContinuousDelivery/images/code_engine_secrets_manager.png)

Figure 4. Code Engine Secrets Manager options
2. Click Continue.
You can only create a rotation policy for root keys that are generated in Key Protect. If you imported the root key initially, you must provide a new base64-encoded key material to rotate the key. For more information, see [Rotating root keys on-demand](https://cloud.ibm.com/docs/key-protect?topic=key-protect-rotate-keys).

ibmcloud kp key policy-update rotation KEY_ID|KEY_ALIAS
[--enable | --disable] [-m MONTHLY_INTERVAL] [-o FORMAT]
[--key-ring KEY_RING_ID]



 Examples 

Rotation policies can be applied at key creation time or after a key has been created. In the first example, we simply create the key without a rotation policy:

 create a root key

ibmcloud kp key create rotateKeyEnableDisableDemo

Creating key: 'rotateKeyEnableDisableDemo', in instance: '5ecfe306-d4at-48c0-ab07-b5c5bc751534'...
OK
Key ID Key Name
4d5540bd-3235-4a46-a8af-974f7ed6558a rotateKeyEnableDisableDemo
---------------------------------------------------------------------------------------------------------------------------------------------------

 set a rotation policy of three months

ibmcloud kp key policy-update rotation 4d5540bd-3235-4a46-a8af-974f7ed6558a -m 3 -e --output json

{
In this scenario, let's use the previously created mydatabasesec and mydatabasesec-writer secrets and reference keys within the secrets from a job.



1. Set an environment variable on a writerpic job to reference specific keys in the mydatabasesec and mydatabasesec-writer secrets. The mydatabasesec secret contains the username=reader, password=abcd, and certificate=cert.pem keys. The mydatabasesec-writer secret contains the username=writer and password=wxyz keys.

ibmcloud ce job create -n writerpick -i busybox -c env --env-from-secret mydatabasesec-writer:username --env-from-secret mydatabasesec-writer:password --env-from-secret mydatabasesec
2. (Optional) View the details of the writerpick job. The output displays the full reference to the mydatabasesec secret and the key reference to the password and username keys of the mydatabasesec-writer secret.

ibmcloud ce job get -n writerpick

Example output

Getting job 'writerpick'...
OK

Name: writerpick
ID: abcdefgh-abcd-abcd-abcd-1a2b3c4d5e6f
Project Name: myproject
Project ID: 01234567-abcd-abcd-abcd-abcdabcd1111
Age: 107s
Created: 2021-02-12T07:16:46-06:00
[...]

Commands:
env
Environment Variables:
Type Name Value
Secret full reference mydatabasesec
Secret key reference password mydatabasesec-writer.password
Secret key reference username mydatabasesec-writer.username
Image: busybox
Resource Allocation:
CPU: 1
Memory: 4G

Runtime:
Array Indices: 0
Max Execution Time: 7200
Retry Limit: 3
For example, you can create and upload a file that is named sample.txt with the following sample text.

A quick brown fox jumped over the lazy dog.







 Prepare your Secrets Manager instance 

Finally, configure your Secrets Manager instance to start working with dynamic secrets.



1. From the command line, verify that you can access the Secrets Manager CLI plug-in.

ibmcloud secrets-manager --help

Don't have the plug-in? To install the Secrets Manager CLI plug-in, run ibmcloud plugin install secrets-manager.
2. Export an environment variable with your unique Secrets Manager API endpoint URL.

export SECRETS_MANAGER_URL="https://$SM_INSTANCE_ID.us-south.secrets-manager.appdomain.cloud"
3. Create a secret group for your instance.

[Secret groups](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-secret-groups) are a way to organize and control who on your team has access to specific secrets in your instance. To create a secret group from the IBM Cloud CLI, run the [ibmcloud secrets-manager secret-group-create](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-cli-plugin-secrets-manager-clisecrets-manager-cli-secret-group-create-command) command.

export SECRET_GROUP_ID=ibmcloud secrets-manager secret-group-create --resources '[{"name":"cloud-object-storage-writers","description":"Read and write to Cloud Object storage buckets."}]' --output json | jq -r ".resources[].id"; echo $SECRET_GROUP_ID

Using a Windows™ command prompt (cmd.exe) or PowerShell? If you encounter errors with passing JSON content on the command line, you might need to adjust the strings for quotation-escaping requirements that are specific to your operating system.
: If you have Writer or Manager access permissions, you can now use the Key Protect REST API to rotate an root key that was initially imported with an import token. To find out more, see [Using an import token to rotate a key](https://cloud.ibm.com/docs/key-protect?topic=key-protect-rotate-keysrotate-keys-secure-api). If you have Manager access permissions, you can now use the the Key Protect REST API to restore a previously imported root key. To find out more, see [Restoring keys](https://cloud.ibm.com/docs/key-protect?topic=key-protect-restore-keys). If you have Manager access permissions, you can now use the the Key Protect REST API to suspend or restore a keys encrypt and decrypt operations. To find out more, see [Disabling keys](https://cloud.ibm.com/docs/key-protect?topic=key-protect-disable-keys).







 April 2020 



 16 April 2020 

Announcing network access policies
: You can set a network access policy to allow API requests to a Key Protect instance from public or private networks. To find out more, see [Managing network access policies](https://cloud.ibm.com/docs/key-protect?topic=key-protect-managing-network-access-policies).







 March 2020 



 14 March 2020 

Announcing support for key metadata and key versions
: If you have Reader access permissions, you can now use the Key Protect REST API to view only details about a specific standard key without retrieving the key itself. To find out more, see [Viewing details about a key](https://cloud.ibm.com/docs/key-protect?topic=key-protect-retrieve-key-metadata). You can now audit the rotation history of a root key by viewing its key versions. After you rotate a root key, the ID of the root key does not change, but Key Protect now returns key version information to help you determine which version of the root key is protecting your data.
[Manage key rotation](https://cloud.ibm.com/docs-content/v1/content/217f108cc44e2ce15fb692d4b57b4c628464e908/hs-crypto//images/uko-key-rotation.svg)

Figure 1. Managed key rotation

Depending on the key type, managed keys in certain states are not eligible for key rotation. Before you rotate a managed key, make sure to check the following table. The Checkmark icon ![checkmark icon](https://cloud.ibm.com/docs-content/v1/content/217f108cc44e2ce15fb692d4b57b4c628464e908/icons/checkmark-icon.svg) indicates that the managed key in this state is eligible for rotation. For more information about the different key states, see [Monitoring the lifecycle of encryption keys in Unified Key Orchestrator](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-uko-key-states).



Table 2. Manage key rotation eligibility

 Key type Pre-active Active Deactivated Destroyed 

 AWS Key Management Service keys N/A ![checkmark icon](https://cloud.ibm.com/docs-content/v1/content/217f108cc44e2ce15fb692d4b57b4c628464e908/icons/checkmark-icon.svg) ![checkmark icon](https://cloud.ibm.com/docs-content/v1/content/217f108cc44e2ce15fb692d4b57b4c628464e908/icons/checkmark-icon.svg) N/A 
 Azure Key Vault keys N/A !
Integrations for Secrets Manager 

With IBM Cloud® Secrets Manager, you can save time with platform integrations that help you to dynamically create and retrieve secrets while you work with supported IBM Cloud services.



 Available integrations 

The following table lists the services that can be authorized to work with Secrets Manager.



Table 1. Available integrations

 Service Supports Description 

 [Application Load Balancer for VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-load-balancers) Certificates Centrally manage the SSL/TLS certificates that are required for load balancers to perform SSL offloading tasks. Create an authorization between VPC Infrastructure Services and Secrets Manager to give a load balancer access to your certificates. [Learn more about this integration](https://cloud.ibm.com/docs/vpc?topic=vpc-load-balancers). 
 [App Configuration](https://cloud.ibm.com/docs/app-configuration) All secret types A property value can be imported from Secrets Manager into the App Configuration service. [Learn more](https://cloud.ibm.com/docs/app-configuration?topic=app-configuration-ac-propertiesproperty-type-secret-reference). 
 [API Connect](https://cloud.ibm.com/docs/apiconnect?topic=apiconnect-getting-started-v5) Certificates Store your custom domain certificates in Secrets Manager, then use certificate CRNs to bind with custom domains in API Gateway. 
 [Catalog management](https://cloud.ibm.com/docs/account?topic=account-create-private-catalog) Arbitrary secrets Centrally manage the credentials for software in your private catalogs. [Learn more about this integration](https://cloud.ibm.com/docs/account?topic=account-create-private-catalog). 
 [Continuous Delivery](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-secretsmanager) Arbitrary secrets <br>IAM credentials Centrally manage the credentials for your Continuous Delivery toolchain.
* UI
* CLI
* API
* Terraform







 Creating IAM credentials 

You can use IBM Cloud® Secrets Manager to dynamically generate IAM credentials for accessing an IBM Cloud resource that requires IAM authentication.

IAM credentials are

dynamic secretsthat you can use to access an IBM Cloud resource. A set of IAM credentials consists of a service ID and an API key that is generated each time that the protected resource is read or accessed. You can define a time-to-live (TTL) or a lease duration for your IAM credential at its creation so that you shorten the amount of time that the secret exists.

To learn more about the types of secrets that you can manage in Secrets Manager, see [What is a secret?](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-what-is-secret)



 Before you begin 

Before you get started, be sure that you have the required level of access. To create or add secrets, you need the [Writer service role or higher](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-iam).

IAM credentials require an extra configuration step before you can start to create or manage them in the service. For more information, see [Configuring the IAM credentials engine](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-configure-iam-engine).

An account administrator (or any entity with the required level of access) can externally alter IAM Credentials that are created and managed by Secrets Manager. If such a service ID or API key is deleted outside of Secrets Manager, the service might behave unexpectedly. For example, you might be unable to create, or rotate credentials.





 Creating IAM credentials in the UI 

To create IAM credentials by using the Secrets Manager UI, complete the following steps.



1. In the console, click the Menu icon !
Why can't I read a locked IAM credentials secret? 

You try to read or access an IAM credentials secret that you manage in IBM Cloud® Secrets Manager, but you get a 412 Precondition Failed response.

  What’s happening 

You have an IAM credentials secret that you want to regenerate for your application. But when you use the Secrets Manager APIs, SDKs, or CLI to get the secret, you see the following 412 Precondition Failed error:

The requested action can't be completed because the secret version is locked.

  Why it’s happening 

A lock on a secret prevents it from being modified or deleted from your instance. IAM credentials are

dynamic secrets. By default, each request to read an IAM credential (for example, a GET request) generates a new service ID API key, deletes the old credentials, and returns the new credentials. Locking the secret overrides this default behavior and returns a 412 Precondition Failed error to indicate that the secret data is locked. A locked IAM credential can't be read, because doing so modifies its secret data.

  How to fix it 

To regenerate your IAM credentials, you can remove all the locks that are associated with your secret, and try again. To delete locks from the Secrets Manager UI, go to Secrets > secret name > Locks > Delete.
To create an IAM credential from the IBM Cloud CLI, run the [ibmcloud secrets-manager secret-create](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-cli-plugin-secrets-manager-clisecrets-manager-cli-secret-create-command) command.

export SECRET_ID=ibmcloud secrets-manager secret-create --resources '[{"name":"test-iam-credentials","description":"Extended description for my secret.","access_groups":"$ACCESS_GROUP_ID"],"secret_group_id":"$SECRET_GROUP_ID","ttl":"2h","labels":"storage","us-south"]}]' --output json | jq -r ".resources[].id"

You can also create an IAM credential by using the Secrets Manager UI. For more information, see [Creating IAM credentials](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-iam-credentials).





 Step 3: Generate an API key 

After you create the IAM credential, a user with lesser privileges can retrieve the secret when it's time to access the storage bucket.

To retrieve an IAM credential from the IBM Cloud CLI, you can run the [ibmcloud secrets-manager secret](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-cli-plugin-secrets-manager-clisecrets-manager-cli-secret-command) command.



1. Retrieve the contents of your IAM credential secret.

ibmcloud secrets-manager secret --id $SECRET_ID --output json

The following JSON snippet shows the example response.

{
"metadata": {
"collection_type": "application/vnd.ibm.secrets-manager.secret+json",
"collection_total": 1
},
"resources": [
{
To configure the IAM secrets engine from the IBM Cloud CLI, run the [ibmcloud secrets-manager config-update](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-cli-plugin-secrets-manager-clisecrets-manager-cli-configuration-update-command) command.

ibmcloud secrets-manager config-update --engine-config '{"api_key": "$API_KEY"}'

Success! Now you can [create an IAM credential](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-iam-credentialsiam-credentials-ui) that you can use to dynamically generate service IDs and API keys. Continue to the next step.









 Step 2: Create an IAM credential 

IAM credentials are dynamic secrets that you can use to access an IBM Cloud resource on-demand, such as a Cloud Object Storage bucket. A set of IAM credentials consists of a service ID and an API key that is generated each time that the protected resource is read or accessed. You can define a time-to-live (TTL) or a lease duration for your IAM credential at its creation so that you shorten the amount of time that the secret exists.

To create an IAM credential from the IBM Cloud CLI, run the [ibmcloud secrets-manager secret-create](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-cli-plugin-secrets-manager-clisecrets-manager-cli-secret-create-command) command.

export SECRET_ID=ibmcloud secrets-manager secret-create --resources '[{"name":"test-iam-credentials","description":"Extended description for my secret."
iam_apikey_description
: Description of IAM API key.

iam_apikey_name
: ID of IAM API key.

iam_role_crn
: The IAM role that the IAM API key has.

iam_serviceid_crn
: The CRN of service ID.

password
: The IBM Cloudant legacy credential password.

port
: IBM Cloudant service port.

url
: IBM Cloudant service URL, including embedded IBM Cloudant legacy credentials.

username
: The IBM Cloudant legacy credential username.

Note the included username and password are always equivalent to IAM's Manager credentials. Therefore, the use of Use both legacy credentials and IAM is insecure when used with Reader, Writer, Monitor, or Checkpointer IAM roles.







 Must I use Use only IAM or Use both legacy credentials and IAM? 

If possible, Use only IAM is preferred. The major advantages for using IBM Cloud IAM are shown in the following list:



* Management of access to IBM Cloudant with the standard tools of IBM Cloud rather than a combination of IBM Cloud and IBM Cloudant-specific credential management.
* Credentials can be easily revoked and rotated when you use IBM Cloud IAM.



Further description of the advantages and disadvantages of each approach follows.

When you use IAM roles other than Manager, such as Reader, Writer, Monitor, or Checkpointer, you must use Use only IAM to avoid supplying users with legacy credentials that include greater access permissions.



 Advantages and disadvantages of the two access control mechanisms 

Overall, IBM Cloud IAM is the recommended authentication model. However, the primary disadvantages to the approach are if you have an existing application or are unable to use an IBM Cloudant-supported client library.



 Advantages of IAM mode 



* Manage access for many services by using one interface.
* Revoke access to a user globally.
* Account-level API keys that use service IDs.
* Easy-to-rotate credentials.
* Activity Tracker logs capture individual humans and services.
* IAM federates with other identity systems, like enterprise LDAP repositories.
The service ID that is associated with your API key must have Editor platform access on the IAM Access Groups Service, Service ID creator access, and Operator platform access on the IAM Identity Service.
6. Click Configure.

Now, your Secrets Manager instance is enabled for IAM credential secrets.







 Setting up the IAM credentials engine from the CLI 

Before you can create dynamic IAM credentials, you must configure the IAM secrets engine for your service instance. Start by entering an [IBM Cloud API key](https://cloud.ibm.com/docs/account?topic=account-serviceidapikeys) that is associated with a service ID in your IBM Cloud account.

To allow your IBM Cloud API key to create and manage other API keys dynamically, its associated service ID must have Editor platform access for the IAM Access Groups Service, Operator platform access for the IAM Identity Service.



1. In a terminal window, log in to IBM Cloud through the [IBM Cloud CLI](https://cloud.ibm.com/docs/cli?topic=cli-install-ibmcloud-cli).

ibmcloud login

If the login fails, run the ibmcloud login --sso command to try again. The --sso parameter is required when you log in with a federated ID. If this option is used, go to the link listed in the CLI output to generate a one-time passcode.
2. Select the account, region, and resource group where your Secrets Manager service instance is located.
3. Create a service ID and set it as an environment variable.

export SERVICE_ID=ibmcloud iam service-id-create <service_ID_name> --description "<description>" --output json | jq -r ".id"; echo $SERVICE_ID
4. Manage access for the service ID.

Assign the service ID permissions to create and manage other service IDs.

ibmcloud iam service-policy-create $SERVICE_ID --roles Operator --service-name "iam-identity"
* UI
* CLI
* API
* Terraform







 Configuring the IAM credentials engine 

You can set up IAM credentials for your IBM Cloud® Secrets Manager service instance by configuring the IAM credentials engine.

In Secrets Manager, the IAM credentials engine serves as the backend for the iam_credentials secret type. Before you can create IAM credentials, you must configure the IAM credentials engine for your service instance. You can enable your instance by entering an [API key](https://cloud.ibm.com/docs/account?topic=account-serviceidapikeys) that is associated with a service ID in your IBM Cloud account.



 Before you begin 

If you're setting up IAM credentials for the first time, be sure that you're assigned the [Manager service role](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-iam) on the Secrets Manager instance. To configure the IAM secrets engine, you need a [service ID API key](https://cloud.ibm.com/docs/account?topic=account-serviceidapikeys) with the following access:



* [Editor platform role](https://cloud.ibm.com/docs/account?topic=account-account-servicesaccess-groups-account-management) on the IAM Access Groups Service.
* [Operator platform role](https://cloud.ibm.com/docs/account?topic=account-account-servicesidentity-service-account-management) on the IAM Identity Service.
* [Service ID creator service role](https://cloud.ibm.com/docs/account?topic=account-account-servicesidentity-service-account-management) on the IAM Identity Service.



The service ID creator service role is only required when you disable the creation of service IDs in your IAM settings. If the account in which you want to generate IAM credentials allows access to only specific IP addresses, you must also update the IP address settings in the account to allow incoming requests from Secrets Manager.
For instance, a user might not be able to create new buckets, but can create and delete objects within existing buckets.







 Manage access 

IAM is based on a fundamental concept: A subject is granted a role on a resource.

There are two basic types of subjects: a user and a service ID.

There is another concept, a service credential. A service credential is a collection of important information that is needed to connect to an instance of IBM Cloud® Object Storage. It gives a user an identifier for the instance of IBM Cloud Object Storage (that is, the Resource Instance ID), service and authorization endpoints, and a means of associating the subject with an API key (that is, Service ID). When you create the service credential you have the option of either associating it with an existing service ID, or creating a new service ID.

You might want your development team to be able to use the console to view Object Storage instances and Kubernetes clusters. They would need Viewer roles on the Object Storage resources and Administrator roles on the Container Service. The Viewer role allows for the user to only see that the instance exists, and to view existing credentials, but not to view buckets and objects. When the service credentials were created, they were associated with a service ID. This service ID would need to have the Manager or Writer role on the instance to be able to create and delete buckets and objects.

For more information on IAM roles and permissions, see [the IAM overview](https://cloud.ibm.com/docs/cloud-object-storage/iam?topic=cloud-object-storage-iam-overview).