vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Zoom

![Schematics Agent Architecture](https://cloud.ibm.com/docs-content/v1/content/ba89e173287b28d1453d2a327d8a1f74ae2f1662/schematics/images/new/sc_agentb0_architecture.svg)

Schematics Agent architecture and its components



1. As the Schematics Agents user, now you can extend the Schematics ability to reach your cloud infrastructure from your IBM Cloud account.
2. Configure the Schematics Agent by using an [Agents infrastructure workspace](https://cloud.ibm.com/docs/schematics?topic=schematics-sch-termsagentsa3) and an [Agents service workspace](https://cloud.ibm.com/docs/schematics?topic=schematics-sch-termsagentsa2) to create your cluster infrastructure.
3. Integrate the Schematics private endpoint with the Schematics service to provision, configure, and monitor your application through Schematics Agent.
4. Manage the tools and softwares through Agents services containing the micro-services.







 Augmenting Schematics with Schematics Agents 

The table describes how the Schematics are augmented with Schematics Agents with the components such as, cluster, cloud providers, compute time, latency, software, tenancy, and network configurations.



Usage of Schematics Agents

 Components Schematics without Agent Schematics with Agent 

 Cluster Runs in Schematics cluster. Runs in customer's cluster. 
 Cloud providers Works primarily with IBM Cloud and not tested with other cloud services. Can integrate with any cloud service providers or private cloud. 
 Compute time Null resources or [local-exec](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-limitationslocal-remote-exec) executes for a maximum of 30 minutes. There is no compute time restrictions. 
 Latency Runs in Schematics clusters provisioned in us or eu region only.
* Install or update the [Schematics plug-in](https://cloud.ibm.com/docs/schematics?topic=schematics-setup-cliinstall-schematics-plugin) version that is greater than the 1.12.5.
* Select the IBM Cloud region that you wish to use to manage your blueprint. Set the region by running [ibmcloud target -r <region>](https://cloud.ibm.com/docs/cli?topic=cli-ibmcloud_cliibmcloud_target) command.
* Check that you have the [IAM permissions](https://cloud.ibm.com/docs/schematics?topic=schematics-accessblueprint-permissions) to create blueprints.



The command example shown here creates a blueprint configuration in Schematics, with the template file basic-blueprint.yaml and input file basic-input.yaml from the source Git repository https://github.com/Cloud-Schematics/blueprint-basic-example. With this basic two module example, the first module reads the existing default resource group and the second, creates a Cloud Object Storage (COS) instance and bucket in the default resource group.

For all the blueprint commands, syntax, and option flag details, see the section [blueprint commands](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceblueprints-cmd).

Full blueprint create command syntax:

ibmcloud schematics blueprint create -name Blueprint_Basic -resource-group Default -bp-git-url https://github.com/Cloud-Schematics/blueprint-basic-example -bp-git-file basic-blueprint.yaml -input-git-url https://github.com/Cloud-Schematics/blueprint-basic-example -input-git-file basic-input.yaml

On successful completion, the config create returns create_success and the unique ID of the blueprint.
Depending on how you change the configuration of existing resources, IBM Cloud Schematics might not be able to update your resource. Instead, IBM Cloud Schematics might decide to apply the change, the resource must be removed, and a new resource must be created. If your resource must be removed, make sure that you do not interrupt a working environment, or delete data.

When I change my configuration file in GitHub, is my change automatically available in the next execution plan?
If you change the code of your Terraform template in GitHub, these changes are not available automatically when you create an execution plan in IBM Cloud Schematics. To pull the latest changes from your GitHub repository, make sure that you click the Pull latest button from the workspace settings page before you create your execution plan.

To update your resources:



1. From the [workspace dashboard](https://cloud.ibm.com/schematics/workspaces), select the workspace that points to the Terraform template that you updated. Sample templates examples are listed in [Cloud Schematics](https://github.com/Cloud-Schematics) GitHub, you can use one of the template for testing. For example, [Easy multizone VPC](https://github.com/Cloud-Schematics/easy-multizone-vpc).
2. Select the Settings tab.
3. In the Summary section, click Pull latest to get the latest version of your Terraform template from the linked GitHub source repository. If you provided your Terraform template by uploading a tape archive file (.tar), you must use the [ibmcloud schematics workspace upload](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-upload) command to provide a new version of your template.
4. If you added, or removed variables in your Terraform configuration files, or if you want to change the variable values that you set when you created the workspace, enter or change the variable values.
5.
[Deploying infrastructure and services with Workspaces](https://cloud.ibm.com/docs-content/v1/content/ba89e173287b28d1453d2a327d8a1f74ae2f1662/schematics//images/new/sc-workspaces.svg)

Deploying infrastructure and services with Workspaces

Using your supplied Terraform template (config), Schematics runs the Terraform engine to execute your config.





 Features 

IBM Cloud® provides built in [remote-state](https://cloud.ibm.com/docs/schematics?topic=schematics-remote-state) management for Terraform. Terraform state files are automatically preserved between runs and are accessible by IBM Cloud® commands and operations. IBM Cloud® remote-state management enables team work and workspace shared operations, with built in state locking preventing concurrent operations against the same state file.

Workspaces is designed for teams. Terraform templates can be stored in a GitHub, GitLab, or Bitbucket repository to ensure source control and enable collaboration, review, and auditing in your organization.

Workspaces supports [drift detection](https://cloud.ibm.com/docs/schematics?topic=schematics-drift-note), detecting when the configuration of your deployed infrastructure differs from the wanted state that is defined in your template configuration. Drift can occur for many reasons. The most frequent cause changes that are applied manually outside of Terraform automation.





 Next steps 

So far you have learned about Schematics Workspaces. The following are some next steps to explore.



* [Getting started usecase](https://cloud.ibm.com/docs/schematics?topic=schematics-get-started-terraform) to understand how to use workspace to deploy and manage your Infrastructure as Code (IaC) in the cloud environments.
[blueprint update flow](https://cloud.ibm.com/docs-content/v1/content/ba89e173287b28d1453d2a327d8a1f74ae2f1662/schematics//images/new/bp-update.svg)

blueprint update flow



1. Edit the blueprint template, and the input YAML files to implement the proposed changes to the environment.



* You can use any text editor. [VSCode with the YAML language extension](https://cloud.ibm.com/docs/schematics?topic=schematics-edit-blueprints) is recommended, as it provides blueprint template syntax validation and autocomplete.
* Push the updated blueprint template and input files to your Git repositories. If needed, create a new Git version release tag for version management.



2. Update the blueprint configuration for the environment in Schematics.



* When using versioning, the new Git release tag or branch must be specified for the modified blueprint template and input value YAML files.
* If no version or branch is specified, Schematics automatically checks the Git repository for a recent commit, and runs a pull latest to pull in any updates.
* For more information on running this operation, see [update a blueprint configuration](https://cloud.ibm.com/docs/schematics?topic=schematics-update-blueprint).



3. Schematics validates the changes made to the configuration.
4. On a successful config update, Schematics automatically reinitializes the modules with any updated input values and updates to the module IaC code.
5. Apply the changes to the updated configuration. The changes are applied with the blueprint apply command or UI apply operation.



* Based on your updated blueprint configuration, runs the IaC modules in dependency order to update the environment. In a future release the plan will be presented for review before apply.
This builds on the IaC best practice of [modular architectures](https://cloud.ibm.com/docs/schematics?topic=schematics-infrastructure-as-codeiac-bp-modularity), where reusable modules implement the layers and components of an infrastructure architecture from well designed, tested and compliant Terraform code.



 Cloud infrastructure lifecycle 

Schematics Blueprints follows an lifecycle operations model. Cloud environments, hosted applications or services all follow a lifecycle from creation to end-of-life. The life of a blueprint environment starts with the initial definition of an infrastructure architecture and configuration. Then, onto deployment of the environment and resources. It will be updated and maintained through its operational life. Which might be hours to years. Finally, to end-of-life when it is torn down, the cloud resources are destroyed, billing gets terminated and the configuration is removed.

The different stages of working with the blueprint lifecycle come under four headings, define, deploy, maintain, and delete. In software lifecycle terms these correspond to Day-0 define, Day-1 deploy, Day-2 operate, through to deletion of the environment.

![Schematics Blueprints lifecycle stages](https://cloud.ibm.com/docs-content/v1/content/ba89e173287b28d1453d2a327d8a1f74ae2f1662/schematics//images/new/bp-lifecycle.svg)

Each of these life stages is supported by blueprint operations. For example, composing and editing the blueprint template, running blueprint operations, monitoring job execution and reviewing the results.

All updates and changes to environments during these lifecycle stages are performed in a controlled sequence of steps:



* Creating or updating the blueprint configuration with changes to the inputs, template or module versions
* Planning the change to identify resource changes, additions or deletions
* Running the plan to apply the changes to create, modify or delete resources







 Working with environments and the blueprint lifecycle 

Explore the following sections to get into the details of working with blueprints and environments over their lifecycle.
Schematics will automatically save and restore them before and after running the Schematics jobs or Terraform command.

Your files must be placed in the /tmp/.schematics folder and the size limit is set to 10 MB. Schematics backups and restores all the files in the /tmp/.schematics folder.
* How do I identify the best way to synchronize a deleted resource with the Terraform state?

Currently, the IBM Cloud Schematics service does not support the ability to import or synchronize the IBM Cloud resource state into the Schematics Workspace. It is planned in the future roadmap.
* How do I overcome the request exceeds the Cluster resource quota of '100' for the account in any region?

Error: Request failed with status code: 403, ServerErrorResponse: {"incidentID":"706efb2c-3461-4b9d-a52c-038fda3929ea,706efb2c-3461-4b9d-a52c-038fda3929ea","code":"E60b6","description":"This request exceeds the 'Cluster' resource quota of '100' for the account in this region. Your account already has '100' of the resource in the region, and the request would add '1'. Revise your request, remove any unnecessary resources, or contact IBM support to increase your quota.","type":"General"}

You see this quota validation error when the Cluster resource quota of 100 for the account in this region is exceeded. You can consider deleting the existing resources and try running operation again.
* While creating Red Hat OpenShift or Kubernetes resources, can I tune 90 minutes time out to higher?

Yes, you can increase the timeout for Red Hat OpenShift or Kubernetes resources. For more information, see [ibm_container_vpc_cluster](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/container_vpc_clustertimeouts) provides the following Timeouts configuration options.
Each of the templates provides guidance specific to the underlying IBM Cloud technology used in one of the [reference architectures](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-aboutframework-reference-architectures). They are referred to as "templates" because they allow you to assess how well your solution meets each of the control requirements. A completed CIO template is a requirement for any ecosystem partner seeking to be Financial Services Validated.





 Reference architectures and deployment guidance 

[Reference architectures](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-reference-architecture-overview) have been defined which can be used as a basis for meeting the security and regulatory requirements defined by the controls. The reference architectures each center on specific IBM Cloud technologies, including:



* [IBM Cloud® Virtual Private Cloud reference architecture](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-vpc-architecture-about), with options for using one or both of:



* [IBM Cloud® Virtual Servers for Virtual Private Cloud](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-vpc-architecture-detailed-vsi)
* [Red Hat® OpenShift® on IBM Cloud®](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-vpc-architecture-detailed-openshift)



* [IBM Cloud Satellite® reference architecture](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-satellite-architecture-about)
* [IBM Cloud® for VMware® Regulated Workloads reference architecture](https://cloud.ibm.com/docs/framework-financial-services?topic=framework-financial-services-vmware-overview)
* Dynamic assignment of workspace or action jobs to the agent.



* What are the costs of installing and using Agents?

The following are the cost break-down for the Schematics Agent.

Pre-requisite: Agent infrastructure



* Cost of VPC infrastructure elements such as, subnet, public gateways.
* Cost of IBM Kubernetes Service (cluster) on VPC, with three-node worker pool.
* Cost of IBM Cloud Object Storage



Agent beta-1 service



* There is no cost involved in running the agent service.
* Post beta, the agent feature may be a priced service.



* Can I install more than one Agent on a cluster?

You can install only one agent on a Kubernetes cluster on IBM Cloud Kubernetes Service. You can install additional agents on different clusters.

You cannot install more than one agent in a single Kubernetes cluster. You will get a failure with namespace conflict error.
* What type of Schematics jobs can I run in my Agent?

You can run Schematics Workspace Terraform jobs on an Agent. You can also run Schematics Action jobs, Ansible playbooks on an Agent.
* How can I see the Schematics job results and logs, for the workloads running on an agent?

The workspace job or action job logs are available in Schematics UI console. You can also access these job logs using the Schematics Workspace API, or CLI.
* How many Schematics jobs can run in parallel in the Agent?

Currently, an agent can run three Schematics jobs in parallel. Any additional jobs are queued and will execute when prior jobs complete execution.

In future, you will be able to customize an agent to increase the number of job Pods, to increase the number of jobs that can run concurrently.
* What is the minimum cluster configuration required in Agent release?

The agent needs IBM Cloud Kubernetes Service Service with minimum three worker nodes, with a flavor of b4x16 or higher.
* How many workspaces can be assigned to an agent?

Currently, you can assign any number of workspaces to an agent.
curl -X GET https://schematics.cloud.ibm.com/v1/workspaces/{w_id} -H 'authorization: <auth>' -H 'cache-control: no-cache' -H 'content-type: application/json'
3. Run [workspace upload](https://cloud.ibm.com/apidocs/schematics/schematicstemplate-repo-upload) to upload agent-service-templates.tar file.

curl -X PUT https://schematics.cloud.ibm.com/v1/workspaces/{w_id}/template_data/{template_id}/template_repo_upload -H 'authorization: <auth>' -H 'cache-control: no-cache' -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' -F file=@agent-service-templates.tar
4. Run [workspace get](https://cloud.ibm.com/apidocs/schematics/schematicsget-workspace) operation to pull the workspace details from the .tar file.

curl -X GET https://schematics.cloud.ibm.com/v1/workspaces/{w_id} -H 'authorization: <auth> ' -H 'cache-control: no-cache' -H 'content-type: application/json'
5. Run [apply command](https://cloud.ibm.com/apidocs/schematics/schematicsapply-workspace-command) to provision an Agent service.

curl -X PUT https://schematics.cloud.ibm.com/v1/workspaces/{w_id}/apply -H 'authorization: <auth>' -H 'cache-control: no-cache' -H 'content-type: application/json' -d '{
"name": "smulampa-schematics-agent-service-workspace",
"type": [
Schematics Blueprints is a [beta feature](https://cloud.ibm.com/docs/schematics?topic=schematics-bp-beta-limitations) that are available for evaluation and testing purposes. It is not intended for production usage. Refer to the list of [limitations](https://cloud.ibm.com/docs/schematics?topic=schematics-bp-beta-limitationssc-bp-beta-limitation) for the beta release.



 Versioning blueprint environments 

Blueprints supports two approaches to managing change to blueprint environments and their constituent templates, modules and inputs. Either a relaxed 'development' mode where the current version of a template or module is always pulled on an update operation, or a 'production' mode where an explicit version must be specified.



 Relaxed versioning 

In some development environments, or where IaC code and blueprint templates are being developed, versioning of IaC definitions is not needed.

Blueprints defaults to always pull the current (latest) template, inputs, or modules on an update operation if no version information is specified in the blueprint config at create time.



 Creating environments using relaxed versioning 

Relaxed versioning is in effect if a blueprint configuration is created without Git version or branch information. To create an environment that does not use versioning, the create command omits the bp-git-release, bp-git-branch and the corresponding input file definitions:

ibmcloud schematics blueprint create -name Blueprint_Basic -resource-group Default -bp-git-url https://github.com/Cloud-Schematics/blueprint-basic-example -bp-git-file basic-blueprint.yaml -input-git-url https://github.com/Cloud-Schematics/blueprint-basic-example -input-git-file basic-input.yaml

Sample template module definition without versioning:

source:
source_type: github
git:
In cookie cutter fashion, several environments can be created from the same blueprint template. Each environment has its own [blueprint configuration](https://cloud.ibm.com/docs/schematics?topic=schematics-sch-termsbpb3) and inputs. This separation of template from its deploy time configuration is illustrated in the figure. Here a template is reused many times to deploy a range of environments such as dev, stage, and production. Each environment is customized with its own input values, all based on the same template. This is illustrated in the following figure.

Zoom

![Blueprint template, modules and inputs define environments](https://cloud.ibm.com/docs-content/v1/content/ba89e173287b28d1453d2a327d8a1f74ae2f1662/schematics//images/new/bp-reuse.svg)

Blueprint template, modules and inputs define environments

The blueprint template determines the architecture. It specifies the modules required for the implementation, infrastructure topology and data dependencies between modules. Schematics deploys the modules as discrete environments, managing their lifecycle and the passing of resource dependency information between modules.

The blueprint environment and the cloud resources to be deployed, are defined by three versioned elements:



1. A blueprint template file specifying the resource topology, infrastructure architecture, IaC automation modules and dependencies.
2. Input values to configure and customize the blueprint template at deployment time.
3. Automation modules written in Terraform to deploy the desired cloud resources.



A blueprint config links a template with the set of inputs values to customize the environment and cloud resources for its target use. The config its relationship to a template and input files are illustrated in the figure below.





 Blueprint template overview 

The infrastructure architecture of a solution is defined by a blueprint template:



* Blueprint templates are written in YAML with a minimum of syntax that specify the automation modules to be used, their versions, source libraries, and relationships for passing resource dependency data between modules.
map(number) {internal = 8080, external = 2020} 
 list(string) ["us-south", "eu-gb"] 
 list ["value", 30] 
 list(list(string)) See [list of String example](https://cloud.ibm.com/docs/schematics?topic=schematics-create-tf-configexample-list-strings). 
 list(object({internal = number </br> external = number </br> protocol = string})) See [list of Object example](https://cloud.ibm.com/docs/schematics?topic=schematics-create-tf-configexample-list-object). 





 Example for list of Strings 

[
{
internal = 8300
external = 8300
protocol = "tcp"
},
{
internal = 8301
external = 8301
protocol = "ldp"
}
]





 Example for list of Objects 

[
{
internal = 8300
external = 8300
protocol = "tcp"
},
{
internal = 8301
external = 8301
protocol = "ldp"
}
]







 Storing your Terraform templates 

Your Terraform configuration files contain infrastructure code that you must treat as regular code. To support collaboration, source, and version control, store your files in a GitHub or GitLab repository. With version control, you can revert to previous versions, audit changes, and share code with multiple teams. If you do not want to store your files in GitHub, provide your template by uploading a [tape archive file or .tar](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-upload) from your local machine instead.