vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Compact config repo download 

Schematics Workspaces act as a container of a Terraform template with the input data, output state, jobs, and job log files. The workspace uses the Git URL of the Terraform template. For example, if the user provides the URL https://github.com/terraform-ibm-modules/terraform-ibm-database/tree/main/examples/simple-etcd to download and securely store the template files. The default execution of the Schematics is to download the whole Git repository and to save securely. Schematics assumes that the Terraform templates have relative references to modules, script, or data files that reside in other folders in the repository.

However, sometimes, the user may be aware that the Terraform templates are isolated to a specific folder and its sub directory, for example, examples/simple-etcd in the Git repository. Now the user wants the workspace to download, only the relevant files from the Git repository. This can be achieved by using the compact download feature. The compact download feature improves the time it takes to download, and process the files from the Git repository while creating, or updating the workspace.



 Using compact download 

You can activate compact download feature through console by unchecking the Download entire repo checkbox on the [Create a Schematics Workspace page](https://cloud.ibm.com/schematics/workspaces/create). By default, the checkbox is selected to download the full Git repository.

You can also activate compact download through the API or CLI by using compact field in the workspace [create](https://cloud.ibm.com/apidocs/schematics/schematicscreate-workspace), or [update](https://cloud.ibm.com/apidocs/schematics/schematicsreplace-workspace) payload, as illustrated in the code block.

{
        "name": "Testmyservice",
        "type": [
                "terraform_v1.0"
        ],
        "description": "Create Terraform workspace to test compact feature",
* Can I upgrade an agent beta-0 to agent beta-1?

No, you cannot upgrade agent beta-0 setup to agent beta-1.
* Does agent beta-1 support backward compatibility?

Agents beta-1 does not support backward compatibility. You need to create a new agent by using the [agent beta-1](https://cloud.ibm.com/docs/schematics?topic=schematics-agentb1-about-intro) setup.
* Are Schematics Agents the same as Terraform Cloud Agents?

Schematics Agents perform a similar role to [Terraform Cloud agents](https://developer.hashicorp.com/terraform/cloud-docs/agents).
* Do the agents run on IBM Cloud cloud resources?

Schematics Agents can run only Terraform and Ansible workloads. For the Beta, the agents are deployed in IBM Cloud IKS Clusters in the user account.
* What are the Git repositories that are supported by Blueprints?



Supported Git repositories

 Repository <br> Template <br>Public repo Template <br>Private repo Module <br>Public repo Module <br>private repo Comment <br> 

 GitHub Yes Git token Yes Git token 
 GitLab Yes Git token Yes Git token 
 IBM GitLab Yes Git token Yes Git token IAM token planned Dec 2022 
 Terraform.io No No Yes NA 



* Are variable operators and functions supported in blueprint templates?

The 1.0.0 release of the Blueprints schema supports a minimal set of variable operators. It is intentionally simple. The values passed between inputs and outputs are passed as is. Functions and variable operators are not supported in this release.

A future release intends to implement functions and operators.
* How are blueprint module dependencies and execution order determined?

Dependencies between blueprint modules are created using the value references between module inputs and outputs. Similar to Terraform, a DAG (Directed Acyclic Graph) is generated from the dependencies and relationships to determine execution order. Dependencies are created using '$module' references.
=> {"msg": "winrm or requests is not installed: No module named 'winrm'"}
2021/12/06 10:15:49 Terraform apply 
2021/12/06 10:15:49 Terraform apply | PLAY RECAP 
2021/12/06 10:15:49 Terraform apply | 161.156.161.7 : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
2021/12/06 10:15:49 Terraform apply 
2021/12/06 10:15:49 Terraform apply 
2021/12/06 10:15:49 Terraform apply 
2021/12/06 10:15:49 Terraform apply | with null_resource.schematics_for_windows,
2021/12/06 10:15:49 Terraform apply | on schematics.tf line 2, in resource "null_resource" "schematics_for_windows":
2021/12/06 10:15:49 Terraform apply | 2: provisioner "ansible" {
2021/12/06 10:15:49 Terraform apply 
2021/12/06 10:15:50 Terraform APPLY error: Terraform APPLY errorexit status 1
2021/12/06 10:15:50 Could not execute action

WinRM is not supported by Schematics Terraform Ansible provisioner. Alternatively you can use the Schematics Actions to run the Ansible playbooks with WinRM. The Schematics Actions support [WinRM](https://cloud.ibm.com/docs/schematics?topic=schematics-action-working).
* When are the new Terraform and Ansible versions added to Schematics?

After new Terraform and Ansible versions are released by the community, the IBM team begins hardening and testing the release for Schematics. Availability of new versions depends on the results of these tests, community updates, security patches, and technology changes between versions. Make sure that your Terraform templates and Ansible playbooks are compatible with one of the supported versions so that you can run them in Schematics.
Repository <br> Template <br>Public repo Template <br>Private repo Module <br>Public repo Module <br>private repo Comment <br> 

 GitHub Yes Git token - 1 Yes Git token - 2 
 GitLab Yes Git token - 1 Yes Git token - 2 
 IBM GitLab Yes Git token - 1 Yes Git token - 2 
 Terraform.io No No Yes NA 





1. Git token defined at workspace create time
2. Git token defined using netrc



When using Schematics, netrcsupport for module credentials can be configured using the __netrc__ environment variable to the pass credentials. The __netrc__ environment variable accepts the list of hostname, username and the password argument. The setting of environment variables is supported only using the Schematics [command-line](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-workspace-new) and [APIs](https://cloud.ibm.com/apidocs/schematics/schematicscreate-workspace). The syntax is provided using the env_values parameter in the JSON payload file.

The __netrc__ expects hostname, username, and password argument in the same order that are listed in the syntax.

Syntax of env_values with list of __netrc__:

"env_values":[
{
"__netrc__":"'example.com', 'user1', 'pass1']'example1.com', 'user2' , 'pass2']]"
}
],



 Using private modules with templates 

Schematics internally creates the .netrc file based on the env_values configured in the JSON file. Here is a syntax and sample testexample.json example file to clone all the files to create and apply the Schematics Workspaces through command-line and API.

Syntax with the description:

{
Alternatively you can use the Schematics Actions to run the Ansible playbooks with WinRM. The Schematics Actions support [WinRM](https://cloud.ibm.com/docs/schematics?topic=schematics-action-working).





 When are the new Terraform and Ansible versions added to Schematics? 

After new Terraform and Ansible versions are released by the community, the IBM team begins hardening and testing the release for Schematics. Availability of new versions depends on the results of these tests, community updates, security patches, and technology changes between versions. Make sure that your Terraform templates and Ansible playbooks are compatible with one of the supported versions so that you can run them in Schematics. For more information, see [Upgrading the Terraform template version](https://cloud.ibm.com/docs/schematics?topic=schematics-migrating-terraform-version) and [Schematics runtime tools](https://cloud.ibm.com/docs/schematics?topic=schematics-sch-utilitiessch-runtime-tf-job).





 Can I run Ansible playbooks with Schematics? 

Yes, you can run Ansible playbooks against your IBM Cloud by using the Schematics Actions or Ansible provisioner in your Terraform configuration file. For example, use the Ansible provisioner to deploy software on IBM Cloud resources or set actions against your resources, such as shutting down a virtual server instance. For more information, see [sample Ansible playbook templates for Schematics Actions](https://cloud.ibm.com/docs/schematics?topic=schematics-sample_actiontemplates).
May 2023 



 18 May 2023 

Version 3.8.3 of the landing zone deployable architectures available
: Version 3.8.3 of the landing zone deployable architectures is available in the IBM Cloud [catalog](https://cloud.ibm.com/catalogreference_architecture). This version fixes the issue that prevented deletion of the SSH key in the VSI on VPC landing zone. For other changes in the release, see the [release notes](https://github.com/terraform-ibm-modules/terraform-ibm-landing-zone/releases/tag/v3.8.3) on GitHub.





 04 May 2023 

Version 3.6.4 of the landing zone deployable architectures available
: Version 3.6.4 of the landing zone deployable architectures is available in the IBM Cloud catalog. This version includes updates to several variable descriptions and support for IBM Cloud Security and Compliance Center v2 rules.







 17 April 2023 

Introducing the landing zone deployable architectures
: Three VPC landing zone deployable architectures are released: VPC landing zone, VSI on VPC landing zone, and Red Hat OpenShift Container Platform on VPC landing zone. You can use the deployable architectures to create a secure and customizable Virtual Private Cloud (VPC) environment. Thesedeployable architecturesare based on the IBM Cloud for Financial Services reference architecture. For more information about using deployable architectures with projects, see the blog posts [Projects and Cost Estimation: How IBM Cloud is Revolutionizing Complex Workloads for Enterprises](https://www.ibm.com/cloud/blog/announcements/projects-and-cost-estimation) and [Turn Your Terraform Templates into Deployable Architectures](https://www.ibm.com/cloud/blog/turn-your-terraform-templates-into-deployable-architectures).
Reusable child modules require additional blueprint template injector parameters to enable usage with templates. Use of injector parameters is described in the next sections.



 Root modules and Terraform configurations 

A root module is the root of an executable Terraform configuration. A root module, also known as a Terraform config or Terraform template and can be run directly by the Terraform command line binary or Schematics.

A Terraform configuration consists of a root module as the main directory that contains .tf files. From the root module, Terraform can call child modules [published](https://developer.hashicorp.com/terraform/language/modulespublished-modules) to public or private repositories. Configurations (root modules) written for IBM Cloud must contain a provider block and a terraform>required_providers block to identify the [IBM Cloud provider](https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs) to Terraform.

A root module can be used unchanged in a blueprint template, by referencing the module source repo and providing input values. It must contain an IBM Cloud provider definition.





 Child and reusable shared modules 

Sharing and reuse of child modules is at the heart of an effective IaC automation strategy and use of Schematics Blueprints.

Child module, is the term given to any module that is called by another module or root module. Child modules cannot be executed stand-alone and the naming distinguishes it from root modules which can be executed directly. Sharing and reuse builds on the Terraform support for loading child modules from shared public and private repositories and the Terraform registry. Remote loading enables the sharing of content by [publishing modules](https://developer.hashicorp.com/terraform/language/modulespublished-modules) for others to use, and to reuse published modules.

Examples of IBM authored (child) modules designed for reuse can be found in the [terraform-ibm-modules](https://github.com/terraform-ibm-modules) GitHub repository.





 Blueprints provider injection
Then, IBM Cloud and the customer configure Border Gateway Protocol (BGP) to exchange routes between the environments. Finally, IBM Cloud places the customer into a VRF to allow for the implementation of non-unique routes to the private address space of the customer's remote network.
* Where do I find cost estimates for Direct Link offerings?

You can estimate the cost of a service using the cost estimator on the provisioning pages for Direct Link on Classic offerings. For example, select a tile from the [IBM Cloud catalog](https://cloud.ibm.com/catalog?category=compute%20network&search=direct%20link%20%22on%20classic%22%20label%3Aibm_createdsearch_results) to view the service's ordering page. As you complete the ordering form, cost estimates appear in the Summary side panel.
* Does IBM Cloud meter bandwidth for Direct Link products?

Yes. Bandwidth usage across the Direct Link service between customers and IBM Cloud is free and unmetered, IBM Cloud does meter outbound bandwidth from IBM Cloud services to the public internet.
* When does billing begin with Direct Link?

The fees for Direct Link Connect cover the cost of service termination on the IBM Cloud infrastructure.

Infrastructure services are billed in advance and begin upon acceptance of a client’s order, however, due to the nature of IBM Cloud Direct Link, the Direct Link service billing begins when a Border Gateway Protocol (BGP) session is established with IBM Cloud, or 30 days after the service key is provided to the client.

Billing stops after (1) a customer requests a circuit to be deleted AND (2) the Connect Provider or Network Service Provider has de-provisioned the circuit.
* What extra charges will I incur from other parties with Direct Link?

You might have extra charges from your exchange provider or network service provider. Refer to your providers for their fee information.
* How can I achieve redundancy with IBM Cloud Direct Link?

Direct Link does not provide an inherently Redundant service. Direct Link can provide Diverse connections that enable customers to create redundancy via BGP. You can achieve diversity with Direct Link by connecting to more than one IBM Cloud Direct Link Dedicated provider or Exchange provider for IBM Cloud.
Using the Security and Compliance Center with DevSecOps toolchains 

IBM Cloud® Security and Compliance Center automates security and compliance postures, enables configuration governance, and detects vulnerabilities and threats. Use the DevSecOps templates and enable the Security and Compliance Center integration to store evidence and monitor deployments.

For more information, see [IBM Cloud Security and Compliance Center](https://www.ibm.com/cloud/security-and-compliance-center)

Every CI Pipeline run that is performed creates evidence that is related to the run. The evidence is made up of the details of the operations that are being performed and the evidence is written to an evidence locker. The evidence locker is a Git repository but a IBM Cloud® Object Storage bucket can also be used if you require the evidence to be more durable (for example, longer TTL). Similarly, the CD pipeline collects evidence that provides an audit trail of the tasks it performs pertaining to acceptance tests, change requests, and so forth. Most importantly, the compliance pipeline writes an aggregation of all the individual evidence fragments to an evidence summary when a deployment completes.

The evidence summary format that is used by the DevSecOps CI/CD templates is published [here](https://cloud.ibm.com/docs/devsecops?topic=devsecops-devsecops-evidencecd-devsecops-evidence-summary). This summary is the link between the DevSecOps CD toolchain and the Security and Compliance Center. Every entry in the evidence summary maps to what is termed a fact in Security and Compliance Center. Figure 1 shows a snapshot of an evidence summary that is produced in the CD pipeline.

Zoom

![CD evidence summary task](https://cloud.ibm.com/docs-content/v1/content/5eb88a2575e30e316dbb83829c7f4c8b1ddbc67c/devsecops/images/cd-devsecops-evidence-summary.png)

Figure 1. CD evidence summary task

Figure 2 shows the evidence summary information that is presented as Facts when the same toolchain is selected in the Security and Compliance Center UI
Using modules in public and private repos 

Schematics and Terraform support downloading Terraform templates and modules from a variety of repository types: Terraform Registry, GitHub, GitLab, S3/COS buckets, IBM Catalog, Artifactory etc. See [Module Sources](https://developer.hashicorp.com/terraform/language/modules/sourcesmodules-in-package-sub-directories) in the Terraform documentation.

When using Schematics, the downloading of Terraform templates and modules prior to performing a Terraform Plan or Apply operation is a two step process. At workspace create time, Schematics will clone only the repository containing your template and any embedded modules in sub-folders. Any modules referenced using the module source parameter are not downloaded at workspace create time. Credentials to access the templates/configs in private repositories, must be passed to Schematics at workspace create time.

Modules referenced with the source parameter are downloaded during the terraform init phase of a plan or apply operation. The terraform init command parses the template files and downloads any modules from the repo's referenced by the source field. Modules residing in private repositories require additional credentials to be passed to Terraform. These credentials are defined and passed separately to those used by Schematics.

To download modules from a private Git repository, an IBM Cloud catalog, or any other repository, Terraform supports the use of a [netrc](https://everything.curl.dev/usingcurl/netrc) configuration to pass any required access id's and tokens.



Supported Git repositories

 Repository <br> Template <br>Public repo Template <br>Private repo Module <br>Public repo Module <br>private repo Comment <br> 

 GitHub Yes Git token - 1 Yes Git token - 2 
 GitLab Yes Git token - 1 Yes Git token - 2 
 IBM GitLab Yes Git token - 1 Yes Git token - 2 
 Terraform.io No No Yes NA 





1. Git token defined at workspace create time
2. Git token defined using netrc
Cloud Type ibm_ctype The cloud type can be public, dedicated, or local. Valid value is public. 
 Location ibm_location The location of the monitored resource, which can be a region, data center, or global. [Valid regions](https://cloud.ibm.com/docs/metrics-router?topic=metrics-router-regions). 
 Resource ibm_resource The resource that is measured by the service, which is typically an identifying name or GUID. 
 Resource Name ibm_resource_name The name of the resource that is measured by the service. 
 Resource Type ibm_resource_type The type of the resource that is measured by the service. Valid value is target. 
 Scope ibm_scope The scope is the account where the metric is generated. 
 Service name ibm_service_name The name of the service that is generating this metric. Valid value is metrics-router. 







 More attributes 

The following attributes are available for one or more attributes described in the previous tables. See the individual metrics for options.



Table 4. Other attributes

 Attribute Attribute Name Attribute Description Valid values 

 Target Type ibm_metrics_router_target_type The resource group where the service instance was created. Valid value is sysdig-monitor. 









 Dashboards templates 

The following table lists the pre-defined dashboard templates that are available to monitor IBM Cloud Metrics Routing:



Table 5. Pre-defined dashboard templates

 Dashboard Name Dashboard description 

 Metrics Routing - Overview Use this dashboard to monitor IBM Cloud Metrics Routing data point processing and target modes.
Selection of template and input file versions is defined on initial creation of the blueprint config for the environment in Schematics. Review the section on [blueprint versioning](https://cloud.ibm.com/docs/schematics?topic=schematics-blueprint-versioning). The omission of template and input file version information at config create time allows for a ‘relaxed’ usage mode. Where any changes to the template or input files are automatically pulled in if Schematics detects a change in the source repository.







 Customizing environments with inputs 

As described, blueprint templates support environment customization and versioning at multiple levels. User customizable values are defined as inputs at the blueprint template level.

The selection of inputs is determined in the following precedence order, lower to higher:



* Blueprint template



* From VCS, version controlled by Git tag, branch or un-versioned (pull-latest).
* Input definition: Default value or no value specified in template YAML file.
* Usage: Opinionated implementation of an application architecture.



* Blueprint input files



* From VCS, version controlled by Git tag, branch or un-versioned (pull-latest).
* Value optionally specified in inputs YAML file.
* Usage: Environment scaling, region, network and security configuration.



* Dynamic (override) inputs



* User provided via CLI or UI, un-versioned
* Value optionally specified
* Usage: SSH and API keys





Inputs that are not satisfied at any level will result in an error at blueprint config create time.





 Using versioned inputs 

Blueprints uses versioning of templates and input files to manage infrastructure change. Review the section on [blueprint versioning](https://cloud.ibm.com/docs/schematics?topic=schematics-blueprint-versioning) to understand how Blueprints versioning can be used.

The use of input versioning is summarized in the table.



Usage of input version

 Blueprint layer Development and unregulated environments Regulated environments 

 Dynamic (override) inputs Any input, no version control.
Each output is identified by a key-value pair.

Refer to the section [modules inputs](https://cloud.ibm.com/docs/schematics?topic=schematics-bp-template-schema-yamlbp-modules-inputs-value) for details on defining the $module interpolation to pass output values.

Example

outputs:
- name: schematics-service-url
value: $module.bp-vsi-vpc-app.outputs.app_url





 settings 

Type: list

Default: []

Optional

Environment variables are used to modify the behavior and execution of Terraform and Ansible without modifying the IaC code itself. Refer to the section [Using environment variables with workspaces](https://cloud.ibm.com/docs/schematics?topic=schematics-set-parallelism) for more information about the environment variables that can be passed to configure Terraform runtime behavior in Schematics.



Settings defined at this level apply to all modules. Additionally env-var settings can also be set at the module level.



 settings.TF_VERSION 

Type: number

Optional

Blueprint templates set the Terraform version to be used at module execution time based on the value of TF_Version. This value can be used to pin the version of Terraform used by Schematics to remain compatible with the template supported version. Updating this value will change the Terraform version that is used on the next execution.

When not specified the required Terraform version is determined by inspecting the TF module code for a required_version definition. Based on the specified version or range of versions, Blueprints will used the most recent version of Terraform supported by Schematics.

Options: Terraform version in SemVer format

Example

settings: Master settings for all modules
- name: TF_VERSION
value: 1.0





 settings.TF_LOGS 

Type: string

Optional

Configure the Terraform logging level for all modules.