Introduction to Terraform Enterprise and Jenkins
By: Jim Ray
Tags: Jenkins, Hashicorp, Hashicorp Terraform

In this post, I will provide background information on using CI/CD with Terraform Enterprise, decision points to select the right Terraform Enterprise package, and finally a detailed walk through of an example Jenkins pipeline using Terraform Enterprise (TFE).

Terraform is the ubiquitous choice for infrastructure and cloud services provisioning. Organizations have built mature processes around the creation, review, and use of Terraform code to manage their infrastructure. One common integration point for Terraform is a CI/CD tool such as Bamboo, Circle CI, or Jenkins. The latter part of this post will focus on Jenkins, but other than pipeline code, the guidance is generally the same across other CI/CD tools.

CI/CD Interacting with TFE

The Tao of HashiCorp has 8 principles that are used to guide the company’s vision, roadmap, and product design. The first principle is Workflows, not technology as we focus on the end goal and workflow, not the underlying technology — because technology changes, but end goals stay the same. This workflow-first principle is very evident with TFE because you have the flexibility of using UI/VCS webhook, CLI, and REST API driven workflows.

high-level components of a basic CI/CD pipeline with infrastructure provisioning with Terraform

How are you using Terraform with your CI/CD today? Are there established processes or controls that need to be respected when considering TFE? These questions and more are used to establish the adoption pattern for TFE relative to your existing pipeline.

The actual migration can be accomplished with minimal refactoring of your pipeline code. This is the approach that most organizations take when implementing TFE where the goal is to focus on extracting value from enterprise features without disrupting established workflows. For example, you have use cases being lead by security and governance for Sentinel Policy as Code and TFE Audit capabilities; the existing workflow and pipeline code should not need to be refactored completely though there will be some minor modifications.

TFE Background

TFE provides features to support the three main teams within IT often interacting with infrastructure and cloud services; Development, Operations, and Security. A sample mapping of enterprise features by team includes API access to Terraform for Development; Team Management, RBAC, and Multi-tenancy for Operations; and Sentinel Policy as Code for Security.

HashiCorp has two consumption models of Terraform that can be purchased with annual or multi-year subscriptions: SaaS and private installation. Recently the offerings were rebranded as Terraform Cloud (SaaS) and Terraform Enterprise (private installation). Terraform Cloud is a viable option for teams looking for enterprise features without a requirement for a single-tenant, self-provisioned implementation. Terraform Enterprise (TFE) is most often deployed by enterprises with compliance, security, and/or scale requirements.

comparison of Terraform packaging and offerings

Which Terraform package is right for you? If you are just starting your journey with Infrastructure as Code (IaC) and have not encountered collaboration or governance requirements, then Open Source is probably fine for the next 6–12 months as you mature your IaC practice. But if you are in search of collaboration or governance features you should be evaluating Terraform Cloud or Terraform Enterprise.

TFE has a few hierarchy constructs that are relevant to your pipeline configuration. The top-level is an organization which can be used as a boundary between business units, environment stages, or other multi-tenancy criteria that may be important to your enterprise.

Within these organizations workspaces are where your Terraform code will be executed and will contain your state, variables, and policy sets will be applied. VCS integration and Sentinel policies defined at the organization level are applied to some or all of these workspaces. Finally, users are assigned to teams that are created within the organization to provide one or more levels of access. Team or user tokens are generated and are used to interact with TFE via the API.

OK, Enough Talking…

I’m going to present a basic Jenkins job file that will demonstrate a CLI-driven workflow by interacting with TFE using a Remote Backend file. The Remote Backend file tells Terraform OSS to use a TFE server to execute the Run remotely. Variables used for the Run and associated state file will reside within the TFE workspace as mentioned earlier. The files referenced in the walkthrough are in this repo, TFE-Jenkins, along with bootstrap scripts.

The next sections are a step-by-step guide to executing a Jenkins job against Terraform Enterprise. Assumptions are that you have a Jenkins instance running on Linux, Terraform Cloud or Enterprise instance, access to a VCS repo, and around 15 minutes. I’ll start with an overview of the steps and then go through TFE and Jenkins configuration.

Overview of High-Level Steps

Terraform

Create a Workspace in TFE

Set the Workspace to auto-apply (to allow Run to auto-apply if it passes Sentinel checks)

Specify the desired TF binary version for the Workspace

Set cloud credentials as Workspace variables (AWS in this example)

Create repo as source of the Workspace’s Terraform code

Seed the code repo with a remote backend file (backend.tf)

Jenkins

Configure .terraformrc file on server

Define job

Run job

TFE Workspace Prep

Navigate in TFE: Workspaces > Jenkins Target Workspace > Settings > General

Apply Method = Auto Apply

Terraform Version = 0.11.14 (note, if you are using 0.12 compliant code, then select the latest release)

Save Settings

Workspace General Settings within TFE UI

Navigate in TFE: Workspaces > Select Jenkins Target Workspace > Variables

Environment Variables

Create the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables

Use the “sensitive” feature of TFE to obfuscate the value in the UI

Click Save Variable

Create the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables using secure variables feature of TFE

Terraform Code Prep

Store main.tf and backend.tf in a repo accessible to the Jenkins server, the repo URL will be specified in the job file as the GIT_REPO value

sample backend.tf file; populate the hostname, organization, and workspace name fields. note if you using a private Terraform Enterprise instance (not Terraform Cloud), the hostname will be your unique fully-qualified domain name.

Jenkins OS Setup

TFE .terraformrc file must be created and configured on Linux-based Jenkins server (see link for notes on configuring terraformrc for Windows)

If not, you will receive “Error configuring the backend “remote”: required token could not be found” in the Stage Log

Write file: sudo nano /var/lib/jenkins/.terraformrc

Install unzip: sudo apt install unzip -y

Jenkins Job File

Update GIT_REPO value to reflect your target environment

Example files are available at https://github.com/raygj/tfe-jenkins

simple Jenkins pipeline with 3 stages

Pipeline Setup from Jenkins UI

http://< IP or DNS of your server>:8080

New Item

<some name>

Pipeline

OK

Description

Pipeline

Definition = Pipeline Script

Script = paste Jenkins file from previous section

Leave all other items default/unchecked

Save

Pipeline Run from Jenkins UI

Jenkins > Build Dashboard

Click on “clock” to schedule a run of your pipeline

Click on the name to see the stages being executed

Jenkins Stage View of our Run

Success from Jenkins UI

Stage Log from Terraform Apply Stage

Success from TFE UI

Output of the Workspace run detail in Terraform Enterprise UI; note the Policy check passed — this is Sentinel Policy as Code running between the Plan and Apply stages

Conclusion

Using the components from this post you have what you need to interact with Terraform Enterprise using a CI/CD tool via Terraform OSS binary. A migration to Terraform Enterprise can be done while respecting your workflow and investments in accompanying pipeline components that may be north or southbound of Terraform.

The net result is that your organization will be positioned to take advantage of Terraform Enterprise features that could unlock new value or opportunities to increase the collaboration, governance, and stability of your infrastructure.