Build Your Own Lint Rules for Terraform with Shisho
Tags: terraform, security, devops, codequality
Published: 2021-08-23 15:44:38

**tl;dr: Shisho is an open-source static code analyzer that lets you build your own lint rules for Terraform codes. You can find and refactor specific code patterns easily with a handy configuration language.**

## Building Linter / Static Analyzer is Too Hard

Every developer wants to avoid embedding issues in their software, while finding issues tends to be boring. So here's where a linter / a static analyzer come: they will be a great supporter of you and your team by detecting common bugs with pre-defined rules *before* the bugs are shipped to the world.

Sometimes, you will want to enforce custom rules for your code to standardize best practices specific to your team. When you want to prevent your team members from using `uniform_bucket_level_access = true` in  `google_storage_bucket` resources like the following snippet, for example, you hope there's a flexible linter that lets you add custom rules quickly:

```tf=
resource "google_storage_bucket" "test" {
  project  = var.project
  name     = "${var.project}-test"
  location = var.location
  uniform_bucket_level_access = true
  force_destroy               = true
}
```

However, **adding and maintaining custom rules is quite hard**! You need to learn how to write custom rules for *each* programming language your team use, although different programming languages have different linters or analyzers, with different DSLs and APIs. This difficulty is one of the severe problems of standard linters / static analyzers. 


## Shisho: A Customizable Static Code Analyzer

Shisho, a lightweight static code analyzer, will help you build custom lint rules for your codebase. I'll explain what and how it is.

{% github flatt-security/shisho %}

### Find Codes


First of all, Shisho enables us to run AST-aware code search over your code. Here's an example command which finds the occurence of `uniform_bucket_level_access = true` inside `google_storage_bucket` resource:

```shell=
docker run -i -v $(pwd):/workspace ghcr.io/flatt-security/shisho-cli:latest find "
resource \"google_storage_bucket\" :[_] {
  :[...]
  uniform_bucket_level_access = true
  :[...]
}
" --lang hcl ./code.tf
```

The command will make the following outputs in your console:

 ![nYvlJB6](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bgpx0xaqkxejfqzmlghj.png)

Here `:[_]` is an **anonymous metavariable**, which matches an arbitrary single node in AST (like a function call, identifier, and so on). Similarly, `:[...]` is an **anonymous ellipsis metavariable**, which matches zero or more nodes in AST. These operators are something like capture groups in regular expressions. They let you search over your code in a structured but flexible manner.

You can also define a *rule*, which includes a pattern and the explaination for it. The following YAML snippet is an example of rules describing the use of `uniform_bucket_level_access` is prohibited:

```yaml=
version: "1"
rules:
  - id: sample-policy
    language: hcl
    pattern: |
      resource "google_storage_bucket" :[_] {
        :[...X]
        uniform_bucket_level_access = true
        :[...Y]
      }
    message: |
      Our team policy prohibits the use of uniform bucket-level access.
```

You can find patterns by executing `shisho find path/to/rule.yaml path/to/search` command, resulting in the following outputs:

![Screenshot from 2021-08-23 19-08-44](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vque1zjoxh4rd8cv8pmb.png) 


This is how Shisho makes it possible to build your own lint rules for Terraform codes. You can use Shisho in the CI pipeline with your own rules, let alone your local machine. Please see [Learn Shisho](https://docs.shisho.dev/learn-shisho/03-constraint) for further details.

### Refactor Codes

Additionally, Shisho rules can include how detected code patterns should be fixed. The following YAML snippet describes a custom lint rule that suggests the use of `uniform_bucket_level_access = true` should be deleted:

```yaml=
version: "1"
rules:
  - id: sample-policy
    language: hcl
    pattern: |
      resource "google_storage_bucket" :[NAME] {
        :[...X]
        uniform_bucket_level_access = true
        :[...Y]
      }
    message: |
      Our team policy prohibits use of uniform bucket-level access.
    rewrite: |
      resource "google_storage_bucket" :[NAME] {
        :[X]
        :[Y]
      }
```

Once this rule is run over your codes and the use of `uniform_bucket_level_access = true` is detected, Shisho suggests changes following the rule's `rewrite` section like:

![Screenshot from 2021-08-23 19-07-30](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4fnmb3zlkc84hup3977q.png)


## Usecases

You can use Shisho **for standardizing your codebase**. In addition, it could be a means of conducting **"security-as-code"** or **"policy-as-code"**! 

For instance, when you want to keep your team's [EBS volumes](https://aws.amazon.com/ebs/) encrypted, you can define a rule as follows:


```yaml=
version: '1'
rules:
  - id: 'unencrypted-ebs-volume'
    language: hcl
    message: |
      There was unencrypted EBS module.
    pattern: |
      resource  "aws_ebs_volume" :[NAME] {
        :[...X]
      }
    constraints:
      - target: X
        should: not-match
        pattern: |
          encrypted = true
    rewrite: |
      resource "aws_ebs_volume" :[NAME] {
        :[X]
        encrypted = true
      }
```

When you want your colleagues to follow the naming convention for resources, the following rule will work well:

```yaml=
version: "1"
rules:
  - id: "invalid-resource-name"
    language: hcl
    message: |
      A resource was named badly.
    pattern: |
      resource  :[_] :[NAME] {
        :[...]
      }
    constraints:
      - target: NAME
        should: not-match-regex
        pattern: '"team1-.*"'

```

The rule will report the result like:

![Screenshot from 2021-08-24 00-34-46](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tbrr64fr1piubmwz6ty4.png) 

## Why Shisho?

["Modern Static Analysis: how the best tools empower creativity"](https://devd.me/log/posts/static-analysis/) explains that good code analyzers or linters are often *interoperable, moldable, efficient, and community-driven* and that [Semgrep](https://semgrep.dev/) works well from these viewpoints. Semgrep is also grep-like (or sed-like) software that lets us find bugs with useful DSLs. 

As for Shisho, it is at least interoperable (since it's open-sourced), moldable (though some efforts are needed; see issue [#7](https://github.com/flatt-security/shisho/issues/7)). Moreover, **Shisho is surprisingly efficient**! Here's the result of a micro-benchmark of [Semgrep](https://semgrep.dev/), [Comby](https://comby.dev/) (a similar tool), and Shisho[^env]:


| Tool | Time | Command |
| --- | --- | --- |
| Comby (1.7.0) | 263.1 ms | `time comby 'len(...)' '' parser.go -match-only &> /dev/null` |
| Semgrep (0.62.0) |  530.0ms  | `time semgrep -e 'len(...)' --lang=go parser.go &> /dev/null` |
| **Shisho (0.1.2-alpha.2)** | **22.8ms** | `time shisho find 'len(:[...])' --lang=go parser.go &> /dev/null` |

In fact, Shisho aims to refine the existing tools and to make it more feasible to run for large projects. You can use Shisho for your monorepo without hesitation. For your information, this speed is supported by [Rust](https://rust-lang.org) [^other].

On the other hand, it's true that Shisho lacks some features of Semgrep and Comby. For instance, Semgrep has [a feature to match patterns with type information](https://semgrep.dev/docs/writing-rules/pattern-syntax/#typed-metavariables) while Shisho doesn't. Semgrep also has [Semgrep Registry](https://semgrep.dev/explore), in which you can share your own lint rules for the worldwide community. Now I'm making efforts to design and implement these features. Stay tuned!

[^other]: Both Semgrep and Comby is written in OCaml.

[^env]: Time is the average of 20 consecutive command executions. The measurement was run on Ubuntu 20.04.2 LTS with AMD Ryzen 5 3600 / 64GB RAM. The scan target was [parser.go](https://sourcegraph.com/github.com/golang/go@5a267c840ae16c1cc7352caa14da5f500d03d338/-/blob/src/go/parser/parser.go).

## Now What?

This article explained the usage of Shisho for Terraform codes, but Shisho is extending other language supports! Especially Dockerfile support will be shipped soon. You can follow [@y0n3uchy](https://twitter.com/y0n3uchy) to see the news on Shisho and star [our GitHub project](https://github.com/flatt-security/shisho) to encourage us :-)

Additionally, I'll release a SaaS which supports your Terraform development workflows with this engine. See [https://shisho.dev/](https://shisho.dev/?utm_source=devto&utm_medium=referral&utm_campaign=post20210824) for further details.