Modern Trunk-Based Development
Tags: github, programming, devops, productivity
Published: 2022-10-17 13:35:43

<!-- wp:paragraph {"className":"excerpt"} -->
<p class="excerpt"><em>For the old guard software developers, trunk-based development is the way to go. For younger developers, used to GitHub pull requests, it is unlikely that they know what trunk-based means. This article shows how we use automation to combine TBD principles with pull requests and get the best of both worlds.</em></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Trunk-based development (TBD) is a software development methodology where the developers continuously merge changes into a single long-lived branch.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Two benefits of TBD stand out:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>TBD avoids branches which save developers’s time from merge hell situations;</li><li>TBD fosters small atomic commits which are essential to build loosely-coupled systems where a developer has the ability to change one part of the system without impacting the rest.</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>For engineering leaders, TBD seems a no brainer and <a href="https://www.devops-research.com/research.html" target="_blank" rel="noreferrer noopener">Google’s State of DevOps Report</a> has consistently associated TBD with a higher software delivery performance.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>At the core, TBD fosters an high-trust and low-blame cultures which tend to have higher organizational performance.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>However, one of the surprises of the <a rel="noreferrer noopener" href="https://cloud.google.com/blog/products/devops-sre/dora-2022-accelerate-state-of-devops-report-now-out" target="_blank">2022 State of DevOps Report</a> is that trunk capabilities had a negative impact on software delivery performance.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>What changed?</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>According to the report, the number of respondents in the study with 16+ years of experience decrease from 40% to 13%.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>Developer experience is a key factor to implement and benefit from TBD practices.</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The report mentions that this difference is <em>likely due to the additional practices required to successfully implement Trunk-based development.</em> These additional practices can be resumed to a robust Continuous Integration (CI) mechanism that ensures that the trunk/main branch is never broken.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>A different perspective</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Over the past years, we have also seen a decline of interest in TBD when we interviewed developers with less than 10 years of experience. However, we observed that the main reason is not related with the lack of a robust CI but with the adoption of git-based version control systems like GitHub, GitLab and Bitbucket.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>These version control systems naturally force developers to use feature branches and asynchronous collaboration via pull requests. There are many junior developers that never heard of TBD.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We also found that teams that rely heavily on feature branches and pull requests typically have strong CI mechanisms to ensure that the pull request merge does not break the long lived branches. In fact, GitHub Actions or GitLab Pipelines simplify the process to quickly get a basic automated CI system up and running.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Pull requests as a git merge blocking mechanism is necessary in low-trust environments found in open source projects where you have contributions from untrusted parties.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Teams practicing TBD understand that it doesn’t make sense to generalize this reality to a team of software developers where high-trust, low-blame culture is key.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>The best of both worlds</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>We see pull requests as a useful mechanism to automate, document and audit three tasks:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>Execute a set of automated tasks, e.g. build, tests, code analysis;</li><li>Code review;</li><li>Git merge approval.</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>We believe the adoption of pull requests will increase in the future for security reasons. For example, a <a rel="noreferrer noopener" href="https://slsa.dev/spec/v0.1/requirements#two-person-reviewed" target="_blank">two-person reviewed</a> requirement must be fulfilled for a team to reach level 4 of the <a rel="noreferrer noopener" href="https://slsa.dev" target="_blank">Supply chain Levels for Software Artifacts</a> standard.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We found that teams doing TBD find it hard to adopt pull requests because it creates a lot of friction in their processes.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>Reviewpad started from the observation that not every pull request is the same. </strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>This fits nicely for teams updating their TBD practices to use pull requests.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>For example, a pull request that introduces a database migration is very different from a pull request that performs a minor refactoring of the code. In the first case, it makes sense to have a more in-depth code review and testing procedures to ensure the correctness of the migration.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>At the moment, you can enforce pull request workflows to deal with this separation of concerns using the <a rel="noreferrer noopener" href="https://github.com/reviewpad/action" target="_blank">GitHub Reviewpad Action</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To install it in your repository you just need to add two files:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>The GitHub Action YAML file that lives in the <code>.github/workflows</code> directory.</li><li>The Reviewpad YAML file that specifies the pull request rules and automation workflows. This file is read by the action to understand which actions to perform. By default, the action assumes that this file is called <code>reviewpad.yml</code> and lives in the root of the repository.</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>The pull request workflows are specified in YAML. The major advantage with respect to other GitHub Actions is that it comes with a domain specific language (DSL) to quickly access contextual information in the pull request and perform actions.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Consider the following configuration:</p>
<!-- /wp:paragraph -->


```yml
workflows:
  - name: label-pull-request-with-size
    if:
      - rule: '$size() < 50'
        extra-actions:
          - '$addLabel("small")'
      - rule: '$size() >= 50 && $size() < 150'
        extra-actions:
          - '$addLabel("medium")'
      - rule: '$size() >= 150'
        extra-actions:
          - '$addLabel("large")'
```

<!-- wp:paragraph -->
<p>It specifies a Reviewpad workflow to automatically add a label to the pull request based on the size of changes (measured in the number of lines added and deleted).</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We currently have over <a rel="noreferrer noopener" href="https://docs.reviewpad.com/guides/built-ins" target="_blank">65 built-ins</a> that allow you to manage labels, comments, reviews and merge pull requests.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Reviewpad’s trunk-based development</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Based on TBD practices, we use Reviewpad to auto-merge pull requests in our main open-source repository <a rel="noreferrer noopener" href="https://www.github.com/reviewpad/reviewpad" target="_blank">reviewpad/reviewpad</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>For us, the first step was to define which scenarios it makes sense to auto-merge pull requests. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Since the repository is public, we need to restrict auto-merges to pull requests authored by owners of the project. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Additionally, to enable this auto-merge feature, the pull requests should be directly flagged by the owners as <code>Ship</code> pull requests. The only requirement was that this flag mechanism should be automatic when opening the pull request.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Finally, we established two criteria for auto-merges <code>Ship</code> pull requests opened by owners:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>Changes to Markdown files should be automatically merged; additionally, such pull requests shouldn't waste resources such as running the CI process;</li><li>Other changes should be automatically merged if the CI process is successful.</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Regardless of the author of the changes, we also wanted to avoid running the CI process unnecessarily. So, pull requests that only modify Markdown files should not trigger the CI process.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>The implementation</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>In this section, we show how we get Reviewpad to automate the merges.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The first step was to find a flexible mechanism for developers to open a pull request and flag them as <code>Ship</code>. We heavily rely on the <a rel="noreferrer noopener" href="https://cli.github.com/" target="_blank">GitHub CLI</a> to open pull requests and we open them as fast as possible. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>So we decided to use pull request templates to allow developers to select which approval/merge mode they want. This is part of our organization <a rel="noreferrer noopener" href="https://raw.githubusercontent.com/reviewpad/.github/main/.github/PULL_REQUEST_TEMPLATE.md" target="_blank">pull request template</a>:</p>
<!-- /wp:paragraph -->


```md
## Description

... for the entire file, please follow the link above :)

## Code review and merge strategy (ship/show/ask) 

- [ ] Ship: this pull request can be automatically merged and does not require code review
- [ ] Show: this pull request can be auto-merged and code review should be done post merge
- [ ] Ask: this pull request requires a code review before merge
```

<!-- wp:paragraph -->
<p>In the final section of the template, authors of pull request select this mode.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>As soon as the pull request is opened, the developer shouldn't have to do anything else.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We achieved this with two GitHub Actions:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li><a style="font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Oxygen-Sans, Ubuntu, Cantarell, &quot;Helvetica Neue&quot;, sans-serif;" href="https://github.com/reviewpad/reviewpad/blob/main/.github/workflows/reviewpad.yml">reviewpad.yml</a> that runs Reviewpad;</li><li><a rel="noreferrer noopener" href="https://github.com/reviewpad/reviewpad/blob/main/.github/workflows/pull_request_build.yml" target="_blank">pull_request_build.yml</a> that runs the CI with a job called <code>pr-build</code>.</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>As soon as a pull request is created, the Reviewpad action will be triggered and reads the <a rel="noreferrer noopener" href="https://github.com/reviewpad/reviewpad/blob/main/reviewpad.yml" target="_blank">configuration file</a>. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We now highlight the relevant sections of the specification.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":4} -->
<h4>Check for ship pull requests authored by owners</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>The first step is to specify a rule that will be true if the pull request has been selected as <code>Ship</code> and is authored by an owner.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We start by defining, with the Reviewpad DSL, the group <code>owners</code> as the list of GitHub usernames <code>marcelosousa</code> and <code>ferreiratiago</code>. </p>
<!-- /wp:paragraph -->

```yml
groups:
  - name: owners
    spec: '["marcelosousa", "ferreiratiago"]'
```

<!-- wp:paragraph -->
<p>Then, we define a rule that checks if the pull request contains in the description the section of the <code>Ship</code> mode. For this we can use the Reviewpad built-ins <code>$contains</code> and <code>$description</code>.</p>
<!-- /wp:paragraph -->

```yml
rules:
  ...
  - name: ship-mode
    spec: '$contains($description(), "[x] Ship:")'
```

<!-- wp:paragraph -->
<p>We can use this rule to define the rule <code>ship-authored-by-owners</code> that checks if the pull request is both authored by an owner and was flagged with <code>Ship</code>.</p>
<!-- /wp:paragraph -->

```yml
rules:
  ...
  - name: ship-authored-by-owners
    spec: '$rule("ship-mode") && $isElementOf($author(), $group("owners"))'
```

<!-- wp:heading {"level":4} -->
<h4>Check for pull requests that only modify Markdown files</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>The first auto-merge criteria is to auto-merge pull requests that only modify Markdown files.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To do this, we define a rule <code>changes-are-in-markdown</code> that checks if the files modified only contain <code>.md</code> extensions.</p>
<!-- /wp:paragraph -->

```yml
rules:
  ...
  - name: changes-are-in-markdown
    spec: '$hasFileExtensions([".md"])'
```

<!-- wp:paragraph -->
<p>Finally, we define the rule <code>ship-markdown-changes</code> that chains the previous rules and if true, should trigger the merge action.</p>
<!-- /wp:paragraph -->

```yml
rules:
  ...
  - name: ship-markdown-changes
    spec: '$rule("ship-authored-by-owners") && $rule("changes-are-in-markdown")'
```

<!-- wp:heading {"level":4} -->
<h4>Check for pull requests that have successful CI</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>The other criteria for the auto-merge involves the following steps:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>Trigger the CI action if the changes are sensitive;</li><li>Check if the CI action was successful.</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>For the first step, we conservatively trigger the CI process if the changes involve other file extensions. We define it by simply negating the result of the rule <code>changes-are-in-markdown</code>.</p>
<!-- /wp:paragraph -->

```yml
rules:
  ...
  - name: changes-should-be-built
    spec: '!$rule("changes-are-in-markdown")'
```

<!-- wp:paragraph -->
<p>We then combine this rule to define a Reviewpad workflow that adds a special label <code>run-build</code>. </p>
<!-- /wp:paragraph -->

```yml
workflows:
  - name: add-label-for-build
    if:
      - rule: changes-should-be-built
    then:
      - '$addLabel("run-build")'
```

<!-- wp:paragraph -->
<p>The event of adding this label triggers the CI run in <a rel="noreferrer noopener" href="https://github.com/reviewpad/reviewpad/blob/main/.github/workflows/pull_request_build.yml" target="_blank">pull_request_build.yml</a> with a job called <code>pr-build</code>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We leverage the fact that you can trigger GitHub Actions based on other GitHub Action workflows. This allows us to trigger Reviewpad once the build finishes. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Reviewpad comes out of the box with built-in functions that allow us to check the result of workflow jobs. So, we defined a rule <code>ci-is-green</code> that checks if the <code>pr-build</code> job succeded.</p>
<!-- /wp:paragraph -->

```yml
rules:
  ...
  - name: ci-is-green
    spec: '$workflowStatus("pr-build") == "success"'
```

<!-- wp:paragraph -->
<p>Finally, we can combine the rules to define the second auto-merge criteria:</p>
<!-- /wp:paragraph -->

```yml
rules:
  ...
  - name: auto-merge-authored-by-owners-with-ship-and-green-ci
    spec: '$rule("ship-authored-by-owners") && $rule("ci-is-green")'
```

<!-- wp:heading {"level":4} -->
<h4>Auto-merge pull requests</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Once we have the rules that define the auto-merge criteria of interest, we just need to use them in a workflow that will trigger the <code>$merge</code> built-in. This built-in receives the merge method - we decided to use <code>rebase</code> to preserve the commits in the pull request.</p>
<!-- /wp:paragraph -->

```yml
workflows:
  ...
  - name: auto-merge-owner-pull-requests
    if:
      - rule: auto-merge-authored-by-owners-with-ship-and-green-ci
      - rule: ship-markdown-changes
    then:
      - '$merge("rebase")'
```

<!-- wp:heading {"level":3} -->
<h3>See it in action</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>When an owner opens a <code>Ship</code> pull request, if the changes are only in Markdown files, the pull request will be automatically merged without wasting the resources to run the CI. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>See the following <a rel="noreferrer noopener" href="https://github.com/reviewpad/reviewpad/pull/388" target="_blank">pull request</a> for an example.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>If the changes are in other file extensions, Reviewpad will add the <code>run-build</code> label and this label will trigger the CI. Once the CI job is completed, the Reviewpad action will run again and now validate the result of the CI.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>This way we can automatically merge pull requests with the guarantee that the CI process was successful. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>See the <a rel="noreferrer noopener" href="https://github.com/reviewpad/reviewpad/pull/390" target="_blank">pull request</a> for an example.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>That's it! This is how we ensure that some pull requests are systematically merged without manual code reviews with the guarantee that the CI process is green.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>If you are curious to see the whole process in action, check out our <a href="https://github.com/reviewpad/reviewpad" target="_blank" rel="noreferrer noopener">GitHub repository</a>. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>PS. Please give us feedback on <a rel="noreferrer noopener" href="http://reviewpad.com/discord" target="_blank">Discord</a> and support our project on <a rel="noreferrer noopener" href="https://github.com/reviewpad/action" target="_blank">GitHub</a> with a star!</p>
<!-- /wp:paragraph -->