Automating Maven Releases with CircleCI
Tags: devops, bash, showdev
Published: 2018-05-27 02:53:02

---
title: Automating Maven Releases with CircleCI
published: true
tags: devops,bash,showdev
canonical_url: https://di.nmfay.com/circle-maven-versions
---

Maven's probably the only all-in-one build tool I've ever really _appreciated_. I'll probably come to like `make` eventually and cement my status as old-before-her-time \*nix crone, but I haven't had a reason to really dig into it yet so Maven it is. And I'm back at a mostly-Java shop, so let's have some fun!

This week's goal: automating releases from our CircleCI instance. Sounds simple enough, right? Bump the version, cut a tag, publish. How hard could it be?

Well, first off, we're using [git-flow](https://nvie.com/posts/a-successful-git-branching-model/), or at least we're preserving `master` for releases and working off a separate `verify` branch. Budget git-flow, if you will. That's one complication, since the release has to be tagged on `master` but `verify` also needs to be updated so the two don't diverge.

If you're familiar with Maven you may already have guessed the second complication. It's trickier. Maven doesn't work in nice, straightforward [semver](https://semver.org): Maven accepts several different versioning schemes and has a special `SNAPSHOT` qualifier for non-release builds. If you're working towards a 1.0 release, your version number is 1.0-SNAPSHOT. After you cut the release, you resume development with 1.1-SNAPSHOT (or 2.0-SNAPSHOT if it really needs a rework already). And so on. It's not _meant_ to be automated, because releases are a _big deal_ in the Maven world and you're expected to have a plan for what you're going to do next instead of reacting to whether you fixed bugs, introduced features, or broke compatibility. And honestly, there are some compelling arguments for doing it this way.

I'm not going to go into them because I'm one half of the software team by myself and they're less applicable working on proprietary stuff at this scale. So let's get to automating!

## Workflow

We're using Circle v2 and its workflow feature to organize the build. Every branch gets built: `verify` and `master` get deployed to Artifactory, while `release` triggers its own job, which latter is the linchpin of the whole structure.

```
workflows:
  version: 2
  build-and-deploy:
    jobs:
      - build
      - deploy:
          requires:
            - build
          filters:
            branches:
              only: /^(master|verify)$/
      - release:
          requires:
            - build
          filters:
            branches:
              only: /^release$/
```

## Just Build

I'll be honest, I copied & pasted most of this job definition right out of the docs:

```
steps:
  - checkout
  - restore_cache:
      keys:
      - v1-dependencies-{{ checksum "pom.xml" }}
      # fallback to using the latest cache if no exact match is found
      - v1-dependencies-
  - run: mvn clean install
  - save_cache:
      paths:
        - ~/.m2
      key: v1-dependencies-{{ checksum "pom.xml" }}
  - persist_to_workspace:
      <<: *source
```

We're caching our dependencies because that's how one does it; `mvn clean install` is likely overkill (we probably don't need to bother with installing the dependency to the local Maven cache) but it builds and runs our tests and generates the artifact. The only really interesting part here is that we're persisting the important files to a workspace so we can recover it later -- `*source` refers to another YAML block with a `root` string and list of `paths`.

## And Deploy

```
steps:
  - attach_workspace:
      at: .
  - run:
      name: Deploy to Artifactory
      command: mvn deploy
```

Here's where we use that workspace. Whenever this job runs, it'll reattach the file structure we saved from the build job. `mvn deploy` still runs all the intermediary lifecycle stages because that's how Maven rolls, but we don't need to check out the code again.

We've got our POMs set up with the [artifactory-maven-plugin](https://www.jfrog.com/confluence/display/RTF/Maven+Artifactory+Plugin) so all we have to do to publish is issue `mvn deploy`. That makes that easy, at least; there's the Artifactory CLI if you prefer, but Maven's whole deal is managing everything so as far as I'm concerned we should let it.

There's just one piece missing, though: how do we actually release a new _version_ of the artifact and set up to begin on the next?

## The Release Trigger

One of the ideas of git-flow is that when you're gearing up for a release, you cut a new branch that only contains work towards that release. This is great if you're working on multiple versions of the code simultaneously and releases can take awhile, so you might cherry-pick a bugfix from current development into a legacy release branch to ensure it doesn't affect a subset of your users. Since we're not a product company, we don't really have to worry about that. We're always working on the next release, and it drops when it's ready to drop.

This is going to get complicated. Here's the `release` build steps in full:

```
steps:
  - checkout
  - run:
      name: Cut new release
      command: |
        # assemble current and new version numbers
        OLD_VERSION=$(mvn -s .circleci/settings.xml -q \
          -Dexec.executable="echo" -Dexec.args='${project.version}' \
          --non-recursive org.codehaus.mojo:exec-maven-plugin:1.3.1:exec)
        NEW_VERSION="${OLD_VERSION/-SNAPSHOT/}"
        echo "Releasing $OLD_VERSION as $NEW_VERSION"

        # ensure dependencies use release versions
        mvn -s .circleci/settings.xml versions:use-releases

        # write release version to POM
        mvn -s .circleci/settings.xml versions:set -DnewVersion="$NEW_VERSION"

        # setup git
        git config user.name "Release Script"
        git config user.email "builds@understoryweather.com"

        # commit and tag
        git add pom.xml
        git commit -m "release: $NEW_VERSION"
        git tag "$NEW_VERSION"

        # land on master and publish
        git checkout master
        git merge --no-edit release
        git push origin master --tags

        # increment minor version number
        MAJ_VERSION=$(echo "$NEW_VERSION" | cut -d '.' -f 1)
        MIN_VERSION=$(echo "$NEW_VERSION" | cut -d '.' -f 2)
        NEW_MINOR=$(($MIN_VERSION + 1))
        DEV_VERSION="$MAJ_VERSION.$NEW_MINOR-SNAPSHOT"

        # ready development branch
        git checkout verify
        git merge --no-edit release
        mvn -s .circleci/settings.xml versions:set -DnewVersion="$DEV_VERSION"
        git add pom.xml
        git commit -m "ready for development: $DEV_VERSION"
        git push origin verify

        # clean up release branch
        git push origin :release
```

It's not _messy_, but that's... a lot of bash script. But just like any sufficiently complicated database task involves writing SQL, any sufficiently complicated ops task involves bash. Let's break it down:

### Getting Version Numbers

```
# assemble current and new version numbers
OLD_VERSION=$(mvn -s .circleci/settings.xml -q \
  -Dexec.executable="echo" -Dexec.args='${project.version}' \
  --non-recursive org.codehaus.mojo:exec-maven-plugin:1.3.1:exec)
NEW_VERSION="${OLD_VERSION/-SNAPSHOT/}"
echo "Releasing $OLD_VERSION as $NEW_VERSION"
```

Note the `-s .circleci/settings.xml`: since Circle's just spinning up a basic OpenJDK image, we have a `settings.xml` checked into source control. Credentials are interpolated through environment variables, but it's still not _great_; at some point, I'll want to come back and create a custom Docker image to centralize our configuration.

Maven stores version numbers in the POM. We could pull them out with XPath, but since this is Maven, there's a plugin for that. The `OLD_VERSION` is the current value; since we're always releasing from the `verify` branch, this is guaranteed to be a snapshot version, and we need to strip that qualifier off to get `NEW_VERSION` for the release.

### Update Versions

```
# ensure dependencies use release versions
mvn -s .circleci/settings.xml versions:use-releases

# write release version to POM
mvn -s .circleci/settings.xml versions:set -DnewVersion="$NEW_VERSION"
```

We don't have a ton of Java libraries, but there are enough that release management is (obviously) a concern. The first statement here makes sure that when we release, we aren't depending on a snapshot version of another of our libraries. The second actually sets the version field in the POM to the release version we generated just now.

You may be asking: why didn't I just alias `mvn` to `mvn -s .circleci/settings.xml`? And the answer is: I did, and spent half a day trying to figure out why it didn't work. I don't know if it's this particular image or Circle in general or what, but aliases are just ignored.

### Release!

```
# setup git
git config user.name "Release Script"
git config user.email "builds@understoryweather.com"

# commit and tag
git add pom.xml
git commit -m "release: $NEW_VERSION"
git tag "$NEW_VERSION"

# land on master and publish
git checkout master
git merge --no-edit release
git push origin master --tags
```

Since we're going to be committing code, we need to do a little more git configuration to attribute the commits properly. This is another element I could streamline with a custom build image later on.

Next, we commit the updated POM and create a tag. When we merge (with `--no-edit` since the script can't change the commit message), the release commit and tag will land on the `master` branch. Then it's just a matter of pushing to the origin.

### Next Up...

We've released, but we're not quite done. If we left it here, the next release from the `verify` branch would run into merge conflicts since `master` has an updated version in the POM. To prevent that, we have to merge _back into `verify`_. Preferably with a snapshot version qualifier, because Maven.

```
# increment minor version number
MAJ_VERSION=$(echo "$NEW_VERSION" | cut -d '.' -f 1)
MIN_VERSION=$(echo "$NEW_VERSION" | cut -d '.' -f 2)
NEW_MINOR=$(($MIN_VERSION + 1))
DEV_VERSION="$MAJ_VERSION.$NEW_MINOR-SNAPSHOT"
```

I switched us over to two-part version numbers strictly out of convenience. Since Maven expects you to know what you're working towards, going from 1.0 to 1.1 is a lot more realistic than trying to suss out whether you're looking at 1.0.1 or 1.1.0 next. We can always update the version ourselves if we decide the next release should actually be 2.0, but I'm trying to minimize human involvement here.

```
# ready development branch
git checkout verify
git merge --no-edit release
mvn -s .circleci/settings.xml versions:set -DnewVersion="$DEV_VERSION"
git add pom.xml
git commit -m "ready for development: $DEV_VERSION"
git push origin verify
```

Merging `release` into `verify` saves us from any potential merge conflicts down the line, since the same release commit now exists both on `master` and in `verify`. The script then adds a second commit to `verify` with the new snapshot version and sends it all up to the origin.

```
# clean up release branch
git push origin :release
```

Finally: when a trigger goes off, it resets. We don't want the `release` branch to hang around long-term. If we did, we'd have to push the release commit up to the origin to avoid merge conflicts in future, and doing that would kick off an infinite loop since the `release` _job_ is watching this branch. So instead we just delete it from the origin, since it's done everything it needed to do.

## Setting it Off

```
git checkout -b release
git push origin release
```

That's the payoff. Whenever we're ready to drop a new version, all that has to happen is a new branch named `release`. You can even do it through the GitHub UI if you're so inclined, in two clicks and seven letters. Once `release` builds and deletes itself, the ordinary build and deploy jobs take over on both updated `master` and `verify` branches. Within a few minutes we've got a release and the first snapshot towards the next landing in Artifactory!