Leveraging Travis-CI for Continuous Deployment to Publish Compiled Binaries to GitHub
Tags: ci, travis, rust, github
Published: 2019-04-25 03:02:32

Recently I wrote a binary called "[Watchdog](https://github.com/hawkins/watchdog)" in Rust on my Mac that I wanted to take with me to Linux systems I frequent, but I couldn't bring a Rust compiler to these systems for unrelated reasons. This meant I had to cross-compile my application and ship just the binary.

Now, I'm taking a few liberties in this post to showcase this as a proof-of-concept, but most notably of these is that *I am not handling dynamically-linked libraries*. I'm shipping a standalone binary, that is to say the one file I build has everything it needs to run, assuming you run it on the correct platform. Or, so I naively hope - it can definitely be more complicated than that.

---

## What is cross-compilation?

Cross-compilation is compiling your code on one platform (for me, Mac OS) to run on another platform (for me, Linux). If you're more familiar with languages like Ruby, JavaScript, and Python, this might not make a lot of sense since your code already runs on different platforms most likely. But for compiled languages like Rust, C++, C, etc., we have to compile it to code that can run on certain platforms. That's what we're doing here, we're compiling my Rust code on my Mac to run on another computer that runs Linux.

You can configure cross-compilation on your local computer, but it can be difficult, and often has some big quirks. I've dealt with this recently when [cross-compiling a lambda function in rust for mac->linux](https://github.com/hawkins/district-doughnut). In particular, getting OpenSSL to behave was an absolute nightmare and killed my motivation for months. Eventually, I kicked the problem to the curb and said I'll bring in Docker just to work around it. Your mileage may vary.

So instead of xcompiling locally, we'll leverage [Travis CI](https://travis-ci.org) to do this natively for free. Technically it's *not cross-compiling, since Travis will compile it in a Linux VM*, but the accomplished goal is the same - building my application for platforms I can't otherwise. So, if you'd like to chew me out for click-baiting, I welcome it - I just couldn't think of a better way to word what we're doing here!

## Where Is Your Code?

You can see all the code I worked on for this post on my GitHub project! {% github https://github.com/hawkins/watchdog %}

## Initial Travis Config

Travis lets us configure it via code, so my initial `.travis.yml` file before I set up "cross-compilation" looked like this:

```yaml
language: rust
rust:
- stable
- beta
- nightly
matrix:
  allow_failures:
  - rust: nightly
  fast_finish: true
os:
- linux
- osx
deploy:
  # This part's not very relevant to this blog post, we'll add a new deploy step though, so I'll keep it as an example of what you might already have
  provider: cargo
  skip_cleanup: true
  on:
    tags: true
    condition: "$TRAVIS_RUST_VERSION = stable && $TRAVIS_OS_NAME = linux"
    branch: master
```

If you're not familiar with Travis, I'll give you a brief summary of what this config does.

1. We run a "matrix" build (multiple platforms and rust versions) to test both stable rust versions and nightly builds of their rust compiler on different platforms.
2. Nightly builds can fail, since those compilers aren't final and may have bugs, so don't mark these build fails as an overall failure if that's all that went wrong
3. When I tag a release in git, publish this as a new version of the crate (rust package) in the global repository so other rust users can grab the latest version just released

Note that there's no talk of publishing binaries yet - because that's the new part we'll add today!

## Travis Releases Provider

GitHub already has a sweet releases feature, and I happen to be using it for Watchdog, so I'd like to publish my built binaries there so users can find it easily if they, like me on my linux machines, can't access a Rust compiler.

Travis CI is super rad, so naturally they already have [swell support for GitHub Releases](https://docs.travis-ci.com/user/deployment/releases/)! Lucky us!

To use this provider, we'll have to:

1. Create a new personal access token on GitHub for Travis to use to upload the binaries to your releases: https://github.com/settings/tokens
2. Add them to your Travis project:
  - In your terminal, run these commands:
  - `cd <Your project>`
  - `gem install travis` (you'll of course need ruby + rubygems, which Mac OS comes with)
  - `travis encrypt` (then follow the provided instructions)
3. Add the below code to your `.travis.yml`:

```yaml
deploy:
  - provider: releases
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable"
      branch: master
    file: target/release/watchdog # where `watchdog` is your built rust binary
    api_token:
      secure: # this part from your `travis encrypt` command
```

## Testing Travis Changes

I'm no Travis expert, so maybe a commenter can help us both learn a better way to test this, but here's how I test changes to my travis config:

1. Commit the change in git
  - To save your sanity, comment out the `branch: master` part so you can do this on a throw-away branch
2. Tag the commit
3. Push to GitHub (remembering to push with tags, i.e. `git push --tags`!)

Travis will then run your new build.

## Connecting the Dots

The above config won't quite work for us though, so we have to be a little more clever about what we send to GitHub. 

If you added that last section like I suggested, you'll notice that `target/release/watchdog` doesn't exist. This is because by default, Travis builds in Debug mode (notice the absence of Rust's required `--release` field for release mode builds in the "Default script" setting, implying it's Debug):

![](https://thepracticaldev.s3.amazonaws.com/i/bbas2erbge80zodm45me.png)

We want to publish our binary in Release mode, so let's modify our yaml to build in Release mode by adding this key to the root of the yaml:

```yaml
script: cargo build --verbose --release; cargo test --verbose
```

Sweet. Now if you tested it, you should see that Travis is a little slower (release builds take a bit more work!) but your file should be uploaded to the GitHub release! :tada:

## But Wait, There's More!

Notice how we build for two platforms, but only see one binary in the release? And we can't tell what platform its built for because its just named `watchdog`? That's no good, if I downloaded that on my linux box, I have no idea if I'm downloading a linux binary or a mac binary. 

**Why is there only one?**

Well, admitting that I didn't encounter this myself (so your exact error may vary) because I did this out of order, these Travis builds run out of order and concurrently, so your linux build may upload a file named "watchdog", only to be overwritten by your mac build when it uploads a different file named "watchdog". So, we have a race condition.

**How can we get both binaries to upload and not overwrite each other?**

Well, we can get two birds with one stone here, by also indicating in the file name whether it is built for linux or mac. The idea is we have to give the files on different builds different names so we don't overwrite them, so we might as well specify which platform they're built for to be clearer to users too.

## Renaming Your Binary

This was the fun part for me, tracking down how to rename binaries so we could upload multiple to GitHub and specify their platform. It only reinforced my love for Travis when I learned that it provided all the building blocks I needed:

- a `before_deploy` [script step](https://docs.travis-ci.com/user/deployment/releases/#setting-the-tag-at-deployment-time) to run whatever I want *after* we run the `script` step yet *before* we run the `deploy` step
- a collection of [environment variables](https://docs.travis-ci.com/user/environment-variables) to tell all sorts of things about our builder, like its platform

With that in mind, we can just connect the dots to rename the binary before we upload it in the deploy step:

```yaml
before_deploy:
  - mv target/release/watchdog "target/release/watchdog-$TRAVIS_TAG-$TRAVIS_OS_NAME"
```

But, if we want to support having multiple deploy steps (publishing to crates.io AND GitHub releases, for instance), this will actually cause an error and fail the whole build. That's because this `before_deploy` script gets called once *for each deploy step*. That actually burned me and caused my "final" build to fail, because the second time it was called, `mv` threw an error because `target/release/watchdog` didn't exist anymore. So, to account for that, you'll need a little bash-fu to update your yaml once more:

```yaml
before_deploy:
 - "if [[ -f target/release/watchdog ]]; then mv target/release/watchdog \"target/release/watchdog-$TRAVIS_TAG-$TRAVIS_OS_NAME\"; fi"
```

What this modification does is use bash to only run `mv` if the file exists, so we won't error out and fail the whole Travis build if we have multiple deploy steps. Anyways....

Rad, now our binary is named something like `watchdog-v0.2.0-linux`! Now we can adjust our deploy step to find this new name pattern:

```yaml
deploy:
  - provider: releases
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable"
      branch: master
    file_glob: true # <-- note this new field
    file: target/release/watchdog-*  # <-- note the `-*`
    api_token:
      secure: # this part from your `travis encrypt` command
```

The key here is to use globbing to find the new binary, since we can't hardcode the name anymore. We have to enable that with `file_glob: true`, then we can add `-*` to the end of our file name to find the new name pattern easily!

Testing this now should show a GitHub Release with your two platform's binaries both uploaded, finally. Way to go! :tada:

| Travis | GitHub |
|:-:|:-:|
| ![](https://thepracticaldev.s3.amazonaws.com/i/sga33kg90g1td9kycr0r.png) | ![](https://thepracticaldev.s3.amazonaws.com/i/xlgt1spmhtlg0qyiihm9.png) |



## Putting it All Together

Here's the final Travis config we used:

```yaml
language: rust
rust:
- stable
- beta
- nightly
matrix:
  allow_failures:
  - rust: nightly
  fast_finish: true
os:
- linux
- osx
before_deploy:
  - mv target/release/watchdog "target/release/watchdog-$TRAVIS_TAG-$TRAVIS_OS_NAME"
deploy:
  - provider: cargo
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable && $TRAVIS_OS_NAME = linux"
      branch: master
    token:
      secure: # ;)
  - provider: releases
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable"
      branch: master
    file:
      - target/release/watchdog-*
    api_key:
      secure: # ;)
```

And that's it! Now we've got a Travis config that allows us to build and publish binaries for any platform Travis supports whenever we tag a release.

You can check out my release I cut to demonstrate this here: https://github.com/hawkins/watchdog/releases/tag/v0.2.5

---

Once more, you can review all this in action with a real Rust project on my GitHub. And if you have any questions, feel free to leave either a comment on this blog post, and issue on the below repo, or shoot me an email!

{% github https://github.com/hawkins/watchdog %}