Of Webpacker Config and Failed Rails App Deploys
Tags: rails, webpack, javascript
Published: 2020-10-08 03:28:38

Earlier this year, deploys of my team's main application began failing with this error:

`FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory`

This is a Rails app with an AngularJS front-end currently being converted to React. In the months leading up to those failures, deploy times had steadily increased. Before they began failing, our longest deploys took 24+ minutes. 😱 Here's how we fixed the issue and what I learned about the cause.

## Attempted Fixes

### The `--max_old_space_size` Setting

We increased Node.js's memory limit to 2GB by setting `--max_old_space_size=2048` as recommended in several Stack Overflow posts and Github issues. While this worked for many others, it did not solve our problem. Deploys continued to fail.

### Node.js Upgrade

We next upgraded the app's Node.js version from 8 to 12 to take advantage of this feature:

> This update will configure the JavaScript heap size based on available memory instead of using defaults that were set by V8 for use with browsers. In previous releases, unless configured, V8 defaulted to limiting the max heap size to 700 MB or 1400MB on 32 and 64-bit platforms respectively. Configuring the heap size based on available memory ensures that Node.js does not try to use more memory than is available and terminating when its memory is exhausted. ([source](https://medium.com/@nodejs/introducing-node-js-12-76c41a1b3f3f))

Upgrading Node.js unblocked our deploys for several weeks. However, during that time, we continued converting our AngularJS code to React and added new features in React. Deploys took longer and longer, and after a while, they began failing again.

## The Fix

Given the attempted fixes above and with the help of infrastructure monitoring already in place, we were pretty sure that we weren't running out of memory on our deploy server. As it turns out, the root cause for this issue was in our Webpacker configuration.

Our `webpacker.yml` contained this:

```yml
default: &default
	source_path: app-web
	source_entry_path: react
...
```

Because of the way our app is structured, this meant we were telling Webpacker to process ALL of our React and Redux-related files, which were increasing in number with every sprint. As I researched the deploy failures, I learned about a helpful of rule of thumb about Webpacker from [Ross Kaffenberger's blog](https://rossta.net/blog/overpacking-a-common-webpacker-mistake.html):

> If any file in Webpacker's "packs" directory does not also have a corresponding `javascript_pack_tag` in your application, then you're overpacking.

Based on this rule, I should've seen just one file in our `packs` directory. What I saw, though, was essentially a replica of the entire structure of our `/app-web/react` directory. We were overpacking.

Ultimately, we moved only the **two** necessary files into a `startup` directory and reconfigured `webpacker.yml` to use that as its entry point:

```yml
default: &default
	source_path: app-web
	source_entry_path: react/startup
...
```

## What I Learned

### What is Webpacker, and what does it do?

[Webpacker](Webpacker) is a gem which allows Rails apps to use [webpack](https://webpack.js.org/) to process and bundle assets, particularly JavaScript.

According to its documentation, webpack "is a static module bundler for modern JavaScript applications. When webpack processes your application, it internally builds a dependency graph which maps every module your project needs and generates one or more bundles."

### Okay, cool. But what does that actually *mean*?

Webpack basically does the work of figuring out what depends on what in your application to generate the minimum "bundles" of assets required to run your app. You include these minimum packs in your application - in Rails, like below - so the app can load with the necessary assets already compiled.

```erb
<%= javascript_pack_tag 'application' %>
```

See [this article](https://medium.com/the-self-taught-programmer/what-is-webpack-and-why-should-i-care-part-1-introduction-ca4da7d0d8dc) for a much more in-depth intro to what webpack actually does and why module bundlers are needed.

### Why was our configuration wrong?

Since webpack builds a dependency graph based on a specified entry point, the greater the number of items in that entry point, the more processing time and resources needed. Because our configuration told Webpacker to process ALL of our React files, this required more time and server resources as we added more files to the React directory.

So, basically, the idea was to not ask Webpacker to process every single file in our React application, but only the entry points to the React app (aka the files that have corresponding `javascript_pack_tag`s), so that they and their immediate dependencies would be ready on initial application load.

## Impact

This fix unblocked our deploys *and* dramatically reduced our deploy times and resource usage on our deploy server.

|        | **Deploy Time** | **Max Deploy CPU Usage** | **Max Deploy Memory Usage** |
|--------|-------------|-----------|--------------|
| **Before Fix** | > 24 min | ~90% | ~2.2GB |
| **After Fix**  | 10 min | ~60% | ~0.28GB |


So, lesson learned - don't overpack with Webpacker! 🧳

<span>*Photo by <a href="https://unsplash.com/@erwanhesry?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Erwan Hesry</a> on <a href="https://unsplash.com/s/photos/suitcase?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a>*</span>