tidepool-org/blip — pull request #30 by nicolashery (2014-03-20 15:16:51+00:00)
Title: Optimize build step and application start

In an email earlier this week, @cheddar pointed out that Blip's build step and start script were taking long and using a lot of machine resources. This was unfortunate because it leads to the following problems:

- Deploys become painful, they should be easy and "boring", so we don't hesitate to push often, delighting our users with frequent updates and bug fixes

- There was a downtime of a couple minutes during a deploy, because the start script took too long, not fun if a user tries to access the application during that time

- Using too much machine resources on the deployment server means we ran the risk of impacting other apps running on that server, not a good thing

This PR is a series of what I hope to be "quick wins" that help tackle the causes of the issues described above. Let me know what you think @cheddar, and thanks for the suggestions on some of these. Here is what I did:

**1) Make build step run faster and use less memory**: I narrowed down the culprit to UglifyJS, which takes up a lot of resources (gulp is fine here, it would be same with grunt or any other tool). What this step does is take all the JS files and minifies them so they weigh less KBs and are downloaded faster in users' browsers.

I did two things here:

- Most vendors provide a minified version of their lib, so use that one directly (instead of minifying the normal version)

- Minify JS files *before* concatenating them, so we're not minifying one big file at once, thus using up more memory

There seems to be somewhat of an improvement. These numbers are totally not scientific, they come from me watching `top(1)` on my machine and running the build step before and after the fix:

- Before: Ran for 14-17s, used up to ~250MB of memory
- After: Runs for 7-11s, uses up to ~175MB

So I guess that's about **40% improvement in running time** and **30% improvement in memory usage**.

**2) Separate dependencies needed to build/run from testing dependencies**: Probably the longest step in `build.sh` is `npm install`. What I did was put everything that's needed for deployment (build and run) in `dependencies` in `package.json`, and leave the things used to test in `devDependencies`. That way we can use `npm install --production` in `build.sh` which will only install `dependencies`.

This means we **go from downloading 28 dependencies to downloading 19**.

**3) Separate config from the rest of the app**: By removing the `config.js` file from the rest of the `all.js` bundle, we're able to move the build task (the one that takes memory) back into the `build.sh` script, where it belongs. In `start.sh`, we only need to run the `gulp scripts-config` task that generates the `config.js` file and runs really fast.

So **the start script should be really fast now**, meaning very little downtime for our users. Also, **the start script should take very little resources now**, meaning less chance of impacting other apps running on the deployment server.

That's it! @cheddar let me know if you think this should help a bit, and if it does feel free to merge.

Thanks