Speed up static file service by caching existence checks

Rails serves assets in development by default, in production this is turned off. The default rails guides require you be running behind nginx. In the last year 800,000+ developers don't want to run behind nginx: https://rubygems.org/gems/rails_serve_static_assets. This PR enables serving assets in Production by default and includes an optimization that makes this option dramatically faster for non-asset requests.

Currently the `ActionDispatch::Static` middleware hits disk on every `GET` and `HEAD` request to check if the given request maps to a file on disk. This means that all requests that do not involve serving an asset are slowed down. If you are using this middleware and serving assets via Nginx, no asset requests will ever reach the `ActionDispatch::Static` middleware so it makes sense as a performance optimization to turn off this middleware. This PR increases the speed of the middleware to the point where the difference on non-asset requests is trivial, so we can enable the middleware by default for all users with minimal (if any) impact.

The optimization works by getting the contents of `/public` (using one call to `Dir.glob`). We then store this information in a hash. Later on each request we check to see if the path of the request is in our hash for instance a request to `/users/new` would not be in the hash because there is no root level `users/` directory, however a request to `assets/application.css` would trigger the middleware because `assets/` is in the `public/` directory. This prevents us form checking the disk except on a request which is more than likely going to contain an asset.

When benchmarked against the current `ActionDispatch::Static` we see roughly a ~68% improvement for non-asset requests:

![](https://www.dropbox.com/s/dcsrhrfh7gb44dc/Screenshot%202014-08-08%2014.05.21.png?dl=1)

While 68% is an impressive speed increase, this is only testing the middleware in isolation. When benchmarked with the entire Rails stack we still see an increase but it is roughly ~2.6% speed increase:

![](https://www.dropbox.com/s/bhb0o3tbupw0hus/Screenshot%202014-08-08%2015.06.11.png?dl=1)

Note: The scale on this one does not start at 0.

There is still a speed cost associated with using this improved middleware versus no middleware. In my benchmarks it resulted in a ~0.7% speed decrease though with a very high standard deviation. You can see in the graph that the speed of the improved middleware falls in-between the speed of the app without middleware. This means that the decrease is fairly trivial in the scope of your entire application stack. You can still disable this middleware for a performance optimization if you are deploying behind NGINX, but there is no reason why using a CDN with your Rails server as the origin server shouldn't work out of the box.

For more information about the purpose of ActionDispatch::Static and how it is currently used (or not used) to serve assets please see this reference document: https://gist.github.com/schneems/5dc42963fb3221a7a089.
> what is "base in root"

It was previously a method `base_in_root?` as in, is the base of the path you're currently passing in in the root directory.

> what is calling it with a path argument checking?

Is it humanly possible that the file exists on disk. If not, don't bother checking the disk.

> how is that different from what we're checking below?

This patch acts as a low pass filter, it will short circuit any request that cannot possibly be an asset and send it straight to Rails. It is not a substitute, but rather an attempt to speed up the common case, which it does fairly drastically, removing 3 disk stats for one hash lookup on most requests.

> if it isn't, why does it get special treatment?

speed

> can the implementation be simplified/unified to avoid special treatment? what are the tradeoffs, and why are we making them?

Begin can of worms. Need to go into why the existing checks exist and what they do:

## Checks

Currently there are checks in Rails. They serve these purposes

1) Ensuring that Rack never serves a 404 (all missing files should be rendered by Rails stack)
2) Act as a flipper allowing asset requests such as `index.html` to take precedence over Rails
3) Add custom behavior (i.e. `domain.com/` will check for a `public/index` and `public/index.html`
4) Allow dynamic file service (i.e. files uploaded via paperclip/carrierwave)

We can't get rid of the existing checks in Rails without removing our custom behavior (3).

We can't modify my new checks to be comprehensive without removing the ability to serve dynamic uploads (4), since the hash is pre-computed.

Deprecating either 3 or 4 would be a nightmare, as i'm sure people are using them.

## Overall

Some of this can be addressed in naming and comments and testing. I think moving the logic back to a method instead of a lambda makes the most sense and allows for testing. I don't thinks we can get this to be faster, maintain all current features, and be super simple.