Speed up static file service by caching existence checks

Rails serves assets in development by default, in production this is turned off. The default rails guides require you be running behind nginx. In the last year 800,000+ developers don't want to run behind nginx: https://rubygems.org/gems/rails_serve_static_assets. This PR enables serving assets in Production by default and includes an optimization that makes this option dramatically faster for non-asset requests.

Currently the `ActionDispatch::Static` middleware hits disk on every `GET` and `HEAD` request to check if the given request maps to a file on disk. This means that all requests that do not involve serving an asset are slowed down. If you are using this middleware and serving assets via Nginx, no asset requests will ever reach the `ActionDispatch::Static` middleware so it makes sense as a performance optimization to turn off this middleware. This PR increases the speed of the middleware to the point where the difference on non-asset requests is trivial, so we can enable the middleware by default for all users with minimal (if any) impact.

The optimization works by getting the contents of `/public` (using one call to `Dir.glob`). We then store this information in a hash. Later on each request we check to see if the path of the request is in our hash for instance a request to `/users/new` would not be in the hash because there is no root level `users/` directory, however a request to `assets/application.css` would trigger the middleware because `assets/` is in the `public/` directory. This prevents us form checking the disk except on a request which is more than likely going to contain an asset.

When benchmarked against the current `ActionDispatch::Static` we see roughly a ~68% improvement for non-asset requests:

![](https://www.dropbox.com/s/dcsrhrfh7gb44dc/Screenshot%202014-08-08%2014.05.21.png?dl=1)

While 68% is an impressive speed increase, this is only testing the middleware in isolation. When benchmarked with the entire Rails stack we still see an increase but it is roughly ~2.6% speed increase:

![](https://www.dropbox.com/s/bhb0o3tbupw0hus/Screenshot%202014-08-08%2015.06.11.png?dl=1)

Note: The scale on this one does not start at 0.

There is still a speed cost associated with using this improved middleware versus no middleware. In my benchmarks it resulted in a ~0.7% speed decrease though with a very high standard deviation. You can see in the graph that the speed of the improved middleware falls in-between the speed of the app without middleware. This means that the decrease is fairly trivial in the scope of your entire application stack. You can still disable this middleware for a performance optimization if you are deploying behind NGINX, but there is no reason why using a CDN with your Rails server as the origin server shouldn't work out of the box.

For more information about the purpose of ActionDispatch::Static and how it is currently used (or not used) to serve assets please see this reference document: https://gist.github.com/schneems/5dc42963fb3221a7a089.
This method is understandable, but one must get into machine mindset and "run the program" to infer its intent. That invites avoidance and regression.

We're building a set of all top-level path elements that we could possibly match, allowing us to quickly reject paths we don't recognize.

That means we must faithfully duplicate behavior of `match?` and `Rack::File`, introducing vulnerability to regression and file-presence security leakage.

Since we build the set of all possible top-level paths when the app is booted, we leave no facility for adding or removing files without restarting the process. This is OK for most production apps, but is problematic for deployments that expect to, say, add or replace a file without restarting app servers.