Instead of hashing trees by mtime and size, hash by file content.

This allows for the cache to be hit far more often, especially when you are modifying the content of a file back and forth (e.g. frequent undo/redo).

Also, added an optional `digestCache` option to `hashTree(...)` which is used to prevent unnecessary file reads (if the mtime and size of the file haven’t changed, assume that we don’t need to re-read & hash its contents).
Got it. Just to make sure that we're on the same page, this PR is definitely experimental and the conversation is just (if not more) important than the code. And as I've said before, I'm new to broccoli so I apologize if I'm rehashing old thoughts, missing the point, etc.

So why do I think content based caching is better...

With mtime-based caching you can only really have one cache, whatever the last state of a file/directory/tree was. It is a binary thing, either you use the last file/tree or your generate a new file/tree (and blow away the previous cache).

However, with content-based caching, you can have several previous states saved to re-use if needed. That is particularly helpful if you undo/revert one or more changes (pretty common in the front-end refresh the browser workflow) or when you do things like change to a new branch, make a fix, and then swap back to your previous branch.

In both of those cases (undoing a few changes and changing back to a previous branch), content-based caching enables those to be pulled directly from the cache and not trigger a rebuild. And to me that is huge, because our environment deals with pretty large and complicated projects. For example, 500-1000+ files, different projects/git repositories that depend on each other (and those dependencies still get modified pretty frequently, so you can't/don't want to copy them, make then an explicit version, etc). Right now, I'm looking at a 20-30 second fresh cache build time, and while I think I can improve that quite a bit, I don't think I'll be able to reduce it by an order of magnitude.

That is why preventing rebuilds in common cases is huge to me. Especially with how you can nests trees inside one another in Broccoli and a rebuild in one tree could force a later tree to rebuild, which could trigger another rebuild, etc. 

That brings up another useful bit, what if what a filter/plugin outputs is identical to a previous rebuild even though the input was different? Content-based caching deals very well with this, but mtime caching less so (unless you can ensure that every plugin is written very well and is good about ensuring it uses its cache and preserves mtimes in all possible scenarios).

Whew, so those are my thoughts. Thanks for listening. And I'm glad to talk more about our environment/team/project sizes if that is helpful. I realize that our slightly more "enterprise-y" (shudder) use-cases can be pretty different than some are used to.

----

ps: Maybe there is a way to try and save multiple "cache states" with a file's mtime and size stats, but can't think of a way do it well (or without lots of complicated hoops to ensure mtimes get reset to exact previous values).

pps: I do realize that content-based caching is more complicated, requires you to has sane heuristics to not grow the cache forever, etc. But IMHO, I this is a case were that complexity is very worth it, at least at the scale and complexity I'm (and other larger projects/engineering teams are dealing with).