jekyll/jekyll — pull request #3116 by alfredxing (2014-11-17 00:19:36+00:00)
Title: Incremental regeneration

Almost complete implementation of incremental regeneration (#380 and #3060) between as well as within a process. Currently handles file content changes and dependency changes (layouts and includes only). This implementation takes a few things from @mattr-'s implementation in #1761.

Here is what happens on each build:

1. The `Metadata` class is initialized and reads in metadata from `.jekyll-metadata` (metadata is not read if the `--clean` flag is set).
2. Jekyll performs `reset`, `read`, and `generate` as it did before.
3. In the render step, all documents, pages, and posts are checked for modifications to itself or its dependencies. A document/page/post is also marked as modified if it has `regenerate: true` in its front-matter.
4. `cleanup` is performed if the `--clean` flag is set.
5. Modified files, as well as the new `.jekyll-metadata`, are written in `write`.

Remarks:

- Although initial (no metadata present) build times went up a tiny bit, builds with metadata are super quick ([here](https://gist.github.com/alfredxing/456cde450dc2e05d423a) are the results from the [jekyllrb.com site](https://github.com/jekyll/jekyll/tree/master/site))
- Dependencies in the form of referencing posts/pages/collections/data in Liquid with `site.x` are not resolved. (This is why I included the `regenerate` front-matter override.) Any ideas on how we can make this work?
- The metadata file is currently YAML (loaded with SafeYAML). Is this optimal, or would another serialization method work better?

---

Todo:

- [x] Implement behaviour
- [ ] Fix failing tests
- [ ] Add tests (cucumber & unit)