taskrabbit/forklift — pull request #14 by evantahler (2014-03-26 23:03:36+00:00)
Title: V2

## V2

This forklift update will be version `1.0.x`, and finally make it onto rubygems.

*This is a major rewrite, and existing plans will not work*

## Major Changes

- **Transports**
- Forklift now contains the infrastructure to work with multiple transports. MySQL and ElasticSearch are included in the package, but you are encouraged to create your own. Forklift now works agnostically with transports, passing arrays of data between `read` and `write` blocks
- inter-transport "speedy" ETL tools are still supported, and are renamed to "pipes"
- **Incremental**
- Forklift now has support for incremental extracts (and overwriting writes). You can now perform incremental ETLs.
- ** Patterns**
- You can now bundle common ETL steps into `patterns ` which wrap common functionality. For example, the [mysql_optimistic_import](https://github.com/taskrabbit/forklift/blob/v2/lib/forklift/patterns.rb) will attempt an incremental extract, and then fallback to a full table import.

## Other Changes

- simplified logger
- more control over your plans
- simplified emailer and support for ERB templates
- project generator `forklift --genearte`
- binary runner `forklift /path/to/my/plan`
- example included within the project
- `mysql lazy table creation`: mySQL can now create tables automatically from the data provided to a `write` operation

## TODO

Pending issues kept in the ["v2" milestone](https://github.com/taskrabbit/forklift/issues?milestone=1&state=open)