activewarehouse/etl — pull request #1 by tylergannon (2013-06-01 21:25:17+00:00)
Title: Make row validation optional, parse XLSX files, add worksheet column to Excel parser

hi, added the following features.

* CSV and Excel parser can now accept `:validate_rows` as a parameter to their source object. If this is turned off, then the process won't die when a row has the wrong number of fields.
* This is because in my application it's pretty annoying to have the whole process die: I have varied result sets in my input files and would rather validate and exclude rows then to have to preprocess my inputs before giving them to the ETL.
* Added in a dependency to [Roo](http://rubygems.org/gems/roo), which is under more active development than the Spreadsheet gem and can parse XLSX files (big bonus for me)
* If the excel parser is given `worksheet_column: :column_name` in the definition configuration, the excel parser will add the name of the worksheet to each row in a field by that name.
* This is great for me since I have a multitude of sheets where the name of the sheet differentiates the data logically.

My change to ETL::Engine is for compatibility with activerecord 3.x, which won't allow mass assignment to fields that have not been declared as accessible in the model. I didn't add any attr_accessible calls on the model because I don't want to mess with backward compatibility.

Lastly, I am not having an easy time with getting the test suite to work. I'm happy to do the leg work in getting these changes properly tested but I wanted to see if you're interested in these changes -- and if so, whether you wouldn't mind helping me out with the test environment.

I have some cool new DSL I've developed, and if I can get my test environment working I'll pull it out of my application and send another pull request to see about adding it to the gem.

Cheers!
Tyler