weecology/retriever — pull request #122 by bendmorris (2013-11-04 14:28:15+00:00)
Title: Starting to refactor engine/table functionality

Everything was previously being handled by the bloated Engine class. I'm splitting this functionality between Engine and Table, and trying to refactor functions such that their dependencies are all explicit, making them much easier to test.

The philosophy of the split:

Engine: functions that take lines of data, do any necessary conversions (e.g. generate SQL statements), and store them in a database/CSV file/etc. In other words, functions that are dependent on the specific engine that was selected.

Table: functions that extract values from a data file, infer data types from input data columns, etc. - functions that are only dependent on the input file, not the engine.

Additionally, functions that aren't dependent on class members are being split out into standalone functions (e.g. split_rows, file_exists, etc. in engine.py.)

A lot of work is still being done in add_to_table, which still needs to be refactored.