activewarehouse/activewarehouse-etl — pull request #63 by pdodds (2011-12-14 21:03:27+00:00)
Title: Ability to use mysql command line to stream data

We have been using ActiveWarehouse on some pretty large datasets - around 70m records and we wanted to be able to pull from staging tables through queries and then apply transformations and upload into a datamart however ran into problems when the data extract was in the millions due to limitations on the MySQL drivers.

To that end we extended the database source to allow you to use the mysql command line with the --quick option to pull the data. It allowed us to get the data out and in much more quickly.

Its not very DB agonistic - but in terms of ETL's its a nifty little feature?