madlib/testsuite — pull request #2 by huor (2012-08-27 06:35:53+00:00)
Title: MADmark improvements: new mechanism to load data and performance improvement of test case generation

This pull request have two big improvements:

1. Data loader tool refinement
The new data loader tool is more flexible, which automatically download, convert, cache, and load dataset into testing databases.
The new features include:
(1) Each algorithm (Decision Tree, Random Forest, etc) has its own 'tables.yaml' to describe which dataset it needs and how they are generated (download origin data set from web, clean raw data, convert them to loadable data format, load them into database tables, etc).
(2) Each algorithm has one or more converting tools to clean and convert related datasets. Users can reuse or modify them to meet their own requirement or add new tools themselves if neccessary.
(3) Users can customize loading process. They can skip datasets, skip algorithms and append tables dynamically.
(3.1) Users can initialize testing database and load datasets for certain algorithms.
(3.2) Users can load all datasets except certain algorithms.
(3.3) Users can skip some big datasets if they want to load only some small/medium datasets and just do a smoke test.
(3.4) Users can add database tables in append mode.
(4) Besides permanent configuration files, Users can also temporarily customize loading process using parameter combinations in 'python run.py'.
(5) All datasets are downloaded dynamically from website (UCI, etc). Thus MADmark is light-weighted, which contains only script and config files.

2. Performance improvement of test case generation
Reduce the time of generating cases from 30min to about 10sec. By default, we remove the time-consuming logging part which takes about 30min. While the logging functionality is kept (use -d to enable it if necessary) as well.