clemente-lab/fresco — pull request #33 by rybern (2013-09-11 11:17:33+00:00)
Title: Refactored, added errors/asserts

I made some progress tonight (/mostly this morning). While working on unit tests, I decided it would be more helpful to start with to write error handling and asserts (so I could test that functionality too). While working on that, I got distracted by doing some refactoring that I've been meaning to do for a while.

I reduced the scope of the wrapper module by a fair bit by moving out the testing code and the problem data building function. I had a pretty frustrating time trying to merge the testing functionality (that was in the wrapper) into the existing cross validation code for scoring feature vectors, and eventually decided that it should be a different module.

In theory, all of the mysterious stitching errors should be gone since that code is gone. Instead, I've simply included the feature vector from each fold in the feature vector output file, which I think makes more sense. Since different cross folds of the optimization can produce different feature vectors, they can't really be averaged. If users want a single feature vector, they shouldn't use cross fold testing (n_trials=0).

The main work was on raising errors/asserts. I think I was pretty thorough with the modules that I covered (so far just the wrapper and group_problem_data), but I haven't done much with it in the past, so let me know what you think if you happen to read through it. I've been assigning functions as a potential public interface pretty liberally, so I do a lot of redundant parameter checking. I really don't like Python's lack of support for private interfaces, but maybe I'm just missing something.

I updated the multiprocessing module to be a little more helpful/flexible by allowing keyword arguments and 'tags' for organizing results.

I changed the masking semantics in problem_data to be a stack of masks, so that testing can be done recursively (currently we're doing cross fold validation over a training partition, so there are two masks). I think this is a lot more elegant, but I might spend more time making it more efficient.

My priorities for the next couple days are to finish my errors/asserts, then unit testing, then finishing docstrings. I'm almost done with errors/asserts, I'll spend some more time on unit testing, and I don't think docstrings will take too long.

However, it's currently passing 5 AM and I just ate a bunch of eggs, so I might be out for a few hours :)