use generators rather than lists for intermediate processing of words
punctuation is removed by strip in norm_words, so it's not necessary in the stopset
ignore compiled python files
split data into training and testing datasets before calculating word scores
added missing tracing
re-factored train_classifier to pull out functions for re-use
GradientBoostingClassifier uses 'learning_rate' not 'learn_rate'
add words to FreqDist, not tuple
make norm_words a default argument to extract_text. handle score_fn not being set
Changed so regardless of whether args.multi and args.binary are set, the
same order of operations happens:
1. corpus read into test_instances, train_instances
2. if feature selection (args.score_fn and args.max_feat), words per category
   are extracted from the train_instances and score_fn is applied to select
   features and define featx
3. features are extracted from instances by applying featx

This order ensures that the corpus is only read once and that the feature
selection is only done over the training set (the classifier can be biased if
feature selection is done over the entire corpus).

Corpus is read differently based if args.multi and args.binary.
Depending on how the corpus is read test_instances and train_instances
are different data structures. category_words is defined based
on how the corpus is read.  extract_features is written to handle
both data structures.
separated out cross-fold execution path and added raising
an exception if trying to cross-fold a multi-binary classifier
Merge pull request #17 from kecaps/master

A few bug fixes and enhancements for evaluating classifiers on large datasets