- Removed some stray punctuation errors from Advertising.dat
- Removed some stray punctuation errors from all three test datasets
- Pickled emotion info for ads, mnemonics, political slogans, and movie quotes, to avoid repeatedly calling the sentiment analysis API.
- Modified sentiment analyzer script so that it takes in any sentence and finds associate emotion info using the pickled info, and only calls API if sentence is unknown.
- Removed (hid) a function that isn't being used right now
.. and got rid of some useless code
- Added syntactic N-gram TF-IDF scores as features
- Added emotion strength and polarity as features
- N-grams are now added to the feature list only if they're not already present.
- Added a new variable current_features: dictionary that maps all N-grams already present in feature list to 1, and all other (unseen) N-grams to 0. (this is faster than searching through the feature list for the N-gram each time)
- Regenerated feature files with updated feature set
- Modified Train class so that BOW features can be excluded if needed, by setting include_BOW_features = False (optional param for constructor)
- Wrote new function in Memorability_training to create feature file for combined set
- Fixed file paths so that they point to the right combined.dat file (in create_datasets/)
- Removed stray newline in test.dat
- Added script to train SVM classifiers using LIBSVM
- Removed some stray descriptions from political slogans dataset
- Regenerated emotion polarities for political slogans
- Annotated all datasets for imperatives
- Added script to return whether a quote is an imperative or not
- Added imperatives feature to training script
- Added new function to try and use LIBSVM's cross-validation tool (doesn't seem to work correctly for some reason..)