remove noisy LDA features by rewriting CSV and omitting features w frequency above given limit
removed snippet for testing that it does what it says
instead of omitting, if a feature's frequency is above limit, set freq to null
extract top entities for top N related wikis by Euclidean distance
tally and filter topics above threshold, set absent frequencies to 0 instead of omitting
clean up write format
write CSV containing top entities common to each topic feature across different wikis
added multiprocessing; why isn't topics dict being modified?
no longer attempting to share state between processes
pull entities from solr instead of TopEntitiesService
handle None argument in to_list
handle empty list in TermFreqField
added logging, added stemmed param to identify_subject
included subjects and jaccard distance
calculate Jaccard distance b/n wiki and topic entities, name after best fit
added terms_only param to identify_subject
change output file name
make sure topic is in entity_counts_for_topic dict
handle empty lists as args to build_dict functions
pickle dicts to avoid data loss upon premature termination