Module ImportFrom alias Import alias identifier Import alias identifier Import alias Import alias Import alias ImportFrom alias alias ImportFrom alias ImportFrom alias Expr Call AttributeLoad NameLoad attr keyword Str keyword AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam NameParam defaults body Return Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults Num body Return DictComp NameLoad NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad NameLoad attr CompareGtE Call NameLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults Num Num Num body Assign NameStore Dict Assign NameStore Call NameLoad ListComp SubscriptLoad NameLoad Index Str comprehension NameStore Call AttributeLoad NameLoad attr comprehension NameStore NameLoad For TupleStore NameStore NameStore Call AttributeLoad NameLoad attr body If CompareLt Call NameLoad NameLoad NameLoad body Continue Assign NameStore DictComp NameLoad Num comprehension NameStore NameLoad For NameStore NameLoad body AugAssignAdd SubscriptStore NameLoad Index SubscriptLoad NameLoad Index Str Num Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Assign NameStore BinOpDiv Call NameLoad Call AttributeLoad NameLoad attr Call NameLoad NameLoad Num If BoolOpAnd CompareGtE NameLoad NameLoad CompareLtE NameLoad NameLoad body Assign SubscriptStore NameLoad Index NameLoad NameLoad Return NameLoad decorator_list Import alias FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad For TupleStore NameStore NameStore Call AttributeLoad NameLoad attr body For NameStore NameLoad body Assign SubscriptStore NameLoad Index Str SubscriptLoad NameLoad Index SubscriptLoad NameLoad Index Str Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return ListComp NameLoad comprehension NameStore NameLoad comprehension NameStore NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Return DictComp NameLoad Call NameLoad ListComp Call AttributeLoad NameLoad attr NameLoad comprehension NameStore NameLoad comprehension NameStore NameLoad decorator_list FunctionDef arguments args TupleStore NameStore NameStore defaults body Expr Call AttributeLoad NameLoad attr BinOpAdd Str NameLoad Assign NameStore Call NameLoad ListComp Call AttributeLoad NameLoad attr NameLoad comprehension NameStore NameLoad Return TupleLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults Num body If CompareEq NameLoad Num body Return DictComp NameLoad Call NameLoad ListComp Call AttributeLoad NameLoad attr NameLoad comprehension NameStore NameLoad comprehension NameStore NameLoad orelse Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore ListComp TupleLoad NameLoad NameLoad comprehension NameStore NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore DictComp NameLoad NameLoad comprehension TupleStore NameStore NameStore NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults Num body If CompareEq NameLoad Num body Return DictComp NameLoad Call AttributeLoad NameLoad attr ListComp Call AttributeLoad NameLoad attr ListComp Call NameLoad TupleLoad NameLoad NameLoad comprehension NameStore NameLoad comprehension NameStore NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad NameLoad attr orelse Assign NameStore Dict Assign NameStore Call AttributeLoad NameLoad attr NameLoad Print TupleLoad Str NameLoad For TupleStore NameStore NameStore Call AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr BinOpAdd BinOpAdd BinOpAdd BinOpAdd Str NameLoad Str Call NameLoad Call NameLoad NameLoad Str Assign NameStore ListLoad For NameStore NameLoad body Assign NameStore ListComp TupleLoad NameLoad NameLoad comprehension NameStore NameLoad Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad NameLoad Assign SubscriptStore NameLoad Index NameLoad Call AttributeLoad NameLoad attr NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults Num body If CompareEq NameLoad Num body Return DictComp NameLoad ListComp ListComp NameLoad comprehension NameStore ListComp Call NameLoad TupleLoad NameLoad NameLoad comprehension NameStore NameLoad comprehension NameStore NameLoad comprehension NameStore NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad NameLoad attr orelse Assign NameStore Dict Assign NameStore Call AttributeLoad NameLoad attr NameLoad Print TupleLoad Str NameLoad For TupleStore NameStore NameStore Call AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr BinOpAdd BinOpAdd BinOpAdd BinOpAdd Str NameLoad Str Call NameLoad Call NameLoad NameLoad Str Assign NameStore ListLoad For NameStore NameLoad body Assign NameStore ListComp TupleLoad NameLoad NameLoad comprehension NameStore NameLoad Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore ListComp ListComp SubscriptLoad NameLoad Index NameLoad comprehension NameStore NameLoad comprehension NameStore Call NameLoad Call NameLoad SubscriptLoad NameLoad Index Num Assign SubscriptStore NameLoad Index NameLoad ListComp Call NameLoad NameLoad comprehension NameStore NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore ListComp NameLoad comprehension NameStore NameLoad comprehension NameStore Call AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return DictComp NameLoad Call AttributeLoad NameLoad attr ListComp SubscriptLoad NameLoad Index Str comprehension NameStore NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr For NameStore Call AttributeLoad NameLoad attr NameLoad body If CompareIn NameLoad NameLoad body Delete SubscriptDel NameLoad Index NameLoad orelse Delete SubscriptDel NameLoad Index NameLoad decorator_list
N/A __future__ division N/A numpy np N/A multiprocessing multi N/A logging N/A types N/A sklearn sklearn.preprocessing LabelBinarizer MultiLabelBinarizer import_utils import_class preprocessing_utils map_feature_extractor N/A N/A N/A logging basicConfig format %(asctime)s : %(levelname)s : %(message)s level N/A logging INFO N/A logger N/A N/A logging getLogger testlogger init_class N/A N/A klass args N/A N/A N/A N/A klass args N/A filter_contexts N/A N/A token_contexts min_total N/A 1 N/A N/A N/A token contexts N/A N/A token contexts N/A N/A token_contexts items N/A N/A len contexts min_total N/A filter_contexts_class N/A N/A token_contexts min_total min_class_count proportion N/A 1 1 2 N/A N/A new_token_contexts N/A N/A classes N/A set N/A N/A cc N/A tag N/A context N/A N/A token_contexts values N/A cc context N/A N/A token contexts N/A N/A token_contexts items N/A N/A N/A N/A len contexts min_total N/A N/A N/A class_counts N/A cl 0 N/A cl classes N/A cc contexts N/A N/A N/A class_counts N/A N/A cc N/A tag 1 N/A min_class N/A min N/A N/A class_counts values N/A cur_proportion N/A N/A max N/A N/A class_counts values N/A max min_class 1 N/A N/A N/A min_class min_class_count N/A cur_proportion proportion N/A N/A N/A new_token_contexts N/A token contexts N/A new_token_contexts N/A N/A copy convert_tagset N/A N/A tagmap tok_contexts N/A N/A N/A tok_contexts_copy N/A N/A copy deepcopy tok_contexts N/A N/A tok contexts N/A N/A tok_contexts_copy iteritems N/A N/A context contexts N/A N/A N/A context N/A tag N/A tagmap N/A N/A context N/A tag N/A tok_contexts_copy N/A flatten N/A N/A lofl N/A N/A N/A N/A item N/A sublist lofl N/A item sublist N/A map_contexts N/A N/A tokens context_creators N/A N/A N/A N/A token N/A flatten N/A N/A N/A creator get_contexts token N/A creator context_creators N/A token tokens N/A map_context_creators N/A N/A N/A token context_creators N/A N/A N/A N/A N/A logger info N/A mapping context creators for token:  token N/A contexts N/A flatten N/A N/A N/A creator get_contexts token N/A creator context_creators N/A N/A token contexts N/A map_contexts N/A N/A tokens context_creators workers N/A 1 N/A N/A N/A workers 1 N/A N/A N/A token N/A flatten N/A N/A N/A creator get_contexts token N/A creator context_creators N/A token tokens N/A N/A pool N/A N/A multi Pool workers N/A tokens_with_extractors N/A N/A token context_creators N/A token tokens N/A res N/A N/A pool map map_context_creators tokens_with_extractors N/A res_dict N/A k v N/A N/A k v res N/A res_dict N/A token_contexts_to_features N/A N/A token_contexts feature_extractors workers N/A 1 N/A N/A N/A workers 1 N/A N/A N/A token N/A N/A np vstack N/A N/A N/A np hstack N/A N/A map_feature_extractor N/A context extractor N/A extractor feature_extractors N/A context contexts N/A N/A token contexts N/A N/A token_contexts items N/A N/A res_dict N/A N/A pool N/A N/A multi Pool workers N/A N/A Feature extractors:  feature_extractors N/A N/A token contexts N/A N/A token_contexts items N/A N/A N/A N/A logger info N/A N/A N/A N/A Multithreaded - Extracting contexts for token:  token  -- with  N/A str N/A len contexts  contexts... N/A extractors_output N/A N/A extractor feature_extractors N/A N/A context_list N/A N/A cont extractor N/A cont contexts N/A N/A N/A extractors_output append N/A N/A np vstack N/A N/A pool map map_feature_extractor context_list N/A N/A res_dict N/A token N/A N/A np hstack extractors_output N/A res_dict N/A token_contexts_to_features_categorical N/A N/A token_contexts feature_extractors workers N/A 1 N/A N/A N/A workers 1 N/A N/A N/A token N/A N/A x N/A a_list N/A N/A map_feature_extractor N/A context extractor N/A extractor feature_extractors N/A x a_list N/A context contexts N/A N/A token contexts N/A N/A token_contexts items N/A N/A res_dict N/A N/A pool N/A N/A multi Pool workers N/A N/A Feature extractors:  feature_extractors N/A N/A token contexts N/A N/A token_contexts items N/A N/A N/A N/A logger info N/A N/A N/A N/A Multithreaded - Extracting categorical contexts for token:  token  -- with  N/A str N/A len contexts  contexts... N/A extractors_output N/A N/A extractor feature_extractors N/A N/A context_list N/A N/A cont extractor N/A cont contexts N/A N/A N/A extractors_output append N/A N/A pool map map_feature_extractor context_list N/A intermediate N/A N/A N/A x N/A i N/A x extractors_output N/A i N/A range N/A len N/A extractors_output N/A 0 N/A N/A res_dict N/A token N/A N/A flatten sl N/A sl intermediate N/A res_dict N/A feature_names_from_extractor_list N/A N/A feature_extractors N/A N/A N/A 
    get a list of feature names from a list of feature extractors
    :param feature_extractors: a list of feature extractors
    :return: a list of the feature names for each extractor (think first row of .csv file)
     N/A feature_names N/A feature_name N/A feature_extractor feature_extractors N/A feature_name N/A N/A feature_extractor get_feature_names N/A feature_names N/A tags_from_contexts N/A N/A token_contexts N/A N/A N/A 
    create a dict mapping tokens to their tags
    :param token_contexts:
    :return: a dict of {<token>: [tag_i, tag_i+1, ...]}
     N/A N/A token N/A N/A np array N/A N/A context N/A tag N/A context contexts N/A N/A token contexts N/A N/A token_contexts items N/A sync_keys N/A N/A dict_a dict_b N/A N/A N/A 
    make sure two dicts have the same keys, delete the ones that are different
    :param dict_a:
    :param dict_b:
    :return:
     N/A dict_a_keys N/A set N/A N/A dict_a keys N/A dict_b_keys N/A set N/A N/A dict_b keys N/A k N/A N/A dict_a_keys symmetric_difference dict_b_keys N/A N/A N/A k dict_a_keys N/A N/A N/A dict_a N/A k N/A N/A N/A dict_b N/A k N/A