Module ImportFrom alias Import alias ImportFrom alias Import alias ImportFrom alias ImportFrom alias ImportFrom alias alias alias ImportFrom alias ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad NameLoad Num body If CompareIs NameLoad NameLoad body If CompareIs NameLoad NameLoad body If CompareIn Str AttributeLoad NameLoad attr body Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index Str orelse Expr Call NameLoad Str Return If CompareIs NameLoad NameLoad body Expr Call NameLoad Str Return Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Str Assign NameStore Call NameLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Str Expr Call NameLoad ListLoad NameLoad Str NameLoad Str NameLoad Str Call NameLoad NameLoad Str NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad For NameStore Call AttributeLoad NameLoad attr NameLoad keyword Str body Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Slice Num attr Str If CompareEq Call NameLoad NameLoad Num body Assign NameStore Call NameLoad Call AttributeLoad SubscriptLoad NameLoad Index Num attr Assign NameStore Call NameLoad SubscriptLoad NameLoad Index Num Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad orelse Expr Call NameLoad Str SubscriptLoad NameLoad Slice Num Str Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults Str body For NameStore Call NameLoad AttributeLoad NameLoad attr Num Num body If CompareEq NameLoad Str body Assign NameStore SubscriptLoad NameLoad Slice BinOpSub Call NameLoad NameLoad NameLoad orelse If CompareEq NameLoad Str body Assign NameStore SubscriptLoad NameLoad Slice NameLoad orelse Expr Call NameLoad Str NameLoad Return Num If CompareIn Call NameLoad NameLoad AttributeLoad NameLoad attr body Return NameLoad Return Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Assert CompareEq Call NameLoad NameLoad Num Assign NameStore Call NameLoad NameLoad If CompareIn NameLoad AttributeLoad NameLoad attr body Return Num orelse If BoolOpAnd CompareIn SubscriptLoad NameLoad Slice Num AttributeLoad NameLoad attr CompareIn SubscriptLoad NameLoad Slice Num AttributeLoad NameLoad attr body Return Num orelse If CompareIn SubscriptLoad NameLoad Slice Num AttributeLoad NameLoad attr body Return Num orelse If BoolOpAnd CompareIn SubscriptLoad NameLoad Slice Num AttributeLoad NameLoad attr CompareIn SubscriptLoad NameLoad Slice Num AttributeLoad NameLoad attr body Return Num orelse If BoolOpAnd CompareIn SubscriptLoad NameLoad Slice Num Num AttributeLoad NameLoad attr CompareIn SubscriptLoad NameLoad Slice Num AttributeLoad NameLoad attr body Return Num orelse If CompareIn SubscriptLoad NameLoad Slice Num AttributeLoad NameLoad attr body Return Num orelse Return Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore SubscriptLoad NameLoad Index Str Assign NameStore BinOpAdd Call NameLoad SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str keyword BinOpSub AttributeLoad NameLoad attr Num keyword NameLoad ListLoad SubscriptLoad NameLoad Index Str Assign NameStore BinOpAdd ListLoad SubscriptLoad NameLoad Index Str Call NameLoad SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str keyword BinOpSub AttributeLoad NameLoad attr Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword Str Assign NameStore BinOpAdd Call NameLoad SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str keyword Num keyword NameLoad ListLoad SubscriptLoad NameLoad Index Str Assign NameStore Call NameLoad SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str keyword NameLoad Assign NameStore BinOpAdd ListLoad SubscriptLoad NameLoad Index Str Call NameLoad SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str keyword Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Return ListLoad NameLoad NameLoad NameLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return ListLoad Str Str Str Str Str decorator_list decorator_list
N/A __future__ print_function N/A codecs subprocess call N/A os collections defaultdict marmot.features.feature_extractor FeatureExtractor marmot.util.ngram_window_extractor extract_window left_context right_context marmot.experiment.import_utils mk_tmp_dir LMFeatureExtractor N/A FeatureExtractor N/A __init__ N/A N/A self ngram_file corpus_file srilm tmp_dir order N/A None None None None 5 N/A N/A N/A ngram_file None N/A N/A N/A srilm None N/A N/A N/A SRILM N/A os environ N/A N/A srilm N/A N/A os environ N/A SRILM N/A N/A N/A print No SRILM found N/A N/A N/A corpus_file None N/A N/A N/A print No corpus for LM generation N/A N/A srilm_ngram_count N/A N/A N/A os path join srilm ngram-count N/A tmp_dir N/A mk_tmp_dir tmp_dir N/A lm_file N/A N/A N/A os path join tmp_dir lm_file N/A ngram_file N/A N/A N/A os path join tmp_dir ngram_count_file N/A N/A call N/A srilm_ngram_count -text corpus_file -lm lm_file -order N/A str order -write ngram_file N/A N/A self lm N/A defaultdict int N/A line N/A N/A codecs open ngram_file encoding utf-8 N/A N/A chunks N/A N/A N/A line N/A -1 split N/A N/A N/A len chunks 2 N/A N/A new_tuple N/A tuple N/A N/A N/A chunks N/A 0 split N/A new_number N/A int N/A chunks N/A 1 N/A N/A N/A self lm N/A new_tuple new_number N/A N/A N/A print Wrong ngram-counts file format at line ' N/A line N/A -1 ' N/A N/A self order order N/A check_lm N/A N/A self ngram side N/A left N/A N/A i N/A range N/A self order 0 -1 N/A N/A N/A side left N/A N/A cur_ngram N/A ngram N/A N/A N/A len ngram i N/A N/A N/A side right N/A N/A cur_ngram N/A ngram N/A i N/A N/A N/A print Unknown parameter 'side' side N/A 0 N/A N/A N/A tuple cur_ngram N/A self lm N/A N/A i N/A 0 N/A get_backoff N/A N/A self ngram N/A N/A N/A N/A N/A len ngram 3 N/A ngram N/A tuple ngram N/A N/A ngram N/A self lm N/A N/A 1.0 N/A N/A N/A N/A N/A ngram N/A 2 N/A self lm N/A N/A ngram N/A 1 N/A self lm N/A N/A 0.8 N/A N/A N/A N/A ngram N/A 1 N/A self lm N/A N/A 0.6 N/A N/A N/A N/A N/A ngram N/A 2 N/A self lm N/A N/A ngram N/A 2 N/A self lm N/A N/A 0.4 N/A N/A N/A N/A N/A ngram N/A 1 2 N/A self lm N/A N/A ngram N/A 2 N/A self lm N/A N/A 0.3 N/A N/A N/A N/A ngram N/A 2 N/A self lm N/A N/A 0.2 N/A N/A 0.1 N/A get_features N/A N/A self context_obj N/A N/A N/A idx N/A context_obj N/A index N/A left_ngram N/A N/A left_context N/A context_obj N/A target N/A context_obj N/A token context_size N/A N/A self order 1 idx idx N/A N/A context_obj N/A token N/A right_ngram N/A N/A N/A context_obj N/A token N/A right_context N/A context_obj N/A target N/A context_obj N/A token context_size N/A N/A self order 1 idx idx N/A left_ngram_order N/A N/A self check_lm left_ngram side left N/A right_ngram_order N/A N/A self check_lm right_ngram side right N/A left_trigram N/A N/A left_context N/A context_obj N/A target N/A context_obj N/A token context_size 2 idx idx N/A N/A context_obj N/A token N/A middle_trigram N/A extract_window N/A context_obj N/A target N/A context_obj N/A token idx idx N/A right_trigram N/A N/A N/A context_obj N/A token N/A right_context N/A context_obj N/A target N/A context_obj N/A token context_size 2 idx idx N/A backoff_left N/A N/A self get_backoff left_trigram N/A backoff_middle N/A N/A self get_backoff middle_trigram N/A backoff_right N/A N/A self get_backoff right_trigram N/A N/A left_ngram_order right_ngram_order backoff_left backoff_middle backoff_right N/A get_feature_names N/A N/A self N/A N/A N/A N/A highest_order_ngram_left highest_order_ngram_right backoff_behavior_left backoff_behavior_middle backoff_behavior_right N/A N/A