Module ImportFrom alias Import alias Import alias Import alias Import alias Import alias ImportFrom alias TryExcept body Import alias identifier handlers ExceptHandler body Import alias Import alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore Dict Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad NameLoad Str Expr Call AttributeLoad NameLoad attr NameLoad NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Dict Assign NameStore Call AttributeLoad NameLoad attr Str Call NameLoad Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad Call NameLoad NameLoad Num For NameStore Call AttributeLoad NameLoad attr NameLoad body Assign SubscriptStore NameLoad Index NameLoad Num For NameStore NameLoad body Assign SubscriptStore NameLoad Index NameLoad Num Return Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore ListLoad For NameStore NameLoad body For NameStore SubscriptLoad NameLoad Index NameLoad body If CompareEq AttributeLoad NameLoad attr Str body For TupleStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad Call AttributeLoad Str attr NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Call NameLoad Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad Call NameLoad NameLoad Num For NameStore NameLoad body AugAssignAdd SubscriptStore NameLoad Index NameLoad Num Return Call NameLoad ListComp NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults Num body Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad Assign NameStore ListLoad For NameStore NameLoad body For NameStore SubscriptLoad NameLoad Index NameLoad body If CompareEq AttributeLoad NameLoad attr Str body Assign NameStore Call AttributeLoad SubscriptLoad Call AttributeLoad Call NameLoad NameLoad NameLoad Str attr Str Index Num attr If UnaryOpNot NameLoad body Continue AugAssignAdd SubscriptStore NameLoad Index NameLoad Num For TupleStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr body Assign NameStore Dict Assign NameStore Call AttributeLoad Call AttributeLoad Str attr NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Call NameLoad Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr For NameStore NameLoad body Assign SubscriptStore NameLoad Index NameLoad Num Assign NameStore Call NameLoad Call NameLoad NameLoad Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad For NameStore NameLoad body Assign SubscriptStore NameLoad Index NameLoad Num Expr Call AttributeLoad NameLoad attr TupleLoad NameLoad NameLoad If BoolOpAnd CompareGt NameLoad Num CompareGt Call NameLoad NameLoad NameLoad body Break Print NameLoad Print Call NameLoad NameLoad Print Str Assign NameStore Call NameLoad Call NameLoad keyword Num Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Call NameLoad NameLoad Str Print Str Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr NameLoad NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults Num body Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad NameLoad Str Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Num Assign NameStore Call NameLoad Assign NameStore ListLoad For NameStore NameLoad body AugAssignAdd NameStore Num For NameStore SubscriptLoad NameLoad Index NameLoad body If CompareEq AttributeLoad NameLoad attr Str body Assign NameStore Call AttributeLoad SubscriptLoad Call AttributeLoad Call NameLoad NameLoad NameLoad Str attr Str Index Num attr If UnaryOpNot NameLoad body Continue AugAssignAdd SubscriptStore NameLoad Index NameLoad Num For TupleStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr body Assign NameStore Dict Assign NameStore Call AttributeLoad Call AttributeLoad Str attr NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Call NameLoad Call AttributeLoad NameLoad attr For NameStore Call AttributeLoad NameLoad attr body Assign SubscriptStore NameLoad Index NameLoad Num Assign NameStore Call NameLoad Call NameLoad NameLoad Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad For NameStore NameLoad body Assign SubscriptStore NameLoad Index NameLoad Num Expr Call AttributeLoad NameLoad attr TupleLoad NameLoad NameLoad Print NameLoad Print Call NameLoad NameLoad Print Str Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr ListComp NameLoad comprehension TupleStore NameStore NameStore NameLoad Assign NameStore Num For TupleStore NameStore NameStore Call NameLoad NameLoad body Assign NameStore Call NameLoad For TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Assign SubscriptStore NameLoad Index NameLoad NameLoad Assign NameStore Call NameLoad ListComp NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad If CompareIn SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index Num NameLoad body AugAssignAdd NameStore Num Print Str NameLoad Str BinOpDiv NameLoad Call NameLoad NameLoad decorator_list If CompareEq NameLoad Str body Assign NameStore Call AttributeLoad NameLoad attr keyword Str Expr Call AttributeLoad NameLoad attr Str keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr Str keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr Str keyword NameLoad keyword Num Expr Call AttributeLoad NameLoad attr Str keyword NameLoad keyword Str keyword Str Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad AttributeLoad NameLoad attr Assign NameStore BinOpMod Str AttributeLoad NameLoad attr If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad NameLoad Str Print Str orelse Print Str Assign NameStore Call NameLoad AttributeLoad NameLoad attr NameLoad Expr Call NameLoad NameLoad NameLoad Expr Call NameLoad BinOpMod Str TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad NameLoad AttributeLoad NameLoad attr Expr Call NameLoad BinOpMod Str TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad NameLoad AttributeLoad NameLoad attr
N/A collections defaultdict N/A argparse N/A sqlite3 N/A sys N/A re N/A os collections Counter N/A N/A N/A cPickle pickle N/A N/A N/A N/A pickle N/A nltk.classify.util nltk.util ngrams unidecode unidecode sklearn.linear_model LogisticRegression nltk.classify.scikitlearn SklearnClassifier util.qdb QuestionDatabase N/A alphanum N/A N/A re compile [\W_]+ N/A merged N/A Biology Physics Chemistry Mathematics Earth Science Astronomy Social Science Geography Science Science Science Science Science Science Social Studies Social Studies write_bigrams N/A N/A bigrams output N/A N/A N/A o N/A open output wb N/A N/A N/A pickle dump bigrams o N/A pickle HIGHEST_PROTOCOL N/A classify_text N/A N/A classifier text all_bigrams N/A N/A N/A feats N/A N/A total N/A N/A alphanum sub N/A unidecode N/A N/A text lower N/A total N/A N/A total split N/A bgs N/A set N/A ngrams total 2 N/A bg N/A N/A bgs intersection all_bigrams N/A N/A N/A feats N/A bg 1.0 N/A word total N/A N/A N/A feats N/A word 1.0 N/A N/A N/A classifier prob_classify feats N/A compute_frequent_bigrams N/A N/A thresh qbdb N/A N/A N/A bcount N/A Counter N/A all_questions N/A N/A qbdb questions_with_pages N/A train N/A N/A page all_questions N/A N/A qq N/A all_questions N/A page N/A N/A N/A N/A qq fold train N/A N/A N/A ss ww tt N/A N/A qq partials N/A N/A total N/A N/A N/A N/A join tt strip N/A total N/A N/A alphanum sub N/A unidecode N/A N/A total lower N/A total N/A N/A total split N/A bgs N/A list N/A ngrams total 2 N/A bg bgs N/A N/A N/A bcount N/A bg 1 N/A N/A set N/A k N/A N/A k v N/A N/A bcount most_common thresh N/A train_classifier N/A N/A out bgset questions attribute limit N/A -1 N/A N/A all_questions N/A N/A questions questions_with_pages N/A c N/A Counter N/A train N/A N/A page all_questions N/A N/A qq N/A all_questions N/A page N/A N/A N/A N/A qq fold train N/A N/A label N/A N/A N/A N/A N/A N/A getattr qq attribute split : N/A 0 lower N/A N/A label N/A N/A N/A N/A c N/A label 1 N/A N/A ss ww tt N/A N/A qq partials N/A N/A feats N/A N/A total N/A N/A N/A N/A join tt strip N/A total N/A N/A alphanum sub N/A unidecode N/A N/A total lower N/A total N/A N/A total split N/A word total N/A N/A N/A feats N/A word 1.0 N/A currbg N/A set N/A ngrams total 2 N/A inter N/A N/A currbg intersection bgset N/A elem inter N/A N/A N/A feats N/A elem 1.0 N/A N/A N/A train append N/A feats label N/A N/A N/A limit 0 N/A N/A len train limit N/A N/A N/A c N/A N/A len train N/A training classifier N/A classifier N/A SklearnClassifier N/A LogisticRegression C 10 N/A N/A N/A classifier train train N/A N/A N/A pickle dump classifier N/A open out wb N/A accuracy@1 train: N/A N/A N/A N/A nltk classify util accuracy classifier train N/A classifier N/A evaluate N/A N/A classifier_file bgset questions attribute top N/A 2 N/A N/A classifier N/A N/A pickle load N/A open classifier_file rb N/A all_questions N/A N/A questions questions_with_pages N/A page_num 0 N/A c N/A Counter N/A dev N/A N/A page all_questions N/A N/A page_num 1 N/A qq N/A all_questions N/A page N/A N/A N/A N/A qq fold dev N/A N/A label N/A N/A N/A N/A N/A N/A getattr qq attribute split : N/A 0 lower N/A N/A label N/A N/A N/A N/A c N/A label 1 N/A N/A ss ww tt N/A N/A qq partials N/A N/A feats N/A N/A total N/A N/A N/A N/A join tt strip N/A total N/A N/A alphanum sub N/A unidecode N/A N/A total lower N/A word N/A N/A total split N/A N/A N/A feats N/A word 1.0 N/A currbg N/A set N/A ngrams total 2 N/A inter N/A N/A currbg intersection bgset N/A elem inter N/A N/A N/A feats N/A elem 1.0 N/A N/A N/A dev append N/A feats label N/A c N/A N/A len dev N/A accuracy@1 dev: N/A N/A N/A N/A nltk classify util accuracy classifier dev N/A probs N/A N/A classifier prob_classify_many N/A f N/A N/A f a dev N/A corr 0.0 N/A N/A index prob N/A enumerate probs N/A N/A c N/A Counter N/A N/A k v N/A N/A N/A prob _prob_dict items N/A N/A N/A c N/A k v N/A topn N/A set N/A k N/A N/A k v N/A N/A c most_common top N/A N/A N/A N/A dev N/A index N/A 1 topn N/A N/A corr 1 N/A top@ top accuracy:  N/A corr N/A len probs N/A N/A N/A __name__ __main__ N/A N/A parser N/A N/A argparse ArgumentParser description N/A N/A N/A parser add_argument --question_db type str default data/questions.db N/A N/A N/A parser add_argument --attribute type str default category N/A N/A N/A parser add_argument --bigram_thresh type int default 1000 N/A N/A N/A parser add_argument --output type str default data/classifier/ help Where we write output file N/A flags N/A N/A parser parse_args N/A questions N/A QuestionDatabase N/A flags question_db N/A bigram_filename N/A %s/bigrams.pkl N/A flags output N/A N/A N/A N/A os path exists bigram_filename N/A N/A bgset N/A N/A pickle load N/A open bigram_filename rb N/A Using previous bigrams N/A N/A computing bigrams... N/A bgset N/A compute_frequent_bigrams N/A flags bigram_thresh questions N/A N/A write_bigrams bgset bigram_filename N/A N/A train_classifier N/A %s/%s.pkl N/A N/A flags output N/A flags attribute bgset questions N/A flags attribute N/A N/A evaluate N/A %s/%s.pkl N/A N/A flags output N/A flags attribute bgset questions N/A flags attribute