Module ImportFrom alias alias Import alias Import alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias Import alias FunctionDef arguments args NameParam NameParam NameParam NameParam defaults Lambda arguments args NameParam defaults NameLoad Num Str body Assign NameStore Call NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad Assign NameStore Call NameLoad Assign NameStore Call AttributeLoad Call NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad For NameStore NameLoad body If CompareEq NameLoad Num body Assign NameStore Call AttributeLoad Call NameLoad AttributeLoad NameLoad attr attr orelse If CompareEq NameLoad Num body Assign NameStore Call AttributeLoad Call NameLoad AttributeLoad NameLoad attr attr Assign NameStore BinOpAdd BinOpAdd ListLoad Str Str NameLoad ListLoad Str Assign NameStore Call NameLoad NameLoad NameLoad For NameStore NameLoad body AugAssignAdd SubscriptStore NameLoad Index NameLoad Num Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults Lambda arguments args NameParam defaults NameLoad Num Str body Assign NameStore Call NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad Assign NameStore Call NameLoad Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call AttributeLoad Call NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call NameLoad NameLoad keyword NameLoad keyword NameLoad keyword NameLoad For TupleStore TupleStore NameStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr body Expr Call NameLoad Call AttributeLoad Str attr NameLoad NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults Lambda arguments args NameParam defaults NameLoad Str body Assign NameStore Call NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad Assign NameStore Call NameLoad Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call AttributeLoad Call NameLoad attr Assign NameStore Call AttributeLoad Call NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad For NameStore NameLoad body If CompareEq NameLoad Num body Assign NameStore Call AttributeLoad Call NameLoad AttributeLoad NameLoad attr attr orelse If CompareEq NameLoad Num body Assign NameStore Call AttributeLoad Call NameLoad AttributeLoad NameLoad attr attr Assign NameStore Call NameLoad NameLoad Num For NameStore NameLoad body AugAssignAdd SubscriptStore NameLoad Index NameLoad Num For TupleStore TupleStore NameStore NameStore Call AttributeLoad NameLoad attr body Expr Call NameLoad Call AttributeLoad Str attr NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults Str body Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr TryExcept body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr NameLoad handlers ExceptHandler type AttributeLoad NameLoad attr body Expr Call NameLoad Call AttributeLoad Str attr NameLoad Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr NameLoad NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults Str body Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call AttributeLoad Call NameLoad attr NameLoad Assign NameStore Call AttributeLoad Call NameLoad attr NameLoad Assign NameStore Call AttributeLoad Call NameLoad attr NameLoad Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad Assign NameStore Call NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad For NameStore NameLoad body Assign TupleStore NameStore NameStore NameStore TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr NameLoad TupleLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr If UnaryOpNot NameLoad body Expr Call NameLoad Str Continue orelse Assign NameStore Num Assign NameStore NameLoad Assign NameStore SubscriptLoad NameLoad Index Num Assign NameStore NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpDiv BinOpAdd NameLoad NameLoad BinOpAdd NameLoad BinOpMult NameLoad NameLoad Expr Call NameLoad Call AttributeLoad Str attr NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpDiv NameLoad BinOpAdd NameLoad BinOpMult NameLoad NameLoad Expr Call NameLoad Call AttributeLoad Str attr NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults Str body Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call AttributeLoad Call NameLoad attr NameLoad Assign NameStore Call AttributeLoad Call NameLoad attr NameLoad Assign NameStore Call AttributeLoad Str attr NameLoad Assign NameStore Call NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad Assign NameStore Call NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Num Assign NameStore Num For NameStore NameLoad body AugAssignAdd NameStore Num AugAssignAdd NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad For NameStore NameLoad body Assign NameStore AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr Assign NameStore Num Assign NameStore NameLoad Assign NameStore NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpDiv BinOpAdd NameLoad NameLoad BinOpAdd NameLoad BinOpMult NameLoad NameLoad Expr Call NameLoad Call AttributeLoad Str attr NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults Lambda arguments args NameParam defaults NameLoad Num Str body Assign NameStore Call AttributeLoad Str attr NameLoad Expr Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad Expr Call NameLoad keyword NameLoad keyword NameLoad Expr Call NameLoad keyword NameLoad keyword NameLoad Expr Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad Expr Call NameLoad keyword NameLoad keyword NameLoad decorator_list If CompareEq NameLoad Str body Pass
N/A __future__ division print_function N/A collections N/A sqlite3 sqlalchemy create_engine sqlalchemy.orm sessionmaker smt.db.tables Tables smt.langmodel.ngram ngram N/A math _create_ngram_count_db N/A N/A lang langmethod n db N/A N/A N/A N/A x N/A x 3 sqilte:///:memory: N/A N/A engine N/A create_engine db N/A Session N/A sessionmaker bind engine N/A session N/A Session N/A Sentence N/A N/A N/A Tables get_sentence_table N/A query N/A N/A session query Sentence N/A ngram_dic N/A N/A collections defaultdict float N/A item query N/A N/A N/A lang 1 N/A N/A sentences N/A N/A N/A langmethod N/A item lang1 split N/A N/A N/A lang 2 N/A N/A sentences N/A N/A N/A langmethod N/A item lang2 split N/A sentences N/A N/A N/A </s> <s> sentences N/A </s> N/A ngrams N/A ngram sentences n N/A tpl ngrams N/A N/A N/A ngram_dic N/A tpl 1 N/A ngram_dic N/A create_ngram_count_db N/A N/A lang langmethod n db N/A N/A N/A N/A x N/A x 3 sqilte:///:memory: N/A N/A engine N/A create_engine db N/A Session N/A sessionmaker bind engine N/A session N/A Session N/A tablename N/A N/A lang{}trigram format lang N/A Trigram N/A N/A N/A Tables get_trigram_table tablename N/A N/A N/A N/A Trigram __table__ drop engine checkfirst True N/A N/A N/A N/A Trigram __table__ create engine N/A ngram_dic N/A _create_ngram_count_db lang langmethod langmethod n n db db N/A N/A N/A first second third count N/A N/A ngram_dic items N/A N/A N/A print N/A N/A inserting {}, {}, {} format first second third N/A item N/A Trigram first first second second third third count count N/A N/A N/A session add item N/A N/A N/A session commit N/A create_unigram_count_db N/A N/A lang langmethod db N/A N/A N/A N/A x N/A x sqilte:///:memory: N/A N/A engine N/A create_engine db N/A Session N/A sessionmaker bind engine N/A session N/A Session N/A tablename N/A N/A lang{}unigram format lang N/A Sentence N/A N/A N/A Tables get_sentence_table N/A Unigram N/A N/A N/A Tables get_unigram_table tablename N/A N/A N/A N/A Unigram __table__ drop engine checkfirst True N/A N/A N/A N/A Unigram __table__ create engine N/A query N/A N/A session query Sentence N/A ngram_dic N/A N/A collections defaultdict int N/A item query N/A N/A N/A lang 1 N/A N/A sentences N/A N/A N/A langmethod N/A item lang1 split N/A N/A N/A lang 2 N/A N/A sentences N/A N/A N/A langmethod N/A item lang2 split N/A ngrams N/A ngram sentences 1 N/A tpl ngrams N/A N/A N/A ngram_dic N/A tpl 1 N/A N/A N/A first count N/A N/A ngram_dic items N/A N/A N/A print N/A N/A inserting {}: {} format first count N/A item N/A Unigram first first count count N/A N/A N/A session add item N/A N/A N/A session commit N/A create_ngram_count_without_last_view N/A N/A lang db N/A :memory: N/A N/A fromtablename N/A N/A lang{}trigram format lang N/A table_name N/A N/A lang{}trigram_without_last format lang N/A con N/A N/A sqlite3 connect db N/A cur N/A N/A con cursor N/A N/A N/A N/A N/A cur execute N/A N/A drop view {0} format table_name N/A N/A N/A N/A sqlite3 Error N/A N/A N/A print N/A N/A {0} view does not exists.
              => creating a new view format table_name N/A N/A N/A cur execute N/A N/A create view {}
                as select first, second, sum(count) as count from
                {} group by first, second order by count
                desc format table_name fromtablename N/A N/A N/A con commit N/A create_ngram_prob N/A N/A lang db N/A :memory: N/A N/A table_name N/A N/A lang{}trigram_without_last format lang N/A con N/A N/A sqlite3 connect db N/A cur N/A N/A con cursor N/A trigram_tablename N/A N/A lang{}trigram format lang N/A trigramprob_tablename N/A N/A lang{}trigramprob format lang N/A trigramprobwithoutlast_tablename N/A N/A lang{}trigramprob_without_last format lang N/A Trigram N/A N/A N/A Tables get_trigram_table trigram_tablename N/A TrigramProb N/A N/A N/A Tables get_trigramprob_table trigramprob_tablename N/A TrigramProbWithoutLast N/A N/A N/A Tables get_trigramprobwithoutlast_table trigramprobwithoutlast_tablename N/A sqlalchemydb N/A N/A sqlite:///{} format db N/A engine N/A create_engine sqlalchemydb N/A Session N/A sessionmaker bind engine N/A session N/A Session N/A N/A N/A N/A TrigramProb __table__ drop engine checkfirst True N/A N/A N/A N/A TrigramProb __table__ create engine N/A N/A N/A N/A TrigramProbWithoutLast __table__ drop engine checkfirst True N/A N/A N/A N/A TrigramProbWithoutLast __table__ create engine N/A query N/A N/A session query Trigram N/A totalnumber N/A len N/A N/A query all N/A query N/A N/A session query Trigram N/A item query N/A N/A N/A first second third N/A N/A item first N/A item second N/A item third N/A count N/A item count N/A N/A N/A cur execute N/A N/A select * from {} where                     first=? and                    second=? format table_name N/A first second N/A one N/A N/A cur fetchone N/A N/A one N/A N/A N/A print not found correspont first and second N/A N/A N/A alpha 0.00017 N/A c count N/A n N/A one N/A 2 N/A v totalnumber N/A logprob N/A N/A math log N/A N/A c alpha N/A n N/A alpha v N/A N/A print N/A N/A {}, {}, {}:                  log({} + {} / {} + {} + {}) = {} format first second third c alpha n alpha v logprob N/A trigramprob N/A TrigramProb first first second second third third prob logprob N/A N/A N/A session add trigramprob N/A logprobwithoutlast N/A N/A math log N/A alpha N/A n N/A alpha v N/A N/A print N/A N/A {}, {}, {}:                  log({} / {} + {} + {}) = {} format first second third alpha n alpha v logprobwithoutlast N/A probwl N/A TrigramProbWithoutLast first first second second prob logprobwithoutlast N/A N/A N/A session add probwl N/A N/A N/A session commit N/A create_unigram_prob N/A N/A lang db N/A :memory: N/A N/A unigram_tablename N/A N/A lang{}unigram format lang N/A unigramprob_tablename N/A N/A lang{}unigramprob format lang N/A Unigram N/A N/A N/A Tables get_unigram_table unigram_tablename N/A UnigramProb N/A N/A N/A Tables get_unigramprob_table unigramprob_tablename N/A sqlalchemydb N/A N/A sqlite:///{} format db N/A engine N/A create_engine sqlalchemydb N/A Session N/A sessionmaker bind engine N/A session N/A Session N/A N/A N/A N/A UnigramProb __table__ drop engine checkfirst True N/A N/A N/A N/A UnigramProb __table__ create engine N/A query N/A N/A session query Unigram N/A sm 0 N/A totalnumber 0 N/A item query N/A N/A totalnumber 1 N/A sm N/A item count N/A query N/A N/A session query Unigram N/A item query N/A N/A first N/A item first N/A count N/A item count N/A alpha 0.00017 N/A c count N/A v totalnumber N/A logprob N/A N/A math log N/A N/A c alpha N/A sm N/A alpha v N/A N/A print N/A N/A {}:              log({}+{} / {} + {}*{}) = {} format first c alpha sm alpha v logprob N/A unigramprob N/A UnigramProb first first prob logprob N/A N/A N/A session add unigramprob N/A N/A N/A session commit N/A create_ngram_db N/A N/A lang langmethod n db N/A N/A N/A N/A x N/A x 3 :memory: N/A N/A sqlalchemydb N/A N/A sqlite:///{} format db N/A N/A create_ngram_count_db lang lang langmethod langmethod n n db sqlalchemydb N/A N/A create_ngram_count_without_last_view lang lang db db N/A N/A create_ngram_prob lang lang db db N/A N/A create_unigram_count_db lang lang langmethod langmethod db sqlalchemydb N/A N/A create_unigram_prob lang lang db db N/A N/A N/A __name__ __main__ N/A N/A