Module Import alias alias alias alias alias alias alias Import alias alias Expr Str Assign NameStore BinOpAdd Call AttributeLoad AttributeLoad NameLoad attr attr Str Str Assign NameStore BinOpAdd Call AttributeLoad AttributeLoad NameLoad attr attr Str Str FunctionDef arguments args defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr ListLoad Str Str Str Str Call NameLoad NameLoad Assign NameStore Call NameLoad Call NameLoad Lambda arguments args NameParam defaults BoolOpAnd NameLoad CompareEq SubscriptLoad NameLoad Index Num Str Call AttributeLoad SubscriptLoad NameLoad Slice Num attr Call AttributeLoad NameLoad attr Str Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad Call NameLoad AttributeLoad NameLoad attr attr If UnaryOpNot NameLoad body Assign NameStore Str orelse Assign NameStore SubscriptLoad NameLoad Index Num If AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad Call NameLoad AttributeLoad AttributeLoad NameLoad attr attr attr If UnaryOpNot NameLoad body Assign NameStore Str orelse Assign NameStore SubscriptLoad NameLoad Index Num orelse Assign NameStore ListLoad Assign NameStore Str Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Assign NameStore Dict Assign SubscriptStore NameLoad Index Str NameLoad Assign SubscriptStore NameLoad Index Str NameLoad Assign SubscriptStore NameLoad Index Str BinOpAdd BinOpAdd NameLoad Str NameLoad Assign NameStore Call NameLoad Num Call NameLoad Call AttributeLoad NameLoad attr Str Str NameLoad If Call AttributeLoad Call AttributeLoad NameLoad attr Str Str attr body Assign SubscriptStore NameLoad Index Str Call NameLoad NameLoad TryExcept body Assign SubscriptStore NameLoad Index Str Call NameLoad Call NameLoad Call AttributeLoad NameLoad attr BinOpAdd Num SubscriptLoad AttributeLoad NameLoad attr Index SubscriptLoad NameLoad Slice Num handlers ExceptHandler body Assign SubscriptStore NameLoad Index Str Call NameLoad Call NameLoad Call AttributeLoad NameLoad attr Num If Call AttributeLoad Call AttributeLoad NameLoad attr Str Str attr body Assign SubscriptStore NameLoad Index Str Call NameLoad Call AttributeLoad SubscriptLoad NameLoad Index Num attr TryExcept body Assign SubscriptStore NameLoad Index Str Call NameLoad Call NameLoad Call AttributeLoad NameLoad attr BinOpAdd Num SubscriptLoad AttributeLoad NameLoad attr Index Call AttributeLoad NameLoad attr handlers ExceptHandler body Assign SubscriptStore NameLoad Index Str Call NameLoad Call NameLoad Call AttributeLoad NameLoad attr Num Assign SubscriptStore NameLoad Index Str BinOpAdd BinOpAdd NameLoad Str Call NameLoad Call AttributeLoad SubscriptLoad NameLoad Index Num attr Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str If UnaryOpNot CompareIn Str NameLoad body Return NameLoad Assign NameStore Call NameLoad NameLoad If Call AttributeLoad NameLoad attr Str body Return NameLoad If Call AttributeLoad NameLoad attr Str NameLoad body Return NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad body Expr Str If CompareEq Call NameLoad NameLoad Call NameLoad Str body Assign NameStore ListLoad NameLoad Assign NameStore NameLoad Assign NameStore Num Assign NameStore Num Assign NameStore ListLoad Assign TupleStore NameStore NameStore TupleLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr For NameStore NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad Assign NameStore Call NameLoad NameLoad For NameStore NameLoad body If BoolOpAnd UnaryOpNot Call AttributeLoad NameLoad attr NameLoad body If UnaryOpNot NameLoad body Assign AttributeStore NameLoad attr NameLoad orelse Assign NameStore Call NameLoad Call AttributeLoad Str attr NameLoad Assign AttributeStore NameLoad attr NameLoad If NameLoad body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad AugAssignAdd NameStore Num Assign NameStore NameLoad Assign NameStore ListLoad For NameStore Call AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr NameLoad If Call NameLoad NameLoad body Assign NameStore Call NameLoad Call AttributeLoad Str attr NameLoad If UnaryOpNot NameLoad body Assign NameStore NameLoad orelse Assign AttributeStore NameLoad attr NameLoad If NameLoad body Assign AttributeStore NameLoad attr Call NameLoad CompareIn Str NameLoad If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr orelse Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Assign AttributeStore NameLoad attr NameLoad AugAssignAdd NameStore Num Assign NameStore NameLoad Assign NameStore ListLoad AugAssignAdd NameStore Num Expr Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad Call AttributeLoad Str attr NameLoad If UnaryOpNot NameLoad body Assign NameStore NameLoad orelse Assign AttributeStore NameLoad attr NameLoad If NameLoad body Assign AttributeStore NameLoad attr Call NameLoad CompareIn Str NameLoad If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr orelse Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad AugAssignAdd NameStore Num If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str TupleLoad NameLoad NameLoad Assign NameStore Call NameLoad NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad body Expr Str Assign NameStore NameLoad Assign NameStore Num Assign NameStore Num Assign NameStore ListLoad Assign TupleStore NameStore NameStore TupleLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr For NameStore Call AttributeLoad NameLoad attr body If BoolOpAnd UnaryOpNot Call AttributeLoad NameLoad attr NameLoad body If UnaryOpNot NameLoad body Assign AttributeStore NameLoad attr NameLoad orelse Assign NameStore Call NameLoad Call AttributeLoad Str attr NameLoad Assign AttributeStore NameLoad attr NameLoad If NameLoad body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad AugAssignAdd NameStore Num Assign NameStore NameLoad Assign NameStore ListLoad For NameStore Call AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr NameLoad If Call NameLoad NameLoad body Assign NameStore Call NameLoad Call AttributeLoad Str attr NameLoad If UnaryOpNot NameLoad body Assign NameStore NameLoad orelse Assign AttributeStore NameLoad attr NameLoad If NameLoad body Assign AttributeStore NameLoad attr Call NameLoad CompareIn Str NameLoad If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr orelse Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Assign AttributeStore NameLoad attr NameLoad AugAssignAdd NameStore Num Assign NameStore NameLoad Assign NameStore ListLoad AugAssignAdd NameStore Num Assign NameStore Call NameLoad Call AttributeLoad Str attr NameLoad If UnaryOpNot NameLoad body Assign NameStore NameLoad orelse Assign AttributeStore NameLoad attr NameLoad If NameLoad body Assign AttributeStore NameLoad attr Call NameLoad CompareIn Str NameLoad If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr orelse Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Str NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad AugAssignAdd NameStore Num If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str TupleLoad NameLoad NameLoad Assign NameStore Call NameLoad NameLoad Return NameLoad decorator_list ClassDef bases body Expr Str FunctionDef arguments args NameParam NameParam defaults body Assign TupleStore AttributeStore NameLoad attr AttributeStore NameLoad attr AttributeStore NameLoad attr TupleLoad Dict Dict Dict Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign TupleStore AttributeStore NameLoad attr AttributeStore NameLoad attr Call AttributeLoad NameLoad attr keyword NameLoad Assign AttributeStore NameLoad attr Call NameLoad AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Expr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Str decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Str Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Str Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Str decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr While NameLoad body For TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body AugAssignAdd SubscriptStore SubscriptLoad NameLoad Index AttributeLoad NameLoad attr Index BinOpAdd BinOpAdd NameLoad Str NameLoad Num AugAssignAdd SubscriptStore NameLoad Index AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore Num Assign NameStore Call AttributeLoad Call NameLoad Call AttributeLoad SubscriptLoad NameLoad Index NameLoad attr attr Call NameLoad Call AttributeLoad SubscriptLoad NameLoad Index NameLoad attr For NameStore ListLoad Num Num body AugAssignAdd SubscriptStore NameLoad Index NameLoad BinOpMult Call NameLoad NameLoad NameLoad For NameStore NameLoad body AugAssignAdd SubscriptStore SubscriptLoad NameLoad Index NameLoad Index NameLoad NameLoad AugAssignDiv SubscriptStore SubscriptLoad NameLoad Index NameLoad Index NameLoad SubscriptLoad NameLoad Index NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index TupleLoad NameLoad NameLoad SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index NameLoad Assign SubscriptStore SubscriptLoad NameLoad Index NameLoad Index Str BinOpDiv SubscriptLoad NameLoad Index NameLoad Call AttributeLoad NameLoad attr Assign SubscriptStore AttributeLoad NameLoad attr Index TupleLoad NameLoad Str SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr ListComp TupleLoad NameLoad BinOpPow SubscriptLoad AttributeLoad NameLoad attr Index TupleLoad NameLoad Str Num comprehension NameStore ListLoad Num Num For NameStore NameLoad body For TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Assign NameStore TupleLoad NameLoad BinOpAdd BinOpAdd NameLoad Str NameLoad If UnaryOpNot CompareIn NameLoad AttributeLoad NameLoad attr body Continue AugAssignMult SubscriptStore NameLoad Index NameLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Return SubscriptLoad NameLoad Index Num decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore AttributeLoad NameLoad attr While NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign NameStore AttributeLoad NameLoad attr If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore BinOpMod Str AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore Call NameLoad Assign NameStore AttributeLoad NameLoad attr While NameLoad body Assign NameStore ListComp BinOpAdd BinOpAdd NameLoad Str NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr For NameStore NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call NameLoad Call NameLoad NameLoad Call NameLoad Num BinOpAdd Call NameLoad NameLoad Num Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore ListLoad Assign NameStore AttributeLoad NameLoad attr While NameLoad body If CompareEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad orelse If CompareGt AttributeLoad NameLoad attr Num body Assign NameStore Str orelse If CompareLt AttributeLoad NameLoad attr Num body Assign NameStore Str orelse Continue Assign NameStore BinOpMod Str NameLoad Assign NameStore ListComp BinOpAdd BinOpAdd NameLoad Str NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore ListComp SubscriptLoad AttributeLoad NameLoad attr Index NameLoad comprehension NameStore NameLoad Expr Call AttributeLoad NameLoad attr Lambda arguments args NameParam NameParam defaults BinOpSub NameLoad NameLoad AugAssignAdd NameStore Call AttributeLoad Str attr ListComp BinOpMod Str NameLoad comprehension NameStore NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore AttributeLoad NameLoad attr Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad Str Expr Call AttributeLoad NameLoad attr BinOpAdd Call AttributeLoad Str attr NameLoad Str Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore Str Assign NameStore BinOpMod Str TupleLoad NameLoad NameLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Assign NameStore BinOpMod Str AttributeLoad NameLoad attr If UnaryOpNot AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Str If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore ListLoad Assign NameStore AttributeLoad NameLoad attr While NameLoad body If CompareEq AttributeLoad NameLoad attr NameLoad body Assign NameStore Str orelse If AttributeLoad NameLoad attr body Assign NameStore Str orelse Assign NameStore Str Assign NameStore BinOpMod Str NameLoad Assign NameStore ListComp BinOpAdd BinOpAdd NameLoad Str NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore ListComp SubscriptLoad AttributeLoad NameLoad attr Index NameLoad comprehension NameStore NameLoad CompareIn NameLoad AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Lambda arguments args NameParam NameParam defaults BinOpSub NameLoad NameLoad AugAssignAdd NameStore Call AttributeLoad Str attr ListComp BinOpMod Str NameLoad comprehension NameStore NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore AttributeLoad NameLoad attr Assign NameStore Str Assign NameStore Call NameLoad NameLoad Str Expr Call AttributeLoad NameLoad attr BinOpAdd Call AttributeLoad Str attr NameLoad Str Expr Call AttributeLoad NameLoad attr Assign NameStore Str Assign NameStore Str Assign NameStore BinOpMod Str TupleLoad NameLoad NameLoad NameLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Num Assign NameStore Call NameLoad NameLoad Str Assign NameStore Call NameLoad NameLoad Call AttributeLoad Call AttributeLoad NameLoad attr attr Assign NameStore AttributeLoad NameLoad attr While NameLoad body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index NameLoad Assign NameStore AttributeLoad NameLoad attr AugAssignAdd NameStore Num Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str decorator_list decorator_list ClassDef bases body Expr Str FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore ListLoad Assign NameStore AttributeLoad NameLoad attr While NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad Return Call AttributeLoad Str attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr While NameLoad body For NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body If Call AttributeLoad Call AttributeLoad NameLoad attr Str Str attr body If Call AttributeLoad NameLoad attr body AugAssignAdd SubscriptStore NameLoad Index Call AttributeLoad NameLoad attr Str Str Num If UnaryOpNot Call AttributeLoad NameLoad attr Str body AugAssignAdd SubscriptStore NameLoad Index NameLoad Num Assign NameStore AttributeLoad NameLoad attr If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str TupleLoad Call NameLoad NameLoad Call NameLoad NameLoad Return TupleLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore AttributeLoad NameLoad attr While NameLoad body Assign AttributeStore NameLoad attr Call NameLoad NameLoad NameLoad Assign NameStore AttributeLoad NameLoad attr If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad NameLoad body Expr Str Assign NameStore ListLoad Assign NameStore Num Assign NameStore ListLoad Assign NameStore AttributeLoad NameLoad attr While NameLoad body If NameLoad body Assign NameStore AttributeLoad NameLoad attr orelse Assign NameStore AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad If BoolOpOr AttributeLoad NameLoad attr BoolOpAnd NameLoad CompareGt AttributeLoad NameLoad attr NameLoad BoolOpAnd UnaryOpNot NameLoad CompareGt AttributeLoad NameLoad attr NameLoad body If UnaryOpNot AttributeLoad NameLoad attr body Break Assign NameStore Call AttributeLoad Str attr NameLoad If AttributeLoad NameLoad attr body Assign NameStore Str orelse Assign NameStore Str If NameLoad body Expr Call AttributeLoad NameLoad attr BinOpAdd NameLoad NameLoad orelse If UnaryOpNot NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpAdd NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore ListLoad Assign NameStore AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults NameLoad body Assign NameStore Num Assign TupleStore NameStore NameStore TupleLoad Num Num Assign NameStore AttributeLoad NameLoad attr While NameLoad body AugAssignAdd NameStore Num If CompareEq AttributeLoad NameLoad attr CompareGt AttributeLoad NameLoad attr NameLoad body AugAssignAdd NameStore Num orelse Assign NameStore Call AttributeLoad Str attr SubscriptLoad Call AttributeLoad AttributeLoad NameLoad attr attr Slice Num If AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad Str attr SubscriptLoad Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Slice Num orelse Assign NameStore Str If NameLoad body Print BinOpMod Str TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore AttributeLoad NameLoad attr Assign NameStore BinOpSub Num BinOpDiv BinOpMult Num NameLoad NameLoad Print BinOpMod Str TupleLoad NameLoad NameLoad NameLoad decorator_list decorator_list ClassDef bases body Expr Str FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body AugAssignAdd NameStore Str Return NameLoad decorator_list decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad NameLoad keyword AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign NameStore Call NameLoad AttributeLoad NameLoad attr orelse Assign NameStore Call NameLoad AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults Str NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad If NameLoad body Assign NameStore Call NameLoad NameLoad orelse Assign NameStore Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Expr Str Assign NameStore Call NameLoad NameLoad keyword NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword NameLoad Return NameLoad decorator_list If CompareEq NameLoad Str body Assign NameStore Str Assign NameStore Call NameLoad NameLoad NameLoad Assign NameStore Str Assign NameStore ListComp NameLoad comprehension NameStore Call NameLoad NameLoad Assign NameStore Str For NameStore NameLoad body If CompareGt Call NameLoad Call AttributeLoad NameLoad attr Num body Print Str NameLoad AugAssignAdd NameStore BinOpAdd Str Call AttributeLoad NameLoad attr Print NameLoad Assign NameStore Call NameLoad NameLoad NameLoad For NameStore NameLoad body Print Str NameLoad Expr Str FunctionDef arguments args defaults body Assign NameStore Str Assign NameStore BinOpAdd NameLoad Str Assign NameStore BinOpAdd NameLoad Str Assign NameStore BinOpAdd NameLoad Str Assign NameStore BinOpAdd NameLoad Str Assign NameStore Str ImportFrom alias Assign NameStore Str Assign NameStore Call NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr Str Str keyword Str keyword NameLoad keyword Str keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword Str keyword NameLoad keyword Str keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword Str keyword Str keyword Str keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword Str keyword Str keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword Str keyword Str keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr Str Str keyword Str keyword NameLoad keyword Str keyword Str Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr If CompareGt Call NameLoad NameLoad Num body Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Num If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr BinOpMod Str AttributeLoad NameLoad attr orelse Assign AttributeStore NameLoad attr NameLoad If UnaryOpNot AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Str If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr Str body AugAssignAdd AttributeStore NameLoad attr Str If AttributeLoad NameLoad attr body If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr BinOpMod Str AttributeLoad NameLoad attr If Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr BinOpMod Str AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr orelse If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr BinOpAdd NameLoad AttributeLoad NameLoad attr If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr BinOpMod Str AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign NameStore Call NameLoad AttributeLoad NameLoad attr NameLoad If UnaryOpNot AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Print AttributeLoad NameLoad attr Print AttributeLoad NameLoad attr If UnaryOpNot AttributeLoad NameLoad attr body If CompareIn Str AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call NameLoad AttributeLoad NameLoad attr Str Assign NameStore Call NameLoad AttributeLoad NameLoad attr keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad keyword NameLoad Print AttributeLoad NameLoad attr Print AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr keyword NameLoad keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr decorator_list
N/A N/A re sys os math tempfile collections subprocess N/A sbd_util word_tokenize N/A 
Utilities for disambiguating sentence boundaries
Copyright Dan Gillick, 2009.

TODO:
- capitalized headlines screw things up?
- deal with ?! maybe just assume these sentence boundaries always
 N/A SVM_LEARN N/A N/A N/A N/A os path abspath /lib/svmlite/svm_learn N/A SVM_CLASSIFY N/A N/A N/A N/A os path abspath /lib/svmlite/svm_classify get_open_fds N/A N/A N/A N/A N/A 
    return the number of open file descriptors for current process

    .. warning: will only work on UNIX-like os-es.
     N/A pid N/A N/A os getpid N/A procs N/A N/A subprocess check_output N/A lsof -w -Ff -p N/A str pid N/A nprocs N/A len N/A filter N/A N/A N/A s N/A N/A s N/A N/A s N/A 0 f N/A N/A N/A s N/A 1 isdigit N/A N/A procs split N/A nprocs N/A unannotate N/A N/A t N/A N/A N/A 
    get rid of a tokenized word's annotations
     N/A t N/A N/A re sub (<A>)?(<E>)?(<S>)?$ t N/A t N/A clean N/A N/A t N/A N/A N/A 
    normalize numbers, discard some punctuation that can be ambiguous
     N/A t N/A N/A re sub [.,\d]*\d <NUM> t N/A t N/A N/A re sub [^a-zA-Z0-9,.;:<>\-'\/?!$% ] t N/A t N/A N/A t replace -- N/A t N/A get_features N/A N/A frag model N/A N/A N/A 
    ... w1. (sb?) w2 ...
    Features, listed roughly in order of importance:

    (1) w1: word that includes a period
    (2) w2: the next word, if it exists
    (3) w1length: number of alphabetic characters in w1
    (4) w2cap: true if w2 is capitalized
    (5) both: w1 and w2
    (6) w1abbr: log count of w1 in training without a final period
    (7) w2lower: log count of w2 in training as lowercased
    (8) w1w2upper: w1 and w2 is capitalized
     N/A words1 N/A N/A N/A clean N/A frag tokenized split N/A N/A words1 N/A N/A w1 N/A N/A w1 N/A words1 N/A -1 N/A N/A frag next N/A N/A words2 N/A N/A N/A clean N/A N/A frag next tokenized split N/A N/A words2 N/A N/A w2 N/A N/A w2 N/A words2 N/A 0 N/A N/A words2 N/A N/A w2 N/A c1 N/A N/A re sub (^.+?\-) w1 N/A c2 N/A N/A re sub (\-.+?)$ w2 N/A feats N/A N/A N/A feats N/A w1 c1 N/A N/A feats N/A w2 c2 N/A N/A feats N/A both N/A N/A c1 _ c2 N/A len1 N/A min 10 N/A len N/A N/A re sub \W c1 N/A N/A N/A N/A N/A c1 replace . isalpha N/A N/A N/A feats N/A w1length N/A str len1 N/A N/A N/A N/A feats N/A w1abbr N/A str N/A int N/A N/A math log N/A 1 N/A N/A model non_abbrs N/A N/A c1 N/A -1 N/A N/A N/A N/A N/A feats N/A w1abbr N/A str N/A int N/A N/A math log 1 N/A N/A N/A N/A N/A c2 replace . isalpha N/A N/A N/A feats N/A w2cap N/A str N/A N/A N/A c2 N/A 0 isupper N/A N/A N/A N/A feats N/A w2lower N/A str N/A int N/A N/A math log N/A 1 N/A N/A model lower_words N/A N/A N/A c2 lower N/A N/A N/A N/A N/A feats N/A w2lower N/A str N/A int N/A N/A math log 1 N/A N/A feats N/A w1w2upper N/A N/A c1 _ N/A str N/A N/A N/A c2 N/A 0 isupper N/A feats N/A is_sbd_hyp N/A N/A word N/A N/A N/A 
    todo: expand to ?!
     N/A N/A N/A . word N/A N/A False N/A c N/A unannotate word N/A N/A N/A c endswith . N/A N/A True N/A N/A N/A re match .*\.["')\]]*$ c N/A N/A True N/A False N/A get_data N/A N/A files expect_labels tokenize verbose N/A True False False N/A N/A 
    load text from files, returning an instance of the Doc class
    doc.frag is the first frag, and each points to the next
     N/A N/A N/A type files N/A type N/A N/A files N/A files N/A frag_list None N/A word_index 0 N/A frag_index 0 N/A curr_words N/A N/A N/A lower_words non_abbrs N/A N/A N/A sbd_util Counter N/A N/A sbd_util Counter N/A file files N/A N/A N/A N/A N/A sys stderr write N/A reading [%s]
 file N/A fh N/A open file N/A line fh N/A N/A N/A N/A N/A N/A line strip frag_list N/A N/A N/A curr_words N/A N/A N/A frag ends_seg True N/A N/A frag N/A Frag N/A N/A join curr_words N/A N/A frag ends_seg True N/A expect_labels N/A N/A N/A frag label True N/A N/A prev next frag N/A tokenize N/A N/A tokens N/A N/A word_tokenize tokenize N/A frag orig N/A N/A frag tokenized tokens N/A frag_index 1 N/A prev frag N/A curr_words N/A N/A word N/A N/A line split N/A N/A N/A N/A curr_words append word N/A N/A is_sbd_hyp word N/A N/A frag N/A Frag N/A N/A join curr_words N/A N/A frag_list N/A N/A frag_list frag N/A N/A N/A prev next frag N/A expect_labels N/A N/A N/A frag label N/A int N/A <S> word N/A tokenize N/A N/A tokens N/A N/A word_tokenize tokenize N/A frag orig N/A N/A tokens N/A frag orig N/A tokens N/A N/A re sub (<A>)|(<E>)|(<S>) tokens N/A N/A frag tokenized tokens N/A frag_index 1 N/A prev frag N/A curr_words N/A N/A word_index 1 N/A N/A N/A fh close N/A frag N/A Frag N/A N/A join curr_words N/A N/A frag_list N/A N/A frag_list frag N/A N/A N/A prev next frag N/A expect_labels N/A N/A N/A frag label N/A int N/A <S> word N/A tokenize N/A N/A tokens N/A N/A word_tokenize tokenize N/A frag orig N/A N/A tokens N/A frag orig N/A tokens N/A N/A re sub (<A>)|(<E>)|(<S>) tokens N/A N/A frag tokenized tokens N/A N/A frag ends_seg True N/A frag_index 1 N/A verbose N/A N/A N/A N/A N/A sys stderr write N/A  words [%d] sbd hyps [%d]
 N/A word_index frag_index N/A doc N/A Doc frag_list N/A doc N/A get_text_data N/A N/A text expect_labels tokenize verbose N/A True False False N/A N/A 
    get text, returning an instance of the Doc class
    doc.frag is the first frag, and each points to the next
     N/A frag_list None N/A word_index 0 N/A frag_index 0 N/A curr_words N/A N/A N/A lower_words non_abbrs N/A N/A N/A sbd_util Counter N/A N/A sbd_util Counter N/A line N/A N/A text splitlines N/A N/A N/A N/A N/A N/A line strip frag_list N/A N/A N/A curr_words N/A N/A N/A frag ends_seg True N/A N/A frag N/A Frag N/A N/A join curr_words N/A N/A frag ends_seg True N/A expect_labels N/A N/A N/A frag label True N/A N/A prev next frag N/A tokenize N/A N/A tokens N/A N/A word_tokenize tokenize N/A frag orig N/A N/A frag tokenized tokens N/A frag_index 1 N/A prev frag N/A curr_words N/A N/A word N/A N/A line split N/A N/A N/A N/A curr_words append word N/A N/A is_sbd_hyp word N/A N/A frag N/A Frag N/A N/A join curr_words N/A N/A frag_list N/A N/A frag_list frag N/A N/A N/A prev next frag N/A expect_labels N/A N/A N/A frag label N/A int N/A <S> word N/A tokenize N/A N/A tokens N/A N/A word_tokenize tokenize N/A frag orig N/A N/A tokens N/A frag orig N/A tokens N/A N/A re sub (<A>)|(<E>)|(<S>) tokens N/A N/A frag tokenized tokens N/A frag_index 1 N/A prev frag N/A curr_words N/A N/A word_index 1 N/A frag N/A Frag N/A N/A join curr_words N/A N/A frag_list N/A N/A frag_list frag N/A N/A N/A prev next frag N/A expect_labels N/A N/A N/A frag label N/A int N/A <S> word N/A tokenize N/A N/A tokens N/A N/A word_tokenize tokenize N/A frag orig N/A N/A tokens N/A frag orig N/A tokens N/A N/A re sub (<A>)|(<E>)|(<S>) tokens N/A N/A frag tokenized tokens N/A N/A frag ends_seg True N/A frag_index 1 N/A verbose N/A N/A N/A N/A N/A sys stderr write N/A  words [%d] sbd hyps [%d]
 N/A word_index frag_index N/A doc N/A Doc frag_list N/A doc N/A Model N/A N/A N/A 
    Abstract Model class holds all relevant information, and includes
    train and classify functions
     __init__ N/A N/A self path N/A N/A N/A N/A N/A self feats N/A self lower_words N/A self non_abbrs N/A N/A N/A N/A N/A N/A self path path N/A prep N/A N/A self doc N/A N/A N/A N/A N/A self lower_words N/A self non_abbrs N/A N/A doc get_stats verbose False N/A N/A self lower_words N/A dict N/A self lower_words N/A N/A self non_abbrs N/A dict N/A self non_abbrs N/A train N/A N/A self doc N/A N/A N/A abstract N/A classify N/A N/A self doc verbose N/A False N/A N/A abstract N/A save N/A N/A self N/A N/A N/A 
        save model objects in self.path
         N/A N/A N/A sbd_util save_pickle N/A self feats N/A N/A self path feats N/A N/A N/A sbd_util save_pickle N/A self lower_words N/A N/A self path lower_words N/A N/A N/A sbd_util save_pickle N/A self non_abbrs N/A N/A self path non_abbrs N/A load N/A N/A self N/A N/A N/A 
        load model objects from p
         N/A N/A self feats N/A N/A sbd_util load_pickle N/A N/A self path feats N/A N/A self lower_words N/A N/A sbd_util load_pickle N/A N/A self path lower_words N/A N/A self non_abbrs N/A N/A sbd_util load_pickle N/A N/A self path non_abbrs N/A N/A NB_Model N/A Model N/A N/A 
    Naive Bayes model, with a few tweaks:
    - all feature types are pooled together for normalization (this might help
      because the independence assumption is so broken for our features)
    - smoothing: add 0.1 to all counts
    - priors are modified for better performance (this is mysterious but works much better)
     train N/A N/A self doc N/A N/A N/A N/A N/A N/A sys stderr write training nb...  N/A feats N/A N/A collections defaultdict N/A sbd_util Counter N/A totals N/A N/A sbd_util Counter N/A frag N/A doc frag N/A frag N/A N/A N/A feat val N/A N/A N/A frag features items N/A N/A N/A N/A feats N/A N/A frag label N/A N/A N/A feat _ val 1 N/A N/A totals N/A N/A frag label N/A len N/A frag features N/A frag N/A frag next N/A N/A N/A N/A sys stderr write smoothing...  N/A smooth_inc 0.1 N/A all_feat_names N/A N/A N/A set N/A N/A N/A feats N/A True keys union N/A set N/A N/A N/A feats N/A False keys N/A label N/A 0 1 N/A N/A N/A totals N/A label N/A N/A len all_feat_names smooth_inc N/A feat all_feat_names N/A N/A N/A N/A feats N/A label N/A feat smooth_inc N/A N/A N/A feats N/A label N/A feat N/A totals N/A label N/A N/A N/A self feats N/A N/A label feat N/A N/A feats N/A label N/A feat N/A N/A N/A feats N/A label N/A <prior> N/A N/A totals N/A label N/A N/A totals totalCount N/A N/A N/A self feats N/A N/A label <prior> N/A N/A feats N/A label N/A <prior> N/A N/A N/A N/A sys stderr write done!
 N/A classify_nb_one N/A N/A self frag N/A N/A N/A probs N/A N/A sbd_util Counter N/A N/A label N/A N/A N/A self feats N/A N/A label <prior> 4 N/A label N/A 0 1 N/A label probs N/A N/A N/A feat val N/A N/A N/A frag features items N/A N/A key N/A label N/A N/A feat _ val N/A N/A N/A key N/A self feats N/A N/A N/A N/A probs N/A label N/A N/A self feats N/A key N/A probs N/A N/A sbd_util normalize probs N/A N/A probs N/A 1 N/A classify N/A N/A self doc verbose N/A False N/A N/A verbose N/A N/A N/A N/A N/A sys stderr write NB classifying...  N/A frag N/A doc frag N/A frag N/A N/A pred N/A N/A self classify_nb_one frag N/A N/A frag pred pred N/A frag N/A frag next N/A verbose N/A N/A N/A N/A N/A sys stderr write done!
 N/A N/A SVM_Model N/A Model N/A N/A 
    SVM model (using SVM Light), with a linear kernel, C parameter set to 1
    Non-exhaustive testing of other kernels and parameters showed no improvement
     train N/A N/A self doc N/A N/A N/A 
        takes training data and a path and creates an svm model
         N/A model_file N/A %ssvm_model N/A self path N/A N/A N/A N/A sys stderr write training. making feat dict...  N/A feat_list N/A set N/A frag N/A doc frag N/A frag N/A N/A feats N/A N/A N/A f _ v N/A N/A f v N/A N/A N/A frag features items N/A feat feats N/A N/A N/A N/A feat_list add feat N/A frag N/A frag next N/A N/A self feats N/A dict N/A zip feat_list N/A range 1 N/A N/A len feat_list 1 N/A N/A N/A N/A sys stderr write writing...  N/A lines N/A N/A frag N/A doc frag N/A frag N/A N/A N/A N/A frag label None N/A N/A N/A N/A sbd_util die N/A expecting labeled data [%s] frag N/A N/A N/A N/A frag label 0.5 N/A N/A svm_label +1 N/A N/A N/A N/A frag label 0.5 N/A N/A svm_label -1 N/A N/A N/A line N/A %s  svm_label N/A feats N/A N/A N/A f _ v N/A N/A f v N/A N/A N/A frag features items N/A svm_feats N/A N/A N/A self feats N/A f N/A f feats N/A N/A N/A svm_feats sort N/A N/A N/A x y N/A N/A x y N/A line N/A N/A join N/A N/A %d:1 x N/A x svm_feats N/A N/A N/A lines append line N/A frag N/A frag next N/A N/A unused train_file N/A N/A tempfile mkstemp N/A fh N/A open train_file w N/A N/A N/A fh write N/A N/A N/A join lines N/A N/A N/A fh close N/A N/A N/A N/A sys stderr write running svm...  N/A options -c 1 -v 0 N/A cmd N/A %s %s %s %s N/A SVM_LEARN options train_file model_file N/A N/A N/A os system cmd N/A N/A N/A N/A sys stderr write done!
 N/A N/A N/A os remove train_file N/A classify N/A N/A self doc verbose N/A False N/A N/A model_file N/A %ssvm_model N/A self path N/A N/A N/A self feats N/A N/A N/A N/A sbd_util die Incomplete model N/A N/A N/A N/A N/A os path isfile model_file N/A N/A N/A N/A sbd_util die N/A no model [%s] model_file N/A verbose N/A N/A N/A N/A N/A sys stderr write SVM classifying...  N/A lines N/A N/A frag N/A doc frag N/A frag N/A N/A N/A N/A frag label None N/A N/A svm_label 0 N/A N/A N/A frag label N/A N/A svm_label +1 N/A N/A svm_label -1 N/A line N/A %s  svm_label N/A feats N/A N/A N/A f _ v N/A N/A f v N/A N/A N/A frag features items N/A svm_feats N/A N/A N/A self feats N/A f N/A f feats N/A f N/A self feats N/A N/A N/A svm_feats sort N/A N/A N/A x y N/A N/A x y N/A line N/A N/A join N/A N/A %d:1 x N/A x svm_feats N/A N/A N/A lines append line N/A frag N/A frag next N/A test_file tmp1 N/A fh N/A open test_file w N/A N/A N/A fh write N/A N/A N/A join lines N/A N/A N/A fh close N/A pred_file tmp2 N/A options -v 0 N/A cmd N/A %s %s %s %s %s N/A SVM_CLASSIFY options test_file model_file pred_file N/A N/A N/A os system cmd N/A total 0 N/A pf N/A open pred_file r N/A preds N/A map float N/A N/A N/A N/A pf read splitlines N/A frag N/A doc frag N/A frag N/A N/A N/A frag pred N/A N/A sbd_util logit N/A preds N/A total N/A frag N/A frag next N/A total 1 N/A N/A N/A pf close N/A N/A N/A os remove test_file N/A N/A N/A os remove pred_file N/A verbose N/A N/A N/A N/A N/A sys stderr write done!
 N/A N/A Doc N/A N/A N/A 
    A Document points to the head of a Frag object
     __init__ N/A N/A self frag N/A N/A N/A N/A self frag frag N/A __str__ N/A N/A self N/A N/A N/A s N/A N/A curr N/A self frag N/A curr N/A N/A N/A N/A s append curr N/A N/A N/A join s N/A get_stats N/A N/A self verbose N/A N/A N/A verbose N/A N/A N/A N/A N/A sys stderr write getting statistics...  N/A lower_words N/A N/A sbd_util Counter N/A non_abbrs N/A N/A sbd_util Counter N/A frag N/A self frag N/A frag N/A N/A word N/A N/A N/A frag tokenized split N/A N/A N/A N/A N/A N/A word replace . isalpha N/A N/A N/A N/A word islower N/A N/A N/A lower_words N/A N/A N/A word replace . 1 N/A N/A N/A N/A word endswith . N/A N/A N/A non_abbrs N/A word 1 N/A frag N/A frag next N/A verbose N/A N/A N/A N/A N/A sys stderr write N/A lowercased [%d] non-abbrs [%d]
 N/A N/A len lower_words N/A len non_abbrs N/A N/A lower_words non_abbrs N/A featurize N/A N/A self model verbose N/A False N/A N/A verbose N/A N/A N/A N/A N/A sys stderr write featurizing...  N/A frag N/A self frag N/A frag N/A N/A N/A frag features N/A get_features frag model N/A frag N/A frag next N/A verbose N/A N/A N/A N/A N/A sys stderr write done!
 N/A segment N/A N/A self use_preds tokenize output list_only N/A False False None False N/A N/A 
        output all the text, split according to predictions or labels
         N/A sents N/A N/A thresh 0.5 N/A sent N/A N/A frag N/A self frag N/A frag N/A N/A tokenize N/A N/A text N/A frag tokenized N/A N/A text N/A frag orig N/A N/A N/A sent append text N/A N/A N/A frag ends_seg N/A use_preds N/A N/A frag pred thresh N/A N/A use_preds N/A N/A frag label thresh N/A N/A N/A N/A frag orig N/A N/A N/A sent_text N/A N/A join sent N/A N/A frag ends_seg N/A N/A spacer N/A N/A spacer N/A output N/A N/A N/A N/A output write N/A sent_text spacer N/A N/A N/A list_only N/A N/A N/A N/A N/A sys stdout write N/A sent_text spacer N/A N/A N/A sents append sent_text N/A sent N/A N/A frag N/A frag next N/A sents N/A show_results N/A N/A self verbose N/A False N/A N/A thresh 0.5 N/A N/A total correct N/A 0 0 N/A frag N/A self frag N/A frag N/A N/A total 1 N/A N/A N/A frag label N/A N/A frag pred thresh N/A N/A correct 1 N/A N/A w1 N/A N/A join N/A N/A N/A N/A frag tokenized split N/A -2 N/A N/A frag next N/A N/A w2 N/A N/A join N/A N/A N/A N/A N/A frag next tokenized split N/A 2 N/A N/A w2 <EOF> N/A verbose N/A N/A N/A [%d] [%1.4f] %s?? %s N/A N/A frag label N/A frag pred w1 w2 N/A frag N/A frag next N/A error N/A 1 N/A N/A 1.0 correct total N/A N/A correct [%d] total [%d] error [%1.4f] N/A correct total error N/A N/A Frag N/A N/A N/A 
    A fragment of text that ends with a possible sentence boundary
     __init__ N/A N/A self orig N/A N/A N/A N/A self orig orig N/A N/A self next None N/A N/A self ends_seg False N/A N/A self tokenized False N/A N/A self pred None N/A N/A self label None N/A N/A self features None N/A __str__ N/A N/A self N/A N/A N/A s N/A self orig N/A N/A self ends_seg N/A N/A s  <EOS>  N/A s N/A N/A build_model N/A N/A files options N/A N/A N/A train_corpus N/A get_data files tokenize N/A options tokenize N/A N/A options svm N/A N/A model N/A SVM_Model N/A options model_path N/A N/A model N/A NB_Model N/A options model_path N/A N/A N/A model prep train_corpus N/A N/A N/A train_corpus featurize model verbose True N/A N/A N/A model train train_corpus N/A N/A N/A model save N/A model N/A load_sbd_model N/A N/A model_path use_svm N/A model_nb/ False N/A N/A N/A N/A N/A sys stderr write N/A loading model from [%s]...  model_path N/A use_svm N/A N/A model N/A SVM_Model model_path N/A N/A model N/A NB_Model model_path N/A N/A N/A model load N/A N/A N/A N/A sys stderr write done!
 N/A model N/A sbd_text N/A N/A model text do_tok N/A True N/A N/A 
    A hook for segmenting text in Python code:

    import sbd
    m = sbd.load_sbd_model('/u/dgillick/sbd/splitta/test_nb/', use_svm=False)
    sents = sbd.sbd_text(m, 'Here is. Some text')
     N/A data N/A get_text_data text expect_labels False tokenize True N/A N/A N/A data featurize model verbose False N/A N/A N/A model classify data verbose False N/A sents N/A N/A data segment use_preds True tokenize do_tok list_only True N/A sents N/A N/A N/A __name__ __main__ N/A N/A modelpath /home/chonger/Downloads/model_svm/ N/A model N/A load_sbd_model modelpath True N/A sam /home/chonger/Downloads/sample.txt N/A sents N/A l N/A l N/A open sam N/A toseg N/A s sents N/A N/A N/A N/A len N/A N/A s strip 0 N/A N/A > s N/A toseg N/A N/A N/A s strip N/A toseg N/A ss N/A sbd_text model toseg N/A s ss N/A N/A ! s N/A 
    test = get_data(
    test.featurize(model, verbose=True)
    model.classify(test, verbose=True)
    test.segment(use_preds=True, tokenize=False, output=None)
     poo N/A N/A N/A N/A N/A data_root /u/dgillick/workspace/sbd/ N/A brown_data N/A data_root whiskey/brown.1 N/A wsj_data N/A data_root whiskey/satz.1 N/A poe_data N/A data_root whiskey/poe.1 N/A new_wsj_data N/A data_root whiskey/wsj.1 N/A install_root /home/chonger/Downloads/splitta/ optparse OptionParser N/A usage usage: %prog [options] <text_file> N/A parser N/A OptionParser usage usage N/A N/A N/A parser add_option -v --verbose dest verbose default False action store_true help verbose output N/A N/A N/A parser add_option -t --tokenize dest tokenize default False action store_true help write tokenized output N/A N/A N/A parser add_option -m --model dest model_path type str default model_nb help model path N/A N/A N/A parser add_option -o --output dest output type str default None help write sentences to this file N/A N/A N/A parser add_option -x --train dest train type str default None help train a new model using this labeled data file N/A N/A N/A parser add_option -c --svm dest svm default False action store_true help use SVM instead of Naive Bayes for training N/A N/A options args N/A N/A parser parse_args N/A N/A N/A len args 0 N/A N/A N/A options test N/A args N/A 0 N/A N/A N/A N/A N/A os path isfile N/A options test N/A N/A N/A N/A sbd_util die N/A test path [%s] does not exist N/A options test N/A N/A N/A options test None N/A N/A N/A options train N/A N/A N/A N/A sbd_util die you did not specify either train or test! N/A N/A N/A N/A N/A options model_path endswith / N/A N/A N/A options model_path / N/A N/A options train N/A N/A N/A N/A N/A N/A os path isfile N/A options train N/A N/A N/A N/A sbd_util die N/A model path [%s] does not exist N/A options train N/A N/A N/A N/A os path isdir N/A options model_path N/A N/A N/A N/A sbd_util die N/A model path [%s] already exists N/A options model_path N/A N/A N/A N/A os mkdir N/A options model_path N/A N/A N/A N/A N/A N/A os path isdir N/A options model_path N/A N/A N/A options model_path N/A install_root N/A options model_path N/A N/A N/A N/A N/A os path isdir N/A options model_path N/A N/A N/A N/A sbd_util die N/A model path [%s] does not exist N/A options model_path N/A N/A options train N/A N/A model N/A build_model N/A options train options N/A N/A N/A options test N/A N/A N/A N/A sys exit N/A N/A options svm N/A N/A options test N/A N/A N/A options train N/A N/A N/A svm N/A options model_path N/A N/A N/A options svm True N/A model N/A load_sbd_model N/A options model_path N/A options svm N/A N/A options output N/A N/A N/A options output N/A open N/A options output w N/A test N/A get_data N/A options test tokenize True N/A N/A N/A test featurize model verbose True N/A N/A N/A model classify test verbose True N/A N/A options tokenize N/A N/A options output N/A N/A N/A test segment use_preds True tokenize N/A options tokenize output N/A options output N/A