Module Expr Str ImportFrom alias Assign NameStore Str Assign NameStore Str Assign NameStore ListLoad Str Assign NameStore Str Assign NameStore Str Assign NameStore Str Import alias Import alias Import alias identifier ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias identifier ImportFrom alias alias ImportFrom alias alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias Assign NameStore Call AttributeLoad NameLoad attr NameLoad ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad NameLoad NameLoad Str NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad ListLoad Num Num Num Num body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num If Call NameLoad NameLoad TupleLoad NameLoad NameLoad NameLoad body Raise Call NameLoad BinOpAdd BinOpAdd Str Str Str If NameLoad body Expr Call AttributeLoad NameLoad attr Str Assign AttributeStore NameLoad attr Call NameLoad NameLoad orelse Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad If CompareIs NameLoad NameLoad body If CompareIsNot NameLoad NameLoad body Raise Call NameLoad BinOpAdd Str Str If CompareIsNot NameLoad NameLoad body Raise Call NameLoad BinOpAdd Str Str Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad If CompareIs NameLoad NameLoad body Assign NameStore Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad keyword ListLoad Str Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str If CompareIs AttributeLoad NameLoad attr NameLoad body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Assign NameStore ListComp NameLoad comprehension NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr If CompareGt Call NameLoad NameLoad Num body Raise Call NameLoad BinOpAdd Str Call NameLoad NameLoad If Call NameLoad ListComp Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr comprehension NameStore Call AttributeLoad NameLoad attr body Assign NameStore ListComp NameLoad comprehension NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Raise Call NameLoad BinOpAdd Str Call NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr If BoolOpAnd Call NameLoad NameLoad Str NameLoad Call NameLoad GeneratorExp CompareNotEq BinOpMod SubscriptLoad AttributeLoad Call AttributeLoad NameLoad attr attr Index Num AttributeLoad NameLoad attr Num comprehension NameStore Call AttributeLoad AttributeLoad NameLoad attr attr UnaryOpNot Call NameLoad AttributeLoad NameLoad attr body Raise Call NameLoad Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num keyword NameLoad Assign NameStore ListLoad For TupleStore NameStore NameStore Call NameLoad NameLoad NameLoad body Assign NameStore BinOpMod Str TupleLoad AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad AttributeLoad NameLoad attr If BoolOpAnd CompareIsNot NameLoad NameLoad CompareIs AttributeLoad NameLoad attr NameLoad body Assign AttributeStore NameLoad attr Str Assign NameStore AttributeLoad NameLoad attr If CompareIsNot AttributeLoad NameLoad attr NameLoad body If BoolOpAnd CompareIs AttributeLoad NameLoad attr NameLoad CompareIs AttributeLoad NameLoad attr NameLoad body Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr Assign NameStore SubscriptLoad Call AttributeLoad AttributeLoad NameLoad attr attr Index Num Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr keyword Str keyword NameLoad keyword NameLoad keyword TupleLoad Call NameLoad Str keyword NameLoad If AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Assert CompareGt Call NameLoad NameLoad Num For TupleStore NameStore NameStore Call NameLoad NameLoad body If CompareIs AttributeLoad NameLoad attr NameLoad body Assign AttributeStore NameLoad attr BinOpMod Str NameLoad Assign AttributeStore NameLoad attr NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad AttributeLoad NameLoad attr If UnaryOpNot Call NameLoad NameLoad NameLoad body Raise Call NameLoad BinOpAdd BinOpAdd BinOpAdd BinOpAdd BinOpAdd Call NameLoad Call NameLoad AttributeLoad NameLoad attr Str Str Call NameLoad Call NameLoad NameLoad Str Str For NameStore NameLoad body Assert CompareIn NameLoad NameLoad For NameStore NameLoad body Assert CompareIn NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr For NameStore NameLoad body If CompareNotIn NameLoad NameLoad body Raise Call NameLoad BinOpAdd BinOpAdd Str Call NameLoad NameLoad Str Assert CompareEq Call NameLoad Call AttributeLoad NameLoad attr Num FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Assign NameStore Call NameLoad Assert BoolOpAnd BoolOpOr NameLoad NameLoad UnaryOpNot BoolOpAnd NameLoad NameLoad If NameLoad body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr orelse Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr FunctionDef arguments args defaults body For NameStore NameLoad body If CompareNotIn NameLoad NameLoad body Assert CompareNotIn NameLoad NameLoad decorator_list Assign NameStore Call NameLoad For NameStore NameLoad body Assign SubscriptStore NameLoad Index NameLoad SubscriptLoad NameLoad Index NameLoad For NameStore NameLoad body If BoolOpAnd CompareIs AttributeLoad SubscriptLoad NameLoad Index NameLoad attr NameLoad CompareIsNot NameLoad NameLoad body Assign AttributeStore SubscriptLoad NameLoad Index NameLoad attr BinOpMod Str Dict Str Str AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assert CompareEq AttributeLoad SubscriptLoad NameLoad Index NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call NameLoad For NameStore NameLoad body If CompareIn NameLoad NameLoad body Assign NameStore SubscriptLoad NameLoad Index NameLoad Assign SubscriptStore NameLoad Index NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Str For NameStore NameLoad body Assign NameStore AttributeLoad NameLoad attr If CompareIs NameLoad NameLoad body Assign NameStore Str Assign NameStore BinOpMult Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Num Expr Call AttributeLoad NameLoad attr BinOpAdd BinOpAdd BinOpAdd Str NameLoad Str Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad NameLoad Expr Call NameLoad For NameStore NameLoad body If CompareIs AttributeLoad SubscriptLoad NameLoad Index NameLoad attr NameLoad body Assign AttributeStore SubscriptLoad NameLoad Index NameLoad attr BinOpAdd BinOpAdd Str AttributeLoad NameLoad attr Str Expr Call NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call NameLoad For NameStore NameLoad body Assign NameStore SubscriptLoad NameLoad Index NameLoad If CompareIs AttributeLoad NameLoad attr NameLoad body Assign AttributeStore NameLoad attr BinOpAdd BinOpAdd Str AttributeLoad NameLoad attr Str For NameStore Call NameLoad NameLoad body If Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad body Raise Call NameLoad BinOpMod Str AttributeLoad NameLoad attr If Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad body Raise Call NameLoad BinOpMod Str AttributeLoad NameLoad attr Expr Call NameLoad If NameLoad body For NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Assert CompareNotIn NameLoad NameLoad With Call NameLoad NameLoad Str body Return Call NameLoad NameLoad keyword NameLoad keyword Str keyword Str keyword AttributeLoad NameLoad attr decorator_list Assign AttributeStore NameLoad attr Call NameLoad Num Num keyword NameLoad Assign AttributeStore NameLoad attr Call NameLoad Num Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If UnaryOpNot Call NameLoad NameLoad Str body Raise Call NameLoad Str For NameStore AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad If BoolOpOr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad body Raise Call NameLoad BinOpAdd Str AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign NameStore AttributeLoad NameLoad attr If UnaryOpNot Call NameLoad AttributeLoad NameLoad attr body Assign NameStore NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num keyword NameLoad If CompareEq Call NameLoad NameLoad Num body Raise Call NameLoad BinOpMod Str Call NameLoad NameLoad Assign NameStore TupleLoad Call NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword NameLoad keyword AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr Assign NameStore Num For NameStore NameLoad body For NameStore NameLoad body Expr Call NameLoad NameLoad If BoolOpAnd AttributeLoad NameLoad attr CompareEq NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Num orelse Expr Call AttributeLoad NameLoad attr NameLoad AugAssignAdd NameStore Num Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Index Num attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad For NameStore AttributeLoad NameLoad attr body Expr Call NameLoad NameLoad For NameStore AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad If BoolOpOr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad body Raise Call NameLoad BinOpAdd Str AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr UnaryOpNot AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareIs AttributeLoad NameLoad attr NameLoad body Return NameLoad orelse Return Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults Num Num Num Num Num Num NameLoad NameLoad body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad If CompareIsNot NameLoad NameLoad body Assign AttributeStore NameLoad attr NameLoad orelse If CompareIsNot NameLoad NameLoad body Assign AttributeStore NameLoad attr BinOpAdd NameLoad Str Assign AttributeStore NameLoad attr NameLoad orelse Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Str Assign NameStore AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assert Call NameLoad NameLoad Str BinOpAdd Str Call NameLoad NameLoad Assign NameStore AttributeLoad NameLoad attr Assign NameStore NameLoad If CompareIs AttributeLoad NameLoad attr NameLoad body Assign NameStore NameLoad Assign NameStore ListComp NameLoad comprehension NameStore AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Str If CompareLt Call NameLoad NameLoad Num body Raise Call NameLoad Str orelse If CompareGt Call NameLoad NameLoad Num body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Raise Call NameLoad BinOpAdd BinOpAdd Str Call NameLoad NameLoad Str orelse Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Num Expr Call AttributeLoad NameLoad attr BinOpAdd BinOpAdd Str Call NameLoad AttributeLoad NameLoad attr Str TryExcept body Assign NameStore AttributeLoad SubscriptLoad AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr attr handlers ExceptHandler type NameLoad body Assign NameStore Str If NameLoad body If AttributeLoad NameLoad attr body Assign NameStore BinOpAdd BinOpAdd Str Call NameLoad AttributeLoad NameLoad attr Str orelse Assign NameStore BinOpAdd BinOpAdd Str Call NameLoad AttributeLoad NameLoad attr Str Assign NameStore BinOpAdd BinOpAdd BinOpAdd BinOpAdd BinOpAdd BinOpAdd Str Call NameLoad AttributeLoad NameLoad attr Str Str Str Str NameLoad Raise Call NameLoad NameLoad If CompareLt Call NameLoad NameLoad Num body If CompareIs AttributeLoad NameLoad attr NameLoad body Assert CompareEq Call NameLoad NameLoad Num Raise Call NameLoad Str Raise Call NameLoad Str If CompareEq Call NameLoad NameLoad Num body Return Assign NameStore NameLoad Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr AttributeLoad NameLoad attr If CompareGt SubscriptLoad NameLoad Index Num BinOpMult AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num body AugAssignMult NameStore AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad orelse If CompareGt SubscriptLoad NameLoad Index Num BinOpMult AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num body AugAssignMult NameStore AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Call SubscriptLoad AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults Num NameLoad body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore AttributeLoad NameLoad attr If CompareIs AttributeLoad NameLoad attr NameLoad body If CompareNotEq Call NameLoad AttributeLoad NameLoad attr Num body Raise Call NameLoad Str Assign NameStore AttributeLoad SubscriptLoad Call AttributeLoad AttributeLoad NameLoad attr attr Index Num attr orelse Assign NameStore AttributeLoad SubscriptLoad AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr attr If CompareLt SubscriptLoad NameLoad Index Num BinOpMult AttributeLoad NameLoad attr BinOpSub Num AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call NameLoad AttributeLoad NameLoad attr BinOpMult Call NameLoad NameLoad AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Num Return CompareLt Call NameLoad NameLoad AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AugAssignAdd AttributeStore NameLoad attr Num Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return BinOpMult AttributeLoad NameLoad attr Call NameLoad Num BinOpDiv AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam defaults body If Call NameLoad NameLoad NameLoad body Assign NameStore Call NameLoad NameLoad If Call NameLoad NameLoad NameLoad body Assign NameStore Call NameLoad NameLoad Assert Call NameLoad NameLoad NameLoad Assert Call NameLoad NameLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad Delete AttributeDel NameLoad attr Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareEq AttributeLoad NameLoad attr Num body Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AugAssignAdd AttributeStore NameLoad attr Num If UnaryOpNot AttributeLoad NameLoad attr body Assign NameStore BinOpDiv AttributeLoad NameLoad attr BinOpPow AttributeLoad NameLoad attr AttributeLoad NameLoad attr If CompareLtE NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr NameLoad Assign NameStore AttributeLoad NameLoad attr orelse Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call SubscriptLoad AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body If Call NameLoad NameLoad NameLoad body Assign NameStore Call NameLoad NameLoad If Call NameLoad NameLoad NameLoad body Assign NameStore Call NameLoad NameLoad If Call NameLoad NameLoad NameLoad body Assign NameStore Call NameLoad NameLoad Assert Call NameLoad NameLoad NameLoad Assert Call NameLoad NameLoad TupleLoad NameLoad NameLoad Assert Call NameLoad NameLoad TupleLoad NameLoad NameLoad Assert CompareGt NameLoad NameLoad Assert CompareGt NameLoad Num Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad Delete AttributeDel NameLoad attr Assign AttributeStore NameLoad attr Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareEq AttributeLoad NameLoad attr Num body Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr BinOpDiv BinOpSub AttributeLoad NameLoad attr BinOpMult AttributeLoad NameLoad attr AttributeLoad NameLoad attr BinOpAdd BinOpSub AttributeLoad NameLoad attr AttributeLoad NameLoad attr Num AugAssignAdd AttributeStore NameLoad attr Num If CompareGtE AttributeLoad NameLoad attr AttributeLoad NameLoad attr body If CompareLt AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Assign NameStore BinOpSub AttributeLoad NameLoad attr BinOpMult AttributeLoad NameLoad attr BinOpAdd BinOpSub AttributeLoad NameLoad attr AttributeLoad NameLoad attr Num orelse Assign NameStore BinOpMult AttributeLoad NameLoad attr AttributeLoad NameLoad attr orelse Assign NameStore AttributeLoad NameLoad attr Assert CompareGt NameLoad Num Assign NameStore Call SubscriptLoad AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr Str Return Call NameLoad NameLoad NameLoad NameLoad decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad Num body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad Delete AttributeDel NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assert CompareGtE NameLoad Num If CompareIs NameLoad NameLoad body Assign AttributeStore NameLoad attr BinOpAdd NameLoad Num orelse Assert CompareGt NameLoad Num decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr If CompareLt AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Raise Call NameLoad BinOpAdd Str Str Assign AttributeStore NameLoad attr NameLoad AugAssignAdd AttributeStore NameLoad attr Num Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call SubscriptLoad AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str If CompareLt AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Assign NameStore Num orelse Assign NameStore BinOpDiv Call NameLoad AttributeLoad NameLoad attr Call NameLoad BinOpAdd BinOpSub AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore BinOpMult AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr NameLoad Return NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad Delete AttributeDel NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assert Call NameLoad NameLoad NameLoad Assert Call NameLoad NameLoad TupleLoad NameLoad NameLoad Assert Call NameLoad NameLoad TupleLoad NameLoad NameLoad Assert CompareGt NameLoad NameLoad Assert CompareGtE NameLoad Num Assert CompareGtE NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr BinOpDiv BinOpSub AttributeLoad NameLoad attr BinOpMult AttributeLoad NameLoad attr AttributeLoad NameLoad attr BinOpAdd BinOpSub AttributeLoad NameLoad attr AttributeLoad NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad AugAssignAdd AttributeStore NameLoad attr Num Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call SubscriptLoad AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str If CompareGtE AttributeLoad NameLoad attr AttributeLoad NameLoad attr body If CompareLt AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Assign NameStore BinOpSub AttributeLoad NameLoad attr BinOpMult AttributeLoad NameLoad attr BinOpAdd BinOpSub AttributeLoad NameLoad attr AttributeLoad NameLoad attr Num orelse Assign NameStore BinOpMult AttributeLoad NameLoad attr AttributeLoad NameLoad attr orelse Assign NameStore AttributeLoad NameLoad attr Assert CompareGt NameLoad Num Return NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad Assign NameStore Call NameLoad Num Assign AttributeStore NameLoad attr Call NameLoad For NameStore Call AttributeLoad NameLoad attr body Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Assert CompareEq Call NameLoad NameLoad Call NameLoad NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad Assign SubscriptStore NameLoad Index NameLoad BinOpSub NameLoad BinOpDiv BinOpSub NameLoad NameLoad NameLoad Assign SubscriptStore NameLoad Index NameLoad BinOpAdd NameLoad Num Assign AttributeStore NameLoad attr Call NameLoad ListLoad keyword NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad Num body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad Delete AttributeDel NameLoad attr Assign AttributeStore NameLoad attr Num Assert Call NameLoad NameLoad NameLoad Assert CompareGtE NameLoad Num decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Str If CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call NameLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr TryExcept body Expr Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Pass orelse If BoolOpAnd CompareIsNot AttributeLoad NameLoad attr NameLoad CompareGt AttributeLoad NameLoad attr AttributeLoad NameLoad attr CompareEq BinOpMod AttributeLoad NameLoad attr AttributeLoad NameLoad attr Num body Assign NameStore Call NameLoad For NameStore Call AttributeLoad NameLoad attr body Assign SubscriptStore NameLoad Index NameLoad Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Call AttributeLoad SubscriptLoad AttributeLoad AttributeLoad NameLoad attr attr Index NameLoad attr Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad For NameStore Call AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index NameLoad AugAssignAdd AttributeStore NameLoad attr Num decorator_list decorator_list
N/A N/A 
Copy of pylearn2's sgd.py, hacked to support alternating between
epochs of updating only the discriminator and epochs of updating
both discriminator and generator. Ideally this would
be accomplished using pylearn2's FixedVarDescr implementation,
but it is currently not very well supported.
 __future__ division N/A __authors__ Ian Goodfellow N/A __copyright__ Copyright 2010-2012, Universite de Montreal N/A __credits__ N/A Ian Goodfellow, David Warde-Farley N/A __license__ 3-clause BSD N/A __maintainer__ David Warde-Farley N/A __email__ pylearn-dev@googlegroups N/A logging N/A warnings N/A numpy np theano config theano function theano.compat.python2x OrderedDict theano.gof.op get_debug_values pylearn2.monitor Monitor pylearn2.space CompositeSpace NullSpace pylearn2.train_extensions TrainExtension pylearn2.training_algorithms.training_algorithm TrainingAlgorithm pylearn2.training_algorithms.learning_rule Momentum pylearn2.training_algorithms.learning_rule MomentumAdjustor LRMomentumAdjustor pylearn2.utils.iteration is_stochastic has_uniform_batch_size pylearn2.utils py_integer_types py_float_types pylearn2.utils safe_zip pylearn2.utils serial pylearn2.utils sharedX pylearn2.utils.data_specs DataSpecsMapping pylearn2.utils.timing log_timing pylearn2.utils.rng make_np_rng N/A log N/A N/A logging getLogger __name__ SGD N/A TrainingAlgorithm N/A N/A 
    SGD = (Minibatch) Stochastic Gradient Descent.
    A TrainingAlgorithm that does stochastic gradient descent on minibatches
    of training examples.

    For theoretical background on this algorithm, see Yoshua Bengio's machine
    learning course notes on the subject:

    http://www.iro.umontreal.ca/~pift6266/H10/notes/gradient.html

    Parameters
    ----------
    learning_rate : float
        The learning rate to use. Train object callbacks can change the
        learning rate after each epoch. SGD update_callbacks can change
        it after each minibatch.
    cost : pylearn2.costs.cost.Cost, optional
        Cost object specifying the objective function to be minimized.
        Optionally, may be None. In this case, SGD will call the model's
        get_default_cost method to obtain the objective function.
    batch_size : int, optional
        The size of the batch to be used.
        If not specified, the model will be asked for the batch size, so
        you must have specified the batch size there.
        (Some models are rigidly defined to only work with one batch size)
    monitoring_batch_size : int, optional
        The size of the monitoring batches.
    monitoring_batches : int, optional
        At the start of each epoch, we run "monitoring", to evaluate
        quantities such as the validation set error.
        monitoring_batches, if specified, determines the number of batches
        to draw from the iterator for each monitoring dataset.
        Unnecessary if not using monitoring or if `monitor_iteration_mode`
        is 'sequential' and `batch_size` is specified (number of
        batches will be calculated based on full dataset size).
        TODO: make it possible to specify different monitoring_batches
        for each monitoring dataset. The Monitor itself already supports
        this.
    monitoring_dataset : Dataset or dictionary, optional
        If not specified, no monitoring is used.
        If specified to be a Dataset, monitor on that Dataset.
        If specified to be dictionary, the keys should be string names
        of datasets, and the values should be Datasets. All monitoring
        channels will be computed for all monitoring Datasets and will
        have the dataset name and an underscore prepended to them.
    monitor_iteration_mode : str, optional
        The iteration mode used to iterate over the examples in all
        monitoring datasets. If not specified, defaults to 'sequential'.
        TODO: make it possible to specify different modes for different
        datasets.
    termination_criterion : instance of         pylearn2.termination_criteria.TerminationCriterion, optional

        Used to determine when the algorithm should stop running.
        If not specified, runs forever--or more realistically, until
        external factors halt the python process (Kansas 1977).
    update_callbacks : list, optional
        If specified, each member of the list should be a callable that
        accepts an SGD instance as its only argument.
        All callbacks will be called with this SGD instance after each
        SGD step.
    learning_rule : training_algorithms.learning_rule.LearningRule, optional
        A learning rule computes the new parameter values given old
        parameters and first-order gradients. If learning_rule is None,
        sgd.SGD will update parameters according to the standard SGD
        learning rule:

        .. code-block:: none

            param := param - learning_rate * d cost / d param

        This argument allows more sophisticated learning rules, such
        as SGD with momentum.
    init_momentum : float, **DEPRECATED** option
        Use learning_rule instead.
        If None, does not use momentum otherwise, use momentum and
        initialize the momentum coefficient to init_momentum. Callbacks
        can change this over time just like the learning rate. If the
        gradient is the same on every step, then the update taken by the
        SGD algorithm is scaled by a factor of 1/(1-momentum). See
        section 9 of Geoffrey Hinton's "A Practical Guide to Training
        Restricted Boltzmann Machines" for details.
    set_batch_size : bool, optional
        Defaults to False.
        If True, and batch_size conflicts with model.force_batch_size,
        will call model.set_batch_size(batch_size) in an attempt to
        change model.force_batch_size
    train_iteration_mode : str, optional
        Defaults to 'shuffled_sequential'.
        The iteration mode to use for iterating through training examples.
    batches_per_iter : int, optional
        The number of batches to draw from the iterator over training
        examples.
        If iteration mode is 'sequential' or 'shuffled_sequential', this
        is unnecessary; when unspecified we will iterate over all examples.
    theano_function_mode : a valid argument to theano.function's         'mode' parameter, optional

        The theano mode to compile the updates function with. Note that
        pylearn2 includes some wraplinker modes that are not bundled with
        theano. See pylearn2.devtools. These extra modes let you do
        things like check for NaNs at every step, or record md5 digests
        of all computations performed by the update function to help
        isolate problems with nondeterminism.
    monitoring_costs : list, optional
        a list of Cost instances. The Monitor will also include all
        channels defined by these Costs, even though we don't train
        using them.
    seed : valid argument to np.random.RandomState, optional
        The seed used for the random number generate to be passed to the
        training dataset iterator (if any)
     __init__ N/A N/A self learning_rate cost batch_size monitoring_batch_size monitoring_batches monitoring_dataset monitor_iteration_mode termination_criterion update_callbacks learning_rule init_momentum set_batch_size train_iteration_mode batches_per_iter theano_function_mode monitoring_costs seed discriminator_steps N/A None None None None None sequential None None None None False None None None None N/A 2012 10 5 1 N/A N/A N/A self discriminator_steps discriminator_steps N/A N/A self train_generator 0 N/A N/A isinstance cost N/A list tuple set N/A N/A N/A TypeError N/A N/A SGD no longer supports using collections of  Costs to represent a sum of Costs. Use  pylearn2.costs.cost.SumOfCosts instead. N/A init_momentum N/A N/A N/A N/A warnings warn init_momentum interface is deprecated and will become officially unsuported as of May 9, 2014. Please use the `learning_rule` parameter instead, providing an object of type `pylearn2.training_algorithms.learning_rule.Momentum` instead N/A N/A self learning_rule N/A Momentum init_momentum N/A N/A N/A self learning_rule learning_rule N/A N/A self learning_rate N/A sharedX learning_rate learning_rate N/A N/A self cost cost N/A N/A self batch_size batch_size N/A N/A self set_batch_size set_batch_size N/A N/A self batches_per_iter batches_per_iter N/A N/A N/A self _set_monitoring_dataset monitoring_dataset N/A N/A self monitoring_batch_size monitoring_batch_size N/A N/A self monitoring_batches monitoring_batches N/A N/A self monitor_iteration_mode monitor_iteration_mode N/A N/A monitoring_dataset None N/A N/A N/A monitoring_batch_size None N/A N/A N/A ValueError N/A Specified a monitoring batch size  but not a monitoring dataset. N/A N/A monitoring_batches None N/A N/A N/A ValueError N/A Specified an amount of monitoring batches  but not a monitoring dataset. N/A N/A self termination_criterion termination_criterion N/A N/A N/A self _register_update_callbacks update_callbacks N/A N/A train_iteration_mode None N/A N/A train_iteration_mode shuffled_sequential N/A N/A self train_iteration_mode train_iteration_mode N/A N/A self first True N/A N/A self rng N/A make_np_rng seed which_method N/A randn randint N/A N/A self theano_function_mode theano_function_mode N/A N/A self monitoring_costs monitoring_costs N/A setup N/A N/A self model dataset N/A N/A N/A 
        Compiles the theano functions needed for the train method.

        Parameters
        ----------
        model : a Model instance
        dataset : Dataset
         N/A N/A N/A self cost None N/A N/A N/A self cost N/A N/A model get_default_cost N/A inf_params N/A param N/A param N/A N/A model get_params N/A N/A np any N/A N/A np isinf N/A N/A param get_value N/A N/A N/A len inf_params 0 N/A N/A N/A ValueError N/A These params are Inf:  N/A str inf_params N/A N/A any N/A N/A N/A np any N/A N/A np isnan N/A N/A param get_value N/A param N/A N/A model get_params N/A N/A nan_params N/A param N/A param N/A N/A model get_params N/A N/A np any N/A N/A np isnan N/A N/A param get_value N/A N/A ValueError N/A These params are NaN:  N/A str nan_params N/A N/A self model model N/A N/A N/A self _synchronize_batch_size model N/A N/A model _test_batch_size N/A self batch_size N/A N/A self monitor N/A N/A Monitor get_monitor model N/A N/A N/A N/A self monitor _sanity_check N/A N/A N/A getattr model force_batch_size False N/A any N/A N/A N/A N/A N/A N/A N/A dataset get_design_matrix shape N/A 0 N/A self batch_size 0 N/A dataset N/A N/A N/A self monitoring_dataset values N/A N/A has_uniform_batch_size N/A self monitor_iteration_mode N/A N/A N/A ValueError Dataset size is not a multiple of batch size.You should set monitor_iteration_mode to even_sequential, even_shuffled_sequential or even_batchwise_shuffled_sequential N/A data_specs N/A N/A N/A self cost get_data_specs N/A self model N/A mapping N/A DataSpecsMapping data_specs N/A space_tuple N/A N/A mapping flatten N/A data_specs N/A 0 return_tuple True N/A source_tuple N/A N/A mapping flatten N/A data_specs N/A 1 return_tuple True N/A theano_args N/A N/A N/A space source N/A safe_zip space_tuple source_tuple N/A N/A name N/A %s[%s] N/A N/A N/A self __class__ __name__ source N/A arg N/A N/A space make_theano_batch name name batch_size N/A self batch_size N/A N/A N/A theano_args append arg N/A theano_args N/A tuple theano_args N/A nested_args N/A N/A mapping nest theano_args N/A fixed_var_descr N/A N/A N/A self cost get_fixed_var_descr model nested_args N/A N/A self on_load_batch N/A fixed_var_descr on_load_batch N/A cost_value N/A N/A N/A self cost expr model nested_args N/A fixed_var_descr fixed_vars N/A N/A N/A cost_value None N/A N/A cost_value name None N/A N/A N/A cost_value name objective N/A learning_rate N/A self learning_rate N/A N/A N/A self monitoring_dataset None N/A N/A N/A N/A N/A self monitoring_batch_size None N/A N/A self monitoring_batches None N/A N/A N/A self monitoring_batch_size N/A self batch_size N/A N/A self monitoring_batches N/A self batches_per_iter N/A N/A N/A N/A self monitor setup dataset N/A self monitoring_dataset cost N/A self cost batch_size N/A self monitoring_batch_size num_batches N/A self monitoring_batches extra_costs N/A self monitoring_costs mode N/A self monitor_iteration_mode N/A dataset_name N/A N/A N/A N/A self monitoring_dataset keys N/A 0 N/A monitoring_dataset N/A N/A self monitoring_dataset N/A dataset_name N/A N/A N/A N/A self monitor add_channel name learning_rate ipt None val learning_rate data_specs N/A N/A NullSpace dataset monitoring_dataset N/A N/A self learning_rule N/A N/A N/A N/A N/A self learning_rule add_channels_to_monitor N/A self monitor monitoring_dataset N/A params N/A list N/A N/A model get_params N/A N/A N/A len params 0 N/A N/A i param N/A enumerate params N/A N/A N/A N/A param name None N/A N/A N/A param name N/A sgd_params[%d] i N/A N/A self params params N/A N/A grads updates N/A N/A N/A self cost get_gradients model nested_args N/A fixed_var_descr fixed_vars N/A N/A N/A isinstance grads OrderedDict N/A N/A N/A TypeError N/A N/A N/A N/A N/A N/A str N/A type N/A self cost .get_gradients returned  something with N/A str N/A type grads as its  first member. Expected OrderedDict. N/A param grads N/A N/A N/A param params N/A param params N/A N/A N/A param grads N/A lr_scalers N/A N/A model get_lr_scalers N/A key lr_scalers N/A N/A N/A key params N/A N/A N/A ValueError N/A N/A Tried to scale the learning rate on  N/A str key  which is not an optimization parameter. N/A N/A N/A len N/A N/A updates keys 0 get_func N/A N/A learn_discriminator learn_generator dont_you_fucking_dare_touch_the_generator N/A False N/A N/A updates N/A OrderedDict N/A N/A N/A learn_discriminator learn_generator N/A N/A learn_discriminator learn_generator N/A learn_discriminator N/A N/A cur_params N/A N/A N/A model discriminator get_params N/A N/A cur_params N/A N/A N/A model generator get_params check N/A N/A N/A N/A N/A param params N/A N/A N/A param cur_params N/A N/A N/A param updates N/A N/A cur_grads N/A OrderedDict N/A param cur_params N/A N/A N/A cur_grads N/A param N/A grads N/A param N/A param grads N/A N/A N/A N/A N/A N/A grads N/A param name None N/A cost_value None N/A N/A N/A N/A grads N/A param name N/A grad(%(costname)s, %(paramname)s) N/A costname paramname N/A cost_value name N/A param name N/A N/A N/A N/A grads N/A param dtype N/A param dtype N/A cur_lr_scalers N/A OrderedDict N/A param cur_params N/A N/A N/A param lr_scalers N/A N/A lr_scaler N/A lr_scalers N/A param N/A N/A cur_lr_scalers N/A param lr_scaler N/A N/A N/A log info Parameter and initial learning rate summary: N/A param cur_params N/A N/A param_name N/A param name N/A N/A param_name None N/A N/A param_name anon_param N/A lr N/A N/A N/A learning_rate get_value N/A N/A cur_lr_scalers get param 1.0 N/A N/A N/A log info N/A N/A N/A param_name :  N/A str lr N/A N/A N/A updates update N/A N/A N/A self learning_rule get_updates learning_rate cur_grads cur_lr_scalers N/A N/A check N/A param cur_params N/A N/A N/A N/A N/A updates N/A param name None N/A N/A N/A N/A updates N/A param name N/A N/A sgd_update( N/A param name ) N/A N/A check N/A N/A N/A model modify_updates updates N/A N/A check N/A param cur_params N/A N/A update N/A updates N/A param N/A N/A N/A update name None N/A N/A N/A update name N/A N/A censor(sgd_update( N/A param name )) N/A update_val N/A get_debug_values update N/A N/A N/A N/A np any N/A N/A np isinf update_val N/A N/A N/A ValueError N/A debug value of %s contains infs N/A update name N/A N/A N/A np any N/A N/A np isnan update_val N/A N/A N/A ValueError N/A debug value of %s contains nans N/A update name N/A N/A check N/A dont_you_fucking_dare_touch_the_generator N/A N/A param N/A N/A N/A model generator get_params N/A N/A N/A param updates N/A N/A log_timing log Compiling sgd_update N/A N/A N/A function theano_args updates updates name sgd_update on_unused_input ignore mode N/A self theano_function_mode N/A N/A N/A self d_func N/A get_func 1 0 dont_you_fucking_dare_touch_the_generator True N/A N/A self g_func N/A get_func 0 1 N/A train N/A N/A self dataset N/A N/A N/A 
        Runs one epoch of SGD training on the specified dataset.

        Parameters
        ----------
        dataset : Dataset
         N/A N/A N/A hasattr self d_func N/A N/A N/A Exception train called without first calling setup N/A param N/A self params N/A N/A value N/A N/A param get_value borrow True N/A N/A N/A N/A np any N/A N/A np isnan value N/A N/A np any N/A N/A np isinf value N/A N/A N/A Exception N/A NaN in  N/A param name N/A N/A self first False N/A rng N/A self rng N/A N/A N/A is_stochastic N/A self train_iteration_mode N/A N/A rng None N/A data_specs N/A N/A N/A self cost get_data_specs N/A self model N/A mapping N/A DataSpecsMapping data_specs N/A space_tuple N/A N/A mapping flatten N/A data_specs N/A 0 return_tuple True N/A source_tuple N/A N/A mapping flatten N/A data_specs N/A 1 return_tuple True N/A N/A N/A len space_tuple 0 N/A N/A N/A NotImplementedError N/A Unable to train with SGD, because the cost does not actually use data from the data set. data_specs: %s N/A str data_specs N/A flat_data_specs N/A N/A CompositeSpace space_tuple source_tuple N/A iterator N/A N/A dataset iterator mode N/A self train_iteration_mode batch_size N/A self batch_size data_specs flat_data_specs return_tuple True rng rng num_batches N/A self batches_per_iter N/A on_load_batch N/A self on_load_batch N/A i 0 N/A batch iterator N/A N/A callback on_load_batch N/A N/A N/A callback batch N/A N/A N/A self train_generator N/A i N/A self discriminator_steps N/A N/A N/A N/A self g_func batch N/A i 0 N/A N/A N/A N/A self d_func batch N/A i 1 N/A actual_batch_size N/A N/A N/A flat_data_specs N/A 0 np_batch_size batch N/A N/A N/A N/A self monitor report_batch actual_batch_size N/A callback N/A self update_callbacks N/A N/A N/A callback self N/A param N/A self params N/A N/A value N/A N/A param get_value borrow True N/A N/A N/A N/A np any N/A N/A np isnan value N/A N/A np any N/A N/A np isinf value N/A N/A N/A Exception N/A NaN in  N/A param name N/A N/A self train_generator N/A N/A self train_generator N/A continue_learning N/A N/A self model N/A N/A N/A 
        Returns True if the algorithm should continue running, or False
        if it has reached convergence / started overfitting and should
        stop.

        Parameters
        ----------
        model : a Model instance
         N/A N/A N/A self termination_criterion None N/A N/A True N/A N/A N/A N/A N/A self termination_criterion continue_learning N/A self model N/A N/A MonitorBasedLRAdjuster N/A TrainExtension N/A N/A 
    A TrainExtension that uses the on_monitor callback to adjust
    the learning rate on each epoch. It pulls out a channel
    from the model's monitor and adjusts the learning rate
    based on what happened to the monitoring channel on the last
    epoch. If the channel is greater than high_trigger times
    its previous value, the learning rate will be scaled by
    shrink_amt (which should be < 1 for this scheme to make
    sense). The idea is that in this case the learning algorithm
    is overshooting the bottom of the objective function.

    If the objective is less than high_trigger but
    greater than low_trigger times its previous value, the
    learning rate will be scaled by grow_amt (which should be > 1
    for this scheme to make sense). The idea is that the learning
    algorithm is making progress but at too slow of a rate.

    Parameters
    ----------
    high_trigger : float, optional
        See class-level docstring
    low_trigger : float, optional
        See class-level docstring
    grow_amt : float, optional
        See class-level docstring
    min_lr : float, optional
        All updates to the learning rate are clipped to be at least
        this value.
    max_lr : float, optional
        All updates to the learning rate are clipped to be at most
        this value.
    dataset_name : str, optional
        If specified, use dataset_name + "_objective" as the channel
        to guide the learning rate adaptation.
    channel_name : str, optional
        If specified, use channel_name as the channel to guide the
        learning rate adaptation. Conflicts with dataset_name.
        If neither dataset_name nor channel_name is specified, uses
        "objective"
     __init__ N/A N/A self high_trigger shrink_amt low_trigger grow_amt min_lr max_lr dataset_name channel_name N/A 1.0 0.99 0.99 1.01 1e-07 1.0 None None N/A N/A N/A self high_trigger high_trigger N/A N/A self shrink_amt shrink_amt N/A N/A self low_trigger low_trigger N/A N/A self grow_amt grow_amt N/A N/A self min_lr min_lr N/A N/A self max_lr max_lr N/A N/A self dataset_name None N/A N/A channel_name None N/A N/A N/A self channel_name channel_name N/A N/A N/A dataset_name None N/A N/A N/A self channel_name N/A dataset_name _objective N/A N/A self dataset_name dataset_name N/A N/A N/A self channel_name None N/A on_monitor N/A N/A self model dataset algorithm N/A N/A N/A 
        Adjusts the learning rate based on the contents of model.monitor

        Parameters
        ----------
        model : a Model instance
        dataset : Dataset
        algorithm : WRITEME
         N/A model N/A algorithm model N/A lr N/A algorithm learning_rate N/A current_learning_rate N/A N/A lr get_value N/A N/A hasattr model monitor N/A no monitor associated with  N/A str model N/A monitor N/A model monitor N/A monitor_channel_specified True N/A N/A N/A self channel_name None N/A N/A monitor_channel_specified False N/A channels N/A elem N/A elem N/A monitor channels N/A N/A elem endswith objective N/A N/A N/A len channels 1 N/A N/A N/A ValueError There are no monitoring channels that end with "objective". Please specify either channel_name or dataset_name. N/A N/A N/A N/A len channels 1 N/A N/A datasets N/A N/A N/A algorithm monitoring_dataset keys N/A N/A ValueError N/A N/A There are multiple monitoring channels thatend with "_objective". The list of available datasets are:  N/A str datasets  . Please specify either channel_name or dataset_name in the MonitorBasedLRAdjuster constructor to disambiguate. N/A N/A N/A self channel_name N/A channels N/A 0 N/A N/A N/A warnings warn N/A N/A The channel that has been chosen for monitoring is:  N/A str N/A self channel_name . N/A N/A N/A v N/A N/A N/A monitor channels N/A N/A self channel_name val_record N/A N/A N/A KeyError N/A N/A err_input N/A monitor_channel_specified N/A N/A N/A self dataset_name N/A N/A err_input N/A N/A The dataset_name ' N/A str N/A self dataset_name ' is not valid. N/A N/A err_input N/A N/A The channel_name ' N/A str N/A self channel_name ' is not valid. N/A err_message N/A N/A N/A N/A N/A N/A There is no monitoring channel named ' N/A str N/A self channel_name '. You probably need to  specify a valid monitoring channel by using either  dataset_name or channel_name in the  MonitorBasedLRAdjuster constructor.  err_input N/A N/A ValueError err_message N/A N/A N/A len v 1 N/A N/A N/A N/A monitor dataset None N/A N/A N/A N/A len v 0 N/A N/A ValueError You're trying to use a monitor-based learning rate adjustor but the monitor has no entries because you didn't specify a monitoring dataset. N/A N/A ValueError For some reason there are no monitor entriesyet the MonitorBasedLRAdjuster has been called. This should never happen. The Train object should call the monitor once on initialization, then call the callbacks. It seems you are either calling the callback manually rather than as part of a training algorithm, or there is a problem with the Train object. N/A N/A N/A len v 1 N/A N/A N/A rval current_learning_rate N/A N/A N/A log info N/A N/A monitoring channel is {0} format N/A self channel_name N/A N/A N/A v N/A -1 N/A N/A self high_trigger N/A v N/A -2 N/A N/A rval N/A self shrink_amt N/A N/A N/A log info N/A shrinking learning rate to %f rval N/A N/A N/A N/A v N/A -1 N/A N/A self low_trigger N/A v N/A -2 N/A N/A rval N/A self grow_amt N/A N/A N/A log info N/A growing learning rate to %f rval N/A rval N/A max N/A self min_lr rval N/A rval N/A min N/A self max_lr rval N/A N/A N/A lr set_value N/A N/A N/A np cast N/A N/A lr dtype rval N/A N/A PatienceBasedTermCrit N/A object N/A N/A 
    A monitor-based termination criterion using a geometrically increasing
    amount of patience. If the selected channel has decreased by a certain
    proportion when comparing to the lowest value seen yet, the patience is
    set to a factor of the number of examples seen, which by default
    (patience_increase=2.) ensures the model has seen as many examples as the
    number of examples that lead to the lowest value before concluding a local
    optima has been reached.

    Note: Technically, the patience corresponds to a number of epochs to be
    independent of the size of the dataset, so be aware of that when choosing
    initial_patience.

    Parameters
    ----------
    prop_decrease : float
        The factor X in the (1 - X) * best_value threshold
    initial_patience : int
        Minimal number of epochs the model has to run before it can stop
    patience_increase : float, optional
        The factor X in the patience = X * n_iter update.
    channel_name : string, optional
        Name of the channel to examine. If None and the monitor
        has only one channel, this channel will be used; otherwise, an
        error will be raised.
     __init__ N/A N/A self prop_decrease initial_patience patience_increase channel_name N/A 2.0 None N/A N/A N/A self _channel_name channel_name N/A N/A self prop_decrease prop_decrease N/A N/A self patience initial_patience N/A N/A self best_value N/A np inf N/A N/A self patience_increase patience_increase N/A __call__ N/A N/A self model N/A N/A N/A 
        Returns True or False depending on whether the optimization should
        stop or not. The optimization should stop if it has run for a number
        of epochs superior to the patience without any improvement.

        Parameters
        ----------
        model : Model
            The model used in the experiment and from which the monitor used
            in the termination criterion will be extracted.

        Returns
        -------
        bool
            True or False, indicating if the optimization should stop or not.
         N/A monitor N/A model monitor N/A N/A N/A self _channel_name None N/A N/A N/A N/A len N/A monitor channels 1 N/A N/A N/A ValueError Only single-channel monitors are supported for channel_name == None N/A v N/A N/A N/A N/A N/A monitor channels values N/A 0 val_record N/A N/A v N/A N/A N/A monitor channels N/A N/A self _channel_name val_record N/A N/A N/A v N/A -1 N/A N/A self best_value N/A 1.0 N/A self prop_decrease N/A N/A N/A self patience N/A max N/A self patience N/A N/A len v N/A self patience_increase N/A N/A self best_value N/A v N/A -1 N/A N/A N/A len v N/A self patience N/A N/A AnnealedLearningRate N/A object N/A N/A 
    This is a callback for the SGD algorithm rather than the Train object.
    This anneals the learning rate to decrease as 1/t where t is the number
    of gradient descent updates done so far. Use OneOverEpoch as Train object
    callback if you would prefer 1/t where t is epochs.

    Parameters
    ----------
    anneal_start : int
        The epoch on which to begin annealing
     __init__ N/A N/A self anneal_start N/A N/A N/A N/A self _initialized False N/A N/A self _count 0 N/A N/A self _anneal_start anneal_start N/A __call__ N/A N/A self algorithm N/A N/A N/A 
        Updates the learning rate according to the annealing schedule.

        Parameters
        ----------
        algorithm : WRITEME
         N/A N/A N/A self _initialized N/A N/A N/A self _base N/A N/A N/A algorithm learning_rate get_value N/A N/A self _count 1 N/A N/A N/A N/A algorithm learning_rate set_value N/A N/A self current_learning_rate N/A current_learning_rate N/A N/A self N/A N/A N/A 
        Returns the current desired learning rate according to the
        annealing schedule.
         N/A N/A N/A self _base N/A min 1 N/A N/A self _anneal_start N/A self _count N/A N/A ExponentialDecay N/A object N/A N/A 
    This is a callback for the `SGD` algorithm rather than the `Train` object.
    This anneals the learning rate by dividing by decay_factor after each
    gradient descent step. It will not shrink the learning rate beyond
    `min_lr`.

    Parameters
    ----------
    decay_factor : float
        The learning rate at step t is given by
        `init_learning_rate / (decay_factor ** t)`
    min_lr : float
        The learning rate will be clipped to be at least this value
     __init__ N/A N/A self decay_factor min_lr N/A N/A N/A N/A isinstance decay_factor str N/A N/A decay_factor N/A float decay_factor N/A N/A isinstance min_lr str N/A N/A min_lr N/A float min_lr N/A N/A isinstance decay_factor float N/A N/A isinstance min_lr float N/A N/A N/A N/A self __dict__ update N/A locals N/A N/A self self N/A N/A self _count 0 N/A N/A self _min_reached False N/A __call__ N/A N/A self algorithm N/A N/A N/A 
        Updates the learning rate according to the exponential decay schedule.

        Parameters
        ----------
        algorithm : SGD
            The SGD instance whose `learning_rate` field should be modified.
         N/A N/A N/A self _count 0 N/A N/A N/A self _base_lr N/A N/A N/A algorithm learning_rate get_value N/A N/A self _count 1 N/A N/A N/A self _min_reached N/A N/A new_lr N/A N/A self _base_lr N/A N/A self decay_factor N/A self _count N/A N/A new_lr N/A self min_lr N/A N/A N/A self _min_reached True N/A new_lr N/A self min_lr N/A N/A new_lr N/A self min_lr N/A new_lr N/A N/A N/A np cast N/A N/A config floatX new_lr N/A N/A N/A N/A algorithm learning_rate set_value new_lr N/A N/A LinearDecay N/A object N/A N/A 
    This is a callback for the SGD algorithm rather than the Train object.
    This anneals the learning rate to decay_factor times of the initial value
    during time start till saturate.

    Parameters
    ----------
    start : int
        The step at which to start decreasing the learning rate
    saturate : int
        The step at which to stop decreating the learning rate
    decay_factor : float
        `final learning rate = decay_factor * initial learning rate`
     __init__ N/A N/A self start saturate decay_factor N/A N/A N/A N/A isinstance decay_factor str N/A N/A decay_factor N/A float decay_factor N/A N/A isinstance start str N/A N/A start N/A float start N/A N/A isinstance saturate str N/A N/A saturate N/A float saturate N/A N/A isinstance decay_factor float N/A N/A isinstance start N/A py_integer_types py_float_types N/A N/A isinstance saturate N/A py_integer_types py_float_types N/A N/A saturate start N/A N/A start 0 N/A N/A N/A N/A self __dict__ update N/A locals N/A N/A self self N/A N/A self _count 0 N/A __call__ N/A N/A self algorithm N/A N/A N/A 
        Adjusts the learning rate according to the linear decay schedule

        Parameters
        ----------
        algorithm : WRITEME
         N/A N/A N/A self _count 0 N/A N/A N/A self _base_lr N/A N/A N/A algorithm learning_rate get_value N/A N/A self _step N/A N/A N/A self _base_lr N/A N/A self _base_lr N/A self decay_factor N/A N/A N/A self saturate N/A self start 1 N/A N/A self _count 1 N/A N/A N/A self _count N/A self start N/A N/A N/A N/A self _count N/A self saturate N/A N/A new_lr N/A N/A self _base_lr N/A N/A self _step N/A N/A N/A self _count N/A self start 1 N/A N/A new_lr N/A N/A self _base_lr N/A self decay_factor N/A N/A new_lr N/A self _base_lr N/A N/A new_lr 0 N/A new_lr N/A N/A N/A np cast N/A N/A config floatX new_lr N/A N/A N/A N/A algorithm learning_rate set_value new_lr N/A N/A MomentumAdjustor N/A N/A final_momentum start saturate N/A N/A N/A 
    Deprecated class used with the deprecated init_momentum argument.
    Use learning_rule.MomentumAdjustor instead.

    Parameters
    ----------
    final_momentum : WRITEME
    start : WRITEME
    saturate : WRITEME
     N/A N/A N/A warnings warn sgd.MomentumAdjustor interface is deprecated and will become officially unsupported as of May 9, 2014. Please use `learning_rule.MomentumAdjustor` instead. N/A N/A LRMomentumAdjustor final_momentum start saturate N/A OneOverEpoch N/A TrainExtension N/A N/A 
    Scales the learning rate like one over # epochs

    Parameters
    ----------
    start : int
        The epoch on which to start shrinking the learning rate
    half_life : int, optional
        How many epochs after start it will take for the learning rate to lose
        half its value for the first time (to lose the next half of its value
        will take twice as long)
    min_lr : float, optional
        The minimum value the learning rate can take on
     __init__ N/A N/A self start half_life min_lr N/A None 1e-06 N/A N/A N/A N/A N/A self __dict__ update N/A locals N/A N/A self self N/A N/A self _initialized False N/A N/A self _count 0 N/A N/A start 0 N/A N/A half_life None N/A N/A N/A self half_life N/A start 1 N/A N/A N/A half_life 0 N/A on_monitor N/A N/A self model dataset algorithm N/A N/A N/A 
        Adjusts the learning rate according to the decay schedule.

        Parameters
        ----------
        model : a Model instance
        dataset : Dataset
        algorithm : WRITEME
         N/A N/A N/A self _initialized N/A N/A N/A self _init_lr N/A N/A N/A algorithm learning_rate get_value N/A N/A N/A self _init_lr N/A self min_lr N/A N/A N/A ValueError N/A The initial learning rate is smaller than  the minimum allowed learning rate. N/A N/A self _initialized True N/A N/A self _count 1 N/A N/A N/A N/A algorithm learning_rate set_value N/A N/A N/A np cast N/A N/A config floatX N/A N/A self current_lr N/A current_lr N/A N/A self N/A N/A N/A 
        Returns the learning rate currently desired by the decay schedule.
         N/A N/A N/A self _count N/A self start N/A N/A scale 1 N/A N/A scale N/A N/A float N/A self half_life N/A float N/A N/A N/A self _count N/A self start N/A self half_life N/A lr N/A N/A self _init_lr scale N/A clipped N/A max N/A self min_lr lr N/A clipped N/A N/A LinearDecayOverEpoch N/A TrainExtension N/A N/A 
    Scales the learning rate linearly on each epochs

    Parameters
    ----------
    start : int
        The epoch on which to start shrinking the learning rate
    saturate : int
        The epoch to saturate the shrinkage
    decay_factor : float
        The final value would be initial learning rate times decay_factor
     __init__ N/A N/A self start saturate decay_factor N/A N/A N/A N/A N/A N/A self __dict__ update N/A locals N/A N/A self self N/A N/A self _initialized False N/A N/A self _count 0 N/A N/A isinstance decay_factor float N/A N/A isinstance start N/A py_integer_types py_float_types N/A N/A isinstance saturate N/A py_integer_types py_float_types N/A N/A saturate start N/A N/A start 0 N/A N/A saturate start N/A on_monitor N/A N/A self model dataset algorithm N/A N/A N/A 
        Updates the learning rate based on the linear decay schedule.

        Parameters
        ----------
        model : a Model instance
        dataset : Dataset
        algorithm : WRITEME
         N/A N/A N/A self _initialized N/A N/A N/A self _init_lr N/A N/A N/A algorithm learning_rate get_value N/A N/A self _step N/A N/A N/A self _init_lr N/A N/A self _init_lr N/A self decay_factor N/A N/A N/A self saturate N/A self start 1 N/A N/A self _initialized True N/A N/A self _count 1 N/A N/A N/A N/A algorithm learning_rate set_value N/A N/A N/A np cast N/A N/A config floatX N/A N/A self current_lr N/A current_lr N/A N/A self N/A N/A N/A 
        Returns the learning rate currently desired by the decay schedule.
         N/A N/A N/A self _count N/A self start N/A N/A N/A N/A self _count N/A self saturate N/A N/A new_lr N/A N/A self _init_lr N/A N/A self _step N/A N/A N/A self _count N/A self start 1 N/A N/A new_lr N/A N/A self _init_lr N/A self decay_factor N/A N/A new_lr N/A self _init_lr N/A N/A new_lr 0 N/A new_lr N/A N/A _PolyakWorker N/A object N/A N/A 
    Only to be used by the PolyakAveraging TrainingCallback below.
    Do not use directly.
    A callback for the SGD class.

    Parameters
    ----------
    model : a Model
        The model whose parameters we want to train with Polyak averaging
     __init__ N/A N/A self model N/A N/A N/A avg_updates N/A OrderedDict N/A t N/A sharedX 1.0 N/A N/A self param_to_mean N/A OrderedDict N/A param N/A N/A model get_params N/A N/A mean N/A sharedX N/A N/A param get_value N/A N/A N/A type mean N/A type param N/A N/A N/A self param_to_mean N/A param mean N/A N/A avg_updates N/A mean N/A mean N/A N/A mean param t N/A N/A avg_updates N/A t N/A t 1.0 N/A N/A self avg N/A function N/A updates avg_updates N/A __call__ N/A N/A self algorithm N/A N/A N/A 
        To be called after each SGD step.
        Updates the Polyak averaged-parameters for this model

        Parameters
        ----------
        algorithm : WRITEME
         N/A N/A N/A self avg N/A N/A PolyakAveraging N/A TrainExtension N/A N/A 
    See "A Tutorial on Stochastic Approximation Algorithms
        for Training Restricted Boltzmann Machines and
        Deep Belief Nets" by Kevin Swersky et al

    This functionality is still a work in progress. Currently,
    your model needs to implement "add_polyak_channels" to
    use it.

    The problem is that Polyak averaging shouldn't modify
    the model parameters. It should keep a second copy
    that it averages in the background. This second copy
    doesn't get to come back in and affect the learning process
    though.

    (IG tried having the second copy get pushed back into
    the model once per epoch, but this turned out to be
    harmful, at least in limited tests)

    So we need a cleaner interface for monitoring the
    averaged copy of the parameters, and we need to make
    sure the saved model at the end uses the averaged
    parameters, not the parameters used for computing
    the gradients during training.

    TODO: make use of the new on_save callback instead
        of duplicating Train's save_freq flag

    Parameters
    ----------
    start : int
        The epoch after which to start averaging (0 = start averaging
        immediately)
    save_path : str, optional
        WRITEME
    save_freq : int, optional
        WRITEME

    Notes
    -----
    This is usually used with a fixed, rather than annealed learning
    rate. It may be used in conjunction with momentum.
     __init__ N/A N/A self start save_path save_freq N/A None 1 N/A N/A N/A N/A N/A self __dict__ update N/A locals N/A N/A self self N/A N/A self _count 0 N/A N/A isinstance start py_integer_types N/A N/A start 0 N/A on_monitor N/A N/A self model dataset algorithm N/A N/A N/A 
        Make sure Polyak-averaged model gets monitored.
        Save the model if necessary.

        Parameters
        ----------
        model : a Model instance
        dataset : Dataset
        algorithm : WRITEME
         N/A N/A N/A self _count N/A self start N/A N/A N/A self _worker N/A _PolyakWorker model N/A N/A N/A N/A algorithm update_callbacks append N/A self _worker N/A N/A N/A N/A N/A model add_polyak_channels N/A N/A self _worker param_to_mean N/A algorithm monitoring_dataset N/A N/A N/A AttributeError N/A N/A N/A N/A N/A N/A N/A self save_path None N/A N/A self _count N/A self start N/A N/A N/A self _count N/A self save_freq 0 N/A N/A saved_params N/A OrderedDict N/A param N/A N/A model get_params N/A N/A N/A saved_params N/A param N/A N/A param get_value N/A N/A N/A param set_value N/A N/A N/A N/A N/A self _worker param_to_mean N/A param get_value N/A N/A N/A serial save N/A self save_path model N/A param N/A N/A model get_params N/A N/A N/A N/A param set_value N/A saved_params N/A param N/A N/A self _count 1 N/A N/A