Module Import alias identifier ImportFrom alias ImportFrom alias identifier ImportFrom alias identifier ImportFrom alias identifier ImportFrom alias ImportFrom alias ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam defaults Num Num body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr Assign AttributeStore NameLoad attr Call NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr BinOpPow NameLoad Num Assign AttributeStore NameLoad attr BinOpDiv Num Num Assign AttributeStore NameLoad attr Call NameLoad Num keyword Str Assign AttributeStore NameLoad attr Call NameLoad BinOpDiv AttributeLoad NameLoad attr BinOpPow Num AttributeLoad NameLoad attr keyword Str decorator_list FunctionDef arguments args NameParam defaults body Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Num Assign SubscriptStore AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr BinOpDiv AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr BinOpAdd Num AttributeLoad NameLoad attr AttributeLoad NameLoad attr For TupleStore NameStore NameStore Call AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad Call AttributeLoad NameLoad attr attr Num AttributeLoad NameLoad attr Assign SubscriptStore NameLoad Index NameLoad BinOpAdd NameLoad NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Dict Str Str Str Str Str Str Str Str Num AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr ListComp Call AttributeLoad SubscriptLoad NameLoad Index Num attr comprehension NameStore AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad NameLoad attr Call NameLoad NameLoad BinOpAdd AttributeLoad Call NameLoad NameLoad attr Str NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad NameLoad BinOpAdd AttributeLoad Call NameLoad NameLoad attr Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr SubscriptLoad NameLoad Index Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr SubscriptLoad NameLoad Index Str Assign SubscriptStore AttributeLoad AttributeLoad NameLoad attr attr Slice SubscriptLoad NameLoad Index Str For TupleStore NameStore NameStore Call NameLoad AttributeLoad AttributeLoad NameLoad attr attr SubscriptLoad NameLoad Index Str body Expr Call AttributeLoad SubscriptLoad NameLoad Index Num attr NameLoad decorator_list decorator_list
N/A N/A numpy np collections OrderedDict theano tensor T os.path join pjoin theano.sandbox.rng_mrg MRG_RandomStreams RandomStreams smartlearner.interfaces.direction_modifier DirectionModifier smartlearner.utils sharedX GradientNoise N/A DirectionModifier N/A N/A 
    Adds annealed gaussian noise to the gradients at every time step, by decaying the variance at each time step

    g_t <- g_t + N(0, sigma_t^2)
    sigma_t^2 = eta / (1 + t)^gamma

    with eta selected from {0.01, 0.3, 1.0) and gamma = 0.55

    See: "Adding gradient noise improves learning for very deep networks", http://arxiv.org/pdf/1511.06807v1.pdf
     __init__ N/A N/A self eta seed N/A 0.01 1234 N/A N/A N/A N/A N/A super GradientNoise self __init__ N/A N/A self _updates N/A OrderedDict N/A N/A self _seed seed N/A N/A self _srng N/A RandomStreams N/A self _seed N/A N/A self _eta N/A eta 0.5 N/A N/A self _gamma N/A 0.55 2 N/A N/A self t N/A sharedX 1 name gradient_noise_t N/A N/A self std N/A sharedX N/A N/A self _eta N/A 1 N/A self _gamma name gradient_noise_std N/A _get_updates N/A N/A self N/A N/A N/A N/A self _updates N/A apply N/A N/A self directions N/A N/A N/A new_directions N/A OrderedDict N/A N/A N/A self _updates N/A N/A self t N/A N/A self t 1 N/A N/A N/A self _updates N/A N/A self std N/A N/A self _eta N/A N/A T pow N/A 1 N/A self t N/A self _gamma N/A N/A param gparam N/A N/A directions items N/A N/A gradient_noise N/A N/A N/A self _srng normal N/A N/A N/A param get_value shape 0 N/A self std N/A N/A new_directions N/A param N/A gparam gradient_noise N/A new_directions N/A save N/A N/A self path N/A N/A N/A state N/A version self._eta self._gamma self._seed t std _srng_rstate _srng_state_updates 1 N/A self _eta N/A self _gamma N/A self _seed N/A N/A N/A self t get_value N/A N/A N/A self std get_value N/A N/A self _srng rstate N/A N/A N/A N/A state_update N/A 0 get_value N/A state_update N/A N/A self _srng state_updates N/A N/A N/A np savez N/A pjoin path N/A N/A N/A type self __name__ .npz state N/A load N/A N/A self path N/A N/A N/A state N/A N/A np load N/A pjoin path N/A N/A N/A type self __name__ .npz N/A N/A N/A N/A self t set_value N/A state N/A t N/A N/A N/A N/A self std set_value N/A state N/A std N/A N/A N/A N/A self _srng rstate N/A N/A state N/A _srng_rstate N/A N/A state_update saved_state N/A zip N/A N/A self _srng state_updates N/A state N/A _srng_state_updates N/A N/A N/A N/A N/A state_update N/A 0 set_value saved_state N/A N/A