Module Expr Str ImportFrom alias alias ImportFrom alias alias Import alias identifier Assign NameStore Str Assign NameStore ListLoad Str Str Str Str Str Assign NameStore Str ClassDef bases NameLoad NameLoad body Expr Str Assign NameStore Num Assign NameStore ListLoad FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults Num kwarg body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr BinOpMult AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr keyword NameLoad keyword NameLoad keyword NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults body Expr Str Pass decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults body Assign NameStore NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad NameLoad NameLoad NameLoad Assign NameStore AttributeLoad NameLoad attr Assign NameStore AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad NameLoad NameLoad Assign NameStore Call NameLoad NameLoad If AttributeLoad NameLoad attr body Assign NameStore BinOpDiv BinOpAdd UnaryOpUSub Call NameLoad AttributeLoad NameLoad attr Call NameLoad NameLoad AttributeLoad AttributeLoad NameLoad attr attr If CompareGt NameLoad Num body Assign AttributeStore NameLoad attr Call NameLoad AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr TupleLoad AttributeLoad AttributeLoad NameLoad attr attr NameLoad AugAssignMult AttributeStore NameLoad attr BinOpMult NameLoad AttributeLoad NameLoad attr AugAssignAdd AttributeStore NameLoad attr NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index CompareGt AttributeLoad NameLoad attr Num Num orelse Assign AttributeStore NameLoad attr NameLoad Assign NameStore BinOpAdd NameLoad Call AttributeLoad NameLoad attr BinOpSub BinOpMult NameLoad NameLoad NameLoad NameLoad If CompareGt NameLoad Num body Expr Call AttributeLoad NameLoad attr NameLoad NameLoad AttributeLoad NameLoad attr NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr AugAssignAdd NameStore BinOpMult BinOpMult BinOpMult AttributeLoad NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad AttributeLoad NameLoad attr If UnaryOpNot Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad body Assign NameStore NameLoad Print Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad NameLoad NameLoad NameLoad If NameLoad body Expr Call AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults body Expr Str Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad NameLoad NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults body Expr Str Return NameLoad decorator_list decorator_list
N/A N/A Control Agents based on TD Learning, i.e., Q-Learning and SARSA Agent Agent DescentAlgorithm rlpy.Tools addNewElementForAllActions count_nonzero N/A numpy np N/A __copyright__ Copyright 2013, RLPy http://acl.mit.edu/RLPy N/A __credits__ N/A Alborz Geramifard Robert H. Klein Christoph Dann William Dabney Jonathan P. How N/A __license__ BSD 3-Clause TDControlAgent N/A DescentAlgorithm Agent N/A N/A 
    abstract class for the control variants of the classical linear TD-Learning.
    It is the parent of SARSA and Q-Learning

    All children must implement the _future_action function.
     N/A lambda_ 0 N/A eligibility_trace N/A __init__ N/A N/A self policy representation discount_factor lambda_ N/A 0 kwargs N/A N/A N/A self eligibility_trace N/A N/A np zeros N/A N/A representation features_num N/A representation actions_num N/A N/A self lambda_ lambda_ N/A N/A N/A N/A super TDControlAgent self __init__ policy policy representation representation discount_factor discount_factor kwargs N/A _future_action N/A N/A self ns terminal np_actions ns_phi na N/A N/A N/A needs to be implemented by children N/A N/A learn N/A N/A self s p_actions a r ns np_actions na terminal N/A N/A N/A prevStateTerminal False N/A N/A N/A N/A self representation pre_discover s prevStateTerminal a ns terminal N/A discount_factor N/A self discount_factor N/A weight_vec N/A N/A self representation weight_vec N/A phi_s N/A N/A N/A self representation phi s prevStateTerminal N/A phi N/A N/A N/A self representation phi_sa s prevStateTerminal a phi_s N/A phi_prime_s N/A N/A N/A self representation phi ns terminal N/A na N/A N/A self _future_action ns terminal np_actions phi_prime_s na N/A phi_prime N/A N/A N/A self representation phi_sa ns terminal na phi_prime_s N/A nnz N/A count_nonzero phi_s N/A N/A self lambda_ N/A N/A expanded N/A N/A N/A N/A len N/A self eligibility_trace N/A len phi N/A N/A self representation actions_num N/A N/A expanded 0 N/A N/A N/A self eligibility_trace N/A addNewElementForAllActions N/A self eligibility_trace N/A N/A self representation actions_num N/A N/A np zeros N/A N/A N/A self representation actions_num expanded N/A N/A self eligibility_trace N/A discount_factor N/A self lambda_ N/A N/A self eligibility_trace phi N/A N/A N/A self eligibility_trace N/A N/A N/A self eligibility_trace 1 1 N/A N/A N/A self eligibility_trace phi N/A td_error N/A r N/A N/A np dot N/A N/A discount_factor phi_prime phi weight_vec N/A N/A nnz 0 N/A N/A N/A N/A self updateLearnRate phi phi_prime N/A self eligibility_trace discount_factor nnz terminal N/A weight_vec_old N/A N/A weight_vec copy N/A weight_vec N/A N/A N/A N/A self learn_rate N/A N/A N/A self representation featureLearningRate td_error N/A self eligibility_trace N/A N/A N/A N/A np all N/A N/A np isfinite weight_vec N/A N/A weight_vec weight_vec_old N/A WARNING: TD-Learning diverged, weight_vec reached infinity! N/A expanded N/A N/A N/A self representation post_discover s prevStateTerminal a td_error phi_s N/A terminal N/A N/A N/A N/A self episodeTerminated N/A N/A Q_Learning N/A TDControlAgent N/A N/A 
    The off-policy variant known as Q-Learning
     _future_action N/A N/A self ns terminal np_actions ns_phi na N/A N/A N/A Q Learning chooses the optimal action N/A N/A N/A N/A self representation bestAction ns terminal np_actions ns_phi N/A N/A SARSA N/A TDControlAgent N/A N/A 
    The on-policy variant known as SARSA.
     _future_action N/A N/A self ns terminal np_actions ns_phi na N/A N/A N/A SARS-->A<--, so SARSA simply chooses the action the agent will follow N/A na N/A N/A