Module Expr Str ImportFrom alias Import alias identifier Import alias Import alias ImportFrom alias Import alias Import alias identifier ClassDef bases AttributeLoad NameLoad attr body FunctionDef arguments args NameParam NameParam NameParam NameParam defaults TupleLoad Num Num body Assign AttributeStore NameLoad attr BinOpMult SubscriptLoad NameLoad Index Num SubscriptLoad NameLoad Index Num Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Call NameLoad TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Dict Num Num Num Num Num Num Num Num decorator_list FunctionDef arguments args NameParam defaults body Return Call AttributeLoad NameLoad attr Num AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body If CompareEq BinOpAdd AttributeLoad NameLoad attr SubscriptLoad AttributeLoad NameLoad attr Index NameLoad BinOpDiv BinOpMult AttributeLoad NameLoad attr Num Num body Return Num Return Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore SubscriptLoad Call NameLoad CompareEq SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Call NameLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index Num Return IfExp CompareLt Call AttributeLoad NameLoad attr Num SubscriptLoad NameLoad Index Call AttributeLoad NameLoad attr Num Call NameLoad NameLoad Call AttributeLoad NameLoad attr Num AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad Call NameLoad BinOpAdd AttributeLoad NameLoad attr NameLoad Num BinOpSub AttributeLoad NameLoad attr Num If BoolOpAnd CompareEq BinOpMod AttributeLoad NameLoad attr SubscriptLoad AttributeLoad NameLoad attr Index Num Num CompareEq BinOpMod AttributeLoad NameLoad attr SubscriptLoad AttributeLoad NameLoad attr Index Num Num body AugAssignSub AttributeStore NameLoad attr Num If BoolOpAnd CompareEq BinOpMod AttributeLoad NameLoad attr SubscriptLoad AttributeLoad NameLoad attr Index Num Num CompareEq BinOpMod AttributeLoad NameLoad attr SubscriptLoad AttributeLoad NameLoad attr Index Num Num body AugAssignAdd AttributeStore NameLoad attr Num Assign SubscriptStore AttributeLoad NameLoad attr Index TupleLoad AttributeLoad NameLoad attr NameLoad BinOpAdd BinOpMult SubscriptLoad AttributeLoad NameLoad attr Index TupleLoad AttributeLoad NameLoad attr NameLoad BinOpSub Num AttributeLoad NameLoad attr BinOpMult BinOpAdd NameLoad BinOpMult AttributeLoad NameLoad attr Call NameLoad SubscriptLoad AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr AttributeLoad NameLoad attr If CompareEq NameLoad Num body Return NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr decorator_list decorator_list If CompareEq NameLoad Str body Assign NameStore Call NameLoad Num Num Expr Call AttributeLoad NameLoad attr Num Print AttributeLoad NameLoad attr
N/A N/A 
Created on Mar 23, 2010
@author: Zachary Varberg
 numpy * N/A scipy Sci N/A scipy.linalg N/A copy matplotlib pyplot N/A abstract_rl N/A lib.matplotlib_util mpl_util Simple_RL N/A N/A abstract_rl AbstractReinforcementLearner N/A __init__ N/A N/A self alpha gamma num_states N/A N/A 15 15 N/A N/A N/A self num_states N/A N/A num_states N/A 0 N/A num_states N/A 1 N/A N/A self dimensions num_states N/A N/A self alpha alpha N/A N/A self gamma gamma N/A N/A self curr_state N/A N/A self get_start_state N/A N/A self num_actions 4 N/A N/A self Q_mat N/A zeros N/A N/A self num_states N/A self num_actions N/A N/A self trans_mat N/A 0 1 2 3 -15 1 15 -1 N/A get_start_state N/A N/A self N/A N/A N/A N/A N/A random randint 0 N/A self num_states N/A get_reward N/A N/A self prev_state action N/A N/A N/A N/A N/A N/A self prev_state N/A N/A self trans_mat N/A action N/A N/A N/A self num_states 3 4 N/A N/A 10 N/A -1 N/A select_action N/A N/A self state N/A N/A N/A best_action N/A N/A nonzero N/A N/A N/A self Q_mat N/A state N/A max N/A N/A self Q_mat N/A state N/A 0 N/A N/A N/A N/A N/A random random 0.9 N/A best_action N/A N/A N/A random randint 0 N/A len best_action N/A N/A random randint 0 N/A self num_actions N/A execute_action N/A N/A self state action N/A N/A N/A move N/A N/A self trans_mat N/A action N/A N/A self prev_state state N/A r N/A N/A self get_reward N/A self prev_state action N/A N/A self curr_state N/A min N/A max N/A N/A self curr_state move 0 N/A N/A self num_states 1 N/A N/A N/A N/A N/A self prev_state N/A N/A self dimensions N/A 0 0 N/A N/A N/A self curr_state N/A N/A self dimensions N/A 0 14 N/A N/A N/A self curr_state 1 N/A N/A N/A N/A N/A self prev_state N/A N/A self dimensions N/A 0 14 N/A N/A N/A self curr_state N/A N/A self dimensions N/A 0 0 N/A N/A N/A self curr_state 1 N/A N/A N/A self Q_mat N/A N/A N/A self prev_state action N/A N/A N/A N/A self Q_mat N/A N/A N/A self prev_state action N/A 1 N/A self alpha N/A N/A r N/A N/A self gamma N/A max N/A N/A self Q_mat N/A N/A self curr_state N/A self alpha N/A N/A r 10 N/A N/A True N/A False N/A display N/A N/A self N/A N/A N/A fig1 N/A N/A pyplot figure 1 N/A N/A N/A mpl_util plot_running_average N/A self tot_reward 100 N/A N/A N/A fig1 suptitle Rewards N/A fig2 N/A N/A pyplot figure 2 N/A N/A N/A mpl_util plot_running_average N/A self tot_steps 100 N/A N/A N/A fig2 suptitle Steps N/A N/A N/A pyplot show N/A N/A N/A N/A __name__ __main__ N/A N/A rl N/A Simple_RL 0.1 0.95 N/A N/A N/A rl run 1000 N/A N/A rl Q_mat