Module Expr Str ImportFrom alias Import alias identifier Import alias Import alias Import alias identifier ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Call NameLoad TupleLoad BinOpMult SubscriptLoad AttributeLoad NameLoad attr Index Num SubscriptLoad AttributeLoad NameLoad attr Index Num AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Dict Num Num Num Num TupleLoad Num Num TupleLoad Num Num TupleLoad Num Num TupleLoad Num Num Assign AttributeStore NameLoad attr ListLoad Call AttributeLoad NameLoad attr Num SubscriptLoad NameLoad Index Num Call AttributeLoad NameLoad attr Num SubscriptLoad NameLoad Index Num Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign SubscriptStore AttributeLoad NameLoad attr Index SubscriptLoad NameLoad Index Num Call NameLoad Call NameLoad BinOpAdd SubscriptLoad AttributeLoad NameLoad attr Index SubscriptLoad NameLoad Index Num SubscriptLoad NameLoad Index Num Num BinOpSub SubscriptLoad AttributeLoad NameLoad attr Index SubscriptLoad NameLoad Index Num Num Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign SubscriptStore AttributeLoad NameLoad attr Index TupleLoad BinOpAdd BinOpMult SubscriptLoad AttributeLoad NameLoad attr Index Num SubscriptLoad NameLoad Index Num SubscriptLoad NameLoad Index Num NameLoad BinOpAdd BinOpMult SubscriptLoad AttributeLoad NameLoad attr Index TupleLoad BinOpAdd BinOpMult SubscriptLoad AttributeLoad NameLoad attr Index Num SubscriptLoad NameLoad Index Num SubscriptLoad NameLoad Index Num NameLoad BinOpSub Num AttributeLoad NameLoad attr BinOpMult BinOpAdd NameLoad BinOpMult AttributeLoad NameLoad attr Call NameLoad SubscriptLoad AttributeLoad NameLoad attr Index BinOpAdd BinOpMult SubscriptLoad AttributeLoad NameLoad attr Index Num SubscriptLoad AttributeLoad NameLoad attr Index Num SubscriptLoad AttributeLoad NameLoad attr Index Num AttributeLoad NameLoad attr If CompareEq Call NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Return NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body If CompareEq Call NameLoad NameLoad AttributeLoad NameLoad attr body Return Num Return Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore ListLoad Assign NameStore ListLoad Assign NameStore Num Assign NameStore Num For NameStore Call NameLoad NameLoad body If BoolOpAnd CompareEq BinOpMod NameLoad Num Num CompareGtE NameLoad Num body Print Str NameLoad BinOpDiv Call NameLoad SubscriptLoad NameLoad Slice BinOpSub NameLoad Num NameLoad Num BinOpDiv Call NameLoad SubscriptLoad NameLoad Slice BinOpSub NameLoad Num NameLoad Num Assign AttributeStore NameLoad attr ListLoad Call AttributeLoad NameLoad attr Num SubscriptLoad AttributeLoad NameLoad attr Index Num Call AttributeLoad NameLoad attr Num SubscriptLoad AttributeLoad NameLoad attr Index Num Assign NameStore Num Assign NameStore Num Assign NameStore NameLoad Assign NameStore ListLoad While NameLoad body AugAssignAdd NameStore Num Assign NameStore SubscriptLoad Call NameLoad CompareEq SubscriptLoad AttributeLoad NameLoad attr Index BinOpAdd BinOpMult SubscriptLoad AttributeLoad NameLoad attr Index Num SubscriptLoad AttributeLoad NameLoad attr Index Num SubscriptLoad AttributeLoad NameLoad attr Index Num Call NameLoad SubscriptLoad AttributeLoad NameLoad attr Index BinOpAdd BinOpMult SubscriptLoad AttributeLoad NameLoad attr Index Num SubscriptLoad AttributeLoad NameLoad attr Index Num SubscriptLoad AttributeLoad NameLoad attr Index Num Index Num Assign NameStore IfExp CompareLt Call AttributeLoad NameLoad attr Num SubscriptLoad NameLoad Index Call AttributeLoad NameLoad attr Num Call NameLoad NameLoad Call AttributeLoad NameLoad attr Num AttributeLoad NameLoad attr Assign NameStore UnaryOpNot Call AttributeLoad NameLoad attr NameLoad AugAssignAdd NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr ListComp IfExp CompareGtE NameLoad Num BinOpDiv Call NameLoad SubscriptLoad NameLoad Slice BinOpSub NameLoad Num NameLoad Num BinOpDiv Call NameLoad SubscriptLoad NameLoad Slice Num NameLoad NameLoad comprehension NameStore Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr ListComp IfExp CompareGtE NameLoad Num BinOpDiv Call NameLoad SubscriptLoad NameLoad Slice BinOpSub NameLoad Num NameLoad Num BinOpDiv Call NameLoad SubscriptLoad NameLoad Slice Num NameLoad NameLoad comprehension NameStore Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr decorator_list decorator_list If CompareEq NameLoad Str body Assign NameStore Call NameLoad TupleLoad Num Num Num Num TupleLoad Num Num Expr Call AttributeLoad NameLoad attr Num Print AttributeLoad NameLoad attr
N/A N/A 
Created on Mar 22, 2010
@author: Zachary Varberg
 numpy * N/A scipy Sci N/A scipy.linalg N/A copy N/A matplotlib.pyplot pyplot simple_RL N/A object N/A __init__ N/A N/A self num_states alpha gamma goal_state N/A N/A N/A N/A self num_states num_states N/A N/A self alpha alpha N/A N/A self gamma gamma N/A N/A self num_actions 4 N/A N/A self Q_mat N/A zeros N/A N/A N/A N/A self num_states N/A 0 N/A N/A self num_states N/A 1 N/A self num_actions N/A N/A self goal_state goal_state N/A N/A self trans_dict N/A 0 1 2 3 N/A 0 -1 N/A 1 1 N/A 0 1 N/A 1 -1 N/A N/A self start_state N/A N/A N/A random randint 0 N/A num_states N/A 0 N/A N/A random randint 0 N/A num_states N/A 1 N/A N/A self curr_state N/A self start_state N/A execute_action N/A N/A self action N/A N/A N/A move N/A N/A self trans_dict N/A action N/A prev_state N/A N/A copy deepcopy N/A self curr_state N/A N/A N/A self curr_state N/A N/A move N/A 0 N/A min N/A max N/A N/A N/A self curr_state N/A N/A move N/A 0 N/A move N/A 1 0 N/A N/A N/A self num_states N/A N/A move N/A 0 1 N/A r N/A N/A self get_reward N/A self curr_state N/A N/A N/A self Q_mat N/A N/A N/A N/A N/A N/A self num_states N/A 0 N/A prev_state N/A 0 N/A prev_state N/A 1 action N/A N/A N/A N/A self Q_mat N/A N/A N/A N/A N/A N/A self num_states N/A 0 N/A prev_state N/A 0 N/A prev_state N/A 1 action N/A 1 N/A self alpha N/A N/A r N/A N/A self gamma N/A max N/A N/A self Q_mat N/A N/A N/A N/A N/A self num_states N/A 0 N/A N/A self curr_state N/A 0 N/A N/A self curr_state N/A 1 N/A self alpha N/A N/A N/A tuple N/A self curr_state N/A self goal_state N/A N/A True N/A False N/A get_reward N/A N/A self state N/A N/A N/A N/A N/A tuple state N/A self goal_state N/A N/A 10 N/A -1 N/A run N/A N/A self num_trials N/A N/A N/A tot_reward N/A N/A tot_steps N/A N/A tri_reward 0 N/A num_steps 0 N/A x N/A xrange num_trials N/A N/A N/A N/A N/A x 10 0 N/A x 10 N/A N/A Trial x N/A N/A sum N/A tot_reward N/A N/A x 10 x 10 N/A N/A sum N/A tot_steps N/A N/A x 10 x 10 N/A N/A self curr_state N/A N/A N/A random randint 0 N/A N/A self num_states N/A 0 N/A N/A random randint 0 N/A N/A self num_states N/A 1 N/A tri_reward 0 N/A num_steps 0 N/A not_done True N/A path N/A N/A not_done N/A N/A num_steps 1 N/A best_action N/A N/A nonzero N/A N/A N/A self Q_mat N/A N/A N/A N/A N/A self num_states N/A 0 N/A N/A self curr_state N/A 0 N/A N/A self curr_state N/A 1 N/A max N/A N/A self Q_mat N/A N/A N/A N/A N/A self num_states N/A 0 N/A N/A self curr_state N/A 0 N/A N/A self curr_state N/A 1 N/A 0 N/A next_action N/A N/A N/A N/A random random 0.9 N/A best_action N/A N/A N/A random randint 0 N/A len best_action N/A N/A random randint 0 N/A self num_actions N/A not_done N/A N/A N/A self execute_action next_action N/A tri_reward N/A N/A self get_reward N/A self curr_state N/A N/A N/A path append N/A N/A copy deepcopy N/A self curr_state N/A N/A N/A tot_reward append tri_reward N/A N/A N/A tot_steps append num_steps N/A fig1 N/A N/A pyplot figure 1 N/A N/A N/A pyplot plot N/A N/A N/A x 100 N/A N/A sum N/A tot_reward N/A N/A x 100 x 100 N/A N/A sum N/A tot_reward N/A 0 x x N/A x N/A xrange num_trials N/A N/A N/A fig1 suptitle Rewards N/A fig2 N/A N/A pyplot figure 2 N/A N/A N/A pyplot plot N/A N/A N/A x 100 N/A N/A sum N/A tot_steps N/A N/A x 100 x 100 N/A N/A sum N/A tot_steps N/A 0 x x N/A x N/A xrange num_trials N/A N/A N/A fig2 suptitle Steps N/A N/A N/A pyplot show N/A N/A N/A N/A __name__ __main__ N/A N/A rl N/A simple_RL N/A 25 25 0.05 0.95 N/A 7 6 N/A N/A N/A rl run 1000 N/A N/A rl Q_mat