Module Import alias Import alias identifier Import alias Import alias ImportFrom alias alias Import alias Import alias Import alias identifier ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad Lambda arguments args defaults BinOpAdd Num BinOpMult Call AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad Num Assign AttributeStore NameLoad attr Call NameLoad Expr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Num decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr Str Print BinOpMod Str NameLoad Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Str Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Str Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr SubscriptLoad AttributeLoad NameLoad attr Index NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body If CompareEq Call NameLoad NameLoad NameLoad body Assign NameStore Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Print Str NameLoad Str SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Str NameLoad Str NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults body If CompareEq Call NameLoad NameLoad NameLoad body Assign TupleStore NameStore NameStore TupleLoad Call NameLoad NameLoad Call NameLoad NameLoad Assign NameStore SubscriptLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index NameLoad Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Assign NameStore BinOpAdd NameLoad BinOpMult AttributeLoad NameLoad attr NameLoad Assign NameStore BinOpAdd BinOpMult BinOpSub Num AttributeLoad NameLoad attr NameLoad BinOpMult AttributeLoad NameLoad attr NameLoad Assign SubscriptStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index NameLoad NameLoad Print Str NameLoad Str NameLoad Str NameLoad Str NameLoad Str NameLoad Str NameLoad Str NameLoad Str NameLoad AugAssignAdd SubscriptStore AttributeLoad NameLoad attr Index NameLoad Num AugAssignAdd AttributeStore NameLoad attr Num If CompareEq BinOpMod AttributeLoad NameLoad attr Num Num body Print Str Print Str Assign NameStore Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr For TupleStore NameStore NameStore Call NameLoad NameLoad keyword Lambda arguments args TupleStore NameStore NameStore defaults UnaryOpUSub NameLoad body If CompareGt NameLoad Num body Print Call AttributeLoad Str attr Call NameLoad NameLoad ListLoad NameLoad NameLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Call AttributeLoad NameLoad attr NameLoad Print Str AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr decorator_list decorator_list
N/A N/A rospy N/A numpy np N/A itertools N/A sys collections defaultdict Counter N/A random N/A states N/A util u DiscreteQTablePolicy N/A object N/A __init__ N/A N/A self num_actions N/A N/A N/A N/A self num_actions num_actions N/A N/A self q_table N/A defaultdict N/A N/A N/A N/A N/A 10 N/A N/A N/A N/A np random uniform size num_actions 3 N/A N/A self state_train_freq N/A Counter N/A N/A N/A self refresh_params N/A N/A self calls_to_train 0 N/A refresh_params N/A N/A self N/A N/A N/A params N/A N/A rospy get_param q_table_policy N/A N/A REFRESH_PARAM	%s params N/A N/A self discount N/A params N/A discount N/A N/A self learning_rate N/A params N/A learning_rate N/A N/A self state_normalisation_squash N/A params N/A state_normalisation_squash N/A q_values_normalised_for_pick N/A N/A self state N/A N/A N/A N/A N/A u normalised N/A N/A u raised N/A N/A self q_table N/A state N/A self state_normalisation_squash N/A action_given_state N/A N/A self state N/A N/A N/A N/A N/A type state list N/A N/A state N/A tuple state N/A normed N/A N/A self q_values_normalised_for_pick state N/A action N/A N/A u weighted_choice normed N/A CHOOSE	 based on state state q_values N/A N/A self q_table N/A state (normed to normed ) => action action N/A action N/A train N/A N/A self state_1 action reward state_2 N/A N/A N/A N/A N/A type state_1 list N/A N/A N/A state_1 state_2 N/A N/A tuple state_1 N/A tuple state_2 N/A current_q_s_a N/A N/A N/A self q_table N/A state_1 N/A action N/A max_possible_return_from_state_2 N/A N/A np max N/A N/A self q_table N/A state_2 N/A candidate_q_s_a N/A reward N/A N/A self discount max_possible_return_from_state_2 N/A updated_q_s_a N/A N/A N/A 1.0 N/A self learning_rate current_q_s_a N/A N/A self learning_rate candidate_q_s_a N/A N/A N/A N/A self q_table N/A state_1 N/A action updated_q_s_a N/A TRAIN	state_1 state_1 action action reward reward state_2 state_2  ... current_q_s_a current_q_s_a max_possible_return_from_state_2 max_possible_return_from_state_2 candidate_q_s_a candidate_q_s_a updated_q_s_a updated_q_s_a N/A N/A N/A self state_train_freq N/A state_1 1 N/A N/A self calls_to_train 1 N/A N/A N/A N/A self calls_to_train 100 0 N/A N/A >>> end of episode stats N/A DEBUG QTABLE: N/A state_freqs N/A list N/A N/A N/A self state_train_freq iteritems N/A N/A state freq N/A sorted state_freqs key N/A N/A N/A N/A s f N/A N/A f N/A N/A N/A freq 2 N/A N/A N/A N/A join N/A map str N/A state freq N/A N/A self q_table N/A state N/A N/A self q_values_normalised_for_pick state N/A state_train_freq N/A self state_train_freq N/A N/A N/A self refresh_params N/A N/A