Module Expr Str Import alias identifier Import alias identifier Import alias identifier Import alias Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Import alias Import alias Import alias Import alias Import alias Import alias ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults NameLoad Str body Expr Str Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Str Str If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Call AttributeLoad Str attr ListLoad NameLoad Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign NameStore BinOpAdd AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign NameStore AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr TupleLoad AttributeLoad NameLoad attr Num Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr ListLoad Assign AttributeStore NameLoad attr ListLoad Assign AttributeStore NameLoad attr UnaryOpUSub AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr BinOpDiv NameLoad BinOpAdd AttributeLoad NameLoad attr AttributeLoad NameLoad attr AugAssignMult AttributeStore NameLoad attr BinOpSub Num AttributeLoad NameLoad attr AugAssignAdd AttributeStore NameLoad attr Num If CompareEq AttributeLoad NameLoad attr Num body Assign NameStore SubscriptLoad NameLoad ExtSlice Slice Index AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr NameLoad If AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore AttributeLoad NameLoad attr attr NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad If CompareIsNot NameLoad NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore NameLoad If NameLoad body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad Num If CompareEq BinOpMod AttributeLoad NameLoad attr AttributeLoad NameLoad attr Num body Expr Call AttributeLoad NameLoad attr AugAssignAdd AttributeStore NameLoad attr NameLoad AugAssignAdd AttributeStore NameLoad attr Num If CompareLt Call AttributeLoad AttributeLoad NameLoad attr attr Num body Expr Call AttributeLoad NameLoad attr Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults NameLoad body Return Call AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Print Call AttributeLoad Str attr ListLoad AttributeLoad NameLoad attr Str Call NameLoad AttributeLoad NameLoad attr Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpDiv Call NameLoad AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Num Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Num Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Print BinOpMod Str NameLoad Expr Call AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad NameLoad body Expr Str If AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr TupleLoad Str AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr If CompareIs NameLoad NameLoad body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr NameLoad keyword Str If NameLoad body Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore NameLoad Assign NameStore NameLoad Print Str TryExcept body With Call NameLoad AttributeLoad NameLoad attr Str NameStore body Expr Call AttributeLoad NameLoad attr NameLoad NameLoad If NameLoad body With Call NameLoad Call AttributeLoad Str attr TupleLoad AttributeLoad NameLoad attr Str Str NameStore body Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Print BinOpAdd BinOpAdd Str Call NameLoad AttributeLoad NameLoad attr Str handlers ExceptHandler type NameLoad name NameStore body Print BinOpAdd BinOpAdd Str Call NameLoad NameLoad Str ExceptHandler type AttributeLoad NameLoad attr name NameStore body Print BinOpAdd BinOpAdd Str Call NameLoad NameLoad Str orelse Assign NameStore NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore NameLoad TryExcept body With Call NameLoad AttributeLoad NameLoad attr Str NameStore body Assign NameStore Call AttributeLoad NameLoad attr NameLoad If BoolOpAnd CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Print Call AttributeLoad Str attr TupleLoad Str Call NameLoad AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Assign NameStore NameLoad orelse Print BinOpAdd BinOpAdd BinOpAdd BinOpAdd Str AttributeLoad NameLoad attr Str Str Str Print Str handlers ExceptHandler type NameLoad body Print BinOpMod Str AttributeLoad NameLoad attr ExceptHandler type AttributeLoad NameLoad attr name NameStore body Print BinOpMod Str NameLoad Return NameLoad decorator_list decorator_list
N/A N/A  the Agent class  N/A cPickle pickle N/A matplotlib.pyplot plt N/A numpy np N/A os N/A mod_path N/A N/A N/A os path dirname N/A N/A N/A os path abspath __file__ N/A arborkey N/A drivetrain N/A hub N/A mainspring N/A spindle N/A tools Agent N/A object N/A N/A  
    A general reinforcement learning agent
    
    It takes in an array of sensor values and 
    a reward and puts out an array of action commands at each time step.
     __init__ N/A N/A self num_sensors num_actions show agent_name N/A True test_agent N/A N/A 
        Configure the Agent

        num_sensors and num_actions are the only absolutely necessary
        arguments. They define the number of elements in the 
        sensors and actions arrays that the agent and the world use to
        communicate with each other. 
         N/A N/A self BACKUP_PERIOD 10000.0 N/A N/A self FORGETTING_RATE 0.001 N/A N/A self show show N/A N/A self name agent_name N/A N/A self log_dir N/A N/A N/A os path normpath N/A N/A N/A os path join mod_path .. log N/A N/A N/A N/A N/A os path isdir N/A self log_dir N/A N/A N/A N/A os makedirs N/A self log_dir N/A N/A self pickle_filename N/A N/A N/A os path join N/A self log_dir N/A N/A . join N/A agent_name pickle N/A N/A self num_sensors num_sensors N/A N/A self num_actions num_actions N/A min_cables N/A N/A self num_actions N/A self num_sensors N/A N/A self drivetrain N/A N/A drivetrain Drivetrain min_cables N/A num_cables N/A N/A self drivetrain cables_per_gearbox N/A N/A self hub N/A N/A hub Hub num_cables N/A N/A self spindle N/A N/A spindle Spindle num_cables N/A N/A self mainspring N/A N/A mainspring Mainspring num_cables N/A N/A self arborkey N/A N/A arborkey Arborkey N/A N/A self action N/A N/A np zeros N/A N/A self num_actions 1 N/A N/A self cumulative_reward 0 N/A N/A self time_since_reward_log 0 N/A N/A self reward_history N/A N/A N/A self reward_steps N/A N/A N/A self reward_max N/A N/A tools BIG N/A N/A self timestep 0 N/A N/A self graphing True N/A step N/A N/A self sensors unscaled_reward N/A N/A N/A N/A self reward_max N/A N/A np maximum N/A N/A np abs unscaled_reward N/A self reward_max N/A N/A self raw_reward N/A unscaled_reward N/A N/A self reward_max N/A tools EPSILON N/A N/A self reward_max N/A 1.0 N/A self FORGETTING_RATE N/A N/A self timestep 1 N/A N/A N/A sensors ndim 1 N/A N/A sensors N/A sensors N/A N/A N/A N/A np newaxis N/A feature_activities N/A N/A N/A self drivetrain step_up N/A self action sensors N/A N/A N/A self drivetrain gearbox_added N/A N/A N/A N/A N/A self hub add_cables N/A N/A self drivetrain cables_per_gearbox N/A N/A N/A N/A self spindle add_cables N/A N/A self drivetrain cables_per_gearbox N/A N/A N/A N/A self mainspring add_cables N/A N/A self drivetrain cables_per_gearbox N/A N/A N/A N/A self arborkey add_cables N/A N/A self drivetrain cables_per_gearbox N/A N/A N/A self drivetrain gearbox_added False N/A N/A hub_goal hub_reward N/A N/A N/A self hub step feature_activities N/A self raw_reward N/A mainspring_reward N/A N/A N/A self mainspring evaluate hub_goal N/A N/A attended_index attended_activity N/A N/A N/A self spindle step feature_activities N/A N/A N/A N/A self mainspring step attended_index attended_activity N/A self raw_reward N/A goal_cable N/A N/A N/A self arborkey step hub_goal mainspring_reward N/A self raw_reward N/A N/A N/A N/A self hub update feature_activities goal_cable N/A N/A goal_cable None N/A N/A N/A N/A N/A self drivetrain assign_goal goal_cable N/A N/A self action N/A N/A N/A self drivetrain step_down N/A random_action False N/A random_action N/A N/A N/A self action N/A N/A np zeros N/A N/A self action shape N/A random_action_index N/A N/A N/A np random randint N/A N/A self action size N/A N/A N/A self action N/A random_action_index 1.0 N/A N/A N/A N/A self timestep N/A self BACKUP_PERIOD 0 N/A N/A N/A N/A self _save N/A N/A self cumulative_reward unscaled_reward N/A N/A self time_since_reward_log 1 N/A N/A N/A N/A N/A np random random_sample 0.001 N/A N/A N/A N/A self visualize N/A N/A self action N/A get_index_projections N/A N/A self to_screen N/A False N/A N/A N/A N/A N/A self drivetrain get_index_projections to_screen to_screen N/A visualize N/A N/A self N/A N/A N/A  Show the current state and some history of the agent  N/A N/A N/A join N/A N/A self name is N/A str N/A self timestep time steps old N/A N/A N/A N/A self reward_history append N/A N/A float N/A self cumulative_reward N/A N/A self time_since_reward_log 1 N/A N/A self cumulative_reward 0 N/A N/A self time_since_reward_log 0 N/A N/A N/A N/A self reward_steps append N/A self timestep N/A N/A N/A self _show_reward_history N/A N/A N/A N/A self drivetrain visualize N/A N/A N/A N/A self spindle visualize N/A N/A N/A N/A self arborkey visualize N/A report_performance N/A N/A self N/A N/A N/A performance N/A N/A np mean N/A self reward_history N/A N/A Final performance is %f performance N/A N/A N/A self _show_reward_history hold_plot N/A self show N/A performance N/A _show_reward_history N/A N/A self hold_plot filename N/A False None N/A N/A  Show the agent's reward history and save it to a file  N/A N/A self graphing N/A N/A fig N/A N/A plt figure 1 N/A N/A N/A plt plot N/A self reward_steps N/A self reward_history N/A N/A N/A plt xlabel time step N/A N/A N/A plt ylabel average reward N/A N/A N/A plt title N/A N/A join N/A Reward history for  N/A self name N/A N/A N/A fig show N/A N/A N/A N/A fig canvas draw N/A N/A filename None N/A N/A filename N/A N/A N/A os path join N/A self log_dir reward_history.png N/A N/A N/A plt savefig filename format png N/A hold_plot N/A N/A N/A N/A plt show N/A _save N/A N/A self N/A N/A N/A  Archive a copy of the agent object for future use  N/A success False N/A make_backup True N/A Attempting to save agent... N/A N/A N/A N/A open N/A self pickle_filename wb agent_data N/A N/A N/A N/A pickle dump self agent_data N/A make_backup N/A N/A N/A open N/A N/A join N/A N/A self pickle_filename .bak wb agent_data_bak N/A N/A N/A N/A pickle dump self agent_data_bak N/A N/A N/A Agent data saved at  N/A str N/A self timestep  time steps N/A N/A N/A IOError N/A err N/A N/A N/A N/A File error:  N/A str err  encountered while saving agent data N/A N/A N/A pickle PickleError N/A perr N/A N/A N/A N/A Pickling error:  N/A str perr  encountered while saving agent data N/A N/A success True N/A success N/A restore N/A N/A self N/A N/A N/A  Reconstitute the agent from a previously saved agent  N/A restored_agent self N/A N/A N/A N/A open N/A self pickle_filename rb agent_data N/A N/A loaded_agent N/A N/A pickle load agent_data N/A N/A N/A N/A loaded_agent num_sensors N/A self num_sensors N/A N/A loaded_agent num_actions N/A self num_actions N/A N/A N/A N/A join N/A Agent restored at timestep  N/A str N/A loaded_agent timestep  from  N/A self pickle_filename N/A restored_agent loaded_agent N/A N/A N/A N/A N/A N/A The agent  N/A self pickle_filename  does not have  the same number of input and output elements as  the world. N/A Creating a new agent from scratch. N/A N/A N/A IOError N/A N/A N/A Couldn't open %s for loading N/A self pickle_filename N/A N/A N/A pickle PickleError N/A e N/A N/A N/A Error unpickling world: %s e N/A restored_agent N/A N/A