Module Expr Str ImportFrom alias alias alias Import alias identifier ImportFrom alias Import alias ImportFrom alias alias Assign NameStore Str Assign NameStore ListLoad Str Str Str Str Str Assign NameStore Str Assign NameStore Str ClassDef bases NameLoad body Expr Str Assign NameStore NameLoad Assign NameStore NameStore Num Assign NameStore Num Assign NameStore Num Assign NameStore Num Assign NameStore Num Assign NameStore Num Assign NameStore Num Assign TupleStore NameStore NameStore NameStore NameStore Call NameLoad Num Assign NameStore Call AttributeLoad NameLoad attr ListLoad ListLoad Num Num ListLoad UnaryOpUAdd Num Num ListLoad Num Num ListLoad Num UnaryOpUAdd Num ListLoad Num Num Assign NameStore NameLoad Assign NameStore NameLoad Assign NameStore NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Str Str FunctionDef arguments args NameParam NameParam defaults Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Str body Expr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr BinOpMult Num BinOpAdd AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr ListLoad ListLoad Num BinOpSub AttributeLoad NameLoad attr Num ListLoad Num BinOpSub AttributeLoad NameLoad attr Num Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad TupleLoad BinOpAdd AttributeLoad NameLoad attr AttributeLoad NameLoad attr Num Assign AttributeStore NameLoad attr BinOpPow Num AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr BinOpMult ListLoad Num AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr ListLoad Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad keyword AttributeLoad NameLoad attr If CompareEq AttributeLoad AttributeLoad NameLoad attr attr Num body Assign AttributeStore NameLoad attr SubscriptLoad AttributeLoad NameLoad attr ExtSlice Index AttributeLoad NameLoad attr Slice Assign TupleStore AttributeStore NameLoad attr AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign TupleStore NameStore NameStore Call AttributeLoad CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr AttributeLoad Call AttributeLoad NameLoad attr ListLoad NameLoad NameLoad attr Assign AttributeStore NameLoad attr Call NameLoad AttributeLoad NameLoad attr Assign TupleStore NameStore NameStore Call AttributeLoad CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr AttributeLoad Call AttributeLoad NameLoad attr ListLoad NameLoad NameLoad attr Assign AttributeStore NameLoad attr Call NameLoad AttributeLoad NameLoad attr Assign TupleStore NameStore NameStore Call AttributeLoad CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr AttributeLoad Call AttributeLoad NameLoad attr ListLoad NameLoad NameLoad attr Assign AttributeStore NameLoad attr Call NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad SubscriptLoad NameLoad Slice BinOpMult AttributeLoad NameLoad attr Num attr Num Num Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad AugAssignAdd NameStore NameLoad Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad SubscriptLoad NameLoad Slice BinOpMult AttributeLoad NameLoad attr Num attr Num Num Assign NameStore ListComp Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index NameLoad comprehension NameStore Call NameLoad AttributeLoad NameLoad attr Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad AugAssignAdd NameStore NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Slice BinOpMult AttributeLoad NameLoad attr Num attr Num Num Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Slice BinOpMult AttributeLoad NameLoad attr Num attr Num Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr Str Assign SubscriptStore NameLoad Index TupleLoad SubscriptLoad NameLoad ExtSlice Slice Index Num SubscriptLoad NameLoad ExtSlice Slice Index Num NameLoad Assign SubscriptStore NameLoad Index TupleLoad SubscriptLoad NameLoad ExtSlice Slice Index Num SubscriptLoad NameLoad ExtSlice Slice Index Num NameLoad Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index TupleLoad SubscriptLoad AttributeLoad NameLoad attr ExtSlice Slice Index Num SubscriptLoad AttributeLoad NameLoad attr ExtSlice Slice Index Num Assign NameStore BinOpMult NameLoad AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Return TupleLoad NameLoad NameLoad NameLoad Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr ListLoad Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr Return TupleLoad Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad ListLoad Call NameLoad AttributeLoad NameLoad attr Num Assign NameStore BinOpAdd NameLoad AttributeLoad NameLoad attr Assign NameStore SubscriptLoad NameLoad ExtSlice Slice Index Num Assign NameStore SubscriptLoad NameLoad ExtSlice Slice Index Num Assign NameStore Call AttributeLoad NameLoad attr CompareLtE Num NameLoad CompareLt NameLoad AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr CompareLtE Num NameLoad CompareLt NameLoad AttributeLoad NameLoad attr Assign TupleStore NameStore NameStore Call AttributeLoad Call AttributeLoad Call AttributeLoad NameLoad attr NameLoad NameLoad attr Num Num attr Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Print Str For NameStore Call NameLoad Num AttributeLoad NameLoad attr body Assign NameStore SubscriptLoad NameLoad Slice BinOpMult NameLoad Num BinOpAdd BinOpMult NameLoad Num Num Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Print Call AttributeLoad Str attr NameLoad NameLoad NameLoad Assign NameStore BinOpMult Num AttributeLoad NameLoad attr For NameStore Call NameLoad Num AttributeLoad NameLoad attr body Assign NameStore SubscriptLoad NameLoad Slice BinOpAdd NameLoad BinOpMult NameLoad Num BinOpAdd BinOpAdd NameLoad BinOpMult NameLoad Num Num Print Str NameLoad Assign TupleStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Print Str NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Return Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore AttributeLoad NameLoad attr If CompareIs AttributeLoad NameLoad attr NameLoad body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr keyword Str keyword Str keyword Num keyword Num Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr keyword NameLoad Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr keyword NameLoad Expr Call AttributeLoad NameLoad attr If CompareIsNot AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr Num attr Expr Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr Num attr Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Slice Num BinOpMult AttributeLoad NameLoad attr Num attr TupleLoad Num Num Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Slice BinOpMult AttributeLoad NameLoad attr Num attr TupleLoad Num Num Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Index Num SubscriptLoad NameLoad ExtSlice Slice Index Num Str keyword Num keyword Num keyword Str keyword Num Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Index Num SubscriptLoad NameLoad ExtSlice Slice Index Num Str keyword Str keyword Num keyword Num keyword Str keyword Num Expr Call AttributeLoad NameLoad attr decorator_list decorator_list
N/A N/A Intruder monitoring task. rlpy.Tools plt id2vec bound_vec N/A numpy np Domain Domain N/A os rlpy.Tools __rlpy_location__ FONTSIZE N/A __copyright__ Copyright 2013, RLPy http://acl.mit.edu/RLPy N/A __credits__ N/A Alborz Geramifard Robert H. Klein Christoph Dann William Dabney Jonathan P. How N/A __license__ BSD 3-Clause N/A __author__ N. Kemal Ure IntruderMonitoring N/A Domain N/A N/A 
    Formulated as an MDP, the intruder monitoring task is to guard danger zones using cameras
    so that if an intruder moves to a danger zone, at least one camera is pointing at that location.

    All locations are on a 2-D grid.

    The episode is finished after 1000 steps.

    **STATE:** 

    Location of: [ Agent_1, Agent_2, ... Agent n ] 

    Location of: [ Intruder_1, Intruder_2, ... Intruder_m ]


    Where *n* is number of agents, *m* is number of intruders.


    **ACTIONS:**
    [Up, Down, Left, Right, Remain]^n (one action for each agent).

    **TRANSITION:**
    Each agent can move in 4 directions + stay still.
    There is no noise on any movements.
    Each intruder moves with a fixed policy (specified by the user)
    By Default, intruder policy is uniform random.

    Map of the world contains fixed number of danger zones. Maps are simple text files
    contained in the ``Domains/IntruderMonitoringMaps/`` directory.

    **REWARD:** 

    -1 for every visit of an intruder to a danger zone with no camera present.

    The team receives a penalty whenever there is an intruder on a danger zone in the
    absence of an agent. The task is to allocate agents on the map so that intruders
    do not enter the danger zones without attendance of an agent.

     N/A map None N/A ROWS COLS 0 N/A NUMBER_OF_AGENTS 0 N/A NUMBER_OF_INTRUDERS 0 N/A NUMBER_OF_DANGER_ZONES 0 N/A discount_factor 0.8 N/A INTRUSION_PENALTY -1.0 N/A episodeCap 100 N/A N/A EMPTY INTRUDER AGENT DANGER N/A xrange 4 N/A ACTIONS_PER_AGENT N/A N/A np array N/A N/A -1 0 N/A N/A 1 0 N/A 0 -1 N/A 0 N/A 1 N/A 0 0 N/A domain_fig None N/A ally_fig None N/A intruder_fig None N/A default_map_dir N/A N/A N/A os path join __rlpy_location__ Domains IntruderMonitoringMaps __init__ N/A N/A self mapname N/A N/A N/A N/A os path join default_map_dir 4x4_2A_3I.txt N/A N/A N/A N/A self setupMap mapname N/A N/A self state_space_dims N/A 2 N/A N/A self NUMBER_OF_AGENTS N/A self NUMBER_OF_INTRUDERS N/A _statespace_limits N/A N/A np vstack N/A N/A 0 N/A N/A self ROWS 1 N/A 0 N/A N/A self COLS 1 N/A N/A self statespace_limits N/A N/A np tile _statespace_limits N/A N/A N/A self NUMBER_OF_AGENTS N/A self NUMBER_OF_INTRUDERS 1 N/A N/A self actions_num N/A 5 N/A self NUMBER_OF_AGENTS N/A N/A self ACTION_LIMITS N/A N/A 5 N/A self NUMBER_OF_AGENTS N/A N/A self DimNames N/A N/A N/A N/A N/A super IntruderMonitoring self __init__ N/A setupMap N/A N/A self mapname N/A N/A N/A N/A self map N/A N/A np loadtxt mapname dtype N/A np uint8 N/A N/A N/A N/A self map ndim 1 N/A N/A N/A self map N/A N/A self map N/A N/A N/A np newaxis N/A N/A N/A N/A self ROWS N/A self COLS N/A N/A np shape N/A self map N/A N/A R C N/A N/A N/A N/A self map N/A self AGENT nonzero N/A N/A self agents_initial_locations N/A N/A N/A np vstack N/A R C T N/A N/A self NUMBER_OF_AGENTS N/A len N/A self agents_initial_locations N/A N/A R C N/A N/A N/A N/A self map N/A self INTRUDER nonzero N/A N/A self intruders_initial_locations N/A N/A N/A np vstack N/A R C T N/A N/A self NUMBER_OF_INTRUDERS N/A len N/A self intruders_initial_locations N/A N/A R C N/A N/A N/A N/A self map N/A self DANGER nonzero N/A N/A self danger_zone_locations N/A N/A N/A np vstack N/A R C T N/A N/A self NUMBER_OF_DANGER_ZONES N/A len N/A self danger_zone_locations N/A step N/A N/A self a N/A N/A N/A 
        Move all intruders according to 
        the :py:meth:`~rlpy.Domains.IntruderMonitoring.IntruderPolicy`, default
        uniform random action.
        Move all agents according to the selected action ``a``.
        Calculate the reward = Number of danger zones being violated by
        intruders while no agents are present (ie, intruder occupies a danger 
        cell with no agent simultaneously occupying the cell).
        
         N/A s N/A self state N/A agents N/A N/A np array N/A N/A N/A s N/A N/A N/A self NUMBER_OF_AGENTS 2 reshape -1 2 N/A actions N/A id2vec a N/A self ACTION_LIMITS N/A actions N/A N/A self ACTIONS_PER_AGENT N/A actions N/A agents actions N/A intruders N/A N/A np array N/A N/A N/A s N/A N/A N/A self NUMBER_OF_AGENTS 2 reshape -1 2 N/A actions N/A N/A N/A self IntruderPolicy N/A intruders N/A i N/A i N/A xrange N/A self NUMBER_OF_INTRUDERS N/A actions N/A N/A self ACTIONS_PER_AGENT N/A actions N/A intruders actions N/A ns N/A N/A np hstack N/A N/A N/A agents ravel N/A N/A intruders ravel N/A ns N/A bound_vec ns N/A self discrete_statespace_limits N/A agents N/A N/A N/A ns N/A N/A N/A self NUMBER_OF_AGENTS 2 reshape -1 2 N/A intruders N/A N/A N/A ns N/A N/A N/A self NUMBER_OF_AGENTS 2 reshape -1 2 N/A map N/A N/A np zeros N/A N/A self ROWS N/A self COLS bool N/A N/A map N/A N/A N/A intruders N/A N/A N/A 0 N/A intruders N/A N/A N/A 1 True N/A N/A map N/A N/A N/A agents N/A N/A N/A 0 N/A agents N/A N/A N/A 1 False N/A intrusion_counter N/A N/A np count_nonzero N/A map N/A N/A N/A N/A self danger_zone_locations N/A N/A N/A 0 N/A N/A self danger_zone_locations N/A N/A N/A 1 N/A r N/A intrusion_counter N/A self INTRUSION_PENALTY N/A ns N/A bound_vec ns N/A self discrete_statespace_limits N/A N/A self state N/A N/A ns copy N/A N/A r ns False N/A N/A self possibleActions N/A s0 N/A N/A self N/A N/A N/A N/A self state N/A N/A np hstack N/A N/A N/A N/A self agents_initial_locations ravel N/A N/A N/A self intruders_initial_locations ravel N/A N/A N/A N/A N/A self state copy N/A N/A self isTerminal N/A N/A self possibleActions N/A possibleActionsPerAgent N/A N/A self s_i N/A N/A N/A 
        Returns all possible actions for a single (2-D) agent state *s_i* 
        (where the domain state s = [s_0, ... s_i ... s_NUMBER_OF_AGENTS])
        
            1. tile the [R,C] for all actions
            2. add all actions to the results
            3. Find feasible rows and add them as possible actions
            
         N/A tile_s N/A N/A np tile s_i N/A N/A len N/A self ACTIONS_PER_AGENT 1 N/A next_states N/A tile_s N/A self ACTIONS_PER_AGENT N/A next_states_rows N/A next_states N/A N/A N/A 0 N/A next_states_cols N/A next_states N/A N/A N/A 1 N/A possibleActions1 N/A N/A np logical_and N/A 0 next_states_rows N/A next_states_rows N/A self ROWS N/A possibleActions2 N/A N/A np logical_and N/A 0 next_states_cols N/A next_states_cols N/A self COLS N/A N/A possibleActions _ N/A N/A N/A N/A N/A N/A np logical_and possibleActions1 possibleActions2 reshape -1 1 nonzero N/A possibleActions N/A printDomain N/A N/A self s a N/A N/A N/A -------------- N/A i N/A xrange 0 N/A self NUMBER_OF_AGENTS N/A N/A s_a N/A s N/A N/A i 2 N/A N/A i 2 2 N/A aa N/A id2vec a N/A self ACTION_LIMITS N/A N/A N/A Agent {} Location: {} Action {} format i s_a aa N/A offset N/A 2 N/A self NUMBER_OF_AGENTS N/A i N/A xrange 0 N/A self NUMBER_OF_INTRUDERS N/A N/A s_i N/A s N/A N/A offset N/A i 2 N/A N/A offset N/A i 2 2 N/A Intruder s_i N/A N/A r ns terminal N/A N/A self step s a N/A Reward  r N/A IntruderPolicy N/A N/A self s_i N/A N/A N/A 
        :param s_i: The state of a single agent
            (where the domain state s = [s_0, ... s_i ... s_NUMBER_OF_AGENTS]).
        :returns: a valid actions for the agent in state **s_i** to take.
        
        Default random action among possible.
        
         N/A N/A N/A N/A self random_state choice N/A N/A self possibleActionsPerAgent s_i N/A showDomain N/A N/A self a N/A N/A N/A s N/A self state N/A N/A N/A self domain_fig None N/A N/A N/A self domain_fig N/A N/A plt imshow N/A self map cmap IntruderMonitoring interpolation nearest vmin 0 vmax 3 N/A N/A N/A plt xticks N/A N/A np arange N/A self COLS fontsize FONTSIZE N/A N/A N/A plt yticks N/A N/A np arange N/A self ROWS fontsize FONTSIZE N/A N/A N/A plt show N/A N/A N/A self ally_fig None N/A N/A N/A N/A N/A N/A N/A self ally_fig pop 0 remove N/A N/A N/A N/A N/A N/A self intruder_fig pop 0 remove N/A s_ally N/A N/A N/A s N/A 0 N/A N/A self NUMBER_OF_AGENTS 2 reshape N/A -1 2 N/A s_intruder N/A N/A N/A s N/A N/A N/A self NUMBER_OF_AGENTS 2 reshape N/A -1 2 N/A N/A self ally_fig N/A N/A plt plot N/A s_ally N/A N/A N/A 1 N/A s_ally N/A N/A N/A 0 bo markersize 30.0 alpha 0.7 markeredgecolor k markeredgewidth 2 N/A N/A self intruder_fig N/A N/A plt plot N/A s_intruder N/A N/A N/A 1 N/A s_intruder N/A N/A N/A 0 g> color gray markersize 30.0 alpha 0.7 markeredgecolor k markeredgewidth 2 N/A N/A N/A plt draw N/A N/A