Module Expr Str ImportFrom alias alias Import alias identifier Import alias Assign NameStore Str Assign NameStore ListLoad Str Str Str Str Str Assign NameStore Str Assign NameStore Str ClassDef bases NameLoad body Expr Str Assign NameStore NameLoad Assign NameStore NameLoad Assign NameStore NameLoad Assign NameStore NameLoad Assign NameStore ListLoad Assign NameStore NameLoad Assign NameStore Num Assign NameStore NameLoad FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults Num kwarg body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr BinOpAdd Str AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Pass decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults body Expr Str Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Str AugAssignAdd AttributeStore NameLoad attr Num If Call NameLoad NameLoad Str body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body Expr Str Assign NameStore Num Assign NameStore Num Assign NameStore ListLoad Assign NameStore NameLoad Assign NameStore Num Assign NameStore Str Assign NameStore ListLoad Str Str Str Str Assign NameStore Num FunctionDef arguments args NameParam NameParam NameParam NameParam defaults Num Str Num kwarg body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad If CompareEq AttributeLoad NameLoad attr Str body Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Num Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults body Expr Str If CompareEq AttributeLoad NameLoad attr Str body If UnaryOpNot NameLoad body Assign NameStore Call AttributeLoad NameLoad attr BinOpSub BinOpMult NameLoad NameLoad NameLoad NameLoad If CompareLt NameLoad Num body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr BinOpDiv Num NameLoad orelse If CompareEq AttributeLoad NameLoad attr Str body Assign AttributeStore NameLoad attr BinOpDiv BinOpMult AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Num BinOpAdd AttributeLoad NameLoad attr BinOpPow BinOpAdd AttributeLoad NameLoad attr Num Num AugAssignDiv AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq AttributeLoad NameLoad attr Str body Assign AttributeStore NameLoad attr BinOpDiv BinOpMult AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Num BinOpAdd AttributeLoad NameLoad attr BinOpPow BinOpAdd AttributeLoad NameLoad attr Num Num orelse If CompareEq AttributeLoad NameLoad attr Str body Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr orelse Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str decorator_list FunctionDef arguments args NameParam defaults body Expr Str AugAssignAdd AttributeStore NameLoad attr Num Expr Call AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr decorator_list decorator_list
N/A N/A Standard Control Agent.  abc ABCMeta abstractmethod N/A numpy np N/A logging N/A __copyright__ Copyright 2013, RLPy http://acl.mit.edu/RLPy N/A __credits__ N/A Alborz Geramifard Robert H. Klein Christoph Dann William Dabney Jonathan P. How N/A __license__ BSD 3-Clause N/A __author__ Alborz Geramifard Agent N/A object N/A N/A Learning Agent for obtaining good policices.

    The Agent receives observations from the Domain and incorporates their
    new information into the representation, policy, etc. as needed.

    In a typical Experiment, the Agent interacts with the Domain in discrete
    timesteps.
    At each Experiment timestep the Agent receives some observations from the Domain
    which it uses to update the value function Representation of the Domain
    (ie, on each call to its :py:meth:`~rlpy.Agents.Agent.Agent.learn` function).
    The Policy is used to select an action to perform.
    This process (observe, update, act) repeats until some goal or fail state,
    determined by the Domain, is reached. At this point the
    :py:class:`~rlpy.Experiments.Experiment.Experiment` determines
    whether the agent starts over or has its current policy tested
    (without any exploration).

    :py:class:`~rlpy.Agents.Agent.Agent` is a base class that provides the basic
    framework for all RL Agents. It provides the methods and attributes that
    allow child classes to interact with the
    :py:class:`~rlpy.Domains.Domain.Domain`,
    :py:class:`~rlpy.Representations.Representation.Representation`,
    :py:class:`~rlpy.Policies.Policy.Policy`, and
    :py:class:`~rlpy.Experiments.Experiment.Experiment` classes within the
    RLPy library.

    .. note::
        All new agent implementations should inherit from this class.

     N/A __metaclass__ ABCMeta N/A representation None N/A discount_factor None N/A policy None N/A eligibility_trace N/A N/A logger None N/A episode_count 0 N/A random_state None __init__ N/A N/A self policy representation discount_factor seed N/A 1 kwargs N/A N/A initialization.

        :param representation: the :py:class:`~rlpy.Representation.Representation.Representation`
            to use in learning the value function.
        :param policy: the :py:class:`~rlpy.Policies.Policy.Policy` to use when selecting actions.
        :param discount_factor: the discount factor of the optimal policy which should be
            learned
        :param initial_learn_rate: Initial learning rate to use (where applicable)

        .. warning::
            ``initial_learn_rate`` should be set to 1 for automatic learning rate;
            otherwise, initial_learn_rate will act as a permanent upper-bound on learn_rate.

        :param learn_rate_decay_mode: The learning rate decay mode (where applicable)
        :param boyan_N0: Initial Boyan rate parameter (when learn_rate_decay_mode='boyan')

         N/A N/A self representation representation N/A N/A self policy policy N/A N/A self discount_factor discount_factor N/A N/A self logger N/A N/A logging getLogger N/A rlpy.Agents. N/A N/A self __class__ __name__ N/A N/A self random_state N/A N/A N/A np random RandomState seed seed N/A init_randomization N/A N/A self N/A N/A N/A 
        Any stochastic behavior in __init__() is broken out into this function
        so that if the random seed is later changed (eg, by the Experiment),
        other member variables and functions are updated accordingly.

         N/A N/A learn N/A N/A self s p_actions a r ns np_actions na terminal N/A N/A N/A 
        This function receives observations of a single transition and
        learns from it.

        .. note::
            Each inheriting class (Agent) must implement this method.

        :param s: original state
        :param p_actions: possible actions in the original state
        :param a: action taken
        :param r: obtained reward
        :param ns: next state
        :param np_actions: possible actions in the next state
        :param na: action taken in the next state
        :param terminal: boolean indicating whether next state (ns) is terminal
         N/A NotImplementedError N/A abstractmethod episodeTerminated N/A N/A self N/A N/A N/A 
        This function adjusts all necessary elements of the agent at the end of
        the episodes.

        .. note::
            Every agent must call this function at the end of the learning if the
            transition led to terminal state.

         N/A N/A self episode_count 1 N/A N/A hasattr self eligibility_trace N/A N/A N/A self eligibility_trace N/A N/A np zeros_like N/A self eligibility_trace N/A N/A DescentAlgorithm N/A object N/A N/A 
    Abstract base class that contains step-size control methods for (stochastic)
    descent algorithms such as TD Learning, Greedy-GQ etc.
     N/A initial_learn_rate 0.1 N/A learn_rate 0 N/A eligibility_trace N/A N/A logger None N/A episode_count 0 N/A learn_rate_decay_mode dabney N/A valid_decay_modes N/A dabney boyan const boyan_const N/A boyan_N0 1000 __init__ N/A N/A self initial_learn_rate learn_rate_decay_mode boyan_N0 N/A 0.1 dabney 1000 kwargs N/A N/A 
        :param initial_learn_rate: Initial learning rate to use (where applicable)

        .. warning::
            ``initial_learn_rate`` should be set to 1 for automatic learning rate;
            otherwise, initial_learn_rate will act as a permanent upper-bound on learn_rate.

        :param learn_rate_decay_mode: The learning rate decay mode (where applicable)
        :param boyan_N0: Initial Boyan rate parameter (when learn_rate_decay_mode='boyan')

         N/A N/A self initial_learn_rate initial_learn_rate N/A N/A self learn_rate initial_learn_rate N/A N/A self learn_rate_decay_mode N/A N/A learn_rate_decay_mode lower N/A N/A self boyan_N0 boyan_N0 N/A N/A N/A self learn_rate_decay_mode dabney N/A N/A N/A self initial_learn_rate 1.0 N/A N/A self learn_rate 1.0 N/A N/A N/A N/A super DescentAlgorithm self __init__ kwargs N/A updateLearnRate N/A N/A self phi phi_prime eligibility_trace discount_factor nnz terminal N/A N/A N/A Computes a new learning rate (learn_rate) for the agent based on
        ``self.learn_rate_decay_mode``.

        :param phi: The feature vector evaluated at state (s) and action (a)
        :param phi_prime_: The feature vector evaluated at the new state (ns) = (s') and action (na)
        :param eligibility_trace: Eligibility trace
        :param discount_factor: The discount factor for learning (gamma)
        :param nnz: The number of nonzero features
        :param terminal: Boolean that determines if the step is terminal or not

         N/A N/A N/A self learn_rate_decay_mode dabney N/A N/A N/A terminal N/A N/A candid_learn_rate N/A N/A np dot N/A N/A discount_factor phi_prime phi eligibility_trace N/A N/A candid_learn_rate 0 N/A N/A N/A self learn_rate N/A N/A np minimum N/A self learn_rate N/A -1.0 candid_learn_rate N/A N/A N/A N/A self learn_rate_decay_mode boyan N/A N/A N/A self learn_rate N/A N/A N/A self initial_learn_rate N/A N/A self boyan_N0 1.0 N/A N/A self boyan_N0 N/A N/A N/A self episode_count 1 1.1 N/A N/A self learn_rate N/A N/A np sum N/A N/A np abs phi N/A N/A N/A N/A self learn_rate_decay_mode boyan_const N/A N/A N/A self learn_rate N/A N/A N/A self initial_learn_rate N/A N/A self boyan_N0 1.0 N/A N/A self boyan_N0 N/A N/A N/A self episode_count 1 1.1 N/A N/A N/A N/A self learn_rate_decay_mode const N/A N/A N/A self learn_rate N/A self initial_learn_rate N/A N/A N/A N/A N/A self logger warn Unrecognized decay mode  N/A episodeTerminated N/A N/A self N/A N/A N/A 
        This function adjusts all necessary elements of the agent at the end of
        the episodes.

        .. note::
            Every Agent must call this function at the end of the learning if the
            transition led to terminal state.

         N/A N/A self episode_count 1 N/A N/A N/A N/A self representation episodeTerminated N/A N/A N/A N/A super DescentAlgorithm self episodeTerminated N/A N/A