  This paper focuses on a class of reinforcement learning problems where
significant events are rare and limited to a single positive reward per
episode. A typical example is that of an agent who has to choose a partner to
cooperate with, while a large number of partners are simply not interested in
cooperating, regardless of what the agent has to offer. We address this problem
in a continuous state and action space with two different kinds of search
methods: a gradient policy search method and a direct policy search method
using an evolution strategy. We show that when significant events are rare,
gradient information is also scarce, making it difficult for policy gradient
search methods to find an optimal policy, with or without a deep neural
architecture. On the other hand, we show that direct policy search methods are
invariant to the rarity of significant events, which is yet another
confirmation of the unique role evolutionary algorithms has to play as a
reinforcement learning method.
