  In this paper, a novel training paradigm inspired by quantum computation is
proposed for deep reinforcement learning (DRL) with experience replay. In
contrast to traditional experience replay mechanism in DRL, the proposed deep
reinforcement learning with quantum-inspired experience replay (DRL-QER)
adaptively chooses experiences from the replay buffer according to the
complexity and the replayed times of each experience (also called transition),
to achieve a balance between exploration and exploitation. In DRL-QER,
transitions are first formulated in quantum representations, and then the
preparation operation and the depreciation operation are performed on the
transitions. In this progress, the preparation operation reflects the
relationship between the temporal difference errors (TD-errors) and the
importance of the experiences, while the depreciation operation is taken into
account to ensure the diversity of the transitions. The experimental results on
Atari 2600 games show that DRL-QER outperforms state-of-the-art algorithms such
as DRL-PER and DCRL on most of these games with improved training efficiency,
and is also applicable to such memory-based DRL approaches as double network
and dueling network.
