  In online advertising, recommender systems try to propose items from a list
of products to potential customers according to their interests. Such systems
have been increasingly deployed in E-commerce due to the rapid growth of
information technology and availability of large datasets. The ever-increasing
progress in the field of artificial intelligence has provided powerful tools
for dealing with such real-life problems. Deep reinforcement learning (RL) that
deploys deep neural networks as universal function approximators can be viewed
as a valid approach for design and implementation of recommender systems. This
paper provides a comparative study between value-based and policy-based deep RL
algorithms for designing recommender systems for online advertising. The
RecoGym environment is adopted for training these RL-based recommender systems,
where the long short term memory (LSTM) is deployed to build value and policy
networks in these two approaches, respectively. LSTM is used to take account of
the key role that order plays in the sequence of item observations by users.
The designed recommender systems aim at maximising the click-through rate (CTR)
for the recommended items. Finally, guidelines are provided for choosing proper
RL algorithms for different scenarios that the recommender system is expected
to handle.
