  In this paper, we study the problem of deceptive reinforcement learning to
preserve the privacy of a reward function. Reinforcement learning is the
problem of finding a behaviour policy based on rewards received from
exploratory behaviour. A key ingredient in reinforcement learning is a reward
function, which determines how much reward (negative or positive) is given and
when. However, in some situations, we may want to keep a reward function
private; that is, to make it difficult for an observer to determine the reward
function used. We define the problem of privacy-preserving reinforcement
learning, and present two models for solving it. These models are based on
dissimulation -- a form of deception that `hides the truth'. We evaluate our
models both computationally and via human behavioural experiments. Results show
that the resulting policies are indeed deceptive, and that participants can
determine the true reward function less reliably than that of an honest agent.
