  This paper presents a secure reinforcement learning (RL) based control method
for unknown linear time-invariant cyber-physical systems (CPSs) that are
subjected to compositional attacks such as eavesdropping and covert attack. We
consider the attack scenario where the attacker learns about the dynamic model
during the exploration phase of the learning conducted by the designer to learn
a linear quadratic regulator (LQR), and thereafter, use such information to
conduct a covert attack on the dynamic system, which we refer to as doubly
learning-based control and attack (DLCA) framework. We propose a dynamic
camouflaging based attack-resilient reinforcement learning (ARRL) algorithm
which can learn the desired optimal controller for the dynamic system, and at
the same time, can inject sufficient misinformation in the estimation of system
dynamics by the attacker. The algorithm is accompanied by theoretical
guarantees and extensive numerical experiments on a consensus multi-agent
system and on a benchmark power grid model.
