  In this paper, we investigate the optimal output tracking problem for linear
discrete-time systems with unknown dynamics using reinforcement learning and
robust output regulation theory. This output tracking problem only allows to
utilize the outputs of the reference system and the controlled system, rather
than their states, and differs from most existing tracking results that depend
on the state of the system. The optimal tracking problem is formulated into a
linear quadratic regulation problem by proposing a family of dynamic
discrete-time controllers. Then, it is shown that solving the output tracking
problem is equivalent to solving output regulation equations, whose solution,
however, requires the knowledge of the complete and accurate system dynamics.
To remove such a requirement, an off-policy reinforcement learning algorithm is
proposed using only the measured output data along the trajectories of the
system and the reference output. By introducing re-expression error and
analyzing the rank condition of the parameterization matrix, we ensure the
uniqueness of the proposed RL based optimal control via output feedback.
