  Model bias is an inherent limitation of the current dominant approach to
optimal quantum control, which relies on a system simulation for optimization
of control policies. To overcome this limitation, we propose a circuit-based
approach for training a reinforcement learning agent on quantum control tasks
in a model-free way. Given a continuously parameterized control circuit, the
agent learns its parameters through trial-and-error interaction with the
quantum system, using measurements as the only source of information about the
quantum state. By focusing on the task of quantum state preparation in a
harmonic oscillator coupled to an ancilla qubit, we show how to reward the
learning agent using measurements of experimentally available observables. We
demonstrate by numerical simulations preparation of arbitrary states using both
open- and closed-loop control through adaptive quantum feedback. Our work is of
immediate relevance to superconducting circuits and trapped ions platforms
where such training can be implemented real-time in an experiment, allowing
complete elimination of model bias and the adaptation of quantum control
policies to the specific system in which they are deployed.
