  Variational quantum eigensolvers have recently received increased attention,
as they enable the use of quantum computing devices to find solutions to
complex problems, such as the ground energy and ground state of
strongly-correlated quantum many-body systems. In many applications, it is the
optimization of both continuous and discrete parameters that poses a formidable
challenge. Using reinforcement learning (RL), we present a hybrid policy
gradient algorithm capable of simultaneously optimizing continuous and discrete
degrees of freedom in an uncertainty-resilient way. The hybrid policy is
modeled by a deep autoregressive neural network to capture causality. We employ
the algorithm to prepare the ground state of the nonintegrable quantum Ising
model in a unitary process, parametrized by a generalized quantum approximate
optimization ansatz: the RL agent solves the discrete combinatorial problem of
constructing the optimal sequences of unitaries out of a predefined set and, at
the same time, it optimizes the continuous durations for which these unitaries
are applied. We demonstrate the noise-robust features of the agent by
considering three sources of uncertainty: classical and quantum measurement
noise, and errors in the control unitary durations. Our work exhibits the
beneficial synergy between reinforcement learning and quantum control.
