  The precision anticipated from next-generation cosmic microwave background
(CMB) surveys will create opportunities for characteristically new insights
into cosmology. Secondary anisotropies of the CMB will have an increased
importance in forthcoming surveys, due both to the cosmological information
they encode and the role they play in obscuring our view of the primary
fluctuations. Quadratic estimators have become the standard tools for
reconstructing the fields that distort the primary CMB and produce secondary
anisotropies. While successful for lensing reconstruction with current data,
quadratic estimators will be sub-optimal for the reconstruction of lensing and
other effects at the expected sensitivity of the upcoming CMB surveys. In this
paper we describe a convolutional neural network, ResUNet-CMB, that is capable
of the simultaneous reconstruction of two sources of secondary CMB
anisotropies, gravitational lensing and patchy reionization. We show that the
ResUNet-CMB network significantly outperforms the quadratic estimator at low
noise levels and is not subject to the lensing-induced bias on the patchy
reionization reconstruction that would be present with a straightforward
application of the quadratic estimator.
