  We consider topologically twisted $\mathcal{N}=2$, $SU(2)$ gauge theory with
a massive adjoint hypermultiplet on a smooth, compact four-manifold $X$. A
consistent formulation requires coupling the theory to a ${\rm Spin}^c$
structure, which is necessarily non-trivial if $X$ is non-spin. We derive
explicit formulae for the topological correlation functions when $b_2^+\geq 1$.
We demonstrate that, when the ${\rm Spin}^c$ structure is canonically
determined by an almost complex structure and the mass is taken to zero, the
path integral reproduces known results for the path integral of the
$\mathcal{N}=4$ gauge theory with Vafa-Witten twist. On the other hand, we
reproduce results from Donaldson-Witten theory after taking a suitable infinite
mass limit. The topological correlators are functions of the UV coupling
constant $\tau_{\rm uv}$ and we confirm that they obey the expected $S$-duality
transformation laws. The holomorphic part of the partition function is a
generating function for the Euler numbers of the matter (or obstruction) bundle
over the instanton moduli space. For $b_2^+=1$, we derive a non-holomorphic
contribution to the path integral, such that the partition function and
correlation functions are mock modular forms rather than modular forms. We
comment on the generalization of this work to the large class of
$\mathcal{N}=2$ theories of class $S$.
