  Utilising optical and near-infrared broadband photometry covering $> 5\,{\rm
deg}^2$ in two of the most well-studied extragalactic legacy fields (COSMOS and
XMM-LSS), we measure the galaxy stellar mass function (GSMF) between $0.1 < z <
2.0$. We explore in detail the effect of two source extraction methods
(SExtractor and ProFound) in addition to the inclusion/exclusion of Spitzer
IRAC 3.6 and 4.5$\mu$m photometry when measuring the GSMF. We find that
including IRAC data reduces the number of massive ($\log_{10}(M/M_\odot) >
11.25$) galaxies found due to improved photometric redshift accuracy, but has
little effect on the more numerous lower-mass galaxies. We fit the resultant
GSMFs with double Schechter functions down to $\log_{10}(M/M_\odot)$ = 7.75
(9.75) at z = 0.1 (2.0) and find that the choice of source extraction software
has no significant effect on the derived best-fit parameters. However, the
choice of methodology used to correct for the Eddington bias has a larger
impact on the high-mass end of the GSMF, which can partly explain the spread in
derived $M^*$ values from previous studies. Using an empirical correction to
model the intrinsic GSMF, we find evidence for an evolving characteristic
stellar mass with $\delta \log_{10}(M^*/M_\odot)/\delta z$ = $-0.16\pm0.05 \,
(-0.11\pm0.05)$, when using SExtractor (ProFound). We argue that with widely
quenched star formation rates in massive galaxies at low redshift ($z<0.5$),
additional growth via mergers is required in order to sustain such an evolution
to a higher characteristic mass.
