  Some ultra-compact dwarf galaxies (UCDs) have elevated observed dynamical
V-band mass-to-light ($M/L_{\rm V}$) ratios with respect to what is expected
from their stellar populations assuming a canonical initial mass function
(IMF). Observations have also revealed the presence of a compact dark object in
the centers of several UCDs, having a mass of a few to 15\% of the present-day
stellar mass of the UCD. This central mass concentration has typically been
interpreted as a super-massive black hole, but can in principle also be a
sub-cluster of stellar remnants. We explore the following two formation
scenarios of UCDs, i) monolithic collapse and ii) mergers of star clusters in
cluster complexes as are observed in massively star-bursting regions. We
explore the physical properties of the UCDs at different evolutionary stages
assuming different initial stellar masses of the UCDs and the IMF being either
universal or changing systematically with metallicity and density according to
the Integrated Galactic IMF (IGIMF) theory. While the observed elevated
$M/L_{\rm V}$ ratios of the UCDs cannot be reproduced if the IMF is invariant
and universal, the empirically derived IMF which varies systematically with
density and metallicity shows agreement with the observations. Incorporating
the UCD-mass-dependent retention fraction of dark remnants improves this
agreement. In addition we apply the results of N-body simulations to young UCDs
and show that the same initial conditions describing the observed $M/L_{\rm V}$
ratios reproduce the observed relation between the half-mass radii and the
present-day masses of the UCDs. The findings thus suggest that the majority of
UCDs that have elevated $M/L_{\rm V}$ ratios could have formed monolithically
with significant remnant-mass components that are centrally concentrated, while
those with small $M/L_V$ values may be merged star-cluster complexes.
