  We use VANDELS spectroscopic data overlapping with the $\simeq$7 Ms Chandra
Deep Field South survey to extend studies of high-mass X-ray binary systems
(XRBs) in 301 normal star-forming galaxies in the redshift range $3 < z < 5.5$.
Our analysis evaluates correlations between X-ray luminosities ($L_X$), star
formation rates (SFR) and stellar metallicities ($Z_\star$) to higher redshifts
and over a wider range in galaxy properties than hitherto. Using a stacking
analysis performed in bins of both redshift and SFR for sources with robust
spectroscopic redshifts without AGN signatures, we find convincing evolutionary
trends in the ratio $L_X$/SFR to the highest redshifts probed, with a stronger
trend for galaxies with lower SFRs. Combining our data with published samples
at lower redshift, the evolution of $L_X$/SFR to $z\simeq5$ proceeds as $(1 +
z)^{1.03 \pm 0.02}$. Using stellar metallicities derived from photospheric
absorption features in our spectroscopic data, we confirm indications at lower
redshifts that $L_X$/SFR is stronger for metal-poor galaxies. We use
semi-analytic models to show that metallicity dependence of $L_X$/SFR alone may
not be sufficient to fully explain the observed redshift evolution of X-ray
emission from high-mass XRBs, particularly for galaxies with SFR $<30$
$M_\odot$ yr$^{-1}$. We speculate that the discrepancy may arise due to reduced
overall stellar ages in the early Universe leading to higher $L_X$/SFR for the
same metallicity. We use our data to define the redshift-dependent contribution
of XRBs to the integrated X-ray luminosity density and, in comparison with
models, find that the contribution of high-mass XRBs to the cosmic X-ray
background at $z>6$ may be $\gtrsim 0.25$ dex higher than previously estimated.
