  We study a linear high-dimensional regression model in a semi-supervised
setting, where for many observations only the vector of covariates $X$ is given
with no response $Y$. We do not make any sparsity assumptions on the vector of
coefficients, and aim at estimating Var$(Y|X)$. We propose an estimator, which
is unbiased, consistent, and asymptotically normal. This estimator can be
improved by adding zero-estimators arising from the unlabelled data. Adding
zero-estimators does not affect the bias and potentially can reduce variance.
  In order to achieve optimal improvement, many zero-estimators should be used,
but this raises the problem of estimating many parameters. Therefore, we
introduce covariate selection algorithms that identify which zero-estimators
should be used in order to improve the above estimator.
  We further illustrate our approach for other estimators, and present an
algorithm that improves estimation for any given variance estimator. Our
theoretical results are demonstrated in a simulation study.
