  In this paper, we propose an abstract procedure for debiasing constrained or
regularized potentially high-dimensional linear models. It is elementary to
show that the proposed procedure can produce $\frac{1}{\sqrt{n}}$-confidence
intervals for individual coordinates (or even bounded contrasts) in models with
unknown covariance, provided that the covariance has bounded spectrum. While
the proof of the statistical guarantees of our procedure is simple, its
implementation requires more care due to the complexity of the optimization
programs we need to solve. We spend the bulk of this paper giving examples in
which the proposed algorithm can be implemented in practice. One fairly general
class of instances which are amenable to applications of our procedure include
convex constrained least squares. We are able to translate the procedure to an
abstract algorithm over this class of models, and we give concrete examples
where efficient polynomial time methods for debiasing exist. Those include the
constrained version of LASSO, regression under monotone constraints, regression
with positive monotone constraints and non-negative least squares. In addition,
we show that our abstract procedure can be applied to efficiently debias SLOPE
and square-root SLOPE, among other popular regularized procedures under certain
assumptions. We provide thorough simulation results in support of our
theoretical findings.
