  We study high-dimensional Bayesian linear regression with product priors.
Using the nascent theory of non-linear large deviations (Chatterjee and
Dembo,2016), we derive sufficient conditions for the leading-order correctness
of the naive mean-field approximation to the log-normalizing constant of the
posterior distribution. Subsequently, assuming a true linear model for the
observed data, we derive a limiting infinite dimensional variational formula
for the log normalizing constant of the posterior. Furthermore, we establish
that under an additional "separation" condition, the variational problem has a
unique optimizer, and this optimizer governs the probabilistic properties of
the posterior distribution. We provide intuitive sufficient conditions for the
validity of this "separation" condition. Finally, we illustrate our results on
concrete examples with specific design matrices.
