Models with a large number of latent variables are often used to fully utilize the information in big or complex data. However, they can be difficult to estimate using standard approaches, and variational inference methods are a popular alternative. Key to the success of these is the selection of an approximation to the target density that is accurate, tractable and fast to calibrate using optimization methods. Mean field or structured Gaussian approximations are common, but these can be inaccurate and slow to calibrate when there are many latent variables. Instead, we propose a family of tractable variational approximations that are more accurate and faster to calibrate for this case. The approximation is a parsimonious copula model for the parameter posterior, combined with the exact conditional posterior of the latent variables. We derive a simplified expression for the re-parameterization gradient of the variational lower bound, which is the main ingredient of efficient optimization algorithms used to implement variational estimation.  We illustrate using two substantive econometric examples. The first is a nonlinear  state space model for U.S. inflation . The second is a random coefficients tobit model applied to a rich marketing dataset with one million sales observations from a panel of 10 ,000 individuals  . In both cases, we show that our approximating family is faster to calibrate than either mean field or structured Gaussian approximations, and that the gains in posterior estimation accuracy are considerable .
Models with a large number of latent variables are often used to fully utilize the information in big or complex data. However, they can be difficult to estimate using standard approaches, and variational inference methods are a popular alternative. Key to the success of these is the selection of an approximation to the target density that is accurate, tractable and fast to calibrate using optimization methods. Most existing choices can be inaccurate or slow to calibrate when there are many latent variables. Instead, we propose a family of tractable variational approximations that are more accurate and faster to calibrate for this case. It combines a parsimonious parametric approximation for the parameter posterior, with the exact conditional posterior of the latent variables. We derive a simplified expression for the re-parameterization gradient of the variational lower bound, which is the main ingredient of efficient optimization algorithms used to implement variational estimation. To do so only requires the ability to generate exactly or approximately from the conditional posterior of the latent variables, rather than to compute its density. We illustrate using two complex contemporary econometric examples. The first is a nonlinear multivariate state space model for U.S. macroeconomic variables . The second is a random coefficients tobit model applied to two million sales by 20 ,000 individuals from a large marketing panel . In both cases, we show that our approximating family is considerably more accurate than mean field or structured Gaussian approximations, and faster than Markov chain Monte Carlo. Last, we show how to implement data sub-sampling in variational inference for our approximation, which can lead to a further reduction in computation time .
R
Mean field or structured Gaussian approximations are common, but these
Most existing choices
R
and
or
R
The approximation is a parsimonious copula model
It combines a parsimonious parametric approximation
D
combined
A
To do so only requires the ability to generate exactly or approximately from the conditional posterior of the latent variables, rather than to compute its density.
R
substantive
complex contemporary
A
multivariate
R
inflation
macroeconomic variables
R
a rich marketing dataset with one million sales observations from a panel of 10
two million sales by 20
A
from a large marketing panel
R
faster to calibrate than either
considerably more accurate than
R
that the gains in posterior estimation accuracy are considerable
faster than Markov chain Monte Carlo. Last, we show how to implement data sub-sampling in variational inference for our approximation, which can lead to a further reduction in computation time