  Researchers have used social media data to estimate various macroeconomic
indicators about public behaviors, mostly as a way to reduce surveying costs.
One of the most widely cited economic indicator is consumer confidence index
(CCI). Numerous studies in the past have focused on using social media,
especially Twitter data, to predict CCI. However, the strong correlations
disappeared when those models were tested with newer data according to a recent
comprehensive survey. In this work, we revisit this problem of assessing the
true potential of using social media data to measure CCI, by proposing a robust
non-parametric Bayesian modeling framework grounded in Gaussian Process
Regression (which provides both an estimate and an uncertainty associated with
it). Integral to our framework is a principled experimentation methodology that
demonstrates how digital data can be employed to reduce the frequency of
surveys, and thus periodic polling would be needed only to calibrate our model.
Via extensive experimentation we show how the choice of different
micro-decisions, such as the smoothing interval, various types of lags etc.
have an important bearing on the results. By using decadal data (2008-2019)
from Reddit, we show that both monthly and daily estimates of CCI can, indeed,
be reliably estimated at least several months in advance, and that our model
estimates are far superior to those generated by the existing methods.
