Rejoinder: Efficiency and Structure in MNIR

project to analyze the Congressional Record from 1873 to the present. With latent sentiment encoded as a random variable v, we can try to measure how it changes over time, such as for a particular lawmaker or author. Specifically, we can embed vt,d in a state-space model (West and Harrison 1997) and model the observed sentiment yt,d as a noisy realization. We can also consider how the relationship between language and sentiment changes, placing each term’s coefficient φw in a similar state-space model. While these innovations lead to more complex inference problems, placing the coefficients and sentiment variables in a time series could capture interesting changes in large collections of texts. • Richer structure with multilevel models. The intercept term α models the language that is unrelated to sentiment. In this article α is a simple unigram model, where there is one value for each term in the vocabulary. With additional structure, we might better tease apart the sentiment words from the background language and rely less on a careful selection of the vocabulary. For example, consider modeling items and their ratings with MIR (such as Amazon reviews). If an item receives universally positive reviews then MIR will attribute words that describe it, such as “DiCaprio,” as positive-sentiment words. This is a deficiency since we would not consider “DiCaprio” to be a word that indicates sentiment. (That said, if our only goal was prediction this might be a perfectly good inference.) Per-item intercepts, shared by all the reviews of a single item, would solve this problem. They can capture that “DiCaprio” is specific to particular movies but not to general sentiment. In MIR, this can be captured by developing the random effect variables uij . In this case, they would be hierarchical random effects (Gelman and Hill 2007) that are conditioned on attributes of the reviewed item. In his article, Taddy does not fully exploit the uij variables as his example problems did not require them. (Further, a model that includes more structured random effects variables will not enjoy the speed-ups that Taddy obtains by collapsing document counts.) With more complicated collections, richer random effects could be important to untangling the text into its sentiment and nonsentiment components. • Unsupervised sentiment analysis. With the view of MIR as a complete hierarchical model, it would be interesting to consider unsupervised sentiment analysis. Given a collection of unlabeled texts, can we identify a dimension of “sentiment” that these texts are expressing? Initially, this might sound far-fetched. But consider the success story of ideal point models in political methodology research (Clinton, Jackman, and Rivers 2004). Based only on a matrix of votes, we can recover the known political spectrum on which the lawmakers lie. In principle, though the data are significantly sparser and have higher dimension, we can do the same for language. Given a large enough collection of texts—again consider Amazon reviews or political speeches—we should be able to distinguish the prominent patterns in descriptive prose from those that express a spectrum of sentiment.