SlowGuess/ACL_2020_OCR huggingface dataset:
To train the TBIP, we perform stochastic gradient ascent using Adam (Kingma and Ba, 2015), with a mini-batch size of 512. To curtail extreme word
count values from long speeches, we take the natural logarithm of the counts matrix before performing inference (appropriately adding 1 and rounding so that a word count of 1 is transformed to still be 1). We use a single Monte Carlo sample to approximate the gradient of each batch. We assume 50 latent topics and posit the following prior distributions:  $\theta_{dk},\beta_{kv}\sim \mathrm{Gamma}(0.3,0.3),\eta_{kv},x_s\sim \mathcal{N}(0,1)$
We train the vote ideal point model by removing all votes that are not cast as "yea" or "nay" and performing mean-field variational inference with Gaussian variational distributions. Since each variational family is Gaussian, we approximate gradients using the reparameterization trick (Rezende et al., 2014; Kingma and Ba, 2015).
For the comparisons against WORDFISH and WORDSHOAL, we preprocess speeches in the same way as Lauderdale and Herzog (2016a). We train each Senate session separately, thereby only including one timestep for WORDFISH. For this reason, our results on the U.S. Senate differ from those reported by Lauderdale and Herzog (2016a), who train a model jointly over all time periods. Additionally, we use variational inference with reparam
eterization gradients to train all methods. Specifically, we perform mean-field variational inference, positing Gaussian variational families on all real variables and lognormal variational families on all positive variables.
Senator tweets Our Senate tweet preprocessing is similar to the Senate speech preprocessing, although we now include all terms that appear in at least  $0.05\%$  of documents rather than  $0.01\%$  to account for the shorter tweet lengths. We remove cities and states in addition to stopwords and the names of politicians. This preprocessing leaves us with 209,779 tweets. We use the same model and hyperparameters as for speeches, although we no longer take the natural logarithm of the counts matrix since individual tweets cannot have extreme word counts due to the character limit. We use a batch size of 1,024.
2020 Democratic candidates We scrape the Twitter feeds of 19 candidates, including all tweets between January 1, 2019 and February 27, 2020. We do not include Andrew Yang, Jay Inslee, and Marianne Williamson since it is difficult to define
Speeches 111	Speeches 112	Speeches 113	Tweets 114
Corr.	SRC	Corr.	SRC	Corr.	SRC	Corr.	SRC
WORDFISH	0.52	0.49	0.51	0.51	0.71	0.65	0.79	0.74
WORDSHOAL	0.62	0.66	0.58	0.51	0.46	0.46	-	-
TBIP	0.82	0.77	0.85	0.85	0.89	0.86	0.94	0.88
Table 4. The TBIP learns ideal points most similar to DW-NOMINATE vote ideal points for U.S. senator speeches and tweets. It learns closer ideal points than WORDFISH and WORDSHOAL in terms of both correlation (Corr.) and Spearman's rank correlation (SRC). The numbers in the column titles refer to the Senate session of the corpus. WORDSHOAL cannot be applied to tweets because there are no debate labels.
the political preferences of non-traditional or single-issue candidates. We follow the same preprocessing we used for the 114th Senate, except we include tokens that are used in more than  $0.05\%$  of documents rather than  $0.1\%$ . We remove phrases used by only one candidate, along with stopwords and candidate names. This preprocessing leaves us with 45,927 tweets for the 19 candidates. We use the same model and hyperparameters as for senator tweets.
C Comparison to DW-Nominate
DW-NOMINATE (Poole, 2005) is a dynamic method for learning ideal points from votes. As opposed to the vote ideal point model in Equation (1), it analyzes votes across multiple Senate sessions. It also learns two latent dimensions per legislator. We also compare text ideal points to the first dimension of DW-Nominate, which corresponds to economic/ redistributive preferences (Lewis et al., 2020). We use the fitted DW-NOMINATE ideal points available on Voteview (Lewis et al., 2020). The TBIP learns ideal points closer to DW-NOMINATE than WORDFISH and WORDSHOAL; see Table 4.
In Section 5, we observed that Bernie Sanders' vote ideal point is somewhat moderate under the scalar ideal point model from Equation (1). It is worth noting that Sanders' vote ideal point is more extreme under DW-NOMINATE than under the scalar model: his DW-NOMINATE ideal point is the third-most extreme among Democrats. Since DW-NOMINATE uses two dimensions to model each legislator's latent preferences, it can more flexibly model Sanders' voting deviations. Additionally, the dynamic nature of DW-NOMINATE may capture salient information from other Senate sessions. However, restricting the vote ideal point to be static and a scalar, like it is for the TBIP, results in the more moderate vote ideal point in Section 5.