SlowGuess/NAACL_2021_OCR huggingface dataset:
The selected values of learning rate and dropout for all our models are shown in Table 11, rounded to two decimal places. We also note the remaining hyperparameters here: our RNN used a hidden dimension of 506, an embedding dimension of 137, and a 2-layer GRU. Additionally, Multi  $D_{r_{FSJ}}$  selected  $\lambda = 0.90$  and Multi  $D_{r_{FSJ}}$ ,  $\lambda = 0.67$ .
All of our models are similar in architecture and therefore take similar runtimes and have a similar number of parameters. Running our entire hyperpa
Table 10: Hyperparameter ranges for our models. BERT-based models tuned the first two; the Multi models additionally tuned  $\lambda$ ; the RNN additionally tuned the remainder.
Model	Learning Rate	Pdropout
RNN	1.40×10-4	0.86
BERT	4.27×10-5	0.13
MultiAltGEA	8.47×10-6	0.40
MultiAltGEE	1.08×10-5	0.00
MultiAltGES	1.69×10-5	0.00
MultiAltGEFSJ	8.98×10-6	0.00
MultiAltVent	4.44×10-5	0.00
MultiDrS	1.14×10-5	0.00
MultiDrFSJ	1.79×10-5	0.00
FTGEA→Dr	7.30×10-5	0.05
FTGEE→Dr	1.35×10-5	0.00
FTGES→Dr	1.95×10-5	0.09
FTGEFSJ→Dr	5.03×10-6	0.03
Table 11: Our models' selected hyperparameters. So that the table fits in a column, GE is GoEmotions, and Dr is Dreaddit.
Model	Binary F1	Accuracy
RNN	72.58 ± 0.50	74.15 ± 1.46
BERT	81.79 ± 0.45	82.97 ± 0.30
MultiAltGEA	81.31 ± 0.81	82.97 ± 0.51
MultiAltGEE	80.30 ± 0.85	82.25 ± 0.59
MultiAltGES	80.79 ± 1.31	82.00 ± 0.74
MultiAltGEFSJ	81.87 ± 2.21	82.61 ± 2.42
MultiAltVent	82.30 ± 1.16	82.49 ± 2.01
MultiDrS	81.40 ± 1.54	82.49 ± 1.20
MultiDrFSSJ	82.58 ± 1.11	83.21 ± 1.46
FTGEA→Dr	82.58 ± 1.53	82.13 ± 1.04
FTGEE→Dr	82.58 ± 1.53	83.57 ± 1.71
FTGES→Dr	80.87 ± 1.15	82.49 ± 0.68
FTGEFSJ→Dr	82.88 ± 0.92	84.54 ± 0.74
Table 12: Results of all our presented models on the Dreadlit development set. So that the table fits in a column, GE is GoEmotions, and Dr is Dreadlit.
rameter tuning setup described above takes about one day, and training one ensemble of three models takes about 25 minutes. BERT makes up the vast majority of our models' parameters, putting all of them at about 109B parameters (aside from the RNN, which has 7B).
Our performance for each model on the dev set is shown in Table 12.
We note the standard equations of our reported metrics. For our Dreaddit models, we report binary F1, i.e., the harmonic mean of precision and recall for the positive class (here, stress):  $2 \cdot \frac{\text{precisionrecall}}{\text{precision+recall}} = \frac{\text{TP}}{\text{TP} + \frac{1}{2}(\text{FP} + \text{FN})}$ , where TP represents the number of examples that were correctly classified as stress (true positives), FP those incorrectly classified as stress (false positives), TN those correctly classified as non-stress (true negatives), and FN those incorrectly classified as non-stress (false negatives). We also report classification accuracy, which is the fraction of samples classified correctly, i.e.,  $\frac{\text{TP} + \text{TN}}{\text{TP} + \text{FP} + \text{TN} + \text{FN}}$ .
We note that Table 4 reports macro-averaged F1 in the multi-label and single-label cases. For both of these, we calculate the macro-average F1, which sums up TP, TN, etc. across all emotion labels and then calculates F1 score. For a multi-label input, we treat each label as a separate classification—e.g., if the model is incorrect with respect to one class but correctly identifies a second, the example counts towards incorrect examples for the first class and then again towards correct examples for the second.
Our data is all English social media data. Dreaddit and GoEmotions are taken from Reddit, and Vent from the social media platform Vent. Dreaddit consists of 3,553 labeled segments of posts taken from 10 subreddits: r/domesticviolence, r/survivorsofabuce, r/anxiety, r/stress, r/almosthomeless, r/assistance, r/food_pantry, r/homeless, r/ptsd, and r/relationships.  $52.3\%$  of the data is labeled stress, with the remaining  $47.4\%$  labeled non-stress. We use the train-dev-test split of Turcan and McKeown (2019) into 2,562 train, 276 development, and 715 test examples. GoEmotions consists of 58,009 labeled Reddit comments taken from non-disclosed selection of subreddits
Demszky et al. (2020). We refer the reader to the original publication's Figure 1 for details on the label distribution; GoEmotions uses 28 labels with a widely varying amount of data for each. We use the label groupings that the authors provide in order to evaluate on the Ekman labels and sentiment labels; these schemes group several of the 28 original labels together into smaller sets.
Our Vent data consists of 1.6 million Vents gathered in collaboration with the Vent platform. A much larger amount of data was collected, but we select the data with self-labeled emotion tags related to joy, sadness, and fear. These data were collected from 2013 to 2016; in their current form, we do not retain metadata about the posters. A group with whom we collaborate has collected this data, and due to licensing and ethics requirements, we are not able to release it publicly. We selected sadness, fear, and happiness based on intuition that they should be relevant to stress; we partitioned off a label-stratified  $10\%$  of the data for development and test each and the training set is 1.3 million examples. The label distribution of this dataset is  $24.0\%$  fear,  $36.0\%$  sadness, and  $40.0\%$  happiness.
We do not filter or remove any examples. The only preprocessing we perform is to apply the pretrained bert-base-uncased tokenizer from Wolf et al. (2019).
We include the full tables of per-emotion correlations for Multi $_{GoEmotions_A}$  in Table 13 and Multi $_{GoEmotions_E}$  in Table 14. We note that the alternating multi-task models do not predict all of their possible emotions on Dreadlit, although all possible emotions do occur in the GoEmotions development set. We also report the correlation coefficients  $\eta$  and mean stress prediction for the Multi $_{Vent}$  model in Table 15.
The full table of LIWC/LIME explanation counts for every model is too large to fit comfortably on a page, so we make a spreadsheet available at www.cs.columbia.edu/~eturcan/data/emotion_infused_explanations.csv.
We include the top 200 relative salience unigrams (in order of relative salience) for stress and non-stress from the Dreaddit train set here. These are tokens as split by the Natural Language Toolkit (NLTK). We reproduce these exactly as they appear, and caution that they may include explicit or
neutral	anger	fear	annoyance	surprise
gold stress + pred. emotion	-0.3761	-	-	-	-
gold emotion + pred. stress	-0.0728	0.2175	0.1066	0.1420	0.0188
gratitude	desire	optimism	admiration	confusion
gold stress + pred. emotion	-	-	-	-	-
gold emotion + pred. stress	-0.0663	-0.0288	-0.0388	-0.1036	0.0471
amusement	approval	caring	embarrass.	realization
gold stress + pred. emotion	-	-	-	-	-
gold emotion + pred. stress	-0.0195	-0.0788	-0.0095	0.0304	-0.0073
disappoint.	grief	sadness	curiosity	joy
gold stress + pred. emotion	-0.1119	-	-	-0.2070	-0.0644
gold emotion + pred. stress	0.0465	0.0320	0.1075	0.0262	-0.0704
love	excitement	disapproval	remorse	disgust
gold stress + pred. emotion	-0.2735	-	-	-	-
gold emotion + pred. stress	-0.0563	-0.0292	0.0342	0.0683	0.1142
relief	pride	nervousness
gold stress + pred. emotion	-0.2070	-	-0.1450
gold emotion + pred. stress	0.0026	-0.0222	0.0318
Table 13: Full emotion correlations for the Multi $_{GoEmotions_A}$  model.
neutral	anger	fear	surprise	joy	sadness	disgust
gold stress + pred. emotion	-0.0419	-	-	-	-0.4986	0.0565	-
gold emotion + pred. stress	-0.0876	0.2936	0.1686	0.0276	-0.3286	0.2762	0.1548
Table 14: Full emotion correlations for the Multi  ${}_{GoEmotions_E}^{Alt}$  model.
fear	sadness	joy	correlation coefficient η
gold stress + pred. emotion	0.9697	0.7113	0.1386	0.3207
gold emtion + pred. stress	0.9545	0.8921	0.0235	0.4115
Table 15: Mean stress predictions by the MultiAlt model for given emotions in Dreadlit and GoEmotions, as well as the correlation coefficient  $\eta$  for those predictions.
BERT	Fine-TuneGEA	Fine-TuneGEE	Fine-TuneGES
stress	33.4%	31.9%	36.2%	33.8%
non-stress	15.1%	17.5%	15.5%	15.3%
Fine-TuneGEFSJ	MultiAltGEA	MultiAltGEE	MultiAltGES
stress	33.3%	33.1%	33.4%	31.9%
non-stress	17.3%	17.5%	16.5%	17.9%
MultiAltGEFSJ	MultiAltVent	MultiGES	MultiGEFSJ
stress	30.9%	31.5%	33.4%	32.0%
non-stress	17.6%	18.7%	17.4%	18.4%
Table 16: Full results for the relative salience analysis. So that the table fits on the page, GE is GoEmotions, and Dr is Dreaddit.
offensive language.
Stress. i, my, me, do, and, 'm, n't, just, ', feel, because, like, am, what, even, ?, he, but, anxiety, m, so, myself, this, know, ca, it, now, have, out, get, no, about, t, feeling, up, bad, how, 've, scared, not, him, over, going, all, tell, right, stop, want, anxious, past, to, fucking, need, hate, s, really, why, panic, where, happened, trying, still, when, days, makes, job, tired, or, shit, hard, getting, day, life, nothing, tl, dr, afraid, has, sorry, boyfriend, felt, crying, school, worse, don, go, attacks, sick, leave, deal, attack, anymore, being, work, im, having, constantly, thinking, almost, feels, been, worried, is, stress, which, family, due, fear, something, keep, everything, enough, every, back, worst,... point home sometimes car down making angry literally feelings actually cry horrible wo think anyone end move ... help terrified fuck head then pain losing situation depression depressed ve made money coming mom safe else everyday gets honestly thing unable turn whole terrible alone room heart saying wake awful sleep against mentally come absolutely nightmares stupid remember lot without does abuse lose class sad stuck hell suffer cant severe emotions leaving / flashbacks hospital close memories off night nowhere abused knowing issues trigger sexually
Non-Stress. , you, the, a, her, she, we, for, , in, your, would, be, ),!, will,  $(*, :, <$ , that, are, who, >, as, was, url, more, if, years, -, first, were, their, thank, us, met, people, his, them, our, an, they, said, one, together, others, share, let, best, food, other, & person, interested, please, study, each, here, asked, link, treatment, those, free, could," take, great, same, support, good," [some, make, months, may, older, finally, bit, research, online, experience, little, through, hope,#,$ many helped edit decided friend see took few homeless wanted nice information thanks around," questions any date went later everyone looking guys ask than relationship ago'll sister post complete'd dating year both current mental's send 18 moved amazing community provide items read how ever name x200b (i.e., a zero-width space), world willing different guy 3 turned area visit health open well case survivors 10 hear're give university own], hi learn couple access old long eventually choose agreed began love reading stories loving hey experiences include preferences forward,, write sub 1 posted also loved
page, email, start, away, sleeping, note, app, liked, helping, seemed, grateful, background, girl, talked, based, amazon, 2
We believe the numbers appearing in the non-stress list are indicative of financial posts where the authors indicate some amount of money has been raised or needs to be raised.
We include the full table of relative salience/LIME explanation counts in Table 16.