SlowGuess/ACL_2020_OCR huggingface dataset:
2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019), pages 2877-2887.
Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019b. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625-641.
Sarah Wiegrefe and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP 2019), pages 11-20.
Ethan Wilcox, Roger Levy, and Richard Futrell. 2019. What syntactic structures block dependencies in RNN language models? In Proceedings of the 41st Annual Meeting of the Cognitive Science Society (CogSci 2019).
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
Arjen Zondervan. 2010. Scalar Implicatures or Focus: An Experimental Approach. LOT Publications, Utrecht.
A Hyperparameter tuning
Figure 6 shows the learning curves averaged over the 5 cross-validation tuning runs for models using different word embeddings. As these plots show, the attention layer improves predictions; contextual word embeddings lead to better results than the static GloVe embeddings; and including the conversational context does not improve predictions and in some cases even lowers prediction accuracy.
Figure 6: Correlation between each model's predictions on valuation set and empirical means, by training epoch.
B Regression analysis on held-out test data
Figure 7 shows the estimates of the predictors in the original and extended Bayesian mixed-effects models estimated only on the held-out test data. We find the same qualitative effects as in Figure 4, but since these models were estimated on much less data (only 408 items), there is a lot of uncertainty in the estimates and therefore quantitative comparisons between the coefficients of the different models are less informative.
Figure 7: Maximum a posteriori estimates and  $95\%$  -credible intervals of coefficients for original and extended Bayesian mixed-effects regression models predicting the inference strength ratings on the held-out test set.  $* / * * / * * *$  indicate that the probability of the coefficient of the original model having a larger magnitude than the coefficient of the extended model is less than 0.05, 0.01, and 0.001, respectively.
C List of manually constructed sentences
Tables 1 and 2 show the 25 manually created sentences for the analyses described in the minimal pairs analysis in Section 5. As described in the main text, we created 16 variants of the sentence with the some-NP in subject position (sentences in the left column), and 16 variants of the sentence with the some-NP in object position (sentences in the right column), yielding in total 800 examples.
Some of the attentive waiters at the gallery opening poured the white wine that my friend really likes.	The attentive waiters at the gallery opening poured some of the white wine that my friend really likes.
Some of the experienced lawyers in the firm negotiated the important terms of the acquisition.	The experienced lawyers in the firm negotiated some of the important terms of the acquisition.
Some of the award-winning chefs at the sushi restaurant cut the red salmon from Alaska.	The award-winning chefs at the sushi restaurant cut some of the red salmon from Alaska.
Some of the brave soldiers who were conducting the midnight raid warned the decorated generals who had served in a previous battle.	The brave soldiers who were conducting the midnight raid warned some of the decorated generals who had served in a previous battle.
Some of the eccentric scholars from the local college returned the old books written by Camus.	The eccentric scholars from the local college returned some of the old books written by Camus.
Some of the entertaining magicians with top hats shuffled the black cards with dots.	The entertaining magicians with top hats shuffled some of the black cards with dots.
Some of the convicted doctors from New York called the former patients with epilepsy.	The convicted doctors from New York called some of the former patients with epilepsy.
Some of the popular artists with multiple albums performed the fast songs from their first album.	The popular artists with multiple albums performed some of the fast songs from their first album.
Some of the angry senators from red states impeached the corrupt presidents from the Republican party.	The angry senators from red states impeached some of the corrupt presidents from the Republican party.
Some of the underfunded researchers without permanent employment transcribed the recorded conversations that they collected while doing fieldwork.	The underfunded researchers without permanent employment transcribed some of the recorded conversations that they collected while doing fieldwork.
Some of the sharp psychoanalysts in training hypnotized the young clients with depression.	The sharp psychoanalysts in training hypnotized some of the young clients with depression.
Some of the harsh critics from the Washington Post read the early chapters of the novel.	The harsh critics from the Washington Post read some of the early chapters of the novel.
Some of the organic farmers in the mountains milked the brown goats who graze on the meadows.	The organic farmers in the mountains milked some of the brown goats who graze on the meadows.
Some of the artisanal bakers who completed an apprenticeship in France kneaded the gluten-free dough made out of spelt.	The artisanal bakers who completed an apprenticeship in France kneaded some of the gluten-free dough made out of spelt.
Some of the violent inmates in the high-security prison reported the sleazy guards with a history of rule violations.	The violent inmates in the high-security prison reported some of the sleazy guards with a history of rule violations.
Table 1: Manually constructed sentences used in the minimal pair analyses. Sentences in the left column have a some-NP in subject position; sentences on the right have a some-NP object position.
Some of the eager managers in the company instructed the hard-working sales representatives in the steel division about the new project management tool.	The eager managers in the company instructed some of the hard-working sales representatives in the steel division about the new project management tool.
Some of the brilliant chemists in the lab oxidized the shiny metals extracted from ores.	The brilliant chemists in the lab oxidized some of the shiny metals extracted from ores.
Some of the adventurous pirates on the boat found the valuable treasure that had been buried in the sand.	The adventurous pirates on the boat found some of the valuable treasure that had been buried in the sand.
Some of the mischievous con artists at the casino tricked the elderly residents of the retirement home.	The mischievous con artists at the casino tricked some of the elderly residents of the retirement home.
Some of the persistent recruiters at the conference hired the smart graduate students who just started a PhD as interns.	The persistent recruiters at the conference hired some of the smart graduate students who just started a PhD as interns.
Some of the established professors in the department supported the controversial petitions that were drafted by the student union.	The established professors in the department supported some of the controversial petitions that were drafted by the student union.
Some of the muscular movers that were hired by the startup loaded the adjustable standing desks made out of oak onto the truck.	The muscular movers that were hired by the startup loaded some of the adjustable standing desks made out of oak onto the truck.
Some of the careful secretaries at the headquarters mailed the confidential envelopes with the bank statements.	The careful secretaries at the headquarters mailed some of the confidential envelopes with the bank statements.
Some of the international stations in South America Televised the early games of the soccer cup.	The international stations in South America Televised some of the early games of the soccer cup.
Some of the wealthy investors of the fund excessively remunerated the successful brokers working at the large bank.	The wealthy investors of the fund excessively remunerated some of the successful brokers working at the large bank.
Table 2: Manually constructed sentences used in the minimal pair analyses (continued).
D Results from no-context experiment
Figure 8 shows the correlation between the mean inference strength ratings for each item in the experiment from Degen (2015) and the mean strength ratings from the new no-context experiment, discussed in Section 6.
Figure 8: Mean inference strength ratings for items without context (new) against items with context (original),  $r = .68$ .
5403