Probabilistic semantic automata in the verification of quantified statements

Probabilistic semantic automata in the verification of quantified statements Jakub Dotlaˇcil (j.dotlacil@gmail.com) Center for Language and Cognition, University of Groningen Jakub Szymanik (jakub.szymanik@gmail.com) Institute for Logic, Language and Computation, University of Amsterdam Marcin Zajenkowski (zajenkowski@psych.uw.edu.pl) Faculty of Psychology, University of Warsaw Abstract Strategies used by people to verify quantified sentences, like ‘Most cars are white’, have been a popular research topic on the intersection of linguistics, computer science, philosophy, and psychology. A prominent computational model of the task, semantic automata, has been introduced by van Benthem in 1983. In this paper we present a probabilistic extension of the model. We show that the model explains counting errors in the verification process. Furthermore, we observe that the variation in quantifier verification data cannot be explained by Approximate Number Sense, a prominent approach to proba- bilistic number estimation. Keywords: quantifier verification; natural language seman- tics; automata theory; probabilistic computational modeling, Approximate Number Sense Introduction. Subjects’ verification strategies used in rejecting/accepting sentences have been a popular research topic in psycholin- guistics (see, e.g. Clark and Chase, 1972). Together with the turn to more linguistically-complex phenomena the topic has also received an increased interest in linguistics, semantics, logic, and computer science. Especially the computational and cognitive capacities of recognizing the truth-value of sen- tences with so-called generalized quantifiers (like ‘some’, ‘an even number of’, ‘more than 7’, ‘less than half’ (Peters and Westerstahl, 2006)) has been intensively studied (see, e.g. Szymanik, 2009; Lidz et al., 2011) A prominent computational model for verification of quan- tifiers employs semantic automata (van Benthem, 1986). In- tuitively, to check whether sentence (1) is true: 1. Every sentence in this paper is grammatically correct. it suffices to read the sentences from this article one by one. If we find an incorrect one, then we know that the statement is false. Otherwise, if we read the entire paper without finding any incorrect sentence, then statement (1) is true (see Fig. 1 for a graphical representation). Analogous strategies exist for all other natural language quantifiers. However, for recognizing some higher-order quantifiers, like “less than half” or “most”, we need computational mod- els making use of internal memory. Intuitively, to check whether sentence (2) is true we must identify the number of correct sentences and store it in working memory to compare with the number of incorrect sentences. 2. Most of the sentences are grammatically correct. correct q 0 correct, incorrect incorrect q 1 Figure 1: This finite automaton checks whether every sen- tence in the text is grammatically correct. It inspects the text sentence by sentence starting in the accepting state (double circled), q o . As long as it does not find an incorrect sentence it stays in the accepting state. If it finds an incorrect sentence, then it already “knows” that the sentence is false and move to the rejecting state, q 1 , where it stays no matter what sentence is next. Mathematically speaking, such an algorithm can be realized by a push-down automaton, PDA, see Fig. 2. PDAs can not only read the input and move to the next state, they also have access to the stack memory and depending on the top ele- ment of the stack they decide what to do next. Graphically, we represent it by the following labeling of each transition: s 1 , x, y → s 2 , w, where s 1 is the current state, x is the current input the machine reads (i.e. the element under considera- tion), y is the top element of the stack, and s 2 is the final state and w shows what element is put on the top of the stack next (when the element is added to the previous top element, w is of length 2 and shows both the previous element and the new element) (Hopcroft et al., 2000). It has been shown that the computational distinction be- tween quantifiers recognized by finite-automata and push- down automata is psychologically relevant, i.e., the more complex the automaton, the longer the reaction time and working memory involvement of subjects asked to solve the verification task (see Szymanik and Zajenkowski, 2010a,b). McMillan et al. (2005), in an fMRI study, have shown that during verification, all sentences recruit the right inferior parietal cortex associated with numerosity, but only proportional quantifiers recruit the prefrontal cortex, which is associated with executive resources, such as working mem- ory. Zajenkowski et al. (2011) have compared the process- ing of natural language quantifiers in a group of patients with schizophrenia and a healthy control group. In both groups, the difficulty of the quantifiers was consistent with the com- putational predictions, and patients with schizophrenia took