On the Quantum-like Contextuality of Ambiguous Phrases (2021)

Abstract: Language is contextual as meanings of words are dependent on their contexts. Contextuality is, concomitantly, a well-defined concept in quantum mechanics where it is considered a major resource for quantum computations. We investigate whether natural language exhibits any of the quantum mechanics’ contextual features. We show that meaning combinations in ambiguous phrases can be modelled in the sheaf-theoretic framework for quantum contextuality, where they can become possibilistically contextual. Using the framework of Contextuality-by-Default (CbD), we explore the probabilistic variants of these and show that CbD-contextuality is also possible.

## introduction we start with a peculiar observation: even though polysemy and homonymy are common phenomena of natural language, i.e. many words have more than one meaning, this does not create a considerable obstacle in our day-to-day comprehension of texts and conversations. for example, the word charge has 40 different senses in english according to wordnet, however, its meaning in the sentence the bull charged. remains fairly unambiguous. on the other hand, polysemy and word sense disambiguation are computationally difficult tasks and is amongst the challenges faced by linguists @xcite @xcite . the emergence of the field of quantum methods in natural language processing offers promising leads for introducing quantum methods to classical nlp tasks, e.g. language modelling @xcite , distributional semantics @xcite , mental lexicon @xcite , narrative structure @xcite , emotion detection @xcite , and classification @xcite . distributional semantics is a natural language semantic framework built on the notion of contextuality. herein, frequencies of co-occurrences of words are computed from their contexts and the resulting vector representations are used in automatic sense discrimination @xcite . an issue with this framework is that the grammatical structure of phrases and sentences is ignored and the focus is mainly on large-scale statistics of data. oppositely, even though the interaction between context and syntax has been studied in the past @xcite , no distributional data has been considered in them. finally, distributional and compositional models of language have been proposed @xcite , small experiments have been implemented on quantum devices @xcite , and choices of meaning in concept combinations have been analysed using superposition @xcite . our work complements these lines of research by modelling the underlying structure of contextuality using distributional data. we investigate the contextual nature of meaning combinations in ambiguous phrases of natural language, using instances of the data gathered in psycholinguistics @xcite @xcite , frequencies mined from large scale corpora @xcite , and models coming from the sheaf-theoretic framework @xcite and the contextuality-by-default (cbd) theory @xcite . we consider phrases with two ambiguous words, in subject-verb and verbobject predicate-argument structures and find instances of logical and cbd contextuality. the structure of the paper is as follows. we start by introducing the main concepts behind quantum contextuality (section 2). we then introduce the sheaf-theoretic framework and logical contextual-ity (section 3), before applying it to possibilistic natural language models (section 4 and 5). in section 6 and 7, we discuss probabilistic models and signalling in natural language respectively. in section 8, we offer the possibility of studying contextuality in signalling models via the contextualityby-default framework and discuss two cbd contextual examples that we found. we then close the paper with insights on how to perform a large scale experiment and the possibility of finding more cbd-contextual examples in natural language. ## quantum contextuality early critics of quantum mechanics claimed that quantum theory was not complete @xcite , but instead was subject to unobserved hidden variables, and claimed that any physical theory should satisfy local realism. by local realism, one means that in a "complete" physical theory, the global behaviour of a system is entirely, and deterministically, fixed by a set of local variables. however, the well-known bell theorem @xcite , supported by experimental data @xcite , shows that a description of quantum mechanics cannot comply with local realism; if quantum systems need to have a "reality" independent of the observers (realism), one should allow interactions between systems to be unrestricted spatially (non-local). the bell inequality offers a proof by contradiction that one cannot extend the probabilistic models obtained from observations of quantum systems to a deterministic hidden-variable model. in @xcite , the authors prove a stronger statement about the existence of hidden-variable models via a logical argument. this more general result provides a description of contextuality as it is understood in quantum mechanics. ## presheaves and logical contextuality the sheaf-theoretic framework of contextuality starts from the observation that contextuality in quantum mechanics translates to the impossibility of finding a global section in special presheaves. in other words, a model is contextual if some of its local features cannot be extended globally. the presheaves considered in the framework developed by abramsky et. al. @xcite is undertaken; it consists of the list of measurements that can be made, which measurements can be made together and what are the associated probability distributions. for example, fig. 1 can represent a standard bell experiment where the list of measurements is the list of all local measurements that can be made on the two qubits involved in the experiment, i.e. {a, a , b, b }, under the condition that each laboratory performs exactly one measurement at each run of the experiment, e.g. (a , b ) can be a joint measurement, but (a, a ) cannot. the distribution presheaf then associates the observed (or theoretical) probabilities for the global measurement outcomes, that is, the joint outcomes of both parties in a bell scenario, for each measurement context. in this framework, a global assignment corresponds to an assignment of an outcome for every local measurement. a global section will on the other hand represent a distribution defined on all global assignments, which is consistent with all the observed probabilities. the framework of @xcite @xcite also introduces a stronger type of contextuality, called logical (or possibilistic) contextuality. indeed, they have found that the contextuality of some systems can be established from the support of each of the contextdependent distributions. these are referred to as possibilistic empirical models. in these models, we are only interested in whether an outcome of a local measurement (given a global measurement context) is possible. a consistent global assignment will then be an assignment of a possible outcome to every measurement, and hence can be represented by a logical statement about a subsystem. a global section will then be a disjunction of consistent global assignments that describes the entirety of the model. hence, one can prove logical contextuality, i.e. the impossibility of being able to write such a logical statement about the system, by finding a locally possible outcome that cannot be extended to a consistent global assignment. for small systems, it is convenient to represent possibilistic models by bundle diagrams @xcite . in these diagrams, we represent each of the local measurements by a vertex. there is an edge 1 between every two of these vertices if the joint measurement is possible. we then depict, for each individual measurements, the set of possible outcomes as a set "sitting" on top of the associated vertex. similarly, an edge is added between two of the "outcome"-vertices if the joint measurement has a non-zero probability (e.g. see fig. 1b ). in particular, global assignments can be seen in these bundle diagrams as shapes going through exactly one outcome for each of the measurements that mirror the structure of the base (measurements). in fig. 1b for example, global assignments correspond to connected loops. the sheaf-theoretic framework relies on the fact that the described distribution presheaf is indeed a presheaf. that is, the distributions associated with measurement contexts that intersect at a local measurement (i.e. two contexts where at least one party performs the same measurement) agree on their restrictions. these are here defined as the marginals of the distributions of interest. this requirement coincides with the non-signalling condition in quantum mechanics. this condition is stated for possibilistic models by requiring that the supports of intersecting distributions coincides. as we will see, many empirical models from natural language will be signalling. that is also the case for many behavioural and psychological experiments (see e.g. @xcite ; dzhafarov et al. ( 2016 )), and in fact, there is no reason why natural language systems should be non-signalling 1 more generally simplices if multiple measurements are carried out simultaneously. and we will discuss this issue in sections 7. ## contextuality and ambiguity in natural language we are interested in studying the influence of the context on the process of meaning selection in ambiguous phrases. indeed, homonymy and polysemy in natural language give rise to an interpretation for context-dependent probability distributions. probabilities will correspond to the likelihood that a certain meaning of a word is selected in the context of interest. by analogy with quantum contextuality, existence of contextual natural language examples confirms that the context in which words are found plays a non-trivial role in the selection of an appropriate interpretation for them and the following question arises: given that a certain interpretation of a word is selected within a certain context, can we use this information to deduce how the same word may be interpreted in a different context (e.g. in different phrases) in the corpus? our intuition is that this is not the case. consider the ambiguous adjective green: this either refers to the colour of its modifier (e.g. a green door), or the environmental-friendly nature of it (e.g. the green party). now, if we consider an unambiguous adjective such as new, then trivially, the same interpretations of new can be selected in both of the phrases new paint and new policy. this, however, does not imply that the same interpretations of green will be selected in green paint and green policy. with this intuition in mind, we start by considering the basic structure of ambiguous phrases of english by considering only the support of probability distributions attributed to these phrases and for now appeal to our common sense to determine the values of these supports. in the first part of the paper, we consider a structure similar to bell scenarios with multiple parties, or agents, each of which will choose one measurement context from a predetermined set. a "measurement" will be associated with each word and will return the activated meaning according to a fixed encoding. for example the two meanings of green could be encoded as: relative to @xmath0, environmental-@xmath1. in a given context, multiple ambiguous words will be allowed to "interact" and form a phrase. a measurement context will then be labelled by the words in this phrase. the interaction will be dictated by some predetermined rules, such as which part-of-speech each word will correspond to. for each global measurement context, the recorded activated meanings will then form a joint distribution. these distributions can be represented in the form of an empirical model as described in section 3. in order to obtain a valid empirical model, all the possible combinations of words need to make sense. for example, take two parties a and b such that a chooses an adjective in the set {green,new} and b chooses its modifier within the set {paint, policy}. all the combinations of a and b are possible, i.e. phrases green paint, green policy, new paint and new policy all make sense and can indeed be found in natural language corpora. however, if the set of adjectives is changed to {blue, new}, we will face a problem since the phrase blue policy does not make much sense and we could not find any occurrence of it in the corpora considered in this paper. 2 in order to keep the models and computations simple, we work with 2-word phrases, where each word of the phrase is ambiguous. from the analogy with bell scenarios, this means that we are working with bipartite scenarios (see fig. 3 ). the set of ambiguous words is taken from experimental data sets from the studies: pickering and frisson (2001); rayner and duffy (1986); tanenhaus et al. @xcite . in sections 5.4 and 6.4, we introduce another kind of experiment which departs from bell scenarios. measurements of these examples have the same interpretation as before, but the focus is on combinations involving a single verb and a single noun for which both of subject-verb and verbobject phrases are possible. this structure is analogous to the scenario in behavioural sciences for the "question order effect" (wang and busemeyer, 2013). in the sheaf-theoretic framework measurement contexts are dictated only by the choices of local measurements and we face two possibilities when modelling these examples. in the first possibility, one can consider the two contexts subjectverb/verb-object as disjoint and as a result lose some semantic information. this is because, for example, adopt in adopt boxer would be treated as completely unrelated to adopt in boxer adopts. in this case, all such systems will be trivially noncontextual, as there will be no intersecting local measurements. in our paper, on the other hand, we choose a second possibility and decide to keep the semantic information but as a result any system for which the distribution arising from the verb-object context differs from the one associated with subjectverb context will be signalling. this type of model does not easily lend itself to a sheaf-theoretic analysis but admits a straightforward cbd analysis foot_1 . ## possibilistic examples we demonstrate the methodology by choosing three sets of phrases from the sets considered by @xcite as well as two verb-object/subject-verb examples. for each of these phrases, we tabulate how we encoded the meanings of each word, provide an empirical bell-style table for the possibilistic cases and outline the different types of contextual features each example demonstrates. ## probabilistic variants we consider the same examples as in the previous section, but from a probabilistic point of view. figure 7 : empirical models for the pairs of words examples. here, the different contexts are depicted as follows: @xmath2. the outcomes labels are the same for both contexts; for example (0, 1) in (a) means @xmath3, @xmath4. and 5a ; for verbs, outcomes 0 and 1 represent literal and figurative meanings, respectively. the measurement contexts (verb-object or subject-verb) are depicted by arrows on the associated edges. the probability distributions are obtained from the british national corpus @xcite and ukwac @xcite . bnc is an open-source text corpus comprising of 100 million words, spread across documents of different nature (including press articles, fiction, transcription of spoken language, and academic publications). ukwac is a 2 billion word corpus constructed from the web limiting the crawl to the .uk domain. both bnc and ukwac are part-of-speech tagged, hence, they provide grammatical relations and the lemma forms of words. the semantic interpretation of the words and phrases are absent from these corpora and had to be decided by the authors manually. recall that the model in section 5.1 was possibilistically signalling. the frequencies mined from corpora were found to have the same support as the model described in section 5.1 (see fig. 9 ), whence the probabilistic analogue remains signalling. ## non-signalling and ambiguity in natural language non-signalling is a necessary condition for demonstrating non-locality in quantum mechanics. in experiments such as the one described in @xcite , this assumption models the space-like separation between the systems, e.g. two entangled qubits which are measured in geographically different labs, or more generally the fact that no communication between these systems is possible after their preparation. non-signalling is a property that ensures some laws of quantum mechanics hold in specific systems and certainly there is no reason to assume it for natural language. in order to understand why not, let's try and use an analogy with quantum systems. in our experiment, ambiguous phrases become analogous to entangled quantum systems and each word within a phrase to a qubit. in the subject-verb phrases we considered, a form of communication between words within a phrase becomes possible if after, say the subjectmeasuring agent determines the meaning of the subject, the verb-measuring agent has a more limited choice in determining the meaning of the verb. a similar situation is true for the verb-object phrases. in these cases, communication between the words of a phrase may seem possible but will definitely not in general. for instance, consider the coach lap phrase, if the subject-measuring agent decides that the meaning of coach is bus, the verb-measuring agent does not get a choice, since buses cannot drink. in this case, communication between the subject and verb-measuring agents is needed. if the subject-agent, however, sets the meaning of coach to be sports trainer, the verb-measuring agent still gets a choice for the meaning of lap, since a trainer can run in circles as well as drink something up. in this case, communication between the agents is not as clearly possible as before. ## contextuality-by-default we will now study the contextuality of the probabilistic signalling systems we obtained in section 6 using the contextuality-by-default framework. in this framework, each set of jointly distributed measurements of the empirical model is called a context, and the contextuality of a system is defined by the impossibility of creating a global joint distribution in which the variables corresponding to each measurement in each pair of contexts where they appear are equal to each other with maximal probability (instead of always). for example, in expression (1) we noticed that the proportions with which the word "box" is assigned the meaning "put in boxes" differs between the contexts with measurements "box leaves" and "box can". this difference makes the system signalling and implies that the two variables cannot be treated as equal to each other within a global assignment. they need to be treated as different random variables. the maximal probability that those two random variables can both receive the assignment "put in boxes" is min{2/3, 7/74} = 7/74. these probabilities can be found for every pair of variables corresponding to each measurement. continuing with the example of the variables corresponding to the measure of "box" from the example in section 6.3, the maximal probability with which they could be assigned the meaning "fight" in the contexts "box leaves" and "box can" is equal to min{0, 0} = 0, and the probability with which they both can be assigned "container" is min{1/3, 67/74} = 1/3. the task of finding whether a global joint distribution that maximizes these probabilities for every measurement exists can be solved by linear programming. dzhafarov and @xcite describe how to define this task for systems that include measurements with a finite number of outcomes by taking all possible dichotomizations of their respective outcome sets. we illustrate the procedure with the proportions of the system in section 6.3. the description of this system simplifies by noting that the word "leaves" could only be assigned two meanings and that for the word "box" a global joint distribution of all dichotomized variables in our system must define probabilities for 2 16 different events. they are the combination of the outcomes of 16 binary random variables: a) 6 in context "press can" including the three dichotomizations of press can| press and press can| can ; b) 4 in context "press leaves"; c) 4 in context "box can"; and d) 2 in context "box leaves". the 2 16 probabilities are restricted by the probabilites estimated in section 6.3, which total 97 linear constrains considering the joint events of the dichotomizations, individual margins (as the ones in expression (1)), and that probabilities in a distribution add to unity. these probabilities are further restricted by the maximal probabilities computed for the pairs of variables corresponding to the same dichotomization of the same measurement. these maximal probabilities are computed by taking the minimum of the two compared probabilities as explained above, and they amount to 8 linear constrains. in all, a total of 105 linear constrains that the probabilities of the 2 16 events must satisfy, and that can be represented in a 105 × 2 16 matrix of coefficients. solving the set of linear equations for this example showed that it was possible to find such a global joint distribution. whence, the system is not contextual. the systems in sections 6.1 and 6.2 can be shown to be non-contextual within the cbd framework from a bell inequality for certain signalling systems which was proved in @xcite . ## conclusions and discussion undoubtedly, the context of ambiguous words plays an important role in their disambiguation process. the nature of this role, on the other hand, is not properly understood and quantified. in this work, we find ambiguous phrases that are possibilistically (i.e. logically) contextual in the sheaftheoretic model, but show that their probabilistic extensions become signalling. in the presence of signalling, we analyse these examples in the cbd framework and discover some of them are not cbdcontextual. at the same time, however, we do find examples that are cbd-contextual albeit signalling. we then argue that the use of different contextuality frameworks allows us to formally study the effect of the context on choices of interpretation of ambiguous phrases, paving the way for a systematic study of general contextual influences in natural language. this study was restricted by the nature of the types of meanings we considered and the size of our corpora. indeed, the observed frequencies of phrases were not always consistent with our intuition, and in some cases, meaningful phrases did not appear in the corpus altogether. an example was the word coach, which could either mean a sports trainer or a type of bus. in the corpora we considered, the latter meaning was in fact quite rare. our conjecture is that this is due to the fact that the corpora we considered were both almost exclusively based on british english, whereas, the bus meaning of coach is mainly american. regarding types of meaning, in order to facilitate our manual search for occurrences of interpretations, we restricted the domain of possible meanings and did not consider figuratively metaphorical options. an example is the verb boxing, which can also mean labelling or ignoring the workings of, but we only considered its putting in a box and fist fighting meanings. in future work, we aim to overcome this restrictions by widening our experimental data and gather human judgement on degrees of likelihood of each meaning combination. this will allow also us to consider a wider range of grammatical relations in the contexts and also study the effects of these structures on the disambiguation process as well as allowing a more reliable estimation of the probability distributions.