DALLE -2 is Seeing Double: Flaws in Word-to-Concept Mapping in T ext2 I mage Models (2022)

Abstract: We study the way DALLE-2 maps symbols (words) in the prompt to their references (entities or properties of entities in the generated image). We show that in stark contrast to the way human process language, DALLE-2 does not follow the constraint that each word has a single role in the interpretation, and sometimes re-use the same symbol for different purposes. We collect a set of stimuli that reflect the phenomenon: we show that DALLE-2 depicts both senses of nouns with multiple senses at once; and that a given word can modify the properties of two distinct entities in the image, or can be depicted as one object and also modify the properties of another object, creating a semantic leakage of properties between entities. Taken together, our study highlights the differences between DALLE-2 and human language processing and opens an avenue for future study on the inductive biases of text-to-image models.

## introduction large diffusion-based text-to-image models, such as dalle-2 @xcite , generate visually compelling images that condition on input texts. yet, the extent to which such models capture properties of human language, such as its compositional structure, has been doubted @xcite . a very basic property in the interpretation of natural language utterances is that each word has a specific role in the interpretation, and there is a oneto-one mapping between symbols and roles. while symbols-as well as sentence structures-may be ambiguous, after an interpretation is constructed this ambiguity is already resolved. for example, while the symbol bat in a flying bat can be interpreted as either a wooden stick or an animal, our possible interpretations of the sentence are either of * equal contribution. a flying wooden stick or a flying animal, but never both at the same time. once the word bat has been used in the interpretation to denote an object (for example a wooden stick), it cannot be re-used to denote another object (an animal) in the same interpretation. similarly, in a fish and a gold ingot, the word gold is used as a modifier of ingot. 1 once it is used, it cannot be re-used to modify another object in the same interpretation, and also cannot be used as a standalone object. 2 some linguists refer to this property as resource sensitivity @xcite . we show evidence that-in stark contrast to humans-the text-to-image model dalle-2 @xcite does not respect this constraint. indeed, we show that dalle-2 exhibits the following behaviors that are inconsistent with the single-role principle: visually, the cases we highlight map to one of two failure modes from the perspective of the image designer / user: (a) sense-ambiguous words are hard to isolate, and the resulting images often exhibit the unintended sense together with the intended one (homonym duplication). (b) visual properties of one object in the image "leak" into other objects in the image (concept leakage). 4 we also observe some second order effects, where a hypernym of the modifier word, or an implicit description of it, also triggers the duplication effect. taken together, the phenomena we examine provides evidence for limitations in the linguistic ability of dalle-2 and opens avenues for future research that would uncover whether those stem from issues with the text encoding, the generative model, or both. more generally, the proposed approach can be extended to other scenarios where the decoding process is used to uncover the inductive bias and the shortcomings of text-to-image models. ## previous work promptly after the introduction of dalle-2 @xcite , analyses of the system began to surface @xcite . @xcite showed that dalle-2 does not understand spatial relations. @xcite reported that if an object in the prompt is said to have some property, then the image will likely show that property somewhere, however, not necessarily on the correct object. we take this analysis a step further, showing that the same property sometimes simultaneously modifies several unrelated objects. @xcite allude to the phenomena we analyze: they mention the issues with binding separate attributes to separate objects, and hypothesize that it is because clip @xcite embeddings do not explicitly bind attributes to objects. accordingly, they observe that that reconstructions from the decoder often mix up attributes and objects. concurrent to this work, @xcite discuss underspecification in text-to-image generation, a condition which we find to be associated with some of the behaviors we identify; and anonymous (2023) discuss several cases of "leakage", and propose an intervention in the decoding process that is aimed to mitigate them. ## methdos stimuli we construct linguistic stimuli (prompts) which evoke a behavior that is inconsistent with the single-use-per-symbol axiom. for the first case (words interpreted as two entities), we use homonyms, words that have two distinct senses. dalle-2 often generates two objects, one corresponding to each of the senses. for the modification cases, we construct the following prompts. for the case of a word that is interpreted as a modifier of two entities, we include prompts with two entities e 1 and e 2 and a modifier word m that is semantically compatible with both of them, but is only syntactically modifying the second one (for instance, "a fish e 1 and a gold m ingot e 2 "). for the case of a word that is simultaneously interpreted as an entity as well as a modifier, we construct stimuli that contain two entities e 1 and e 2 , such as that one of them is semantically associated with the other noun-for instance, "classic butter e 1 and peanuts e 2 /m ". the complete list of stimuli is presented in tables 1, 2 and 3 in appendix a.1, together with samples of the generated images. ## results we generate 12 images per stimulus, and report the average over all stimuli and images. ## conclusions we demonstrate that across a set of stimuli, dalle-2 does not follow basic principles of symbol-to-entity mapping in language. this flaw is especially pressing given that dalle-2 is directed to the general population, where the majority of the users are probably not aware of the phenomenon. 7 future work should trace back the issues we diagnose to specific components in the model, such as the text encoding or the generative decoder; and study its dependence on scale and architecture. additionally, the development of post-hoc mitigation techniques is especially important, given that most users lack the resources needed to train the models from scratch.