Meaning Representations from Trajectories in Autoregressive Models
Authors: Tian Yu Liu
Matthew Trager
Alessandro Achille
Pramuditha Perera
Luca Zancato
Stefano Soatto
Venue: ICLR 2024 poster (2024)
Keywords: definitions of meaning
semantic similarity
sentence embeddings
Primary area: visualization or interpretation of learned representations

Abstract: We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. Moreover, unlike vector-based representations,  distribution-based representations can also model asymmetric relations (e.g., direction of logical entailment, hypernym/hyponym relations) by using algebraic operations between likelihood functions. These ideas are grounded in distributional perspectives on semantics and are connected to standard constructions in automata theory, but to our knowledge they have not been applied to modern language models. We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle. Finally, we extend our method  to represent data from different modalities (e.g., image and text) using multimodal autoregressive models. Our code is available at: https://github.com/tianyu139/meaning-as-trajectories

MEANING REPRESENTATIONS FROM TRAJECTORIES IN AUTOREGRESSIVE MODELS
Tian Yu Liut* UCLA!
Matthew Trageri
Alessandro Achille AWS AI Labs²
AWS AT Labs²
Pramuditha Perera AWS AT Labs?
Luca Zancato AWS Al Labs²
Stefano Soatto AWS AI Labs²
ABSTRACT
We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. Moreover, unlike vector- based representations, distribution-based representations can also model asym- metric relations (e.g., direction of logical entailment, hypernmmhhyonyym rela- tions) by using algebraic operations between likelihood functions. These ideas are grounded in distributional perspectives on semanties and are connected standard constructions in automata theory, but to our knowledge they have not been applied to modern language models. We empirically show that the repre- sentations obtained from large models align well with human annotations, out- perform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle. Finally, we extend our method to represent data from different modalities (c.g., image and text) using multimodal autoregres- sive models. Our code available at: https: //gitthuu com/ tianyul39/ mean meaning-as ng- traa-aattaaaaecrrieeee fectories
INTRODUCTION
Generative Large Scale Models today are capable generating remarkably coherent text by itera- tively predicting individual tokens. Contrary to encoder-only encoder-decoder models, however, autoregressive models do not construct explicit representations of sentences: the model's represen- tation of given input is distributed across layers and attention heads, making it difficult analyze how the model "understands" and contextualizes language. This lack of transparency and inter- pretability is challenge for the responsible deployment of these models.
In this paper, we propose a simple way to explore how autoregressive models manipulate text. Whereas standard methods represent sentences by embedding them in vector space, propose to represent sentences, or parts of sentences, as the distribution of their possible continuations, or trajectories. This can be seen as practical embodiment of classical distributional approaches to semantics (Boleda, 2020; Sahlgren, 2008), according to which the meaning of linguistic items is tied the distribution of their usage. is also related to standard constructions in formal language and automata theory which associate the "behavior" of prefix with the set of its possible future continuations (Hoperoft et al., 2007).
Prior work has mostly focused on representing sentences via pooling output tokens. For example, BERT-based models (Devlin al., 2018) include a [CLS] token that meant capture semantic information. Sentence encoder models such as ST5 (Ni et al., 2021) represent sentences by pooling encoder or decoder outputs. These strategies are most effective after fine-tuning using a contrastive
i Equal contribution
Work done during an internship at AWS Labs. Tlanyu@eesucca.cco 2 [[mtrrggeraacchlle...man www.:/aammaaiiiaaaaoooooootooooo @ @amazon.com

---

leamingg objective (Gao et al., 2021), but this requires additional data and modifies the weights of the original model. In particular, these methods do not faithfully reflect the original model's s internal representation of the input sentence, and may be skewed by biases present in the data. Moreover, at the time of writing, the most powerful large language models are based on autoregressive archi- tectures (Touvron al., 2023; Brown al., 2020; OpenAT, 2023). For such architectures, similar strategies based on averages of output tokens significantly underperform (Table 1). Instead, prompr engineering usually regarded as the de facto standard solve semantic tasks without further fine- tuning. For example, Jiang et al. (2023) craft careful prompts to elicit better token representations from autoregressive models. However, such approaches are problematic, since: 1) they can be highly susceptible to the (language dependent) choice of prompt; 2) the answer may not faithfully capture how the model actually interprets the sentence - model may reply that two sentences are similar, but not necessarily represent them internally in the same way; 3) they do not provide any structured (toplogicc/metriccommssttoooo
semantic space in which sentences are embedded.
Representing the meaning of a sentence as distribution of trajectories provides a straightforward way of bypassing these limitations. First, the method is general and can be applied to any autore- gressive model without assumptions on architecture
on the language that it was trained on -ndd does not require fine-tuning or carefully crafted prompts. Since modern pre-trained large models are very capable of processing text, they should provide strong meaning representations oul- of-the-box. Second, using representations based on trajectories allows not only measuring semantic distances, but also defining compositional operations on meanings. In particular, we can define Boolcan-like operations between meanings, and use them for example to infer the direction of log- ical entailment between sentences as perceived by the model, or determine hypernym/hyponn ponym relation between words (Figure 2b). Third, our method can be applied without any modification to multimodal autoregressive models that encode images as sequences of tokens, and used to compare the meaning representation of data from different modalities. The main technical challenge of our definition is that the space of all possible continuations of a sentence too large to explored directly. We show however that, with appropriate sampling strategy. 10-20 sampled trajectories for each prompt are sufficient to approximate pairwise distances in semantic space (see Figure 1).
The focus of our work is to propose general way to extract "canonical" and interpretable mean- ing representations from pre-trained autoregressive models. Nonetheless, we empirically show that method achieves competitive results on prompt-free, zero-shot semantic textual similarity (STS) tasks and that the representation we obtain applying our method to the LLaVA (Liu et al., 2023) vision-language model outperforms CLIP embeddings (Radford et al., 2021) on semantic image- image and image-text similarity tasks on the Crisscrossed Captions (Parekh et al., 2020) dataset. We show that the representation we obtain, although based solely on distributions of token sequences, largely agrees with human annotations both on semantic similarity, logical entailment and contain- ment relations. These results support the idea that autoregressive models can represent sentences in a semantically meaningful way even if the representations are not explicit in their activations.
In summary, our main contributions are as follows:
1. We propose canonical meaning representation for autoregressive models distribution over trajectories extending sentence. Unlike vector space representations, this definition can directly capture asymmetric relations like logical entailments and hypenyym/hyponym relations.
2. We show that the representations obtained from modern Large Language Models (LLMs) align well with conventional linguistic meanings: our method achieves competitive performance on Se- mantic Textual Similarity (STS) benchmarks, outperforming comparable zero-shot and prompt- free baselines using the same architectures.
3. Our method can be extended without any modification Vision Language Models (VLMs) for the purpose of quantifying semantic image-image and image-text similarity, outperforming even CLIP embeddings (Radford et al., 2021) when applied L.LaVA (Liu et al., 2023).
RELATED WORK
Our work unifies two lines of work which are highly synergistic yet largely disjoint up until now the investigation of "meaning" within the internal representations of pre-trained LLMs and the com- putation of sequence embeddings for semantic comparison tasks.
2

---

Input Sentences
Sampled Trajectories
Compare Score Distributions
Semerticaly Simier
"This is great news for everyone. He then open...
The sound was deafening. didn't understand Then, rotating he changed ...! As he was hoiding the bottle in the air while cutting....
$1: The used sword to sllce plastic bottle.
What he did showed power of teamwork, 'He then poured some coloured water into the bottle. man slices a plastic bottle with sward during "Whe the plastic shattered, there noise
S2: man sliced a plastic bottle with a sword.
Trajecsoin
Semantically Diffrent
'He walking post shop window, smiling to himself ... "ninn my office. have a book, which I have been 'I want make this work because love flutes, i/s he happy, he sad? What is he feeling?... ?..'
S1:A man playing a fluto.
Liwibaod Soue
The dog's is owner telling him stop, but avail... "nAnother dog jumps in. Then another. Soon they're.. 'nehe dog's barking wakes me in fly can only fly... 'nit's like the fly has superpowers. The dog's brain..."
$2: dag is barking at fly.
Figure 1: Sentences with similar meanings produce similar score distributions over their continua- tions (top), while sentences with different meanings produce different score distributions over their continuations (bottom).
The close relationship between the statistical distribution of linguistic items and their meaning is the basis of the Distributional Hypothexis (Harris, 1954). This perspective also draws insight from Wittgenstein's 's use theory of meaning (Wittgenstein, 1953), commonly sloganized as "meaning is use." " In the field of natural language processing (NLP), semantic representations are frequently constructed based on statistical co-occurrences. However, conventional computational approaches such as word2vec (Mikolov et al., 2013), typically involve computing statistics from a text corpus and then constructing vector representations for words sentences. In contrast, in this work propose to directly leverage the distribution itself as a fundamental representation of meaning. This possible since LLMs offer a way to efficiently sample from such distributions, thereby providing an intrinsic notion of meaning from the perspective of the model.
Recently, several authors have argued that models trained on language alone, or more generally on "form," are necessarily incapable of learning and representing conventional meaning. In particular, Bender & Koller (2020) propose definition of meaning as relation between language expressions and "communicative intents" which are, by definition, external to the language. Therefore, they conclude that LLMS trained on language expressions cannot principle learn meanings. This leads to characterizing LLMs as "stochastic parrots" (Bender et al., 2021) capable of modeling the statis- tical form of the language (syntax) but intrinsically incapable of representing meaning (semanties). Merrill et al. (2021) investigate the role of assertions in both code and language, suggesting that ungrounded language models cannot fully emulate semantic representations.
However, semantic structures can be constructed from syntactic ones: For instance, Wu et al. (2023) show that models trained on synthetic languages with "strong transparency" (defined as those where expressions have montext-independee gest, however, that the senexxt-dependency of natural language limits language models from learning the semantic representations within. Using the language of category theory, Bradley al. (2022) describe a functor between a syntactic category of text probabilities and a semantic category of meanings. While purely theoretical, their construction is closely related our distribution-based meaning representation, thus providing further support for our proposed method. Soatto et al. (2023) define meanings in L.LMs as equivalence classes of sentences induced by the trained model. This definition generalizes that of Bender & Koller (2020), since "communicative intent" can be latent in the expressions used for training the LLM, which induces partitions the set of complete sen- tences. But while this characterization is suitable for analyzing the controllability of the model in the corresponding metric space, the resulting meaning representation does not exhibit any obvious compositional structure. Our definition is more general, and provides us with means to compose meaning directly in representation space, unlike all other works. While we do not wish to focus on the high-level aspects of the debate on "meaning" and "understanding" (or lack thereof) in LLMs,
denotations) can emulate semantic representations. They sug-
3

---

results provide evidence that autoregressive models actually have rich latent semantic represen- tations within their internal structure.
Encoder-based architectures have traditionally been the main tool for embedding sequences in common vector space, in which they can be easily compared. Apart from BERT (Devlin et al., 2018) and ST5 (Ni et al., 2021), Centence-BERT (Reimers & Gurevych, 2019) fine-tunes modified BERT architecture to improve sentence embeddings. Opitz & Frank (2022) improves the interpretability of Sentence-BERT embeddings while preserving their effectiveness. Zhang al. (2020) and Gao al. (2021) propose contrastive fine-tuning objectives to obtain more effective embeddings. In contrast, our method does not require any fine-tuning, hence can faithfully reflect the original model's internal representation of an input string. Prompting is also commonly used to extract embeddlnngs. Jiang et al. (2022) search over prompts to improve the embeddings obtained from BERT. Jiang et al. (2023) propose PromptHOI. to summarize sentences as a single word for comparisons. Similar to fine-tuning, prompting alters/biases the meaning of the original string, and further requires sufficient command over the language being used to engincer effective prompt. The latter also fails to scale with model sizes, generally performing worse on semantic similarity tasks as model size increases.
Most related to our work, Muennighoff (2022) applies decoder-only models for semantic search by computing pairwise conditional likelihood scores between a query and each document in the search database. Our experiments show that this conditional likelihood is insufficient to fully capture relative semantic meaning. Our method is prompt-free, and scales well with model size and human perception of model performances. Unlike prompt-based methods, the meaning space resulting from our method can also be composed to compute more complex relations between strings.
3 METHOD
Preliminaries. We use A to denote a finite vocabulary of tokens and A to indicate the set of all variable-length finite sequences of tokens in A. We view language model as map M : A x A'' - [0. 1] associating a "prompt" sequence s E AA and a possible continuation sequence I E A" with score M([ss) [0, 1]. Intuitively, this score represents the likelihood of the model sampling / as a continuation of 8. For our experiments, we use as score the inverse perplexity:
M(L- = (a1 am))))= = II
where PM is the probability over the next token defined by the model. When s the empty string, we write (t) instead of (tle).
Meaning representation for prompts. We define the syntactic meaning representation of a prompt string for the model M as the function M, := (--ls) A' + [0, 1]. This definition fully captures the way in which the model interprets the string s. For example, if M,- = Mt. then the prompts s and are indistinguishable based their continuations for the model. Note that the function (I) that represents the string 8 is an infinite dimensional object, since its domain are all finite sequences & A One of the challenges will handle later is how effectively use this representation through sampling.
We remark that we can consider particular case of eq. (1) where the domain of M, is restricted only to strings E A comprising single tokens, instead of arbitrary length strings. This represents sentence using the score distribution over the immediate next token, and is common baseline used to embed sentences with autoregressive models. However, it is easy to see that this is a very incomplete semantic representation: for example, common tokens such as "The" are often the most likely next token regardless of the actual meaning of the prompt. This limitation will be evident in our experimental results.
Sets of continuations. as the distribution trajectories that extend that string. To guide intuitions, it often useful to con- sider the more restricted setting where scores are binary M, A + {0,1). 1). This can be interpreted as the characteristic function of the set of strings t that are feasible continuations of s according to the model. One advantage of this interpretation is that it makes explicit that meaning representations in our framework are not simple vectors, but rich objects that can be naturally manipulated though set-theoretic operations such as intersections fact that we will use later. This simpler setting
The function
+ [0, 1] essentially represents the meaning of a string
4

---

also allows direct connection with automata theory: For any language L A", the sets of fea- sible continuations s s := {t: st E L} of prefixes can seen as the set of states canonical "minimal automaton" accepting the language (Hopcroft et al., 2007). In a similar fashion, the sets M, of strings accepted by the model prompted with & can be interpreted as a canonical "model of behaviors" for the I.M.. We refer to Appendix G for discussion on these topics.
Semantic similarity. Given two prompts u and v, we define their semantic distance as the distance d(M,, My) between their representation M,, and M., where d denotes distance function that can be picked arbitrarily. For our experiments, we use:
d( Mu, M,) = E (M,+MM) flog (t) - log (o)
(2)
log Ma
(M
plas|u,aes)
This amounts comparing the expected difference in log-likelihood between the models on continuations ~~ 2 {M,, + M,) sampled with equal probability from either prompts. We ablate on other natural choices of distances in Appendix A.2. As noted above, explicitly integrating eq. (2) over all possible trajectories is not feasible. Rather, we approximate the expectation through Monte Carlo sampling. More precisely, we sample n trajectories Tú { 1 for the prompt u, where t. ~ Mu. and trajectories T, for the prompt each of length up fixed hyperparameter m.
We then approximate eq. (2) as:
d(Mu: My)
Σ
llog Mt) - log M, (t)
2n
LET,UIT
The steps we follow are detailed in Algorithm 1. More sophisticated approaches for approximating the distance could be explored in future work.
related baseline for comparing the similarity of two sentences and v is the likelihood of their concatenation, M (wv) M (vlu). However, perplexity-baseee measures are known be unreliable when directly used to compare different sentences, even when the sentences have the same length (Wang et al., 2022; Meister & Cotterell, 2021). Moreover, the fact that is likely continuation of u does not necessarily imply that u and u have the same meaning. Our method circumvents these problems: rather than computing M (ulu), M,(() M (1lv) on a common set of continuations E Tu T., This strategy is arguably more natural and also, as our experiments will demonstrate, much more effective. While our notions of semantic similarity are defined from the perspective of language models, our experiments in Section 4 suggest that they increasingly align with that of human annotators model size and training data increases, and vastly outperform that of next-token/iikllioodd baselines.
compare the values of (t) (t]u)) and
Algorithm Similarity in Meaning Space
Require: Model M, Strings u and v, num. trajectories n, max trajectory length m, distance d TN Sample trajectories from up [EOS] or length m, whichever occurs sooner
T, + Sample n trajectories from to [EOS] or length m, whichever occurs sooner
Initialize M, = M,- = 0
for ? = am.. E T. T. do
b> Compute trajectory likelihood
M, [t] + |=1
mic
PM (a: uu
0i a;-1) 1 1/me
[. + I Par (a, ai a;-1) 1/ml
end for return d(M,, M.)
Return similarity score
Containments of semantic representations. Our representations M, belong the space of functions [0, 1]-4", which we can view as the "meaning space" for the vocabulary A. Note that this space has natural partial order: given M, E [0, 1]4", we say that M < N if E A we have (tt < (t),, which intuitively means that any feasible sentence for M is also feasible for N. More generally,
can define operations of meet and join as M A N min min(M, .N and M V N := max(M, N), respectively. These Boolean-like operations on meanings can be used to investigate more complex (even asymmetric) meanings relationships, in addition to similarity.
5

---

These definitions require using unnormalized scores, which is why we consider [0, 1| instead of the set of probabilities over AA as our meaning space. Note that, in contrast, traditional vector-space embeddings are ill-suited for representing such relationships.
In our experiments, we explore how this sort of (syntactic) meaning containment is related to en- tailment (=) in the conventional sense. As we discuss in Appendix F, given two sentences and such that u = 2, the relation M, < M,, is "more true" than M, < M,, Note that, for our particular score representation in eq. (I), neither My < My. nor M, M, can hold exactly; however we can quantify how far they are from being true. Based on this, we define the Entailment Test: If d(M, A M,, M.) < d(M, Me. Ma). then + v; otherwise, = u.
Semantic representation for substrings. The meaning representation M, for string considered until now assumes that is used as prompt, i... as a prefix within a longer string. We can also modify our definition to account for strings in any position, and in particular to words. Specifically, for any string u, we consider a meaning representation M, : A x A" - [0, 1] defined by:
Mu(s,t) == (sstt).
Intuitively, the meaning of a word/string is the likelihood function of it appearing in between all "contexts" (s, t) very natural idea in distributional semanties, resembling for example the skip- gram model used in word2vec (Mikolov et al., 2013).
Using this representation, above for prompts. However, unlike the previous setting, sampling the support of M, or M. (con- texts that contain u and v) is not trivial, since LLMs can only sample "forward" trajectories. In practice, we circumvent this issue by using a text corpus, WikiText (Merity et al., 2016), to re- trieve, rather than sample, paragraphs containing the given word to use context. In Section 4, our experiments show that the partial ordering in semantic space aligns quite well with "meaning con- tainment" in natural language. i.e., with hyponym/hypemyym relations defined in WordNet (Miller, 1995). Specifically, if v is a hyponym of u, then it is natural to expect that M. M, (see Ap- pendix FF for a justification). Thus, given two words (u, u) between which hyponym relation exists, we define the Hyponym Test: If d(M, A M. M,) < d(M, a M., M,,MM M.), then is a hyponym of u; otherwise, is a hyponym of v. We refer to Algorithm 2 in the Appendix for full details.
can define partial ordering of meanings in the same way considered
Semantic similarity for different modalities. The meaning representations we consider are appli- cable to any model that assigns likelihoods to sequences of tokens. In particular, they can be ap- plied without modification to multimodal autoregressive models which accept both image and text prompts. In Section 4. we show how meaning representations obtained from the multimodal model LLaVA (Liu et al., 2023) can effectively compute semantic image-text and image-image distances.
4 EXPERIMENTS
Implementation details. Apart from adding a full stop (".") at the end of each sequence that does not already end with punctuation complete the sentence, we evaluate each dataset verbatim (in Table 6 in the Appendix. we show results obtained without this step). For our baseline methods, we report the best result with or without adding a full stop, to ensure fair comparison. For experiments on LLaVA (Liu et al., 2023), we use the default query format to structure the input data. We do not apply any additional prompts/formattigg for all other models unless otherwise mentioned. We use eq. (2) as our distance function. We report results using other metrrissdiverreenes in the Appendix. We use multinomial sampling for all experiments on our method with sampling temperature À = 1.0. We set n 20 and 20 for sampling trajectories, based on ablations in Appendix A. Distance metric and hyperparameter choices for semantic similarity are based on a search using the validation set of the STS-B dataset, and are then fixed when evaluating all test datasets.
Evaluation procedure. We evaluate our method on the following tasks:
Semantic Textual Similarity (STS) (Agirre al., 2012; 2013; 2014; 2015; 2016; Cer et al., 2017): The STS dataset scores how similar two pieces of texts are. We use the Spearman coefficient (scaled by 100x) to evaluate correlation with the human-annotate
similarity scores.
Stanford Natural Language Inference (SNLI) (Bowman et al., 2015): SNLI labels pairs of strings based on the categories (entailment, neutral, contradiction). The latter two are symmetric and can

---

Table 1: Comparison with other prompt-free and zero-shot methods on Semantic Textual Similarity benchmarks. *; indicate results taken from Ni et al. (2021); Gao et al. (2021) respectively. Our method outperforms all baselines, and even mcoder-based methods like ST5-Enc-ment (11B). As model size scales, our method approaches the paragon of contrastive-trained models, even though models we use have been trained only on unsupervised next-token prediction.
STS-B STS12 STS13 STS14 STSI5 STS16 SICK-R Avg
Paragon: Contrastive-Trained Trained Models
CLIP- CLIP-VIT114 (Radford et al., 2021)
63.5
67.7
68.5
58.0
67.1
73.6
68.6
67.0
IS-BERT (Zhang al., 2020)
56.8
69.2
61.2
75.2
70.2
69.2
64.3
66.6
SimCSE-BBe BERT (Gao et al., 2021)
68.4
82.4
74.4
80.9
78.6
76.9
72.2
76.3
Zero-Shot Encoder-based Models
BERT-CLS' (Devlin al., 2018)
16.5
20.2
30.0
20.1
36.9
38.1
42.6
29.2
BERT-mean" (Devlin al., 2018)
45.4
38.8
58.0
58.0
63.1
61.1
58.4
54.8
BERT Large-mean" (Devlin et al., 2018)
47.0
27.7
44.5
51.7
61.9
53.9
48.9
RoBERTa Large-mean" (Liu et al., 2019)
50.6
33.6
57.2
45.7
63.0
61.2
58.4
52.8
STS-Enc-firsh (Base)" (Ni et al., 2021)
16.7
17.5
6.3
-20.7
2.3
21.9
28.6
10.4
ST5-EncDec- first (Base)* (Ni et al., 2021)
9.4
10.9
29.6
14.9
28.9
30.6
39.3
23.4
STS-Enc-mean (Large)"
2021)
56.3
28.0
52.6
41.4
63.6
59.5
51.8
ST5-Enc-mean (11B)* (Ni et al., 2021)
35.0
60.2
47.6
66.4
70.6
63.6
58.0
Autoregressive Model Baselines: Falcon-7B
Cross Encoder (Muennighoff, 2022)
46.7
25.1
53.9
41.9
53.7
54.2
57.2
47.5
Joint Likelihood
38.1
40.8
32.7
35.7
47.6
33.5
Last token
23.1
27.0
20.1
8.5
18.7
18.3
40.8
22.4
Mean token
18.8
18.0
25.9
18.5
25.8
27.5
37.3
24.5
Autoregressive Models (Ours)
Ours (GPT-2)
55.2
39.9
42.6
30.5
52.4
62.7
62.0
49.3
Ours (GPT-2-XI.)
62.1
43.6
54.8
37.7
61.3
68.2
68.4
56.5
Ours (Falcon-7B)
67.7
56.3
66.5
53.0
67.4
75.5
73.5
65.7
Ours (LLaMA-13B)
70.6
52.5
65.9
53.2
67.8
74.1
73.0
65.3
Ours (LLaMA-33B)
52.5
54.6
75.2
73.0
66.6
be quantified via similarity. To evaluate our method's ability to compute asymmetric relationships, we restrict SNLI to only pairs of sentences labelled with the "entailment" relation. We express this as a binary classification task determine the direction of entailment, i.c., given pair (u, v), wish to determine if u + v, or + u. We term this resultant task SNLI-Entailment,
WordNet (Miller, 1995): WordNet establishes hierarchy among English words through semantics- based hypernym/hyponym relations. We sample branches from the WordNet hierarchy (see Ap- pendix C.1), and recover their pairwise relations using operations syntactic meaning space.
Crisscrossed Captions (CxC) (Parekh et al., 2020): CxC extends MS-COCO (Lin et al., 2014) with numan-labellee semantic similarity scores ranging from 0-5 for image-image, caption-caption and image-caption pairs. Since most scores are close to 5 (e.g. original image-caption pairs from COCO) for which ranking comparisons would be vacuous, we subsample balanced subset 1000 pairs each from the image-image (CxC-SIS) and image-caption (CXC-SITS) dataset for our experiments.
Semantic similarity. Our main baselines for comparison are methods which are 1) zero-shot, and 2) prompt-free. As such, compare our method presented in Algorithm against encoder-based models, and the following baselines for autoregressive models given a pair strings (u, v):
Conditional Likelihood Pross-Encoder (Muennighoff, 2022): computes Mulu).
Joint Likelihood: measures the likelihood of the concatenation of and v, MM (uv) M (wvle) where e is begin-of-sentence token [BOS], normalized by number of tokens. If [BOS] is not sup- ported by the model, we the M (u C, ... v1) instead where v (U,,.. (..
3. (Last/Mean) Token: represent and using the model's output distribution for the next
token immediately following the sentence (last) or the average next-token predictions over the input sentence (mean), and produce similarity score via cosine similarity.

---

22.32
05.10
70-
65.
62:45
4154
40-
30--
20-
Besee
Jon Lawibedd
Gool Lhaaked (Pacos 78)
70)
dee FPP (CPP-))
chn (aatoe- 19)
den 180)
Random
Our (GPP2)
Ours com3-
fatoon
Durk GLaMA 158)
(c)
Figure 2: (a) Accuracy in inferring the entailment direction. On SNLI-Entailment, our method outperforms existing baselines applied to the best model, Falcon-7B, showing that our notion of meaning containment aligns with that of natural language when quantifying "entailment" relation- ships between statements. (b) Accuracy in inferring hypernym/hyponym direction. Our method performs significantly better than chance WordNet hyponym/hypernye prediction. (c) Visual- ization of word hierarchy recovered by our method on subset of words using Falcon-7B (red indicates predictions that differ from the WordNet ground-truth).
On the Semantic Textual Similarity benchmark, Table shows that our method uniformly outper- forms all baselines on one of the best autoregressive models, Falcon-7B, by minimum relative improvement of 38.3%. Even when applied to GPT-2, a much smaller model, our method improves over Falcon-7B baselines by 3.8%. While our method is expectedly outperformed by models which are explicitly fine-tuned on contrastive-learning objectives, such as SimCSE (Gao et al., 2021), it performs comparably CLIP (Radford et al., 2021), and when applied to Falcon-7B and LLaMA- 33B respectively, significantly outperforms the best zero-shot encoder- based model (ST5-Enc Enn--maan IBB) by relative margin of 13.3% and 14.8%. We achieve this without any fine-tuning or prompt- ing. We further highlight that our results do not rely any pairs, since the models we use have been trained only on unsupervised next-token prediction.
data or contrastive
Lastly, our method shows an improvement in performance that correlates with model size, suggest- ing that further performance gains could be obtained larger/better autoregressive models are used. Our results also suggest that the proposed method can be used to evaluate pre-trained models in zero-shot manner without requiring instruction-tuning or RLHF, since their alignment with human labelers seems to correlate with human perception of how good model is.
Entailment via meaning containment. We show accuracies obtained on SNLI-Entailment in Fig- ure 2a when applying the Entailment Test described in Section 3. We compare against Cond. Likeli- hood (u + if (vlu) > (ulvv) else v = u) and Joint Likelihood (u = v if M(v) > M ((w)) on the best performing model, Falcon-7B 7B. Our results show that the trajectories sampled from all LLMs that we tested align with the assumptions of the Entailment Test with significantly high prob- ability, outperforming both random and likelihood baselines by 15.7% and 9.3% respectively.
Meaning containment of individual words. We apply the above-defined Hyponym Test to recover hypernymhhyooyym relations from WordNet. Our results in Figure 2b and Figure 2c show that the Hyponym Test is mostly able to recover semantic containment relations between words, with an absolute improvement of 12.7% to 18.1% over the random baseline, depending on the model. Note that our computation of the hierarchy is based entirely pairwise comparisons and does not explicitly enforce the transitivity of containments; however, transitivity is almost always already satisfied by the predictions of our method (i.e., the recovered hierarchy is an acyelic graph). We present more qualitative examples in Appendix D.2.
Vision-language experiments. Our method can be applied without any modification to models that accept multimodal token sequences. In Table 2, we apply our method to CxC (Parekh et al., 2020) using the vision-language model LLaVA (Liu et al., 2023) to show that we can measure semantic distances between not only text, but also between image-image (CxC-SIS) and image-text (CxC- SITS) pairs that align with that of human annotators. Our method outperforms all decoder-only baselines on both SIS and SITS. On SIS. our method even outperforms CLIP (Radford et al., 2021) which is trained explicitly contrastive image-text objective.
8

---

Table Image Similarity and Image-Text (Caption) Similarity on balanced subsets of CxC- SIS and CXC-SITS respectively. Even without any prompts, our method outperforms all zero-shot baselines on both modalities. The performance on the image-text similarity can be further boosted with alignment prompt, allowing our method to outperform even CLIP which is explicitly trained with a contrastive objective to output aligned image-text embeddings. For CLIP (Vision), we use image embeddings prior to projection onto text embedding space.
Architecture
Method
CxC-SIS
CXC-SITS
Average
66.33
64.25 64.60
65.29
CLIP-ViTB/I
66.95
65.78
CLIP (Radford et al., 2021)
CLIP-ViTI/14 (Vision)
71.45
(Vision)
72.08
Cond. Likelihood
29.46
Mean Token
32.76
-0.52
16.12
LLaVA (Liu et al., 2023)
Last Token
26.91
2.43
14.67
Ours
81.47
57.14
69.31
Mcan Token (Prompt)
32.76
-0.07
16.35
LLaVA (Liu et al., 2023) w/ Alignment Prompt
Last Token (Prompt)
26.91
6.21
16.56
Ours (Prompt)
81.47
67.63
74.55
We highlight that while the "Cond. Likelihood" baseline SITS should most directly capture M (caption[image) our experiments show that it fares poorly compared to our method. We hy- pothesize that this results from the limitations of perplexity. For instance, likelihood scores directly compared across captions of various lengths, for which length normalization does not suf- ficiently mitigate the bias towards shorter sentences (Wang al., 2022). Our method avoids this issue entirely by construction, since we compare distributions across the same set of trajectories. We can optionally make use of "alignment prompts" to ensure that the trajectories from image and text modalities are more similar. This improves the resulting performance on the CXC-SITS task, outperforming the CLIP paragon by 13.3% (Table 2). We discuss this in Appendix E.2.
CONCLUSIONS
We proposed a strategy to investigate how autoregressive language models interpret text. By identi- fying "meaning" - from the perspective of the model with score distributions over text continu- ations, we can compare the meaning of arbitrary strings. This notion of meaning correlates with that of human annotators, outperforming comparable zero-shot and prompt-free baselines on semantic textual similarity tasks using the same architectures. We further defined composition operators on meanings and showed how autoregressive language models can be used to quantify entailment be- tween sentence pairs and hyponym relations between individual words. Our method can further be applied without modification to autoregressive vision-language architectures to define and compute meaning representations of images, outperforming even CLIP semantic image similarity tasks.
A key limitation of our approach its computational cost compared to embedding methods that require only single forward pass. However, our ablations in Appendix A show that that using 10- 20 trajectories of 10-20 tokens each is sufficient to achieve most of the performance gain compared using more trajectories or tokens. We also note that both the sampling and score evaluation processes can be easily parallelized. Our approach in its current form is also not computationally efficient for semantic search, since the computation of pairwise similarities between queries and database elements is performed using different set of trajectories for each new query. We explore ways to mitigate this in Appendix A.4 and the potential performance trade-offs that they incur.
Our method is intentionally prompt-free, our goal in this work was to define the most canonical meaning representation of string for given model. Nevertheless, our experiments in Appendix strongly suggest that designing appropriate "alignment" prompts could further significantly improve quantitative results on semantic similarity tasks. Our method can also be generalized to compare semantic distances between autoregressive models from the same family of architectures sharing common vocabulary, since their meaning representations belong to the same space. We leave these directions for future work.

---

REFERENCES
Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. Semeval-2012 task 6: A pilot on semantic textual similarity, In * SEM 2012: The First Joint Conference on Lexical and Computa- tional Semantics-Volumey 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), pp- 385-393, 2012.
Eneko Agirre, Daniel Cer, Mona Diab, Aitor Monzalez-Aginta and Weiwei Guo. sem 2013 shared task: Semantic textual similarity. In Second joint conference on lexical and computational se- mantics (* SEM), volume I: proceedings of the Main conference and the shared task: semantic textual similarity, pp. 32-43, 2013.
Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Wei- wei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. Semeval-2014 4 task 10: Multilingual semantic textual similarity. In Proceedings of the 8th international workshop semantic evalu- ation (SemEval 2014), pp- 81-91, 2014.
Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre Wei- wei Guo, Inigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, al. Semeval-2015 task 2: Semantic textual similarity, english, spanish and pilot on interpretability. In Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pp. 252-263, 2015.
Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez Agirre, Rada Mihalcea, German Rigau Claramunt, and Janyce Wiebe. Semeval-2016 task I: Semantic textual similarity, monolingual and cross-lingual evaluation. In SemEval-2016. 10th International Workshop on Semantic Evaluation; 2016 Jun 16-17, 17; San Diego, CA. Stroudsburg (PA): ACL; 2016. p. 497-511. ACL (Association for Computational Linguistics), 2016.
Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Co- jocaru, Merouane Debbah, Etienne Goffinet, Daniel Heslow, Julien Launay, Quentin Malartic, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. Falcon-40B: an open large lan- guage model with state-of-the-ee performance. 2023.
Emily M Bender and Alexander Koller. Climbing towards nlu: On meaning, form, and under- standing in the age of data. In Proceedings of the 58th annual meeting of the association for computational linguistics, pp. 5185-5198, 2020.
Emily M Bender, Timnit Gebru, Angelina McMillan- Majorr and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp. 610 623, 2021.
Gemma Boleda. Distributional semantics and linguistic theory. Annual Review of Linguistics, 6: 213-234, 2020.
Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher Manning. A large anno- tated corpus for learning natural language inference. arXiv preprint 15Xiv:1508.0000 05326, 2015.
Tai-Danae Bradley, John Terilla, and Yiannis Vlassopoulos. An enriched category theory of lan- guage: from syntax to semantics. La Matematica, 1(2):551-580, 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 13:187771901, 2020.
Daniel Cer, Mona Diab, Eneko Agirre, Inigo Jopez-Gazpio, and Lucia Specia. Semeval-2017 task 1: Semantic textual similarity-mutiiinnnnn and cross-lingun focused evaluation. arXiv preprint arXiv: 7708. 00055, 2017.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, lon Stoica, and Eric P. Xing. Vicuma: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https: //1msys..rg//lll/
10

---

Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint MrXiv:8110004805 2018.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simcse: Simple contrastive learning of sentence 2021.
embeddings. arXiv preprint
Zellig S Harris. Distributional structure. Word, 10(2-3):146-16) 1954.
John E. Hoperoft, Rajeev Motwani, and Jeffrey D. Ullman. Introduction to automata theory, lan-
guages, and computation. Pcarson/Addison Wesley, Boston, 3rd ed edition, 2007. ISBN 978-0- 321-45536-9 118-00331--462555 978-0-321-45555 OCLC: ocm69013079.
Bart Jacobs. Introduction to coalgebra. Towards Mathematics of States and Observations, Version, 2, 2012.
Ting Jiang, Jian Jiao, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haizhen Huang, Denvy Deng, and Qi Zhang. Promptbert: Improving bert sentence embeddings with prompts. arXiv preprint (tIv:2201.03337 2022.
Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang. Scaling sentence embeddings with large language models. arXiv preprini arXiv:2307. 16645, 2023.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint ArXv::2304. 008488200 2023.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: robustly optimized bert pretraining approach. arXiv preprint 1907 1692, 2019.
Clara Meister and Ryan Cotterell. Language model evaluation beyond perplexity. arXiv preprint arXiv:2106.0000 2021.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016.
William Merrill, Yoav Goldberg, Roy Schwartz, and Noah Smith. Provable limitations of acquir- ing meaning from ungrounded form: What will future language models understand? Transactions of the Association for Computational Linguistics, 9: 9:1047-1060, 2021.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. In Advances in neural information processing systems, pp. 3111-3119, 2013.
George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11): 39 41, 1995.
Niklas Muennighoff. Sgpt: Gpt sentence embeddings for semantic search. arXiv preprint arXiv:2202.08904 2022.
Jianmo Ni, Gustavo Hernández Ábrego, Noah Constant, Ji Ma, Keith B Hall, Daniel Cer, and Yin- fei Yang, Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. arXiv preprint arXiv:2108.0885 08877, 2021.
OpenAl. Gpt-4 technical report, 2023.
Juri Opitz and Anette Frank. Sbert studies meaning representations: Decomposing sentence embed- dings into explainable semantic features. In Proceedings of the 2nd Conference of the ssia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Con- ference Natural Language Processing, pp. 625-638, 2022. 2022
11

---

Zarana Parekh, Jason Baldridge, Daniel Cer, Austin Waters, and Yinfei Yang. Crisscrossed captions: Extended intramodal and intermodal semantic similarity judgments for ms-coco. arXiv preprint 15Xi: 2020.
Jean-Eric Pin. Mathematical Foundations of Automata Theory. 2022.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748 9748-8763. PMLR, 2021.
Nils Reimers and Iryna Gurevych. networks. arXiv preprint arXiv: 1908: 10084, 2019.
Sentence-bert: Sentence embeddings using siamese bert-
Magnus Sahlgren. The distributional hypothesis. The Italian Journal of Linguistics, 20:33-54, 2008. URL httph:ttp/s//...t //ppi sema nt www.cccclllacc ar org/ Corrpuustii ID: 2375::2::000 23750999.
Stefano Soatto, Paulo Tabuada, Pratik Chaudhari, and Tian Yu Liu. Taming AI Bots: Controllability of Neural States Large Language Models, May 2023. URL http:://rxttt org/ abs/ 2305. 18449. arXiv:2305. 18449 [cs, eess].
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint 27X71 2023.
Yequan Wang.
Jiawen Deng, Aixin Sun, and Xuying Meng. Perplexity from plm is unreliable for evaluating text quality, arXiv preprint arXiv:2210.0580 2022.
Ludwig Wittgenstein. Philosophical Investigations. Macmillan Publishing Company, edition, 1953.
Zhaofeng Wu, William Merrill, Hao Peng, Iz Beltagy, and Noah A Smith. Transparency helps reveal when language models learn meaning. Transactions of the Association for Computational Lingnistics, 11:6176634, 2023.
Yan Zhang, Ruidan He, Zuozhu Liu, Kwan Hui Lim, and Lidong Bing. An unsupervised sentence embedding method by mutual information maximization. arXiv preprint arXiv:2009, /2061, 2020.
12

---

Supplementary Material
A ABLATION STUDIES
In this section, we present ablation studies how trajectories sampled in Appendix A.1, choice of distance function in Appendix A.2, completing "incomplete" sentences with a single full stop in Appendix A.3, and discuss extensions perform computationally efficient semantic search in Appendix A.4.
A.1 ABLATION ON TRAJECTORIES
We present ablations on Algorithm investigate the impact of (1) number of trajectories (2) length of trajectories and (3) sampling temperature. All experiments are done on the validation set of STS-B instead of test set to avoid over-fitting results the test set.
Figure 3(a) shows that performance on evaluating semantic similarity increases with both number and length of trajectories sampled, the cost computational time. We use m 20 for all of our main experiments, which is sufficient to yield most of the performance. We also ablate of sampling temperature À in Figure 3(b), where we show that sampling trajectories that are either too diverse or lack diversity (as measured by A) tends to harm performance. Instead, the standard temperature value 1.0 yields the best results.
Spearman Corr (x100)
Spearman Corr (x100)
70
60
60
50
50-
40
m=1
40
20:
m-20
10
m-40
15
20
35
2.0 5.0
10.0
Number of Trajectories (n)
Temperature (A)
Figure 3: Ablation over maximum length (M), number (N) of trajectories, and sampling temperature (A) on STS-B validation dataset using the Falcon-7B model. While only small number of short trajectories is sufficient to yield good results, performance semantic similarity generally increases with both number and length of trajectories. Too much diversity and lack of diversity in the sampled trajectories both harm performance, shown by higher and lower values of À respectively.
A.2 CHOICE OF DISTANCE FUNCTION
We further ablate the choice of distance function in Table 3. For distance functions proba- bility spaces (Hellinger, Total Variation, Symmetric KL-Divergence), we normalize the scores M, using
M,(t)
M (t) = Erea M,(t")+
(3)
to convert them into a probability distribution summing to We use T 0.5 which we experimen- tally determined to perform best. We compare against our choice of distance function defined in eq. (2), and modified version that uses L2 instead of LI loss, which we refer to as Log-LI and L2 respectively. We show that most choices of distance functions (Symmetric KL Divergence, Hellinger distance, Log-L2, Log-L1) work reasonably well for computing the semantic distance between strings. We chose Log-L.1 in main experiments, which performs best.
13

---

Table 3: Ablation over the choice of distance function LLaMA-13B and LLaVA. For Hellinger distance, Total Variation (TV), and Symmetric KL--Divergence, we set = 0.5 as in eq. (3). Here we use the prompt-aligned version of our method for SITS.
Metric
STS-B
STS-12
STS-13
STS-14
STS-15
STS-16
SNLI
SIS
SITS
Hellinger
69.7
53.0
65.5
50.7
65.6
71.2
65.8
81.3
67.6
TV
51.2
40.9
47.8
32.8
42.4
50.9
64.1
78.5
62.4
Sym-KL
69.7
52.9
65.4
50.6
65.5
71.1
65.9
80.9
67.6
Log-L1
70.6
52.5
65.9
53.2
67.8
74.1
63.6
81.0
67.6
Log-L2
69.2
51.4
64.1
48.2
65.7
71.6
65.4
80.8
67.0
A.3 COMPLETING THE SENTENCE WITH FULL STOP
According to our definitions, the meaning representations of complete and incomplete sentences dif- fer, since the distributions over their trajectories are likely to very different. To this, consider the following pair of semantically similar sentences that differ by the last punctuation: "The dog ate the bone" and "The dog ate the bone.". The continuations the latter are likely to start with capital letter, but this does not hold for the former. Hence, our method is likely to attribute larger distances between these two sentences than humans. In Table 4, we show that by ensuring all sentences we compare are complete, by appending full stop when necessary, the similarity scores computed between sentences align better with that human annotators. We also show that completing the sentence occasionally improve results for certain baselines well.
A.4 EXTENSION TO SEMANTIC SEARCH
computationally expensive for semantic search, where we wish to retrieve the most similar sample in search database D given query g, since it requires multiple sampling and forward pass operations for each pairwise comparison d(g, s) for all s E D. As such, an inference cost of O(DD) is incurred each time new query is received. This holds true even for previously proposed methods for semantic search using decoder-only models, e.g., (Muennighoff,, 2022). Instead, if there exists a fixed set of trajectories TD for the search database D that can be used instead of A" in eq. (1), then M, for each item E 'D can be pre-computed beforehand, incurring one-time cost of O(D)). Hence, for each subsequent query, only need to incur ((1) inference cost to compute M, on Tp. This can be compared against the pre-computed embeddings D using standard distance functions such as LI. We present proof-of-concepl experiment on the STS-B B validation dataset to observe the trade-off in performance that this incurs in Table 5, where we obtain TD by naively selecting n examples from the dataset uniformly at random, from each of which we generate single trajectory. Nevertheless, indeed possible to achieve satisfactory performance using fixed sets of trajectories. We leave investigating more sophisticated methods to construct Tp for future work.
We note that Algorithm
preliminary results demonstrate that
B FURTHER BASELINES
We provided baseline comparisons in Table of the main body of the paper against one of the best model tested, Falcon-7B. In Table 6, we provide additional baseline results for several other autoregressive architectures used.
C ADDITIONAL IMPLEMENTATION DETAILS
We use the base GPT-2, GPT-2-XL (Radford et al., 2019), LLaMA-13B (Touvron et al., 2023) and Falcon-7B (Almazrouei et al., 2023) for experiments models trained with unsupervised pre-training objectives. We use Vicuna-13B (Chiang et al., 2023) and Stable Vicuna-133 as the instruction-tuune version and the Rriieer (reinforcement learning with human feedback) ver-
3b-delta
14

---

Table 4: Ablation over adding full stop (FS) incomplete sentences. Based on our definitions, meaning representations of complete and incomplete sentences differ. We show that the meaning similarities of complete sentences align better with that of human annotators.
Method
FS? STS-B STSI2
STSI3 STS14 STSI5 STSI6 SICK-R Avg
Baselines (Falcon-7B)
Cond. Likelihod
46.0
22.3
52.5
41.7
46.9
51.8
54.3
45.1
Joint Likelihood
38.3
4.5
38.3
32.4
28.3
34.4
43.3
31.4
Last token
24.9
18.9
13.6
4.2
4.7
18.5
34.1
17.0
Mean token
18.8
18.0
25.9
18.5
25.8
27.5
37.3
24.5
Cond. Likelihood
46.7
25.1
53.9
41.9
53.7
54.2
57.2
47.5
Joint Likelihood
38.1
6.0
40.8
32.7
33.7
35.7
47.6
33.5
Last token
23.1
27.0
20.1
8.5
18.7
18.3
40.8
22.4
Mean token
18.1
21.7
25,4
16.9
26.3
26.6
33,9
24.1
Baselines (LLaMA-13B)
Cond. Likelihood
41.9
19.8
54.6
40.1
54.6
52.2
55.0
45.5
Joint Likelihood
X
36.6
-0.6
34.8
27.8
28.2
32.6
43.2
28.9
Last token
18.8
15.9
18.2
5.6
2.3
9.9
35.6
15.2
Mean token
28.0
22.0
27.5
19.6
30.8
35.8
29.6 43.7
Cond. Likelihood
44.3
20.8
51.8
38.6
56.0
50.9
56.7
45.6
Joint Likelihood
36.7
1.1
35.0
27.7
33.0
32.4
48.0
30.6
Last Token
18.2
24.2
29.0
16.8
21.9
10.2
40.8
23.0
Mean Token
28.8
25.2
30.2
20.2
31.5
35.1
45.0
30.9
Ours
Ours (GPT-2)
48.3
28.7
39.7
23.8
35.7
60.0
56.6
41.8
Ours (GPT-2-XL)
56.8
32.5
49.5
29.1
45.2
66.0
63.1
48.9
Ours (Falcon-7B)
67.2
44.0
62.1
44.7
57.5
76.1
69.3
60.1
Ours (LLaMA-13B)
66.9
39.9
61.2
45.0
56.4
74.4
68.7
58.9
Ours (GPT-2)
55.2
39.9
42.6
30.5
52.4
62.7
62.0
49.3
Ours (GPT-2-XL)
62.1
43.6
54.8
37.7
61.3
68.2
68.4
56.6
Ours (Falcon-7B)
67.7
56.3
66.5
53.0
67.4
75.5
73.5
65.7
Ours (I..aM--13B)
70.6
52.5
65.9
53.2
67.8
74.1
73.0
65.3
Table 5: Trade-off in performance STS-B validation set from using fixed set of trajectories
(m = 20) for all pairwise distance comparisons.
Method
Spearman Corr (x100)
Ours (Falcon-7B)
74.74
Fixed Traj (n 20)
49.41
Fixed Traj (n = 40)
53.29
sion of LLaMA-13B respectively. We use LLaVA! (Liu et al., 2023) for our multimodal experi- ments, which trained accept both image and inputs.
Technically, computing distances with Equation (2) when compositional terms are involved would require sampling trajectories from the composed distributions. In particular, evaluating d(M, M., M,) in the Entailment test would require sampling trajectories Tune from the composed dis- tribution M,, A M,, then approximating Equation (2) with the set trajectories Tuxe L Ty, For the sake of simplicity and computational efficiency, we instead compute M,, A M, over the set of trajectories T, T. sampled from u and v, which we empirically found be similarly effective when applied to downstream tasks.
15

---

Table 6: Comparison our method against baselines (best among with/without fullstop) for each model architecture on STS tasks.
Model
Method
STS-B
STS12
STS13
STS14
STSI5
STSI6 SICK-R Avg
Cond. Likelihood
37.9
28.6
39.1
34.3
50.5
53.0
Joint Likelihood
27.9
18.4
22.0
23.1
32.8
27.2
44.0
27.9
GPT-2
Last token
27.7
8.4
23.0
10.5
31.0
26.6
41.9
24.1
Mcan token
20.4
17.0
22.0
17.6
36.2
31.5
38.4
26.2
Ours
55.2
39.9
42.6
30.5
52.4
62.7
62.0
49.3
Cond. Likelihood
40.3
23.8
43.2
33.6
51.3
48.7
55.0
42.3
Joint Likelihood
31.4
13.3
29.3
23.8
35.2
28.6
46.0
GPT-2-XL
Last token
24.1
-5.9
20.9
5.0
25.6
21.4
40.8
18.8
Mean token
21.1
13,0
28.0
16.7
34,9
33,0
37.8
26.4
Ours
62.1
43.6
54.8
37.7
61.3
68.2
56.6 68.4
Cond. Likelihood
46.7
25.1
53.9
41.9
53.7
54.2
57.2 47.5
Joint Likelihood
38.1
6.0
40.8
32.7
33.7
35.7
47.6
33.5
Falcon-7B
Last token
23.1
27.0
20.1
8.5
18.7
18.3
40.8
22.4
Mcan token
18.8
18.0
25.9
18.5
25.8
27.5
37.3
24.5
Ours
67.7
56.3
66.5
53.0
67.4
75.5
65.7 73.5
Cond. Likelihood
44.3
20.8
51.8
38.6
56.0
50.9
56.7 45.6
Joint Likelihood
36.7
1.1
35.0
27.7
33.0
32.4
48.0
30.6
LLaMA-13B
Last token
18.2
24.2
29.0
16.8
21.9
10.2
40.8
23.0
Mean token
28.8
25.2
30.2
20.2
31.5
35.1
45.0
30.9
Ours
70.6
52.5
65.9
53.2
67.8
74.1
73.0
Cond. Likelihood
31.4
21.5
41.5
35.3
38.8
38.3
56.3
37.6
Likelihood
36.2
4.9
35.6
27.7
30.3
32.3
47.8
30.7
LLaMA-33B
Last token
21.8
20.1
13.2
9.4
22.5
11.5
40.8
19.9
Mean token
27.9
24.0
29.6
21.7
35.4
34.6
43.5
31.0
Ours
71.5
52.5
70.6
54.6
69.1
75.2
73.0
66.6
Vicuna-138 133
Ours
70.2
53.4
62.4
52.0
68.5
73.9
75.3 65.1
Vicuna- 13B
Ours
70.5
56.2
63.9
52.5
67.9
74.8
75.3 65.9
For experiments on WordNet hyponyn/hypermym relations, leverage the WikiText (Merity al., 2016) corpus by sampling up to n 100 contexts (i.e. paragraphs in the WikiText dataset) contain- ing each given word.
C.I WORDNET HYPONYM/HYPERNVMM SUBSET:
For our experiments, we use total of 166 pairwise hyponym/hypemnym relation between the following sets sampled from WordNet (Miller, 1995), enumerated in order of meaning containment:
1. {puppy, dog, canine, carnivore, predator, animal, organism}
2. {storybook, book, publication, work, product, creation, artifact)
3. (dine, eat, consume}
4. (soar, fly, travel)
5. (chuckle, laugh, express emotion)
6. (bobcat, lynx, wildeat, cat, feline, carnivore)
7. (penthouse, apartment, housing, structure]
8. (recliner, armchair, chair, seat, furniture, furnishing, instrumentality)
9. (neurosurgeon, surgeon, doctor, medical practitioner, professional, adult, person)
10. (brunch, meal, food, substance)
11. (hydrofoil, speedboat, motorboat, boat, vessel, craft, vehicle, conveyance)
16

---

12. {
consult, research, investigate, analyze
13. { symposium, conference, meeting, gathering)
14. { hacker, programmer, engineer, person]
C.2 HYPONYM TEST
detail the Hyponym Test for quantifying meaning containment between words in algorithm
Algorithm 2 Hyponym Test (Words)
Require:
Model M, Words and v, number of trajectories distance d, Text Corpus D corpus
Tu {81, 4- == < Sample n paragraphs (8; 44) containing u from Deorpus
T. [(,,, 1.)) f- Sample paragraphs (s; 0t) containing from D Deorpus
T LT.
Initialize M M = M, A M =
for L (a) ........ Um, bi dmú) Tu U T., do
Mu[t] + (II Par (a: 01 ... a;-1) PMs (ula1 ame)-
II
PM (bila1 am, ub b)-1)) 1/(metm/(1)
Mo (II Par(ala a.-)) (M al ... ame))
PM (bsla1 (b:la1.....e ame bb bz-1)) 1/ 1/mm+t//+1
Muli A M, [t] + (M [t]. M,)
end for
if d(M u. M A M, < d(M,, F. M, A M.) then
return a hyponym of
else
return hyponym of end if
D ADDITIONAL VISUALIZATIONS
D.1 PERFORMANCE SCALES WITH MODEL SIZE
70
Spearman Corr (x100)
60
50
Average SICKR
STS12
40
51514
STS16
30
103
10a
Number of Model Parameters (M)
Figure 4: Plot of performance on semantic textual tasks vs number of model parameters, as measured using GPT-2, GPT-2-XI., Falcon-7B, LLaMA-13B, and LLaMA-33B.
We show Figure 4 that the alignment of our method on semantic textual similarity with human annotators scales with model size.
17

---

D.2 WORDNET HYPONYM//YPERNYNN RELATIONS
In Figure 5, we show further visualizations of the hyponym/hypemym hierarchies established by method on WordNet using Falcon-7B.
structure
ON
Food
prodect
subsiannc
housing
publcction
travel
penthouse
nly
bubcca
brunch
apartment
Sear
Tangh
anumal
zbuckle
vehicle
deg
CEpRco CEnÉiOS
bust
organism
predator
camiisore
speedbont
monorrbead
puppy
hycrofoil
meeting
research
engineer
parson
gathering
consult
Tosical postiione
programizer
conference
investigate
symposium
Ana Tyze
Figure 5: WordNet Hyponym/Hypemyn Relation predictions using Falcon-7B
PROMPTING FOR DOWNSTREAM TASKS
Our definition of meaning in the context of large language is prompt-free, and hence not subject to the drawbacks and variabilities that arise from prompt-enginnering. However, prompts can naturally be used to improve performances on downstream tasks by conditioning the trajectories obtained from input strings.
18

---

Table We implemented Jiang et al. (2023) for LLaMA-13B, and show that prompt-based methods are brittle and generalize poorly to other architectures apart from that which they were tuned on. Nevertheless, we show that prompting, while detracting from retaining our "pure" notion of mean- ing, can be used to further improve our results on downstream semantic textual similarity tasks. We also compare against other prompt-based methods here. For Ours-Prompt-l,, we simply prepend "The meaning of this sentence is: to each input string. For Ours-Prompt we append "TThis sen- tence implics to the end of each input string to condition the set of trajectories towards logical implications, achieving superior results across zero-shot, prompt-based methods. i indicates results taken from Jiang et al. (2023).
Method
STS-B STS12 STSI3
STS14 STSIS STSI6 SICK-R Avg.
Pentrastive-rrannee Models
PromptBERT (Jiang al., 2022)
81.6
71,6
84.6
77.0
84.5
80.6
69.9 78.5
PromptRoBERT (Jiang al., 2022)
81.9
73,9
84.7
77.3
85.0
81.7
69.5
79.2
Autoregressive Models
PromptEOL (OPT-1.3B)
73.2
64.6
79.1
68.5
78.9
78.6
69.4
73.2
PromptEOL (OPT-13B)
70.7
60.2
81.4
67.0
75.5
79.6
66.0
71.9
PromptEOL (OPT-66B)
71.7
55.7
74.6
64.9
72.3
75.2
67.4
68.8
PromptEOL (LLaMA-13B)
63.4
52.3
75.3
64.0
70.5
73.2
60.5
65.6
Ours (LLaMA-133)
70.6
52.5
65.9
53.2
67.8
74.1
73.0
65.3
Ours-Prompt-1 (LLaMA-13B)
72.2
61.6
68.4
66.9
72.7
75.6
76.3
70.5
Ours-Prompt-2 (LLaMA-13B)
81.5
67.9
79.9
75.3
82.9
82.3
74.6
77.8
E. PROMPTING FOR SEMANTIC TEXTUAL SIMILARITY
By implementing an existing prompt-based method (Jiang et al., 2023) on LLaMA-13B, we show in Table 7 that prompt-based methods are brittle and model-specific, hence require careful tuning for each specific architecture. In contrast, our original method prompt-free and robust against such variances arising from prompt-engineering.
Nevertheless, we present some preliminary investigations for augmenting our method with prompts in Table for the STS task. We also compare against existing zero-shot prompt-based methods. We show that prompting can also significantly improve performance over the prompt-free approach for the STS task, by appending "The meaning of this sentence is: to each input string when generating trajectories. We note that we did not carefully search over prompts, and simply tried the first ones (above) that came mind. It is likely that there exist others which work better.
E.2 ALIGNMENT PROMPTS FOR VISION-LANGUAGE MODELS
In the main paper, we presented prompt-free approaches for extracting similarity scores from mul- timodal inputs. However, we note that by our definitions, LLaVA (Liu et al., 2023) does not tech- nically attribute the same meaning to images and captions. We visualize this in Figure 6, where we show that image and caption inputs are continued very differently by the model. For instance, given an image. LLaVA generally attempts generate caption. On the other hand, when given caption, LLaVA simply continues it, often in an unpredictable manner. In spite of this misalignment, there exists sufficient overlap likelihood scores outperform all baselines as observed in Table the main paper.
We demonstrate that a prompt can optionally be used to align the meaning representations for vision and text modalities for the purposes of semantic comparison. We achieve this by conditioning the caption continuations to match the continuations of images. In particular, we append "This caption for image. Describe this image. This image shows" to the text inputs, and "Describe this image. This image shows" to image inputs. Figure (Right) shows that this successfully aligns the trajectories of both modalities. Indeed Table of the main paper shows that this improves the prompt-fre free version of our method on the CxC-SITS task by 18.4%, and outperforms the CLIP paragon by 13.3%.
19

---

Simier Image Caption Pair
Without Alignment Promol
With Aiigiment Prompt
Iraoe Traledories
young Inaaa Toilccatic
sonding amongee
A young tn likal tay, stande amonget
pretumarily
The image shross young buy holding kong un The imape stows yourg and group image,
a small chrss, poss Biy a yourg boy standing
yourg chil saandngg in bemboo
Tout Truncholia
Sap Iatacictia
"trrrggnng pep
young mon standing nent io hand youing mon akanding neot in hand slanding noud hard
prompt
hoiling
Pionot a vil
overy Loader ing standing hard
young holding umbrolla next cattle.
Different Image: Caption Pair
Trojectoiies
Image Trancctotica
The image fustunes a boeed mountal the acloe
DOX and beby sow standing logether
with
grartng
diepbaye sowic
gresey landiccpp feeturing
This image whowe call and a cow ges image lusturea beo catle gresing
lanchcape, plctureqqu
1 Toueccodess
actodie
donkay standing hort of restaumant"
"Aoat recaal back again"
conkay standing outside a restaument could donkay wi which is domesilaaned cnimall donkay starding outsklo restarient The dor" metaurant!
paaning of donkeyy eownding
donidey
sonkay standdnng
A donkey standing in front of restaurant.
Figure 6: A prompt can be used to align the meaning representations (i.e., distribution over trajec- tories) for vision and text modalities to measure image-caption similarities. We obtain trajectories on the right by appending "Describe this image. This image shows" to image inputs, and appending "This is a caption for an image. Describe this image. This image shows" to caption text inputs.
F MEANING CONTAINMENTS
In this section, we discuss how the partial ordering defined our meaning representations is related to entailment (=) between statements and to hyponym/hypernym relations between words.
F.1 ENTAILMENT TEST
In the main body of the paper, we claimed that if u > v then M, S M2 is more natural than M, Me. Our empirical experiments indeed show that our Entailment Test succeeds significantly more often than chance. Intuitively, this means that if u = v then more continuations for e are feasible continuations for u, instead of the other way around.
To understand why this is the case, we consider the sets Ca, C. of consequents of u and U. that is, the set of sentences t such that entailment we have C. C C. (since + 1 implies u + l). Thinking M,, and M, as sets, then if M,, C, and M,- C. were true (i.e., if the of feasible continuations coincided with the set consequents), then this would justify our claim. In practice, continuations and consequents do not coincide, especially because some continuations are not consequents. However, is generally true that consequents are valid continuations- so approximately C. C. Ma. C, C. M, M,-and overall consequents seem sufficiently frequent as continuations to dictate the containment relation among general continuations.
t respectively. If u = v, then by transitivity of
To make this argument more concrete, consider the sentences u. ="oody, the neighbor's dog, is barking." and = "A dog is barking.", so that + v. If t is a continuation of v, then & could in general be
a consequent of v (v + t) for example "Therefore, can't 't sleep."
an antecedent of v + v) for example t "Indeed, there a cat.".
non-comparable with (neither v +> nor 1 = hold) for example, "The dog is brown." or "The dog 's name is Spot".
From these examples, we see that: 1) consequent continuations for e are also valid continuations for w; 2) antecedent continuations are somewhat unnatural and likely uncommon; 3) non-comparable
20

---

continuations of may may not be valid continuations of Overall, if we think of consequent continuations as the default, then we expect the containment direction M, I My to hold more than M, M,
F.2 HYPONYM TEST
In the paper, we claimed that if e: is hyponym of then M. S M,, is more natural relation than M, I M,. Our experiments indeed suggest that M. S M occurs significantly more often than chance. Intuitively, this means that if is a hyponym of then can be substituted with more often than the other way around.
To investigate why this is the case, distinguish between two types of usages of common noun v (e.g., "dog"):
Definite reference: when e refers to a specific instance or set of instances the noun, for example "The dog is burking."
Generic reference: when refers to all instances of the noun, for example "Any dog is an animal".
Now, if is sentence that hypernym in s ("The dog is barking." "The animal is barking.") "). In contrast, if uses v with generic reference, then we can replace with hyponym ("Any dog is an animal." -+a German Shepherd is animal."). Thus, whether hyponym relations correspond to M, S M or M, S M, depends on which type of reference more common. Our empirical results suggest that definite reference is more common-an one might probably expect, particularly for singular nouns.
v with definite reference, then it is possible to replace with
We note that in practice, M,, S M,, rarely happens. As an alternative, we can express this relation as d( ((M,A A M., M,)) = 0, where A represents the meet operation given by M, A,, == min( M.., M,). In other words, M,, I M. M, is contained within their intersection. This alternative formulation offers crucial advantage, as it provides soft measure of containment that quantifies the strength of this relation.
G LANGUAGES AND MEANINGS
In this section, we present a more theoretical discussion that motivates our notion of meaning repre- sentation. We also introduce some definitions and perspectives on LLMs that were not required for describing the methods proposed in the paper but that may be of independent interest. We recall that in the main body of the paper, we identified a language model with a map M A x A [0, 1]. Here we take a step back and start from a more primitive notion of autoregressive token generator.
Definition 1. An autoregressive token generator is a map G A" x A + [0. 1] associating any string with a score G(als) 9) for identifying the next token in A. Given any A", we use G2 to denote the prompted token generator, that is, token generator defined by G.(aas) == G(a)us) We write ((A) for the set of all autoregressive token generators with tokens from
Starting with an initial prompt ug, greedy text generation process using the generator G returns sequence of strings u,+1 = u,a, or trajectory, where a is a token recovered from G(u;) according some sampling scheme.
Given a candidate trajectory u = Q1 ... the token generator provides a sequence of token-level scores G(a ... (a,) in [o, 1]. These scores can be aggregated, for example by simply taking their product. In practice, it is more common to normalize by sequence length. Thus, we consider the sequence level score as given by
II G(aalar......
This choice allows [0, 1]. We think such a map as a (soft) "predicate" characterizing strings that the model considers "feasible."
generator evaluate candidate trajectories, obtaining map AA
21

---

Definition 2. A linguistic predicate is map L A" + [0, 1]. We write ((A) for the set of all linguistic predicates with tokens from A.
Any autoregressive language generator
thus be uniquely associated with linguistic predicate T(G) & ((A) using eq. (4). Conversely, any linguistic predicate T. determines associated token generator G(L) E ((C) by setting
G(L)(alu) min (Tlua)leltt) L(u)lu| (L(a)ial .1).
where this minimum is taken to be whenever the denominator of the left term is zero. Thus, obtain two "dual" maps L ((A) - L(A) and G L(A) + G(A). These maps are not full inverses but satisfy Go I Idg.
A generator G also uniquely determines a language model Mg AA" x A' + [0, 1], as defined in the main body of the paper, by simply setting (s(s) == L(G.) (s). Using these definitions, the meaning representation of the prompt u for the token generator G (or the model Mc) is the linguistic predicate L(G.) associated with the prompted generator.
As mentioned in the paper. these ideas are closely connected to the theory of automata and formal languages. To see this, consider "crisp" token generator whose values are always {0, 13. In this setting, we say that string u .1.... On (or trajectory) is "feasible" if G(a, 41.... Q;-1) for all The associated predicate L(G) also takes values in {0. 1} and can be seen the formal language consisting of all feasible strings (we identify {0, J--vllued functions with subsets of the domain). We now remark that, if a string u is acceptable for G, then
I(G,)= =
where - (u A uy E L.} the "left quotient" T. by и (sometimes also known as the Brzozowski derivative). The set u T is class of the equivalence relation on A given by
~ u2 + u1s E L iff u2s E L.
Thus, My ~L if and We have no "distinguishing continuations." This equivalence relation fea- tures in the construction of the minimal automaton that accepts given language Pin (2022). More precisely, for any language L, minimal automaton for L has states identified with left quotients Lu E }.}. accepting states corresponding F {s- - L} (classes of strings in L), and actions for each token a E A described by
(s L) a (sa)
Thus, the meaning representations L(G..) = L(G) for acceptable strings u correspond exactly to the states of a minimal automaton accepting the language L(G). We also remark that a different equivalence relation L, sometimes known the syntactic congruence. is given by
W1 25 U2 + suit L iff su2t E L,
and has the property that W1, U2 E A" induce the same action on states of the minimal automa- ton if and only if in Rt U2; in other words, the monoid of transformations on states is given by A+ Semtt (Pin, 2022). The equivalence classes for this relation correspond to the meaning repre- sentation for substrings that we consider in the paper, a refinement of the meaning representation for prefixes.
Remark 3. If string u is not feasible for crisp generator G, then we have u LLG) Z, since the product in eq. (4) is zero when at least one token not acceptable. On the other hand, according and thus may a priori
Definition 1, the language L(Gu) depends only tokens following be arbitrary and unrelated to the language L(G)). Practically, this means that infeasible prompts may lead to completely unpredicatable continuations. This intuition may also be useful for general (non-crisp) language models, by thinking of infeasible prompts as strings with very low likelihood for the model.
According to our definitions, the language L(G) associated with a generator is always prefix-closed set. If the vocabulary A has "end of sentence" [EOS] token, could also consider the language of all strings that feasible and also "complete," i.c., such that G(FOS]]m)) 1.
22

---

We conclude by revisiting these ideas in the actual [0, 1]-valued setting considered in the paper. To do so, we take a "coalgebraic" perspective on automata, as described in (Jacobs, 2012). We view a deterministic automaton with [0. 1|-valued outputs and action set A as triple (S, A) where S is set and 6, À are maps
S -
S:S [0. 1].
Here d describes state transitions and À describes (soft) acceptance of states (note that we do not model the initial state; for this reason we sometimes also use the term "process" instead of automa- ton). An autoregressive token generator G can be seen automaton in which 5 = A", 8 is string concatenation, and is the predicate L(G).
Given two automata (S, X) and (T,8', X'), morphism between the two is defined by map between states S -
T such that
f(8(s, a)) ((((s), and ((s) = N(F(s)), V.ES,aEA.
We write such morphism as ( + (T)
Intuitively, a "semantic interpretation" of an automaton (5, v) is given by a morphism m (S, 6, A) + (U,7.)) where U is some sort of "meaning space" and y correspond transitions and observations of S within U.! Moreover, a desirable property would be that the meaning pro- cess (U. %. ()) is also universal: this would mean that any automaton (S. 6, X) can be interpreted in (U,Y,P) (4) and in unique way. The following standard result shows such process actually exists and its states correspond to predicates.
Proposition 4 (Proposition 2.3.5 Jacobs (2012)). Let (U, ()) be the process given by U = [0, 1| and for any L E U:
(a) = Lo: where Ls(s) == L(as), p(L)= I(c),
where e is the empty string, Then, for any automaton (S, 6,,6, A), there exists unique morphism m (S) + (U, m)). Moreover, (U, %.pp) is determined (up to isomorphism) by this property.
The unique map (S A) (U, p4) from this Proposition can be described as
S U [0, 1
(v (s,"ss,))))
SE E A",
where 8 . S is the interated transition function. In particular, if we consider the process (A', L(G)) associated with a token generator G, this unique map is
u + (u + (uv)),
u, U A".
The association on the right analogous to the left-quotient set u u- LGG) considered for crisp models and motivates the semantic representation considered in this work.
'This kind interpretation assumes that the set A also acts on meanings. When modeling natural language, it is probably more natural to think of A some collection of "meaningful sentences" (as opposed to tokens) so that the state space 5 A' of language generator is the set of concatenations of such sentences. This would not significantly change discussion nor the construction of representations.
23


Reviews:
[{"review_id": "okY2McAvq0", "reviewer": "Reviewer_Lfxp", "summary": "The paper presents an innovative approach to interpret and represent the meaning of text in autoregressive LLMs through the concept of trajectories -- distributions of potential text continuations. This method diverges from traditional vector space embeddings, offering a a semantic interpretation that aligns with the actual use and context of language as understood by LLMs. It effectively overcomes the challenges posed by other methods, such as prompt dependency and the need for fine-tuning, providing a more faithful reflection of the model's internal representations without additional data or model modifications.\n\nEmpirical results demonstrate that this trajectory-based approach can successfully capture complex linguistic relationships and perform competitively on semantic textual similarity tasks without any fine-tuning or prompts. Furthermore, the paper extends this approach to multimodal models, where it outperforms established benchmarks like CLIP embeddings in understanding semantic similarities across images. The main contributions of the study include a new interpretable semantic representation for autoregressive models, the alignment of these representations with human linguistic understanding, and the applicability of the method to multimodal contexts.", "questions": "How does this model perform relative to more interpretable baselines - like random word embeddings?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper's trajectory-based method for understanding LLMs is original, diverging from typical vector space models and prompt-based approaches, and introducing a new angle to distributional semantics.\n\n2. The approach has been empirically tested against established benchmarks, indicating robust methodology and results that surpass existing techniques like CLIP embeddings on image-image similarity tasks.\n\n3. The paper is clearly articulated, systematically presenting the new method and its implications, with illustrative examples that enhance comprehension of the proposed concepts.\n\n4. This work is significant for its practical application in making LLMs more interpretable without extra training, it is an approach that is original as far as I know.", "weaknesses": "My main criticism of the paper is in the results for semantic similarity, they are far below those of contrastive methods (like 10 pts or so). I also the Sentence-T5 results are a bit misleading, that is the case without any fine-tuning. This isn't explicitly stated. There are other approaches that achieve far higher results on these tasks that do not use the training data for these tasks. I think the Sentence-T5 results in this paper are actually worse than a random encoder where random word embeddings are average together.\n\nThere are also a few typos:\n\nSection 3: “Speicifically”\nAppendix E: “presenedt”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ohHc7V1j2R", "reviewer": "Reviewer_ecJy", "summary": "The paper proposes a new method of sentence or phrase similarity measures using Decoder-only language models.  Decoder-only language models generate a continual string of tokens given an input. The proposed method measures two given sentences as individual inputs to a decoder-only model and measures the distributional similarity between the probabilities of multiple possible continual strings, named trajectories, for both of the inputs.\nWhile it cannot catch up with the recent contrastive learning-based sentence similarity models, it outperforms most off-the-shelf encoder-based sentence representation models.\nAlthough there are limitations especially high computational costs, the proposed method is interesting and unique.", "questions": "- For the Hyponym test, the contexts for a given word u are not sampled but retrieved from. It is not clear how the value Mu(s u t) is obtained.\n- Is it guaranteed that the set of trajectories for an input u always the same? Or, does it vary each time sampling for a sentence u is conducted?\n- The Autoregressive Model Baselines look very weak. I am not sure they are worth being included.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is overall well-written. I had no difficulty in reading and understanding this paper.\n- This paper presents an interesting and unique usage of decoder-only language models for measuring sentence similarity.\n- The proposed method shows better performance on sentence similarity tasks over encoder-based baselines.", "weaknesses": "- For the baseline encoder-only models, it is better to include larger models like BERT-large and RoBERTa-large using their CLS tokens and token averages.\n- The discussion about partial ordering between sentences is a bit puzzling. Since Tu and Tv are samples from u and v respectively, the former set of trajectories usually gives high Mu values for u and vise versa, so it hardly happens that Mu < Mv or Mu > Mv for all t in Tu U Tv. Besides, the discussion of entailment suddenly shifts from the comparison between Mu and Mv to the comparison between d(Mu cup Mv, Mu) and d(Mu cup Mv, Mv). This part is also quite puzzling.\n- For the experiments of entailment and hypernym/hyponym evaluation, there is an assumption that either of those relations exists between the given input pairs, which is not realistic.\n- As is pointed out in the limitation paragraph, the proposed model is computationally higher than other baseline models. In the Appendix, it is tested using a fixed set of trajectories, which seems to cause a big performance degradation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IT3rsdpVVy", "reviewer": "Reviewer_7D5R", "summary": "This work proposes an alternative sentence meaning representation (specifically for sentence similarity) for zero-shot use with auto-regressive LMs.   The method is to sample N possible following texts for two sentences A and B, and score each of those 2N following text for its probability of following A and the probability of following B, and then compare difference between the probabilities. They explore the ability to modify the notion of similarity by augmenting those sentences with prompts, and evaluate on zero-shot Semeval STS tasks, a task of comparing captions to CLIP outputs, and (using a modified approach to focus on single-word characterization) a lexical semantics task (WordNet hypernym modeling).", "questions": "- As mentioned above: do the authors feel that this would be viable to fine-tune, or is it limited entirely to zero-shot STS settings (plus entailment/hypernymy)? \n- Wouldn't the general hypernymy assumptions often be violated under negation?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The work seems to outperform other methods for autoregressive representation of sentences on the sentence similarity tasks studied, indicating its potential for continued relevance and utility. \n- The ability to modify similarity using prompting is very clever. \n- On a theoretical level, thinking about sentence meaning in terms of this theoretical notion of a trajectory of meanings is a great framing.", "weaknesses": "- It has a relatively rigid and narrow use case, since this method can only be used for pairwise comparison and since it's not obvious how to fine-tune it.  \n- The work frames it as producing \"interpretable\" vectors, but the work was somewhat lacking in an actual exploration of that interpretability. \n- I liked the idea of the entailment and hypernymy work, but it felt a bit convoluted: the way they approached both tasks seems to have lead to them comparing to weak baselines, despite NLI and wordnet link detection being well-explored areas.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "F7PxfrJy4p", "reviewer": "Reviewer_xcZs", "summary": "This paper proposes a \"meaning representation\" of a text based on inverse perplexity for the continuation sequence of the text.\nThe authors define a semantic distance between two prompts using their meaning representations.\nThis is useful for capturing the similarity between texts and for testing hyponym relationships.\nAdditionally, it can be applied to multimodal autoregressive models.", "questions": "1) The similarity distance depends on the performance of the autoregressive model. If the model is fine-tuned, e.g., for a chatbot, then how would the similarity change?\n\n2) In the first sentence of the second paragraph under **Meaning representation for prompts**, I think $M_u$ should be changed to $M_s$ and $t\\in \\mathcal{A}^1$ should be changed to $t\\in \\mathcal{A}^*$.\n\n3) In the second sentence of the paragraph **Containments of semantic representations**, the definition of 'partial order' should be revised. Since $\\sum_{\\mathrm{len}(t) = m} M_u(t)^{m} = \\sum_{\\mathrm{len}(t) = m} M_v (t)^{m} = 1$, we cannot have $M_u(t) < M_v(t)$ for all $t\\in \\mathcal{A}^*$. Please update the definition of 'partial order.'\n\n4) What is the temperature $\\lambda$ in the experiment? Is it the temperature parameter used during the inference process in the autoregressive model? Please clarify this.\n\n5) I believe we should use the training set for the model to compute $\\overline{M}_u$. Does the text corpus WikiText reflect the distribution of the training set?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper is well-written, featuring concrete formulations and structured experiments.\nThe authors introduce a novel and powerful semantic distance measurement that captures sentence similarity and hyponym relationships. Furthermore, they demonstrate that this distance can be computed more efficiently than initially concerned, alleviating worries about sampling a lot of trajectories.", "weaknesses": "1) The authors define a 'syntactic meaning representation' as the function $M_s$. I'm unclear as to why it is named 'syntactic meaning representation' (I'm also unsure why 'syntactic' is included). It is simply the conditional probability of the prompt. I don't find it to be a useful 'representation' for training or other applications like vector-space representation. In fact, they merely use the divergence between the conditional probabilities to measure text similarity. If this approach hasn't been taken by others, then this divergence could be considered novel. Thus, I cautiously suggest they use 'inverse perplexity mapping' rather than 'meaning representation'. The paper's title could then be 'A New Measurement for Sentence Similarity via Sampling Trajectories in Autoregressive Models'. Furthermore, 'semantic representations' appear suddenly following Algorithm 1. I recommend they use 'inverse perplexity mapping for substrings' instead of 'semantic representations'.\n\n2) I'm unsure what is meant by 'meaning containment'. Why use 'containment', which typically refers to 'the action of keeping something harmful under control or within limits'? Please provide a definition or explanation, along with some references.\n\n3) For the final version, sharing the code on GitHub would be beneficial for readers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)