The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities
Authors: Zhaofeng Wu
Xinyan Velocity Yu
Dani Yogatama
Jiasen Lu
Yoon Kim
Venue: ICLR 2025 Poster (2025)
Keywords: interpretability
representation
multilingual
multimodal
Primary area: interpretability and explainable AI

Abstract: Modern language models can process inputs across diverse languages and modalities. We hypothesize that models acquire this capability through learning a _shared representation space_ across heterogeneous data types (e.g., different languages and modalities), which places semantically similar inputs near one another, even if they are from different modalities/languages. We term this the _semantic hub hypothesis_, following the hub-and-spoke model from neuroscience (Patterson et al., 2007) which posits that semantic knowledge in the human brain is organized through a transmodal semantic "hub" which integrates information from various modality-specific ``spokes'' regions. We first show that model representations for semantically equivalent inputs in different languages are similar in the intermediate layers,  and that this space can be interpreted using the model's dominant pretraining language via the logit lens. This tendency extends to other data types, including arithmetic expressions, code, and visual/audio inputs. Interventions in the shared representation space in one data type also predictably affect model outputs in other data types, suggesting that this shared representations space is not simply a vestigial byproduct of large-scale training on broad data, but something that is actively utilized by the model during input processing.

Published as a conference paper at ICLR 2025
THE SEMANTIC HUB HYPOTHESIS: LANGUAGE MODELS SHARE SEMANTIC REPRESENTA- TIONS ACROSS LANGUAGES AND MODALITIES
Zhaofeng Wue Xinyan Velocity Yu"
Dani Yogatama"
Jiasen Lu²
Yoon Kim
MIT University of Southern California 2fwecsail. mit.edu edu
Allen Institute for AI
ABSTRACT
Modern language models can process inputs across diverse languages and modali- ties. We hypothesize that models acquire this capability through learning a shared representation space across heterogeneous data types (e.g., different languages and modalities), which places semantically similar inputs near one another, even if they are from different modalities/languaage hypothesis, following the hub-and-spoke model from neuroscience (Patterson et al. 2007) which posits that semantic knowledge in the human brain organized through a transmodal semantic "hub" which integrates information from various modality-specific "spokes" regions. We first show that model representations for semantically equivalent inputs in different languages are similar in the intermediate layers, and that this space can be interpreted using the model's 's dominant pretraining language via the logit lens. This tendency extends to other data types, including arithmetic expressions, code, and visual/audio inputs. Interventions the shared representation space in one data type also predictably affect model outputs in other data types, suggesting that this shared representations space is not simply vestigial byproduct of large-scale training broad data, but something that is actively utilized by the model during input processing.
We term this the semantic hub
Output
Industry
-In
In
Industry
Closest Token to Each Layer's Hidden
Industry
five
sky
piano
industry
stop
piano
industry
five
Intervene by modifying representation English "four"
industry
anstime
interal
bacoground
plano
baceground
plana
whis
CRVPT
bacoground
piano
Dackground
are
plane
Engg meeningful though English
lrfrff
recij
Window
CMIT
reactj
ub
ENTION
graccl
8 Input ( #5 3
75-70+
geint_evernnuee))
rangech,
- M
Transiaion: may not have another chance enter the entertaiement
<piano sound>
Figure 1: Examples of the semantic hub effect across input data types. For every other layer, we show the closest output token to the hidden state based on the logit lens. Llama-3's hidden states are often closest to English tokens when processing Chinese texts, arithmetic expressions (in ȘCin the appendix). and code, in semantically corresponding way. LLaVA, "siounlanguage model, SALMONN, an audio-language model. have similar behavior when processing images/audio. As shown for the arithmetic expression example, models can be intervened cross- Progu-lingually or cross-modally. such as using English even though the input is non-English, and be steered towards corresponding effects. Boldface is only for emphasis.

---

Published as a conference paper at ICLR 2025
INTRODUCTION
Modern language and multimodal models (LMs can process heterogeneous data types: text in different languages, non-linguistic inputs such as code and math expressions, and even other modali- ties such as images and sound. How do LMs process these distinet data types with a single set of parameters? One strategy might be to learn specialized subspaces for each data type that are only employed when processing it. Often, however, surface-distin distinct data types share underlying semantic concepts. This is most obvious for sentences in different languages with the same meaning; but such shared concepts are present across other data types, e.g., between an image and its caption, or piece of code and its natural language description. The human brain, for example, is believed to have a transmodal "semantic hub" (Patterson al.) 2007 Ralph et al.) (017) that integrates semantic information from modality-specifft "spokes" (e.g., visual/auditory cortices). A model, leveraging structural commonalities across data types, could similarly project their surface forms into a shared representation space, perform computations in it, and then project back out into surface forms.
To what extent is this idealized strategy adopted by actual models? Wendler et al. (2024) find that on simple synthetic tasks, Llama-2 Touvron et al. 2023b) maps various input languages into shared "English space", hinting that it leverages this shared representation scheme to an extent. We show that this is in fact much more general phenomenon: when model processes inputs from multiple data types, there is a shared representation space, and this space can be interpreted to an extent in the LM's inherently dominant language (usually English) via the logit lens (nostalgebraist, 2020). Concretely, we say that this space is scaffolded by tokens in the LM's dominant language.
We first show that LMs represent semantically similar inputs from distinct data types (across lan- guages, or between natural language and arithmetic expressions, code, formal semantic structures, and multimodal inputs) to be close one another in intermediate LM layers. We further show that we can interpret these hidden representations to an extent using the LM's s dominant data type e.g.. when processing a Chinese input, an English-dominn dommnnant LM "thinks" in English before projecting back out to a Chinese space. Finally, we perform intervention experiments showing that intervening in the shared representation space using the LM's dominant data type, predictably affects model output when processing other data types; that is, the shared representation space (and the processing of these representations through subsequent layers) is not a vestigial byproduct of the model's being trained on (say) English-dominant text, but causally impacts model behavior.
Our work is complementary and distinct from prior work which finds structural similarities between the representation spaces of models trained (usually independently) on different data types, such as those showing that text representations from text-only LMs can be aligned, via a transformation. to vision/audio representations of modality-specife
models (Ilharco al. 2021 Merullo et al., 2022 Li et al. 2023] Ngo & Kim, 2024; Huh et al. [2024] i.a.). the literature on cross- cross-lingual lingual word embedding
alignment (Mikolov al., [013]] Artetxe al. 2017 Conneau et al.) 2018 Schuster al.. 2019 i.a.),
and work on cross-task transfer (Moschella et al. 2023 Wu et al. [022]] i.a.). We instead show that an
LM trained on multiple data types represents and processes them in a shared space without requiring
explicit alignment transformation. We hope our findings inspire ways to more easily interpret the mechanisms of current models and future work aimed at better controlling models.
THE SEMANTIC HUB HYPOTHESIS
Let X² be the domain of data type E Z where Z denotes all model-supported data types. E.g., for languages, X'Chinse could be all Chinese tokens, while for images Xmmage = [0, 255juxhx3 could be the RGB values for an x h-sized patch. Consider a function M, X + S2 mapping input sequence into a semantic representation space Sy (the "hub"), and a verbalization function V2 Va:0 S2 + X Given an input prefix wf., E X of length i where wf E Xz, a sensible (implementation-agnossis way to continue the sequence is to first encode it into a modality-agnostie representation, m = MM (wra), reason and formulate representation of futures oat E S., and finally verbalizing via (mmt).
An LM parameterizes a similar process: it uses MLM to map various input data types into repre- sentation space SIM C IRd (early layers), performs computations in the space (middle layers), and verbalizes the output via VLM (end layers and the LM head). However, it is unknown how different
'ereaatter, we use the term "language model" loosely and also consider multimodal language models that process additional data modalities, since such models are commonly trained top of a text LMM hackhone.
2

---

Published as a conference paper at ICLR 2025
data types are structured in the representation space. E.g., one possibility is that the LM partitions I into disjoint subspaces for each data type and processes them separately. We instead hypothesize that LMs learn to represent and process different data types in shared representation space that functions as a modality-gnnostic "semantic hub." That is, semantically similar inputs wia and w from distinct data types-es e.g., texts in different languages that are mutual translations-are similarly mapped in SIM: informally, MLM(WiH 2 (irr) However, absolute similarity measures (i.e., sim( MMM (M((()))) (wft),, (w) )) are generally difficult and unintuitive to interpret high dimensional spaces P We thus focus on relative similarity measures, taking a semantically unrelated sequence "i:t" and evaluating whether wit is closer to wis than u 1a.. We formalize this hypothesis as:
sim (MLM (w (LM((((.)) sim (MIM MLM(u(.))).
Moreover, when the LM has a dominant data rype 2" in training (e.g., English for Llama-2) 2), we hypothesize that this shared representation space is "anchored" by 2", in the sense that Eq. [ holld strongly enough that we can probe out to z* from MMM(w..)) We further expect this to hold for model representations of the future, which autoregressive LMS trained to model. I.e., the representation of prefix should better align with verbalization of the future in 2* than a non-dominant 2 (though we now need a different kind of model representation, denoted with re)
sim (w repr (w.)) sim (MLM( (ff)) repr (M (w²))
We hypothesize that this holds even when 2 = 2 E.g. with an English- LM, its encoding of the Chinese prefix wf. = ml ie Acll" (trans. "This paper is so hard to") should be closer to the representation of the English word wł41 ='write" than its Chinese translation wi """"
2.1 METHОD: TESTING THE SEMANTIC HUB HYPOTHESIS
We test the semantic hub hypothesis by considering pairs of distinct data types, the dominant one x* and non-dominant one z², different for each experiment. Whenever semantically related inputs are available (e.g.. an image and its caption), we directly test Eq. by using h4, the LM's hidden state at position t and layer É, as MLM(W..r ), and further using cosine similarity for the similarity function.
We operationalize Eq. 2 via the logit lens (nostalgebraist, 2020), a simple training- freee approach for interpreting the hidden states of model. Transformer-basee LMs produce the next-token distribution using softmax (Oh.) (omitting the bias term) where O is the output token embeddings (or "unembeddings") '' and h is the final layer hidden state, Logit lens applies the same operation to the intermediate layers to obtain p inggitlens ht)= == softmax (Oh?). Logit lens has been found to produce meaningful distributions that shed light on an LM's internal representations and computations.
Under the logit lens, repr in Eq. 2 considers the output embedding of single tokens, 7² E X=-
and -=² E Xyo. We use the first token of w and wše. Using the dot product for sim(-), Eq. 2 is
cquivalent to comparing the logit lens probabilities.
p Jogitlens (7= (1))
> pogittens
(3)
i.e., testing whether the probability of the continuation in the dominant language is more likely than the continuation in the original input data type. Since the logit lens is tailored for probing out a single token, we usually consider short-enough verbalizations such that a single BPE token can reliability identify it. This often means that the two verbalizations are two single words that are semantic equivalents. Nevertheless, we also consider longer future verbalizations when its first token unanbiguously suggests one interpretation in that context, which allows more flexibility.
While the above test is simple, -= is unavailable in many multimodal models without vocabulary tokens for 20. We thus only focus on testing Eq. [ though logit lens enables an additional test:
y Pogitless (7= mt)
p
logillens
where e is an unrelated token. Also, using the next token as 7= unsuitable for multimodal models never been trained to output at non-dominant data type positions but the non-language data types are
"See for example Beyer et al. 1999). Most prior work in the probing literature also implicitly uses relative similarity measures since the similarity scores are normalized over a finite label set.
3

---

Published as a conference paper at ICLR 2025
Oos 55m Increase Over Basellan
Tranlition (Buglish)
Next tolen (Chinene)
Cosmne Sanllsty
0.5
Randon bandline
Loget Lema Log
0:33
0.
Liama-s
Bloom
Liuma
5loam
Layer
Layer
(a) The cosine similarity of interme- (b) Same as (a), but subtracted by (c) IJama-3 logit lens log prob. of parallel English vs. Chinese tokens when processing Chinese text.
diate representations of English and baseline over non- non-parallel texts.
Chinese parallel texts.
Figure 2: Results for the multilingual experiments. The 95% CI is plotted in all. Parallel texts have similar representations. Hidden states for Chinese texts are close to the unembedding of English tokens.
only encoded as prefixes. For such models, we hypothesize that they still fully represent the current input in a shared space. We thus take -= to be the last token of wi. (which is the interpretation of wf. under st; e.g., if wf: is image, wi.. can be the objects in the image described in language).
EVIDENCE OF A SEMANTIC HUB
We apply our tests across diverse data types and find evidence of shared representation space in all cases. We show additional experiments arithmetic data in with similar trends.
3.1 MULTILINGUAL
Wendler et al. (2024) find that when processing specific in-context learning (ICL) templates for highly synthetic lexical- tasks (word repetition, word translation, etc.) in non-Englise languages, the intermediate hidden states of 2lama-2 closer the unembeddings of English tokens than the output language. This is consistent with our hypothesis, albeit constrained to a simple synthetic task and one LM. We show that this shared representation space is general property of LMs when they face naturally occurring text.
Experiment 1: Representation similarity of mutual translations. Translation datasets enable a direct test for Eq. [] with semantically equivalent cross-lingual sentences as wit and w and randomly chosen non-matching sentence as u "ize. We use the professionally-transiiie Chinese parallel sentences from Chen et al. (206) (N = 5260). For each sentence pair, we use template transform each sentence and compute the representation cosine similarity for each layer, using the last token position as the sentence representation following Wu et al. (2023), ) which preserves sentence information (Morris et al. 2023). We consider two English-dominant LMs, Llama- 2 and Llama-3, one Chinese-dominant LM, Baichuan-2 (Yang et al. 2023). and one multilingual LM, BLOOM (BigScience) 2023), specifically the 7B/8B variants. AA. contains more details.
English-
In Figure 2a the raw cosine similarity is high, up to >80% (left), and it is also significantly higher
than the non-matching pairs' similarities (right), but only in the middle layers. These trends support the hypothesis that the middle layers act as the semantic hub of the LM. Notably, this trend also exists for BLOOM which does not have a dominant pretraining language.
Experiment 2: Probing out continuations in the dominant language. We next test Eq whether continuations in the dominant language have higher probability than those in the input language. We use 1,000 Chinese and English sentences from Wikipedia English-dominant LLama-3, we use a Chinese prefix w as input and take -a to be the next Chinese token (i.e., wi and x to be the (first token of the) English translation of w $A.1 has further details. Figure 2c plots the logit lens probability for the two tokens well the uniform distribution probability. In early layers, we cannot read out either token better than random chance. After layer 17, the model representations
Pikkimedia-Foundatee
2023). For the
substantially closer the English token than the Chinese token until layer 31, showing that the model hidden space is indeed better scaffolded by English than Chinese.
Next, extend this analysis consider global language-leve trends across languages. We first compute p(w 2). the token distribution under a language 2, by running the LM tokenizer on the language-specific split of the mC4 dataset (Xue et al. 2021). We then use Bayes' rule estimate
4

---

Published as a conference paper at ICLR 2025
1.ama----ff Taglish tmpuc
1lama-a-yfi; legur
Liama RR English tepur
31ama-5- Chinese Ingut
Barchhuaa-- 7R English impur
Stochumssyiis ytt Cámese imput
Bloom 1itt Engitsh impuz
Bloom-7fi Chinme impat
26
15
Eeglih Eott-rta Lingsage
Eaglsh artap legguge
Chisss at nelvicg Lengage
Chasmin atrop lingsage
Acather Lingu-gr
Language
Figure 3: Language prob. English and Chinese (and the top language when it neither. which only happens for Bloom). Regardless of the input language, the dominant LM language is more salient in early-middle layers, and the input language is more salient in the final layers. Bloom has no clear intermediate language.
p(2 w) x p(w: (=)p(z) with a uniform prior p(z) We finally compute the probability of hf belonging to each language as p(= h.) x Ewey p(2 w)ppoggile" (w: hG). If our hypothesis that the universal representation space is better scaffolded by the dominant language is true, we expect the dominant language =* to have the highest probability across input languages in the middle layers.
Figure 3 shows the layer-wise probability of English and Chinese on 10,000 English/Chinese Wikipedia sentences. When English-dominan models process Chinese text. Wendler et al.]s finding generalizes, where English dominates in intermediate layers and Chinese only dominates in the final layers. On the Chinese-dominant LM, this trend flips: when processing English text, its intermediate layers are closer to Chinese tokens and the final layers are closer to English tokens. For BLOOM, a multilingual model with a relatively balanced training language mixture, we do not see a clear domi- nating language in intermediate layers; when we manually inspect the closest tokens, in most cases they are symbols with no clear semantics (though this does not mean lacks a unified representation space: see Exp. 1). 70B model trends in T are highly similar to the 7B/8B ones.
3.2 CODE
Output
<whitespa0e>
Many recent LMs are trained on code corpora (Touvron et al.] 2023a b. 3-ma---Team| 2024], Gemini- Team 2024). We find that they similarly process code by projecting it into unified representation space shared with regular language tokens. Figure ] shows examples, where LMs in the
Closest Tokeo to Exch Layer's Hiddon
reverte
and
and
everpt
voho
returro
Inperil
intermediate layers tend to verbalize the future in free-form English, un- constrained by program syntax. F.g.,
uulti
in the first program, given the Python for idx, elem for idx2, elem2 in enumerate (numbers"" in- stead of the groundtruth continuation in Python "): if idx 1= idx2: ...", the most salient intermediate token is "except", likely attempting to predict in English "(for each element in numbers) except if it is equal to idx". Similarly, in
prefix
enumerate(numbers)
Input dosoenottooss
diet
suibers: Liceflot))
C1-9, 2.8, S.P...
idb,
threabaic: ficet.
Figure 4: Logit lens analysis on Llama-2 2 processing Python pro- grams. For every other layer, we show the closest token (sometimes whitespace) to the hidden states before the grayed-out texts. The model tends to verbalize the future prediction in English that corresponds to the code continuations gray). a list expression "[ 2.0," instead of continuing in Python " 3.0", it predicts "and", which is natural way to continue in English. In these cases, it is difficult to obtain semantically equivalent English-Python pairs, so we only test Eq. acroos targeted cases in Python below.
Experiment 1: Simple Python list literals. We systematically test the list case, where ht is the hidden state after processing
r² ="and", and T² is the actual next token. Figure 5a shows
This prior obviously does not reflect the training language distribution, but in fact makes our trends even more salient, since using a real (or estimated) p(2) would make p(2 w) even larger for the dominant language.
5

---

Published as a conference paper at ICLR 2025
-25
Login Prob.
token
Hidden closer surface argument
400
Hidden closer semantic role
-23
Layer
32
45.0
100
200
300
400
500
All Function Arguments in MBPP
Layer
(a) Llama-2 logit lens log probabilities (and the 95% CT) at commas in Python list literals, the English "and" token (and baseline tokens) vs. the actual next token in the program, from MBPP.
(b) The distance between Ilama-2 hidden states when predicting function argument, the unembedding of the argument's (its semantic role) vs. the actual argument expression, MBPP.
Figure 5: Results for the code experiments. Code expressions are close to semantically meaningful free-form English words in carly-middle layers, such as "and" in list literals and the argument's semantic role in function calls; in the final layers, the representation converges the context-constrained Python token.
Cos, Sim. Inccoase Over Baseine
Presurmmc as Top-r Tokem (S)
CarespondngGooo
Cos, Sim. En.coaus Overr Beseine
Chomeleon
White
002
00L-
0.0-
0.00
0.00
-2.00
17
35 Layer
23
06
35
Layer
Lay
(a) The cosine similarity difference (b) The frequency of the closest to- (c) The cosine similarity difference between intermediate representations ken to LLaVA's hidden states de- between intermediate SALMONN of matching images and captions. scribing image color, against representations of matching audios over non- non-matching ones.
baseline using "white".
and labels, over non-matonn non-matching ones.
plotted in all. Model representation (a) visual and (c) audio inputs and their textual labels are similar. Furthermore, in (b), model representations of individual color patch tokens are close to English words for those colors.
Figure Results for the multimodal experiments. The
that this trend holds systematically on all such commas in the MBPP dataset (Austin et al. 202 (N 6923, including unit tests): as expected, in the final layers, the representation closer the ground truth next token's 's unembedding, and closer to "and" in the middle layers. We also show the probability with two other tokens, "ar" and "not". as baselines, both of which are lower than "and".
Experiment 2: Python function call arguments. Function arguments have names the defini- tion, such "range (start, end, step)"; but when invoked, they are filled with actual context- appropriate expressions. We call the argument names "semantic roles", and the context-specific expressions the "surface forms", inspired by thematic relations in linguistics (Fillmore 1968). In the second example in Figure T we show that LMs predict the arguments by first "thinking" about their semantic role (7*") and then instantiating with crrface-constraince expressions (7* ). We extract all function calls and arguments from MBPP with simple filtering, resulting in 540 arguments (see $A.2 for details). For each argument, we use the logit lens to inspect the hidden states at the preceding token ("(" or ", "). For each argument, Figure 5b visualizes if the semantic role the surface form is closer each layer's hidden state of Llama-2. The semantic role (7*) dominates for the early to middle layers, even though the role token usually does not appear in the context at all, and only the final layers do the representations converge towards the surface form argument (x²
3.3 VISUAL INPUT
Past work has investigated the representation of separately trained vision and text models, often finding that their representation spaces are similarly structured and alignable (Merullo et al. 2022 Liet al. 2023 Huh et al. 2024 i.a.). We show that when trained together, vision- vision-language models learn to project both modalities into a joint representation space. Current language models typically represent images by segmenting them into patches, embedding them into "image tokens".

---

Published as a conference paper at ICLR 2025
Sogghin-ogghe oegiti
Myrckngpphon coptic
Marking Eiahhe
Layer
Layaz
Layer
Layar
(a) Caption, LLaVA (b) Caption, Chameleon
(c) Segment., LL
(d) Segment., Chameleon
Figure 7: When processing an image patch, model logit lens probabilities of either the nouns in the corresponding caption or the patch segmentation label, as well as a baseline for each with no correspondence between the patch and the label. The image representations better match the semantically corresponding English words.
and then feeding them into the transformer model along with other text tokens (Lu et al. 2023 2024 Liu et al. 2023 i.a.) We hypothesize that the intermediate representations of the image patches are close to the corresponding language tokens that describe the scene. Experimental details are in $A.3
Representation similarity between image and its caption. Though not constituting exact semantic equivalence, an image paired with its caption provides one possible test for Eq. We take 1000 images and corresponding captions in the MSCOCO dataset (Lin
Experiment 1:
ctt al. 2014) and measure their hidden states cosine similarity in LLaVA-7B (Liu et al. 2023) and
Chameleon-7B (Chameleon-Teaal 2024 As in Eq. I we subtract the average cosine similarity between non-matching image-caption pairs as a baseline, separately for each layer. Figure 66 shows
that semantically matching inputs, even though in different modalities, are more similarly mapped in the models' hidden space, though the similarities are lower than for mutual translations ($.1)
Patch-level analysis using logit lens. We now test the image-description similarity using the logit lens, in Eq, 4 First, as a toy setting for illustration, we inspect LLaVA's 's representations of pure color images, specifically those in red, green. blue, and black. Figure 66 shows that, in up to more than 20% of the time in the intermediate layers (averaged across the patches and the four colors, N 2304), the closest token is the corresponding color word (out of all vocabulary tokens).
Experiment 2:
We next consider the image captions, the same 1000. For each im- age patch, we compute scalar patch-caption alignment score (for each layer separately) by summing over the logit lens probabilities for all nouns in its caption (as proxy for objects in the image) at that image patch position. For the irrelevant token baseline, we compute the alignment score in the same way but using an unrelated caption. We normalize by the number of nouns so that the score is comparable. We average this alignment score over all patches in all images. Figures 7a and [bb show that the matching caption better aligns with the image patch representations than an unmatched caption, reliable across all layers for LLaVA and consistently in the middle-upie upper layers for Chameleon.
Machingt- Shbel
Label Logle Lest Frob.
RandemLabel
Layer
Figure 8: When SALMONN pro- cesses audio clip, the logit lens prob. of the English words in the audio label vs. another random la- bel. The audio representations better match the semantically corresponding label in English.
Finally, we perform finer-grate finer-grained study using not caption infor- mation but segmentation information, with object labels in specific image locations. The setup is near-identical captions, but the scalar alignment score for a given patch is not computed using its logit but of its corresponding object label. For the irrelevant token baseline, aligning each patch with a different randomly chosen object category from all categories. Figures [c and 7 show that, for LLaVA, the patches are much better aligned to the corresponding labels than randomly assigned labels (which have near-O logit lens probability). For Chameleon, this is the case for only one middle layer, and not in a statistically significant way, though we will show in $ that Chameleon's 's latent space can be reliably steered using English tokens.
lens probability of caption nouns,
compute the alignment by
3.4 AUDIO
Audio is another modality that is often modeled jointly with text (Lu et al. 2024; Gong et al. 2024 2023; i.a.), and perform similar experiments using SALMONN, an audio-text model. We use the VGGSound dataset (Chen et al. 2020) which contains 10-second audio clips with labels, e.g.. "duck quacking" "playing cello". We use the same two multimodal tests in the vision case.
7

---

Published as a conference paper at ICLR 2025
Table 1:
Steering Llama-3's output sentiments using trigger words in English the input language (either Spanish or Chinese). report the mean sentiment, disfluency (perplexity), relevance the continuation. as well as the standard deviation across 10 seeds. Cross Cross-lingual steering is consistently successful, sometimes even more than monolingual steering, without substantial damage in text fluency and relevance.
Text Lang.
Steering Dir.
Steering Lang.
Sentiment
Disfluency ())
Relevance (t)
None
None
0.143
2.35
0.861 +0.002
Spanish
0.125+00033
10.54=2.39
0.842
+
Spanish
English
0.139 10.026
8.75
0.857
Spanish
0.175 033
7.98=2.01
0,856110.002
1
English
0.159 10.096
7.35
0.85910.003
None
None
0.178+0.038
11.06_3.12
0.869+0.004
Chinese
0.152
10.78
0.86610.005
+
Chinese
English
0.161
11.36_1.13
0.864
Chinese
0.153+40004
11.12
0.870=0.004
English
0.17940.032
10.90 23.25
0.869 +0.03
Experiment 1: Representation similarity between audio and its label. We study the represen- tation cosine similarity between an audio and its label description, and subtract from a baseline which is the average cosine similarity between non-matching pairs, separately for each layer. On 1000 samples from VGGSound, we see in Figure (egure/6cjthr that semantically matching audios and labels have more similar representations in the intermediate layers.
Token-level analysis using logit lens. Unlike for vision-language models where we can map individual image patches model token positions, such correspondence does not exist in SALMONN. This limits to position-agnostic evaluations like the captioning study, preventing fine- grained analysis such as segmentation. Similar to the captioning experimental design, we measure the average logit lens probabilities of the words in the label, and consider a random label in the dataset with no word overlap as the baseline. On the same 1000 samples, Figure X shows familiar trend, where the audio hidden states are closer to semantically corresponding label words. We note that this is lower bound-ooal words in some labels, such as the prepositions in the label "writing on blackboard with chalk", are unlikely to be represented in the audio hidden states.
Experiment 2:
INTERVENING IN THE SEMANTIC HUB
Interpretability results should be tested under causal framework to ensure that the observation is not vestigial byproduct of model training with no effect on model behavior (Vig et al. 2020 Ravichander et al.] 202T] Elazar et al., 202T Chan et al. 2022 i.a.). Here, we show that the semantic hub does causally affect model output: Semantically transforming T= in English interpretably leads to corresponding behavior changes the non-dominant data type. $B reports relevant hyperparameters.
Multilingual. Past work has shown that (monolingual) interventions in the middle layers can steer LM outputs predictably (Subramani et al., 2022) Turner et al.) 2024] Rimsky et al.) 2024) i.a.). If English-dominant LMs have shared representation space, should be able to intervene on it in English even with non- Enggish inputs. We use common intervention method, Activation Addition (ActAdd; Turner al. 2024), which has two stages: (I) taking steering word (and optionally contrastive one) that semantically represents the steering effect, passing it through the same model, and taking its hidden states at an intermediate layer; (2) adding the steering hidden states to those in the original forward pass of the regular generation process, at the same layer. We take their sentiment-steerte steering experiment but generalize it cross-lingually. See Tuurner al. (2024) for details.
We consider two non-dominant languages, Spanish and Chinese, and take 1000 texts in the InterTASS dataset (Spanish: Keung et al., 2020), and generate continuations either without modifications or intervened using ActAdd. As the steering vector, we use the difference between a positive sentiment trigger word and negative one, in the appropriate direction for negative or positive steering. Specifically, we "Good" and "Bad" for English, "Bueno" " and "Malo" for Spanish, and "IJ" and "ff"" for Chinese. In addition to sentiment evaluation, we also measure the generation fluency and compute the relevance
et al. (2018) and the multilingual Amazon reviews corpus (Chinese;
8

---

Published as a conference paper at ICLR 2025
Poputtion of Enstaabrrs (66)
Correce Uinchnggd
Interventioon Accurary (N)
0.93
Obe
Mammals to memmils
60
047
Non-mummals Memmels
40
60-
0325
00-
0875
20
70
Intervention Strength Coefficient
Layer
Intervention Strength Coefficient
(a) Steering single-argument (b) Replacing image representations (c) Steering mammal sounds to be " "range(end)" call to be predicted of color with language tokens of predicted as non-mammal sounds, another color, and expecting the and vice versa. model to predict the new color.
double argument.
Figure 9: For (a) code, (b) images, and (c) audio, steering model output using English words, for various intervention strengths ((a) and (c)) and layers ((b)). (a) and (b) measure successfulness with the proportion of instances steered to the correct output, and (c) measures the probability of predicting mammals. Overall, intervening in the unified representation space in English reliably leads to desired model output changes.
of the generation with the prefix using trained models, following Turner et al. (0224). Ideally, the intervention should achieve the desired sentiment without hurting text fluency/relevance (see (B.I).
Because we take intermediate layer representations of the steering words (step (1)), if the semantic hub is language-agnostic, we expect similar representations and similar steering effectiveness across steering languages. [ that this is indeed true Llama-3: ActAdd in the text language is in most cases effective, achieving the intended effect on sentiment, with usually only a modest decrease in fluency and relevance, often statistically insignificant. This aligns with the English-only findings in Turner et al. 2024). And intervening in English is similarly effective as using the text language. Table (appendix) shows the results for Llama-2, with very similar trends
Code. Based on our semantic role observation $3.2 we rum a intervention experiment using the "range" function. We focus on two overloaded versions of "range": "range end)" and "range (end)". If the semantic hub causally affects model output, then intervening using the semantic role tokens should affect the function version used in given context.
We take all single-argument "range calls in the MBPP dataset (N = 159) and attempt to expand it into "range end)". As the intervention, we use an even simpler method than ActAdd: because the unembedding vectors of the semantic roles are close to the intermediate hidden states, we simply compute the difference between the unembeddings of two contrasting trigger tokens ("start" "eed""), scale it by constant coefficient, and add it to the hidden representation corresponding to the open parenthesis "(" at an intermediate layer (layer 17). For all these "range" call in the dataset, we let 2 generate without and with intervention. Figure 9a shows that, with increasing intervention strength, more instances are successfully steered to "range end)", up to 67%.
Visual inputs. We show that we can steer the output of vasion-language models by intervening on image patches using language tokens and analyze how this affects textual output. We focus on Chameleon which showed a weaker trend $3.3 Focusing on the color setup, if the representation of color is similar between visual and language inputs, hypothesize that hidden states corresponding to one pure color image patch with the unembedding of the language token for another color, and mislead the model to "perceive" the new color when asked about it. Note that replacing the hidden state is a more invasive intervention than addition. But there is one confounder: the intervened word may lexically bias the model to generate the same word, without any reasoning that incorporates the new color. To control for this, we show two colors in one image and only intervene at the positions corresponding one color: if the intervention unconditionally and lexically biases the generation to the new color, this effect would (incorrectly) affect both colors
replace the image
"We do not consider Llama-3 in this case because its default behavior usually generates "ra end)" in the first place, and it is unclear how steer from "range(0, end)" "
'ee tested settings that require more sophisticated reasoning such as asking for country flag with the two colors, or asking about spatial relationships of the colors. They seem to be beyond the capability of Chameleon-7B. -en without interventions, the model cannot answer the questions correctly.

---

Published as a conference paper at ICLR 2025
We consider all color pairs using the same colors as in $3.3 red, green, blue, and black, and picking one color in the pair and intervene to a new third color (N = 48). As the intervention, we start from layer l and replace all hidden states at and after & to be the unembedding of the new color minus the old color. We ask the model what the two colors in the image are, and only consider the intervention successful if the model answers both the new color and the other unintervened color correctly. Figure Shooressssswws the success rate across all é it gets as high as above 80% o We highlight that, for both this experiment and the earlier ones in this section, the interventions are not even necessarily guaranteed to lead sensible outputs, let alone correct ones.
Audio. We perform similar intervention with SALMONN, with the same desideratum that the QA process should require reasoning rather than outputring the intervened token as-is. We consider 1000 animal sounds in the VGGSound dataset, specifically only single-word animals, and ask "Is this animal a mammal?" !?" We intervene both on mammal sounds with a random non mon-mammal word and vice versa, in case the intervention only biases the model in a certain direction. We perform the invention similarly to the code case, adding the unembedding difference between the new trigger word and the original animal name, scaled by a constant, layer 13. We measure the probability of the "Yes" token and the "No" token and compute the normalized "Yes" probability. Figure 9 visualizes the two cases across intervention strengths. As the strength increases, the model is more likely to predict in the steered direction, again demonstrating cross-data-type intervention effectiveness.
5 RELATED WORK
Representation alignment between separately trained models. A long line of work has investi- gated the representations of separately trained mono-data-type models, and showed that they can be aligned through transformation. In the multilingual case, it has been found that separately trained word embeddings for different languages can be aligned (Mikolov et al., 2013) /Smith et al., [017 Cao al. 2020; i.a.). Similarly, prior work has shown that visual representations and text representations
from different models can be mapped together (Merullo et al., 2022 Koh et al. 2023 Maniparambil
al. 2024 i.a.). Huh et al. (2024) argued that these are possible because the different data modalities are projections of the same underlying reality. Our work, in contrast, looks at a single model that
processes multiple input data types and finds that the resulting representations align, without needing a transformation. Xia et al. (2023) considered an objective that explicitly trains the model to increase such alignment, while we analyze how it organically emerges through autoregressive training.
Representation evolution throughout layers. Past work has analyzed the representation evolution throughout transformer layers, inspecting how it affects reasoning (Yang et al., 2024 factual- ity (Chuang et al. (2044) knowledge (Jin et al. 2024). etc. From another angle. work on layer pruning and early l exiting also speaks to representation dynamics across layers (Gromov et al. 2024 Sanyal let al. 2024. i.a.) Mechanistically, EElhage et al. (202T), Merullo et al. (2024), Todd et al. (0024).
Hendel et al. 2003)) i.a., more precisely characterized the representation changes algorithmically.
Inspecting model hidden states. We adopted the logit lens for its simplicity which brings few confounders. However, alternatives exist, usually requiring some training (Belrose et al. 2023 Ghandeharioun et al.) 2024 Templeton et al. (2024, i.a.). They allow for more expressive explanations, though at the risk of overfitting. Similar methods have been developed for other modalities, such as Toker et al. 2024).. Testing our hypothesis using these methods would be valuable future work.
6 CONCLUSION
We proposed and investigated the semantic hub hypothesis, which posits that LMS represent semanti- cally similar inputs from distinct modalities near one another in intermediate layers. We find evidence of this phenomenon across multiple LMs and data types, and further observe that intervening in this space in the model's 's dominant language (usually English) leads to predictable behavior changes.
'One may argue this conceptually similar to half langulae half ialf-image input. There are many distinctions: most importantly, a half-image is not processable by Chameleon and severely goes out of its training distribution, since it only ever processes images of size exactly 512 x 512. Other distinctions include: the presence of a special token marking the beginning of the image; our intervention repeats the new color token, once for each patch, rather than just one; and the token representation is held constant across layers rather than evolving; etc.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
We thank Alex Gu, Alexis Ross, Alisa Liu, Aryaman Arora, Asma Ghandcharioun, Cedegao E. Zhang, Freda Shi, Han Guo, Jack Merullo, Linlu Qiu, Mor Geva, Naman Jain, Ruochen Zhang, Sarah Wiegreffe, Shushan Arakelyan, and Yung-Sung Chuang for discussions and help at various stages of this project. Figure [ uses icons from flaticon. com This study was supported by funds from MIT-IBM Watson Al Lab.
REFERENCES
Mikel Artetxe, Gorka Labaka, and Eneko Agirre. Learning bilingual word embeddings with (almost) no bilingual data. In Proceedings of Annual Meeting of the Association for Computational Linguistics (ACI.), 2017. URL. htt.://nntnooooooooo////-//- wtt///rgg/////../
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language
models. arXiv preprint, 2021. URL WTRLhttps:/// https:://axii.. htttps////rrkv.orrr/b/////66 2210/00332
Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens. arXiv preprint, 2023. URL URLp:tss:///ax wwwp://wwww.rrgp///3a3//// 08112
Kevin S. Beyer, Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft. When is "nearest neighbor" meaningful? Proceedings of the 7th International Conference on Database Theory, ICDT '99, pp. 217-235, Berlin, Heidelberg, 1999. Springer- Verlag. ISBN 3540654526.
BigScience. Bloom: A porb-parmeeer open-access multilingual language model. arXiv Preprint, 2023. https/sts////aaa...oom wttp://wwww....a/a///1.......
Steven Cao, Nikita Kitaev, and Dan Klein. Multilingual alignment of contextual word representations. In Proceedings of the International Conference on Learning Representations (ICLR), 2020. URL
URL http:s//pppaarreevvvveeeeeerrrrr
Chamelcon-Team, Chameleon: mixed-modal farly-fusion foundation models. arXiv preprint, 2024.
09818
Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldwosky-Dill, Ryan Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. Causal scrubbing, a method for rigorously testing interpretability hypotheses. ment Forum, 2022. URL https:///ww/ aliggnentoormmm wttpw/rwww....////////t////////a Sousal-scrubine scrubbing a method for rigorous Ly- testing
Align-
causal-
Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman. Vggsound: A large-scale audio- visual dataset. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2020. Utt/ http::/:///eeeeeeoreee..ee ieee. prg//occumen//965337
Song Chen, Gary Krug, and Stephanie Strassel. Gale phase and 4 chinese newswire parallel text. Linguistic Data Consortium, 2016. URL https:://caallo.. Ldw.ealllol...........o.oo.. www.uu.eweeeeneeee..e ed//DD22016725
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. UtL/tttps://ppeereviie neti/forum1dd-Thhlyy0007z
Alexis Conneau, Guillaume Lample, Mare' Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. Word translation without parallel data. In Proceedings of International Conference on Learning Reprexentations (ICLR), 2018. URL http:::://aaaiiooog/aab/////////
04087
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsuper- vised cross- lingual representation learning scale. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2020. URL
acl www...nn..... 747
11

---

Published as a conference paper at ICLR 2025
Manuel Carlos Díaz- Malz-,aiano,, Eugenio Martinez-Cánen Miguel Ángel García Cumbreras, Manuel García Vega, and Julio Villena-Román The democratization of deep learning in tass 2017. Proces. del Leng. Natural, 2018. URL https://api. 13667878
Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg. Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals. Transactions of the Association for Computational Linguistics ( (TACL), 2021.
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Danny Hemnandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. https:/transformer.ciccitttt http:/aannnppput22222aa/aaaee/ne html.
Zac Hatfield-Dodee
Charles J. Fillmore. The case for case. In Emmon Bach and Robert T. Harms (eds.), Universals in Linguistic Theory, pp. 0-88. Holt, Rinehart and Winston, New York. 1968.
Gemini-Team. Gemini: A family of highly capable multimodal models. arXiv preprint, 2024. URL https://arxxivorrr/bb///2 11805
Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. Patchscopes: A unifying framework for inspecting hidden representations of language models, arXiv preprint, 2024. httpss///axxii wttpaaawwwi..o.a//////aaa// 06102
Yuan Gong, Alexander H. Liu, Hongyin Luo, Leonid Karlinsky, and James Glass. Joint audio and speech understanding. In Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2023. URLp:tts:://are http:////aaaa..orr/////////// wrtp:/s////b//////////
14405
Yuan Gong, Hongyin Luo, Alexander H. Liu, Leonid Karlinsky, and James R. Glass. Listen, think, and understand. In Proceedings of the International Conference Learning Representations (ICLR), 2024. BtRL/httss///peerreeee
Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Daniel A. Roberts. The unreasonable ineffectiveness of the deeper layers. arXiv preprint, 2024. URL https://arxiv org/abs/2403.1778
Roee Hendel, Mor Geva, and Amir Globerson. In-context learning creates task vectors. In Findings of the Association for Computational Linguistics: EMNLP, December 2023. URL https://
Cclantholoogy ooggolgg/ooooo/2333 findings emnlp.624 624
Matthew Honnibal and Ines Montani. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear, 2017. URL https:///pacy..2
Minyoung Huh, Brian Cheung. Tongzhou Wang, and Phillip Isola. The platonic representa-
tion hypothesis. In Proceedings of Machine Learning Research (ICML), 2024. URL https
//proceedings.mtt mlr. press//235/huh244 /trpreddddngnm.....rrsss///////
html
Gabriel Ilharco, Rowan Zellers, Ali Farhadi, and Hannaneh Hajishirzi. Probing contextual language models for common ground with visual representations. In Proceedings of the Conference the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HILT), 2021. URL htt.tps:/aallaantoooo..ooo
-aa.ll wwt-maain.2.... .422
Mingyu Jin. Qinkai Yu. Jingyuan Huang. Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, et al. Exploring concept depth: How large language models acquire knowledge different layers? arXiv preprint, 2024. URI htttshhtts://axi.v... org/ abs/2494.00 078666
Phillip Keung, Yichao Lu, György Szarvas, and Noah A. Smith. The multilingual Amazon reviews corpus. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.
orgg////oooooooooo222eo emnlp-main.33 www.mainnnn.. 369
12

---

Published as a conference paper at ICLR 2025
Jing Yu Koh, Ruslan Salakhutdinov, and Daniel Fried. Grounding language models to images for multimodal inputs and outputs. In Proceedings of the International Conference on Machine Learning (ICLR), 2023. URL ptttps://axxiv.orr/ass..t 13822
Jiaang Li, Yova Kementchedjhiiva, and Anders Søgaard. Implications of the convergence of language and vision model geometries. arXiv preprins, 2023. URL
06555
Tsung-Vo Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and Larry Zitnick. Microsoft coco: Common objects in context. In Proceedings of the Europeun Conference on Computer Vision (ECCV), 2014. URL https:/: //wpps::///www/// microsoft. com/ coco- common objects-----eeeooooo in- context/
US/ hts.rreeearccchoooolaatttoooooos
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Advances in Neural Information Processing Systems (NeurIPS), 2023. URL.
08485
The Llama Team. The Ilama 3 herd of models. arXiv Preprint, 2024. URL https://arxiv.org/
abs/2487.00 .21783
Jiasen Lu, Christopher Clark, Rowan Zellers, Roozbeh Mottaghi, and Aniruddha Kembhavi. UNIFIED-IO: A unified model for vision, language, and multi-modal tasks. In Proceed- ings of the International Conference on Learning Representations (ICLR), 2023. https //openreview. wttp://wwwne/tommmoo/ooolllloooo
Jiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang, Savya Khosla, Ryan Marten, Derek Hoiem, and Aniruddha Kembhavi. Unified-io 2: Scaling autoregressive multimodal models with vision language audio and action. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. /RLp:ttss://artt 111///22w...0//////331111111
Mayug Maniparambil, Raiymbek Akshulakov, Yasser Abdelaziz Dahou Djilali, Mohamed El Amine Seddik, Sanath Narayan, Karttikeya Mangalam, and Noel E. O'Connor. Do vision and language encoders represent the world similarly? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. http:ttt/////aa..org////a/
05224
MDBG. Mdbg chinese-english dictionary (cc-cedict). MBDG, 2024. URL https://ww.nddg net/ chinese/olccioonry/ppeeeee
cedict Downloaded: 2024-09-25.
Jack Merullo, Louis Castricato, Carsten Eickhoff, and Ellie Pavlick. Linearly mapping from image to text space. arXiv preprint, 2022. URL httpps///arrxx..rrrrga///2299 wwtp:////bb.....ba../g 15162
Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Language models implement simple Word2Vec- style vector arithmetic. In Proceedings of the Conference of the North American Chapter of the Association for Computatioonal Linguistics: Human Language Technologies (NAACL-ILLT), 2024.
URL http:::///ccaaattoooooo//ooo
org/ 2024. naacl- long. .281
Tomas Mikolov, Quoe V Le, and Ilya Sutskever. Exploiting similarities among languages for machine translation. arXiv preprint, 2013. URL http:ss:/aarxiv..rr/abb////////s 4468
John Morris, Volodymyr Kuleshov, Vitaly Shmatikov, and Alexander Rush. Text embeddings reveal (almost) as much as text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023. URL
765
Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space communication. The Eleventh International Conference on Learning Representations, 2023. ItLl/https://openreviie
Jerry Ngo and Yoon Kim. What do language models hear? probing for auditory representations in language models. In Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL.), 2024. htl.htttps://arxi..rr////t
16998
13

---

Published as a conference paper at ICLR 2025
nostalgebraist. Interpreting gpt: the logit lens. LessWrong, 2020. URL http::://w..weeetttette:: wtwps//wwww.sssrrog. ccwp///ots//kkaaaaaaaaaaab/rrr/p
logit lens
Karalyn Patterson, Peter J Nestor, and Timothy T Rogers. Where do you know what you know? the representation of semantic knowledge in the human brain. Nature reviews neuroscience, 8(12): 976-987, 2007.
Matthew Lambon Ralph, Elizabeth Jefferies, Karalyn Patterson, and Timothy T Rogers. The neural and computational bases of semantic cognition. Nature reviews neuroscience, 18(1):42-555 55, 2017.
Abhilasha Ravichander, Yonatan Belinkov, and Eduard Hovy. Probing the probing paradigm: Does probing accuracy entail task relevance? In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2021. URL https: logy org/2021 www.:ee211.cleee. eacl main. 295
Nina Rimsky, Nick Gabrieli, Julian Schulz. Meg Tong. Evan Hubinger, and Alexander Turner. Stecring llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024, URL https://aclanthoologg... org/
2824. acl- long. 828
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint, 2020. URL httpp:://axxiv.orrg///. / 1910.011080
Sunny Sanyal, Sujay Sanghavi, and Alexandros G. Dimakis. Pre-training small base Ims with fewer tokens. arXiv preprint, 2024. URL
08634
Tal Schuster, Ori Ram, Regina Barzilay, and Amir Globerson. Cross-lingual alignment of contextual word embeddings, with applications to zero-shot dependency parsing. In Proceedings of the Con- ference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLLT) 2019, /RL httphttt///alaaatoooooyy//
1162
Samuel L. Smith, David H. P. Turban, Steven Hamblin, and Nils Y. Hammerla. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. In Proceedings of the International Conference on Learning Representations (ICLR), 2017. URL 1d=rr\ab855g
Nishant Subramani, Nivedita Suresh, and Matthew Peters. Extracting latent steering vectors from pretrained language models. In Findings of the Association for Computational Linguistics: ACL., 2022. URL https:://claantholog.. orgp/o/2...../222.... findings-acl acl.48
Junyi Sun. Jieba: Chinese text segmentation tool. Github, 2024. URL https://giitub..com//// jieba, Accessed: 2024-09-25.
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam
Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. Transformer Circuits Thread, 2024. URL https://trans fo former circuits pubp:iiits.ppp///// 20tp///22242c///// ling monosemanticitty 1
Eric Todd, Millicent Li, Arnab Sen Sharma, Aaron Mueller, Byron C Wallace, and David Bau. Funetion vectors in large language models. In Proceedings of the International Conference Learning Representations (ICLR), 2024. /RRLitttss///oeeeeiiee..eee http://wwwww.eeeeeooo//o///aa//e
Michael Toker, Hadas Orgad, Mor Ventura, Dana Arad, and Yonatan Belinkov. Diffusion lens: Interpreting text encoders in text-to-image pipelines. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2024. URL
acl- L24g.524
14

---

Published as a conference paper at ICLR 2025
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie Anne Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. arXiv preprint, 2023a. URL httptttts////aiiii http:::://axxl..rrg/ab..ss
13971
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint, 2023b. URL https://arxiv, http://.ww.rg/aa///...000
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization. arXiv preprint, 2024. URL https::/ www.ooggg////////////// htt///bs//22// 2200 10248
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems (NeurIPS), 2020. URL httts::/prrccedins...urre. wttpripps.cpppperfiileees/////// 925502222211555112ffffaa/a332333
Paper
Liang Wang. Nan Yang. Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multilinguul e5 text embeddings: A technical report. arXiv preprint, 2024. URL https:
Tlaxxiv.org/abs/24000 056722
Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. Do Ilamas work in English? on the latent language of multilingual transformers. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2024. URL httpss///aacaaattooooo.oo///////
acc- Long.ong.BB0 820
Skkimedia-Foundattion Wikimedia downloads, 2023. URL
org
Zhaofeng Wu, William Merrill, Hao Peng. Iz Beltagy, and Noah A. Smith. Transparency Helps Reveal When Language Models Learn Meaning. Transactions of the Association for Computational Linguistics (TACL), 2023. URL http:: http:/doooo.gggg/ 1112//aac..... a a0055
Zijun Wu, Yongkang Wu, and Lili Mou. Zero-shot continuous prompt transfer: Generalizing task semantics across language models. In The Twelfth International Conference on Learning Representations, 2024. URL /tt.t//tt//////ooooo//////////rn
Yan Xia, Hai Huang, Jieming Zhu, and Zhou Zhao. Achieving cross modal generalization with multimodal unified representation. In seveth- Conference on Neural Information Processing Systems, 2023.
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami AI-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the Conference the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2021. URL https:://cclattttth Logy orgg 2ww...2acncaca....m main.41
Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan. Fan Yang. Fei Deng, Feng Wang, Feng Liu, Guangwei Ai, Guosheng Dong, Haizhou Zhao, Hang Xu, Haoze Sun, Hongda Zhang, Hui Liu, Jiaming Ji, Jian Xie, Jan Dai, Kun Fang, Lei Su, Liang Song. Lifeng Liu, Liyun Ru, Luyao Ma, Mang Wang. Mickel Liu, MingAn Lin, Nuolan Nie, Peidong Guo, Ruiyang Sun, Tao Zhang, Tianpeng Li, Tianyu Li, Wei Cheng, Weipeng Chen, Xiangrong Zeng, Xiaochuan Wang, Xiaoxi Chen, Xin Men, Xin Yu, Xuehai Pan, Yanjun Shen, Yiding Wang. Yiyu Li, Youxin Jiang, Yuchen Gao, Yupeng Zhang, Zenan Zhou, and Zhiying Wu. Baichuan 2: Open large-scale language models. arXiv Preprint, 2023. URL https://arxiv.orr/abssss
10305
Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. Do large language models latently perform multi-hop reasoning? Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2024. URL
acl- 550
15

---

Published as a conference paper at ICLR 2025
Englch tuput
inpet
Enghish teput
Chinen ingut
p0B, English laput
a poB; Chiinii input
poB, English input
Zlama 5-808 laput
10
Baichaanz 28, English input
Beidhuana 73, Chisese input
Bloons 7B, English input
Blocm 28, Chiness ixput
Lapsr
Luyar
Enginsh langaage
Engiehh
Chenono. 0ce-1p
Chansio.
pinggage
Another
Language probabilities of English and Chinese (and the top language, when it is neither, which only happens for Bloom). Regardless of the input language, the dominant LM language is more salient in the marly- layers, and the input language is more salient the final layers. Bloom does not have a clear intermediate latent language.
Figure 10:
A EXPERIMENTAL DETAILS FOR 3
A.1 MULTILINGUAL
For Experiment 1, for each sentence pair, we use a template to transform each sentence. This is due to the automatic code-switching behavior of IMs.. For an English model processing Chinese text, we expect the Chinese tokens to have high probabilities in the final layer because they need to be output; however, observe these models tend to code-switch back to their dominant language after a full sentence, which confounds our analysis. We therefore put the parallel sentences into a template, "I (English Sentence) | This represents" (and the corresponding Chinese version), the model is less likely to code-switch mid-sentence after "represents" We experimented with other templates that led to similar results. Furthermore, for sentence in GALE (Chen al., (016)), we make sure the t transcript;
is not empty for both the source and the translation.
For Experiment 2, due to tokenization, it is challenging to obtain exactly parallel English- tokens, and hence we perform aggressive filtering. We consider only text positions where the next BPE token (1) is valid Chinese word (as segmented by Jieba (Sun 2024 and (2) has an English
translation (using the English- Chinnee dictionary CC-CEDICT (MDBG) 2024 For example, "4
H (Today is a happy day). Llama-3 tokenizes as I K. E FF' '0' AA9
E' }. while Jieba segments it to be (' L "T '0''' "-
J. We only keep ( R' Furthermore, only 4 R'' translation a single token, the only token that survives the cutoff
is R.
The full result of Figure is in Figure TO We observe that the 7B/8B and 70B models of the same model family have highly similar trends, so we only consider the 7B/SB models in other experiments.
A.2 CODE
We consider all non-zero-argument function calls in the MBPP dataset, excluding unit tests. We automatically identify the argument names (the "semantic roles") by function inspection for built-in functions and by looking at the function definition for those defined in-context, and skip when this is not possible. We also ignore arguments whose semantic roles are generically called "obj" or "obj bct"e,"" and instances where the instantiated surface-form argument is the same as the semantic role. We look the hidden state corresponding to the previous token, either "(" or " ",, except when tokenization renders this impossible (c.g., when the previous token is merged with a part of the surface argument). This leaves. 540 arguments.
16

---

Published as a conference paper at ICLR 2025
Table 2: Steering Llama-2's output sentiments using trigger words in English the input language (either Spanish Chinese). report the mean sentiment, disfluency (perplexity), relevance the continuation. as well as the standard deviation across 10 seeds. Cross Cross-lingual steering is consistently successful, sometimes even more than monolingual steering, without substantial damage in text fluency and relevance.
Text Lang.
Steering Dir.
Steering Lang.
Sentiment
Disfluency (L)
Relevance (1)
None
None
0.144
8.58
0.850
Spanish
0.143
8.84
0.847 #0.006
Spanish
English
0.097
8.991
0.847 10.005
Spanish
0.164
9.11
0.844
English
0.149
8.350
0.849
None
None
0.223=0.030
14.63
0.844
Chinese
0.117
15.29 1:2.47
0.840
+
Chinese
English
0.156=0.076
14.80+2.24
0.842
Chinese
0.359
545.94
$45.9411544.36
00.091
English
0.227=0.038
14.14
0.845 00.009
A.3 VISION-LANGUAGE
To pass the images through the model, embed them in templates, only for the logit lens experiments. For the color experiment, we use the template "USER: what is the color in the image?<image>An ASSISTANT:". For the caption and segmentation experiments, use "USER: what is in the image?\nsimage>" for Chamelcon.
ASSISTANT: " for LLaVA and "What is in the
For all caption and segmentation experiments, we use the MSCOCO 2017 dataset. For the caption evaluation, for each image patch (with the associated caption for the entire image), we compute scalar patch-caption alignment score (for each layer separately) with nouns in the caption. We consider words with NOUN or PROPN tags given by SpaCy's en_ core.web. trf model (Honnibal Montani 2017).
For the segmentation evaluation, we use the MSCOCO 2017 panoptic segmentation labels. The metric calculation is similar as above. Instead of scalar patch-caption alignment score, we consider a scalar patch-label alignment score between a patch and its matching segmentation label, computed likewise using the logit lens. We consider a patch and a label as matching if there is an image segment with that label that occupies more than half the pixels in the patch. Under this definition, a patch cannot have more than one label. When patch is not matched with any label, we disregard it. In the baseline, we use a randomly chosen incorrect label from all possible labels for the alignment score. Finally, we average this alignment score across all patches in all images obtain the curves in Figure 7c and and 7d
B EXPERIMENTAL DETAILS FOR
For both the code and lision-language intervention experiments, we use argmax decoding.
B. MULTILINGUAL
For each language, we sample N 1000 instances from the training set of InterTASS, for Spanish and the multilingual Amazon reviews corpus for Chinese. Following Turner et al. (2024), we use trained models for various metrics. We automatically evaluate the sentiment the generation using a DistillBERT-baase (Sanh al. 2020) model finetuned for multilingual sentiment analysis, judge the generation fluency by taking the conditional perplexity of the generation given the prefix from
httpss//iuuiinaaae..o///wws/sssr has.rarittaateemmmiiiiinuuglall cased- sentiments student
17

---

Published as a conference paper at ICLR 2025
Ons 55m Increaae Over Basellne
6/7
0.20
Caass
Liama-
taglah
Cosine Siniaccyy
0.65
0.00
Laats 5
Logit Lets Log Pooh
Lians
Arbbic mumced)
0.00
000
0.00
030
Layer
Lager
(a) Cosine similarity between an (b) Same as (a), but only the exact (c) Logit lens log probability when arithmetic expression in Arahic nu- translation similarities subtracted by predicting number, between either merals vs. English words, broken the others.
the number itself or its English equiv- alent.
down into separate categories.
Figure 11: Results for the arithmetic experiments. The 95% Cs plotted in all. Expressions in Arabie numerals have similar representation corresponding expressions in English, as well the unembeddings of corresponding number words in English.
Llama-3.1-700 (Llama-3- Team 2024 and compute the relevance of the generation with the prefix by computing the cosine similarity between the generation and the prefix using a KLM-R-Larger based (Conneau et al. 2020) model finetuned for sentence representation (Wang et al. 2024). All these models support both Spanish and Chinese.
We perform ActAdd by passing both the positive and negative steering words through the LM, taking their hidden states at layer 17, computing their difference, scaling by a constant, and adding it to the normal generation forward pass also at layer 17, exactly following Turner et al. (2024), except we use a scaling coefficient of 5, rather than 2 in their experiments, for which we observed larger effect. For generation, we use temperature of 1, top-p=0.3, and frequency penalty of 1, all following Turner et al. (2024), without tuning.
We showed the Llama-3 intervention results in Table 1 and here in Table 2 we show the results on Llama2, with similar trends.
C EXPERIMENTS WITH ARITHMETIC EXPRESSIONS
We hypothesize that similar trend exists when LMs process arithmetic expressions where they route shared space anchored by numerical words in English intermediate layers. We consider simple expressions in the form of "a=b+c" or "a=bvc"; for simplicity, we restrict "a" and "b" to be at most two digits and "c" to single positive digit.
Experiment 1: Representations are similar for translations. Here, we only consider the right- hand side, "b+c" and "bxc", as 1i in Eq. [ Like in the multilingual case, we translate them into English (e.g., "five plus three") as wirrr and evaluate the representation cosine similarity between every English expression and every numeric expression, throughout layers. We group the pairwise cosine similarities in three buckets: (I) exact translation (c.g., "5+3" and "five plus three"; N 1123), (2) non-exact but same value (e.g., "5+3" and "two plus six"; N 13293), and (3) different value (N 1247836). Figure Tha that exact translations have high cosine similarity, although this is be expected since embeddings of numbers and their corresponding English words are near one another (thus even a bag-of-word word-embeddings should also have high similarity). More interestingly, we that the similarities are still higher when the surface forms distinct but the "meaning" of the expression (i.e., the value of the expression) is the same. Next, like 1 we subtract the cosine similarities among con-translation pairs as baseline (wizz) Figure shows high similarity in the carly-middle layers for translations over the bascline, but gradually decreasing to near O.
Base Mistral- Nemo- Base 2407), and found similar trends.
We
also
tried using
https://huggingfacee.c///ssssss
18

---

Published as a conference paper at ICLR 2025
Second Principal Component
25
Proportion Instances (%)
80
ECED THEN BRga
ungea Other
Liama
20
Layer
10
Intervention Strength Coefficient
First Principal Component
Figure 12: Steering arithmetic expressions' re- sults different value.
Figure 13: The Llama-3 hidden representation evolution when predicting number, projected by PCA where the principal components learned the output embeddings of 20 number tokens, 10 in English and 10 numerals.
Experiment 2: Representations are anchored by semantically-eqquivaai English words. We hypothesize that, for some prefix such as "a=b+", the intermediate representations fi are close the English word for "c" that would make the equality hold. First, we randomly sample 100 such prefixes and take the representation of the last token all layers. For each prefix, we plot the representation evolution throughout layers using PCA, as well as the unembeddings of numbers in English y vs. numerals 7= Figure 13 shows that the representations indeed go through the space occupied by the English words in intermediate layers. Next, we repeat our logit lens experiments, inspecting the log probability of the following numeral token vs. its English version (N 1123). Figure TIc shows that the two tokens have similar log probability until around layer 25, after which the numeral token dominates.
Experiment 3: Intervention. We perform intervention using our arithmetic expressions, for example "4=1+3". We intervene by attempting modify the token after "+" be smaller, e.g. "2" here, and expect this to not only lead the model to output "2" instead of "3", but also fundamentally affects the model's reasoning process and causes the model to patch this error with an additional suffix "+1", i.e., "4=1+2+1". We use ActAdd except for adding the intervention vector (e.g.. "three"
"two"") only at the position of "+" "+"P For all addition expressions in our data (N 846), we perform such intervention at an intermediate layer (25 for Llama-3 and 30 for Llama-2) and measure how often this leads to the model correctly outputting the decremented number followed by "+1", versus unchanged, or changed to some other output. Figure 12 shows that, as the intervention coefficient (i.e., the scaling constant the vector) increases, this procedure leads to the expected output for up to > 90% of the instances.
Another difference is that we do not use the hidden representation after seeing e.g. "three", because that usually represents the next token. Instead, equals five plus", and take the last token hidden representation, which is supposed to represent "three".
prefix that uniquely determines the number, e.g. "Eight
19


Reviews:
[{"review_id": "E9RsDvmW0k", "reviewer": "Reviewer_id3E", "summary": "This paper aims to show that models process inputs from different languages/modalities primarily by first projecting them into a unified representation space, before projecting them out into their language/modality for the output. They do so for multilingual, code, visual and audio settings. They then perform interventions in the unified space across those settings to support the hypothesis that the models operate in that space in intermediate layers.", "questions": "* Section 2: Notation wise, it is not clear what $M_{LM}$ is. It strictly reads as just the embedded tokens in this section (L102), but it also looks like a general, vague reference to mid-layer representations later in the paper.\n* Equation 1 needs to be explained: is this the formulation of your hypothesis? If so, it should be introduced like: \"Formally, our hypothesis can be formulated as:\"\n* What do you mean by \"scaffolded\" as used throughout the paper? The term is not defined but used early on in the introduction.\n* Typos: (1) L288: Should this refer to Figure 8? (2) All references to Llava should be 'LLaVA' as per the original paper.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "* The paper builds on Wendler et al. (2024), which only looked at Llama trained pre-dominantly on English, to show that models trained predominantly on other languages do indeed have that language as their intermediate representation space\n* They show that this also extends to multimodal input, which is what supports the claim of a 'unified' representation space.\n* They perform some intervention experiments to support the hypothesis, and show that intervening in the unified representation space can affect outputs in other modalities/languages.", "weaknesses": "* In Line 108, they mention that 'absolute similarity measures are generally difficult and unintuitive to interpret in high dimensional spaces' and hence choose relative similarity measures instead. This is a key choice in the experimental setup, and I would hope to see more discussions on whether prior work have chosen to use relative over absolute similarity measures.\n* The intervention experiments seem to be somewhat weak. For them to strongly support the unified representation space hypothesis, I would expect a comparison showing that  dominant data type steering is comparable or more effective than non-dominant steering. This doesn't seem to be the case as monolingual steering seems to be on the whole better than crosslingual steering (Table 1). However, I do think that the fact that English steering works at all weakly suggests that hypothesis is true.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JHYfhnf7pS", "reviewer": "Reviewer_EcEH", "summary": "The paper proposes a hypothesis that models can learn a unified representation space across multiple modalities, such as language and vision, as well as across multiple languages. Specifically, the authors hypothesize that: (1) Semantically related concept pairs from different modalities or linguistic spaces are represented more closely than unrelated pairs. (2) When given an input, the model continues in the dominant modality (e.g., language for a language model), even when the input format is symbolic, such as code.\n\nTo validate the first hypothesis, the authors calculate the cosine similarity between paired data (e.g., parallel translations, images and captions) and non-paired data, observing that semantically paired data have higher similarity scores. For the second hypothesis, they evaluate whether tokens that match the input context have a higher probability than random tokens. The results suggest that the model favors continuations in the dominant modality.\n\nThe authors further examine cross-modal correspondence by intervening in one modality and observing changes in others, covering multilingual, code, image, and video data. For example, by replacing image hidden states corresponding to one color with the text embedding for another color, they find that the model correctly predicts the new color with 80% accuracy when replacements are made from the first layer onward.\n\nIn conclusion, the authors present empirical evidence through similarity analysis and cross-modal interventions to support that models can learn a semantically unified space across languages and modalities.", "questions": "1. The description of Figure1, and Figure2 is missing?\n\n2. In Figure 6, the intermediate tokens shown are predominantly English words rather than the matched label token. Could the authors clarify whether the tokens from the last layer align with the matched token? If so, the results may not be surprising, as the intermediate layers likely serve to associate the input with internal knowledge in the dominant space, while the later layers have output circuits that transform this dominant representation back into the output format. This aligns with my earlier point about the expected behavior of the model’s translation mechanism (see Weakness Point 2)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The topic of interpretability in multi-modal representations is highly relevant, and the hypotheses and conclusions presented in this paper—along with the supporting experiments—are likely to inspire further research. \n2. The paper is well-written and easy to follow. \n3. The authors validate their hypotheses across multiple modalities, including multilingual settings, text-code, language-vision, and language-audio, providing a thorough examination of their proposed unified representation space.", "weaknesses": "1: Sub-hypothesis 1, “semantically related concepts are closer than unrelated concepts,” is not novel, as it aligns with the standard training objective of many multi-modal models. This alignment is already an expected outcome, though recent research has highlighted its limitations and proposed methods to improve it [1, 2, 3].\n\n2: Sub-hypothesis 2 suggests that inputs will continue in the dominant space, even if the input format is not in that space. The authors support this by observing that, in models like Llama, intermediate representations shift to the dominant language (e.g., English) when the input is Chinese, with the representation returning to Chinese dominance in later layers. However, this hypothesis seems unrelated to the main idea of a unified representation space, and its findings are largely anticipated. A deeper analysis is needed, potentially identifying specific transformation mechanisms, such as attention heads or MLPs, that facilitate cross-lingual translation. Insights into how these transformations occur would add value, as explored in the Induction Head blog[3], which discusses how output circuits transform intermediate representations into the output space.\n\n3: Lack of discussion of related work in unified representation[1,2] and information flow, especially the intermediate representation to input/output space [3]  \n\nReferences: \n\n[1] Achieving Cross Modal Generalization with Multimodal Unified Representation\n\n[2] UNIFIED-IO: A Unified Model for Vision, Language, and Multi-modal Tasks\n\n[3] A Mathematical Framework for Transformer Circuits", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0dpW8cIj2N", "reviewer": "Reviewer_nLRX", "summary": "This paper proposes the Unified Representation Hypothesis, which predicts that—regardless of the type of data of the inputs being processed—the latent representations of a given concept will be aligned with the dominant data type that the model was trained on. For example, given a multimodal language model trained primarily on English text, the latent representations will reliably align with English text, even if the input is only an image, or in another language.\n\nThe study tests and finds consistent support for this hypothesis in cross-lingual and cross-modal settings, including text-audio, text-image, and multilingual models. These alignments are largely verified using the logit lens, but many experiments also entail counterfactual interventions to hidden states.", "questions": "* Figure 11: The experimental setup should be clarified. Is this a single comparison of the similarity between the logit lens and the mean representation across all nouns in the caption? Or is this an average similarity across separate similarities per noun? Or something else?\n* L360-368: This is pretty light on details.  appendix explaining the experimental settings would be very helpful.\n* Figure 8: There is plenty of vertical space here; consider enlarging this figure. Also: this looks pretty noisy. It could be revealing to add a color for a control setting, where the hidden representation is closer to a token other than these two possibilities. I hypothesize that much of this graph will be filled with the control color, but also that the rows with blue color will be much more consistent across examples.\n* Figures 12 and 13: These values seem small on an absolute scale, and the difference between baseline and the correct answer is also small on a relative scale.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* The study advances an idea that has been more casually discussed in past work: that the representation space of models representing many types of data will be biased toward the dominant data type. This paper proposes and presents evidence for a stronger version: that, when given data of any type, the model’s representation of what to predict next will be more closely aligned with the dominant data type (even in the absence of any data of the dominant type in the input).\n* The ideas are presented in a precise way. While it may take more effort to parse than verbal intuitions, it makes the theoretical assumptions underlying the work clearer.\n* Thorough experimentation. Convincing evidence (often causal) from many models and in many settings.\n* Table 2: It’s nice that the quality of the outputs is evaluated along multiple axes, rather than just overlap with respect to a reference.", "weaknesses": "* The proposed idea does not seem distinct from that proposed in the Platonic Representation Hypothesis paper (Huh et al., 2024), which is cited. That paper also presented similarly diverse and thorough empirical evidence in favor of unified representations across modalities. It would be helpful to explicitly contrast the Unified Representation Space Hypothesis from the Platonic Representation Hypothesis in the paper, or at least to contextualize the main idea and findings here more thoroughly with it.\n* Due to the quantity of experiments in the paper, it was probably difficult to fit the necessary details for each in such a way that makes them all clear. Most of them are well-defined, but I was sometimes not sure what, exactly, was being evaluated. See Questions/Suggestions for details.\n* L108-L113: Relative similarity captures a notion of distance, but it doesn’t capture whether there is underlying structural similarity (e.g., isotropy). Token distance is a nice start, but I have a hunch that going deep rather than wide would yield even more significant insights.\n* L257-258: Speculative. It would be interesting to test this by projecting the inputs onto the space of the dominant data type, and then generating.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5H31NbZNLi", "reviewer": "Reviewer_MWcS", "summary": "This paper investigates whether multimodal LLM  process diverse input types using a unified representation space. The authors utilized \"logitlens\" as an interpretability tool to extract representations from each layer of the LLM. They show that model representations for semantically equivalent inputs from different modalities are similar in intermediate layers and that interventions in this space lead to predictable changes in model behavior.", "questions": "1. Why Bloom is not a english-dominant LLM but Llama is english-dominant? Does it mean that they were trained differently?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.  Extends prior research (Wendler et al.) from multilingual to multimodal settings on unified representation in the middle layers of LLM, offering new insights.\n2. The experiments are interesting and thorough across modalities, including text, code, image, audio, adding robustness to its claims.\n3. Good experiments on intervention in the unified representation space which shows the space causally steers model output.", "weaknesses": "1. The paper is largely an empirical study, and the findings may heavily depend on experimental settings. Some details are unclear, e.g., in multilingual experiment 1, the baseline for non-parallel texts is not well explained.\n2. The paper consistently use the last token's representations over layers for measuring similarities interpretation, which maybe a limitation.\n3. In Figure 9, the cosine similarity increase over baseline is marginal, with a maximum of only 0.03.\n4. In line 435, the paper hypothesizes that the unified representation space is language-agnostic, but the experiments primarily use the English-dominant Llama-3. This needs more justification.\n5. Lack of related work for representation alignment, such as alignment through relative space [1] and its application in LLMs [2].\n\n[1] Moschella et al. \"Relative representations enable zero-shot latent space communication\"  \n[2] Wu et al. \"Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)