Shared Imagination: LLMs Hallucinate Alike
Authors: Yilun Zhou
Caiming Xiong
Silvio Savarese
Chien-Sheng Wu
Venue: Accepted by TMLR ()

Abstract: Despite the recent proliferation of large language models (LLMs), their training recipes -- model architecture, pre-training data and optimization algorithm -- are often very similar. This naturally raises the question of the similarity among the resulting models. In this paper, we propose a novel setting, imaginary question answering (IQA), to better understand model similarity. In IQA, we ask one model to generate purely imaginary questions (e.g., on completely made-up concepts in physics) and prompt another model to answer. Surprisingly, despite the total fictionality of these questions, all models can answer each other's questions with remarkable consistency, suggesting a "shared imagination space" in which these models operate during such hallucinations. We conduct a series of investigations into this phenomenon and discuss the implications of such model homogeneity on hallucination detection and computational creativity.

Published in Transactions on Machine Learning Research (07/2025)
Shared Imagination: LLMs Hallucinate Alike
Yilun Zhou
yikn.zhhonc@sekefooer.ccom
Salesforce Al Research
Caiming Xiong Salesforce Al Research
criong@@aaleforee.com
Silvio Savarese Salesforce Al Research
sanwrresecoollesforee.com
Chien-Sheng Wu
ww.jaason@salesoreees
Salesforce Al Research
Project website: https: /ttps://yilee //yiluzzhuu. io/ io
Reviewed on OpenReview: https: openreview. net/ formmm id- NUXpBMt.DY
Abstract
Despite the recent proliferation of large language models (LLMs), their training recipes model architecture, pre-training data and optimization algorithm are often very similar. This naturally raises the question of the similarity among the resulting models. In this pa- per, we propose novel setting, imaginary question answering (IQA), better understand model similarity. In TQA, ask model to generate purely imaginary questions (e-g., completely made-up concepts in physies) and prompt another model to answer. Surpris- ingly, despite the total fictionality of these questions, all models can answer each other's questions with remarkable consistency, suggesting a "shared imagination space" in which those models operate during such hallucinations. We conduct a series investigations into this phenomenon and discuss the implications of such model homogeneity on hallucination detection and computational creativity. We will release and maintain code and data on public website.
a multrle fe cho on quaedarr medeeup comeapt
parayraph mede-sp cantept
Arpwest The following quaodion by chooolig the baot
98%
89%
mechariam
Quedtio particee Invclved the
Question: Joussiioon
(Nots: thereia Veolooo
Basod on the porograph abovo. wiih tiple choice
quectoon so ind cote the correct answer.
[cholce (choow
62
54%
Priton and elesmmn
Noultino and neution
down quork
Queston:
purposo Peterson Intoroation?
D. (chike D
Direct Quentoree Dontest Questions
Bootion
Answen cannot sppperss be about fetiooal concept
Cornect arooer:
Coreetterss
Arcwerin
Figure Imaginary question answering (IQA). Prompt texts are for illustrative purposes, with exact ones shown in Tab. 2-4. Left. two: question model (QM) prompted to generate an imaginary multiple-choice question and indicate the correct answer, either directly (left) or based the previously generated context (right). Middle right: an answer model (AM) answers the question (with the four choices shuffled and without the context paragraph in the second generation mode), or refuses to answer, citing reasons such as the fictionality of the question. Rightmost: we observe non-trivial correct rale and relatively high answering rate (i.e., low refusal rate), with higher values when AM and QM are the same or from the same model family (shown Fig. 2), and significantly higher values for context-basd based questions.

---

Published in Transactions on Machine Learning Research (07/2025)
1 Introduction
Recently, LLMs have been increasingly used in various applications. Although these models occupy a wide spectrum of model sizes and benchmark performances (Tiang et al., 2022), they also share high degrees of similarities: decoder-only transformer architecture (Radford et al., 2018) with one of a few positional embedding designs (Dufter et al., 2022), pre-training corpus consisting of books, Internet texts and codes (Gao et al., 2020), stochastic gradient descent. (SGD)-based optimization (Kingma & Ba, 2014), and similar procedures for instruction tuning and alignment after pre-training (Ouyang et al., 2022; Rafailov et al., 2024). As a result, is it possible that these models share certain fundamental commonalities?
In this paper, we identify one such commonality: these models agree, to surprising extent, on purely imaginary contents, or hallucinations. Specifically, we propose the imaginary question answering (IQA) task, shown in Fig. 1. A question model (QM) is prompted to generate multiple-choice question, either directly about a fictional concept (top left), called direct question (DQ), or based on previously generated context paragraph about a fictional concept (top right), called a context question (CQ). Although they are impossible to answer with rational decision-malingg we still ask the QM to specify a "correct" answer. Then, in a new session, without any previous interactions or contexts with the QM, we solicit an answer from an answer model (AM), which may be the same as or different from the QM (bottom left).
13 LLMs from four model families (GPT, Claude, Mistral, and Llama 3) achieve an average 54% correctness rale on DQs (with random chance being 25%), with higher accuracy when the AM is the same, or in the same model family, as the QM. More surprisingly, for CQs, which are generated based on an imaginary concept paragraph, the correctness rate increases significantly to 86%, with certain (QM, AM) pairs achieving as high as 96%.
These results show high degrees of agrecment among models on what they hallucinate, which we call "sharod imagination". Focusing on this phenomenon, we study six research questions via carefully designed experi- ments, listed in Tab. 1. These results shed light on fundamental properties of LLMs and suggest that, despite their highly varying benchmark results, they exhibit more homogeneous behaviors than expected.
This homogeneity could have broad implications on model hallucination and its detection if models hal- lucinate homogeneously with each other, can one model really detect that generated by another model? Indeed, recent works identify such difficulty (Chen & Shu, 2023), for which shared imagination could be the fundamental reason.
In addition, this work also casts doubt on LLM creativity if models generate highly homogeneous imaginary contents, can we really use them creative agent/ agent/assistant /assistant in artistic expressions? Similar issues are raised
RQI: Exploratury Data Analysis (Sec. 3.1): What do the generated IQA data (in. questions and context paragrplas) look like? Anewzrr: While data are well-clusteed by topies, questious generated by different models, as well as direct vs. contect questions, look very homogeneous both in the embedding space and by cosine similarity metrics. Word cloud visualization also coufirms the homogeneity.
RQ2: Heuristics for Correct Choice (Sec. 3.2): Are there simple explanations for the high correctnees rate?
Answer: While data inspection and model evaluation identify ways to make predictions better than random chance (e.g,, the correct choice of DQs is most likely to be the longest), none of them suffice to schieve the observed correctness rate. In addition, the correctuess rate is sensitive to the orders.
RQ3: Fictionality Awareness (See. 3.3): Are models aware of the fictiouality of these guestions and context. paragraphs?? Answer:: They ran deteet fictionality oasier in DQ than CQ2 and they can identify fictionality botter when divectly usking a Yrs/No question, but struggle more multiple-choice QA.
RQ4: Effect of Model "Warm-Up" (Sec. 3.4); Does model generation general make model converge to the "shared inagination space" and strengthen the pheuomemun??
Answer: Yes, when a QM generates soveral questions sequcutially, they becomne intressingly casy to answer. For individual questions louger ones also easier answer.
RQG: Universality of the Pheuomenon (Sec. 3.5): Can models other than recent Instruction-tunat models schieve high correctuess rate? Answzer: Pre-ChatGPT models cannot, large ones such as GPT-NeeX 20B, but base versions of receut small models (e.g. Mistral 7B) can.
RQ6: Other Content Types (Sec. 3.6): Does this phenomenon oceur for other content types than (simulated) knowledge concepts?
Answer: Yes, we observe similar results for questions ahout (imagined) short stories in creative writing.
RQ7: Prompt Varlations (Sec. 3.7): Does this phenomenon preserve under different prompting setups?
Answer: Yes, we observe similar results acress different setupe, including three additional question wordings, prompt that elicit sero-shot mhuin-of-thonghe auswer, and usking the question without candidate optious for the model freely generate answer.
Table 1: Six research questions into the shared imagination phenomenon and summary answers.
2

---

Published in Transactions on Machine Learning Research (07/2025)
Role
Mussage
User
Ou the topic of physics, please write multiple choice question around a concept that completely made up. Try to make the problem hard and challenging. In your question, do not say that the concept is hypothetical or fictional. Instead, treat it as if were real and widely accepted. Use the following template:
Question: [question statement)
A. [choice A]
B. choice B)
C. (choice C]
D. [choice D]
Answer:
[the correct choice]
Model (the generated question and onswer)
Table 2: The prompt for direct question generation. The underlined topic is replaced accordingly.
by (Chakrabarty al., 2024), for which we provide a more systematic study with 13 LLMs on a wide range
of topics.
2
Imaginary Question Answering (IQA)
2.1 Framework Description
The setup for the IQA procedure is summarized in Fig. 1. In the direct question generation mode (Tab. 2), the QM is asked to generate a standalone question. In the context-basee question generation mode (Tab. 3), the model first writes paragraph on fictional concept, and then generates question based on it. We call the former direct question (DQ) and the latter contect question (CQ). To prevent biasing the generation in any way, we use zero-shot prompting with no examples.
To elicit answers from the AM, we use the prompt in Tab. 4, with the four choices shuffled in all experiments except for that in Sec. 3.2. When context question is posed, the context paragraph is not provided. Although the format template explicitly instructs the model make selection, there are occasional refuse- to-answer responses, such as "I apologize, but the question seems to be asking about a fictional concept, and hence cannol answer it." We deterministically decode from the AM. Since for all language models, the tokenization of the string of "Answer: X" (where "X" is one of "A", "B", "C", "D") puts "X" on its own token, the doterministic decoding equivalent to selecting the most likely choice letter as the next token, given that. the model does not refuse to answer.
Formally, the QM generates set of questions {(z, vt / where y is the assigned correct answer. Then, the AM predicts Di, where is either one of the choices or R. for refuso-to-answer On these problems, we compute correctness rate as the fraction of correctly answered questions among answered ones, and
Role
Message
User
Imagine that you are writing textbook. Please make up a concept in physics explain with a single paragraph of text. Please write as if the concept is real, and completely avoid saying that the concept is fictional or made-up. Be creative.
Use the Kollowing template:
Concept:
[the of the concept that you writing about]
Content:
la single paragraph of text explaining the concept
Model
(the generated concept and peragraph)
User
Now, based on the paragraph above, write a multiple choice question about this concept. The question should be auswerable using the paragraph. Try to make the problem hard aud (sanc as Tob. 2 aftewwrd))
Model
(the generated question and onswer)
Table 3: The prompt for context-based question generation. The underlined topic is replaced accordingly.
3

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Role
Mussage
User
Answer the following question. Be concise and give the answer only.
(the question and its four choices)
Write your respouse in the following format:
Answer: [the letter (A, B, C or D) of the selected choice)
Model
(the genevaled question and onswer)
Table 4:
The prompt for the answer model. Note that while the instruction does not "allow" the refusal behavior, it still occurs occasionally.
answering rate as the fraction of answered questions among all questions:
Σ hi
Σ 1
(1)
Eisl 1 1
N
2.2 Experiment Setup and Results
We study 13 models from four model families, listed in Tab. 5. QMs use temperature 1 to balance output quality and stochasticity, and AMs use temperature 0 for groedy answer selection.
Taking inspirations from the MMLU benchmark (Hendrycks et al., 2020), we select 17 lopics on com- mon college subjects: mathematics, computer science, physics, chemistry, biology, geography, sociology, psychology, oconomics, accounting, marketing, law, politics, history, literature, philosophy, and religion. Each QM in Tab. generates 20 direct questions and 20 context questions for each topic, for a total of 13 x 17 (20 20) 8840 questions. Tab. 9 of App. A presents some generated questions.
Fig. 2 shows the correctness and answering rates, along with the respective averages. Notably, all correctness rates are higher than random chance of 25%. For DQs, most of the bigh-performing AMs (i.e., shaded cells) are the same (diagonal) or the same model family (block diagonal) as the QMs. On context questions (CQs), the correctness rate increases significantly, from 54% to 86% on average. While the performance difference among AMs are smaller (uniform vertical color patterns), GPT-4 omni, Claude Opus, Claude 3.5 Sonnet, and Llama 70B are the slightly better ones for almost all QMs (horizontal shaded cells).
The answering rate on DQs mostly depend AM some AMs, such as GPT-4, Claude 3 Opus and Llama 3 70B, consistently refuse quest from all QMs, and do not particularly lavor/dislavor their own questions. Quite remarkably, the refusal behavior virtually disappears on CQs, except for Claude 3 Opus, which nonetheless answers much more froquently (44% vs. 87%).
3 Further Analyses
Given the large variance of model capabilities measured by numerous benchmarks, the findings that models tacitly agree with each other on purely imaginary contents are surprising. In the next section, we conduct in-depth analyses by focusing on six research questions listed in Tab. 1.
Model
API or Hugginglace ID
Abbr.
Model
API or Huggingface ID
Abbr.
GPT-3.5
gpt-3.5-turbo-0125
G-3.5
Claude Haiku
Blaude--haiiln-22222 20240307
C-3H
GPT-4
gpt-4-0613
G-4
Clande Sonnet
claude-3-sonmet 20240229
C-38
GPT44 Turbo
spt-4-turbo-2024404444
G-4T
Claude Opus
claud----ppus222242222
C-30
GPT-4 omni
gpt-44-2024-05-13
G-4o
Claude 3.5 Sonnet
C-3.58
Mistral
Mistral-7B. Mistral-7B--usttruc--e -.-tuuuuuut-
M-7
Llama 8B
L3-8
Mixtral Sx7B
Eixtral-&x7B-Insttruot.
M-8x7
Llama 3 70B
Meta-Llama- --0-
L3-70
Mistral Large
mistral- largral-large-21022
M-Lg
Table 5: A summary of models used in the experiments. Model abbreviations are used figures.
4

---

Published in Transactions on Machine Learning Research (07/2025)
Corretneess Direct Questions
Correctrec
Andwaring Rate Direct Queztions
Questions
Anver Model
210
M1u
Question Model
Question Model
Question Model
Question Model
The correctness and answering rate on direct and context questions for each pair of question model (QM) and answer model (AM). Each pink rectangle represents model family. For correctness rate, the top-4 highest performing AMs for each QM are shaded. An enlarged version is reproduced in Fig. 13 of App. A.
Figure 2:
3.1 RQ1: Data Characteristics
By studying the word frequencies of DQs, CQs and context paragraphs, we find that they share many common words, such as "individual", "principle", "phenomenon" and "time" in word cloud visualization (Fig, 14 of App. B).
333
4T
C3H
cat
6:0
c3.55
Mix
L
134
13770
03.5
math
G-3.5
0.70
6:3
physks
G44T
chemistry
G44
biology
C:3H
640
C3H
geography
C:25
0.60
6-H
6-5
socielogy
peychology
C:35
C30
C-3.55
C:30
C3.55
accounting
M.7
63355
marketing
M-1g
0.50
M4gg
history
politics
(unused)
13.8
Mig
L38
literature
134
13-70
philosophy
(unused)
13
religion
(unused)
-0.40
6:3.5
G4T
c38
6:30
63.55
M7
Nig
L38
11-70
Figure 3: Top: question embeddings (computed by text-embbedinng---aarge by UMAP, color-coded by topic (left) and question model (right). A triangle marker indicates DQ, and circle marker indicates CQ. Bottom left: average intra-topic cosine similarity between questions generated by different models, with DQs on the lower-left half and CQs on the upper-right half. Bottom right: the color legends for the three plots.
5

---

Published in Transactions on Machine Learning Research (07 (07/2025)
To visualize the questions generated across topics and models, use OpenAI's rext-embedding---lrgg to compute the embeddings for each question (including the four choices), and visualize them with UMAP dimensionality reduction (Melnnes et al., 2018) in the top panels of Fig. 3. We can see that while questions from different topics are well-chustered (top left), there are no clear patterns for questions from different QMs. Furthermore, there are no obvious distinctions between DQs (triangle markers) and CQs (circle markers).
The lower-left panel shows the cosine similarity between each QM pairs, for DQs on the lower triangle and CQs the upper triangle. The similarity between two QMs my and my is defined as
1 Enco." Exser 92)
(2)
10T
10
where T is the set of topics, and for each E T, Qi" is the set of (direct context) questions generated by QM m. o (gn, 42) is the embedding cosine similarity of and In our experiment, we have [T] = 17 and 12Th = 20 for all m and t.
The similarity values are generally quite similar for each model pair, ranging from 0.44 to 0.63, with Mistral models being most dissimilar from the rest, consistent with the upper-right embedding visualization showing different QMs generate highly similar and homogeneous questions. In Fig. 15 of App. B, we performed analogous analyses on the context paragraphs, and obtained similar findings.
3.2 RQ2: Heuristics for Correct Choice
Human Answer Guessing To intuitively understand the generated questions, we manually answered 340 questions: 10 DQs and 10 CQs randomly sampled from each topic. We try guess answers in the most rational way. While some clues hint at the correct choice or allow for the climination of likely wrong ones (see App. C for details), we struggle to answer most of the questions.
Fig. shows the correctness rate per topic for direct (top) and context (bottom) questions, alongside a few representative models and their averages. Human performance much lower than that of all models, especially on context questions. Interestingly, although we do not perceive any difference between DQs and CQs (questions of both types are shuffled and unlabeled during our answering), our correctness rate on CQs is also higher.
G-3.5
51
50
C.3H
40
52
42
43
52
48
67
63
52
54
63
51
58
41
65
64
51
53
49
60
48
L3-8
33
54
37
37
49
43
64
65
54
62
51.
59
49
61
65
55
52
Avg
63
51
62
51
Human
20
40
30
20
40
30
10
20
60
50
400
1.0
70
60
30
35
G-3.5
93
82
C-3H
89
84
83
79
DO
95
B4
82
85
94
U5
94
70
87
90
09
15
75
L3-8
79
85
85
84
79
83
93
86
85
83
95
86
92
74
86
89
90
86
Avg
92
81
84
Human
70
60
40
40
40
60
70
90
50
80
40
50
40
55
math
comp. sd..
physics
chemistry
biology
geoprantt
sociology
psychology
econonics
accounting
marketing
law
nolitics
history
literturre
ehitsophy
religion
average
Figure 4: Per topic correctness rate of (a subset of) AMs, their average and human guessing, for direct questions (top) and context questions (bottom). Results for other AMs are similar and presented in Fig. 16 of App. C.

---

Published in Transactions on Machine Learning Research (07/202) 20225)
1.0
0.75
0.5
0.25
0.0
G-3.5
G-
G-4T
G-40
C-3H
C-35
C-30
C-3.55
M-7
M-8x7
M-Lg
L3-8
L3-70
Avg
G-3.5
G-4
G-4T
6-40
C-3H
C-35
C:30
C-3.55
M-7
M-8x7
M-Lg
L3-8
L3-70
Avg
Figure 5: Fraction of questions whose correct choice is the shortest (in blue), 2nd shortest (in orange), 3rd
shortest (in green) and the longest (in red), by each QM, for direct questions (left) and context questions (right).
Length Ranking of Correct Choice For each QM, we compute the distribution of the ranking of the correct choice length among the four choices. Concretely, we find the fraction of questions whose correct choice is the shortest (in the number of characters) among the four, the 2nd shortest, the 3rd shortest and the longest. Fig. 5 shows the iostributions, with the four segments in the order above.
For DQs (left panel), the correct answer is almost equally likely to be of any length, with a slight tendency towards being the longest (red). However, for CQs, the correct answer much more likely to be the longest than the rest for most QMs, over half of the questions, the correct answer is the longest. The strong contrast between the two settings, yet the consistency across different QMs, strongly indicate that these QMs share fundamental similarities in their generations, and thus it not surprising that AMs have much higher correctness rate (86,, Fig. 1) CQs. However, simply choosing the longest answer alone is not sufficient, so AMs must have also exploited other signals.
Answer Perplexity In this experiment, we study the model perplexity of each choice as a free-lext response and see whether the correct answer has the lowest perplexity. Specifically, we use the conversation shown Tab. 6. After applying the appropriate chat template the conversation, we evaluate the perplexity of each of the four choice texts as possible complctions at the square location.
Since perplexity calculation requires token-level log- likelihhodd, we only study the four open-source models: Mistral 7B, Mixtral 8x7B, Llama 3 8B and Llama 3 70B. Their perplexities questions generated by each QM is shown in Fig. 6. Selecting the lowest-perplexity answer yields significantly lower correctness rate, on par with human guessing (Fig. 4), further suggesting that the models are using more complex features, possibly interactions among answer choices, to make final predictions.
Answer Choice Shuffling In all experiments, we randomly shuffle the choices before presenting them to the AM. Here, replicate our main experiment (Sec. 2.2) but instead present the answer choices in the native orders generated by QMs. Fig. 7 presents the difference when use the native order, with a green color denoting an increase under the nalive order, and red color decrease. Generally, correctness rate increases, most remarkably on the diagonal (i.e., same AM and QM) for DQs as much as 10%. In addition, the answering rates for DQs decrease, suggesting that AMs are also better at identifying question fictionality (though see Sec. 3.3 for more nuanced analyses).
Role
Message
User
Answer the following question.
fquestion statement without the four choices)
Model
Answer: D
Table 6: The prompt for perplexity evaluation.
7

---

Published in Transactions on Machine Learning Research (07 (07/2025)
M-7
32
28
30
29
37
28
33
29
40
39
47
33
29
33
Answer Model
M-8x7
34
36
29
36
27
33
25
31
28
39
34
39
36
28
32
L3-70
37
31
36
28
31
29
31
30
43
36
42
37
36
34
M-7
72
55
54
59
56
61
53
43
53
62
47
44
56
Answer Model
53
61
54
44
71
45
41
49
50
53
48
38
55
53
53
47
38
49
L3700
76
51
46
56
54
54
50
38
58
61
58
52
45
54
6-
G-T
G-4a
c-1
0-30
c-1.55
M-7
M-txx
M4g
13-
L3-70
Avg
Question Model
Figure 6: Correctness rate for four open-source AMs if the choice with the lowest perplexity were chosen, on direct questions (top) and context questions (bottom).
Summary This section reveals factors that lead to above-random correctness rates, but none them could adequately explains the observed high values. The answer shuflling result implies that there are complex and hidden rules for the correct choice shared among different models.
3.3
RQ3: Fictionality Awareness
The fact that most AMs exhibit high answering rate (i.c., low refusal rate) raises the qucstion of whether the content truly appear fictional the models. In this section, we assess fictionality awareness via two metrics. First, we augment each question with a fifth choice stating "E. This question cannot be answered since the concept does not exist." Second, we directly query the model on the fictionality of the contexts (associated with CQs), using the prompt "Does the following paragraph describe real concept in (topic)?" and compute the detection rate as the fraction of negative answers.
These values are shown in Fig. 8. The choice E selection rate for DQs (left) is higher than that for CQs (right), consistent with the finding that answering rates are lower on DQs than on CQs (cf. Fig. 2). However,
Direct Questions
Context Questions
Correctness Rate
Answering Rate
G
M
Figure 7: The effect of native choice order for GPT, Claude, Mistral, Llama 3 and their Average, with a green color representing an increase under the native order, and a red color a decrease. An enlarged plot with annotated cell values is presented in Fig. 17 of App. D.
8

---

Published in Transactions on Machine Learning Research (07/202) 2025)
100
75
50
25
G-3.5
G-4T
G4o
C-3H
C-35
C-30
C-3.55
M-Lg
L3-8
L3-70
Avg
Chaice Sel Rate for DQs
Choice E Sel Rate for COs
Context Fictionality Detection Rate
Figure 8: Choice selection rate on direct and context questions, and ficitonality detection rate on context paragraphs, for AMs and their average. Breakdown by QM is shown in Fig. 18 of App. E.
the selection rate for CQs (middle) is average much lower than context fictionality detection rate (right), indicating that models can often identify content fictionality when directly queried, but often cannot translate this knowledge to downstream tasks such as question answering.
3.4 RQ4: Effect of Model "Warm-Up"
One possible explanation for the correctness increase from DQ CQ is that the QMs has more tokens to "warm-up", and models warm up in highly similar manners. Thus, hypothesize that generating any preceding content is likely to help the model converge onto this shared imagination space.
Warm-Up Up With Previous Questions We propose new data generation setup, where the prompt (Tab. 10 of App. F) asks the model to generate five questions sequentially. If the hypothesis is true, then we should expect the correctness rate to increase from the first questions to the fifth questions.
For each QM, we run this prompt times on each lopic. The correctness and answering rales per AM on these five question groups are plotted in Fig. (left and middle), with the average shown in the pink line, Both metrics exhibit clear increasing trend from the first to the last generated questions, mirroring that from DQs to CQs in Fig. 2.
Nonetheless, the length ranking distribution of the correct choice, shown on the right panel, does not shift towards the longest, but instead is very consistent across the five question groups, which may explain the small magnitude of the increase.
Warm-U With Current Question If models "converge" while generating previous questions, do they also converge when generating the current question itself? If so, then should expect longer questions to be easier to answer. To study this, we took the original sets of DQs and CQs, and partition each set into 10 subsets according to their length (i.e., number of characters in the question statement and four choices combined). For each subset, we compute the correctness and answering rate for each AM, and plot them in Fig. 10, along with their average in pink. Those for direct questions are plotted in solid lines, and those for
Correctr Rate
Answering
Correct Choice Longth Ranking
0.63
G-41
0.59
0.85
G-40
0.55
0.70
C-1.55
0.51
0.95
(unusodi
L3.70
0.47
(unused)
3st
21d
3rd
4th
5th
2nd
3rd
4th
5th
1st
2nd
3rd
4th
5th
Question Order
Quastion Order
Question Order
Figure 9: Left and middle: the correctness and answering rates for each AM and their average in pink, for questions of different generation order (e.g., 1 on r-axis means first-generated questions and 5 means gestreenera Right: the length ranking distribution the correct choice for the five groups of questions (the shortest in blue, 2nd shortest in orange, 3rd shortest in green and the longest in red).
9

---

Published in Transactions on Machine Learning Research (07 (025)
Correctness Rate
Answering Rate
0.9
0.9
0.7
0.7
0.5
0.5
Shortest
Longest
Shortest
Longest
Figure 10: Correctness and answering rates for each of 10% questions ranked by question + choices length.
context questions are plotted dashed lines. A similar trend is observed longer questions are answered more correctly and frequently.
Summary While both question order and question length are potentially correlated with many other factors, for which a thorough study is left to future work, the consistency of both trends highly suggests that as the generation goes on, the generated content looks increasingly familiar and predictable to not only itself but also other models.
3.5
RQ5: Universality of the Phenomenon
We evaluate whether pre-Chate Dre-ChatGPPT or base models, listed in Tab. 7, can achieve the same high correctness rale (note that M-7 Tnst and L3-8 Inst. are the same as before). We use the prompt below:
Questiou: (question statement
A: [choine
B: choice B
C: choice C]
D: (choice D]
Answer: [
For BERT, we replace the square marker with the [MASK] token and extract the predicted probability on tokens 'A', 'B', 'C', and 'D' respectively. For all other models, we set the context be everything before the square marker and extract the next-token prediction probability on the four tokens (with preceding space needed). Even for the two instruction-tunel models, we use the same method, remove the effect of the chat template.
The correctness rate for each model is shown in Fig. 11. As we can see, most pro-ChatGPT models perform at or marginally better than the chance level, even for the larger 6B and 20B models. By comparison, for M-7 and L3-8, both the base and instruction-tuned (but without chat-template) models perform on par with chat-templated prompts, implying that neither instruction tuning nor chat-templating is required for the high correctness rate.
Model
Huggingface ID
# Params
Model
Huggingface ID
# Params
BERT
bert-largo-cased gpt2-xl
0.3B
M-77 Base
Mistral-7BB-00.
7B
GPT-2
1.5B
M-77 Inst
Mistral-7BB Ins--rutttut--00.22
7B
GPT-Neo
gpt-neo-2.7B
2.7B
L3-8 Base
Meta-Llamat 3-BB
8B
GPT-J
gpt-j-6b
6B
L3-8 Inst
Litta-lamma--B1 3-B-nnssretttt
8B
GPT-NeoX
gpt-neox-200
20b 20b
20B
Table 7: Models used in the universality study.
10

---

Published in Transactions on Machine Learning Research (07 (07/202) 2025)
Direct Question Correctness Rate
Context Question Correctness Rate
Random Chance Level
20
GPT-2
GPT-Neo
GPT:J
GPT-NeXX
L3-8
Figure 11:
Correctness rate of other LLM types as AMs. Breakdown by QM is shown in Fig. 19 in App. G.
After ruling out the factors of model size and instruction tuning, we hypothesize that the shared imagination behavior emerges from pre-training data. However, such details are often not public, even for open-source models such M-7, and some models could even be trained on synthetic data generated by other models, such the ShareGPT datasct. Hence, leave additional investigation to future work.
Even with possibly high degrees of training data overlap, however, we argue that the shared imagination phenomenon is significant in that it reveals common extrapolation tendency. Since we empirically found that Google search on over 90% of the generated concepts returns no mcaningfully relevant results, is very unlikely that these concepts are present in any training corpus, and hence the shared imagination phenomenon likely due to the similarity of other parts of the training recipe.
3.6 RQ6: Other Content Types
We investigate if shared imagination transfers to other content types, in particular creative writing. We generate DQs from an imagined "story about ftopic) with an intricate story plot" with abstract concrete topics such as "friendship", or "an ancient empire". For CQs, the model needs to "write a short story of 3-5 paragraphs about (topic)" and then "write a question about one of its details." See App. H for details.
These questions appear even more "hallucinated", such as "What message was hidden in the antique locket that Sarah discovered in the basement? A: Map coordinates. B: An old photograph. C: A secret code. D: A heartfelt apology note." (correct answer: C). Nonetheless, as shown in Tab. 8, models again achieve higher-than-random correctness rate for DQs and CQs, although the difference smaller. By contrast, the answering rates are similar, both at 84%, which is still very high considering the un-answerability nature of these questions. The detailed brcakdown in Fig. 20 of App. H show the same trend as before: high correctness rate for same or intra-family (AM, QM) pairs for DQs and concentration of high-performing models to a few AMs (horizontal shaded patterns) and the disappearance of self-model advantage (the lack of diagonal shaded patterns) for CQs.
3.7 RQ7: Prompt Variations
Last, we assess the persistence of shared imagination phenomenon with three experiment setup variations: different prompt wordings, zero-shot CoT reasoning (Kojima et al., 2022) and free-response answers evaluated by LLM-as-a-judge. The four different wordings and the CoT prompt used in the first two experiments are shown in Tab. 14. For the last setup, we remove the four candidate options in the prompt of Tab. 4 of App. I. We employ an GPT--4o as judge to score the answer on a scale of 0 to 100 or recognize refusal-to-answer (for calculating answering rate), with 100 being close to or the same as the correct option, 50 being close to or the same as one of the incorrect options (which still exhibits shared imagination), and different from any options. The exact judge instruction shown in Fig. 21 of App I).
Correctness Rate
Answering Rate
Direct Questions
54%
84%
Context Questions
58%
84%
Table 8: Summary result for creative writing questions.
11

---

Published in Transactions on Machine Learning Research (07/202) 2025)
100
Prompt
Prompt
80
Prompt
ohhouhhf
60
Free Recponse
Correct
Correct
Answering
Answering
Figure 12: Correctness and answering rates for four GPT models across different prompt setups.
We query the four GPT models on questions generated by all models, and plot aggregate correctness and answering rates in Fig. 12. Detailed breakdowns shown in Fig. 22-24 of App. I. As wordings and the CoT prompt give very similar responses. Most notably, the free response answers demon- strate similar trends CQs scores higher than DQs, both significantly greater than 0, and DQ answering rate greater than CQs. All such evidence suggests the robustness of the shared imagination phenomenon.
see, different
4 Related Work
Model Homogeneity Most related to our work the platonic representation hypothesis (Huh et al., 2024), which hypothesizes that models learn convergent representations of the real world. We take this hypothesis step further, and demonstrate convergence even on purely imaginary contents. Tn addition, Yax et al. (2024) uses genetics-inspired approach to construct phylogenetic tree based model's next token prediction results, while we propose IQA as a new probe into model similarity.
Model Hallucination The generated context paragraphs and questions can be considered as (intentional) hallucinations (Huang et al., 2023). Laban et al. (2023) and Chen & Shu (2023) found that model-generated misinformation is harder to detect than human-written one, which is reflected in our findings of the higher answering rates on context questions (Fig. 2). On the other hand, the decreased answering rate without answer shuffling (Fig. 7) also suggests that models could be aware of their hallucination, explored by CH- Wang et al. (2023). Our IQA setup could also present interesting challenge for hallucination detection algorithm (e.g. Li et al., 2023; Manakul et al., 2023).
Computational Creativity People have found that LLMs tend to produce repeated syntactic structure (Shaib et al., 2024), score less on creativity metrics than human writers (Chakrabarty al., 2024), and have homogenization effect on people's writing when used as creative support tools (Anderson et al., 2024). Along with these studies, our results, particularly the creative writing task in Sec. 3.6, shed light on the potential limit of creativity (DeLorenzo et al., 2024) that can be produced by these models.
Model Evaluation Many evaluation benchmarks have been proposed that focus diverse model abilities such as math reasoning (Hendrycks et al., 2021; Mao et al., 2024), factual knowledge (Hendrycks et al., 2020; Wang et al., 2024), commonsense knowledge (Sap et al., 2019) and instruction following (Zheng et al., 2024:
Zeng et al., 2023). While these benchmarks demonstrate key capability differences among models, our IQA task reveals striking similaritics among them, lcading a new dimension of model understanding.
Multiple-Choice QA MCQA benchmarks (e.g. Rajpurkar et al., 2016; Hendrycks et al., 2020; Sakaguchi et al., 2021) are widely used to evaluate LLM capabilities, and recent works propose more critical looks into this capability. For example, Zong et al. (2023), Li & Gao (2024) and Gupta et al. (2024) all found that changing the answer orders could decrease model performance MCQA, suggesting possible data contami- nation and model robustness issues. In addition, Xu et al. (2024) found that LLMs often cannot return the "none of the above" choice, even when explicitly instructed. We take inspiration from these studics and incorporate them in our analyses on answer order shuffling and fictionality detection. While existing analyses use written benchmarks, we use purely model-generated,, intentionally fictional questions.
12

---

Published in Transactions on Machine Learning Research (07/2025)
Do-Not-Answer A key aspect of LLM trustworthiness (Sun et al., 2024; Wang et al., 2023) is its ability to refuse answering nonsensical questions (Peng et al., 2024; Brahman et al., 2024), or at least convey its uncertainty (Kadavath et al., 2022; Xiong et al., 2023). However, our results on the high answering rate suggest that many models lack this ability on model-generated contents, which could lead to certain trust issues.
5 Discussion
In this section, we dive deeper into several higher-level discussion questions related to the shared imagination
phenomenon.
The role of training data While the fundamental reason for the shared imagination phenomenon is an open question, for which we hope that our paper more would invite further investigation, we believe that converging pre-training corpus is the biggest contributing factor. Modern model training employs highly overlapping training data, in terms of both the source Common Crawl' for general web content, GitHub² for code, Project Gutenberg for book and Wikipedia for factual knowledge, and the size of the training corpus the order of 10 trillion training data (Grattafiori et al., 2024; Soldaini et al., 2024; Groeneveld et al., 2024). By comparison, the Pile (Gao et al., 2020) used in to train GPT-NeoX (Black et al., 2022), an 20B model that notably lacks shared imagination (Fig. 11), is much smaller in size, at only around 240B tokens, despite also being a mixture diverse sources.
Nonetheless, we emphasize that the shared imagination phenomenon cannot be solely explained by mem- orization of common training data: because the generated concepts, paragraphs and questions are rarely "Google-able", the resulting models extrapolate from similar pro-training data in a highly similar manner, which could be due lo subtle linguistic cues in the training corpus (Shaib et al., 2024) or any inductive biases of the transformer architecture (Lavic et al., 2024).
Mechanisms promoting shared imagination Mcchanistic interpretability (Rai et al., 2024) has been developed to study the internal mechanisms of certain model behaviors, such as arithmetics (Nikankin et al., 2024), compositional reasoning (Biran et al., 2024), question answering (Licherum et al., 2023) and in-context learning (Olsson et al., 2022), by focusing on identifying the circuits (Elhage et al., 2021) or latent features (Shu et al., 2025) that elicit them. In similar vein, see from Sec. 3.4 that the model exhibits gradually stronger shared imagination during token generation, so it could be fruitful to identify mechanisms that promote this phenomenon during generation, which could also inform ways to suppress it when desirable.
Broader implications on LLM trustworthiness Perhaps the most significant implication the shared imagination phenomenon of LLMs is on their trustworthiness (Wang et al., 2023). In particular, Sec. 3.3 presents alarming findings on the fictionality detection rate of LLMs on each other's 's generation. This suggests potential intrinsic limitations of LLMs detectors misinformation generated by other LLMs (Chen Shu, 2023), prompting new paradigms for such endeavors.
Moreover, the shared imagination patterns is only one (though quite extreme) aspect in the broader context of model commonality, as also evidenced by the platonic representation hypothesis (Huh et al., 2024) and grammatical idiosyncrasy of model generation (Shaib et al., 2024). Such commonality may cause diverse LLMs to "mysteriously" fail on the same task, or pose linitations to the performance of ensemble across different LLMs (Chen et al., 2025). Being able to identify and mitigate the negative impact of model commonality could be the key to improve LIM trustworthiness in these cases.
6 Conclusion and Future Work
In this paper, we propose the imaginary question answering (IQA) task, which reveals an intriguing behavior that models can answer each other's purely hypothetical questions with a surprisingly high correctness rate. These results reveal fundamental similarities between models, likely acquired during pre-training with either
'nttps//ccoooncrawl.com
hhttps://github.co com
onttpe://ww..uteee www.nwwwwwteebegggggg org/
'httpe://www.rikkippp http://wwww.kkipi.i..... org/
13

---

Published in Transactions on Machine Learning Research (07/2025)
similar training corpus models training on each other's synthetic outputs, and may lead to more model merging possibilities (Goddard et al., 2024). Furthermore, due to the imaginary and hallucinatory nature of these question contents, such behaviors suggest potential difficulty and open questions in model-based hallucination detection and computational creativity.
For future work; additional model families could be included to identify the extent and root cause of the shared imagination phenomenon, such as Qwen (Yang al., 2024) and Gemini (Team et al., 2023), as well reasoning-finttueed
finetuned models including DeepSeek-RI (Guo et al., 2025) and GPT-o1 (Jacch et al., 2024), and language models that are not based on the ango-segrsssive transformer model (Gulrajani & Hashimoto, 2023; Nie et al., 2025).
A direction complementary to these algorithmic studies is to assess human behavior. In Sec. 3.2 and App. C, we conducted a small-scale human study with paper authors trying to answer the generated questions. Despite getting non-random accuracy, our performance lags behind that of all models, suggesting currently unknown strategies used by them. Future human studies could involve participants with respective domain expertise allow them to use Internet, and reverse the design by asking humans to write such hypothetical questions for LLMs to answer.
Last, we could also study the shared imagination phenomenon beyond the text modality, and incorporate image (Liu et al., 2023) and even video (Tang et al., 2023). The addition vision modality opens up several possibilities for experiment design. For example, the question model could be provided with a vision input and asked to formulate a tnon-answerable" question based on Alternatively, the QM could be presented with four pieces of vision input and asked to generate question with each choice being vision input. Finally, for QMs with image/vidoo vidoo generation abilitics, (Ho et al., 2020) the question model could synthesize its own vision component as part of the question.
References
Barrett R. Anderson, Jash Hemant Shah, and Max Kreminski. Homogenization effects of large language models human creative ideation. In Proceedings of the 16th Conference on Crealivily & Cognilion, pp. 413 42-,425,200 2024.
Eden Biran, Daniela Gottesman, Sohee Yang, Mor Geva, and Amir Globerson. Hopping too late: Exploring the limitations of large language models on multi-hop queries. arXiv preprint arXiv:2406.4 12775, 2024.
Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leaby, Kyle McDonell, Jason Phang, et al. Gpt-neox-20b: An open-source autoregressive language model. arXiv preprint Xiv: 06745, 2022.
Faeze Brahmaan Sachin Kumar, Vidhisha Balachandran, Pradeep Dasigi, Valentina Pyatkin, Abhilasha Ravichander, Sarah WiegrefTe, Nouha Dziri, Khyathi Chandu, Jack Hessel, el al. The art of saying no: Contextual noncompliance in language models. Advances in Neural Information Processing Systems, 37: 49706-49744 2024.
Sky Wang,ag, Benjamin Van Durme, Jason Eisner, and Chris Kedzic. Do androids know they're only dreaming of electric sheep? arXiv preprinl ar Xiv: 17249, 2023.
Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal, Smaranda Muresan, and Chien-Sheng Wu. Art or artifice? large language models and the false promise of creativity. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 1-34, 2024.
Canyu Chen and Kai Shu. Can I1m -ln-geeertted misinformation be detected? arXiv preprint 2rXiv:2309.1177 2023.
Zhijun Chen, Jingzheng Li, Pengpeng Chen, Zhuoran Li, Kai Sun, Yuankai Luo, Qianren Mao, Dingqi Yang, Hailong Sun, and Philip S Yu. Harnessing multiple large language models: A survey on Ilm ensemble. arXiv preprint ar Xic:2502. 18036, 2025.
14

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Matthew DeLorenzo, Vasudev Gohil, and Jeyavijayan Rajendran. Creativeval: Evaluating creativity of Ilm- based hardware code generation. In 2024 IEEE LLM Aided Design Workshop (LAD), pp. 1-5. TEEE, 2024.
Philipp Dufter, Martin Schmitt, and Hinrich Schütze. Position information in transformers: An overview. Compulational Linguistics, 48(3):733-707 763, 2022.
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. A mathematical framework for transformer circuits. Trans- former Circuits Thread, 1(1):12, 2021.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint trXic:2101.0002 2020.
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee's mergekit: A toolkit for merging large language models. arXiv preprint Xrx::22403.138 13257, 2024.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llaa 3 herd of models. arXiv preprint Xrx::24407.2178. 2024.
Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafford, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, et al. Olmo: Accelerating the science of language models. arXiv preprint arXiv:2402.00833 2024.
Ishaan Gulrajani and Tatsunori B Hashimoto. Likelihood based diffusion language models. Advances in Newral Information Processing Systems, 36:16699 16715, 2023.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. ar preprint arXiv:2501. 12948, 2025.
Vipul Gupta, David Pantoja, Candace Ross, Adina Williams, and Megan Ung. Changing answer order can decrease mmlu accuracy. arXiv preprint orXiv:2406.19/ 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXie preprint arXiv:2009.0330 2020.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. ar preprinl arXiv:2103.0388 2021.
Jonathan Ho, Ajay Jain, and Picter Abbecl. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840 -851, 2020.
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaochong Feng, Bing Qin, et al. A survey on hallucination large language models: Principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:2311.05232, 2023.
Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis. arXiv preprint ar arXim:2405.000 07907777 2024.
Aaron Jacch, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Alck- sander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint 2rXiv:2,12.1672) 2024.
15

---

Published in Transactions on Machine Learning Research (07 (07/)025)
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, 7ac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05222 2022.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXi preprint arXiv:1412.6800 6980, 2014.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shol reasoners. Advances in neural informalion processing systems, 35:22199 2222239-2221,, 2022.
Philippe Laban, Wojeiech Kryscinski, Divyansh Agarwal, Alexander Fabbri, Caiming Xiong, Shafiq Joty, and Chien-Sheng Wu. SummEdits: Measuring LLM ability at factual reasoning through the lens of summarization. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 9662-9676, Singapore, Decem- ber 2023. Association for Computational Linguistics. doi: 10.18653 1a//////11222eem//m wwnlm-main.600 main.600.. URL emnlp-maiee 600.
org//023.
Itay Lavie, Guy Gur-Ari, and Zohaa Ringel. Towards understanding inductive bias in transformers: A view from infinity. arXiv preprint arX2::242.00511 05173, 2024.
Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian- Nie, and Ji-Rong Wen. Halueval: A large-scale hallucination evaluation benchmark for large language models. arXie preprint orXiw:2305.11747, 2023.
Ruizhe Li and Yanjun Gao. Auchored answers: Unravelling positional bias in gpt-2's multiple-choice ques-
tions. arxev preprint arXiv:2405.03m00 03200, 2024.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint ar (xXi:222110091 1.09110, 2022.
Tom Lieberum, Matthew Rahtz, János Kramár, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla. arXiv preprint ar xrX::2307700455, 2023.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34911 2023.
Potsawee Manakul, Adian Liusie, and Mark JF Gales, Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303. 08896, 2023.
Yujun Mao, Yoon Kim, and Yilun Zhou. Champ: A competition-level dalaset for fine-grained analyses of Ilms' mathematical reasoning capabilities. arXiv preprint arXto:2401.06961, 2024.
Leland MeInnes, John Healy, Nathaniel Saul, and Lukas Grossberger. Umap: Uniform manifold approxima- tion and projection. The Journal of Open Source Software, 3(29):861, 2018
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. ai preprint ar Xix:22500 099992, 2025.
Yaniv Nikankin, Anja Reusch, Aaron Mueller, and Yonatan Belinkov. Arithmetic without algorithms: Language models solve math with a bag of heuristics. arXiv preprinl arXim: 10.22224, 2024.
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads. arXiv preprint 1TK::2200.11800 2022.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models follow instructions with human feedback. Advances in neuraa information processing systems, (197730-277445 2022.
16

---

Published in Transactions on Machine Learning Research (07/2025)
Xiangyu Peng, Prafulla Kumar Choubey, Caiming Xiong, and Chien-Sheng Wu. Unanswerability evaluation for retreival augmented generation. arXie preprint orXiv:24 12. 12300, 2024.
Alec Radford, Karthik Narasimhan, Tim Salimans, Nya Sutskever, et al. Improving language understanding by generative pre-training. 2018.
Rafacl Rafailov, Archit Sharma, Eric Mitchell, Christopher Manning, Stefano Ermon, and Chelsea. Finn. Direct preference oplimization: Your language model is secretly a reward model. Advances in Newral Information Processing Systems, 36, 2024.
Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. A practical review of mechanistic interpretability for tramsformer-based language models. arXiv preprint arXiv:2407.0264 2024.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint 666..05250055 2016.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99- 106, 2021.
Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. Atomic: An atlas of machine commonsense for if-then reasoning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 3027-3035, 2019.
Chantal Shaib, Yanai Flazar, Junyi Jessy Li, and Byron C Wallace. Detection and measurement of syntactic templates in generated text. ar preprint 000::4007.0211, 2024.
Dong Shu, Xuansheng Wu, Haiyan Zhao, Daking Rai, Ziyu Yao, Ninghao Liu, and Mengnan Du. A survey on sparse onnoodde: Interpreting the internal mechanisms of large language models. arXic preprinl WwXXvv2503.0561. 2025.
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, al. Dolma: An open corpus of three trillion tokens for language model pretraining research. arXiv preprint arXiv:2402.00 001129002001, 2024.
Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, et al. Trustlim: Trustworthiness in large language models. arXie preprint arXiv:2401.05500 2024.
Yumlong Tang, Jing Bi, Siting Xu, Luchuan Song, Susan Liang, Teng Wang. Daoan Zhang, Jie An, Jingyang Lin, Rongyi Zhu, et al. Video understanding with large language models: A survey. arXio preprinl arXim:2312.17, 17432, 2023.
Gemini Team, Rohan Anil, Sebastian Borgcaud, Yonghui Wa, Jean-Baptiste Alayrac, Jiahui Yu, Radu Sori- cut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. aixio preprint arxi::2312. 1805, 2023.
Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, et al. Decodingtrust: comprehensive assessment of trustworthiness in gpt models. In NewrIPS, 2023.
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406001574 2024.
Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can 1lms express their uncertainty? an empirical evaluation of confidence clicitation in 1lms. arXiv preprint orXiw:2306.13063 2023.
17

---

Published in Transactions on Machine Learning Research (07/2025)
Hanzi Xu, Renze Lou, Jiangshu Du, Vahid Mahzoon, Elmira Talebianaraki, Zhuoan Zhou, Elizabeth Garri- son, Slobodan Vucetic, and Wenpeng Yin. Llms' classification performance is overclaimed. arxiv preprinl arXiv:2406.1620 16203, 2024.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint or 3rXiv2412.15111, 2024.
Nicolas Yax, Pierre- Yves Oudeyer, and Stefano Palminteri. Inferring the phylogeny of large language models and prodicting their performances in benchmarks. arXi preprint vrXim:2404.44077 2024.
Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. Evaluating large language models at evaluating instruction following, aiie preprinl Xxx:2330.00041, 2023.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mi-bench and chatbot arena. Advances in Newral Information Processing Systems, 36, 2024.
Yongshuo Zong, Tingyang Yu, Bingchen Zhao, Ruchika Chavhan, and Timothy Hospedales. Fool your (vision and) language model with embarrassingly simple permutations. ar preprint 0rx::22310.016 2023.
18

---

Published in Transactions on Machine Learning Research (07 (07/2025)
A
Additional Results on the IQA Main Experiment
Tab. 9 presents several randomly sampled direct and context questions. The correct choices in bold.
DQL: Claude 3.5 Sonvel, Politics
Which of the following best describes the impact of the Quantum Resonance Voting (QRV) system voter turnout in the 2028 U.S. Presidential election?
A. QRV dramatically increased voter turnout by enabling voters to participate in multiple parallel universes simultancously
QRV had significant impact on voter tarnout, as citizens preferred traditional voting methods
QRV increased voter turnout by allowing citizens cast votes through quantum enfanglement, but raised concerns about privacy
QRV decreased voter turnout due to technical difficulties in implementing the system across rural areas
DQ2: GPT-4 omod, Roligion In the sacred texts of the Zorban faith, which celestial event is believed to herald the imminent return of the Enlightened Sage Paulasha?
A. The Convergence of the Five Stars The Dance of the Solar Serpents The Eclipsc of Twin Moons
The Rising of the Blue Comet
DQ3: Mistra Large, Chemistry Which of the following clements is most likely to undergo process known as "quantum tunneling" in order to form stable compound with noble gas?
A. Nitrogen
B. Carbou
Oxygen
Hydrogen
CQI: Ilama 8B, Physics
The Fractal Permeability QuarkNodes (FPQN) fundamental property of quantum chromodyuamics describes the rate at which entangled particles exchange information through the fabric of spacetime, In a recent study, researchers found that the FPQN of a quark-autiquark pair is directly proportional to the square of the particle's angular momentum. What is the relationship between the FPQN and the energy density of the particle-antipartae A. The FPQN is directly proportional to the energy density.
pair?
B. The FPQN is indepeudent of the energy density.
C. The FPQN is inversely proportional to the energy density.
D. The FPQN is inversely proportional the square of the energy density.
CQ2: Llanaa 3 70B, Mathematics
Which the following statements a direct consequence of geometric structure having high fluxionality?
The structure's curvature remains constant under varying external influencrs.
B. The structure's adaptability to external forces enables it to maintain a stable shape.
C. The structure's geometry is more resistant to changes in its surroundings.
D. The structure's trajectory is more likely to exhibit chaotic behavior over time.
CQS: Cloude 3 Soanet, Litenature
According the theory Narrative Resonance, which of the following factors is NOT believed to contribute to narrative's ability to resouate deeply with readers on subconscious level?
Universal human experiences
B. Archetypical themes
C. Adherence to established literary conventions
D. Symbolic motifs
Table 9: Example direct questions (DQs) and context questions (CQs), with correct choices in bold.
19

---

Published in Transactions on Machine Learning Research (07 (025)
Fig. 13 reproduces enlarged version of Fig. 2.
Correctness Rate Direct Questions
Correctness Rate Context Questions
G:3.5
55
55
50
47
47
44
48
41
55
58
94
89
91
88
82
57
84
77
76
85
G-4
57
82
8
53
25
50
58
46
56
56
93
92
89
91
87
88
93
86
77
85
80
75
87
G-4T
53
56
84
54
50
50
51
46
31
58
46
42
54
95
91
89
92
84
89
89
87
79
83
94
79
77
87
G-40
54
13/
56
18
52
15
59
11
48
10
36
96
93
92
96
B8
90
92
91
R1
82
81
89
C-3H
50
54
54
52
53
33
50
48
56
56
54
44
55
95
90
86
91
63
88
89
192
81
81
74
74
85
Answer Model
C-35
51
55
59
48
51
51
52
48
55
56
46
43
52
95
90
91
92
87
89
90
85
79
C:30
91
58
88
69
68
83
57
135
55
91
90
96
90
92
95
89
90
93
86
79
89
C-3.55
55
82
58
14
03
51
56
18
106
96
93
93
89
92
M-7
53
45
91
84
80
73
M-8x7
55
50
45
93
86
M-Lg
50
L3-8
58
54
L3-70
Avg
Answering Rate Direct Questions
Answering Rate on Context Questions
G:3.5
100
100
100
100
100
100
100
100
100 190 100 100
100
100
200
100
100
100
100
100
100
100
100 190 100 100 100 100
G-4
72
77
85
75
57
75
73
82
80
64
75
99
96
99
97
37
96
97
97
96 100 95
97
G-4T
1.00
100
100
100
100
100
300
100 100 100 100 100 100
100
300
100
100
100
100
100
100 100 100 100 100 100 100 100
G-40
98
130
160 100
99
100 100 100 100 99 100 100 160
100
300
100
100 100 130 100 100 100 100 100 100 100 100 100
C-3H
1.00
100
100
100 100 100 100 100 100 100 100
100
100
100
100
100
100
100 100 100 100 100 100 100 100 100 100
100
Answer Model
C-35
94
94
94
93
84
93
93
94
95
97
91
91
93
100
100
99
99 100 99
99
99
99
100 100
99
99
99
C:30
55
30
29
21
22
46
48
44
70
34
92
87
86
86
92
86
90
93
98
96
C-3.55
76
84
89
73
60
81
79
85
83
300
100
100
89
99 98
99
99
99
M-7
90
96
98
55
300
59
99
99 1.0 100 100 100
100
1.00
M-8x7
100
100
180
1.0 100 100
M-Lg
100
100
100
L3-8
100
99 100
L3-70
Avg
G-3.57
G4
G-4
G-40
C.3H
C-35
C330
C3.55
M-7
M-8x7
M.lg
L3-8
L3-70
Avg
G-3.5
G-4
G-41
C-40
C-3H
C330
C3.55
M-7
M-8x7
M-Lg
L3-8
L3-7D
Avg
Question Model
Question Model
Figure 13: Enlarged version of Fig. 2.
B Dataset Visualization
Fig. 14 visualizes word cloud of the generated questions and context paragraphs.
Fig. 15 presents analogous figure to Fig. 3, but instead studies the context paragraphs. In particular, the left column visualizes the paragraph embeddings computed by OpenAT's text-embedding-3-lar color- coded by topic and question model respectively. The upper-left panel shows the average intra-topic cosine embedding similarity, and the bottom panel shows the cosine similarity on term frequency-invers document frequency (TF-IDF) vectors. We observe similarity trends as in Fig. 3.
C Human Answer Guessing Details
When trying guess answers, we were able to identify some signals that suggest the correct choice least eliminate some wrong ones. Below some findings:
20

---

Published in Transactions on Machine Learning Research (07/202) 2025)
Direct Questions
Context Questions
Context Paragraphs
system
duestudy number
reseurce
quantuuuuun um state
result change even
p
tempor
quantum
effect
phenomenon
incip
Echrone
based
al
use
e
product due toldd
Jeasing
arhance
factor spectriecohsumer
phenomenortheor 1ikely
use Felat aattiinn tionship
pr r incc wwccpplllllln
Callow
atiie
gunng cono
lext
rather
unique
ecorooiics tasestttteee tocctti
postt
individee
syst
phenomenoo
peeaan potettia
quantunt f
ield
paittes
enaweven even
priaue
behavior fect
link
value
undividual
alllo
ooe soc
exper
ience
allowing
Fiee
change
in Tuen er
en
impact
leading
specttic influence
Interattioniio
Luence
known
theorys
resulting
ro
e
company
ressan
traddittiion
process
partic
mntecata
ethi
value sys
change
one ecoioonc
llce tempora aarl
sense
erier
telr
Spraiin Spryttes aonuatttins
pertain
value process nar ative
va
mayaf
unders Landin ing
mayrolssonhipn
ffffect
time
Sunee
tern
ee
principle
Ereating lead
rate
onedec
state
enotional
time increased
Hight
company
new
Cer from
behaviii ior form
temporal
dynami
time increase
leading T
Increase create
experiencebasediee
"wy refe fe
Wherge
off
Figure 14: Word cloud for direct and context questions (including the four choices), and context paragraphs.
Color-Coded by Topic
Embedding Similarity
G-3.5
G-4
G-4T
C-35
C:30
C:3.58
M-7
0
Mig
L38
L3-70
Coder-Coded Model
TF-IDF Similarity
0.35
G-3.5
G44
0.30
G-40
C-3H
0.25
C-35
C.30
0,20
M-7
0.15
M-8x7
Mig
0,10
L3-70
G:355
G-4
G-4T
C:35
C:30
C3355
M-8x7
M-gg
L3-8
L3-70
Figure 15: Visualization and similarity analyses for generated context paragraphs.
1. When the question introduces term and only some choices mention this term, the correct answer is more likely to be among them. As example, question starts with "A company is experiencing a high rate of Flibulation", and only one choice mentions "Flibulation", which turns out to be the correct choice.
Related the heuristic above, sometimes there are common sub-words between question statements and choice texts. For example, a question asks about "mucleocytokinesis;, and one choice includes "cytoplasm" while others do not have any words that share sub-words with the concept.
21

---

Published in Transactions on Machine Learning Research (07 (07/2025)
3. The question describes concept in neutral to positive manner, but some choices sound slightly negative. They can often be eliminated. An example is CQ3 in Tab. 9, where the correct choice is less positive than the rest (note that the negation in the question).
4. There may be semantic matches between the question statement and choice texts. For example, the question asks "Which phenomenon is used to seloctively reduce the strength of correlations in entangled quantum systems?" The answers are "A. Quantum superposition decay", "B. Quantum interference enhancement", "C. Quantum entanglement dampening", "D. Quantum entanglement crcation". We can be pretty cortain that the correct choice is between A and C.
It should be noted, however, that these identified heuristics are far from sufficient to achieve the observed model correctness rate, as shown in Fig. 16, which presents an expanded version of the correctness rate visualization in Fig. 4, with all answer models included. As we can see, models score very similarly to each other on each topic, but show variance across different topics.
Direct Questions
Context Questions
G-3.5
G-4
50
82
46
80
91
G-4
46
C-31-
C-35
24
37
22
61
52
82
32
26
83
06
50
55
74
30
30
C:30
C.3.55
87
19
30
26
52
82
15
8-8x7
26
20
23
8
92
72
15
70
76
90
M-Lg
13-8
23
24
37
21
8
x
#
22
84
86
8
1 12
28
23
29
8
M
R
2
#
82
8
Human
math-
phypics
chem strry
bology
geed sociology
poychology:
economies
accounting
marketing
law
politics
llterature
philosaphy-
relgionn
averaage
moth-
comp. sc.
physics-
chemistry-
Nology
enograp
soctology
exycholoe
economic
accouning
marketing
law-
history
llteroiture
philosaphy
raligion-
average-
Figure 16: An expanded version of Fig. 4, showing the correctness rates of all AM models and their average.
22

---

Published in Transactions on Machine Learning Research (07 (07/2025)
D Answer Shuffling Result Details
Fig. 17 reproduces enlarged version of Fig. 7, with annotated cell values.
4 Correctness Rate on Direct Questions
A Correctness Rate on Context Questions
G-3.5
-1
6
1
G-4
4
4
i
4
G-4T
G-40
C-3H
Answer Model
C-35
C-30
M-Lg
L3-70
Avg
A Answering Rate on Direct Questions
A Answering Rate on Context Questions
G-3.5
0
G-4
-1
4
0
2
4
4
4
G-4T
9
0
G-40
C-3H
Answer Model
C-35
C-30
C-3.55
M-Lg
L3-70
Avg
9
9
9
9
C3.55 8
9
G-3.5-
G-4
G-4T
G-40
C33H-
C335
C:30
M-7
M.8x7
M-Ig
L3 8--
L3-70
Av9
G33.5
G-4
G-4T
G-40
C-3H
C.35
C3.55
M-8x7
M-Lg
L3-8
L3-70
Avg
Question Model
Question Model
Figure 17: An emlarged version of Fig. 7, showing the effect of removing answer shuflling on correctness and
answering rates. A positive (green) value means that the value would higher without answer shuffling, and a negative (red) value means that the value would lower.
23

---

Published in Transactions on Machine Learning Research (07 (07/2025)
E Fictionality Awareness Experiment Details
Fig. 18 shows the choice E. selection rates on direct and question paragraphs, well fictionality detection rate for each (AM, QM) pair.
Choice E Selection Rate
Choice E Selection Rate Context Questions
Context Fictionality Detection Rate
Direct Questions
G-3.5
11
12
14
22
11
69
G-40
68
C:3H
46
42
32
51
54
34
26
32
16
23
32
40
55
12
10
10
16
15
11
45
31
35
52
39
28
40
17
16
35
61
53
35
Answer Model
C:35
61
39
46
69
34
24
24
27
82
63
52
40
74
94
74
C-3.55
52
M-7
28
11
M-8x7
44
42
15
47
59
41
14
25
22
22
14
16
37
14
16
12
15
14
3.9
1.0
1.9
16
12
19
44
34
55
2%
38
1.6
15
26
69
59
14
M-1g
5/
24
70
64
L3-70
Avg
61
70
41
40
59
14
32
28
24
29
39
14
14
28
61
68
37
73
55
6:3.5
3
C33
C35
C30
03.35
M-7
M-8X7
Mi
L3-8
L3-70
ANg
G-33
6-00
C3H
C35
C330
L3-70
Ava
G35
G-4T
G-40
C3H
C35
G3.55
W--
M-8x7
L3-70
Avg
Question Model
Question Model
Question Model
Figure 18: Choice E selection rate and fictionality detection rate for each (AM, QM) pair.
24

---

Published in Transactions on Machine Learning Research (07 (07/2025)
F
Prompt for Sequential Question Generation
The prompt for generating five sequential questions is shown in Tab. 10.
Role
Message
User
On the topic of physies, please write five multiple choice questions around concepts that are completely made up. Make sure that these questions are cistinct from each other. Try to make each problem hard and challenging, In your question, do not say that the concept is hypothetical or lictional. Instead, treat were real and widely accepted. Use the following template:
Question [question statement]
A. [choice A]
B. choire
ccoocee C
D. [choice D]
Answer: [the correct choice]
Question Iquestion statement]
A. [choice A]
B. cchoice
(choice C
D. [choice D]
Answer: [the correct choice]
Question iquestion statement]
(elsoice
cchoice
(choine
D. (choice D)
Answer: [the correct choice]
Question "question statement]
[choice
choice
[choice
D. [choice D]
Answer: [the correct choice]
Question iqesstion statement]
[choice
chaice
choice
D. [choice D]
Answer:
[the currect choice)
Model
(the gemernted question and answer)
Table 10: The prompt for direct question generation, The underlined topic replaced accordingly.
25

---

Published in Transactions on Machine Learning Research (07 (02))
G
Universality Result Details
Fig. 19 presents the result of evaluating the correctness rates of other L.M types AMs for all QMs, expanding the average values across QMs shown in Fig. 11.
Direct Questions
Context Questions
BERT
25
21
24
25
23
21
24
25
21
21
23
24
25
23
26
26
23
26
24
28
25
22
21
25
22
24
GPT-2
25
26
23
27
24
Answer Model
GPT-Neo
25
21
25
25
26
22
24
29
22
27
28
27
25
25
25
22
27
28
24
23
24
27
30
25
28
25
24
25
GPT-1
24
28
26
31
31
23
33
27
27
30
32
29
30
34
31
30
GPT-NeoX
26
25
26
29
27
25
23
26
26
35
30
26
24
24
32
22
32
30
27
29
M-7 Base
47
52
41
42
42
44
42
59
55
59
47
40
48
90
84
86
86
79
84
84
81
79
91
76
71
82
M-7 Inst
53
L3-8 Base
50
46
48
45
46
47
49
40
51
50
55
48
43
48
91
85
86
86
81
84
85
84
77
76
89
75
66
82
L3-8 Inst
51
45
43
48
58
52
93
86
86
78
89
84
G3355
6.4
C3H
C-35
C:30
C33555
M-7
M-8x7
M-Lg
L3-8
L3-70
Ava
G3355
6-41
G-40
C3HH
C-35
C-30
C33555
M-7
M-8x7
M-Lg
L3-8
L3-70
Avg
Question Model
Question Model
Figure 19: Correctness rates of other LLM types on all QMs.
H Creative Writing Experiment Details
Tab. 11 and 12 present the prompts for generating direct and context questions in the creative writing experiment. The prompt used by the answer model is the same as before (Tab. 4). We generated 20 DQs and 20 CQs for each of the following 10 topics: friendship, family relationship, young adulthood, an ancient empire, an interpersonal conflict, roadtrip, a childhood in poverty, future technology, long-lasting war, an intergalactic civilization.
Rale
Message
User
Imagine story about friendship with intricute story plot. Without telling the story, write question about of its details, and also indicate the correct answer. Do not explicitly reference the story in the question (Le., do not plrases such 'in the story' 'according to the text"). Use the following template:
Question: [guestion statemeut]
A. (choice A]
choice
cchoice
[choice D]
Answer:
[the correct choice]
Model
(the genernted question und answer)
Table 11: The prompt for direct question generation. The underlined topic is replaced accordingly.
26

---

Published in Transactions on Machine Learning Research (07/2025)
Role
Message
User
You excellenn writer. Write short story of 3-5 paragraphs about friendship. Be creative, develop intricate story plot and include lots of details.
Model
(the generated concept and paragraph)
User
Now, write question about its details, and also indicate the correct Do explicitly reference the story the question (Le., do not use phrases such 'in the story" "according to the text"). Use the following template:
Question: (question statement]
A. [choice A]
choice
cchoice C]
(chuice D]
Answer:
[the correct choice]
Model
(the genevated question and answer)
Table 12:
The prompt for context-based question generation. The underlined topic is replaced accordingly.
Tab. 13 presents questions for the creative writing setup in Sec. 3.6, with correct choices in bold.
DQ1: Mistrol &r1B, Priendship
Which character, before the climax, reveals their hidden identity to the protagonist?
The wise old mentor
B. The long-lost sibling
C. The jealous rival
D. The unsuspecting friend
DQ2: GPT-4 Turbo, An ancient empine
What the name of festival during which High Priest Zoroth attempted his betrayal?
Festival the Harvest
Festival of the Sands
C. Festival of Lights
D. Festival of Solaris
DQ3: GPT-4, An interpersonal conftict
What the real reason for Sophia secretly reading Luey's diary?
Sophia attempting plagiarize Lucy's poems
Sophia was looking incriminating evidence against Lucy
Sophia was trying to understand Lucy's feelings towards her Sophia was snooping into Lucy's 's personal life out curiosity
DQ2: Mistral 7B, A childhood poverty
Which flower Maria find and carefully murture?
A. A red rose
B. A jasmine plant
C. A wilting daisy
D. An abandoned sunflower seed
CQ2: Cloude 3 Hatuu, Future technology
Which advanced technology
by the protagonist track the location their missing loved one? Quantum entanglement communication device
Nanite swarm reconnaissance drones
C. Gravitational warp field generator
D. Holographic survelllance system
CQ3: Llama 3 70B, long lasting
What the primary reason for the disputed border between the Eastern Realm and the Western Empire?
powerful artifact was discovered in the contested territory.
centuries-old trade agreement was never formally ratified.
lorg-togg treaty was intentionally mistranslated to conceal hidden clanse. series brutal skirmishes sparked cycle revenge attacks.
Table 13: Example direct questions (DQs) and context questions (CQs), with correct choices in bold.
27

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Fig. 20 shows the detailed breakdown of statistics presented in Tab. 8.
Correctness Rate on Direct Questions
Correctness Rate on Context Questions
G-3.5
51
G-4
68
G-4T
58
G-40
C-3H
Answer Model
58
64
50
44
64
47
M-8x7
50
44
48
40
57
51
46
62
65
52
44
55
47
62
40
53
50
52
51
59
53
45
58
47
42
45
66
58
57
52
66
63
61
46
58
51
65
44
58
56
L3-8
53
68
51
31
59
49
51
44
60
61
57
79
53
64
68
55
40
63
41
71
47
54
54
48
49
55
L3-70
36
52
52
42
40
52
48
60
50
31
65
54
61
68
48
50
60
34
69
66
67
61
86
17
60
Avg
55
57
52
48
66
47
49
52
84
58
50
48
54
54
61
66
60
45
58
48
67
50
61
82
50
52
52
56
Answering Rate on Direct Questions
Answering Rate on Context Questions
G-3.5
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
G-4
56
G-4T
100
100
100
100
100
100
G-40
C-3H
100
Answer Model
100
100
300
100
18
18
18
54
55
60
60
48
46
631
45
98
96
85
91
93
93
86
M-8x7
100
100
100
99
100
99
97
100
100
1.0
100
300
98
100
100
100
100
100
100
190
100
100
M-Lg
94
99
96
98
90
91
98
99
97
54
95
85
98
91
93
94
97
96
94
L3-8
94
89
97
53
98
84
98
93
99
98
99
89
94
98
95
93
94
93
90
96
92
90
51
57
93
L3-70
94
97
98
99
93
90
99
97
97
96
99
84
100
96
93
93
97
96
92
95
94
96
89
94
98
Avg
82
85
86
85
81
78
84
83
84
81
90
86
84
8
83
89
80
83
84
8*
G-3.5-
6.4
G-1
G46
C3H
C30
C3.55-
M-7
M-8x7
M.Lg-
L3-8
L3-70
Avg
G-3.5
G-4
G-1
G-40
C3H
C30
C3.55-
M-7
M-8x7
M-L.
L3-8
L3-70
Avg
Question Model
Question Model
Figure 20: Correctness and answering rates for creative writing questions.
Prompt Variation Experiment Details
Tab. 14 presents the exact prompts used in the prompt variation studies.
Prompt
Text
Answer the following question. Be concise and give the answer only. (the defoslt prompt)
Answer the following question with the letter for best option. You need explain.
Choose the best option for the question below. There is no need for any explanation.
4
Please select the best cholce to answer the question, without giving any reasoning.
CoT
Answer the following question. First, paragraph think step by step through the question. Then, output the final answer.
Table 14: Four different prompt wordings (the first being the default one used in other experiments) and the zero-shot thouughh prompt.
28

---

Published in Transactions on Machine Learning Research (07/2025)
Fig. 21 lists the exact prompt template used by GPT-40 to grade models' free response answers.
You are an impartial teacher to grade the answer written by a student. Below, you are given multiple-choice question and four choices, along with the correct choice. The student, however, answered the question based on the question statement only, without seeing the four choices. So he wrote his answer in free text format. You need to score the quality of the student answer, on scale of 0 to 100. Assign a score of 100 if the student answer is essentially the same the correct choice, and lower score accordingly for any deviation. Assign a score of 50 if the student answer is essentially the same as one of the wrong choices, and a lower score accordingly for any deviation. If the student answer is very different from any of the choices, assign a score close to O. However, if the student refuses to auswer the question by pointing out that the concept asked in the question appears to be fake or fictional, output 'Refuse to answer' instead. Below is the question:
[Question)
A. [Choice A}
B. [Choice B}
C. (Choice C)
D. {Choice D}
The correct answer is: {Correct Choice Letter). {Correct Choice Text)
The student answer is: (Answer Model Response]
Below, output an integer score for the student answer, or 'Refuse to answer'. Do not output any- thing else.
Figure 21: LLM-as-a-judge prompt template used to grade the free response answers.
Fig. 22 plot the detailed breakdown of correctness and answering rates under each of the four prompt wordings, grouped by the (AM, QM) pair. Except for some decrease on DQ answering rates under the first prompt (i.e., the default one used in other experiments), there are minimal effects of wording differences.
Correciness
Questions 5o
Content Questions
55155 14155
25141
25
47 15
50 6 na 50
4:0 41
91
DR
0T
BE 07
BE ST BE ES
E5 BI 77 82 16
81
76
15,76
70
05
B3
40
50150
55145
12
60
50
16:30:0
90
95
25
50
10
15
26
51
22
53
46
92
91
19
26
19
87
56
92
51
92
28
79
76
39
50
46
50
92
93
05
25
12
11
52
54:40
41
14
02
91
91
82
05
10
10
14/57
40
53
53
16:46
55:48
57
56
57
47
48:45
43
05
91
91
79.76
87
87
15
02
635
G4
G-TT
6-6e
CAH
C-15
C-10
C-..S
H-7
H-677
Mig
138
13-70
Ang
6-3.5
G-1
G-4T
G-la
C-HH
CSS
C-30
N-7 Mi-c2
Mrlg
134
15-70
Avg
Queeton
Quedion Model
Figure 22: Correctness and answering rate breakdown for four different prompt wordings. Within each block of (AM, QM) pair, the four numbers represent those under the four wordings. Asterisks denote 100%.
29

---

Published in Transactions on Machine Learning Research (07 (025))
Fig. 23 plot the detailed breakdown of correctness and answering rates under the chain-of- prompt. Fig. 24 plot the detailed breakdown of correctness scores and answering rates in the free-response answering setup (as judged by GPT-4o using the prompt in Fig. 21). The CoT result is very similar to the non-CoTT ones, and while the correctness scores for the freo-response setup is significantly lower, we see that most trends, notably the higher correctness scores and answering rates on CQs than DQs, are still preserved.
Correctness Rate on Direct Questions
Correctness Rate on Context Questions
G-3.5
51
49
49
46
45
48
48
42
54
53
63
45
42
49
93
87
88
88
82
86
87
83
77
81
91
73
74
84
Answer Model
G-4
G-4T
53
62
56
50
49
47
57
47
53
55
68
48
42
53
94
90
90
84
87
88
88
82
79
80
87
G-40
54
57
46
53
47
57
49
41
94
92
86
91
84
81
79
Avg
53
55
48
49
54
45
57
68
49
43
94
90
85
88
89
87
83
79
86
Answering Rate on Direct Questions
Answering Rate Context Questions
G-3.5
100
Answer Model
G4
71
84
66
51
77
66
80
83
81
75
70
99
95
97
93
95
97
95
85
92
95
G-4T
94
98
100
99
96
97
98
98
97
98
100
100
100
100
100
100
99
100
100
100
99
100
99
100
100
94
99
97
97
98
98
96
97
98
100
100
100
99
100
100
99
100
100
Avg
89
96
85
92
90
94
94
93
92
92
92
100
99
99
98
99
99
98
96
99
G-3.5
G4
G44
C.HH
C3355
M-7
M--X77
M-Lg
L3-8
L3-70
Avg
G-3.5
G-
G-4T
C-3H1 G-40
C:30
C3.55
M-7
M-8x7
M-lg
L3-8
L3-70
Avg
Question Model
Question Model
Figure 23: Correctness and answering rate breakdown for the chain-of-thought prompt.
Correctness Rate on Direct Questions
Correctness Rate Context Questions
G-3.5
49
66
Answer Model
G-
36
36
38
30
48
50
41
40
42
51
54
38
29
41
76
60
53
60
69
65
63
59
60
68
69
45
51
61
G-4T
G-40
39
32
38
32
36
48
43
40
45
48
59
41
29
41
82
61
55
63
70
67
64
56
65
69
70
48
49
6.3
Avg
39
37
29
40
47
38
48
56
39
30
79
53
68
65
63
51
68
46
49
61
Answering Rate on Direct Que stioons
Answering Rate on Context Questions
G-3.5
92
97
96
96
96
95
94
99
97
97
96
97
97
96
100
98
99
100
99
98
100
96
100
98
98
99
98
99
Ánswer Model
G-4
G-TT
79
90
93
87
85
80
84
89
91
89
82
92
89
87
99
94
97
99
97
96
97
92
99
95
96
93
96
96
G-4o
85
94
92
91
88
92
94
89
94
92
91
99
98
97
96
99
91
98
96
98
97
Avg.
81
90
81
83
89
88
87
89
85
99
95
96
96
88
96
92
95
G:3.5
J
3
C3H
C35
C:30
C3.55
M-7
M-8K7
M-Lg
L3-8
13-70
Avg
G:3.5
G4
G.4
G-40
C3H-+
C335
C:30
C3.55
M-7
M-8x7
M-1g
38
13:70
Avg
Question Model
Question Model
Figure 24: Correctnoss and answering rate brcakdown for the chain-of-thought prompt.
30


Reviews:
[{"review_id": "W4GQJjNukq", "reviewer": "Reviewer_RgkF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "S1. The paper introduces a creative concept IQA, which provides a new lens for understanding LLM behavior. Compared to standard factual QA, IQA spotlights how models handle content that is intentionally fictional or impossible to verify by external references. Findings are interesting and could reshape how hallucination and creativity in LLMs are understood.\n\nS2. Methodology is clear and robust, systematically exploring numerous dimensions such as question types (direct and context-based), model types, and various experimental setups.\n\nS3. The analysis is thorough. The authors test multiple models (GPT-3.5, GPT-4, Claude variants, Mistral, Llama 3) and systematically vary prompts, question-generation procedures, and topics, and delve deeply into key dimensions (e.g., impact of question order, question length, base vs. instruction-tuned models, and variations in prompt structure). It explores multiple plausible heuristics and explanations for observed phenomena.\n\n---\n\nW1. The mechanisms driving the \"shared imagination\" phenomenon are not deeply explained; the theoretical underpinnings remain somewhat speculative. While the authors thoroughly document the shared imagination phenomenon, the manuscript stops short of providing an in-depth mechanistic explanation of why these models converge on similar fictitious content. Including at least a speculative discussion on possible reasons (e.g., overlap in training data or alignment processes) would strengthen the paper. \n\nW2. The manuscript uses external embeddings to cluster or visualize the imaginary questions. This is a minor critique, but the choice itself might introduce an external bias. While those results are indicative, an ablation or sanity check with alternative embeddings could add robustness.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "- This paper introduces the Imaginary Question Answering (IQA) task, where one LLM generates imaginary multiple-choice questions on entirely fictional concepts, and another LLM attempts to answer these questions.\n\n- The authors experiment with multiple recent LLMs (e.g., GPT series, Claude, Mistral, and Llama 3) and systematically prompt one model (the “question model”) to generate fictional multiple-choice questions, then ask another model (the “answer model”) to respond.\n\n- Despite generating entirely fictional and thus logically unanswerable content, many models demonstrate a high degree of consistency and correctness when answering questions posed by other models. This phenomenon, referred to as \"shared imagination\" by authors, suggests significant underlying similarities among various LLMs.\n\n- The work systematically explores this phenomenon across different settings, question generation modes, content types, context paragraphs, prompt lengths, sequential “warm-up” effects, and differences in model families.\n\n- Their findings suggest that modern LLMs might share “imaginative” representations not strictly found in real data, raising questions regarding hallucination detection and the true creativity and independence of these models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I found the work quite interesting with thorough experimentation and engaging content. I would like to suggest a few improvements on certain aspects of the writing to significantly enhance the paper’s potential impact.\n\nRC1. It is strongly recommended to expand explicitly on potential reasons why large language models (LLMs) frequently converge on similar fictional content in mechanical (or theoretical) aspects. Possible contributing factors to explore include overlaps in training datasets (this is mentioned in the paper but could be discussed more thoroughly), specific alignment or fine-tuning procedures, and inherent properties of autoregressive architectures. The authors could provide a detailed section or an extended paragraph to hypothesize how these factors might reinforce each other, resulting in a phenomenon akin to \"shared imagination\".\n\nRC2. Clearly articulating how the proposed framework can be effectively utilized by researchers and practitioners would significantly improve the paper. Additionally, highlighting possible issues regarding user trust (particularly scenarios where the model’s fabricated outputs might mistakenly be perceived as factual) would strengthen the relevance of the discussion from an AI safety perspective. Explicitly addressing how incorrect agreement or shared illusions among models may undermine trustworthiness would also be valuable.\n\nRC3. The paper touches upon crucial ethical considerations regarding misinformation and the trustworthiness of language models. To enhance this important discussion, the authors should systematically address potential risks associated with generating convincing yet completely fabricated content, outlining clearly the dangers of misuse. Additionally, greater clarity on recommended mitigation strategies and ethical guidelines for both developers and end-users of LLM technologies would significantly reinforce the practical utility and impact of this work.", "broader_impact_concerns": "Please see above.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0i7TVLfgs9", "reviewer": "Reviewer_1Uhb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1.The proposed task is novel and new. The task sounds an interesting point to view some aspect of LLMs and it is also interesting to measure the similarity among different LLMs. \n\n2.The paper is easy to follow and understand.\n\n3.The proposed methods sound easy to implement and reproduce. The authors also release the source codes.\n\n4.The experiments are solid and they authors collect a great number of  phenomenon under their proposed setting.\n\nWeaknesses:\n\n1.Although the proposed methods is easy and clear, the technoical novelty of the method is limited. Although this paper’s main contribution is around the proposed new task instead of the method, it would be better to xxx\n\n2.  Some figures need to be refined. E.g. the figure for prompt text in section 3.5.\n\n3.In the analysis section, the RQ should be a research question, which should not be a term (RQ1: Data Characteristics).\n\n4.The related work does not provide a detailed description of the related research. It involves too much research topics and the connections between the topics are not so dense. For each research area, the author did not provide a comprehensive analyses of the related papers.\n\n5.The authors provide many experimental results in multiple aspects but the insights and the in-depth analysis behind the results are not so sufficient. It is better the see some qualitative and generalized conclusion through the  experimental results. If there is no qualitative conclusion out-of-the-scope of the proposed task (IQA), the motivation of proposing such a task is not so strong.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new task, imaginary question answering (IQA), with its benchmarks. This new task aims to better model and measure the similarity among different large language models (LLMs). In that task, the authors propose to prompt one LLM to generate purely imaginary questions (e.g.,on completely made-up concepts in physics) and prompt another model to answer. They conduct a series of experiments and collect the phenomenon.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "see the weaknesses", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mrrZzsmdfm", "reviewer": "Reviewer_K71b", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n1. Very interesting experimental setup that is automatically generated by different combinations of models using a pool of concepts\n2. Seven different research questions to assess different capabilities and behaviours of the models\n3. The authors test fourteen different models including both closed source and open source ones\n\n## Weaknesses\n1. Please report some additional details regarding the human evaluation. How did you set up this study? How many human annotators were involved? I think it's really important to have a robust human baseline for this study.\n2. [minor] It would be interesting to see how Qwen-2 performs considering that it's been showing very good performance across the board \n3. I find this work very interesting from a philosophical point of view (i.e., what's an imaginative world when everything is expressed via text and how do LLMs process it?) but I think the authors should spend some time in their manuscript to discuss the implications of this work for additional downstream applications and more fundamental research in the development of LLMs. I think this will make the manuscript even appealing to a wider audience.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present a new study that addresses a very interesting question: what would LLM answer to questions about imaginative objects and scenarios? More concretely, the authors propose the imaginary question answering task where a model is prompted to generate multiple-choice questions about a given topic, and then ask another model to generate an answer. The authors propose this task in two settings: 1) where the model has to generate an answer based on just the question; 2) where the answer model has also access to additional context that can be used to provide an answer. \n\nThis authors complete this study using 14 different LLMs including both open-source and closed-source models. This provides a very comprehensive set of results for the evaluation that is used to shed light on 7 different interesting research questions such as different data characteristics, heuristics used by models to generate the correct choice, whether the models are actually able to spot imaginative contexts, and others. Overall, this represents a very interesting investigation in the overall internal representations that these models learn.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Please clarify the criterion you used to select the different types of concepts you have used for the evaluation.\n2. In paragraph 2.1, it would be useful to clarify whether you're following a likelihood-based evaluation or a generative one\n3. I find the caption of Figure 1 a bit confusing because the authors refer to \"bottom left\" and \"bottom right\" but the image is on the same level (at least as far as I can tell).\n4. Figure 6 at the moment it's supposed to show correctness rate for open-source models but there are also closed source ones. I think this is confusing because it's not possible to evaluate closed source models without access to likelihood scores.\n5. Figure 9 refers to Figure 3 for the legend. This is a bit inconvenient and I would suggest the authors to report the legend in the same image.\n6. Consider adding a final section where you discuss the results as a whole (think about it as a \"Discussion\" section) where you provide some indications regarding the implications of this work. For instance, it would be interesting to see how this work could be extend to multimodal models such as Vision and Language models or image/video generation models.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)