FlexAC: Towards Flexible Control of Associative Reasoning in Multimodal Large Language Models
Authors: Shengming Yuan
Xinyu Lyu
Shuailong Wang
Beitao Chen
Jingkuan Song
Lianli Gao
Venue: NeurIPS 2025 poster ()
Keywords: hallucination
creativity
multimodal large language model
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We propose FlexAC, a training-free method that enables multimodal large language models to flexibly control their creativity and factuality by modulating associative reasoning.

Abstract: Multimodal large language models (MLLMs) face an inherent trade-off between faithfulness and creativity, as different tasks require varying degrees of associative reasoning. However, existing methods lack the flexibility to modulate this reasoning strength, limiting MLLMs' adaptability across factual and creative scenarios. To bridge this gap, we propose equipping MLLMs with mechanisms that enable flexible control over associative reasoning. We begin by investigating the internal mechanisms underlying associative behavior in MLLMs and  find that: (1) middle layers play a pivotal role in shaping model’s associative tendencies,  (2) modifying representations in these layers effectively regulates associative reasoning strength, and  (3) hallucinations can be exploited to derive steering vectors that guide this modulation. Building on these findings, we introduce Flexible Association Control (FlexAC), a lightweight and training-free framework for modulating associative behavior in MLLMs. FlexAC first induces hallucination-guided intermediate representations to encode associative directions. Then, it selects high-association instances to construct effective associative steering vectors, whose strengths are adaptively calibrated to balance creative guidance with output stability. Finally, recognizing the multi-dimensional nature of associative reasoning, FlexAC incorporates task-specific associative vectors derived from a forward pass on a few target-domain samples, enabling models to follow diverse associative directions and better adapt to creative tasks. Notably, our method achieves up to a 5.8× improvement in creativity on Creation-MMBench and a 29\% reduction in hallucination rate on CHAIR, surpassing existing baselines and demonstrating its effectiveness in enabling flexible control over associative reasoning in MLLMs. Our code is available at https://github.com/ylhz/FlexAC.

FlexAC C
Towards Flexible Control of Associative Reasoning in Multimodal Large Language Models
Shengming Yuan
Xinyu Lyu xinyulyu680gmail.cm
Shuailong Wang' welliongglong@@mall.coom
shengming yuan@outlook.coom
Beitao Chen' chenbei chenbeitao@gmaiin www.gmaall..ooom com
Jingkuan Song' Langkuan.song@@eee...
Lianli Gao't
lianli, gww.uestc. edu. cn
University of Electronic Science and Technology of China Southweesernn University of Finance and Economics, Chengdu, China TTongii University
Abstract
Multimodal large language models (MLLMs) face an inherent trade-off between faithfulness and creativity, as different tasks require varying degrees of associative reusoning. However, existing methods lack the flexibility to modulate this reason- ing strength, limiting MI.LMs' adaptability across factual and creative scenarios. To bridge this gap, propose equipping MLLMs with mechanisms that enable flexible control over associative reasoning. We begin by investigating the internal mechanisms underlying associative behavior in MLLMs and find that: (1) middle layers play pivotal role shaping model's associative tendencies, (2) modifying representations in these layers effectively regulates associative reasoning strength, and (3) hallucinations can be exploited to derive steering vectors that guide this modulation. Building these findings, introduce Flexible Association Control (FlexAC), lightweight and training-free framework for modulating associative intermediate rep-
behavior in MILMs. FlexAC first induces hallucinatiin--uuide resentations to encode associative directions. Then, it selects high-association instances construct effective associative steering vectors, whose strengths are adaptively calibrated to balance creative guidance with output stability. Finally, recognizing the multi-dimensional nature of associative reasoning, FlexAC incor- porates task-specific associative vectors derived from a forward pass on few target-domain samples, enabling models to follow diverse associative directions and better adapt to creative tasks. Notably, our method achieves up to 5.8x improvement creativity on Creation-MMBent
and 29% reduction in hal- lucination rate on CHAIR, surpassing existing baselines and demonstrating its effectiveness in enabling flexible control over associative reasoning in MIL.Ms. Our code is available at https: //gt thub. com/ htte/ylhz/FFe FexxA.C
Introduction
In cognitive science, divergent and convergent thinking represent two distinct modes of human associative behavior: convergent thinking relies on typical, fact-based associations support faithful
"Co-first author.
CCoresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Image Caption
Associative Strength
Event Planning
Describe the main content of the image.
Plan a grassland camping event from the image.
A brown and while dog runs on grass with a rope toy. Trees and buildings are visible.
1. Set up tents for camping.
14.00
14.2 CHAIR
2. Organize ouldoor games.
CHAIR
3. Host barbecue in the open ait.
happy beagle races across the park with its favorite loy, heading back to its owner during fun game of fetch.
Others
FlexAC
I. Host "Mountain Treasure Hunt" for plants,
2. Create mini eco-photo exhibit with collected
1,78
high
1.6
3. Host movie screening with barbecue.
VDAT
VDAT
4. Conclude the day with stargaring.
Figure 1: Different tasks require different levels of associative reasoning: factual tasks (e-g-, image caption) benefit from lower association, while creative tasks (e.g., event planning) thrive on higher association. Existing methods suppress hallucinations at the cost of creativity (e.g..-1.78 on VDAT: "Others" " from Ha-DPO). FlexAC enables MLLMs to adjust associative reasoning strength accordingly.
reasoning, whereas divergent thinking engages atypical, context-dependent associations to foster creativity [1]. Recent studies show that multimodal large language models (MLLMs) [2, 3, 4] exhibit brain-like properties, such structured embedding spaces [5], cross-modal integration [6], and higher-order cognitive functions [7], indicating that they emulate human associative processes. Consequently, like the human brain, MILMs require the capacity to flexibly regulate associative reasoning strength to support both faithful reasoning and creative generation.
However, existing methods lack the flexibility to modulate associative reasoning strength, limiting MLLMs' adaptability across factual and creative scenarios. On one hand, current hallucination miti- gation techniques, such Contrastive Decoding [8, 9, 10] and Direct Preference Optimization [11], focus on improving faithfulness but often suppress associative reasoning capabilities, thereby hin- dering performance on tasks involving imaginative understanding and literary expression. On the other hand, how to enhance MLLMs' creativity in a controllable and task-specific manner remains underexplored. For instance, as illustrated in Figure 1, existing hallucination mitigation techniques improve faithfulness (14.0 J. in CHAIR) but lack mechanisms for enhancing creativity, resulting in reduced associative reasoning strength (1.78 in VDAT) and poor performance on tasks such as event planning. This gap highlights the need for equipping MLLMs with controllable mechanisms to flexibly modulate associative reasoning strength based on task demands.
To enable controllable modulation of associative reasoning strength, we begin by examining how associative behavior emerges within MILMs. Drawing inspiration from prior works [12, 13], we hypothesize that hallucination and creativity arise from shared associative mechanisms, whose manifestations vary with task demands. To validate this, we collect input-response pairs containing both grounded (ow-association) and hallucinated (high-associattion) outputs, and analyze their internal representations to uncover how associative behavior is reflected within the model. Our analysis reveals three key findings (see Section 2.1 and Section 2.2): (I) Associative behaviors are primarily encoded in the middle layers, where the representations of grounded and hallucinated responses become distinctly separable; (2) Modifying internal representations at these layers can effectively alter the strength and direction of associative reasoning; (3) Direction of hallucinated representations can stimulate associative reasoning capability, offering potential control signal for this modulation. These findings indicate that associative tendencies are encoded in middle layers and can be modulated through targeted interventions guided by hallucination.
Motivated by these findings, we propose Flexible Association Control (FlexAC), lightweight and training-free framework for modulating associative behavior in MILMs. The core idea is to first extract the associative vector from hallucinated responses (Phase I: Offline Control Vector Construction), which exhibit strong associative tendencies, and then apply it at inference time to guide model behavior (Phase II: Inference-Time Control). In the Offline Control Vector Construction Phase, FlexAC performs three key steps: (1) Hallucination- Guided Intermediate States: We collect grounded hallucinated response pairs, and measuring the differences between their hidden states within model's 's middle layers, which encode the associative direction. (2) Instance Selection: To reduce noise from individual samples, we select the top-K response pairs with the largest association shifts and average their differences to obtain a reliable steering vector. (3) Directional Integration: To further support tasks requiring multi-dimensional associations (e.g., storytelling or metaphor generation), augment the general associative vector with task-specific associative vectors derived

---

from GPT-4o-genne generated, high-association samples. These vectors are incorporated at inference time for fine-grained and controllable modulation. In the Inference-Time Control Phase, we apply the combined steering vector during inference. However, uniformly applying this vector can lead to over-steering, especially for inputs already exhibit strong associative behavior, causing irrelevant outputs or stylistic drift. To mitigate this, we introduce Steering Intensity Calibration, which adaptively scales the steering vector: amplifying it when associative behavior is weak, and attenuating it when the desired level has been reached.
To evaluate the effectiveness of FlexAC in controlling associative behavior, we conduct experiments across three fronts: hallucination mitigation (CHAIR [14] and POPE [15] for low- aswocisciitionn tasks), creativity enhancement (VDAT and Creation-MMBench [16] for high-association tasks), and general- purpose evaluation (MME [17], MMMU [18], and MMStar [19]). Results show that FlexAC enables flexible modulation of associative reasoning capability, achieving state-of-the-art performance on both low- and association tasks while enhancing general capabilities.
In summary, our contributions are fourfold: (1)We present a unified perspective that links hallucination and creativity to associative reasoning, identifying middle-layer representations as key control points. (2)We propose FlexAC, a lightweight and training-free framework for flexible modulation of associative strength, enabling task-aware switching between hallucination suppression and creativity enhancement. (3) We introduce VDAT, a benchmark specifically designed to evaluate associative reasoning strength. (4)We conduct comprchensive experiments demonstrating that FlexAC effectively controls associative behavior across hallucination, creativity, and general purpose benchmarks.
2
Analyzing and modulating associative behavior in MLLMs
2.1 Analyzing layer-wise localization of associative processes
Feature Distance Analysis: Quantifying layer-wise differences between associative and non- associative representations. To identify where associative behavior emerges, we analyze layer-wise representations in LLaVA-115-7b using 1000 images from COCO2024. For each image, we collect two type of responses: a grounded aoo-ciasoiat response from the model's default output, and a hallucinated (associative) response induced via blurred inputs and specific prompts [8]. Here, we use hallucinated responses to represent associative behavior, as they often include many imaginative contents, objects that do not exist in the image but are semantically related to the scene, reflecting the model's associative tendencies. We then extract the associative features f(a) and non-associative features ((() from all intermediate layers for both data types (visualized in Figure 4). The full data construction and feature extraction process is detailed in Appendix B. Next, we compute the cosine distance and Euclidean distance between ((e) and f(r) across all layers. The cosine distance Doos is used to evaluate the directional aligmment between associative and non-associative features, while Euclidean distance Duc measures the spatial distribution differences.
shallow
middle
deep
shallow
middle
shallow middle
deep
shallow
middle
Eudcce Desence
1st
ept
Cone Teteeve
Eucldban ditarce
opt
Rest
Layer
Layer
Layar
Layer
(a) Cosine distance
(b) Euclidean distance
(c) Cosine distance
(d) Euclidean distance
Figure 2: (a) and (b) show the cosine and Euclidean distances between associative and non-associative features across layers. (c) and (d) illustrate the impact of replacing associative features in different layers on subsequent layers. "Last" and "Rest" denotes the final layer difference dy and the average layer difference dm=.L, respectively. "Rest-ori" represents the original mean feature distance dmu without replacement.
As shown in Figures 2a and 2b, both cosine and Euclidean distances remain consistently low in the shallow layers (layers 0-9), indicating shared low-level perception. However, for middle and deep layers, we observe distinct patterns between cosine and Euclidean distance when comparing grounded
3

---

CHAIR VDAT
Layer
Layer
Layer
Layer
Layer
CHAIR
VOAT
05
4.5
-0.5
1.5
control coefficient
Figure 3: Impact of Middle Layer Control on mallucination-Drrien Behavior. Adjusting increases both hallucination (CHAIR) and creativity (VDAT), suggesting that associative strength can be modulated through middle-layer control using hallucination representations.
Figure 4: Visualization of feature representations in LLaVA-1.5-7b. reduced via PCA, shows red (as- sociative) and blue (non-associative) points. The feature distributions show increasing separation in deeper layers, illustrating how associative distinc- tions are formed. See Appendix F.1 for all layers.
and hallucinated responses across layers. Cosine distance peaks in the middle layers (layers 10-15), indicating that this stage is where feature directions diverge most aegnfiiantlyysuugeeee associative behavior is primarily introduced and shaped in this range. In contrast, Euclidean distance increases steadily across both middle and deep layers (layers 10-31), implying that the overall feature magnitudes continue to drift even in later stages. This discrepancy raises a key question: associative behavior actively introduced in the deep layers, or are these differences merely the propagated result of associative shifts originating in the middle layers?
that
Layer Intervention: Veritying the source of as- sociative signals. To answer this, conduct layer intervention experiment (Figure 5), in which replace the associative feature f(a) with the corresponding non-au assoccicie feature fii)) dif- ferent layers m, and observe the influence on downstream representations. The modified fea- ture propagation is defined as:
the m-th layer, use to replace
to giet
m L-1
Non-A ssoclatin
A ssociatio
Intervened A ssociation
(1(0]
fmoddified
1<
o (
12 m,
m+1
(1)
dm (D
$
))
d. = =
where M denotes the I-th layer of the model. We evaluate the impact by calculating the final layer difference dy and the average layer differ- ence dm+1.L as follows:
m+1
Figure Layer Intervention for Association Localization. The goal is that locating the key (2) layers for associative feature generation. Asso- ciative features replaced with non-associative ones at different layers, and the impact on subse- quent layers is evaluated using d. and dm:t.-
d,. D..= pmmodiiie ((())
dm
Σ 1=M+1
mmodiified (())..
L
where D(-) denotes either cosine Euclidean distance.
Results Figures 2c and 2d show that replacing features in shallow layers (layers 0-9) leads to minimal changes in downstream representations, indicating limited influence associative processing. In contrast, replacing features middle layers (layers 10-15) significantly reduces divergence in later layers, suggesting that these layers the primary source of associative behavior. Replacements in deep layers (layers 16-31) again have limited impact, implying that these layers mainly propagate rather than generate associative features. More visualization in Appendix F.2.
Finding 1: Middle layers are critical for shaping MILM's associative behavior.

---

Phase I: Offline Control Vector Construction
Step I: Hallucination-guudee Intermediate States
Step Instance Selection
Directional Integration
ASSOCIATION
domet-domain exampics
seleet
original response
essociation stirt
mesaphorial asocialion
assocsation
hall aemaaldddd response
0.0.0
layeers MLLMs
Flay
associate voctor
(g.... (e)
Top - castnols, spane)
erost-modal association
association
-1
Phase II: Inference-Time Time Control
General Vector
Task-specifle Vector
Step III: Steering Intensity Calibration (SIC)
General
Task-speeifle
Auifann shift
f
over-steering alreudy aligned feature f)
SIC
SIC
cq
90* wasket
sigmoid (mas(
f f + ap + agv
Layer m++1
Uniform Intensity
Intensity Calibration
MLLMs
Figure 6: Overview of the proposed FlexAC framework. Phase I: Offline Control Vector Construction extracts general associative vector from hallucination- guided intermediate features (Step I). by selecting Top-K instance pairs with maximal association shifts (Step II). It also constructs task-specific associative vectors from a few target-domain examples (Step III), reflecting diverse associative needs. Phase II: Inference- Cemee Control injects these vectors into middle-layer features. Steering Intensity Calibration (SIC) module adaptively adjusts the influence of each vector per sample to achieve controllable associative reasoning strength.
2.2 Analyzing control strategies for associative behavior modulation
This analysis investigates whether associative behavior can modulated by manipulating middle- layer representations, and whether hallucinated responses reveal effective directions for such control. Using the same grounded and hallucinated feature pairs from Scction 2.1, we compute feature differences layer by layer to derive the control direction:
V (a) F!").
(4)
We then apply this steering vector during inference modulate the model's output by adjusting the middle-layer features with control coefficient a:
feeminul a
To assess the impact of steering on associative behavior, we introduce VDAT (Visual-Divergent Association Test), benchmark that evaluates a model's associative reasoning by prompting it to generate unrelated nouns the input image, thereby measuring its capacity for disual-driven divergent thinking (details in Section 3.1). As shown in Figure 3, increasing from -1.5 to 1.5 raises CHAIR from approximately 38.8 53.6 and VDAT from around 83 87.9, indicating that higher values lead to both more hallucination and stronger associative ability. Conversely, decreasing reduces both scores. These results highlight that a provides a controllable mechanism for modulating associative behavior in MILMMs. These results yield two key findings:
Finding 2: Modifying middle layers enables control over associative reasoning strength.
Finding 3: Hallucinations help derive steering vectors to guide associative reasoning.
2.3 Flexible association control
Based on our findings in Sections 2.1 and 2.2, we propose Flexible Association Control (FlexAC), lightweight, training-free framework for modulating associative behavior MILMs. As illustrated in Figure 6, FlexAC operates in two phases: (I) Offline Control Vector Construction, which derives general and task-specific associative directions, and (II) Inference-Time Control, which injects these directions into middle-layer features for dynamic modulation.
5

---

Phase I: Offline Control Vector Construction. To capture a general associative direction, we first induce hallucinated responses that exhibit high associative behavior (Finding 3). For each input, we extract hidden features from the middle layer 1, where associative distinctions are most prominent (Finding 1), resulting in paired features 1/0)) and f!() We select the top-K pairs with the highest cosine distances to construct representative direction vector:
I = Top K (Deal (1. )( i
1
(6)
I
EET
To handle tasks requiring diverse associative patterns (e.g., metaphorical, contextual), we further construct task-specific associative vectors from a few high-association, instruction-aligned examples. As vanilla MLLMs struggle to produce such outputs, we leverage GPT-4o to generate high-quality associative outputs.
Phase II: Inference- Time Control. During inference- time phase, we adjust the hidden state fr at middle layer (Finding 2) by injecting combination of general associative vector 2i and task-specific associative vector ulask
flontol fi Olgen
1/
gen
+ Ckaak ui
where a is the tunable coefficient that controls the steering intensity. This formulation is grounded in recent theoretical findings [20], which reveal that task task-specific differences in model weights exhibit linearly decomposable structures. This property supports our assumption that associative directions can be independently extracted and combined within the hidden space.
However, directly applying a uniform steering vector across all inputs can lead to over-steering, especially when the input already exhibits strong associative behavior, causing deviation from the intended semantic space (see Step III of Figure 6). To mitigate it, we introduce steering intensity calibration strategy, which adjusts the steering strength a based on:
max (- ft 11-00 U1
| 0))
a = sigmoid
This formulation increase steering strength when the current representations is misaligned with the associate direction, and suppresses it when already aligned. We further normalize the modulated feature to preserve its scale:
llfell
peontroo
f
pcuntrol
[
This mechanism enables precise, interpretable modulation of associative behavior, allowing MLLMs shift smoothly between factual accuracy and creative generation (Figure 8).
3 Experiments
3.1 Experimental setup
Evaluation Metric: To evaluate the effectiveness of FlexAC, we conduct experiments on three benchmark types: (1) hallucination, using CHAIR [14] and POPE [15] to assess object-level
factual consistency; (2) creativity, using our proposed VDAT for associative reasoning and Creation- MMBench [21] for open-ended image-grounded generation; and (3) general-purpose capability, using MME [17], MMMU [18] and MMStar [19] ensure core perception and reasoning are
preserved. Metric details are in Appendix C.
VDAT: Visual Divergent Association Test. To measure model's associative reasoning and creative potential more directly, we introduce VDAT, a diagnostic benchmark that complements Creation- MMBench by focusing specifically associative reasoning strength. Inspired by [22], VDAT prompts the model to generate multiple nouns that are unrelated both to the input image, capturing its capacity for visual-driven divergent thinking (Figure 7). The metric is computed using CLIP ViT-L/14 embeddings.
Implementation Details. We evaluate the effectiveness of our FlexAC on LLaVA-1.5 [23], Qwen- VL [3], and Deepseek- VL [4], comparing with Ha-DPO [1I], VCD [8], and VAF [24]. From the COC02014 [25] dataset, randomly selected 2000 images and then applied Instance Selection

---

to choose 50 images for generating the general association vector. For the layer intervention, we manipulated the following layers based on each model's associative strength: Qwen- VI. (layers 15, 16, 17), LLaVA-1.5 (layers 11, 12, 13), and Deepseek- VL (layers 4, 5, 6). For FlexAC-P (faithfulness-cnhancee) and FlexAC- (creativity-enhancee), the control coefficient is set to -1 and respectively. All experi- ments were conducted on 8xRTX 4090 GPUs. The parameter analysis of the number of images provided in Appendix E.
MLLMMS
VDAT cos(e".et)
Figure 7: Visual Divergent Association Test (VDAT) evaluates model's associative reason- ing by prompting it to generate unrelated nouns from image, and quantifies performance through image-text measured using CLIP embeddings.
Table I: Performance on hallucination benchmarks. FlexAC here denotes the version configured to suppress associative behavior, aiming to improve factual accuracy (faithfulness).
CHAIR
POPE
Models
Methods
CHAIRs
CHAIR,
Recall
Fl-score
Accuracy
Precision f Recall
Regular
40.6
12.5
71.7
94.6
85.6
86.6
92,9
79.3
VCD
42.0
11.2
71.7
91.2
86.3
87.2
92.4
81.0
Qwen VI.
38.0
FlexAC (Ours)
19.2
5.4
62.5
74.8
87.1
87.4
89.3
85.1
Regular
50.8
14.3
79.7
97.3
86.5
87.2
91.5
82.0
Ha-DPO)
36.8
10.4
74.0
88.3
83.9
85.3
92.6
76.7
LLaVA-L5
VCD
51.0
15.5
79.1
98.9
84.3
84.9
88.1
80.7
VAF
47.8
13.7
79.2
96.1
86.9
87.1
87.9
85.9
FlexAC (Ours)
36.6
10.4
75.0
95.1
87.9
87.8
87.1
88.8
Regular
32.6
9.2
121.0
88.5
4
88.1
88.8
Deepseck- VL2
VCD
36.6
11.3
67.2
128.2
87.9
87.8
87.6
88.1
VAF
32.0
9.2
66.2
119.0
88.5
88.4
87.6
89.4
(Ours)
3.2
Main results
Results on Hallucination Benchmark. To evaluate FlexAC's ability to improve factual accuracy in faithfulness- focussed tasks, we conduct experiments on CHAIR and POPE. To this end, we set a in FlexAC 1, selecting the precision-optimized variant. As shown in Table 1, FlexAC consistently achieves the lowest hallucination scores most models and metrics. For examples, on CHAIRs, FlexAC reduces hallucination 19.2 (f21.4) on Qwen- VI., 36.6 (11. vs. Regular) on LLaVAA1155 and 28.6 (f4.0) Deepseck-VL2 On CHAIR, similarly achieves the best scores (5.4, 10.4, and 8.1 respectively). In terms POPE accuracy, FlexAC achieves the highest Fl-score on VA-15 (87.9) and comparable superior precision and recall across the board. These results highlight FlexAC's ability to flexibly suppress excessive associative behavior in factual tasks, leading to improved accuracy across models.
Results on Creativity Benchmark. To evaluate FlexAC's ability to en- hance associative reasoning in cre- ative tasks, we conduct experiments on VDAT(Table 2) and Creation- XMBBench(Talle 3).
Table 2: Performance VDAT. FlexAC here denotes the version optimized enhance associative behavior for cre- ative tasks (creativity).
Methods
QW
LLaVA-1.5
DeepScck-VIA
Regular Ha-DPO
84.85
86.89
84.54
As shown in Table 2, hallucination mitigation methods like Ha-DPO re- duce hallucinations but impair asso- ciative capacity, leading lower cre- ativity (e.g., VDAT score of 85.11 vs.
VCD
83.69
86.83
84.62
VAF
84.95
86.79
84.61
FlexAC (Ours)
86.58
88.49
84.76
86.89 for the regular model). In contrast, FlexAC improves remote associative reasoning, achieving higher VDAT score of 88. 48.49. To further verify the validity of the VDAT metric, we conduct user study presented in Appendix D.I. Further, on Creation-MMBench Table 3, we report VFS (Visual Fidelity Score), which evaluates image-text alignment, and Reward, which quantifies creativity
7

---

Creation MMBench
Q: Write prese inspired by Van Gogh's 1889 artwork, capturing its essence and depth.
Title: Whestfield Cypresses
The wheatfield, Vincent van Goghs masterpiece, testament to the power of nature and the human spirit, The painting depicts wheatfeedd micst stomm, with cypress trees slanding proud against the backdrop a drumatic sky...
FlexAC-P
-1
Title: Whestfield with Cypresses
The wheatfield, Vincent van Goglls masterpicce, is testament to the power of nature and the human experience. painting. the cuptures the essence of death, cycle growth decay, and the fleeting nature of existence...
FlexAC-C 1)
VDAT
Q: List 10 unrelated, langible nouns that are not connected to the image or each other.
snowboarder, black, white, snow, mountain, glove helmet, goggles, boand, jump FlexAC-P
ssoap. train, piano, cake, guitar, apple, lamp, book, bed, chair
FlexAC-C
Figure 8: Visualization of FlexAC's Control on Associative Reasoning. This figure illustrates example outputs from Creation MMBench and VDAT, showing how FlexAC-P (faithfulness) and FlexAC-C (creativity) adjust the level of associative strength in the generated responses.
improvements relative to the base model (Qwen- Dowe--VL). FlexAC achieves the highest Reward (10.92), outperforming methods like VCD (-3.86) and VAF (-1.63), while maintaining competitive VFS.
Qualitative examples in Figure 8 further support this: in Creation FlexAC-P focuses concrete visual elements (e.g.. "cypress trees"), while FlexAC-C introduces abstract themes (e.g.. "life and death"). In VDAT, FlexAC-P outputs image-relevant nouns (e.g., "snowboarder"), whereas FlexAC-C generates semantically distant words (e.g.. "guitar", "apple"), demonstrating enhanced divergent thinking. These examples confirm that FlexAC effectively modulates associative strength to meet diverse creative demands. For additional examples, see Appendix F.3.
Table 3: Performance Creation-MMMBench. We report results four subcategories: Literary Writing (LW), Common Functional Writing (CFW), Professional Functional Writing (PFW), and Creative Multimodal Understanding (CMU). FIexAC here denotes the version optimized to enhance associative behavior for creative tasks (creativity).
Overall
LW
CMU
PFW
CFW
Methods
VFS
Reward
VFS
Reward
VFS
Reward
VFS
Reward
VFS
Reward
Regular
6.10
0.00
6.83
0.00
5.53
0.00
5.58
0.00
6.66
0.00
VCD
6.05
-3.86
6.68
-2.11
5.67
2.50
5.61
-3.77
6.46
-6.57
VAF
6.06
-1.63
6.39
-3.96
5.57
-4.17
5.61
-0.53
6.64
-0.93
FlexAC (Ours)
6.25
10,92
7.20
15.63
5.83
6.11
5.43
5.96
7.00
15.65
Results on General-Purposs Benchmark. To evaluate the generalization capabilities of FlexAC across range of tasks, we conduct experiments on three standard multimodal benchmarks using Qwen-VL: VL: MME, MMMU, and MMStar. These benchmarks cover wide range of capabilities including fine- -me-irri grounding, reasoning, and instruction following.
As shown in Figure 9, both FlexAC-P (faitffulnes-eehhnneed) and FlexAC-C (creativity-enhanced) maintain performance similar to the vanilla model across most categories, indicating significant compromise in general capabilities. Notably, AC- outperforms the baseline on the OCR task in MME, likely due its enhanced ability to associate text with related visual entities, improving inference and disambiguation under challenging conditions.
3.3 Ablation study
Layer-wise Control Analysis. We investigate the impact middle layers on associative reasoning and identify the optimal control layers by testing interventions shallow, middle, and deep layers, evaluating their effects on both CHAIR and VDAT metrics.
The results in Figure 10 demonstrate that middle layers have the most significant impact perfor- mance: FlexAC-P achieves the best CHAIR results when suppressing associative behavior, while FlexAC-C shows the highest VDAT scores when enhancing creativity. In contrast, controlling shallow
8

---

Regular
FlexAC-P
FlexAC-C
code reASOPing
celebriy
Srtwork
Fine grained perception
Buciness
desig
gercmption
Instance
reasoning:
count
Rccestion
Health
medocm
Overa
6.22
Oversil
existence
leat
Logical reasoring
0.2
Tendmark
Numan Bite
numerical
posters
engineeing
technology
Math
(a) MME
(b) MMMU
(c) MMStar
Figure 9: Performance on general-purper purpose benchmarks. Comparison of Regular, FlexAC-P (faihffulness-enhnncedd a 1)), and FlexAC-C (creativity-enhaneed, a = 1).
or deep layers has minimal effect. Based on these findings, we select layers 15, 16, and 17 as the control layers for Qwen-VI,; results for other models provided in Appendix E.2.
Effectiveness of different Components. We conducted ablation study assess the impact of components within FlexAC, including Instance Selection (IS), Steering Intensity Calibration (SIC), and Directional Integration (DI), on faithfulness (CHAIR) and creativity (VDAT).
CHAIR
VDAT
30
86
40
FoxACC
FevAC C
84-
FesACCC
FlecA--P
FlesAc-P
FexAC-P
Regu
Regular
Regular
1.23
435
916-12
11:1--1
12.14-4
121448
14-14-38
15:1617
16-2733
36-33-22
123
9-10-11
11-22:13
12-18-14
1214.15
1416-.1
15-16-17
1637-18
17-28-99
32
1:2.9
80
43.
9-16-1
11-18-13
22-18-14
22344-15
14-1--6
15:16-17
10-17-18
27.28-29
30 20-3--22
Figure Layer-wise analysis of control effectiveness in FlexAC. The x-axis represents the control
layers, while the y-axis shows the performance of the model on CHAIR and VDAT metrics.
As shown in Figure 11, for CHAIR, FlexAC(P) achieves the lowest CHAIRs score (19.2), indicat- ing effective hallucination reduction compared to the regular model (40.6). When IS and SIC are removed from FlexAC (FlexAC-IS-SICO, perfor- mance slightly worsens (30.4), confirming their role in enhancing faithfulness. Similarly, for
Falthfulness
Creativity
Regular
FloxAC
FlexAC-15
RexAC DI
RexAC
FlexAC(P)
RexACi()
20
B6
creativity, FlexAC-C scores the highest VDAT (86.58). Removing IS and SIC in FlexAC-IS-SIC leads to a small decrease (85.05), while FlexAC- DI results in slight improvement, highlighting for creativity. In summary, FlexAC enables flexible adjustment of associa- tive strength to meet the needs of different tasks, balancing hallucination reduction and creativity enhancement effectively.
CHAIRG! 4)
CHAIR(
(DAT(1)
the importance
Ablation study components, showing the impact of Instance Selection (IS), Steering Intensity Calibration (SIC), and Direc- tional Integration (DI).
Figure 11:
4 Related work
Multimodal Large Language Models. Recent advances in large language models (LLMs) [26, 27, 28] have led to the emergence of multimodal LLMs (MLLMs) that incorporate visual inputs for enhanced capabilities [29, 30). LLaVA [2, 23] improves cnatucttion-oolowinn via visual instruc- tion tuning, while Qwen Owen-VLBB [3] enhances spatial reasoning through visual grounding. DeepSeek- dVL.2 [4] adopts Mixture-of-Experts. architecture improve multimodal comprehension.
Hallucination in MILMs. MILMS still face various safety risks [31, 32, 33). Hallucination is one of the core challenges, where MLLMs generate content misaligned with visual input [34]. To address

---

this, VCD [8] employs contrastive decoding, and VAF [24] enhances visual signal processing during fusion. HA-DPO [11] reduces ballucinations via preference optimization.
Creativity in Large Models. Creativity, involving divergent thinking and novel associations [35, 36], has been explored LLMs via cognitive theories like dual-pathway [37]. Olson ef al. [38] and Chen & Ding [39] promote remote associations; MacGyver [40] and CLOT [41] tackle functional fixedness and divergent thinking, respectively. Creation-MMBench [21] provides a benchmark for evaluating image-grounded creative generation.
Conclusion
In this work, we investigate the root of associative behavior in MLLMs, finding that middle-layer representations govern associative reasoning strength and that hallucinated responses encode reli- able steering directions. Based on these insights, we propose FlexAC, a lightweight, training-free framework that combines hallucination-guiie augmentation. FlexAC enables controllable creativity and achieves state-of-the-art performance across hallucination, creativity, and general-purpose benchmarks. Limitations: FIexAC requires white-box access to hidden states and is not applicable to black-box models like ChatGPT.
steering with adaptive calibration and in-context
Acknowledgements
This study is supported by grants from the National Natural Science Foundation of China (Grant No. U23A20315, No. 62425208. No. U22A2097, No. 62122018. 8, No. 62020106008), Shen- zhen Science and Technology Program (No.JCYJ202408 42080112), Fundamental Research Funds for the Central Universities, and Natural Science Foundation of Sichuan Province (Grant No. 20252NSFSC1463).
References
[1] Liane Gabora. The neural basis and evolution of divergent and convergent thought. The Cambridge handbook of the neuroscience of creativity, pages 58-70, 2018. 2
[2] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023.
2, 9
[3] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou Qwen-vl: A frontier large vision-language model with versatile abilities. CoRR, 2023. 2, 6,
[4] Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wang. and Chong Ruan. Deepseek-v12: Mixture-of-experss lasion-language models for advanced multimodal understanding, 2024. 2, 6,
[5] Ariel Goldstein, Haocheng Wang. Leonard Niekerken, Mariano Schain, Zaid Zada, Bobbi Aubrey, Tom Sheffer, Samuel A Nastase, Harshvardhan Gazula, Aditi Singh, et al. A unified cooustictoo-spechh speech-to-language embedding space captures the neural basis of natural language processing in everyday conversations. Nature Human Behaviour, pages 1-15, 2025.
[6] Jerry Tang, Meng Du, Vy Vo. Vasudev Lal, and Alexander Huth. Brain encoding models based on multimodal transformers can transfer across language and vision. Advances in neural information processing systems, 36:29654-29666, 2023. 2
[7] Xuhui Jiang, Yuxing Tian, Fengrui Hua, Chengjin Xu, Yuanzhuo Wang, and Jian Guo. A survey on large language model hallucination via a creativity perspective. arXiv preprint arXiv:2400. 066447 2024.7
[8] Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In CVPR, pages 13872-13882, 2024. 2, 3, 6, 10, 22
10

---

[9] Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. In Findings of the Association for Computational Linguistics, ACI. 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024. Association for Computational Linguisties, 2024.
[10] Xinyu Lyu, Beitao Chen, Lianli Gao, Hengtao Shen, and Jingkuan Song. Alleviating hallucina- tions in large vision-language models through hallucination-indueee optimization. In NeurIPS, 2024. 2
[11] Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. Beyond hallucinations: Enhancing Ivims through hallucination-avee direct preference optimization, 2023. 2, 6, 10
[12] Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering llama via contrastive activation addition. In ACL. pages 35504-15522, 2024. 2, 22
[13] Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. In ICLR, 2024. 2
[14] Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. In EMNLP, pages 4035-4045, 2018. 3, 6, 22
[15] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In EMNLP, pages 292-202222202 305, 2023. 3, 6, 23
[16] Xinyu Fang, Zhijian Chen, Kai Lan, Lixin Ma, Shengyuan Ding, Yingji Liang, Xiangyu Zhao, Farong Wen, Zicheng Zhang, Guofeng Zhang, Haodong Duan, Kai Chen, and Dahua Lin. Creation-mmbench: Assessing context-aware creative intelligence in MILM. CORR, 1as/25033114478. 2025. 3, 22
[17] Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and Rongrong Ji. MME: A comprehensive evaluation benchmark for multimodal large language models. CoRR, 2023. 3, 6, 23
[18] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang. Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In CVPR, 2024. 3, 6, 23
[19] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and Feng Zhao. Are we the right way for evaluating large vision-language models? In NeurIPS, 2024. 3, 6, 23
[20] Hongkang Li, Yihua Zhang, Shuai Zhang, Meng Wang, Sijia Liu, and Pin- Yu Chen. When task vector provably effective for model editing? a generalization analysis of nonlinear transformers. In ICLR, 2025. 6
[21] Xinyu Fang, Zhijian Chen, Kai Lan, Lixin Ma, Shengyuan Ding, Yingji Liang, Xiangyu Zhao, Farong Wen, Zicheng Zhang, Guofeng Zhang, Haodong Duan, Kai Chen, and Dahua Lin. Creation-mmbendee Assessing context-aware creative intelligence in MLLM. CoRR, abs/2503.14478 2025. 6, 10
[22] Honghua Chen and Nai Ding. Probing the "creativity" of large language models: Can models produce divergent semantic association? In EMNLP, pages 12881-12888, 2023, 6
[23] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR. pages 26286-26296 2024. 6,9
[24] Hao Yin, Guangzong Si, and Zilei Wang. Clearsight: Visual signal enhancement for object hallucination mitigation in multimodal large language models. In CVPR, 2025. 6, 10
11

---

[25] Tsung- Yi Lin, Michael Maire, Serge J. Belongie. James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitmick. Microsoft COCO: common objects in context. In ECCV, pages 740-755, 2014. 6, 23
[26]
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. Chatgpt outperforms text aexttannotation tasks. arXiv preprint arXiv:2303. 5556, 2023.
for
[27] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anm Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 2023. 9
[28] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 100209966600 /6609, 2023. 9
[29] Wenliang Dai, Jumnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-pun purpose vision-language models with instruction tuning. arXiv preprint arXiv:2306. 04387, 2023. 9
[30] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified vision- understanding and generation. In International Conference on Machine Learning, pages 12888 12880-12900. PMLR, 2022. 9
[31] Shengming Yuan, Qilong Zhang, Lianli Gao, Yaya Cheng, and Jingkuan Song. Natural color fool: Towards boosting black-box unrestricted attacks. In NeurIPS, 2022. 9
[32] Youheng Sun, Shengming Yuan, Xuanhan Wang, Lianli Gao, and Jingkuan Song. Any target can be offense: Adversarial example generation via generalized latent infection. In ECCV, 2024.
9
[33] Beitao Chen, Xinyu Lyu, Shengming Yuan, Jingkuan Song, Heng Tao Shen, and Lianli Gao. SafePTR: Token-level jailbreak defense in multimodal LLMs via prune-then-restor mechanism. In NeurlPS, 2025. 9
[34] Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. survey on hallucination in large vision- languag models. CORR, abs/2402.00253. 2024. 9
[35] P. GUILFORD. Creativity: Yesterday, today and tomorrow. The Journal of Creative Behavior, pages 3-14, 1967. 10
[36] Mark Runco and Garrett Jaeger. The standard definition of creativity. Creativity Research Journal- CREATIVITY RES J. pages 92-96, 2012. 10
[37] Roger E. Beaty, Paul J. Silvia, Emily C. Nusbaum, Emanuel Jauk, and Mathias Benedek. The roles of associative and executive processes in creative cognition. Memory & Cognition, pages 1 1186-1197 119,, 2014. 10
[38] Jay A. Olson, Johnny Nahas, Denis Chmoulevitch, Simon J. Cropper, and Margaret F. Webb. Naming unrelated words predicts creativity. Proceedings of the National Academy of Sciences, page e20223401 18, 2021. 10
[39] Honghua Chen and Nai Ding. Probing the "creativity" of large language models: Can models produce divergent semantic association? In EMNLP, pages 12881-1222 12888, 2023. 10
[40] Yufei Tian, Abhilasha Ravichander, Lianhui Qin, Ronan Le Bras, Raja Marjich, Nanyun Peng, Yejin Choi, Thomas Griffiths, and Faeze Brahman. Macgyver: Are large language models creative problem solvers? In NAACL, pages 5303-5324, 2024. 10
[41] Shanshan Zhong, Zhongzhan Huang, Shanghua Gao, Wushao Wen, Liang Lin, Marinka Zitnik, and Pan Zhou. Let's think outside the box: Exploring leap-of-thought in large language models with creative humor generation. In CVPR, pages 13246-13257, 2024. 10
[42] Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. Mitigating hallucinations in large vasion-language models with instruction contrastive decoding. Findings of the Association for Computational Linguistics ACL 2024, pages 15840-15853, 2024. 22
12

---

[43] Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang. and Lijuan Wang. Mm-vet: evaluating large multimodal models for integrated capabilities. In ICMI,, pages 57730-57754, 2024. 27
[44] Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang. Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference computer vision, pages 216-233. Springer, 2024. 27
[45] Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed- bench: Benchmarking multimodal Ilms with generative comprehension. arXiv preprint wrXv:23307.6115, 2023. 27
[46] Hai-Long Sun, Pei Zhou, Yang Li, Shiyin Lu, Chao Yi, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, and Han-Jia Ye. Parrot: Multilingual visual instruction tuning. In ICML, 2025. 27
[47] xAI. Grok-1.3 1.5 vision preview. 2025-10-14. 27
wwwph/wwwwwwwwwoooee April 2024. Accessed:
[48] Nannan Zhu, Yonghao Dong, Teng Wang, Xueqian Li, Shengjun Deng, Yijia Wang, Zheng Hong, Tiantian Geng, Guo Niu, Hanyan Huang, et al. Cvbench: Evaluating cross-video synergies for complex multimodal understanding and reasoning. arXiv preprint arXiv:2508. /9542, 2025. 27
[49] Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images, 2016. 27
[50] Amanpreet Singh, Vivek Natarjan, Meet Shah, Yu Jiang, Xinlei Chen, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In CVPR, pages 8317-8326, 8326, 2019. 27
[51] Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2263-2279, 2022. 27
[52] Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. document images. In WACV, pages 2200-2209, 2021. 27
Docvqa: dataset for vqa on
[53] Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. In ICDAR, 2019. 27
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims in the abstract and introduction accurately reflect the con- tributions of the paper, including introducing the Flex AC framework control associative reasoning in MILMs and demonstrating its effectiveness in modulating creativity and factual accuracy across various tasks.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in conclusion.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
14

---

Answer: [NA]
Justification: The paper does not include formal theoretical proofs. It is empirical in nature, with focus on the practical application of the FlexAC framework.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Section 3, we provide comprehensive details of the experimental setup and all results required to reproduce our main findings, accompanied by thorough analysis and corresponding conclusions.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: The code is included in the supplementary material for reproducibility. and we plan to release the full codebase publicly upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ /nips. htppppubbuuddeeeeeeeppppppbppeee t
/CodeSubmic cerbbbisssiooi weriiisssinnooiiiioeeeoo onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper provides sufficient experimental details, including the datasets used, model configurations, and the benchmarks in Section 3.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Section 3 and the appendix include detailed analysis of hyperparameters and consistent results across multiple datasets and models, demonstrating robustness even without explicit error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Section 3.1 provides details on the compute resources and settings used in our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https //neurrris.... cetpurrps..cppulll1/e
Answer: [Yes]
Justification:
The research conform to the NeurIPS Code of Ethics. There is no indication any ethical concerns, and the work focuses on improving deep learning techniques with no harmful implications.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The broader societal impacts of this work, including both the potential benefits and risks, are discussed in Appendix I.
17

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications,
authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not release any models or datasets that pose a high risk for misuse. All experiments conducted using existing publicly available MILMs and datasets, and no new potentially sensitive or high-risk assets are introduced.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external assets used in the paper, including datasets (e.g., COCO2014) and pretrained models (e.g., LLaVA, Qwen-VL,, (eppSeek-V..). are properly cited in the references. Their usage complies with the respective licenses and terms of use as outlined by the original creators.
Guidelines:
The answer NA means that the paper does not existing assets.
18

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces new method (FlexAC) and provides its core implemen- tation code in the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
The paper includes user study in which internal volunteers rated model out- puts. Participants were informed of the study's purpose and procedure prior to participation. Although they were internal to the organization, compensation was provided in accordance with fair labor practices.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
19

---

Answer: [Yes]
Justification: The paper includes a user study in which participants rated model outputs. Al- though the task posed minimal risk, participants' involvement disclosed, and appropriate ethical considerations were followed.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: The paper discusses the use of multimodal large language models (MLLMs) as part of the core methodology demonstrate the effectiveness the FlexAC framework, including its application in hallucination reduction and creativity enhancement.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy (https: http/enursp.../// /cc//oofeeeeccc//22///////////// for what should or should not be described.
20

---

FlexA C
Towards Flexible Control of Associative Reasoning in Multimodal Large Language Models
(Supplementary Material)
CONTENTS
Broader Impacts
Data Generation and Feature Extraction
22
C Metries details
22
D Detailed Experimental Results
24
D.I User study
24
D.2
Extended results on Creation-MMBBench
26
D.3
Efficiency comparison
................................
27
D.4
Extended Evaluation on General- Purpose Benchmarks
27
D.5
Detailed results on POPE
28
E
Ablation Study
29
E.1
Effect of dataset Sizes
29
E.2
Effect of control layer.
30
................................
F
Visualizations
31
F.1
Detailed Feature Representation Analysis Using PCA
................................
31
F.2
Detailed Layer Intervention for Association Localization
32
F3
Visualization of more examples
F4
Feature Distance Analysis on Additional Models
42
21

---

A
Broader Impacts
FIexAC introduces finer control over the associative behavior of MLLMs, enabling safer and more context-appropee responses across tasks. This may benefit applications requiring factual precision (e.g., education, medical support) creative output (e.g., storytelling, art generation). However, en- hancing associative capacity also increases the model's expressive power, which-if misused-cout lead to persuasive but unfounded generations. As with all generation-controlinng techniques, FCex should deployed alongside robust safeguards ensure alignment with human intent and ethical use.
B Data Generation and Feature Extraction
Inducing and Representing Model Associations: To investigate the causes of model association,
we generate two data distributions: one from the model's original outputs (non-associative) and another with induced associative content using blurred images and tailored prompts [8, 42]. For example, the model is prompted with: "Describe the image and include some hallucinated objects that are imagined but do not exist in the image, as if they were real." Following [12], construct multiple-choice dataset to capture feature distributions. The model is given an image and prompted generate detailed responses, with two predefined options (Figure 12): [1] non-associative (factual) and [2] associative (creative). The hidden states corresponding to these inputs are extracted to obtain distinct feature representations, F! nea-assoc and Ft assoee capturing the model's 's internal response both associative and non non-associative prompts across different layers.
<image>
Question: Please describe this image in detail.
[1] responses without association
[2] responses with association
Please select the most appropriate answer: or 2]
Figure 12: The prompt for extracting associative and non-associative features
C Metrics details
All comparative experiments are conducted using the VLMEvalKit', For binary choice questions, we prompt the model with: "Please answer Yes or No." We evaluate three models in our experiments: LLaVA-1.5 (Luuhotiaan/laava-ll..55 5--7), Qwen-VL (Qwen/Qwen-VL-CK)) and DeepSeek- VL2 (deppeekkaa//eesseekk2222
VDAT
VDAT fills a gap in evaluating the creative potential of multimodal models, which previous metrics did not adequately address. To ensure consistency, both CHAIR and VDAT were evaluated using the same 500 images, randomly selected from the MSCOCO dataset.
Creation-MMBenne [16]:
is multimodal benchmark designed to evaluate scenarios. contains 765 test
the creative capabilities of MLLMs in real-world, cases across 51 fine-grained tasks, with instance-specific criteria that assess both imaginative quality and visual consistency. In contrast to prior work that compares models GPT-4o, our evaluation focuses measuring improvements over each model's 's own vanilla baseline.
Caption Hallucination Assessment with Image Relevance (CHAIR) is metric designed to evaluate the hallucination of image caption task. It measures the hallucination rate of the generated text by comparing the generated caption with the ground-truth caption. CHAIR consists of
CHAIR [14]:
Phttps://githu/ http::///ggthuuu..ooooeeeett http://a......oopooooccoc//a/aaa
22

---

two metrics: S and ,
They can be calculated as follows:
Khallucinated objects]|
CHAIRs
(10)
Mall mentioned objects]]
CHAIR Heaptions with hallucinateddbbjects]]
(11)
Hall captions)
POPE [15]: The Polling-based Object Probing Evaluation (POPE) metric developed to evaluate object hallucination in MLLMs. By framing the evaluation as a series of Yes-or-No questions about specific objects in images, POPE avoids issues related instruction sensitivity. Using three sampling strategies- Random, Popular, and Adversarial-it effectively examines models' tendencies hallucinate frequently occurring or co-occurring objects. providing stable and reliable assessment of object hallucination. Refer to [15], we built POPE on 500 randomly selected MSCOCO [25] validation images, each containing over three ground-truth objects and six constructed questions.
MME [17]: MLLM Evaluation benchmark (MME) is a benchmark designed to assess multimodal large language models (MLLMs) across core skills in perception and cognition, such as object recogni- tion, attribute identification, reasoning, and translation. Using accuracy- based metrics, MME provides objective insights into model capabilities, highlighting areas for improvement understanding and reasoning.
MMMU [18]: MMMU (Massive Multi-discipling Multimodal Understanding and Reasoning) is large-scale benchmark targeting expert-level multimodal understanding and reasoning. It comprises 11.5K college-level questions across disciplines and 30 subjects, featuring diverse image types such charts, medical scans, diagrams, and chemical structures. MMMU emphasizes deep domain knowledge and deliberate reasoning. challenging models integrate perception, knowledge, and logic in complex tasks. It serves necessary testbed for evaluating progress toward Expert AGI.
MMStar [19]: MMStar a high-quality benchmark designed to evaluate vision-language models on truly vision-dependent tasks. It includes 1,500 human-curated samples across 6 core capabilities and 18 fine-grained skills, ensuring minimal data leakage and strong visual grounding.
23

---

D Detailed Experimental Results
D.1
User study
To validate the effectiveness of the VDAT metric as a measure of associative creativity, we conducted human evaluation study comparing FlexAC against several baselines. Specifically, we randomly selected 30 image-response examples from the Qwen-VL evaluation set and presented them to 15 human raters. For each example, two responses were shown one from FlexAC and one from baseline method (Regular, VAF, or VCD). Participants were asked to judge which response contained objects more unrelated to the image. as a proxy for stronger remote association. The response options were presented as "Answer A" and "Answer B," with the method-to-label mapping randomized in each trial to eliminate bias. Raters evaluated each pair on a five-point scale ranging from "A much better than B" to "B is much better than A." These choices were then converted to numeric scores for aggregation for example, "A >> 3" assigns points to A, "A B" assigns point to both A and B.
Figure 13 shows that FlexAC consistently receives higher average scores than all baselines, with low variance across users. Figure 14 further reveals that over 70% of responses favored FlexAC (A > B or A >> B), while fewer than 6% favored the baseline. These results demonstrate strong alignment between the VDAT metric and human judgment. Figure 15 provides screenshot of the evaluation interface, Together, these findings support VDAT as a valid and human-aligned metric measuring associative creativity in vision-language generation tasks.
FlexAC vs Regular
FlexAC VAF
FlexAC VCD
3.0
3.0
3.0
2.5
2.5
2.5
2.0
2.0
1.5
1.5
1.5
01.0
1.0
0.5
0.5
0.5
0.0
0.0
0.0
FlexAC
Regular
FlexAC
VÁF
FlexAC
VČD
Figure 13: Average user ratings comparing FlexAC with baseline methods on the VDAT task. Each bar represents the average score across 15 users for 30 randomly selected image-response pairs. Error bars indicate the maximum and minimum individual user scores, reflecting rating consistency. Higher scores indicate stronger perceived remote association ability.
FlexAC Regular (8)
FlexAC VAF
FlexAC VCD
A
AsoB 33.78%
A-B 31.56%
A=-B
A-8 34.00%
35.335%
35 33%
35.56%
A-B $1. 7878%
A-R
Aroi b47%
5.09%
26 44%
26.44%
Av-n
0.67%
X-B
1.33%
Figure 14: Distribution of user rating preferences when comparing FlexAC with each baseline on the VDAT task. A = B indicates equal preference; A >> B and A > B mean FlexAC is preferred; << B and A < B mean the baseline is preferred. Results show strong preference for FlexAC in most cases.
24

---

Task: Test the model's remote association ability
Question: Please generate 10 objects that are most unrelated to the image
Name (Required)
Please enter your name
Image 21
Comparison: Method A Mothod (Item 1/90)
Taylor
each image, please compare the two answers. more unrelated answers are image content, the better consider the model's performance. Objects generated more important!!!
Answer
Answer
book, apple, guitar, cake, tv. car, fridge, plane, computer, lamp
train, platform, trees, signal, signal
fence, signal, signal, signal, signal,
Your Evaluation (Which answer better? Single choice)
much than
ArB somewhat
A-B and
A-B somewhat
much than
Previous
Progress: 1/90 (0%)
Next
Submit Survey
Interface of the user study for evaluating remote association. Participants are presented with an image and two model-generated answers, and asked to judge which set of objects more unrelated to the image. The label A was randomly assigned to FlexAC or bascline in each trial to prevent method identification.
Figure 15:
25

---

D.2 Extended results on Creation-MMBennch
To further evaluate FlexAC's effectiveness in enhancing associative behavior for creative genera- tion, we report additional results on the
benchmark using two base models: LLaVA-1.5 and DeepSeek- VLL2, as shown in Table 4. This benchmark covers four creative sub- -Literrr Writing (I.W). Common Functional Writing (CFW), Professional Functional Writing (PFW), and Creative Multimodal Understanding (CMU). For each subtask, we report two metrics: VFS (Visual Fidelity Score), which measures the alignment between the image and the generated response, and Reward, which quantifies creativity improvements relative the base model (i.e., vanilla LLa or vanilla DeepSeek-VL2. VL2, respectively).
In this experiment, FlexAC is configured to enhance associative behavior, with the goal of generating more creative content. Across both models, FlexAC achieves the highest overall reward scores, demonstrating its effectiveness promoting creative generation without sacrificing visual grounding. Notably, on DeepSeek- FlexAC obtains reward of +10.35 on PFW and +6.73 overall, clearly outperforming all baselines. test whether performance gains stem from meaningful control or arbitrary perturbation, we also evaluate variant that injects random vectors into the representation. As shown in the "Random" rows, this leads large performance drops across all metrics metrics--highlggttinn that FlexAC's improvements do not come from noise randomness, but from targeted modulation of associative features. These results further support FlexAC's ability to improve creative reasoning across diverse multimodal architectures.
Table Performance on Creation-MMBenchh We report results on four subcategories: Literary Writing (LW), Common Functional Writing (CFW), Professional Functional Writing (PFW), and Creative Multimodal Understanding (CMU). FlexAC here denotes the version optimized to enhance associative behavior for creative tasks (creativity).
Models
Methods
Overall
L.W
CFW
PFW
CMU
VFS
Reward
VFS
Reward
VFS
Reward
VFS
Rewand
VFS
Reward
Regular
5.32
0.00
6.28
0.00
nan
0.00
4,26
0.00
6.08
0.00
Random
3.53
-60.49
3.11
-69.58
2.19
-72.22
2.93
-60.35
4.80
-52.69
LLVALS
Ha-DPO
4.84
226.41
5.09
330.00
3.68
19.72
4.37
-26.2
5.67
-2.22
VCD
5.56
2,00
6.69
7.08
4.87
5,00
4.86
3,00
6.23
-2.11
VAF
5.30
-5.86
6.15
-3.54
4.27
-5.00
4.74
-6.34
6.01
-6.67
AlexAC (Ours)
5.45
4.39
6.52
11.88
4.76
-3.89
4.72
3.62
4.63
Regular
6.12
0.00
6.98
0.00
6.35
0.00
5.71
0.00
6.21
0.00
DeepSeek-VL22
Random
2.34
-77.47
1.32
-78.96
3.28
-75.83
1.96
-82.46
2.75
-72.08
VCD
6.42
4,80
7.37
5.63
6.58
-3.33
5.98
6.40
6.55
5.46
VAF
6.26
-0.39
6.70
-1,25
6.46
-3.06
5.93
2.46
6.42
-2.13
ACexAC (Ours)
6.29
6.73
6.76
0.63
6.37
4.17
5.99
10.38
6.44
6.48
26

---

D.3 Efficiency comparison
To assess the computational efficiency of FlexAC, we compare the inference runtime of different methods on the Qwen-VL VL model when evaluating the CHAIR benchmark. Specifically, we measure the total time required process the full test set under each method's configuration. As shown in Figure 16, AC incurs only minimal additional overhead compared to the original model, demonstrating that its control mechanism introduces negligible runtime cost. contrast, VCD exhibits significantly higher latency due to its reliance dual forward passes-on one for the original image and another for perturbed version- aassion-highlightingg its inefficiency. These results confirm that AC achieves controllable reasoning with minimal impact inference speed.
Runtime andr))
Regulaar
VCD
VAF
FexAC
Figure 16: Inference runtime (in seconds) of different methods on Qwen-Vi when evaluating CHAIR. AIexAC adds minimal overhead, while VCD incurs high cost due to dual- sual-pass processing.
D.4 Extended Evaluation on General-Purpose Benchmarks
To rigorously evaluate FlexAC's impact on general capabilities, we extended our analysis to a diverse suite of benchmarks, as detailed Table 5. Our evaluation spans three representative categories: general multimodal reasoning, comprising MM-Ve3(43). MMBBnch [44], SEED-Bench [45], and MMMB [46]; vision-centric understanding. which includes RealWorldQA [47], CVBench [48], and A12D [49]; and OCR/document-based question answering, covering TextVQA [50], ChartQA [5 l], VQA [52], and OCRVQA [53]. This comprehensive approach verifies that our control mechanism
does not introduce performance degradation.
The results are presented in Table Across all three categories, both FlexAC-C greeaivittyenhhaneee and FlexAC-P mhithfulness-enhaneee model. This provides strong evidence that our targeted control mechanism effectively modulates associative reasoning without degrading the model's 's fundamental, general-purpose capabilities.
maintain performance closely comparable to the baseline
Table 5: Performance of FlexAC extended suite of 11 general-purpose benchmarks, grouped by capability. The results demonstrate that FlexAC maintains performance comparable to the baseline across general multimodal, vision-centric. and OCR/document tasks, indicating method does harm general capabilities.
Category
Benchmark
Regular
FlexAC-C
FlexAC-P
MM-Vet
39.81
38.17
37.33
MMBench
0.581
0.598
0.576
General Multimodal
SEED-Bench
0.638
0.625
0.640
MMMB
0.703
0.678
0,699
WorldAA
0.486
0.490
0.495
Vision-centric
CVBench
0.549
0.524
0.560
A12D
0.612
0.614
0.616
TextVQA
60.66
60.78
59.81
ChartQA
48.36
49.40
45.92
OCR & Document
VQA
57.79
56.85
57.59
OCRVQA
47.46
49.74
45.83
27

---

D.5 Detailed results on POPE
To complement the summary results in Figure 1, we report detailed POPE evaluation metrics across all settings (random, popular, adversarial) and models in Table 6. These include accuracy, precision, recall, and F1 scores for all baselines and our FCexAC variants.
Table 6: Performance on POPE. FlexAC here denotes the version configured to suppress associative behavior, aiming to improve factual accuracy (faithfulness).
Modal
Setting
Method
Accuracyt
Precision
Recall
F1 Scoret
Regular
86.64
92.92
79.33
85.59
VCD
87.62
91.91
82.53
86.97
Overall
VAF
87.17
91.45
82.0
86,47
FlexAC (Ours)
87.44
89.31
85.07
87.14
Regular
88.6
97.38
79.33
87.44
VCD
89.97
97.02
82.53
89.19
random
VAF
89.5
96.47
82.0
88,65
Qwen-Ve VL
FlexAC (Ours)
90.0
94.38
85.07
89.48
Regular
87.0
93.7
79.33
85,92
VCD
87.97
92.6
82.53
87.28
popular
VAF
87.7
92.55
82.0
86,96
FlexAC (Ours)
88.47
91.27
85.07
88.06
Regular
84.33
88.15
79.33
83,51
VCD
84.93
86.69
82.53
84.56
adversarial
VAF
84.3
85.95
82.0
83.93
FlexAC (Ours)
83.87
83.07
85.07
84.06
Regular
87.18
91.47
82.0
86.48
HA-DPO
85.29
92.57
76.73
83.91
Overall
VCD
84.91
88.09
80.73
84,25
VAF
87.07
87.93
85.93
86.92
FlexAC (Ours)
87.84
87.13
88.8
87.96
Regular
89.3
96.02
82.0
88.46
HA-DPO
86.97
96.48
76.73
85,48
random
VCD
87.5
93.37
80.73
86.59
VAF
90.07
93.68
85.93
89.64
L.5
FlexAC (Ours)
91.43
93.74
88.8
91.2
Regular
87.53
92.2
82.0
86.8
HA-DPO
86.0
94.19
76.73
84.57
popular
VCD
85.27
88.78
80.73
84.57
VAF
87.93
89.51
85.93
87.69
FlexAC (Ours)
88.7
88.62
88.8
88.71
Regular
84.7
86.68
82.0
84.28
HA-DPO
82.9
87.53
76.73
81.78
adversarial
VCD
81.97
82.78
80.73
81.74
VAF
83.2
81.48
85.93
83.65
FlexAC (Ours)
83.4
80.14
88.8
84.25
Regular
88.42
88.13
88.8
88.47
VCD
87.82
87.64
88.07
87.85
Overall
VAF
88.37
87.59
89.4
88.49
FlexAC (Ours)
88.52
88.36
88.73
88.55
Regular
92.0
94.87
88.8
91.74
VCD
91.03
93.62
88.07
90.76
random
VAF
91.87
94.04
89.4
91.66
DeepSeek-VL.
FlexAC (Ours)
91.8
94.53
88.73
91.54
Regular
88.13
87.63
88.8
88.21
VCD
87.27
86.68
88.07
87.37
popular
VAF
88.13
87.19
89.4
88.28
FlexAC (Ours)
88.37
88.09
88.73
88.41
Regular
85.13
82.73
88.8
85,66
VCD
85.17
83.24
88.07
85.58
adversarial
VAF
85.1
82.32
89.4
85.71
FlexAC (Ours)
85.4
83.19
88.73
85.87
28

---

E
Ablation Study
E.1 Effect of dataset Sizes
To analyze the sensitivity of FlexAC to the number of instances used in control vector construction.
we vary Top-K over wide range: {1, 5, 10, 20, 50, 100, 200, 500, 1000, 1500, 2000), and evaluate performance on CHAIRs (L), CHAIRi (J), and VDAT (1) using Qwen-VL.
As shown in Figure/ 17, both FlexAC-P (measured CHAIR for faithfulness) and FlexAC-C (measured on VDAT for creativity) exhibit similar trends: performance is relatively high but unstable when K is very small, and stabilizes near its peak around K 50. Further increasing K leads to slight performance degradation, likely due to noise introduced by instances. These results highlight the effectiveness of our Instance Selection strategy, which focuses selecting small, high-quality set of associative and non-associativ samples.
Notably, across all K values, FlexAC-C and FlexAC-P consistently appear opposite sides of the Regular baseline, reflecting two associative reasoning strength. This clear separation demonstrates FlexAC's s capacity to bidirectionally modulate reasoning behavior, enabling controllable transitions between creative and faithful outputs.
50
CHAlR
40
k=50
FlaxAC-C C
Regual
k=50
FlexAC-C
Regular
VOAT
86
K250
FlexXC-P
Regular
FlexAC-C
30
84
20
1500 Top-K
1000
1000
Top-K
(a) CHAIR
(b) VDAT
Figure 17: Sensitivity analysis of the Top-K hyperparameter used in general control vector con- struction Qwen-VL. We vary the number of selected instances (K) and evaluate performance on CHAIRs, CHAIRi, and VDAT benchmarks.
29

---

E.2 Effect of control layer.
To validate the generality our control layer findings beyond Qwen- VL, we conduct additional layer-wise control effectiveness analysis on LLaVA-1.5 and DeepSeek- VL2, VL2, as shown in Appendix Figure 18 and Figure 19. Similar the trends observed in Qwen- VL. we find that both FlexAC-C and FlexAC-P exhibit consistent improvements in their respective metrics (VDAT and CHAIR) when applied to middle layers. Specifically, the performance peaks around middle layers (layers 10-15) for V.L and Layers 4-6 for DeepSeek-VEl VI.2, which aligns with our feature distance analysis (see Appendix F.4). These results further support our choice of control layers and demonstrate that the effectiveness of FlexAC's modulation strategy generalizes across diverse MLLM architectures.
18
90
FlexAC-E
FlacAC-P
Regular
CHARR
CHAIRI
VOAT
FlexAC
FlexAC-C
FlexAC-P
12
FlexAC-P
80
Regular
Regular
82
1.2.
4.56
9:10-11
21.12-13
12:13-14
1911-115
4425-16
25-16:17
16.17-16
27.28-29
30-31-32
122
4.5.6
9-10-11
11-12-13
12-19-14
13:14-25
14:15-16
25-16-17
26-17-19
27-28-22
30 10-33-21
1:2.3
255
9-10-11
11-12-13
12-13-4
1324-15
1625-16
15-26-12
1617731
27-28-20
20
(a)
CHAIR
(b) VDAT
Figure 18: Layer-wise analysis of control effectiveness in FlexAC on V.. The x-axis
represents the control layers, while the y-axis shows the performance of the model on CHAIR and VDAT metrics.
86.0
34
FlexAC-C
85.5
RiexAC--P
10
Reqular
QUARR
CHAIRI
VOAT
85.0
9
30
FlaxAC-C FlexAC-P Regular
84.5
FlexAC-P Regular
84.0
28
1-2-3
4-5-6
5.6-7
8-9.10
9-10.11
10-11-12
1-2-3
4-5-6
5-6-7
8-9-10
9.10.11
10. 10-11-12
83.5
1-2-3
45.6
5.6-7
8-9-10
9.10-12
10.11.12
(a) CHAIR
(b) VDAT
Figure 19: Layer-wise analysis control effectiveness in FlexAC on DeepSeek-VL. VL2. The x-axis represents the control layers, while the y-axis shows the performance of the model on CHAIR and VDAT metrics.
30

---

Visualizations
F.1 Detailed Feature Representation Analysis Using PCA
To provide more detailed view of how associative and non-associative representations evolve across the model, we present an expanded version of Figure Figure 20. This visualization shows the PCA-reduced features layer by layer in LLa with red points representing associative features and blue points representing non-associative ones. Compared to the summary visualization, this version reveals how feature separation progressively emerges across layers. In shallow layers (c.g.. Layer O), the two feature types show significant overlap, indicating similar low-level representations. However, starting from the middle layers (around Layer 12), the separation becomes increasingly distinct, highlighting that the model's 's associative behavior is primarily shaped in the deeper stages.
Laper 11
Layer II
Lp iD
21
Laper 25
Layet
ape
Laper
Lover
Lover
Lover
Loyer
Layn to
LANE
L2
JANI 13
Le
LENe'E
(b)
Figure 20: Visualization of feature representations in LLaVA, reduced via PCA, shows the distribution of associative and non-associative data points, represented by red and blue colors, respectively. Subplots (a) and (b) represent the results for different option orders. In deeper layers, the red and blue points exhibit clearer separation, indicating enhanced differentiation between associative and non-associative representations.
31

---

F.2 Detailed Layer Intervention for Association Localization
To gain more comprehensive understanding of how different layers contribute associative content generation, we expanded on the analysis presented in Figures 2c and 2d by examining each layer individually. As shown in Figure 21 and Figure 22, the detailed version presented here provides layer-by-layer breakdown of how the interventions affect the model's internal representations.
In each subplot of this detailed version, intervened specific layer (denoted by the subplot title, e.g.. "Layer 0," "Layer 1," etc.) by replacing its associative features with non-associative features. We then analyzed the impact of this intervention on feature distances across all layers. Figure 21 and Figure 22 shows that when shallow layers (c.g., layers before Layer 11) are replaced, the feature distances in subsequent layers do change significantly. However, when middle layers such as Layer 11 are replaced, the subsequent feature distances drop sharply, indicating that these layers have crucial impact on the model's associative tendencies. contrast, when deeper layers (e.g., layers after Layer 14) are replaced, the changes in subsequent layers become more stable, suggesting that deeper layers have a weaker influence on associative tendencies.
This detailed analysis highlights that replacing features at specific layers has a distinct influence on subsequent layers, with the greatest impact often observed in middle layers. This is consistent with the averaged results in Figures 2c and 2d, which pointed towards the critical role of middle layers in maintaining associative characteristics.
32

---

Layer
Layer?
Layer
Layer
Layer 10
Layer
Layer 12
Layer
Layer 14
Layer 15
Layer 26
Layer
Layer
Layer
Layer
Layer
Laver
Layer 30
Figure 21: Feature distance analysis across layers after layer intervention. For example, the subplot titled "Layer 12" shows the feature distances across all layers after replacing associative features Layer 12 with non-associative features. The X-axis represents the different layers, and the Y-axis represents the Cosine distance between associative and non-associative data.
33

---

L
Layer7
Layer
Layer
Layer 10
Layer
Layer 12
Layer
Layer 24
Layer 15
Layer 18
Layer
22
Layyer
Leyer
Laver
Layer
Layer
Laver
Layer 30
Figure 22: Feature distance analysis across layers after layer intervention. For example, the subplot titled "Layer 12" shows the feature distances across all layers after replacing associative features Layer 12 with non-associative features. The X-axis represents the different layers, and the Y-axis represents the Euclidean distance between associative and non-associative data.
34

---

F.3 Visualization of more examples
We visualize sample outputs from both the creativity- and -enasioinnn variants FlexAC on Qwen-VLL VL. Figure 23 shows examples from the VDAT benchmark, highlighting differences in asso- ciative strength. Additional qualitative results on Creatio--MMeot are provided Figures 24-29, illustrating FlexAC's ability to adjust associative reasoning across creative tasks.
VDAT
Q: List unrelated, tangible nouns that are not connected to the image or cach other.
grill, food, plate, vegetable, meat, oven, table, kitchen, cooking, dinner
FlexAC-P (at
book, chair, Iv, bag, car, computer, phone, clock, guitar, cake
(ax 1)
plate, broccoli, food, plastic, container, table, fork, dinner, meal, plate
FlexAC-P (at = -1)
banana, car, book, cake, chair, cup, dog, fork, guitar, house
FlexAC-C (a
lamp post, traffic light, building, street, sidewalk, traffic, people, traffic signal, street light, street sign.
FlexAC-P (at 1)
umbrella, elephant, pizza, guitar, book, cake, car, chair, cat, lamp FlexAC-C (a 1)
kitchen, dishwasher, cabincts, windows, curtains, sink, counter, floor, wall, light
FlexAC-P (a = -1)
table, chair, book, window, curtain, wall, floor, counter, cabinet, drawer FlexAC-C (a 1)
zebra, grass, field, animal, grassy, meadow, herd, grazing, grasslands, wild animal
FlexAC-P (a=
book, cat, pizza, guitar, moon, elephant, cake, plane
FlexAC-C (a 1)
motorcycle, sidewalk, brick, pavement, bike, street, sidewalk, bike, street, pavement
FlexAC-P (a -1)
book, chair, car, clock, cake, cat, couch, cup, door, dog
FlexAC-C 1)
watermelon, scooter, helmet, market, people, car, street, helmet, scooter, market
FlexAC-P (a = -1)
apple, car, truck, helmet. bag, chair, table, umbrella, person, market
FlexAC-C
Figure 23: Visualization of FlexAC's Control on VDAT, showing how FlexAC-P (faithfulness) and FCexAC-C (creativity) adjust the level of associative strength in the generated responses.
35

---

Creation MMBench
Literary Writing
Assume you renowned prose writer with exceptional writing skills and outstanding insight into works art, good drawing inspiration from artistic works for literary writing.
This painting work by 19th-century Dutch artist Pierre Tetar Elven, depicting banquet from the year 1867.
Title:
The Banquet in the Garden
Pierre Tetar van Elven's painting, "The Banquet in the Garden," captures the essenee of a grand outdoor celebration in the late 19th century. The painting depicts group of people dressed in their finest attire, gathered in lush garden for formal banquet. The guests mingling and enjoying the festivities, with the glow of candles illuminating the scene.
The painting exudes sense joy and celebration, with the guests' smiles and laughter reflecting the happiness of the moment. The lush gurden setting adds the
opulence and grandeur, with the vibrant colors of the flowers and greenery contrasting with lhe attire the guests.
The painting also captures social aspect banquet, with guests mingling and chatting with one another. The sense community and connection evident the painting, people come together to celebrate enjoy each other's 's company. Overall, "The Banquet the Garden" is beautiful painting that captures the essence a grand outdoor celebration in the late 19th century. The painting's vivid colors and sense of community and connection make it timeless piece of art that continues to inspire and caplivate viewers.
FlexAC-P (a
"Banquet at the Luxembourg Palace, 1867"
The painting by Pierre Telar van Elven cuptures the essence grand event, a bunquet held at the Luxembourg Palace in Paris in the year 1867. The painting depicts group of peuple dressed in their finest attire, gathered in the courtyard of the palace, which is illuminated by hundreds of candles and lamps. The atmosphere is one of opulence and extravagance, with the guests enjoying themselves as they move between the tables, which laden with fine food and drink.
In the center of the seene, young couple can be seen dancing, their faces illuminated by the glow of the candles. The man, dressed in suit and tic, holds bouquet flowers, while the woman, wearing white gown, holds a small bouquet her hands. They seern to be center attention, their dance watched by the rest the guests, who stand around them, their faces illuminated the light the candles.
The painting a restament to the opulence excess of the Victorian era, time when wealth status were highly valued, and the move between the tables, their faces illuminated by the glow the candles, as they enjoy the fruits of their labor. The painting is a reminder of the importance of social status and the rolc that art and culture played in shaping the values of the time.
entertaining was key part society. The guests, dressed in their finest clothes,
FlexAC-C (a
Figure 24: Visualization FlexAC's Control on Creation MMBench, showing how FlexAC-P (faithfulness) and FlexAC-C (creativity) adjust the level of associative strength in the generated responses.
36

---

Creation M.Bennc
Literary_ Writing
Assume you are graduates who have obtained a bachelor's degree from university. These are scries of pictures I took when I graduared
Pleuse follow the requirements below to Illlp write Facebook summary about this ceremony.
1.Describe the process of participating in the graduation ceremony and the unforgettable moments.
Share your re reflectios and insights gained from the graduation ceremony. Let others know the significance of this honor, well the achievements and experiences you gained during study.
A. The graduation ceremony was a momentous occasion for me, filled with emotions and memories that will be cherished forever. remember walking across the stage, receiving my degree, and feeling a sense of accomplishment and pride that words cannot express.
Reflecting the graduation ceremonny. realize that not just about receiving degree, but also about celebrating the hard work, dedication, and perseverance that went into achieving it. It was reminder that every achievement the result of countless hours of effort and sacrifice.
3. The significance of my graduation cannot be overstated. is symbol of my academic achievements and milestone my personal professional joumey. marks the heginning new chapter, filled with opportunitics and possibilitics.
FlexAC-P (at
Today, have the honor of participating in the graduation ceremony of University Ottawa. The ceremony took place in the university's auditorium, where I was scated in the front row along with my fellow graduates. The atmosphere was electric. with everyone eagerly awaiting the start of the ceremony. The university president and vice-chuncellor delivered a speech, highlighting the achievements of the graduating class and encouraging lined up in our cups und gowns, and the university president and vice-chancellor led us to the stage. We received degrees and shook hands with the university president. It unforgettable moment, and felt sense pride and accomplishment wash graduation ceremony. realize that is celebrating the work dedication the experiences have had, the people have met, and the challenges have the opportunitiss the University of Ottawa given me, and am excited see where this jourcyy take me. am also proud of the achievements of my fellow graduates, and I am inspired by their stories and experiences.
pursuc dreams. After the specch, was time for the graduation procession. We
Reflecting
about receiving degree, also about have put over the past years. is time to reflect on
grateful for
3. The significance of this honor lies in the fact that it represents the culmination of years of hard work and dedication. is a recognition of the knowledge and skills that we have acquired, and is a testament to our commitment to excellence. The graduation ceremony is not just about receiving degree, but also about celebrating the achievements of group of individuals who have come together to pursuc their dreams. It is time reflect on the experiences we have had, the people we have met, and the challenges we have overcome. It is time look forward the future with hope and excitement, knowing that have the skills and knowledge to succeed in whatever choose to do.
FlexAC-C (a 1)
Figure 25: Visualization FlexAC's Control on Creation MMBench, showing how FlexAC-P (faithfulness) and FlexAC-C (creativity) adjust the level of associative strength in the generated responses.
37

---

Creation MMBench
Literary Writing
Assume you are talented children's animation crcators who have a profound insight into the story plot and a unique understanding of the picture content, and have the ability to continue to narrate and complete wonderful story according to the content of the previous foreshadows. These are the first few scenes of the children's animation: group of dinosaur friends standing grassy field.
As the sun began to set, the group of dinosaur friends, including cute little T-Rex, friendly Triceratops, and wise Stegosaurus, gathered together on the grassy field. They had been a long adventure, exploring the lush green landscapes and encountering new friends along the way. As they sat together, they shared stories of their adventures and the fun they had had together. The T-Rex, who known for his love of adventure, spoke of the excitement of discovering new lands and meeting new creatures. The Triceratops, who was known for his calm and steady nature, spoke of the joy of simply being surrounded by the beauty of nature. The Stegosaurus, who was known for his wisc words, spoke of the importance of friendship and the value of the bonds form with others. As the set behind them, the group of dinosaur friends in silence, content knowledge they had each other
memor ftheir adventures.
FlexAC-P (a
As the group of dinosaur friends stands on the grassy field, they notice a small, colorful butterfly flitting about. The baby dinosaur, who is still bit too small to walk, is fascinated by the fluttering insect.
"Wow, look that pretty butterfly!" the baby dinosaur says, pointing excitedly the tiny creature.
before," says the baby triceratops, who standing the baby dinosaur. you think it's dragonfly?"
"I've
"No, it's a butterfly," baby pteranodon corrects, with a grin on face. "Butterflics are way cooler than dragonflies."
The baby lyrannosaurus, who is standing the back the group, chimes with giggle. want to catch one!
Can we, Mommy?"
The mother triccratops, who is standing nearby, gives the baby warming look. "We can't catch the butterfly, honcy. It's too fast and too fragile. We have to be careful not hurt it."
The baby dinosaur pouts. "But want to play with it! Can't just touch it?"
The mother dinosaur sighs. "I'm sorry, but can't touch the butterfly. It's delicate. We have careful not harm
The baby dinosaur huffs, but the mother dinosaur is right. Butterflies are very fragile creatures, and they need to be treated with care. The baby dinosaur decides to watch the butterfly as it flutters around them, fascinated by its colors and movements.
As they watch, the baby dinosaur notices a small, wiggly worm crawling nearby. "Hey, look at that!" he says, pointing excitedly at the worm. "It's cool! Can touch it?"
The mother dinosaur shakes her head. "No, honey. That's worm. They're not very pretty, but they're important the ecosystem. We have cureful not harm them.
The baby dinosaur pouts again, but the mother dinosaur right. Worms are important the soil plants, and they help the plants grow. The baby dinosaur decides to watch the worm as it wriggles around them, fascinated by its movements.
As they watch, the baby dinosaur notices small, colorful flower blooming nearby. "Wow, look that!" he says, pointing excitedly at the flower. "It's so pretty! Can
FlexAC-C(== 1)
Figure 26: Visualization of FlexAC's Control on Creation MMBench, showing how FlexAC-P (faithfulness) and FlexAC-C (creativity) adjust the level of associative strength in the generated responses.
38

---

Creation MMBench
Professional Functionality Writing
Assume you are an experienced psychotherapist who can infer owner's personality traits through their living style.
This view
room of the house where client resides.
Please follow the requirements below to make analysis bused on the given image.
Describe the furmishings and style the given image
2. Speculate about his lifestyle, interests, or personality traits
3. Create first-peroon inner monologue reflecting his life
1. The room has gym setup with exercise equipment, including a bench, barbell, and window for natural light. The room is also furiished with a couch and a chair, suggesting that the client values comfort and relaxation.
2. The presence of exercise equipment in the room indicates that the client is wealth-conscious and likely leads an active lifestyle. The gym sctup the living room also suggests that the client values convenience and has strong desire stay in shape.
3. The first-person inner monologue of the client might be: "I'm really glad up the gym in the living room. so easier to stay on top of my fitness routine when can just walk over bench and quick workout during commercial breaks. also appreciate having a comfortable couch and chair to relax on after long day work."
FlexAC-P (at -1)
1. The furnishings of the room include variely of exercise equipment, such as bench, barbell, and a mirror. There is also window in the room, which allows natural light enter. The walls are painted gray, and the floor has brown rug. The room be interested in reading. There is also a TV in the which could be used for watching fitness videos other content.
total books scattered around, which suggests that the owner might
likely be active and health-conscious individual, They have invested a home gym, which indicates commilment fitness wellness. presence of a bench, barbell, and mirror suggests that they are interested in strength training and possibly other forms of exercise as well. The books in the room may indicate that owner is interested learning about fitness and nutrition, or they simply enjoy reading for leisure.
2. Based on the image,
be inferred that
3. "I love having a home gym in my apartment. allows me to exercise whenever want, without having to go to the gym. I usually start day by doing some cardio on the treadmill, and then I'll do some strength training with the barbell and beneh. like read fitness magazines and books while I'm working always have something keep me entertained. The natural light from the window is great for motivation, and I love having TV watch fitness videos documentaries. find that having home gym makes consistent with my exercise routine, and I feel great knowing that I'm taking care of my body." .
I
FlexAC-C
Figure 27: Visualization of FlexAC's Control on Creation MMBench, showing how FlexAC-P (faithfulness) and FlexAC-C (creativity) adjust the level of associative strength in the generated responses.
39

---

Creation MMBench
Professional Functionality. Writing
Assume you high-level photography critic with a thcoretical foundation.
This is a work captured by the photographer Ed Gordeev.
Please follow the requirementt below to appreciate and analyze the given work. 1. Deseribe the content of the image.
Analyze the brilliancc of the work from aspocts such color, composition, and
perspective.
Uncover the possible idcas messages behind the work.
1. The image captures train station with train on the tracks and sunlight shining through the station. The train is surrounded by sunlight and uppears be the main subject of the image.
2. The work is well-composed, with the train taking up significant portion of the image and the sunlight creating strong focal point. The perspective of the image is interesting, as it captures the train station from low angle, emphasizing the size and scale
train.
3. The work likely conveys sense of motion and activity, the train is motion and the sunlight suggests it daytime. image may also evoke sense of nostalgia or a connection to past, as trains are oflen associated with travel exploration.
FlexAC-P
The image features a train station with train on the tracks, surrounded by sunlight streaming through the arched windows overhcad. The train is in the center of the frame, with the station platform stretching out on cither side. The train is dark blue color, and the platform is empty. The train tracks curve off to the right of the frame.
2. The color in the image muted and carthy, with a warm cast the sunlight. The contrast between the dark blue train and the lighter colors of the station balanced, with the truin and station platform roughly equal in size, and the sunbcams creating focal point the
of depth and dimension. The composition well-
the image.
3. The perspective in image is interesting, with the sunbeams appearing to come from above and behind the train. This creates a sense mystery and wonder, the viewer is looking through a keyhole at a magical moment. The empty platform adds to this sense of anticipation and wonder, as if the train is waiting for something or someone to arrive.
4. The possible ideas or messages behind the work could be about the passage of time, the anticipation of arrival or departure, and the beauty of everyday life. The empty platform and the sunbeams could represent the fleeting nature of moments and the imporfance of cherishing them. The contrast between the dark blue train and the lighter colors of the station could symbolize the duality of light and shadow, the duality of life and death. The curve of the train tracks could represent the journey of life, with the train metaphor for the human experience.
FlexAC-C (a 1)
Figure 28: Visualization FlexAC's Control on Creation MMBench, showing how FlexAC-P (faithfulness) and FlexAC-C (creativity) adjust the level of associative strength in the generated responses.
40

---

Creation MMBench
Common Coammon__Pnctioniilln Functionality
Writing
Assume you experienced adventurer. You have the habit of writing travel journal every day. You tell about the beautiful things you have seen, the interesting things you have encountered, and the feelings you have had during your trip. This is scenic photo taken during the tour loday.
Please follow the roquirements below to write today's travel journal.
Describe the Scenery:
Use sensory dctails vividly convey what you saw, heard, felt landscape.
Chromological Storytelling: Nurrate day's events from first-person perspective, starting your arrival and ending with your reflections.
Highlihgh Adventure: Include one two adventurous moments challenges (e.gr, navigating tough terrain encountering wildlife).
4. A.Emotional Reflection: Share how scenery and the day's experiences made you feel adventurer.
Tie to Photo: Explain why you captured photo and how connects most exciting or meuningful part of your day.
Today was an incredible day of adventure and exploration. arrived at my destination and was immediately cuptured perfectly captures the beauty the landscape, the bright green Northem Lights dancing across the sky.
struck by the stunning scenery. photo
As explored the area, heard the sound of rushing water, which led me to beautiful waterfull. The sound of the water was so peaceful and calming, was a welcome break from the excitement of the day.
One of the most exciting moments of the day was when encountered a group of hikers who were also out to see Northem Lights. We huddled logether and watched awe as the Northem Lights up sky in stunning display.
As the day cume to un end, reflected on the beauty of nature and the sense of adventure that comes with exploring new places. am so grateful have captured moment time, and know that it will a memory that I will treasure for years to come.
F-exAC-P (a -1)
Day 3: Northem I. Lights Tour
I woke early this morning, excited to embark Northem Lights tour. The sky clear and the stars were shining bright as made way the tour bus. The tour guide, friendly and knowledgeable fellow. greeted us all and gave We drove for few hours, driving through the rugged terrain of Iecland, passing by stunning landscapes and waterfalls. The tour group was made of travelers from all over the world, and we all shared our excitement and anticipation for what was come.
quick briefing on what expect during the tour.
As the night fell, the sky turned beautiful shade of pink and purple, and we could see the Northern Lights starting to dance in the distance. The tour guide explained that the Northern Lights are caused by charged particles from the sun colliding with the Earth's amoospere, creating bcautiful display of light and color. We drove further into the wilderness, and soon we were surrounded by nothing but the stars and the Norther Lights. The display was like nothing had ever seen befure a heautiful, otherworddly sight that left awe. The Northem Lights were like dance, moving and changing colors in way that was almost too beautiful to be real.
We spent the next few hours watching the Northem Lights, tuking pictures und simply enjoying the moment. It was truly magical experience, and one that will never forget. As the night came end, we headed back to our hotel, tired but huppy. felt grateful to have been able to witness such beautiful natural phenomenon, and I knew that this would be trip that I would always remember.
I took photo of the Northern Lights, knowing that this would be a memory that I would always trensure. The Northern Lights were like a reminder that there is so much more to this world than we can ever imagine, and that every day, we have the opportunity to be amazed by something new.
went to bed that night feeling grateful and humbled by the beauty of the world around us. knew that would never forget this trip, and that I would always look back on it with sense of wonder and awe.
FlexAC-C (a= 1)
Figure 29: Visualization of FlexAC's Control on Creation MMBench, showing how FlexAC-P (faithfulness) and FlexAC-C (creativity) adjust the level of associative strength in the generated responses.
41

---

F.4 Feature Distance Analysis on Additional Models
To complement the analysis in Section 3.1, we extend the feature distance evaluation two additional MLLMs: Qwen-V VL and Deepseek- VL2. As in the main study, compute the cosine and Euclidean distances between associative and non-associative representations extracted from each transformer layer. The results are shown in Figure 30.
Consistent with findings LLaVA, observe that cosine distance peaks in the middle layers, while Euclidean distance gradually increases throughout the network. These patterns reinforce the conclusion that associative behavior primarily emerges and diverges in the middle layers, while deep layers largely propagate those effects.
Importantly, this analysis also informs the design of our control strategy. In Qwen- VL, the middle layers are approximately 13 20, and in DeepSeek- VL2, 3 - 7. Accordingly, we select Layers 15 17 for Qwen- VL and Layers - for DeepSeek-Vii as control points FlexAC. These ranges correspond to the regions maximal divergence between associative and non-ase con-associative features, enabling targeted yet lightweight intervention.
0.00
340
opt 1
Euclidean Distance
opt 1
Cosine Distance
220
006
opt 2
500
opt 2
opt m
opt m
0.04
0.2
0.00
10
15
Layer
Layer
(a) Qwen- -L
(b) Qwen- VL
opt
Euclidean Distance
opt 1
Cosine Distance
0.035
opt 2
opt 2
0025
opt_m m
opt_m
0015
0.01
0.005
0.000
10
Layer
Layer
(c) Deepseek- VV2
(d) Deepseck-VLL
Figure 30: Layer-wise feature distance trends between associative and non associative representations on Qwen- VL and Deepseek- VL2, extending the LLaVA results from Section 3.1.
42


Reviews:
[{"review_id": "DqLXLoN8yS", "reviewer": "Reviewer_oRMg", "summary": "This paper proposes FlexAC, a lightweight and training-free method for flexibly modulating associative reasoning strength in multimodal large language models (MLLMs) to balance faithfulness and creativity. The key contributions include: (1) identifying the critical role of middle layers in associative reasoning; (2) proposing a hallucination-guided steering vector for control; and (3) introducing the VDAT benchmark to evaluate associative reasoning capabilities. Experiments demonstrate that FlexAC outperforms existing methods on multiple benchmarks. The paper is well-structured and the experimental design is sound, though some formulations could be refined for greater rigor.", "questions": "Why was no comparison made with closed-source models like GPT-4V?", "limitations": "The experiments primarily relied on the COCO dataset (object-centric static images) and did not validate performance in video temporal reasoning, abstract art comprehension, or cross-cultural scenarios. If the model fails in dynamic or multimodal associative tasks , would FlexAC's modulation mechanism remain effective?", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Offers a fresh perspective by unifying hallucination and creativity as manifestations of associative reasoning and explores modulation via middle layers.\n2. The FlexAC method is simple yet effective, requiring no additional training, making it practical for deployment.\n3. Thorough evaluations across three dimensions: hallucination mitigation (CHAIR, POPE), creativity enhancement (VDAT, Creation-MMBench), and general capabilities (MME, MMMU).\n4. Ablation studies validate key design choices.\n5. The proposed VDAT benchmark addresses the lack of metrics for visual-driven divergent thinking.\n6. The method is compatible with diverse MLLMs, demonstrating strong generalizability.\n\nWeaknesses:\n1. While the paper identifies middle layers as pivotal for associative behavior, it does not explain why these layers exhibit this property .", "was_revised": true, "final_justification": "Thank you for your comprehensive response. They have addressed my main concerns, and I maintain my recommendation score of 4 points.", "rebuttal": "_**Weakness 1. While the paper identifies middle layers as pivotal for associative behavior, it does not explain why these layers exhibit this property.**_\n\n**Response:** We thank the reviewer for this insightful question. We can conceptualize the MLLM's information flow as a process from **Perception**, to **Concept**, and finally to **Expression**:\n1.  **Shallow Layers (Perception):** The initial layers are focused on processing raw inputs. In MLLMs, they primarily inject general visual features into the text representations to form a basic cross-modal understanding [1,2]. At this stage, high-level associations have not yet been formed.\n2.  **Middle Layers (Concept):** This is where abstract concepts are formed and factual knowledge is localized[3,4]. Research shows that these layers integrate specific, relevant visual details with the linguistic context [2] and are where the probabilities of factually correct tokens sharply increase[4]. This suggests the middle layers function as the primary locus for **knowledge association**, where the model transitions from low-level feature processing to high-level semantic reasoning.\n3.  **Deeper Layers (Expression):** The final layers are less about forming new associations and more about refining the already-integrated information for coherent linguistic output[1,2]. For instance, they handle tasks like syntactic formatting and ensuring the final answer is grammatically correct[2].\n\n---\n\n_**Question 1. Why was no comparison made with closed-source models like GPT-4V?**_\n\n**Response:** Our method requires **'white-box' access** to intervene on **internal hidden states**, which is impossible with closed-source models like GPT-4V. We explicitly acknowledge this limitation in our conclusion. \n\nMoreover, a direct performance comparison is not meaningful due to the significant disparity in model scale between GPT-4V and our 7B-scale models (Qwen-VL, LLaVA-1.5, and DeepSeek-VL2). And model scale is a key factor influencing the associative reasoning capabilities of MLLMs. This effect is confirmed in Table I, within the same model family (Qwen-VL), the performance gap between the 7B and 72B variants is substantial.\nTable I. Compare with GPT-4V on Creation-MMBench.\n\n| Methods               | Overall Reward(↑) | LW Reward | CMU Reward | PFW Reward | CFW Reward |\n| --------------------- | ----------------- | --------- | ---------- | ---------- | ---------- |\n| Qwen-VL               | 0.00              | 0.00      | 0.00       | 0.00       | 0.00       |\n| Qwen-VL+FlexAC        | 10.92             | 15.63     | 6.11       | 5.96       | 15.65      |\n| Qwen2-VL-72B-Instruct | 75.96             | 81.04     | 69.05      | 82.98      | 68.42      |\n| GPT-4V                | 85.17             | 92.10     | 76.12      | 91.92      | 87.12      |\n\n---\n\n_**Limitations 1. The experiments primarily relied on the COCO dataset (object-centric static images) and did not validate performance in video temporal reasoning, abstract art comprehension, or cross-cultural scenarios. If the model fails in dynamic or multimodal associative tasks , would FlexAC's modulation mechanism remain effective?**_\n\n**Response:** We would like to clarify that our evaluation on Creation-MMBench[5]—a diverse benchmark comprising **51 creative tasks** and no any COCO data—already includes subtasks covering Video Temporal Reasoning, Abstract Art Comprehension, and Cross-Cultural Scenarios:\n- **Video Temporal Reasoning**: This is directly addressed by tasks like **\"story continue\"** of Creation-MMBench (**Reward Score: 20.0**). In this task, the model is presented with **multiple sequential frames from an animation** and must continue the narrative. This directly tests the model's ability to understand the temporal progression of events in the visual input. \n- **Abstract Art Comprehension**: The benchmark includes the \"**Art inspired prose**\" task (**Reward Score: 30.0**) and a dedicated \"Art\" image category (5% of the dataset), which involves interpreting various paintings. \n- **Cross-Cultural Scenarios**: Creation-MMBench is rich in cross-cultural contexts, featuring a \"History & Culture\" image category and tasks like **\"social media travel content\"**. (**Reward Score: 16.7**)\n\n[1] Yin, Hao, Guangzong Si, and Zilei Wang. \"Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference.\" CVPR. 2025.\n\n[2] Zhang, Zhi, et al. \"Cross-modal information flow in multimodal large language models.\" CVPR. 2025.\n\n[3] Meng, Kevin, et al. \"Locating and editing factual associations in gpt.\" _Advances in neural information processing systems_ 35 (2022): 17359-17372.\n\n[4] Chuang, Yung-Sung, et al. \"DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models.\" _The Twelfth International Conference on Learning Representations_. 2023.\n\n[5] Fang, Xinyu, et al. \"Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM.\" ICCV. 2025."}, {"review_id": "1da1Iz5Kfn", "reviewer": "Reviewer_g1qC", "summary": "This paper presents FlexAC, a training-free framework for controlling associative reasoning in MLLMs. Through layer-wise analysis, the authors identify that middle layers are critical for encoding associative behavior. FlexAC constructs steering vectors from the difference between hallucinated and grounded representations in these layers, then applies them during inference to modulate the model's behavior between factuality and creativity. The paper also introduces VDAT, a new benchmark for measuring associative reasoning strength. Experiments on three MLLMs demonstrate that FlexAC can significantly reduce hallucination and enhance creativity depending on the steering direction, while maintaining general task performance.**Strengths:**\n\n1. **Well-executed empirical analysis**: The layer-wise localization experiments clearly demonstrate that middle layers control associative behavior, using both feature analysis and intervention studies.\n\n2. **Practical method**: Training-free, works on existing MLLMs, computationally efficient - just adds vector operations during inference.\n\n3. **Effective bidirectional control**: Successfully reduces hallucination and enhances creativity based on steering direction, addressing different application needs.\n\n4. **New benchmark (VDAT)**: Provides a simple, direct metric for measuring visual associative reasoning strength, filling a gap between hallucination metrics and subjective creativity evaluation.\n5. **Consistent results across architectures**: The method shows improvements across three different model families with varying architectures.\n\n**Weaknesses:**\n\n1. **Similar findings in prior work**: Although well-executed empirical analysis, the observation that middle layers control hallucination has been reported in previous studies[1].  This paper builds upon these existing findings by adding steering vector techniques.\n\n2. **Model-specific tuning**: Optimal layers vary dramatically (DeepSeek: 4-6, Qwen: 15-17), requiring manual search for each architecture. \n\n3. **Heavy reliance on task-specific non-associative features**: The method depends on non-associative features extracted from specific hallucination scenarios, but these features vary significantly across tasks, limiting generalization to different applications.\n\n4. **Limited evaluation on general benchmarks**: The authors only evaluate on 3 general benchmarks (MME, MMMU, MMStar), which is insufficient to fully assess the impact on model's general capabilities. \n\n[1]. Wang, Sudong, et al. \"Towards understanding how knowledge evolves in large vision-language models.\" *Proceedings of the Computer Vision and Pattern Recognition Conference*. 2025.", "questions": "1. **Model generalization and comprehensive evaluation**: My biggest concern is the generalization ability of FlexAC. As mentioned in W3 and W4, could you provide evaluation results on more diverse benchmarks to better assess the impact on general capabilities? (You don't need to evaluate on all of these, but results from a representative subset would help demonstrate that the non-associative steering doesn't harm performance on diverse tasks requiring different types of reasoning and association.)\n   - General multimodal benchmarks: MMBench, SEED-Image, MMVet.\n   - Vision-centric benchmarks: CVBench, RealWorldQA.\n   - OCR and document understanding: TextVQA, ChartQA, DocVQA.\n2. **Architectural dependence**: The optimal layers vary dramatically across models. How would you determine optimal control layers for a new model architecture? Is manual search always required, or can this be automated?\n3. **Steering vector stability**: How stable are the derived steering vectors across different prompts and domains? Do they need to be recomputed frequently, or can a fixed set be reused effectively?\n4. **VDAT validity**: Can you provide more rigorous validation of VDAT beyond CLIP similarity? How does it correlate with human judgments of creativity quality? Automatic metrics may not fully capture quality differences in creative tasks.", "limitations": "The authors mention that FlexAC requires white-box access to hidden states. This issue is related to whether the model is open-sourced or not.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n1. **Well-executed empirical analysis**: The layer-wise localization experiments clearly demonstrate that middle layers control associative behavior, using both feature analysis and intervention studies.\n\n2. **Practical method**: Training-free, works on existing MLLMs, computationally efficient - just adds vector operations during inference.\n\n3. **Effective bidirectional control**: Successfully reduces hallucination and enhances creativity based on steering direction, addressing different application needs.\n\n4. **New benchmark (VDAT)**: Provides a simple, direct metric for measuring visual associative reasoning strength, filling a gap between hallucination metrics and subjective creativity evaluation.\n5. **Consistent results across architectures**: The method shows improvements across three different model families with varying architectures.\n\n**Weaknesses:**\n\n1. **Similar findings in prior work**: Although well-executed empirical analysis, the observation that middle layers control hallucination has been reported in previous studies[1].  This paper builds upon these existing findings by adding steering vector techniques.\n\n2. **Model-specific tuning**: Optimal layers vary dramatically (DeepSeek: 4-6, Qwen: 15-17), requiring manual search for each architecture. \n\n3. **Heavy reliance on task-specific non-associative features**: The method depends on non-associative features extracted from specific hallucination scenarios, but these features vary significantly across tasks, limiting generalization to different applications.\n\n4. **Limited evaluation on general benchmarks**: The authors only evaluate on 3 general benchmarks (MME, MMMU, MMStar), which is insufficient to fully assess the impact on model's general capabilities. \n\n[1] Towards understanding how knowledge evolves in large vision-language models.", "was_revised": true, "final_justification": "I would like to thank the authors for their detailed rebuttal and comprehensive effort in providing additional experimental results. My concerns have been addressed, and I will raise my score to 4.", "rebuttal": "_**Weakness 1. Similar findings in prior work: Although well-executed empirical analysis, the observation that middle layers control hallucination has been reported in previous studies[1]. This paper builds upon these existing findings by adding steering vector techniques.**_\n\n**Response:** We would like to clarify that FlexAC is an independent study conducted without prior knowledge of [1]. We will add a discussion to the Related Work section to elaborate on the key differences between the two approaches.\n1. **Different Research Goals and Findings with [1].** \n\t- **Different Research Objectives:** The goal of [1] is _diagnostic_—to investigate \"where and how do hallucinations arise?\". In contrast, FlexAC is the first work to **explore hallucination's intrinsic link with creativity**, and provide a new method for bidirectional control. Our focus is not on controlling hallucination, but rather on understanding and leveraging its generative potential.\n\t- **Different Core Findings:** The finding in [1] is that **\"middle layers control hallucination.\"** Our core finding is different: we found that the state of hallucination can be harnessed to **guide creative behavior**.\n2. **Methodological Novelty Beyond \"Adding Steering Vectors\":** Our method is more than adding steering vectors. We introduce **three key innovations**:\n\t- Hallucination-Guided Steering: We introduce a novel method to **derive association steering vectors by leveraging hallucinated states** to enhance the model's associative capabilities.\n\t- Steering Intensity Calibration (SIC): To prevent over-steering, we designed the SIC module, which **adaptively scales steering vectors**.\n\t- Directional Integration: To further support tasks requiring multi-dimensional associations, we **augment general associative vector  $v_{l}^{\\mathrm{gen}}$ in Eq.7 with task-specific associative vectors $v_{l}^{\\mathrm{task}}$ in Eq.7** derived from GPT-4o-generated, high-association samples.\n\n[1] Wang, Sudong, et al. \"Towards understanding how knowledge evolves in large vision-language models.\" CVPR. 2025.  \n\n\n---\n\n_**Weakness 2.  Model-specific tuning: Optimal layers vary dramatically (DeepSeek: 4-6, Qwen: 15-17), requiring manual search for each architecture.**_\n\n**Response:**  We clarify that our method does **not** require manual search. For any MLLM, the optimal layers can be automatically identified via a **single forward pass** by maximizing the cosine distance between features from associative and non-associative inputs (Section 2.1 and Appendix 6.4).\n$$L^∗ = \\{l^∗, l^∗ + 1, l^∗ + 2\\}, \\quad \\text{where} \\quad l^∗ = \\arg\\max_{l} \\sum_{i=0}^{2} D_{\\cos}(f_{l+i}^{(a)}, f_{l+i}^{(n)})$$\nHere, $f^{(a)}$ and $f^{(n)}$ are features from associative and non-associative inputs, and $D_{\\cos}$ denotes cosine distance.\n\n---\n\n_**Weakness 3. Heavy reliance on task-specific non-associative features: The method depends on non-associative features extracted from specific hallucination scenarios, but these features vary significantly across tasks, limiting generalization to different applications.**_\n\n**Response:**  Our method does not rely on features from \"specific\" scenarios; its core is a _general steering vector_ designed for broad applicability.：\n- **General Hallucination Scenarios**: Our primary steering vector is computed from a large, diverse set of images (_randomly sampled from COCO_) to capture a fundamental and universal associative pattern of the model. This ensures it has strong generalization for new tasks.\n- **Broad Experiments Directly Prove Generalization**: Its generalization is directly proven by our results on **Creation-MMBench**[1] (Table 3). This benchmark includes **51 distinct creative tasks**, where our method achieves a significant **+10.92 Reward**, while competing methods show performance degradation (-3.86 and -1.63).\n\n[1] Fang, Xinyu, et al. \"Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM.\" ICCV. 2025.\n\n---\n\n_**Weakness 4.  Limited evaluation on general benchmarks: The authors only evaluate on 3 general benchmarks (MME, MMMU, MMStar), which is insufficient to fully assess the impact on model's general capabilities.**_\n\n**Response:**  In direct response, we conducted a comprehensive new evaluation on **11 diverse benchmarks**, including those you recommended, to assess the impact on the model's general capabilities.\n\nTable I: Performance of FlexAC on 11 general-purpose benchmarks with the Qwen-VL model.\n\n|Category|Benchmark|Regular|FlexAC-C|FlexAC-P|\n|-|-|-|-|-|\n|**GeneralMultimodal**|MM-Vet|39.81|38.17|37.33|\n||MMBench|0.581|0.598|0.576|\n||SEED-Bench|0.638|0.625|0.640|\n||MMMB|0.703|0.678|0.699|\n|**Vision-centric**|RealWorldQA|0.486|0.490|0.495|\n||CVBench|0.549|0.524|0.560|\n||AI2D|0.612|0.614|0.616|\n|**OCR&Document**|TextVQA|60.66|60.78|59.81|\n||ChartQA|48.36|49.40|45.92|\n||DocVQA|57.79|56.85|57.59|\n||OCRVAQ|47.46|49.74|45.83|\n\nAcross all 11 benchmarks, FlexAC maintains performance **closely comparable to the baseline**. This provides strong evidence that our targeted control mechanism **does not harm the model's general capabilities**.\n\n---\n\n_**Question 1. Model generalization and comprehensive evaluation: My biggest concern is the generalization ability of FlexAC. As mentioned in W3 and W4, could you provide evaluation results on more diverse benchmarks to better assess the impact on general capabilities? (You don't need to evaluate on all of these, but results from a representative subset would help demonstrate that the non-associative steering doesn't harm performance on diverse tasks requiring different types of reasoning and association.)\n    - General multimodal benchmarks: MMBench, SEED-Image, MMVet.\n    - Vision-centric benchmarks: CVBench, RealWorldQA.\n    - OCR and document understanding: TextVQA, ChartQA, DocVQA.**_\n\n**Response:**  Thank you. We have addressed this directly in our response to **Weakness 4** and present new results on 11 benchmarks in **Table I**. These comprehensive results confirm our method generalizes well without harming core capabilities.\n\n---\n\n_**Question 2.  Architectural dependence: The optimal layers vary dramatically across models. How would you determine optimal control layers for a new model architecture? Is manual search always required, or can this be automated?**_\n\n**Response:** \nAs detailed in our response to **Weakness 2**, this process is **straightforward**. For any new model, we use a lightweight analytical method (a single forward pass and cosine distance check) to identify optimal layers, **requiring no manual search**.\n\n---\n\n_**Question 3.  Steering vector stability: How stable are the derived steering vectors across different prompts and domains? Do they need to be recomputed frequently, or can a fixed set be reused effectively?**_\n\n**Response:**  \n- The vector is highly stable. We proved this by successfully applying a **single, fixed vector, i.e., $v_{l}^{\\mathrm{gen}}$** in Eq.7 of manuscript—without modification—across our entire range of diverse benchmarks (CHAIR, MME, MMMU, MMStar).\n- Our vector was **computed only once** from diverse data (COCO) and then used for all subsequent experiments, demonstrating that frequent re-computation is not necessary. A fixed set can be **reused effectively**.\n\n---\n\n\n_**Question 4.  VDAT validity: Can you provide more rigorous validation of VDAT beyond CLIP similarity? How does it correlate with human judgments of creativity quality? Automatic metrics may not fully capture quality differences in creative tasks.**_\n\n**Response:**  To answer your question, we would like to clarify your misconception regarding the distinction between VDAT evaluation and creativity assessment.\n\nAs stated in the paper (line 189), **VDAT is designed as a diagnostic benchmark to assess associative reasoning ability**, rather than as a direct measure of functional creativity. Creativity is a multifaceted construct; therefore, we supplement VDAT with **Creation-MMBench[1]**, which evaluates open-ended, task-level creative performance (Table 3).\n- **Empirical Validation via User Study**: To assess the validity of VDAT as a measure of associative creativity, we conducted a human evaluation comparing FlexAC with several baselines (Appendix 4.1).\n- **Correlation with  Human Judgments**: A user study involving 15 raters demonstrated a strong correlation between VDAT scores and human assessments of associative ability (Appendix 4.1).\n- **Foundation of VDAT's Validity**:  Our VDAT builds upon the DAT benchmark [3], which employs automatic metrics to evaluate associative reasoning in LLMs. DAT has been both theoretically [2] and empirically [3] validated as effective. Moreover, models (e.g., FlexAC) with higher VDAT scores (Table 2) consistently outperformed others on Creation-MMBench (Table 3), indicating that VDAT reliably captures key aspects of creative quality in open-ended tasks.\n\n[1] Fang, Xinyu, et al. \"Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM.\" ICCV. 2025.\n\n[2] Jay A. Olson, Johnny Nahas, Denis Chmoulevitch, Simon J. Cropper, and Margaret E. Webb. Naming unrelated words predicts creativity. Proceedings of the National Academy of Sciences, 2021.\n\n[3] Honghua Chen and Nai Ding. Probing the “creativity” of large language models: Can models produce divergent semantic association? In EMNLP, 2023."}, {"review_id": "nqbnlfMwGW", "reviewer": "Reviewer_2JXU", "summary": "This paper proposes FlexAC (Flexible Association Control), a training-free framework for modulating associative reasoning strength in multimodal large language models (MLLMs). The authors investigate how associative behavior emerges in MLLMs and find that middle layers are critical for shaping associative tendencies. Based on these findings, they develop a method that uses hallucination-guided steering vectors to control the trade-off between faithfulness and creativity. The approach is evaluated on hallucination benchmarks (CHAIR, POPE), creativity benchmarks (VDAT, Creation-MMBench), and general-purpose benchmarks (MME, MMMU, MMStar).", "questions": "1. The steering vector is defined as the difference between hallucinated and grounded representations. Could this vector be capturing \"non-factualness\" or \"context-ignorance\" rather than genuine creativity? Have you explored alternative methods for deriving creativity-enhancing vectors using high-quality human-generated creative content instead of model-induced hallucinations?\n2. How does VDAT performance correlate with functional, grounded creative tasks? Is there evidence that pushing VDAT scores higher doesn't simply train the model to ignore visual input, undermining its utility as a multimodal model?\n3. The method requires empirical identification of optimal control layers for each model. How sensitive is performance to layer selection? For practitioners with new MLLMs, is there a principled way to identify these layers beyond exhaustive testing?\n4. How well does the method transfer across different types of creative tasks beyond the evaluated benchmarks? What evidence supports that the observed behavior represents genuine creativity rather than reduced factual grounding?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The paper provides an interesting unified view linking hallucination and creativity through associative reasoning mechanisms, which offers a fresh theoretical perspective on MLLM behavior.\n- The layer-wise analysis identifying middle layers as critical for associative behavior is well-conducted and provides valuable insights into MLLM internal mechanisms.\n- The proposed method requires no additional training, making it practically appealing and computationally efficient.\n\nWeaknesses:\n- The central premise that creativity and hallucination represent two ends of a single \"associative strength\" spectrum is a significant oversimplification.\n- The core technique of manipulating model behavior through activation steering is well-established. This work is essentially a direct application of existing representation steering concepts to MLLMs.\n- A significant portion of creativity evaluation depends on VDAT, a new benchmark that measures ability to generate nouns unrelated to an image. This is an extremely narrow and potentially misleading proxy for divergent thinking. A model could score highly on VDAT by simply ignoring the image, which contradicts the goal of image-grounded creativity.", "was_revised": true, "final_justification": "Overall, the additional explanations and evidence provided in the rebuttal have addressed several of the original concerns. Although some aspects could still be refined, the progress is clear and encouraging. I raised the score.", "rebuttal": "_**Weakness 1. Associative Strength Spectrum Is Oversimplified.**_\n\n**Responses:** We agree this is a simplification, but argue it's an **effective operational model** rather than an oversimplification.\n- **Theoretical Grounding**:  Inspired by neuroscience research linking creativity and hallucination[1], we propose an 'associative strength' spectrum to model this connection. Drawing on theories of **convergent and divergent thinking**[2], we frame faithfulness as low associative strength (convergent) and creativity as high (divergent), with hallucination seen as an extreme form of uncontrolled divergence\n- **Empirical Validation**: Critically, Fig. 3 validates this model. By adjusting a single coefficient, α, we make it success to control both hallucination (CHAIR: 38.8→53.6) and associative strength (VDAT: 83→87.9). This provides strong empirical evidence that the Associative Strength Spectrum is an effective and practical abstraction for steering between creativity and hallucination\n- **Beyond a Single Spectrum**: Our framework goes **beyond** a unidimensional two-end simplification. We acknowledge the existence of **diverse creative trajectories between creativity and hallucination**. To capture this complexity, we introduce task-specific associative vectors that guide the model along multiple creative pathways, thereby addressing the inherently multidimensional nature of creativity\n\n[1] To create or to recall? Neural mechanisms underlying the generation of creative new ideas(NeuroImage 2014)\n\n[2] The standard definition of creativity (Creativity research journal 2012)\n\n---\n_**Weakness 2.  This work is essentially a direct application of existing representation steering concepts to MLLMs.**_\n\n**Responses:** While our work utilizes established steering principles, it introduces two novel contributions specifically for controlling associative reasoning in MLLMs:\n- **A Novel Control Signal**: We are the first to discover that the direction of **hallucinated representations can be used to steer associative reasoning**. This offers a new perspective on the relationship between hallucination and creativity.\n- **Multi-Directional Steering for Creativity**: Recognizing **the multi-dimensional nature of creativity**, we introduce Directional Integration. This technique combines both **general $v_{l}^{gen}$ and task-specific vectors $v_{l}^{task}$ in Eq.7** to adapt to varied creative tasks, a significant advance over conventional single-vector steering\n\n---\n_**Weakness 3. VDAT is a narrow and potentially misleading proxy for divergent thinking. A model could achieve a high score by simply ignoring the visual input, which undermines the goal of image-grounded creativity.**_\n\n**Responses:** \n- **VDAT and Divergent Thinking:** VDAT is a principled, multimodal adaptation of the validated Divergent Association Task (DAT), a paradigm from cognitive science[1]. Divergent thinking, a core component of creativity, is defined as the ability to generate diverse ideas from a single stimulus[2]. Research has established that the semantic distance between these ideas serves as a robust, objective measure of this ability[1,3]. VDAT operationalizes this principle by using an image as the stimulus.\n- **High VDAT ≠ Ignoring the Image**:  Outputs with high VDAT often include objects that are relevant to the image's scene but are not visually present. As shown in Fig. 23 in Appendix, an indoor cooking scene includes \"chair\" and \"table\"—typical kitchen items not pictured. \n- **Consistency Performance with Creation-MMBench**: For a holistic assessment, we rely on Creation-MMBench[4], which evaluates overall image-grounded creative generation in terms of VFS scores. Higher associative strength of our FlexAC on VDAT (Table 2) directly translates to higher-quality, image-grounded creative generation on Creation-MMBench[4] (Table 3) with high VFS scores. This consistency validates that VDAT effectively measures a fundamental mechanism contributing to broader creative intelligence.\n\n[1] Naming unrelated words predicts creativity (National Academy of Sciences 2021)\n\n[2] Creativity: Yesterday, today and tomorrow (Journal of Creative Behavior 1967)\n\n[3] Probing the “creativity” of large language models: Can models produce divergent semantic association (EMNLP 2023)\n\n[4] Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM (ICCV 2025)\n\n---\n_**Question 1.  Does the Vector Reflect Creativity or Context-Ignorance? Why Not Use Human Data?**_\n\n**Responses:** \n1. **On Why Hallucination is a Valid Signal for Creativity, Not \"Context-Ignorance\"** \n\t- Our use of hallucinations as a signal source is grounded in cognitive science: studies show that the brain regions involved in creative thinking significantly overlap with those that produce hallucinations[1]. This suggests hallucinations may reflect an intensified form of divergent thinking—semantically rich, contextually relevant, yet unconstrained by reality\n\t- The ultimate proof is empirical. If the vector merely captured \"context-ignorance,\" it would degrade performance on functional creative tasks. Instead, FlexAC achieves a **+10.92 Reward on Creation-MMBench**, while competing methods are negative (Table 3). This provides strong evidence that our vector is steering towards genuine, functional creativity\n2. **On Using Human-Generated Content**: We did experiment with using human-written captions to extract creativity vectors. Annotators were instructed to “be creative,” resulting in diverse styles—narrative, humorous, metaphorical, and anthropomorphic. While imaginative, this diversity introduced **high variance**, making the averaged vectors noisy and inconsistent.\n\nAs shown in Table I, models guided by these human-derived vectors performed worse (Reward: -22.25), indicating their instability as guidance signals. In contrast, hallucination-guided vectors, though model-generated, provided more consistent and controllable cues, leading to improved image-grounded creativity. This may be because human creativity often **relies on loosely grounded rhetorical styles** that are **less suitable for generating coherent guidance**.\n\nTable I: Performance of human-generated vs. hallucination-derived steering vectors on Qwen-VL\n\n|Methods|OverallVFS(↑)|OverallReward(↑)|\n|-|-|-|\n|Regular|6.10|0.00|\n|FlexAC|6.25|10.92|\n|Human-Generated|5.12|-22.25|\n\n[1] To create or to recall? Neural mechanisms underlying the generation of creative new ideas (NeuroImage 2014)\n\n---\n_**Question 2. How does VDAT performance correlate with grounded creative tasks? Is there evidence that pushing VDAT scores higher doesn't simply train the model to ignore visual input, undermining its utility as a multimodal model?**_\n\n**Responses:** As stated in our paper (line 189), VDAT is a **diagnostic benchmark** designed to measure a model's **associative reasoning strength**, which serves to verify our control mechanism's effectiveness. The evaluation of functional **creativity quality** is primarily conducted using **Creation-MMBench**. Our results show:\n- **Positive Correlation with Functional Creative Tasks**: Higher VDAT scores of our FlexAC (Table 2) consistently lead to better creative performance on the grounded tasks in Creation-MMBench (Table 3). This consistent trend validates that the capability measured by VDAT translates to better performance on practical creative tasks. \n- **Evidence Against Ignoring Visual Input**: We explicitly test for \"ignoring visual input\" using the **Visual Fidelity Score (VFS)** in Creation-MMBench, which measures the visual-text alignment degree. Table 3 shows our FlexAC improves creative Reward while VFS remains high and comparable to the baseline (6.25 vs 6.1). This provides strong, quantitative evidence that our method enhances creativity without sacrificing visual grounding\n\n---\n_**Question 3. The method requires identifying optimal control layers for each model. How sensitive is performance to layer selection? Is there a principled way to select layers for new MLLMs without exhaustive testing?**_\n\n**Responses:** Layer selection is a **principled and simple procedure**, not an exhaustive search. \n- **On Sensitivity to Layer Selection**: The sensitivity analysis for Qwen-VL, LLaVA, and DeepSeek-VL is shown in Fig. 10 (main paper) and expanded in Fig. 18–19 (appendix).\n- **On the Method for Identifying Optimal Layers**: Layer selection is **not an exhaustive search**, but a principled, simple procedure. For any new model, we simply perform a single forward pass with associative/non-associative pairs ($f^{(a)}$, $f^{(n)}$) and identify the three layers where the **cosine distance $D_{\\cos}$ between hidden states peaks** $$L^∗ = \\{l^∗,l^∗+1,l^∗+2\\}, \\quad \\text{where} \\quad l^∗ = \\arg\\max_{l} \\sum_{i=0}^{2} D_{\\cos}(f_{l+i}^{(a)}, f_{l+i}^{(n)})$$\n\n\n---\n_**Question 4. How does VDAT performance correlate with functional, grounded creative tasks? What evidence shows that improving VDAT scores doesn't cause the model to ignore visual input?**_\n\n**Responses:** \n- **Transferring Across Creative Tasks**: Transferability is directly validated by its strong performance on **Creation-MMBench**, which comprises **51 diverse creative tasks** across varied domains (e.g., Art, Animation, UI).\n- **Genuine Creativity vs. Reduced Factual Grounding**: \n\t- **Quantitative Evidence**: Concerns about reduced factual grounding are refuted by the **Visual Fidelity Score (VFS)**, which evaluates image-text alignment in Creation-MMBench (Table 3): despite a +10.92 gain in creativity reward, our method maintains a high VFS (6.25), slightly above the baseline (6.10), showing creativity improves without compromising factual grounding.\n\t- **Qualitative Evidence**: As shown in Fig.26 (dinosaur story), FlexAC-C creates a novel, coherent scene by adding plausible elements (e.g., \"butterfly\", \"worm\") and using them in **specific actions and dialogue** to develop character—demonstrating genuine narrative creation."}, {"review_id": "NMfXwJk5Z6", "reviewer": "Reviewer_Up2p", "summary": "The paper presents FlexAC, a lightweight method for steering associative reasoning in multimodal LLMs by modifying middle-layer model parameters. Using parameter vectors extracted from hallucinated outputs, the approach adjusts the model’s balance between factual accuracy and creativity. Experiment results validate the steerability on hallucination and creativity.", "questions": "1. Since the non-middle layers show minimal change on hallucination, have the authors explored applying steering vectors to all layers as an ablation?\n\n2. Does FlexAC operate solely on the language-side layers, or is the vision encoder also considered? A brief discussion would clarify the scope of parameter mixing.\n\n3. Equations (7) and (9) suggest both unnormalized and normalized vector mixing. Given the presence of LayerNorm in the transformer, can the authors comment on whether there are redundant computations involved?\n\n4. While most prior efforts on steerable LLMs focus on trainable approaches, they are still relevant and should be discussed in the related work section. Including these works would help position FlexAC more clearly within the broader landscape of controllable and steerable LLM/MLLMs.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n1. The identification of middle layers as critical for hallucination control is novel and well-supported by empirical analysis. The proposed approach for steering between faithfulness and creativity is thoughtfully designed.\n\n2. The analysis of associative behavior is well-motivated and convincingly executed, especially the layer-wise representation study. These analyses provide solid empirical grounding for the design of the control mechanism.\n\n3. The introduction of the VDAT benchmark fills an evaluation gap for associative reasoning in MLLMs.\n\n\n\n**Weaknesses**\n\n1. **Data quality**: The construction of the steering vector depends on hallucinated input-response pairs, which are derived via blurred inputs and specific prompting. This may introduce distributional biases, e.g., capturing only specific hallucination types. Besides that, top-k selection can be sensitive to outlier data. Some robustness or sensitivity analysis of the vector construction would strengthen the approach.\n\n2. **Connection between hallucination vector and associative reasoning**: While the extracted steering vector clearly controls the factual grounding, it's less certain whether it directly reflects a model’s associative reasoning capability. The paper seems to assume that hallucinated features correspond to associative reasoning, but this isn't explicitly validated. For instance, in Figure 9 (b)–(c), FlexAC-P (presumably more powerful on associative reasoning) performs similarly or worse than FlexAC-C on reasoning-heavy tasks like MMMU and Math. Therefore, it seems to me this method is for hallucination control instead of associative reasoning control.\n\n3. **Model Architecture**: The empirical analysis on layer-wise impact on associative reasoning are demonstrated only on LLaVA model. It remains unclear whether the same behavior holds for other architectures, particularly mixture-of-experts (MoE) models.", "was_revised": true, "final_justification": "I thank the authors for their effective rebuttal, which has clarified my main questions regarding the method's robustness and generalization.\n\nWhile my major concerns have been resolved, I feel the connection drawn between the 'hallucination vector' and 'associative reasoning' could be presented with more nuance. I believe that further clarifying the assumptions behind this conceptual link would be a valuable addition to the final paper.\n\nIn light of the authors' successful efforts in addressing my primary concerns, I have increased my score from 3 to 4.", "rebuttal": "_**Weakness 1. Data quality: The construction of the steering vector depends on hallucinated input-response pairs, which are derived via blurred inputs and specific prompting. This may introduce distributional biases, e.g., capturing only specific hallucination types. Besides that, top-k selection can be sensitive to outlier data. Some robustness or sensitivity analysis of the vector construction would strengthen the approach.**_\n\n**Responses:** \n- **Hallucination Distribution**: We analyzed our data and confirmed it contains a rich mixture of both **object-level (86.35%) and relation-level (94.8%) hallucinations**, ensuring it is not biased toward a specific type.\n- **Sensitivity Analysis of Top-K**: Our method demonstrates robustness in two key aspects: the `top-k` selection is not sensitive to outliers, and the optimal `k` value is generalizable across datasets.\n\t- **Generalizability of `k=50`:** As shown in the table I and our original appendix (Figure. 17), `k=50` is a robust choice that delivers near-peak performance across all tested benchmarks (CHAIR, VDAT, MME, HallusionBench), making it a reliable and practical default setting.\n\t- **Robustness to Outliers:** The method is not sensitive to including a large number of instances (and potential outliers). For instance, on MME (Perception), performance degrades gracefully from the peak at `k=50` (1427.8) to `k=2000` (1390.0), rather than collapsing. This confirms the robustness of our vector construction.\n\nTable I: Sensitivity analysis of Top-K selection. \n\n|Benchmark|k=1|k=50|k=100|k=1000|k=2000|\n|-|-|-|-|-|-|\n|MME(Perception)|1397.3|1427.8|1423.7|1397.4|1390.0|\n|MME(reasoning)|371.8|413.6|413.8|406.1|405.7|\n|HallusionBench|50.6|50.6|50.3|49.1|49.2|\n\n---\n_**Weakness 2. Connection between hallucination vector and associative reasoning: While the extracted steering vector clearly controls the factual grounding, it's less certain whether it directly reflects a model’s associative reasoning capability. The paper seems to assume that hallucinated features correspond to associative reasoning, but this isn't explicitly validated. For instance, in Figure 9 (b)–(c), FlexAC-P (presumably more powerful on associative reasoning) performs similarly or worse than FlexAC-C on reasoning-heavy tasks like MMMU and Math. Therefore, it seems to me this method is for hallucination control instead of associative reasoning control.**_\n\n**Responses:**  We believe there may be a critical misunderstanding of our notation which, once clarified, resolves the core concern.\n\n- **A Crucial Clarification: The Reviewer's Premise on P vs. C is Inverted.**\nThe reviewer’s analysis mistakenly assumes that `FlexAC-P` enhances associative reasoning, which contradicts our definition. In fact, the correct premise is as follows: **`FlexAC-C` (Creativity, α = 1)** is designed to **ENHANCE** associative reasoning; **`FlexAC-P` (Precision, α=-1)** is designed to **SUPPRESS** it.\n\n- **Re-interpreting Figure 9 with the Correct Premise：** With the correct premise, the data in Figure 9 now strongly supports our claim. It shows that `FlexAC-C` (the creativity-enhanced version) maintains stable performance on logical reasoning tasks (MMMU/Math). This is a key feature: **our method precisely targets _associative reasoning_ without disrupting the model's separate, core _logical reasoning_ capabilities.**\n- **Direct Validation on Creativity Tasks.** The fact that our vector controls more than just hallucination is validated by `FlexAC-C`'s strong performance on creativity-focused benchmarks: It generates abstract, metaphorical content on **Creation-MMBench**. It improves divergent thinking scores on **VDAT**, a result confirmed by our **human study (Appendix, Sec. 4.1)**.\n\n---\n_**Weakness 3. Model Architecture: The empirical analysis on layer-wise impact on associative reasoning are demonstrated only on LLaVA model. It remains unclear whether the same behavior holds for other architectures, particularly mixture-of-experts (MoE) models.**_\n\n**Responses:** Our submitted appendix (App. 6.4, Fig. 30) already provides feature analysis for Qwen-VL and **the MoE-based DeepSeek-VL2**. To provide even more direct evidence, the following intervention analysis for DeepSeek-VL2 shows each layer's impact; **a larger '$\\Delta$' value signifies a greater influence on the final result:**\n\n|Intervention Layer|Last Layer Euclidean Distance|$\\Delta$ (↓)|\n|:-:|:-:|:-:|\n|0|8.8|–|\n|1|8.6|-0.2|\n|2|8.6|-0.0|\n|3|8.1|-0.5|\n|**4**|**6.8**|**-1.3**|\n|**5**|**5.5**|**-1.3**|\n|**6**|**4.1**|**-1.4**|\n|7|2.9|-1.2|\n|8|2.3|-0.6|\n|9|1.7|-0.6|\n|10|1.4|-0.3|\n|11|1.2|-0.2|\n\nThe table's \"$\\Delta$\" column peaks at Layer 6 (-1.4), confirming our paper's finding that the middle layers (4-7) are the critical control region. This result from the MoE model is consistent with our main analysis. The full plots will be added to the camera-ready version.\n\n---\n_**Question 1. Since the non-middle layers show minimal change on hallucination, have the authors explored applying steering vectors to all layers as an ablation?**_\n\n**Responses:** Per your advice, we conducted an ablation study applying the steering vector to all layers.\n\nTable II: Comparison of Middle-Layer vs. All-Layer Intervention in LLaVA.\n\n| Methods               | CHAIRs | CHAIRi | Recall | Len  |\n| --------------------- | ------ | ------ | ------ | ---- |\n| Regular               | 50.8   | 14.3   | 79.7   | 97.3 |\n| FlexAC (middle-layer) | 36.6   | 10.5   | 75     | 95.1 |\n| FlexAC (all-layers)   | 0.2    | 0.6    | 7.1    | 22.3 |\n\nWhile the 'all-layers' intervention drastically reduces CHAIR scores, it comes at the cost of catastrophic damage to the model's generative ability. This is evidenced by the collapse in performance (e.g., LLaVA's Recall dropped from 75.0 to 7.1) and the frequent generation of meaningless outputs like _empty strings or gibberish_. This result confirms that all-layer intervention is harmful, thus validating our design choice to precisely target only the middle layers. The possible reason is that all-layer intervention disrupts both low-level representations and high-level semantics, leading to unstable decoding and degraded output.\n\n---\n_**Question 2. Does FlexAC operate solely on the language-side layers, or is the vision encoder also considered? A brief discussion would clarify the scope of parameter mixing**_\n\n**Responses:**  FlexAC operates **solely on the language-side layers**, a deliberate choice based on the standard MLLM architecture.\n\nIn MLLMs, the vision encoder functions as the perceptual \"eyes,\" while the LLM is the cognitive \"brain\" responsible for reasoning[1]. Since hallucination and creativity are **cognitive phenomena** that occur in the \"brain,\" our intervention is precisely targeted at the LLM's layers where this reasoning takes place\n\n[1] A survey on multimodal large language models (National Science Review 2024)\n\n---\n_**Question 3. Equations (7) and (9) suggest both unnormalized and normalized vector mixing. Given the presence of LayerNorm in the transformer, can the authors comment on whether there are redundant computations involved?**_\n\n**Responses:** **No redundant computation is involved.** Our normalization (Eq. 9) and the transformer's LayerNorm serve distinct, complementary purposes:\n- **Our Eq. 9 (Norm Preservation):** \n\t- **Target:** A **single** hidden state vector after we have steered it.\n\t- **Purpose:** To reset the steered vector's magnitude to match the **original** vector's magnitude. This is a stability measure ensuring our intervention only changes the representation's _direction_, not its strength. This is a common practice in activation steering[1,2,3].\n- **Transformer's LayerNorm:** \n\t- **Target:** The **entire set** of all token vectors in the sequence.\n\t- **Purpose:** To rescale the _distribution_ of this entire group of vectors (e.g., to zero mean, unit variance) to ensure a stable input for the **next** network layer.\n\n[1] Reducing hallucinations in large vision-language models via latent space steering (ICLR 2025)\n\n[2] Steering away from harm: An adaptive approach to defending vision language model against jailbreaks(CVPR 2025)\n\n[3] Safety arithmetic: A framework for test-time safety alignment of language models by steering parameters and activations(EMNLP 2024)\n\n---\n_**Question 4. While most prior efforts on steerable LLMs focus on trainable approaches, they are still relevant and should be discussed in the related work section. Including these works would help position FlexAC more clearly within the broader landscape of controllable and steerable LLM/MLLMs.**_\n\n**Responses:** We agree and will update the \"Related Work\" section to better position FlexAC relative to trainable steering methods. The discussion will include:\n- **Preference-Based Alignment:** Methods like Reinforcement Learning from Human Feedback (RLHF)[1] and Direct Preference Optimization (DPO)[2], which fine-tune the model on preference data.\n- **Targeted Fine-tuning:** Approaches like CogSteer[3], which fine-tunes lightweight adapter modules in specific middle layers to steer behavior.\n- **Auxiliary Module Training:** Frameworks that train a separate module to guide the LLM, such as SCAV[4] (trains a linear classifier to find a steering vector) or TSV[5] (trains a lightweight vector to be added during inference).\n\n[1] Training language models to follow instructions with human feedback (NeurIPS 2022)\n\n[2] Direct preference optimization: Your language model is secretly a reward model (NeurIPS 2023)\n\n[3] Cogsteer: Cognition-inspired selective layer intervention for efficiently steering large language models. (ACL Finding 2024)\n\n[4] Uncovering Safety Risks of Large Language Models through Concept Activation Vector (NeurIPS 2024)\n\n[5] Steer LLM Latents for Hallucination Detection (ICML 2025)"}]

Author rebuttal:

Meta-review: This paper presents FlexAC, a training-free framework for controlling associative reasoning in multimodal large language models by modulating the trade-off between faithfulness and creativity through middle-layer interventions. The reviewers recognized the work's significant contributions, particularly its novel unification of hallucination and creativity as manifestations of associative reasoning, the identification of middle layers as critical control points, and the introduction of the VDAT benchmark for measuring visual associative reasoning. While initial concerns were raised by the several reviewers, the authors provided comprehensive responses including extensive experiments on eleven diverse benchmarks, human evaluation studies, and rigorous sanity checks with masked image experiments. Given the paper's novel theoretical insights, strong empirical validation, practical significance for controllable AI systems, and thorough addressing of reviewer concerns, I recommend accepting this paper for NeurIPS 2025.
Decision: Accept (poster)