MAI: A Multi-turn Aggregation-Iteration Model for Composed Image Retrieval
Authors: Yanzhe Chen
Zhiwen Yang
Jinglin Xu
Yuxin Peng
Venue: ICLR 2025 Poster (2025)
Keywords: Multi-modal
Multi-turn retrieval
Retrospective-based
Composed image retrieval
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Multi-Turn Composed Image Retrieval (MTCIR) addresses a real-world scenario where users iteratively refine retrieval results by providing additional information until a target meeting all their requirements is found. Existing methods primarily achieve MTCIR through a "multiple single-turn" paradigm, wherein methods incorrectly converge on shortcuts that only utilize the most recent turn's image, ignoring attributes from historical turns. Consequently, retrieval failures occur when modification requests involve historical information. We argue that explicitly incorporating historical information into the modified text is crucial to addressing this issue. To this end, we build a new retrospective-based MTCIR dataset, **FashionMT**, wherein modification demands are highly associated with historical turns. We also propose a Multi-turn Aggregation-Iteration (**MAI**) model, emphasizing efficient aggregation of multimodal semantics and optimization of information propagation in multi-turn retrieval. Specifically, we propose a new Two-stage Semantic Aggregation (TSA) paradigm coupled with a Cyclic Combination Loss (CCL), achieving improved semantic consistency and modality alignment by progressively interacting the reference image with its caption and the modified text. In addition, we design a Multi-turn Iterative Optimization (MIO) mechanism that dynamically selects representative tokens and reduces redundancy during multi-turn iterations. Extensive experiments demonstrate that the proposed MAI model achieves substantial improvements over state-of-the-art methods.

Published as a conference paper at ICLR 2025
MAI: A MULTI-TURN AGGREGATION-ITERATION MODEL FOR COMPOSED IMAGE RETRIEVAL
Yanzhe Chen', Zhiwen Yang', Jinglin Xu", Yuxin Peng
WWanxuuan Institute of Computer Technology, Peking University
School of Intelligence Science and Technology, University of Science and Technology Beijing chenyanzhe@stu. pku. edu. en, yangzhiwen@pkuu edu. cn, iujinglinnove@@gee com, pengyuxin@pku eew.e//pkkkuuuuuucomooomoo edu.cn
ABSTRACT
Multi-Turn Composed Image Retrieval (MTCIR) addresses a real-world scenario where users iteratively refine retrieval results by providing additional informa- tion until target meeting all their requirements is found. Existing methods primarily achieve MTCIR through a "multiple single-turn" paradigm, wherein methods incorrectly converge on shortcuts that only utilize the most recent turn's ' image, ignoring attributes from historical turns. Consequently, retrieval fail- ures occur when modification requests involve historical information. We argue that explicitly incorporating historical information into the modified text cru- cial to addressing this issue. To this end, we build new retrospective-bat based MTCIR dataset, FashionMT, wherein modification demands are highly associ- ated with historical turns. We also propose Multi-turn Aggregation-Iteran Iteration (MAD) model, emphasizing efficient aggregation multimodal semantics and optimization of information propagation in multi-turn retrieval. Specifically, we propose a new Two-stage Semantic Aggregation (TSA) paradigm coupled with a Cyclic Combination Loss (CCL). achieving improved semantic consis- tency and modality alignment by progressively interacting the reference image with its caption and the modified text. In addition, we design a Multi-turn It- erative Optimization (MIO) mechanism that dynamically selects representative tokens and reduces redundancy during multi-turn iterations. Extensive experi- ments demonstrate that the proposed MAI model achieves substantial improve- ments over of-te-of-the-art methods. The dataset and source code are available at bttps://qithun www.://oocccCCCO/. RAS-0--CST-RR MMMD-IIIII-MMMPAPMAAMM MAI ICLR2025
INTRODUCTION
Image retrieval remains a longstanding task computer vision et al. (2023) Levy et al. (2024a). gaining continuous attention in practical applications such e-commerce in recent years Jin et al.
22023; Park et al. (2019). However, relying solely on images may fall short of practical needs, as users often modify these images better match their requirements Chen et al. (2020); Guo et al. (018).. In response, Composed Image Retrieval (CIR) has been introduced to locate target images by combining reference images and modified text Wenn et al. (2023); Shoib et al. (2023). Due to
the interactive nature of retrieval scenario Xu & Sundar (2014)) Adhikari et al. (2018), multi-turn systems can leverage more user feedback. fulfilling user needs better than single-turn systems Ag- nolucci et al. (2023): Chen et al.) (2023a). Therefore, Multi-turn Composed Image Retrieval (MT- CIR), which aims to retrieve the most suitable target image by allowing users to iteratively select images and provide modification feedback, as illustrated in Figure has garnered increasing atten- Liu al. (002b)).
tion recent years Guo et al. (2018); Yuan & Lam 2021):
Due to the lack of dedicated datasets for the MTCIR task Pal et al. (2023). existing methods typically construct multi-turn datasets by concatenating single-turn CIR datasets Wu al. (2021): Guo et al.) 20018), using the target image from the historical turn the reference image for the next turn. However, datasets constructed in this manner exhibit the following limitations: (i) Lack of historical
"Corresponding author.

---

Published as a conference paper at ICLR 2025
I'want buy dress.
How about this one?
about this dress?
What about this dress?
Tur
Turn
Tum
Turn
Retrospective phenomenon
don't like V-neck design longer sleeves.
preferr one- and lighter color.
keep two color style and like keep shoulder design.
Thanks FashionMel, dress exactly buy.
prefer
Figure 1: The definition of Multi-Turn Composed Image Retrieval (MTCIR). The retrospective phenomenon is common in the MTCIR task, wherein a user's new turn modification request often involves the attributes of images from historical turns.
context. Modified text in existing MTCIR datasets lacks image information from historical turns, resembling "multiple single-turn retrievals and deviating from real real-world scenarios. (ii) Small data scale. Existing single-tur targlll-tnnnae datasets face challenges due to their limited scale Saito et al. (2023) Feng et al. (2024); Zhao et al. (2024b). Moreover, this concatenation method further diminishes the size of multi-turn datasets, lagging behind current trends Chen et al. (202bb); Baldrati et al. (2023).
The deficiencies outlined in existing MTCIR datasets have hindered the development of methods in this domain. Existing methods typically employ "multiple single-turn" paradigm for multi- turn retrieval. However, this paradigm causes methods to incorrectly converge on shortcuts that only utilize the most recent turn's 's image, neglecting attributes from previous turns. Consequently, retrieval failures arise when modification requests involve attributes or modifications from previous images. Additionally, existing methods lack designs leverage inherent multimodal information in images Chen et al. 2023b): Li et al. (2024)), and to store multi-turn information effectively.
To address these issues and align with existing MTCIR datasets, we construct new dataset, Fash- ionMT, tailored for e-commerce scenarios characterized by typical multi-turn interactions. Fash- ionMT has the following characteristics: (i) Retrospective-based. It simulates real-world MTCIR scenarios, where the modified text in each new turn may involve information from historical refer- ence images, such as preserving certain attributes. This necessitates retrieval algorithms to utilize multi-turn historical information retrospectively. (ii) Massive and diverse. FashionMT contains 14 times more fashion images and 30 times more categories than MT FashionIQ Yuan & Lam (2021) Our Modification Generation Framework generates multi-turn transactions nearly 27 times larger than MT FashionIQ, offering rich multimodal data, including images, text, attributes, etc.
We further propose multi-turn key information-awan approach, the Multi-turn Aggregation- Iteration (MAI) model, which focuses on two challenges in MTCIR: (i) multimodal semantics aggregation and (ii) multi-turn information optimization. Specifically, MAI introduces new Two-stage Semantic Aggregation (TSA) paradigm coupled with a Cyclic Combination Loss (CCL). TSA introduces captions as transition, progressively aggregating the image with its caption and then with the modified text. The CCL's cyclic structure further enhances semantic consistency and modality alignment. We also provide theoretical insights into the rationale behind introducing cap- tions for two-stage fusion. Furthermore, we design parameter-free Multi-turn Iterative Optimiza- tion (MIO) mechanism that dynamically selects representative tokens with high semantic diversity, effectively reducing the storage space for historical information tokens.
Our contributions are summarized as follows:
We build the first dataset specifically designed for multi-turn composed image retrieval, named FashionMT, characterized by its retrospective-based nature and massive diversity.
We propose the Multi-turn engregattionIIeratioon (MAI) model, focusing efficient aggregation and iterative optimization of multimodal semantics in multi-turn composed image retrieval.
We provide theoretical insights that our modality fusion approach effectively bridges the modality and semantic gaps, which informs the design of our loss function.
Extensive experiments demonstrate that our proposed MAI model obtains substantial improve- ments and achieves state-of-the-art performance.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
Single-turn Composed Image Retrieval. In existing works on composed image retrieval, the focus has mainly been on single-turn retrieval Wen et al. (2023); Chen et al. (2024c), which can be categorized based on the amount of training data into fully trained on all data Goenka et al. (2022); Levy et al. (2024b) and zero-shot Karthik et al.] (2024); Gu et al. (2023): :: Chen & Lai (2023) or few-shot Wu et al. (202)) settings. Currently, composed image retrieval methods can be broadly categorized into two paradigms |Bii et al. (2024): late fusion Chen et al.] (2024b); /Zhang et al. al. (2023); Liu al. (2024b); [Suo et al. 2024.. In the first paradigm type, [paldattt et al. (2022) propose a simple yet effective fusion model, Combiner, combine features extracted by the CLIP Radford al. (2021) model. In the second paradigm type,Saito et al. (2023) propose an LLAVA-like Liu et et. (2024a) method to convert visual features into tokens for a text encoder. Bai al. (2024) propose a method similar to the BLIP-2 Li et al. (2233) to learn sentence-level prompts, achieving state-of-the-an results. However, the above methods are limited to single-turn retrieval scenarios and are challenging to apply directly to the more user-demand-oriented MTCIR tasks.
(2024) pseudo-word embedding methods Baldrati
MTCIR Methods. Several recent methods have emerged in the fusion of visual and textual in- puts across multiple exchanges of information Zhu et al. ) Li et al. (2024b): Hu et al. (2024). Due to the inherent multi-turn nature of dialogues Zhang et al. (2022); ;) Yu et al. (2019), com- mon application scenario is multi-turn dialogue systems Colkepli et al. (2024); ZZeng et al. 2022). In the prevalent retrieval tasks of the fashion domain, multi-turn retrieval has emerged a more comprehensive approach compared to single-turn retrieval, offering enhanced user interaction and feedback to better cater user needs Zhang al. (2019); Agnolucci et al. (2023). There have been several groundbreaking studies in multi-turn composed image retrieval in recent years. Guo et al.) (2018) propose modeling images and text using CNN networks. capturing sequential information with RNNs, and employing reinforcement learning for constraint, Yuan & Lam (2021) construct the first multi-turn composed retrieval dataset based on the single-turn retrieval dataset Fashion Wu et al. (2021). Pal et al. (2023) introduce a memory network to retain historical retrieval information and further develop a multi-turn retrieval dataset based on the single-turn dataset Shoes Guo et al. 2018)) However, the above methods fail to leverage the multimodal content naturally present in fashion images, such as captions and titles. Additionally, these methods do not consider optimizing the storage overhead of multi-turn representations.
Fashion Datasets. In the past few years, a large number of datasets have been proposed for trieval Corbiere et al. (2017): Ge et al. (2019); Rostamzadeh et al. (2018): Han el al. (2017); Zhan al. (2021). Due the inherent inclusion of a vast amount of data and extensive user interactions in the e-commerce domain, existing fashion datasets exhibit large scale. The Product IM Zhan al. (202T) contains 1,182,083 cosmetic samples. The M5Product Dong et al. (2022) encompasses 6,131,064 samples with 5 modalities. A massive amount of data also contributes to the model acquir- ing capabilities closer to practical usage Chen et al. (2023b). the composed image retrieval task,
FashionlQ Wu et al." (202T) and Shoes Guo et al. 2018) represent pioneering works, being more user-friendly compared to direct image retrieval. However, the MTCIR task still lacks a dedicated custom dataset. Constructing modifications by concatenating single-turn datasets fails to capture the historical context crucial for multi-tune tulti--uurnan scenarios. Our proposed FashionMT offers essential and timely data support to advance this task.
3 THE FASHIONMT DATASET
3.1 DATA COLLECTIONS AND CONSTRUCTION
Our data primarily originates from two sources: (i Gathering images and associated text from existing single-turn composed image retrieval datasets Wu et al. (2021): Guo al. (018); Han et al. (017). (ii) Crawling images and related text from multiple e-commerce platforms. We clean the scraped images, including removing damaged, unclear, and non non-product images.
Inspired by the manual annotation process of modified text Wu et al. (202T); ) Liu et al. (02T)) 1 we propose Modification Generation Framework (MGF) to automate the construction of dataset
3

---

Published as a conference paper at ICLR 2025
Sand
Main Product Categories (12)
Ran DOUrS
Dress
Shirts
Skürts
O Jacket
Outdour
O Ornament
Sonttal
Consutie
Polo
Outdoor
Mibie
Underwest
Ltai
Tops
Shows
Loafe
Brecedle
Pants
Tops
Shoes
Winter
O Rags
Underwear
Keee lengin
148 leiget
Skirts
teare
Meng
Balt
Ornamnnt
Pants
Shorts
Product Subcategories (Top 56)
Ruated
Jacket
Wide
Lang: 46044
@ Mid length 25520
@ Blazer: 236.0
Rain 137903
Dress
Winter
Coopree
legginge
Croust: 25178
25796
Most 22366
leegthr
Gasal
Waee 22876
Loster 17500
Galans Formal
Sinke
Swenter
Goann: 331341
Formal: 25026
@ Trunk: 38500
Badipadk 13540
Derim: 13528
Fadled. 17488
Bracest 27580
Stanttrr 41120
Cencal
Corte
Doort
end king
Cas
Srad
Robe:
Canisole 15400
Bag: 13120
@ Jeans. 126240
Shorts
Scart 30050
Stick
Soore
Oxload
Mee
haok
Cardiart 17840
Chernis 20260
Cass 46750
23758
Wiie eg 26662
Logaings 24216
Tunic 27500
Tank: 18420
Sungracue 1540
Be exaso
Croppect 77988
Veet 18400
Hus 12250
Ring: 34203
Striggte 22020
Parka 17560
Swewandurt 40000
Oufond 24790
Mlousee 73014
Figure 2: Top categories and distribution in our proposed FashionMT dataset. We have listed 12 main product categories and the top 56 product subcategories to provide a clearer presentation.
Table 1:
Comparison with other MTCIR datasets. MT stands for Multi-turn. In a transaction, there are multiple turss. Length denotes the modified text's average length. Categories denotes the number of finest subcategories, while Product type lists the typical product categories.
Datasets
# Images
# Transactions
# Turns
# Categories
Length
Product type
MT F
74,381
11,505
26,506
3
10.7
shirt, top-tee, dress
Yuan & Lam
MI Shoes
15,661
4,097
11,346
5.2
hoots, sneakers, clogs, etc. shirt, top-tee, dress, shoes, coat, pants, bag, ornament, etc.
Pal et al.
FT (ours))
1,067,688
247,911
743,733
95
by capturing the distinctions between reference and target image pairs. The framework consists of the following steps: (i) Image Selection: Selecting N ++1 images from product subcategory for N turns in transaction. (ii) Caption Generation: Generating captions for these images using an image captioning model. (iii Base Modification Generation: Employing large language model (LLM) to describe the differences between image captions from adjacent turns. (iv) Retrospective Modi- fication Generation: Determining the specific turns requiring retrospective analysis and generating corresponding modified text based on the intersection of attributes between the most recent image and images from previous turns.
Specifically, we generate the captions using the prompt: "Question: Describe the product. Answer:". For generating base modified text, the prompt is: "The reference depicts {REF), and the target depicts [TAR]. Describe the modifications to transform the reference into the target", where REF and TAR represent the captions of the reference and target images within single turn, respectively.
To better align with retrospective needs in real-world scenarios, we have established two scenarios for generating retrospective- based modified text: rollback and combination. In the rollback setting, similar to base transaction generation, modifications are generated between specified reference and the target by rolling back. An example under this setup "Compared to the most recent turn, I still prefer the item from the second furn. Building that, I like.... In the combination setting, users combine attributes from multiple images in historical turns to formulate modification requests. An example under this setup would be: "I like ... from the first turn, and ... from the second turn". In this setup, the modified text consists of two parts: the initial segment encompasses common at- tributes earmarked for retention, prefaced by the prompt "Keep the (Attr) in the {ID} turn" where Attr represents common properties like color, logo, pattern, etc., and ID signifies the turns sharing commonalities with the target. Meanwhile, the subsequent segment delineates additional modifica-
4

---

Published as a conference paper at ICLR 2025
tion requisites, prefaced by the prompt "the reference images depict REFs, the target depicts TAR. Describe the distinctiveness of the target:".
3.2 DATASET STATISTICS
The data distribution of FashionMT is illustrated in Figure 2 Detailed information and a compari- with existing datasets, MT FashionIQ Yuan & Lam (222 and MT Shoes Pal et al. (2023), presented in [ FashionMT significantly surpasses existing datasets in both scale and richness, featuring 14 times more images than MT FashionIQ and nearly 10 times more categories than MT Shoes. By leveraging the Modification Generation Framework, FashionMT enables the efficient construction of high-quality transactions, resulting in a dataset that 27 times larger than MT Fash- ionIQ. Additionally, FashionMT provides more detailed modified text, with an average length twice that of MT FashionIQ. As dataset tailored specifically comprehensive and realistic data support. For more details on our proposed dataset, including setup explanations and quality control, please refer to Section 7.3
MTCIR task, FashionMT offers more
4 APPROACH
Madification Semantic Aggregation (MSA)
Cyclic Combination Loss (CCL)
Multi-tum Iterative Optimization (MIO)
Fusion Modality
Tum 2
S
Modifier
DIC DKC-WN
dann Buy ling
modality align
semantic align
Self Attention
Seif Attention
1
000 3
000
MIO
000000
MIO
Modality
Bimodal Semantic Aggregation (BSA)
TSA BSA MSA
Target
Image
Image
FFN
Visual patch embedding
Cross Attention
Cross Attention
Cross Attention
Caption embedding
Self Attention
Self Attention
Self Attention
Modified text embeddlng
000
000
000 odo o
00O
O
Learned token
Turn
Turn 2
Turn N (Trrgot)
Illustrations
Figure 3: The architecture the MAI model. For each turn, images, captions, and modified text are progressively aggregated through BSA and MSA, with MIO preserving core information across turns, and CCL constraining the training process. For simplicity, we illustrate two retrieval turns.
4.1 PROBLEM FORMULATION
In our task setup, we provide the previous N turns' multimodal data, which consists of predefined reference images with captions and modified text, aiming to retrieve the most suitable target image based on the final modified text. We represent the image patch embedding, caption embedding, and modified text embedding of the neth turn un. E V, Cn E C, and m. E M respectively. Furthermore, we distinguish among the reference image embedding, reference caption embedding. target image embedding, and target caption embedding vi c% of and d; respectively.
4.2 MULTI-TURN AGGREGATION OATREGGATIOON-IERATIOOO (MAI) MODEL
The architecture of MAI is depicted in Figure 3 We will introduce Bimodal Semantic Aggregation (BSA) and Modification Semantic Aggregation (MSA), which are part of the Two-stage Semantic Aggregation (TSA), along with the Multi-tur Iterative Optimization (MIO).
Bimodal Semantic Aggregation (BSA). In the n-th turn, first conduct lexical analysis the modified text to determine if there is a rollback operation. We established a template for auto- matically generating modified text with Rollback instructions to facilitate benchmark construction.
5

---

Published as a conference paper at ICLR 2025
The template includes phrases such as: "Compared to this one I prefer the {). and", "I would rather choose the {,, and", where { denotes rollback turn descriptions, such as "Turn 2: White off-shoulder lace short sleeve." Rollback operations are executed when modified text match the tem- plate. If so, the reference image is designated as the image from the specified rollback turn. If not, the default reference image for the n-th turn is adopted. We extract visual patch embeddings Un. of images using a frozen visual encoder. The effectiveness of the Q-Former architecture Li et al. (2023) in integrating vision-text embeddings has been validated in prior studies Bai al. (2024); Hu et al. (2024). Hence, our BSA transfers this framework to adapt to the MTCIR task. Through learned tokens t. BSA initially learns the bimodal semantics of images and their corresponding captions before interacting with modified text. This strategy employs captions
transition, en-
hancing modality relevance during interaction with modified text, as elaborated in Section 4.3 A fixed text encoder extracts caption embeddings Cn. interacting with learned tokens L, in BSA's attention layers. In the cross-attenticn layers, they engage with visual patch embeddings Un. After BSA, learned tokens aggregate multimodal semantics from reference images and captions, denoted as BSA, As the target side lacks modified text, this embedding is directly used for training loss constraints and inference distance measurement.
Multi-fury Iterative Optimization (MIO), Despite learned tokens being more space-efficient than visual embeddings Li al. (2023), storing these tokens for each turn still results in significant space consumption. Additionally, fashion images encompass various attributes such as color, style, size, etc Tian et al. (203); Chen et al. (003b): Han et al. (2023). Multi-turn retrieval often revolves around the same subcategory of products, resulting in similar attributes across the images involved multiple turns. Therefore, we propose a parameter-free mechanism to optimize and retain the key attributes throughout multi-turn interactions.
Specifically, concatenate to from the previous turn with f BSA from the current turn to obtain MT, Our objective is to preserve key semantic tokens while discarding redundant ones from the learned tokens 1 This process involves several steps. (i) Clustering, We apply an efficient density peaks elustering based on k-nearest neighbors (DPC-KNN) algorithm Du et al. (2016). The learned tokens tú are clustered into k groups and the clustering operation is formulated as follows:
k Σ
cluster(t,, k) arg min
1 CEC
(1)
C
where C' represents the clusters, C; represents the i-th cluster, and represents the centroid of the i-th cluster. (ii) Density Estimation. After clustering, the density of each cluster is estimated based on the distances between the tokens within the cluster and learned tokens with lower densities are filtered out to enhance clustering efficiency. The density estimation is calculated as follows:
density(v) = exp(
Σ ||e 1v--w ul
2
k
uENei(e)
where Nei(v) represents the neighboring tokens of v. (iii) Pruning. Tokens with low density are eliminated to ensure that only the most semantically significant tokens are retained. To achieve this, each token is assigned a score, computed as the product of its density and its distance to the nearest neighbor. The top & tokens with the highest scores are then selected as the optimized tokens.
score(v) (r) = density(v) x dist(u)
where dist(v) represents the distance of token v to its nearest neighbor. The final tokens, denoted as gr, are obtained by selecting the tokens with the top-k scores. Through the optimization process described above, MIO effectively preserves learned tokens carrying discriminative semantics while discarding tokens with relatively less semantic importance, thereby saving computational resources.
"n
Modification Semantic Aggregation (MSA). During the MSA stage, we engage the tokens t grMio which encapsulate reference semantics, with the modified text embedding m, By em- ploying a frozen text encoder to extract embeddings mn, we concatenate them with learned tokens ty
Kr,MIO before feeding them into the self-attentio layer. Subsequently, we employ linear and nor-
malization layer on the learned tokens to map them, producing a reference-side embedding t,, This embedding concurrently embodies multimodal semantics from the reference and modified text.

---

Published as a conference paper at ICLR 2025
It is important to note that in the combination setting, due the involvement of multiple historical images, BSA aggregates the bimodal embeddings by concatenating the learned tokens from previous turns with their corresponding image captions. Subsequently, these embeddings are semantically aggregated with the modified text in the MSA.
4.3 THFORETICAL INSIGHTS
In this section, we justify the rationale behind introducing captions for bimodal semantic aggregation and explain how our approach outperforms naive solution. In the MTCIR task, we hypothesize that the transition from the initial image to the final target image occurs by gradually introducing specific attributes related to the modified text in each turn, denoted as v on N (m,, 1). Ideally, the
visual increments o oo should correspond the textual modifications Mas i.e., on + m. 2
which can be supervised using the following similarity loss:
Csim
Σ B
1
B
+
mm
(4)
2
i=1
where for simplicity, given an embedding 4, |a| stands for the normalized form [|
However, the effectiveness of the aforementioned supervision constrained by: (i) the inherent modality gap between texts and images; (ii) the semantic disparity between the additional textual attributes and visual items. To mitigate these gaps, we propose leveraging image captions to (i) align with the modality of modified texts; (ii) match the semantics of visual items. We can adopt a naive method methodHHaang et al. (202)), replacing the visual embeddings in Eq, I with the corresponding caption text embeddings for modal constraints. This results in the following naive cross-modal loss by replacing images with their corresponding captions:
Luaive -Lsim
Σ 2
B
1 2
+
|m
lc +
lmm
|0
(5)
B
2
Although Lnavve aligns visual increments with textual modifications, bridging both modality and semantic gaps in the meantime, the separate optimization modality and semantic space may affect each other in the training process. To make further efforts, we propose that the reference and target images should undergo pre-fusion with caption text to achieve an intermediate state that closer in modality and semanties to the modified text. The paradigm of this process is represented as follows:
Cpre fisim + 1 E1-
1
(luz
|mm + (1o71 + mmm
B
4
2
Furthermore, we give theoretical justifications for the effectiveness of the proposed pre-fusion loss:
Proposition 1: Let O (pre)) and O(GErroo(Lnaai be the upper bound of generalization error of the above two losses. Then for any hypothesis fppe; Luaive H. V x C M + [0, and 1>6 8 > 0, it holds that:
(Grror)(Cpre) S O(Geroo((naaive))
(7)
with probability at least 1 - 8, given that the visual and textual encoders are reliable for generating positively correlated embeddings in the n-th turn and clustering embeddings with the same modality.
4.4 OPTIMIZATION AND INFERENCE
Training. Given the guiding role of modified text in retrieval Chen al. (2024a), we design the Cyclic Combination Loss (CCL) to align semantically similar fused modality with text modality, thereby preserving the critical semantics within the textual modality. Specifically, we employ batch-based classification loss commonly used in CIR and MTCIR tasks Pal et al. (2023): Wen et al. 20044; Chen al. (2024a), which is defined as:
B Σ 1=1
exp K(r E exp exprirr
CB(rg, r.) = B
log
(8)
7

---

Published as a conference paper at ICLR 2025
where B represents the batch size, the kernel x() is the inner product resulting in cosine similarity. rq denotes the reference- side representation, and Ft signifies the target-side representation.
Inspired by pree 's paradigm in Section 4.3 and 7.1 our loss function incorporates three constraints on embeddings after Bimodal Semantic Aggregation pre-fusion, along with an additional constraint the text modality. For the n-th turn, the cyclic constraints involve the following four sets of embeddings: learned tokens f", from MSA, encompassing semantics the reference image, caption, and modified text; learned tokens t from BSA, containing semantics of the target image and its caption; the modified text embedding m, and the caption text feature of the target image c2
In line with previous works in the MTCIR task, overall Cyclic Combination Loss Loct for N turns is composed of the cumulative losses from each turn:
N
LOOL
Σ 7=
C (2) + LB mn) + LB (2) + LB ,
encompasses key multimodal semantics from prior turs.. Upon interacting with the modified text through MSA, we derive the reference-side embed- ding fy.. Meanwhile, on the gallery side, the bimodal embedding of the image and its caption. t is computed. Retrieval matching ensues by evaluating the similarity between tyy and t
Inference. the conclusive N-th turn, MIO
5
EXPERIMENT
5.1 EXPERIMENTAL SETTING
Implementation Details. We adopt BLIP-2 Li al. (023) with the Flan-t5-xxl language model Chung et al. (024) for image captioning and Xwin 13B V0.2 Ni et al. (2024) the LLM. Optimization performed using Adam Loshchilov Hutter (2019) with a batch size of 16, an initial learning rate of le-5, and cosine annealing. Training runs for 50 epochs, while inference uses a batch size 2048. All model training and inference are conducted on 8 V100 GPUs. The number of learned tokens is fixed at 32, and 32 tokens are retained each turn through the MIO. Q-Former parameters are initialized with blip2 pretrain vitL, consistent with SPRC Bai et al. (002)).
Representative Methods. We select representative methods from five different categories for comprehensive performance comparison: (i) MTCIR methods including FashionNTM Pal et al. (023) and CFIR Yuan & Lam (202T) (ii) STFIR+NTM: DQU-CIR Wen al. (2024), single-turn methods CashionERN[Chen et al. (2024)). and SPRC Bai et al. (2024), integrated with the multi-turn method FashionNTM. (iii) ZS-CIR: Pic2word Saito al. (2023), Cow Tang et al. (2024), and Image2Sentence Du et al. (2024), also integrated with Fashion Additionally, since LLMS inherently support multi-turn interactions, we select several MILMS as stronger baselines for com- parison and fine-tune them FashionMT using their original training methods. The selected base- lines include: (iv) Retrieval-capable MLLMs: Fromage Koh et al. (2023), GILL et al. 20244) We use the [IM]] and [RET] tags provided by these methods for retrieval. (v) Interleaved MLLM:
MLLMS designed for interleaved multiple images and text, including MMICL Zhao et al. 2024a)
and Flamingo Alayrac et al. (2022)-95 -BB. For these methods, perform retrieval by encoding the target's s description text with the final-round LLM. All methods use ViT-L, Radford et al. 2021 as the visual backbone for fair comparison.
Evaluation Metrics. Consistent with existing multimodal retrieval tasks Pal et al. 2023 Wen al. (2024), we use the standard top-K recall metric evaluate models" performance, denoted as R@k. Specifically, we adopt R@1, @@,, R@ 10, R@20 and their mean as the evaluation metrics.
5.2 RESULTS
Quantitative Analysis. Experimental results on FashionMT are shown in Table Benefiting from the strong multimodal fusion capability of the BLIP-2 architecture, methods such SPRC Bai et al. (202)) demonstrate performance advantages. Building upon this, our TSA and CCL incorporate captions transition, leveraging their semantic alignment with references and consistency with
8

---

Published as a conference paper at ICLR 2025
modified text. Furthermore, the proposed MIO effectively retains key semantics across multiple turs.. Consequently, MAI significantly outperforms existing methods, achieving a remarkable 8.63 improvement in the Mcan metric over the SOTA method SPRC.
Table 2: Results on our proposed FashionMT dataset.
Combination
Rollback
Method
Mean
R@D
R@5
@@ 10
@@20
@@l
R@5
R@ 10
R@20
CFIR (SIGIR'21)
11.70
23.09
30.89
40.14
8.25
22.63
31.04
41.79
26.19
Pic2word (CVPR 23)
13.35
27.12
35.42
45.40
8.69
23.98
33.15
44.41
28.94
GILL (NeurIPS'2) 23)
19.54
38.17
47.63
56.14
9.12
24.56
33.62
42.77
33.95
Fromage (ICML'23)
19.45
39.12
49.00
59.65
10.12
26.54
34.97
45.61
35.56
FashionNTM (ICCV'23)
18.98
38.51
48.35
58.30
10.73
27.71
37.66
49.85
36.26
FashionERN (AAAI'24)
20.36
41.37
50.18
60.51
11.42
29.67
41.02
52.98
38.44
Flamingo (NeurIPS'22)
21.38
44.17
$5.16
63.09
11.55
28.18
37.81
48.76
38.76
DQU-CIR (SIGIR 224)
20.57
42.32
52.33
62.03
12.59
31.69
42.79
54.68
39.88
Context- (AAAI'24)
30.62
51.84
62.50
71.75
12.63
32.98
45.48
59.30
45.89
Image2Sentence (ICLR'24
32.44
53.71
65.16
74.52
15.79
36.87
50.17
64.56
49.15
MMICL (ICLR'24 24)
39.17
60.89
70.28
79.89
18.46
43.53
57.05
69.66
54.87
SPRC (ICLR'24)
39.28
62.42
72.11
80.23
23.31
49.79
62.
74.82
58.01
MAI (ours)
51.51
74.67
80.66
86.52
28.94
58.89
70.42
81.50
66.64
Qualitative Analyses. In Figure we compare MAI with two representative methods, Fashion- NTM and SPRC. MAI effectively handles fine-grained demands by leveraging TSA and CCL for efficient aggregation of image-caption semantics, making it responsive to domain-specific terms like "crepe fabric" and "vintage design." Furthermore, MAI addresses retrospective- based needs by utilizing the MIO component to retain multi-tur turn historical key information, enabling precise interpretation of vague expressions such as "strap design."
Initial query
GT target Initial
Tum
Turn
Tarn
Turn
Tumn
Tum
light blue and
a ai front top leopard-
crape fabric. kewo the shoulder vintage
is champugne
white faux
cream :colored lavo Initil's strap
Fashion
K
NTM
(Durs)
Rank
Figure 4: Qualitative results for the last turn in the FashionMT dataset. The top retrieval results of MAI compared with two representative methods are shown.
Table 3: Ablation study on different components of the MAI model. Mean-Com Combination and Mean- Rollback denote the mean recall under the combination and rollback settings.
Settings
Cean-Coomination
Mean
Base
58,69
41.49
50.04
Base + TSA
69,22
55.74
62.48
Base + MIO
64.17
47.03
55.60
Base + TSA + CCL
72.31
58.83
65.57
Base + TSA MIO
71.19
58.19
64.69
Base + TSA CCL + MIO (MAI)
73.34
59.94
66.64

---

Published as a conference paper at ICLR 2025
5.3 ABLATION STUDIES
Effects of Different Components. Our baseline method employs Q-Former from BLIP-2 et al. (2023) for reference and modified text semantic fusion and adopts the multi-turn information aggre- gation model from FashionNTM for task adaptation. We gradually add the TSA, CCL and MIO, comparing their performance both combination and rollback settings. Table demonstrates (eeonnsdeosstttatess the positive contribution of each component performance improvement in both settings.
Table 4; Effects of TSA and CCL. Mcan-C and Mean-S denote using caption adaptation and single-turn results.
Table 5: The comparison between MIO and other methods on memory cost and average re- trieval metrics. "N" represents turns.
Method
Mean-C
Mcan-S
Method
Memory Cost (MB)
Mean
FashionERN 2024a
44.47
50.29
None
0
50,04
Image2Sentence 2024
47.61
51.64
Concat
64
50,56
Fashion 2023
48.03
45.75
L.TM 2012
957
64x N
52.08
MMICL 2024a
58.67
47.28
GRU 2017
858 6
51.80
SPRC 2024
62.90
52.66
NTM 2023
1270 64x
53.19
TSA CCL
65.57
53.73
MIO
64
55.60
Table 6: Effects of each loss in CCL. For simplicity, we denote LB(E, y) as L(x, y).
Settings
w/ CCL (total)
w/o CCL
w/o C(
w/o C(t mm)
w/o L(m,, d.))
w/o L(c),
Recall
65.57
62.48 -3.09
63.54
64.39
64.51
64.90
A
-2,03
-1.18
-1.06
-0.67
Effects of TSA and CCL. We further conduct two sets of experiments, shown in Table with Mean-C and Mean-S. (i) Caption adaptation. We adapt several representative methods to two-stage fusion process, allowing the reference image to interact with both the caption text and the modified text. Specifically, FashionERN, Image2Sentence, and FashionNTM utilize the Combiner Baldrati et al. (2022) widely employed in this field, for interaction with caption embeddings. (ii) Single-tur turn retrieval. We evaluate performance using the first turn from FashionMT. The results in Table 4 indi- cate that the two-stage fusion significantly improves the performance of the methods. Additionally, the combination of TSA and CCI. effectively integrates the critical semantics from the modified text. Consequently, it achieves superior retrieval performance compared to existing methods in both settings, shown in Table 4 We also conduct ablation experiments for each loss in CCL. Since CCL computes losses based the outputs from TSA, requires TSA to be present, resulting settings. Results in ( that each loss contributes to performance gains.
For further ablation studies on performance in existing datasets, reducing modality gap. and rollback setting, please refer to Section 7.1
6 CONCLUSION AND DISCUSSION
In this paper, we have constructed the first dataset specifically designed for Multi-turn Composed Image Retrieval, named FashionMT. We also propose MAI model, multi-turn key information- aware approach that uses paired captions as a transition for better semantic consistency and modality alignment while adaptively filtering and preserving significant attributes to reduce spatial occupancy. We have conducted extensive experiments on FashionMT and observed that MAI achieves state-of- the-art performance, demonstrating its usefulness and effectiveness.
Limitations. As the first dedicated MTCIR dataset, we standardize the number of turns to 3 for method comparison. However, real-world scenarios may involve more diverse transactions and cover more general contexts beyond e-commerce, aligning with our ongoing development efforts. Furthermore, we aim upgrade our model with integrated dialogue and retrieval capabilities.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by the grants from the National Natural Science Foundation of China (61925201, 62132001, 62432001, 62373043) and Beijing Natural Science Foundation (L247006, 4252020).
REFERENCES
Bijaya Adhikari, Parikshit Sondhi, Wenke Zhang, Mohit Sharma, and B Aditya Prakash. Mining e-commerce query relations using customer interaction networks. Proceedings of the 2018 World Wide Web Conference, pp. 1805-1814, 2018
Lorenzo Agnolucci, Alberto Baldrati, Marco Bertini, and Alberto Del Bimbo. Zero-shot image retrieval with human feedback. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 9417-9419, 2023.
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in neural information processing systems, 35:23716- 23736, 2022.
Yang Bai, Xinxing Xu, Yong Liu, Salman Khan, Fahad Shahbaz Khan, Wangmeng Zuo, Rick Siow Mong Goh, and Chun-Mei Feng. Sentence-level prompts benefit composed image retrieval. In ICLR. OpenReview.net, 2024.
Alberto Baldrati, Marco Bertini, Tiberio Uricehio, and Alberto Del Bimbo. Effective conditioned and composed image retrieval combining clip-based features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 21466-21474, 2022.
Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, and Alberto Del Bimbo. Zero-shot composed image retrieval with textual inversion. Proceedings of the IEEE/CVE International Conference Computer Vision, pp. 15338-153477 2023.
Junyang Chen and Hanjiang Lai. Pretrain like you inference: Masked tuning improves zero-shot composed image retrieval. arXiv preprint arXiv:2311.011 1.07222, 2023.
Yanbei Chen, Shaogang Gong, and Loris Bazzani. Image search with text feedback by visiolinguistic attention learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3001-3011, 2020.
Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, and Ajay Divakaran. Dress: Instructing large vision-language models to align and interact with humans via natural language feedback. arXiv preprint arXiv:2311.10081 2023a.
Yanzhe Chen, Huasong Zhong, Xiangteng He, Yuxin Peng, and Lele Cheng. Real20m: A large-scale e-commerce dataset for cross-domain retrieval. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 4939-4948, 2023b.
Yanzhe Chen, Huasong Zhong, Xiangteng He, Yuxin Peng, Jiahuan Zhou, and Lele Cheng. Fash- ionern: Stshance-andrrffin network for composed fashion image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 128--236, 2024a.
Yanzhe Chen, Jiahuan Zhou, and Yuxin Peng. Spirit: Style-guided patch interaction for fashion im- age retrieval with text feedback. ACM Transactions on Multimedia Computing, Communications and Applications, 2024b.
Yiyang Chen, Zhedong Zheng, Wei Ji, Leigang Qu, and Tat-Seng Chua. Composed image retrieval with text feedback via multi-grained uncertainty regularization. In ICLR. OpenReview.net, 2024c.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetun lan- guage models. Journal Machine Learning Research, 25(70): 2024.
11

---

Published as a conference paper at ICLR 2025
Charles Corbiere, Hedi Ben- Younes, Alexandre Ramé, and Charles Ollion. Leveraging weakly annotated data for fashion image retrieval and label prediction. In Proceedings of the IEEE inter- national conference computer vision workshops, pp. 2274,, 2017.
Rahul Dey and Fathi M Salem. Gate-variants of gated recurrent unit (gru) neural networks. In 2017
IEEE 60th international midwest symposium on circuits and systems (MWSCAS), pp. 1597-1600.
IEEE, 2017.
Xiao Dong, Xunlin Zhan, Yangxin Wu, Yunchao Wei, Michael C Kampffineye, Xiaoyong Wei, Minlong Lu, Yaowei Wang, and Xiaodan Liang. MSproduct: Self-harmonized contrastive learn- ing for e-commercial multi-modal pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21252 21262, 2022.
Mingjing Du, Shifei Ding, and Hongje Jia. Study on density peaks clustering based on k-nearest neighbors and principal component analysis. Knowledge-Baad Based Systems, 99:135-144 145, 2016.
Yongchao Du, Min Wang, Wengang Zhou, Shuping Hui, and Houqiang Li. Image2sentence based asymmetrical zero-shot composed image retrieval. In ICLR. OpenReview.net, 2024.
Zhangchi Feng, Richong Zhang, and Zhijie Nie. Improving composed image retrieval via contrastive learning with scaling positives and negatives. arXiv preprint arXiv:2404.1 2024.
Yuying Ge, Ruimao Zhang, Xiaogang Wang, Xiaoou Tang, and Ping Luo. Deepfashion2: versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp-
5337-5345, 2019.
Sonam Goenka, Zhaoheng Zheng, Ayush Jaiswal, Rakesh Chada, Yue Wu, Varsha Hedau, and Pradeep Natarajan. Fashionvlp: Vision language transformer for fashion retrieval with feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
14105-14115, 2022.
Alex Graves and Alex Graves. Long short-term memory. Supervised sequence labelling with recur- rent neural networks, pp. 37-45, 2012.
Geonmo Gu, Sanghyuk Chun, Wonjae Kim, Yoohoon Kang, and Sangdoo Yun. Language-only efficient training of zero-shot composed image retrieval. arXiv preprint arXiv:2312.01990 2023.
Geonmo Gu, Sanghyuk Chun, Wonjae Kim, Yoohoon Kang, and Sangdoo Yun. Language-only training of zero-shot composed image retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13225-13234, 2024.
Xiaoxiao Guo, Hui Wu, Yu Cheng, Steven Rennie, Gerald Tesauro, and Rogerio Feris. Dialog- based interactive image retrieval. Advances in neural information processing systems, 31, 2018.
Xintong Han, Zuxuan Wu, Phoenix Huang, Xiao Zhang, Menglong Zhu, Yuan Li, Yang Zhao, and Larry S Davis. Automatic spatially-aware fashion concept discovery. In Proceedings of the IEEE international conference on computer vision, pp. 1463-1471, 2017.
Yunpeng Han, Lisai Zhang, Qingcai Chen, Zhijian Chen, Zhonghua Li, Jianxin Yang, and Zhao Cao. Fashionsap: Symbols and attributes prompt for fine-grained fashion vision-language pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 15028-15038, 2023.
Wenbo Hu, Yifan Xu, Li, Weiyue Li, Zeyuan Chen, and Zhuowen Tu. Bliva: simple multimodal Ilm for better handling of text-rich visual questions. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2256-2264, 2024.
Jingjia Huang, Yinan Li, Jiashi Feng, Xinglong Wu, Xiaoshuai Sun, and Rongrong Ji. Clover: Towards a unified vineo-language alignment and fusion model. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14856-1445 14866, 2023.
12

---

Published as a conference paper at ICLR 2025
Yang Jin, Yongzhi Li, Zehuan Yuan, and Yadong Mu. Learning instance-level representation for large-scale modti-modal pretraining in e-commerce. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp. 11060-11069, 2023.
Shyamgopal Karthik, Karsten Roth, Massimiliano Mancini, and Zeynep Akata. Uision-yyllanuen for training- compositional image retrieval. In ICLR. OpenReview.net, 2024.
Jing Yu Koh, Ruslan Salakhutdinov, and Daniel Fried. Grounding language models to images for multimodal inputs and outputs. International Conference on Machine Learning, pp. 17283- 17300. PMLR, 2023.
Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. Generating images with multimodal language models. Advances in Neural Information Processing Systems, 36, 2024.
Matan Levy, Rami Ben-Ari, Nir Darshan, and Dani Lischinski. Chatting makes perfect: Chat-based image retrieval. Advances in Neural Information Processing Systems, 36, 2024a.
Matan Levy, Rami Ben-Ari, Nir Darshan, and Dani Lischinski. Data roaming and quality assessment for composed image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2991-2999, 2024b.
Huayang Li, Siheng Li, Deng Cai, Longyue Wang. Lemao Liu, Taro Watanabe, Yujiu Yang, and Shuming Shi. Textbind: Multi-turn interleaved multimodal eestruction-oollwivii in the wild. In ACL (Findings). pp. 9053-9076 Association for Computational Linguistics, 2024a.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023.
Wenhao Li, Mengyuan Liu, Hong Liu, Pichao Wang, Jialun Cai, and Nicu Sebe. Hourglass tokenizer for efficient transformer-basee 3d human pose estimation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 604-613, 2024b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024a.
Zheyuan Liu, Cristian Kodriguez-Opaza, Damien Teney, and Stephen Gould. Image retrieval on real-life images with pre-trained vision-and-lane lan-uan models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2125 22142224,,222 2021.
Zheyuan Liu, Weixuan Sun, Yicong Hong. Damien Teney, and Stephen Gould. Bi-directional train- ing for composed image retrieval via text prompt learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5753-5762, 2024b.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR (Poster). Open- Review.net, 2019.
Mehryar Mohri. Foundations of machine learning, 2018.
Bolin Ni. JingCheng Hu, Yixuan Wei, Houwen Peng. Zheng Zhang, Gaofeng Meng, and Han Hu. Xwin-Im: Strong and scalable alignment practice for 1lms. arXiv preprint 203:555 2024.
Anwesan Pal, Sahil Wadhwa, Ayush Jaiswal, Xu Zhang, Yue Wu, Rakesh Chada, Pradeep Natara- jan, and Henrik Christensen. Fashionntm: Multi-turn fashion image retrieval via cascaded memory. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 11323-11334, 2023.
Sanghyuk Park, Minchul Shin, Sungho Ham, Seungkwon Choe, and Yoohoon Kang. Study on fash- ion image retrieval methods for efficient fashion visual search. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 0-0, 2019.
13

---

Published as a conference paper at ICLR 2025
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PMLR, 2021.
Negar Rostamzadeh, Seyedarian Hosseini, Thomas Boquet, Wojeiech Stokowiec, Ying Zhang, Christian Jauvin, and Chris Pal. Fashion-gen: The generative fashion dataset and challenge. arXiv preprint arXiv: (et::88066..1111 2018.
Aneeshan Sain, Ayan Kumar Bhunia, Pinaki Nath Chowdhury, Subhadeep Koley, Tao Xiang, and Yi-Zhe Song. Clip for all things zero-shot sketch-based image retrieval, fine-grained or not. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2775-2775, 2023.
Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen- Yu Lec, Kate Saenko, and Tomas Pfister. Pic2word: Mapping pictures to words for zero-shot composed image retrieval. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19305- 19314, 2023.
Teresa Schubert. People choose options that leave future options open. Nature Reviews Psychology, 2(3):155-135, 2023.
Amin Muhammad Shoib, Jabeen Summaira, Changbo Wang, and Abdul Jabbar. Methods and ad- vancement of content-based fashion image retrieval: A review. arXiv preprint 1rXiv:2303.17371, 2023.
Eunsuk Sung, Won Young Chung, and Daeho Lee. Factors that affect consumer trust in product quality: a focus on online reviews and shopping platforms. Humanities and Social Sciences Communications, 10(1):1-10, 2023.
Yucheng Suo, Fan Ma, Linchao Zhu, and Yang. Knowledge-enhanced dual-stream zero-shot composed image retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Yuanmin Tang, Jing Yu, Keke Gai, Jiamin Zhuang, Gang Xiong, Yue Hu, and Qi Wu. Context-i2w: Mapping images to context-dependent words for accurate zero-shot composed image retrieval. In 2024.
Proceedings of the AAAI Conference on Artificial Intelligence, pp.
Yuxin Tian, Shawn Newsam, and Kofi Boakye. Fashion image retrieval with text feedback by additive attention compositional learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp- 1011-1021, 2023.
Peter M Todd and Gerd Gigerenzer. Précis of simple heuristics that make us smart. Behavioral and brain sciences, 727777777 2000.
Noël Vouitsis, Zhaoyan Liu, Satya Krishna Gorti, Valentin Villecroze, Jesse C Cresswell, Guangwei Yu, Gabriel Loaiza-Ganem and Maksims Volkovs. Data-efficient multimodal fusion single gpu. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 27239-27251, 2024.
Haokun Wen, Xian Zhang, Xuemeng Song, Yinwei Wei, and Liqiang Nie. Target-guided composed image retrieval. In Proceedings of the 31st ACM International Conference on Multimedia, pp- 915-923, 2023.
Haokun Wen, Xuemeng Song, Xiaolin Chen, Yinwei Wei, Liqiang Nie, and Tat-Seng Chua. Simple but effective raw-data level multimodal fusion for composed image retrieval. In SIGIR, pp. 229- 239. ACM, 2024.
Hui Wu, Yupeng Gao, Xiaoxiao Guo, Ziad Al-Halah, Steven Rennie, Kristen Grauman, and Roge- rio Feris. Fashion iq: A new dataset towards retrieving images by natural language feedback. In Proceedings of the JEEE/CVF Conference on computer vision and pattern recognition, pp-
11 11307-11317, 2021.
14

---

Published as a conference paper at ICLR 2025
Junda Wu, Rui Wang. Handong Zhao, Ruiyi Zhang, Chaochao Lu, Shuai Li, and Ricardo Henao. Fhw-shot composition learning for image retrieval with prompt tuning. In Proceedings of the AAAI Conference Artificial Intelligence, pp. 4729-4737, 2023.
Qian Xu and S Shyam Sundar. Lights, camera, music, interaction! interactive persuasion in e- commerce. Communication Research, (1(2):282-308 2014.
Onesun Steve Yoo and Rakesh Sarin. Consumer choice and market outcomes under ambiguity in product quality. Marketing Science, 37(3):445-468, 2018.
Tong Yu, Yilin Shen, and Hongxia Jin. A visual dialog augmented interactive recommender system. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 165, 2019.
Yifei Yuan and Wai Lam. Conversational fashion image retrieval via multiturn natural language feedback. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development Information Retrieval, pp. 839-848, 2021.
Xunlin Zhan, Yangxin Wu, Xiao Dong, Yunchao Wei, Minlong Lu, Yichi Zhang, Hang Xu, and Xi- aodan Liang. ProductI m: Towards weakly supervised instance-level product retrieval via cross- modal pretraining. Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, 11782-11791, 2021.
Gangjian Zhang, Shikun Li, Shikui Wei, Shiming Ge, Na Cai, and Yao Zhao, Multimodal composi- tion example mining for composed query image retrieval. IEEE Transactions on Image Process- ing, 2024.
Ruiyi Zhang, Tong Yu, Yilin Shen, Hongxia Jin, and Changyou Chen. Text-based interactive rec- ommendation via constraint-augmentee reinforcement learning. Advances in neural information processing systems, 32, 2019.
Ruiyi Zhang, Tong Yu, Yilin Shen, and Hongxia Jin. Text-based interactive recommendation via offline reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 11694-11702. 2022.
Haozhe Zhao, Zefan Cai, Shuzheng Si, Xiaojian Ma, Kaikai An, Liang Chen, Zixuan Liu, Sheng Wang, Wenjuan Han, and Baobao Chang. MMICL: empowering vision-language model with multi-modal in-context learning. In ICLR. OpenReview.net, 2024a.
Xiangyu Zhao, Yuehan Zhang, Wenlong Zhang, and Xiao-Ming Wu. Unifashion: A uni- fied vision-language model for multimodal fashion retrieval and generation. arXiv preprint arXiv:2408.1 10000//1303. 11305, 2024b.
Yinhe Zheng, Guanyi Chen, Xin Liu, and Jian Sun. Mmchat: Multi-modal chat dataset on social media. In LREC, pp. 5778-5786. European Language Resources Association, 2022.
Hongyi Zhu, Jia- Hong Huang, Stevan Rudinac, and Evangelos Kanoulas. Enhancing interactive image retrieval with query rewriting using large language models and vision language models. In Proceedings of the 2024 International Conference Multimedia Retrieval, pp. 978-987, 2024.
Husein Zolkepli, Aisyah Razak, Kamarul Adha, and Ariff Nazhan. Mmmmodal-mutti-naagee multi-audio multi-turn multi-modal. arXiv preprint arXiv:2402. 1297, 2024.
7
APPENDIX
7.1 PROOF OF PROPOSITION
For simplicity, given two representations æ, y. we denote their similarity score as s(x y) = x We first compare the i-th similarity score terms in Tesin and Tapre respectively:
15

---

Published as a conference paper at ICLR 2025
1 +
+
|mm
S
|c/9 +
naive
2
2
+ (eff)) + s(cc])
(m
S
S
naive
[((])) (w ((C55 Ic s(u 1e <(/c" bo
The modality gaps between visual images and textual captions indicate that representations within the same modality are closer to each other, leading higher similarity scores, ie., s(lu Ix > a( y Ie and <(cct) 1 > a(1e% (o%)). Therefore, Spze
Snaive > O. Notice that:
B Σ
Lnaive =Lsim +
(1
S Suaive):
B
i=1
1 Σ 1=1
Lpr -Igiin
Spre);
where Daavve > Tpre- Furthermore, based on the Rademacher Complexity Theory Mohri (2018). the upper bound of generalization errors is estimated follows, with probability at least 1 0:
B
log 2B
E[Lnaine] <E(Lsim)
Σ
(1
St Stavee) 2Rz(G)
B
1=1
=O(GError([uuivee)
B
log 2B
Elprr SE(Lsim] +
Σ
(1
Sprc) + 22Rn(G) +
B
2m1
=O(GErrro(Lppee))
where (G) the Rademacher Complexity of the family of all possible loss functions, inde- pendent of our design for loss functions. From the above analysis, we have ((GEroo([ppe))) < O(Errror [naive)), which indicates the superiority of our pre loss Lpre the original naive cross- modal loss I-aive-
7.2 MORE ABLATION STUDIES
Validation on Existing Datasets. Despite the limitations of existing datasets, we further validate the effectiveness and generalization of our approach by adding performance comparisons on the real datasets MT FashionlQ Yuan & Lam (20TT) and MT Shoes et al. (2023). The results from Table 7 shows that our proposed approach. MAI, achieves the best performance in all settings due its fine-grained semantic capture and efficient modality alignment.
Table 7: Validation existing datasets MT FashionIQ and MT Shoes.
Train FashionMT Test on MT FashionIQ
Train FashionMT Test on MT Shoes
Train on FashionM'T
Method
Means
Test on FashionMT
FashionNTM Pal et al.
42.3
25.6
36.26
34.72
Image2Sentence al.
43.8
28.2
49.15
40.38
MMICL Zhao al.
46.9
29,8
54.87
43.86
SPRC Bai al.
48.0
58.01
44.74
MAI (ours)
50.6
33.8
66.64
50.35
16

---

Published as a conference paper at ICLR 2025
Modality Gap. We observe that recent works enhance feature-level representations to reduce modality gaps. We compare our approach with these methods, shown the Table g The re- sults show that noise addition methods are effective for large modality gaps, but approach reduces the gap through aligning modalities and semantics, the gains are limited.
Table 8: Comparison of various methods for reducing modality gaps.
Settings
Recall
Settings
Recall
Base
50.04
Base
50.04
Mixing Vouitsis al.
49.87
65.57
+ NN
52.17
+ ours + ())
65,23
+ 111,,1
52.09
ours 1)
65.30
+ N( 1) 1
Gt al.
54.88
+ ours + N(0, U--1,1
65.66
Ablation Study on Rollback. In the current setup for handling Rollback operations, the reference image for the current turn is replaced with the specified rollback image. We conduct comparative experiments under various Rollback settings: (1) Replace: the currett setting. (2) Ignore: no re-
placement is performed. (3) Random: selecting randomly from previous turns. (4) Blend: using
the PIL library's ' Image. blend() to merge two images into one. The results from Table / indicate that since the Rollback operation approximates redefining the current local optimal point, the setting Replace achieves the best performance.
Table 9: Ablation study on Rollback setting
Settings
Replace
Ignore
Random
Blend
Recall
59.94
49.59
53.62
57.80
7.3 FURTHER CLARIFICATION ON THE FASHIOOMT DATASET
We further clarify the setting, utility, quality control, and benefits of our FashionMT dataset below.
Explanation of Modified Text in Multi-turn. Our current approach for constructing modified text is based on two main reasons:
User Target Ambiguity. Humans often make decisions heuristically, so selecting while browsing aligns with human intuition Todd Gigerenzer (2000); [Schubert (2023). In the e-commerce domain, our analysis of multi-turn interaction data from well-known platform shows that users frequently experience "target ambiguity" during online shopping. Initially, users are unsure of the exact target and its details, and they need to select and refine attributes throughout the multi- turn interaction process. This behavior also supported by psychological studies Yoo & Sarin (2018): Sung al. (2023). To simulate this, mirror real-world scenarios.
combination and rollback settings better
Benchmark for Multi-turn. Initially, explored constructing the dataset by describing the difference hetween the current and target images. However, this "clear goal" setting resulted in models achieving precise retrieval within 1-2 turs.. This results in the multi-turn retrieval task degrading into single-turn retrieval task, thus failing to serve as benchmark requiring algorithms to integrate information from multiple historical interactions.
Utility. Although the FashionMT is synthetically constructed, we conduct an in-depth analysis of user behaviors during multi-turn purchases on famous e-commerce platform. We categorize these behaviors into two representative scenarios: "combination" and "rollback", aiming to replicate the real-world process where refine their choices through iterative comparisons. Compared to existing datasets that concatenate single-turn retrieval data, FashionMT more accurately reflects real-world scenarios.
17

---

Published as a conference paper at ICLR 2025
Table 10: Quality assessments among multi-turn datasets.
Datasets
Acc
HA
Gra
Con
Cov
Mean
MT F Yuan & Lam MT Shoes Pal et
93.3
43.1
67.2
86.4
70.2
72.0
95.1
65.6
76.3
91.3
80.9
81.8
MAI (ours)
96.2
98.7
91.3
90.5
93.2
94.0
Combination Setting
Rollback Setting
query
target
Initial query
target
Tum
Turn
Turn
Turn
Tumn
Turn
light and has subile ruffled shoulder design
fabric, Siitia's shoulder style & Secona's vintage design
brown peep- wedking shoe
has higher heels and more elegant style
this prefer the Second one, with embellished strap on the instep
with leopard- spot print style
Tum
is pleated
stripes,
comparees
colorful fashvonable
leaping color Second's
of parterrs
knees
added to hemn of the skirt
Turn
Tur
Turn
Tum
Turn
Tumn
5 sportier style and more inchion- design
denim moterial with fewer pattems
siimmer fit, keeping inital's light blue color, Second sleeve design, logo
the necklnee round, with
white with higher neckine more stylish look
co red this, prefer the Initial, and has warmer color
lace round neckline
Turn
is champagne strap higher
white foux cream -oolored keep initiats strap desion
more formal apacarance,
more vibrant as
compared this prefer the Initial, patterns
Examples of image sequences across multiple turns in the Combination and Rollback settings from proposed FashionM'T dataset.
Figure 5:
Quality Control. Additionally, to validate its utility, we conduct a quality assessment of Fash- ionMT and existing datasets, scoring them scale from to 5 in the following aspects:
Accuracy (Ace): Whether the modified text reflects the actual differences between images, with being very inaccurate and 5 being very accurate.
Historical Awareness (HA): Whether the modified text involves attributes from previous turns, with being not involved and 5 being fully involved.
Granularity (Gra): Whether the text provides enough detail cover subtle differences between images, with being lacking detail and 5 being very detailed.
Consistency (Con): Whether the differences between items in multi-turn retrieval are realistic, with being unrealistic and being very realistic.
Coverage (Cov): Whether the description covers all major differences between items, with being minimal coverage and being comprehensive coverage.
We provided explanations of the rating requirements to 20 e-commerce platform staff members and calculated the average scores independently. The scores were then converted into percentages, as shown below. Specifically, we randomly selected 20% of the data from each dataset for manual scoring without informing the evaluators of the data source. Quality assessments among multi-turn datasets are shown in Table [T
7.4 MORE VISUALIZATION
Dataset Examples. To facilitate better understanding of our newly proposed dataset, Fash- ionMT, present data samples under the Combination and Rollback settings in Figure In each
18

---

Published as a conference paper at ICLR 2025
Combination
Rollback
Rollback
Combination
Dress
Shirts a Painss
Dress
Shirs
Pants
Shoes
(a) Distributions the two settings and main categories
Image:
Avg Caption
Rollback
Transaction =100
Triplet
Combination
Aug
Length
Category
x:l
(b) Average text length
Dataset scale comparison
Figure 6: (1) Proportions of Combination and Rollback settings and main category distributions; (2) Average lengths of Modified text and captions; (3) Scale comparison of FashionMT with existing multi-turn datasets.
transaction, the Ground Truth is highlighted with green bounding box, and the retrospective-base modified text marked in dark blue.
Dataset Statistics Visualizations. Figure 6 provides visualizations of various statistics in the FashionMT dataset. (1) The proportions of Combination and Rollback settings and their respective main categories. (2) The average length of modified text, along with separate averages for Com- bination and Rollback settings, and the average caption length. (3) A scale comparison between FashionMT and existing multi-turn datasets, MT FashionIQ and MT Shoes.
Modality gap. As shown Figure we visualize the modality gap between the query and target sides in the final round using t-SNE on FashionMT and existing datasets. Leveraging captions bridge between visual and textual modalities, our proposed MAI approach effectively reduces the modality gap between the query and target sides.
Effeets of MIO. Due to the extensive storage of historical tokens in multi-turn retrieval, Table 5 presents the memory cost and mean retrieval performance of various methods. None denotes ran- domly initialized learned tokens tv. Concat is concatenating all tM [1, 1]. and LSTM and GRU respectively indicate using Graves & Graves (2012) and GRU Dey & Salem (017)) to aggregate all MIO
19

---

Published as a conference paper at ICLR 2025
sum Queryours
Tangeraaigi
Query-ariin
0-
20-
(a) Visualization on FashionMT
Tarpet ours
Drjptt ours
Target-ergg
Target-or
20-
-20
-20-
-60-
(b) Visualization on MT FashionlQ
(c) Visualization on MT Shoes
Figure 7: Visualization of modality gaps in FashionMT and existing datasets. Our approach signifi- cantly reduces the gap between the query and target sides.
20


Reviews:
[{"review_id": "cCKL5wzyaV", "reviewer": "Reviewer_TgqM", "summary": "Existing multi-turn composed image retrieval (MTCIR) methods adopt the “multiple single-turn” paradigm and neglect the historical correlation information in multi-turn interactions. To address this problem, this paper first builds a new retrospective-based MTCIR dataset, wherein modification demands are highly associated with historical turns. Then, the authors develop a new Multi-turn Aggregation-Iteration (MAI) model, which contains a two-stage semantic aggregation paradigm coupled with a cyclic combination loss. Besides, a multi-turn iterative optimization mechanism is designed to select representative tokens and reduce redundancy dynamically. Experimental results demonstrate the effectiveness of the proposed method.", "questions": "please try to address the weaknesses.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1)\tThis paper focuses on an interesting multi-turn composed image retrieval task, and points out the critical “multiple single-turn” problem in the existing research field. This research is very valuable and provides significant insights for further research.\n2)\tThe authors have constructed a new dataset for the multi-turn composed image retrieval task, which is more similar to real-world scenarios, and is more massive and diverse. This is an important contribution if the dataset is made publicly available.\n3)\tA multi-turn key information-aware approach, named the Multi-turn Aggregation model, is proposed to achieve multimodal semantics aggregation and multi-turn information optimization. The proposed method is reasonable and the parameter-free multi-turn iterative optimization (MIO) mechanism is interesting.\n4)\tThe effectiveness of the proposed method has been demonstrated by extensive experimental results.\n5)\tThe writing of this paper is good and easy to follow.", "weaknesses": "1)\tWill the collected dataset be made publicly available? Some key characteristics of the dataset are still not clear, such as the average and variance of turn number for each query, the average length for modification text, etc. The dataset will be more comprehensible as more detailed statistics are provided.\n2)\tIt seems MIO is executed for each input, and it contains a DPC-kNN for clustering and density computation. The complexity of this module should be analyzed.\n3)\tIn Table 6, what does the w/o CCL mean? It seems w/o CCL has outperformed most of the compared methods. Are all the methods evaluated on the same backbone? If they are not, the comparison results may not be fair.\n4)\tExcept for the proposed dataset, the proposed method should be also evaluated on existing multi-turn composed image retrieval datasets to further demonstrate the effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2VijqPzmrz", "reviewer": "Reviewer_WPSv", "summary": "The paper proposes a new dataset, FashionMT, specifically designed for multi-turn composed image retrieval (MTCIR) tasks. FashionMT is characterized by its retrospective-based nature, where the modified text in each new turn may involve information from historical reference images, and its massive diversity, containing 14 times more fashion images and 30 times more categories than previous MTCIR datasets. The authors also introduce a new Multi-turn Aggregation-Iteration (MAI) model that focuses on efficient aggregation and iterative optimization of multimodal semantics in MTCIR. The MAI model includes a Two-stage Semantic Aggregation (TSA) paradigm and a Cyclic Combination Loss (CCL) to enhance semantic consistency and modality alignment, as well as a Multi-turn Iterative Optimization (MIO) mechanism to dynamically select representative tokens and reduce redundancy during multi-turn iterations.", "questions": "refer to the weaknesses.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "a.\tThe paper proposes a new dataset, FashionMT, for multi-turn composed image retrieval (MTCIR) tasks\nb.\tFashionMT is characterized by its retrospective-based nature and massive diversity, containing 14 times more fashion images and 30 times more categories than previous MTCIR datasets\nc.\tThe authors introduce a new Multi-turn Aggregation-Iteration (MAI) model that focuses on efficient aggregation and iterative optimization of multimodal semantics in MTCIR\nd.\tThe MAI model includes a Two-stage Semantic Aggregation (TSA) paradigm and a Cyclic Combination Loss (CCL) to enhance semantic consistency and modality alignment\ne.\tThe MAI model also includes a Multi-turn Iterative Optimization (MIO) mechanism to dynamically select representative tokens and reduce redundancy during multi-turn iterations", "weaknesses": "a.\tHow to avoid false negative samples during FashionMT dataset construction? The existing single-turn dataset already has the presence of false negative samples, will it be more obvious with multiple rounds?.\nb.\tFor the ZS-CIR method, how was it trained and tested on FashionMT? They are designed for zero-shot, is the adaptation process also zero-shot paradigm?\nc.\tHow does MAI perform for the single-turn dataset? The authors should add the results of MAI for datasets such as FashionIQ, CIRR, etc. to confirm the superiority of MAI even with turn=1.\nd.\tThe clustering used in MIO is affected by the number of cluster centers, denoted as k, and the authors do not mention in the implementation details the value of k, and the number of iterations used when performing k-means, which affects the accuracy of the clustering. And does clustering increase the time overhead of the whole model? Authors need to increase the comparison of model training and inference time.\ne.\tThe Q-former used in MAI is initialized by BLIP-2 with the Flan-t5-xxl language model, while the version of BLIP-2 used in SPRC is blip2-pretrain, right? If MAI uses the same version of BLIP-2 as SPRC to initialize the Q-former parameters, how does the result behave? The authors need to add this experiment to further demonstrate the superiority of MAI?\nf.Typos and minors. There are non-standard punctuation marks, such as Line 431, P8, \"thanks to TSA\", etc. It is recommended to check the whole paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WMDd7To9a2", "reviewer": "Reviewer_JTgq", "summary": "This paper introduces MAI (Multi-turn Aggregation-Iteration), a model for multi-turn composed image retrieval (MTCIR), along with FashionMT, a new large-scale dataset specifically designed for MTCIR. The key innovation is a two-stage semantic aggregation approach that uses image captions as a bridge between visual and textual modalities, plus a memory-efficient mechanism for retaining historical information across multiple turns.", "questions": "1. Statistics for Modification text types: is possible to supply a category of the modified text type, so that the performance can be compared according to different categories? such as Rollback operations and combination operations.\n2. Will the dataset be open-source? \n3. How does the model perform with varying numbers of turns beyond the fixed 4-turn setup?\n4. Could the approach be extended to other domains beyond fashion?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Creation of a large-scale, diverse dataset (FashionMT) that better reflects real-world scenarios\n2. Memory-efficient design through the Multi-turn Iterative Optimization mechanism", "weaknesses": "1. Limited evaluation of existing datasets (mostly focused on their new dataset), how is the model's performance on existing benchmark like CIRR, FashionIQ and CIRCO. \n2. The fixed number of turns (4) in the dataset may not reflect varying real-world scenarios, does the author have plan to extend the dataset with a different number of turns? \n3. More visualization of the dataset is expected: I'd like to know (1)sample image sequences across multiple turns: namely the quality of the dataset, the relation of images in different turns, and whether the modified text can describe the relationship between images. (2) visualizations of the distribution of different types of modifications: such as Rollback operations and combination operations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pFMXlkHExN", "reviewer": "Reviewer_GvMC", "summary": "The paper introduces FashionMT and the MAI model. While the method shows strong results, its complexity and potential scalability challenges could limit its immediate applicability in more generalized settings. Further testing on diverse datasets and longer interaction sequences, along with analysis of the model's efficiency, would enhance the practical relevance of this work.", "questions": "- Can the MAI model be adapted to other MTCIR applications outside fashion, such as general e-commerce, furniture, or other products?\n\n- It would be beneficial to know how this impacts performance when dealing with significantly larger datasets. Is the trade-off between memory efficiency and retrieval performance consistent across different dataset sizes?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- FashionMT is a significant contribution. The dataset provides richer and more realistic interactions with over 1 million images and 95 categories. This dataset fills a critical gap in the field where existing datasets fail to incorporate historical context across multiple turns.\nThe model’s approach to maintaining and optimizing key tokens during multiple retrieval iterations is highly relevant to real-world e-commerce scenarios, where users iteratively refine their search queries. The MAI method effectively addresses the shortcomings of the “multiple single-turn” paradigm that fails to leverage historical turn information.\n- Extensive experiments show that MAI achieves significant improvements over existing methods in both the combination and rollback settings.", "weaknesses": "- While the model is innovative, it introduces considerable complexity with multiple components (TSA, CCL,). The addition of multiple layers, clustering mechanisms, and token filtering might make the model difficult to implement and optimize in real-world settings where computational efficiency is key.\n- While the results on FashionMT are strong, the paper does not provide comparisons on non-fashion datasets. The model's application to a more diverse range of image retrieval tasks, such as general object retrieval or scene retrieval, would provide a stronger claim to its versatility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)