QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval
Authors: Jaehyun Kwak
Ramahdani Muhammad Izaaz Inhar
Se-Young Yun
Sung-Ju Lee
Venue: ICML 2025 poster (2025)
Keywords: Composed Image Retrieval
Hard Negative Sampling
Primary area: general_machine_learning->everything_else
TL;DR: We propose QuRe, CIR method to retrieve both target and relevant images with novel hard negative sampling strategy. We created Human-Preference FashionIQ dataset to evaluate CIR models CIR models on their alignment with human preferences.

Abstract: Composed Image Retrieval (CIR) retrieves relevant images based on a reference image and accompanying text describing desired modifications. However, existing CIR methods only focus on retrieving the target image and disregard the relevance of other images. This limitation arises because most methods employing contrastive learning-which treats the target image as positive and all other images in the batch as negatives-can inadvertently include false negatives. This may result in retrieving irrelevant images, reducing user satisfaction even when the target image is retrieved. To address this issue, we propose Query-Relevant Retrieval through Hard Negative Sampling (QuRe), which optimizes a reward model objective to reduce false negatives. Additionally, we introduce a hard negative sampling strategy that selects images positioned between two steep drops in relevance scores following the target image, to effectively filter false negatives. In order to evaluate CIR models on their alignment with human satisfaction, we create Human-Preference FashionIQ (HP-FashionIQ), a new dataset that explicitly captures user preferences beyond target retrieval. Extensive experiments demonstrate that QuRe achieves state-of-the-art performance on FashionIQ and CIRR datasets while exhibiting the strongest alignment with human preferences on the HP-FashionIQ dataset. The source code is available at https://github.com/jackwaky/QuRe.

QuRe: Query-Relevant Retrieval through
Hard Negative Sampling in Composed Image Retrieval
Jaehyun Kwak Ramahdani Muhammad Izaaz Inhar'
Se- Ye-Young Yun
Sung-Ju Lee
Abstract
"Blue t-ahirt with short sieeves"
Existing CIR
QuRe
Composed Image Retrieval (CIR) retrieves rel- evant images based on a reference image and accompanying text describing desired modifi- cations. However, existing CIR methods only focus on retrieving the target image and disre- gard the relevance of other images. This limi- tation arises because most methods employing contrastive learning-which treats the target im- age as positive and all other images the batch as negatives-can inadvertently include false neg- atives. This may result in retrieving irrelevant images, reducing user satisfaction even when the target image is retrieved. To address this issue, we propose Query-Relevant Retrieval through Hard Negative Sampling (QURE), which optimizes a reward model objective to reduce false negatives. Additionally, we introduce hard negative sam- pling strategy that selects images positioned be- tween two steep drops in relevance scores fol- lowing the target image, to effectively filter false negatives. In order to evaluate CIR models on their alignment with human satisfaction, we create Hluman-Preferee FashionI Fashionnn,,,)), a new dataset that explicitly captures user prefer- ences beyond target retrieval. Extensive exper- iments demonstrate that QURE achieves state- of-the of-the-art art performance on FashionIQ and CIRR datasets while exhibiting the strongest alignment with human preferences on the HP-FashionlQ dataset. The source code available at https: / //github. com/ jac.wjkycwwkkkkkk /OuRe.
Sort
Image Corpus
Hard Negetive Set
X
Contrastive Learning
Optimication
Target
Urrelevant
Target
Relevant
Figure I. Comparison of existing CIR methods and QURE. Tradi- tional CTR approaches all non-target images negatives in contrastive learning. In contrast, QURE ranks the image corpus using learned relevance score identify a hard negative set. then applies preference-basee optimization to distinguish not only target image also other relevant images from hard negatives, leading to improved retrieval performance.
search capabilities in scenarios where textual descriptions alone are insufficient. This task is critical for applications such as e-commerce and Internet search, where users often seek results that match complex, visually nuanced specifica- tions. For instance, as shown in Figure ing image of shirt with the text 'blue t-shirt with short sleeves' requires CIR to retrieve matching images, including variations in style or color.
query combin-
Existing CIR methods focus on retrieving the target image and overlook the broader relevance other images. This limitation arises from the structure of CIR datasets, which typically annotate only a single target image per query and lack annotations for false relevant images marked as targets. Furthermore, most CIR methods (Bai et al., 2024; Li et al.; Zhang et al., 2024) adopt contrastive learning, treating the target image as positive and all other images in the batch as negatives, inevitably including false negatives. While effective ranking the target image within the top-k results, this approach frequently retrieves irrele- vant images, as illustrated in Figure 1. Such irrelevance can reduce user satisfaction, retrieval experience largely depends on the proportion of relevant items in the retrieved set (Al-Maskan & Sanderson, 2010).
1. Introduction
Composed Image Retrieval (CIR) retrieves images from large corpus using text and image inputs, enabling precise
KAIST. Correspondence
to:
Sung-Ju Lee
<profsj @ kr.>..
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
We propose Query-Relevant Relevant Retrieval through Hard Neg- ative Sampling (QURE), which aims to retrieve not only

---

QuRe: Query-Relevane Retricval through Hard Negative Sampling in Composed Image Retrieval
the target image but also other relevant images with high ranks, thereby improving user satisfaction. To mitigate the inclusion of multiple false negatives, QURE adopts a reward model training objective (Ouyang et al., 2022), optimizing the likelihood of ranking a positive image above a single negative image, with the target image designated as positive for each query. A key challenge lies sampling appropri- ate negatives, excluding false negatives while incorporating hard negative.
identifies images between two sharp declines in relevance scores after the target image. It optimizes model training by leveraging highly challenging samples while ensuring they are less relevant to the query than the target.
We achieve state-of-the-art performance on the FashionIQ and CIRR datasets, and demonstrate superior alignment with human preferences on the newly introduced Human- Preference F (HP- Fas-ooshion))) dataset.
Hard negatives are generally defined as samples that (1) be- long to a different class than the anchor and (2) have embed- dings close to the anchor (Robinson et al., 2020; Ma et al., 2020; Tabassum et al., 2022; Huynh et al., 2022). Tradi- tional hard negative selection relies on class labels, but in CIR, each query has unique target, making class-based distinctions impractical. Moreover, randomly selecting neg- atives from the entire corpus or choosing those too similar to the target often leads to suboptimal training (Figure 4). To address these challenges, we redefine the first condition as "less relevant to the query than the target,' ensuring that hard negatives differ from the query in at least one key attribute, such as color or shape.
2. Related Work
Vision-language foundation model. Vision-Language Models (VLMs) have gained attention for their ability to integrate multimodal data. Transformer- effectively handle both visual and language inputs (Li et al., 2019; Lu et al., 2019). Contrastive learning methods, which align visual and language modalities, have significantly im- proved performance in VLMs (Jia al., 2021; Radford et al., 2021). New architectures combine features from both modalities. For instance, Flamingo (Alayrac et al., 2022) and BLIP (Li et al., 2022) use cross-attention, where visual hidden states from the vision encoder are inserted into cross- attention layers within the text encoder layers. BI.IP-2 (Li et al., 2023) and QWEN (Bai al., 2023) utilize pre-trained image and text encoders with learnable networks that bridge the gap between modalities.
architectures
To balance both conditions for selecting proper hard neg- atives in CIR, QURE periodically sorts the images in the corpus based on their relevance scores, calculated using the training model. During training, with the target image marked as positive, visually similar images (false negatives) tend to rank near the top of the sorted list. The sorted im- ages are then divided into three groups: (1) false negatives, including the target image, (2) hard negatives, and (3) easy negatives. Hard negatives are defined as images that fall between two sharp declines in relevance scores, which occur after the target image. These steep drops indicate signifi- cant changes in relevance (Xia et al., 2024), ensuring that hard negatives differ from the query in at least one key at- tribute (Figure 6). This distinction makes hard negatives particularly valuable as challenging examples for training.
QURE fine-tunes BLIP-2, using its image and text encoders with the Q-former module to handle modality gaps. We chose BLIP-2 for its efficient combination of image and text processing, requiring minimal training of the Q-former module.
Composed image retrieval. The CIR task fetches im- ages using multimodal input features. A common approach is feature fusion, where the reference image and text are jointly embedded and compared against embeddings of can- didate images (Vo et al., 2019; Dodds et al., 2020; Liu et al., 2021; Baldrati et al., 2023). Bi-BLIP4CIR (Liu et al., 2024) trains the text encoder using bi-directional training to cap- ture both text directions of given relation. CASE (Levy et al., 2024) leverages BLIP (Li et al., 2022) cross-attention architecture to perform an early fusion between the modali- ties. Other approaches transform images into sseudo-word embeddings or sentence-level prompts for text-to-image re- trieval (Liu et al., 2023; Saito et al., 2023; Bai et al., 2024). MGUR (Chen et al., 2024) introduces an uncertainty loss for saarse-grained retrieval, and SPN4CIR (Feng al., 2024) proposes data generation method to scale positive and negative samples using multimodal L.MSs.
Our approach achieves otate-of-the-art performance on the FashionIQ and CIRR datasets. To further evaluate alignment with human preferences, we created the -reee--Peeeeeece FashionIQ (HP-FashionIQ) dataset, where human annota- tions indicate preferences between two retrieved sets for a given query. Experiments on Ha-FashionIQ demonstrate that QURE achieves the best alignment with human prefer- ences compared to baseline methods.
Our contributions are as follows:
We propose QURE, CIR algorithm that retrieves not only the target image but also other relevant images with high ranks to enhance user satisfaction.
However, all previous works trained models using con- trastive loss, treating the target image as positive and all other images in the batch as negatives. This approach risks
We introduce a novel hard negative sampling strategy that
2

---

QuRe: Query-Relevan Retrieval through Hard Negative Sampling in Composed Image Retrieval
Training Objective
Hard Negative Sampling
Reference Image
"Two dogs of the same breed on the floor"
Target Image
Query
Image Encoder
Q Q-rrerr
Tokens d
Q-Fomrer
Image Encoder
Steep
Query Features
Negative Features
After target, Between Steeps (Hard Negatives)
Negative Image
Positive Image
Positive Features
Positive
Negative
Score
Score
Steep
Image Encoder
Q-Foree
KL-Div Loss
Figure Overview QURE. During training, QURE periodically ranks corpus images by relevance score using the current model. Hard negatives are selected from the range between two sharp drops in relevance scores following the target image. A KI. divergence loss is then used to train the model to assign higher relevance to the positive image than to randomly chosen hard negative.
including false negatives negatives, which may lead to irreeevant retrieval results. QURE overcomes this issue by employing the reward model objective, pairing each positive image with sampled single hard negative.
3. Methodology
We denote CTR dataset as ID {d. ii= 1,..., Naata where each data point consists of a reference image, relative text, and target image. i.e., d, {aı, TT, 31, }. The goal of CIR is to retrieve a set of images from the image corpus I = (T, i = 1..... Nimg }. including the target image yı. where the retrieved images reflect the specified relative text #y while preserving the visual properties of the reference image Fy. The training algorithm of QURE provided in Appendix A.
Hard negative sampling. Hard negative sampling is a common technique in contrastive learning that selects more informative negatives rather than treating all images in the batch equally. HCL (Robinson et al., 2020) suggests that hard negatives should (1) belong to a different class than the anchor, and (2) have embeddings closer to it. However, in unsupervised settings like the CIR dataset, the lack of annotations makes it challenging to identify appropriate hard negatives. Embedding similarity alone can lead to including false negatives, prompting some approaches to incorporate model uncertainty, higher uncertainty often indicates proximity to class boundaries (Ma et al., 2020). UnReMix (Tabassum et al., 2022) addresses this by combin- ing embedding similarity with model uncertainty to identify suitable hard negatives. Mcanwhile, FNC (Huynh et al., 2022) uses predefined threshold to filter out false nega- tives and treats such samples as positives.
3.1. QURE: Query-Relevant Retrieval through Hard Negative Sampling
Relevance score. For each image in corpus / we define the relevance score the query as the inner product of the bi-modal query and image embeddings:
(())ig(@1).a8 IT) ((Eimg(L))
s(11, IT,
(I)
Here, Esny is the BLIP-2 image encoder, Q denotes the Q- Former, and is the learned BLIP-2 temperature. Q-Former processes the reference image embedding and relative text to align image and text modalities through cross-attention.
To address these challenges, QURE introduces novel hard negative sampling strategy. It selects hard negatives from images positioned between two sharp drops in relevance scores following the target image. The steep drop after the target ensures that the selected image differs from the query at least one key attribute (e.g., color or shape), making it suitable hard negative for training.
Training objective. QURE trained to maximize the prob- ability of preferring the highly relevant image, positive, over the less relevant image, negative, We model the latent prefer- ence distribution p" using the Bradley-Tere model (Bradley & Terry, 1952), where I, and T, denote the positive and negative images, respectively.
p² (I, > In xI,0T) = a (s(21,T,,, Ip) s(x1, ær. In)). (2)
3

---

QuRe: Query-Relevany Retricval through Hard Negative Sampling in Composed Image Retrieval
We set the target image as positive l yr- To in- clude the negative image I, construct the dataset DD* 4(I,, Ip. In) (I), E D, In E [i. where In is drawn from the hard negative set HI, defined in Section 3.2. The model is optimized by minimizing the negative log-likelihood (NLL) loss:
Finally, the hard negative set for the i-th query is defined as:
H. = (2, li E [min((1, + 1, max(k1 k2)], $1.j < ST, 9:)}
Sampling hard negatives. To ensure diverse and informa- tive negatives, a single image is sampled from the defined hard negative set for every epoch based on a uniform distri- bution.
L == - (p,))- Dog((" (T, > In 21, [- (3)
This objective is equivalent to minimizing the KL. diver- gence between p" and a target distribution p = [1, 0j. ensur- ing the model prefers the positive image over the negative.
4. HP-FashionIQ Dataset
The commonly used evaluation metric, Recall@k, fails to capture user satisfaction. While user satisfaction increases with the number of relevant items retrieved (AI-Maskari & Sanderson, 2010), Recall@K only checks whether the target image is retrieved, disregarding the relevance of other im- ages in the fetched result. However, assessing the relevance of retrieved images is challenging, as it requires evaluating how well the images align with both the text and image in- puts, which in CIR involves considering numerous complex attributes.
3.2. Hard Negative Set Sampling
Defining hard negative set. We defined the conditions for an appropriate negative image In the CIR setting as follows:
C1. The negative image I, should be less relevant to the query than the target image Ip- C2. The relevance score of the negative image In should be similar to that of the target image Tp-
However, identifying false negatives is practically infeasible as the CIR dataset annotates only the target image for each query. Additionally, selecting images that are overly dissim- ilar from the target may satisfy C1 but fail to meet C2, or vice versa, further complicating the selection process.
Human evaluation remains the most reliable way to measure image relevance, as humans can accurately assess how well image matches a multi-modal query. To facilitate such evaluation, we created the Human- Preference FashionIQ (HP- Pa--aashonn)) dataset, using the validation set of the Fash- ionIQ dataset (Wu et al., 2021) with 61 participants. We selected the FashionIQ dataset for its high relevance and broad applicability, mirroring the search functionalities of e-commerce platforms.
To balance these conditions, we sort the images in corpus by their relevance scores obtained from the training model, forming an ordered set:
S.- ....1...., Nemg } where 81,1 2 ... 2 St.Nemg
(4)
Table 1. Statisties of HP-FashionIQ Dataset: Each query has two sets of retrieved images from different CIR models, annotated based on their preferences.
where S; represents the relevance scores sorted in descend- ing order for the i-th query.
# Total Queries
# Shirts Queries
# Toptee Queries
# Valid Queries
Based on the ordered set, we categorize images into three groups: (I) false negatives, which include the target image, (2) hard negatives, and (3) easy negatives. To identify hard negatives, we select images positioned between two steep relevance score drops occurring after the target image. The steep drop in relevance score indicates a noticeable shift in semantic similarity (Xia et al., 2024), helping to exclude false negatives while selecting negatives that are still chal- lenging for the model. By focusing this transition zone, we ensure that the selected negatives maintain balance between similarity and distinction from the target image.
3,050
1,800
1,250
2,715
Data collection setting. Each question consisted of two retrieved image sets, each with the top 5 results from differ- ent CIR models. For every question, two CIR models were randomly selected from the following four: CLIP4CIR (Bal- drati et al., 2023), Bi-BLIP4CIR (Liu et al., 2024), CoVR- BLIP (Ventura et al., 2024b), and SPRC (Bai et al., 2024). We provided queries and retrieved images from the 'shirts" or 'top tees' categories of the FashionIQ dataset partici- pants. Each participant was given 50 questions with total of 100 sets of retrieved images, covering 3,050 queries in the FashionlQ validation set.
The subset of scores lower than the target score is defined as:
S. Sxtarg {8i.j 8i.j < 5(x1,, TT, J:) From this subset, the indices of the top two largest degrada- tions are identified as:
(5)
Annotation methodology. For each question in the survey, participants chose the preferred set between the two pro- vided sets, assessing the alignment with human preferences.
k hog- = arg 20p-2,(815 - 81.41 8i.5 E Sataro).
(6)
4

---

QuRe: Queey-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval
Table Performance comparison the FashionIQ validation dataset different methods. The best results highlighted in bold, and the second-best are underlined.
Dress
Shirt
Toptee
Average
Method
R 10
R 50
R@10
R@50
R 10
R@50
R 10
R@
Avg.
CoSMo (Lee al., 2021)
23,60
49.18
18.11
43.18
24.63
54.31
22.11
48.89
35.50
MGUR (Chen et al., 2024)
23.15
48.74
18.99
43.47
25.55
52.83
22.56
48.35
35,46
CLIP4CIR (Baldrati et al., 2023)
38.32
63,90
44.31
65.41
47.27
70.98
43.30
66.76
55.03
Bi-BLIP4CIR (Liu et al., 2024)
39.12
62.92
39.21
62.81
44.37
67.06
40,90
64.26
52.58
CoVR-BLIP (Ventura 2024b)
44.55
69.03
48.43
67.42
52.60
74.31
48.53
70,25
60.24
SPRC (Bai et al., 2024)
45.71
70.00
51.37
72.77
55.48
77.46
50.86
73.41
62.13
QURE
46.80
53.53
72.87
57.47
77.77
52.60
73.48
63.04
To our knowledge, this is the first CIR dataset with human preference-annotated retrieved images. An example of data from Q is shown in Figure 3, with a detailed explanation of the data collection process in Appendix C.
centration during the annotation period, users rated the rel- evance of the retrieved sets before annotating for their pre- ferred choice. Users rated each on 5-point Likert scale (Likert, 1932), with score of 5 indicating strong match with the query. Queries were discarded if the user did not prefer the set with the higher relevance score. As result, 28 queries were excluded, leaving 2,715 valid queries. The total number of queries in the IP--aasionnQ dataset shown in Table
Reference Image
Relative Text
+
"Has long sleeves and brighter in color, white long-sleeve
5. Experiments
Retrieved 1
5.1. Experimental Setup
Datasets. We evaluate the models on widely used CIR datasets, FashionlQ (Wu et al., 2021) and CIRR (Suhr et al., 2018), to assess their ability to retrieve the target image. Additionally, we evaluate them on the II--aahioonQ dataset to assess their alignment with human preferences.
Retrieved 2
Implementation. We used BLIP-2 (Li al., 2023) with ViT-L image encoder. Following previous work (Baldrati et al., 2023), we resized images to 224x224 with a 1.25 padding ratio. QURE is trained using the Adam opti- mizer (Loshchilov. 2017) for 50 epochs on CIRR and 30 epochs on FashionIQ. The hard negative set HI was defined Rdef times, starting with warm-up phase where H. initially included the entire corpus except for the target during the first |eepoch/Ddee] epochs. The hard negative set H. is up- dated every [neeochh/ /n epochs. We set Tade to six for both FashionIQ and CIRR. All experiments were conducted using single Nvidia RTX 3090 GPU.
Figure 3. An example from the HP- Pashionio dataset: Given a query, retrieved image user preferred Set 2, as it better preserves the visual properties of the reference image.
presented. In this example, the
Modality redundancy check. While CIR should consider both image and text, some examples focus solely on the image or the text. CASE (Levy et al., 2024) highlighted modality redundancy FashionIQ, indicating that text is sometimes more influential than the image. We instructed participants to consider both input modalities equally. We asked them to flag instances where one modality seemed irreeevat to the retrieved images to exclude data that was unclear for human evaluation. A total of 307 queries were treated as irrelevant and excluded.
5.2. Comparison with
CIR Models.
Table presents the evaluation of CIR models on the Fash- ionIQ dataset. QURE consistently achieves the best or second-best performance across all categories, attaining the highest overall average. Notably, QURE demonstrates
Sanity check. To identify instances of decreased user con-
5

---

QuRc: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval
Table Performance comparison on the CTRR test dataset across different methods, where Recall, @ @ represents Recallsubset@1 K. The best results are highlighted in bold, and the second-best are underlined.
Recall @ K
Recall @K
Average
Method
K=1
K=5
K=10
K=50
K=1
K=2
K=3
R@5 R @1
CosMo (Lee et al., 2021)
6.48
23.11
34.63
67.33
20.29
40.22
60.80
43.55
MGUR (Chen et al., 2024)
5.78
21.45
33.42
67.06
20.29
40.22
60.80
42.91
CLIPACIR (Baldrati al., 2023)
44.12
77.23
86.51
97.95
73.11
89.11
95.42
75.17
Bi-BLIP4CIR (Liu al., 2024)
32.55
64.36
76.53
91.61
63.54
82.46
92.48
63.95
CoVR-BLIP (Ventura al., 2024b)
39.76
70.15
80.89
95.01
72.46
87.86
94.77
71.30
SPRC (Bai al., 2024)
50.75
80.58
88.72
97.59
79.57
91.76
96.70
80.07
QURE
52.22
82.53
90.31
98.17
78.51
91.28
96.48
80.52
sigmificant improvements over SPRC (Bai et al., 2024) when the retrieved set size is small, such as in Recall 10, with gains of 1.09%, 2. 2.,, and 1.99% for the dress, shirt, and toptee categories, respectively. Previous work (Levy et al., 2024) has identified high modality redundancy in the Fash- ionIQ dataset, where text dominates the retrieval process, favoring text-based methods such as Bi Bi-BLIP4CIR (Liu et al., 2024) and SPRC (Bai et al., 2024). Despite this bias, QURE achieves state-of-the-ar performance, improving the overall average recall by 10.46% and 0.91% % compared with Bi-BLIPACIR and SPRC, respectively. These results high- light QURE's effectiveness in accurately retrieving the target image, even in the presence of modality redundancy.
Set 2, and annotated their preferences between them. To calculate the overall relevance score of CIR model for each set, we averaged the relevance scores of the five retrieved images within the set:
(Set i) = Σ
(r,,
(8)
TESet
where 8yat denotes the relevance score (e.g., cosine similar- ity) computed by CIR models.
The aligmment with human preferences is measured through the preference rate, which represents the conditional prob- ability that Set 1 is preferred when its relevance score is greater than that of Set 2. Formally, we define the prefer- ence rate as:
Table 3 shows the evaluation results on CIRR, general- domain dataset. QURE achieves the highest performance across all @ metrics, particularly excelling in Re- call@1 and Recall@5 surpassing the current state-of-the-arr method, SPRC (Bai et al., 2024), by 1.47% and 1.95%, re- spectively. Regarding Recall, @k, which measures retrieval performance from subset containing relevant images and the target, QURE achieves the second-bec best results. This result is attributed to the design of QURE, where even false negative images can receive higher scores than the target as they closely match the query. This behavior arises from our hard negative set definition, which excludes false negatives, ensuring that relevant images are not treated as negatives and be ranked higher than the target. Notably, QURE achieves sate-of-theearr performance the combined Re- c Recall, @1 average.
P(Sel 1 > Sel 2 Syet(Set 1) Sree(Se 2)).
(9)
Table 4 shows that the ranking of CIR models based on their alignment with human preferences on the HP-FashionIQ dataset differs from their performance rankings on the Fash- ionIQ dataset (Table using the Recall @k metric. For in- stance, MGUR achieves performance comparable CosMo in retrieving the target image but aligns more closely with human preferences. This discrepancy stems from MGUR's s additional coarse-grained loss, which considers both the target image and visually similar alternatives as positives. Moreover, while SPRC, CoVR-BLIP and Bi-BLIP4CIR surpass CLIP4CIR in terms of Recall @k on the FashionIQ dataset, CLIP4CIR aligns better with human preferences the FaP-FoshionI dataset, despite its lower accuracy in retrieving the exact target image.
5.3. Evaluation with the HP-FashionIQ Dataset
We evaluate the alignment of CIR models with human pref- erences. Given query {xr, J. each participant was pre- sented with two different sets of retrieved images, Set 1 and
QURE achieves the best alignment with human preferences, which shows that Set is preferred 74.55% of the time when its relevance score exceeds that of Set 2. This result
6

---

QuRc: Query-Relevany Retrieval through Hard Negative Sampling in Composed Image Retrieval
Table Preference comparison HP-FashionlQ dataset across different methods. The best results are highlighted in bold, the second- best are underlined.
Table 5. Zero-shot performance comparison the CIRCO dataset across different methods. The best results are highlighted in bold. and second-best are underlined.
Method
Preference Rate (%)
Method
mAP@5
mAP@ 10
mAP@25
mAP@50
CosMo (Lee et al., 2021)
72.96
CosMo
0.31
0.40
0.47
MGUR (Chen al., 2024)
73.99
MGUR
0.14
0.17
0.25
0.30
CLIP4CIR (Baldrati et al., 2023)
74.45
CLIP4CIR
10.58
11.18
12.32
12.96
Bi-BLIP4CIR (Liu 2024)
67.33
Bi-BLIPACIR
4.74
4.97
5.69
6.10
CoVR-BLIP (Ventura et al., 2024b)
73.15
CoVR-BLIP
18.35
19.25
21.02
21.88
SPRC (Bai al., 2024)
73.82
SPRC
17.57
18.48
20.14
20.98
QURE
74.55
QURE
23.22
24.23
26.26
27.24
Top-k
After target, top-k
All corpus
QuRe
FashionlQ
CIRR
Average Recall
Average Set Size
58
600
54
400
200
Epoch
Figure 4. Average recall on the FashionlQ validation set using four hard negative definitions. After the initial hard negative is established (e.g.. epoch 4), the choice of definition significantly influences average recall throughout training.
30
40
50
Epoch
Figure 5. Average size of the hard negative per epoch Fash- ionIQ and CIRR. This plot shows how the average number of hard negatives per query evolves across epochs in which the defined.
demonstrates QURE's ability not only to retrieve the correct target image but also relevant images that best align with human preferences.
Table indicates SPRC (Bai et al., 2024) significantly out- performs CoVR-BLIP (Ventura et al., 2024b) on the CIRR dataset, CoVR-BLIP shows better performance in the zero- shot setting. This suggests that CoVR-BLID generalizes un- seen tasks better. The results highlight that QURE achieves the highest generalizability, significantly improving over other baselines.
5.4. Ablation Studies
We present ablation results under various scenarios, with additional experiments included in Appendix B.
Zero-shot performance comparison. We evaluated the zero-shot performance of the models the CIRCO dataset using those pre-trained on the CIRR dataset. Although QURE and baseline methods are not explicitly designed for zero-shot tasks, models that effectively retrieve relevant images are expected to perform well in such scenarios. Fur- thermore, CIRCO is the first CIR dataset to include multiple ground truths, addressing the issue false negatives in ex- isting datasets. Thus, evaluating the mean average precision at (mAP@k k) on this dataset provides a reliable measure of the model's ''s ability to retrieve relevant items.
Effect of hard negative set definition strategy. To assess the effectiveness of the hard negative set definition approach, we compare four strategies on the FashionIQ dataset. The hard negative set is defined as follows: (1) the entire image corpus (All corpus), (2) the top-k images based on relevance scores (Top-k), (3) the top-k images with relevance scores lower than the target (After target, top-k), and (4) images between sharp drops in relevance scores, occurring after the target (QURE).
Table 5 reveals that QURE achieves the best performance among all baselines, outperforming the second-best method, CoVR-BLIP by an average margin of 5.13 mAP. While
Figure 4 presents the average recall of each strategy across training epochs. The results indicate that the All corpus approach shows consistent improvement; however, ul-

---

QuRe: Query- Relev-Relevvat Retricval through Hard Negative Sampling in Composed Image Retrieval
Reference Image
Relative Text
Target Reference Image
Relative Text
Target image
short sleeved and has collar, is grey with shorter sleeves"
"has vertical and horizontal stripes, darker and has plaid pattern"
Hard Negative Set
Hard Negative Set
Figure 6. Hard negative set examples for two shirt-category queries in FashionIQ. Each set contains images that are semantically similar the target but deemed less relevant by the model.
timately leads to suboptimal performance as the selected negatives from the entire corpus may be too irrelevant. In- spired by HCL (Robinson et al., 2020), we define hard negative set by selecting the top-100 100 images based on rele- vance scores (Top-k). This approach occasionally enhances model performance (e.g., at epochs 10 and 20), but it of- ten degrades due to the inclusion of false negatives. As training progresses, since the target image is treated as posi- tive, visually similar images (false negatives) tend to rank higher. To mitigate the issue in the Top-k approach, we analyze the performance of the After target, top-k strategy, which excludes false negatives by selecting the top-k images following the target. Unlike Top-k, this method provides consistent improvements without fluctuations but gradually leads to slight performance degradation over time. This decline results from the heuristic assumption that all images right after the target are reliable hard negatives. In contrast, QURE identifies points where relevance drops sharply and selects images between two such points, resulting in stable performance improvements throughout training. These re- sults show that, with a novel hard negative sampling method, QURE effectively balances the conditions C1 and C2 of se- lecting proper hard negatives.
throughout training. Initially, the model identifies a broader range of images as hard negatives due to lower confidence. As training progresses, it refines this selection, yielding a smaller yet more challenging hard negative set. This dy- namic resembles curriculum learning, where increasingly difficult samples accelerate model convergence.
Visualization of hard negative set. To evaluate our hard negative set sampling strategy, we present qualitative ex- amples from the FashionI dataset in Figure 6, illustrating two queries. In the first query. the hard negatives either lack collar or text from the reference image 'ECK' or depict shirts instead of t-shirts, differing from the input image. In the second query, all retrieved shirts contain only vertical stripes while successfully retrieving darker shirts. These examples demonstrate that the hard negative sets defined by our method consist of images that are less relevant than the target (C1) while remaining semantically similar (C2), thereby satisfying both conditions outlined in Section 3.2.
6. Conclusion
We present QURE, a CIR model designed to retrieve both target and relevant images to enhance user satisfaction. Ex- isting CIR datasets typically annotate only the target image per query, leading prior methods to rely on contrastive learn- ing that treats all non-target images as negatives. QURE mit- igates false negatives by leveraging reward model objectives and introduces a novel hard negative sampling strategy, se- lecting images between two sharp relevance score drops af- ter the target. To evaluate the alignment of CIR models with human preferences, we introduce the Human-Preferenn FashionIQ (HP-FashionIQ) dataset. QURE achieves state- of-the-aut art performance both the FashionIQ and CIRR datasets and demonstrates the highest alignment with human preferences on the HP-FashionIQ dataset.
Size of hard negative set. The number of hard negative images varies significantly depending on the complexity of bi-modal queries and the corpus. For instance, when the input text contains fewer attributes, such as 'blue shirt with short sleeves', the hard negative set may include images that mathh either 'blue' or 'short sleeves', resulting in a larger set. Therefore, it crucial to define a query-specific hard negative set size (Xia et al., 2024).
Figure 5 shows the average size of the hard negative set across all queries in the FashionIQ and CIRR datasets. While QURE does not explicitly define the size of the hard negative set, the results indicate a consistent decrease
8

---

QuRc:
Retrieval through Hard Negative Sampling in Composed Image Retrieval
Acknowledgements
Composed image retrieval using contrastive learning and task-oriented clip-based features. ACM Transactions on Multimedia Computing, Communications and Applica- tions, 20(3):1-24, 2023.
This work was supported by the National Research Founda- tion of Korea (NRF) grant funded by the Korea government DSSTTXXS-2024-00337000
50%), Institute of Informa- tion & Communications Technology Planning & Evalua- tion (ITTP) grant funded by the Korea government (MSIT) URS-2019-1119005, Artificial Intelligence Graduate School Program (KAIST), 5%), and the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2022- 0-00871, Development of AI Autonomy and Knowledge Enhancement for AI Agent Collaboration, 45%)
Bradley, R. A. and Terry, M. E. Rank Analysis of Incom- plete Block Designs: The Method of Paired Compar- isons. Biometrika, 39(3-4):324-345, 12 1952. ISSN 0006- 3444, doi: 10. 10.1093/biomee/333334.2222 URL https: 4.324.
/ //doi.org// 10. wwt.:199//iimme//333
Chen, Y., Zheng, Z., Ji, W., Qu, I.., and Chua, T.-S. Composed image retrieval with text feedback via multi- grained uncertainty regularization. In The Twelfth In- ternational Conference on Learning Representations, 2024. URL https////pppeeeeeeeeeee net/ forum?
Impact Statement
This paper introduces Query-Relevve Retrieval through Hard Negative Sampling (QURE), an approach to Com- posed Image Retrieval (CIR) that improves retrieval qual- ity by retrieving both the target and other relevant images, improving user satisfaction. It addresses the false nega- tive problem in CIR with novel hard negative sampling strategy, advancing Machine Learning and Information Re- trieval. QURE contributes to improving search efficiency in e-commerce and visual search platforms, which bene- fits businesses and consumers by delivering more relevant search results. Additionally, the human preference-basee evaluation (HP- HiP-FashionnQ dataset) aligns CIR models with human expectations, making Al-powered retrieval more user-centric.
Dodds, E., Culpepper, J., Herdade, S., Zhang, Y., and Boakye, K. Modality-agnostic attention fusion for visual search with text feedback. arXiv preprint arXiv:2007.00145, 2020.
Feng, Z., Zhang, R., and Nie, Z. Improving composed image retrieval via contrastive learning with scaling positives and negatives. arXiv preprint arXiv:2404. //17, 2024.
Huynh, T., Kornblith, S., Walter, M. R., Maire, M., and Khademi, M. Boosting contrastive sulf-super learn- ing with false negative cancellation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 2785-2795 2022.
Jia, C., Yang. Y., Xia, Y. Chen, Y.T.,, Parekh, Z., Pham, H., Le, Q., Sung, Y.-H., Li, Z., and Duerig, T. Scaling up visual and vasion-language representation learning with noisy text supervision. In International conference on machine learning, pp. 4904-4916. PMLR, 2021.
References
Al-Maskari, A. and Sanderson, M. A review of factors influ- encing user satisfaction in information retrieval. Journal the American Society for Information Science and Tech- nology, 61 61(5):859-888 868, 2010.
Lee, S., Kim, D., and Han, B. Cosmo: Content-style modu- lation for image retrieval with text feedback. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 802-812 2021.
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lene, K., Mensch, A., Millican, K., Reynolds, M., et al. Flamingo: a visual language model for few- shot learning. Advances in neural information processing systems, 35:237 23736, 2022.
Levy, M., Ben-Ari, R., Darshan, N., and Lischinski, D. Data roaming and quality assessment for composed image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 2991-2999, 2024.
Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y. Huang, F., et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
Li, J., Li, D., Xiong, C., and Hoi, S. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International confer- ence on machine learning, pp. 12980. PMLR, 2022.
Bai, Y., Xu, X., Liu, Y., Khan, S., Khan, F., Zuo, W., Goh, R. S. M., and Feng, C.-M. Sentence-level prompts benefit composed image retrieval. In The Twelfth In- ternational Conference on Learning Representations, 2024. URL https::////....... http:///o/eeeeeeeeeeeeeeeee net/ forum?
Li, J., Li, D.,, Savarese, S., and Hoi, S. Blip-2: Bootstrapping language-imnn -igguage-imageeaae pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMIRR, 2023.
id-m3ch3kJ17g
Baldrati, A., Bertini, M., Uricchio, T., and Del Bimbo, A.

---

QuRc:
Retrieval through Hard Negative Sampling in Composed Image Retrieval
Li, L.. H., Yatskar, M., Yin, D., Hsieh, C.-J,, and Chang, K.-W. Visualbert: A simple and performant baseline for vision and language. arXiv preprint S.Xiv:198.003555 2019.
Saito, K., Sohn, K., Zhang, X., Li, C.-L., Lee, C.-Y., Saenko, K., and Pfister, T. Pic2word: Mapping pictures words for zero-shot composed image retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1 19305-19314, 2023.
Li, W., Fan, H., Wong, Y. Yang, Y., and Kankanhalli, M. Improving context understanding in multimodal large lan- guage models via multimodal composition learning. In Forty-first International Conference on Machine Learn- ing.
Suhr, A., Zhou, S., Zhang, A., Zhang, I., Bai, H., and Artzi, Y. A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv: htt::/811.0000 1.00499, 2018.
Likert, R. A technique for the measurement of attitudes. Archives of psychology, 1932.
Tabassum, A., Wahed, M., Eldardiry, H., and Lourentzou, I. Hard negative sampling strategies for contrastive rep- resentation learning. arXiv preprint arXiv:2206.00 197, 2022.
Liu, Z., Rodriguez- Opazzo, C., Teney, D., and Gould, S. Image retrieval on real-life images with pre-trained vision- and and-language models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2125- 2134, 2021.
Ventura, L., Yang, A., Schmid, C., and Varol, G. Covr-2: Automatic data construction for composed video retrieval. IEEE Transactions on Pattern Analysis and Machine In- telligence, 2024a.
Liu, Z., Sun, W. Teney, D., and Gould, S. Candidate set re-ranking for composed image retrieval with dual multi- modal encoder. arXiv preprint 4rX10:2305.16304, 2023.
Ventura, L., Yang, A., Schmid, C., and Varol, G. CoVR: Learning composed video retrieval from web video cap- tions. AAAI, 2024b.
Liu, Z., Sun, W., Hong. Y., Teney, D., and Gould, Bi- directional training for composed image retrieval via text prompt learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5753- 5762, 2024.
Vo, N., Jiang, L.. Sun. C., Murphy, K., Li, L.-J., Fei-Fei, L., and Hays. J. Composing text and image for image retrieval-an empirical odyssey. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6439-6448 2019.
Loshchilov, L. Decoupled weight decay regularization. arXiv preprint arXiv: 1.05 2017.
Wu, H., Gao, Y., Guo, X., Al-Halah, Z., Rennie, S., Grau- man, K., and Feris, R. Fashion iq: A new dataset towards retrieving images by natural language feedback. In Pro- ceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp. 11307-11317, 2021.
Lu, J., Batra, D., Parikh, D., and Lee. S. Vilbert: Pre- training task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural informa- tion processing systems, 32, 2019.
Xia, P., Zhu, K., Li, H., Wang, T., Shi, W., Wang, S., Zhang, L., Zou, J., and Yao, H. Mmed-rag: Versatile multimodal rag system for medical vision language models. arXiv preprint arXiv:2410. 13085, 2024.
Ma, S., Zeng, Z., McDuff, D., and Song, Y. Active con- trastive learning of audio-visual video representations. arXiv preprint arXiv:2009.09800 2020.
Zhang, K., Luan, Y., Hu, H., Lee, K., Qiao, S., Chen, W., Su, Y., and Chang, M.-W. Magiclens: Self-supervised image retrieval with open-ended instructions. Forty-first International Conference on Machine Learning, 2024.
Ouyang, I., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
Radford, A., Kim, W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp.
PMLR, 2021.
Robinson, J., Chuang, Y. Sra, S., and Jegelka, S. Con- trastive learning with hard negative samples. arXiv preprint XrXiv:2000.0455 2020.
10

---

QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval
A. Algorithm
Algorithm Training Flow QURE
1: Input: Parameters 9, CIR dataset D, Image corpus I, Number of defining negative set Biats Total epochs Nepoch
2: for each epoch do
3: if
0 then
4: H I yr
// Warmup: using all candidates as the negalive
5: else if e 0 and mod [epooch/ nact 0 then
6: H + (T PT,, Ur) < s(11, Pr, I), I E range between two largest score degradations)
I/ /'quation (7)
7:
end if
8:
for each batch b do
9:
In - Sample(HH))
I/ Sample one negative for every query // Equation (3)
t (( Pr,, Ipa) s(x1,, Fr,, Imn)))
11: 0 VoL
12: end for
13: end for
B. Additional Experiments
B.1. Ablations with Consistent Model Backbone
In Section 5, compare QURE with existing baselines FashionIQ, CIRR, HP-FashionlQ, and CIRCO. However, model backbones not unified, as we follow the comparison settings used in prior CIR methods (Baldrati et al., 2023; Liu et al., 2024; Bai et al., 2024). To ensure fairer comparison, we conduct experiments using both BLIP and BLIP-2 backbones. Specifically, we train QURE with the BLIP backbone to compare against Bi- BIIBLIP,,,R, We also identify CoVR-2 (Ventura et al., 2024a), the latest version of CoVR-BLIP that adopts BLII-22, and compare with our original QURE model.
Table Table 7, and Table present results on the CIRR, FashionIQ, HP-FashionIQ. and CIRCO datasets. QURE with BLIP backbone consistently outperforms Si-BLIP4CIR. Moreover, CoVR-2, which adopts BLIP-2 backbone, still underperforms compared to original QURE model with the same backbone.
Table Performance comparison the CIRR test dataset with consistent model backbones, where Recall, @ represents Recallsub- set@ K. The best results are highlighted in bold.
R
Recall @K
Average
Method
Backbone
K=1
K=5
K=10
K=50
K=1
K=2
K=3
R@S + R @1
BI-BLIPACIR
BLIP
32.55
64.36
76.53
91.61
63.54
82.46
92.48
63.95
QURE
BLIP
51.52
80.29
88.89
97.74
78.02
91.23
96.55
79.16
CoVR-2
BLIP-2
42.80
74.60
83.90
96.22
69.49
86.22
93.98
72.05
QURE
BLIP-2
52.22
82.53
90.31
98.17
78.51
91.28
96.48
80.52
B.2. Visualization of Score Steepness
QURE defines a query-specific hard negative set by excluding false and easy negatives, aiming to enhance training effectiveness. Specifically, selects images between the largest drops in relevance scores following the target image. To analyze this steepness, visualize the sorted relevance scores.
Figure 7 illustrates the relevance scores for sample query from the FashionIQ dataset at two stages: before training and after the warm-up phase. QURE identifies hard negatives as the images between the red and green lines, determined by the top two largest drops relevance scores after the target. These steep declines- visible immediately before the red line and after the green lne-suggest substantial drops in relevance (Xia et al., 2024), effectively separating false negatives, hard negatives, and easy negatives.

---

QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval
Table Performance comparison the PashionlQ validation dataset with consistent model hackbones, The hest results highlighted
in bold.
Dress
Shirt
Toptee
Average
Method
Backbone
R@10
R@
R@10
R@
R@
R@
R@
Avg.
Bi-BLIP4CIR
BLIP
39.12
62.92
39.21
62.81
44.37
67.06
40.90
64.26
52.58
QURE
BLIP
40.80
64.90
45.93
65.90
52.07
72.87
46.27
67.89
57.08
CoVR-2
BI.I--2
46.41
69.51
49.75
67.76
51,86
72.46
49.34
69.91
59.63
QURE
BLIP-2
46.80
69.81
53.53
72.87
57.47
77.77
52.60
73.48
63.04
Table Comparison F--Fhsshoon Preference Rate and CIRCO zero-shot performance with consistent model backbones. The best results highlighted in bold.
HP--FashioIQQ
CIRCO (Zero-shot)
Method
Backbone
Preference Rate (%)
Method
Backhone
m@5
m @ 10
m
25
m @5
Bi-BLIPACIR
BLIP
67.33
Bi-BLIPPCIR
BL.P
4.74
4.97
5.69
6.1
QURE
BLIP
75.28
QuRk
BLIP
20.85
21.48
23.35
24.31
CoVR-2
BLIP-2
71.99
CoVR-2
BLIP-2
23.18
23.59
25.57
26.49
QURE
BLIP-2
74.55
QURE
BLIP-2
23.22
24.23
26.26
27.24
Since Figure 7 shows only a single query, we also present aggregated results across all queries Figure 8. This aggregated visualization shows that, after the warm-up phase, hard negatives shift toward higher-ranked positions, likely capturing more true hard negatives. This supports our design choice of introducing a warm-up stage prior hard negative selection. Without this stage, the selected negatives tend include many easy examples, observed in the left panel of the figure.
Sorted Score Plot after the Target
Epoch 6 (After warm-up)
Epoch 0
Scare
0.98
Icote Hard regatives stat indes
repelives
03723
0.43
63813
0.48
61060
0.42
300
150
200
250
300
Figure 7. Visualization of sorted relevance scores for a FashionlQ query two stages: prior to training and after completing the warm-up phase.
C. HP-FashionIQ Dataset
C.1. Data Annotation Examples
We conducted a data collection via Google Forms. Each form consisted of instructions and 25 questions, with each question including a query and two different sets retrieved images. Each participant completed two forms, covering 50 queries from the FashionlQ validation dataset, with no overlap between participants.
12

---

QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval
Aggregated Sorted Score Plot after the Target
Epoch
Epoch (Atter warm-up)
040
Wereeg legec rnto
Avange lard regariuss and lile
639
039
Asstage Fidh
035
2000
2003
400 20300 9000
Figure 8. Visualization of sorted relevance scores aggregated over all FashionIQ queries two stages: before training and after the warm-up phase.
Fig 9 shows the guidelines provided to participants, who were asked to score the retrieved image sets based on relevance to the query using a 5-point Likert scale. We gave participants relatable scenario that required them to evaluate the results from two different online shopping malls based on their input. Figure 10 illustrates an example image), relative text (user text), and two Participants (1) rated the relevance of each set, (2) indicated which set they preferred, and (3) noted whether any results were irrelevant to the reference image text, as shown in Figure 11.
reference image (original retrieved images from different CIR models (Shopping Mall I and 2).
Original Image: The image have.
2. User Text: description of how the image you are looking for differs from the original Image. sets of image search results: contains images searched through two different shopping malls.
Evaluation Method:
Please evaluate how well the two sets of image search results match the original image and the user text. Each set consists images,
higher the match with original image and user the higher whole, not each individual image.
you should give. You should evaluate
Does not metch at
Does not match.
3: Matches an average degree.
Matches.
very
Notes:
completing survey on
device
images.
The order of the images within the image search result sets does not matter.
The search results not generated images but found from foxed set of images that best match the original image and user text. Therefore, the quality of the search results may not meet user expectations. Please score as consistently as possible.
The user text in this survey is from the dataset as ia, so there may be typos or duplicated expressions.
There questions throughout the survey you asked explain
your
There simple meth problems before proceeding to the next question. you get these problems wrong. you
page.
Figure 9, Guidelines for user survey.
13

---

QuRe: Recry-Relevaat Retrieval through Hard Negative Sampling in Composed Image Retrieval
Original Image User Text: is more green and has 3/4 sleeves and has more colors and sportier
Suinness
Shopping Mail
PADDYS
QUIMEESS
59
Kiss ME
IRISH
Shopping Mell
Guinies
59
FOOTBALL
Figure 10. Example of query and two set of retrieved images in user survey,
14

---

QuRe: Query- Relev-Re Retrieval through Hard Negative Sampling in Composed Image Retrieval
Score for shopping mail 1
Does match all, not match, Matches average degree, Matches, & Matches very
1
3
4
O
O
O
O
O
Score for shopping mail 2
Does match all, 2: Does match, Matches average degree, Matches, 5: Matches very
2
4
O
O
O
O
Preference Question
Which shopping mall's results do you prefer?
Shopping
Shopping
(Optional) Irrelevance Check Question
Two set of search results are irrelevan to original image user text
Figure 11. Example of questions in survey.
15


Reviews:
[{"review_id": "3LMcjNWmhs", "reviewer": "Reviewer_13Df", "summary": "This work introduces a new method of QuRe to tackle the problem of composed image retrieval. The proposed method adopts and tailors the hard negative mining to emphasize not only the ranking of the target image, but also other relevant images in the dataset, aiming at improving the overall recall. Experiments on benchmark dataset demonstrate improved performance of the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer 13Df for the insightful comments and the time dedicated to reviewing our manuscript. Below, we provide detailed responses to each of your points.\n\n**Q1 : Justification and thorough study to the motivation of the proposed method** \n\nThe primary motivation behind our method stems from a key limitation of existing CIR datasets, where only one or a few target images are annotated per query. As a result, false negatives are often included as negatives during training. Existing baselines typically treat all non-target images as negatives, which delays convergence and degrades performance [r1].\n\n\nOur method addresses this issue by modifying the contrastive learning objective to compare the true positive against a single negative with increasing difficulty. This is achieved by selecting a small set of hard negatives for each query. We validate the effectiveness of our approach by achieving state-of-the-art performance on both CIRR and FashionIQ, and demonstrating the highest correlation with human preferences on HP-FashionIQ.\n\n\nWe clarify the motivation of our study, we added visualizations of the steep drops in relevance scores (see $\\text{\\color{blue}\\textbf{Q2}}$) and provided a mathematical justification in response to $\\text{\\color{blue}\\textbf{Q1 in Reviewer 6GLy}}$.\n\n\n[r1] Huynh, Tri, et al. \"Boosting contrastive self-supervised learning with false negative cancellation.\" Proceedings of the IEEE/CVF winter conference on applications of computer vision. 2022.\n\n\n**Q2 : Visualizations and discussions on the steep** \n\n\nThank you for pointing this out. We agree that visualizing the steeps during optimization is beneficial. To illustrate this, we plotted the relevance scores for one sample from the FashionIQ dataset at two stages: before training and after the warm-up training. Please refer to the following link:\n\n\n### **<https://anonymous.4open.science/r/QuRe-ICML-Rebuttal/QuRe_steep_one.png>**\n\n\nWe defined the hard negatives as the images between the red and green lines. By selecting the top-2 largest drops in relevance score following the target, we observed steep degradations immediately before the red line and after the green line. These steep drops indicate substantial decreases in relevance[r2], suggesting that the selected boundaries effectively separate false negatives, hard negatives, and easy negatives.\n\nAs the above figure shows only a single query, we also aggregated the results across all queries, which can be found here:\n\n\n### **<https://anonymous.4open.science/r/QuRe-ICML-Rebuttal/QuRe_steep_agg.png>**\n\n\nThis aggregated visualization demonstrates that after the warm-up phase, the hard negatives shift toward higher ranks, likely capturing more true hard negatives. This supports our design choice of including a warm-up stage before defining hard negatives. Without a warm-up, the hard negative set would contain many easy negatives, as shown in the left figure.\n\n\n[r2] Xia, Peng, et al. \"Mmed-rag: Versatile multimodal rag system for medical vision language models.\" ICLR 2025.\n\n\n**Q3 : Apply QuRe on a wide range of the datasets beyond the HP-FashionIQ and CIRR** \n\n\nWe evaluated our method on four datasets: CIRR, FashionIQ, HP-FashionIQ, and CIRCO. CIRR and FashionIQ are two representative datasets in CIR, where CIRR covers general domains (e.g., people, animals, food) and FashionIQ focuses on fashion-related queries (e.g., shirt, dress, etc). We agree that evaluating on a broader range of datasets would further validate our approach. To this end, we created the HP-FashionIQ dataset to better capture human preferences. We additionally evaluated on CIRCO, a dataset derived from COCO, which enables relevance-based retrieval evaluation using mAP in a zero-shot setting.\n\n\n**Q4 : How to make sure the quality of the ranking and steeps especially at the initial stage?** \n\n\nIf the model is not sufficiently trained and the relevance scores are not reliable, our proposed method, which defines the hard negative set based on relevance scores, might produce inaccurate results. To address this, we include a warm-up phase where the model is trained by sampling negatives from the entire corpus without defining a hard negative set. As shown in $\\text{\\color{red}\\textbf{Figure 4}}$, the model achieves comparable performance even when negatives are sampled from the full corpus (\"All corpus\"). Additionally, the **QuRe_steep_agg.png** visualization in $\\text{\\color{blue}\\textbf{Q2}}$ demonstrates that defining the hard negative set without warm-up training tends to select easy negatives, which undermines the effectiveness of hard negative mining.\n\nWe will revise the manuscript to reflect the points raised in this rebuttal. We hope our responses and the novelty of our contributions have adequately addressed your concerns. Should any issues remain, we are happy to provide further clarification.", "claims_and_evidence": "The claims could be problematic. See weakness.", "methods_and_evaluation_criteria": "The method makes sense.", "theoretical_claims": "No proofs needed.", "experimental_designs_or_analyses": "The experimental designs could be biased. See weakness.", "relation_to_broader_scientific_literature": "Could be benificial to multimodal learning.", "essential_references_not_discussed": "The related works are essential.", "other_strengths_and_weaknesses": "The motivation of the proposed method could be problematic. The proposed method challenges the data labeling of existing benchmark datasets – existing benchmark datasets can mislabel relevant images as the false negative ones and thus could be highly biased. In this case, detecting and prioritizing hard negatives could be beneficial.  Yet, there lacks justification and thorough study to this motivation. \n\nIt seems that no visualizations and discussions on the steeps during the optimization and for different datasets.", "other_comments_or_suggestions": "Another strategy to validate the effectiveness of the proposed motivation is to apply the proposed method on a wide range of the datasets beyond the HP-FashionIQ and CIRR. Is the motivation to take advantage of the inherent bias of above datasets?", "questions_for_authors": "The proposed method remains unclear to me. The proposed  method monitors the steeps to distinguish between different levels of the negatives. Yet, how to make sure the quality of the ranking and steeps during optimization, especially at the initial stage? More illustrations or empirical evidence expected.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "768AMGEEQf", "reviewer": "Reviewer_6GLy", "summary": "The paper introduces the QURE algorithm, leveraging the BLIP-2 framework and a Hard Negative Sampling strategy to address the challenges in Cross-Image Retrieval (CIR). The novel approach is demonstrated using a custom dataset, HP-FashionIQ. While the approach is innovative and effectively addresses key pain points, the paper does not provide ablation studies for the BLIP-2 model, making it difficult to discern the individual contributions of the model enhancements from the sampling strategy", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer 6GLy for recognizing the contributions of our work and for providing valuable suggestions.\n\n\n**Q1 : Mathematical justification for the Hard Negative Sampling strategy** \n\n\nWe re-emphasize an important limitation in CIR datasets. Since only one or a few target images are annotated per query, **it is not feasible to accurately determine which images are false, hard, or easy negatives**. Hence, we first provide a theoretical justification for our approach, which is why defining the hard negative set and sampling from it in each epoch is effective in our scenario. The justification shows sampling negative from the hard negative set yields a higher expected loss than sampling from all corpus, thereby guiding the model to focus more on the defined hard negatives during training. Please refer to the following link:\n\n\n### **<https://anonymous.4open.science/r/QuRe-ICML-Rebuttal/QuRe_Justification.pdf>**\n\n\nHowever, if the defined hard negative set is improperly constructed and includes false negatives, it can misguide training. In $\\text{\\color{red}Figure 4}$, we observe a clear performance improvement immediately after introducing the hard negative set, compared to sampling from the full corpus. Conversely, sampling from the Top-K set, which contains more false negatives than our defined set, leads to performance degradation.\n\n\n**Q2 : Ablation studies for the BLIP-2 model architecture** \n\n\nWe conducted additional experiments using both BLIP and BLIP-2 model structures. Specifically, we trained QuRe with the BLIP backbone to compare with Bi-BLIP4CIR. We also identified the latest version of CoVR-BLIP, CoVR-2[r1], which uses BLIP-2, and compared it with our original QuRe model. We report results on the CIRR, FashionIQ, HP-FashionIQ, and CIRCO datasets. Notably, the current state-of-the-art method SPRC originally used BLIP-2, and QuRe already outperformed it in our original submission.\n\n\n| CIRR | backbone | Recall@1 | Recall@5 | Recall@10 | Recall@50 | Recall s@1 | Recall s@2 | Recall s@3 | Mean 5 + 1 |\n|:------------:|:--------:|:--------:|:--------:|:---------:|:---------:|:----------:|:----------:|:----------:|:----------:|\n| Bi-BLIP4CIR  | BLIP     | 32.55    | 64.36    | 76.53     | 91.61     | 63.54      | 82.46      | 92.48      | 63.95      |\n| **QuRe**     | **BLIP** | **51.52**| **80.29**| **88.89** | **97.74** | **78.02**  | **91.23**  | **96.55**  | **79.16**  |\n| CoVR-2       | BLIP-2   | 42.80    | 74.60    | 83.90     | 96.22     | 69.49      | 86.22      | 93.98      | 72.05      |\n| **QuRe**     | **BLIP-2**| **52.22**| **82.53**| **90.31** | **98.17** | **78.51**  | **91.28**  | **96.48**  | **80.52**  |\n\n\n\n\n| **FashionIQ** | **backbone** | **Dress - Recall@10** | **Dress - Recall@50** | **Shirt - Recall@10** | **Shirt - Recall@50** | **TopTee - Recall@10** | **TopTee - Recall@50** | **Recall@10** | **Recall@50** | **Mean** |\n|:---------------------:|:-------------------------:|:---------------------:|:---------------------:|:---------------------:|:---------------------:|:----------------------:|:----------------------:|:-------------:|:-------------:|:---------:|\n| Bi-BLIP4CIR | BLIP | 39.12 | 62.92 | 39.21 | 62.81 | 44.37 | 67.06 | 40.90 | 64.26 | 52.58 |\n| **QuRe** | **BLIP** | **40.80** | **64.90** | **45.93** | **65.90** | **52.07** | **72.87** | **46.27** | **67.89** | **57.08** |\n| CoVR-2 | BLIP-2 | 46.41 | 69.51 | 49.75 | 67.76 | 51.86 | 72.46 | 49.34 | 69.91 | 59.63 |\n| **QuRe** | **BLIP-2** | **46.80** | **69.81** | **53.53** | **72.87** | **57.47** | **77.77** | **52.60** | **73.48** | **63.04** |\n\n\n| **CIRCO** | **backbone** | **mAP@5** | **mAP@10** | **mAP@25** | **mAP@50** |\n|-------------------|:-------------------------:|-----------|------------|------------|------------|\n| Bi-BLIP4CIR | BLIP | 4.74 | 4.97 | 5.69 | 6.1 |\n| **QuRe** | **BLIP** | **20.85** | **21.48** | **23.35** | **24.31** |\n| CoVR-2 | BLIP-2 | 23.18 | 23.59 | 25.57 | 26.49 |\n| **QuRe** | **BLIP-2** | **23.22** | **24.23** | **26.26** | **27.24** |\n\n\n\n| **HP-FashionIQ** | **backbone** | **Preference Rate (%)** |\n|--------------------------|:-------------------------:|-------------------------|\n| Bi-BLIP4CIR | BLIP | 67.33 |\n| **QuRe** | **BLIP** | **75.28** |\n| CoVR-2 | BLIP-2 | 71.99 |\n| **QuRe** | **BLIP-2** | **74.55** |\n\n\n\nThe results show that QuRe with a BLIP backbone consistently outperforms Bi-BLIP4CIR. CoVR-2, which uses a BLIP-2 backbone, still underperforms our original QuRe with BLIP-2.\n\n\n[r1] Ventura, Lucas, et al. \"CoVR-2: Automatic Data Construction for Composed Video Retrieval.\" IEEE Transactions on Pattern Analysis and Machine Intelligence (2024).\n\n\nWe will revise the manuscript to incorporate the points addressed in this rebuttal. We hope our responses and the novelty of our contributions have sufficiently addressed your concerns. If there are any remaining issues or points that need further clarification, we would be more than happy to provide additional details.", "claims_and_evidence": "The effectiveness of the Hard Negative Sampling strategy is well-supported by empirical evidence. However, the paper lacks a crucial ablation study of the BLIP-2 framework, which is necessary to isolate and understand its specific contributions relative to the overall performance improvements claimed.", "methods_and_evaluation_criteria": "The QURE algorithm and Hard Negative Sampling approach are clearly articulated and address the identified issues in CIR tasks effectively. The validation on the HP-FashionIQ dataset appropriately benchmarks the model's performance. Nevertheless, the absence of BLIP-2 specific ablation studies undermines the ability to fully evaluate the method's effectiveness.", "theoretical_claims": "While practical in nature, the paper does not adequately justify the theoretical basis for the Hard Negative Sampling strategy's relevance score methodology. A rigorous mathematical or logical explanation is required to substantiate the claims made about the selection of negatives based on their relevance scores.", "experimental_designs_or_analyses": "The experimental design demonstrates a thorough understanding of the practical applications but lacks detailed ablation studies on the BLIP-2 model. This omission is critical as it hinders the clear differentiation of the effects of the model's capabilities from those of the sampling strategy.", "relation_to_broader_scientific_literature": "The paper positions itself well within the existing literature and effectively addresses significant challenges in the field. It brings innovative solutions to the forefront, though it could benefit from a more detailed discussion on theoretical frameworks related to negative sampling.", "essential_references_not_discussed": "The current literature review is sufficient but would be enhanced by including discussions on theoretical frameworks that specifically address the role and impact of relevance scores in negative sampling within similar contexts.", "other_strengths_and_weaknesses": "Strengths: \n1.the paper is well-written, and easy to understand.\n2.It effectively tackles key pain points in Cross-Image Retrieval tasks, notably through its novel Hard Negative Sampling strategy which improves model robustness\n3.The detailed analysis and clear visualizations of the Hard Negative Sampling strategy aid in understanding and highlight its practical impact.\nWeaknesses: \n1.the lack of ablation studies for the BLIP-2 framework\n2.insufficient theoretical justification for the relevance score methodology in Hard Negative Sampling are significant drawbacks", "other_comments_or_suggestions": "It is recommended that the authors include ablation studies for the BLIP-2 model to clarify its contributions and provide a detailed mathematical justification for the relevance-based hard negative selection process.", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7evQkOdLOI", "reviewer": "Reviewer_c7Tq", "summary": "This paper proposes a QURE method to retrieval the target image and mitigate the false negatives for the task of Composed Image Retrieval (CIR). The authors introduce a hard negative sampling strategy that select images positioned between two sharp relevance score drops after the target to filter false negative. The paper also introduces human-annotated HP-FashionIQ dataset that explicitly captures user preferences beyond target retrieval. Experiments on the FashionIQ, CIRR and CIRCO datasets validate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer c7Tq for the positive feedback and for recognizing the novelty of our work in both the proposed method and the dataset.\n\n**Q1 : There may be cases where there are no hard-negative samples?** \n\n\nIt is true that the number of hard negatives can vary significantly depending on the query (e.g., if the user asks for a \"blue short-sleeve t-shirt,\" then \"blue long-sleeve shirts\" might be considered hard negatives) and the available candidate images in the corpus (e.g., if there are no blue shirts, no hard negatives might exist). However, identifying true hard negatives for all queries is impractical in both real-world applications and existing CIR datasets. Instead, we empirically demonstrate that QuRe improves model performance by selecting **relatively** hard negatives from the corpus.\n\n\n**Q2 : Ablation studies for the BLIP-2 model architecture** \n\n\nWe conducted additional experiments using both BLIP and BLIP-2 architectures to ensure a fair comparison between QuRe and other baselines. Specifically, we trained QuRe on the BLIP backbone to compare with Bi-BLIP4CIR. For CoVR, we identified its latest version, CoVR-2[r1], which adopts BLIP-2 as its backbone, and compared it with our original QuRe model. Notably, the current state-of-the-art method SPRC originally used BLIP-2, and QuRe already outperformed it in our original submission. The results are presented across the following datasets:\n\n\n- **CIRR**: QuRe outperforms Bi-BLIP4CIR and CoVR-2 with average Recall improvements of **11.53** and **6.04**, respectively.\n- **FashionIQ**: QuRe outperforms Bi-BLIP4CIR and CoVR-2 with average Recall improvements of **4.50** and **3.41**, respectively.\n- **CIRCO**: QuRe outperforms Bi-BLIP4CIR and CoVR-2 with average mAP improvements of **17.12** and **0.53**, respectively.\n- **HP-FashionIQ**: QuRe outperforms Bi-BLIP4CIR and CoVR-2 with higher human preference rates of **7.95** and **2.56**, respectively.\n\n\nPlease refer to our response in $\\text{\\color{blue}\\textbf{Q2 under Reviewer 6GLy}}$ for detailed results.\n\n\n\n**Q3 : Report on the improvement of false negative results** \n\n\nWe agree that providing additional qualitative results, similar to $\\text{\\color{red}\\textbf{Figure 6}}$, helps illustrate the improvements in handling false negatives. Using the given query, we extracted the top 4 retrieved results from Bi-BLIP4CIR, CoVR-BLIP, SPRC, and our method, QuRe. The results are shown in the following figure:\n\n\n### **<https://anonymous.4open.science/r/QuRe-ICML-Rebuttal/QuRe_FalseNegatives.png>**\n\nAs shown, QuRe successfully retrieves relevant t-shirts that match the query attributes, such as navy blue color (close to black, like the query image) and the presence of a chicken or bird in the center. In contrast, other baselines retrieve less relevant images, such as those in light blue or without the chicken/bird. Such irrelevant results can lead to user dissatisfaction, as user satisfaction is generally proportional to the number of relevant items in the retrieved set. These results highlight the effectiveness of QuRe in assigning higher scores to relevant images by explicitly addressing false and easy negatives during training.\n\n**Q4 : Add target image and novelty of the method in Figure 1** \n\n\nThank you for pointing this out. We agree that highlighting the target image for each query in $\\text{\\color{red}\\textbf{Figure 1}}$ would improve clarity and overall understanding. Additionally, we will include the $\\text{\\color{blue}\\textbf{figure provided in Q3}}$ ([**QuRe_FalseNegatives.png**](https://anonymous.4open.science/r/QuRe-ICML-Rebuttal/QuRe_FalseNegatives.png)) in the appendix to illustrate the effectiveness of our method, QuRe, compared with existing baselines.\n\n\n**Q5 : How is sharpness defined, and what if multiple sharp drops exist?** \n\n\nTo determine whether the drop is sharp, we compute the relevance scores of all candidate images for each query. We then sort the scores and identify the top-2 largest drops occurring after the target. The corresponding images are selected as the hard negative set. An example of these steep degradation patterns is shown in $\\text{\\color{blue}\\textbf{Q2 under Reviewer 13Df}}$ ([**QuRe_steep_one.png**](https://anonymous.4open.science/r/QuRe-ICML-Rebuttal/QuRe_steep_one.png)). While there may be multiple sharp drops, we believe that the top-2 degradations after the target likely represent the boundaries between false negatives, hard negatives, and easy negatives. We agree that more advanced approaches to identifying such transitions could be explored in the future. We will include this as a discussion point in the final version of the paper. \n\n\nWe will update the manuscript to incorporate the feedback discussed in this rebuttal. We sincerely hope that our responses, along with the originality of our contributions, have addressed your concerns. If any questions remain, we would be glad to offer further clarification.", "claims_and_evidence": "The article is well written, there are no typos or spelling errors, and the arguments are well stated. The state-of-the-art and related topics are comprehensive. However, it is suggested to supplement some improved experiments about false negative results to facilitate readers' understanding.", "methods_and_evaluation_criteria": "The idea of QURE is novel and the HP-FashionIQ dataset provides a new evaluation metric for CIR. This contribution looks like alleviating the problem of false negative results in CIR research.", "theoretical_claims": "I'm not sure if the hard-negative sampling strategy can be used to filter out false negatives for all queries. There may be cases where there are no hard-negative samples?", "experimental_designs_or_analyses": "(1) Tables 2 and 3 compare different methods to the paper result (QURE). As described in Section 5, the authors use the BLIP-2 for the CIR task, but not all of the methods shown in table use BLIP-2. This makes it questionable whether much of the improvement in model performance is due to the introduction of BLIP-2.\n\n(2) The authors show in Figure 6 the selection of the QURE method for hard-negative samples, but there is no report on the improvement of false negative results.", "relation_to_broader_scientific_literature": "The paper alleviates the problem of false negative results in the CIR dataset (FashionIQ) with a hard negative sampling strategy, which is one of the research directions in CIR.", "essential_references_not_discussed": "The paper is well-written and fully cited.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The idea of a hard negative sampling strategy is good.\n2. The creation of the HP-FashionIQ dataset is a valuable contribution to the CIR field. It addresses the core limitations of current CIR evaluation metrics by focusing on user preferences rather than just retrieving target images.\n3. The paper shows SoTA performance on the main CIR datasets.\n\nWeaknesses:\n1. The paper does not sufficiently report the improvement of the false negative of the proposed approach.", "other_comments_or_suggestions": "The example provided in figure 1 is helpful, but it would be beneficial to highlight the target image and the novelty of method for clarity. This is important for readers who may not be familiar with CIR.", "questions_for_authors": "The paper uses candidate images between two correlation sharp drops as hard negative samples is novel, but I am confused about how to determine whether it is a sharp drop or not. In addition, whether there will be some queries with multiple sharp drops affects the selection of hard negative samples.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes a retrieval framework that improves composed image retrieval by introducing a hard negative sampling strategy based on steep drops in relevance scores. It also presents HP-FashionIQ, a new benchmark for evaluating alignment with human satisfaction. The reviewers find the motivation clear and the method well-executed, with good empirical performance across multiple CIR datasets. While some concerns are raised regarding in-depth discussion, theoretical basis, architectural ablations, etc, the authors provide additional experiments and visualizations that help clarify these aspects. Several questions remain regarding the technical novelty and the generalizability of the method, as the focus is primarily on fashion items. Overall, the paper offers certain contribution to the CIR community.
Decision: Accept (poster)