TOMCAT: Test-time Comprehensive Knowledge Accumulation for Compositional Zero-Shot Learning
Authors: Xudong Yan
Songhe Feng
Venue: NeurIPS 2025 poster ()
Keywords: Compositional Zero-Shot Learning
Compositionality
Visual-Attribute
Label Space Shift
Primary area: applications
TL;DR: We propose to accumulate multimodal knowledge to overcome the label distribution shift caused by unseen compositions recombined from attributes and objects by leveraging unsupervised data at test time.

Abstract: Compositional Zero-Shot Learning (CZSL) aims to recognize novel attribute-object compositions based on the knowledge learned from seen ones. 
Existing methods suffer from performance degradation caused by the distribution shift of label space at test time, which stems from the inclusion of unseen compositions recombined from attributes and objects. 
To overcome the challenge, we propose a novel approach that accumulates comprehensive knowledge in both textual and visual modalities from unsupervised data to update multimodal prototypes at test time. Building on this, we further design an adaptive update weight to control the degree of prototype adjustment, enabling the model to flexibly adapt to distribution shift during testing. Moreover, a dynamic priority queue is introduced that stores high-confidence images to acquire visual knowledge from historical images for inference. Considering the semantic consistency of multimodal knowledge, we align textual and visual prototypes by multimodal collaborative representation learning. Extensive experiments indicate that our approach achieves state-of-the-art performance on four benchmark datasets under both closed-world and open-world settings. 
Code will be available at [https://github.com/xud-yan/TOMCAT](https://github.com/xud-yan/TOMCAT).

TOMCAT Test-time Comprehensive Knowledge A cccumulation for Compositional Zero-Shot Learning
Xudong Yan 1.2 Songhe Feng 1.2
School of Computer Science and Technology, Beijing Jiaotong University Key Laboratory of Big Data and Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education {xud_yan, shfeng)lbjtu. edu.
Abstract
Compositional Zero-Shot Learning (CZSL) aims recognize novel attribute- object compositions based on the knowledge learned from seen ones. Existing methods suffer from performance degradation caused by the distribution shift of label space at test time, which stems from the inclusion of unseen compositions recombined from attributes and objects. To overcome the challenge, we propose a novel approach that accumulates comprehensive knowledge in both textual and visual modalities from unsupervised data update multimodal prototypes at test time. Building on this, we further design an adaptive update weight to control the degree of prototype adjustment, enabling the model to flexibly adapt to distribution shift during testing. Moreover, dynamic priority queue is introduced that stores nigh-confidence images acquire visual knowledge from historical images for inference. Considering the semantic consistency of multimodal knowledge, we align textual and visual prototypes by multimodal collaborative representation learning. Extensive experiments indicate that our approach achieves state-of-the-ate performance on four benchmark datasets under both closed- world and open-world world settings. Code will be available at https: :// //gi github. com/ com/xud-aa yand TTMCTT.
1 Introduction
Integrating familiar concepts and expanding novel compositions stand as one the striking hall- marks of human intelligence [9, 10, 24J. For example, even when confronted with completely new composition browned cheese, humans can swiftly apprehend and recognize it by combining known atomic concepts browned and cheese. These capacities for compositionality and generalization enable us to infer a wide range of combinations from limited atomic concepts [1], and have spurred the emergence of Compositional Zero-Shot Learning (CZSL) [39, 46, 2, 26, 35]. Specifically, CZSL aims to recognize unseen attribute-object compositions based on the primitive (i.e., attributes and objects) knowledge learned from ones [60].
Traditional CZSL approaches either project both images and attribute-object labels into a joint space to learn compositional representations [40, 36, 22]. use two classifiers to predict attributes and objects separately [50, 25, 13). Recently, benefiting from the impressive multimodal representational abilities of large pre-trained vision-language models (VLMs) like CLIP [49], there have been further significant advancements in CZSL [42, 34, 14, 28]. These methods employ the prompt tuning technique [73], i.e., replacing the hard prompt like "a photo of [attribute] [object]] with learnable soft prompt tokens to align images and textual composition labels, thereby mun-tuning
VLMs for adaptation the CZSL task.
Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

test Image
Priority
Visua Proterypes
test image
Inference Based on Multimodal Prototypas
Encoder
photo of ald cae jeounng cot.
Terial Protonypas
Text Encoder
yesing
Encoder
wrong prediction
distritttion
Ihage dog
right prediction
Figure 1: At test time, existing methods (left) fail leverage test images for model updates, obtaining wrong prediction distribution caused by label space shift, whereas our TOMCAT (right) accumulates multimodal knowledge from unsupervised data at test time to overcome the challenge.
While these pioneering methods have achieved significant progress, they fail to address the per- formance degradation due to the distribution shift of label space at test time. Specifically, models are assigned to identify unseen attribute-object compositions that are absent during training. This pattern leads to mismatch between the learned and actual test-time label distribution, bringing about inaccurate predictions and poor generalization. The key reason why the above issue has yet to be effectively addressed lies in the fact that the model parameters and class prototypes frozen after training, preventing the model from leveraging the test data to adapt towards the new distribution.
However, in more real-world scenarios, an outstanding intelligent system should continuously accumulate knowledge and thus overcome distribution changes after deployment by utilizing the unlabeled data provided through user interactions during the testing phase. In CZSL, the following three aspects should taken into consideration when using test-time unsupervised data to adjust the model: (1) Accumulation. The process of observing test-time images can be regarded as form of knowledge accumulation, rather than a complete adaptation to unseen compositions at the cost of forgetting knowledge of seen compositions learned during training [23, 43, 58]. (2) Knowledge comprehensiveness. Existing methods treat compositions encoded by the text encoder of VLMs textual-moda modal prototypes, and predict compositions by computing similarity between the visual features of images and prototypes. However, they overlook the potential of utilizing visual information in the historical images encountered at test time. (3) Efficiency. Considering the practical scenario of interaction with users, the method should be time-efficient with low latency [43, 29, 7J.
To this end, we propose Test-time cOMprehensive knowledge aCcumulATion (TOMCAT) ap- proach for CZSL, a novel framework that leverages multimodal knowledge of unlabeled data at test time overcome label distribution shift, illustrated in Fig. 1. During the test phase, we keep the textual-modal prototypes frozen and progressively learn a Knowledge Accumulation Module (KAM) for prototype adjustment to bridge the label distribution gap with the continual influx of test samples. Subsequently, we determine the extent to which KAM updates the prototypes by employing a de- signed adaptive update weight strategy, based similarities between the image and the prototypes. To take full advantage of the visual knowledge from previously seen images, TOMCAT maintains a dynamic priority queue to store high-confidence images for each class. Building this, visual-moda. prototypes are constructed from the images stored in the queue and are dynamically updated-similarly to the textual counterpart-by visual KAM with higher-confidence images enqueued. As testing pro- gresses, the textual and visual prototypes are updated by the entropy minimization objective [54, 52), and are jointly used to facilitate composition recognition under the new label distribution. addition, given the inherent semantic interdependence between multimodal knowledge, we align textual and visual prototypes by multimodal collaborative representation learning.
Notably, in practical applications where user interaction imposes high requirements on time efficiency, keep the original model frozen during inference and optimize only the parameters of KAMs via gradient backpropagation once the prediction for a test sample is completed. Meanwhile, extensive experiments on four benchmark datasets (i.e., UT-Zappos [61], MIT-States [15], C-GQA [40], and Clothing 16K [68]) demonstrate that our TOMCAT outperforms the state-of-the-art by significant margins in both doosddwwrrd and open-world settings (Sec. 3.1..
In summary, the main contributions of our work are three-fold:
We propose TOMCAT, novel framework that accumulates multimodal knowledge from unlabeled data and updates prototypes at test time to bridge the label distribution shift. To the best of our knowledge, we are the first to leverage unsupervised data test time improve models in CZSL.

---

TOMCAT adopts a priority queue that stores historical high- ceghhconfiee images to calculate visual prototypes and adaptively updates multimodal prototypes by knowledge accumulation modules. Extensive experiments conducted on four benchmark datasets demonstrate that our TOMCAT achieves state-of-the performance in both closed world and open-world settings.
2 Related Work
Compositional Zero-Shot Learning (CZSL). CZSL requires the model to recognize unseen com- positions with the attribute and object knowledge learned from seen compositions. Previous works in CZSL can be broadly divided into two main streams. One main stream aims at learning repre- sentations of compositions by aligning images and textual labels in a shared space, and predicting compositions with the lowest distance [39, 41, 46, 40, 36, 22]. The other stream concentrates on disentangling visual representations of compositions and predicting individual primitives separately to reduce composition learning into primitive learning since both seen and unseen compositions inherently share the same attributes and objects [50, 27, 51, 56, 33, 71]. Rather than learning image- composition association from scratch, recent approaches have increasingly shifted focus to exploiting the multimodal representational capacity of VLMs (e.g., CLIP (49]) for CZSL [42, 34, 14, 28, 3, 59]. For example, Troika [14] exploits multi-path paradigm and cross-modal traction modules to jointly model attributes, objects, and compositions. CDS-CZSL [28] leverages context- based diversity- driven specificity learning prioritize specific attributes with richer information. ClusPro [47] learns multi-prototyy by within- within-primitive clustering and dynamically updates them. Although notable progress has been made, these methods freeze model parameters and prototypes after training, which hinders them from using test-time data for further improvement.
Vision-Language Model (VLM). VLMs (e.g., CLIP [49] and ALIGN [16]) pre-trained on web-scale datasets have recently attracted considerable attention due to their impressive capability in aligning visual and textual modalities. Current approaches typically seek to repurpose the multimodal ability of VLMs for various downstream tasks through prompt tuning or adapter tuning. On the text side, prompt tuning is a parameter-efficient technique that replaces manually crafted prompts with sequence of Iearnable soft tokens while keeping the text encoder frozen [73, 72, 74J. For example, CoOp [73] eliminates the requirement of manually crafting prompts by prompt tuning under the few-shot setting. CoCoOp [72] extends CoOp by introducing instance- wise conditional prompt learning. On the visual side, adapter tuning refers to injecting lightweight trainable modules (e-g., residual adapters and attention- based adapters) into the visual encoder without fine-tuning the entire backbone [62, 8, 11, 48]. For instance, AdapterFormer [5] augments each Transformer block of the visual backbone with a parallel adapter module and fuses their outputs via element-wise addition. In this work, following CoOp [73] and AdapterFormer [5], we fine-tune CLIP using training data to obtain a simple base model for subsequent testing.
Online Test-time Adaptation (OTTA). OTTA refers to a practical technique where trained model continually adapts itself by exploiting a stream of unsupervised test data-cach test sample is used exactly once, and the model is required to retain and leverage the knowledge gained from earlier test images to improve its performance on later ones in an online manner [54, 55, 44, 32, 4, 6, 30]. For example, Tent [54] proposes to optimize batch normalization layer by minimizing prediction entropy on test data. SAR [44] enhances generalization ability in OTTA by eliminating partially noisy samples with high gradients and promoting the convergence of model weights flat minimum. CoTTA [55] is the first work to decrease the accumulation error and avoid catastrophic forgetting through using averaged pseudo-labels and retaining the knowledge of the original model enhance long-term adaptation. Recently, activating the zero-shot capability of VI.Ms at test time to mitigate domain shift in downstream tasks has increasingly attracted significant research attention [52. 19, 63, 66, 64, 53]. For instance, TPT [52] develops a prompt tuning approach that learns adaptive prompts on the fly using each test sample with its augmented views. TDA [19] proposes a training- uses lightweight key-value cache and progressive pseudo-label refinement without backpropagation. DynaPrompt [57] adaptively selects and optimizes the relevant prompts for each test built on an online prompt buffer. TPS [53] proposes a straightforward and efficient prototype shifting approach that adjusts per-class prototypes within the embedding space. Notably, OTTA mainly focuses on addressing distribution shifts in the feature domain while we aim to resolve the challenge of label distribution shift caused by unseen compositions recombined from attributes and objects in CZSL.. The CZSL task does not have access any data of unseen compositions during training.
adapter that
3

---

Priority Queue
Lre (Eq.
huge dog
(b) mon
Textual KAM AGAA ArAA ASAA
AUW
AUW
Adaptive Update Weights (AUW)
young
Encoder
Signald
buge dog
Figure 2: The overall architecture of our proposed TOMCAT at test time. The model accumulates multimodal knowledge to update prototypes to overcome the label distribution shift.
3 Method
3.1 Task Formulation
In this section, we provide a formal description of the CZSL task. Given an attribute set A and an object set O, the composition set can be defined by the Cartesian product of A and O,i.e., === AxO. And the seen and unseen composition set C and C" are two disjoint subsets of i.e.e. C* nC" Ø. During the training phase, the model learns from a seen training where X' is the image space, and c the composition label of image In the closed setting [46]. the test composition set C is defined as the union of C and C" i.e. C C C", where only the presupposed known composition space is considered. For the open-world setting [35], the test composition set expands to all possible attribute-object pairs, i.e., C C.
D {(x, E X' C*).
3.2 Training Phase
As a preparatory step, we train a simple CLIP- model by adjusting only its visual and text encoders the training set, which serves the foundation for subsequent testing.
Textual Representation Extraction. CLIP treats each attribute-object composition label as prompt-based textual input, i.e., "a photo of [attribute] [object]", which is subsequently encoded via the text encoder 1b. Given composition c = (a, o). we create a learnable prompt p . Pt, w w*], where P represents the learnable prompt tokens. w and w are the learnable word tokens and o within the attribute and object vocabularies, respectively. Therefore, the textual representation of composition is obtained, i.e.. f (PP)).
Visual Representation Extraction. Given an input image x Nx x3 we feed it into the CLIP visual encoder & and employ the output [CLS] token as its visual representation, i.e., f ((x). Based on AdapterFormer [5], a set of learnable adapters injected into the visual encoder of CLIP, while keeping the initial parameters frozen. Refer to Appendix A for more details.
Training Objective. Given the textual and visual representations, we compute the probability p(c[#) and use the cross-entropy loss to align them:
exp(cos(f". .f.)/
Lir - log p(c)æ). p(c]x) " exp(cos( f2)T)
(1)
where denotes the temperature parameter of pre-trained CLIP and cos(..)) is used to compute cosine similarity, After the training phase, simple base model is obtained by only tuning adapters and prompt within CLIP, without introducing any additional complex modules to accelerate inference.
3.3 TOMCAT at Test Time
As the major novelty, we introduce TOMCAT to overcome the challenge of label distribution shift by employing unsupervised data at test time, shown in Fig. 2. Specifically, textual and visual

---

composition prototypes are obtained by CLIP-encoded textual labels and dynamic priority queue of historical images, respectively. These multimodal prototypes are then updated by Knowledge Accumulation Modules (KAM) and adaptive update weights, with the objective of minimizing prediction entropy at test time.
Textual Prototype Construction. In line with CLIP's principles, we treat the embeddings of both seen and unseen composition labels-encoded by the text encoder of the base model after training-as the textual-modal prototypes, ie., [Le Lzz be tggo T E IR [=
Visual Prototype Construction. Inspired by TDA [19] and DPE [63], complementary the text modality, we recognize that prior visual knowledge-capture leveraged to further enhance the discriminative capability of CLIP. Therefore, a dynamic priority queue is designed to selectively preserve K (K" 3) high-confidence images, which enables TOMCAT to retain representative and reliable exemplars throughout testing. Specifically, for each seen and unseen composition we maintain a "confalence-feature" queue, i.e., of [(hs, J { where f:" E I$ is the visual feature of the image z; in the queue and h; E R is its prediction entropy as confidence:
from historical test images-can be
exp(cos(Jf", (.)/T) H(r)) = Σ plclu,, p(cks,)) p(clar, i) Edecie exp(cos( (f. /
(2)
cEC
where i is the updated textual prototypes (defined below in Eq. 5). The lower prediction entropy means higher confidence, and the samples in the queue are sorted by their confidence, i.e,, hy S ((S)))
The priority queue corresponding to each composition is initialized as an empty set 2. For an incoming image x, we first calculate its visual feature f" and the prediction entropy h, and obtain its pseudo-label Cp by assigning the composition with the highest predicted probability, i... Cp = arg max.gC** p(c|2r, t). If the queue for Cp is not full, the pair (h, l") is directly inserted into the queue. If the queue is full, only if the new image has prediction entropy lower than the highest element in the queue, the highest one is replaced with (h, f"); otherwise, the queue remains unchanged. Based on the priority queue, the visual-modal prototype for each composition is computed by averaging the visual features, i.e., Vo = + Σ Sr. Subsequently, the visual prototypes are denoted as v [Voy Vogi..., Vejooe IT E IR [xddd"
Knowledge Accumulation Module (KAM). We aim to continuously acquire information about the new distribution from test samples, while avoiding both catastrophic forgetting of existing knowledge and excessive increases in inference latency. To this end, we introduce learnable KAMs instead of directly modifying the parameters of the base model. and employ adaptive update weights to control the extent to which the KAMs adjust the prototypes. Specifically, for multimodal prototypes t and v, multimodal KAMs consist of two sets of learnable parameters, which are initialized to zero:
At Atee Atc., Ate 1 Rlo Av= [Aver: Avey....,
Avgote) l_ R|o (3)
Adaptive update weight. Taking textual prototypes as an example, we demonstrate the process by which the prototypes are updated with newly arrived image Specifically, the cosine similarity is computed between the visual feature fr of the image and each original prototype tc. based on which the adaptive update weight is calculated as follows:
We = 0-00 8c), 8. =z= cos(f, t.))
where denotes the Sigmoid activation function and 0 is the hyperparameter that controls the degree of update. Therefore, the updated textual prototypes can be denoted as follows:
t + w.
i = [tc, Eez Eojett JT, E.
(5)
t + w. w.Atell
Accordingly, we can obtain the updated visual prototypes V. This adaptive weighting mechanism enables more controlled updates of the prototypes, thereby avoiding treating all compositions equally regardless of familiarity. Intuitively, when the test image closely matches the original prototype, it is likely associated with seen composition, and thus excessive adjustments should be avoided. Conversely, large discrepancy between the test image and the prototype suggests a potentially unseen composition, permitting stronger updates to improve adaptability.
5

---

Prediction Entropy Minimization. Following Tip-Adapter [65], the final prediction for the input image is determined as follows:
exp (f" Ee + aAlf", :ee)
p(cl,r, (,l,, Edeecte exp (f to (a(ff,,ee)) A(f".v.) e- v.)) (6)
where a and are hyperparameters controlling multimodal balance and modulating visual-modal sharpness, respectively. At test time, minimizing prediction entropy serves unsupervised learning signal that encourages the model to produce more confident predictions in the target label space, thereby enhancing generalization. By reducing uncertainty, the model progressively adjusts to better aligning with the test distribution, which includes unseen compositions. Further explanations provided in Appendix B. The loss for multimodal prediction entropy is formulated follows:
LP Execie p(c|,, te, Ve) poga(, Le; Ve).
Multimodal Collaborative Representation Learning. Considering the intrinsic semantic interde- pendence between multimodal knowledge, we align the textual and visual prototypes by employing multimodal collaborative representation learning. This strategy effectively facilitates the integration of both modalities, thereby enhancing the representation of both textual and visual information in a unified framework. Specifically, contrastive learning is exploited to bring the visual and textual prototypes corresponding to the same composition closer while pushing apart non-corresponding ones, as formulated below:
exp (cos(te, vc)/T)
exp (cos( Ve)/T)
LMCRL = 2
Σ CEC
log
log Eeecre exp (cus(te' vo)/T)
Le'eote exp (te V///T)
(8)
Upon receiving a test image. first extract its visual feature and compute the prediction entropy with the original textual prototypes. We then determine whether to update the priority queue of its weedo-composition label based on the entropy. After performing multi- modal prototype refinement by KAMs and adaptive update weights, the final inference prediction for this image is obtained. To reduce latency, the backpropagation update of KAMs is deferred until after the inference step by minimizing the total loss of multimodal prediction entropy and multimodal collaborative representation learning follows:
Testing Pipeline Overview.
CTOMCAT = Lp + ALMORL
(9)
where À is the weighting coefficient of LMCRL.
4 Experiment
4.1 Experiment Setup
Datasets. Our proposed TOMCAT is evaluated on four commonly used datasets: UT-Zappos [61], MIT-States [15], C-GQA [40], and Clothing16K [68]. UT-Zappos and Clothing 16K are two fine- grained fashion datasets, whereas MIT-States and C-GQA consist of images depicting real-world objects. In addition, prior studies [2, 68, 60] have evidenced that MIT-States [15] suffers from considerable noise, with approximately 70% of the labels being incorrect. The detailed introduction and common data splits of the four datasets are presented in Appendix C.
Metrics.
Following the evaluation protocol of previous works [46, 35, 42], a bias term from -00 to +0o introduced trade off the prediction logits between seen and unseen compositions. By varying the bias term, we calculate the best Seen accuracy, best Unseen accuracy, best Harmonic Mean (HM) of seen and unseen accuracies [70]. and the Area Under the Curve (AUC) drawn with seen and unseen accuracies. In the open-world setting, post-training feasibility calibration is applied filter out infeasible compositions within vast search space [35].
Implementation Details. We implement the base model with CLIP ViT-L/14 architecture in the training phase and TOMCAT test time in PyTorch [45] framework on single NVIDIA RTX 3090 GPU, Refer to Appendix D for more implementation details. The source code will also be released at this website to provide all implementation details and thus facilitate reproducibility.
Baselines. We compare TOMCAT with recent and prominent approaches on UT-Zappos [61], MIT- States [15], and C-GQA [40], including CLIP [49], CoOp [73], CLiP-based Co-CGE [36], DFSP [34],

---

Table 1: bossed-worle and open-world results on UT-Zappos, MIT-States, and C-GQA. The best results are displayed in boldface, and the second- best results are underlined. The four indicators are explained in Metrics (Sec. 4. In the open-world setting, we report the results EDS-"ZSI" [28] using the same post-training feasibility calibration [35] as our TOMCAT and other baselines use.
UTP-aapos
MIT States
GQA
Methods
AUC
HM
Seen
Unseen
AUC
Seen
Unseen
AUC
Seen
Unseen
Closed- world
CLIP (CM222
5.0
15.6
15.8
49.1
11.0
26.1
30.2
46.0
1.4
7.5
25.0
CoOp [73] (0C222
18.8
34.6
52.1
49.3
13.5
29.8
34.4
47.6
4.4
17.1
20.5
26.8
Co-CGB [36] (7940)
36.3
49.7
63.4
71.3
17.0
33.1
46.7
45.9
5.7
18.9
34.1
21.2
CSP
33.0
46.6
64.2
66.2
36.3
46.6
49.9
6.2
28.8
26.8
DFSP [34] (2V
36.0
47.2
66.7
71.7
20.6
37.3
46.9
52.0
10.5
27.1
38.2
32.0
GIPCOL [59] (MNCC)224)
36.2
48.8
65.0
68.5
19.9
36.6
48.5
49.6
7.1
31.9
28.4
Troika (C19900
41.7
54.6
66.8
73.8
22.1
39.3
49.0
53.0
12.4
29.4
41.0
35.7
CDS- E---CZSL (28) (CW9913))
39.5
52.7
63.9
74.8
22.4
39.2
50.3
$2.9
11.1
28.1
38.3
34.2
PLID [3] (80C1124)
38.7
52.4
67.3
68.8
22.1
39.0
49.7
52.4
11.0
27.9
38.8
33.0
IMAX [17] (794M)25)
40.6
54.2
69.3
70.7
21.9
39.1
48.7
53.8
12.8
29.8
39.7
35.8
ClusPro [47] (CC8225)
46.6
58.5
70.7
76.0
23.8
40.7
$2.1
54.0
14.9
32.8
44.3
37.8
TOMCAT (Ours)
48.3
60.2
74.5
72.8
22.6
39.5
50.3
53.0
16.0
34.0
45.3
40.1
Open-world Results
CLIP [49] (CM222
2.2
11.2
15.7
20.6
3.0
12.8
30.1
14.3
0.3
4.0
7.5
4,6
CoDp [73] (2C122)
13.2
28.9
52.1
31.5
2.8
12.3
34.6
9.3
0.7
5.5
21.0
4.6
Co-CGB [36] (29M22
56.2
5.6
38.1
20.0
0.9
33.2
CSP [42] acue
22.7
38.9
64.1
44.1
5.7
17.4
46.3
15.7
1.2
6.9
28.7
5.2
DFSP [34] (c
30.3
44.0
66.8
60.0
6.8
19.3
47.5
18.5
2.4
10.4
38.3
7,2
GIPCOL [59] (WACC)22)
23.5
45.0
6.3
17.9
48.5
16.0
1.3
31.6
Troika [14] (CVP822))
33.0
47.8
66.4
61.2
7.2
20.1
48.8
18.7
2.7
10.9
40.8
7,9
CDS-CZSL" [28] (CV9922))
32.1
48.0
64.7
60.4
2.6
10.9
38.2
8.0
[3] (SCC))0))
55.5
IMAX [17] (7240025))
32.3
47.5
68.4
57.3
7.6
21.4
50.2
18.6
2.6
11.2
38.7
7.9
ClusPro [47] (cue)2)
39.5
54.1
71.0
66.2
9.3
23.0
51.2
22.1
3.0
11.6
41.6
TOMCAT (Ours)
43.7
57.9
74.1
65.8
8.2
21.7
49.2
21.0
4.2
14.2
45.1
10.6
Table 2: Closed-word and open- open-world world results on Clothing 16K.
Closed- World
Open-Workh World
Dataset
Methods
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
SymNet [26] (CVPR'20)
78.8
79.3
98.0
85.1
57.4
68.3
98.2
60.7
CompCos [35] (CVPR22)
90.3
87.2
98.5
96.8
64.1
70.8
98.2
69.8
Clothing l6K
CGE [40] (CVPX'2/)
89.2
84.2
98.0
97.4
62.0
68.3
98.5
69.7
Co-CGE [36] (7PAM/22)
88.3
94.7
69.2
98.7
SCEN [25]
78.8
78.5
98.0
89.6
53.7
61.5
96.7
62.3
INV [68] (ECCV/22)
90.6
86.6
99.0
97.0
63.6
72.0
98.7
69.0
OADis [S1] (CVPR122)
88.4
86.1
97.7
94.2
$3.4
63.2
98.0
58.6
ADR [13] (CVPR122)
92.4
88.7
98.2
97.7
68.0
74.2
99.0
73.1
CLPS [18] (2MM (25)
96.2
91.7
99.2
98.9
71.5
76.1
99.2
76.1
ATIF [38] (XMM)225)
96.6
95.4
99.4
92.0
99.2
93.9
ATF [38] (TMM3225)
98.2
96.3
99.5
99.6
89.9
90.3
99.7
92.2
TOMCAT (Ours)
99.5
98.4
99.9
100
95.8
95.0
100
96.4
GIPCOL [59], Troika [14], CDS-CZSL [28], PLID [3], IMAX [17], and ClusPro [47]. TOMCAT is also compared with some canonical methods that conduct their experiments Clothing [68], including SymNet [26], CompCos [35], CGE [40], Co-CGE [36], SCEN [25], INV [68], OADis [51],
ADE [13), CLPS [18], ATIF [38], and AIF [38].
4.2 Main Results
Closed-world results are presented in Table and Table 2. For two fine- ine-grained fashion datasets, our TOMCAT outperforms previous state-of-the-art methods with the improvement of AUC by 1.7%, 1.3%, and HM by 1.7%, 2.1 % respectively on UT-Zappos and Clothing16k Our model also surpasses baselines by 1.1% in AUC and 1.2% in HM on C-GQA. However, TOMCAT falls behind ClusPro and achieves second-bess performance MIT-States, which may be attributed to the substantial

---

Table 3: Abaltion study of our proposed modules on UT-Zappos and MIT-States. Queue means visual priority queue. T- and V. KAM denote textual and visual KAM. AUW is adaptive update weights.
Module
UT-7appos
MIT-States
Queue
T-KAM
V-KAM
AUW
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
43.57
55.54
68.72
74.30
22.12
38.97
49.50
52.61
43.28
55.54
68.43
74.14
22.12
39.15
49.45
52.88
45.68
58.22
74.39
69.01
22.18
39.22
49.50
52.95
43.28
55.54
68.43
74.14
22.32
39.32
49.54
52.93
46.64
58.83
76.44
68.38
22.34
39.39
49.58
52.93
48.31
74.99
72.77
22.55
39,45
50.32
52.95
Table 4: Ablation study of our designed loss on UT-Zappos and MIT-States.
Table 5: Influence of initialization strategies of KAMs on UT-Zappos and MIT-States.
Coss
UT-7appos
MIT States
UT-Zappos
MIT-Staaes
Initialization
Cpg
LMCRE
AUC
HM
AUC
HM
AUC
HM
AUC
HM
43.57
55.54
22.12
38.97
Uniform Random
43.49
56.23
21.81
38.59
44.59
57.29
22.35
39.32
Normal Random
45.50
57.98
21.32
38.36
42.46
53.97
22.29
39.42
Random Walking
47.08
59.12
22.14
39.16
48.31
60.18
22.55
39.45
All Zeros
48.31
60.18
22.55
39.45
Table 6: Influence test order on UT-Zappos, MIT-States, and C-GQA.
UT-7appos
MIT-States
C-GQA
Test Order
AUC
Scen
Unseen
AUC
HM
Scen
Unseen
AUC
HM
Soen
Unseen
Order
48.31
60.18
74.49
72.77
22.55
39.45
50.32
52.95
15.98
33.95
45.30
40.12
Order 2
48.52
60.82
74.64
72.95
22.46
39.33
49.84
53.05
15.71
33.75
45.02
39.95
Order 3
46.44
58.72
72.63
72.03
22.31
39.02
49.62
53.09
15.86
34.17
45.42
39.35
noise present in the dataset, leading to inaccurate label supervision and hindering the learning of discriminative features.
Open-world results are shown in Table and Table Our method performs much better than the second-best method by 4.2%, 1.2%, 4.0% in AUC, and 3.8%, 2.6%. 3,0% in HM UT-Zappos, C-GQA, and Clothing1668 16K, respectively, which indicates that TOMCAT has great potential in the open-world setting.
Discussion. TOMCAT achieves state-of-the-ae performance both closed-world and open-world world settings, demonstrating that our proposed method enables the model adapt the distribution shift of label space caused by unseen compositions during testing. Particularly in the open-world setting, TOMCAT shows significant improvement. This can be attributed to the fact that all test images are derived from feasible compositions (both seen and unseen ones), which reinforces our model's 's focus on the feasible compositions and reduces its attention to infeasible ones.
4.3 Ablation Study
In this section, we conduct extensive ablation studies to evaluate the contribution each component within proposed TOMCAT UT-Zappos, MIT-States, and C-GQA in the closed-won world setting.
Ablation Study of Main Modules. According to Table 3, for MIT-States, empirical results indicate that all proposed modules contribute to the performance improvement of TOMCAT, including the priority queue, multimodal KAMs, and the adaptive update weights. However, on UT-Zappos, incorporating the priority queue and the subsequent visual KAM leads to performance degradation compared the base model. This phenomenon is attributed the uniform style of shoes and the subtle inter-composition difference in this dataset, which makes the model insufficient to capture fine-grained semantic distinctions. In contrast, when textual KAM is added, it enables targeted refinement of the textual prototypes based on the stored images. Ultimately, the best performance is

---

49
AUC
HM
69
61
80
22.5
AUC
HM
58.4
38.6
48.24
48.4
AUC
60,0
22.60
39.4
68
48
66
57
22.4
38.0
38.8
47.8
58.0
39.3
65
1.5
Que Images
Queus Images
Upditt Contral Factor DAW
Upditte Control Factor DAW
(a) UT-Zappos
(b) MIT-States
(a) UT-Zappos
(b) MIT-States
Figure 3: Influence of the number of images in each priority queue K on two datasets.
Figure 4: Influence of the value of update control factor on two datasets.
Claceiiiation Accursoy
Base Model
TOMCAT
Glass Mfccation Acairacy
Model
TOMCAT
Clussification Accuracy
Base Model
TOMCAT
0.45
0.285
0.44
0.43
0.250
8.27
0.42
500
1800
1900 2000
2300
3000
5000
7600
10000
12600
1000
2000
5000
4600
5000
Test Sample Number
Test Sample Number
Test Sample Number
(a) UT-ZZappss
(b) MIT-States
(c) C-GQA
Figure 5: Trend of top-1 classification accuracy with increasing test sample size on three datasets.
achieved by TOMCAT with all modules, which indicates that all components are beneficial and that the multimodal KAMs and priority queue offer complementary advantages.
Ablation Study of Each Loss. As shown in Table 4, the improvement of adding Lps to the base model indicates that prediction entropy loss helps the model to adapt towards the new label distribution of seen and unseen compositions. However, using only the constraint term LMCRL may cause the model blindly reduce the distance between multimodal prototypes without considering other factors, thereby degrading the model's performance. Finally, by combining LPF and LMORL. TOMCAT achieves the best performance, which confirms effectiveness our algorithmic designs.
Influence of Different Initialization Strategies of Multimodal KAMs. Table 5 shows that the acro-initialization outperforms other random initialization strategies. The reason is that starting with random initialization at test-time causes the model to forget the knowledge gained during training.
Influence of Test Order. Since TOMCAT continually accumulates knowledge during testing, the test order may affect the results. We conduct three experiments with different random seeds to vary the sample order. The results in Table 6 suggest that there exists performance variance among different orders, although the differences are not statistically significant.
Influence of Hyperparameters. In Fig. 3 and Fig. 4, we explore the influence of the number of images stored in priority queue and the update control factor 0 respectively on UT-ZZapos and MIT- States. TOMCAT achieves the best performance when K 3 both datasets, small number of images causes instability in the visual prototypes, while a large number may include low-confidence images. Moreover, as 0 increases, the performance initially improves but subsequently deteriorates. Analysis reveals small ( leads to insufficient adaptation of the prototypes when encountering images with substantial variation, whereas large results in over-updating of the prototypes.
4.4 Qualitative Analysis
In this section, we conduct qualitative analysis of TOMCAT to visualize its predictions on UT- Zappos, MIT-States, and C-GQA in the closed-world setting.
Trend of Classification Accuracy. To verify whether TOMCAT continuously improves, we visualize Fig. how top-1 classification accuracy evolves with the number test samples on three datasets. At the beginning, TOMCAT performs comparably to the base model; however, as more test samples arrive, TOMCAT progressively demonstrates its superiority. The performance gap suggests TOMCAT continuously accumulates useful knowledge to overcome the distribution shift of the label space.

---

UT-Zappos
MIT-States
Satin
Lesther
Leather
OT: Whipped Me
G1: Folded Bike
GI: Frousn Ice
Sunin of
Scandli
Andle-Baott
v
Arite-Balis
You
Clarm: Whipped
Dure Foided like
Qure: Frozen Ice
V
of
of
of
wihipped CIT intinted Time
Foldod
lce
Scandals
Antie-Bodit
Antle-Rodts
GT: Surda
Slppers
d1: Synthatic
Clogs Shoes
di: Loather
Anéie koots
Cn Fresh Garli
Ranny Mud
Ounc Sunde o
Dure Symthetic
Gune Leather v
Inflated
Garlic
OursRurny Med
Skppers
Clogs Shocs
Antil
v
V
BMC LeatherXX
Stapers
aM leather x
Clogs
BM: Leather
X
DMte Painted Wheee
Durt: Large Candla
X
DM: Spilled Oett
X
Flass
Scandals
GI Fresh Rig
CTt Mohen Capper
Thowed Pizza
LoutersX
Flats Shoes
Heals
Durs: tipe Fig
X
Oun: Mohien Bronge
X
Oun: Thin Pusts
X
Leather
X
Slpperic
Plats Show
Heels
Figure 6: Case study on UT-Zappos and MIT-States. We compare TOMCAT (Ours) with the base model (BM) after training. The successful and failure results are marked in green and red, respectively.
chicken 0.28
0.21
0.18
0.19
0.18
0.19
0.20
chicken 0.29
0.21
0.19
0.21
0.20
0.18 0.18
-30
Protolyp Umilariiio
4.28
0.25
0.18
0.21
0.24
0.20
0.21
0.20
3.0
Vieual Prototypes
Visual Prototypes
pasta
0.10
0.16
4.24
Profotype
pasta
0.29
0.21
salad
0.19 0.19
0.19
0.25
0.20
0.21 0.20
4.29
salad
0.20
0.21
0.19
0.28
0.20
0.20
0.22
salmon
0.18 0.19 0.17 0.18
0.25 0.17 0.18
4.2
salmon 0.19 6.20
0.16 0.19
0.27 0.18
0.18
820
seafood
0.18
Waitt
seafood
vegetable 0.19
0.21 0.20 0.20 0.20 0.21 0.23
vegetable 0.20 0.22 0.21 6.22 0.21 0.20
0.25
chicko heat pasta salad
salmon seafood
vegetable
chicke
pasta salad salmon
seafood
vegutable
Textual Prototypes
Textual Prototypes
(a) Test sample size of 2500.
(b) Test sample size of 12000.
Figure 7: Similarity heatmap of multimodal prototypes on MIT-States. All unseen compositions consisting of the attribute cooked and its corresponding objects (e.g., chicken, meat...) are selected.
Successful and Failure Cases. In Fig. 6, we report top-1 predictions for some image cases. can be observed that benefiting from adapting to the new distribution, TOMCAT shows superior performance with higher accuracy. For the failure cases, the predictions of TOMCAT can also describe them; for instance, the image of satin sandals also be categorized satin heels.
Similarity Heatmap of Multimodal Prototypes. We visualize the cosine similarities between multimodal prototypes of all unseen compositions consisting of the attribute cooked and its corre- sponding objects in Fig. 7. Compared to the test size of 2500, the diagonal elements become larger at the test size of 12000, indicating that TOMCAT enhances the connection between multimodal information. Meanwhile, the increase in off-diagonal elements is attributed to all used compositions involving cooked food (almost in the same cluster), where larger similarities can better characterize their semantic structure.
5 Conclusion
In this work, we consider the issue of label distribution shift in CZSL, which arises from novel compositions recomposed from attributes and objects. Therefore, we propose TOMCAT to accumulate comprehensive knowledge of visual and textual modalities from unsupervised data test time to overcome the challenge. Specifically, we update textual and visual prototypes by multimodal knowledge accumulation modules, and use adaptive update weights control the degree. Meanwhile, priority queue is leveraged that stores historical images to obtain the visual knowledge. We hope that our work will inspire further research into exploring label space shift at test time in CZSL.
Limitation. While our proposed TOMCAT accumulates knowledge from test samples to overcome distribution shift of label space at time, two potential limitations are identified: (1) TOMCAT accumulates comprehensive knowledge only at the level of compositions, neglecting to exploit the information contained within each primitive. (2) Without proper initialization of the priority queue. the model is prone to bias toward the compositions of images already stored during the testing phase.
10

---

Acknowledgements
This work was supported by the Fundamental Research Funds for the Central Universities (2025YIS052 and 2025JBZX059), the Beijing Natural Science Foundation (4242046), and the Hebei Province Natural Science Foundation (F2025105018).
References
[1]
Yuval Atzmon, Jonathan Berant, Vahid Kezami, Amir Gloherson, and Gal Chechik. Learning to generalize to new compositions in image undcrstanding. arXiv preprint N.ki::6608.066:30. 2016.
[2] Yuval Atzmon, Felix Kreuk, Uri Shalit, and Gal Chechik. A causal view of compositional zero-shot recognition. In Advances in Neural Information Processing Systems, volume 33, pages 1462 1473, 2020.
1, 6, 17
[3] Wentao Bao, Lichang Chen, Heng Huang, and Yu Kong, Prompting ianguage-informet distribution for compositional zero-shot learning. In Proceedings of the European Conference Computer Vision, pages 107-123, 2024. 3, 7, 22
[4] Haoran Chen, Micah Goldblum. Zuxuan Wu, and Yu-Gang Jiang. Adaptive retention & correction: Test-time training continual learning. In Proceedings of the International Conference on Learning Representations, 2025. 3
[5] Shoufa Chen, Chongjian Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. Adapt- former: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems, 35: 35:16664 16678, 2022. 3, 4, 16, 21
[6] Zhixiang Chi, Li Gu, Huan Liu, Ziqiang Wang. Yanan Wu, Yang Wang, and Konstantinos N Plataniotis. Learning to adapt frozen clip for few-shot test-time domain adaptation. In Proceedings of the International Conference Learning Representations, 2025. 3
[7] Zhixiang Chi, Yanan Wu, Li Gu, Huan Liu, Ziqiang Wang, Yang Zhang, Yang Wang, and Konstantinos N Plataniotis. Plug-in feedback self-adaptive attention in clip for training- free open-vocabulary segmentation. arXiv preprint MiXi::508..22666 2025. 2
[8] Sanjoy Chowdhury, Sayan Nag. and Dinesh Manocha. Apollo: Unified adapter and prompt learning for vision language models. Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 10173-1011 10187,1170000 2023. 3
[9] Gottlob Frege. Sense and reference. The Philosophical Review, 57(3):209-230, 1948.
[10] Liane Gabora and Kirsty Kitto. Concept Combination and the Origins of Complex Cognition, pages 361-381. Springer Netherlands, 2013.
[11] Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang. Yongfeng Zhang, Hongsheng and Qiao. Clip-adapter: Better liniun-language models with feature adapters. International Journal of Computer Vision, (32(())581-595 2024. 3
[12] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. Advances in Neural Information Processing Systems, pages 529-536, 2004. 16
[13] Shaozhe Hao, Kai Han, and Kwan Yee K. Wong. Learning attention as disentangler for compositional zero- shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1.531515322, 2023. 7, 18, 22
[14] Siteng Huang, Biao Gong, Yutong Feng. Min Zhang, Yiliang Lv, and Donglin Wang. Troika: Multi-path cross-modal traction for compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24005-24014, 2024. 1, 1, 3, 21,
[15] Phillip Isola, Joseph Lim, and Edward H Adelson. Discovering states and transformations in image collections. Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 1383-1391, 2015. 2, 6, 17, 18,
[16] Chao Jia, Yinfei Yang. Xia, Yi- Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, fun-Hsuan Sung. Zhen Li. and Tom Ducrig. Scaling up visual and vision- languaee representation learning with noisy text supervision. In Proceedings of the International Conference on Machine Learning, pages 4904 4916. 2021. 3

---

[17] Chenyi Jiang. Shidong Wang. Yang Long. Zechao Li. Haofeng Zhang, and Ling Shao, Imaginary conneceeed embedding in complex space for unseen attribute-object discrimination. IEEE Transactions Puttern Analysis and Machine Intelligence, 47(3):195-1413,, 2025. 22
[18] Han Jiang, Chaofan Chen, Xiaoshan Yang. and Changsheng Xu. Compact latent primitive space learning for compositional zero-shot learning. IEEE Transactions on Multimedia, 27:4297 4308, 2025. 7, 18, 22
[19] Adilbek Karmanov, Dayan Guan, Shijian Lu, Abdulmotaleb El Saddik. and Eric Xing. Efficient test-time adaptation of vision-language models. In Proceedings of the IEEF/CVF Conference Computer Vision and Partern Recognition, pages 14162-14171, 2024, 3, 5, 18, 19, 21
[20] Shyamgopal Karthik, Massimiliano Mancini, and Zeynep Akata. Revisiting visual product for composi- tional zero-shot learning. In NeurlPS 202/ Workshop Distribution Shifts: Connecting Methods and Applicaations, 2021. 18, 22
[21] Shyamgopal Karthik, Massimiliano Mancini, and Zeynep Akata. Kg-sp: Knowledge guided simple primitives for open world compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9336 9345, 2022. 18, 22
[22] Muhammad Gul Zain Ali Khan, Muhammad Ferjad Naeem, Luc Van Gool, Alain Pagani, Didier Stricker, and Muhammad Zeshan Afzal. Learning attention propagation for compositional zero-shot learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3828-3837, 2023. 3,3
[23] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinss Barwinska, al. Overcoming catastrophic forgetting neural networks. Proceedings of the National Academy of Sciences, 114(13):3521-3566 2017. 2
[24] Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman. Building machines that learn and think like people. Behavioral and Bruin Sciences, 40:e253, 2017. 1
[25] Xiangyu Li, Xu Yang. Kun Wei, Cheng Deng. and Muli Yang. Siamese contrastive embedding network for compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9326 9335, 2022. 1, 7, 18, 22
[26] Yong-Lu Lu Li, Yue Xu, Xiaohan Mao, and Cewu Lu. Symmetry and group in attribute-object compositions. In Proceedings of the IRFF/CVF Conference on Computer Vision and Pattern Recognition, pages 11313- 1322, 2020. 1, 7, 18, 22
[27] Yong-Lu Li, Yue Xu, Xinyu Xu, Xiaohan Mao, and Cewu Lu. Learning singlchmulti-attribuct of object with symmetry and group. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12)9043-9055 2022. 3
[28] Yun Li, Zhe Liu, Hang Chen, and Lina Yao. Context-based and diversity-driven specificity in composi- tional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17037-17046, 2024. 1, 7,
[29] Yushu Li. Xun Xu, Yongyi Su, and Kui Jia. On the robustness of open-world test-time training: Self- training with dynamic prototype expansion. In Proceedings of the IEEÉ/CVF International Conference on Computer Vision, pages
2023. 2
[30] Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribution shifts. International Journal of Computer Vision, 133(1):31-64 2025. 3
[31] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jac Lee. Improved baselines with visual instruction tnning. In Proceedings of the JEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 26396, 2024. 18, 19
[32] Jiaming Liu, Senqiao Yang, Peidong Jia, Renrui Zhang, Ming Lu, Yandong Guo, Wei Xue, and Shanghang Zhang. Vida: Homeostatic visual domain adapter for continual test time adaptation. In Proceedings of the International Conference on Learning Representations, 2024. 3
[33] Zhe Liu, Yun Li, Lina Yao, Xiaojun Chang, Wei Fang, Xiaojun Wu, and Abdulmotaleb El Saddik. Simple primitives with feasibility- and contextualty-dependdnne for woen-world compositional sero-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1):543-56 560. 2024. 3, 18, 22
12

---

[34] Xiaocheng Lu, Song Guo, Ziming Liu, and Jingcai Guo. Decomposed soft prompt guided fusion enhancing for compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 23560-23569, 2023. 1,
[35] Massimiliano Mancini, Muhammad Ferjad Naeem, Yonggin Xian, and Zeynep Akata. Open world compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2021. 1,4, 7, 22
[36] Massimiliano Mancini, Muhammad Ferjad Naeem, Yongqin Xian, and Zeynep Akata. Learning graph embeddings for open world compositional zcro-shot learning. IEEE Transactions Pattern Analysis and Machine Intelligence, 46(3):1545- 1560, 2022. 1, 1,3,6 6, 22
[37] Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. In Proceedings of the International Conference on Learning Representations, 2018. 18
[38] Lingtong Min, Ziman Fan, Shunzhou Wang, Feiyang Dou, Xin Li, and Binglu Wang. Adaptive fusion lcarning for compositional zero-shot recognition. IEEE Transactions on Multimedia, 27:1193-1200 2025.
[39] Ishan Misra, Abhinav Gupta, and Martial Hebert. From red wine to red tomato: Composition with context. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pages 1160-1169,
2017. 3, 18, 22
[40] Muhammad Ferjad Nacem, Yonggin Xian, Federico Tombari, and Zeynep Akata. Learning graph embed- dings for compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 953-962, 2021. 1, 3, 17, 18,21
[41] Tushar Nagarajan and Kristen Grauman. Auributes operators: Factorizing unseen attribute-object compositions. In Proceedings of the European Conference on Computer Vision, pages 169-185, 2018. 3,
18, 22
[42] Nihal V. Nayak, Peilin Yu, and Stephen H. Bach. Learning to compose soft prompts for compositional zero-shot learning. In Proceedings of the International Conference on Learning Representations, 2023. 1, 3, 6, 7,
[43] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International Conference on Machine Learning, volume 162. pages 16888-16905, 2022. 2
[44] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In Proceedings of the International Conference on Learning Representations, 2023. 3
[45] Adam Paszke, Sam Gross, Francisco Massa, al. Pytorch: An imperative style, high-performance deep learning library. In Advances Neural Information Processing Systems, volume 32, 2019. 6, 18
[46] Senthil Purushwalkam, Maximilian Nickel, Abhinav Gupta, and Marc' Aurelio Ranzato. Task-driven modular networks for zero-shot compositional learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3592-3601, 2019. 1, 4, 6, 18, 22
[47] Hongyu Qu, Jianan Wei, Xiangho Shu, and Wenguan Wang. Learning clustering-based prototypes for compositional zero-shot learning. In Proceedings of the International Conference on Learning Representations, 2025. 3, 7,22 22
[48] Hongyu Qu, Rui Yan, Xiangbo Shu, Hailiang Gao, Peng Huang, and Guo-Sen Xie. Mvp-shot: Multi- velocity progressiv--aignment framework for few-shot action recognition. IEEE Transactions on Multime- dia, 27:6593-660 2025.
[49] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Leamingg trans- ferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning, volume 139, pages 8748-87887888 8763, 2021. 1, 3, 6, 7, 21, 22
[50] Frank Ruis, Gertjan Burghouts, and Doina Bucur Independent prototype propagation for zero-shot compositionality, In Advances in Neural Information Processing Systems, volume 34, pages 10641-10653, 2021. 1,3
13

---

[51] Nirat Saini, Khoi Pham, and Abhinav Shrivastava. Disentangling visual embeddings for attributes and objects. In Proceedingy of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13658-13667, 2022. 3, 18, 22
[52] Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in Asionnlanguage models. Advances in Neural Information Processing Systems, 35:14274 14289, 2022. 2.3
[53] Elaine Sui, Xiaohan Wang, and Serena Yeung-Levv. Just shift it: Test-time prototype shifting for zero- shot generalization with vasion-language models. In Proceedings of IEEF/CVF Winter Conference on Applications of Computer Vision, pages 825-835. IEEE, 2025. 3, 18, 19,
[54] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: fully test- time adaptation by entropy minimization. In Proceedings of the International Conference on Learning Representations, 2021. 2,3
[55] Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7201-7211 2022. 3
[56] Qingsheng Wang, Lingqiao Liu, Chenchen Jing, Hao Chen, Guoqiang Liang, Peng Wang. and Chunhua Shen. Learning conditional attributes for compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11197-11206, 2023, 3, 18, 22
[57] Zehao Xiao, Shilin Yan, Jack Hong, Jiayin Cai, Xiaolong Jiang, Yao Hu, Jiayi Shen, Cheems Wang, and Cees GM Snoek. Dynaprompt: Dynamic test-time prompt tuning. In Proceedings of the International Conference on Learning Representations, 2025. 3
[58] Ling Xing, Hongyu Qu, Rui Yan, Xiangbo Shu, and Jinhui Tang. Locality-aware cross-modal corre- spondence learning for dense audio-visual events localization. arXiv preprint arXiv:2409.0796. 2024.
[59] Guangyue Xu, Joyce Chai, and Parisa Kordjamshidi. Gipcol: graph-injected soft prompting for composi- tional zero-shot learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5774-5783, 2024. 3, 7, 22
[60] Xudong Yan, Songhe Feng. Yang Zhang, Jian Yang. Yueguan Lin, and Haojun Fei. Leveraging milm beddings and attribute smoothing for compositional zero-shot learning. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 21 2179-2187, 2025. 1, 6, 17
[61] Aron Yu and Kristen Grauman. Fine-grained visual comparisons with local learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 192-199, 2014. 2, 6, 17, 18,21
[62] Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, and Xinchao Wang. Task residual for tuning vision-language models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 10899-10909, 2023. 3
[63] Ce Zhang, Simon Stepputtis. Katia Sycara, and Yaqi Xie. Dual prototype evolving for test-time generaliza- tion of vision-language models. In Advances in Neural Information Processing Systems, volume 37, pages 32111-32136, 2024. 3, 18, 19,
[64] Jingyi Zhang, Jiaxing Huang, Xiaoqin Zhang, Ling Shao, and Shijian Lu. Historical test-time prompt tuning for vision foundation models. In Advances in Neural Information Processing Systems, volume 37, pages 1 12872-12896, 2024.
[65] Renrui Zhang, Wei Zhang, Rongyao Fang. Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free adaption of clip for few-shot classification. In Proceedings of the European Conference on Computer Vision, pages 493-510 510, 2022. 6
[66] Taolin Zhang, Jinpeng Wang. Hang Guo. Tao Dai, Bin Chen, and Shu-tao Xia. Boostadapter: Improving vision-language test-time adaptation via regional bootstrapping. In Advances in Neural Information Processing Systems, volume 37, pages 67795 67825, 2024.
[67] Tian Zhang, Kongming Liang, Ruoyi Du. Wei Chen, and Zhanyu Ma. Disentangling before composing: Learning invariant disentangled features for compositional zero-shot learning. IEEE Transactions on Pattern Analysis and Mochine Intelligence, 47(2):1132 1147, 2025. 18, 22
14

---

[68] Tian Zhang, Kongming Liang, Ruoyi Du, Xian Sun, Zhanyu Ma, and Jun Guo. Learning invariant visual representations for compositional zero-shot learning. In Proceedings of the European Conference Computer Vision, pages 339-355, 2022. 2, 6, 7, 17,
[69] Tian Zhang, Kongming Liang, Ke Zhang. and Zhanyu Ma. Learning conditional prompt for compositional zero-shot learning. Proceedings of the IEEE International Conference on Multimedia and Expo, pages 1-6, 2024. 18, 22
[70] Yang Zhang and Songhe Feng. Enhancing domain-invanne invaaiant parts for generalized zero shot learning. In Proceedings of the ACM International Conference on Multimedia, page 6283 12911, 2023,
[71] Yang Zhang, Songhe Feng, and Jiazheng Yuan. Continual compositional zero-shot learning. Proceedings of the International Joint Conference on Artificial Intelligence, pages 1724-1732, 2024.
[72] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning vision- language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16816-16825, 2022. 3
[73] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9)22337-22 -3448, 2022. 1, 6, 7, 21 22
[74] Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. Prompt-aligned gradient for prompt tuning. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 15659- 15669, 2023.
[75] Jinguo Zhu, Weiyun Wang, Zhe Chen, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:250411449, 2025. 18, 19
15

---

Appendix
A
Visual Adapter
Based on AdapterFormer [5], set of learnable adapters injected in parallel with the multi-head attention layer and feed-forward layer, respectively within each Transformer block of the CLIP visual encoder, while keeping the initial parameters frozen, as shown in Fig. Given the input embedding of each layer, the adapter is formulated as:
Adapter(e) = w (ReLU (Wrowne)).
(10)
where Walown and WuP are Iearnable linear layers employed for down-sampling and up-sampling, respectively, For each layer, the final output is obtained by adding the adapter output the original layer output through a residual connection.
Up
MLP
MLP
ReLU
LayerNorm
LayerNorm
Down
Adapter
Multi- Head Attention
Multi-Head
Up
Attention
4
ReLU
LayerNorm
LayerNorm
Down
(a) Transformer Block Without Adapter
(b) Transformer Block
With Adapter
Figure 8: Transformer block within visual encoder of CLIP without and with Adapter.
B Theoretical Explanation of Prediction Entropy
In the absence of labeled target-composition samples, the Minimum Entropy Principle is adopted to improve model predictions. This principle grounded the cluster assumption learning, which posits that the decision boundary should pass through low-density regions in the prediction space. By minimizing the entropy of the model predictions, we encourage high-confidence outputs and implicitly shift the decision boundary away from high-density data regions, thus reducing classification ambiguity.
supevisee
With reference to [12], we formally define DT unlabeled input, and define p(y(xr) the predicted class distribution from the model. The prediction entropy of a single example is defined as:
Ne
H(p(y(z)) = Σ ply clz) llg) ply = clx)
(11)
c=1
where No is the total number of classes. The entropy minimization objective expressed
Lentropy = E,~DY [?(([(}[=))
(12)
Assuming the target data follows continuous density distribution (<x), the entropy loss becomes:
Lentropy
H((py))))4 dx.
(13)
16

---

This formulation reveals that high-density regions (i.e., large q(=)) contribute more to the loss. Therefore, minimizing this loss leads the model to produce low-entropy (i.e., high-confidence) predictions in those regions, which implicitly encourages the decision boundary to lie in low-density areas of the prediction space.
On the other hand, to understand how entropy minimization sharpens model predictions, we analyze the gradient behavior in the prediction space. Let the model output logits .. , ) and the predicted probabilities be given by the softmax function:
Ps
(14)
Žj=1 e
Based this, the gradient of entropy is computed with respect the logits using the chain rule. The derivative of Mp)) with respect to p; is:
0H.
log pt
(15)
Öp:
and the Jacobian of the softmax function is:
Op:
(:((1 - pi),
if
(16)
0z,
-PiPj.
if
Therefore, the gradient of entropy with respect to 2j becomes:
Nc OH 0p;
OH
(17)
0z,
0p; Ozi
This gradient encourages the logits diverge, making one class score dominant over others, and thus increasing the confidence of the softmax prediction (i.e., moving p closer to a one-hot vector). As result, entropy minimization leads to sharper decision boundaries and effectively separates data clusters the prediction space.
This microscopic gradient behavior aligns with the macroscopic Minimum Entropy Principle, both of which serve place the decision boundary in low-density regions and enforce prediction consistency within high-density clusters.
C Dataset Introduction
We conduct our experiments on four CZSL datasets, i.e., UT-Zappos [61]. MIT-States [15], C- GQA [40], and Clothing16K [68]. UT-Zappos is fine-grained fashion dataset consisting of different shoes (e.g., heels, sandals) with their material (e.g., leather, satin). MIT-States contains diverse real-world objects (e.g., envelope, tower) and attributes (e.g., folded, tight) collected by early image search engine technology. However, suffers from considerable noise, with approximately 70% of the labels being incorrect [2, 68, 60). C-GQA is a large-scale dataset composed of in-the- wild images with more general compositions (e.g., marble countertop, striped pillow). For the low-noise dataset Clothing 16K, the objects mainly various kinds of clothing (e.g., dress, shirt), while the attributes are the clothing colors (e.g., pink, white). Table reports the common data splits of the four datasets.
Table Summary statistics of the four datasets used our experiments.
Composition
Train
Validation
Test
Dataset
||]
|A
[**l
[
[C*|
|C"
[2]
[C*]
|C*)
|
UT-Zappos [61]
16
192
22998
3214
18
2914
MIT-States [15]
115
245
28175
1262
30338
300
300
10420
400
400
12995
C-GQA [40]
413
674
278362
5592
26920
1252
1040
7280
888
923
5098
1lothing16K [68]
9
72
18
7242
10
10
5515
9
3413
17

---

D Implementation Details
We use single NVIDIA RTX 3090 GPU train the base model during training and TOMCAT at test time using mixed-precision prompt of CLIP ViT-L/14 in the training phase is initialized by "a photo of"". The hyperparameters for the four datasets are listed in Table 8.
training [37] under the PyTorch framework [45]. The trainable
Table 8: Hyperparameter settings for UT-Zappos, MIT-States, C-GQA, and Clothing 16K.
Hyperparameters
UT-Zaappss
MIT-States
C-GQA
Clothing
The Base Model (Training Phase)
Batch Size
128
64
16
128
Epochs
20
20
20
20
Prompt Dropout Rate
0.3
0.3
0.3
Adapter Downsampling Dimension
64
64
64
64
Adapter Dropout
0.1
0.1
0.1
0.1
Optimizer
Adam
Adam
Adam
Adam
Optimizer- Weight Decay
le-5
le-4
le-5
le-5
Optimizer-Leamiing Rate
5c-4
lc-4
lc-4
5c-4
Scheduler
StepLR
StepLR
StepLR
StepLR
Scheduler-Ster Step Size
5
Scheduler- Ganmaa
0.5
TOMCAT (Test Phase)
Batch Size
1
Image Number of Priority Queue
3
3
3
Optimizer
AdamW
AdamW
AdamW
AdamW
Optimizer-Epsilon
le-3
le-3
le-3
le-3
Optimizer- Weight Decay
lc-3
lc-4
lc-4
lc-3
Slumizer-Learnnng Rate
5e-6
1e-6
6.25e-6
1.25
0.5
0.25
10
7.5
1.5
3.5
2.5
1.75
3.5
E Comparison with More Baselines
In this section, we compare our TOMCAT with three types of baselines on U'T-Zappos, MIT-States, and C-GQA: (1) more CZSL methods; (2) online test-time adaptation (OTTA) of VLM methods; and (3) multimodal large language models (MLLMs).
Comparison with More CZSL Methods, TOMCAT is compared with more saute-of-thh-art CZSL methods on UT-Zappos [61]. J. MIT-States [15], and C-GQA [40] in both closed-wond and open-world settings in Table 17 and Table 18, respectively. Relative to Table 1, these additional baselines include LE+ [39], AtOPp [41], TMN [46], SymNet [26], SCEN [25], OADis [51], ADE [13], CANET [56],
CLPS [18), DBC [67]. CoP [69]. VisProd [20], KG-SP [21], and SAD-SP [33]. Our TOMCAT surpasses all other models and achieves ofate-of-the- art performance.
Comparison with OTTA of VLM Methods. Building the base model obtained after training, we conduct experiments on the three datasets with OTTA VLM approaches, including online-TPS [53], TDA [19], and DPE [63]. These methods also utilize unlabeled test data, making the comparison fairer, although the original comparison with CZSL methods fair due to the absence of labels of test samples. In Table 9, TOMCAT yields 3.7%. 0.4%, and 0.4% AUC gains compared with OTTA of VLM methods on the three datasets, which demonstrates that TOMCAT also exhibits superior performance over OTTA methods on CZSL task under fair conditions.
Comparison with MILMs. With the advancement of MLLMs, they can also be employed to generate image descriptions or categories. Therefore, we compare TOMCAT with two outstanding MLLMS (i.e., LLaVA v1.5 [31] and InternVL.33 [75]) on UT-Zappos and MIT-States. Specifically, we provide the prompt "Classify the image and output strictly two words in the form: 'ultribute object' (an attribute must be generated). No other text. Example: 'red apple'. Choose the autribute and object from their respective list. Attribute list: [Attribute List]. Object list: [Object List]." to
18

---

Table Comparison with OTTA of VLM methods in the closed-wordd setting on UT-Zappos, MIT- States, and C-GQA.
Closed-worke Methods
UTppoappss
MtTe-states
C-GQA
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
AUC
HM
Soen
Unseen
online-TPP [53]
44.6
71.3
71.4
22.2
39.1
49.6
52.8
33.8
45.0
39.6
TDA [19]
41.6
54.6
69.0
70.6
22.0
38.7
49.5
52.9
14.4
32.0
44.7
37.2
DPE [63]
43.3
56.0
68.5
74.2
22.0
38.9
49.2
52.8
15.3
33.5
44.0
39.7
TOМCКT (Ours)
48.3
60.2
74.5
72.8
22.6
39.5
50.3
53.0
16.0
34,0
45,3
40.1
Table 10: Comparison with MILMs on UT-Zappos and MIT-Sttate.
UT-Zaappss
MTT-States
Methods
Comp.
Attr.t
Ohj.
Time (ms)
Comp.
Attr.
Obj. 1
Time (ms)
LLaVA v1.5 [31)
0.32
10.17
2.13
463
5.72
1.06
21.77
857
InternVL-3 [75]
2.06
8.77
23.71
5770
4.43
8.53
48.28
728
TOМCAT (Ours)
44.88
58.54
77.45
30.92
41.00
56.58
145
MLLMs and obtaín their predictions. Since MILMS cannot produce class probabilities for computing AUC, we report the top-1 classification accuracy (%) for compositions (Comp.), attributes (Attr.), objects (Obj.), and the inference Time in Table 10. TOMCAT significantly outperforms them with less time overhead, indicating that MLLMs are primarily designed for text generation and are less effective for representation learning tasks (e.g.. recognition and retrieval). Moreover, the behavior of generative models inherently difficult to control, making them unsuitable for recognition tasks.
F More Ablation Study
In this section, more ablation studies are presented in the closed-world setting.
Ablation study fine-tuning CLIP with the training set. In Table 11, we respectively the naive CLIP without fine-tuning on the training set and the base model fine-tuned on the training data as the starting points for testing. The latter significantly outperforms the former, indicating that while the naive CLIP exhibits strong zero-shot abilities, its general -urpose nature limits its effectiveness in the specific downstream task requiring composition reasoning under the zero-shot setting.
Influence of learning rate. Since our method simulates test-time improvement in real-world applications, the value of learning rate is crucial. According to Table 12, TOMCAT achieves optimal performance when the learning rate is set to 5e-6.
Table 11: Ablation study of fine-tuning CLIP the training set of three datasets, respectively.
Table 12: Influence of learning rate (Ir) of TOMCAT on UT-Zappos.
Dataset
tuning
Seen
Unseen
AUC
HM
Seen
Unseen
UTPpaapps
1.54
7.29
4.79
50.66
le-3
17.11
17.11
$4.25
39.77
48.31
60.18
74.49
72.77
lc-4
28.66
46.14
70.19
45.21
Stae-
11.42
26.79
30.88
46.50
lc-5
47.28
58.96
76.25
69.65
22.55
39.45
50.32
52.95
5e-6
48.31
60.18
74.49
72.77
1.47
7.79
25.09
lc-6
45.78
57.65
70.97
74.25
C-GQA
15.98
33.95
45.30
40.12
1c-7
43.69
55.77
68.82
74.35
Influence of the hyperparameter controlling multimodal balance a. In table 13, TOMCAT achieves the best performance MIT-States when is set to 1.25. An excessively large small value of a leads the model to overemphasize one modality while neglecting the other, thereby undermining its discriminative capability.
Influence the weighting coefficient of LMCRL X. In TOMCAT, balancing the contributions of each loss term plays crucial role. As shown in Table 14, the best performance on MIT-States is
19

---

achieved when À = 1.5, indicating that an appropriate trade-off between the competing objectives leads to optimal model behavior.
Table 13: Influence of a on MIT-States.
Table 14: Influence of À on MIT-States.
AUC
Seen
Unseen
AUC
HM
Seen
Unseen
0.25
22.24
39.27
49.58
52.86
0.25
22.50
39.46
50.08
$2.99
0.5
22.20
39.22
49.50
52.81
0.5
22.53
39.41
50.13
52.93
0.75
22.34
39.47
49.75
52.89
0.75
22.50
39.40
50.04
52.92
22.36
39.38
49.92
52.83
22.51
39,47
50.04
52.91
1.25
22.55
39.45
50.32
52.95
1.25
22.48
39.42
50.00
52.93
1.5
22.43
39.31
49.96
52.95
1.5
22.55
39.45
$0.32
52.95
1.75
22.41
39.31
49.79
53.04
1.75
22.53
39.48
50.17
52.91
2
22.42
39.17
49.79
53.12
22.46
39.45
50.01
52.92
Ablation Study of Prediction Probability in Eq. In Table 15, we observe that relying solely on visual prototypes yields poor performance, likely due to the large intra-class variance in images, which limits the discriminative power of the model. Incorporating both textual and visual prototypes better exploits the complementary multimodal strengths, enabling TOMCAT to achieve good performance.
Table 15: Ablation Study of prediction probability on three datasets. Text- and visual-only denote that the prediction probability derived from only textual prototypes and visual prototypes, respectively.
UTppas
MtT-Stutes
C-GQA
Prediction Probability
AUC
HM
Seen
Unseen
AUC
HM
Soen
Unseen
AUC
HM
Seen
Unseen
Text-only
48.31
60.18
74.49
72.77
22.1
38.95
49.33
52.8
15.79
33.76
45.03
39.86
Visual-onl
1.66
67.45
4.92
15.66
33.76
16.76
41.48
3.95
16.65
29.29
14.95
Multimodal (TOMCAT)
48.31
60.18
74.49
72.77
22.55
39.45
50.32
$2.95
15.98
33.95
45,30
40.12
G More Qualitative Analysis
In this section, we report more qualitative analysis of TOMCAT in the floeed-wrrke setting.
Time and memory analysis. As shown in Table 16, we present the time, latency, and memory occupation the base model and TOMCAT. Our TOMCAT achieves substantial performance improvement with minimal additional computational resources test time.
Table 16: Comparison of time, latency, and memory between TOMCAT and the base model (w/o TOMCAT) on UT-ZZappss and MIT-States. Time means that average testing time across all samples. Latency indicates time from input prediction output for per sample (excluding backpropagation). Memory represents the GPU memory occupied by the model during testing.
Time
Latency
Memory
Dataset
TOМCAT
w/ TOMCAT
TOMCAT
w/ TOMCAT
TOMCAT
w/o TOMCAT
UT-Zappos
47ms
36ms
35ms 112ms
32ms 37ms
4044MB
3801MB
MIT-States
145ms
70ms
4180MB
4175MB
More study. In Fig. 9 and Fig. 10, present more studies UT-Zappos and MIT-States, respectively. As shown, by leveraging our designed algorithm with unlabeled data at test time, TOMCAT significantly improves the accuracy of composition predictions compared to the base model. In some failure cases (e.g., the image labeled as new bus in MIT-States), our method is also capable of making correct predictions, though it focuses aspects different from the labels.
H Broader Impacts
In this work, we recognize that the unseen compositions resulting from the recombination of attributes and objects lead to the distribution shift of label space. Therefore, aim to establish more reliable
20

---

Midcalf
Ouns Leather Midcall Boots
Ours: Pateri Louther fluts shaes
Carraa Flats Stoes
Oura Loather Antle Boots
BM: leather Midcalff Brons
BM: Putent Leather flni-shae
BM: Careaa fliss Shots
BM: Loather Ankle Bood
GT: Sunde Sippers
GT: synthetic Midcall-Boots
GT: Satin Scundalls
GT: Patent-Lenthhe Heals
Ours: Suedi Sippers
Systhatic Midesil-ooots V
Sipppers
Systhetic Midcclf Boons
BM: Satin Scamiils
BM Patent Leather
Sheepalnn Ankle- Ankls- toots
GT: Sunde Midcshh-oots
GT: Camas Flab-SStos
GT: Leather Ankle-Soch
Ours Sheepskin Arile Boots
Oure: Suede Mideiit-Boots
V
Ours: Carras Fixt, Shoes
Ounc Leather Ankie-Soots
BM: Sheopskin Midcall Boots
BM: Suode Kinco High Boots
X
BM: Camas Secokers
BM Loather Midcalf Bouts
X
X
X
Ours Call-hair Hesb
V
Ours: Camous flats Shoei v
Oun: Suade Midcaal--oons
Ours Canves Flats Shoes
BMC Suede Neels
X
BM: Leather Fluts-Shees
X
BM: Leather Mideak- Boots
DM: Leather Skppers
GT: Symthtic Aride- Boons
GT: Syntheric widccif-Boort
GTi Syothetic Roors
GTi Cal--iär
Ouns Leather Ankle- Soots
X
Ours Suede Knee-kil Biee--iig--Boots
Ourci Leather Anots
Ours Calf-Hiar Rats Shoes
Syntheti
Midzalh-Boots v
Syothetic Antle foots
x
V
Cotton Sandels
GT. Nebuck Onford Shoes
6f: Nylon Anble Books
Gf: Syrthetic Midcalf Boots
Oune Sotin Iteels X
Gare: Suede Codord-Shoes X
Oun: Myton Mideal-itotss X
Ourc Rubber Mideall-Botts X
BM: Sutin Heds X
BM: Sucdle Oxford Shoes
X
BM: hylon Midcall Boots X
Bic Rubber Midcall Boots X
Figure 9; More case study on UT-Zappos. We compare TOMCAT (Ours) with the base model (BM)
after training. The successful and failure results are marked in green and red, respectively.
GT: Unripe Tomato
GT: Cut Diamond
GT: Carmmellad Milk
GT: Dry Forest
GT: Fresh Mest
Ours: Unripe Tomaoo
Ours: Cut Diamond y
Oursi Caramelieed y
Oursi Dry Forset o
Ouesi Fresh Moat v
Terrat
BM: Cut Diamond !
BM: Cammeilae Mil
Foreet o
Fresh v
influted
Colled Mat
6T:Small Dog
Dented
Sunry Beach
Oure: infiseed Castle /
Ours Colled Mat /
Oun: Small Dog v
Oun: Derted Truck of
Ours Sunny beach v/
BMI infoned Castlny /
BM: Colled Mat /
BM: Small Dag /
BM: Dented Truck o
BMt Sunny Beach v
dt: Called Busket
Rusty Truck
GT: Poggr Window
GT: Broken Bottle
Sunny Beach
/
Foggy
Ours:
v
X
Grimy Window X
Crumpled
City X
GT: Dry Basement
GT: Whethered Door
GT: Ancient House
GT: Cooked Chicisen
CT: Whipped Max
Ours: Dry Basemens
Oure Whethered Dear
Ours: Anciont House v
Ours: Cooked Chicke
Ours: Whisped Wax
Camp Easement
aM: Crocked Wall
BM: Andient Town
Cooted Salmon
Crinkled Cookic
X
X
X
levelry
Sunny
GTi Muddy River
Oure Thick Rie
Dure large Dax
Our: Broken Windeys
Oun: Westhered Granit
Dun: Muddy Straam
BM: Andeni
BMt New Bus vf
BM: Sunny Window
BM: Weathered Rock
BM: Muddy River
GT: Engroved Sword
GT Andiet Dock
Worn
GT Whipord Ocean
Gurved Hondle
Straight
Heavy
Windblswn
X
Striigtt
BMt Small Clock
BM Hoavy Gear X
Windilown
BM: Engraved Hands
X
Figure 10: More case study on MIT-States.
compositional reasoning system by leveraging test samples to enhance composition modeling, which more in line with the application and development in real-world scenarios. To the best of our knowledge, our method has no potential negative societal impact.
I Licenses
We have explicitly cited the datasets and models used in our paper. Their licenses are listed as follows.
Datasets.
UT-Zappos [61] and MIT-States [15] are two carly-proposed datasets commonly used in CZSL. Their ereators and owners have not declared any license and have allowed com-commercial research use. C-GQA [40] is under CC BY 4.0 license. Clothing16K [68] is under CCO license.
Models. In this work, our TOMCAT is built on existing code repositories: CLIP [49], CoOp [73], FdapterFormer [5], Troika [14], TDA [19], TPS [53], and DPE [63]. The code implementations used in our paper are all under MIT license.
21

---

Table Comparison with more baselines the closed- world setting on UT-Zappos, MIT-States, and C-GQA. The best results are displayed in boldface, and the second-best results are underlined.
UT-Zappos
MIT-States
C-GQA
Closed-world Methods
HM
Seen
Unseen
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
[39] (CVPX9F)
41.0
10.7
15.0
16.1
AnOP [41]
25.9
40.8
59.8
54.2
1.6
9.9
14.3
17.4
0.3
2.9
11.8
3.9
TMN [46] UCCV199
29.3
45.0
58.7
60.0
2.9
13.0
20.2
20.1
1.1
7.7
21.6
6,3
SymNet [26] (C1PR22))
39,2
16.1
24.4
9.8
SCHN (CVPR22)
32.0
47.8
63.5
63.1
18.4
29.9
25.2
12.4
28.9
12.1
OADis [51] (CVAP122)
30.0
44.4
59.5
65.5
5.9
18.9
31.1
25.6
ADE [13] (CVPR22)
51.1
18.0
35.0
17.7
CANET [56]
33.1
47.3
66.3
17.9
26.2
14.5
30.0
13.2
CLPS [18] (7MM29))
37.2
51.8
63.2
70.1
5.6
18.9
35.1
19.0
DBC [67] 07AMP225)
51.3
35.2
17.8
CLIP
5.0
15.6
15.8
49.1
11.0
26.1
30.2
46.0
8.6
25.0
CoOp [75] (UCV22))
18.8
34.6
52.1
49.3
13.5
29.8
34.4
47.6
4.4
17.1
20.5
26.8
Co-CGE [36] (790022))
17.0
33.1
21.2
CSP [42] (C1222
33.0
46.6
64.2
66.2
19.4
36.3
46.6
39.9
6.2
20.5
28.8
26.8
DFSP [34] (CVPR22)
32.1
45.1
64.2
66.4
20.7
37.2
47.4
52.4
8.7
24.3
35.6
29.3
DFSP(BiP) [34] (CРP)2))
47.1
37.7
26.2
36.5
32.0
DESP((2) [34]
36.0
47.2
66.7
71.7
20.6
37.3
46.9
52.0
10.5
27.1
38.2
32.0
GIPCOL [59] (RACP20
36.2
48.8
65.0
68.5
19.9
36.6
48.5
49.6
7.1
22.5
31.9
28.4
Troika [14] (CV9220)
41.7
54.6
73.8
22.1
39.3
49.0
53.0
12.4
29.4
41.0
35.7
CDS-C7SL, [28]
39.5
52.7
63.9
74.8
22.4
39.2
50.3
52.9
11.1
28.1
38.3
34.2
CoP [69] (CME)24)
36.2
51,2
64.8
67.3
19.7
36.4
47.0
50.9
7.3
21.7
33.3
27.0
PLID [3] (ECCV200
52.4
22.1
39.0
49.7
52.4
11.0
27.9
38.8
33.0
IMAX
40.6
54.2
69.3
21.9
39.1
48.7
12.8
29.8
39.7
35.8
ClusPro [47] (CLP25)
46.6
58.5
70.7
76.0
23.8
40.7
52.1
54.0
14.9
32.8
44.3
37.8
TOMCAT (Ours)
48.3
60.2
74.5
72.8
22.6
39.5
50.3
53.0
16.0
34.0
45.3
40.1
Table 18: Comparison with more baselines the open- setting on UT-Zappos, MIT-States, and C-GQA. We report the results of CDS-CZSL [28] using the same post-training feasibility calibration [35] as our TOMCAT and other baselines use.
UT-ZZappos
MIT-States
C-GQA
Open-world Methods
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
AUC
HM
Seen
Unseen
LE+ [39] (CV9917)
16,3
30.5
60.4
36.5
0.3
2.7
14.2
2.5
0.1
1.0
19.2
0.7
AttOp [41] (CVP8:0)) 18)
13.7
29.4
50.9
34.2
0.7
16.6
5.7
TMN
8.4
21.7
55.9
18.1
0.1
1.2
12.6
0.9
SymNet [26] (CV9920))
18.5
34.5
53.3
44.6
0.8
5.8
21.4
7.0
0.4
3.3
26.7
2.2
VisProd [20] (Wer
5.8
KG-SP [21] (CV99:222)
26.5
42.3
61.8
52.1
1.3
7.4
28.4
7.5
0.8
4.7
31,5
2.9
ADE [13] (CVAR220)
27.1
44.8
62.4
50.7
1.4
7.6
35.1
4.8
SAD-SP [33] 09P0022
54.7
1.4
29.1
7.6
1.0
31.0
DBC [67] (795M)25)
28.6
44.9
63.0
52.8
1.4
35.6
4.7
CLIP [49] 0cM0/20
2.2
11.2
15.7
20.6
3.0
12.8
30.1
14.3
D.3
4.0
7.5
CoOp [73] (UCV222)
28.9
$2.1
31.5
2.8
12.3
34.6
Co-CGE [36] (ZPAM)22))
28.4
45.3
59.9
56.2
5.6
17.7
38.1
20.0
0.9
5.3
33.2
3.9
CSP [42] (cuu222
22.7
38.9
64.1
44.1
5.7
17.4
46.3
15.7
1.2
6.9
28.7
5.2
DFSP(120 [34] (CVA9133)
DISP(BiFF) (34]:VV9822
27.6
42.7
63.5
57.2
6.7
19.2
47.1
18.1
2.4
10.6
36.4
7.6
DFSP((2)) [34] (CV99123)
30.3
44.0
66.8
60.0
6.8
19.3
47.5
18.5
2.4
10.4
38.3
GIPCOL [59]
23.5
40.1
65.0
45.0
6.3
48.5
16.0
1.3
31.6
Troika [14] (c199330)
33.0
47.8
66.4
61.2
7.2
20.1
48.8
18.7
2.7
10.9
40.8
7.9
CDS- CZSL [28] (CVIe:2))
32.1
48.0
64.7
60.4
2.6
10.9
38.2
PLID [3] (сceи22)
30.8
46.6
67.6
55.5
7.3
20.4
49.1
18.7
2.5
10.6
39,1
7.5
IMAX [17] (794)25))
32.3
47.5
68.4
57.3
7.6
21.4
50.2
18.6
2.6
11.2
38.7
7.9
ClusPro [47] (CDR25)
39.5
54.1
71.0
66.2
9.3
23.0
51.2
22.1
3.0
41.6
TOMCAT (Ours)
43.7
57.9
74.1
65.8
8.2
21.7
49.2
21.0
4.2
14.2
45.1
10.6
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have clearly stated our contributions and scope of the paper in Abstract and Introduction (Sec. 1).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of our work in Conclusion (Sec. 5),
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
23

---

Justification:
This paper does not provide any theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the model framework and implementation in Method (Sec. 3), Experiment Setup (Sec. 4.1), and Implementation Details (Appendix D). Therefore, we believe that the readers can replicate the proposed TOMCAT by following our instructions.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
24

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All datasets are publicly available through the links provided in the respective papers where they are proposed. The source code of our method will be released at attps://githin com com/xud- wwwudddannnna yan/TOMCAT.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.. cc/
Suuidee/CooeSubiisee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
gui des/ Codeeubmies iss ss onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We report all the training and test details in our paper, including data splits (Appendix C), analysis of
(Sec. 4.3 ), and the implementation details
(Appendix D).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We vary the test order using different random seeds to conduct statistical significance tests in Ablation Study (Appendix 4.3).
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We present the time consumption and GPU memory usage of method in More Qualitative Analysis (Appendix G).
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't fidn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: / //t:://neuure //nuurnuusp.../ wwtrriis.ccppuuuiie cctppblic//aieecccccccccc Beteee/ttiiiiieeeeeeeeee csGuidelines"
Answer: [Yes]
Justification: We have reviewed the NeurIPS Code of Ethics and make sure that our research conform with it in every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed both potential positive and negative socictal impacts of our work in Broader Impacts (Appendix H).
Guidelines:
26

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We are convinced that the paper dose not pose any risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have explicitly cited all the assets used in our paper. Additionally, the licenses our used datasets and code are presented in Licenses section (Appendix I).
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
27

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This paper does not introduce any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
28

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
Our proposed method does not involve LLM.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: Inttps://netll https//////uueeeeeeeeeeeeeer http:pps...//ooeeeeeeeee/////220 for what should or should not be described.
29


Reviews:
[{"review_id": "G8krXNqKcn", "reviewer": "Reviewer_hxuG", "summary": "This paper proposes a new method named TOMCAT. By leveraging unlabeled data to accumulate knowledge in both text and visual modalities during the test-time, it aims to address the problem of label space distribution shift caused by unseen combinations in Combinatorial Zero-Shot Learning (CZSL).", "questions": "1. **Limited Performance Gain on MIT-States and C-GQA**: Currently, TOMCAT only achieves significant gains on the UT-Zappos dataset (+6.6 AUC for the closed-world setting, +10.7 AUC for the open-world setting), while the gains on the other two datasets are only ≤ 1 AUC. The authors need to provide further explanations for such differences at the data level.​\n2. **Add Significance Discussion**: With reference to the statement in Weakness #1, it is recommended to discuss the differences and relationships between the current non-MLLM-based CZSL and MLLM-based attribute recognition/reasoning (e.g., Attribute Recognition and Attribute Reasoning in MMbench [1]) from a more global perspective.\n\n[1] MMBench: Is Your Multi-modal Model an All-around Player? Y Liu and et-al.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strength**:\n\n1. **Good Paper Quality**: This work proposes a logically clear and experimentally well-validated solution to the label shift problem in CZSL.\n2. **Good Paper Clarity**: The work has a clear structure, rigorous and sufficient arguments, and accurate and appropriate expressions.\n\n**Weakness**:\n1. **Limited Significance Regarding Real-world Deployment**: The CZSL paradigm adopted in this paper, when viewed in the context of current powerful MLLMs, has debatable uniqueness and practical significance. For instance, for the task of recognizing a \"folded bike\", a powerful MLLM can directly perform fine-grained attribute reasoning. In contrast, the setting of CZSL artificially restricts it to a specific scenario where one has seen \"folded\" and \"bike\" but not their combination, which deviates from the complexity of the real world.\n2. **Limited Originality**: TOMCAT is original at the framework level, but its internal components are inspired by existing works. For example, the idea of TTA has been proven effective in the ZSL field; the dynamic priority queue idea references TDA; COOP is used during the training phase; the fusion method for prediction references Tip-adapter; and the design of KAM may also refer to the prototype shift idea in TPS. Although the authors have clearly cited most of these references, the overall originality is somewhat limited.", "was_revised": true, "final_justification": "The author's response has addressed most of my concerns, I keep positive attitude towards the acceptance of this paper.", "rebuttal": "Dear Reviewer hxuG,\n\nWe sincerely and deeply thank you for your valuable reviews, which truly contribute a lot to improving the quality of our paper. First, we appreciate the recognition of our paper: (1) TOMCAT is a logically clear and experimentally well-validated solution for CZSL; (2) Our work is well-structured, well-argued, and clearly expressed. Below, we address your questions and clarify the misunderstanding.\n\n---\n\n**W1:** The significance of CZSL in the context of current powerful MLLMs.\n\n**R1:** At its core, CZSL is a **representation learning task** rather than a **generative task**; hence, MLLMs are not suitable for this purpose, for the following reasons: \n\n\n1. In the current situation where the community pays excessive attention to generative models such as LLMs and MLLMs, representation models like CLIP have the advantages of **strong feature extraction ability**. \n2. MLLMs are used for text generation, so **their model behavior is difficult to control**.\n3. MLLMs have billions of parameters, whose inference **needs too much GPU memory and time**. CLIP based methods are more flexible under resource-constrained and latency-sensitive conditions.\n4. As shown in Table 1, we conduct the experiments on the three datasets in CZSL using MLLMs (e.g., LLaVA v1.5[1], and InternVL-3[2]). **Our TOMCAT outperforms the MLLMs** significantly, indicating that MLLMs, whose generation is hard to control, are not suitable for representational tasks like CZSL, recognition, and retrieval.\n    \n| Method     | UT-Zappos     |             |                  |            | MIT-States |             |                  |       |\n|------------|---------------|-------------|------------------|------------|---------------------|-------------|------------------|-------|\n|            | Attribute Acc↑ | Object Acc↑ | Composition Acc↑ | Time (ms)↓ | Attribute Acc↑       | Object Acc↑ | Composition Acc↑ | Time (ms)↓ |\n| LLaVA v.15[1] | 2.13          | 10.17       | 0.32             | 463        | 1.06                | 21.77       | 5.72             | 857   |\n| InternVL-3[2] | 8.77          | 23.71       | 2.06             | 5770       | 8.53                | 48.28       | 4.43             | 728   |\n| TOMCAT     | **58.54**         | **77.45**       | **44.88**            | **47**        | **41.00**                  | **56.58**       | **30.92**            | **145**   |\n\nTable 1. The classification accuracy of TOMCAT and two MLLMs. Since MLLMs cannot generate classification probability, we report the top-1 accuracy instead of AUC and HM.\n\n5. **CZSL has broad applicability**, such as enabling home robots to recognize novel attribute-object compositions like “broken glass” or “dirty plate” for accurate classification; and facilitating drug discovery by composing different chemical molecules to predict bioactivity.\n6. For the sake of research convenience, CZSL simplifies compositions into single-attribute single-object pairs. However, CZSL can **easily generalize to multi-attribute single-object**, encoded by CLIP text encoder, such as \"old and folded bike\".  Some recent works, such as Trident[3] and MAC[4], have started to address this aspect.\n\n---\n\n**W2:** TOMCAT has limited originality.\n\n**R2:** We appreciate the reviewer’s concern regarding the originality of our work, but we would like to clarify our motivation more clearly:\n\n1. Our work is **the first to pay attention to the issue of label space shift in CZSL**, and for the first time use unsupervised test data to adapt to the new label distribution at test time, improving performance on both seen and unseen categories. The entropy loss used during test overcomes the drawback that unseen compositions can not be included for optimization during the training stage. We believe that our method, from the angle of label space shift to solve the CZSL problem, will provide significant inspiration for future research in the CZSL community.\n2. Our method is inspired by TDA[5], TPS[6], and DPE[7]. **However, TOMCAT is different from them at the methodology level**. TDA records historical images without focusing text information; TPS concentrates on shift vectors but fails to use visual information; DPE updates all prototypes by directly adding vectors to them **in a offline manner** (**failing to accumulate knowledge**), which treats all categories equally regardless of familiarity.  \nOur work overcomes their limitations with visual priority queue, **online** knowledge accumulation modules, and **adaptive update weights** based on the similarity of test image and multi-modal prototypes. **Our work is quite different from the above ones, so we believe it still has considerable originality.**\n    \n---\n\n**Q1:** Explanation on the varying degree of improvement on datasets.\n\n**R3:** Your observation on the phenomenon is very perceptive, which we have noticed when conducting experiments and briefly mentioned in Sec.4.2 Main Results. \n\n1. MIT-States and C-GQA are two datasets containing real-world and daily compositions (e.g., tiny dog). These datasets share a similar image style with the CLIP pre-training datasets, which determines that the CZSL **training** set is good to fine-tune CLIP for the CZSL task, leaving limited room for improvement through **test-time adaptation**. \n2. However, UT-Zappos and Clothing16K are fine-grained fashion datasets, whose image style and label space shift are very different from CLIP pre-training datasets. At this point, the CZSL training set can only adapt CLIP to the seen compositions in CZSL, without enabling it to learn knowledge about the unseen ones. TOMCAT has great potential in learning unseen compositions as time goes by.\n\n**In conclusion, The greater the label shift in the CZSL dataset, the more obvious the advantage of our method over other CZSL methods.**\n\n---\n\n**Q2:** The difference between CLIP-based CZSL and MLLM-based attribute recognition.\n\n**R4:** The significance of CZSL is responded in **R1**. Here we pay more attention to the difference between the two settings. **As general-purpose models, the attribute recognition capability of MLLMs is limited to coarse-grained scenarios. However, CZSL focuses on fine-grained compositional reasoning under zero-shot setting (a kind of representational task).** \n\nAs shown in Table 1, for more fine-grained dataset UT-Zappos and Clothing16K, MLLMs can not systematically classify the compositions well. Moreover, fine-tuning MLLMs on specific datasets is challenging due to the limited data size, label format constraints, and memory requirements. Instead, our CLIP-based TOMCAT performs well with less time usage.\n\n---\n\nThe above is my response to the question and concern raised. Once again, thank you for your time and thoughtful consideration. We would be happy to discuss any remaining concerns in the next phase.\n\nBest regards,\n\nAuthors.\n\n&nbsp;\n\n[1] Haotian Liu, et al. Improved baselines with visual instruction tuning. CVPR 2024.\n\n[2] Jinguo Zhu, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint 2025.\n\n[3] Xudong Yan, et al. Leveraging MLLM Embeddings and Attribute Smoothing for Compositional Zero-Shot Learning. arXiv preprint 2024.\n\n[4] Shuo Xu, et al. MAC: A Benchmark for Multiple Attributes Compositional Zero-Shot Learning. arXiv preprint 2024.\n\n[5] Adilbek Karmanov, et al. Efficient test-time adaptation of vision-language models. CVPR 2024.\n\n[6] Elaine Sui, et al. Just shift it: Test-time prototype shifting for zero-shot generalization with vision-language models. WACV 2025.\n\n[7] Ce Zhang, et al. Dual prototype evolving for test-time generalization of vision-language models. NeurIPS 2024."}, {"review_id": "Nv0cW7IJis", "reviewer": "Reviewer_4ZG6", "summary": "Authors present TOMCAT, a compositional zero-shot learning approach that leverages test-time knowledge accumulation to identify novel compositions of attributes and objects not seen during training. Authors learn multi-modal prototypes for each combination of attributes and categories that are updated during inference using a confidence-based dynamic priority queue. Authors demonstrate that their proposed approach works better than their base model over time (e.g. as class prototypes are refined throughout inference), and significantly outperforms prior work on four datasets in both closed-world and open-world settings.", "questions": "- Authors train the base model on in-domain data and see examples of typical compositions during training. How is the proposed method zero-shot? Although it may generalize to novel compositions, it has already seen the domain, making it difficult to evaluate how well the model is generalizing. Instead, one can consider using the CLIP model as the base architecture and demonstrate compositional generalization without any training. \n- How does the ordering of test images impact final performance? Do we expect to see significant variance? \n- What is the impact of the size of the label space C on the model performance? \n- Authors highlight latency during inference time as a primary consideration in the design of their method. What is the latency of this method compared to prior work?", "limitations": "Yes, authors address the limitations of their work in the supplement.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strenghts\n- Strong Results. Authors demonstrate that that their proposed approach significantly outperforms prior work.\n- Extensive Baseline Comparisons. Authors compare against many baselines (and include even more results in the supplement). \n- Informative Visuals. The visuals help explain the core components of the method, making it easy to understand the proposed approach. \n\nWeaknesses\n- Unclear Writing. The writing is unclear at times, making it difficult to understand the fine-grained details. Authors naming each individual component of their method makes it more confusing to follow their approach. \n- Limited Open-World Benchmark. Although the total number of potential attribute-object pairs is large (especially for the C-GQA) benchmark, many of the attribute-object pairs are not realistic, limiting the total size of the \"open-world\". This textual bias may make it easier to ignore certain attribute-object pairs. Instead, real attribute-object pairs should be sequestered during training. \n- Bias Towards Compositional Datasets. The proposed approach seemingly has a bias towards compositional objects. Although I understand that this is the primary focus of the paper, it is unlikely that real-world scenarios will only exclusively have compositional objects. This paper can be strengthened by evaluating on standard classification datasets to validate that the method does not improve compositional performance at the cost of non-compositional classification accuracy.", "was_revised": true, "final_justification": "Authors have sufficiently addressed my questions and provide strong experimental evidence to support their claims. I would recommend accepting this paper.", "rebuttal": "Dear Reviewer 4ZG6,\n\nWe are truly and profoundly grateful to you for your insightful suggestions and careful evaluation, which have been invaluable in enhancing our work. First, we appreciate your recognition of our work: (1) TOMCAT has strong results. (2) We provide extensive comparisons with many baselines and informative visual. We now provide detailed response to your comments.\n\n---\n\n**W1:** Unclear writing.\n\n**R1:** We sincerely apologize for the confusion caused by our writing and naming conventions. We will take the reviewer's suggestions seriously and refine our writing and naming practices in this paper and our future works after rebuttal. We would be happy to explain if you point out the unclear points. In addition, we promise to immediately release the code after the final decision to facilitate readers’ understanding.\n\n---\n\n**W2:** Limited Open-World Benchmark caused by infeasible compositions.\n\n**R2:** We use a post-training feasibility calibration (computed by GloVE[1] word embedding) to filter out part of infeasible compositions following CSP[2], Troika[3] and PLID[4]. During training, we sequester the real attribute-object pairs. In the open-world setting, we do not sequester them at test time. Instead, **the model is required to infer them autonomously, assigning lower prediction probabilities to infeasible combinations.** We believe this setting is more challenging than assuming a predefined set of feasible compositions (closed-world setting).\n\n---\n\n**W3:** Bias Towards compositional datasets.\n\n**R3:** Our method is proposed to overcome the label space shift caused by the unseen compositions recombined by attributes and objects at test time. However, as you say, a robust model does not improve compositional performance at the cost of standard classification accuracy. \n\nTherefore, on the one hand, following your suggestion, **we evaluate TOMCAT on four standard datasets (two ImageNet datasets and two fine-grained datasets) in Table 1**. On the other hand, **we report the pure object classification accuracy on CZSL datasets in Table 2.** Both of the results demonstrate that our TOMCAT performs well on standard object recognition too.\n\n| Method | ImageNet | ImageNet V2 | Flower 102 | DTD   |\n|--------|----------|-------------|------------|-------|\n| TPT[7]    | 69.54    | 63.95       | 69.71      | 46.93 |\n| TPS[8]    | 71.45    | 64.91       | 71.54      | 50.47 |\n| TDA[9]   | 69.51    | 64.67       | 71.42      | 47.40 |\n| DPE[10]    | 71.91    | 65.44       | 75.07      | 54.20  |\n| TOMCAT | **72.09**   | **64.83**       | **75.46**      | **56.03** |\n\nTable 1. The top-1 classification accuracy on four standard classification datasets.\n\n| Method | UT-Zappos    |            | MIT-States   |            | C-GQA        |            |\n|--------|--------------|------------|--------------|------------|--------------|------------|\n|        | Attribute Acc | Object Acc | Attribute Acc | Object Acc | Attribute Acc | Object Acc |\n| SCEN[5]   | 47.3         | 75.6       | 28.2         | 32.2       | 13.6         | 27.9       |\n| CANet[6]  | 48.4         | 72.6       | 30.2         | 32.6       | 17.5         | 22.3       |\n| CSP[2]    | 47.9        | 73.1       | 38.4        | 54.9      | 30.4        | 44.7      |\n| TOMCAT | **58.5**        | **77.5**      | **41.0**           | **56.6**      | **42.3**        | **54.8**      |\n\nTable 2. The top-1 object classification accuracy on three CZSL datasets (only few works report their classification accuracy).\n\n---\n\n**Q1:** Explain the compositional zero-shot learning setting.\n\n**R4:** CZSL aims to recognize unseen compositions that have no training data. In other words, the model is trained only on seen compositions during training, but is required to recognize unseen (novel) compositions during testing. **The “zero-shot” is reflected in the fact that the training data of unseen compositions is \"zero-shot\".**\n\n---\n\n**Q2:** The impact of ordering of test images.\n\n**R5:** Your question on the test ordering is very insightful. Table 3 provides three different ordering, from which we observe that **there is variance among different orderings, but they are not significant.** \n\nWe attribute this to the fact that adaptive update weights enhance the stability of the model, allowing the model to adaptively determine the extent to which the prototypes are updated. In addition, the regularization item of multimodal collaborative representation learning also maintains the stability of the model. Also, we look forward to more future works that can better address this issue.\n\n| Ordering | UT-Zappos |       |       |        | MIT-States |       |       |        | C-GQA |       |       |        |\n|-------|------------|-------|-------|--------|-----------|-------|-------|--------|-------|-------|-------|--------|\n|       | AUC        | HM    | Seen  | Unseen | AUC       | HM    | Seen  | Unseen | AUC   | HM    | Seen  | Unseen |\n| 1     | 48.31      | 60.18 | 74.49 | 72.77  | 22.55     | 39.45 | 50.22 | 52.95  | 11.05 | 28.48  | 37.66  | 34.18   |\n| 2     | 48.52      | 60.82 | 74.64 | 72.95  | 22.46     | 39.33 | 49.84 | 53.05  | 11.38 | 28.93 | 38.44 | 34.18  |\n| 3     | 46.44      | 58.72 | 72.63 | 72.03  | 22.31     | 39.02 | 49.62 | 53.09  | 11.04 | 28.51 | 38.09 | 33.48  |\n\nTable 3. The results of different test ordering on three CZSL datasets.\n\n---\n\n**Q3:** The impact of the size of label space C.\n\n**R6:** **The size of label space is quite different among the four datasets**: UT-Zappos, MIT-States, C-GQA, and Clothing16K. TOMCAT achieves the state-of-the-art performance on the four datasets, indicating that our method is robust to the size of label space. \n\nIn addition, for each dataset, we conduct our experiments **in both closed setting (small label space) and open-world setting (large one)** in Table 1 of the paper. In general, TOMCAT has more obvious advantages over baselines in open-world setting when label space gets larger.\n\n---\n\n**Q4:** Explain what is the decay.\n\n**R7:** We apologize for only providing a brief description of the decay in the caption of Table 13 in the Appendix G. After the user submits an image, Tomcat first generates a prediction and then performs backpropagation to update the parameters. **Decay denotes the time interval between user’s submission and model’s prediction, including the calculation of logits based on textual and visual prototypes, but excluding the backpropagation.** TOMCAT exhibits minimal decay due to our deliberate design.\n\n---\n\nOur response to your concerns and questions are provided above. Once again, we appreciate your time and insightful consideration. Please feel free to let us know if you have additional questions.\n\nBest regards,\n\nAuthors.\n\n&nbsp;\n\n[1] Jeffrey Pennington, et al. GloVe: Global vectors for word representation. NeurIPS 2014.\n\n[2] Nihal V. Nayak, et al. Learning to compose soft prompts for compositional zero-shot learning. ICLR 2023.\n\n[3] Siteng Huang, et al. Troika: Multi-path cross-modal traction for compositional zero-shot learning. CVPR 2024.\n\n[4] Wentao Bao, et al. Prompting language-informed distribution for compositional zero-shot learning. ECCV 2024.\n\n[5] Xiangyu Li, et al. Siamese contrastive embedding network for compositional zero-shot learning. CVPR 2022.\n\n[6] Qingsheng Wang, et al. Learning conditional attributes for compositional zero-shot learning. CVPR 2023.\n\n[7] Manli Shu, et al.  Test-time prompt tuning for zero-shot generalization in vision-language models. NeurIPS 2022.\n\n[8] Elaine Sui, et al. Just shift it: Test-time prototype shifting for zero-shot generalization with vision-language models. WACV 2025.\n\n[9] Adilbek Karmanov, et al. Efficient test-time adaptation of vision-language models. CVPR 2024.\n\n[10] Ce Zhang, et al. Dual prototype evolving for test-time generalization of vision-language models. NeurIPS 2024."}, {"review_id": "OHpK0AAnVq", "reviewer": "Reviewer_uhct", "summary": "Current Compositional Zero-Shot Learning (CZSL) methods often struggle with performance declines during testing due to shifts in label space distribution resulting from unseen attribute-object combinations. This paper proposes a novel approach that collects extensive knowledge from unsupervised data in both textual and visual domains to update multi-modal prototypes at test time. An adaptive update weight is introduced to control prototype adjustments, enhancing the model's adaptability to distribution shifts. A dynamic priority queue is also employed to store high-confidence images, allowing the model to leverage visual knowledge from historical data for better inference. To maintain semantic consistency, textual and visual prototypes are aligned through multimodal collaborative representation learning. Extensive experiments show the method surpasses existing ones on four challenging benchmark datasets, under both closed-world and open-world conditions.", "questions": "1. Does the performance on test data improve progressively with the inference order? In other words, would the prediction results be relatively better for data that comes later in the sequence?\n2 . Will the performance increase if the method uses BLIP2 or a more powerful foundational model?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "[s1] The proposed TOMCAT can accumulate multimodal knowledge from unlabeled data and updates prototypes at test time to bridge the label distribution shift.\n\n[s2] A priority queue stores historical information and adaptively updates multimodal prototypes.\n\n[s3]  The performance of TOMCAT on four benchmark datasets is the best in both closed-world and open-world settings.\n\n[w1] The main issue is that the evaluation is not entirely fair. The evaluation of existing CZSL methods does not rely on test data. However, the proposed method utilizes historical data to obtain visual and textual information through CLIP. Alternatively, from one perspective, it leverages CLIP to annotate a portion of the data, which is not as fair in comparison to previous approaches. It might be the primary reason for the significant performance improvement observed in this paper.", "was_revised": true, "final_justification": "Thank the authors for the detailed response. The proposed method of the paper can improve upon existing methods on the CZSL dataset and also demonstrates outstanding performance on standard TTA classification datasets. Therefore, I consider raising my score and tend to accept this work.", "rebuttal": "Dear Reviewer uhct,\n\nWe would like to wholeheartedly and sincerely thank you for your constructive comments, which we found immensely helpful. First, we appreciate your recognition of our TOMCAT: (1) This paper proposes a novel approach with some useful components. (2) Our method achieves SOTA performance across four datasets. We now provide detailed explanations on each of your concerns.\n\n---\n\n**W1:** The evaluation is not entirely fair.\n\n**Q1:** First, we sincerely thank you for providing the insightful concern about fairness of comparison, which is understandable, but we want to explain our comparison is reasonably fair, based on the following reasons:\n\n1. Compared with the existing methods, **there is no complex disentanglement or other modules for training in TOMCAT.** However, TOMCAT still outperforms them.\n2. Since the model simulates the scenario where a user submits a single test image **without accessing its ground-truth label**, we consider this a fair evaluation.\n3. Existing methods simply discard the test data after evaluation, which we believe is a big waste. Instead, we propose to make use of this unlabeled data.\n4. You have provided a very insightful and profound comment--it seems that we leverage CLIP to annotate a portion of the data at test time. However, we have to emphasize that **CLIP is not an external auxiliary helper--it is the core predictive model in current mainstream methods** (such as all baselines in Table 1 in the paper). In standard evaluation setting, it has to predict an image; TOMCAT just uses the prediction.  \nOn the other hand, **adapting the model using its own predictions follows a self-supervised paradigm** (e.g., BLIP[7]), which is a widely acceptable and reasonable practice.\n5. To further alleviate the reviewer’s concerns about fairness, we provide experimental results under three different settings:  \n**(1) comparing different CZSL methods using test-time TOMCAT on CZSL datasets** (disentanglement-based models like Troika are not compatible with TOMCAT due to disentangling multi-prototypes) in Table 1;  \n**(2) comparing TOMCAT with TTA methods on standard TTA classification datasets** (e.g., TPT[4], TPS[5], TDA[2], and DPE[6]) in Table 2.  \n**(3) comparing TOMCAT and different TTA methods based on the Base Model on CZSL datasets** in Table 3.  \nThe results show that TOMCAT performs better than baselines with fairness in different settings.\n\n| Method            | UT-Zappos |       |       |        | MIT-States |       |       |        | C-GQA |       |       |        |\n|-------------------|------------|-------|-------|--------|-----------|-------|-------|--------|-------|-------|-------|--------|\n|                   | AUC        | HM    | Seen  | Unseen | AUC       | HM    | Seen  | Unseen | AUC   | HM    | Seen  | Unseen |\n| CSP[3]               | 31.24      | 46.55 | 60.9  | 62.62  | 19.13     | 25.92 | 46.68 | 49.67  | 5.85  | 19.45 | 27.54 | 26.92  |\n| CSP[3] + TOMCAT        | 32.26      | 49.27 | 64.32 | 57.69  | 19.48     | 26.77 | 49.96 | 49.96  | 6.09  | 19.8  | 28.88 | 26.75  |\n| Base Model        | 43.57      | 55.54 | 68.72 | **74.30**   | 22.12     | 38.97 | 49.50  | 52.61  | 10.73 | 27.92 | 36.63 | 24.39  |\n| Base Model + TOMCAT | **48.31**      | **60.18** | **74.49** | 72.77  | **22.55**     | **39.45** | **50.22** | **52.95**  | **11.05** | **28.48**  | **37.66**  | **34.18**  |\n\nTable 1. Comparison the results of CSP and the Base Model with and without TOMCAT on three CZSL datasets.\n\n| Method | ImageNet | ImageNet V2 | Flower102 | DTD   |\n|--------|----------|-------------|------------|-------|\n| TPT[4]    | 69.54    | 63.95       | 69.71      | 46.93 |\n| TPS[5]    | 71.45    | 64.91       | 71.54      | 50.47 |\n| TDA[2]    | 69.51    | 64.67       | 71.42      | 47.40 |\n| DPE[6]    | 71.91    | **65.44**       | 75.07      | 54.20  |\n| TOMCAT | **72.09**   | 64.83       | **75.46**      | **56.03** |\n\nTable 2. Comparison of the top-1 classification accuracy among TOMCAT with TTA methods on four standard TTA classification datasets.\n\n| Method            | UT-Zappos |       |       |        | MIT-States |       |       |        | C-GQA |       |       |        |\n|-------------------|------------|-------|-------|--------|-----------|-------|-------|--------|-------|-------|-------|--------|\n|                   | AUC        | HM    | Seen  | Unseen | AUC       | HM    | Seen  | Unseen | AUC   | HM    | Seen  | Unseen |\n| online-TPS[5] | 44.59 | 57.29 | 71.26 | 71.44 | 22.21 | 39.09 | 49.62 | 52.8  | 10.84 | 27.87 | 37.51 | 33.65 |\n| TDA[2]        | 41.57 | 54.61 | 69.01 | 70.6  | 22.01 | 88.71 | 49.5  | 52.86 | 9.41  | 25.3  | 37.25 | 30.33 |\n| DPE[6]       | 43.32 | 55.97 | 68.49 | **74.17** | 22.02 | 38.9  | 49.24 | 52.78 | 10.69 | 27.88 | 36.57 | 34    |\n| TOMCAT | **48.31**      | **60.18** | **74.49** | 72.77  | **22.55**     | **39.45** | **50.22** | **52.95**  | **11.05** | **28.48**  | **37.66**  | **34.18**  |\n\nTable 3. Comparison of TOMCAT and different TTA methods based on the Base Model on CZSL datasets.\n\n---\n\n**Q1:** The impact of different inference order.\n\n**R2:** Generally, the images coming later are predicted relatively better due to the knowledge accumulation, where the model is anticipated to evolve better during test. As shown in Figure 5 in the paper, TOMCAT start obviously higher than base model from about 1/3 point of the sequence. However, we focus more on the overall performance of the model on the dataset. \n\nIn addition, Table 4 provides three different ordering, from which we observe that **there is variance among different ordering, but they are not significant.** We attribute this to the fact that adaptive weights and the regularization item of multimodal collaborative representation learning enhance the stability of the model, allowing the model to adaptively determine the extent to which the prototypes are updated.\n\n| Order | UT-Zappos |       |       |        | MIT-States |       |       |        | C-GQA |       |       |        |\n|-------|------------|-------|-------|--------|-----------|-------|-------|--------|-------|-------|-------|--------|\n|       | AUC        | HM    | Seen  | Unseen | AUC       | HM    | Seen  | Unseen | AUC   | HM    | Seen  | Unseen |\n| 1     | 48.31      | 60.18 | 74.49 | 72.77  | 22.55     | 39.45 | 50.22 | 52.95  | 11.05 | 28.48  | 37.66  | 34.18   |\n| 2     | 48.52      | 60.82 | 74.64 | 72.95  | 22.46     | 39.33 | 49.84 | 53.05  | 11.38 | 28.93 | 38.44 | 34.18  |\n| 3     | 46.44      | 58.72 | 72.63 | 72.03  | 22.31     | 39.02 | 49.62 | 53.09  | 11.04 | 28.51 | 38.09 | 33.48  |\n\nTable 4. The reusults of different test ordering on three CZSL datasets.\n\n---\n\n**Q2:** Comparison with a more powerful foundational model like BLIP-2.\n\n**R3:** At its core, CZSL is a representation learning task rather than a generative task; hence, MLLMs like BLIP-2 and LLaVA, whose behaviour is hard to control, are not suitable for this purpose. \n\nHowever, we still conduct this experiment following your suggestions using MLLMs. BLIP-2 exhibits limited instruction-following capability and can only generate a caption of the whole image such as \"a pair of knives with a box next to them” instead of the composition \"folded knife\". Therefore, we use two stronger MLLMs, LLaVA v1.5[8] and InternVL-3[1]. **Table 5 shows that TOMCAT outperforms them on three datasets in CZSL, indicating CLIP-based models are more suitable for representation tasks** (e.g., CZSL, recognition, and retrieval).\n\n| Method     | UT-Zappos     |             |                  |            | MIT-States |             |                  |       |\n|------------|---------------|-------------|------------------|------------|---------------------|-------------|------------------|-------|\n|            | Attribute Acc↑ | Object Acc↑ | Composition Acc↑ | Time (ms)↓ | Attribute Acc↑       | Object Acc↑ | Composition Acc↑ | Time (ms)↓ |\n| LLaVA v.15[8] | 2.13          | 10.17       | 0.32             | 463        | 1.06                | 21.77       | 5.72             | 857   |\n| InternVL-3[1] | 8.77          | 23.71       | 2.06             | 5770       | 8.53                | 48.28       | 4.43             | 728   |\n| TOMCAT     | **58.54**         | **77.45**       | **44.88**            | **47**        | **41.00**                  | **56.58**       | **30.92**            | **145**   |\n\nTable 5. The classification accuracy of TOMCAT and two MLLMs. Since MLLMs cannot generate classification probability, we report the top-1 accuracy instead of AUC and HM.\n\n---\n\n**PFC:** The order of presentation can be adjusted appropriately.\n\n**R4:** We are sincerely sorry to mislead the readers due to our writing order. We will carefully adjust the order of our presentation following your valuable suggestion after rebuttal.\n\n---\n\nThe above provides our responses to your comments. Once again, we are grateful for your time and thoughtful feedback. We would be happy to further discuss if any concerns remain.\n\nBest regards,\n\nAuthors.\n\n&nbsp;\n\n[1] Jinguo Zhu, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint 2025.\n\n[2] Adilbek Karmanov, et al. Efficient test-time adaptation of vision-language models. CVPR 2024.\n\n[3] Nihal V. Nayak, et al. Learning to compose soft prompts for compositional zero-shot learning. ICLR 2023.\n\n[4] Manli Shu, et al. Test-time prompt tuning for zero-shot generalization in vision-language models. NeurIPS 2022.\n\n[5] Elaine Sui, et al. Just shift it: Test-time prototype shifting for zero-shot generalization with vision-language models. WACV 2025.\n\n[6] Ce Zhang, et al. Dual prototype evolving for test-time generalization of vision-language models. NeurIPS 2024.\n\n[7] Junnan Li, et al. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. ICML 2022.\n\n[8] Haotian Liu, et al. Improved baselines with visual instruction tuning. CVPR 2024."}, {"review_id": "44h3AiowEE", "reviewer": "Reviewer_45LQ", "summary": "This paper introduces TOMCAT, a novel approach for addressing label distribution shift in Compositional Zero-Shot Learning (CZSL) by dynamically updating multimodal prototypes at test time. The method demonstrates significant improvements in performance on multiple benchmark datasets, particularly in the open-world setting.", "questions": "NO", "limitations": "NO", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Innovation: TOMCAT introduces a novel approach to address the label distribution shift in Compositional Zero-Shot Learning (CZSL) by dynamically updating multimodal prototypes during testing, making it a significant advancement in the field.\n2. Writing: The paper is well-written and clearly structured, making the methodology easy to understand and follow. The experiments are presented effectively, and the results are clearly communicated, enhancing the overall readability.\n\nWeaknesses:\n1. Comparison with Recent Methods: The paper mostly compares TOMCAT to methods from a year ago or older. Given the fast-moving nature of the field, it would be beneficial to compare TOMCAT against more recent methods. For example, the work from 2024 or 2025 in test-time adaptation and zero-shot learning would provide a more compelling argument for TOMCAT's advantages.\n2. Resource and Latency Analysis: While the authors mention deferred backpropagation to reduce latency during testing, there is little discussion on the scalability and resource consumption of TOMCAT in more complex, real-world environments. Further analysis on memory usage and computational demands, especially for large-scale deployments, would be valuable.\n3. Limited Discussion of Limitations: The paper does not sufficiently discuss the limitations of the approach. For example, how well does TOMCAT handle extreme label distribution shifts or ambiguous compositions in real-world settings? The model’s performance on datasets with noise or extremely variable compositions should be further explored.", "was_revised": true, "final_justification": "Thanks for the author's reply. I tend to give it a positive score but not a completely acceptable rate.", "rebuttal": "Dear Reviewer 45LQ,\n\nWe would like to express our deepest and most sincere gratitude to you for your constructive comments, which significantly improve the quality of our work. First, we appreciate your recognition of our work: (1) TOMCAT introduces a novel approach to address the label distribution shift in CZSL. (2) The paper is well-written and clearly structured.\n\nFor the three weaknesses raised, we are sincerely sorry for putting them in the Appendix due to the page limitation. Specifically, comparison with more baselines is presented in Appendix E (Table 14 and Table 15); Resource and Latency Analysis is provided in Appendix G (Table 13); Discussion of Limitations is presented in Appendix H. Now we would like to provide a more detailed response to each comment below:\n\n---\n\n**W1:** Comparison with Recent Methods.\n\n**R1:** In Table 14 and Table 15 in the Appendix, we compare many recent and prominent CZSL approaches, including six methods proposed in 2024 and 2025. After rebuttal, we will add more baselines in the manuscript, including CoP[1](ICME 2024), CAGC[2](TNNLS 2025), CLPS[3](TMM 2025), and more. \n\nIn addition, you have mentioned that it is better to compare with test-time adaptation (TTA) baselines, so **we compare TOMCAT with them on CZSL datasets** in Table 1. Also, **we compare TOMCAT with them under standard TTA setting** in Table 2. TOMCAT outperforms them under CZSL and TTA setting, indicating that TOMCAT has great potential in both CZSL and TTA tasks.\n\n| Method            | UT-Zappos |       |       |        | MIT-States |       |       |        | C-GQA |       |       |        |\n|-------------------|------------|-------|-------|--------|-----------|-------|-------|--------|-------|-------|-------|--------|\n|                   | AUC        | HM    | Seen  | Unseen | AUC       | HM    | Seen  | Unseen | AUC   | HM    | Seen  | Unseen |\n| online-TPS[5] | 44.59 | 57.29 | 71.26 | 71.44 | 22.21 | 39.09 | 49.62 | 52.8  | 10.84 | 27.87 | 37.51 | 33.65 |\n| TDA[6]        | 41.57 | 54.61 | 69.01 | 70.6  | 22.01 | 88.71 | 49.5  | 52.86 | 9.41  | 25.3  | 37.25 | 30.33 |\n| DPE[7]       | 43.32 | 55.97 | 68.49 | **74.17** | 22.02 | 38.9  | 49.24 | 52.78 | 10.69 | 27.88 | 36.57 | 34    |\n| TOMCAT | **48.31**      | **60.18** | **74.49** | 72.77  | **22.55**     | **39.45** | **50.22** | **52.95**  | **11.05** | **28.48**  | **37.66**  | **34.18**  |\n\nTable 1. Comparison of TOMCAT and different TTA methods based on the Base Model on CZSL datasets.\n\n| Method | ImageNet | ImageNet V2 | Flower 102 | DTD   |\n|--------|----------|-------------|------------|-------|\n| TPT[4]    | 69.54    | 63.95       | 69.71      | 46.93 |\n| TPS[5]    | 71.45    | 64.91       | 71.54      | 50.47 |\n| TDA[6]   | 69.51    | 64.67       | 71.42      | 47.40 |\n| DPE[7]    | 71.91    | 65.44       | 75.07      | 54.20  |\n| TOMCAT | **72.09**   | **64.83**       | **75.46**      | **56.03** |\n\nTable 2. Comparison of the top-1 classification accuracy on four standard TTA classification datasets.\n\n---\n\n**W2:** Resource and Latency Analysis.\n\n**R2:** As shown in Table 3 (Table 13 in the Appendix G), we present the time, decay, and GPU memory occupation of the base model under standard test setting and TOMCAT. **Our TOMCAT only performs minimal additional decay and computational resources at test time.**\n\n|Dataset | Time |  | Latency |  | GPU Memory |  |\n|------------|--------|------------|--------|------------|--------|------------|\n|            | TOMCAT | w/o TOMCAT | TOMCAT | w/o TOMCAT | TOMCAT | w/o TOMCAT |\n| UT-Zappos  | 47ms   | 36ms       | 35ms   | 32ms       | 4044MB | 3801MB     |\n| MIT-States | 145ms  | 70ms       | 112ms  | 37ms       | 4180MB | 4175MB     |\n\nTable 3. Comparison of time, decay, and memory between TOMCAT and the base model (w/o TOMCAT) on UT-Zappos and MIT-States. Time means that average testing time across all samples. Latency indicates time from input to prediction output for per sample (excluding backpropagation). Memory represents the GPU memory occupied by the model during testing.\n\n---\n\n**W3:** Discussion of limitation.\n\n**R3:** As presented in Appendix H, two potential limitations of TOMCAT are identified: (1) The prediction entropy incorporates unseen compositions into optimization to adapt to the new distribution, but it may cause the model to make incorrect predictions with greater confidence. (2) Although many measures have been taken to reduce latency, the backpropagation still increases the time overhead at test time. We hope that future research can focus more on the label distribution shift in CZSL and better address these two limitations.\n\n---\n\nThe above summarizes our response to your concerns. Once again, we sincerely appreciate your constructive and thoughtful comments. And we would be happy to further discuss if there are any remaining concerns.\n\nBest regards,\n\nAuthors.\n\n&nbsp;\n\n[1] Tian Zhang, et al. Learning Conditional Prompt for Compositional Zero-Shot Learning. ICME 2024.\n\n[2] Yang Liu, et al. Concept-Aware Graph Convolutional Network for Compositional Zero-Shot Learning. TNNLS 2025.\n\n[3] Han Jiang. Compact Latent Primitive Space Learning for Compositional Zero-Shot Learning. TMM 2025.\n\n[4] Manli Shu, et al.  Test-time prompt tuning for zero-shot generalization in vision-language models. NeurIPS 2022.\n\n[5] Elaine Sui, et al. Just shift it: Test-time prototype shifting for zero-shot generalization with vision-language models. WACV 2025.\n\n[6] Adilbek Karmanov, et al. Efficient test-time adaptation of vision-language models. CVPR 2024.\n\n[7] Ce Zhang, et al. Dual prototype evolving for test-time generalization of vision-language models. NeurIPS 2024."}, {"review_id": "Puly8XYetG", "reviewer": "Reviewer_PjJh", "summary": "The core contribution of this paper lies in introducing prompt tuning—commonly used in test-time adaptation tasks—into the context of compositional zero-shot learning. By incorporating several off-the-shelf strategies, the authors achieve promising performance improvements.", "questions": "1. The task formulation appears to be a straightforward combination of TTA and CZSL, while the introduced modules—VPC and KAM—are directly adopted from the TTA method TDA[1] and DPE[2] with minor modification. The paper offers only minor modifications that resemble superficial tricks, rather than introducing substantive methodological innovations.While I do not object to integrative works of the \"A + B\" type, I believe that the components drawn from B should at least be carefully adapted to address the specific challenges of A.\n\n\n2. The article lacks proper citations, e.g., the KAM module has a very high similarity to the DPE[2] in terms of prototype update strategies in the queue. But unfortunately, I did not find any references to them in this part. Alternatively, the authors should clearly articulate the distinctions between the two approaches.\n\n\n3. Unfairness in the experimental setup: How can the TTA-based strategies be fairly compared with compositional zero-shot learning models, given the fundamental discrepancy in the amount of accessible information? If the goal is to demonstrate the effectiveness of the proposed method, it is essential to include comparisons with other TTA approaches, rather than only comparing with standard CZSL methods that have no access to test data.\n\n[1]Karmanov, Adilbek, et al. \"Efficient test-time adaptation of vision-language models.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n[2]Zhang, Ce, et al. \"Dual prototype evolving for test-time generalization of vision-language models.\" Advances in Neural Information Processing Systems 37 (2024): 32111-32136.", "limitations": "Yes", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The paper conducts extensive experiments and demonstrates consistently strong results across various settings.", "was_revised": true, "final_justification": "After reviewing the paper and the authors’ rebuttal, I find the attempt to bridge TTA and CZSL commendable. The motivation for combining these two areas is well-articulated and represents a meaningful direction. However, I believe the methodological changes introduced in adapting TTA to the CZSL setting are relatively modest and lack surprising innovations. As a result, I can only assign a score that reflects a middle-ground evaluation.", "rebuttal": "Dear Reviewer PjJh,\n\nWe would like to express our deepest and most sincere gratitude to you for your thoughtful and constructive comments, which have significantly improved the quality of our work. First, we appreciate your recognition of the extensive experiments and strong results of TOMCAT. Below, we provide our detailed response to each of your valuable comments.\n\n---\n\nQ1: TOMCAT seems to lack substantive methodological innovations.\n\nR1: First, e are sincerely grateful for your thoughtful concern about methodological innovations; however, we cannot fully agree with this comment with our highest respect. Here we seriously address your questions with great care, based on the following reasons:\n\n1. Let us elaborate on our motivation in more detail. **Our work is the first to pay attention to the issue of label space shift in CZSL, and for the first time use unsupervised test data to adapt to the new label distribution at test time**, which improves performance in both seen and unseen compositions. The entropy loss used during test overcomes the drawback that unseen compositions cannot be included for optimization during training stage.  \nWe believe that our method, from the angle of label space shift to solve the CZSL problem, will provide significant inspiration for future research in the CZSL community.\n\n2. Let us clarify the difference between our method and existing TTA models. TDA[2] records historical images without focusing text information; TPS[3] concentrates on shift vectors but fails to use visual information; DPE[4] updates all prototypes by directly adding shift vectors to them **offline** (failing to accumulate knowledge), which treats all categories equally regardless of familiarity.  \nOur work overcomes their limitations with visual priority queue, **online** knowledge accumulation modules, and adaptive update weights based on the similarity of test image and multi-modal prototypes. The adaptive update weight is very effective in addressing the challenge in CZSL where some compositions are seen while others are unseen. Therefore, **at methodology level, we refine the TTA methods, and carefully adjust the refined method with novel components to address the challenge of label space shift in CZSL**, instead of “A+B”.\n\n3. **There are many outstanding and valuable works building upon prior works in TTA and CZSL, instead of starting from scratch.** In TTA task, the core idea of **TPT**[1] is prompt tuning and data augmentation, which come from **CoOp**[5] and **MEMO**[7], respectively; the core component of **TDA**[2]--visual priority queue--comes from **Tip-Adapter**[6]; the core shift vector in **TPS**[3] is proposed by **SSF**[8]; in **DPE**[4], the visual cache and shift vector are inspired by **TDA**[2] and **TPS**[3]. In CZSL, the proposal of **CSP**[9] is to introduce prompt tuning to CZSL field from **CoOp**[5], inspiring many prompt-tuning based works; the core idea of **Troika**[10]--disentanglement--has shown its effectiveness in non-CLIP-based models[11,12]; the main component GNN in **GIPCOL**[13] is inspired by **Co-CGE**[14].  \n**The usage of prompt tuning technology in CZSL has in turn contributed to the development of prompt-tuning itself[10,15].** **Similarly, our work provides a new test-time approach to address CZSL, and it will in turn promote the development of TTA.**  \nSimilar to our work, these high-quality methods have **applied a particular technique to another practical problems after refinement in a adaptive and flexible manner and achieved strong results**. We cannot deny the valuable contributions they have made to the community.\n\n4. After rebuttal, we will revise our paper following your valuable suggestions--**further clarify the novelty and the differences with related methods in more detail**. In addition, considering that Reviewers hxuG, uhct and 45LQ have agreed that our method is novel (new) with good merits, we think our work presents an important insight for advancing CZSL, and believe that our research is worth sharing with the community.\n\n---\n\nQ2: Unclear citation.\n\nR2: We are very sorry that the citations in our paper have caused your misunderstandings. First, we have cited all methods and code inspiring our work in Related Work and Appendix J. In addition, we have made appropriate citations to Tip-Adapter[6], AdapterFormer[16], TDA[2], and CoOp[5] in Method Section; and we will appropriately add more citations of models such as TPS[3] and DPE[4] in this section.\n\nThank you for your kind reminder regarding the citations, which are now included in the current manuscript.\n\n---\n\nQ3: Unfairness in the experimental setup.\n\nR3: To alleviate the reviewer’s concerns about fairness, we provide experimental results under three different settings:  \n**(1) comparing different CZSL methods using test-time TOMCAT on CZSL datasets** (disentanglement-based models like Troika are not compatible with TOMCAT due to disentangling multi-prototypes) in Table 1;  \n**(2) comparing TOMCAT with TTA methods on standard TTA classification datasets** (e.g., TPT[1], TDA[2], TPS[3], and DPE[4]) in Table 2.  \n**(3) comparing TOMCAT and different TTA methods based on the Base Model on CZSL datasets** in Table 3.  \nThe results show that TOMCAT performs better than baselines with fairness in different fair settings.\n\n| Method            | UT-Zappos |       |       |        | MIT-States |       |       |        | C-GQA |       |       |        |\n|-------------------|------------|-------|-------|--------|-----------|-------|-------|--------|-------|-------|-------|--------|\n|                   | AUC        | HM    | Seen  | Unseen | AUC       | HM    | Seen  | Unseen | AUC   | HM    | Seen  | Unseen |\n| CSP[9]               | 31.24      | 46.55 | 60.9  | 62.62  | 19.13     | 25.92 | 46.68 | 49.67  | 5.85  | 19.45 | 27.54 | 26.92  |\n| CSP[9] + TOMCAT        | 32.26      | 49.27 | 64.32 | 57.69  | 19.48     | 26.77 | 49.96 | 49.96  | 6.09  | 19.8  | 28.88 | 26.75  |\n| Base Model        | 43.57      | 55.54 | 68.72 | **74.30**   | 22.12     | 38.97 | 49.50  | 52.61  | 10.73 | 27.92 | 36.63 | 24.39  |\n| Base Model + TOMCAT | **48.31**      | **60.18** | **74.49** | 72.77  | **22.55**     | **39.45** | **50.22** | **52.95**  | **11.05** | **28.48**  | **37.66**  | **34.18**  |\n\nTable 1. Comparison of the results of models with and without TOMCAT on three CZSL datasets.\n\n| Method | ImageNet | ImageNet V2 | Flower102 | DTD   |\n|--------|----------|-------------|------------|-------|\n| TPT[1]    | 69.54    | 63.95       | 69.71      | 46.93 |\n| TDA[2]    | 69.51    | 64.67       | 71.42      | 47.40 |\n| TPS[3]    | 71.45    | 64.91       | 71.54      | 50.47 |\n| DPE[4]    | 71.91    | **65.44**       | 75.07      | 54.20  |\n| TOMCAT | **72.09**   | 64.83       | **75.46**      | **56.03** |\n\nTable 2. Comparison of the top-1 classification accuracy among TOMCAT with TTA methods on four standard TTA classification datasets.\n\n| Method            | UT-Zappos |       |       |        | MIT-States |       |       |        | C-GQA |       |       |        |\n|-------------------|------------|-------|-------|--------|-----------|-------|-------|--------|-------|-------|-------|--------|\n|                   | AUC        | HM    | Seen  | Unseen | AUC       | HM    | Seen  | Unseen | AUC   | HM    | Seen  | Unseen |\n| online-TPS[3] | 44.59 | 57.29 | 71.26 | 71.44 | 22.21 | 39.09 | 49.62 | 52.8  | 10.84 | 27.87 | 37.51 | 33.65 |\n| TDA[2]        | 41.57 | 54.61 | 69.01 | 70.6  | 22.01 | 88.71 | 49.5  | 52.86 | 9.41  | 25.3  | 37.25 | 30.33 |\n| DPE[4]       | 43.32 | 55.97 | 68.49 | **74.17** | 22.02 | 38.9  | 49.24 | 52.78 | 10.69 | 27.88 | 36.57 | 34    |\n| TOMCAT | **48.31**      | **60.18** | **74.49** | 72.77  | **22.55**     | **39.45** | **50.22** | **52.95**  | **11.05** | **28.48**  | **37.66**  | **34.18**  |\n\nTable 3. Comparison of TOMCAT and different TTA methods based on the Base Model on CZSL datasets.\n\n---\n\nThis concludes our response to each of your comments. Once again, we wholeheartedly and sincerely express our gratitude to your time and thoughtful consideration. We would be happy to further discuss if you have any remaining concerns.\n\nBest regards,\n\nAuthors.\n\n&nbsp;\n\n[1] Manli Shu, et al. Test-time prompt tuning for zero-shot generalization in vision-language models. NeurIPS 2022.\n\n[2] Adilbek Karmanov, et al. Efficient test-time adaptation of vision-language models. CVPR 2024.\n\n[3] Just shift it: Test-time prototype shifting for zero-shot generalization with vision-language models. WACV 2025.\n\n[4] Ce Zhang, et al. Dual prototype evolving for test-time generalization of vision-language models. NeurIPS 2024.\n\n[5] Kaiyang Zhou, et al. Conditional prompt learning for vision-language models. CVPR 2022.\n\n[6] Renrui Zhang, et al. Training-free adaption of clip for few-shot classification. ECCV 2022.\n\n[7] Marvin Zhang, et al. MEMO: test time robustness via adaptation and augmentation. arXiv preprint 2021.\n\n[8] Dongze Lian, et al. Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning. NeurIPS 2022.\n\n[9] Nihal V. Nayak, et al. Learning to compose soft prompts for compositional zero-shot learning. ICLR 2023.\n\n[10] Siteng Huang, et al. Troika: Multi-path cross-modal traction for compositional zero-shot learning. CVPR 2024.\n\n[11] Nirat Saini, et al.  Disentangling visual embeddings for attributes and objects. CVPR 2022.\n\n[12] Shaozhe Hao, et al. Learning attention as disentangler for compositional zero shot learning. CVPR 2023.\n\n[13] Guangyue Xu, et al. GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning. WACV 2024.\n\n[14] Massimiliano Mancini, et al. Learning graph embeddings for open world compositional zero-shot learning. TPAMI 2022.\n\n[15] Wentao Bao, et al. Prompting language-informed distribution for compositional zero-shot learning. ECCV 2024.\n\n[16] Shoufa Chen, et al. Adaptformer: Adapting vision transformers for scalable visual recognition. NeurIPS 2022."}]

Author rebuttal:

Meta-review: The work addresses a meaningful challenge in CZSL, i.e., the distribution shift caused by unseen attribute-object compositions at test time. The authors propose a novel method that leverages unlabeled test data to dynamically update multimodal prototypes, incorporating an adaptive weighting mechanism and a priority queue to accumulate knowledge. This approach is justified as a way to adapt to the shifting label space without ground-truth labels.

The paper's strengths include its good empirical performance on standard CZSL benchmarks under both closed and open-world settings. The comprehensive evaluation and clear presentation are notable. Some concerns were raised regarding the fairness of comparison to methods that do not use test-time data, the scalability of the approach, and the degree of methodological novelty relative to existing test-time adaptation techniques. The authors' rebuttal has provided additional experiments comparing their method to MLLMs and other TTA techniques, demonstrated robustness to test-time ordering, and included results on standard classification datasets to show broader applicability. These responses adequately addressed most reviewer concerns, leading several reviewers to raise their scores. 

Given the above conditions, I recommend an acceptance.
Decision: Accept (poster)