Visual-Language Collaborative Representation Network for Broad-Domain Few-Shot Image Classification

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-01 04:03:40

Abstract: Visual-language models based on CLIP have shown remarkable abilities in general few-shot image classification. However, their performance drops in specialized fields such as healthcare or agriculture, because CLIP's pre-training does not cover all category data. Existing methods excessively depend on the multi-modal information representation and alignment capabilities acquired from CLIP pre training, which hinders accurate generalization to unfamiliar domains. To address this issue, this paper introduces a novel visual-language collaborative representation network (MCRNet), aiming at acquiring a generalized capability for collaborative fusion and representation of multi-modal information. Specifically, MCRNet learns to generate relational matrices from an information fusion perspective to acquire aligned multi-modal features. This relationship generation strategy is category-agnostic, so it can be generalized to new domains. A class adaptive fine-tuning inference technique is also introduced to help MCRNet efficiently learn alignment knowledge for new categories using limited data. Additionally, the paper establishes a new broad-domain few-shot image classification benchmark containing seven evaluation datasets from five domains. Comparative experiments demonstrate that MCRNet outperforms current state-of-the-art models, achieving an average improvement of 13.06% and 13.73% in the 1-shot and 5-shot settings, highlighting the superior performance and applicability of MCRNet across various domains.

Main paper: 
 introduction: 
 Few-shot image classification (FSIC) is a fundamental task in computer vision that has garnered widespread attention in recent years [38, 40, 42] . FSIC aims for models to acquire meta-knowledge from a large number of base classes and subsequently adapt rapidly to novel classes using a few support images, enabling the classification of query images. Numerous visual few-shot learning (FSL) models based on meta-learning [12, 23, 25, 30] or metric learning [17, 39, 44, 48] have been employed to tackle FSIC, but they have not yielded satisfactory performance. Recently, the emergence of CLIP [34] presents a new multi-modal view to address FSIC. Based on CLIP, the state-of-the-art (SOTA) visual-language methods (VLMs) [14, 51, 53] have showcased impressive performance on general domain datasets, achieving over 60% accuracy on Im-ageNet [8] and 90% accuracy on Caltech-101 [11] with only one support image.However, when existing VLMs are applied to specific fields such as medicine [33, 45] , agriculture [24] , or industry [15] , their performance is less than ideal. This deficiency arises from the challenge that CLIP's pre-training classes are difficult to encompass all categories across various domains, leading to its limited capability in representing and aligning images and text from unfamiliar categories. Existing works focus on enhancing CLIP by designing new text prompts or adapters, yet they fail to address CLIP's poor generalization when confronted with unfamiliar domains. For example, in fine-grained butterfly classification, as shown in Fig. 1 (a ) and (b), CLIP struggles to accurately extract information from new categories such as "cabbage butterfly" or "pachliopta aristolochiae butterfly", resulting in biased matching computations. Other models based on CLIP fail to align features of images and text from new classes, hence yielding inaccurate results.To address the issue, this paper introduces a novel visual-language collaborative representation network (MCRNet) that aims to learn a generalized capability for aligning and representing multi-modal information. As depicted in Fig. 1 (c ), MCRNet comprises three components: visual-text encoders, a collaborative relation learner, and a multi-feature re-presentation module. The visual-text encoders are used to extract prototype features of multi-modal information. The collaborative relation learner extensively interacts with prototype features of support or query images and texts and generates relationship matrices for support or query. The re-presentation module recalculates the prototype features and relationship matrices through weighted computations. What sets it apart from existing methods is that MCRNet coordinates the representation and fusion processes of visual and language modalities, which strengthens the alignment between multi-modal information by learning relationships among different modal semantics. This ability to generate relationships between different modalities to enhance representation can be widely applied in various tasks, allowing MCRNet to adapt more quickly to new category knowledge. To better apply to new category tasks, a class-adaptive fine-tuning inference method is also proposed, aiming to rapidly learn from the given support data and save testing time under the 𝑛-support images scenario.Furthermore, this paper establishes a new broad-domain fewshot image classification (BD-FSIC) benchmark to evaluate the generalization and applicability of existing models across a wide range of domain tasks. This benchmark comprises seven evaluation datasets covering five domains including biology, agriculture, medicine, mining industry, and archaeology. Finally, the evaluation studies on the BD-FSIC benchmark validate that existing visuallanguage models exhibit notably lower performance on unfamiliar domain tasks when compared to general datasets. The comparative experiments show that MCRNet surpasses the current SOTA models, achieving an average improvement of 13.06% in the 1-shot scenario and 13.73% in the 5-shot scenario, outperforming both existing visual-language models and visual few-shot learning models. These results demonstrate the superiority of MCRNet, as well as its transferability and generalizability across multi-domain tasks. In summary, this paper has the following contributions:  
related work 2.1 few-shot learning: 
 Visual few-shot learning (FSL) is the primary approach used to address few-shot image classification tasks. Existing FSL models mainly focus on a purely visual perspective and can be categorized into three types. The first type involves prototype representation learning [6, 28, 35, 36, 42, 47, 49] , which focuses on learning more generalizable feature representations to classify query images quickly on new categories after fine-tuning. The second type is based on metric learning [17, 39, 44, 48] , which deals with how to measure feature distances in the manifold space, and this metaability of measurement can be transferred to new categories. The third type is based on meta-learning [12, 23, 25, 30] , known as "learning to learn", where multiple different tasks are constructed during pre-training to learn a general classification ability from these tasks, which is then applied to new categories. Recently, more research has been focusing on transferring FSL models to specific domains such as healthcare [5, 16, 31] and industry [13, 21] . However, their performance decreases compared to general domains because, in specific domain applications, textual descriptions play a crucial role in capturing important information in images. Therefore, this paper aims to draw inspiration from representation learning methods in visual FSL and fully utilize the textual information provided by support to address the aforementioned challenges.In recent years, there has been a growing focus on leveraging language cues to enhance the performance of visual tasks. This has led to significant attention being drawn to vision-language models (VLMs) [26, 32, 50] , which are pre-trained using a vast amount of image-text pairs readily available on the internet and can be directly applied to downstream visual tasks. In particular, the introduction of the CLIP [34] in the embedding space. This approach allows pre-trained VLMs to capture intricate vision-language correspondence knowledge. Subsequently, CLIP-based models [14, [51] [52] [53] have been proposed to enhance CLIP's performance in few-shot image classification. These models primarily achieve this by designing adapters that can be quickly fine-tuned or by incorporating methods such as image matching. They leverage the multi-modal information features generated by CLIP to compute similarities between the test image and prompt text or between the test image and known images to determine the category of the test image. However, excessive reliance on features generated by CLIP can lead to representation biases when the model encounters text and images of categories that were rarely seen or unseen during pre-training. This bias in representations can result in classification errors when further matching is performed based on these representations. To improve the performance of existing VLMs in unfamiliar domains or categories, such as fine-grained butterfly classification in the field of biology or plant virus classification in agriculture, this study proposes a novel visual-language cooperative representation method to learn a highly generalizable multi-modal information representation for multi-domain few-shot image classification.FSL models and VLMs are evaluated on general domain benchmarks such as miniImageNet [20] , CIFAR [11] , CUB [9] , or tiered-ImageNet [11] . However, there is a scarcity of evaluation datasets specific to domains. Despite the introduction of a cross-domain few-shot learning benchmark by Guo Y et al. [18] , which only includes two datasets from the medical domain and one from the agricultural domain. To bridge this gap, this study establishes a new benchmark encompassing five domains with seven datasets, to offer a comprehensive platform for assessing model transferability and generalization across diverse domains in image classification. 
method 3.1 problem formulation: 
 In broad-domain few-shot image classification (BD-FSIC), we adhere to the classic FSIC problem setting, where the model is pretrained on a large-scale base class dataset C base and then evaluated on novel classes C novel in unfamiliar domains. Both training and evaluation are conducted in 𝑁 -way-𝐾-shot episodes [37] , and calculates the model's accuracy. Hence, for visual FSL models and VLMs, the key to addressing BD-FSIC lies in learning more generalized representations or aligning meta-knowledge from C base and effectively leveraging the relevant information from 𝐼 𝑠 𝑘 𝑖 in C novel .The proposed MCRNet is designed to achieve feature alignment with generalization capabilities through a collaborative fusion and representation method for vision and text. As illustrated in Fig. 2 , MCRNet consists of three components: the prototype representation based on CLIP, the collaborative relation learning, and the re-representation part. The prototype representation comprises text and visual encoders from CLIP, mapping input images and text information into prototype features. The collaborative relation learner, including a fusion layer and two unidirectional 3Dconvolutional layers, fuses visual-text prototype features and learns the relationship matrix between support and query information.The multi-feature re-presentation learner incorporates a simple self-attention layer to relearn the alignment of the relationship matrix and prototype features for a refined feature representation. The entire network is supervised by a contrastive learning loss, aiming to bring similar support and query features closer while pushing different-class features apart. Additionally, a category-adaptive finetuning method was proposed to assist MCRNet in rapid learning on limited data. . Following this, an attention mechanism is utilized, defined as:EQUATIONWe perform self-attention calculations on 𝑢 𝑠 𝑖 and 𝑢 𝑞 𝑖 separately. Since the fused features include both image and text features, important category features in the fused features are assigned higher weights during multiple similarity calculations. This is expressed as: 𝑖 , we assume that 𝐿 × 𝑀 × 𝑁 is the shape of the 3D convolutional kernel. 𝑊 represents the weight at position 𝑙, 𝑚, 𝑛 of the kernel, and we set 𝑁 to 1. When learning the relational matrix 𝜔 𝑞 for 𝑓 𝑢 𝑞 𝑖 , we set 𝐿 to 1. The specific operations are as follows:EQUATION𝜔 𝑠 =𝐹 ( 𝐶 ∑︁ 𝑐 1 ∑︁ 𝑛 𝑙 =0 𝑊 ∑︁ 𝑤 𝑙 =0 𝐻 ∑︁ ℎ 𝑙 =0 𝑊 (ℎ 𝑙 ,𝑤 𝑙 ,𝑛 𝑙 )𝑓 𝑢 𝑐,(𝑙+ℎ 𝑙 ),(𝑚+𝑤 𝑙 ),(𝑛+𝑛 𝑙 ) ) + 𝑏 (ℎ 𝑙 ,𝑤 𝑙 ,𝑛 𝑙 ) ,(3)EQUATIONwhere 𝐹 (.) is an activation function and 𝑏 (ℎ 𝑙 ,𝑤 𝑙 ,𝑛 𝑙 ) is the bias of the computed feature map. Thus, Based on the collaborative relation learner, we obtain the weight relational matrix 𝜔 𝑞 representing the impact of query fused features on support fused features, and the relational matrix 𝜔 𝑠 representing the weighted impact of support fused features on query fused features.We concatenate the obtained relational matrices 𝜔 𝑠 and 𝜔 𝑞 with the prototype features of support and query, 𝑓 𝑠 𝑖 and 𝑓 𝑞 𝑖 respectively. Subsequently, we pass them through a self-attention layer and a linear mapping layer to regenerate the final fused features of support and query, 𝐹 𝑠 𝑖 and 𝐹 𝑞 𝑖 :EQUATIONThe resulting features are 𝐹 process beyond the prototype representation:EQUATIONwhere I(𝑦 𝑠 𝑖 == 𝑦 𝑞 𝑖 ) indicates that if the class labels of support and query are the same, it is 1; otherwise, it is 0. The 𝑑 (𝐹 𝑞 ,𝐹 𝑠 refers to the L2 distance. By reducing the distance between support and query instances of the same class and increasing the distance between instances of different classes, the goal is to enable the Collaborative Relation Learner to align multi-modal information of the same class and separate multi-modal information of different classes. This process helps in learning the final support-query relational matrix. The ability to incorporate multi-modal information of the same and different classes into the relation learning process is category-agnostic and can be generalized to new classes.During the evaluation process, to fully utilize the support information, we design a fast fine-tuning method for MCRNet. Since the relational matrices learned in the collaborative relation learner exhibit strong generalization properties, we only fine-tune the Rerepresentation part. This is why the Re-representation learner is designed to be lightweight. Taking 5-way-5-shot as an example, where 5 classes are randomly selected from C novel , each with five support images 𝐼 𝑠 𝑖 and 𝑇 𝑠 𝑖 , we augment each 𝐼 𝑠 𝑖 into 𝑁 images using random rotations, cropping, and other data augmentation techniques. These augmented images are then combined with different 𝑇 𝑠 𝑖 and fed into MCRNet to obtain multiple relational matrices. By randomly combining an augmented support image with the relational matrix generated from that image, we can create multiple sets of new class data for re-representation. These class data are sequentially input into the re-representation network as either the same class or a different class to fine-tune MCRNet. This method alleviates overfitting issues caused by the limited number of support instances and helps the model learn more accurate class distributions for new classes. 
the proposed benchmark: 
 This paper constructs a new broad-domain few-shot image classification benchmark (BD-FSIC), aiming to provide a comprehensive evaluation platform for existing methods in the field of image classification. As shown in Fig. 3 , this benchmark covers five domains: Biology, Agriculture, Medicine, Mining, and Archaeology, encompassing evaluation datasets for seven different domain-specific tasks. The biological domain includes three datasets for different classification tasks, while each of the other domains contains one dataset. Furthermore, except for Animal, the other seven datasets are finegrained classification datasets as these tasks are more challenging and have practical applications. Therefore, except for Animal, the other seven datasets are fine-grained. The descriptions of these datasets are provided below:• Animal is a coarse-grained dataset containing 34 animal categories, with a total of 50, 304 images sourced primarily from the COD10K dataset [10] and collected from the web. • Insect is a fine-grained classification dataset comprising 70 categories and 22, 242 images. It is sourced from the InsectD dataset [43] and collected from the web. Notably, the category of butterflies alone includes 18 species, challenging models to accurately differentiate between closely related categories with minimal intra-class variations. All the above methods were evaluated on the seven datasets of the BD-FSIC benchmark mentioned in Sec. 4. Testing Strategy. During the test phase, a standardized 𝑁 -way-𝐾shot approach was used to select support images, with 15 query images sampled per class. The reported results in the tables are presented in both 5-way-1-shot and 5-way-5-shot formats, where 5 novel classes are randomly selected each time, with 1 or 5 support images per class. This constitutes a single-episode test. To ensure fairness, each method underwent 600 random tests. The reported metrics include the average accuracy and a 95% confidence interval.All results are presented as accuracy, representing the proportion of correctly predicted outcomes to the total count. Implementation Details. All models were trained and tested on the same GPU. In the case of MCRNet, the training process was parallelized across eight NVIDIA A800-SXM4-80GB GPUs. AfterVisual-Language Collaborative Representation Network for Broad-Domain Few-Shot Image Classification MM '24, 28 October -1 November 2024, Melbourne, Australia We employed data augmentation techniques such as rotation and cropping to generate five images per support image, forming 100 image-text triplets for 5-way-5-shot. In the 𝑛-shot setting, the prototype features of each class's images were averaged to obtain class prototype features for subsequent predictions. 
comparative experiments and analysis: 
 We compared MCRNet with SOTA visual FSL models, VLMs, and the baseline CLIP on the BD-FSIC benchmark, as shown in Tab. 1 and 2. Apart from a slight 0.51% lower performance compare to HTCTrans in the 5-way-1-shot setting on the ISIC dataset, MCR-Net outperform all other models in all settings on the remaining datasets. On average across the seven datasets, MCRNet surpass the second-best model by 13.06% in the 1-shot and 13.73% in the 5-shot. Specifically, on the coarse-grained Animal, MCRNet outperform VLMs, particularly surpassing APE-T by 4.36% in the 5-shot setting and the best FSL model by 32.26%. On the Insect dataset, MCRNet's performance is even more remarkable, significantly outperforming existing methods by 22.80% and 22.06%. This demonstrates MCRNet's ability to generalize effectively across different classification granularities, handling scenarios with small intra-class variances. MCRNet maintains a stable advantage on other finegrained datasets as well, especially on ISIC where its 5-shot results exceed the second-best model. Compared to MCRNet's baseline CLIP, MCRNet show an improvement of 17.43% and 29.76% on average. These results collectively showcase the superiority of our approach, highlighting its strong generalization and practical applicability across multiple domain datasets. Furthermore, we have the following discoveries and analyses: 1) Weaknesses of VLMs: Visual FSL models outperform VLMs on OracleScript and LeafVirus. This demonstrates that existing VLMs overly rely on the representational abilities learned during CLIP pretraining. When faced with unfamiliar tasks, the textual features in CLIP fail to provide the image encoder with accurate cues, resulting in the image encoder's performance being inferior to that of pretraining models trained solely on visual data. The proposed MCRNet effectively addresses this limitation by incorporating relationship learners that conduct relation learning between support and query images with text. These fused image-text features undergo a re-representation process, correcting the representational biases introduced by CLIP's unfamiliarity. Besides, this also suggests the need to design supplementary representational structures when applying VLMs in specific domains, rather than solely relying on simple adapters or metric enhancements.2) N-shot inference comparison: We conducted a comparison between the performance of MCRNet and two top-performing models across different 𝑛-shot scenarios, as in Fig. 4 . It is clear that as the amount of provided support data increases, the growth trend of conventional VLMs is not as significant as that of visual FSL models and MCRNet. For example, in the case of 1-shot scenarios, CLIP's accuracy is 4% lower than MCRNet, but by the 15-shot mark, CLIP lags behind MCRNet by almost 25%. Results from CLIP-adapter show a slight improvement but still trail MCRNet by 4% in the 1-shot scenario, surpassing it by more than 15% in the 15-shot scenario. These comparisons underscore that, in contrast to existing VLMs, MCRNet effectively utilizes support image information, swiftly grasping the data distribution of new classes through categoryadaptive fine-tuning methods. Furthermore, when compared to FSL models, MCRNet adeptly employs textual cues, maintaining its superiority over them.3) Dataset analysis: Existing models perform well in general domains, but they show overall poor performance on the BD-FSIC benchmark, especially on the skin disease classification dataset ISIC and the OracleScript dataset for Oracle bone script recognition. Even with an increase in the number of support provided, their performance improvement remains slow. This is because the representation attention of these medical data often focuses on local features such as color and texture, rather than the target shapes Figure 5 : The comparison between the results of MCRNet after fine-tuning and the original results. The baseline is CLIP [34] .in general data. Oracle bone script fonts are more sensitive to extracted features because of the significant feature variance within similar fonts. These results indicate the need to pay more attention to the generalization performance of enhancement methods in domain applications to meet practical requirements, while also highlighting the importance of the proposed BD-FSIC benchmark. The Effectiveness of the Class-adaptive Fine-tuning Inference. To fully utilize support images and text information, we designed a class-adaptive fine-tuning inference method for MCRNet. In the 5-way-5-shot scenario, 100 image-text pairs were constructed using support images, and so forth. The results in Fig. 5 demonstrate its effectiveness. It can be observed that with five or more support images, this fine-tuning technique consistently boosts performance by 6% on Ore, 2% on Animal, and around 5% on LeafVirus compared to the non-fine-tuned model. The results in the figure are based on a single iteration of constructed data. Tab. 3 illustrates the impact of different numbers of iterations on the results. It is evident that after more than 5 iterations, the model tends to overfit due to the limited data for fine-tuning. Iterating once yields the optimal performance on average with the least time consumption. Therefore, we set the standard number of fine-tuning iterations for MCRNet as one.The Flexibility of MCRNet. MCRNet integrates and re-represents multi-modal information relationships based on prototype features, To address the under-performance of existing visual few-shot learning models and CLIP-based vision-language models in domainspecific tasks, this paper introduces a novel vision-language collaborative representation network. Building upon CLIP, this network innovatively integrates and collaborates visual and textual features for joint feature fusion and representation, enabling the learning of aligned representations that generalize to new classes. Furthermore, a new evaluation benchmark comprising five domains with seven datasets is proposed to offer a comprehensive domain image classification assessment platform. Comparative experiments demonstrate the superiority and generalization of our approach, with extension experiments showing MCRNet's flexibility in integration with other methods to enhance performance. In the future, we aim to incorporate large language models to enrich existing semantic information, and explore more detailed descriptions to guide multi-domain visual tasks.  
 
 Appendix:  


Decision: This paper introduces MCRNet, a novel visual-language collaborative representation network to improve few-shot image classification in specialized fields like healthcare and agriculture. Unlike existing CLIP-based models, MCRNet generates relational matrices for multi-modal feature alignment, which is category-agnostic and generalizes to new domains. It also employs class adaptive fine-tuning for better learning with limited data. The new broad-domain few-shot benchmark shows MCRNet's superior performance, achieving significant improvements over state-of-the-art models.

Pros
- Establishes a comprehensive benchmark covering multiple domains, providing a robust platform for evaluating few-shot learning models.
- MCRNet significantly outperforms existing state-of-the-art models, with notable improvements in 1-shot and 5-shot settings.
- The paper is well-written, well-organized, and easy to follow.
- Conducts thorough experiments to validate MCRNet's effectiveness across various domains.
- Addresses the practical issue of CLIP's generalization capability in few-shot classification tasks involving arbitrary categories.
- Introduces a novel approach for category-agnostic representation learning, enhancing semantic alignment through collaborative relation modeling.

Cons
- The class-adaptive fine-tuning method might lead to overfitting with limited support data, as indicated by performance drops after more iterations.
- MCRNet relies on pre-trained CLIP parameters, which may limit applicability if these parameters are unavailable or if there are significant domain shifts.
- While the paper includes fine-grained datasets, further evaluation on more diverse and larger-scale datasets could strengthen the evidence of MCRNet's generalizability.
- Important related works, such as Li and Wang (2023), which address similar problems of visual-language integration, are not compared.
- The paper mentions issues with CLIP handling unfamiliar domains and fine-grained classification but does not provide separate discussions or solutions for these scenarios.
- The paper should include results on commonly used datasets to further demonstrate the robustness and generalizability of MCRNet.

The rebuttal has addressed most of the cons from reviewers, and all reviewers agree this paper should be accepted.

Reviews: review: paper_summary: paper_summary: The paper presents MCRNet, a visual-language collaborative representation network for improving few-shot image classification in specialized fields. MCRNet enhances multi-modal feature alignment through relational matrices and includes visual-text encoders, a collaborative relation learner, and a re-presentation module, plus a class-adaptive fine-tuning technique. A new benchmark with seven datasets from five domains is introduced. Experiments indicate that MCRNet outperforms current state-of-the-art models across various domains.
strength_weakness: strength_weakness: 1. The paper establishes a new benchmark that comprehensively covers multiple domains, providing a robust platform for evaluating few-shot learning models.
2. MCRNet significantly outperforms existing state-of-the-art models, achieving notable improvements in both 1-shot and 5-shot settings across various datasets.
limitations: limitations: 1. The class-adaptive fine-tuning method might lead to overfitting, especially with limited support data, as indicated by the performance drop after more iterations of fine-tuning.
2. MCRNet relies on pre-trained CLIP parameters, which might limit its applicability if these parameters are not available or if there are significant domain shifts from the pre-training data.
3. While the paper includes fine-grained datasets, further evaluation on more diverse and larger-scale datasets could strengthen the evidence of MCRNet's generalizability.
paper_summary: paper_summary: This work proposes a new method (MCRNet) to integrate the vision and language information for few-shot image classification.
Empirical results demonstrate the effectiveness of MCRNet.
strength_weakness: strength_weakness: 1. The paper is well-written and the idea is easy to follow.
2. Extensive experiments are conducted to demonstrate the effectiveness of MCRNet.
limitations: limitations: Some important related works are missing:
[1] deals with the same problems of visual-language integration, but not compared with.
[1] Li, Zhuoling, and Yong Wang. "Better Integrating Vision and Semantics for Improving Few-shot Classification." Proceedings of the 31st ACM International Conference on Multimedia. 2023.
paper_summary: paper_summary: This paper presents a novel multi-modal collaborative representation network (MCRNet) designed to enhance the alignment and representation capabilities of multi-modal information in unfamiliar domains. Contrary to existing approaches, MCRNet employs a universal relational matrix learning methodology to streamline the fusion and feature representation of multi-modal information. This paper establishes a new benchmark for broad-domain few-shot image classification, comprising seven evaluation datasets from five diverse domains.
strength_weakness: strength_weakness: +This paper is well-written, well-organized, and easy to understand.

+The application scenario of few-shot classification tasks inherently involves arbitrary categories. Addressing CLIP's generalization capability in such scenarios is a very practical issue.

+This paper constructs a meaningful benchmark and proposes an effective method to address the issue.

+This paper conducts extensive experiments to validate the performances of this method under various domains.
limitations: limitations: -In the introduction, the authors mention that CLIP is not adept at handling unfamiliar domains. The authors cite examples of fields such as medicine and industry that are not well-covered by CLIP, and also points out issues with fine-grained classification. Whether these two scenarios require separate discussions and solutions?

-Apart from conducting experiments on domains that CLIP is unfamiliar with, it is recommended for the authors to also report the results on the commonly used datasets. This would further demonstrate the robustness and generalizability of the proposed approach.
paper_summary: paper_summary: The paper introduces MCRNet, a visual-language collaborative representation network designed to improve few-shot image classification by generating category-agnostic relational matrices for better multi-modal feature alignment. Additionally, the authors also introduce a class-adaptive fine-tuning inference technique and a brand-new benchmark. Experiments on this benchmark show that MCRNet outperforms current models, achieving significant improvements in 1-shot and 5-shot settings.
strength_weakness: strength_weakness: 1. This paper addresses a fundamental problem in few-shot learning, which is the category-agnostic representation learning. It differs from existing CLIP-based methods in that it introduces an extra step to modeling the collaborative relation between the text, query, and support, enabling more comprehensive semantic alignment. It is not an intricate idea, but personally I think the novelty is acceptable.
2. The authors propose a new benchmark for broad-domain few-shot learning in this work.
3. The experimental design is good, and the results are satisfactory.
limitations: limitations: Major concerns:
1. Besides accuracy, other metrics such as F1 should also be reported in the table.

Minor concerns:
1. Please improve consistency throughout the paper. It seems like "re-presentation" and "re-representation" are chaosly used in the paper.
2. I would suggest the authors add some more intuitive introduction to existing few-shot learning methods if that is possible.