PS-TTL: Prototype-based Soft-labels and Test-Time Learning for Few-shot Object Detection

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 05:15:02

Abstract: In recent years, Few-Shot Object Detection (FSOD) has gained widespread attention and made significant progress due to its ability to learn models with strong generalization power using extremely limited annotated data. Although the fine-tuning-based paradigm for FSOD has become mainstream, where detectors are initially pretrained on base classes with sufficient samples and then fine-tuned on novel classes with few annotated samples, the scarcity of samples in novel classes hampers the precise capture of their data distribution. To address this issue, we propose a novel framework for few-shot object detection, namely Prototype-based Soft-labels and Test-Time Learning (PS-TTL). Specifically, we design a Test-Time Learning (TTL) module that employs a mean-teacher network for self-training to discover novel instances on test data, effectively alleviating the problem of overfitting to the distribution of base class. Furthermore, we develop a Prototype-based Soft-labels (PS) strategy via assessing similarities between pseudo-labels and category prototypes to unleash the potential of low-quality pseudo-labels, thereby significantly mitigating the constraints posed by few-shot samples. Extensive experiments on both the VOC and COCO benchmarks show that PS-TTL achieves a new state-of-the-art, highlighting its effectiveness.

Main paper: 
 introduction: 
 Object detection [24, 39, 44, 50] is a fundamental computer vision task and has a variety of applications, including autonomous driving [58, 63] , robotics [31, 37] , medicine [23, 27] , etc. Although significant progress has been archived in recent years [35, 43, 57, 60] , these detectors heavily rely on a large number of training samples. On the other hand, humans can quickly extract novel concepts from a small amount of data. For example, children can learn to identify objects of novel categories after viewing a few pictures. The deep object detectors are also supposed to be able to learn effectively in data-limited scenarios because labelling data is quite expensive, Few-Shot Object Detection (FSOD) is a promising way to address this issue. It aims to train an object detector using only a few samples on novel classes with the help of abundant data on base classes, attracting widespread attention from researchers. Early FSOD methods typically adopt the meta-learning paradigm, organizing the task into a series of episodes simulating FSOD scenarios, where each episode includes few-shot training (support) and test (query) sets. The support set is utilized for model training with a limited number of samples, while the query set is employed to assess the model's detection performance on novel objects. Kang et al. [16] propose a lightweight feature reweighting module that learns to capture the global features of support images and embeds such features into reweighting coefficients to adjust the meta features of the query image. Meta R-CNN [52] develops a meta-learner, known as the Predictor-head Remodeling Network (PRN), leveraging a common Faster R-CNN [39] backbone to efficiently extract features from support images. Subsequently, meta-learning based works progress from optimizing both classification and localization features [6, 9, 11, 46, 53] , employing Transformer for capturing spatial relationships between support and query classes [13, 56] , as well as exploring inter-class relationships [14, 17, 29, 59] . However, such methods involve complex architectures and training procedures, leading to increased computational complexity and costs. Additionally, they suffer from poor interpretability of what the model learned in the novel stage.To facilitate faster training and simple deployment for rapid adaptation to novel classes, most existing FSOD methods employ a fine-tuning based paradigm. The detector is first pre-trained on base classes with adequate samples, then fine-tuned on novel classes with few annotated samples. Early methods [4, 48] employ a jointly fine-tuning based architecture, where the entire pre-trained base model, comprising both the class-agnostic and class-specific layers, is updated simultaneously during training on the novel task. Later, the two-stage fine-tuning approaches [2, 8, 32, 40, 45, 47, 62] demonstrate that maintaining the feature extraction part of the model unchanged and solely fine-tuning the last layer can significantly enhance detection accuracy. Based on this fact, most of the subsequent methods combined with knowledge distillation [33, 34, 49] , context reasoning [19, 65] , or decoupling detection networks [30, 36, 54] to further improve the detection performance. However, constrained by the limited samples for novel classes, they struggle to accurately capture the data distribution. Some works attempt to address this issue by generating synthetic data for novel classes [61, 64] or mining implicit novel instances from the training set [3, 18, 41] . However, the former method relies on information from base classes to synthesize novel samples, which may not accurately reflect the true distribution as shown in Fig. 1(a) . The latter approach relies on the assumption that unlabeled novel instances are widely present in abundant base data as shown in Fig. 1 (b), which may not hold true in real-world scenarios.Considering the accessibility of novel instances in the test data, it motivates us to explore fine-tuning an object detection model at test-time as shown in Fig. 1(c ). Compared to mining novel instances from base class data (the presence of unlabeled novel instances in base class data essentially represents a loophole in few-shot object detection settings), conducting online learning on test data is a more realistic approach aligned with real-world applications. In this paper, we first propose a Test-Time Learning (TTL) module, which utilizes a mean-teacher network for self-training to simultaneously train and test on test data, effectively leveraging the novel class data present in the test data. Specifically, both the student and teacher networks are first initialized by the FSOD detector fine-tuned on novel data. Then, the teacher network takes test data as input to generate pseudolabels. The student model is trained using the pseudo-labels after post-processing and N-way-K-shot data as supervision signals and updates the teacher network through exponential moving average. Additionally, considering the limited number of high-quality pseudo labels and the fact that a large number of low-quality pseudo labels can recall most of the foreground but exhibit low classification accuracy, we develop a Prototype-based Soft-labels (PS) strategy to unlock the potential of these low-quality pseudo-labels. Specifically, we maintain class prototypes and compute the feature similarity between low-confidence pseudo-labels and class prototypes to replace them with soft-labels. Class prototypes are initialized using N-way-K shot data and dynamically updated during online learning using the instance features of high-confidence pseudo-labels. Finally, we integrate the aforementioned two modules into a novel framework for few-shot object detection, dubbed PS-TTL.In summary, the major contributions of this paper are:• We propose a novel PS-TTL framework for few-shot object detection, which effectively mines new instances from test data to address the issue of limited novel class samples. To the best of our knowledge, it is the first attempt to explore fitting novel class data distributions in a way that is more in line with real-world scenarios.  
related work 2.1 object detection: 
 Object detection aims to identify and localize objects within images, constituting a fundamental challenge in computer vision. Recently, the success of deep learning has yielded numerous effective object detection methods. These methods can be categorized into two main groups: two-stage and one-stage.Single-stage detectors (e.g., SSD [26] and RetinaNet [24] ) predict bounding boxes and classification scores based on predefined anchor boxes, exhibiting strong real-time performance. Subsequent anchor-free detectors [43, 60] alleviate the prior constraints of predefined anchors, further streamlining the detection process. The YOLO [38, 44] series, by continuously assimilating the latest advancements in object detection, such as label assignment and multi-scale feature fusion techniques, has achieved high-precision real-time object detection. Although the structure of single-stage detectors is straightforward, their integrated design also makes them less adaptable to FSOD tasks.Two-stage detectors (e.g., Faster R-CNN [39] and Double-Head [50] ) usually first use an region proposal network (RPN) to propose potential proposals, which are then refined by other modules. Methods such as Cascade R-CNN [1] and HTC [5] employ multi-stage refinement, further enhancing the detection precision. Compared to single-stage detectors, two-stage detection frameworks achieve higher detection performance. The concept of multi-stage refinement is also widely employed in the recently transformer-based detectors [35, 57] . Due to the design of multi-stage refinement, FSOD can achieve few-shot fine-tuning by controlling the gradients obtained by each stage of the detection module, effectively mitigating the issue of knowledge forgetting [45] . Two-stage detectors also facilitate the extraction of instance features for metric learning, making them commonly used in FSOD research [51] .The FSOD methods enable detectors to swiftly adapt to new tasks with minimal data while preserving their original performance, enhancing the adaptability of models under data-constrained circumstances. FSOD methods can be broadly categorized into two paradigms: meta-learning based and fine-tuning based.Meta-learning based methods employ numerous N-way K-shot detection tasks [55] for training, aiming to quickly adapt to new tasks with few support samples. FSRW [16] and Meta R-CNN [52] propose feature reweighting strategies on single-stage and two-stage detectors, respectively. They extract class-specific representations from support images and combine them with weighted queries to achieve detection for specific categories. Attention-RPN [9] integrates support information into the Region Proposal Network (RPN) and employs a contrastive training strategy to enhance the relevance between proposals and support classes. MetaDet [46] disentangle the learning of category-agnostic and category-specific components in detectors better to tackle few-shot classification and localization in a unified way. Recent efforts to improve meta-learning based approaches include introducing metric learning to enhance feature discriminability. CME [22] utilizes class margin loss to preserve sufficient margin space for novel classes. TIP [21] introduces consistency regularization on image transformation to enhance the model's generalization ability. Meta-learning methods enable detectors to rapidly adapt to new categories. However, the training and inference processes of these methods are highly complex, making them challenging to deploy in real-world scenarios.Fine-tuning based methods adopt the two-stage training strategy, i.e., base training and then few-shot fine-tuning, which expects to transfer the prior knowledge from base classes to the novel classes. LSTD [4] is the earliest method to employ the two-stage training strategy for FSOD, using regularization to retain base knowledge. TFA [45] simply freezes the backbone and only fine-tunes the box classifier with instance-level normalization. Based on TFA, FSCE [40] introduces contrastive learning to learn the discriminative object proposal representations, alleviating the misclassification issues in novel classes. Subsequent research refine the TFA method and integrate it with other techniques to further enhancing the FSOD performance [10, 19, 30, 33, 34, 36, 49, 54, 65] . DeFRCN [36] employs the gradient decoupled layer to stop the gradient from the detector head, aiming to preserve generic knowledge of base classes while gradually extracting novel information in examples. PTF [54] devise an effective method for initializing novel class weights and propose an adaptive length re-scaling strategy to enhance classification precision. Although fine-tuning-based methods are simple for deployment, their generalization primarily relies on extensive pretraining with base data. When the disparity between novel and base classes is significant, detectors still struggle to adapt to novel domains with few samples.Semi-supervised learning has been applied in FSOD to enrich the diversity of novel samples and address the issue of inconsistent label assignment for novel classes [3, 18, 41] . Kaul et al. [18] introduce a simple pseudo labelling strategy to detect potential novel instances in the base dataset. MINI [3] introduces a teacher-student framework and performs online parameter updates, enabling better novel instance mining. Tang et al. [41] propose a class-adaptive threshold filtering strategy to select more valuable pseudo labels. The core assumption of these methods is that novel instances appear frequently in the base dataset, which may not hold in real-world scenarios. However, novel instances are guaranteed to appear in the test set, making our TTL method more practically valuable. 
methods: 
 In this section, we initially review the problem setting of conventional few-shot object detection in Section 3.1, followed by a brief introduction to our baseline method, DeFRCN [36] , in Section 3.2. Subsequently, we elaborate on our Test-Time Learning (TTL) module in Section 3.3 and Prototype-based Soft-labels (PS) strategy in Section 3.4. Finally, we delve into the training process of the entire framework in Section 3.5.We follow the standard few-shot object detection setting introduced in [45] . There are two disjoint training sets: a base dataset 𝐷 𝑏 = {𝑥 𝑏 𝑖 , 𝑦 𝑏 𝑖 } with exhaustively annotated instances for each base class 𝐶 𝑏 and a novel dataset 𝐷 𝑛 = {𝑥 𝑛 𝑖 , 𝑦 𝑛 𝑖 } with only 𝐾 (usually less than 30) instances for each novel class 𝐶 𝑛 . In which 𝑥 𝑖 and 𝑦 𝑖 refer to the input image and the ground truth, respectively. It is worth noting that there is no intersection between the base classes and the novel classes, i.e., 𝐶 𝑏 ∩ 𝐶 𝑛 = ∅. Therefore, the ultimate goal of the FSOD is to train a robust detector based on the 𝐷 𝑏 and 𝐷 𝑛 to detect objects in the test set that contains both instances in 𝐶 𝑏 ∪ 𝐶 𝑛 .DeFRCN [36] is a state-of-the-art fine-tuning based few-shot object detector, consisting of two training stages. In the first phase, the Faster-RCNN is trained on the base classes 𝐶 𝑏 with sufficient samples. In the second phase, the transfer learning is performed, by fine-tuning the Faster-RCNN on the base classes and novel classes 𝐶 𝑏 ∪ 𝐶 𝑛 with 𝐾 instances per class. Fine-tuning on a balanced set 𝐷 𝑛𝑘 containing training samples for base and novel classes can help preserve the performance on base classes. The overall procedure of the fine-tuning based methods is summarized as follows:EQUATIONwhere 𝐹 𝑖𝑛𝑖𝑡 , 𝐹 𝑏𝑎𝑠𝑒 , and 𝐹 𝑛𝑜𝑣𝑒𝑙 indicate the detector in the initialization, base training, and novel fine-tuning stages, respectively. Different from previous fine-tuning based methods, which only fine-tune a small number of parameters of the Faster-RCNN, such as the prediction head, to prevent overfitting of the detector. DeFRCN introduces a Gradient Decoupled Layer during fine-tuning to stop the gradient between RPN and backbone while scaling the gradient between RCNN and backbone. It allows the detector to learn sufficiently about the novel data while preventing overfitting and is remarkably superior to other existing approaches.Despite the significant progress made by the fine-tuning based methods, given only 𝐾 novel instances, researchers fail to capture the data distribution accurately. To overcome the obstacles, we propose Prototype-based Soft-labels and Test-Time Learning (PS-TTL) to mine novel instances in the test data. The overall architecture of the model is illustrated in Fig. 2 . Self-training has promising performance for semi-supervised object detection [20, 28, 42] . It typically generates pseudo labels for the unlabeled data, and then the high-confidence pseudo labels are used to supervise the detector training.In this work, we hope to fully leverage novel instances in the test data, especially in the scenario of online learning, called Test-Time Learning (TTL). Therefore, we employ a mean-teacher self-training paradigm [42] . This paradigm mainly consists of two architecturally identical detectors, namely the student network and the teacher network. The teacher network first detects objects in the test data. Then we can obtain pseudo labels from the detection results through some post-processing procedures (e.g., non-maximum suppression and filtering using a confidence threshold). The high-quality pseudo labels are used to supervise the student network, enhancing its detection capability.Since we rely on the teacher network to predict reliable pseudo labels of novel classes in the test data, we utilize the FSOD detector 𝐹 𝑛𝑜𝑣𝑒𝑙 which has fine-tuned on novel data as the initialization of both the student and teacher network. However, the self-training paradigm inevitably generates noisy pseudo labels, especially in the novel classes. If we use excessively noisy pseudo labels to train the student network, the performance of the detector would deteriorate as training progresses. To filter out the noisy pseudo labels, we first apply non-maximum suppression for each class to remove duplicate detection boxes. Then, we set a high confidence threshold 𝛿 𝑢𝑝𝑝𝑒𝑟 to exclude uncertain labels. Finally, we optimize the student network using the remaining high-quality pseudo labels with the loss function as follows:EQUATIONwhere 𝑋 𝑡 is the input test image, and Ŷ𝑡 denotes the filtered pseudo labels. Note that the unsupervised loss is only applied to the classification heads of the Region Proposal Network (RPN) and Region of Interest (ROI) head.Even after filtering out low-confidence predictions, the pseudo labels are still noisy due to the poor detection performance of the FSOD detector 𝐹 𝑛𝑜𝑣𝑒𝑙 . Therefore, to alleviate the degradation of the FSOD detector during test-time learning, we propose using N-way-K-shot data 𝐷 𝑛𝑘 as supervision signals. Hence, the supervised loss for training the student network can be defined as:EQUATIONWhere {𝑋 𝑠 , 𝑌 𝑠 } ∈ 𝐷 𝑛𝑘 . Both RPN and ROI head adopt classification loss and bounding box regression loss. Following the mean-teacher [42] , to obtain strong pseudo labels from the test data, we update the teacher network weights via Exponential Moving Average (EMA) of student ones as below:EQUATIONwhere 𝜃 𝑡 and 𝜃 𝑠 are the network parameters of the teacher network and the student network, respectively. And 𝛼 is the EMA momentum coefficient.Utilizing the mean-teacher self-training framework proposed in Section 3.3 for test-time learning on the test data can promote the detection performance. Through experiments, we found that it is necessary to choose a large threshold 𝛿 𝑢𝑝𝑝𝑒𝑟 to filter the generated pseudo labels. However, this leads to severely missed detections, indicating that many test images have no pseudo labels. Different from semi-supervised object detection, where multiple rounds of fine-tuning can be conducted on the unlabeled data. Under the testtime learning setting, we can only perform one epoch of training on the test data. How to fully utilize every input test image is crucial.As shown in the Fig. 3 , we observed that relatively low-confidence pseudo labels, despite having classification confusion, mostly recall the foreground. Based on this phenomenon, we propose a Prototypebased Soft-labels (PS) strategy to replace the hard labels of these implicit foreground predictions with soft labels for fully unleashing the potential of low-quality pseudo-labels.Firstly, we introduce a lower bound confidence threshold 𝛿 𝑙𝑜𝑤𝑒𝑟 ; the predicted results between 𝛿 𝑙𝑜𝑤𝑒𝑟 and 𝛿 𝑢𝑝𝑝𝑒𝑟 are also assigned as foreground. Due to the increased class confusion in these implicit foreground predictions, employing class-specific NMS in the teacher network fails to effectively remove redundant boxes. Therefore, after removing the hard labels of these implicit foreground predictions, we apply class-agnostic NMS to them using every highconfidence pseudo prediction (i.e., whose confidence score is greater than 𝛿 𝑢𝑝𝑝𝑒𝑟 ) to filter the redundant ones.We then generate soft labels for the implicit foreground predictions by measuring their similarities to each class. Formally, given a implicit foreground prediction 𝑟 , we define its similarity to a class 𝑐 as the cosine distance between its ROI feature 𝑉 𝑟 and the prototype 𝑃 𝑐 of the class 𝑐:EQUATIONFinally, 𝑠 𝑟 = [𝑠 1 𝑟 , 𝑠 2 𝑟 , ..., 𝑠 𝑁 𝑟 ] followed a softmax function to generate 𝑞 𝑟 , which represents the soft label of the implicit foreground prediction 𝑟 . And we minimize the Kullback-Leibler (KL) divergence between the soft label and the class logits 𝑝 𝑟 of each implicit foreground prediction 𝑟 :EQUATIONwhere 𝑁 + 1 denotes 𝑁 foreground classes and one background class. Additionally, we set 𝑞 𝑁 +1 𝑟 = 0. To leverage soft labels of the implicit foreground predictions at the early stage during test-time learning, we initialize the class prototypes with N-way-K-shot data:𝑃 𝑐 = 1 𝐾 𝐾 ∑︁ 𝑖=1 𝑓 𝑖 𝑐 (7)where 𝑓 𝑖 𝑐 is the ROI feature of the 𝑖-th instance for class 𝑐. Because N-way-K-shot data cannot accurately represent the class prototypes, we propose dynamically updating the class prototypes using both labeled data and test data with high-confidence pseudo labels, aiming for the class prototypes to converge to the true representations as training progresses. Specifically, we update the class prototypes using the following formula:𝑃 𝑐 = 𝑃 𝑐 (1 -𝑠𝑖𝑚(𝑃 𝑐 , 𝑓 𝑐 )) + 𝑓 𝑐 𝑠𝑖𝑚(𝑃 𝑐 , 𝑓 𝑐 ) (8)where 𝑓 𝑐 is the averaged ROI features for class 𝑐. And 𝑠𝑖𝑚(•, •) is the cosine similarity function. During test-time learning, the total loss we optimize is as follows:EQUATION)which consists of supervised loss of N-way-K-shot data, unsupervised loss of pseudo labels, and KL loss of soft labels. Here, 𝜆 1 and 𝜆 2 are hyper-parameters to balance among losses. Then, we summarize the overall algorithm. We aim for the FSOD detector to learn from the test data. When a mini-batch of testing samples arrives, we update the model weights through the total loss. A detailed description is provided in Algorithm 1. L total = L sup + 𝜆 1 L unsup + 𝜆 2 L KL ; Student update (Gradient descent): Θ 𝑠 = Θ 𝑠 -𝜂∇L totalTeacher update (EMA):Θ 𝑡 = 𝛼Θ 𝑡 + (1 -𝛼)Θ 𝑠 
experiments: 
 In this section, we first introduce the experimental benchmarks in Section 4.1 and then describe the implementation details of our method in Section 4.2. Following this, we conduct extensive experiments on PASCAL VOC and MS COCO to compare our method with previous state-of-the-art approaches in Section 4.3. Finally, we provide extensive ablation studies on different components in Section 4.4. PASCAL VOC. For PASCAL VOC [7] , the overall 20 classes are divided into 15 base classes and 5 novel classes. Following TFA [45] , we utilize three different class splits, namely split 1, 2, and 3.For each split, base classes are exhaustively annotated, but novel classes only have 𝐾 = 1, 2, 3, 5, 10 annotated instances per class. Both base and novel class instances are sampled from the PASCAL VOC (07+12) trainval set, and the model is tested on the PASCAL VOC07 test set. We report AP50 for novel classes during evaluation. MS COCO. MS COCO [25] has 80 classes, we selecte the 20 classes that overlapped with PASCAL VOC as novel classes and the remaining 60 classes as base classes. In this case, we evaluate our method with 𝐾 = 10, 30 shots for each novel class. And we report mAP, and AP75, respectively.Our method can be combined with majority fine-tuning based fewshot object detector. For simplicity, we chose the most representative SOTA method, DeFRCN [36] , as our baseline. DeFRCN uses Faster-RCNN [39] as the detection model and ImageNet pre-trained ResNet-101. We use DeFRCN, which has been pre-trained on base classes and fine-tuned on novel classes, as the initialization of our model, and then fine-tune on the test data. During test-time learning, we fine-tune our model with a mini-batch of 2 on single GPU, which simulate the real inference process of the FSOD detector. Besides, we adopt a one-pass setting, where we fine-tune on the test data for only one epoch. We also utilize the N-way K-shot data used for novel fine-tuning during the testing process. Due to the poor performance of the FSOD detector, we apply weak data augmentation to both the N-way K-shot data and the test data, including random resize and random horizontal flip. For the hyperparameter, we set the 𝜆 1 = 0.5 and 𝜆 2 = 0.1 for all the experiments for simplicity. We set the thresholds 𝛿 𝑢𝑝𝑝𝑒𝑟 = 0.9 and 𝛿 𝑙𝑜𝑤𝑒𝑟 = 0.7. We optimize the network PASCAL VOC. Experimental results on the PASCAL VOC dataset are shown in Table 1 . We use DeFRCN as our baseline, which incorporates an additional Prototypical Calibration Block (PCB) for refining the predictions. However, we find that the N-way K-shot data utilized by the PCB may not align with that used during the novel fine-tuning stage. Therefore, we exclude the PCB and present our re-implementation results DeFRCN* in Tables 1 and 2 . It can be observed that our method achieves improvements across various splits and different shots on PASCAL VOC benchmark. Our method outperforms HallucFsDet [61] and LVC [18] , which represent synthetic novel class data and semi-supervised learning on the base data, respectively. Meanwhile, we find that the improvement gained from test-time learning becomes more significant as the shot decreases, especially in the 1-shot scenario. MS COCO. Table 2 shows the detection results on MS COCO. The MS COCO dataset contains more categories, and typically, a single image contains multiple instances. FSOD detectors generally perform poorly on MS COCO due to these factors, which undermine the performance of our method. However, we observe that our method achieves a significant improvement compared to the baseline, especially in the mAP75 metric. There is a 5.0% improvement in AP75 at 10 shots and a 9.2% improvement in AP75 at 30 shots. LVC [18] demonstrates a noticeable improvement on the MS COCO dataset, because the base data in the MS COCO benchmark contains a large number of implicit novel instances. However, this issue arises from the setting of few-shot detection, which could not reflect the In this section, we conduct ablation studies on novel split 1 of the PASCAL VOC benchmark to reveal the effectiveness of each individual component.We conduct a detailed ablation study on each component of the model, as shown in Table 3 .The first row presents our baseline, which is the result of DeFRCN. Initially, we attempted to solely utilize N-way K-shot data for supervised learning during testing but found that the model tended to overfit to these K-shot data, resulting in decreased performance.In the third row, we only fine-tuned the model using high-quality pseudo labels during testing, yielding results superior to the baseline.To further enhance FSOD performance in low-sample scenarios, we combined N-way K-shot data with pseudo-labels for training. Interestingly, except for the 3-shot setting, the model achieved further optimization in other cases, suggesting that this training approach effectively prevented the accumulation of biases in the model. Finally, by introducing 𝐿 𝐾𝐿 , i.e., employing a prototype-based soft-label strategy during testing, the model significantly improved its performance across various sample sizes. This also indicates that our proposed method can more efficiently utilize pseudo-labels.4.4.2 Upper and lower thresholds setting. Threshold selection has always been crucial in pseudo-label training, so we conducted ablation experiments on pseudo-label thresholds, as shown in Table 4 . Firstly, we used a large threshold 𝛿 𝑢𝑝𝑝𝑒𝑟 to filter high-quality pseudo-labels as hard labels for training the student network. To determine the appropriate value of 𝛿 𝑢𝑝𝑝𝑒𝑟 , we performed standard self-training on the test data without using soft labels. From Table 4 , it can be observed that using a larger threshold may result in only a few pseudo-labels available as hard labels, which could lead to many foreground objects being mistakenly classified as background, damaging the model's detection performance. Conversely, setting a threshold too low introduces excessive noise labels, which also affects performance. By comparing the results from rows 1 to 4, we set 𝛿 𝑢𝑝𝑝𝑒𝑟 to 0.9. Next, we conducted experiments on the low threshold 𝛿 𝑙𝑜𝑤𝑒𝑟 , where prediction boxes with confidence scores between the high threshold 𝛿 𝑢𝑝𝑝𝑒𝑟 and the low threshold 𝛿 𝑙𝑜𝑤𝑒𝑟 were considered implicit foreground predictions and assigned soft labels.Setting 𝛿 𝑙𝑜𝑤𝑒𝑟 too high may result in only a few implicit foreground predictions available as soft labels, while setting it too low may lead to many false positives, mistaking background as implicit foreground. From Table 4 , we chose 𝛿 𝑙𝑜𝑤𝑒𝑟 = 0.7, which helps the model efficiently utilize implicit foreground predictions, especially in extremely low-shot scenarios (i.e., shot=1). Additionally, we noticed that when the FSOD detector performs well, it is not sensitive to the threshold 𝛿 𝑙𝑜𝑤𝑒𝑟 . We speculate that this is because implicit foreground predictions are correctly assigned higher confidence scores, while background is given lower confidence scores.As mentioned earlier, we utilize the similarity between the ROI features of implicit foreground predictions and each class prototype to generate soft labels, where welldefined class prototypes can produce more accurate soft labels for implicit foreground predictions. However, since we initialize class prototypes using N-way K-shot data. During the teacher-student learning phase, predictions for the features of objects in each class are changing. Therefore, static prototype features cannot accurately represent their respective classes. We propose dynamically updating class prototypes using high-confidence pseudo-labels, aiming to gradually converge the prototypes to their true class distributions during test-time learning. In Table 5 , we compare the results of static class prototypes and dynamically updated class prototypes across multiple samplings, with the latter showing consistent improvements. We observe that the improvement brought by dynamically updating class prototypes becomes more pronounced as the number of samples decreases.We also validated the data augmentation used for both the student and teacher networks. Generally, in semi-supervised object detection, weak augmentation is applied to input images for the teacher network, while strong augmentation is used for the student network. For details on data augmentation, readers are advised to refer to [28] . However, in our case, we found that even when employing weak data augmentation for the student network, its performance improved. As shown in Table 6 , consistently using weak-weak data augmentation enhanced performance across all settings. This is because, during test-time learning, we can only fine-tune on the test data for one epoch. Additionally, in scenarios of data scarcity, strong data augmentation disrupts the original data distribution, impeding the model convergence.4.4.5 Qualitative evaluation. We visualize the detection results of 1-shot of PASCAL VOC in Fig. 4 . Our method can significantly alleviate the problem of classification confusion between base classes and novel classes. In the first column, DeFRCN misclassifies a base class (horse) as a novel class (cow), and in the second column, DeFRCN misclassifies a novel class (motorcycle) as a base class (bicycle). Our method addresses this issue through test-time learning.In the third column, DeFRCN predicts multiple local regions of a bus (novel class) as the bus category. Although we doesn't design any loss specifically for regression, the improvement in classification performance also helps the model alleviate this issue. Additionally, our method also improves the performance on base classes. For example, in column 4 of Fig. 4 , DeFRCN incorrectly identifies a newsstand as a bottle and misses dense cars, both of which have been corrected by our method. 
conclusion: 
 This paper proposes a novel framework for few-shot object detection, namely Prototype-based Soft-labels and Test-Time Learning (PS-TTL). It aims to address the challenge of accurately capturing the real data distribution under the condition of scarce samples from novel classes. To this end, we propose a Test-Time Learning (TTL) module to discover novel instances of test data, effectively alleviating the problem of overfitting to the distribution of base class. Furthermore, we design a Prototype-based Soft-labels (PS) strategy to unleash the potential of low-quality pseudo-labels, thereby significantly mitigating the constraints posed by few-shot samples. Extensive experiments are conducted on VOC and COCO, and PS-TTL reaches state-of-the-art performance, validating its effectiveness. 
 
 Appendix:  


Decision: This paper proposes to a Test-Time Learning (TTL) framework for Few-Shot Object Detection (FSOD), leveraging test data to improve the performance of FSOD. The idea of using test data to improve FSOD is novel and makes sense. The author did a great rebuttal. Most of the reviewers are positive about this work after the rebuttal, despite one reviewer not providing feedback. Therefore, my recommendation is Accept (Poster), and hope the authors can incorporate the reviewers' comments into the final version.

Reviews: review: paper_summary: paper_summary: Few-Shot Object Detection is a challenging task as it need to capture the distribution of unseen classes with quite limited instances. This paper proposed to address this problem by designing a novel framework named prototype-based soft-labels and test-time learning. The test-time learning module use a teacher network for self-training. Prototype-based soft-labels strategy asses the similarities between pseudo-labels ans category prototypes to unleash the potential of low quality pseudo-labels. It achieves good performance on several benchmarks.
strength_weakness: strength_weakness: The experimental results are satisfactory, demonstrating a commendable outcome. Using prototype-based soft-label method is a interesting way to introduce more fine-tuning samples to improve the performance. Test-Time Learning (TTL) that employs a mean-teacher network for self-training is proposed to discover novel instances on test data and Prototype based Soft-labels (PS) strategy is proposed  to unleash the potential of low-quality pseudo-labels.
limitations: limitations: 1) Motivation Clarity: The paper fails to clearly explain why adopting a test-time learning method can address the shortcomings of previous methods in accurately capturing data distribution. The motivation behind this approach is not adequately explained.

2) Limited Novelty: The approach lacks novelty. [A] was the first to use a teacher and student model in test-time adaptation, and many subsequent works have followed this method. Hence, this is not a new approach.

3)The paper mentions that prototypes were initialized with novel training data, but it does not clarify how base classes are handled.

4) Comparison with Recent SOTA Methods: The paper does not compare its results with several recent state-of-the-art (SOTA) methods such as [B] and others. I hope these SOTA results will be supplemented or explanation of why they are not being compared.

5) Presentation of Results: The presentation of results in Table 2 can be potentially misleading. Highlighting data that is neither the best nor the worst in terms of performance could cause confusion.

6) Effectiveness of Prototype-Based Soft-Label Method: The paper needs to provide stronger evidence to convince readers of the effectiveness of the prototype-based soft-label method. 


References:

[A] Wang, Qin, et al. "Continual test-time domain adaptation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[B] Zhang, Xinyu, Yuting Wang, and Abdeslam Boularias. "Detect everything with few examples." arXiv preprint arXiv:2309.12969 (2023).
paper_summary: paper_summary: This paper targets on Few-Shot Object Detection (FSOD). Concretely, it proposed a framework called Prototype-based Soft-labels and Test-Time Learning (PS-TTL). They propose Test-Time Learning (TTL) to alleviate overfitting to base classes. It also develop prototype-based pseudo labels to enhance the quality of pseudo labels. Extensive experiment results prove the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. This paper is overall well-written and easy to read. The figures are clear, demonstrating the idea in a good manner. 
2. The method section is clear, the algorithm depicts the proposed method well. 
3. I think the experiment results are abundant, with detailed analysis.
limitations: limitations: 1. I think overall this paper is an ok submission to ACMMM. My major concern is that test-time learning and prototype-based method is not new to me (eg. they are widely applied in few-shot learning or transfer learning). So I need to see more discussions about the novelty of this paper. More discussion about the differences between the proposed method and previous methods are also welcome.
paper_summary: paper_summary: This paper proposes a new PS-TTL method for FSOD to alleviate the sample scarcity issue of novel classes by taking advantage of the test-time samples with a well-designed test-time learning scheme. In particular, a teacher network which is EMA updated with the student network is used to produce high-confidence hard pseudo class labels for test-time samples while for those detection results with mild confidence, soft pseudo class labels are generated based on the similarities between the region features and the maintained class-wise prototypes. Then both kinds of pseudo labels are used in test-time learning for improving the generalization performance of baseline detectors. The experiments are conducted on two public datasets and PS-TTL shows improved performances compared to the baseline methods.
strength_weakness: strength_weakness: 1.	This paper is well organized while the proposed method is well presented and is easy to follow.
2.	It is an appropriate and preferable way to consider both the hard and soft pseudo labels for test-time learning.
3.	The experimental results on VOC dataset show clear improvements of PS-TTL compared to the baseline methods. The authors also conduct extensive ablation studies to show the effect of each component in PS-TTL.
limitations: limitations: 1.	The authors claim that the proposed method could be applied to different fine-tuning based baseline detectors, however only one existing baseline detector DeFRCN is considered in the experiments. I think more baseline detectors should also be considered to illustrate the effectiveness of PS-TTL.
2.	On MS COCO dataset, although DeFRCN+PS-TTL performs worse than LVC, I wonder will LVC+PS-TTL achieve a better performance than LVC.
3.	In some scenarios, the EMA updated teacher network will achieve a better test performance than the online updated student network at the end of training. Have the authors used the teacher network instead of the student network for testing and I wonder will the teacher network perform better.
4.	In Section 4.2, does the test-time learning be conducted on 2 samples and then these 2 samples are tested immediately? Does the batch size in this procedure influence the test performance?
5.	I recommend the authors to add a computational time comparison between DeFRCN and DeFRCN+PS-TTL to show the efficiency of the proposed method.
6.	I think the title of this paper needs to be further improved, as prototype-based soft-labels is indeed a part of the proposed test-time learning framework.
7.	There are some small typos and grammatical mistakes. For example, in Section 3.1, $C_b$ and $C^b$ are inconsistent. At line 334 and 849, the sentences are ended improperly.
paper_summary: paper_summary: The paper  proposes a new famework named Prototype-based Soft-labels and Test-Time Learning(PS-TTL) for few-shot object detection. The authors design a Test-Time Learning(TTL) module to discover novel instances on test data, and develop a Prototype-based Soft-labels (PS) strategy to unleash the potential of low-quality pseudo-labels. The authors believe it can enhance the accuracy capturing the real data distribution under the condition of scarce samples from novel classes.
strength_weakness: strength_weakness: The work introduces a strong and a lower bound confidence threshold remaining high-quality pseudo labels and generating  soft labels for images that have relatively low-confidence pseudo labels. This approach could utilize more input test images and improve the performance in the few-shot learning.
limitations: limitations: 1. The authors mention that the weakness of mining implicit novel instances from the training set is that it relies on the assumption that unlabeled novel instances are widely present in abundant base which may not hold true in real-world scenarios. This is not articulated very clearly.  Figure 1(b) does not clearly elucidate the weakness of the approach either. Please clarify it specifically.

2. The authors mention that this work is the first attempt to explore fitting novel class data distributions in a way that is more in line with real-world scenarios. To my knowledge, several published papers have similar design before.  《Uncertainty-aware Pseudo Label Refinery for Domain Adaptive Semantic Segmentation》denoises pseudo labels by estimating uncertainty to a bimodal distribution without any manually designed thresholds. 《Consistent-Teacher: Towards Reducing Inconsistent Pseudo-targets in Semi-supervised Object Detection》 develops a Consistent-Teacher approach instead of the Mean-Teacher and sets adaptive assignment assigns anchor and dynamic thresholds to efficiently utilize the data. Please clarify the advantages of your work compared to these methods.