Original: Prototypical VoteNet for Few-Shot 3D Point Cloud Object Detection
Revised: Prototypical VoteNet for Few-Shot 3D Point Cloud Object Detection

Original: Abstract
Revised: Abstract

Original: Most existing 3D point cloud object detection approaches heavily rely on large amounts of labeled training data.
Revised: Most existing 3D point cloud object detection approaches heavily rely on large amounts of labeled training data.

Original: However, the labeling process is costly and time-consuming.
Revised: However, the labeling process is costly and time-consuming.

Original: This paper considers few-shot 3D point cloud object detection, where only a few annotated samples of novel classes are needed with abundant samples of base classes.
Revised: This paper considers few-shot 3D point cloud object detection, where only a few annotated samples of novel classes are needed with abundant samples of base classes.

Original: To this end, we propose Prototypical VoteNet to recognize and localize novel instances, which incorporates two new modules: Prototypical
Revised: To this end, we propose Prototypical VoteNet to recognize and localize novel instances, which incorporates two new modules: Prototypical

Original: Vote Module (PVM) and Prototypical Head Module (PHM).
Revised: Vote Module (PVM) and Prototypical Head Module (PHM).

Original: Specifically, as the 3D basic geometric structures can be shared among categories, PVM is designed to leverage class-agnostic geometric prototypes, which are learned from base classes, to refine local features of novel categories.
Revised: Specifically, as the 3D basic geometric structures can be shared among categories, PVM is designed to leverage class-agnostic geometric prototypes, which are learned from base classes, to refine local features of novel categories.

Original: Then PHM is proposed to utilize class prototypes to enhance the global feature of each object, facilitating subsequent object localization and classification, which is trained by the episodic training strategy.
Revised: Then PHM is proposed to utilize class prototypes to enhance the global feature of each object, facilitating subsequent object localization and classification, which is trained by the episodic training strategy.

Original: To evaluate the model in this new setting, we contribute two new benchmark datasets, FS-ScanNet and FS-SUNRGBD.
Revised: To evaluate the model in this new setting, we contribute two new benchmark datasets, FS-ScanNet and FS-SUNRGBD.

Original: We conduct extensive experiments to demonstrate the effectiveness of Prototypical VoteNet, and our proposed method shows significant and consistent improvements compared to baselines on two benchmark datasets.
Revised: We conduct extensive experiments to demonstrate the effectiveness of Prototypical VoteNet, and our proposed method shows significant and consistent improvements compared to baselines on two benchmark datasets.

Original: This project will be available at https: //github.com/CVMI-Lab/FS3D .
Revised: This project will be available at https: //github.com/CVMI-Lab/FS3D .

Original:  Introduction
Revised:  Introduction

Original: 3D object detection aims to localize and recognize objects from point clouds with many applications in augmented reality, autonomous driving, and robotics manipulation.
Revised: 3D object detection aims to localize and recognize objects from point clouds with many applications in augmented reality, autonomous driving, and robotics manipulation.

Original: Recently, a number of fully supervised 3D object detection approaches have made remarkable progress with deep learning [23, 19, 32, 25].
Revised: Recently, a number of fully supervised 3D object detection approaches have made remarkable progress with deep learning [22, 18, 31, 24].

Original: Nonetheless, their success heavily relies on large amounts of labeled training data, which are time-consuming and costly to obtain.
Revised: Nonetheless, their success heavily relies on large amounts of labeled training data, which are time-consuming and costly to obtain.

Original: On the contrary, a human can quickly learn to recognize novel classes by seeing only a few samples.
Revised: On the contrary, a human can quickly learn to recognize novel classes by seeing only a few samples.

Original: To imitate such human ability, we consider few-shot point cloud 3D object detection, which aims to train a model to recognize novel categorizes from limited annotated samples of novel classes together with sufficient annotated data of base classes.
Revised: To imitate such human ability, we consider few-shot point cloud 3D object detection, which aims to train a model to recognize novel categorizes from limited annotated samples of novel classes together with sufficient annotated data of base classes.

Original: Few-shot learning has been extensively studied in various 2D visual understanding tasks such as object detection [40, 41, 44, 47], image classification [15, 10, 3, 33], and semantic segmentation [24, 22, 50, 20].
Revised: Few-shot learning has been extensively studied in various 2D visual understanding tasks such as object detection [39, 40, 43, 45], image classification [14, 9, 3, 32], and semantic segmentation [23, 21, 48, 19].

Original: Early attempts [10, 17, 12, 39] employ meta-learning to learn transferable knowledge from a collection of tasks and attained remarkable progress.
Revised: Early attempts [9, 16, 11, 38] employ meta-learning to learn transferable knowledge from a collection of tasks and attained remarkable progress.

Original: Recently, benefited from large-scale datasets ( e.g. ImageNet [7]) and advanced pre-training methods [28, 51, 11, 56], finetuning large-scale pre-trained visual models on down-stream few-shot datasets emerges as an effective approach to address this problem [34, 40, 57].
Revised: Recently, benefited from large-scale datasets ( e.g. ImageNet [6]) and advanced pre-training methods [27, 49, 10, 54], finetuning large-scale pretrained visual models on down-stream few-shot datasets emerges as an effective approach to address this problem [33, 39, 55].

Original: Among different streams of work, prototype-based methods [43, 55, 21, 18] have been incorporated into both streams and show the great advantages, since they can capture the representative features of categories that can be further utilized for feature refinement [47, 53] or classification [27, 33].
Revised: Among different streams of work, prototype-based methods [42, 53, 20, 17] have been incorporated into both streams and show the great advantages, since they can capture the representative features of categories that can be further utilized for feature refinement [45, 51] or classification [26, 32].

Original: This motivates us to explore effective 3D cues to build prototypes for few-shot 3D detection.
Revised: This motivates us to explore effective 3D cues to build prototypes for few-shot 3D detection.

Original: Different from 2D visual data, 3D data can get rid of distortions caused by perspective projections, and offer geometric cues with accurate shape and scale information.
Revised: Different from 2D visual data, 3D data can get rid of distortions caused by perspective projections, and offer geometric cues with accurate shape and scale information.

Original: Besides, 3D primitives to constitute objects can often be shared among different categories.
Revised: Besides, 3D primitives to constitute objects can often be shared among different categories.

Original: For instance, as shown in Figure 1, rectangular plates and corners can be found in many categories.
Revised: For instance, as shown in Figure 1, rectangular plates and corners can be found in many categories.

Original: Based on these observations, in this work, we propose
Revised: Based on these observations, in this work, we propose

Original: Prototypical VoteNet, which employs such robust 3D shape and primitive clues to design geometric prototypes to facilitate representation learning in the few-shot setting.
Revised: Prototypical VoteNet, which employs such robust 3D shape and primitive clues to design geometric prototypes to facilitate representation learning in the few-shot setting.

Original: Prototypical VoteNet incorporates two new modules, namely Prototypical Vote Module (PVM) and Prototypical Head Module (PHM), to enhance local and global feature learning, respectively, for few-shot 3D detection.
Revised: Prototypical VoteNet incorporates two new modules, namely Prototypical Vote Module (PVM) and Prototypical Head Module (PHM), to enhance local and global feature learning, respectively, for few-shot 3D detection.

Original: Specifically, based on extracted features from a backbone network ( i.e. PointNet++ [26]), PVM firstly constructs a class-agnostic 3D primitive memory bank to store geometric prototypes, which are shared by all categories and updated iteratively during training.
Revised: Specifically, based on extracted features from a backbone network ( i.e. PointNet++ [25]), PVM firstly constructs a class-agnostic 3D primitive memory bank to store geometric prototypes, which are shared by all categories and updated iteratively during training.

Original: To exploit the transferability of geometric structures, PVM then incorporates a multi-head cross-attention module to associate geometric prototypes with points in a given scene and utilize them to refine their feature representations.
Revised: To exploit the transferability of geometric structures, PVM then incorporates a multi-head cross-attention module to associate geometric prototypes with points in a given scene and utilize them to refine their feature representations.

Original: PVM is majorly developed to exploit shared geometric structures among base and novel categories to enhance feature learning of local information in the few-shot setting.
Revised: PVM is majorly developed to exploit shared geometric structures among base and novel categories to enhance feature learning of local information in the few-shot setting.

Original: Further, to facilitate learning discriminative features for object categorization, PHM is designed to employ a multi-head cross-attention module and leverage class-specific prototypes from a few support samples to refine global representations of objects.
Revised: Further, to facilitate learning discriminative features for object categorization, PHM is designed to employ a multi-head cross-attention module and leverage class-specific prototypes from a few support samples to refine global representations of objects.

Original: Moreover, episodic training [33, 39] is adopted to simulate few-shot circumstances, where PHM is trained by a distribution of similar few-shot tasks instead of only one target object detection task.
Revised: Moreover, episodic training [32, 38] is adopted to simulate few-shot circumstances, where PHM is trained by a distribution of similar few-shot tasks instead of only one target object detection task.

Original: Our contributions are listed as follows:
Revised: Our contributions are listed as follows:

Original: • We are the first to study the promising few-shot 3D point cloud object detection task, which allows a model to detect new classes, given a few examples.
Revised: • We are the first to study the promising few-shot 3D point cloud object detection task, which allows a model to detect new classes, given a few examples.

Original: • We propose Prototypical VoteNet, which incorporates Prototypical Vote Module and Prototypical Head Module, to address this new challenge.
Revised: • We propose Prototypical VoteNet, which incorporates Prototypical Vote Module and Prototypical Head Module, to address this new challenge.

Original: Prototypical Vote Module leverages class-agnostic geometric prototypes to enhance the local features of novel samples.
Revised: Prototypical Vote Module leverages class-agnostic geometric prototypes to enhance the local features of novel samples.

Original: Prototypical Head Module utilizes the class-specific prototypes to refine the object features with the aid of episode training.
Revised: Prototypical Head Module utilizes the class-specific prototypes to refine the object features with the aid of episode training.

Original: • We contribute two new benchmark dataset settings called FS-ScanNet and FS-SUNRGBD, which are specifically designed for this problem.
Revised: • We contribute two new benchmark dataset settings called FS-ScanNet and FS-SUNRGBD, which are specifically designed for this problem.

Original: Our experimental results on these two benchmark datasets show that the proposed model effectively addresses the few-shot 3D point cloud object detection problem, yielding significant improvement over several competitive baseline approaches.
Revised: Our experimental results on these two benchmark datasets show that the proposed model effectively addresses the few-shot 3D point cloud object detection problem, yielding significant improvement over several competitive baseline approaches.

Original:  Related Work
Revised:  Related Work

Original: 3D Point Cloud Object Detection.
Revised: 3D Point Cloud Object Detection.

Original: Current 3D point cloud object detection approaches can be divided into two streams: Grid Projection/Voxelization based [48, 16, 36, 4, 54, 42] and pointbased [31, 23, 19, 9, 2].
Revised: Current 3D point cloud object detection approaches can be divided into two streams: Grid Projection/Voxelization based [46, 15, 35, 4, 52, 41] and pointbased [30, 22, 18, 8, 2].

Original: The former projects point cloud to 2D grids or 3D voxels so that the advanced convolutional networks can be directly applied.
Revised: The former projects point cloud to 2D grids or 3D voxels so that the advanced convolutional networks can be directly applied.

Original: The latter methods take the raw point cloud
Revised: The latter methods take the raw point cloud

Original: [Figure]
Revised: [Figure]

Original: feature extraction network such as PointNet++ [26] to generate point-wise features for the subsequent detection.
Revised: feature extraction network such as PointNet++ [25] to generate point-wise features for the subsequent detection.

Original: Although these fully supervised approaches achieved promising 3D detection performance, their requirement for large amounts of training data precludes their application in many real-world scenarios where training data is costly or hard to acquire.
Revised: Although these fully supervised approaches achieved promising 3D detection performance, their requirement for large amounts of training data precludes their application in many real-world scenarios where training data is costly or hard to acquire.

Original: To alleviate this limitation, we explore the direction of few-shot 3D object detection in this paper.
Revised: To alleviate this limitation, we explore the direction of few-shot 3D object detection in this paper.

Original: Few-Shot Recognition.
Revised: Few-Shot Recognition.

Original: Few-shot recognition aims to classify novel instances with abundant base samples and a few novel samples.
Revised: Few-shot recognition aims to classify novel instances with abundant base samples and a few novel samples.

Original: Simple pre-training and finetuning approaches first train the model on the base classes, then finetune the model on the novel categories [3, 8].
Revised: Simple pre-training and finetuning approaches first train the model on the base classes, then finetune the model on the novel categories [3, 7].

Original: Meta-learning based methods [10, 17, 12, 39, 33] are proposed to learn classifier across tasks and then transfer to the few-shot classification task.
Revised: Meta-learning based methods [9, 16, 11, 38, 32] are proposed to learn classifier across tasks and then transfer to the few-shot classification task.

Original: The most related work is Prototypical Network [33], which represents a class as one prototype so that classification can be performed by computing distances to the prototype representation of each class.
Revised: The most related work is Prototypical Network [32], which represents a class as one prototype so that classification can be performed by computing distances to the prototype representation of each class.

Original: The above works mainly focus on 2D image understanding.
Revised: The above works mainly focus on 2D image understanding.

Original: Recently, some few-shot learning approaches for point cloud understanding [30, 53, 49] are proposed.
Revised: Recently, some few-shot learning approaches for point cloud understanding [29, 51, 47] are proposed.

Original: For instance, Sharma et al.
Revised: For instance, Sharma et al.

Original: [53] propose a graph-based method to propagate the knowledge from few-shot samples to the input point cloud.
Revised: [51] propose a graph-based method to propagate the knowledge from few-shot samples to the input point cloud.

Original: However, there is no work studying few-shot 3D point cloud object detection.
Revised: However, there is no work studying few-shot 3D point cloud object detection.

Original: In this paper, we first study this problem and introduce the spirit of Prototypical Network into few-shot 3D object detection with 3D geometric prototypes and 3D class-specific prototypes.
Revised: In this paper, we first study this problem and introduce the spirit of Prototypical Network into few-shot 3D object detection with 3D geometric prototypes and 3D class-specific prototypes.

Original: 2D Few-shot Object Detection.
Revised: 2D Few-shot Object Detection.

Original: Most existing 2D few-shot detectors employ a meta-learning [41, 15, 47] or fine-tuning based mechanism [45, 44, 27, 37].
Revised: Most existing 2D few-shot detectors employ a meta-learning [40, 14, 45] or fine-tuning based mechanism [44, 43, 26, 36].

Original: Particularly, Kang et al.
Revised: Particularly, Kang et al.

Original: [15] propose a one-stage few-shot detector which contains a meta feature learner and a feature re-weighting module.
Revised: [14] propose a one-stage few-shot detector which contains a meta feature learner and a feature re-weighting module.

Original: Meta R-CNN [47] presents meta-learning over RoI (Region-of-Interest) features and incorporates it into Faster R-CNN [29] and Mask R-CNN [12].
Revised: Meta R-CNN [45] presents meta-learning over RoI (Region-of-Interest) features and incorporates it into Faster R-CNN [28] and Mask R-CNN [11].

Original: TFA [40] reveals that simply fine-tuning the box classifier and regressor outperforms many meta-learning based methods.
Revised: TFA [39] reveals that simply fine-tuning the box classifier and regressor outperforms many meta-learning based methods.

Original: Cao et al.
Revised: Cao et al.

Original: [1] improve the few-shot detection performance by associating each novel class with a well-trained base class based on their semantic similarity.
Revised: [1] improve the few-shot detection performance by associating each novel class with a well-trained base class based on their semantic similarity.

Original:  Our Approach
Revised:  Our Approach

Original: In few-shot 3D point cloud object detection (FS3D), the object class set C is split into C base and C novel such that C = C base ∪ C novel and C base ∩ C novel = ∅ .
Revised: In few-shot 3D point cloud object detection (FS3D), the object class set C is split into C base and C novel such that C = C base ∪ C novel and C base ∩ C novel = ∅ .

Original: For each class r ∈ C , its annotation dataset T r contains all the data samples with object bounding boxes, that is T r = { ( u, P ) | u ∈ R 6 , P ∈ R N × 3 } .
Revised: For each class r ∈ C , its annotation dataset T r contains all the data samples with object bounding boxes, that is T r = { ( u, P ) | u ∈ R 6 , P ∈ R N × 3 } .

Original: Here, ( u, P ) is a 3D object bounding box u
Revised: Here, ( u, P ) is a 3D object bounding box u

Original: = ( x, y, z, h, w, l ) , representing box center locations and box dimensions, in a point cloud scene P .
Revised: = ( x, y, z, h, w, l ) , representing box center locations and box dimensions, in a point cloud scene P .

Original: There are only a few examples/shots for each novel class r ∈ C novel , which are known as support samples.
Revised: There are only a few examples/shots for each novel class r ∈ C novel , which are known as support samples.

Original: Besides, there are plenty of annotated samples for each base class r ∈ C base .
Revised: Besides, there are plenty of annotated samples for each base class r ∈ C base .

Original: Given the above dataset, FS3D aims to train a model to detect object instances in the novel classes leveraging such sufficient annotations for base categories C base and limited annotations for novel categories C novel .
Revised: Given the above dataset, FS3D aims to train a model to detect object instances in the novel classes leveraging such sufficient annotations for base categories C base and limited annotations for novel categories C novel .

Original: In the following, we introduce Prototypical VoteNet for few-shot 3D object detection.
Revised: In the following, we introduce Prototypical VoteNet for few-shot 3D object detection.

Original: We will describe the preliminaries of our framework in Section 3.1, which adopts the architecture of VoteNetstyle 3D detectors [25, 52, 2].
Revised: We will describe the preliminaries of our framework in Section 3.1, which adopts the architecture of VoteNetstyle 3D detectors [24, 50, 2].

Original: Then, we present Prototypical VoteNet consisting of Prototypical Vote
Revised: Then, we present Prototypical VoteNet consisting of Prototypical Vote

Original: Module (Section 3.2.1) and Prototypical Head Module (Section 3.2.2) to enhance feature learning for
Revised: Module (Section 3.2.1) and Prototypical Head Module (Section 3.2.2) to enhance feature learning for

Original: FS3D.
Revised: FS3D.

Original: 3.1 Preliminaries
Revised: 3.1 Preliminaries

Original: VoteNet-style 3D detectors [25, 52, 2] takes a point cloud scene P i as input, and localizes and categorizes 3D objects.
Revised: VoteNet-style 3D detectors [24, 50, 2] takes a point cloud scene P i as input, and localizes and categorizes 3D objects.

Original: As shown in Figure 2, it firstly incorporates a 3D backbone network ( i.e.
Revised: As shown in Figure 2, it firstly incorporates a 3D backbone network ( i.e.

Original: PointNet++ [26]) parameterized by θ 1 with downsampling layers for point feature extraction as
Revised: PointNet++ [25]) parameterized by θ 1 with downsampling layers for point feature extraction as

Original: Equation (1).
Revised: Equation (1).

Original: [Equation]
Revised: [Equation]

Original: where N and M represent the original and subsampled number of points, respectively, P i ∈
Revised: where N and M represent the original and subsampled number of points, respectively, P i ∈

Original: R N ×represents an input point cloud scene i , and F i ∈ R M × (3+ d ) is the subsampled scene points (also called seeds) with d -dimensional features and 3 -dimensional location coordinates.
Revised: R N ×represents an input point cloud scene i , and F i ∈ R M × (3+ d ) is the subsampled scene points (also called seeds) with d -dimensional features and 3 -dimensional location coordinates.

Original: Then, F i is fed into the vote module with parameters θ 2 which outputs a 3 -dimensional coordinate offset ∆ d j = (∆ x j , ∆ y j , ∆ z j ) relative to its corresponding object center c
Revised: Then, F i is fed into the vote module with parameters θ 2 which outputs a 3 -dimensional coordinate offset ∆ d j = (∆ x j , ∆ y j , ∆ z j ) relative to its corresponding object center c

Original: = ( c x , c y , c z ) and a residual feature vector ∆ f j for each point j in F i = { f j } i as in Equation (2).
Revised: = ( c x , c y , c z ) and a residual feature vector ∆ f j for each point j in F i = { f j } i as in Equation (2).

Original: [Equation]
Revised: [Equation]

Original: Given the predicted offset ∆ d j , the estimated corresponding object center c j = ( c x j , c y j , c z j ) that point j belongs to can be calculated as Equation (3).
Revised: Given the predicted offset ∆ d j , the estimated corresponding object center c j = ( c x j , c y j , c z j ) that point j belongs to can be calculated as Equation (3).

Original: [Equation]
Revised: [Equation]

Original: Similarly, the point features are updated as F i ← F i + ∆ F i where ∆ F i = { ∆ f j } i .
Revised: Similarly, the point features are updated as F i ← F i + ∆ F i where ∆ F i = { ∆ f j } i .

Original: Next, the detector samples object centers from { ( c x j , c y j , c z j ) } i using farthest point sampling and group points with nearby centers together (see Figure 2: Sampling & grouping) to form a set of object proposals O i = { o t } i .
Revised: Next, the detector samples object centers from { ( c x j , c y j , c z j ) } i using farthest point sampling and group points with nearby centers together (see Figure 2: Sampling & grouping) to form a set of object proposals O i = { o t } i .

Original: Each object proposal is characterized by a feature vector f o t which is obtained by applying a max pooling operation on features of all points belonging to o t .
Revised: Each object proposal is characterized by a feature vector f o t which is obtained by applying a max pooling operation on features of all points belonging to o t .

Original: Further, equipped with object features { f o t } i , the prediction layer with parameters θ 3 is adopted to yield the bounding boxes b t , objectiveness scores s t , and classification logits r t for each object proposal o t following Equation (4).
Revised: Further, equipped with object features { f o t } i , the prediction layer with parameters θ 3 is adopted to yield the bounding boxes b t , objectiveness scores s t , and classification logits r t for each object proposal o t following Equation (4).

Original: [Equation]
Revised: [Equation]

Original: 3.2 Prototypical VoteNet
Revised: 3.2 Prototypical VoteNet

Original: Here, we present Prototypical VoteNet which incorporates two new designs – Prototypical Vote
Revised: Here, we present Prototypical VoteNet which incorporates two new designs – Prototypical Vote

Original: Module (PVM) and Prototypical Head Module (PHM) to improve feature learning for novel categories with few annotated samples (see Figure 2).
Revised: Module (PVM) and Prototypical Head Module (PHM) to improve feature learning for novel categories with few annotated samples (see Figure 2).

Original: Specifically, PVM builds a class-agnostic memory bank of geometric prototypes G = { g k } Kk =1 with a size of K , which models transferable class-agnostic 3D primitives learned from rich base categories, and further employs them to enhance local feature representation for novel categories via a multi-head cross-attention module.
Revised: Specifically, PVM builds a class-agnostic memory bank of geometric prototypes G = { g k } Kk =1 with a size of K , which models transferable class-agnostic 3D primitives learned from rich base categories, and further employs them to enhance local feature representation for novel categories via a multi-head cross-attention module.

Original: The enhanced features are then utilized by the Vote Layer to output the offset of coordinates and features as Equation (2).
Revised: The enhanced features are then utilized by the Vote Layer to output the offset of coordinates and features as Equation (2).

Original: Second, to facilitate learning discriminative features for novel class prediction, PHM employs an attention-based design to leverage class-specific prototypes E = { e r } Rr =1 extracted from the support set D support with R categories to refine global discriminate feature for representing each object proposal (see Figure 2).
Revised: Second, to facilitate learning discriminative features for novel class prediction, PHM employs an attention-based design to leverage class-specific prototypes E = { e r } Rr =1 extracted from the support set D support with R categories to refine global discriminate feature for representing each object proposal (see Figure 2).

Original: The output features are fed to the prediction layer for producing results as Equation (4).
Revised: The output features are fed to the prediction layer for producing results as Equation (4).

Original: To make the model more generalizable to novel classes, we exploit the episodic training [33, 39] strategy to train PHM, where a distribution of similar few-shot tasks instead of only one object detection task is learned in the training phase.
Revised: To make the model more generalizable to novel classes, we exploit the episodic training [32, 38] strategy to train PHM, where a distribution of similar few-shot tasks instead of only one object detection task is learned in the training phase.

Original: PVM and PHM are elaborated in the following sections.
Revised: PVM and PHM are elaborated in the following sections.

Original: 3.2.1 Prototypical Vote Module
Revised: 3.2.1 Prototypical Vote Module

Original: Given input features F i extracted by a backbone network, Prototypical Vote Module constructs class-agnostic geometric prototypes G = { g k } Kk =1 and then uses them to enhance local point features with an attention module.
Revised: Given input features F i extracted by a backbone network, Prototypical Vote Module constructs class-agnostic geometric prototypes G = { g k } Kk =1 and then uses them to enhance local point features with an attention module.

Original: Geometric Prototype Construction.
Revised: Geometric Prototype Construction.

Original: At the beginning, G = { g k } Kk =1 ∈ R d × K is randomly initialized.
Revised: At the beginning, G = { g k } Kk =1 ∈ R d × K is randomly initialized.

Original: During training, G is iteratively updated with a momentum based on point features of foreground objects.
Revised: During training, G is iteratively updated with a momentum based on point features of foreground objects.

Original: Specifically, for each update, given G = { g k } Kk =1 and all the foreground points { p m } M f m =1 with features { f m } M f m =1 , where M f is the number of foreground points in the current batch, we assign each point to its nearest geometric prototype based on feature space distance.
Revised: Specifically, for each update, given G = { g k } Kk =1 and all the foreground points { p m } M f m =1 with features { f m } M f m =1 , where M f is the number of foreground points in the current batch, we assign each point to its nearest geometric prototype based on feature space distance.

Original: Then, for each prototype g k , we have a group of points { p m } k with features represented as { f m } k assigned to it.
Revised: Then, for each prototype g k , we have a group of points { p m } k with features represented as { f m } k assigned to it.

Original: Point features in one group are averaged to update the corresponding geometric prototype as
Revised: Point features in one group are averaged to update the corresponding geometric prototype as

Original: Equation (5).
Revised: Equation (5).

Original: [Equation]
Revised: [Equation]

Original: Here γ ∈ [0 , 1] is the momentum coefficient for updating geometric prototypes in a momentum manner, serving as a moving average over all training samples.
Revised: Here γ ∈ [0 , 1] is the momentum coefficient for updating geometric prototypes in a momentum manner, serving as a moving average over all training samples.

Original: Since one point feature is related to one geometric prototype, we call this one-hot assignment strategy as hard assignment.
Revised: Since one point feature is related to one geometric prototype, we call this one-hot assignment strategy as hard assignment.

Original: An alternative to the hard assignment is the soft assignment, which calculates the similarity between a point features with all geometric prototypes.
Revised: An alternative to the hard assignment is the soft assignment, which calculates the similarity between a point features with all geometric prototypes.

Original: Empirically, we found that hard assignment results in more effective grouping versus soft assignment.
Revised: Empirically, we found that hard assignment results in more effective grouping versus soft assignment.

Original: More details can be found in the supplementary material.
Revised: More details can be found in the supplementary material.

Original: Geometric Prototypical Guided Local Feature Enrichment.
Revised: Geometric Prototypical Guided Local Feature Enrichment.

Original: Given the geometric prototypes
Revised: Given the geometric prototypes

Original: G = { g k } Kk =1 and point features F i = { f j } i of a scene i , PVM further employs a multi-head cross-attention module [38] to refine the point features.
Revised: G = { g k } Kk =1 and point features F i = { f j } i of a scene i , PVM further employs a multi-head cross-attention module [37] to refine the point features.

Original: Specifically, the multi-head attention network uses the point features F i = { f j } i as query, geometric prototypes G = { g k } Kk =1 as key and value where linear transformation functions with weights represented as Q h , U h , V h are applied to encode query, key and value respectively.
Revised: Specifically, the multi-head attention network uses the point features F i = { f j } i as query, geometric prototypes G = { g k } Kk =1 as key and value where linear transformation functions with weights represented as Q h , U h , V h are applied to encode query, key and value respectively.

Original: Here, h represents the head index.
Revised: Here, h represents the head index.

Original: Then, for each head h , the query point feature is updated by softly aggregating the value features where the soft aggregation weight is determined by the similarity between the query point feature and corresponding key feature.
Revised: Then, for each head h , the query point feature is updated by softly aggregating the value features where the soft aggregation weight is determined by the similarity between the query point feature and corresponding key feature.

Original: The final point feature f j is updated by summing over outputs from all heads as Equation (6).
Revised: The final point feature f j is updated by summing over outputs from all heads as Equation (6).

Original: f
Revised: f

Original: j ← Cross
Revised: j ← Cross

Original: [Equation]
Revised: [Equation]

Original: Att
Revised: Att

Original: [Equation]
Revised: [Equation]

Original: Here, A h j,k is the soft aggregation weight considering the similarity between the j -th query point feature and the k -th key feature and used to weight the k -th value feature.
Revised: Here, A h j,k is the soft aggregation weight considering the similarity between the j -th query point feature and the k -th key feature and used to weight the k -th value feature.

Original: Through this process, the point feature is refined using geometric prototypes in a weighted manner where prototypes similar to the query point feature will have higher attention weights.
Revised: Through this process, the point feature is refined using geometric prototypes in a weighted manner where prototypes similar to the query point feature will have higher attention weights.

Original: This mechanism transfers geometric prototypes learned from base categories with abundant data to model novel points.
Revised: This mechanism transfers geometric prototypes learned from base categories with abundant data to model novel points.

Original: The multi-head design enables the model to seek similarity measurements from different angles in a learnable manner
Revised: The multi-head design enables the model to seek similarity measurements from different angles in a learnable manner

Original: [Figure]
Revised: [Figure]

Original: to improve robustness.
Revised: to improve robustness.

Original: Additionally, in both PHM and PVM, the multi-head attention layer are combined with feed forward FC layers.
Revised: Additionally, in both PHM and PVM, the multi-head attention layer are combined with feed forward FC layers.

Original: After refining point features { f j } i , PVM predicts the point offset and residual feature vector { ∆ d j , ∆ F j } i as stated in Equation (2).
Revised: After refining point features { f j } i , PVM predicts the point offset and residual feature vector { ∆ d j , ∆ F j } i as stated in Equation (2).

Original: ∆ d j is explicitly supervised by a regression loss L vote used in [26].
Revised: ∆ d j is explicitly supervised by a regression loss L vote used in [25].

Original: What do the geometric prototypes store?
Revised: What do the geometric prototypes store?

Original: To shed more insights on what the geometric prototypes represent, we visualize the frequency of geometry prototypes in different categories using the “assignment”.
Revised: To shed more insights on what the geometric prototypes represent, we visualize the frequency of geometry prototypes in different categories using the “assignment”.

Original: The number of “assignment”s of object point features to the geometric prototypes is shown in Figure 3, where a point is assigned to the geometric prototype with the highest similarity.
Revised: The number of “assignment”s of object point features to the geometric prototypes is shown in Figure 3, where a point is assigned to the geometric prototype with the highest similarity.

Original: In each histogram, the horizontal axis represents the index of geometric prototypes and the vertical axis represents the number of assignments.
Revised: In each histogram, the horizontal axis represents the index of geometric prototypes and the vertical axis represents the number of assignments.

Original: Note that the first row is the novel classes and the second row is the base classes.
Revised: Note that the first row is the novel classes and the second row is the base classes.

Original: Figure 3 shows that two visually similar categories have a similar assignment histogram since they share the basic geometric structures.
Revised: Figure 3 shows that two visually similar categories have a similar assignment histogram since they share the basic geometric structures.

Original: This indicates that the memory bank of geometric prototypes successfully learns the 3D basic geometric knowledge, which can be a bridge to transfer the geometric knowledge from base classes to novel ones.
Revised: This indicates that the memory bank of geometric prototypes successfully learns the 3D basic geometric knowledge, which can be a bridge to transfer the geometric knowledge from base classes to novel ones.

Original: 3.2.2 Prototypical Head Module
Revised: 3.2.2 Prototypical Head Module

Original: As shown in Figure 2, given object proposals O i =
Revised: As shown in Figure 2, given object proposals O i =

Original: { o t } i with features { f o t } i from Sampling &
Revised: { o t } i with features { f o t } i from Sampling &

Original: Grouping module, PHM module leverages class-specific prototypes { e r } to refine the object features f o t for the subsequent object classification and localization.
Revised: Grouping module, PHM module leverages class-specific prototypes { e r } to refine the object features f o t for the subsequent object classification and localization.

Original: Moreover, for better generalizing to novel categories, PHM is trained by the episodic training scheme, where PHM learns a large number of similar few-shot tasks instead of only one task.
Revised: Moreover, for better generalizing to novel categories, PHM is trained by the episodic training scheme, where PHM learns a large number of similar few-shot tasks instead of only one task.

Original: Considering the function of PHM, we construct the few-shot tasks that, in each iteration, PHM refines the object features { f o t } i with the aid of class-specific prototypes, which are extracted from randomly sampled support samples.
Revised: Considering the function of PHM, we construct the few-shot tasks that, in each iteration, PHM refines the object features { f o t } i with the aid of class-specific prototypes, which are extracted from randomly sampled support samples.

Original: In each few-shot task, class-specific prototypes are built based on support sets that are randomly sampled.
Revised: In each few-shot task, class-specific prototypes are built based on support sets that are randomly sampled.

Original: For class r , the class-specific prototype e r is obtained by averaging the instance features for all support samples in class r .
Revised: For class r , the class-specific prototype e r is obtained by averaging the instance features for all support samples in class r .

Original: The instance feature is derived by applying a max pooling operation over the features of all points belonging to that instance.
Revised: The instance feature is derived by applying a max pooling operation over the features of all points belonging to that instance.

Original: As shown in Figure 2, with class-specific prototypes E = { e r } Rr =1 for a total of R categories, PHM further employs a multi-head cross-attention module to refine object features { f o t } i .
Revised: As shown in Figure 2, with class-specific prototypes E = { e r } Rr =1 for a total of R categories, PHM further employs a multi-head cross-attention module to refine object features { f o t } i .

Original: Here, the object features { f o t } i serve as the query features, class-specific prototypes are used to build value features and key features similar as what has been described in Section 3.2.1.
Revised: Here, the object features { f o t } i serve as the query features, class-specific prototypes are used to build value features and key features similar as what has been described in Section 3.2.1.

Original: Then, the representation f o t of each proposal o t is refined using the outputs of the multi-head attention module, which are weighted sum over the value features and the weight is proportionally to the similarity between the query feature and corresponding key features.
Revised: Then, the representation f o t of each proposal o t is refined using the outputs of the multi-head attention module, which are weighted sum over the value features and the weight is proportionally to the similarity between the query feature and corresponding key features.

Original: This process can be formulated as Equation (7), which is similar to Equation (6).
Revised: This process can be formulated as Equation (7), which is similar to Equation (6).

Original: [Equation]
Revised: [Equation]

Original: Until now, f o t is refined using class-specific prototypes, injecting class-specific features from given support samples into object-level features.
Revised: Until now, f o t is refined using class-specific prototypes, injecting class-specific features from given support samples into object-level features.

Original: Finally, the refined object features { f o t } i are fed into the prediction module following Equation (4).
Revised: Finally, the refined object features { f o t } i are fed into the prediction module following Equation (4).

Original: What does PHM help?
Revised: What does PHM help?

Original: Figure 4 visualizes the effect of feature refinement by PHM.
Revised: Figure 4 visualizes the effect of feature refinement by PHM.

Original: The experiment is conducted on FS-ScanNet.
Revised: The experiment is conducted on FS-ScanNet.

Original: Figure 4(a) shows the object features, which have not been processed by PHM.
Revised: Figure 4(a) shows the object features, which have not been processed by PHM.

Original: Figure 4(b) shows the object features processed by PHM.
Revised: Figure 4(b) shows the object features processed by PHM.

Original: We could observe that, after the feature refinement by PHM, the features of each classes become more compact compared to the non-PHM one, which further validates the effectiveness of PHM.
Revised: We could observe that, after the feature refinement by PHM, the features of each classes become more compact compared to the non-PHM one, which further validates the effectiveness of PHM.

Original: 3.3 Model Training
Revised: 3.3 Model Training

Original: The model is trained by the episodic training strategy [33, 39].
Revised: The model is trained by the episodic training strategy [32, 38].

Original: The detailed training strategy is included in the supplementary material.
Revised: The detailed training strategy is included in the supplementary material.

Original: We use the standard cross entropy loss L cls and smooth- Lloss [29] L reg to supervise the classification and the bounding box regression, respectively.
Revised: We use the standard cross entropy loss L cls and smooth- Lloss [28] L reg to supervise the classification and the bounding box regression, respectively.

Original: As for the objectness prediction, if a vote is located either within 0.3 meters to a ground truth object center or more than 0.6 meters from any center, it is considered to be positive [25], which is supervised by a cross entropy loss L obj .
Revised: As for the objectness prediction, if a vote is located either within 0.3 meters to a ground truth object center or more than 0.6 meters from any center, it is considered to be positive [24], which is supervised by a cross entropy loss L obj .

Original: Therefore, the overall loss for Prototypical VoteNet is given by,
Revised: Therefore, the overall loss for Prototypical VoteNet is given by,

Original: [Equation]
Revised: [Equation]

Original: where α 1 , α 2 , α 3 is the coefficients to balance the loss contributions.
Revised: where α 1 , α 2 , α 3 is the coefficients to balance the loss contributions.

Original:  Experiments
Revised:  Experiments

Original: To our best knowledge, there is no prior study of few-shot point cloud object detection.
Revised: To our best knowledge, there is no prior study of few-shot point cloud object detection.

Original: Therefore, we setup a new benchmark which is described in Section 4.1 & 4.2.
Revised: Therefore, we setup a new benchmark which is described in Section 4.1 & 4.2.

Original: Then, we conduct experiments and compare our method with baseline methods in Section 4.3.
Revised: Then, we conduct experiments and compare our method with baseline methods in Section 4.3.

Original: Third, a series of ablation studies are performed for further analyzing Prototypical VoteNet in Section 4.4.
Revised: Third, a series of ablation studies are performed for further analyzing Prototypical VoteNet in Section 4.4.

Original: In addition, the implementation details are included in the supplementary material.
Revised: In addition, the implementation details are included in the supplementary material.

Original: 4.1 Benchmark Setup
Revised: 4.1 Benchmark Setup

Original: Datasets.
Revised: Datasets.

Original: We construct two new benchmark datasets FS-SUNRGBD and FS-ScanNet.
Revised: We construct two new benchmark datasets FS-SUNRGBD and FS-ScanNet.

Original: Specifically,
Revised: Specifically,

Original: FS-SUNRGBD is derived from SUNRGBD [35].
Revised: FS-SUNRGBD is derived from SUNRGBD [34].

Original: SUNRGBD consists of 5K RGB-D training images annotated, and the standard evaluation protocol reports performance on 10 categories.
Revised: SUNRGBD consists of 5K RGB-D training images annotated, and the standard evaluation protocol reports performance on 10 categories.

Original: We randomly select 4 classes as the novel ones while keeping the remaining ones as the base.
Revised: We randomly select 4 classes as the novel ones while keeping the remaining ones as the base.

Original: In the training set, only K annotated bounding boxes for each novel class are given, where k equals 1, 2, 3, 4 and 5.
Revised: In the training set, only K annotated bounding boxes for each novel class are given, where k equals 1, 2, 3, 4 and 5.

Original: FS-ScanNet is derived from ScanNet [6].
Revised: FS-ScanNet is derived from ScanNet [5].

Original: ScanNet consists of 1,513 point clouds, and the annotation of the point clouds corresponds to 18 semantic classes plus one for the unannotated space.
Revised: ScanNet consists of 1,513 point clouds, and the annotation of the point clouds corresponds to 18 semantic classes plus one for the unannotated space.

Original: Out of its 18 object categories, we randomly select 6 classes as the novel ones, while keeping the remaining as the base.
Revised: Out of its 18 object categories, we randomly select 6 classes as the novel ones, while keeping the remaining as the base.

Original: We evaluate with 2 different base/novel splits.
Revised: We evaluate with 2 different base/novel splits.

Original: In the training set, only K annotated bounding boxes for each novel class are given, where k equals 1, 3 and 5.
Revised: In the training set, only K annotated bounding boxes for each novel class are given, where k equals 1, 3 and 5.

Original: More details about the new benchmark datasets can be referred to the supplementary material.
Revised: More details about the new benchmark datasets can be referred to the supplementary material.

Original: Evaluation Metrics.
Revised: Evaluation Metrics.

Original: We follow the standard evaluation protocol [25] in 3D point cloud object detection by using mean Average Precision(mAP) under different IoU thresholds ( i.e. 0.25, 0.50), denoted as AP 25 and AP 50 .
Revised: We follow the standard evaluation protocol [24] in 3D point cloud object detection by using mean Average Precision(mAP) under different IoU thresholds ( i.e. 0.25, 0.50), denoted as AP 25 and AP 50 .

Original: In addition, in the inference stage, we detect both novel classes and base classes.
Revised: In addition, in the inference stage, we detect both novel classes and base classes.

Original: The performance on base classes is included in the supplementary material.
Revised: The performance on base classes is included in the supplementary material.

Original: 4.2 Benchmark Few-shot 3D Object Detection
Revised: 4.2 Benchmark Few-shot 3D Object Detection

Original: We build the first benchmark for few-shot 3D object detection.
Revised: We build the first benchmark for few-shot 3D object detection.

Original: The benchmark incorporatescompetitive methods, and three of them are built with few-shot learning strategies, which have been shown to be successful in 2D few-shot object detection.
Revised: The benchmark incorporatescompetitive methods, and three of them are built with few-shot learning strategies, which have been shown to be successful in 2D few-shot object detection.

Original: • Baseline: We abandon PVM and PHM, and train the detector on the base and novel classes together.
Revised: • Baseline: We abandon PVM and PHM, and train the detector on the base and novel classes together.

Original: In this way, it can learn good features from the base classes that are applicable for detecting novel classes.
Revised: In this way, it can learn good features from the base classes that are applicable for detecting novel classes.

Original: • VoteNet+TFA [40]: We incorporate a well-designed few-shot object detection method TFA [40] with VoteNet.
Revised: • VoteNet+TFA [39]: We incorporate a well-designed few-shot object detection method TFA [39] with VoteNet.

Original: TFA first trains VoteNet on the training set with abundant samples of base classes.
Revised: TFA first trains VoteNet on the training set with abundant samples of base classes.

Original: Then only the classifier and the regressor are finetuned with a small balance set containing both base classes and novel classes.
Revised: Then only the classifier and the regressor are finetuned with a small balance set containing both base classes and novel classes.

Original: • VoteNet+PT+TFA: The pretraining is proven to be important in 2D few-shot learning, as it learns more generic features, facilitating knowledge transfer from base classes to novel classes.
Revised: • VoteNet+PT+TFA: The pretraining is proven to be important in 2D few-shot learning, as it learns more generic features, facilitating knowledge transfer from base classes to novel classes.

Original: Therefore, we add a pretraining stage, which is borrowed from a self-supervised point cloud contrastive learning method [14], before the training stage of VoteNet+TFA.
Revised: Therefore, we add a pretraining stage, which is borrowed from a self-supervised point cloud contrastive learning method [13], before the training stage of VoteNet+TFA.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: • Meta VoteNet: Meta VoteNet is inspired by Meta RCNN [47].
Revised: • Meta VoteNet: Meta VoteNet is inspired by Meta RCNN [45].

Original: Meta RCNN develops a meta-learning prediction head, which leverages class prototypes to multiply the RoI features and then predicts if a RoI feature is the category that the class prototype belongs to.
Revised: Meta RCNN develops a meta-learning prediction head, which leverages class prototypes to multiply the RoI features and then predicts if a RoI feature is the category that the class prototype belongs to.

Original: In VoteNet, the grouped point features after the vote stage can be treated as the RoI features
Revised: In VoteNet, the grouped point features after the vote stage can be treated as the RoI features

Original: 4.3 Main Results
Revised: 4.3 Main Results

Original: FS-ScanNet.
Revised: FS-ScanNet.

Original: We first compare the proposed approach with the baselines on FS-ScanNet in Table 1.
Revised: We first compare the proposed approach with the baselines on FS-ScanNet in Table 1.

Original: These results show: 1) VoteNet+TFA has the worst performance.
Revised: These results show: 1) VoteNet+TFA has the worst performance.

Original: For example, it only achieves 8.07%
Revised: For example, it only achieves 8.07%

Original: AP 25 and 1.03% AP 50 on 3-shot in Novel Split-1.
Revised: AP 25 and 1.03% AP 50 on 3-shot in Novel Split-1.

Original: The reason is that VoteNet+TFA is trained from scratch, and only a few layers are funetuned by novel samples.
Revised: The reason is that VoteNet+TFA is trained from scratch, and only a few layers are funetuned by novel samples.

Original: Therefore, it tends to be overfitting with poor generalization ability on novel classes.
Revised: Therefore, it tends to be overfitting with poor generalization ability on novel classes.

Original: Even adding a pretaining stage to VoteNet+TFA, which is termed as VoteNet+PT+TFA, it still performs poorly.
Revised: Even adding a pretaining stage to VoteNet+TFA, which is termed as VoteNet+PT+TFA, it still performs poorly.

Original: For example, it only contributes the improvement of +2.30% AP 25 and +1.10% AP 50 to VoteNet+TFA on 3-shot in Novel Split-1.
Revised: For example, it only contributes the improvement of +2.30% AP 25 and +1.10% AP 50 to VoteNet+TFA on 3-shot in Novel Split-1.

Original: This is different from 2D few-shot object detection.
Revised: This is different from 2D few-shot object detection.

Original: This is potentially caused by the lack of large-scale datasets for pre-training in 3D, while the 2D backbone ResNet [13] has a dataset of over one million images for pre-training.
Revised: This is potentially caused by the lack of large-scale datasets for pre-training in 3D, while the 2D backbone ResNet [12] has a dataset of over one million images for pre-training.

Original: Besides our designed method, Meta VoteNet is superior.
Revised: Besides our designed method, Meta VoteNet is superior.

Original: For example,
Revised: For example,

Original: Meta VoteNet reaches 25.73% AP 25 and 16.01% AP 50 on 3-shot in Novel Split-1.
Revised: Meta VoteNet reaches 25.73% AP 25 and 16.01% AP 50 on 3-shot in Novel Split-1.

Original: This shows the meta-learning prediction head, derived from Meta RCNN [47], helps to generalize from base classes to novel classes.
Revised: This shows the meta-learning prediction head, derived from Meta RCNN [45], helps to generalize from base classes to novel classes.

Original: Our new model Prototypical VoteNet surpasses all competitors by significant margins.
Revised: Our new model Prototypical VoteNet surpasses all competitors by significant margins.

Original: For example, Prototypical VoteNet achieves 32.25% AP 25 and 16.01% AP 50 on 3-shot in
Revised: For example, Prototypical VoteNet achieves 32.25% AP 25 and 16.01% AP 50 on 3-shot in

Original: Novel Split-1.
Revised: Novel Split-1.

Original: This is because our proposed method is equipped with a more generic vote module by learning geometric prototypes, and leverage class prototypes to promote the discriminative feature learning.
Revised: This is because our proposed method is equipped with a more generic vote module by learning geometric prototypes, and leverage class prototypes to promote the discriminative feature learning.

Original: FS-SUNRGBD.
Revised: FS-SUNRGBD.

Original: Table 2 shows the result comparison on FS-SUNRGBD.
Revised: Table 2 shows the result comparison on FS-SUNRGBD.

Original: In FS-SUNRGBD, we find that VoteNet+TFA and VoteNet+PT+TFA still have a large gap with other methods due to the overfitting and insufficient pretraining.
Revised: In FS-SUNRGBD, we find that VoteNet+TFA and VoteNet+PT+TFA still have a large gap with other methods due to the overfitting and insufficient pretraining.

Original: Meanwhile, Meta VoteNet outperforms other baseline methods.
Revised: Meanwhile, Meta VoteNet outperforms other baseline methods.

Original: For example, on 3-shot, Meta VoteNet surpasses VoteNet+PT+TFA by +8.34%
Revised: For example, on 3-shot, Meta VoteNet surpasses VoteNet+PT+TFA by +8.34%

Original: APand +0.81% AP 50 and achieves 16.24% AP 25 and 3.12% AP 50 .
Revised: APand +0.81% AP 50 and achieves 16.24% AP 25 and 3.12% AP 50 .

Original: Finally, our designed Prototypical
Revised: Finally, our designed Prototypical

Original: VoteNet outperforms all the methods.
Revised: VoteNet outperforms all the methods.

Original: For instance, on 3-shot, Prototypical VoteNet surpasses Meta
Revised: For instance, on 3-shot, Prototypical VoteNet surpasses Meta

Original: VoteNet by +5.37% AP 25 and +3.01% AP 50 , which further validates the effectiveness of our proposed
Revised: VoteNet by +5.37% AP 25 and +3.01% AP 50 , which further validates the effectiveness of our proposed

Original: PVM and PHM.
Revised: PVM and PHM.

Original: 4.4 Further Analysis
Revised: 4.4 Further Analysis

Original: [Table]
Revised: [Table]

Original: +PVM+
Revised: +PVM+

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Contributions of individual components.
Revised: Contributions of individual components.

Original: We conduct ablation studies on 3-shot and 5-shot in split-1 of FS-ScanNet with results shown in Table 3.
Revised: We conduct ablation studies on 3-shot and 5-shot in split-1 of FS-ScanNet with results shown in Table 3.

Original: The results show that both of them are effective on their own.
Revised: The results show that both of them are effective on their own.

Original: For example, on 3-shot, PVM contributes the improvement of +4.19% AP 25 and +4.59% AP 50 , and PHM contributes the improvement of +6.12% AP 25 and +5.00% AP 50 .
Revised: For example, on 3-shot, PVM contributes the improvement of +4.19% AP 25 and +4.59% AP 50 , and PHM contributes the improvement of +6.12% AP 25 and +5.00% AP 50 .

Original: Moreover, when combined, the best performance, 31.25% AP 25 and 16.01% AP 50 , is achieved.
Revised: Moreover, when combined, the best performance, 31.25% AP 25 and 16.01% AP 50 , is achieved.

Original: Effectiveness of Prototypes.
Revised: Effectiveness of Prototypes.

Original: Table 4 shows the effectiveness of two kinds of prototypes.
Revised: Table 4 shows the effectiveness of two kinds of prototypes.

Original: In order to validate the geometric prototypes, we displace them by the self-learning ones, which are randomly initialized and updated by the gradient descend during the model training.
Revised: In order to validate the geometric prototypes, we displace them by the self-learning ones, which are randomly initialized and updated by the gradient descend during the model training.

Original: Table 4 results show that the performance significantly degrades.
Revised: Table 4 results show that the performance significantly degrades.

Original: To validate the effectiveness of class prototypes, we also alter them by the randomly initialized self-learning prototypes.
Revised: To validate the effectiveness of class prototypes, we also alter them by the randomly initialized self-learning prototypes.

Original: Similarly, the performance drops drastically due to the lack of class prototypes.
Revised: Similarly, the performance drops drastically due to the lack of class prototypes.

Original: [Table]
Revised: [Table]

Original: memory bank size.
Revised: memory bank size.

Original: Coefficient m
Revised: Coefficient m

Original: AP 0 . AP 0 .
Revised: AP 0 . AP 0 .

Original: 50 γ = 0 .29.
Revised: 50 γ = 0 .29.

Original: γ = 0 .30. γ = 0 .30.
Revised: γ = 0 .30. γ = 0 .30.

Original: γ = 0 .31.
Revised: γ = 0 .31.

Original: γ = 0 .30.
Revised: γ = 0 .30.

Original: Size of Memory Bank.
Revised: Size of Memory Bank.

Original: Table 5 studies the size of the memory bank containing the geometric prototypes.
Revised: Table 5 studies the size of the memory bank containing the geometric prototypes.

Original: This ablation study is performed on 3-shot in split-1 of FS-ScanNet.
Revised: This ablation study is performed on 3-shot in split-1 of FS-ScanNet.

Original: The value of K is set to { 30 , 60 , 90 , 120 , 150 } .
Revised: The value of K is set to { 30 , 60 , 90 , 120 , 150 } .

Original: For K = 30 , the memory bank only contains 30 geometric prototypes, which only achieves 29.98% AP 25 and 15.01% AP 50 .
Revised: For K = 30 , the memory bank only contains 30 geometric prototypes, which only achieves 29.98% AP 25 and 15.01% AP 50 .

Original: Furthermore, when using more prototypes (i.e., K = 120 ), there will be an obvious performance improvement, which reaches 31.25% AP 25 and 16.01% AP 50 .
Revised: Furthermore, when using more prototypes (i.e., K = 120 ), there will be an obvious performance improvement, which reaches 31.25% AP 25 and 16.01% AP 50 .

Original: However, when continuous increasing K, there will be no improvement.
Revised: However, when continuous increasing K, there will be no improvement.

Original: Therefore, we set the size of the memory bank to be 120.
Revised: Therefore, we set the size of the memory bank to be 120.

Original: Coefficient γ .
Revised: Coefficient γ .

Original: Table 6 shows the effect of momentum coefficient ( γ in Equation (5)).
Revised: Table 6 shows the effect of momentum coefficient ( γ in Equation (5)).

Original: The experiment is performed on 3-shot in split 1 of FS-ScanNet.
Revised: The experiment is performed on 3-shot in split 1 of FS-ScanNet.

Original: The results show that, when using a relatively large coefficient ( i.e. , γ ∈ [0 . 999 , 0 . 9999] ), the model performs well, compared with the model using a small momentum coefficient ( i.e. , γ ∈ [0 . 9 , 0 . 99] ).
Revised: The results show that, when using a relatively large coefficient ( i.e. , γ ∈ [0 . 999 , 0 . 9999] ), the model performs well, compared with the model using a small momentum coefficient ( i.e. , γ ∈ [0 . 9 , 0 . 99] ).

Original: Moreover, the performance drops when using a small value of γ = 0 .
Revised: Moreover, the performance drops when using a small value of γ = 0 .

Original: The is potentially because a small momentum coefficient might bring about unstable prototype representation with rapid prototype updating.
Revised: The is potentially because a small momentum coefficient might bring about unstable prototype representation with rapid prototype updating.

Original: [Table]
Revised: [Table]

Original: More Analysis on TFA.
Revised: More Analysis on TFA.

Original: TFA [40] is a simple yet effective method in few-shot 2D object detection.
Revised: TFA [39] is a simple yet effective method in few-shot 2D object detection.

Original: Specifically, TFA first trains a detector on the training set with abundant samples of base classes
Revised: Specifically, TFA first trains a detector on the training set with abundant samples of base classes

Original: [Table]
Revised: [Table]

Original: and then finetunes the classifier and the regressor on a small balanced set, which contains both base classes and novel classes.
Revised: and then finetunes the classifier and the regressor on a small balanced set, which contains both base classes and novel classes.

Original: More details can be referred to in [40].
Revised: More details can be referred to in [39].

Original: However, as shown in Table 1 and Table 2 in the main paper, VoteNet+TFA performs poorly in few-shot 3D object detection.
Revised: However, as shown in Table 1 and Table 2 in the main paper, VoteNet+TFA performs poorly in few-shot 3D object detection.

Original: As discussed in Section 4.3 in the main paper, the reason is that a few layers are finetuned by the samples of novel classes.
Revised: As discussed in Section 4.3 in the main paper, the reason is that a few layers are finetuned by the samples of novel classes.

Original: A question arises here: why does TFA in few-shot 2D object detection only need to train the classifier and the regressor on the novel samples?
Revised: A question arises here: why does TFA in few-shot 2D object detection only need to train the classifier and the regressor on the novel samples?

Original: We speculate that the large-scale pre-training on ImageNet [7] helps.
Revised: We speculate that the large-scale pre-training on ImageNet [6] helps.

Original: To overcome this problem, in the first stage of TFA, we train the
Revised: To overcome this problem, in the first stage of TFA, we train the

Original: VoteNet on the training set with both novel classes and base classes.
Revised: VoteNet on the training set with both novel classes and base classes.

Original: Then, in the second stage, we use a small balanced set containing both novel classes and base classes to finetune the classifier and the regressor.
Revised: Then, in the second stage, we use a small balanced set containing both novel classes and base classes to finetune the classifier and the regressor.

Original: We denote this new baseline as VoteNet+TFA ∗ .
Revised: We denote this new baseline as VoteNet+TFA ∗ .

Original: As shown in Table 7, VoteNet+TFA ∗ boosts the performance significantly.
Revised: As shown in Table 7, VoteNet+TFA ∗ boosts the performance significantly.

Original: For example, in 3-shot in split-1 of FS-ScanNet, VoteNet+TFA ∗ can achieve 26.02% AP 25 and 13.96% AP 50 , while VoteNet+TFA only reaches 16.36% AP 25 and 7.91% AP 50 .
Revised: For example, in 3-shot in split-1 of FS-ScanNet, VoteNet+TFA ∗ can achieve 26.02% AP 25 and 13.96% AP 50 , while VoteNet+TFA only reaches 16.36% AP 25 and 7.91% AP 50 .

Original: More Methods Borrowed From 2D Few-Shot Object detection.
Revised: More Methods Borrowed From 2D Few-Shot Object detection.

Original: We combine two SOTA 2D few-shot object detection techniques (i.e. DeFRCN [27], FADI [1]) and two SOTA 3D detectors (i.e. GroupFree [19], 3DETR [23]).
Revised: We combine two SOTA 2D few-shot object detection techniques (i.e. DeFRCN [26], FADI [1]) and two SOTA 3D detectors (i.e. GroupFree [18], 3DETR [22]).

Original: These two few-shot techniques are plug-in-play modules and can be easily incorporated into the different detection architectures.
Revised: These two few-shot techniques are plug-in-play modules and can be easily incorporated into the different detection architectures.

Original: We conducted this experiment on 3-shot and 5-shot in split-1 of FS-ScanNet.
Revised: We conducted this experiment on 3-shot and 5-shot in split-1 of FS-ScanNet.

Original: The results in Table 8 show that our method still surpasses these methods by a large margin.
Revised: The results in Table 8 show that our method still surpasses these methods by a large margin.

Original: This is potentially because, in the 2D domain, they often build their model upon a large-scale pre-trained model on ImageNet.
Revised: This is potentially because, in the 2D domain, they often build their model upon a large-scale pre-trained model on ImageNet.

Original: However, in the 3D community, there does not exist a large-scale dataset for model pre-training, which requires future investigations.
Revised: However, in the 3D community, there does not exist a large-scale dataset for model pre-training, which requires future investigations.

Original: Therefore, these 2D few-shot object detection techniques might not be directly transferable to the 3D domain.
Revised: Therefore, these 2D few-shot object detection techniques might not be directly transferable to the 3D domain.

Original: For future works, we might resort to the pre-training models in the 2D domain to facilitate the few-shot generalization on 3D few-shot learning and how these techniques can be combined with our method.
Revised: For future works, we might resort to the pre-training models in the 2D domain to facilitate the few-shot generalization on 3D few-shot learning and how these techniques can be combined with our method.

Original:  Concluding Remarks
Revised:  Concluding Remarks

Original: In this paper, we have presented Prototypical VoteNet for FS3D along with a new benchmark for evaluation.
Revised: In this paper, we have presented Prototypical VoteNet for FS3D along with a new benchmark for evaluation.

Original: Prototypical VoteNet enjoys the advantages of two new designs, namely Prototypical
Revised: Prototypical VoteNet enjoys the advantages of two new designs, namely Prototypical

Original: Vote Module (PVM) and Prototypical Head Module (PHM), for enhancing feature learning in the few-shot setting.
Revised: Vote Module (PVM) and Prototypical Head Module (PHM), for enhancing feature learning in the few-shot setting.

Original: Specifically, PVM exploits geometric prototypes learned from base categories to refine local features of novel categories.
Revised: Specifically, PVM exploits geometric prototypes learned from base categories to refine local features of novel categories.

Original: PHM is proposed to utilize class-specific prototypes to promote discriminativeness of object-level features.
Revised: PHM is proposed to utilize class-specific prototypes to promote discriminativeness of object-level features.

Original: Extensive experiments on two new benchmark datasets demonstrate the superiority of our approach.
Revised: Extensive experiments on two new benchmark datasets demonstrate the superiority of our approach.

Original: We hope our studies on 3D propotypes and proposed new benchmark could inspire further investigations in few-shot 3D object detection.
Revised: We hope our studies on 3D propotypes and proposed new benchmark could inspire further investigations in few-shot 3D object detection.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: This work has been supported by Hong Kong Research Grant Council - Early Career Scheme (Grant
Revised: This work has been supported by Hong Kong Research Grant Council - Early Career Scheme (Grant

Original: No. 27209621), HKU Startup Fund, HKU Seed Fund for Basic Research, and Tencent Research
Revised: No. 27209621), HKU Startup Fund, HKU Seed Fund for Basic Research, and Tencent Research

Original: Fund.
Revised: Fund.

Original: Part of the described research work is conducted in the JC STEM Lab of Robotics for Soft
Revised: Part of the described research work is conducted in the JC STEM Lab of Robotics for Soft

Original: Materials funded by The Hong Kong Jockey Club Charities Trust.
Revised: Materials funded by The Hong Kong Jockey Club Charities Trust.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information on how to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No], or [N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or [N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .
Revised: [Yes] See Section ?? .

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data are proprietary.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] we have included limitation discussion in the supplementary material Section A.7.
Revised: [Yes] we have included limitation discussion in the supplementary material Section A.7.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [Yes] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [Yes] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [Yes] 5.
Revised: [Yes] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?[N/A] A Appendix
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: In this supplemental material, we will include the details of dataset split in Section A.1, the experimental results on base classes in Section A.2, the ablation study of hard and soft assignment in Section A.3, implementation and training details of Prototypical VoteNet in Section A.4, implementation details of the baseline method Meta VoteNet in Section A.5, visualization of basic geometric primitives in

Original: Section A.6, KNN baselines in Section A.7, non-updated prototypes in Section A.8, performance on the unbalance Problem in Section A.9 and limitation analysis in Section A.10.

Original: A.1 Dataset Split

Original: Table 9 lists the names of novel classes for FS-SUNRGBD and FS-ScanNet.

Original: [Table]

Original: table, bed, nightstand, toilet

Original: [Table]

Original: bookshelf

Original: [Table]

Original: A.2 Results on Base Classes

Original: [Table]

Original: thresholds of

Original: AP 25 and AP 50 .

Original: Since we detect both base and novel classes during inference, we also report the performance on base classes in Table 10.

Original: For simplicity, we average the results of all k-shot ( e.g. k=1,3,5) in each split.

Original: Table 10 shows VoteNet achieves the best performance on base classes, because it is specifically designed as a fully supervised method for base classes.

Original: Compared with VoteNet, the performance of

Original: VoteNet+TFA and VoteNet+PT+TFA on base classes degrades by -6.96% AP 25 and -8.38% AP 50 , and -5.18% AP 25 and -7.14% AP 50 , respectively, on split-1 of FS-ScanNet.

Original: Moreover, Prototypical

Original: VoteNet retains much more ability to recognize base classes, which achieves 53.83% AP 25 and 28.20% AP 50 on split-1 of FS-ScanNet, and 46.07% AP 25 and 25.26% AP 50 on FS-SUNRGBD.

Original: Compared with our method, we can observe a significant performance degradation in finetuning based methods.

Original: The reason is that, without the large-scale pre-training, a detector cannot learn more generic feature representation so as to transfer knowledge from base classes to novel classes.

Original: Under this circumstance, if we force the knowledge transferring by finetuning, it will distort the learned feature space for base classes.

Original: A.3 Hard vs. Soft Assignment

Original: [Table]
Revised: [N/A]

Original: We leverage the hard assignment in Section 3.2.1: Geometric Prototype Construction, in the main paper.

Original: Here, we compare the original hard assignment in our implemented method with the soft assignment, which calculates the similarity between a point feature with all geometric prototypes and updates all geometric prototypes in a soft manner by the similarity scores between a point feature and the geometric prototypes.

Original: We conduct the experiment on 3-shot and 5-shot in split-1 of FS-ScanNet.

Original: The results in Table 11 indicate that the hard assignment is more effective than the soft assignment.

Original: This is because the geometric prototypes by the hard assignment are more distinctive than those using the soft assignment, since one geometric prototype is updated using the nearest point features without considering the others.

Original: A.4 Implementation and Training Details

Original: We follow recent practice [25, 46] to use PointNet++ [26] as a default backbone network.

Original: The backbone has 4 set abstraction layers and 2 feature propagation layers.

Original: For each set abstraction layer, the input point cloud is sub-sampled to 2048, 1024, 512, and 256 points with the increasing receptive radius of 0.2, 0.4, 0.8, and 1.2, respectively.

Original: Then, two feature propagation layers successively up-sample the points to 512 and 1024.

Original: Additionally, in both PHM and PVM, the multi-head attention layer are combined with feed forward FC layers.

Original: The details can be referred to our code.

Original: The size of memory bank of geometric prototypes is set to 120.

Original: The number of heads in both multi-head attention networks is empirically set to 4.

Original: The momentum coefficient is set to 0.999.

Original: We use 40k points in

Original: FS-ScanNet and 20k points in FS-SUNRGBD as input and adopt the same data augmentation as in [25], including a random flip, a random rotation, and a random scaling of the point cloud by [0.9, 1.1].

Original: Following the episodic training strategy [47], a training mini-batch in Prototypical VoteNet is comprised of a K-shot R-class support set D support and a R-class query set D train (the classes in D support and D train is consistent).

Original: Each sample in D support is the whole point cloud scene.

Original: Before the pooling step, we use the groud-truth bounding boxes to obtain point features of target objects.

Original: The network is trained from scratch by the AdamW optimizer with 36 epochs.

Original: The weight decay is set to 0.01.

Original: The initial learning rate is 0.008 in FS-ScanNet and 0.001 in FS-SUNRGBD.

Original: Additionally, during the inference stage, we input both the query point cloud and the given support set to Prototypical VoteNet.

Original: Note that the features of the support point cloud only need to be extracted once, as they are independent of the query feature extraction.

Original: A.5 Implementation Details of Meta VoteNet

Original: We provide more details of the competitive baseline corresponding to Meta VoteNet in our main paper.

Original: It is derived from an effective few-shot 2D object detection approach - Meta RCNN [47].

Original: In

Original: Meta RCNN, each RoI feature is fused with R class prototypes using the channel-wise multiplication operator.

Original: As a result, a number of R fused RoI features for each RoI are generated.

Original: Then each fused

Original: RoI feature is fed into the prediction layer for the binary prediction (whether the RoI feature is the category that the class prototype belongs to).

Original: More details can be referred to in [47].

Original: We incorporate this meta-learning approach into VoteNet, namely Meta VoteNet.

Original: Similarly, after Sampling &

Original: Grouping (see Figure 2 in the main paper), we fuse point features of objects with R class prototypes by the channel-wise multiplication operator.

Original: Then the binary class prediction and bounding box regression are conducted based on the fused features for classification and location prediction, respectively.

Original: For a fair comparison, Meta VoteNet shares the same architecture with Prototypical

Original: VoteNet, except Prototypical Vote Module and Prototypical Prediction Module.

Original: The training details follow the proposed Prototypical VoteNet.

Original: A.6 Visualization of Basic Geometric Primitives

Original: In Figure 5, we visualize the relation between the learned geometric prototypes and the 3D points by searching points with features that are similar to a given geometric prototype.

Original: First, we feed object point clouds to a trained Prototypical VoteNet.

Original: Second, for each point feature, we can search for its most similar prototype.

Original: If the similarity is above a threshold, we can assign the point to that prototype.

Original: Third, we use a density-based clustering algorithm DBSCAN to cluster the point groups, and we draw the minimum 3D bounding box around each point group.

Original: As shown in the figure, all the red bounding boxes within each subfigure belong to the same prototype.

Original: The result shows that in each subfigure,

Original: [Figure]

Original: the enclosed geometric structures are similar.

Original: For example, subfigure (a) illustrates that the prototype learns the feature of corners, while subfigure (b) shows that the prototype learns the long stick.

Original: A.7 KNN Baselines

Original: [Table]

Original: We apply KNN assignment to VoteNet and two SOTA 3D detectors GroupFree [19] and 3DETR [23].

Original: We conducted this experiment on 3-shot and 5-shot in split-1 of FS-ScanNet.

Original: The KNN assignment is realized by calculating the distance between each object feature and features of all training objects in the classification step, and assigning the sample to the class based on voting from its k-nearest objects of the training set.

Original: Here, we take k as one since we find increasing the value k doesn’t improve performance.

Original: The results are shown in Table 12.

Original: Comparing the performance of “VoteNet +

Original: KNN” and “ours”, we see that the non-parametric KNN classifier will not help improve few-shot learning much (“VoteNet” vs “VoteNet+KNN”).

Original: Comparing the performance of different detectors (“VoetNet+KNN”, “GroupFree + KNN”, and “3DETR+KNN”), we observe that a better detection architecture does not bring large performance gains in the few-shot 3D detection scenario.

Original: The most challenging issue for few-shot 3D object detection still lies in how to learn effective representation if only a few training samples are provided.

Original: The classifier and architecture don’t help much if the model cannot effectively extract features to represent novel categories with only a few samples.

Original: A.8 Non-Updated Prototypes

Original: [Table]

Original: As shown in Table 13, for the proposed Prototypical VoteNet, if we don’t update the prototype in

Original: PVM, the performance would degrade significantly.

Original: Without updating, the randomly initialized prototypes can not learn the geometry information from base classes in the training phase.

Original: In this case, it is hard to transfer the basic geometry information from base classes to the novel classes as the prototypes are meaningless.

Original: A.9 Performance on the Unbalance Problem

Original: [Table]

Original: [Table]

Original: To analyze the performance of the proposed model on the imbalance problem, we conduct experiments using all the classes.

Original: Note that we conduct the experiments not only on ScanNet V2, but also on the more unbalanced counterparts.

Original: We follow the benchmark [5], to create these counterparts: 1) sorting the classes in descending order according to number of samples in each class, then we have n i > n j if i < j , where n is the number of samples, i and j denote the index of the classes.

Original: 2) reducing the number of training samples per class according to an exponential function n = n i ∗ u i , where u ∈ (0 , 1) .

Original: The test set remains unchanged.

Original: According to the benchmark [5], we define the imbalance factor of a dataset as the number of training samples in the largest class divided by the smallest.

Original: Note that we use P as the value of the imbalance factor in ScanNet V2.

Original: Additionally, we add another three sets, whose values of imbalance factor are 10P, 25P and 50P for both ScanNet V2.

Original: As shown in

Original: Table 14, we achieve comparable performance in the original dataset setting.

Original: With the imbalance becoming more severe (e.g., 25P, 50P), our approach outperforms the baseline more.

Original: Note that our focus is on few-shot 3D object detection, where representation learning of new categories becomes the top consideration of algorithm design.

Original: This few-shot problem is more useful for scenarios where many new categories appear frequently and require the system to quickly adapt to recognize them.

Original: However, the long-tailed problem focuses on how to learn good representations and classifiers that can deliver good performance for both head and tail categories.

Original: We believe that dedicated designs can further improve the performance of long-tailed 3D object detection.

Original: We will also add the results and analysis for the long-tailed setting in our paper and hope to inspire more future investigations.

Original: A.10 Limitation Analysis

Original: Although the 3D cues of point clouds are more stable since they can get rid of some visual distractors, such as lighting and perspectives, some factors still impede the model from better generalization.

Original: For instance, in 3D scene understanding, if the point cloud in the training set is dense and that of the test set is sparse, a model often performs poorly, which can be treated as a cross-domain problem.

Original: Regarding few-shot 3D object detection, the performance might degrade if there is such a large domain gap between base classes and novel classes.

Original: Even though the basic geometric features are learned in the base classes, they might not be generalized well to the novel classes due to the difference in point cloud sparsity.

Original: The performance of this model has much room for improvement.

Original: One way to achieve better performance is large-scale pre-training.

Original: Large-scale pre-training enables the model to learn more generic features for transfer learning using limited samples, which benefits the community of 2D few-shot learning (i.e., ImageNet Pre-training).

Original: For future works, we might resort to the pre-training models in the 2D domain to facilitate the few-shot generalization on 3D few-shot learning and how these techniques can be combined with our method.