Original: O PEN -S ET 3D D ETECTION VIA I MAGE - LEVEL C LASS AND D E - BIASED C ROSS - MODAL C ONTRASTIVE L EARNING
Revised: O PEN -S ET 3D D ETECTION VIA I MAGE - LEVEL C LASS AND D E - BIASED C ROSS - MODAL C ONTRASTIVE L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Current point-cloud detectors have difficulty in detecting the open-set objects in the real world, due to their limited generalization capability.
Revised: Current point-cloud detectors have difficulty in detecting the open-set objects in the real world, due to their limited generalization capability.

Original: Moreover, collecting and fully annotating a point-cloud detection dataset with a large number of classes of objects is extremely laborious and expensive, leading to the limited class size of existing point-cloud datasets and hindering the model from learning general representations for open-set point-cloud detection.
Revised: Moreover, collecting and fully annotating a point-cloud detection dataset with a large number of classes of objects is extremely laborious and expensive, leading to the limited class size of existing point-cloud datasets and hindering the model from learning general representations for open-set point-cloud detection.

Original: Instead of seeking well-annotated point-cloud datasets, we resort to ImageNet1K to broaden the vocabulary of pointcloud detectors.
Revised: Instead of seeking well-annotated point-cloud datasets, we resort to ImageNet1K to broaden the vocabulary of pointcloud detectors.

Original: Specifically, we propose OS-3DETIC, an O pen- S et 3 D DET ector using I mage-level C lass supervision.
Revised: Specifically, we propose OS-3DETIC, an O pen- S et 3 D DET ector using I mage-level C lass supervision.

Original: Intuitively, we take advantage of two modalities, the image modality for recognition and the point-cloud modality for localization, to generate pseudo-labels for unseen classes.
Revised: Intuitively, we take advantage of two modalities, the image modality for recognition and the point-cloud modality for localization, to generate pseudo-labels for unseen classes.

Original: We then propose a novel de-biased cross-modal contrastive learning strategy to transfer knowledge from image to point-cloud.
Revised: We then propose a novel de-biased cross-modal contrastive learning strategy to transfer knowledge from image to point-cloud.

Original: Without hurting the latency during inference, OS-3DETIC makes the well-known point-cloud detector capable of achieving open-set detection.
Revised: Without hurting the latency during inference, OS-3DETIC makes the well-known point-cloud detector capable of achieving open-set detection.

Original: Extensive experiments demonstrate that the proposed OS-3DETIC achieves at least 10.77% mAP improvement (absolute value) and 9.56% mAP improvement (absolute value) by a wide range of baselines on the SUN-RGBD and ScanNetV2, respectively.
Revised: Extensive experiments demonstrate that the proposed OS-3DETIC achieves at least 10.77% mAP improvement (absolute value) and 9.56% mAP improvement (absolute value) by a wide range of baselines on the SUN-RGBD and ScanNetV2, respectively.

Original: Moreover, we conduct sufficient experiments to shed light on why the proposed OS-3DETIC works.
Revised: Moreover, we conduct sufficient experiments to shed light on why the proposed OS-3DETIC works.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: 3D point-cloud detection is defined as finding objects (localization) in point-cloud and naming them (classification).
Revised: 3D point-cloud detection is defined as finding objects (localization) in point-cloud and naming them (classification).

Original: Recently, deep learning based 3D detectors have achieved significant progress.
Revised: Recently, deep learning based 3D detectors have achieved significant progress.

Original: However, most methods are developed on point-cloud detection datasets with limited classes, whereas the real world has a cornucopia of classes.
Revised: However, most methods are developed on point-cloud detection datasets with limited classes, whereas the real world has a cornucopia of classes.

Original: It is common for 3D detectors to encounter objects that had never occurred during training, resulting in failure to generalize to real-life scenarios.
Revised: It is common for 3D detectors to encounter objects that had never occurred during training, resulting in failure to generalize to real-life scenarios.

Original: Therefore, it is extremely important to design an open-set point-cloud detector which is able to generalize to unseen classes.
Revised: Therefore, it is extremely important to design an open-set point-cloud detector which is able to generalize to unseen classes.

Original: The key ingredient of open-set detection is that the model learns sufficient knowledge thus is able to output general representations.
Revised: The key ingredient of open-set detection is that the model learns sufficient knowledge thus is able to output general representations.

Original: To achieve this, in the image field, typical open-set classification and detection either require to introduce large-scale image-text pairs or image datasets with sufficient labels.
Revised: To achieve this, in the image field, typical open-set classification and detection either require to introduce large-scale image-text pairs or image datasets with sufficient labels.

Original: For example, CLIP Radford et al.
Revised: For example, CLIP Radford et al.

Original: (2021) introducedmillion image-text pairs for pre-training to help visual models learn general representation.
Revised: (2021) introducedmillion image-text pairs for pre-training to help visual models learn general representation.

Original: Detic
Revised: Detic

Original: Zhou et al.
Revised: Zhou et al.

Original: leverages ImageNet21K to extend the knowledge of image detectors.
Revised: leverages ImageNet21K to extend the knowledge of image detectors.

Original: OVR-CNN
Revised: OVR-CNN

Original: Zareian et al.
Revised: Zareian et al.

Original: (2021) uses the language pretrained embedding layer to broaden the vocabulary of the 2D detector.
Revised: (2021) uses the language pretrained embedding layer to broaden the vocabulary of the 2D detector.

Original: However in the point-cloud field, as far as we know, there are few studies for the open-set pointcloud detection.
Revised: However in the point-cloud field, as far as we know, there are few studies for the open-set pointcloud detection.

Original: The most notable hindrance is that we can hardly obtain large-scale point-cloud data and labels (or optionally, captions like the aforementioned image field), due to the difficulty of collection and annotation.
Revised: The most notable hindrance is that we can hardly obtain large-scale point-cloud data and labels (or optionally, captions like the aforementioned image field), due to the difficulty of collection and annotation.

Original: The scarcity of point-cloud data and labels drastically restricts the pointcloud model from learning sufficient knowledge and obtaining general representations.
Revised: The scarcity of point-cloud data and labels drastically restricts the pointcloud model from learning sufficient knowledge and obtaining general representations.

Original: Therefore, this limitation motivates us to ask - Can we transfer the knowledge from images to point-cloud so that the point-cloud model is capable of learning general representations?
Revised: Therefore, this limitation motivates us to ask - Can we transfer the knowledge from images to point-cloud so that the point-cloud model is capable of learning general representations?

Original: [Figure]
Revised: [Figure]

Original: al.
Revised: al.

Original: (2021a)
Revised: (2021a)

Original: ; Park et al.
Revised: ; Park et al.

Original: Human-beings have no problem understanding both modalities.
Revised: Human-beings have no problem understanding both modalities.

Original: But two issues still exist: what kind of image data can we use and how to use the image data?
Revised: But two issues still exist: what kind of image data can we use and how to use the image data?

Original: It is straightforward that we directly utilize the image detection dataset with fully bounding boxes and class labels, and transfer the knowledge from the 2D detector to 3D.
Revised: It is straightforward that we directly utilize the image detection dataset with fully bounding boxes and class labels, and transfer the knowledge from the 2D detector to 3D.

Original: However, bounding-box level annotation is still laborious and difficult to scale, while open-set detection requires rich labels to help the detector learn sufficient knowledge Zhou et al.
Revised: However, bounding-box level annotation is still laborious and difficult to scale, while open-set detection requires rich labels to help the detector learn sufficient knowledge Zhou et al.

Original: Therefore, in this paper, instead of seeking to build a large-scale point-cloud dataset or use 2D detection datasets, we open-up another path by resorting to large-scale image datasets with image-level class supervision, ImageNet1K Krizhevsky et al.
Revised: Therefore, in this paper, instead of seeking to build a large-scale point-cloud dataset or use 2D detection datasets, we open-up another path by resorting to large-scale image datasets with image-level class supervision, ImageNet1K Krizhevsky et al.

Original: (2012), to enable the point-cloud detector capable of learning general representations, thus broadening the vocabulary of the point-cloud detector, as shown in Fig.
Revised: (2012), to enable the point-cloud detector capable of learning general representations, thus broadening the vocabulary of the point-cloud detector, as shown in Fig.

Original: 1(a).
Revised: 1(a).

Original: Open-set 3D detection it to detect unseen categories without corresponding 3D labels.
Revised: Open-set 3D detection it to detect unseen categories without corresponding 3D labels.

Original: Note that, in this setting, open-set is defined in terms of the 3D detection, we can make use of external knowledge such as ImageNet, which only provides Image-level category knowledge.
Revised: Note that, in this setting, open-set is defined in terms of the 3D detection, we can make use of external knowledge such as ImageNet, which only provides Image-level category knowledge.

Original: Specifically, the proposed
Revised: Specifically, the proposed

Original: OS-3DETIC makes the 3D detector learning sufficient knowledge from image-level supervision thus achieving open-set point-cloud detection, and it is a synergy of two components: 1) Make full use of knowledge learning from ImageNet and generalizability of localization on point-cloud to generate pseudo-labels for unseen class.
Revised: OS-3DETIC makes the 3D detector learning sufficient knowledge from image-level supervision thus achieving open-set point-cloud detection, and it is a synergy of two components: 1) Make full use of knowledge learning from ImageNet and generalizability of localization on point-cloud to generate pseudo-labels for unseen class.

Original: We design a de-biased cross-modal contrastive learning with distance-aware temperature to capture the shared low-dimensional space within and across modalities, thus better transferring the sufficient knowledge from image domain to point-cloud domain.
Revised: We design a de-biased cross-modal contrastive learning with distance-aware temperature to capture the shared low-dimensional space within and across modalities, thus better transferring the sufficient knowledge from image domain to point-cloud domain.

Original: It is noteworthy to mention that during training, we introduce the paired images to narrow the gap between the point-cloud data and the images from ImageNet, but we do not need any extra annotations except the Lidar-Camera transformation matrix.
Revised: It is noteworthy to mention that during training, we introduce the paired images to narrow the gap between the point-cloud data and the images from ImageNet, but we do not need any extra annotations except the Lidar-Camera transformation matrix.

Original: Extensive experiments show that OS-3DETIC outperforms a wide range of state-of-the-art baselines by at least 10.77% mAP (absolute) and 9.56% mAP (absolute) without hurting the latency of the original 3D detector, on the unseen classes of SUN RGB-D Song et al.
Revised: Extensive experiments show that OS-3DETIC outperforms a wide range of state-of-the-art baselines by at least 10.77% mAP (absolute) and 9.56% mAP (absolute) without hurting the latency of the original 3D detector, on the unseen classes of SUN RGB-D Song et al.

Original: (2015) and the ScanNet Dai et al.
Revised: (2015) and the ScanNet Dai et al.

Original: (2017), respectively.
Revised: (2017), respectively.

Original: An example on the ScanNet dataset is shown in Fig.
Revised: An example on the ScanNet dataset is shown in Fig.

Original: 1(c).
Revised: 1(c).

Original: Sufficient ablation studies shed light on why the OS-3DETIC works.
Revised: Sufficient ablation studies shed light on why the OS-3DETIC works.

Original: Overall, our contribution is as follows:
Revised: Overall, our contribution is as follows:

Original: • We propose an open-set 3D detector with image-level class, termed as OS-3DETIC, which is a synergy of two components: the pseudo-label generation from two modalities, and the de-biased cross-modal contrastive learning with distance-aware temperature.
Revised: • We propose an open-set 3D detector with image-level class, termed as OS-3DETIC, which is a synergy of two components: the pseudo-label generation from two modalities, and the de-biased cross-modal contrastive learning with distance-aware temperature.

Original: • OS-3DETIC can be regarded as a superior baseline in open-set 3D point-cloud detection.
Revised: • OS-3DETIC can be regarded as a superior baseline in open-set 3D point-cloud detection.

Original: • Extensive experiments demostrate the effectiveness of OS-3DETIC, and we also provide sufficient analysis to uncover why it works.
Revised: • Extensive experiments demostrate the effectiveness of OS-3DETIC, and we also provide sufficient analysis to uncover why it works.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: open-set object detection targets to detect novel classes that are never provided labels during the training Bansal et al.
Revised: open-set object detection targets to detect novel classes that are never provided labels during the training Bansal et al.

Original: ; Gu et al.
Revised: ; Gu et al.

Original: ; Rahman et al.
Revised: ; Rahman et al.

Original: (2020a;b); Zhou et al.
Revised: (2020a;b); Zhou et al.

Original: ; Zareian et al.
Revised: ; Zareian et al.

Original: The most similar work to us is Detic Zhou et al.
Revised: The most similar work to us is Detic Zhou et al.

Original: (2022), which utilizes ImageNet21K to broaden the classifier of the 2D detector.
Revised: (2022), which utilizes ImageNet21K to broaden the classifier of the 2D detector.

Original: Yet, it is infeasible to directly use the same method to broaden the classifier of the point-cloud detector, due to the large gap between image and pointcloud.
Revised: Yet, it is infeasible to directly use the same method to broaden the classifier of the point-cloud detector, due to the large gap between image and pointcloud.

Original: Different from Detic that transfers knowledge within the same modality, we propose to transfer knowledge from ImageNet to a totally different modality.
Revised: Different from Detic that transfers knowledge within the same modality, we propose to transfer knowledge from ImageNet to a totally different modality.

Original: Point-Cloud Detection has been actively researched Yang et al.
Revised: Point-Cloud Detection has been actively researched Yang et al.

Original: ; Chen et al.
Revised: ; Chen et al.

Original: ; Wu et al.
Revised: ; Wu et al.

Original: Xu et al.
Revised: Xu et al.

Original: (2020); Shi et al.
Revised: (2020); Shi et al.

Original: ; Xu et al.
Revised: ; Xu et al.

Original: (2018; 2021b)
Revised: (2018; 2021b)

Original: ; Zhou & Tuzel (2018)
Revised: ; Zhou & Tuzel (2018)

Original: ; Yan et al.
Revised: ; Yan et al.

Original: ; Shi et al.
Revised: ; Shi et al.

Original: (2020b;a);
Revised: (2020b;a);

Original: Dosovitskiy et al.
Revised: Dosovitskiy et al.

Original: ; Wu et al.
Revised: ; Wu et al.

Original: ; Liu et al.
Revised: ; Liu et al.

Original: (2021c)
Revised: (2021c)

Original: ; Misra et al.
Revised: ; Misra et al.

Original: ; Zhao et al.
Revised: ; Zhao et al.

Original: (2021), yet to the best of our knowledge, open-set 3D point-cloud detection has not been widely explored, existing methods Cen et al.
Revised: (2021), yet to the best of our knowledge, open-set 3D point-cloud detection has not been widely explored, existing methods Cen et al.

Original: ; Wong et al.
Revised: ; Wong et al.

Original: (2020) mainly focus on identify the unknown objects from known one, whereas in our case, we further assign a name to each unknown object, which is similar to Xu et al.
Revised: (2020) mainly focus on identify the unknown objects from known one, whereas in our case, we further assign a name to each unknown object, which is similar to Xu et al.

Original: (2021a)
Revised: (2021a)

Original: ; Zhang et al.
Revised: ; Zhang et al.

Original: ; Cheraghian et al.
Revised: ; Cheraghian et al.

Original: (2019b;a; 2021), yet these works mainly study classification.
Revised: (2019b;a; 2021), but these works mainly study classification.

Original:  M ETHOD 3. N OTATION AND P RELIMINARIES
Revised:  M ETHOD 3. N OTATION AND P RELIMINARIES

Original: We use I ∈ R 3 × H × W to represent image, and P = { p i ∈ R 3 , i = 1 , 2 , 3 ..., N } to represent point-cloud, where N is the point number in the point-cloud.
Revised: We use I ∈ R 3 × H × W to represent image, and P = { p i ∈ R 3 , i = 1 , 2 , 3 ..., N } to represent point-cloud, where N is the point number in the point-cloud.

Original: During training, we use 1) point-cloud dataset denoted as D pc = { ( P , ( b 3 D ∈ R 7 , c
Revised: During training, we use 1) point-cloud dataset denoted as D pc = { ( P , ( b 3 D ∈ R 7 , c

Original: 3 D ) k ) j } |D pc | j =1 , with vocabulary size
Revised: 3 D ) k ) j } |D pc | j =1 , with vocabulary size

Original: C pc , where b 3 D is the annotation of 3D bounding box, c
Revised: C pc , where b 3 D is the annotation of 3D bounding box, c

Original: 3 D is the corresponding classification label; 2) paired image dataset denoted as D img
Revised: 3 D is the corresponding classification label; 2) paired image dataset denoted as D img

Original: = { I j } |D img | j = ; 3) ImageNet1K dataset denoted as
Revised: = { I j } |D img | j = ; 3) ImageNet1K dataset denoted as

Original: D ign =
Revised: D ign =

Original: ( I , c ign 2 D ) j } |D ign | j =1 , with vocabulary C ign , where c ign 2 D denotes classfication label of the image in ImageNet1K. During test, we evaluate on the vocabulary C test , where C ign ≥ C test > C pc .
Revised: ( I , c ign 2 D ) j } |D ign | j =1 , with vocabulary C ign , where c ign 2 D denotes classfication label of the image in ImageNet1K. During test, we evaluate on the vocabulary C test , where C ign ≥ C test > C pc .

Original: A typical point-cloud detector deals with localization and classification, where the localization module outputs bounding boxes ˆ b 3 D ∈ R 7 and we could get its corresponding point-cloud ROI features f 3 D .
Revised: A typical point-cloud detector deals with localization and classification, where the localization module outputs bounding boxes ˆ b 3 D ∈ R 7 and we could get its corresponding point-cloud ROI features f 3 D .

Original: Similarly, we can project the 3D bounding box into 2D image via projection matrix K , i.e., ˆ b 2 D ∈ R 4 and index the corresponding image ROI feature f 2 D .
Revised: Similarly, we can project the 3D bounding box into 2D image via projection matrix K , i.e., ˆ b 2 D ∈ R 4 and index the corresponding image ROI feature f 2 D .

Original: Then we can use the f 3 D and f 2 D to predict the class of each object.
Revised: Then we can use the f 3 D and f 2 D to predict the class of each object.

Original: 3. OS-3DETIC: OPEN - SET 3D D ETECTOR WITH I MAGE C LASSES
Revised: 3. OS-3DETIC: OPEN - SET 3D D ETECTOR WITH I MAGE C LASSES

Original: We use ImageNet1K to broaden the vocabulary of the point-cloud detector.
Revised: We use ImageNet1K to broaden the vocabulary of the point-cloud detector.

Original: To transfer the knowledge contained in ImageNet1K to the point-cloud detector, paired image is introduced to bridge these two modalities.
Revised: To transfer the knowledge contained in ImageNet1K to the point-cloud detector, paired image is introduced to bridge these two modalities.

Original: Specifically, We design a two-phase training strategy to enable open-set point-cloud detection.
Revised: Specifically, We design a two-phase training strategy to enable open-set point-cloud detection.

Original: The first-phase training is similar to Detic Zhou et al.
Revised: The first-phase training is similar to Detic Zhou et al.

Original: (2022), which aims at leveraging
Revised: (2022), which aims at leveraging

Original: ImageNet to help the 2D detector be able to learn sufficient knowledge.
Revised: ImageNet to help the 2D detector be able to learn sufficient knowledge.

Original: The second-phase is the core of the proposed OS-3DETIC, which aims at transferring the knowledge of the 2D detector to the 3D detector via customized pseudo-label strategy and de-biased cross-modal contrastive learning, as shown in Fig.
Revised: The second-phase is the core of the proposed OS-3DETIC, which aims at transferring the knowledge of the 2D detector to the 3D detector via customized pseudo-label strategy and de-biased cross-modal contrastive learning, as shown in Fig.

Original: During inference, we only need the 3D detector without any extra models or modalities.
Revised: During inference, we only need the 3D detector without any extra models or modalities.

Original: OS-3DETIC is developed on top of 3DETR Misra et al.
Revised: OS-3DETIC is developed on top of 3DETR Misra et al.

Original: , and we use DETR
Revised: , and we use DETR

Original: Carion et al.
Revised: Carion et al.

Original: as the 2D detector.
Revised: as the 2D detector.

Original: The details are illustrated below.
Revised: The details are illustrated below.

Original: In the first stage, the point-cloud P in D pc is injected to the 3D detector, which is supervised by the ground truth { b 3 D , c 3 D } of seen objects.
Revised: In the first stage, the point-cloud P in D pc is injected to the 3D detector, which is supervised by the ground truth { b 3 D , c 3 D } of seen objects.

Original: The paired images I from D img are input into the 2D detector, which is supervised by projected 3D boxes and the corresponding class, denoted as { K × b 3 D , c 3 D } .
Revised: The paired images I from D img are input into the 2D detector, which is supervised by projected 3D boxes and the corresponding class, denoted as { K × b 3 D , c 3 D } .

Original: It is noteworthy to mention that the images I from from ImageNet1K D ign are also input into the same 2D detector with only image-level labels (category) are provided.
Revised: It is noteworthy to mention that the images I from from ImageNet1K D ign are also input into the same 2D detector with only image-level labels (category) are provided.

Original: Following
Revised: Following

Original: Detic Zhou et al.
Revised: Detic Zhou et al.

Original: (2022), we choose the max-size proposal f maxsize and apply classification label c ign 2 D to supervised the classifier in the 2D detector.
Revised: (2022), we choose the max-size proposal f maxsize and apply classification label c ign 2 D to supervised the classifier in the 2D detector.

Original: The loss in the phase 1 training is given by
Revised: The loss in the phase 1 training is given by

Original: [Equation]
Revised: [Equation]

Original: where L 3 Dbox and L 2 Dbox follow Misra et al.
Revised: where L 3 Dbox and L 2 Dbox follow Misra et al.

Original: (2021) and Carion et al.
Revised: (2021) and Carion et al.

Original: L 3 Dcls and L 2 Dcls are crossentropy loss for classification.
Revised: L 3 Dcls and L 2 Dcls are crossentropy loss for classification.

Original: W 3 D and W 2 D denote the classifier in the 3D detector and the 2D detector, respectively.
Revised: W 3 D and W 2 D denote the classifier in the 3D detector and the 2D detector, respectively.

Original: In phase 2, we first generate pseudo-labels for unseen classes.
Revised: In phase 2, we first generate pseudo-labels for unseen classes.

Original: The pseudo-labels contain two parts: bounding box from the 3D detector, and class from the 2D detector.
Revised: The pseudo-labels contain two parts: bounding box from the 3D detector, and class from the 2D detector.

Original: Specifically, since we already exploit ImageNet1K to train the 2D detector in the phase 1, as similar to Detic Zhou et al.
Revised: Specifically, since we already exploit ImageNet1K to train the 2D detector in the phase 1, as similar to Detic Zhou et al.

Original: (2022), the 2D detector is able to classify unseen classes so that we can use its classification results as relatively accurate pseudo-labels.
Revised: (2022), the 2D detector is able to classify unseen classes so that we can use its classification results as relatively accurate pseudo-labels.

Original: To leverage this, we crop the image region of the projected 3D detection and use the 2D detector’s classifier to generate its class label.
Revised: To leverage this, we crop the image region of the projected 3D detection and use the 2D detector’s classifier to generate its class label.

Original: For the bounding box pseudo-labels, we take advantage of the generalizability of localization for the point-cloud detector, as mentioned in Fig. 1(b).
Revised: For the bounding box pseudo-labels, we take advantage of the generalizability of localization for the point-cloud detector, as mentioned in Fig. 1(b).

Original: Besides using pseudo-labels, we also design a de-biased cross-modal contrastive learning to better transfer the knowledge from 2D modality to point-cloud modality.
Revised: Besides using pseudo-labels, we also design a de-biased cross-modal contrastive learning to better transfer the knowledge from 2D modality to point-cloud modality.

Original: Note that there is significant synergy between this pseudo-label strategy and our proposed de-biased cross-modal contrastive learning.
Revised: Note that there is significant synergy between this pseudo-label strategy and our proposed de-biased cross-modal contrastive learning.

Original: The pseudo-label is beneficial for true positive sampling of crossmodal contrastive learning, and cross-modal contrastive learning transfer the knowledge from image to point-cloud gradually, in turn helping to generate better pseudo-labels.
Revised: The pseudo-label is beneficial for true positive sampling of crossmodal contrastive learning, and cross-modal contrastive learning transfer the knowledge from image to point-cloud gradually, in turn helping to generate better pseudo-labels.

Original: Thus the pseudo-labels are iteratively updated to be of higher quality.
Revised: Thus the pseudo-labels are iteratively updated to be of higher quality.

Original: Overall, the total loss in phase 2 is given by
Revised: Overall, the total loss in phase 2 is given by

Original: [Equation]
Revised: [Equation]

Original: where ¯b 3 D and ¯c 3 D come from either ground truth or pseudo-labels, and L DECC denotes the loss function of the de-biased cross-modal contrastive learning.
Revised: where ¯b 3 D and ¯c 3 D come from either ground truth or pseudo-labels, and L DECC denotes the loss function of the de-biased cross-modal contrastive learning.

Original: 3. D E - BIASED C ROSS - MODAL C ONTRASTIVE L EARNING
Revised: 3. D E - BIASED C ROSS - MODAL C ONTRASTIVE L EARNING

Original: Before we in-depth illustrate the proposed de-biased cross-modal contrastive learning, we first discuss the typical biased contrastive learning in the following.
Revised: Before we in-depth illustrate the proposed de-biased cross-modal contrastive learning, we first discuss the typical biased contrastive learning in the following.

Original: Cross-modal contrastive learning has been demonstrated as a powerful method used for knowledge transferring Jiang et al.
Revised: Cross-modal contrastive learning has been demonstrated as a powerful method used for knowledge transferring Jiang et al.

Original: ; Radford et al.
Revised: ; Radford et al.

Original: ; Minderer et al.
Revised: ; Minderer et al.

Original: and enabling zero-shot/open vocabulary classification and detection.
Revised: and enabling zero-shot/open vocabulary classification and detection.

Original: Using instance discrimination as the pretext task, typical contrastive learning Radford et al. (2021); Chen et al.
Revised: Using instance discrimination as the pretext task, typical contrastive learning Radford et al. (2021); Chen et al.

Original: ; He et al.
Revised: ; He et al.

Original: Khosla et al.
Revised: Khosla et al.

Original: (2020) pulls the semantically-close neighbors together and pushes away non-neighbors.
Revised: (2020) pulls the semantically-close neighbors together and pushes away non-neighbors.

Original: In previous works Radford et al.
Revised: In previous works Radford et al.

Original: ; Minderer et al.
Revised: ; Minderer et al.

Original: (2022), dissimilar (negative) points are typically taken to be randomly sampled datapoints, implicitly accepting that these points may actually express the same underlying concepts.
Revised: (2022), dissimilar (negative) points are typically taken to be randomly sampled datapoints, implicitly accepting that these points may actually express the same underlying concepts.

Original: For example, CLIP Radford et al.
Revised: For example, CLIP Radford et al.

Original: (2021) conducts contrastive learning on coupled text and image, while the text caption may describe a bunch of images that have similar semantics.
Revised: (2021) conducts contrastive learning on coupled text and image, while the text caption may describe a bunch of images that have similar semantics.

Original: This issue is studied as noisy matching Wu et al.
Revised: This issue is studied as noisy matching Wu et al.

Original: or biased contrastive learning Chuang et al.
Revised: or biased contrastive learning Chuang et al.

Original: In the image-point-cloud field, existing studies mainly focused on matching the features from the same 3D positions Hou et al.
Revised: In the image-point-cloud field, existing studies mainly focused on matching the features from the same 3D positions Hou et al.

Original: ; Liu et al.
Revised: ; Liu et al.

Original: (2021b)
Revised: (2021b)

Original: ; Park et al.
Revised: ; Park et al.

Original: This results in biases, as shown in left part of Fig.
Revised: This results in biases, as shown in left part of Fig.

Original: We can observe that both within and across modalities, some objects with the same classes are inaccurately assigned as negative samples.
Revised: We can observe that both within and across modalities, some objects with the same classes are inaccurately assigned as negative samples.

Original: In order to mitigate the biased issue in contrastive learning, we propose a de-biased cross-modal contrastive learning (DECC), as shown in right part of Fig.
Revised: In order to mitigate the biased issue in contrastive learning, we propose a de-biased cross-modal contrastive learning (DECC), as shown in right part of Fig.

Original: We take the ROI features f 3 D from the 3D detector and f 2 D from the 2D detector, and use the ground truth or pseudo-labels ¯c 3 D to assign the positive and negative samples.
Revised: We take the ROI features f 3 D from the 3D detector and f 2 D from the 2D detector, and use the ground truth or pseudo-labels ¯c 3 D to assign the positive and negative samples.

Original: For a mini-batch of f 3 D and f 2 D with total M features from two modalities, we further use a linear projection to transform these feature into hidden representations, which is denoted as h i , where i = 1 , 2 , 3 , ..., M .
Revised: For a mini-batch of f 3 D and f 2 D with total M features from two modalities, we further use a linear projection to transform these feature into hidden representations, which is denoted as h i , where i = 1 , 2 , 3 , ..., M .

Original: Then the loss is given by
Revised: Then the loss is given by

Original: [Equation]
Revised: [Equation]

Original: where m is the number of positive sample corresponding to h i ( m < = M ), τ 0 is the base temperature.
Revised: where m is the number of positive sample corresponding to h i ( m < = M ), τ 0 is the base temperature.

Original: It is noteworthy to mention that different from previous contrastive learning with constant temperature τ for each sample, we here adopt a distance-aware temperature τ ( dist ij ) =
Revised: It is noteworthy to mention that different from previous contrastive learning with constant temperature τ for each sample, we here adopt a distance-aware temperature τ ( dist ij ) =

Original: τ 0 × γ dist ij for positive sample, where γ are hyperparameters.
Revised: τ 0 × γ dist ij for positive sample, where γ are hyperparameters.

Original: In particular, we calculate the Euclidean distance of the two samples in 3D space.
Revised: In particular, we calculate the Euclidean distance of the two samples in 3D space.

Original: Note that if the one of the sample from ImageNet, we consider the distance between ImageNet and any of other sample as 1.
Revised: Note that if the one of the sample from ImageNet, we consider the distance between ImageNet and any of other sample as 1.

Original: Intuitively, the correlation between two close-by objects is greater than that of two distant object, thus we scale the temperature with distance-aware strategy to facilitate this correlation.
Revised: Intuitively, the correlation between two close-by objects is greater than that of two distant object, thus we scale the temperature with distance-aware strategy to facilitate this correlation.

Original: Specifically, If γ < 1 . 0 , then L DECC pays more attention to connect close-by objects, and if γ = 1 . 0 , L DECC degenerates to de-biased crossmodal contrastive learning with constant temperature, while if γ < 1 . 0 , L DECC focuses on bridge two distant objects.
Revised: Specifically, If γ < 1 . 0 , then L DECC pays more attention to connect close-by objects, and if γ = 1 . 0 , L DECC degenerates to de-biased crossmodal contrastive learning with constant temperature, while if γ < 1 . 0 , L DECC focuses on bridge two distant objects.

Original: In practices, we set γ > 1 . 0 is to enforce the alignment of close-by objects.
Revised: In practices, we set γ > 1 . 0 is to enforce the alignment of close-by objects.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we compare the proposed OS-3DETIC with popular baselines on two widely used 3D detection datasets, SUN RGB-D Song et al.
Revised: In this section, we compare the proposed OS-3DETIC with popular baselines on two widely used 3D detection datasets, SUN RGB-D Song et al.

Original: and ScanNetDai et al.
Revised: and ScanNetDai et al.

Original: Then we conduct sufficient analysis and ablation studies to explore why OS-3DETIC works.
Revised: Then we conduct sufficient analysis and ablation studies to explore why OS-3DETIC works.

Original: The details are illustrated in the below.
Revised: The details are illustrated in the below.

Original: 4. D ATASETS AND E VALUATION M ETRIC
Revised: 4. D ATASETS AND E VALUATION M ETRIC

Original: SUN RGB-D Song et al.
Revised: SUN RGB-D Song et al.

Original: and ScanNetV2 Dai et al.
Revised: and ScanNetV2 Dai et al.

Original: (2017) are two widely used 3D detection datasets.
Revised: (2017) are two widely used 3D detection datasets.

Original: We follow the data preprocessing procedure from VoteNet Qi et al.
Revised: We follow the data preprocessing procedure from VoteNet Qi et al.

Original: (2019b), except for the sampling of unseen classes.
Revised: (2019b), except for the sampling of unseen classes.

Original: We randomly select 10 unseen classes for both two datasets.
Revised: We randomly select 10 unseen classes for both two datasets.

Original: Specifically, toilet, bed, chair, bathtub, sofa, dresser, scanner, fridge, lamp and desk are unseen classes for
Revised: Specifically, toilet, bed, chair, bathtub, sofa, dresser, scanner, fridge, lamp and desk are unseen classes for

Original: SUN RGB-D, while table, night stand, cabinet, counter, garbage bin, bookshelf, pillow, microwave, sink and stool are seen classes.
Revised: SUN RGB-D, while table, night stand, cabinet, counter, garbage bin, bookshelf, pillow, microwave, sink and stool are seen classes.

Original: toilet, bed, chair, sofa, dresser, table, cabinet, bookshelf and pillow are unseen classes for ScanNet, and bathtub, fridge, desk, night stand, counter, door, curtain, box, lamp and bag are seen classes.
Revised: toilet, bed, chair, sofa, dresser, table, cabinet, bookshelf and pillow are unseen classes for ScanNet, and bathtub, fridge, desk, night stand, counter, door, curtain, box, lamp and bag are seen classes.

Original: The metrics we use in the experiments are Average Precision (AP), mean Average Precision (mAP) and Average Recall (AR) at IoU thresholds of 0.25, denotes as AP 25 , mAP 25 , AR 25 , respectively.
Revised: The metrics we use in the experiments are Average Precision (AP), mean Average Precision (mAP) and Average Recall (AR) at IoU thresholds of 0.25, denotes as AP 25 , mAP 25 , AR 25 , respectively.

Original: mAP takes both classification and localization in to consideration, the large the mAP means the better the detection, whereas AR mainly focuses on localization.
Revised: mAP takes both classification and localization in to consideration, the large the mAP means the better the detection, whereas AR mainly focuses on localization.

Original: Therefore, compared to AR, mAP is a better metric for evaluating detection result.
Revised: Therefore, compared to AR, mAP is a better metric for evaluating detection result.

Original: 4. E XPERIMENTAL S ETUP AND I MPLEMENTATION D ETAILS
Revised: 4. E XPERIMENTAL S ETUP AND I MPLEMENTATION D ETAILS

Original: Implementation The proposed OS-3DETIC is mainly based on 3DETR Misra et al.
Revised: Implementation The proposed OS-3DETIC is mainly based on 3DETR Misra et al.

Original: (2021), yet it can also generalize to other point-cloud detectors.
Revised: (2021), yet it can also generalize to other point-cloud detectors.

Original: During training, a minibatch of data consists of the point-cloud with its paired image and several images that from ImageNet.
Revised: During training, a minibatch of data consists of the point-cloud with its paired image and several images that from ImageNet.

Original: 3DETR as the 3D detector consumes the point-cloud, and DETR Carion et al.
Revised: 3DETR as the 3D detector consumes the point-cloud, and DETR Carion et al.

Original: (2020) as the 2D detector deals with images.
Revised: (2020) as the 2D detector deals with images.

Original: Note that only classification loss is used in the DETR when the input is images from
Revised: Note that only classification loss is used in the DETR when the input is images from

Original: ImageNet.
Revised: ImageNet.

Original: We select the max-size ROI features detected by DETR and apply the classfication loss on it, as following Detic Zhou et al.
Revised: We select the max-size ROI features detected by DETR and apply the classfication loss on it, as following Detic Zhou et al.

Original: τ 0 and γ are two hyperparameters in the distance-aware temperature strategy, in practice, τ 0 is set to 0.2, γ is set to 1.1, and the distance between image from
Revised: τ 0 and γ are two hyperparameters in the distance-aware temperature strategy, in practice, τ 0 is set to 0.2, γ is set to 1.1, and the distance between image from

Original: ImageNet and any other sample is 1.0.
Revised: ImageNet and any other sample is 1.0.

Original: For both 3DETR and DETR, a unified learning rate is set to 2 × 10 − 5 , with batch size of 4 for each GPU, and we train our model on 8 RTX 2080 TI gpus.
Revised: For both 3DETR and DETR, a unified learning rate is set to 2 × 10 − 5 , with batch size of 4 for each GPU, and we train our model on 8 RTX 2080 TI gpus.

Original: We train 200 epochs for the phase 1, and 200 epochs for the phase 2.
Revised: We train 200 epochs for the phase 1, and 200 epochs for the phase 2.

Original: Pseudo-Label Generation Pseudo-label consists of two elements: 3D bounding box and category.
Revised: Pseudo-Label Generation Pseudo-label consists of two elements: 3D bounding box and category.

Original: We first generate initial pseudo-labels after finishing phase 1, and update them iteratively.
Revised: We first generate initial pseudo-labels after finishing phase 1, and update them iteratively.

Original: As we discussed in Section 3.2, the 3D detector performs generalizable object localization, while the 2D detector is trained on the ImageNet dataset with sufficient knowledge.
Revised: As we discussed in Section 3.2, the 3D detector performs generalizable object localization, while the 2D detector is trained on the ImageNet dataset with sufficient knowledge.

Original: Therefore, the bounding box comes from the 3D detector and the class label comes from the 2D detector.
Revised: Therefore, the bounding box comes from the 3D detector and the class label comes from the 2D detector.

Original: Furthermore, we only keep proposals with high confidence and filter out the proposals that are duplicated with ground truth or no point in it.
Revised: Furthermore, we only keep proposals with high confidence and filter out the proposals that are duplicated with ground truth or no point in it.

Original: Finally, only top- k proposals are for
Revised: Finally, only top- k proposals are for

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 4. M AIN R ESULTS
Revised: 4. M AIN R ESULTS

Original: As there is no baseline directly solving the problem of both open-set 3D point-cloud localization and classification, we mainly compare OS-3DETIC with state-of-the-art 3D point-cloud detectors
Revised: As there is no baseline directly solving the problem of both open-set 3D point-cloud localization and classification, we mainly compare OS-3DETIC with state-of-the-art 3D point-cloud detectors

Original: Liu et al.
Revised: Liu et al.

Original: (2021d)
Revised: (2021d)

Original: ; Qi et al.
Revised: ; Qi et al.

Original: (2019a)
Revised: (2019a)

Original: ; Zhang et al.
Revised: ; Zhang et al.

Original: ; Misra et al.
Revised: ; Misra et al.

Original: (2021) and some well-known works Zhang et al.
Revised: (2021) and some well-known works Zhang et al.

Original: ; Xu et al.
Revised: ; Xu et al.

Original: (2021a)
Revised: (2021a)

Original: ; Zhou et al.
Revised: ; Zhou et al.

Original: (2022) that study either transferability in point-cloud or 2D open-set detection.
Revised: (2022) that study either transferability in point-cloud or 2D open-set detection.

Original: Specifically, the baselines we use include:
Revised: Specifically, the baselines we use include:

Original: • GroupFree3D Liu et al.
Revised: • GroupFree3D Liu et al.

Original: (2021d),
Revised: (2021d),

Original: VoteNet Qi et al. (2019a), H3DNet Zhang et al.
Revised: VoteNet Qi et al. (2019a), H3DNet Zhang et al.

Original: (2020), 3DETR Misra et al.
Revised: (2020), 3DETR Misra et al.

Original: (2021) are well-known and representative 3D point-cloud detectors that are chosen as our baselines.
Revised: (2021) are well-known and representative 3D point-cloud detectors that are chosen as our baselines.

Original: Specifically, these four baselines are trained on the seen classes while being tested on the unseen.
Revised: Specifically, these four baselines are trained on the seen classes while being tested on the unseen.

Original: • The second is PointCLIP Zhang et al.
Revised: • The second is PointCLIP Zhang et al.

Original: (2021) which bridges the point-cloud and text domain.
Revised: (2021) which bridges the point-cloud and text domain.

Original: We use it directly as a pre-trained open-set 3D classifier and replace the classifier of 3DETR with PointCLIP.
Revised: We use it directly as a pre-trained open-set 3D classifier and replace the classifier of 3DETR with PointCLIP.

Original: This baseline is denoted as OS-PointCLIP , which is similar to well-known 2D open-set detection works Bansal et al.
Revised: This baseline is denoted as OS-PointCLIP , which is similar to well-known 2D open-set detection works Bansal et al.

Original: ; Gu et al.
Revised: ; Gu et al.

Original: ; Zhou et al.
Revised: ; Zhou et al.

Original: (2022) that replace the classifier in the detector with the generalizable classifier.
Revised: (2022) that replace the classifier in the detector with the generalizable classifier.

Original: • Besides, Xu et al.
Revised: • Besides, Xu et al.

Original: Xu et al.
Revised: Xu et al.

Original: (2021a) transfer the image pre-trained transformer to the pointcloud by copying or inflating the weights.
Revised: (2021a) transfer the image pre-trained transformer to the pointcloud by copying or inflating the weights.

Original: Similarly, we copy the weights of the transformer and the classifier from pre-trained DETR (pre-trained on COCO Lin et al.
Revised: Similarly, we copy the weights of the transformer and the classifier from pre-trained DETR (pre-trained on COCO Lin et al.

Original: to 3DETR and finetune the set aggregation module and the 3D box head.
Revised: to 3DETR and finetune the set aggregation module and the 3D box head.

Original: We term this baseline as OS-Image2Point .
Revised: We term this baseline as OS-Image2Point .

Original: Moreover, Detic Zhou et al.
Revised: Moreover, Detic Zhou et al.

Original: (2022) leverages large-scale classification dataset (ImageNet) to broaden the 2D detector, here we directly extend the idea to 3D open-set detection.
Revised: (2022) leverages large-scale classification dataset (ImageNet) to broaden the 2D detector, here we directly extend the idea to 3D open-set detection.

Original: Specifically, we consider two manners, extend the classifier via ModelNet or ImageNet, and term them as Detic-ModelNet and Detic-ImageNet , respectively.
Revised: Specifically, we consider two manners, extend the classifier via ModelNet or ImageNet, and term them as Detic-ModelNet and Detic-ImageNet , respectively.

Original: The results are presented in Tables 1 and 2.
Revised: The results are presented in Tables 1 and 2.

Original: We can observe that OS-PointCLIP achieves mAPof 2.26% and 3.09%, outperforming the other baselines on both SUN RGB-D and ScanNet.
Revised: We can observe that OS-PointCLIP achieves mAPof 2.26% and 3.09%, outperforming the other baselines on both SUN RGB-D and ScanNet.

Original: Furthermore, Detic-ModelNet reaches mAP 25 of 1.52% and 1.69% on SUN RGB-D and ScanNet, respectively.
Revised: Furthermore, Detic-ModelNet reaches mAP 25 of 1.52% and 1.69% on SUN RGB-D and ScanNet, respectively.

Original: It can be observed that both OS-PointCLIP and Detic-ModelNet outperforms OSImage2Point and Detic-ImageNet .
Revised: It can be observed that both OS-PointCLIP and Detic-ModelNet outperforms OSImage2Point and Detic-ImageNet .

Original: Indeed, OS-PointCLIP and Detic-ModelNet enhances the classifier of the detector via introducing the pretraining on ModelNet, which is a 3D classification
Revised: Indeed, OS-PointCLIP and Detic-ModelNet enhances the classifier of the detector via introducing the pretraining on ModelNet, which is a 3D classification

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: dataset, while OS-Image2Point and Detic-ImageNet try to transfer knowledge from image (COCO and ImageNet) to point-cloud.
Revised: dataset, while OS-Image2Point and Detic-ImageNet try to transfer knowledge from image (COCO and ImageNet) to point-cloud.

Original: This contrast shows that the modality gap between 2D and 3D does indeed hinder knowledge transfer.
Revised: This contrast shows that the modality gap between 2D and 3D does indeed hinder knowledge transfer.

Original: The results also demonstrate that directly plugging and playing the Detic method on 3DETR with introducing ImageNet is infeasible to transferring the knowledge from 2D image to 3D. Nonetheless, our method achieves mAP 25 of 13.03% and 12.65% on SUN
Revised: The results also demonstrate that directly plugging and playing the Detic method on 3DETR with introducing ImageNet is infeasible to transferring the knowledge from 2D image to 3D. Nonetheless, our method achieves mAP 25 of 13.03% and 12.65% on SUN

Original: RGB-D and ScanNet, respectively, which proves the proposed OS-3DETIC can indeed make use of the image knowledge, to achieve open-set 3D detection.
Revised: RGB-D and ScanNet, respectively, which proves the proposed OS-3DETIC can indeed make use of the image knowledge, to achieve open-set 3D detection.

Original: 4. A NALYSIS AND A BLATION S TUDY
Revised: 4. A NALYSIS AND A BLATION S TUDY

Original: Ablation Study on Different Components We conduct an ablation study on SUN RGB-D, the results of unseen classes are reported in Table 3.
Revised: Ablation Study on Different Components We conduct an ablation study on SUN RGB-D, the results of unseen classes are reported in Table 3.

Original: Our baseline is 3DETR which is trained only on seen categories. ”
Revised: Our baseline is 3DETR which is trained only on seen categories. ”

Original: Pseudo-Label” denotes to train with pseudo-label. ”
Revised: Pseudo-Label” denotes to train with pseudo-label. ”

Original: Augmentation Based CL” denotes we follow SimCLR
Revised: Augmentation Based CL” denotes we follow SimCLR

Original: Chen et al.
Revised: Chen et al.

Original: ; He et al.
Revised: ; He et al.

Original: (2020) that utilize data augmentation strategy in contrastive learning, which however, is a biased contrastive learning setting as well.
Revised: (2020) that utilize data augmentation strategy in contrastive learning, which however, is a biased contrastive learning setting as well.

Original: Positionbased contrastive learning takes only the same instance from the paired image/point-cloud as positive.
Revised: Positionbased contrastive learning takes only the same instance from the paired image/point-cloud as positive.

Original: Class based contrasitve learning takes positive so long as two samples belong to the same category. ”
Revised: Class based contrasitve learning takes positive so long as two samples belong to the same category. ”

Original: Distance-Aware Temperature” refer to the strategy that is proposed in Section 3.3.
Revised: Distance-Aware Temperature” refer to the strategy that is proposed in Section 3.3.

Original: We can observe that, first, pseudo-label brings the largest improvement, actually, pseudo-label implicitly transfers the knowledge contained in ImageNet to a totally different modality, the point-cloud modality, via providing useful class labels.
Revised: We can observe that, first, pseudo-label brings the largest improvement, actually, pseudo-label implicitly transfers the knowledge contained in ImageNet to a totally different modality, the point-cloud modality, via providing useful class labels.

Original: This not only validates our hypothesis that the 3D detector functions as a general region proposal network, but it also demonstrates the effectiveness of introducing large-scale image-level supervision for classification.
Revised: This not only validates our hypothesis that the 3D detector functions as a general region proposal network, but it also demonstrates the effectiveness of introducing large-scale image-level supervision for classification.

Original: Second, class-based contrastive learning and distance-aware temperature further significantly improve the performance, while traditional augmentation-based contrastive learning and position-based contrastive learning hurt the performance, which indicates the weakness of biased contrastive learning, and demonstrates the proposed de-biased cross-modal contrastive learning indeed benefits the point-cloud detector to learn general representations.
Revised: Second, class-based contrastive learning and distance-aware temperature further significantly improve the performance, while traditional augmentation-based contrastive learning and position-based contrastive learning hurt the performance, which indicates the weakness of biased contrastive learning, and demonstrates the proposed de-biased cross-modal contrastive learning indeed benefits the point-cloud detector to learn general representations.

Original: Analysis
Revised: Analysis

Original: of
Revised: of

Original: Performance on Different Training Data
Revised: Performance on Different Training Data

Original: Ratio Fig. 4(a) illustrate relation between mAP 25 , AR 25 and the different training data ratio.
Revised: Ratio Fig. 4(a) illustrate relation between mAP 25 , AR 25 and the different training data ratio.

Original: the left y axis denotes mAP 25 , the right y axis denotes AR 25 , and the x axis represents the data ratio that used during training.
Revised: the left y axis denotes mAP 25 , the right y axis denotes AR 25 , and the x axis represents the data ratio that used during training.

Original: We draw both mAP 25 , AR 25 results of the point-cloud and the paired image branch.
Revised: We draw both mAP 25 , AR 25 results of the point-cloud and the paired image branch.

Original: On one hand, compared with mAP 25 , AR 25 converges with relatively less training data, which exists in both the image and the point-cloud branches.
Revised: On one hand, compared with mAP 25 , AR 25 converges with relatively less training data, which exists in both the image and the point-cloud branches.

Original: AR 25 partially reflects localization ability, which means we can train a generalizable bounding box detector with few annotations.
Revised: AR 25 partially reflects localization ability, which means we can train a generalizable bounding box detector with few annotations.

Original: On the other hand, compared with ARof image branch, the AR 25 of point-cloud branch converges with less training data, which means the localization ability of 3D detector are better than 2D detector, verifying the strategy of using the point-cloud detector to generate bounding box pseudo-labels.
Revised: On the other hand, compared with ARof image branch, the AR 25 of point-cloud branch converges with less training data, which means the localization ability of 3D detector are better than 2D detector, verifying the strategy of using the point-cloud detector to generate bounding box pseudo-labels.

Original: Moreover, the converged mAPof the image branch is better than the point-cloud branch.
Revised: Moreover, the converged mAPof the image branch is better than the point-cloud branch.

Original: This may be because texture and detail account a lot for classification, which however is the weakness of the point-cloud detector.
Revised: This may be because texture and detail account a lot for classification, which however is the weakness of the point-cloud detector.

Original: Analysis on Pseudo-Label Effects Fig. 4(b) illustrates the relation between mAP 25 , AR 25 and pseudo-label iteration.
Revised: Analysis on Pseudo-Label Effects Fig. 4(b) illustrates the relation between mAP 25 , AR 25 and pseudo-label iteration.

Original: During the second phase, we iteratively update the pseudo-label everyepochs.
Revised: During the second phase, we iteratively update the pseudo-label everyepochs.

Original: The results show that the more the iteration, the better the pseudo-labels should be, thus leading to the better the performance.
Revised: The results show that the more the iteration, the better the pseudo-labels should be, thus leading to the better the performance.

Original: 4. Q UALITATIVE R ESULTS
Revised: 4. Q UALITATIVE R ESULTS

Original: Fig.
Revised: Fig.

Original: 5 provides the qualitative results.
Revised: 5 provides the qualitative results.

Original: We can observe that Baseline is able to predict the relatively accurate locations compared with ground truth, but the size, center, and direction are incorrect, especially the unseen class bounding boxes (chair in Scene 1, sofa in scene 2, bed in scene 3 and sofa in scene 4).
Revised: We can observe that Baseline is able to predict the relatively accurate locations compared with ground truth, but the size, center, and direction are incorrect, especially the unseen class bounding boxes (chair in Scene 1, sofa in scene 2, bed in scene 3 and sofa in scene 4).

Original: Compare baseline with OS-3DETIC, OS-3DETIC performs much better on the unseen classes.
Revised: Compare baseline with OS-3DETIC, OS-3DETIC performs much better on the unseen classes.

Original: [Figure]
Revised: [Figure]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we study a new problem of open-set 3D detection.
Revised: In this paper, we study a new problem of open-set 3D detection.

Original: The proposed method OS-3DETIC introduces ImageNet1K to help open-set point-cloud detection.
Revised: The proposed method OS-3DETIC introduces ImageNet1K to help open-set point-cloud detection.

Original: OS-3DETIC consists of two components: 1) we take advantage of two modalities — the image modality for classification and the point-cloud modality for localization, to generate pseudo-labels for unseen classes, and 2) de-biased cross-modal contrastive learning to transfer the knowledge from images to point-clouds.
Revised: OS-3DETIC consists of two components: 1) we take advantage of two modalities — the image modality for classification and the point-cloud modality for localization, to generate pseudo-labels for unseen classes, and 2) de-biased cross-modal contrastive learning to transfer the knowledge from images to point-clouds.

Original: Extensive experiments show that we improve a wide range of baselines by a large margin, demonstrating the effectiveness of the proposed method.
Revised: Extensive experiments show that we improve a wide range of baselines by a large margin, demonstrating the effectiveness of the proposed method.

Original: We also provide explanations of why it works via ablation studies and analyzing the representations.
Revised: We also provide explanations of why it works via ablation studies and analyzing the representations.

Original: We do hope our work could inspire the research community to further explore this field.
Revised: We do hope our work could inspire the research community to further explore this field.