Original: I MAGE 2P OINT : 3D P OINT -C LOUD U NDERSTANDING WITH 2D I MAGE P RETRAINED M ODELS
Revised: I MAGE 2P OINT : 3D P OINT -C LOUD U NDERSTANDING WITH 2D I MAGE P RETRAINED M ODELS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: 3D point-clouds and 2D images are different visual representations of the physical world.
Revised: 3D point-clouds and 2D images are different visual representations of the physical world.

Original: While human vision can understand both representations, computer vision models designed for 2D image and 3D point-cloud understanding are quite different.
Revised: While human vision can understand both representations, computer vision models designed for 2D image and 3D point-cloud understanding are quite different.

Original: Our paper explores the potential for transferring between these two representations by empirically investigating the feasibility of the transfer, the beneﬁts of the transfer, and shedding light on why the transfer works.
Revised: Our paper explores the potential of transferring 2D model architectures and weights to understand 3D point-clouds, by empirically investigating the feasibility of the transfer, the beneﬁts of the transfer, and shedding light on why the transfer works.

Original: We discovered that we can indeed use the same architecture and pretrained weights of a neural net model to understand both images and point-clouds.
Revised: We discover that we can indeed use the same architecture and pretrained weights of a neural net model to understand both images and point-clouds.

Original: Speciﬁcally, wecan transfer the pretrained image model to a point-cloud model by inﬂating 2D convolutional ﬁlters to 3D  andthen f inetuning the  i mage- p retrained models (FIP).
Revised: Speciﬁcally, we transfer the image-pretrained model to a point-cloud model by inﬂating 2D convolutional ﬁlters to 3D convolutional ﬁlters and f inetuning the inﬂated i magep retrained models (FIP).

Original: We discover that, surprisingly, models with minimal ﬁnetuning efforts — only on input, output, and optionally batch normalization layers, can achieve competitive performance on 3D point-cloud classiﬁcation, beating a wide range of point-cloud models that adopt task-speciﬁc architectures and use a variety of tricks.
Revised: We ﬁnd that models with minimal ﬁnetuning efforts — only on input, output, and optionally, batch normalization layers — can achieve competitive performance on 3D point-cloud classiﬁcation, beating a wide range of point-cloud models that adopt task-speciﬁc architectures and use a variety of tricks.

Original: When ﬁnetuning the whole model, the performance  further improves signiﬁcantly.
Revised: When ﬁnetuning the whole model, the performance gets further improved.

Original: Meanwhile,we also ﬁnd that FIP improves data efﬁciency, achieving up to 10.0 points top-1 accuracy gain on few-shot classiﬁcation.
Revised: Meanwhile, FIP improves data efﬁciency, reaching up to 10.0 points top-1 accuracy gain on few-shot classiﬁcation.

Original: It also speeds up training of point-cloud models by up to 11.1x to reach a target accuracy.
Revised: It also speeds up training of point-cloud models by up to 11.1x for a target accuracy.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Point-cloud is an important visual representation for 3D computer vision.
Revised: Point-cloud is an important visual representation for 3D computer vision.

Original: It is widely used in applications such as autonomous driving (Behley et al., 2019;
Revised: It is widely used in applications such as autonomous driving (Behley et al., 2019;

Original: Caesar et al., 2020;
Revised: Caesar et al., 2020;

Original: Yue et al., 2018), robotics (Armeni et al., 2017; Pomerleau et al., 2015; Xu et al., 2021), augmented and virtual reality (Sketchup, 2021; Wu et al., 2015; Shi et al., 2015), etc.
Revised: Yue et al., 2018), robotics (Armeni et al., 2017; Pomerleau et al., 2015; Xu et al., 2021), augmented and virtual reality (Sketchup, 2021; Wu et al., 2015; Shi et al., 2015), etc.

Original: A point-cloud represents visual information in a highly different way from a 2D image.
Revised: A point-cloud represents visual information in a highly different way from a 2D image.

Original: A point-cloud consists of a set of unordered points lying on object surface, with each point encoding its spatial x, y, z coordinates and potentially other features.
Revised: A point-cloud consists of a set of unordered points lying on the object’s surface, with each point encoding its spatial x, y, z coordinates and potentially other features.

Original: In contrast, a 2D image organizes visual features as a dense 2D pixel array.
Revised: In contrast, a 2D image organizes visual features as a dense 2D pixel array.

Original: Due to the representation differences, 2D image and 3D point-cloud understanding are treated as separate problems.
Revised: Due to the representation differences, 2D image and 3D point-cloud understanding are treated as separate problems.

Original: Image models and point-cloud models are designed to have different architectures and are trained on different types of data.
Revised: Image models and point-cloud models are designed to have different architectures and are trained on different types of data.

Original: Few research efforts have tried to directly transfer models from images to point-clouds or vice versa.
Revised: Few research efforts have tried to directly transfer models from images to point-clouds or vice versa.

Original: Intuitively, both 3D point-clouds and 2D images are visual representations of the physical world.
Revised: Intuitively, both 3D point-clouds and 2D images are visual representations of the physical world.

Original: Their low-level representations are drastically different, but they can represent the same underlying visual concept.
Revised: Their low-level representations are drastically different, but they can represent the same underlying visual concept.

Original: Furthermore, human vision has no problem understanding both representations.
Revised: Furthermore, human vision has no problem understanding both representations.

Original: However, can computer vision modelsbe trained on one modality understand the other?
Revised: However, can computer vision models trained on one modality understand the other?

Original: Somewhat surprisingly, the answer to the question above is: Yes, 2D image models trained on image datasets can be transferred to understand 3D point-clouds with minimal efforts.
Revised: Somewhat remarkably, the answer to the question above is: Yes. 2D image models trained on image datasets can be transferred to understand 3D point-clouds with minimal efforts.

Original: As illustrated in
Revised: As illustrated in

Original: Figure 1, we transfer a 2D ConvNet to a 3D ConvNet whose input is a 3D voxel representation converted from a point-cloud.
Revised: Figure 1, we transfer a 2D ConvNet to a 3D ConvNet whose input is a 3D voxel representation converted from a point-cloud.

Original: Based on a pretrained 2D ConvNet, we inﬂate its 2D convolutional ﬁlters to 3D by copying the ﬁlter weights along a third dimension.
Revised: Based on a pretrained 2D ConvNet, we inﬂate its 2D convolutional ﬁlters to 3D by copying the ﬁlter weights along a third dimension.

Original: We add linear input and output layers to the network; and on a target point-cloud dataset, we only ﬁnetune the input/output layers and optionally the normalization layers, while keeping the original model weights untouched.
Revised: We add linear input and output layers to the network; and on a target point-cloud dataset, we only ﬁnetune the input/output layers, and optionally, the normalization layers, while keeping the pretrained model weights untouched.

Original: We term such partially-ﬁnetuned-image-pretrained models as FIP-IO (ﬁnetuning only input and output layer) or FIP-IO+BN (ﬁnetuning input, output, and BN layers).
Revised: We term such partially-ﬁnetuned-image-pretrained models as FIP-IO (ﬁnetuning only input and output layers) or FIP-IO+BN (ﬁnetuning input, output, and BN layers).

Original: FIP-IO+BN can achieve competitive
Revised: FIP-IO+BN can achieve competitive

Original: Pretrained 2D ConvNet
Revised: Pretrained 2D ConvNet

Original: [Figure]
Revised: [Figure]

Original: performance up to 90.8% top-1 accuracy on the ModelNet 3D Wharehouse dataset, on top a ResNet50, outperformingmany previous point-cloud models that adopt task-speciﬁc model architectures and tricks.
Revised: performance up to 90.8% top-1 accuracy on the ModelNet 3D Warehouse dataset, on top a ResNet50, outperforming previous point-cloud models that adopt task-speciﬁc model architectures and tricks.

Original: Most point-cloud models except projection-based models are only trained from scratch.
Revised: Most point-cloud models except projection-based models are only trained from scratch.

Original: Based on oursurprising discovery, we further investigate fully-ﬁnetuned-image-pretrained models (termed as FIP-ALL ).
Revised: Based on our discovery, we further investigate fully-ﬁnetuned-image-pretrained models (termed as FIP-ALL ).

Original: We observe that FIP-ALL brings signiﬁcant improvement on top of ResNet series.
Revised: We observe that FIP-ALL brings signiﬁcant improvement on top of ResNet series.

Original: Besides applying FIP-ALL to voxel-based method, we also ﬁnd  it generalizes to other popular methods, such as point-based method (PointNet++ (Qi et al., 2017)) and projection-based method (SimpleView (Goyal et al., 2021a)), as well as current popular vision transformers (ViT (Dosovitskiy et al., 2020)).
Revised: Besides applying FIP-ALL to voxel-based method, we also ﬁnd that it generalizes to other popular methods, such as point-based method (PointNet++ (Qi et al., 2017)) and projection-based method (SimpleView (Goyal et al., 2021a)), as well as current popular vision transformers (ViT (Dosovitskiy et al., 2020)).

Original: Speciﬁcally, FIP-ALL largely outperforms the training-from-scratch by 0.88, 0.50, 3.50, 4.18 points top-1 accuracy on top of PointNet++, SimpleView, ViT-B-16, and ViT-L-16, respectively.
Revised: Speciﬁcally, FIP-ALL largely outperforms the training-from-scratch by 0.88, 0.50, 3.50, 4. points top-1 accuracy on top of PointNet++, SimpleView, ViT-B-16, and ViT-L-16, respectively.

Original: In addition to the performance gain, FIP-ALL exhibits superior data efﬁciency with up to 10.0 points improvement in few-shot classiﬁcation on the ModelNet 3D Wharehouse dataset.
Revised: In addition to the performance gain, FIP-ALL exhibits superior data efﬁciency with up to 10.0 points improvement in few-shot classiﬁcation on the ModelNet 3D Warehouse dataset.

Original: We also ﬁnd that comparing with training-from-scratch, FIP-ALL  dramatically speeds up the training by using 11. times fewer epochs to reach a target validation accuracy.
Revised: Comparing with training-from-scratch, FIP-ALL also dramatically speeds up the training by using 11.1 times fewer epochs to reach a target validation accuracy.

Original: In order to understand why the image pretraining can be utilized and beneﬁt point-cloud understanding, we conduct experimentto shed light on this by studying the network dissection (Bau et al., 2017), text-shape representation transferring (Geirhos et al., 2018), and distribution distance, hoping these can inspire the research community to explore further.
Revised: In order to understand why the image pretraining can be utilized to beneﬁt point-cloud understanding, we conduct experiment studying the network dissection (Bau et al., 2017), text-shape representation transferring (Geirhos et al., 2018), and distribution distance.

Original:  R ELATED W ORK 2. P OINT - CLOUD PROCESSING MODEL
Revised:  R ELATED W ORK 2. P OINT - CLOUD PROCESSING MODEL

Original: 3D convolution-based method is one of the mainstreams in point-cloud processing approaches which efﬁciently process point-clouds based on voxelization.
Revised: 3D convolution-based method is one of the mainstream point-cloud processing approaches which efﬁciently process point-clouds based on voxelization.

Original: Inparticular, in this approach, voxelization is used to rasterize point-clouds into regular grids (called voxels), thus conventional 3D convolutions can be applied.
Revised: In this approach, voxelization is used to rasterize point-clouds into regular grids (called voxels) so that conventional 3D convolutions can be applied.

Original: Sparse convolution is proposed to apply on the non-empty voxels (Liu et al., 2015; Choy et al., 2019; Tang et al., 2020;
Revised: Sparse convolution is proposed to apply on the non-empty voxels (Liu et al., 2015; Choy et al., 2019; Tang et al., 2020;

Original: Zhou et al., 2020; Yan et al., 2018; Feng et al., 2021), largely improving the efﬁciency of 3D convolutions.
Revised: Zhou et al., 2020; Yan et al., 2018; Feng et al., 2021), largely improving the efﬁciency of 3D convolutions.

Original: Projection-based method attempts to project a 3D point-cloud to a 2D plane and uses 2D convolution to extract features (Wang et al., 2018; Wu et al., 2018; 2019; Xu et al., 2020; Su et al., 2015; Lawin et al., 2017; Boulch et al., 2017).
Revised: Projection-based method attempts to project a 3D point-cloud to a 2D plane and uses 2D convolution to extract features (Wang et al., 2018; Wu et al., 2018; 2019; Xu et al., 2020; Su et al., 2015; Lawin et al., 2017; Boulch et al., 2017).

Original: Speciﬁcally,the bird-eye-view projection (Yang et al., 2018; Lang et al., 2019) andthe spherical projection (Wu et al., 2018; 2019; Xu et al., 2020; Milioto et al., 2019) make great progress in outdoor point-cloud tasks.
Revised: Speciﬁcally, bird-eye-view projection (Yang et al., 2018; Lang et al., 2019) and spherical projection (Wu et al., 2018; 2019; Xu et al., 2020; Milioto et al., 2019) make great progress in outdoor point-cloud tasks.

Original: Point-based method directly processes the point-cloud data.
Revised: Point-based method directly processes the point-cloud data.

Original: The most classic methods, PointNet (Qi et al., 2016) and PointNet++ (Qi et al., 2017), consume points by customized feature aggregation.
Revised: The most classic methods, PointNet (Qi et al., 2016) and PointNet++ (Qi et al., 2017), consume points by customized feature aggregation.

Original: Many works further develop advanced local-feature aggregation operators that mimic the convolution to structure data (Xu et al., 2021;
Revised: Many works further develop advanced local-feature aggregation operators that mimic the convolution to structure data (Xu et al., 2021;

Original: Li et al., 2018b; Hua et al., 2018; Liu et al., 2019; 2020; Wang et al., 2017; Li et al., 2018a; Komarichev et al., 2019).
Revised: Li et al., 2018b; Hua et al., 2018; Liu et al., 2019; 2020; Wang et al., 2017; Li et al., 2018a; Komarichev et al., 2019).

Original: 2. P RETRAINING IN 2D AND 3D VISION
Revised: 2. P RETRAINING IN 2D AND 3D VISION

Original: Pretraining in 2D vision has shown effectiveness under supervised (Dosovitskiy et al., 2020;
Revised: Pretraining in 2D vision has shown effectiveness under supervised (Dosovitskiy et al., 2020;

Original: Girshick et al., 2014), self-supervised (Jing & Tian, 2020; Goyal et al., 2021b), and unsupervised contrastive approach (He et al., 2020; Bachman et al., 2019; Chen et al., 2020a; Caron et al., 2020;
Revised: Girshick et al., 2014), self-supervised (Jing & Tian, 2020; Goyal et al., 2021b), and unsupervised contrastive approach (He et al., 2020; Bachman et al., 2019; Chen et al., 2020a; Caron et al., 2020;

Original: Chen et al., 2020c; Hjelm et al., 2018).
Revised: Chen et al., 2020c; Hjelm et al., 2018).

Original: After pretraining on a large amount of data, a 2D model requires much fewer computational resources and data for ﬁnetuning to reach competitive performance on downstream tasks (Kataoka et al., 2020; Caron et al., 2019; Chen et al., 2020b; Henaff, 2020).
Revised: After pretraining on a large amount of data, a 2D model requires much fewer computational resources and data for ﬁnetuning to reach competitive performance on downstream tasks (Kataoka et al., 2020; Caron et al., 2019; Chen et al., 2020b; Henaff, 2020).

Original: Pretraining in 3D vision has been studied similarly as pretraining in 2D vision: both self-supervised and contrastive pretraining (Xie et al., 2020) show promising results.
Revised: Pretraining in 3D vision has been studied similarly as pretraining in 2D vision: both self-supervised and contrastive pretraining (Xie et al., 2020) show promising results.

Original: Due to the lack of large, annotated point-cloud datasets, pretraining in 3D vision is motivated for data efﬁciency (Xu &
Revised: Due to the lack of large, annotated point-cloud datasets, pretraining in 3D vision is motivated to achieve data efﬁciency (Xu &

Original: Lee, 2020).
Revised: Lee, 2020).

Original: Recent works (Hou et al., 2020; Zhang et al., 2021) consider pretraining methods, for example, Contrastive Scene Contents which making use of both point-level correspondences and spatial contexts, with data efﬁciency in mind.
Revised: Recent works (Hou et al., 2020; Zhang et al., 2021b) consider pretraining methods with data efﬁciency in mind, for example, Contrastive Scene Contents which makes use of both point-level correspondences and spatial contexts.

Original: 2. C ROSS - MODAL TRANSFER LEARNING
Revised: 2. C ROSS - MODAL TRANSFER LEARNING

Original: Cross-modal transfer learning attempts to take advantage of data from different modalities (Dai &
Revised: Cross-modal transfer learning attempts to take advantage of data from different modalities (Dai &

Original: Nießner, 2018; Liu et al., 2021b).
Revised: Nießner, 2018; Liu et al., 2021b).

Original: For example, Liu et al.
Revised: For example, Liu et al.

Original: (2021a)
Revised: (2021a)

Original: proposed pixel-to-point knowledge transfer (PPKT) from 2D to 3D which uses aligned RGB and RGB-D images during pretraining.
Revised: proposed pixel-to-point knowledge transfer (PPKT) from 2D to 3D which uses aligned RGB and RGB-D images during pretraining.

Original: Our work does not rely on joint image-point-cloud pretraining.
Revised: Our work does not rely on joint image-point-cloud pretraining.

Original: Instead, we directly transfer an image-pretrained model to point-cloud with the simplest pretraining-ﬁnetuning scheme.
Revised: Instead, we directly transfer an image-pretrained model to point-cloud with the simplest pretraining-ﬁnetuning scheme.

Original: Some of the previous works for video and medical images (Carreira & Zisserman, 2017; Shan et al., 2018) have adopted the method of simply extending a pretrained 2D convolutional ﬁlter along time or depth direction for transferring to 3D models.
Revised: Some of the previous works for video and medical images (Carreira & Zisserman, 2017; Shan et al., 2018) have adopted the method of simply extending a pretrained 2D convolutional ﬁlter along time or depth direction for transferring to 3D models.

Original: Between language and image modality, transfer learning with minimal ﬁnetuning also shows a competitive performance (Lu et al., 2021).
Revised: Between language and image modality, transfer learning with minimal ﬁnetuning also shows a competitive performance (Lu et al., 2021).

Original:  C ONVERTING A 2D C ONV N ET TO A 3D C ONV N ET
Revised:  C ONVERTING A 2D C ONV N ET TO A 3D C ONV N ET

Original: In this paper, we primarily focus on the 3D sparse-convolution based method to process point-clouds because it is ﬂexible to all point-cloud tasks.
Revised: In this paper, we primarily focus on the 3D sparse-convolution based method to process point-clouds because it can be extended to all point-cloud tasks.

Original: As discussed in 2.1, we consider a set of points where each point is represented by its 3D coordinates andoptionally additional features such as intensity and RGB.
Revised: As discussed in 2.1, we consider a set of points where each point is represented by its 3D coordinates and additional features such as intensity and RGB.

Original: We then voxelize/quantize these points into voxels according to their 3D space coordinates, following Choy et al.
Revised: We then voxelize/quantize these points into voxels according to their 3D space coordinates, following Choy et al.

Original: A voxel’s feature is inherited from the point that lies in the voxel.
Revised: A voxel’s feature is inherited from the point that lies in the voxel.

Original: If there are multiple points in a voxel, then we average all points’ feature and assign the mean to the voxel.
Revised: If there are multiple points in a voxel, then we average all points’ feature and assign the mean to the voxel.

Original: If there is no point in the voxel, then we simply set the voxel’s feature to 0.
Revised: If there is no point in the voxel, then we simply set the voxel’s feature to 0.

Original: When using sparse convolution, we skip the computation on empty voxels.
Revised: When using sparse convolution, we skip the computation on empty voxels.

Original: Given a pretrained 2D ConvNet, we convert it to a 3D ConvNet that takes 3D voxels as input.
Revised: Given a pretrained 2D ConvNet, we convert it to a 3D ConvNet that takes 3D voxels as input.

Original: The key element of this procedure is to convert 2D convolution ﬁlters to 3D, i.e. constructing 3D ﬁlters with the weights directly inherited from 2D ﬁlters.
Revised: The key element of this procedure is to convert 2D convolution ﬁlters to 3D, i.e. constructing 3D ﬁlters with the weights directly inherited from 2D ﬁlters.

Original: A 2D convolutional ﬁlter can be represented as a 4D tensor of shape [ M, N, K, K ] , representing output dimension, input dimension, and two spatial kernel sizes, respectively.
Revised: A 2D convolutional ﬁlter can be represented with a 4D tensor of shape [ M, N, K, K ] , representing output dimension, input dimension, and two spatial kernel sizes, respectively.

Original: A 3D convolutional ﬁlter has an extra dimension, and its shape is [ M, N, K, K, K ] .
Revised: A 3D convolutional ﬁlter has an extra dimension, and its shape is [ M, N, K, K, K ] .

Original: To better illustrate, we ignore the output and input dimensions and only consider a spatial slice of the 2D ﬁlter with shape
Revised: To better illustrate, we ignore the output and input dimensions and only consider a spatial slice of the 2D ﬁlter with shape

Original: [ K, K ] .
Revised: [ K, K ] .

Original: The simplest way to convert this 2D ﬁlter to 3D is to copy the 2D ﬁlter and repeat it by K times along a third dimension.
Revised: The simplest way to convert this 2D ﬁlter to 3D is to copy the 2D ﬁlter and repeat it by K times along a third dimension.

Original: This operation is the same as the inﬂation technique used by (Carreira & Zisserman, 2017) to initialize a video model with a pretrained 2D ConvNet.
Revised: This operation is the same as the inﬂation technique used by (Carreira & Zisserman, 2017) to initialize a video model with a pretrained 2D ConvNet.

Original: Besides convolution, other operations such as downsampling, BN, nonlinear activation can be easily migrated to 3D. Our 3D model inherits the architecture of the original 2D ConvNet, but we also add a linear layer as the input layer and an output layer depending on the target task.
Revised: Besides convolution, other operations such as downsampling, BN, nonlinear activation can be easily migrated to 3D. Our 3D model inherits the architecture of the original 2D ConvNet, but we also add a linear layer as the input layer and an output layer depending on the target task.

Original: For classiﬁcation, we use a global average pooling layer followed by one fully connected layer to get the ﬁnal prediction.
Revised: For classiﬁcation, we use a global average pooling layer followed by one fully connected layer to get the ﬁnal prediction.

Original: For semantic segmentation, the output layer is a U-Net style decoder (Ronneberger et al., 2015).
Revised: For semantic segmentation, the output layer is a U-Net style decoder (Ronneberger et al., 2015).

Original: The architecture of the input/output layers is described in more detail in supplementary A.5.
Revised: The architecture of the input/output layers is described in more detail in Appendix A.9. [Figure]

Original:  E MPIRICAL E VALUATION
Revised:  E MPIRICAL E VALUATION

Original: To explore the image to point-cloud transfer, we study three settings: 1) partially-ﬁnetuned-imagepretrained model, only ﬁnetuning input and output layers (FIP-IO), 2) ﬁnetuning input, output, and batch normalization layers (FIP-IO+BN), and 3) ﬁnetuning the whole pretrained network (FIP-ALL).
Revised: To explore the image to point-cloud transfer, we study three settings: 1) partially-ﬁnetuned-imagepretrained model, only ﬁnetuning input and output layers (FIP-IO), 2) ﬁnetuning input, output, and batch normalization layers (FIP-IO+BN), and 3) ﬁnetuning the whole pretrained network (FIP-ALL).

Original: Under the three settings, we extensively explore the feasibility of transferring the image-pretrained model for point-cloud understanding and the beneﬁts of this.
Revised: Under the three settings, we extensively explore the feasibility of transferring the image-pretrained model for point-cloud understanding and its beneﬁts.

Original: The entire empirical evaluation is organized as four questions: 1) Can we transfer pretrained-image models to recognize point-clouds?
Revised: The entire empirical evaluation is organized as four questions: 1) Can we transfer pretrained-image models to recognize point-clouds?

Original: (Section 4.1) 2)
Revised: (Section 4.1) 2)

Original: Can image-pretraining beneﬁt the performance of point-cloud recognition?
Revised: Can image-pretraining beneﬁt the performance of point-cloud recognition?

Original: (Section 4.2) 3) Can image-pretrained model improve the data efﬁciency on point-cloud recognition?
Revised: (Section 4.2) 3) Can image-pretrained model improve the data efﬁciency on point-cloud recognition?

Original: (Section 4.3) 4) Can image-pretrained model accelerate training point-cloud models?
Revised: (Section 4.3) 4) Can image-pretrained model accelerate training point-cloud models?

Original: (Section 4.4)
Revised: (Section 4.4)

Original: Datasets.
Revised: Datasets.

Original: We benchmark the transferred models on ModelNet 3D Wharehouse classiﬁcation (Wu et al., 2015), S3DIS indoor segmentation (Armeni et al., 2017), and SemanticKITTI outdoor segmentation (Behley et al., 2019) tasks.
Revised: We benchmark the transferred models on ModelNet 3D Warehouse classiﬁcation (Wu et al., 2015), S3DIS indoor segmentation (Armeni et al., 2017), and SemanticKITTI outdoor segmentation (Behley et al., 2019) tasks.

Original: ModelNet 3D Wharehouse is a CAD model classiﬁcation dataset that consists of point-clouds with 40 categories.
Revised: ModelNet 3D Warehouse is a CAD model classiﬁcation dataset that consists of point-clouds with 40 categories.

Original: CAD models in this benchmark come from 3D Warehouse (Sketchup, 2021).
Revised: CAD models in this benchmark come from 3D Warehouse (Sketchup, 2021).

Original: In this benchmduiark, we only utilize x, y, z coordinates as features.
Revised: In this benchmark, we only utilize x, y, z coordinates as features.

Original: S3DIS is a dataset collected from real-world indoor scenes and includes 3D scans of Matterport Scanners from 6 areas.
Revised: S3DIS is a dataset collected from real-world indoor scenes and includes 3D scans of Matterport Scanners from 6 areas.

Original: It provides point-wise annotations for indoor objects like chair, table, and bookshelf, etc.
Revised: It provides point-wise annotations for indoor objects like chair, table, and bookshelf, etc.

Original: SemanticKITTI dataset from KITTI Vision Odometry (Geiger et al., 2012) is a driving scene dataset.
Revised: SemanticKITTI dataset from KITTI Vision Odometry (Geiger et al., 2012) is a driving scene dataset.

Original: It provides dense point-wise annotations for the complete 360 degrees ﬁeld-of-view of the deployed automotive lidar, which is currently one of the most challenging datasets.
Revised: It provides dense point-wise annotations for the complete 360 degrees ﬁeld-of-view of the deployed automotive lidar, which is currently one of the most challenging datasets.

Original: ResNet (He et al., 2016a) series is used mostly throughout our experiments.
Revised: ResNet (He et al., 2016a) series is used mostly throughout our experiments.

Original: Depending on the experiments, ResNets are pretrained on Tiny-ImageNet, ImageNet-1K, ImageNet-21K (Deng et al., 2009), and Fractal database (FractalDB)
Revised: Depending on the experiments, ResNets are pretrained on Tiny-ImageNet, ImageNet-1K, ImageNet-21K (Deng et al., 2009), and Fractal database (FractalDB)

Original: (Kataoka et al., 2020).
Revised: (Kataoka et al., 2020).

Original: Our pretrained models are directly downloaded from various sources, with detailed links provided in Section A.1.
Revised: Our pretrained models are directly downloaded from various sources, with detailed links provided in Section A.1.

Original: To study the beneﬁts of using pretrained image models, we also utilize PointNet++ (Qi et al., 2017), ViT (Dosovitskiy et al., 2020), and SimpleView (Goyal et al., 2021a) as our baselines.
Revised: To study the beneﬁts of using pretrained image models, we also utilize PointNet++ (Qi et al., 2017), ViT (Dosovitskiy et al., 2020), and SimpleView (Goyal et al., 2021a) as our baselines.

Original: 4. C AN WE TRANSFER PRETRAINED - IMAGE MODELS TO RECOGNIZE POINT - CLOUDS ?
Revised: 4. C AN WE TRANSFER PRETRAINED - IMAGE MODELS TO RECOGNIZE POINT - CLOUDS ?

Original: To evaluate the feasibility of transferring pretrained 2D image models to 3D point-cloud tasks, we conduct experiments on top of the ResNet series since there are abundant open-source pretrained
Revised: To evaluate the feasibility of transferring pretrained 2D image models to 3D point-cloud tasks, we conduct experiments on top of the ResNet series since there are abundant open-source pretrained

Original: ResNet available.
Revised: ResNet available.

Original: In particular, we convert 2D ConvNets into 3D ConvNets using the procedure described in Section 3.
Revised: In particular, we convert 2D ConvNets into 3D ConvNets using the procedure described in Section 3.

Original: We hypothesize that, if a pretrained 2D image model is capable of understanding point-clouds directly, we can see an nontrivial performance by only ﬁnetuning input and output layers of the transferred model.
Revised: We hypothesize that, if a pretrained 2D image model is capable of understanding point-clouds directly, we can see a non-trivial performance by only ﬁnetuning input and output layers of the transferred model.

Original: Further, as we gradually relax the frozen parameters—ﬁnetuning
Revised: Further, as we gradually relax the frozen parameters, ﬁnetuning

Original: BN parameters as well, the transferred model can achieve better performance, even surpassing training-from-scratch performance. 
Revised: BN parameters as well, the transferred model can achieve better performance, even surpassing training-from-scratch performance. [Table]

Original:  We conduct two groups of experiments with FIP-IO and FIP-IO+BN, with the results shown in Figure 2.
Revised: [Table] [Table] We conduct two groups of experiments with FIP-IO and FIP-IO+BN, with the results shown in Figure 2.

Original: The ﬁrst is toinﬂate ResNet50 pretrained from different image datasets, including Tiny-ImageNet, ImageNet1K, ImageNet21K, FractalDB1K and FractalDB10K, and evaluateon the ModelNet 3D
Revised: The ﬁrst is to evaluate the performance as the trainable parameters gradually increase. As shown in Figure 2 (a), training no more than 0.3 % (345.5x fewer) of the whole parameters, the image pretraining even beats the training-from-scratch (100 % trainable parameters). Speciﬁcally,

Original: Wharehouse. We surprisingly discovered that, even if we only ﬁnetune the input and output layers while keeping the image-pretrained weights freezed, the FIP-IO pretrained from ImageNet1K, FractalDB1K and FractalDB10K achieve competitive performance.
Revised: ResNet152 FIP-IO+BN with ImageNet1K pretraining improves training-from-scratch by 0.16 points, and ResNet50 FIP-IO+BN with ImageNet21K pretraining improves 0.48 points. Meanwhile, FIP-IO also reaches a non-trivial performance.

Original: Speciﬁcally, ResNet50 FIP-IO performance with
Revised: ResNet50 FIP-IO pretrained on ImageNet1K achieves 81.

Original: ImageNet1K pretraining outperforms 3D ShapeNet (Wu et al., 2015) and DeepPano (Shi et al., 2015), which were the state-of-the-arts in 2015, by 4.2 and 3.6 points respectively in top-1 accuracy on ModelNet 3D Wharehouse. More importantly, with ImageNet21K pretrained model, ResNet FIP-IO+BN surpass training-from-scratch by 0.48 points, even beating a variety of well-known methods including PointNet (Qi et al., 2016),
Revised: % top-1 accuracy, only 9.12 points worse than training-from-scratch with approximately 0.1 % trainable parameters.

Original: MVCNN (Su et al., 2015), DGCNN (Wang et al., 2019),

Original: KDNet (Klokov & Lempitsky, 2017), etc . [Figure] [Figure] [Figure]
Revised: Furthermore, to investigate the effect of different datasets, as shown in the right ﬁgure of Figure 2, we inﬂate ResNet50 pretrained from different image datasets, including Tiny-ImageNet, ImageNet1K,

Original: [Figure] D [Figure] h a p e N e t s D ee p P a n o P
Revised: ImageNet21K, FractalDB1K, and FractalDB10K, then evaluate on the ModelNet 3D Warehouse.

Original: o

Original: i n t N e t P [Figure] n [Figure]
Revised: We discover that, even if we only ﬁnetune the input and output layers while keeping the imagepretrained weights frozen, the FIP-IO pretrained from ImageNet1K, FractalDB1K, and FractalDB10K achieves competitive performance.

Original: N e

Original: [Figure] D [Figure] NN
Revised: Speciﬁcally, ResNet50 FIP-IO with ImageNet1K pretraining outperforms 3D ShapeNet (Wu et al., 2015) and DeepPano (Shi et al., 2015), which were the state-ofthe-arts in 2015, by 4.2 and 3.6 points respectively in top-1 accuracy on ModelNet 3D Warehouse.

Original: M

Original: [Figure] NN K [Figure]
Revised: More importantly, with ImageNet21K pretrained model, ResNet50 FIP-IO+BN surpasses trainingfrom-scratch by 0.48 points, even beating a variety of well-known methods including PointNet (Qi et al., 2016),

Original: [Figure]

Original: [Figure] [Figure] [Figure] [Table]
Revised: MVCNN (Su et al., 2015), DGCNN (Wang et al., 2019), KDNet (Klokov & Lempitsky, 2017), etc .

Original: [Table] [Table] The second group of experiments are based on different ResNets, as shown in the right ﬁgure of Figure 2.
Revised: Notably, we ﬁnd out the answer to "Can we transfer pretrained-image models to recognize pointclouds?"

Original: All the ResNet models are pretrained on ImageNet1K. We observe that FIP-IO+BN on top of different ResNet models is highly competitive to training-from-scratch, and FIP-IO+BN on top of ResNet152 is even better than training-from-scratch by 0.16 points top-1 accuracy. Therefore, we surprisingly found out the answer of "Can we transfer pretrained-image models to recognize point-clouds?"
Revised: : Yes.

Original: : Yes, the pretrained 2D image models can be directly used for recognizing point-cloud.
Revised: The pretrained 2D image models can be directly used for recognizing point-clouds.

Original: It is also noteworthy that the pretraining dataset is not restricted to natural but also synthetic images like those in FractalDB1K/10K.
Revised: It is also noteworthy that the pretraining dataset is not restricted to natural but also synthetic images like those in FractalDB1K/10K.

Original: 4. C AN IMAGE - PRETRAINING BENEFIT POINT - CLOUD RECOGNITION ?
Revised: 4. C AN IMAGE - PRETRAINING BENEFIT POINT - CLOUD RECOGNITION ?

Original: From the previous subsection, we ﬁnd surprising that image-pretrained model can be directly used for point-cloud understanding.
Revised: From the previous subsection, we ﬁnd unexpectedly that image-pretrained model can be directly used for point-cloud understanding.

Original: In this subsection, we investigate whether image-pretrained modelis helpful to improve the performance on point-cloud tasks.
Revised: In this subsection, we investigate whether image-pretrained model

Original: We use different baselines, including voxelization-based method (simply ResNet), point-based method (PointNet++ (Qi et al., 2017)), projection-based method (SimpleView (Goyal et al., 2021a)), and current popular transformer-based

Original: [Table]
Revised: [Table]

Original:  method (ViT-B-16 and ViT-L-16 (Dosovitskiy et al., 2020)). We fully ﬁnetune them on three pointcloud datasets: classiﬁcation on ModelNet 3D Wharehouse, indoor scene segmentation on S3DIS, and outdoor scene segmentation on SemanticKITTI, as shown in Table 1 and Table 2.
Revised: is helpful to improve the performance on point-cloud tasks. We use different baselines, including voxelization-based method (simply ResNet), point-based method (PointNet++ (Qi et al., 2017)), projection-based method (SimpleView (Goyal et al., 2021a)), and current popular transformer-based method (ViT-B-16 and ViT-L-16 (Dosovitskiy et al., 2020)), and fully ﬁnetune them on three pointcloud datasets: classiﬁcation on ModelNet 3D Warehouse, indoor scene segmentation on S3DIS, and outdoor scene segmentation on SemanticKITTI, as shown in Table 1 and Table 2.

Original: For PointNet++, we use ImageNet1K to pretrain: we break each image into pixels and regard it as a point-cloud.
Revised: For PointNet++, we use ImageNet1K to pretrain: we break each image into pixels and regard it as a point-cloud.

Original: For ViT, we directly use the open-source pretrained model and ﬁnetune it on ModelNet 3D Wharehouse.
Revised: For ViT, we directly use the open-source pretrained model and ﬁnetune it on ModelNet 3D Warehouse.

Original: All the implementation details are illustrated in supplementary A. 1.
Revised: All the implementation details are illustrated in Appendix A.1.

Original: Table 1 presents performance on ModelNet 3D Wharehouse dataset.
Revised: Table 1 presents performance on ModelNet 3D Warehouse dataset.

Original: We observe that FIP-ALLsteadily and signiﬁcantly improves all the baselines.
Revised: We observe that FIP-ALL improves all baselines steadily and signiﬁcantly.

Original: With pretraining, deeper models improve more.
Revised: Besides, pretraining brings more improvements to deeper models.

Original: For example, ResNet18 canbe only  improved by 0.13% top-1 accuracy, but pretraining on ImageNet1K leads to 0.81 points top-1 accuracy improvement on top of ResNet152.
Revised: For example, ResNet18 can only be improved by 0.13% top-1 accuracy, but pretraining on ImageNet1K leads to 0.81 points top-1 accuracy improvement on top of ResNet152.

Original: Moreover, larger pretrained datasets also bring larger performance gain.
Revised: Moreover, larger pretrained datasets also lead to better performance.

Original: Speciﬁcally, ResNet50 FIP-ALL from ImageNet21K can reach 91.05% top-1 acc, with 0.73 points improvement over training-from-scratch.
Revised: Speciﬁcally, ResNet50 FIP-ALL from ImageNet21K can reach 91.05% top-1 acc, with 0.73 points improvement over training-fromscratch.

Original: Such FIP-ALL signiﬁcantly outperforms a series of well-known methods such as (Qi et al., 2016; 2017; Klokov & Lempitsky, 2017; Wang et al., 2019;
Revised: Such FIP-ALL signiﬁcantly outperforms a series of well-known methods such as (Qi et al., 2016; 2017; Klokov & Lempitsky, 2017; Wang et al., 2019;

Original: Su et al., 2015; Li et al., 2018a).
Revised: Su et al., 2015; Li et al., 2018a).

Original: We also explore FIP-ALL on different architectures, as shown in the second group of Table 1.
Revised: We also explore FIP-ALL on different architectures, as shown in the second group of Table 1.

Original: In particular, FIP-ALL on top of PointNet++, ViT-B-16, ViT-L-16 and SimpleView with image dataset pretraining improve the training-from-scratch by 0.88, 3.50, 4.18, 0.50 points, respectively.
Revised: In particular, FIP-ALL on top of PointNet++, ViT-B-16, ViT-L-16 and SimpleView with image dataset pretraining improve the training-from-scratch by 0.88, 3.50, 4.18, 0.50 points, respectively.

Original: Especially for the current superior baseline in image recognition, ViT-B-16 and ViT-L-16, the improved performance is quite signiﬁcant, revealing the huge potential of using image-pretrained models for point cloud recognition.
Revised: Especially for the current superior baseline in image recognition, ViT-B-16 and ViT-L-16, the improved performance is quite signiﬁcant, revealing the huge potential of using image-pretrained models for point cloud recognition.

Original: For the challenging indoor and outdoor scene segmentation, using ImageNet1K pretrained models (FIP-ALL on ImageNet1K) alsoconsistently improve the training-from-scratch, as shown in Table 2.
Revised: For the challenging indoor and outdoor scene segmentation, using ImageNet1K pretrained models (FIP-ALL on ImageNet1K) also improve the training-from-scratch consistently, as shown in Table 2.

Original: PointNet++ (resp.
Revised: PointNet++ (resp.

Original: ResNet18) pretrained on ImageNet1K outperforms the training-from-scratch by 2.56 points (resp.
Revised: ResNet18) pretrained on ImageNet1K outperforms the training-from-scratch by 2.56 points (resp.

Original: 1.53 points) mIoU on S3DIS dataset.
Revised: 1.53 points) mIoU on S3DIS dataset.

Original: For SemanticKITTI, we utilize the commonly used projection-based method with 2D ConvNet HRNet.
Revised: For SemanticKITTI, we utilize the commonly used projection-based method with 2D ConvNet HRNet.

Original: With ImageNet1K pretraining, we observe 3.41 points mIoU improvement, a large margin in such a difﬁcult task.
Revised: With ImageNet1K pretraining, we observe 3.41 points mIoU improvement, a large margin in such a challenging task.

Original: Since HRNetV2-W48 has rich pretrained models, we ﬁnetune Cityscapes pretrained HRNetV2-W48 and observe this enhances more (5.25% mIoU improvement over training from scratch).
Revised: Since HRNetV2-W48 has rich pretrained models, we ﬁnetune Cityscapes pretrained HRNetV2-W48 and observe this enhances more (5.25% mIoU improvement over training from scratch).

Original: Even for the ResNet18 with a high from-scratch performance of 64.75% mIoU, the ImageNet1K pretraining can also bring 0.82 points mIoU improvement.
Revised: Even for the ResNet18 with a high from-scratch performance of 64.75% mIoU, the ImageNet1K pretraining can also bring 0.82 points mIoU improvement.

Revised: Finally, we compare the performance gain with the well-known point-cloud self-supervised method

Revised: PointContrast (Xie et al., 2020), as presented in Table 3.

Revised: We use the same model architecture and ﬁnetuning recipe, and the only difference is the pretraining weights.

Revised: We can observe that imagepretraining on ImageNet1K signiﬁcantly boosts the training-from-scratch by 0.93 points, surpassing the PointContrast by at least 0.64 points.

Original: Therefore, the answer to "Can image-pretraining beneﬁt point-cloud recognition" is Yes.
Revised: Therefore, the answer to "Can image-pretraining beneﬁt point-cloud recognition" is: Yes.

Original: Imagepretraining can indeed improve point-cloud recognition, which can generalize to a wide range of backbones as well as beneﬁt more challenging tasks.
Revised: Imagepretraining can indeed improve point-cloud recognition, which can generalize to a wide range of backbones and beneﬁt more challenging tasks.

Original: 4. C AN IMAGE - PRETRAINED MODEL IMPROVE THE DATA EFFICIENCY ON POINT - CLOUD RECOGNITION ?
Revised: 4. C AN IMAGE - PRETRAINED MODELS IMPROVE THE DATA EFFICIENCY ON POINT - CLOUD RECOGNITION ?

Original: Data efﬁciency is extremely important in point-cloud understanding due to the huge labor of collecting and annotating point-cloud data.
Revised: Data efﬁciency is extremely important in point-cloud understanding due to the huge labor of collecting and annotating point-cloud data.

Original: In this subsection, we investigate whether the image-pretrained  model can help to improve the data efﬁciency by conducting few-shot setting experiments, including 1-shot, 5-shot, and 10-shot.
Revised: In this subsection, we investigate whether the image-pretrained [Table] [Figure] model can help to improve the data efﬁciency by conducting few-shot setting experiments, including 1-shot, 5-shot, and 10-shot.

Original: We conduct 3 trials for each setting and report the results as mean ± std.
Revised: We conduct 3 trials for each setting and report the results as mean ± std.

Original: In detail, for each class (ModelNet 3D Wharehouse involves 40 classes), we randomly choose a few point-clouds as training data, and still evaluate on the whole test set.
Revised: In detail, for each class (ModelNet 3D Warehouse involves 40 classes), we randomly choose a few point-clouds as training data, and still evaluate on the whole test set.

Original: We compare the results between training-from-scratch and FIP-ALL pretrained on the ImageNet1K dataset.
Revised: We compare the results between training-from-scratch and FIP-ALL pretrained on the ImageNet1K dataset.

Original: The experimental results are shown in Table 3.
Revised: The experimental results are shown in Table 4.

Original: We observe that FIP-ALL dramatically surpasses training-from-scratch on the low data regime (1-shot): pretraining on ImageNet1K brings 10.0, 6.0, and 9.9 points top-1 accuracy[Figure] improvement for ResNet18, ResNet50, and ResNet152, respectively.
Revised: We observe that FIP-ALL dramatically surpasses training-from-scratch on the low data regime (1-shot): pretraining on ImageNet1K brings 10.0, 6.0, and 9.9 points top-1 accuracy improvement for ResNet18, ResNet50, and ResNet152, respectively.

Original: For 5-shot and 10-shot settings, using ImageNet1K pretraining can still consistently improve the performance.
Revised: For 5-shot and 10-shot settings, using ImageNet1K pretraining can still consistently improve the performance.

Original: However, we also observe that as the amount of training data increases, the performance gain becomes saturated.
Revised: However, we also observe that as the amount of training data increases, the performance gain becomes saturated.

Original: Therefore, our answer to "Can image-pretrained model improve the data efﬁciency on point-cloud recognition?"
Revised: Therefore, our answer to "Can image-pretrained model improve the data efﬁciency on point-cloud recognition?"

Original: is: Yes.
Revised: is: Yes.

Original: Image-pretrained model can improve the data efﬁciency on point-cloud recognition, especially onthe low data regime.
Revised: Image-pretrained model can improve the data efﬁciency on point-cloud recognition, especially on low data regime.

Original: When the training data increases, it can still improve the performance, but the gain becomes marginal.
Revised: When the training data increases, it can still improve the performance, but the gain becomes marginal.

Original: 4. C AN IMAGE - PRETRAINED MODEL ACCELERATE POINT - CLOUD TRAINING ?
Revised: 4. C AN IMAGE - PRETRAINED MODELS ACCELERATE POINT - CLOUD TRAINING ?

Original: We also investigate whether image-pretrained model can help point-cloud task train faster.
Revised: We also investigate whether image-pretrained model can help point-cloud task train faster.

Original: The results are shown in Figure 3.
Revised: The results are shown in Figure 3.

Original: We ﬁnd that surprisingly, after training only one epoch on ModelNet 3D Wharehouse dataset, 
Revised: We discover that, after training only one epoch on ModelNet 3D Warehouse dataset, FIP-ALL on

Original: FIP-ALL on ImageNet1K achieves very impressive performance yet the performance of trainingfrom-scratch is still at a low level.
Revised: ImageNet1K achieves very impressive performance, yet the performance of training-from-scratch is still at a low level.

Original: For example, after the ﬁrst epoch, ResNet50 (resp.
Revised: For example, after the ﬁrst epoch, ResNet50 (resp.

Original: ResNet152) with training from scratch can only achieve 28.48% (resp. 13.94%) top-1 accuracy while ResNet (resp.
Revised: ResNet152) with training from scratch can only achieve 28.48% (resp. 13.94%) top-1 accuracy while ResNet50 (resp.

Original: ResNet152) with ImageNet1K pretraining reaches 80.11% (resp.
Revised: ResNet152) with ImageNet1K pretraining reaches 80.11% (resp.

Original: 79.34%) top-1 accuracy.
Revised: 79.34%) top-1 accuracy.

Original: Moreover, to reach 90% top-1 accuracy, a non-trivial performance, FIP-ALL signiﬁcantly accelerates the training by 2.14x (28 vs. 60 epoch), 11.1x (11 vs. 
Revised: Moreover, to reach 90% top-1 accuracy, a non-trivial performance, FIP-ALL signiﬁcantly accelerates the training by 2.14x (28 vs. 60 epoch), 11.1x (11 vs. 122 epoch), 2.95x (19 vs. 56 epoch) over training-from-scratch, on top of ResNet18, ResNet50, and ResNet152, respectively.

Original: Therefore, our answer to "Can image-pretrained model accelerate point-cloud training?"
Revised: Therefore, our answer to "Can image-pretrained model accelerate point-cloud training?"

Original: is still: Yes.
Revised: is still: Yes.

Original: The image-pretrained model can signiﬁcantly accelerate the training speed of point-cloud tasks. 
Revised: The image-pretrained model can signiﬁcantly accelerate the training speed of point-cloud tasks. [Table] B r o d e n

Revised: M o d e l n e t

Revised: [Figure]

Original: D ISCUSSION
Revised: o [Figure] e l n e t [Figure]

Original:  In this section, we attempt to shed light on why transferring image-pretrained models for point-cloud understanding works.
Revised: n [Figure]  D ISCUSSION In this section, we attempt to shed light on why transferring image-pretrained models for point-cloud understanding works.

Original: Inspired by recent related works (Geirhos et al., 2018; Brendel & Bethge, 2019;
Revised: Inspired by recent related works (Geirhos et al., 2018; Brendel & Bethge, 2019;

Original: He et al., 2015; 2016b; Bau et al., 2017), we explore this from the aspects of the network dissection, texture-shape representation transferring, the distance between feature distributions.
Revised: He et al., 2015; 2016b; Bau et al., 2017), we explore this from the aspects of the network dissection, texture-shape representation transferring, and feature distribution distance.

Original: 5. W HAT DOES THE PRETRAINED - IMAGE MODEL TRANSFER TO POINT - CLOUD MODEL ?
Revised: 5. W HAT DO IMAGE - PRETRAINED MODELS TRANSFER TO POINT - CLOUD MODEL ?

Original: Does the image-pretrained model transfer the visual concepts?
Revised: Does the image-pretrained model transfer the visual concepts?

Original: Inspired by the network dissection for 2D Broden dataset (Bau et al., 2017), we also attempt to explore what the transferred units are looking at in the ModelNet 3D Wharehouse dataset.
Revised: Inspired by the network dissection for 2D Broden dataset (Bau et al., 2017), we also attempt to explore what the transferred units are focus on in the ModelNet 3D Warehouse dataset.

Original: We present the visualization of FIP-IO+BN pretrained on ImageNet1K.
Revised: We present the visualization of FIP-IO+BN pretrained on ImageNet1K.

Original: The visualization is to show the most activated cases when the whole dataset passes through each unit of the last model stage, as displayed in Figure 4.More visualization [Table] [Figure]
Revised: The visualization shows the most activated cases when the whole dataset passes through each unit of the last model stage, as displayed in Figure 4.

Original: r o d e n

Original: M o d e l n e t

Original: [Figure]

Original: l

Original: [Figure] t [Figure] can be found in supplementary 5.
Revised: More visualization can be found in Appendix 7.

Original: From the visualization alone, we do not get obvious cues of what visual concepts are transferred between the two modalities.
Revised: From the visualization alone, we do not get obvious cues of what visual concepts are transferred between the two modalities.

Original: For example, unit 161 strongly activating to computer screens in the Broden, yet it is looking at cars and shelves in the ModelNet 3D Wharehouse.
Revised: For example, unit 161 strongly activating to computer screens in the Broden, yet it attends to cars and shelves in the ModelNet 3D Warehouse.

Original: However, weunexpectedly ﬁnd that the pretrained units are prone to cluster similar objects.
Revised: Surprisingly, we ﬁnd that the pretrained units are prone to cluster similar objects.

Original: In fact, such clustering ability is an important cue of performing well on classiﬁcation tasks (Hartigan & Wong, 1979; Caron et al., 2021).
Revised: In fact, such clustering ability is an important cue of performing well on classiﬁcation tasks (Hartigan & Wong, 1979; Caron et al., 2021).

Original: Does pretrained-image model transfer shape or texture representation?
Revised: Do pretrained-image models transfer shape or texture representation?

Original: Recent work (Geirhos et al., 2018) proposed that models learn texture and shape from ImageNet.
Revised: Recent work (Geirhos et al., 2018) proposed that models learn texture and shape from ImageNet.

Original: We follow this direction tofuther explore what pretrained-image mode transfers.
Revised: We follow this direction to explore what pretrained-image mode transfers.

Original: For the experiment, we take two imagepretrained models with either moreof the shape or texture representation and compare the FIP-IO performance.
Revised: For the experiment, we take two image-pretrained models with either more shape or texture representation and compare their FIP-IO performance.

Original: To force the model to acquire more shape representation, (Geirhos et al., 2018) stylizes the images in
Revised: In order to force the pretrained models to acquire more shape representation, Geirhos et al.

Original:  ImageNet into artwork style, such that the models trained on that are confused by variant textures, hence having stronger shape representation.
Revised: (2018) stylizes the images in ImageNet into artwork style, such that the models trained on those images are confused by variant textures, hence having stronger shape representation.

Original: We directly take the pretrained ResNet50 on stylized ImageNet as the stronger shape representation model.
Revised: We directly take the pretrained ResNet50 on stylized ImageNet as the stronger shape representation model.

Original: To get a stronger texture representation model, we are inspired by BagNet (Brendel & Bethge, 2019).
Revised: To get a stronger texture representation model, we are inspired by BagNet (Brendel & Bethge, 2019).

Original: By controlling the receptive ﬁeld, BagNet breaks the shape in a image and focuses more on the texture information.
Revised: By controlling the receptive ﬁeld, BagNet breaks the shape in an image and focuses more on the texture information.

Original: Our experimental result is shown in Table 4.
Revised: Our experimental result is shown in Table 5.

Original: BagNet17 means the size of attended patches is 17 × 17 , and BagNet33 means the size of attended patches is 33 × 33 .
Revised: BagNet17 means the size of attended patches is 17 × 17 , and BagNet33 means the size of attended patches is 33 × 33 .

Original: Note that after inﬂating the BagNet17 and BagNet33, bothof the architectures are totally as same asthe inﬂated ResNet50.
Revised: Note that after inﬂating the BagNet17 and BagNet33, both architectures are the same as inﬂated ResNet50.

Original: Besides, both BagNet17/33 and ResNet pretrained on stylized-ImageNet1K perform worse than the original ResNet50 on ImageNetclassiﬁcation (Geirhos et al., 2018; Brendel & Bethge, 2019).
Revised: Besides, both BagNet17/33 and ResNet pretrained on stylized-ImageNet1K perform worse than the original ResNet50 on ImageNet (Geirhos et al., 2018; Brendel & Bethge, 2019).

Original: We can observe that the ResNet50 FIP-IO on Stylized-ImageNet (with stronger shape representation) outperforms the baseline ResNet50 FIP-IO on ImageNet1K over 2.32 points top-1 accuracy, while both inﬂated BagNets performs dramatically worse than the baseline.
Revised: We can observe that the ResNet50 FIP-IO on Stylized-ImageNet (with stronger shape representation) outperforms the baseline ResNet50 FIP-IO on ImageNet1K over 2.32 points top-1 accuracy, while both inﬂated BagNets perform dramatically worse than the baseline.

Original: This shows that the shape representations are better transferred from image to point-cloud modality.
Revised: This shows that the shape representations are transferred.

Original: [Table]
Revised: To make clear what parts of the image are critical for the transfer, we further conduct experiments using different image processing methods for pretraining.

Original: [Table]
Revised: The results are reported in Appendix A.7.

Original:  W HY DOES FINETUNING BATCH NORMALIZATION HELP THE TRANSFERRING ?
Revised: 5. W HY DOES FINETUNING BATCH NORMALIZATION HELP THE TRANSFERRING ?

Original: Our experiment in Figure 2 shows that ﬁnetuning BN, in addition to the input and output layer, can greatly improve the transfer performance compared with only ﬁnetuning input/output.
Revised: Our experiment in Figure 2 shows that ﬁnetuning BN, in addition to the input and output layer, can greatly improve the transfer performance.

Original: It is interesting why such a small part of the network, in terms of parameter size and FLOPs, can have a big impact.
Revised: It is interesting why such a small part of the network, in terms of parameter size and FLOPs, can have a big impact.

Original: We hypothesize that batch normalization layers shift and scale the feature distribution. To measure the effect, we leverage the ﬁrst-wasserstein distance  (Rubner et al., 2000), a way to calculate the distance between  distributions.
Revised: We calculate the ﬁrst-wasserstein distance (FWD) (Rubner et al., 2000), a measurement of Gaussian distribution distance, between the 2D image and 3D point-cloud feature distributions.

Original: For estimating the  feature distribution of the pretrained-image model, we pass the whole ImageNet1K dataset into the pretrained-image models, collect the pre-activation features after each convolution layer, then sample 15,000 data points from the element-wise distribution of collected features (we assume the pre-activation features presentGaussian distribution).
Revised: For estimating the 2D feature distribution, we pass the whole ImageNet1K dataset into the pretrained-image models, collect the pre-activation features after each convolution layer, then sample 15,000 data points from the element-wise distribution of collected features (we assume the pre-activation features present

Original: For the
Revised: Gaussian distribution).

Original: ModelNet 3D Wharehouse dataset, we use FIP-IO, FIP-IO+BN, and FIP-ALL to conduct the same operation on the ModelNet 3D Wharehouse dataset and also collect 15,000 data points.
Revised: For 3D feature distribution, we use FIP-IO, FIP-IO+BN, and FIP-ALL to conduct the same operation on the ModelNet 3D Warehouse dataset and also collect 15,000 data points.

Original: We then calculate the FWD between the element-wise feature distribution of image-pretrained model’s and each of the FIP-IO, FIP-IO+BN, FIP-ALL model’s.
Revised: We then calculate the FWD between the element-wise feature distribution of image-pretrained model and each of the FIP-IO, FIP-IO+BN, FIP-ALL model.

Original: The example results are shown in Table.
Revised: We observe that the layer-wise average FWD between FIP-IO (point-cloud features) and image model (image features) is 2 .

Original: We observe that, the FWD between FIP-IO (point-cloud features) and image model (image features) is very large, yet after ﬁnetuning batch normalization layers, the distance is dramatically reduced. 
Revised: 1 × 10 2 , yet after ﬁnetuning batch normalization layers, the distance is dramatically reduced. Particularly, average FWD between FIP-IO+BN and image model is 0 . 27 , and FWD between FIP-ALL and image model is only 0 . 093 . Complete FWD for each layer is shown in

Original:  This suggests that batch normalization plays a critical role of transferring the point-cloud representation to be closer to the image representation.
Revised: Appendix Table 11. This suggests that batch normalization plays a critical role in transforming the point-cloud representation to be closer to the image representation.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we use ﬁnetuned-image-pretrained models (FIP) to explore the feasibility of transferringthe image-pretrained model for point-cloud understanding, and the beneﬁts of using image-pretrained models on point-cloud tasks.
Revised: In this work, we use ﬁnetuned-image-pretrained models (FIP) to explore the feasibility of transferring image-pretrained models for point-cloud understanding and the beneﬁts of using image-pretrained models on point-cloud tasks.

Original: We surprisingly discover that, with simply inﬂating a 2D pretrained
Revised: We surprisingly discover that, with simply inﬂating a 2D pretrained

Original: ConvNet and minimal ﬁnetuning — input, output, and optionally batch normalization layer (FIP-IO or FIP-IO+BN), the image pretrained models can achieve very competitive performance on 3D point-cloud classiﬁcation,even beating a wide range of point-cloud models that adopt a variety of tricks.
Revised: ConvNet and minimal ﬁnetuning — input, output, and optionally, batch normalization layer (FIP-IO or FIP-IO+BN), FIP can achieve very competitive performance on 3D point-cloud classiﬁcation, beating a wide range of point-cloud models that adopt a variety of tricks.

Original: Moreover, we ﬁnd that when ﬁnetuning all the parameters of the pretrained models (FIP-ALL), the performance can be signiﬁcantly improved on point-cloud classiﬁcation, indoor and outdoor scene segmentation.
Revised: Moreover, we ﬁnd that when ﬁnetuning all the parameters of the pretrained models (FIP-ALL), the performance can be signiﬁcantly improved on point-cloud classiﬁcation, indoor and outdoor scene segmentation.

Original: Full ﬁnetuning generalizes to most of the popular point-cloud methods.
Revised: Fully ﬁnetuned models generalizes to most of the popular point-cloud methods.

Original: Besides, we also ﬁnd that FIP-ALL can improve the data efﬁciency on few-shot learning, and accelerate the training speed by a large margin.
Revised: We also ﬁnd that FIP-ALL can improve the data efﬁciency on few-shot learning and accelerate the training speed by a large margin.

Original: After this, we shed light on whythe image-pretrained model can be used for point-cloud understanding from three aspects, network dissection, texture-shape representation transferring,  feature distribution distance.
Revised: Additionally, we shed light on why image-pretrained models can be used for point-cloud understanding from three aspects: network dissection, texture-shape representation transferring, and feature distribution distance.

Original: Compared with previous works that seek improvements from designing architectures and pretraining only onthe point-cloud modality, our work is not limited by the architecture design and small scale point-cloud dataset.
Revised: Compared with previous works that seek improvements from designing architectures and pretraining only on point-cloud modality, our work is not limited by the architecture design and the small-scale point-cloud dataset.

Original: We believe that image pretraining is one of the solutions to the bottleneck of point-cloud understanding anddo hope this direction can inspire the research community in the future.
Revised: We believe that image pretraining is one of the solutions to the bottleneck of point-cloud understanding and hope this direction can inspire the research community.

Original: A A PPENDIX A. I MPLEMENTATION DETAIL 
Revised: A A PPENDIX A. I MPLEMENTATION DETAIL .

Original: Our experiments are conducted on ModelNet 3D Wharehouse, S3DIS, and SemanticKITTI dataset.
Revised: Our experiments are conducted on ModelNet 3D Warehouse, S3DIS, and SemanticKITTI datasets.

Original: For the ModelNet 3D Wharehouse dataset, we train all models on the train set and evaluate on the validation set.
Revised: For the ModelNet 3D Warehouse dataset, we train all models on the train set and evaluate on the validation set.

Original: For the S3DIS, we train all models on area 1, 2, 3, 4, 6 and evaluate on area 5.
Revised: For the S3DIS, we train all models on area 1, 2, 3, 4, 6 and evaluate on area 5.

Original: For the
Revised: For the

Original: SemanticKITTI dataset, we train all models on splits 00-10 except 08 which is used for evaluation.
Revised: SemanticKITTI dataset, we train all models on splits 00-10 except 08 which is used for evaluation.

Original: For each of the dataset, all ResNet-series models use the same training scheme, and all experiments are implemented with Pytorch.
Revised: For each of the dataset, all ResNet series models use the same training scheme, and all experiments are implemented with PyTorch.

Original: When training on the ModelNet 3D Wharehouse dataset , coordinates of point-cloud are randomly scaled, translated, and jittered.
Revised: When training on the ModelNet 3D Warehouse dataset , coordinates of point-clouds are randomly scaled, translated, and jittered.

Original: We use SGD optimizer with momentum 0.9, weight-decay 10 − 4 , and initial learning rate 0.1 with cosine learning rate scheduler.
Revised: We use SGD optimizer with momentum 0.9, weight-decay 10 − 4 , and initial learning rate 0.1 with cosine learning rate scheduler.

Original: Each mini batch is set to 32, and models are trained for 300 epochs.
Revised: Each mini batch is set to 32, and models are trained for 300 epochs.

Original: For both training and inference phase, we only utilize x, y, z coordinates without other features and set voxel size to 0.05.
Revised: For both training and inference phase, we only utilize x, y, z coordinates without other features and set voxel size to 0.05.

Original: The experiments for ModelNet 3D Wharehouse are all conducted on a Titan RTX GPU.
Revised: The experiments for ModelNet 3D Warehouse are all conducted on a Titan RTX GPU.

Original: When training on the S3DIS dataset , we concatenate all subparts of an indoor scene to train and validate on.
Revised: When training on the S3DIS dataset , we concatenate all subparts of an indoor scene to train and validate on.

Original: Along x, y directions, scenes arerandomly applied horizontal ﬂip.
Revised: Along x, y directions, scenes are applied horizontal ﬂip randomly.

Original: RGB features are randomly jittered, translated, and auto contrasted.
Revised: RGB features are randomly jittered, translated, and auto contrasted.

Original: Finally, we normalize and clip point-clouds.
Revised: Finally, we normalize and clip point-clouds.

Original: We set voxel size to 0.05. We use SGD optimizer with momentum 0.9, weight-decay 10 − 4 , and initial learning rate  0.1 with polynomial learning rate scheduler.
Revised: We set voxel size to 0.05, use SGD optimizer with momentum 0.9, weight-decay 10 − 4 , and initialize learning rate to 0.1 with polynomial learning rate scheduler.

Original: Each mini batch is set to 3, and models are trained for 400 epochs on 2 Titan RTX GPUs.
Revised: Each mini batch is set to 3, and models are trained for 400 epochs on 2 Titan RTX GPUs.

Original: When training on the SemanticKITTI dataset , coordinates of each point-cloud are randomly scaled and rotated.
Revised: When training on the SemanticKITTI dataset , coordinates of each point-cloud are randomly scaled and rotated.

Original: We use SGD optimizer with momentum 0.9, weight-decay 10 − 4 , and initial learning rate 0.24 with cosine warmup learning rate scheduler.
Revised: We use SGD optimizer with momentum 0.9, weight-decay 10 − 4 , and initial learning rate 0.24 with cosine warmup learning rate scheduler.

Original: Each mini batch is set to 2, and models are trained for 15 epochs on 4 Titan RTX GPUs.
Revised: Each mini batch is set to 2, and models are trained for 15 epochs on 4 Titan RTX GPUs.

Original: For both training and inference phase, we utilize x, y, z coordinates as well as intensity feature and set voxel size to 0.05.
Revised: For both training and inference phase, we utilize x, y, z coordinates as well as intensity feature and set voxel size to 0.05.

Original: Most of our pretrained models come from open-sources, 1 2 3 4 5 6 , so we do not need to take time and computational resources for pretraining.
Revised: Most of our pretrained models come from open-sources, 1 2 3 4 5 6 , so we do not need to take time and computational resources for pretraining.

Original: We use torchsparse 7 to produce sparse 3D convolutions.
Revised: We use torchsparse 7 to produce sparse 3D convolutions.

Original: Details of Section 4.1.
Revised: Details of Section 4.1.

Original: In this section, we take the ResNet architecture, inﬂate the pretrained models of different image dataset, and add linear input and output layers as shown in Section A.5.
Revised: In this section, we take the ResNet architecture, inﬂate the pretrained models of different image datasets, and add linear input and output layers as shown in Section A.9.

Original: The
Revised: The

Original: ResNet50 pretrained on ImageNet1K is directly taken from Pytorch.
Revised: ResNet50 pretrained on ImageNet1K is directly taken from PyTorch.

Original: We use the same training recipe provided by Pytorch to train the ResNet50 on Tiny-ImageNet.
Revised: We use the same training recipe provided by PyTorch to train the ResNet50 on Tiny-ImageNet.

Original: The pretrained ResNet50 on
Revised: The pretrained ResNet50 on

Original: ImageNet21K comes from (Ridnik et al., 2021).
Revised: ImageNet21K comes from Ridnik et al.

Original: Details of Section 4.2, Section 4.3 and 4.4.
Revised: Details of Section 4.2, Section 4.3 and 4.4.

Original: In this section, the ResNet pretrained models are taken from the same sources as illustrated above.
Revised: In this section, the ResNet pretrained models are taken from the same sources as illustrated above.

Original: For PointNet++ pretraining on ImageNet1K, we utilize the PointNet++ SSG version (Qi et al., 2017).
Revised: For PointNet++ pretraining on ImageNet1K, we utilize the PointNet++ SSG version (Qi et al., 2017).

Original: We break the images into pixels, and regard the  pixels as a point-cloud, coordinates ofwhich are the x, y position in  original image, with appending z = 1 to all pixels.
Revised: We break the image into pixels and regard the group of pixels as a point-cloud with coordinates of x, y positions in the original image and appending z = 1 to all pixels.

Original: Then we set center sampling number to 1024 and 256 for ﬁrst and second stage,  the radius are set into 8 and 64 for them respectively.
Revised: Then, we set center sampling number to 1024 and 256 for ﬁrst and second stage, and the radius is set into 8 and 64, respectively.

Original: For each center point, wewill query 64 neighbouring points.
Revised: For each center point, we query 64 neighbouring points.

Original: The training recipe is also provided by
Revised: The training recipe is also provided by

Original: Pytorch.
Revised: PyTorch.

Original: For ViT models, we directly take the pretrained weights from Dosovitskiy et al.
Revised: For ViT models, we directly take the pretrained weights from Dosovitskiy et al.

Original: To apply it on ModelNet 3D Wharehouse, we sample 256 centers and group 64 nearby points, regarding these as "point-cloud patches".
Revised: To apply it on ModelNet 3D Warehouse, we sample 256 centers and group 64 nearby points, regarding these as "point-cloud patches".

Original: Then we use a linear embedding to project the point-cloud patches into a sequence, andthe ViT process themas same as the images.
Revised: Then, we use a linear embedding to project the point-cloud patches into a sequence, and ViT processes them same as image patches.

Original: Except the linear embedding and the ﬁnal
Revised: Except the linear embedding and the ﬁnal

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: output classiﬁer, all the models are kept same as the origin version.
Revised: output classiﬁer, all the models are kept same as the origin version.

Original: For the experiments on S3DIS and SemanticKITTI, the architecture detail of ResNet18 is shown in A.5 listing 2.
Revised: For the experiments on S3DIS and SemanticKITTI, the architecture detail of ResNet18 is shown in A.5 listing 2.

Original: For SimpleView model, all the experiment settings are as same as Goyal et al.
Revised: For SimpleView model, all the experiment settings are the same as Goyal et al.

Original: (2021a).
Revised: (2021a).

Original: The only difference is  to useor not use the pretrained ResNet18.
Revised: The only difference is whether to use the pretrained ResNet18.

Original: For HRNetV2-W48, we directly use the
Revised: For HRNetV2-W48, we directly use the

Original: ImageNet1Kpretrained model and Cityscape pretrained model from (Sun et al., 2019).
Revised: ImageNet1K and Cityscape pretrained models from (Sun et al., 2019).

Original: We conduct three trials on the few-shot experiments.
Revised: We conduct three trials on the few-shot experiments.

Original: For each trial, we change the random seed but keep all the others are same.
Revised: For each trial, we change the random seed but keep all the other settings the same.

Original: To plot the training speed curve, we directly use the training log without any other change like smoothing.
Revised: To plot the training speed curve, we directly use the training log without any other changes, such as smoothing.

Original: A. F INETUNING THE MEAN AND VARIANCE OF BATCH NORMALIZATION .
Revised: A. F INETUNING THE MEAN AND VARIANCE OF BATCH NORMALIZATION .

Original: For the ﬁrst group of experiments, ResNet50 FIP either has IO or IO+BN ﬁnetuned.
Revised: For the ﬁrst group of experiments, ResNet50 FIP either has IO or IO+BN ﬁnetuned.

Original: In addition to these two experimental setting, we also investigate ﬁntuning input, output layers, and mean, variance of normalization layers, while ﬁxing the convolution layer weights, normalization layer weights and bias.
Revised: In addition to these two experimental settings, we also investigate ﬁntuning input, output layers, and mean, variance of normalization layers, while ﬁxing the convolution layer weights, normalization layer weights, and bias.

Original: The full experiments result with this extra setting is reported in Table 6 and 7.
Revised: The full experiment results with this extra setting is reported in Table 6 and 7.

Original: We can observe that compared with only ﬁnetuning input and output layers, updating mean and variance can also largely improve the performance of point-cloud recognition.
Revised: We can observe that compared with only ﬁnetuning input and output layers, updating mean and variance can also largely improve the performance of point-cloud recognition.

Original: As suggested in Section 5.2 in the main paper, updating mean and variance isalso to push the ﬁnetuned pretrained image model to generate point-cloud representation as similar as image representation.
Revised: As suggested in Section 5.2 in the main paper, updating mean and variance is to push the FIP models to generate point-cloud representation that is similar to image representation.

Original: A. M ORE VISUALIZATION OF N ETWORK DISSECTION .
Revised: A. A BLATION STUDY OF INFLATING TOWARDS DIFFERENT DIRECTIONS .

Revised: We conduct experiments of inﬂating ﬁlters along different directions, the illustration ﬁgure is shown in Figure 5, and the results are shown in Table 8.

Revised: We ﬁnd that the performance is different when using different inﬂation methods.

Revised: In particular, with ResNet50 pretrained on ImageNet1K, inﬂating along x axis and the y axis leads to better performance compared with inﬂating along z axis for both

Revised: FIP-IO and FIP-IO+BN.

Revised: More importantly, the minimally ﬁnetuned FIP-IO+BN with inﬂating along the x and y axis even surpasses the training-from-scratch.

Revised: A. A BLATION STUDY OF LOADING DIFFERENT STAGES OF IMAGE - PRETRAINED MODEL .

Revised: We investigate the effect of loading different subset of stages.

Revised: The results are shown in Table 9.

Revised: In detail, we load the pretrained weights partially while keeping the other weights randomly initialized.

Revised: [Figure]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: We observe that excluding the weights of the ﬁrst stage achieves the best performance, bringing 0.

Revised: points improvement.

Revised: A. A N INTUITION OF WHY THE IMAGE - PRETRAINED WEIGHTS CAN BE USED FOR POINT - CLOUD RECOGNITION .

Revised: In this subsection

Revised: [Table]

Revised: we try to give an intuition of why the weights pretrained on images can be used for point-cloud tasks from the aspect of local feature projection.

Revised: Let’s only look at the one time convolution on a local point-cloud feature X local ∈ R C in × C out × K 3 .

Revised: In fact

Revised: [Table]

Revised: a ﬁlter inﬂation can be represented as linear projection, given by W 2 d ⊗ T

Revised: = W 3 d , where W 2 d ∈ R C in × C out ×

Revised: Kis pretrained 2D weight, W 3 d ∈ R C in × C out × K 3 is the transformed 3D weight from W 2 d

Revised: [Table]

Revised: and

Revised: T ∈ R K 2 × K 3 is the projection matrix.

Revised: Therefore, for a local 3D feature X local ∈ R C in × C out × K 3 , the convolution can be formed as W 3 d ⊗ X local

Revised: =

Revised: W 2 d ⊗ T ⊗ X local

Revised: =

Revised: W 2 d ⊗ X local − 2 d , where

Revised: X local − 2 d ∈ R C in × C out × K 2 is the 2D feature projected from X local .

Revised: Essentially, applying the inﬂated ﬁlters on 3D point-clouds is equal to the local projection of 3D point-clouds to 2D then applying the pretrained 2D ﬁlters.

Revised: A visual explanation is in Figure 6.

Revised: [Figure]

Revised: [Table]

Revised: A. M ORE VISUALIZATION OF NETWORK DISSECTION .

Original: We present more visualization results based on the technique of network dissection, as shown in
Revised: We present more visualization results based on the technique of network dissection, as shown in

Original: Figure 5.
Revised: Figure 7.

Original: We can observe that for most cases, although there is no obvious visual concept transferring, the pretrained ﬁlters are prone to cluster similar objects on ModelNet 3D Wharehouse dataset. 
Revised: We can observe that for most cases, although there is no obvious visual concept transferred, the pretrained ﬁlters are prone to cluster similar objects on ModelNet 3D Warehouse dataset. A. U SING DIFFERENT IMAGE PROCESSING METHODS FOR PRETRAINING .

Revised: To further investigate the image-to-point-cloud transfer in a more rigorous manner, we design a new set of experiments and report them in Appendix A.7 in the paper revision.

Revised: The basic idea is that we perform a transformation (such as a low-pass ﬁlter) on the original image to destroy certain properties (high frequency components).

Revised: Then, we train a ResNet18 on the transformed images, then transfer to point-cloud recognition.

Revised: This way, we can better attribute which parts of the images are useful when transferred to point-cloud recognition.

Original: A. D ETAILS OF F IRST -W ASSERSTEIN D ISTANCE ON R ES N ET 18.
Revised: We perform the following transformations. Implementation code is taken from external sources 8 9 10 . 1) Canny edge detection (Canny, 1986) transforms the images into high-frequency edge maps, discarding the low-frequency such as the surface of objects; 2) Bilateral ﬁlters (Tomasi & Manduchi, 1998) preserve the low-frequency properties of images while ﬁltering out the high-frequency noise. Note that the high-frequency edges are still preserved and even sharpened; 3) Haar wavelet image compression (Talukder & Harada, 2010).

Revised: The wavelet transform divides the information of an image into approximation and detailed sub-signals.

Revised: If the detail is small, then it would be thrown; 4) Patch shufﬂing divides an image into patches with different patch sizes, shufﬂe the patches, and rearrange them into a new image.

Revised: This transformation preserves the statistics of the images, while destroying the original order of the pixels.

Revised: “Shufﬂe patch 56” means the patch size is 56 × 56 .

Original: We list all the results  in each layer of ResNet18, as shown in Table 8.
Revised: The results are reported in the Table 10. Speciﬁcally, the edge images from canny edge detection are dramatically different from original images while keeping minimal performance drop. The bilateral ﬁlters discard the high-frequency noise, sharpening the high-frequency edge lead to the best performance, even outperforming the result of ﬁnetuning from the original image pretraining. As details of images are thrown (haar transformation), the performance slightly drops.

Original: We can observe that for each layer, the ﬁrst-wasserstein distance is largely reduced when ﬁnetuning more parameters.
Revised: On the other hand, shufﬂing the patches results in worse performance as the patch size gradually becomes small, which indicates

Original: Interestingly, we also ﬁnd that the distance between training-from-scratch on ModelNet 3D Wharehouse and image models are not too far, which may indicate the image and point-cloud could be represented similar even no transferring. [Table]
Revised: that the order of the pixels are important to the transfer. Therefore, we may infer that the structured (ordered) edges are important to the transfer.

Original: A. 
Revised: A. D ETAILS OF FIRST - WASSERSTEIN DISTANCE ON R ES N ET 18.

Revised: We list all the results in each layer of ResNet18, as shown in Table 11. We can observe that for each layer, the ﬁrst-wasserstein distance is largely reduced when ﬁnetuning more parameters.

Original: [Table]
Revised: [Table]

Original:  D ETAILS OF USED ARCHITECTURES 
Revised: A. D ETAILS OF USED ARCHITECTURES .

Original: [Table]
Revised: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: sparse

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: 24self.up2 = sparse_deconv(layer3_Odim, layer3_Odim, k=2, s=2) self.decode

Revised: 24self.up2 = sparse_deconv(layer3_Odim, layer3_Odim, k=2, s=2) self.decode

Original: = self.
Revised: = self.

Original: Sequential( res_block(layer3_Odim+layer2_Odim, layer2_Odim), res_block(layer2_Odim, layer2_Odim))
Revised: Sequential( res_block(layer3_Odim+layer2_Odim, layer2_Odim), res_block(layer2_Odim, layer2_Odim))

Original: 29self.up3 = sparse_deconv(layer2_Odim, layer2_Odim, k=2, s=2) self.decode3 = self.
Revised: 29self.up3 = sparse_deconv(layer2_Odim, layer2_Odim, k=2, s=2) self.decode3 = self.

Original: Sequential( res_block(layer2_Odim+layer1_Odim, layer1_Odim), res_block(layer1_Odim, layer1_Odim))
Revised: Sequential( res_block(layer2_Odim+layer1_Odim, layer1_Odim), res_block(layer1_Odim, layer1_Odim))

Original: 34self.up4 = sparse_deconv(layer1_Odim, layer1_Odim, k=2, s=2) self.decode

Revised: 34self.up4 = sparse_deconv(layer1_Odim, layer1_Odim, k=2, s=2) self.decode

Original: = self.
Revised: = self.

Original: Sequential( res_block(layer1_Odim+layer1_Odim, layer1_Odim), res_block(layer1_Odim, layer1_Odim))
Revised: Sequential( res_block(layer1_Odim+layer1_Odim, layer1_Odim), res_block(layer1_Odim, layer1_Odim))

Original: 39self.output_layer = nn.
Revised: 39self.output_layer = nn.

Original: Sequential( nn.Linear(layer1_Odim, class_num)) 42def forward(self, x): x_i = self.input_layer(x) x1 = self.layer1(x_i) x2 = self.layer2(x1) x3 = self.layer3(x2)
Revised: Sequential( nn.Linear(layer1_Odim, class_num)) 42def forward(self, x): x_i = self.input_layer(x) x1 = self.layer1(x_i) x2 = self.layer2(x1) x3 = self.layer3(x2)

Original: x4 = self.layer4(x3) 49x

Revised: x4 = self.layer4(x3) 49x

Original: _ = self.decoder1(cat(x3, self.up1(x4)))
Revised: _ = self.decoder1(cat(x3, self.up1(x4)))

Original: x2_
Revised: x2_

Original: =
Revised: =

Original: self.decoder2(cat(x2, self.up2(x3_)))
Revised: self.decoder2(cat(x2, self.up2(x3_)))

Original: x

Revised: x

Original: _ = self.decoder3(cat(x1, self.up3(x2_)))
Revised: _ = self.decoder3(cat(x1, self.up3(x2_)))

Original: xi
Revised: xi

Original: _ = self.decoder4(cat(x_i, self.up
Revised: _ = self.decoder4(cat(x_i, self.up

Original: [Figure]
Revised: [Figure]

Original: B r o d e n
Revised: B r o d e n

Original: M o d e l n e t
Revised: M o d e l n e t

Original: [Figure] 
Revised: [Figure] [Figure]

Revised: o d e l n e t

Original: [Figure]
Revised: [Figure]

Original: M o d e l n e t
Revised: r o d [Figure] n [Figure]

Original: [Figure]
Revised: [Figure]

Original: B r o d e n 
Revised: r o d e n M o d e l n e t

Original: [Figure]
Revised: [Figure]

Revised: M o d e l n e t

Original: [Figure]
Revised: [Figure]

Original: B r o d e n
Revised: B r o d e n

Original: M o d e l n e t

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: M o d e l n e t

Original: [Figure]
Revised: [Figure]

Original: B r o d e n
Revised: B r o d e n

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: B r o d e n

Original: M o d e l n e t
Revised: M o d e l n e t

Original: [Figure]
Revised: [Figure]

Original: M
Revised: d

Original: [Figure]
Revised: [Figure]

Original: d e l n e t
Revised: l n e t

Original: [Figure]
Revised: [Figure]

Original: n

Original: [Figure]

Original: t
Revised: t

Original: [Figure]
Revised: [Figure]