Original: A DAPTIVE W AVELET T RANSFORMER N ETWORK FOR 3D S HAPE R EPRESENTATION L EARNING
Revised: A DAPTIVE W AVELET T RANSFORMER N ETWORK FOR 3D S HAPE R EPRESENTATION L EARNING

Original: A BSTRACT
Revised: A BSTRACT

Original: We present a novel method for 3D shape representation learning using multi-scale wavelet decomposition.
Revised: We present a novel method for 3D shape representation learning using multi-scale wavelet decomposition.

Original: Distinct from previous works that either decompose 3D shapes into complementary components  at a single scale, or naively adopt up- /down-sampling to build hierarchies and treat all points or local regions equally, we  decompose 3D shapes into sub-bands components  at multiple scales and all scales form a  decomposition tree in a principled manner rooted in multi-resolution wavelet analysis .
Revised: Previous works often decompose 3D shapes into complementary components in spatial domain at a single scale. In this work, we study to decompose 3D shapes into sub-bands components in frequency domain at multiple scales, resulting in a hierarchical decomposition tree in a principled manner rooted in multi-resolution wavelet analysis .

Original: Specifically, we propose Adaptive Wavelet Transformer Network (AWT-Net) that firstly generates approximation or detail wavelet coefficients per point, classifying each point into high or low sub-bands components, using lifting scheme at multiple scales recursively and hierarchically.
Revised: Specifically, we propose Adaptive Wavelet Transformer Network (AWT-Net) that firstly generates approximation or detail wavelet coefficients per point, classifying each point into high or low sub-bands components, using lifting scheme at multiple scales recursively and hierarchically.

Original: Then, AWT-Net exploits Transformer that treats the  features from different butcomplementary components as two integrated representations, and fuses them with the original shape features with different attentions.
Revised: Then, AWT-Net exploits Transformer to enhance the original shape features by querying and fusing features from different but integrated sub-bands.

Original: The wavelet coefficients can be learned without direct supervision on coefficients, and AWT-Net is fully differentiable and can be learned in an end-to-end fashion.
Revised: The wavelet coefficients can be learned without direct supervision on coefficients, and AWTNet is fully differentiable and can be learned in an end-to-end fashion.

Original: Extensive experiments demonstrate that AWT-Net achieves competitive performance on 3D shape classification and segmentation benchmarks.
Revised: Extensive experiments demonstrate that AWT-Net achieves competitive performance on 3D shape classification and segmentation benchmarks.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Analysis of 3D point cloud is a crucial topic in computer vision and graphics, and has wide applications in robotics (Chen et al., 2019), autonomous driving (Qi et al., 2018), and visual SLAM (Hitchcox & Forbes, 2020), etc .
Revised: Analysis of 3D point cloud is a crucial topic in computer vision and graphics, and has wide applications in robotics (Chen et al., 2019), autonomous driving (Qi et al., 2018), and visual SLAM (Hitchcox & Forbes, 2020), etc .

Original: To better understand the 3D shapes, effective point cloud analysis approaches and methods are in great demand.
Revised: To better understand the 3D shapes, effective point cloud analysis approaches and methods are in great demand.

Original: With the thriving of deep learning, numerous well-designed neural networks are applied to extract expressive semantics of 3D point clouds with layered operations, in contrast to low-levelhand-crafted shape descriptors.
Revised: With the thriving of deep learning, numerous task-driven neural networks are created to generate high-level 3D shape representations with learnable neurons, rather than designing low-level shape descriptors manually.

Original: The pioneering work PointNet (Qi et al., 2017a) learns a spatial encoding of each point using multi-layer perceptrons and aggregates all point features to a global shape representation.The follow-on work PointNet++ (Qi et al., 2017b) adopts a hierarchical encoder-decoder structure to process local regions, which down-samples point clouds in layers and gradually interpolates them back to the original resolution.
Revised: PointNet (Qi et al., 2017a) is firstly proposed to use a deep neural network to directly consumes take as input a raw point cloud and max-pools per-point features to obtain the global shape representation. PointNet++ (Qi et al., 2017b) introduces feature aggregation operations at multi-scales to group of the entire point clouds in both down- and up-sampling layers.

Original: The following works learn 3D shape representation from different perspectives: (Xu et al., 2018; Thomas et al., 2019) extend regular grid convolution to irregular 3D point clouds, (Li et al., 2018; Wu et al., 2019) design local operations on point patches, (Wang et al., 2019; Shi & Rajkumar, 2020) treat point clouds as graphs andapply graph convolution networks, to name a few.
Revised: The following works learn 3D shape representation from different perspectives: SO-Net (Li et al., 2018a) utilizes self-organizing map to model the spatial distribution of points, SpiderCNN (Xu et al., 2018) exploits step function and

Original:  An interesting yet challenging problem is how to decompose 3D shapes into interpretable representations and exploit them for better shape analysis. For instance, some attempts (Wang et al., 2019; Lan et al., 2019) capture local geometric relations among center points and their neighbors. However, these works treat all points or local regions equally, making it hard to extract features of the most relevant and crucial geometric parts, e.g .,  areas containing flat planes or areas containing edges.
Revised: Taylor polynomial to captures local geodesic efficiently, KPConv (Thomas et al., 2019) designs de- formable convolutions to localize kernel points for local geometry learning, DGCNN (Wang et al., 2019) treats point clouds as graphs and applies graph convolution networks, to name a few. An interesting yet challenging problem is how to decompose 3D shapes into interpretable representations and exploit them for better shape analysis, e.g ., decomposing shapes into areas containing flat planes or areas containing edges.

Original: Another challenging problem is how to analyze 3D shapes at multiple scales/resolutions in an explainable way.
Revised: Another challenging problem is how to analyze 3D shapes at multiple scales/resolutions in an explainable way.

Original: Though some works (Qi et al., 2017b; Jiang et al., 2019) extract point cloud features hierarchically, these works build the hierarchy in spatial/semantic spaces based on pairwise point distance, which is less interpretable w.r.t .
Revised: Though some works (Qi et al., 2017b; Jiang et al., 2019) extract point cloud features hierarchically, these works build the hierarchy in spatial/semantic spaces based on pairwise point distance, which is less interpretable w.r.t .

Original: shape geometry.
Revised: shape geometry.

Original: The geometric components with different frequencies in 3D shapes contain distinct geometric characteristics ( e.g ., high-frequency components correspond to edges or sharp parts, while lowfrequency components correspond to flat areas or smooth parts) and provide complementary geometric information for shape representation.
Revised: As indicated in GDA-Net (Xu et al., 2021b), the geometric components with different frequencies in 3D shapes contain distinct geometric characteristics ( e.g ., high-frequency components correspond to edges or sharp parts, while low-frequency components correspond to flat areas or smooth parts) and provide complementary geometric information for shape representation.

Original: To effectively learn 3D shape representation, it is desirable to separate these components and process them differently, rather than feed the whole shape into a single processor.
Revised: To effectively learn 3D shape representation, it is desirable to separate these components and process them differently, rather than feed the whole shape into a single processor.

Original: It, therefore, motivates us to decompose 3D shapes into sub-bands components.
Revised: GDA-Net (Xu et al., 2021b) disentangles 3D shapes into sharp and gentle variation components using spectral graph analysis. Generally, GDANet starts from building a dynamic graph from a given point cloud, then applies a 2-order polynomial approximation (Laplacian) as filter to the graph to disentangle the graph into high and low-frequency components, and finally fuses the two component features with the original shape features through two attention modules. GDA-Net has achieved excellent performance on 3D shape classification and segmentation. However, for a given graph, the spectral analysis based approach adopted in GDANet provides a static and deterministic graph decomposition, posing a limitation that disconnects the graph decomposition process from downstream tasks of shape representation learning.

Original: In this work, we propose a principled method  for 3D shape representation learning rooted in wavelet transform.
Revised: Therefore, to address the limitation, we propose a novel wavelet analysis based method to decompose 3D shapes into sub-bands components for 3D shape representation learning. The decomposition process is realized by neural networks in a learnable fashion, thus connected to the downstream tasks of shape representation learning.

Original: Theoretical properties of wavelets in signal processing, such as time-frequency localization and multi-resolution analysis, have been well studied, making it appropriate to fuse into deep neural networks with interpretability.
Revised: Theoretical properties of wavelets in signal processing, such as time-frequency localization and multi-resolution analysis, have been well studied, making it appropriate to fuse into deep neural networks with interpretability.

Original: Inspired by that, we propose to perform multi-resolution analysis within neural networks through lifting scheme (Sweldens, 1998) to achieve a data-driven wavelet transform.
Revised: Inspired by that, we propose to perform multi-resolution analysis within neural networks through lifting scheme (Sweldens, 1998) to achieve a data-driven wavelet transform.

Original: The multi-resolution analysis generates decomposed visual representation at each scale, which contributes to the interpretability of the network.
Revised: The multi-resolution analysis generates decomposed visual representation at each scale, which contributes to the interpretability of the network.

Original: The generated approximation or detail coefficients per point at each scale, corresponding to high or low-frequency components, capture essential geometric interest for down-streaming tasks.
Revised: The generated approximation or detail coefficients per point at each scale, corresponding to high- or low-frequency components, capture essential geometry for downstream tasks.

Original: Specifically, we first adapt wavelet transform that analyzes 1D temporal or 2D image signals to 3D shapes at different semantic levels and decompose the original point cloud into the approximation/coarse and detail components, as explained in Section 4.1 and 4.2.
Revised: Specifically, we first adapt wavelet transform that analyzes 1D temporal or 2D image signals to 3D shapes at different semantic levels and decompose the original point cloud into the approximation/coarse and detail components, as explained in Section 4.1 and 4.2.

Original: Next, in Section 4.3, we utilize Transfomer (Vaswani et al., 2017; Choromanski et al., 2021) to pay different attention to features from approximation and detail variation components, and then fuse them with the original point features, respectively.
Revised: Next, in Section 4.3, we utilize Transfomer (Vaswani et al., 2017) to pay different attention to features from approximation and detail variation components, and then fuse them with the original point features, respectively.

Original: As shown in Figure 1, the approximation and detail components have distinct but complementary effects on reflecting the geometry of 3D shapes.
Revised: As shown in Figure 1, the approximation and detail components have distinct but complementary effects on reflecting the geometry of 3D shapes.

Original: Equipped with wavelet transform and Transformer, we propose Adaptive Wavelet Transformer Network (AWT-Net) that captures and refines the integrated and complementary geometry of 3D shapes to supplement neighboring local information.
Revised: Equipped with wavelet transform and Transformer, we propose Adaptive Wavelet Transformer Network (AWT-Net) that captures and refines the integrated and complementary geometry of 3D shapes to supplement neighboring local information.

Original: AWT-Net is trained using the loss functions proposed in Section 4.4.
Revised: AWT-Net is trained using the loss functions proposed in Section 4.4.

Original: Experimental results on challenging benchmarks demonstrate that our AWT-Net achieves competitive performance on 3D shape classification and segmentation tasks.
Revised: Experimental results on challenging benchmarks demonstrate that our AWT-Net achieves competitive performance on 3D shape classification and segmentation tasks.

Original: Comprehensive analysis and visualizations verify that our model effectively extracts the complementary geometric features from two decomposed components at multiple scales.
Revised: Comprehensive analysis and visualizations verify that our model effectively extracts the complementary geometric features from two decomposed components at multiple scales.

Original: For instance, as shown in Figure 1, we observe that our model can learn meaningfully samplings of point clouds in different categories and can detect key regions consistently within a semantic class, e.g ., it separates chair legs and backs from chair seat, and separates a cup handle from a body.
Revised: For instance, as shown in Figure 1, we observe that our model can learn meaningfully samplings of point clouds in different categories and can detect key regions consistently within different categories, e.g ., it separates chair legs and backs from chair seat, and separates a cup handle from a body.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Geometric Point Cloud Models.
Revised: Geometric Point Cloud Models.

Original: PointNet (Qi et al., 2017a) and DeepSet (Zaheer et al., 2017) are pioneering neural network models that directly process point cloud to learn a spatial encoding of each point and then aggregate all individual point features to an integrated representation.
Revised: PointNet (Qi et al., 2017a) and DeepSet (Zaheer et al., 2017) are pioneering neural network models that directly process point cloud to learn a spatial encoding of each point and then aggregate all individual point features to an integrated representation.

Original: Point-Net++ (Qi et al., 2017b) partitions point cloud into overlapping local regions by the distance metric of spatial and/or semantic spaces and extracts local features capturing fine geometric structures from neighbors hierarchically.
Revised: PointNet++ (Qi et al., 2017b) partitions point cloud into overlapping local regions by the distance metric of spatial and/or semantic spaces and extracts local features capturing fine geometric structures from neighbors hierarchically.

Original: DGCNN (Wang et al., 2019) reconstructs k-NN graphs using nearest neighbors in feature space to capture local geometric structure while maintaining permuta- tion invariance. 
Revised: DGCNN (Wang et al., 2019) reconstructs k-NN graphs using nearest neighbors in feature space to capture local geometric structure while maintaining permutation invariance. KC-Net

Original: Geo-CNN (Lan et al., 2019) defines a convolution operation that aggregates edge features to capture the local geometric relations. (Xu et al., 2020) aggregates points from local neighbors with similar semantics in both Euclidean and Eigenvalue space.
Revised: (Shen et al., 2018) proposes kernel correlation layers to extract the local geometric features. Point2Sequence (Liu et al., 2019a) designs a sequence model to capture the correlations among points. FPConv (Lin et al., 2020a) introduces a surface-style convolution operator to directly work on point cloud surface.

Original: RS-CNN (Liu et al., 2019c) extends regular CNN to encode geometric relation in a localpoint set by learning shapetopology. DensePoint (Liu et al., 2019b) recognizes the implicit geometry by extracting contextual shape semantics.
Revised: Point-GNN (Shi & Rajkumar, 2020) learns local shape geometry by capturing interactions among points in local patches through node and edge feature aggregation. PAConv (Xu et al., 2021a) proposes a convolution kernel that adaptively assembles different weight matrices using coefficients learned from point positions.

Original: GDA-Net
Revised: GDA-Net

Original: (Xu et al., 2021) is built on graph spectral theory and uses a 2-order polynomial approximation (Laplacian) as the filter and applies it to a graph to disentangle the graph into high and low-frequency components, thus resulting in deterministic and static decomposition.
Revised: (Xu et al., 2021b) is built on graph spectral theory and uses a 2-order polynomial approximation (Laplacian) as a filter and applies it to a graph to disentangle the graph into high and low-frequency components, thus resulting in deterministic and static decomposition.

Original: Our model learns point cloud geometry from a novel perspective, i.e ., multi-resolution wavelet analysis, which has been extensively applied in signal processing but seldom explored for 3D shapes.
Revised: Our model learns point cloud geometry from a novel perspective, i.e ., multi-resolution wavelet analysis, which has been extensively applied in signal processing but seldom explored for 3D shapes.

Original: Wavelet in Vision.
Revised: Wavelet in Vision.

Original: Theoretical properties of multi-resolution analysis using wavelets have been well studied, making such approaches more interpretable than CNNs.
Revised: Theoretical properties of multi-resolution analysis using wavelets have been well studied, making such approaches more interpretable than CNNs.

Original: Several prior works incorporate wavelet representations into CNNs.
Revised: Several prior works incorporate wavelet representations into CNNs.

Original: (Oyallon et al., 2017) proposes a hybrid network that replaces the first layers of a ResNet with a wavelet scattering network.
Revised: (Oyallon et al., 2017) proposes a hybrid network that replaces the first layers of a ResNet with a wavelet scattering network.

Original: This modified ResNet resulted in comparable performance to the original ResNet but has fewer trainable parameters.
Revised: This modified ResNet resulted in comparable performance to the original ResNet but has fewer trainable parameters.

Original: The wavelet pooling layer proposed in (Williams & Li, 2018) subsamples image features based on second-level wavelet decomposition.
Revised: The wavelet pooling layer proposed in (Williams & Li, 2018) subsamples image features based on second-level wavelet decomposition.

Original: Wavelet-SRNet (Huang et al., 2017) reconstructs super-resolution images from multi-scale wavelet coefficients estimated from low-resolution images under coefficient supervision.
Revised: Wavelet-SRNet (Huang et al., 2017) reconstructs super-resolution images from multi-scale wavelet coefficients estimated from low-resolution images under coefficient supervision.

Original: Similar scheme isadapted in (Liu et al., 2020) for image demoir ´ eing and (Rong et al., 2020) for burst denoising. In (Ramamonjisoa et al., 2021), they decompose a single image using Haar
Revised: Similar scheme is

Original: Transform and then reconstruct depth images hierarchically through inverse discrete wavelet transform.
Revised: Transform and then reconstruct depth images hierarchically through inverse discrete wavelet transform.

Original: All of the mentioned works use pre-defined Haar wavelet which is less flexible for complex 3D shapes.
Revised: All of the mentioned works use pre-defined Haar wavelet which is less flexible for complex 3D shapes.

Original: (Rustamov & Guibas, 2013) adopts lifting scheme but still restricts to linear transformations of Haar coefficients for graph signals.
Revised: (Rustamov & Guibas, 2013) adopts lifting scheme but still restricts to linear transformations of Haar coefficients for graph signals.

Original: DAWN-Net
Revised: DAWN-Net

Original: (Rodriguez et al., 2020) uses lifting scheme to implicitly define wavelet transform functions for image recognition.
Revised: (Rodriguez et al., 2020) uses lifting scheme to implicitly define wavelet transform functions for image recognition.

Original: Applying lifting scheme to learn 3D shape representations is more challenging as amounts of shape points are arranged irregularly, and as far as we know, we are among the first to tackle this problem.
Revised: Due to the irregularity of 3D shape point arrangement, applying lifting scheme to learn 3D shape representation is more challenging than to 2D images proposed in DAWN-Net, and as far as we know, we are among the first to tackle this problem.

Original: Transformer in Vision.
Revised: Transformer in Vision.

Original: The seminal work to adapt Transformer (Vaswani et al., 2017) to vision tasks commences with Vision Transformer (ViT) (Dosovitskiy et al., 2020) and Detection Transformer (Carion et al., 2020).
Revised: The seminal work to adapt Transformer (Vaswani et al., 2017) to vision tasks commences with Vision Transformer (ViT) (Dosovitskiy et al., 2020) and Detection Transformer (Carion et al., 2020).

Original: Since then, various applications of Transformer in vision are proposed, including but not limited to: semantic segmentation (Wang et al., 2021; Zhang et al., 2020), generative modeling (Chen et al., 2020; Hudson & Zitnick, 2021), object detection (Zhu et al., 2021; Dai et al., 2021), video analysis (Gabeur et al., 2020; Zeng et al., 2020), to name a few.
Revised: Since then, various applications of Transformer in vision are proposed, including but not limited to: semantic segmentation (Wang et al., 2021; Zhang et al., 2020), generative modeling (Chen et al., 2020; Hudson & Zitnick, 2021), object detection (Zhu et al., 2021; Dai et al., 2021), video analysis (Gabeur et al., 2020; Zeng et al., 2020), to name a few.

Original: PCT (Guo et al., 2021) applies Transformer in 3D point cloud classification and segmentation.
Revised: PCT (Guo et al., 2021) applies Transformer in 3D point cloud classification and segmentation.

Original: Concurrent works include the other two types of Point Transfomer proposed by (Zhao et al., 2021) and (Engel et al., 2021).
Revised: Concurrent works include the other two types of Point Transfomer proposed by (Zhao et al., 2021) and (Engel et al., 2021).

Original: PoinTr (Yu et al., 2021) converts a point cloud to a sequence of point proxies and employs
Revised: PoinTr (Yu et al., 2021) converts a point cloud to a sequence of point proxies and employs

Original: Transformer for point cloud completion.
Revised: Transformer for point cloud completion.

Original: PPT-Net (Hui et al., 2021) adopts Transformer for largescale place recognition.
Revised: PPT-Net (Hui et al., 2021) adopts Transformer for largescale place recognition.

Original: MViT (Fan et al., 2021) and Swin (Liu et al., 2021) connect multi-scale feature hierarchies with transformer models by applying different Transformers for features with distinct spatial resolutions.
Revised: MViT (Fan et al., 2021) and Swin (Liu et al., 2021) connect multi-scale feature hierarchies with transformer models by applying different Transformers for features with distinct spatial resolutions.

Original: Since multi-scale discrete wavelet transform decomposes point clouds at each subsequent scales, it is intrinsically essential to combine with Transformer in a similar way as in MViT and Swin.
Revised: Since multi-scale discrete wavelet transform decomposes point clouds at each subsequent scales, it is intrinsically essential to combine with Transformer in a similar way as in MViT and Swin.

Original: Due to quadratic computational complexity of self-attention, Fast Autoregressive Transformer (Katharopoulos et al., 2020) is among the first to address the quadratic complexity by using the associativity property of matrix products.
Revised: Due to quadratic computational complexity of self-attention, Fast Autoregressive Transformer (Katharopoulos et al., 2020) is among the first to address the quadratic complexity by using the associativity property of matrix products.

Original: Some recent works (Kitaev et al., 2020; Dai et al., 2020; Tay et al., 2020; Choromanski et al., 2021) propose efficient Transformers for long sequences. These models can be also adapted for vision tasks and the one proposed in (Choromanski et al., 2021) is adopted in our work.
Revised: Some recent works (Kitaev et al., 2020; Dai et al., 2020; Tay et al., 2020; Choromanski et al., 2021) propose efficient Transformers for long sequences and these models can be also adapted for vision tasks.

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: We briefly review wavelet transform and lifting scheme as the building blocks of our model.
Revised: We briefly review wavelet transform and lifting scheme as the building blocks of our model.

Original: We refer the reader to (Sweldens, 1996;  Claypoole et al., 2003; Rodriguez et al., 2020) for more information.
Revised: We refer the reader to (Sweldens, 1996; 1998; Claypoole et al., 2003; Rodriguez et al., 2020) for more information.

Revised: Multi-head attention

Revised: Add & norm

Original: Wavelet Transform/Decomposition.
Revised: Feedforward Add & norm : − :  Transformer Transformer

Revised: C :  : − : − ⨂ |  | : − ⨂ |  |

Revised: Split

Revised: U

Revised: P []   []   [] [] []

Revised: Wavelet Transform/Decomposition.

Original: Let us consider a continuous scalar function of a single variable f ( x ) .
Revised: Let us consider a continuous scalar function of a single variable f ( x ) .

Original: Wavelet transform is to find a basis generated by a mother wavelet, denoted by ψ ( x ) , to represent f ( x ) with a few parameters.
Revised: Wavelet transform is to find a basis generated by a mother wavelet, denoted by ψ ( x ) , to represent f ( x ) with a few parameters.

Original: The basic rule for generating this basis is to translate and dilate ψ ( x ) , i.e ., ψ (2 j x − i ) , where j and i are integers.
Revised: The basic rule for generating this basis is to translate and dilate ψ ( x ) , i.e ., ψ (2 j x − i ) , where j and i are integers.

Original: The basis is localized in both time (controlled by i ) and scale/frequency (controlled by j ) domains.
Revised: The basis is localized in both time (controlled by i ) and scale/frequency (controlled by j ) domains.

Original: The original function f ( x ) is decomposed as a linear combination of the basis:
Revised: The original function f ( x ) is decomposed as a linear combination of the basis:

Original: [Equation]
Revised: [Equation]

Original: where a j,i are the wavelet coefficients of f ( x ) and can be computed as the inner product of ψ (2 j x − i ) with f ( x ) .
Revised: where a j,i are the wavelet coefficients of f ( x ) and can be computed as the inner product of ψ (2 j x − i ) with f ( x ) .

Original: For a discretized signal x [ n ] ( n ∈ Z ), we sample points from ψ (2 j x − i ) to form filter banks to decompose x [ n ] .
Revised: For a discretized signal x [ n ] ( n ∈ Z ), we sample points from ψ (2 j x − i ) to form filter banks to decompose x [ n ] .

Original: This is classical or first-generation wavelet (Santosa, 2011).
Revised: This is classical or first-generation wavelet (Santosa, 2011).

Original: Some commonly used mother wavelets include Haar wavelet (Haar, 1910), Daubechies wavelet,
Revised: Some commonly used mother wavelets include Haar wavelet (Haar, 1910), Daubechies wavelet,

Original: Biorthogonal wavelet (Daubechies, 1992), to name a few.
Revised: Biorthogonal wavelet (Daubechies, 1992), to name a few.

Original: Wavelet transform provides a nice framework for multi-resolution analysis of functions, i.e ., wavelet transform generates coarser and coarser versions of the same function while keeping track of the details missed going from fine to coarse.
Revised: Wavelet transform provides a nice framework for multi-resolution analysis of functions, i.e ., wavelet transform generates coarser and coarser versions of the same function while keeping track of the details missed going from fine to coarse.

Original: However, the first-generation wavelet mainly works well for infinite or periodic signals, which are rare in many applications.
Revised: However, the first-generation wavelet mainly works well for infinite or periodic signals, which are rare in many applications.

Original: It is still unclear how to design a proper basis for any arbitrary signals sampled irregularly from an unbounded domain.
Revised: It is still unclear how to design a proper basis for any arbitrary signals sampled irregularly from an unbounded domain.

Original: One solution is to give up designing a mother wavelet manually.
Revised: One solution is to give up designing a mother wavelet manually.

Original: Instead, the decomposition is entirely based on an approach called lifting scheme .
Revised: Instead, the decomposition is entirely based on an approach called lifting scheme .

Original: Lifting Scheme.
Revised: Lifting Scheme.

Original: Lifting scheme, a.k.a ., second-generation wavelets (Sweldens, 1998; Daubechies & Sweldens, 1998), is a simple yet effective approach to define wavelets sharing the same properties as the first-generation wavelets.
Revised: Lifting scheme, a.k.a ., second-generation wavelets (Sweldens, 1998; Daubechies & Sweldens, 1998), is a simple yet effective approach to define wavelets sharing the same properties as the first-generation wavelets.

Original: The lifting scheme takes as input a signal x and generates as outputs the approximation c and the detail d sub-bands of the wavelet transform.
Revised: The lifting scheme takes as input a signal x and generates as outputs the approximation c and the detail d sub-bands of the wavelet transform.

Original: The lifting scheme consists of three stages (Claypoole et al., 2003) as on the left panel of Figure 2.
Revised: The lifting scheme consists of three stages (Claypoole et al., 2003; Rodriguez et al., 2020) as on the left panel of Figure 2.

Original: • Split.
Revised: • Split.

Original: This stage splits the input signal into two non-overlapping partitions, e.g ., the input signal x is divided into even x e and odd x o components, which are defined as x e [ n ] = x [2 n ] and x o [ n ] = x [2 n + 1] .
Revised: This stage splits an input signal into two non-overlapping partitions, e g ., the input signal x is divided into even x e and odd x o components, which are defined as x e [ n ] = x [2 n ] and x o [ n ] = x [2 n + 1] .

Original: • Update (U).
Revised: • Update (U).

Original: This stage separates the signal in frequency domain, generating the approximation c that has the same running average as the original input signal.
Revised: This stage separates the signal in frequency domain, generating the approximation c that has the same running average as the original input signal.

Original: Let x
Revised: Let x

Original: L
Revised: L

Original: U
Revised: U

Original: o
Revised: o

Original: [ n ] = x o [ n − L U ] , x o [ n − L U + 1] , · · · , x o [ n + L U − 1] , x o [ n + L U ] denote the sequence of 2 L U + 1 neighboring odd polyphase samples around x e [ n ] .
Revised: [ n ] = x o [ n − L U ] , x o [ n − L U + 1] , · · · , x o [ n + L U − 1] , x o [ n + L U ] denote the sequence of 2 L U + 1 neighboring odd polyphase samples around x e [ n ] .

Original: The even polyphase samples are updated through an update operator U ( · ) on x L U o [ n ] , resulting the approximation c as:
Revised: The even polyphase samples are updated through an update operator U ( · ) on x L U o [ n ] , resulting the approximation c as:

Original: [Equation]
Revised: [Equation]

Original: • Predict (P).
Revised: • Predict (P).

Original: As the partitions x e and x o are usually closely correlated, given one of them, it is feasible to build a predictor P ( · ) for the other, by tracking the difference (or detail)
Revised: As the partitions x e and x o are usually closely correlated, given one of them, it is feasible to build a predictor P ( · ) for the other, by tracking the difference (or detail)

Original: d among them.
Revised: d among them.

Original: As the even component x e corresponds to the approximation c [ n ] , it is possible to define P ( · ) as a function of c [ n ] .
Revised: As the even component x e corresponds to the approximation c [ n ] , it is possible to define P ( · ) as a function of c [ n ] .

Original: Let c L P [ n ] =
Revised: Let c L P [ n ] =

Original: c
Revised: c

Original: [ n − L P ] , c [ n − L P + 1] , · · · , c [ n + L P − 1] , c [ n + L P ] denote a sequence of 2 L P + 1 approximation coefficients.
Revised: [ n − L P ] , c [ n − L P + 1] , · · · , c [ n + L P − 1] , c [ n + L P ] denote a sequence of 2 L P + 1 approximation coefficients.

Original: The odd polyphase samples are predicted as P ( c L P [ n ]) .
Revised: The odd polyphase samples are predicted as P ( c L P [ n ]) .

Original: The resulting prediction residual, or detail d , are computed as:
Revised: The resulting prediction residual, or detail d , are computed as:

Original: [Equation]
Revised: [Equation]

Original: Note that the update and predict operators U and P implicitly and effectively define the wavelet for  decomposition.
Revised: Note that the update and predict operators U and P implicitly and effectively define the wavelet for signal decomposition.

Original:  A DAPTIVE W AVELET T RANSFORMER N ETWORK
Revised:  A DAPTIVE W AVELET T RANSFORMER N ETWORK

Original: Overview.
Revised: Overview.

Original: The update and predict operators in Eq. 2 and Eq. 3 are linear, which are incapable of learning 3D shape geometry. In this section, we propose a data-driven adaptive lifting scheme that introduces non-linearity into wavelet.
Revised: In this section, we propose an Adaptive Wavelet Transformer Network, dubbed as AWTNet, for 3D shape representation learning, as shown on the right panel of Figure 3.

Original: Based on this scheme, we develop an Adaptive Wavelet Transformer Network, dubbed as AWT-Net , for 3D shape representation learning.
Revised: Similar to the overall structure of GDA-Net (Xu et al., 2021b), AWT-Net consists of three modules: 1) k-NN graph construction module to build graphs from point clouds, 2) adaptive lifting module, reflecting our main contribution that decomposes 3D shapes into high- and low-frequency components using wavelet analysis, and 3) transformer module to fuse decomposed component features.

Original: The AWT-Net generates shape representation hierarchically, and the decomposition tree is shown on the left panel of Figure 3.
Revised: The AWTNet generates shape representation hierarchically, and the decomposition tree is shown on the left panel of Figure 3.

Original: Here we show three scales where each scale corresponds to a separate lifting process as illustrated on the left panel of Figure 2.
Revised: Here we show three scales where each scale corresponds to a separate lifting process as illustrated on the left panel of Figure 2.

Original: We omit the split, update and predict blocks for conciseness.
Revised: We omit the split, update and predict blocks for conciseness.

Original: At each scale, the approximation c i serves as a coarse version of the input signal while the detail d i maintains the detailed difference between c
Revised: At each scale, the approximation c i serves as a coarse version of the input signal while the detail d i maintains the detailed difference between c

Original: i and the input signal.
Revised: i and the input signal.

Original: The approximation c i is further decomposed by another lifting process at the next scale.
Revised: The approximation c i is further decomposed by another lifting process at the next scale.

Original: The AWT-Net is built based on this decomposition tree but replaces the linear operators U and P  with non-linear neural networks.
Revised: The AWT-Net is built based on this decomposition tree but replaces the linear operators U and P in Eq. 2 and Eq. 3 with non-linear neural networks.

Original: Once trained and fixed, the  update U and predict P operators can be regarded as two sub-band basis/filters to subdivide 3D shapes into high-frequency and low-frequency components, in analogy to the basis in Eq.
Revised: Once trained and fixed, the non-linear update U and predict P operators can be regarded as two sub-band basis/filters to subdivide 3D shapes into high-frequency and low-frequency components, in analogy to the basis in Eq.

Original: Its unrolled version is shown on the right panel of Figure 3.

Original: Each module in AWT-Net is described as follows.

Original: [Figure]
Revised: [Figure]

Original: (cid:2870)
Revised:  Split U: GCN P: GCN

Original: Transformer
Revised: Transformer

Original: Transformer (cid:2870) C G C (cid:2869) (cid:2869)
Revised: Transformer   C G C S p a t i a l t r a n s f o r m Graph Construction (GC)

Original: 4. E VEN -O DD S PLIT
Revised: 4. E VEN -O DD S PLIT

Original: Graph Construction.
Revised: Graph Construction.

Original: Assume a point cloud containing N points associated with features x i ∈ R d , i ∈ { 1 , · · · , N } .
Revised: Assume a point cloud containing N points associated with features x i ∈ R d , i ∈ { 1 , · · · , N } .

Original: Features can be coordinates, normals and/or other high-dimensional features.
Revised: Features can be coordinates, normals and/or other high-dimensional features.

Original: We construct a k-NN graph G =
Revised: We construct a k-NN graph G =

Original: ( V , ˜
Revised: ( V , ˜

Original: A u , ˜
Revised: A u , ˜

Original: A w ) , where each point x i corresponds to a node v i ∈ V , ˜
Revised: A w ) , where each point x i corresponds to a node v i ∈ V , ˜

Original: A u and ˜
Revised: A u and ˜

Original: A w ∈ R N × N are unweighted and weighted adjacency matrices encoding point similarity in feature space.
Revised: A w ∈ R N × N are unweighted and weighted adjacency matrices encoding point similarity in feature space.

Original: The unweighted and weighted edges connecting two nodes v i and v j are defined as:
Revised: The unweighted and weighted edges connecting two nodes v i and v j are defined as:

Original: [Equation]
Revised: [Equation]

Original: where κ ( · ) is an non-negative function, e.g ., Gaussian function, to ensure that
Revised: where κ ( · ) is an non-negative function, e.g ., Gaussian function, to ensure that

Original: ˜
Revised: ˜

Original: A w is a diagonally dominant matrix and N represents neighborhood.
Revised: A w is a diagonally dominant matrix and N represents neighborhood.

Revised: Before constructing k-NN graph, we compute point features through a spatial transform layer (Wang et al., 2019), also termed as local operator in (Xu et al., 2021b).

Original: Following (Xu et al., 2021), we normalizeall edge weights in each row in A w to remove the impacts of varying neighborhood sizes and feature scales.
Revised: After graph construction, we normalize edge weights in each row in A w to remove the impacts of varying neighborhood sizes and feature scales as in (Xu et al., 2021b).

Original: Graph Split.
Revised: Graph Split.

Original: We partition all nodes in a graph into two sets, containing even and odd nodes, respectively.
Revised: We partition all nodes in a graph into two sets, containing even and odd nodes, respectively.

Original: Since lifting scheme uses information from odd ( resp .
Revised: Since lifting scheme uses information from odd ( resp .

Original: even) nodes to update ( resp .
Revised: even) nodes to update ( resp .

Original: predict) even ( resp . odd) nodes as in Eq.
Revised: predict) even ( resp . odd) nodes as in Eq.

Original: 2 ( resp .
Revised: 2 ( resp .

Original: Eq. 3), having all neighboring nodes with same parity means that local information cannot be used.
Revised: Eq. 3), having all neighboring nodes with same parity means that local information cannot be used.

Original: Following (Narang & Ortega, 2009), we define the number ofconflicts as the percentage of neighboring nodes that have the same parity after partition. The optimal splitting minimizes the number of conflicts. Theoretically, it is an NP-hard bipartite sub-graph problem, but a heuristic solution can be found. We adapt the conservative fixed probability colorer algorithm in (Fitzpatrick & Meertens, 2001), which is based on iteratively greedy local heuristics:
Revised: Following (Narang & Ortega, 2009), we define the number of

Original: Require: A graph G =
Revised: Require: A graph G =

Original: ( V , A u ) with unweighted adjacency, maximum number of iterations M 1 Randomly assign initial parity ( e.g ., 1 for odd, 0 for even) to each node 2 for m = 1: M do
Revised: ( V , A u ) with unweighted adjacency, maximum number of iterations M 1 Randomly assign initial parity ( e.g ., 1 for odd, 0 for even) to each node 2 for m = 1: M do

Original: Activate each  with a probability  from a uniform distribution independently for each activated node do
Revised: Activate each node with a probability sampled from a uniform distribution independently for each activated node do

Original: Choose a parity that minimizes its conflict with neighboring nodes /
Revised: Choose a parity that minimizes its conflict with neighboring nodes /

Original: * Keep the number of odd and even nodes roughly the same */
Revised: * Keep the number of odd and even nodes roughly the same */

Original: N o , N e ← number of nodes with parity 1 , parity if N o > N e then
Revised: N o , N e ← number of nodes with parity 1 , parity if N o > N e then

Original: Randomly flip ⌊ N o − N e⌋ odd nodes’ parity else
Revised: Randomly flip ⌊ N o − N e⌋ odd nodes’ parity else

Original: Randomly flip ⌊ N e − N o⌋ even nodes’ parity  at each iteration, a few randomly chosen nodes are activated, and each activated node counts the number of conflicting edges ( i.e ., connecting neighboring nodes with the same parity) and changes its parity based on the conflict.
Revised: Randomly flip ⌊ N e − N o⌋ even nodes’ parity conflicts as the percentage of neighboring nodes that have the same parity after partition. The optimal splitting minimizes the number of conflicts. Theoretically, it is an NP-hard bipartite sub-graph problem, but a heuristic solution can be found. We adapt the conservative fixed probability colorer algorithm in (Fitzpatrick & Meertens, 2001), which is based on iteratively greedy local heuristics: at each iteration, a few randomly chosen nodes are activated, and each activated node counts the number of conflicting edges ( i.e ., connecting neighboring nodes with the same parity) and changes its parity based on the conflict.

Original: The algorithm is summarized in Algorithm 1.
Revised: The algorithm is summarized in Algorithm 1.

Original: 4. A DAPTIVE L IFTING S CHEME
Revised: 4. A DAPTIVE L IFTING S CHEME

Original: Based on nodes’ parity after even-odd split, we re-arrange nodes V to gather odd and even nodes, and also re-arrange the weighted adjacency matrix A w accordingly:
Revised: Based on nodes’ parity after even-odd split, we re-arrange nodes V to gather odd and even nodes, and also re-arrange the weighted adjacency matrix A w accordingly:

Original: [Equation]
Revised: [Equation]

Original: where V odd and V even represent the sets of odd and even nodes.
Revised: where V odd and V even represent the sets of odd and even nodes.

Original: The sub-matrix F is the adjacency matrix of a subgraph containing odd nodes only.
Revised: The sub-matrix F is the adjacency matrix of a subgraph containing odd nodes only.

Original: The submatrix J contains edges that do not have conflicts, i.e ., edges connecting nodes with the opposite parity.
Revised: The submatrix J contains edges that do not have conflicts, i.e ., edges connecting nodes with the opposite parity.

Original: Similar definitions for K and L .
Revised: Similar definitions for K and L .

Original: Unlike the linear lifting scheme introduced in Section 3, we propose data-dependent non-linear update and predict operators, implemented as neural networks whose parameters are adaptively optimized to capture complex shape geometry.
Revised: Unlike the linear lifting scheme introduced in Section 3, we propose data-dependent non-linear update and predict operators, implemented as neural networks whose parameters are adaptively optimized to capture complex shape geometry.

Original: Specifically, we apply two Graph Convolutional Networks (GCNs) proposed in (Kipf & Welling, 2017) as the update and predict operators, separately.
Revised: Specifically, we apply two Graph Convolutional Networks (GCNs) proposed in (Kipf & Welling, 2017) as the update and predict operators, separately.

Original: The graph convolution allows us to compute the response of a node based on its neighbors decided by the adjacency matrix.
Revised: The graph convolution allows us to compute the response of a node based on its neighbors decided by the adjacency matrix.

Original: Performing graph convolution is equal to performing message gathering within a neighborhood around each center node, sharing similarities with the definitions of x L u o [ n ] and c L p [ n ] in Eq. 2 and Eq.
Revised: Performing graph convolution is equal to performing message gathering within a neighborhood around each center node, sharing similarities with the definitions of x L u o [ n ] and c L p [ n ] in Eq. 2 and Eq.

Original: The non-linear update and predict stages at i -th scale are defined as:
Revised: The non-linear update and predict stages at i -th scale are defined as:

Original: [Equation]
Revised: [Equation]

Original: where c i and d i represent the approximation and detail components at i -th scale.
Revised: where c i and d i represent the approximation and detail components at i -th scale.

Original: We repeat this process at each scale recursively and form a hierarchical decomposition tree, capturing shape geometry at multi-scales.
Revised: We repeat this process at each scale recursively and form a hierarchical decomposition tree, capturing shape geometry at multi-scales.

Original: The maximum number of scales can be decided analytically as log 2 N and the coarsest scale contains only a single point.
Revised: The maximum number of scales can be decided analytically as log 2 N and the coarsest scale contains only a single point.

Original: 4. T RANSFORMER FOR C ROSS A TTENTION
Revised: 4. T RANSFORMER FOR C ROSS A TTENTION

Original: The approximation and detail components act in different but complementary roles in representing 3D shape geometry, and thus should not be treated equally.
Revised: The approximation and detail components depict 3D shape geometry from different but complementary aspects, and thus can be exploited to enhance the original input shape features. Similar to GDA-Net

Original: To utilize different variation components gained from lifting scheme, we adopt Transformer (Vaswani et al., 2017) to integrate different components with the original input shape features.
Revised: (Xu et al., 2021b) which adopts an attention module to fuse variation components with the original shape features, we adopt Transformer (Vaswani et al., 2017) to query and fuse features from different sub-bands gained from lifting scheme in a principled way.

Original: The architecture of Transformer is shown on the right panel of Figure 2, and multi-head attention is defined as:
Revised: The architecture of Transformer is shown on the right panel of Figure 2, and multi-head attention is defined as:

Original: [Equation]
Revised: [Equation]

Original: [Equation] where Q, K, V denote query, key and value, respectively.
Revised: ⊤ ) V , (7) where Q, K, V denote query, key and value, respectively.

Original: For approximation c i as coarse component at i -th scale, we set query, key and value to:
Revised: For approximation c i as coarse component at i -th scale, we set query, key and value to:

Original: [Equation]
Revised: [Equation]

Original: where the definition of z i is shown in Figure 3.
Revised: where the definition of z i is shown in Figure 3.

Original: Note that for the first scale ( i.e ., i = 1 ), we set Q = {+z i − 
 =+} x as input.
Revised: Note that for the first scale ( i.e ., i = 1 ), we set Q = {+z i − 
 =+} x as input.

Original: For detail d i , as most of the values are close to zeros (explained in Section 4.4), directly feeding them into Transformer yields sub-optimal results.
Revised: For detail d i , as most of the values are close to zeros (explained in Section 4.4), directly feeding them into Transformer yields sub-optimal results.

Original: Utilizing a nice property of wavelets, namely time-frequency localization 1 , we regard d
Revised: Utilizing a nice property of wavelets, namely time-frequency localization 1 , we regard d

Original: i as a multiplier to strengthen or weaken the context features.
Revised: i as a multiplier to strengthen or weaken the context features.

Original: Formally, the query, key and value for detail d i at i -th scale are set as:
Revised: Formally, the query, key and value for detail d i at i -th scale are set as:

Original: [Equation]
Revised: [Equation]

Original: where ⊗ represents element-wise multiplication.
Revised: where ⊗ represents element-wise multiplication.

Original: For the first scale, we set both z
Revised: For the first scale, we set both z

Original: i − 

Revised: i −

Original: and c i − 1 to x .
Revised: and c i − 1 to x .

Original: To efficiently compute multi-head attention, we adopt Performer (Choromanski et al., 2021) as a variant of Transformer (Vaswani et al., 2017).

Original: Briefly, Performer represents the (attention) softmax kernel in terms of the Gaussian kernel, and approximates the latter by sampling from the Fourier

Original: Transform (Rahimi et al., 2007).

Original: 4. L OSS F UNCTION
Revised: 4. L OSS F UNCTION

Original: End-to-end training is performed using task-specific cross-entropy loss, in combination with regularization terms, to enforce a wavelet decomposition structure during training.
Revised: End-to-end training is performed using task-specific cross-entropy loss, in combination with regularization terms, to enforce a wavelet decomposition structure during training.

Original: The loss function takes the form of Eq.
Revised: The loss function takes the form of Eq.

Original: 10, where K in the first term denotes the number of classes, y i and p i are the ground-truth and the predicted probability of belonging to class i , respectively.
Revised: 10, where K in the first term denotes the number of classes, y i and p i are the ground-truth and the predicted probability of belonging to class i , respectively.

Original: [Equation]
Revised: [Equation]

Original: 1 ( d i ) . In lifting scheme, a perfect output of Eq. 2, i.e ., the series c [ n ] , would be a coarse approximation of x [ n ] at half the resolution.
Revised: In lifting scheme, a perfect output of Eq. 2, i.e ., the series c [ n ] , would be a coarse approximation of x [ n ] at half the resolution.

Original: In the second term, we enforce each lifted even node has the same local average as its corresponding node in the original graph.
Revised: In the second term, we enforce each lifted even node has the same local average as its corresponding node in the original graph.

Original: Due to the half resolution of c i compared with the input c
Revised: Due to the half resolution of c i compared with the input c

Original: i − 1 , we raise the adjacent matrix A to the power of two to capture the node features up to second-order neighbors, and minimize the difference between the input graph features and the coarser version across all the decomposition scales.
Revised: i − 1 , we raise the adjacent matrix A to the power of two to capture the node features up to second-order neighbors, and minimize the difference between the input graph features and the coarser version across all the decomposition scales.

Original: The rationale for the third term is: if the underlying signal x [ n ] is locally smooth, the prediction residuals d [ n ] should be small.
Revised: The rationale for the third term is: if the underlying signal x [ n ] is locally smooth, the prediction residuals d [ n ] should be small.

Original: Thus, the third term promotes the minimization of the magnitude of detail coefficients
Revised: Thus, the third term promotes the minimization of the magnitude of detail coefficients

Original: d [ n ] in Eq. 3, indicating smaller energy is contained in d [ n ] .
Revised: d [ n ] in Eq. 3, indicating smaller energy is contained in d [ n ] .

Original:  E XPERIMENTS AND R ESULTS
Revised:  E XPERIMENTS AND R ESULTS

Original: We evaluate our model on 3D shape classification and part segmentation tasks on three datasets:
Revised: We evaluate our model on 3D shape classification and part segmentation tasks on three datasets following the settings (Qi et al., 2017a;b; Wang et al., 2019; Xu et al., 2021b; Zhao et al., 2021):

Original: ModelNet40 (Wu et al., 2015), ScanObjectNN (Uy et al., 2019) and ShapeNet Part (Yi et al., 2016).
Revised: ModelNet40 (Wu et al., 2015), ScanObjectNN (Uy et al., 2019) and ShapeNet Part (Yi et al., 2016).

Original: 5. I MPLEMENTATION DETAILS
Revised: 5. I MPLEMENTATION DETAILS

Original: The number of the constructed neighborhoods in k-NN graphs at each stage is 64.
Revised: The number of the constructed neighborhoods in k-NN graphs at each stage is 64.

Original: The maximum number of iterations M used in the split Algorithm 1 is set to 10, which strikes a balance between efficiency and accuracy.
Revised: The maximum number of iterations M used in the split Algorithm 1 is set to 10, which strikes a balance between efficiency and accuracy.

Original: The update U ( · ) and P ( · ) operators are implemented as two GCNs (Kipf &
Revised: The update U ( · ) and P ( · ) operators are implemented as two GCNs (Kipf &

Original: Welling, 2017) and the internal structure of each GCN is: Conv-BN-ReLU-Dropout-Conv-BN.
Revised: Welling, 2017) and the internal structure of each GCN is: Conv-BN-ReLU-Dropout-Conv-BN.

Original: We use a two-heads attention in Transformer.
Revised: We use a two-heads attention in Transformer.

Original: We implement our model in PyTorch (Paszke et al., 2019).
Revised: We implement our model in PyTorch (Paszke et al., 2019).

Original: We use SGD optimizer with momentum and weight decay set to 0.9 and 0.0001, respectively.
Revised: We use SGD optimizer with momentum and weight decay set to 0.9 and 0.0001, respectively.

Original: For both 3D shape classification and 3D object part segmentation tasks, we train for 350 epochs.
Revised: For both 3D shape classification and 3D object part segmentation tasks, we train for 350 epochs.

Original: The initial learning rate is set to 0.1  and is dropped by 5x every 40 epochs and clipped at 0 .
Revised: The initial learning rate is set to 0.1 for shape classification and 0.003 for object part segmentation, and clipped at 0 .

Original: 9 e − 5 .
Revised: 9 e − 5 .

Original: The values of λ 1 and λ 2 are set to 0.1 for all tasks.
Revised: The values of λ 1 and λ 2 are set to 0.1 for all tasks.

Original: 5. O BJECT C LASSIFICATION
Revised: 5. O BJECT C LASSIFICATION

Original: We evaluate AWT-Net on ModelNet40 (Wu et al., 2015) which contains 9,843 training shapes and 2,468 test shapes across 40 categories.
Revised: We evaluate AWT-Net on ModelNet40 (Wu et al., 2015) which contains 9,843 training shapes and 2,468 test shapes across 40 categories.

Original: Points are uniformly sampled from mesh models by (Qi et al., 2017a).
Revised: Points are uniformly sampled from mesh models by (Qi et al., 2017a).

Original: Following (Wang et al., 2019; Xu et al., 2021), we translate shapes and randomly shuffle points in the training set.
Revised: Following (Wang et al., 2019; Xu et al., 2021b), we translate shapes and randomly shuffle points in the training set.

Original: Similar to (Qi et al., 2017a;b), we conduct several voting tests with random scaling and average the predictions during testing.
Revised: Similar to (Qi et al., 2017a;b), we conduct several voting tests with random scaling and average the predictions during testing.

Original: The quantitative comparisons of the overall classification accuracy with the state-of-the-art methods are in Table 1.
Revised: The quantitative comparisons of the overall classification accuracy with the state-of-the-art methods are in Table 1.

Original: AWT-Net with two scales achieves the best performance.
Revised: AWT-Net with two scales achieves the best performance.

Original: Beyond quantitative evaluation, we exhibit that our method can decompose shapes into approximation and detail components as shown in Figure 1 and Figure 6.
Revised: Beyond quantitative evaluation, we exhibit that our method can decompose shapes into approximation and detail components as shown in Figure 1 and Figure 6.

Original: We also evaluate AWT-Net on ScanObjectNN by (Uy et al., 2019) to examine the robustness to deformed and noisy objects scanned in the real world.
Revised: We also validate the robustness of AWT-Net on a real-scanned dataset (Uy et al., 2019) following the experimental settings in GDA-Net (Xu et al., 2021b).

Original: We test on the OBJ ONLY and noisy OBJ BG splits
Revised: We test on the OBJ ONLY and noisy OBJ BG splits, and the results are summarized in Table

Original: [Table]
Revised: [Table]

Original: and the results are summarized in Table 2, where our model achieves comparable accuracy and performance drop from OBJ ONLY to OBJ BG.
Revised: where our model achieves comparable accuracy and performance drop from OBJ ONLY to OBJ BG.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 5. P ART S EGMENTATION
Revised: 5. P ART S EGMENTATION

Original: We evaluate AWT-Net with three scales for shape part segmentation on ShapeNet Part
Revised: We evaluate AWT-Net with three scales for shape part segmentation on ShapeNet Part

Original: (Yi et al., 2016) dataset, which contains 16,881 shapes in 16 categories and is labeled in 50 parts where each shape has 2˜5 parts.
Revised: (Yi et al., 2016) dataset [Table] which contains 16,881 shapes in 16 categories and is labeled in 50 parts where each shape has 2 ˜ 5 parts.

Original: The same voting test in classification is adopted.
Revised: The same voting test in classification is adopted.

Original: The instance average, the class average
Revised: The instance average, the class average

Original: [Table]
Revised: [Table]

Original: and each class mean Inter-over-Union (mIoU) are summarized in Table 3.
Revised: and each class mean Inter-over-Union (mIoU) are summarized in Table 3.

Original: AWTNet achieves the best performance with class mIoU of 85.0% and instance mIoU of 86.6%.
Revised: AWTNet achieves the best performance with class mIoU of 85.0% and instance mIoU of 86.6%.

Original: It is worth noticing that AWT-Net performs better on objects with obvious boundaries such as bag, knife, laptop

Original: [Table]

Original: and skateboard.

Original: Some qualitative segmentation results are visualized in Figure
Revised: Some qualitative segmentation results are visualized in Figure

Original: [Table]
Revised: [Table]

Original:  A BLATION S TUDY
Revised:  A BLATION S TUDY

Original: We conduct ablative experiments on ModelNet40 and no voting during testing (unless specified).
Revised: We conduct ablative experiments on ModelNet40 and no voting during testing (unless specified).

Revised: In all ablative experiments, we follow the same experimental settings DGA-Net (Xu et al., 2021b).

Revised: In addition

Revised: [Table]

Revised: we adopt the same performance measurement metrics that were used in DGA-Net to report our results in Table 4 to Table 8 and Figures 5 in validating and analyzing the effectiveness of our 3D shape representation learning approach based on wavelet analysis for shape decomposition.

Original: Module Analysis.
Revised: Module Analysis.

Original: We ablate each module in AWT-Net, and the results are listed in Table 4.
Revised: We ablate each module in AWT-Net, and the results are listed in Table 4.

Original: When the Transformer takes as input the original point cloud and both approximation and detail components, AWT-Net achieves the best accuracy with 93.4%.
Revised: When the Transformer takes as input the original point cloud features fused with both approximation and detail components, AWT-Net achieves the best accuracy with 93.4%.

Original: AWT-Net also surpasses the architecture of only using k-NN with 1.1% improvement.

Original: Our method ultimately obtains an accuracy of 93.9% with voting tests.
Revised: Our method ultimately obtains an accuracy of 93.9% with voting.

Original: This experiment shows that the decomposed approximation and detail components capture different but complementary geometric features to get complete shape representation.
Revised: This experiment shows that the decomposed approximation and detail component complement mutually capture different but complementary geometric features to get complete shape representation.

Original: Split Scheme.
Revised: Split Scheme.

Original: We replace the split of approximation and detail components with random and farthest point sampling (FPS).
Revised: We validate the split of approximation and detail components against random and farthest point sampling (FPS).

Original: The results are listed in Table 5, where decomposing point clouds into approximation and detail components gets the highest accuracy.
Revised: The results are listed in Table 5, where decomposing point clouds into approximation and detail components gets the highest accuracy.

Original: Empirically, our wavelet decomposition strategy localizes points carried with informative geometry instead of noisy points, which improves the robustness against noise.

Original: Adjacency Matrix 
Revised: Adjacency Matrix Computation.

Original: Computation.

Original: The result of dynamic adjacency matrix calculation in Section 4.1 is shown in Table 6.
Revised: The result of dynamic adjacency matrix computation in Section 4.1 is shown in Table 6.

Original: We precompute the adjacency matrix on 3D coordinates to split point clouds, which achieves the accuracy of 91.8%.
Revised: We pre-compute the adjacency matrix on 3D coordinates to split point clouds, which achieves the accuracy of 91.8%.

Original: Nevertheless, by dynamically calculating the adjacency matrix in high dimensional semantic space at different scales, our network gains 1.6% improvement.
Revised: Nevertheless, constructing the adjacency matrix in high dimensional semantic space at different scales gains 1.6% improvement.

Original: Figure 1 and Figure 6 show that our decomposition strategy successfully decomposes points into flat areas and areas containing edges.
Revised: Figure 1 and Figure 6 show that our decomposition strategy successfully decomposes points into flat areas and areas containing edges.

Original: Note that we do not provide any supervision w.r.t .
Revised: Note that we do not provide any supervision w.r.t .

Original: shape geometry.
Revised: shape geometry.

Original: Decomposition Levels.
Revised: Decomposition Levels.

Original: We investigate the impact ofthe number of decomposition levels ( i.e ., resolution scales), and the results are summarized in Table 7.
Revised: We investigate the impact of decomposition levels ( i.e ., resolution scales), and the results are summarized in Table 7.

Original: AWT-Net performs best on the overall accuracy when the decomposition level is set to 2.
Revised: AWT-Net performs best on the overall accuracy when the decomposition level is set to 2.

Original: We also observe that for shapes with simple geometric structures ( e.g ., radio and wardrobe), more scales have a negative effect on the performance, as more scales make the network prone to over-fitting on these simple shapes.
Revised: We also observe that for shapes with simple geometric structures ( e.g ., radio and wardrobe), more scales have a negative effect on the performance, as more scales make the network prone to over-fitting on these simple shapes.

Original: While for shapes with complex structures ( e.g ., stool and person), more scales are needed to capture both coarse and finegrained geometric features.
Revised: While for shapes with complex structures ( e.g ., stool and person), more scales are needed to capture both coarse and fine-grained geometric features.

Original: That is, the accuracy increases with more scales applied to geometrically more complex objects.
Revised: That is, the accuracy increases with more scales applied to geometrically more complex objects.

Original: We find out that the ModelNet40 dataset contains more geometrically simple objects than complex ones, which leads to the overall accuracy drops with more scales adopted.
Revised: We find out that the ModelNet40 dataset contains more geometrically simple objects than complex ones, which leads to the overall accuracy drops with more scales adopted.

Original: Robustness Analysis.
Revised: Sampling and Rotation Test.

Original: First, we testthe robustness of AWT-Net onsampling density is tested by using sparser points as the input to AWT-Net pretrained with 1024 points. Figure 5 lists the results.
Revised: First, we test AWT-Net on the 3D shapes with different numbers of randomly sampled points and Figure 5 lists the results.

Original: Although sparser points bring difficulties, AWT-Net still performs consistently across all density settings. In addition, Table 8 summarizes the results of  rotationrobustness, where AWT-Net outperforms other compared methods, especially with 2.4% improvement than the second-best at (s/s). Last, AWT-Net is evaluated on ScanObjectNN (Uy et al., 2019) that consists of noisy and deformed scanned objects with non-uniform surface density.
Revised: Next, we evaluate the robustness of AWTNet against rotation and Table 8 summarizes the results.

Original: Table 2 summarizes AWT-Net gains the comparable accuracy drop from OBJ ONLY to OBJ BG, proving that AWT-Net is robust against noise.
Revised: We note that AWT-Net outperforms the compared methods.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We propose AWT-Net for 3D shape representation learning. Rooted in wavelet transform, AWT-Net dynamically decomposes point clouds into approximation and detail components at multiple resolution scales, which respectively denotes the low and high sub-bands of point clouds recursively.
Revised: Previous works such as GDA-Net have observed the effectiveness of learning shape representation via decomposing the 3D shapes into low- and high-frequency components. In this work, we further advance the shape decomposition with a novel wavelet analysis based method that decomposes 3D shapes into sub-bands components.

Original: Another core component is Transformer, which explores the relations between the original complete points and coarse and detail components to provide complementary geometric information for learning shape representations.
Revised: Our work proposed to use the deep neural networks to realize the shape decomposition in a learnable fashion that seamlessly connects to the downstream tasks of shape representation learning.

Original: Extensive experiments show that our method achieves comparable performance  and is robust to noise.
Revised: Extensive experiments show that our AWT-Net achieves excellent performance on 3D shape classification and segmentation tasks which are on par with the state-ofthe-art approaches.

Original: A CKNOWLEDGMENTS
Revised: A CKNOWLEDGMENTS

Original: We would like to thank the reviewers  for their comments and efforts towards improving our manuscript.
Revised: We would like to thank the reviewers and the authors of GDA-Net (Xu et al., 2021b) for their comments and efforts towards improving our manuscript.

Original: The authors appreciate the generous support provided by Inception Institute of Artificial Intelligence (IIAI) in the form of NYUAD Global Ph.D. Student Fellowship.
Revised: The authors appreciate the generous support provided by Inception Institute of Artificial Intelligence (IIAI) in the form of NYUAD Global Ph.D. Student Fellowship.

Revised: This work was also partially supported by the NYUAD Center for Artificial

Revised: Intelligence and Robotics (CAIR), funded by Tamkeen under the NYUAD Research Institute Award

Revised: CG010.

Original: A A PPENDIX A. T ABLES AND F IGURES FOR A BLATION S TUDY
Revised: A A PPENDIX A. T ABLES AND F IGURES FOR A BLATION S TUDY

Original: In this section, we list tables and figures for ablation study in Section 6 for a better understanding of the proposed model.
Revised: In this section, we list tables and figures for ablation study in Section 6 for a better understanding of the proposed model.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: 256 128 64 Number of points A cc u r a c y GDANetSpiderCNNDGCNNPointNet++PointNetAWT-Net +P (ours)

Original: A. M ORE VISUALIZATION OF DECOMPOSITION OF SHAPES
Revised: A. M ORE VISUALIZATION OF DECOMPOSITION OF SHAPES

Original: We present more visualization of detail components of multiple shapes from the test set of ModelNet40 (Wu et al., 2015) in Figure 6.
Revised: We present more visualization of detail components of multiple shapes from the test set of ModelNet40 (Wu et al., 2015) in Figure 6.

Original: Such geometric representations are learned implicitly in 3D shape classification task without component supervision
Revised: Such geometric representations are learned implicitly in 3D shape classification task without component supervision

Original: . A. E XECUTION TIME AND MODEL PARAMETERS
Revised: . A. E XECUTION TIME AND MODEL PARAMETERS

Original: We compare the number of parameters and computational overhead w.r.t .
Revised: We compare the number of parameters and computational overhead w.r.t .

Original: GDA-Net
Revised: GDA-Net

Original: (Xu et al., 2021) on ModelNet

Revised: (Xu et al., 2021b) on ModelNet

Original: and the results are listed in Table 9.
Revised: and the results are listed in Table 9.

Original: We measure four scales.
Revised: We measure four scales.

Original: Note thatGDA-Net
Revised: Note that

Original: [Table] consists of two building blocks, which is fairly comparable to “scale 2” in our model.
Revised: GDA-Net consists of two building blocks, which is fairly comparable to “scale 2” in our model.

Original: In this case, though our model contains around 25% more parameters, the execution time is only 13% slower than GDA-Net, which shows the practicability of our model. 
Revised: In this case, though our model contains around 25% more parameters, the execution time is only 13% slower than GDA-Net, which shows the practicability of our model. [Table]

Original: A. R ANDOMNESS IN EVEN - ODD NODE SPLIT
Revised: A. R ANDOMNESS IN EVEN - ODD NODE SPLIT

Original: We compare the effect of using even-odd split and random split in Algorithm 1 and the results are listed in Table 10.
Revised: We compare the effect of using even-odd split and random split in Algorithm 1 and the results are listed in Table 10.

Original: Using random split performers slightly worse than even-odd split.
Revised: Using random split performers slightly worse than even-odd split.

Original: With evenodd split, we can separate any two connected nodes into two different sets to the best extent, thus ensuring each local area contains both even and odd nodes.
Revised: With evenodd split, we can separate any two connected nodes into two different sets to the best extent, thus ensuring each local area contains both even and odd nodes.

Original: However, with random split, some local areas might contain only one type of nodes, thus failing to be decomposed properly.
Revised: However, with random split, some local areas might contain only one type of nodes, thus failing to be decomposed properly.

Original: [Table]
Revised: [Table]